diff --git a/linux/include/tensorflow/c/LICENSE b/linux/include/tensorflow/c/LICENSE new file mode 100644 index 000000000..02c27d14a --- /dev/null +++ b/linux/include/tensorflow/c/LICENSE @@ -0,0 +1,7668 @@ +-------------------------------------------------------------------------------- +BEGIN LICENSE FOR third_party/hadoop/LICENSE.txt +-------------------------------------------------------------------------------- + + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + + +APACHE HADOOP SUBCOMPONENTS: + +The Apache Hadoop project contains subcomponents with separate copyright +notices and license terms. Your use of the source code for the these +subcomponents is subject to the terms and conditions of the following +licenses. + +For the org.apache.hadoop.util.bloom.* classes: + +/** + * + * Copyright (c) 2005, European Commission project OneLab under contract + * 034819 (http://www.one-lab.org) + * All rights reserved. + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * - Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * - Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the distribution. + * - Neither the name of the University Catholique de Louvain - UCL + * nor the names of its contributors may be used to endorse or + * promote products derived from this software without specific prior + * written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS + * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE + * COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, + * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, + * BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; + * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER + * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN + * ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE + * POSSIBILITY OF SUCH DAMAGE. + */ + +For portions of the native implementation of slicing-by-8 CRC calculation +in src/main/native/src/org/apache/hadoop/util: + +/** + * Copyright 2008,2009,2010 Massachusetts Institute of Technology. + * All rights reserved. Use of this source code is governed by a + * BSD-style license that can be found in the LICENSE file. + */ + + For src/main/native/src/org/apache/hadoop/io/compress/lz4/lz4.c: + +/* + LZ4 - Fast LZ compression algorithm + Copyright (C) 2011, Yann Collet. + BSD License + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions are + met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above + copyright notice, this list of conditions and the following disclaimer + in the documentation and/or other materials provided with the + distribution. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +*/ +-------------------------------------------------------------------------------- +END LICENSE FOR third_party/hadoop/LICENSE.txt +-------------------------------------------------------------------------------- +-------------------------------------------------------------------------------- +BEGIN LICENSE FOR third_party/eigen3/LICENSE +-------------------------------------------------------------------------------- +Eigen is primarily MPL2 licensed. See COPYING.MPL2 and these links: + http://www.mozilla.org/MPL/2.0/ + http://www.mozilla.org/MPL/2.0/FAQ.html + +Some files contain third-party code under BSD or LGPL licenses, whence +the other COPYING.* files here. + +All the LGPL code is either LGPL 2.1-only, or LGPL 2.1-or-later. +For this reason, the COPYING.LGPL file contains the LGPL 2.1 text. + +If you want to guarantee that the Eigen code that you are #including +is licensed under the MPL2 and possibly more permissive licenses (like +BSD), #define this preprocessor symbol: EIGEN_MPL2_ONLY +For example, with most compilers, you could add this to your project + CXXFLAGS: -DEIGEN_MPL2_ONLY +This will cause a compilation error to be generated if you #include +any code that is LGPL licensed. + +---------------------------------------------------------------------- +Following applies to: +./test/mapstaticmethods.cpp +./test/schur_real.cpp +./test/prec_inverse_4x4.cpp +./test/smallvectors.cpp +./test/redux.cpp +./test/special_numbers.cpp +./test/adjoint.cpp +./test/resize.cpp +./test/mixingtypes.cpp +./test/product_trmv.cpp +./test/sparse_solvers.cpp +./test/cholesky.cpp +./test/geo_quaternion.cpp +./test/miscmatrices.cpp +./test/stddeque.cpp +./test/integer_types.cpp +./test/product_large.cpp +./test/eigensolver_generic.cpp +./test/householder.cpp +./test/geo_orthomethods.cpp +./test/array_for_matrix.cpp +./test/sparseLM.cpp +./test/upperbidiagonalization.cpp +./test/nomalloc.cpp +./test/packetmath.cpp +./test/jacobisvd.cpp +./test/geo_transformations.cpp +./test/swap.cpp +./test/eigensolver_selfadjoint.cpp +./test/inverse.cpp +./test/product_selfadjoint.cpp +./test/product_trsolve.cpp +./test/product_extra.cpp +./test/sparse_solver.h +./test/mapstride.cpp +./test/mapped_matrix.cpp +./test/geo_eulerangles.cpp +./test/eigen2support.cpp +./test/denseLM.cpp +./test/stdvector.cpp +./test/nesting_ops.cpp +./test/sparse_permutations.cpp +./test/zerosized.cpp +./test/exceptions.cpp +./test/vectorwiseop.cpp +./test/cwiseop.cpp +./test/basicstuff.cpp +./test/product_trmm.cpp +./test/linearstructure.cpp +./test/sparse_product.cpp +./test/stdvector_overload.cpp +./test/stable_norm.cpp +./test/umeyama.cpp +./test/unalignedcount.cpp +./test/triangular.cpp +./test/product_mmtr.cpp +./test/sparse_basic.cpp +./test/sparse_vector.cpp +./test/meta.cpp +./test/real_qz.cpp +./test/ref.cpp +./test/eigensolver_complex.cpp +./test/cholmod_support.cpp +./test/conjugate_gradient.cpp +./test/sparse.h +./test/simplicial_cholesky.cpp +./test/bicgstab.cpp +./test/dynalloc.cpp +./test/product_notemporary.cpp +./test/geo_hyperplane.cpp +./test/lu.cpp +./test/qr.cpp +./test/hessenberg.cpp +./test/sizeof.cpp +./test/main.h +./test/selfadjoint.cpp +./test/permutationmatrices.cpp +./test/superlu_support.cpp +./test/qtvector.cpp +./test/geo_homogeneous.cpp +./test/determinant.cpp +./test/array_reverse.cpp +./test/unalignedassert.cpp +./test/stdlist.cpp +./test/product_symm.cpp +./test/corners.cpp +./test/dontalign.cpp +./test/visitor.cpp +./test/geo_alignedbox.cpp +./test/diagonalmatrices.cpp +./test/product_small.cpp +./test/eigensolver_generalized_real.cpp +./test/umfpack_support.cpp +./test/first_aligned.cpp +./test/qr_fullpivoting.cpp +./test/array_replicate.cpp +./test/geo_parametrizedline.cpp +./test/eigen2/eigen2_unalignedassert.cpp +./test/eigen2/eigen2_prec_inverse_4x4.cpp +./test/eigen2/eigen2_alignedbox.cpp +./test/eigen2/eigen2_sparse_product.cpp +./test/eigen2/eigen2_meta.cpp +./test/eigen2/eigen2_nomalloc.cpp +./test/eigen2/eigen2_visitor.cpp +./test/eigen2/eigen2_packetmath.cpp +./test/eigen2/eigen2_svd.cpp +./test/eigen2/eigen2_mixingtypes.cpp +./test/eigen2/eigen2_qr.cpp +./test/eigen2/eigen2_cwiseop.cpp +./test/eigen2/eigen2_geometry_with_eigen2_prefix.cpp +./test/eigen2/eigen2_smallvectors.cpp +./test/eigen2/eigen2_commainitializer.cpp +./test/eigen2/eigen2_sparse_solvers.cpp +./test/eigen2/eigen2_hyperplane.cpp +./test/eigen2/eigen2_eigensolver.cpp +./test/eigen2/eigen2_linearstructure.cpp +./test/eigen2/eigen2_sizeof.cpp +./test/eigen2/eigen2_parametrizedline.cpp +./test/eigen2/eigen2_lu.cpp +./test/eigen2/eigen2_adjoint.cpp +./test/eigen2/eigen2_geometry.cpp +./test/eigen2/eigen2_stdvector.cpp +./test/eigen2/eigen2_newstdvector.cpp +./test/eigen2/eigen2_submatrices.cpp +./test/eigen2/sparse.h +./test/eigen2/eigen2_swap.cpp +./test/eigen2/eigen2_triangular.cpp +./test/eigen2/eigen2_basicstuff.cpp +./test/eigen2/gsl_helper.h +./test/eigen2/eigen2_dynalloc.cpp +./test/eigen2/eigen2_array.cpp +./test/eigen2/eigen2_map.cpp +./test/eigen2/main.h +./test/eigen2/eigen2_miscmatrices.cpp +./test/eigen2/eigen2_product_large.cpp +./test/eigen2/eigen2_first_aligned.cpp +./test/eigen2/eigen2_cholesky.cpp +./test/eigen2/eigen2_determinant.cpp +./test/eigen2/eigen2_sum.cpp +./test/eigen2/eigen2_inverse.cpp +./test/eigen2/eigen2_regression.cpp +./test/eigen2/eigen2_product_small.cpp +./test/eigen2/eigen2_qtvector.cpp +./test/eigen2/eigen2_sparse_vector.cpp +./test/eigen2/product.h +./test/eigen2/eigen2_sparse_basic.cpp +./test/eigen2/eigen2_bug_132.cpp +./test/array.cpp +./test/product_syrk.cpp +./test/commainitializer.cpp +./test/conservative_resize.cpp +./test/qr_colpivoting.cpp +./test/nullary.cpp +./test/bandmatrix.cpp +./test/pastix_support.cpp +./test/product.h +./test/block.cpp +./test/vectorization_logic.cpp +./test/jacobi.cpp +./test/diagonal.cpp +./test/schur_complex.cpp +./test/sizeoverflow.cpp +./bench/BenchTimer.h +./bench/benchFFT.cpp +./bench/eig33.cpp +./bench/spbench/spbenchsolver.h +./bench/spbench/spbenchstyle.h +./lapack/complex_double.cpp +./lapack/cholesky.cpp +./lapack/lapack_common.h +./lapack/eigenvalues.cpp +./lapack/single.cpp +./lapack/lu.cpp +./lapack/complex_single.cpp +./lapack/double.cpp +./demos/mix_eigen_and_c/binary_library.cpp +./demos/mix_eigen_and_c/binary_library.h +./demos/mix_eigen_and_c/example.c +./demos/mandelbrot/mandelbrot.cpp +./demos/mandelbrot/mandelbrot.h +./demos/opengl/icosphere.cpp +./demos/opengl/icosphere.h +./demos/opengl/camera.cpp +./demos/opengl/quaternion_demo.h +./demos/opengl/camera.h +./demos/opengl/trackball.h +./demos/opengl/gpuhelper.h +./demos/opengl/trackball.cpp +./demos/opengl/gpuhelper.cpp +./demos/opengl/quaternion_demo.cpp +./debug/gdb/printers.py +./unsupported/test/minres.cpp +./unsupported/test/openglsupport.cpp +./unsupported/test/jacobisvd.cpp +./unsupported/test/dgmres.cpp +./unsupported/test/matrix_square_root.cpp +./unsupported/test/bdcsvd.cpp +./unsupported/test/matrix_exponential.cpp +./unsupported/test/forward_adolc.cpp +./unsupported/test/polynomialsolver.cpp +./unsupported/test/matrix_function.cpp +./unsupported/test/sparse_extra.cpp +./unsupported/test/matrix_functions.h +./unsupported/test/svd_common.h +./unsupported/test/FFTW.cpp +./unsupported/test/alignedvector3.cpp +./unsupported/test/autodiff.cpp +./unsupported/test/gmres.cpp +./unsupported/test/BVH.cpp +./unsupported/test/levenberg_marquardt.cpp +./unsupported/test/matrix_power.cpp +./unsupported/test/kronecker_product.cpp +./unsupported/test/splines.cpp +./unsupported/test/polynomialutils.cpp +./unsupported/bench/bench_svd.cpp +./unsupported/Eigen/IterativeSolvers +./unsupported/Eigen/src/IterativeSolvers/DGMRES.h +./unsupported/Eigen/src/IterativeSolvers/IncompleteLU.h +./unsupported/Eigen/src/IterativeSolvers/GMRES.h +./unsupported/Eigen/src/IterativeSolvers/IncompleteCholesky.h +./unsupported/Eigen/src/IterativeSolvers/Scaling.h +./unsupported/Eigen/src/IterativeSolvers/MINRES.h +./unsupported/Eigen/src/SparseExtra/RandomSetter.h +./unsupported/Eigen/src/SparseExtra/MatrixMarketIterator.h +./unsupported/Eigen/src/SparseExtra/DynamicSparseMatrix.h +./unsupported/Eigen/src/SparseExtra/MarketIO.h +./unsupported/Eigen/src/SparseExtra/BlockOfDynamicSparseMatrix.h +./unsupported/Eigen/src/KroneckerProduct/KroneckerTensorProduct.h +./unsupported/Eigen/src/NonLinearOptimization/LevenbergMarquardt.h +./unsupported/Eigen/src/NonLinearOptimization/HybridNonLinearSolver.h +./unsupported/Eigen/src/BVH/BVAlgorithms.h +./unsupported/Eigen/src/BVH/KdBVH.h +./unsupported/Eigen/src/AutoDiff/AutoDiffScalar.h +./unsupported/Eigen/src/AutoDiff/AutoDiffJacobian.h +./unsupported/Eigen/src/AutoDiff/AutoDiffVector.h +./unsupported/Eigen/src/Splines/Spline.h +./unsupported/Eigen/src/Splines/SplineFitting.h +./unsupported/Eigen/src/Splines/SplineFwd.h +./unsupported/Eigen/src/SVD/JacobiSVD.h +./unsupported/Eigen/src/SVD/BDCSVD.h +./unsupported/Eigen/src/SVD/SVDBase.h +./unsupported/Eigen/src/MatrixFunctions/MatrixFunction.h +./unsupported/Eigen/src/MatrixFunctions/MatrixSquareRoot.h +./unsupported/Eigen/src/MatrixFunctions/MatrixLogarithm.h +./unsupported/Eigen/src/MatrixFunctions/StemFunction.h +./unsupported/Eigen/src/MatrixFunctions/MatrixPower.h +./unsupported/Eigen/src/MatrixFunctions/MatrixExponential.h +./unsupported/Eigen/src/MatrixFunctions/MatrixFunctionAtomic.h +./unsupported/Eigen/src/MoreVectorization/MathFunctions.h +./unsupported/Eigen/src/LevenbergMarquardt/LevenbergMarquardt.h +./unsupported/Eigen/src/FFT/ei_fftw_impl.h +./unsupported/Eigen/src/FFT/ei_kissfft_impl.h +./unsupported/Eigen/src/Polynomials/PolynomialSolver.h +./unsupported/Eigen/src/Polynomials/Companion.h +./unsupported/Eigen/src/Polynomials/PolynomialUtils.h +./unsupported/Eigen/src/NumericalDiff/NumericalDiff.h +./unsupported/Eigen/src/Skyline/SkylineProduct.h +./unsupported/Eigen/src/Skyline/SkylineMatrixBase.h +./unsupported/Eigen/src/Skyline/SkylineStorage.h +./unsupported/Eigen/src/Skyline/SkylineUtil.h +./unsupported/Eigen/src/Skyline/SkylineInplaceLU.h +./unsupported/Eigen/src/Skyline/SkylineMatrix.h +./unsupported/Eigen/SparseExtra +./unsupported/Eigen/AdolcForward +./unsupported/Eigen/KroneckerProduct +./unsupported/Eigen/NonLinearOptimization +./unsupported/Eigen/BVH +./unsupported/Eigen/OpenGLSupport +./unsupported/Eigen/ArpackSupport +./unsupported/Eigen/AutoDiff +./unsupported/Eigen/Splines +./unsupported/Eigen/MPRealSupport +./unsupported/Eigen/MatrixFunctions +./unsupported/Eigen/MoreVectorization +./unsupported/Eigen/LevenbergMarquardt +./unsupported/Eigen/AlignedVector3 +./unsupported/Eigen/FFT +./unsupported/Eigen/Polynomials +./unsupported/Eigen/NumericalDiff +./unsupported/Eigen/Skyline +./COPYING.README +./COPYING.README +./LICENSE +./LICENSE +./LICENSE +./Eigen/Eigen2Support +./Eigen/src/Eigen2Support/VectorBlock.h +./Eigen/src/Eigen2Support/Cwise.h +./Eigen/src/Eigen2Support/Minor.h +./Eigen/src/Eigen2Support/Lazy.h +./Eigen/src/Eigen2Support/Memory.h +./Eigen/src/Eigen2Support/MathFunctions.h +./Eigen/src/Eigen2Support/Geometry/AlignedBox.h +./Eigen/src/Eigen2Support/Geometry/Hyperplane.h +./Eigen/src/Eigen2Support/Geometry/Quaternion.h +./Eigen/src/Eigen2Support/Geometry/Rotation2D.h +./Eigen/src/Eigen2Support/Geometry/ParametrizedLine.h +./Eigen/src/Eigen2Support/Geometry/RotationBase.h +./Eigen/src/Eigen2Support/Geometry/Translation.h +./Eigen/src/Eigen2Support/Geometry/Scaling.h +./Eigen/src/Eigen2Support/Geometry/AngleAxis.h +./Eigen/src/Eigen2Support/Geometry/Transform.h +./Eigen/src/Eigen2Support/TriangularSolver.h +./Eigen/src/Eigen2Support/LU.h +./Eigen/src/Eigen2Support/QR.h +./Eigen/src/Eigen2Support/SVD.h +./Eigen/src/Eigen2Support/Meta.h +./Eigen/src/Eigen2Support/Block.h +./Eigen/src/Eigen2Support/Macros.h +./Eigen/src/Eigen2Support/LeastSquares.h +./Eigen/src/Eigen2Support/CwiseOperators.h +./Eigen/src/Jacobi/Jacobi.h +./Eigen/src/misc/Kernel.h +./Eigen/src/misc/SparseSolve.h +./Eigen/src/misc/Solve.h +./Eigen/src/misc/Image.h +./Eigen/src/SparseCore/SparseColEtree.h +./Eigen/src/SparseCore/SparseTranspose.h +./Eigen/src/SparseCore/SparseUtil.h +./Eigen/src/SparseCore/SparseCwiseBinaryOp.h +./Eigen/src/SparseCore/SparseDiagonalProduct.h +./Eigen/src/SparseCore/SparseProduct.h +./Eigen/src/SparseCore/SparseDot.h +./Eigen/src/SparseCore/SparseCwiseUnaryOp.h +./Eigen/src/SparseCore/SparseSparseProductWithPruning.h +./Eigen/src/SparseCore/SparseBlock.h +./Eigen/src/SparseCore/SparseDenseProduct.h +./Eigen/src/SparseCore/CompressedStorage.h +./Eigen/src/SparseCore/SparseMatrixBase.h +./Eigen/src/SparseCore/MappedSparseMatrix.h +./Eigen/src/SparseCore/SparseTriangularView.h +./Eigen/src/SparseCore/SparseView.h +./Eigen/src/SparseCore/SparseFuzzy.h +./Eigen/src/SparseCore/TriangularSolver.h +./Eigen/src/SparseCore/SparseSelfAdjointView.h +./Eigen/src/SparseCore/SparseMatrix.h +./Eigen/src/SparseCore/SparseVector.h +./Eigen/src/SparseCore/AmbiVector.h +./Eigen/src/SparseCore/ConservativeSparseSparseProduct.h +./Eigen/src/SparseCore/SparseRedux.h +./Eigen/src/SparseCore/SparsePermutation.h +./Eigen/src/Eigenvalues/RealSchur.h +./Eigen/src/Eigenvalues/ComplexEigenSolver.h +./Eigen/src/Eigenvalues/GeneralizedEigenSolver.h +./Eigen/src/Eigenvalues/ComplexSchur.h +./Eigen/src/Eigenvalues/RealQZ.h +./Eigen/src/Eigenvalues/EigenSolver.h +./Eigen/src/Eigenvalues/HessenbergDecomposition.h +./Eigen/src/Eigenvalues/GeneralizedSelfAdjointEigenSolver.h +./Eigen/src/Eigenvalues/Tridiagonalization.h +./Eigen/src/Eigenvalues/SelfAdjointEigenSolver.h +./Eigen/src/Eigenvalues/MatrixBaseEigenvalues.h +./Eigen/src/SuperLUSupport/SuperLUSupport.h +./Eigen/src/StlSupport/StdDeque.h +./Eigen/src/StlSupport/StdVector.h +./Eigen/src/StlSupport/StdList.h +./Eigen/src/StlSupport/details.h +./Eigen/src/SparseQR/SparseQR.h +./Eigen/src/LU/Inverse.h +./Eigen/src/LU/arch/Inverse_SSE.h +./Eigen/src/LU/Determinant.h +./Eigen/src/LU/PartialPivLU.h +./Eigen/src/LU/FullPivLU.h +./Eigen/src/UmfPackSupport/UmfPackSupport.h +./Eigen/src/OrderingMethods/Ordering.h +./Eigen/src/OrderingMethods/Eigen_Colamd.h +./Eigen/src/QR/HouseholderQR.h +./Eigen/src/QR/ColPivHouseholderQR.h +./Eigen/src/QR/FullPivHouseholderQR.h +./Eigen/src/SVD/JacobiSVD.h +./Eigen/src/SVD/UpperBidiagonalization.h +./Eigen/src/Geometry/OrthoMethods.h +./Eigen/src/Geometry/AlignedBox.h +./Eigen/src/Geometry/Hyperplane.h +./Eigen/src/Geometry/Quaternion.h +./Eigen/src/Geometry/EulerAngles.h +./Eigen/src/Geometry/Rotation2D.h +./Eigen/src/Geometry/ParametrizedLine.h +./Eigen/src/Geometry/RotationBase.h +./Eigen/src/Geometry/arch/Geometry_SSE.h +./Eigen/src/Geometry/Umeyama.h +./Eigen/src/Geometry/Homogeneous.h +./Eigen/src/Geometry/Translation.h +./Eigen/src/Geometry/Scaling.h +./Eigen/src/Geometry/AngleAxis.h +./Eigen/src/Geometry/Transform.h +./Eigen/src/plugins/BlockMethods.h +./Eigen/src/plugins/CommonCwiseUnaryOps.h +./Eigen/src/plugins/CommonCwiseBinaryOps.h +./Eigen/src/plugins/MatrixCwiseUnaryOps.h +./Eigen/src/plugins/MatrixCwiseBinaryOps.h +./Eigen/src/Householder/Householder.h +./Eigen/src/Householder/HouseholderSequence.h +./Eigen/src/Householder/BlockHouseholder.h +./Eigen/src/Core/VectorBlock.h +./Eigen/src/Core/Matrix.h +./Eigen/src/Core/Ref.h +./Eigen/src/Core/SelfAdjointView.h +./Eigen/src/Core/MathFunctions.h +./Eigen/src/Core/GlobalFunctions.h +./Eigen/src/Core/MapBase.h +./Eigen/src/Core/EigenBase.h +./Eigen/src/Core/GenericPacketMath.h +./Eigen/src/Core/NestByValue.h +./Eigen/src/Core/CwiseUnaryOp.h +./Eigen/src/Core/SolveTriangular.h +./Eigen/src/Core/Fuzzy.h +./Eigen/src/Core/Visitor.h +./Eigen/src/Core/Map.h +./Eigen/src/Core/NoAlias.h +./Eigen/src/Core/Diagonal.h +./Eigen/src/Core/StableNorm.h +./Eigen/src/Core/CoreIterators.h +./Eigen/src/Core/products/Parallelizer.h +./Eigen/src/Core/products/SelfadjointMatrixVector.h +./Eigen/src/Core/products/GeneralMatrixMatrixTriangular.h +./Eigen/src/Core/products/TriangularSolverMatrix.h +./Eigen/src/Core/products/GeneralMatrixMatrix.h +./Eigen/src/Core/products/SelfadjointProduct.h +./Eigen/src/Core/products/CoeffBasedProduct.h +./Eigen/src/Core/products/TriangularMatrixVector.h +./Eigen/src/Core/products/SelfadjointMatrixMatrix.h +./Eigen/src/Core/products/TriangularSolverVector.h +./Eigen/src/Core/products/SelfadjointRank2Update.h +./Eigen/src/Core/products/GeneralBlockPanelKernel.h +./Eigen/src/Core/products/GeneralMatrixVector.h +./Eigen/src/Core/products/TriangularMatrixMatrix.h +./Eigen/src/Core/Reverse.h +./Eigen/src/Core/BooleanRedux.h +./Eigen/src/Core/Replicate.h +./Eigen/src/Core/arch/AltiVec/PacketMath.h +./Eigen/src/Core/arch/AltiVec/Complex.h +./Eigen/src/Core/arch/SSE/PacketMath.h +./Eigen/src/Core/arch/SSE/Complex.h +./Eigen/src/Core/arch/SSE/MathFunctions.h +./Eigen/src/Core/arch/NEON/PacketMath.h +./Eigen/src/Core/arch/NEON/Complex.h +./Eigen/src/Core/arch/Default/Settings.h +./Eigen/src/Core/CwiseUnaryView.h +./Eigen/src/Core/Array.h +./Eigen/src/Core/ArrayWrapper.h +./Eigen/src/Core/Swap.h +./Eigen/src/Core/Transpositions.h +./Eigen/src/Core/Random.h +./Eigen/src/Core/IO.h +./Eigen/src/Core/SelfCwiseBinaryOp.h +./Eigen/src/Core/VectorwiseOp.h +./Eigen/src/Core/Select.h +./Eigen/src/Core/ArrayBase.h +./Eigen/src/Core/DenseCoeffsBase.h +./Eigen/src/Core/DiagonalProduct.h +./Eigen/src/Core/Assign.h +./Eigen/src/Core/Redux.h +./Eigen/src/Core/ForceAlignedAccess.h +./Eigen/src/Core/BandMatrix.h +./Eigen/src/Core/PlainObjectBase.h +./Eigen/src/Core/DenseBase.h +./Eigen/src/Core/Flagged.h +./Eigen/src/Core/CwiseBinaryOp.h +./Eigen/src/Core/ProductBase.h +./Eigen/src/Core/TriangularMatrix.h +./Eigen/src/Core/Transpose.h +./Eigen/src/Core/DiagonalMatrix.h +./Eigen/src/Core/Dot.h +./Eigen/src/Core/Functors.h +./Eigen/src/Core/PermutationMatrix.h +./Eigen/src/Core/NumTraits.h +./Eigen/src/Core/MatrixBase.h +./Eigen/src/Core/DenseStorage.h +./Eigen/src/Core/util/Memory.h +./Eigen/src/Core/util/StaticAssert.h +./Eigen/src/Core/util/BlasUtil.h +./Eigen/src/Core/util/MatrixMapper.h +./Eigen/src/Core/util/XprHelper.h +./Eigen/src/Core/util/ForwardDeclarations.h +./Eigen/src/Core/util/Meta.h +./Eigen/src/Core/util/Macros.h +./Eigen/src/Core/util/Constants.h +./Eigen/src/Core/CwiseNullaryOp.h +./Eigen/src/Core/Block.h +./Eigen/src/Core/GeneralProduct.h +./Eigen/src/Core/CommaInitializer.h +./Eigen/src/Core/ReturnByValue.h +./Eigen/src/Core/Stride.h +./Eigen/src/SPQRSupport/SuiteSparseQRSupport.h +./Eigen/src/SparseLU/SparseLU_column_dfs.h +./Eigen/src/SparseLU/SparseLU_panel_dfs.h +./Eigen/src/SparseLU/SparseLU_relax_snode.h +./Eigen/src/SparseLU/SparseLU_panel_bmod.h +./Eigen/src/SparseLU/SparseLU_SupernodalMatrix.h +./Eigen/src/SparseLU/SparseLU_Utils.h +./Eigen/src/SparseLU/SparseLU_gemm_kernel.h +./Eigen/src/SparseLU/SparseLU_kernel_bmod.h +./Eigen/src/SparseLU/SparseLU_pivotL.h +./Eigen/src/SparseLU/SparseLU_Memory.h +./Eigen/src/SparseLU/SparseLU_heap_relax_snode.h +./Eigen/src/SparseLU/SparseLUImpl.h +./Eigen/src/SparseLU/SparseLU_copy_to_ucol.h +./Eigen/src/SparseLU/SparseLU_Structs.h +./Eigen/src/SparseLU/SparseLU.h +./Eigen/src/SparseLU/SparseLU_column_bmod.h +./Eigen/src/SparseLU/SparseLU_pruneL.h +./Eigen/src/IterativeLinearSolvers/IncompleteLUT.h +./Eigen/src/IterativeLinearSolvers/BasicPreconditioners.h +./Eigen/src/IterativeLinearSolvers/IterativeSolverBase.h +./Eigen/src/IterativeLinearSolvers/ConjugateGradient.h +./Eigen/src/IterativeLinearSolvers/BiCGSTAB.h +./Eigen/src/SparseCholesky/SimplicialCholesky.h +./Eigen/src/Cholesky/LDLT.h +./Eigen/src/Cholesky/LLT.h +./Eigen/src/CholmodSupport/CholmodSupport.h +./Eigen/src/PaStiXSupport/PaStiXSupport.h +./Eigen/src/MetisSupport/MetisSupport.h +./Eigen/StdVector +./Eigen/Core +./Eigen/SparseLU +./Eigen/StdList +./Eigen/StdDeque +./Eigen/SparseCholesky +./scripts/relicense.py +./scripts/relicense.py +./blas/BandTriangularSolver.h +./blas/PackedTriangularMatrixVector.h +./blas/complex_double.cpp +./blas/level2_real_impl.h +./blas/level1_cplx_impl.h +./blas/level1_impl.h +./blas/level1_real_impl.h +./blas/level3_impl.h +./blas/single.cpp +./blas/level2_cplx_impl.h +./blas/PackedSelfadjointProduct.h +./blas/Rank2Update.h +./blas/complex_single.cpp +./blas/PackedTriangularSolverVector.h +./blas/double.cpp +./blas/common.h +./blas/level2_impl.h +./blas/GeneralRank1Update.h + +Mozilla Public License Version 2.0 +================================== + +1. Definitions +-------------- + +1.1. "Contributor" + means each individual or legal entity that creates, contributes to + the creation of, or owns Covered Software. + +1.2. "Contributor Version" + means the combination of the Contributions of others (if any) used + by a Contributor and that particular Contributor's Contribution. + +1.3. "Contribution" + means Covered Software of a particular Contributor. + +1.4. "Covered Software" + means Source Code Form to which the initial Contributor has attached + the notice in Exhibit A, the Executable Form of such Source Code + Form, and Modifications of such Source Code Form, in each case + including portions thereof. + +1.5. "Incompatible With Secondary Licenses" + means + + (a) that the initial Contributor has attached the notice described + in Exhibit B to the Covered Software; or + + (b) that the Covered Software was made available under the terms of + version 1.1 or earlier of the License, but not also under the + terms of a Secondary License. + +1.6. "Executable Form" + means any form of the work other than Source Code Form. + +1.7. "Larger Work" + means a work that combines Covered Software with other material, in + a separate file or files, that is not Covered Software. + +1.8. "License" + means this document. + +1.9. "Licensable" + means having the right to grant, to the maximum extent possible, + whether at the time of the initial grant or subsequently, any and + all of the rights conveyed by this License. + +1.10. "Modifications" + means any of the following: + + (a) any file in Source Code Form that results from an addition to, + deletion from, or modification of the contents of Covered + Software; or + + (b) any new file in Source Code Form that contains any Covered + Software. + +1.11. "Patent Claims" of a Contributor + means any patent claim(s), including without limitation, method, + process, and apparatus claims, in any patent Licensable by such + Contributor that would be infringed, but for the grant of the + License, by the making, using, selling, offering for sale, having + made, import, or transfer of either its Contributions or its + Contributor Version. + +1.12. "Secondary License" + means either the GNU General Public License, Version 2.0, the GNU + Lesser General Public License, Version 2.1, the GNU Affero General + Public License, Version 3.0, or any later versions of those + licenses. + +1.13. "Source Code Form" + means the form of the work preferred for making modifications. + +1.14. "You" (or "Your") + means an individual or a legal entity exercising rights under this + License. For legal entities, "You" includes any entity that + controls, is controlled by, or is under common control with You. For + purposes of this definition, "control" means (a) the power, direct + or indirect, to cause the direction or management of such entity, + whether by contract or otherwise, or (b) ownership of more than + fifty percent (50%) of the outstanding shares or beneficial + ownership of such entity. + +2. License Grants and Conditions +-------------------------------- + +2.1. Grants + +Each Contributor hereby grants You a world-wide, royalty-free, +non-exclusive license: + +(a) under intellectual property rights (other than patent or trademark) + Licensable by such Contributor to use, reproduce, make available, + modify, display, perform, distribute, and otherwise exploit its + Contributions, either on an unmodified basis, with Modifications, or + as part of a Larger Work; and + +(b) under Patent Claims of such Contributor to make, use, sell, offer + for sale, have made, import, and otherwise transfer either its + Contributions or its Contributor Version. + +2.2. Effective Date + +The licenses granted in Section 2.1 with respect to any Contribution +become effective for each Contribution on the date the Contributor first +distributes such Contribution. + +2.3. Limitations on Grant Scope + +The licenses granted in this Section 2 are the only rights granted under +this License. No additional rights or licenses will be implied from the +distribution or licensing of Covered Software under this License. +Notwithstanding Section 2.1(b) above, no patent license is granted by a +Contributor: + +(a) for any code that a Contributor has removed from Covered Software; + or + +(b) for infringements caused by: (i) Your and any other third party's + modifications of Covered Software, or (ii) the combination of its + Contributions with other software (except as part of its Contributor + Version); or + +(c) under Patent Claims infringed by Covered Software in the absence of + its Contributions. + +This License does not grant any rights in the trademarks, service marks, +or logos of any Contributor (except as may be necessary to comply with +the notice requirements in Section 3.4). + +2.4. Subsequent Licenses + +No Contributor makes additional grants as a result of Your choice to +distribute the Covered Software under a subsequent version of this +License (see Section 10.2) or under the terms of a Secondary License (if +permitted under the terms of Section 3.3). + +2.5. Representation + +Each Contributor represents that the Contributor believes its +Contributions are its original creation(s) or it has sufficient rights +to grant the rights to its Contributions conveyed by this License. + +2.6. Fair Use + +This License is not intended to limit any rights You have under +applicable copyright doctrines of fair use, fair dealing, or other +equivalents. + +2.7. Conditions + +Sections 3.1, 3.2, 3.3, and 3.4 are conditions of the licenses granted +in Section 2.1. + +3. Responsibilities +------------------- + +3.1. Distribution of Source Form + +All distribution of Covered Software in Source Code Form, including any +Modifications that You create or to which You contribute, must be under +the terms of this License. You must inform recipients that the Source +Code Form of the Covered Software is governed by the terms of this +License, and how they can obtain a copy of this License. You may not +attempt to alter or restrict the recipients' rights in the Source Code +Form. + +3.2. Distribution of Executable Form + +If You distribute Covered Software in Executable Form then: + +(a) such Covered Software must also be made available in Source Code + Form, as described in Section 3.1, and You must inform recipients of + the Executable Form how they can obtain a copy of such Source Code + Form by reasonable means in a timely manner, at a charge no more + than the cost of distribution to the recipient; and + +(b) You may distribute such Executable Form under the terms of this + License, or sublicense it under different terms, provided that the + license for the Executable Form does not attempt to limit or alter + the recipients' rights in the Source Code Form under this License. + +3.3. Distribution of a Larger Work + +You may create and distribute a Larger Work under terms of Your choice, +provided that You also comply with the requirements of this License for +the Covered Software. If the Larger Work is a combination of Covered +Software with a work governed by one or more Secondary Licenses, and the +Covered Software is not Incompatible With Secondary Licenses, this +License permits You to additionally distribute such Covered Software +under the terms of such Secondary License(s), so that the recipient of +the Larger Work may, at their option, further distribute the Covered +Software under the terms of either this License or such Secondary +License(s). + +3.4. Notices + +You may not remove or alter the substance of any license notices +(including copyright notices, patent notices, disclaimers of warranty, +or limitations of liability) contained within the Source Code Form of +the Covered Software, except that You may alter any license notices to +the extent required to remedy known factual inaccuracies. + +3.5. Application of Additional Terms + +You may choose to offer, and to charge a fee for, warranty, support, +indemnity or liability obligations to one or more recipients of Covered +Software. However, You may do so only on Your own behalf, and not on +behalf of any Contributor. You must make it absolutely clear that any +such warranty, support, indemnity, or liability obligation is offered by +You alone, and You hereby agree to indemnify every Contributor for any +liability incurred by such Contributor as a result of warranty, support, +indemnity or liability terms You offer. You may include additional +disclaimers of warranty and limitations of liability specific to any +jurisdiction. + +4. Inability to Comply Due to Statute or Regulation +--------------------------------------------------- + +If it is impossible for You to comply with any of the terms of this +License with respect to some or all of the Covered Software due to +statute, judicial order, or regulation then You must: (a) comply with +the terms of this License to the maximum extent possible; and (b) +describe the limitations and the code they affect. Such description must +be placed in a text file included with all distributions of the Covered +Software under this License. Except to the extent prohibited by statute +or regulation, such description must be sufficiently detailed for a +recipient of ordinary skill to be able to understand it. + +5. Termination +-------------- + +5.1. The rights granted under this License will terminate automatically +if You fail to comply with any of its terms. However, if You become +compliant, then the rights granted under this License from a particular +Contributor are reinstated (a) provisionally, unless and until such +Contributor explicitly and finally terminates Your grants, and (b) on an +ongoing basis, if such Contributor fails to notify You of the +non-compliance by some reasonable means prior to 60 days after You have +come back into compliance. Moreover, Your grants from a particular +Contributor are reinstated on an ongoing basis if such Contributor +notifies You of the non-compliance by some reasonable means, this is the +first time You have received notice of non-compliance with this License +from such Contributor, and You become compliant prior to 30 days after +Your receipt of the notice. + +5.2. If You initiate litigation against any entity by asserting a patent +infringement claim (excluding declaratory judgment actions, +counter-claims, and cross-claims) alleging that a Contributor Version +directly or indirectly infringes any patent, then the rights granted to +You by any and all Contributors for the Covered Software under Section +2.1 of this License shall terminate. + +5.3. In the event of termination under Sections 5.1 or 5.2 above, all +end user license agreements (excluding distributors and resellers) which +have been validly granted by You or Your distributors under this License +prior to termination shall survive termination. + +************************************************************************ +* * +* 6. Disclaimer of Warranty * +* ------------------------- * +* * +* Covered Software is provided under this License on an "as is" * +* basis, without warranty of any kind, either expressed, implied, or * +* statutory, including, without limitation, warranties that the * +* Covered Software is free of defects, merchantable, fit for a * +* particular purpose or non-infringing. The entire risk as to the * +* quality and performance of the Covered Software is with You. * +* Should any Covered Software prove defective in any respect, You * +* (not any Contributor) assume the cost of any necessary servicing, * +* repair, or correction. This disclaimer of warranty constitutes an * +* essential part of this License. No use of any Covered Software is * +* authorized under this License except under this disclaimer. * +* * +************************************************************************ + +************************************************************************ +* * +* 7. Limitation of Liability * +* -------------------------- * +* * +* Under no circumstances and under no legal theory, whether tort * +* (including negligence), contract, or otherwise, shall any * +* Contributor, or anyone who distributes Covered Software as * +* permitted above, be liable to You for any direct, indirect, * +* special, incidental, or consequential damages of any character * +* including, without limitation, damages for lost profits, loss of * +* goodwill, work stoppage, computer failure or malfunction, or any * +* and all other commercial damages or losses, even if such party * +* shall have been informed of the possibility of such damages. This * +* limitation of liability shall not apply to liability for death or * +* personal injury resulting from such party's negligence to the * +* extent applicable law prohibits such limitation. Some * +* jurisdictions do not allow the exclusion or limitation of * +* incidental or consequential damages, so this exclusion and * +* limitation may not apply to You. * +* * +************************************************************************ + +8. Litigation +------------- + +Any litigation relating to this License may be brought only in the +courts of a jurisdiction where the defendant maintains its principal +place of business and such litigation shall be governed by laws of that +jurisdiction, without reference to its conflict-of-law provisions. +Nothing in this Section shall prevent a party's ability to bring +cross-claims or counter-claims. + +9. Miscellaneous +---------------- + +This License represents the complete agreement concerning the subject +matter hereof. If any provision of this License is held to be +unenforceable, such provision shall be reformed only to the extent +necessary to make it enforceable. Any law or regulation which provides +that the language of a contract shall be construed against the drafter +shall not be used to construe this License against a Contributor. + +10. Versions of the License +--------------------------- + +10.1. New Versions + +Mozilla Foundation is the license steward. Except as provided in Section +10.3, no one other than the license steward has the right to modify or +publish new versions of this License. Each version will be given a +distinguishing version number. + +10.2. Effect of New Versions + +You may distribute the Covered Software under the terms of the version +of the License under which You originally received the Covered Software, +or under the terms of any subsequent version published by the license +steward. + +10.3. Modified Versions + +If you create software not governed by this License, and you want to +create a new license for such software, you may create and use a +modified version of this License if you rename the license and remove +any references to the name of the license steward (except to note that +such modified license differs from this License). + +10.4. Distributing Source Code Form that is Incompatible With Secondary +Licenses + +If You choose to distribute Source Code Form that is Incompatible With +Secondary Licenses under the terms of this version of the License, the +notice described in Exhibit B of this License must be attached. + +Exhibit A - Source Code Form License Notice +------------------------------------------- + + This Source Code Form is subject to the terms of the Mozilla Public + License, v. 2.0. If a copy of the MPL was not distributed with this + file, You can obtain one at http://mozilla.org/MPL/2.0/. + +If it is not possible or desirable to put the notice in a particular +file, then You may include the notice in a location (such as a LICENSE +file in a relevant directory) where a recipient would be likely to look +for such a notice. + +You may add additional accurate notices of copyright ownership. + +Exhibit B - "Incompatible With Secondary Licenses" Notice +--------------------------------------------------------- + + This Source Code Form is "Incompatible With Secondary Licenses", as + defined by the Mozilla Public License, v. 2.0. + +---------------------------------------------------------------------- +Following applies to: +./doc/UsingIntelMKL.dox +./doc/UsingIntelMKL.dox +./Eigen/src/Eigenvalues/ComplexSchur_MKL.h +./Eigen/src/Eigenvalues/ComplexSchur_MKL.h +./Eigen/src/Eigenvalues/SelfAdjointEigenSolver_MKL.h +./Eigen/src/Eigenvalues/SelfAdjointEigenSolver_MKL.h +./Eigen/src/Eigenvalues/RealSchur_MKL.h +./Eigen/src/Eigenvalues/RealSchur_MKL.h +./Eigen/src/LU/arch/Inverse_SSE.h +./Eigen/src/LU/arch/Inverse_SSE.h +./Eigen/src/LU/PartialPivLU_MKL.h +./Eigen/src/LU/PartialPivLU_MKL.h +./Eigen/src/QR/HouseholderQR_MKL.h +./Eigen/src/QR/HouseholderQR_MKL.h +./Eigen/src/QR/ColPivHouseholderQR_MKL.h +./Eigen/src/QR/ColPivHouseholderQR_MKL.h +./Eigen/src/SVD/JacobiSVD_MKL.h +./Eigen/src/SVD/JacobiSVD_MKL.h +./Eigen/src/PardisoSupport/PardisoSupport.h +./Eigen/src/PardisoSupport/PardisoSupport.h +./Eigen/src/Core/Assign_MKL.h +./Eigen/src/Core/Assign_MKL.h +./Eigen/src/Core/products/SelfadjointMatrixVector_MKL.h +./Eigen/src/Core/products/SelfadjointMatrixVector_MKL.h +./Eigen/src/Core/products/GeneralMatrixVector_MKL.h +./Eigen/src/Core/products/GeneralMatrixVector_MKL.h +./Eigen/src/Core/products/SelfadjointMatrixMatrix_MKL.h +./Eigen/src/Core/products/SelfadjointMatrixMatrix_MKL.h +./Eigen/src/Core/products/TriangularMatrixMatrix_MKL.h +./Eigen/src/Core/products/TriangularMatrixMatrix_MKL.h +./Eigen/src/Core/products/GeneralMatrixMatrix_MKL.h +./Eigen/src/Core/products/GeneralMatrixMatrix_MKL.h +./Eigen/src/Core/products/TriangularMatrixVector_MKL.h +./Eigen/src/Core/products/TriangularMatrixVector_MKL.h +./Eigen/src/Core/products/GeneralMatrixMatrixTriangular_MKL.h +./Eigen/src/Core/products/GeneralMatrixMatrixTriangular_MKL.h +./Eigen/src/Core/products/TriangularSolverMatrix_MKL.h +./Eigen/src/Core/products/TriangularSolverMatrix_MKL.h +./Eigen/src/Core/util/MKL_support.h +./Eigen/src/Core/util/MKL_support.h +./Eigen/src/Cholesky/LLT_MKL.h +./Eigen/src/Cholesky/LLT_MKL.h + +/* + Copyright (c) 2011, Intel Corporation. All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. * + Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in the + documentation and/or other materials provided with the + distribution. * Neither the name of Intel Corporation nor the + names of its contributors may be used to endorse or promote + products derived from this software without specific prior written + permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +---------------------------------------------------------------------- +Following applies to: + everything under ./bench/btl + + GNU GENERAL PUBLIC LICENSE + Version 3, 29 June 2007 + + Copyright (C) 2007 Free Software Foundation, Inc. + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + Preamble + + The GNU General Public License is a free, copyleft license for +software and other kinds of works. + + The licenses for most software and other practical works are designed +to take away your freedom to share and change the works. By contrast, +the GNU General Public License is intended to guarantee your freedom to +share and change all versions of a program--to make sure it remains free +software for all its users. We, the Free Software Foundation, use the +GNU General Public License for most of our software; it applies also to +any other work released this way by its authors. You can apply it to +your programs, too. + + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +them if you wish), that you receive source code or can get it if you +want it, that you can change the software or use pieces of it in new +free programs, and that you know you can do these things. + + To protect your rights, we need to prevent others from denying you +these rights or asking you to surrender the rights. Therefore, you have +certain responsibilities if you distribute copies of the software, or if +you modify it: responsibilities to respect the freedom of others. + + For example, if you distribute copies of such a program, whether +gratis or for a fee, you must pass on to the recipients the same +freedoms that you received. You must make sure that they, too, receive +or can get the source code. And you must show them these terms so they +know their rights. + + Developers that use the GNU GPL protect your rights with two steps: +(1) assert copyright on the software, and (2) offer you this License +giving you legal permission to copy, distribute and/or modify it. + + For the developers' and authors' protection, the GPL clearly explains +that there is no warranty for this free software. For both users' and +authors' sake, the GPL requires that modified versions be marked as +changed, so that their problems will not be attributed erroneously to +authors of previous versions. + + Some devices are designed to deny users access to install or run +modified versions of the software inside them, although the manufacturer +can do so. This is fundamentally incompatible with the aim of +protecting users' freedom to change the software. The systematic +pattern of such abuse occurs in the area of products for individuals to +use, which is precisely where it is most unacceptable. Therefore, we +have designed this version of the GPL to prohibit the practice for those +products. If such problems arise substantially in other domains, we +stand ready to extend this provision to those domains in future versions +of the GPL, as needed to protect the freedom of users. + + Finally, every program is threatened constantly by software patents. +States should not allow patents to restrict development and use of +software on general-purpose computers, but in those that do, we wish to +avoid the special danger that patents applied to a free program could +make it effectively proprietary. To prevent this, the GPL assures that +patents cannot be used to render the program non-free. + + The precise terms and conditions for copying, distribution and +modification follow. + + TERMS AND CONDITIONS + + 0. Definitions. + + "This License" refers to version 3 of the GNU General Public License. + + "Copyright" also means copyright-like laws that apply to other kinds +of works, such as semiconductor masks. + + "The Program" refers to any copyrightable work licensed under this +License. Each licensee is addressed as "you". "Licensees" and +"recipients" may be individuals or organizations. + + To "modify" a work means to copy from or adapt all or part of the work +in a fashion requiring copyright permission, other than the making of an +exact copy. The resulting work is called a "modified version" of the +earlier work or a work "based on" the earlier work. + + A "covered work" means either the unmodified Program or a work based +on the Program. + + To "propagate" a work means to do anything with it that, without +permission, would make you directly or secondarily liable for +infringement under applicable copyright law, except executing it on a +computer or modifying a private copy. Propagation includes copying, +distribution (with or without modification), making available to the +public, and in some countries other activities as well. + + To "convey" a work means any kind of propagation that enables other +parties to make or receive copies. Mere interaction with a user through +a computer network, with no transfer of a copy, is not conveying. + + An interactive user interface displays "Appropriate Legal Notices" +to the extent that it includes a convenient and prominently visible +feature that (1) displays an appropriate copyright notice, and (2) +tells the user that there is no warranty for the work (except to the +extent that warranties are provided), that licensees may convey the +work under this License, and how to view a copy of this License. If +the interface presents a list of user commands or options, such as a +menu, a prominent item in the list meets this criterion. + + 1. Source Code. + + The "source code" for a work means the preferred form of the work +for making modifications to it. "Object code" means any non-source +form of a work. + + A "Standard Interface" means an interface that either is an official +standard defined by a recognized standards body, or, in the case of +interfaces specified for a particular programming language, one that +is widely used among developers working in that language. + + The "System Libraries" of an executable work include anything, other +than the work as a whole, that (a) is included in the normal form of +packaging a Major Component, but which is not part of that Major +Component, and (b) serves only to enable use of the work with that +Major Component, or to implement a Standard Interface for which an +implementation is available to the public in source code form. A +"Major Component", in this context, means a major essential component +(kernel, window system, and so on) of the specific operating system +(if any) on which the executable work runs, or a compiler used to +produce the work, or an object code interpreter used to run it. + + The "Corresponding Source" for a work in object code form means all +the source code needed to generate, install, and (for an executable +work) run the object code and to modify the work, including scripts to +control those activities. However, it does not include the work's +System Libraries, or general-purpose tools or generally available free +programs which are used unmodified in performing those activities but +which are not part of the work. For example, Corresponding Source +includes interface definition files associated with source files for +the work, and the source code for shared libraries and dynamically +linked subprograms that the work is specifically designed to require, +such as by intimate data communication or control flow between those +subprograms and other parts of the work. + + The Corresponding Source need not include anything that users +can regenerate automatically from other parts of the Corresponding +Source. + + The Corresponding Source for a work in source code form is that +same work. + + 2. Basic Permissions. + + All rights granted under this License are granted for the term of +copyright on the Program, and are irrevocable provided the stated +conditions are met. This License explicitly affirms your unlimited +permission to run the unmodified Program. The output from running a +covered work is covered by this License only if the output, given its +content, constitutes a covered work. This License acknowledges your +rights of fair use or other equivalent, as provided by copyright law. + + You may make, run and propagate covered works that you do not +convey, without conditions so long as your license otherwise remains +in force. You may convey covered works to others for the sole purpose +of having them make modifications exclusively for you, or provide you +with facilities for running those works, provided that you comply with +the terms of this License in conveying all material for which you do +not control copyright. Those thus making or running the covered works +for you must do so exclusively on your behalf, under your direction +and control, on terms that prohibit them from making any copies of +your copyrighted material outside their relationship with you. + + Conveying under any other circumstances is permitted solely under +the conditions stated below. Sublicensing is not allowed; section 10 +makes it unnecessary. + + 3. Protecting Users' Legal Rights From Anti-Circumvention Law. + + No covered work shall be deemed part of an effective technological +measure under any applicable law fulfilling obligations under article +11 of the WIPO copyright treaty adopted on 20 December 1996, or +similar laws prohibiting or restricting circumvention of such +measures. + + When you convey a covered work, you waive any legal power to forbid +circumvention of technological measures to the extent such circumvention +is effected by exercising rights under this License with respect to +the covered work, and you disclaim any intention to limit operation or +modification of the work as a means of enforcing, against the work's +users, your or third parties' legal rights to forbid circumvention of +technological measures. + + 4. Conveying Verbatim Copies. + + You may convey verbatim copies of the Program's source code as you +receive it, in any medium, provided that you conspicuously and +appropriately publish on each copy an appropriate copyright notice; +keep intact all notices stating that this License and any +non-permissive terms added in accord with section 7 apply to the code; +keep intact all notices of the absence of any warranty; and give all +recipients a copy of this License along with the Program. + + You may charge any price or no price for each copy that you convey, +and you may offer support or warranty protection for a fee. + + 5. Conveying Modified Source Versions. + + You may convey a work based on the Program, or the modifications to +produce it from the Program, in the form of source code under the +terms of section 4, provided that you also meet all of these conditions: + + a) The work must carry prominent notices stating that you modified + it, and giving a relevant date. + + b) The work must carry prominent notices stating that it is + released under this License and any conditions added under section + 7. This requirement modifies the requirement in section 4 to + "keep intact all notices". + + c) You must license the entire work, as a whole, under this + License to anyone who comes into possession of a copy. This + License will therefore apply, along with any applicable section 7 + additional terms, to the whole of the work, and all its parts, + regardless of how they are packaged. This License gives no + permission to license the work in any other way, but it does not + invalidate such permission if you have separately received it. + + d) If the work has interactive user interfaces, each must display + Appropriate Legal Notices; however, if the Program has interactive + interfaces that do not display Appropriate Legal Notices, your + work need not make them do so. + + A compilation of a covered work with other separate and independent +works, which are not by their nature extensions of the covered work, +and which are not combined with it such as to form a larger program, +in or on a volume of a storage or distribution medium, is called an +"aggregate" if the compilation and its resulting copyright are not +used to limit the access or legal rights of the compilation's users +beyond what the individual works permit. Inclusion of a covered work +in an aggregate does not cause this License to apply to the other +parts of the aggregate. + + 6. Conveying Non-Source Forms. + + You may convey a covered work in object code form under the terms +of sections 4 and 5, provided that you also convey the +machine-readable Corresponding Source under the terms of this License, +in one of these ways: + + a) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by the + Corresponding Source fixed on a durable physical medium + customarily used for software interchange. + + b) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by a + written offer, valid for at least three years and valid for as + long as you offer spare parts or customer support for that product + model, to give anyone who possesses the object code either (1) a + copy of the Corresponding Source for all the software in the + product that is covered by this License, on a durable physical + medium customarily used for software interchange, for a price no + more than your reasonable cost of physically performing this + conveying of source, or (2) access to copy the + Corresponding Source from a network server at no charge. + + c) Convey individual copies of the object code with a copy of the + written offer to provide the Corresponding Source. This + alternative is allowed only occasionally and noncommercially, and + only if you received the object code with such an offer, in accord + with subsection 6b. + + d) Convey the object code by offering access from a designated + place (gratis or for a charge), and offer equivalent access to the + Corresponding Source in the same way through the same place at no + further charge. You need not require recipients to copy the + Corresponding Source along with the object code. If the place to + copy the object code is a network server, the Corresponding Source + may be on a different server (operated by you or a third party) + that supports equivalent copying facilities, provided you maintain + clear directions next to the object code saying where to find the + Corresponding Source. Regardless of what server hosts the + Corresponding Source, you remain obligated to ensure that it is + available for as long as needed to satisfy these requirements. + + e) Convey the object code using peer-to-peer transmission, provided + you inform other peers where the object code and Corresponding + Source of the work are being offered to the general public at no + charge under subsection 6d. + + A separable portion of the object code, whose source code is excluded +from the Corresponding Source as a System Library, need not be +included in conveying the object code work. + + A "User Product" is either (1) a "consumer product", which means any +tangible personal property which is normally used for personal, +family, or household purposes, or (2) anything designed or sold for +incorporation into a dwelling. In determining whether a product is a +consumer product, doubtful cases shall be resolved in favor of +coverage. For a particular product received by a particular user, +"normally used" refers to a typical or common use of that class of +product, regardless of the status of the particular user or of the way +in which the particular user actually uses, or expects or is expected +to use, the product. A product is a consumer product regardless of +whether the product has substantial commercial, industrial or +non-consumer uses, unless such uses represent the only significant +mode of use of the product. + + "Installation Information" for a User Product means any methods, +procedures, authorization keys, or other information required to +install and execute modified versions of a covered work in that User +Product from a modified version of its Corresponding Source. The +information must suffice to ensure that the continued functioning of +the modified object code is in no case prevented or interfered with +solely because modification has been made. + + If you convey an object code work under this section in, or with, or +specifically for use in, a User Product, and the conveying occurs as +part of a transaction in which the right of possession and use of the +User Product is transferred to the recipient in perpetuity or for a +fixed term (regardless of how the transaction is characterized), the +Corresponding Source conveyed under this section must be accompanied +by the Installation Information. But this requirement does not apply +if neither you nor any third party retains the ability to install +modified object code on the User Product (for example, the work has +been installed in ROM). + + The requirement to provide Installation Information does not include +a requirement to continue to provide support service, warranty, or +updates for a work that has been modified or installed by the +recipient, or for the User Product in which it has been modified or +installed. Access to a network may be denied when the modification +itself materially and adversely affects the operation of the network +or violates the rules and protocols for communication across the +network. + + Corresponding Source conveyed, and Installation Information provided, +in accord with this section must be in a format that is publicly +documented (and with an implementation available to the public in +source code form), and must require no special password or key for +unpacking, reading or copying. + + 7. Additional Terms. + + "Additional permissions" are terms that supplement the terms of this +License by making exceptions from one or more of its conditions. +Additional permissions that are applicable to the entire Program shall +be treated as though they were included in this License, to the extent +that they are valid under applicable law. If additional permissions +apply only to part of the Program, that part may be used separately +under those permissions, but the entire Program remains governed by +this License without regard to the additional permissions. + + When you convey a copy of a covered work, you may at your option +remove any additional permissions from that copy, or from any part of +it. (Additional permissions may be written to require their own +removal in certain cases when you modify the work.) You may place +additional permissions on material, added by you to a covered work, +for which you have or can give appropriate copyright permission. + + Notwithstanding any other provision of this License, for material +you add to a covered work, you may (if authorized by the copyright +holders of that material) supplement the terms of this License with +terms: + + a) Disclaiming warranty or limiting liability differently from the + terms of sections 15 and 16 of this License; or + + b) Requiring preservation of specified reasonable legal notices or + author attributions in that material or in the Appropriate Legal + Notices displayed by works containing it; or + + c) Prohibiting misrepresentation of the origin of that material, or + requiring that modified versions of such material be marked in + reasonable ways as different from the original version; or + + d) Limiting the use for publicity purposes of names of licensors or + authors of the material; or + + e) Declining to grant rights under trademark law for use of some + trade names, trademarks, or service marks; or + + f) Requiring indemnification of licensors and authors of that + material by anyone who conveys the material (or modified versions + of it) with contractual assumptions of liability to the recipient, + for any liability that these contractual assumptions directly + impose on those licensors and authors. + + All other non-permissive additional terms are considered "further +restrictions" within the meaning of section 10. If the Program as you +received it, or any part of it, contains a notice stating that it is +governed by this License along with a term that is a further +restriction, you may remove that term. If a license document contains +a further restriction but permits relicensing or conveying under this +License, you may add to a covered work material governed by the terms +of that license document, provided that the further restriction does +not survive such relicensing or conveying. + + If you add terms to a covered work in accord with this section, you +must place, in the relevant source files, a statement of the +additional terms that apply to those files, or a notice indicating +where to find the applicable terms. + + Additional terms, permissive or non-permissive, may be stated in the +form of a separately written license, or stated as exceptions; +the above requirements apply either way. + + 8. Termination. + + You may not propagate or modify a covered work except as expressly +provided under this License. Any attempt otherwise to propagate or +modify it is void, and will automatically terminate your rights under +this License (including any patent licenses granted under the third +paragraph of section 11). + + However, if you cease all violation of this License, then your +license from a particular copyright holder is reinstated (a) +provisionally, unless and until the copyright holder explicitly and +finally terminates your license, and (b) permanently, if the copyright +holder fails to notify you of the violation by some reasonable means +prior to 60 days after the cessation. + + Moreover, your license from a particular copyright holder is +reinstated permanently if the copyright holder notifies you of the +violation by some reasonable means, this is the first time you have +received notice of violation of this License (for any work) from that +copyright holder, and you cure the violation prior to 30 days after +your receipt of the notice. + + Termination of your rights under this section does not terminate the +licenses of parties who have received copies or rights from you under +this License. If your rights have been terminated and not permanently +reinstated, you do not qualify to receive new licenses for the same +material under section 10. + + 9. Acceptance Not Required for Having Copies. + + You are not required to accept this License in order to receive or +run a copy of the Program. Ancillary propagation of a covered work +occurring solely as a consequence of using peer-to-peer transmission +to receive a copy likewise does not require acceptance. However, +nothing other than this License grants you permission to propagate or +modify any covered work. These actions infringe copyright if you do +not accept this License. Therefore, by modifying or propagating a +covered work, you indicate your acceptance of this License to do so. + + 10. Automatic Licensing of Downstream Recipients. + + Each time you convey a covered work, the recipient automatically +receives a license from the original licensors, to run, modify and +propagate that work, subject to this License. You are not responsible +for enforcing compliance by third parties with this License. + + An "entity transaction" is a transaction transferring control of an +organization, or substantially all assets of one, or subdividing an +organization, or merging organizations. If propagation of a covered +work results from an entity transaction, each party to that +transaction who receives a copy of the work also receives whatever +licenses to the work the party's predecessor in interest had or could +give under the previous paragraph, plus a right to possession of the +Corresponding Source of the work from the predecessor in interest, if +the predecessor has it or can get it with reasonable efforts. + + You may not impose any further restrictions on the exercise of the +rights granted or affirmed under this License. For example, you may +not impose a license fee, royalty, or other charge for exercise of +rights granted under this License, and you may not initiate litigation +(including a cross-claim or counterclaim in a lawsuit) alleging that +any patent claim is infringed by making, using, selling, offering for +sale, or importing the Program or any portion of it. + + 11. Patents. + + A "contributor" is a copyright holder who authorizes use under this +License of the Program or a work on which the Program is based. The +work thus licensed is called the contributor's "contributor version". + + A contributor's "essential patent claims" are all patent claims +owned or controlled by the contributor, whether already acquired or +hereafter acquired, that would be infringed by some manner, permitted +by this License, of making, using, or selling its contributor version, +but do not include claims that would be infringed only as a +consequence of further modification of the contributor version. For +purposes of this definition, "control" includes the right to grant +patent sublicenses in a manner consistent with the requirements of +this License. + + Each contributor grants you a non-exclusive, worldwide, royalty-free +patent license under the contributor's essential patent claims, to +make, use, sell, offer for sale, import and otherwise run, modify and +propagate the contents of its contributor version. + + In the following three paragraphs, a "patent license" is any express +agreement or commitment, however denominated, not to enforce a patent +(such as an express permission to practice a patent or covenant not to +sue for patent infringement). To "grant" such a patent license to a +party means to make such an agreement or commitment not to enforce a +patent against the party. + + If you convey a covered work, knowingly relying on a patent license, +and the Corresponding Source of the work is not available for anyone +to copy, free of charge and under the terms of this License, through a +publicly available network server or other readily accessible means, +then you must either (1) cause the Corresponding Source to be so +available, or (2) arrange to deprive yourself of the benefit of the +patent license for this particular work, or (3) arrange, in a manner +consistent with the requirements of this License, to extend the patent +license to downstream recipients. "Knowingly relying" means you have +actual knowledge that, but for the patent license, your conveying the +covered work in a country, or your recipient's use of the covered work +in a country, would infringe one or more identifiable patents in that +country that you have reason to believe are valid. + + If, pursuant to or in connection with a single transaction or +arrangement, you convey, or propagate by procuring conveyance of, a +covered work, and grant a patent license to some of the parties +receiving the covered work authorizing them to use, propagate, modify +or convey a specific copy of the covered work, then the patent license +you grant is automatically extended to all recipients of the covered +work and works based on it. + + A patent license is "discriminatory" if it does not include within +the scope of its coverage, prohibits the exercise of, or is +conditioned on the non-exercise of one or more of the rights that are +specifically granted under this License. You may not convey a covered +work if you are a party to an arrangement with a third party that is +in the business of distributing software, under which you make payment +to the third party based on the extent of your activity of conveying +the work, and under which the third party grants, to any of the +parties who would receive the covered work from you, a discriminatory +patent license (a) in connection with copies of the covered work +conveyed by you (or copies made from those copies), or (b) primarily +for and in connection with specific products or compilations that +contain the covered work, unless you entered into that arrangement, +or that patent license was granted, prior to 28 March 2007. + + Nothing in this License shall be construed as excluding or limiting +any implied license or other defenses to infringement that may +otherwise be available to you under applicable patent law. + + 12. No Surrender of Others' Freedom. + + If conditions are imposed on you (whether by court order, agreement +or otherwise) that contradict the conditions of this License, they do +not excuse you from the conditions of this License. If you cannot +convey a covered work so as to satisfy simultaneously your obligations +under this License and any other pertinent obligations, then as a +consequence you may not convey it at all. For example, if you agree +to terms that obligate you to collect a royalty for further conveying +from those to whom you convey the Program, the only way you could +satisfy both those terms and this License would be to refrain entirely +from conveying the Program. + + 13. Use with the GNU Affero General Public License. + + Notwithstanding any other provision of this License, you have +permission to link or combine any covered work with a work licensed +under version 3 of the GNU Affero General Public License into a single +combined work, and to convey the resulting work. The terms of this +License will continue to apply to the part which is the covered work, +but the special requirements of the GNU Affero General Public License, +section 13, concerning interaction through a network will apply to the +combination as such. + + 14. Revised Versions of this License. + + The Free Software Foundation may publish revised and/or new versions +of the GNU General Public License from time to time. Such new +versions will be similar in spirit to the present version, but may +differ in detail to address new problems or concerns. + + Each version is given a distinguishing version number. If the +Program specifies that a certain numbered version of the GNU General +Public License "or any later version" applies to it, you have the +option of following the terms and conditions either of that numbered +version or of any later version published by the Free Software +Foundation. If the Program does not specify a version number of the +GNU General Public License, you may choose any version ever published +by the Free Software Foundation. + + If the Program specifies that a proxy can decide which future +versions of the GNU General Public License can be used, that proxy's +public statement of acceptance of a version permanently authorizes you +to choose that version for the Program. + + Later license versions may give you additional or different +permissions. However, no additional obligations are imposed on any +author or copyright holder as a result of your choosing to follow a +later version. + + 15. Disclaimer of Warranty. + + THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY +APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT +HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT +WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND +PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE +DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR +CORRECTION. + + 16. Limitation of Liability. + + IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN +WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES +AND/OR CONVEYS THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR +DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL +DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM +(INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED +INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF +THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER +OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. + + 17. Interpretation of Sections 15 and 16. + + If the disclaimer of warranty and limitation of liability provided +above cannot be given local legal effect according to their terms, +reviewing courts shall apply local law that most closely approximates +an absolute waiver of all civil liability in connection with the +Program, unless a warranty or assumption of liability accompanies a +copy of the Program in return for a fee. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Programs + + If you develop a new program, and you want it to be of the greatest +possible use to the public, the best way to achieve this is to make it +free software which everyone can redistribute and change under these +terms. + + To do so, attach the following notices to the program. It is safest +to attach them to the start of each source file to most effectively +state the exclusion of warranty; and each file should have at least +the "copyright" line and a pointer to where the full notice is found. + + + Copyright (C) + + This program is free software: you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation, either version 3 of the + License, or (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see + . + +Also add information on how to contact you by electronic and paper mail. + + If the program does terminal interaction, make it output a short +notice like this when it starts in an interactive mode: + + Copyright (C) This program comes + with ABSOLUTELY NO WARRANTY; for details type `show w'. This is + free software, and you are welcome to redistribute it under + certain conditions; type `show c' for details. + +The hypothetical commands `show w' and `show c' should show the +appropriate parts of the General Public License. Of course, your +program's commands might be different; for a GUI interface, you would +use an "about box". + + You should also get your employer (if you work as a programmer) or +school, if any, to sign a "copyright disclaimer" for the program, if +necessary. For more information on this, and how to apply and follow +the GNU GPL, see . + + The GNU General Public License does not permit incorporating your +program into proprietary programs. If your program is a subroutine +library, you may consider it more useful to permit linking proprietary +applications with the library. If this is what you want to do, use +the GNU Lesser General Public License instead of this License. But +first, please read . + + +---------------------------------------------------------------------- +Following applies to: +./test/metis_support.cpp +./test/sparselu.cpp +./unsupported/test/mpreal/mpreal.h +./unsupported/Eigen/src/IterativeSolvers/IterationController.h +./unsupported/Eigen/src/IterativeSolvers/ConstrainedConjGrad.h +./unsupported/Eigen/src/Eigenvalues/ArpackSelfAdjointEigenSolver.h +./Eigen/src/OrderingMethods/Amd.h +./Eigen/src/SparseCholesky/SimplicialCholesky_impl.h + + GNU LESSER GENERAL PUBLIC LICENSE + Version 3, 29 June 2007 + + Copyright (C) 2007 Free Software Foundation, Inc. + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + + This version of the GNU Lesser General Public License incorporates +the terms and conditions of version 3 of the GNU General Public +License, supplemented by the additional permissions listed below. + + 0. Additional Definitions. + + As used herein, "this License" refers to version 3 of the GNU Lesser +General Public License, and the "GNU GPL" refers to version 3 of the +GNU General Public License. + + "The Library" refers to a covered work governed by this License, +other than an Application or a Combined Work as defined below. + + An "Application" is any work that makes use of an interface provided +by the Library, but which is not otherwise based on the Library. +Defining a subclass of a class defined by the Library is deemed a mode +of using an interface provided by the Library. + + A "Combined Work" is a work produced by combining or linking an +Application with the Library. The particular version of the Library +with which the Combined Work was made is also called the "Linked +Version". + + The "Minimal Corresponding Source" for a Combined Work means the +Corresponding Source for the Combined Work, excluding any source code +for portions of the Combined Work that, considered in isolation, are +based on the Application, and not on the Linked Version. + + The "Corresponding Application Code" for a Combined Work means the +object code and/or source code for the Application, including any data +and utility programs needed for reproducing the Combined Work from the +Application, but excluding the System Libraries of the Combined Work. + + 1. Exception to Section 3 of the GNU GPL. + + You may convey a covered work under sections 3 and 4 of this License +without being bound by section 3 of the GNU GPL. + + 2. Conveying Modified Versions. + + If you modify a copy of the Library, and, in your modifications, a +facility refers to a function or data to be supplied by an Application +that uses the facility (other than as an argument passed when the +facility is invoked), then you may convey a copy of the modified +version: + + a) under this License, provided that you make a good faith effort to + ensure that, in the event an Application does not supply the + function or data, the facility still operates, and performs + whatever part of its purpose remains meaningful, or + + b) under the GNU GPL, with none of the additional permissions of + this License applicable to that copy. + + 3. Object Code Incorporating Material from Library Header Files. + + The object code form of an Application may incorporate material from +a header file that is part of the Library. You may convey such object +code under terms of your choice, provided that, if the incorporated +material is not limited to numerical parameters, data structure +layouts and accessors, or small macros, inline functions and templates +(ten or fewer lines in length), you do both of the following: + + a) Give prominent notice with each copy of the object code that the + Library is used in it and that the Library and its use are + covered by this License. + + b) Accompany the object code with a copy of the GNU GPL and this + license document. + + 4. Combined Works. + + You may convey a Combined Work under terms of your choice that, +taken together, effectively do not restrict modification of the +portions of the Library contained in the Combined Work and reverse +engineering for debugging such modifications, if you also do each of +the following: + + a) Give prominent notice with each copy of the Combined Work that + the Library is used in it and that the Library and its use are + covered by this License. + + b) Accompany the Combined Work with a copy of the GNU GPL and this + license document. + + c) For a Combined Work that displays copyright notices during + execution, include the copyright notice for the Library among + these notices, as well as a reference directing the user to the + copies of the GNU GPL and this license document. + + d) Do one of the following: + + 0) Convey the Minimal Corresponding Source under the terms of + this License, and the Corresponding Application Code in a form + suitable for, and under terms that permit, the user to + recombine or relink the Application with a modified version of + the Linked Version to produce a modified Combined Work, in the + manner specified by section 6 of the GNU GPL for conveying + Corresponding Source. + + 1) Use a suitable shared library mechanism for linking with the + Library. A suitable mechanism is one that (a) uses at run time + a copy of the Library already present on the user's computer + system, and (b) will operate properly with a modified version + of the Library that is interface-compatible with the Linked + Version. + + e) Provide Installation Information, but only if you would otherwise + be required to provide such information under section 6 of the + GNU GPL, and only to the extent that such information is + necessary to install and execute a modified version of the + Combined Work produced by recombining or relinking the + Application with a modified version of the Linked Version. (If + you use option 4d0, the Installation Information must accompany + the Minimal Corresponding Source and Corresponding Application + Code. If you use option 4d1, you must provide the Installation + Information in the manner specified by section 6 of the GNU GPL + for conveying Corresponding Source.) + + 5. Combined Libraries. + + You may place library facilities that are a work based on the +Library side by side in a single library together with other library +facilities that are not Applications and are not covered by this +License, and convey such a combined library under terms of your +choice, if you do both of the following: + + a) Accompany the combined library with a copy of the same work based + on the Library, uncombined with any other library facilities, + conveyed under the terms of this License. + + b) Give prominent notice with the combined library that part of it + is a work based on the Library, and explaining where to find the + accompanying uncombined form of the same work. + + 6. Revised Versions of the GNU Lesser General Public License. + + The Free Software Foundation may publish revised and/or new versions +of the GNU Lesser General Public License from time to time. Such new +versions will be similar in spirit to the present version, but may +differ in detail to address new problems or concerns. + + Each version is given a distinguishing version number. If the +Library as you received it specifies that a certain numbered version +of the GNU Lesser General Public License "or any later version" +applies to it, you have the option of following the terms and +conditions either of that published version or of any later version +published by the Free Software Foundation. If the Library as you +received it does not specify a version number of the GNU Lesser +General Public License, you may choose any version of the GNU Lesser +General Public License ever published by the Free Software Foundation. + + If the Library as you received it specifies that a proxy can decide +whether future versions of the GNU Lesser General Public License shall +apply, that proxy's public statement of acceptance of any version is +permanent authorization for you to choose that version for the +Library. + + +---------------------------------------------------------------------- +Following applies to: +./unsupported/Eigen/src/LevenbergMarquardt/LevenbergMarquardt.h +./unsupported/Eigen/src/LevenbergMarquardt/LMcovar.h +./unsupported/Eigen/src/LevenbergMarquardt/LMonestep.h +./unsupported/Eigen/src/LevenbergMarquardt/LMpar.h +./unsupported/Eigen/src/LevenbergMarquardt/LMqrsolv.h + +Minpack Copyright Notice (1999) University of Chicago. All rights +reserved + +Redistribution and use in source and binary forms, with or +without modification, are permitted provided that the +following conditions are met: + +1. Redistributions of source code must retain the above +copyright notice, this list of conditions and the following +disclaimer. + +2. Redistributions in binary form must reproduce the above +copyright notice, this list of conditions and the following +disclaimer in the documentation and/or other materials +provided with the distribution. + +3. The end-user documentation included with the +redistribution, if any, must include the following +acknowledgment: + + "This product includes software developed by the + University of Chicago, as Operator of Argonne National + Laboratory. + +Alternately, this acknowledgment may appear in the software +itself, if and wherever such third-party acknowledgments +normally appear. + +4. WARRANTY DISCLAIMER. THE SOFTWARE IS SUPPLIED "AS IS" +WITHOUT WARRANTY OF ANY KIND. THE COPYRIGHT HOLDER, THE +UNITED STATES, THE UNITED STATES DEPARTMENT OF ENERGY, AND +THEIR EMPLOYEES: (1) DISCLAIM ANY WARRANTIES, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO ANY IMPLIED WARRANTIES +OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, TITLE +OR NON-INFRINGEMENT, (2) DO NOT ASSUME ANY LEGAL LIABILITY +OR RESPONSIBILITY FOR THE ACCURACY, COMPLETENESS, OR +USEFULNESS OF THE SOFTWARE, (3) DO NOT REPRESENT THAT USE OF +THE SOFTWARE WOULD NOT INFRINGE PRIVATELY OWNED RIGHTS, (4) +DO NOT WARRANT THAT THE SOFTWARE WILL FUNCTION +UNINTERRUPTED, THAT IT IS ERROR-FREE OR THAT ANY ERRORS WILL +BE CORRECTED. + +5. LIMITATION OF LIABILITY. IN NO EVENT WILL THE COPYRIGHT +HOLDER, THE UNITED STATES, THE UNITED STATES DEPARTMENT OF +ENERGY, OR THEIR EMPLOYEES: BE LIABLE FOR ANY INDIRECT, +INCIDENTAL, CONSEQUENTIAL, SPECIAL OR PUNITIVE DAMAGES OF +ANY KIND OR NATURE, INCLUDING BUT NOT LIMITED TO LOSS OF +PROFITS OR LOSS OF DATA, FOR ANY REASON WHATSOEVER, WHETHER +SUCH LIABILITY IS ASSERTED ON THE BASIS OF CONTRACT, TORT +(INCLUDING NEGLIGENCE OR STRICT LIABILITY), OR OTHERWISE, +EVEN IF ANY OF SAID PARTIES HAS BEEN WARNED OF THE +POSSIBILITY OF SUCH LOSS OR DAMAGES. +-------------------------------------------------------------------------------- +END LICENSE FOR third_party/eigen3/LICENSE +-------------------------------------------------------------------------------- +-------------------------------------------------------------------------------- +BEGIN LICENSE FOR third_party/fft2d/LICENSE +-------------------------------------------------------------------------------- +Copyright(C) 1997,2001 Takuya OOURA (email: ooura@kurims.kyoto-u.ac.jp). +You may use, copy, modify this code for any purpose and +without fee. You may distribute this ORIGINAL package. +-------------------------------------------------------------------------------- +END LICENSE FOR third_party/fft2d/LICENSE +-------------------------------------------------------------------------------- +-------------------------------------------------------------------------------- +BEGIN LICENSE FOR external/boringssl/LICENSE +-------------------------------------------------------------------------------- +BoringSSL is a fork of OpenSSL. As such, large parts of it fall under OpenSSL +licensing. Files that are completely new have a Google copyright and an ISC +license. This license is reproduced at the bottom of this file. + +Contributors to BoringSSL are required to follow the CLA rules for Chromium: +https://cla.developers.google.com/clas + +Files in third_party/ have their own licenses, as described therein. The MIT +license, for third_party/fiat, which, unlike other third_party directories, is +compiled into non-test libraries, is included below. + +The OpenSSL toolkit stays under a dual license, i.e. both the conditions of the +OpenSSL License and the original SSLeay license apply to the toolkit. See below +for the actual license texts. Actually both licenses are BSD-style Open Source +licenses. In case of any license issues related to OpenSSL please contact +openssl-core@openssl.org. + +The following are Google-internal bug numbers where explicit permission from +some authors is recorded for use of their work. (This is purely for our own +record keeping.) + 27287199 + 27287880 + 27287883 + + OpenSSL License + --------------- + +/* ==================================================================== + * Copyright (c) 1998-2011 The OpenSSL Project. All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * + * 3. All advertising materials mentioning features or use of this + * software must display the following acknowledgment: + * "This product includes software developed by the OpenSSL Project + * for use in the OpenSSL Toolkit. (http://www.openssl.org/)" + * + * 4. The names "OpenSSL Toolkit" and "OpenSSL Project" must not be used to + * endorse or promote products derived from this software without + * prior written permission. For written permission, please contact + * openssl-core@openssl.org. + * + * 5. Products derived from this software may not be called "OpenSSL" + * nor may "OpenSSL" appear in their names without prior written + * permission of the OpenSSL Project. + * + * 6. Redistributions of any form whatsoever must retain the following + * acknowledgment: + * "This product includes software developed by the OpenSSL Project + * for use in the OpenSSL Toolkit (http://www.openssl.org/)" + * + * THIS SOFTWARE IS PROVIDED BY THE OpenSSL PROJECT ``AS IS'' AND ANY + * EXPRESSED OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR + * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE OpenSSL PROJECT OR + * ITS CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT + * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; + * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, + * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) + * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED + * OF THE POSSIBILITY OF SUCH DAMAGE. + * ==================================================================== + * + * This product includes cryptographic software written by Eric Young + * (eay@cryptsoft.com). This product includes software written by Tim + * Hudson (tjh@cryptsoft.com). + * + */ + + Original SSLeay License + ----------------------- + +/* Copyright (C) 1995-1998 Eric Young (eay@cryptsoft.com) + * All rights reserved. + * + * This package is an SSL implementation written + * by Eric Young (eay@cryptsoft.com). + * The implementation was written so as to conform with Netscapes SSL. + * + * This library is free for commercial and non-commercial use as long as + * the following conditions are aheared to. The following conditions + * apply to all code found in this distribution, be it the RC4, RSA, + * lhash, DES, etc., code; not just the SSL code. The SSL documentation + * included with this distribution is covered by the same copyright terms + * except that the holder is Tim Hudson (tjh@cryptsoft.com). + * + * Copyright remains Eric Young's, and as such any Copyright notices in + * the code are not to be removed. + * If this package is used in a product, Eric Young should be given attribution + * as the author of the parts of the library used. + * This can be in the form of a textual message at program startup or + * in documentation (online or textual) provided with the package. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. All advertising materials mentioning features or use of this software + * must display the following acknowledgement: + * "This product includes cryptographic software written by + * Eric Young (eay@cryptsoft.com)" + * The word 'cryptographic' can be left out if the rouines from the library + * being used are not cryptographic related :-). + * 4. If you include any Windows specific code (or a derivative thereof) from + * the apps directory (application code) you must include an acknowledgement: + * "This product includes software written by Tim Hudson (tjh@cryptsoft.com)" + * + * THIS SOFTWARE IS PROVIDED BY ERIC YOUNG ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * The licence and distribution terms for any publically available version or + * derivative of this code cannot be changed. i.e. this code cannot simply be + * copied and put under another distribution licence + * [including the GNU Public Licence.] + */ + + +ISC license used for completely new code in BoringSSL: + +/* Copyright (c) 2015, Google Inc. + * + * Permission to use, copy, modify, and/or distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY + * SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION + * OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN + * CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. */ + + +The code in third_party/fiat carries the MIT license: + +Copyright (c) 2015-2016 the fiat-crypto authors (see +https://github.com/mit-plv/fiat-crypto/blob/master/AUTHORS). + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. + + +Licenses for support code +------------------------- + +Parts of the TLS test suite are under the Go license. This code is not included +in BoringSSL (i.e. libcrypto and libssl) when compiled, however, so +distributing code linked against BoringSSL does not trigger this license: + +Copyright (c) 2009 The Go Authors. All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are +met: + + * Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above +copyright notice, this list of conditions and the following disclaimer +in the documentation and/or other materials provided with the +distribution. + * Neither the name of Google Inc. nor the names of its +contributors may be used to endorse or promote products derived from +this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + + +BoringSSL uses the Chromium test infrastructure to run a continuous build, +trybots etc. The scripts which manage this, and the script for generating build +metadata, are under the Chromium license. Distributing code linked against +BoringSSL does not trigger this license. + +Copyright 2015 The Chromium Authors. All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are +met: + + * Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above +copyright notice, this list of conditions and the following disclaimer +in the documentation and/or other materials provided with the +distribution. + * Neither the name of Google Inc. nor the names of its +contributors may be used to endorse or promote products derived from +this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +-------------------------------------------------------------------------------- +END LICENSE FOR external/boringssl/LICENSE +-------------------------------------------------------------------------------- +-------------------------------------------------------------------------------- +BEGIN LICENSE FOR external/com_googlesource_code_re2/LICENSE +-------------------------------------------------------------------------------- +// Copyright (c) 2009 The RE2 Authors. All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +-------------------------------------------------------------------------------- +END LICENSE FOR external/com_googlesource_code_re2/LICENSE +-------------------------------------------------------------------------------- +-------------------------------------------------------------------------------- +BEGIN LICENSE FOR external/curl/COPYING +-------------------------------------------------------------------------------- +COPYRIGHT AND PERMISSION NOTICE + +Copyright (c) 1996 - 2018, Daniel Stenberg, , and many +contributors, see the THANKS file. + +All rights reserved. + +Permission to use, copy, modify, and distribute this software for any purpose +with or without fee is hereby granted, provided that the above copyright +notice and this permission notice appear in all copies. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OF THIRD PARTY RIGHTS. IN +NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, +DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR +OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE +OR OTHER DEALINGS IN THE SOFTWARE. + +Except as contained in this notice, the name of a copyright holder shall not +be used in advertising or otherwise to promote the sale, use or other dealings +in this Software without prior written authorization of the copyright holder. +-------------------------------------------------------------------------------- +END LICENSE FOR external/curl/COPYING +-------------------------------------------------------------------------------- +-------------------------------------------------------------------------------- +BEGIN LICENSE FOR external/double_conversion/LICENSE +-------------------------------------------------------------------------------- +Copyright 2006-2011, the V8 project authors. All rights reserved. +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are +met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above + copyright notice, this list of conditions and the following + disclaimer in the documentation and/or other materials provided + with the distribution. + * Neither the name of Google Inc. nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +-------------------------------------------------------------------------------- +END LICENSE FOR external/double_conversion/LICENSE +-------------------------------------------------------------------------------- +-------------------------------------------------------------------------------- +BEGIN LICENSE FOR external/eigen_archive/COPYING.MPL2 +-------------------------------------------------------------------------------- +Mozilla Public License Version 2.0 +================================== + +1. Definitions +-------------- + +1.1. "Contributor" + means each individual or legal entity that creates, contributes to + the creation of, or owns Covered Software. + +1.2. "Contributor Version" + means the combination of the Contributions of others (if any) used + by a Contributor and that particular Contributor's Contribution. + +1.3. "Contribution" + means Covered Software of a particular Contributor. + +1.4. "Covered Software" + means Source Code Form to which the initial Contributor has attached + the notice in Exhibit A, the Executable Form of such Source Code + Form, and Modifications of such Source Code Form, in each case + including portions thereof. + +1.5. "Incompatible With Secondary Licenses" + means + + (a) that the initial Contributor has attached the notice described + in Exhibit B to the Covered Software; or + + (b) that the Covered Software was made available under the terms of + version 1.1 or earlier of the License, but not also under the + terms of a Secondary License. + +1.6. "Executable Form" + means any form of the work other than Source Code Form. + +1.7. "Larger Work" + means a work that combines Covered Software with other material, in + a separate file or files, that is not Covered Software. + +1.8. "License" + means this document. + +1.9. "Licensable" + means having the right to grant, to the maximum extent possible, + whether at the time of the initial grant or subsequently, any and + all of the rights conveyed by this License. + +1.10. "Modifications" + means any of the following: + + (a) any file in Source Code Form that results from an addition to, + deletion from, or modification of the contents of Covered + Software; or + + (b) any new file in Source Code Form that contains any Covered + Software. + +1.11. "Patent Claims" of a Contributor + means any patent claim(s), including without limitation, method, + process, and apparatus claims, in any patent Licensable by such + Contributor that would be infringed, but for the grant of the + License, by the making, using, selling, offering for sale, having + made, import, or transfer of either its Contributions or its + Contributor Version. + +1.12. "Secondary License" + means either the GNU General Public License, Version 2.0, the GNU + Lesser General Public License, Version 2.1, the GNU Affero General + Public License, Version 3.0, or any later versions of those + licenses. + +1.13. "Source Code Form" + means the form of the work preferred for making modifications. + +1.14. "You" (or "Your") + means an individual or a legal entity exercising rights under this + License. For legal entities, "You" includes any entity that + controls, is controlled by, or is under common control with You. For + purposes of this definition, "control" means (a) the power, direct + or indirect, to cause the direction or management of such entity, + whether by contract or otherwise, or (b) ownership of more than + fifty percent (50%) of the outstanding shares or beneficial + ownership of such entity. + +2. License Grants and Conditions +-------------------------------- + +2.1. Grants + +Each Contributor hereby grants You a world-wide, royalty-free, +non-exclusive license: + +(a) under intellectual property rights (other than patent or trademark) + Licensable by such Contributor to use, reproduce, make available, + modify, display, perform, distribute, and otherwise exploit its + Contributions, either on an unmodified basis, with Modifications, or + as part of a Larger Work; and + +(b) under Patent Claims of such Contributor to make, use, sell, offer + for sale, have made, import, and otherwise transfer either its + Contributions or its Contributor Version. + +2.2. Effective Date + +The licenses granted in Section 2.1 with respect to any Contribution +become effective for each Contribution on the date the Contributor first +distributes such Contribution. + +2.3. Limitations on Grant Scope + +The licenses granted in this Section 2 are the only rights granted under +this License. No additional rights or licenses will be implied from the +distribution or licensing of Covered Software under this License. +Notwithstanding Section 2.1(b) above, no patent license is granted by a +Contributor: + +(a) for any code that a Contributor has removed from Covered Software; + or + +(b) for infringements caused by: (i) Your and any other third party's + modifications of Covered Software, or (ii) the combination of its + Contributions with other software (except as part of its Contributor + Version); or + +(c) under Patent Claims infringed by Covered Software in the absence of + its Contributions. + +This License does not grant any rights in the trademarks, service marks, +or logos of any Contributor (except as may be necessary to comply with +the notice requirements in Section 3.4). + +2.4. Subsequent Licenses + +No Contributor makes additional grants as a result of Your choice to +distribute the Covered Software under a subsequent version of this +License (see Section 10.2) or under the terms of a Secondary License (if +permitted under the terms of Section 3.3). + +2.5. Representation + +Each Contributor represents that the Contributor believes its +Contributions are its original creation(s) or it has sufficient rights +to grant the rights to its Contributions conveyed by this License. + +2.6. Fair Use + +This License is not intended to limit any rights You have under +applicable copyright doctrines of fair use, fair dealing, or other +equivalents. + +2.7. Conditions + +Sections 3.1, 3.2, 3.3, and 3.4 are conditions of the licenses granted +in Section 2.1. + +3. Responsibilities +------------------- + +3.1. Distribution of Source Form + +All distribution of Covered Software in Source Code Form, including any +Modifications that You create or to which You contribute, must be under +the terms of this License. You must inform recipients that the Source +Code Form of the Covered Software is governed by the terms of this +License, and how they can obtain a copy of this License. You may not +attempt to alter or restrict the recipients' rights in the Source Code +Form. + +3.2. Distribution of Executable Form + +If You distribute Covered Software in Executable Form then: + +(a) such Covered Software must also be made available in Source Code + Form, as described in Section 3.1, and You must inform recipients of + the Executable Form how they can obtain a copy of such Source Code + Form by reasonable means in a timely manner, at a charge no more + than the cost of distribution to the recipient; and + +(b) You may distribute such Executable Form under the terms of this + License, or sublicense it under different terms, provided that the + license for the Executable Form does not attempt to limit or alter + the recipients' rights in the Source Code Form under this License. + +3.3. Distribution of a Larger Work + +You may create and distribute a Larger Work under terms of Your choice, +provided that You also comply with the requirements of this License for +the Covered Software. If the Larger Work is a combination of Covered +Software with a work governed by one or more Secondary Licenses, and the +Covered Software is not Incompatible With Secondary Licenses, this +License permits You to additionally distribute such Covered Software +under the terms of such Secondary License(s), so that the recipient of +the Larger Work may, at their option, further distribute the Covered +Software under the terms of either this License or such Secondary +License(s). + +3.4. Notices + +You may not remove or alter the substance of any license notices +(including copyright notices, patent notices, disclaimers of warranty, +or limitations of liability) contained within the Source Code Form of +the Covered Software, except that You may alter any license notices to +the extent required to remedy known factual inaccuracies. + +3.5. Application of Additional Terms + +You may choose to offer, and to charge a fee for, warranty, support, +indemnity or liability obligations to one or more recipients of Covered +Software. However, You may do so only on Your own behalf, and not on +behalf of any Contributor. You must make it absolutely clear that any +such warranty, support, indemnity, or liability obligation is offered by +You alone, and You hereby agree to indemnify every Contributor for any +liability incurred by such Contributor as a result of warranty, support, +indemnity or liability terms You offer. You may include additional +disclaimers of warranty and limitations of liability specific to any +jurisdiction. + +4. Inability to Comply Due to Statute or Regulation +--------------------------------------------------- + +If it is impossible for You to comply with any of the terms of this +License with respect to some or all of the Covered Software due to +statute, judicial order, or regulation then You must: (a) comply with +the terms of this License to the maximum extent possible; and (b) +describe the limitations and the code they affect. Such description must +be placed in a text file included with all distributions of the Covered +Software under this License. Except to the extent prohibited by statute +or regulation, such description must be sufficiently detailed for a +recipient of ordinary skill to be able to understand it. + +5. Termination +-------------- + +5.1. The rights granted under this License will terminate automatically +if You fail to comply with any of its terms. However, if You become +compliant, then the rights granted under this License from a particular +Contributor are reinstated (a) provisionally, unless and until such +Contributor explicitly and finally terminates Your grants, and (b) on an +ongoing basis, if such Contributor fails to notify You of the +non-compliance by some reasonable means prior to 60 days after You have +come back into compliance. Moreover, Your grants from a particular +Contributor are reinstated on an ongoing basis if such Contributor +notifies You of the non-compliance by some reasonable means, this is the +first time You have received notice of non-compliance with this License +from such Contributor, and You become compliant prior to 30 days after +Your receipt of the notice. + +5.2. If You initiate litigation against any entity by asserting a patent +infringement claim (excluding declaratory judgment actions, +counter-claims, and cross-claims) alleging that a Contributor Version +directly or indirectly infringes any patent, then the rights granted to +You by any and all Contributors for the Covered Software under Section +2.1 of this License shall terminate. + +5.3. In the event of termination under Sections 5.1 or 5.2 above, all +end user license agreements (excluding distributors and resellers) which +have been validly granted by You or Your distributors under this License +prior to termination shall survive termination. + +************************************************************************ +* * +* 6. Disclaimer of Warranty * +* ------------------------- * +* * +* Covered Software is provided under this License on an "as is" * +* basis, without warranty of any kind, either expressed, implied, or * +* statutory, including, without limitation, warranties that the * +* Covered Software is free of defects, merchantable, fit for a * +* particular purpose or non-infringing. The entire risk as to the * +* quality and performance of the Covered Software is with You. * +* Should any Covered Software prove defective in any respect, You * +* (not any Contributor) assume the cost of any necessary servicing, * +* repair, or correction. This disclaimer of warranty constitutes an * +* essential part of this License. No use of any Covered Software is * +* authorized under this License except under this disclaimer. * +* * +************************************************************************ + +************************************************************************ +* * +* 7. Limitation of Liability * +* -------------------------- * +* * +* Under no circumstances and under no legal theory, whether tort * +* (including negligence), contract, or otherwise, shall any * +* Contributor, or anyone who distributes Covered Software as * +* permitted above, be liable to You for any direct, indirect, * +* special, incidental, or consequential damages of any character * +* including, without limitation, damages for lost profits, loss of * +* goodwill, work stoppage, computer failure or malfunction, or any * +* and all other commercial damages or losses, even if such party * +* shall have been informed of the possibility of such damages. This * +* limitation of liability shall not apply to liability for death or * +* personal injury resulting from such party's negligence to the * +* extent applicable law prohibits such limitation. Some * +* jurisdictions do not allow the exclusion or limitation of * +* incidental or consequential damages, so this exclusion and * +* limitation may not apply to You. * +* * +************************************************************************ + +8. Litigation +------------- + +Any litigation relating to this License may be brought only in the +courts of a jurisdiction where the defendant maintains its principal +place of business and such litigation shall be governed by laws of that +jurisdiction, without reference to its conflict-of-law provisions. +Nothing in this Section shall prevent a party's ability to bring +cross-claims or counter-claims. + +9. Miscellaneous +---------------- + +This License represents the complete agreement concerning the subject +matter hereof. If any provision of this License is held to be +unenforceable, such provision shall be reformed only to the extent +necessary to make it enforceable. Any law or regulation which provides +that the language of a contract shall be construed against the drafter +shall not be used to construe this License against a Contributor. + +10. Versions of the License +--------------------------- + +10.1. New Versions + +Mozilla Foundation is the license steward. Except as provided in Section +10.3, no one other than the license steward has the right to modify or +publish new versions of this License. Each version will be given a +distinguishing version number. + +10.2. Effect of New Versions + +You may distribute the Covered Software under the terms of the version +of the License under which You originally received the Covered Software, +or under the terms of any subsequent version published by the license +steward. + +10.3. Modified Versions + +If you create software not governed by this License, and you want to +create a new license for such software, you may create and use a +modified version of this License if you rename the license and remove +any references to the name of the license steward (except to note that +such modified license differs from this License). + +10.4. Distributing Source Code Form that is Incompatible With Secondary +Licenses + +If You choose to distribute Source Code Form that is Incompatible With +Secondary Licenses under the terms of this version of the License, the +notice described in Exhibit B of this License must be attached. + +Exhibit A - Source Code Form License Notice +------------------------------------------- + + This Source Code Form is subject to the terms of the Mozilla Public + License, v. 2.0. If a copy of the MPL was not distributed with this + file, You can obtain one at http://mozilla.org/MPL/2.0/. + +If it is not possible or desirable to put the notice in a particular +file, then You may include the notice in a location (such as a LICENSE +file in a relevant directory) where a recipient would be likely to look +for such a notice. + +You may add additional accurate notices of copyright ownership. + +Exhibit B - "Incompatible With Secondary Licenses" Notice +--------------------------------------------------------- + + This Source Code Form is "Incompatible With Secondary Licenses", as + defined by the Mozilla Public License, v. 2.0. +-------------------------------------------------------------------------------- +END LICENSE FOR external/eigen_archive/COPYING.MPL2 +-------------------------------------------------------------------------------- +-------------------------------------------------------------------------------- +BEGIN LICENSE FOR external/farmhash_archive/COPYING +-------------------------------------------------------------------------------- +// Copyright (c) 2014 Google, Inc. +// +// Permission is hereby granted, free of charge, to any person obtaining a copy +// of this software and associated documentation files (the "Software"), to deal +// in the Software without restriction, including without limitation the rights +// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +// copies of the Software, and to permit persons to whom the Software is +// furnished to do so, subject to the following conditions: +// +// The above copyright notice and this permission notice shall be included in +// all copies or substantial portions of the Software. +// +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +// THE SOFTWARE. +-------------------------------------------------------------------------------- +END LICENSE FOR external/farmhash_archive/COPYING +-------------------------------------------------------------------------------- +-------------------------------------------------------------------------------- +BEGIN LICENSE FOR external/fft2d/fft/readme.txt +-------------------------------------------------------------------------------- +General Purpose FFT (Fast Fourier/Cosine/Sine Transform) Package + +Description: + A package to calculate Discrete Fourier/Cosine/Sine Transforms of + 1-dimensional sequences of length 2^N. + +Files: + fft4g.c : FFT Package in C - Fast Version I (radix 4,2) + fft4g.f : FFT Package in Fortran - Fast Version I (radix 4,2) + fft4g_h.c : FFT Package in C - Simple Version I (radix 4,2) + fft8g.c : FFT Package in C - Fast Version II (radix 8,4,2) + fft8g.f : FFT Package in Fortran - Fast Version II (radix 8,4,2) + fft8g_h.c : FFT Package in C - Simple Version II (radix 8,4,2) + fftsg.c : FFT Package in C - Fast Version III (Split-Radix) + fftsg.f : FFT Package in Fortran - Fast Version III (Split-Radix) + fftsg_h.c : FFT Package in C - Simple Version III (Split-Radix) + readme.txt : Readme File + sample1/ : Test Directory + Makefile : for gcc, cc + Makefile.f77: for Fortran + testxg.c : Test Program for "fft*g.c" + testxg.f : Test Program for "fft*g.f" + testxg_h.c : Test Program for "fft*g_h.c" + sample2/ : Benchmark Directory + Makefile : for gcc, cc + Makefile.pth: POSIX Thread version + pi_fft.c : PI(= 3.1415926535897932384626...) Calculation Program + for a Benchmark Test for "fft*g.c" + +Difference of the Files: + C and Fortran versions are equal and + the same routines are in each version. + "fft4g*.*" are optimized for most machines. + "fft8g*.*" are fast on the UltraSPARC. + "fftsg*.*" are optimized for the machines that + have the multi-level (L1,L2,etc) cache. + The simple versions "fft*g_h.c" use no work area, but + the fast versions "fft*g.*" use work areas. + The fast versions "fft*g.*" have the same specification. + +Routines in the Package: + cdft: Complex Discrete Fourier Transform + rdft: Real Discrete Fourier Transform + ddct: Discrete Cosine Transform + ddst: Discrete Sine Transform + dfct: Cosine Transform of RDFT (Real Symmetric DFT) + dfst: Sine Transform of RDFT (Real Anti-symmetric DFT) + +Usage: + Please refer to the comments in the "fft**.*" file which + you want to use. Brief explanations are in the block + comments of each package. The examples are also given in + the test programs. + +Method: + -------- cdft -------- + fft4g*.*, fft8g*.*: + A method of in-place, radix 2^M, Sande-Tukey (decimation in + frequency). Index of the butterfly loop is in bit + reverse order to keep continuous memory access. + fftsg*.*: + A method of in-place, Split-Radix, recursive fast + algorithm. + -------- rdft -------- + A method with a following butterfly operation appended to "cdft". + In forward transform : + A[k] = sum_j=0^n-1 a[j]*W(n)^(j*k), 0<=k<=n/2, + W(n) = exp(2*pi*i/n), + this routine makes an array x[] : + x[j] = a[2*j] + i*a[2*j+1], 0<=j \. All Rights Reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + +- Redistributions of source code must retain the above copyright notice, + this list of conditions and the following disclaimer. +- Redistributions in binary form must reproduce the above copyright notice, + this list of conditions and the following disclaimer in the documentation + and/or other materials provided with the distribution. +- Neither the name of the libjpeg-turbo Project nor the names of its + contributors may be used to endorse or promote products derived from this + software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS", +AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDERS OR CONTRIBUTORS BE +LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR +CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN +CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) +ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE +POSSIBILITY OF SUCH DAMAGE. + + +The zlib License +================ + +Copyright (C) \, \. + +This software is provided 'as-is', without any express or implied +warranty. In no event will the authors be held liable for any damages +arising from the use of this software. + +Permission is granted to anyone to use this software for any purpose, +including commercial applications, and to alter it and redistribute it +freely, subject to the following restrictions: + +1. The origin of this software must not be misrepresented; you must not + claim that you wrote the original software. If you use this software + in a product, an acknowledgment in the product documentation would be + appreciated but is not required. +2. Altered source versions must be plainly marked as such, and must not be + misrepresented as being the original software. +3. This notice may not be removed or altered from any source distribution. + + +Why Three Licenses? +=================== + +The zlib License could have been used instead of the Modified (3-clause) BSD +License, and since the IJG License effectively subsumes the distribution +conditions of the zlib License, this would have effectively placed +libjpeg-turbo binary distributions under the IJG License. However, the IJG +License specifically refers to the Independent JPEG Group and does not extend +attribution and endorsement protections to other entities. Thus, it was +desirable to choose a license that granted us the same protections for new code +that were granted to the IJG for code derived from their software. +-------------------------------------------------------------------------------- +END LICENSE FOR external/jpeg/LICENSE.md +-------------------------------------------------------------------------------- +-------------------------------------------------------------------------------- +BEGIN LICENSE FOR external/llvm/LICENSE.TXT +-------------------------------------------------------------------------------- +============================================================================== +LLVM Release License +============================================================================== +University of Illinois/NCSA +Open Source License + +Copyright (c) 2003-2018 University of Illinois at Urbana-Champaign. +All rights reserved. + +Developed by: + + LLVM Team + + University of Illinois at Urbana-Champaign + + http://llvm.org + +Permission is hereby granted, free of charge, to any person obtaining a copy of +this software and associated documentation files (the "Software"), to deal with +the Software without restriction, including without limitation the rights to +use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies +of the Software, and to permit persons to whom the Software is furnished to do +so, subject to the following conditions: + + * Redistributions of source code must retain the above copyright notice, + this list of conditions and the following disclaimers. + + * Redistributions in binary form must reproduce the above copyright notice, + this list of conditions and the following disclaimers in the + documentation and/or other materials provided with the distribution. + + * Neither the names of the LLVM Team, University of Illinois at + Urbana-Champaign, nor the names of its contributors may be used to + endorse or promote products derived from this Software without specific + prior written permission. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS +FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +CONTRIBUTORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS WITH THE +SOFTWARE. + +============================================================================== +Copyrights and Licenses for Third Party Software Distributed with LLVM: +============================================================================== +The LLVM software contains code written by third parties. Such software will +have its own individual LICENSE.TXT file in the directory in which it appears. +This file will describe the copyrights, license, and restrictions which apply +to that code. + +The disclaimer of warranty in the University of Illinois Open Source License +applies to all code in the LLVM Distribution, and nothing in any of the +other licenses gives permission to use the names of the LLVM Team or the +University of Illinois to endorse or promote products derived from this +Software. + +The following pieces of software have additional or alternate copyrights, +licenses, and/or restrictions: + +Program Directory +------- --------- +Google Test llvm/utils/unittest/googletest +OpenBSD regex llvm/lib/Support/{reg*, COPYRIGHT.regex} +pyyaml tests llvm/test/YAMLParser/{*.data, LICENSE.TXT} +ARM contributions llvm/lib/Target/ARM/LICENSE.TXT +md5 contributions llvm/lib/Support/MD5.cpp llvm/include/llvm/Support/MD5.h +-------------------------------------------------------------------------------- +END LICENSE FOR external/llvm/LICENSE.TXT +-------------------------------------------------------------------------------- +-------------------------------------------------------------------------------- +BEGIN LICENSE FOR external/lmdb/LICENSE +-------------------------------------------------------------------------------- +The OpenLDAP Public License + Version 2.8, 17 August 2003 + +Redistribution and use of this software and associated documentation +("Software"), with or without modification, are permitted provided +that the following conditions are met: + +1. Redistributions in source form must retain copyright statements + and notices, + +2. Redistributions in binary form must reproduce applicable copyright + statements and notices, this list of conditions, and the following + disclaimer in the documentation and/or other materials provided + with the distribution, and + +3. Redistributions must contain a verbatim copy of this document. + +The OpenLDAP Foundation may revise this license from time to time. +Each revision is distinguished by a version number. You may use +this Software under terms of this license revision or under the +terms of any subsequent revision of the license. + +THIS SOFTWARE IS PROVIDED BY THE OPENLDAP FOUNDATION AND ITS +CONTRIBUTORS ``AS IS'' AND ANY EXPRESSED OR IMPLIED WARRANTIES, +INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY +AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT +SHALL THE OPENLDAP FOUNDATION, ITS CONTRIBUTORS, OR THE AUTHOR(S) +OR OWNER(S) OF THE SOFTWARE BE LIABLE FOR ANY DIRECT, INDIRECT, +INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, +BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; +LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER +CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN +ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE +POSSIBILITY OF SUCH DAMAGE. + +The names of the authors and copyright holders must not be used in +advertising or otherwise to promote the sale, use or other dealing +in this Software without specific, written prior permission. Title +to copyright in this Software shall at all times remain with copyright +holders. + +OpenLDAP is a registered trademark of the OpenLDAP Foundation. + +Copyright 1999-2003 The OpenLDAP Foundation, Redwood City, +California, USA. All Rights Reserved. Permission to copy and +distribute verbatim copies of this document is granted. +-------------------------------------------------------------------------------- +END LICENSE FOR external/lmdb/LICENSE +-------------------------------------------------------------------------------- +-------------------------------------------------------------------------------- +BEGIN LICENSE FOR external/local_config_sycl/sycl/LICENSE.text +-------------------------------------------------------------------------------- + +--------------------------------------------------------------------- + +SOFTWARE LICENSE AGREEMENT + +--------------------------------------------------------------------- +--------------------------------------------------------------------- + +By downloading, installing, copying, or otherwise using the +ComputeCpp Community Edition software, including any associated +components, media, printed materials, and electronic documentation +("Software"), the user agrees to the following terms and conditions +of this Software License Agreement ("Agreement"). Please read the +terms of this Agreement carefully before beginning your download, as +pressing the "I AGREE" button at the end of this Agreement will +confirm your assent. If you do not agree to these terms, then +Codeplay Software Limited is unwilling to license the Software to +you; so please press the "CANCEL" button to cancel your download. + + 1. License. Codeplay Software Ltd., a company incorporated in + England and Wales with registered number 04567874 and having its + registered office at Regent House, 316 Beulah Hill, London, + United Kingdom, SE19 3HF ("Codeplay") hereby grants the user, + free of charge, a non-exclusive worldwide license to use and + replicate (but not modify) the Software for any use, whether + commercial or non-commercial, in accordance with this Agreement. + Codeplay reserves all rights to the Software that are not + expressly granted by this Agreement. + 2. Redistribution. The user may copy and redistribute unmodified + copies of only those components of the Software which are + specified below ("Redistributable Components"), in object code + form, as part of the user’s software applications or libraries + ("Applications"). The user acknowledges and agrees that it has no + right to modify the Redistributable Components in any way. Any + use of the Redistributable Components within the user’s + Applications will continue to be subject to the terms and + conditions of this Agreement, and the user must also distribute a + copy of this Agreement and reproduce and include all notices of + copyrights or other proprietary rights in the Software. The + user’s redistribution of the Redistributable Components will not + entitle it to any payment from Codeplay. The user may not + transfer any of its rights or obligations under this Agreement. + ++-------------------------------------------+ +|Redistributable Component|File Name | +|-------------------------+-----------------| +|Runtime (for Linux) |libComputeCpp.so | +|-------------------------+-----------------| +|Runtime (for Windows) |libComputeCpp.dll| ++-------------------------------------------+ + + 3. Restrictions. The user shall not: + + a. circumvent or bypass any technological protection measures in + or relating to the Software; + b. use the Software to perform any unauthorized transfer of + information or for any illegal purpose; + c. de-compile, decrypt, disassemble, hack, emulate, exploit or + reverse-engineer the Software (other than to the limited + extent permitted by law); + d. copy or redistribute any components of the Software that are + not listed in the table of Redistributable Components; + e. publish, rent, lease, sell, export, import, or lend the + Software; + f. represent in any way that it is selling the Software itself + or any license to use the Software, nor refer to Codeplay or + ComputeCpp within its marketing materials, without the + express prior written permission of Codeplay. + 4. Support. Codeplay does not provide any guarantees of support for + the Software to the user. Codeplay will use reasonable endeavors + to respond to users' support requests, for the most recent + release only, via the community support website at https:// + computecpp.codeplay.com. + 5. Intellectual Property. The Software is owned by Codeplay or its + licensors, and is protected by the copyright laws of the United + Kingdom and other countries and international treaty provisions. + Codeplay (and/or its licensors, as the case may be) retains all + copyrights, trade secrets and other proprietary rights in the + Software, including the rights to make and license the use of all + copies. To the extent that any patents owned by Codeplay or its + licensors relate to any component of the Software, the license + granted to the user in accordance with this Agreement allows for + the lawful use of such patents but only for the purposes of this + Agreement and not further or otherwise. Therefore, the user may + make no copies of the Software, or the written materials that + accompany the Software, or reproduce it in any way, except as set + forth above. + 6. Terms. This Agreement is effective until terminated. Codeplay or + the user may terminate it immediately at any time. Any violation + of the terms of this Agreement by the user will result in + immediate termination by Codeplay. Upon termination, the user + must return or destroy the Software and accompanying materials + and notify Codeplay of its actions by email to info@codeplay.com. + 7. NO WARRANTIES. Codeplay expressly disclaims any warranty for the + Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF + ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE + WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE + AND NON-INFRINGEMENT. IN NO EVENT SHALL CODEPLAY BE LIABLE FOR + ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF + CONTRACT, DELICT OR TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN + CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + SOFTWARE. In particular, Codeplay provides no guarantees of + application performance on the target hardware. + 8. General. The invalidity of any portion or provision of this + Agreement shall not affect any other portions or provisions. This + Agreement shall be governed by the laws of Scotland. This + Agreement is the complete and exclusive agreement between the + user and Codeplay regarding the Software, and it supersedes any + prior agreement, oral or written, and any other communication + between the user and Codeplay relating to the subject matter of + the Agreement. Any amendment or modification of this Agreement + must be in writing and signed by both parties. If the user does + not agree to the terms of this Agreement, the user must not + install or use the Software. + 9. Third Party Licenses. The following licenses are for third-party + components included in the software. + + a. License for Clang/LLVM compiler technology components: + +============================================================================== + +LLVM Release License + +============================================================================== + +University of Illinois/NCSA + +Open Source License + +Copyright (c) 2007-2014 University of Illinois at Urbana-Champaign. + +All rights reserved. + +Developed by: + + LLVM Team + + University of Illinois at Urbana-Champaign + + http://llvm.org + +Permission is hereby granted, free of charge, to any person obtaining a copy of +this software and associated documentation files (the "Software"), to deal with +the Software without restriction, including without limitation the rights to +use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies +of the Software, and to permit persons to whom the Software is furnished to do +so, subject to the following conditions: + + * Redistributions of source code must retain the above copyright notice, + this list of conditions and the following disclaimers. + + * Redistributions in binary form must reproduce the above copyright notice, + this list of conditions and the following disclaimers in the + documentation and/or other materials provided with the distribution. + + * Neither the names of the LLVM Team, University of Illinois at + Urbana-Champaign, nor the names of its contributors may be used to + endorse or promote products derived from this Software without specific + prior written permission. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS +FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +CONTRIBUTORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS WITH THE +SOFTWARE. + +============================================================================== + + b. License for OpenBSD regex components: + +$OpenBSD: COPYRIGHT,v 1.3 2003/06/02 20:18:36 millert Exp $ +Copyright 1992, 1993, 1994 Henry Spencer. All rights reserved. +This software is not subject to any license of the American Telephone +and Telegraph Company or of the Regents of the University of California. +Permission is granted to anyone to use this software for any purpose on +any computer system, and to alter it and redistribute it, subject +to the following restrictions: + +1. The author is not responsible for the consequences of use of this + software, no matter how awful, even if they arise from flaws in it. + +2. The origin of this software must not be misrepresented, either by + explicit claim or by omission. Since few users ever read sources, + credits must appear in the documentation. + +3. Altered versions must be plainly marked as such, and must not be + misrepresented as being the original software. Since few users + ever read sources, credits must appear in the documentation. + +4. This notice may not be removed or altered. + +=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= + +/*- + * Copyright (c) 1994 + * The Regents of the University of California. All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. Neither the name of the University nor the names of its contributors + * may be used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * @(#)COPYRIGHT8.1 (Berkeley) 3/16/94 + */ + + c. License for MD5 components: + +/* + * This code is derived from (original license follows): + * + * This is an OpenSSL-compatible implementation of the RSA Data Security, Inc. + * MD5 Message-Digest Algorithm (RFC 1321). + * + * Homepage: + * http://openwall.info/wiki/people/solar/software/public-domain-source-code/md5 + * + * Author: + * Alexander Peslyak, better known as Solar Designer + * + * This software was written by Alexander Peslyak in 2001. No copyright is + * claimed, and the software is hereby placed in the public domain. + * In case this attempt to disclaim copyright and place the software in the + * public domain is deemed null and void, then the software is + * Copyright (c) 2001 Alexander Peslyak and it is hereby released to the + * general public under the following terms: + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted. + * + * There's ABSOLUTELY NO WARRANTY, express or implied. + * + * (This is a heavily cut-down "BSD license".) + * + * This differs from Colin Plumb's older public domain implementation in that + * no exactly 32-bit integer data type is required (any 32-bit or wider + * unsigned integer data type will do), there's no compile-time endianness + * configuration, and the function prototypes match OpenSSL's. No code from + * Colin Plumb's implementation has been reused; this comment merely compares + * the properties of the two independent implementations. + * + * The primary goals of this implementation are portability and ease of use. + * It is meant to be fast, but not as fast as possible. Some known + * optimizations are not included to reduce source code size and avoid + * compile-time configuration. + */ + + +-------------------------------------------------------------------------------- +END LICENSE FOR external/local_config_sycl/sycl/LICENSE.text +-------------------------------------------------------------------------------- +-------------------------------------------------------------------------------- +BEGIN LICENSE FOR external/nasm/LICENSE +-------------------------------------------------------------------------------- +NASM is now licensed under the 2-clause BSD license, also known as the +simplified BSD license. + + Copyright 1996-2010 the NASM Authors - All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following + conditions are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above + copyright notice, this list of conditions and the following + disclaimer in the documentation and/or other materials provided + with the distribution. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND + CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, + INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF + MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE + DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR + CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT + NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; + LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN + CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR + OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, + EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +-------------------------------------------------------------------------------- +END LICENSE FOR external/nasm/LICENSE +-------------------------------------------------------------------------------- +-------------------------------------------------------------------------------- +BEGIN LICENSE FOR external/nsync/LICENSE +-------------------------------------------------------------------------------- + + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +-------------------------------------------------------------------------------- +END LICENSE FOR external/nsync/LICENSE +-------------------------------------------------------------------------------- +-------------------------------------------------------------------------------- +BEGIN LICENSE FOR external/png_archive/LICENSE +-------------------------------------------------------------------------------- + +This copy of the libpng notices is provided for your convenience. In case of +any discrepancy between this copy and the notices in the file png.h that is +included in the libpng distribution, the latter shall prevail. + +COPYRIGHT NOTICE, DISCLAIMER, and LICENSE: + +If you modify libpng you may insert additional notices immediately following +this sentence. + +This code is released under the libpng license. + +libpng versions 1.0.7, July 1, 2000 through 1.6.34, September 29, 2017 are +Copyright (c) 2000-2002, 2004, 2006-2017 Glenn Randers-Pehrson, are +derived from libpng-1.0.6, and are distributed according to the same +disclaimer and license as libpng-1.0.6 with the following individuals +added to the list of Contributing Authors: + + Simon-Pierre Cadieux + Eric S. Raymond + Mans Rullgard + Cosmin Truta + Gilles Vollant + James Yu + Mandar Sahastrabuddhe + Google Inc. + Vadim Barkov + +and with the following additions to the disclaimer: + + There is no warranty against interference with your enjoyment of the + library or against infringement. There is no warranty that our + efforts or the library will fulfill any of your particular purposes + or needs. This library is provided with all faults, and the entire + risk of satisfactory quality, performance, accuracy, and effort is with + the user. + +Some files in the "contrib" directory and some configure-generated +files that are distributed with libpng have other copyright owners and +are released under other open source licenses. + +libpng versions 0.97, January 1998, through 1.0.6, March 20, 2000, are +Copyright (c) 1998-2000 Glenn Randers-Pehrson, are derived from +libpng-0.96, and are distributed according to the same disclaimer and +license as libpng-0.96, with the following individuals added to the list +of Contributing Authors: + + Tom Lane + Glenn Randers-Pehrson + Willem van Schaik + +libpng versions 0.89, June 1996, through 0.96, May 1997, are +Copyright (c) 1996-1997 Andreas Dilger, are derived from libpng-0.88, +and are distributed according to the same disclaimer and license as +libpng-0.88, with the following individuals added to the list of +Contributing Authors: + + John Bowler + Kevin Bracey + Sam Bushell + Magnus Holmgren + Greg Roelofs + Tom Tanner + +Some files in the "scripts" directory have other copyright owners +but are released under this license. + +libpng versions 0.5, May 1995, through 0.88, January 1996, are +Copyright (c) 1995-1996 Guy Eric Schalnat, Group 42, Inc. + +For the purposes of this copyright and license, "Contributing Authors" +is defined as the following set of individuals: + + Andreas Dilger + Dave Martindale + Guy Eric Schalnat + Paul Schmidt + Tim Wegner + +The PNG Reference Library is supplied "AS IS". The Contributing Authors +and Group 42, Inc. disclaim all warranties, expressed or implied, +including, without limitation, the warranties of merchantability and of +fitness for any purpose. The Contributing Authors and Group 42, Inc. +assume no liability for direct, indirect, incidental, special, exemplary, +or consequential damages, which may result from the use of the PNG +Reference Library, even if advised of the possibility of such damage. + +Permission is hereby granted to use, copy, modify, and distribute this +source code, or portions hereof, for any purpose, without fee, subject +to the following restrictions: + + 1. The origin of this source code must not be misrepresented. + + 2. Altered versions must be plainly marked as such and must not + be misrepresented as being the original source. + + 3. This Copyright notice may not be removed or altered from any + source or altered source distribution. + +The Contributing Authors and Group 42, Inc. specifically permit, without +fee, and encourage the use of this source code as a component to +supporting the PNG file format in commercial products. If you use this +source code in a product, acknowledgment is not required but would be +appreciated. + +END OF COPYRIGHT NOTICE, DISCLAIMER, and LICENSE. + +TRADEMARK: + +The name "libpng" has not been registered by the Copyright owner +as a trademark in any jurisdiction. However, because libpng has +been distributed and maintained world-wide, continually since 1995, +the Copyright owner claims "common-law trademark protection" in any +jurisdiction where common-law trademark is recognized. + +OSI CERTIFICATION: + +Libpng is OSI Certified Open Source Software. OSI Certified Open Source is +a certification mark of the Open Source Initiative. OSI has not addressed +the additional disclaimers inserted at version 1.0.7. + +EXPORT CONTROL: + +The Copyright owner believes that the Export Control Classification +Number (ECCN) for libpng is EAR99, which means not subject to export +controls or International Traffic in Arms Regulations (ITAR) because +it is open source, publicly available software, that does not contain +any encryption software. See the EAR, paragraphs 734.3(b)(3) and +734.7(b). + +Glenn Randers-Pehrson +glennrp at users.sourceforge.net +September 29, 2017 +-------------------------------------------------------------------------------- +END LICENSE FOR external/png_archive/LICENSE +-------------------------------------------------------------------------------- +-------------------------------------------------------------------------------- +BEGIN LICENSE FOR external/protobuf_archive/LICENSE +-------------------------------------------------------------------------------- +Copyright 2008 Google Inc. All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are +met: + + * Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above +copyright notice, this list of conditions and the following disclaimer +in the documentation and/or other materials provided with the +distribution. + * Neither the name of Google Inc. nor the names of its +contributors may be used to endorse or promote products derived from +this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +Code generated by the Protocol Buffer compiler is owned by the owner +of the input file used when generating it. This code is not +standalone and requires a support library to be linked with it. This +support library is itself covered by the above license. +-------------------------------------------------------------------------------- +END LICENSE FOR external/protobuf_archive/LICENSE +-------------------------------------------------------------------------------- +-------------------------------------------------------------------------------- +BEGIN LICENSE FOR external/snappy/COPYING +-------------------------------------------------------------------------------- +Copyright 2011, Google Inc. +All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are +met: + + * Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above +copyright notice, this list of conditions and the following disclaimer +in the documentation and/or other materials provided with the +distribution. + * Neither the name of Google Inc. nor the names of its +contributors may be used to endorse or promote products derived from +this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +=== + +Some of the benchmark data in testdata/ is licensed differently: + + - fireworks.jpeg is Copyright 2013 Steinar H. Gunderson, and + is licensed under the Creative Commons Attribution 3.0 license + (CC-BY-3.0). See https://creativecommons.org/licenses/by/3.0/ + for more information. + + - kppkn.gtb is taken from the Gaviota chess tablebase set, and + is licensed under the MIT License. See + https://sites.google.com/site/gaviotachessengine/Home/endgame-tablebases-1 + for more information. + + - paper-100k.pdf is an excerpt (bytes 92160 to 194560) from the paper + “Combinatorial Modeling of Chromatin Features Quantitatively Predicts DNA + Replication Timing in _Drosophila_” by Federico Comoglio and Renato Paro, + which is licensed under the CC-BY license. See + http://www.ploscompbiol.org/static/license for more ifnormation. + + - alice29.txt, asyoulik.txt, plrabn12.txt and lcet10.txt are from Project + Gutenberg. The first three have expired copyrights and are in the public + domain; the latter does not have expired copyright, but is still in the + public domain according to the license information + (http://www.gutenberg.org/ebooks/53). +-------------------------------------------------------------------------------- +END LICENSE FOR external/snappy/COPYING +-------------------------------------------------------------------------------- +-------------------------------------------------------------------------------- +BEGIN LICENSE FOR external/zlib_archive/zlib.h +-------------------------------------------------------------------------------- +/* zlib.h -- interface of the 'zlib' general purpose compression library + version 1.2.11, January 15th, 2017 + + Copyright (C) 1995-2017 Jean-loup Gailly and Mark Adler + + This software is provided 'as-is', without any express or implied + warranty. In no event will the authors be held liable for any damages + arising from the use of this software. + + Permission is granted to anyone to use this software for any purpose, + including commercial applications, and to alter it and redistribute it + freely, subject to the following restrictions: + + 1. The origin of this software must not be misrepresented; you must not + claim that you wrote the original software. If you use this software + in a product, an acknowledgment in the product documentation would be + appreciated but is not required. + 2. Altered source versions must be plainly marked as such, and must not be + misrepresented as being the original software. + 3. This notice may not be removed or altered from any source distribution. + + Jean-loup Gailly Mark Adler + jloup@gzip.org madler@alumni.caltech.edu + + + The data format used by the zlib library is described by RFCs (Request for + Comments) 1950 to 1952 in the files http://tools.ietf.org/html/rfc1950 + (zlib format), rfc1951 (deflate format) and rfc1952 (gzip format). +*/ + +#ifndef ZLIB_H +#define ZLIB_H + +#include "zconf.h" + +#ifdef __cplusplus +extern "C" { +#endif + +#define ZLIB_VERSION "1.2.11" +#define ZLIB_VERNUM 0x12b0 +#define ZLIB_VER_MAJOR 1 +#define ZLIB_VER_MINOR 2 +#define ZLIB_VER_REVISION 11 +#define ZLIB_VER_SUBREVISION 0 + +/* + The 'zlib' compression library provides in-memory compression and + decompression functions, including integrity checks of the uncompressed data. + This version of the library supports only one compression method (deflation) + but other algorithms will be added later and will have the same stream + interface. + + Compression can be done in a single step if the buffers are large enough, + or can be done by repeated calls of the compression function. In the latter + case, the application must provide more input and/or consume the output + (providing more output space) before each call. + + The compressed data format used by default by the in-memory functions is + the zlib format, which is a zlib wrapper documented in RFC 1950, wrapped + around a deflate stream, which is itself documented in RFC 1951. + + The library also supports reading and writing files in gzip (.gz) format + with an interface similar to that of stdio using the functions that start + with "gz". The gzip format is different from the zlib format. gzip is a + gzip wrapper, documented in RFC 1952, wrapped around a deflate stream. + + This library can optionally read and write gzip and raw deflate streams in + memory as well. + + The zlib format was designed to be compact and fast for use in memory + and on communications channels. The gzip format was designed for single- + file compression on file systems, has a larger header than zlib to maintain + directory information, and uses a different, slower check method than zlib. + + The library does not install any signal handler. The decoder checks + the consistency of the compressed data, so the library should never crash + even in the case of corrupted input. +*/ + +typedef voidpf (*alloc_func) OF((voidpf opaque, uInt items, uInt size)); +typedef void (*free_func) OF((voidpf opaque, voidpf address)); + +struct internal_state; + +typedef struct z_stream_s { + z_const Bytef *next_in; /* next input byte */ + uInt avail_in; /* number of bytes available at next_in */ + uLong total_in; /* total number of input bytes read so far */ + + Bytef *next_out; /* next output byte will go here */ + uInt avail_out; /* remaining free space at next_out */ + uLong total_out; /* total number of bytes output so far */ + + z_const char *msg; /* last error message, NULL if no error */ + struct internal_state FAR *state; /* not visible by applications */ + + alloc_func zalloc; /* used to allocate the internal state */ + free_func zfree; /* used to free the internal state */ + voidpf opaque; /* private data object passed to zalloc and zfree */ + + int data_type; /* best guess about the data type: binary or text + for deflate, or the decoding state for inflate */ + uLong adler; /* Adler-32 or CRC-32 value of the uncompressed data */ + uLong reserved; /* reserved for future use */ +} z_stream; + +typedef z_stream FAR *z_streamp; + +/* + gzip header information passed to and from zlib routines. See RFC 1952 + for more details on the meanings of these fields. +*/ +typedef struct gz_header_s { + int text; /* true if compressed data believed to be text */ + uLong time; /* modification time */ + int xflags; /* extra flags (not used when writing a gzip file) */ + int os; /* operating system */ + Bytef *extra; /* pointer to extra field or Z_NULL if none */ + uInt extra_len; /* extra field length (valid if extra != Z_NULL) */ + uInt extra_max; /* space at extra (only when reading header) */ + Bytef *name; /* pointer to zero-terminated file name or Z_NULL */ + uInt name_max; /* space at name (only when reading header) */ + Bytef *comment; /* pointer to zero-terminated comment or Z_NULL */ + uInt comm_max; /* space at comment (only when reading header) */ + int hcrc; /* true if there was or will be a header crc */ + int done; /* true when done reading gzip header (not used + when writing a gzip file) */ +} gz_header; + +typedef gz_header FAR *gz_headerp; + +/* + The application must update next_in and avail_in when avail_in has dropped + to zero. It must update next_out and avail_out when avail_out has dropped + to zero. The application must initialize zalloc, zfree and opaque before + calling the init function. All other fields are set by the compression + library and must not be updated by the application. + + The opaque value provided by the application will be passed as the first + parameter for calls of zalloc and zfree. This can be useful for custom + memory management. The compression library attaches no meaning to the + opaque value. + + zalloc must return Z_NULL if there is not enough memory for the object. + If zlib is used in a multi-threaded application, zalloc and zfree must be + thread safe. In that case, zlib is thread-safe. When zalloc and zfree are + Z_NULL on entry to the initialization function, they are set to internal + routines that use the standard library functions malloc() and free(). + + On 16-bit systems, the functions zalloc and zfree must be able to allocate + exactly 65536 bytes, but will not be required to allocate more than this if + the symbol MAXSEG_64K is defined (see zconf.h). WARNING: On MSDOS, pointers + returned by zalloc for objects of exactly 65536 bytes *must* have their + offset normalized to zero. The default allocation function provided by this + library ensures this (see zutil.c). To reduce memory requirements and avoid + any allocation of 64K objects, at the expense of compression ratio, compile + the library with -DMAX_WBITS=14 (see zconf.h). + + The fields total_in and total_out can be used for statistics or progress + reports. After compression, total_in holds the total size of the + uncompressed data and may be saved for use by the decompressor (particularly + if the decompressor wants to decompress everything in a single step). +*/ + + /* constants */ + +#define Z_NO_FLUSH 0 +#define Z_PARTIAL_FLUSH 1 +#define Z_SYNC_FLUSH 2 +#define Z_FULL_FLUSH 3 +#define Z_FINISH 4 +#define Z_BLOCK 5 +#define Z_TREES 6 +/* Allowed flush values; see deflate() and inflate() below for details */ + +#define Z_OK 0 +#define Z_STREAM_END 1 +#define Z_NEED_DICT 2 +#define Z_ERRNO (-1) +#define Z_STREAM_ERROR (-2) +#define Z_DATA_ERROR (-3) +#define Z_MEM_ERROR (-4) +#define Z_BUF_ERROR (-5) +#define Z_VERSION_ERROR (-6) +/* Return codes for the compression/decompression functions. Negative values + * are errors, positive values are used for special but normal events. + */ + +#define Z_NO_COMPRESSION 0 +#define Z_BEST_SPEED 1 +#define Z_BEST_COMPRESSION 9 +#define Z_DEFAULT_COMPRESSION (-1) +/* compression levels */ + +#define Z_FILTERED 1 +#define Z_HUFFMAN_ONLY 2 +#define Z_RLE 3 +#define Z_FIXED 4 +#define Z_DEFAULT_STRATEGY 0 +/* compression strategy; see deflateInit2() below for details */ + +#define Z_BINARY 0 +#define Z_TEXT 1 +#define Z_ASCII Z_TEXT /* for compatibility with 1.2.2 and earlier */ +#define Z_UNKNOWN 2 +/* Possible values of the data_type field for deflate() */ + +#define Z_DEFLATED 8 +/* The deflate compression method (the only one supported in this version) */ + +#define Z_NULL 0 /* for initializing zalloc, zfree, opaque */ + +#define zlib_version zlibVersion() +/* for compatibility with versions < 1.0.2 */ + + + /* basic functions */ + +ZEXTERN const char * ZEXPORT zlibVersion OF((void)); +/* The application can compare zlibVersion and ZLIB_VERSION for consistency. + If the first character differs, the library code actually used is not + compatible with the zlib.h header file used by the application. This check + is automatically made by deflateInit and inflateInit. + */ + +/* +ZEXTERN int ZEXPORT deflateInit OF((z_streamp strm, int level)); + + Initializes the internal stream state for compression. The fields + zalloc, zfree and opaque must be initialized before by the caller. If + zalloc and zfree are set to Z_NULL, deflateInit updates them to use default + allocation functions. + + The compression level must be Z_DEFAULT_COMPRESSION, or between 0 and 9: + 1 gives best speed, 9 gives best compression, 0 gives no compression at all + (the input data is simply copied a block at a time). Z_DEFAULT_COMPRESSION + requests a default compromise between speed and compression (currently + equivalent to level 6). + + deflateInit returns Z_OK if success, Z_MEM_ERROR if there was not enough + memory, Z_STREAM_ERROR if level is not a valid compression level, or + Z_VERSION_ERROR if the zlib library version (zlib_version) is incompatible + with the version assumed by the caller (ZLIB_VERSION). msg is set to null + if there is no error message. deflateInit does not perform any compression: + this will be done by deflate(). +*/ + + +ZEXTERN int ZEXPORT deflate OF((z_streamp strm, int flush)); +/* + deflate compresses as much data as possible, and stops when the input + buffer becomes empty or the output buffer becomes full. It may introduce + some output latency (reading input without producing any output) except when + forced to flush. + + The detailed semantics are as follows. deflate performs one or both of the + following actions: + + - Compress more input starting at next_in and update next_in and avail_in + accordingly. If not all input can be processed (because there is not + enough room in the output buffer), next_in and avail_in are updated and + processing will resume at this point for the next call of deflate(). + + - Generate more output starting at next_out and update next_out and avail_out + accordingly. This action is forced if the parameter flush is non zero. + Forcing flush frequently degrades the compression ratio, so this parameter + should be set only when necessary. Some output may be provided even if + flush is zero. + + Before the call of deflate(), the application should ensure that at least + one of the actions is possible, by providing more input and/or consuming more + output, and updating avail_in or avail_out accordingly; avail_out should + never be zero before the call. The application can consume the compressed + output when it wants, for example when the output buffer is full (avail_out + == 0), or after each call of deflate(). If deflate returns Z_OK and with + zero avail_out, it must be called again after making room in the output + buffer because there might be more output pending. See deflatePending(), + which can be used if desired to determine whether or not there is more ouput + in that case. + + Normally the parameter flush is set to Z_NO_FLUSH, which allows deflate to + decide how much data to accumulate before producing output, in order to + maximize compression. + + If the parameter flush is set to Z_SYNC_FLUSH, all pending output is + flushed to the output buffer and the output is aligned on a byte boundary, so + that the decompressor can get all input data available so far. (In + particular avail_in is zero after the call if enough output space has been + provided before the call.) Flushing may degrade compression for some + compression algorithms and so it should be used only when necessary. This + completes the current deflate block and follows it with an empty stored block + that is three bits plus filler bits to the next byte, followed by four bytes + (00 00 ff ff). + + If flush is set to Z_PARTIAL_FLUSH, all pending output is flushed to the + output buffer, but the output is not aligned to a byte boundary. All of the + input data so far will be available to the decompressor, as for Z_SYNC_FLUSH. + This completes the current deflate block and follows it with an empty fixed + codes block that is 10 bits long. This assures that enough bytes are output + in order for the decompressor to finish the block before the empty fixed + codes block. + + If flush is set to Z_BLOCK, a deflate block is completed and emitted, as + for Z_SYNC_FLUSH, but the output is not aligned on a byte boundary, and up to + seven bits of the current block are held to be written as the next byte after + the next deflate block is completed. In this case, the decompressor may not + be provided enough bits at this point in order to complete decompression of + the data provided so far to the compressor. It may need to wait for the next + block to be emitted. This is for advanced applications that need to control + the emission of deflate blocks. + + If flush is set to Z_FULL_FLUSH, all output is flushed as with + Z_SYNC_FLUSH, and the compression state is reset so that decompression can + restart from this point if previous compressed data has been damaged or if + random access is desired. Using Z_FULL_FLUSH too often can seriously degrade + compression. + + If deflate returns with avail_out == 0, this function must be called again + with the same value of the flush parameter and more output space (updated + avail_out), until the flush is complete (deflate returns with non-zero + avail_out). In the case of a Z_FULL_FLUSH or Z_SYNC_FLUSH, make sure that + avail_out is greater than six to avoid repeated flush markers due to + avail_out == 0 on return. + + If the parameter flush is set to Z_FINISH, pending input is processed, + pending output is flushed and deflate returns with Z_STREAM_END if there was + enough output space. If deflate returns with Z_OK or Z_BUF_ERROR, this + function must be called again with Z_FINISH and more output space (updated + avail_out) but no more input data, until it returns with Z_STREAM_END or an + error. After deflate has returned Z_STREAM_END, the only possible operations + on the stream are deflateReset or deflateEnd. + + Z_FINISH can be used in the first deflate call after deflateInit if all the + compression is to be done in a single step. In order to complete in one + call, avail_out must be at least the value returned by deflateBound (see + below). Then deflate is guaranteed to return Z_STREAM_END. If not enough + output space is provided, deflate will not return Z_STREAM_END, and it must + be called again as described above. + + deflate() sets strm->adler to the Adler-32 checksum of all input read + so far (that is, total_in bytes). If a gzip stream is being generated, then + strm->adler will be the CRC-32 checksum of the input read so far. (See + deflateInit2 below.) + + deflate() may update strm->data_type if it can make a good guess about + the input data type (Z_BINARY or Z_TEXT). If in doubt, the data is + considered binary. This field is only for information purposes and does not + affect the compression algorithm in any manner. + + deflate() returns Z_OK if some progress has been made (more input + processed or more output produced), Z_STREAM_END if all input has been + consumed and all output has been produced (only when flush is set to + Z_FINISH), Z_STREAM_ERROR if the stream state was inconsistent (for example + if next_in or next_out was Z_NULL or the state was inadvertently written over + by the application), or Z_BUF_ERROR if no progress is possible (for example + avail_in or avail_out was zero). Note that Z_BUF_ERROR is not fatal, and + deflate() can be called again with more input and more output space to + continue compressing. +*/ + + +ZEXTERN int ZEXPORT deflateEnd OF((z_streamp strm)); +/* + All dynamically allocated data structures for this stream are freed. + This function discards any unprocessed input and does not flush any pending + output. + + deflateEnd returns Z_OK if success, Z_STREAM_ERROR if the + stream state was inconsistent, Z_DATA_ERROR if the stream was freed + prematurely (some input or output was discarded). In the error case, msg + may be set but then points to a static string (which must not be + deallocated). +*/ + + +/* +ZEXTERN int ZEXPORT inflateInit OF((z_streamp strm)); + + Initializes the internal stream state for decompression. The fields + next_in, avail_in, zalloc, zfree and opaque must be initialized before by + the caller. In the current version of inflate, the provided input is not + read or consumed. The allocation of a sliding window will be deferred to + the first call of inflate (if the decompression does not complete on the + first call). If zalloc and zfree are set to Z_NULL, inflateInit updates + them to use default allocation functions. + + inflateInit returns Z_OK if success, Z_MEM_ERROR if there was not enough + memory, Z_VERSION_ERROR if the zlib library version is incompatible with the + version assumed by the caller, or Z_STREAM_ERROR if the parameters are + invalid, such as a null pointer to the structure. msg is set to null if + there is no error message. inflateInit does not perform any decompression. + Actual decompression will be done by inflate(). So next_in, and avail_in, + next_out, and avail_out are unused and unchanged. The current + implementation of inflateInit() does not process any header information -- + that is deferred until inflate() is called. +*/ + + +ZEXTERN int ZEXPORT inflate OF((z_streamp strm, int flush)); +/* + inflate decompresses as much data as possible, and stops when the input + buffer becomes empty or the output buffer becomes full. It may introduce + some output latency (reading input without producing any output) except when + forced to flush. + + The detailed semantics are as follows. inflate performs one or both of the + following actions: + + - Decompress more input starting at next_in and update next_in and avail_in + accordingly. If not all input can be processed (because there is not + enough room in the output buffer), then next_in and avail_in are updated + accordingly, and processing will resume at this point for the next call of + inflate(). + + - Generate more output starting at next_out and update next_out and avail_out + accordingly. inflate() provides as much output as possible, until there is + no more input data or no more space in the output buffer (see below about + the flush parameter). + + Before the call of inflate(), the application should ensure that at least + one of the actions is possible, by providing more input and/or consuming more + output, and updating the next_* and avail_* values accordingly. If the + caller of inflate() does not provide both available input and available + output space, it is possible that there will be no progress made. The + application can consume the uncompressed output when it wants, for example + when the output buffer is full (avail_out == 0), or after each call of + inflate(). If inflate returns Z_OK and with zero avail_out, it must be + called again after making room in the output buffer because there might be + more output pending. + + The flush parameter of inflate() can be Z_NO_FLUSH, Z_SYNC_FLUSH, Z_FINISH, + Z_BLOCK, or Z_TREES. Z_SYNC_FLUSH requests that inflate() flush as much + output as possible to the output buffer. Z_BLOCK requests that inflate() + stop if and when it gets to the next deflate block boundary. When decoding + the zlib or gzip format, this will cause inflate() to return immediately + after the header and before the first block. When doing a raw inflate, + inflate() will go ahead and process the first block, and will return when it + gets to the end of that block, or when it runs out of data. + + The Z_BLOCK option assists in appending to or combining deflate streams. + To assist in this, on return inflate() always sets strm->data_type to the + number of unused bits in the last byte taken from strm->next_in, plus 64 if + inflate() is currently decoding the last block in the deflate stream, plus + 128 if inflate() returned immediately after decoding an end-of-block code or + decoding the complete header up to just before the first byte of the deflate + stream. The end-of-block will not be indicated until all of the uncompressed + data from that block has been written to strm->next_out. The number of + unused bits may in general be greater than seven, except when bit 7 of + data_type is set, in which case the number of unused bits will be less than + eight. data_type is set as noted here every time inflate() returns for all + flush options, and so can be used to determine the amount of currently + consumed input in bits. + + The Z_TREES option behaves as Z_BLOCK does, but it also returns when the + end of each deflate block header is reached, before any actual data in that + block is decoded. This allows the caller to determine the length of the + deflate block header for later use in random access within a deflate block. + 256 is added to the value of strm->data_type when inflate() returns + immediately after reaching the end of the deflate block header. + + inflate() should normally be called until it returns Z_STREAM_END or an + error. However if all decompression is to be performed in a single step (a + single call of inflate), the parameter flush should be set to Z_FINISH. In + this case all pending input is processed and all pending output is flushed; + avail_out must be large enough to hold all of the uncompressed data for the + operation to complete. (The size of the uncompressed data may have been + saved by the compressor for this purpose.) The use of Z_FINISH is not + required to perform an inflation in one step. However it may be used to + inform inflate that a faster approach can be used for the single inflate() + call. Z_FINISH also informs inflate to not maintain a sliding window if the + stream completes, which reduces inflate's memory footprint. If the stream + does not complete, either because not all of the stream is provided or not + enough output space is provided, then a sliding window will be allocated and + inflate() can be called again to continue the operation as if Z_NO_FLUSH had + been used. + + In this implementation, inflate() always flushes as much output as + possible to the output buffer, and always uses the faster approach on the + first call. So the effects of the flush parameter in this implementation are + on the return value of inflate() as noted below, when inflate() returns early + when Z_BLOCK or Z_TREES is used, and when inflate() avoids the allocation of + memory for a sliding window when Z_FINISH is used. + + If a preset dictionary is needed after this call (see inflateSetDictionary + below), inflate sets strm->adler to the Adler-32 checksum of the dictionary + chosen by the compressor and returns Z_NEED_DICT; otherwise it sets + strm->adler to the Adler-32 checksum of all output produced so far (that is, + total_out bytes) and returns Z_OK, Z_STREAM_END or an error code as described + below. At the end of the stream, inflate() checks that its computed Adler-32 + checksum is equal to that saved by the compressor and returns Z_STREAM_END + only if the checksum is correct. + + inflate() can decompress and check either zlib-wrapped or gzip-wrapped + deflate data. The header type is detected automatically, if requested when + initializing with inflateInit2(). Any information contained in the gzip + header is not retained unless inflateGetHeader() is used. When processing + gzip-wrapped deflate data, strm->adler32 is set to the CRC-32 of the output + produced so far. The CRC-32 is checked against the gzip trailer, as is the + uncompressed length, modulo 2^32. + + inflate() returns Z_OK if some progress has been made (more input processed + or more output produced), Z_STREAM_END if the end of the compressed data has + been reached and all uncompressed output has been produced, Z_NEED_DICT if a + preset dictionary is needed at this point, Z_DATA_ERROR if the input data was + corrupted (input stream not conforming to the zlib format or incorrect check + value, in which case strm->msg points to a string with a more specific + error), Z_STREAM_ERROR if the stream structure was inconsistent (for example + next_in or next_out was Z_NULL, or the state was inadvertently written over + by the application), Z_MEM_ERROR if there was not enough memory, Z_BUF_ERROR + if no progress was possible or if there was not enough room in the output + buffer when Z_FINISH is used. Note that Z_BUF_ERROR is not fatal, and + inflate() can be called again with more input and more output space to + continue decompressing. If Z_DATA_ERROR is returned, the application may + then call inflateSync() to look for a good compression block if a partial + recovery of the data is to be attempted. +*/ + + +ZEXTERN int ZEXPORT inflateEnd OF((z_streamp strm)); +/* + All dynamically allocated data structures for this stream are freed. + This function discards any unprocessed input and does not flush any pending + output. + + inflateEnd returns Z_OK if success, or Z_STREAM_ERROR if the stream state + was inconsistent. +*/ + + + /* Advanced functions */ + +/* + The following functions are needed only in some special applications. +*/ + +/* +ZEXTERN int ZEXPORT deflateInit2 OF((z_streamp strm, + int level, + int method, + int windowBits, + int memLevel, + int strategy)); + + This is another version of deflateInit with more compression options. The + fields next_in, zalloc, zfree and opaque must be initialized before by the + caller. + + The method parameter is the compression method. It must be Z_DEFLATED in + this version of the library. + + The windowBits parameter is the base two logarithm of the window size + (the size of the history buffer). It should be in the range 8..15 for this + version of the library. Larger values of this parameter result in better + compression at the expense of memory usage. The default value is 15 if + deflateInit is used instead. + + For the current implementation of deflate(), a windowBits value of 8 (a + window size of 256 bytes) is not supported. As a result, a request for 8 + will result in 9 (a 512-byte window). In that case, providing 8 to + inflateInit2() will result in an error when the zlib header with 9 is + checked against the initialization of inflate(). The remedy is to not use 8 + with deflateInit2() with this initialization, or at least in that case use 9 + with inflateInit2(). + + windowBits can also be -8..-15 for raw deflate. In this case, -windowBits + determines the window size. deflate() will then generate raw deflate data + with no zlib header or trailer, and will not compute a check value. + + windowBits can also be greater than 15 for optional gzip encoding. Add + 16 to windowBits to write a simple gzip header and trailer around the + compressed data instead of a zlib wrapper. The gzip header will have no + file name, no extra data, no comment, no modification time (set to zero), no + header crc, and the operating system will be set to the appropriate value, + if the operating system was determined at compile time. If a gzip stream is + being written, strm->adler is a CRC-32 instead of an Adler-32. + + For raw deflate or gzip encoding, a request for a 256-byte window is + rejected as invalid, since only the zlib header provides a means of + transmitting the window size to the decompressor. + + The memLevel parameter specifies how much memory should be allocated + for the internal compression state. memLevel=1 uses minimum memory but is + slow and reduces compression ratio; memLevel=9 uses maximum memory for + optimal speed. The default value is 8. See zconf.h for total memory usage + as a function of windowBits and memLevel. + + The strategy parameter is used to tune the compression algorithm. Use the + value Z_DEFAULT_STRATEGY for normal data, Z_FILTERED for data produced by a + filter (or predictor), Z_HUFFMAN_ONLY to force Huffman encoding only (no + string match), or Z_RLE to limit match distances to one (run-length + encoding). Filtered data consists mostly of small values with a somewhat + random distribution. In this case, the compression algorithm is tuned to + compress them better. The effect of Z_FILTERED is to force more Huffman + coding and less string matching; it is somewhat intermediate between + Z_DEFAULT_STRATEGY and Z_HUFFMAN_ONLY. Z_RLE is designed to be almost as + fast as Z_HUFFMAN_ONLY, but give better compression for PNG image data. The + strategy parameter only affects the compression ratio but not the + correctness of the compressed output even if it is not set appropriately. + Z_FIXED prevents the use of dynamic Huffman codes, allowing for a simpler + decoder for special applications. + + deflateInit2 returns Z_OK if success, Z_MEM_ERROR if there was not enough + memory, Z_STREAM_ERROR if any parameter is invalid (such as an invalid + method), or Z_VERSION_ERROR if the zlib library version (zlib_version) is + incompatible with the version assumed by the caller (ZLIB_VERSION). msg is + set to null if there is no error message. deflateInit2 does not perform any + compression: this will be done by deflate(). +*/ + +ZEXTERN int ZEXPORT deflateSetDictionary OF((z_streamp strm, + const Bytef *dictionary, + uInt dictLength)); +/* + Initializes the compression dictionary from the given byte sequence + without producing any compressed output. When using the zlib format, this + function must be called immediately after deflateInit, deflateInit2 or + deflateReset, and before any call of deflate. When doing raw deflate, this + function must be called either before any call of deflate, or immediately + after the completion of a deflate block, i.e. after all input has been + consumed and all output has been delivered when using any of the flush + options Z_BLOCK, Z_PARTIAL_FLUSH, Z_SYNC_FLUSH, or Z_FULL_FLUSH. The + compressor and decompressor must use exactly the same dictionary (see + inflateSetDictionary). + + The dictionary should consist of strings (byte sequences) that are likely + to be encountered later in the data to be compressed, with the most commonly + used strings preferably put towards the end of the dictionary. Using a + dictionary is most useful when the data to be compressed is short and can be + predicted with good accuracy; the data can then be compressed better than + with the default empty dictionary. + + Depending on the size of the compression data structures selected by + deflateInit or deflateInit2, a part of the dictionary may in effect be + discarded, for example if the dictionary is larger than the window size + provided in deflateInit or deflateInit2. Thus the strings most likely to be + useful should be put at the end of the dictionary, not at the front. In + addition, the current implementation of deflate will use at most the window + size minus 262 bytes of the provided dictionary. + + Upon return of this function, strm->adler is set to the Adler-32 value + of the dictionary; the decompressor may later use this value to determine + which dictionary has been used by the compressor. (The Adler-32 value + applies to the whole dictionary even if only a subset of the dictionary is + actually used by the compressor.) If a raw deflate was requested, then the + Adler-32 value is not computed and strm->adler is not set. + + deflateSetDictionary returns Z_OK if success, or Z_STREAM_ERROR if a + parameter is invalid (e.g. dictionary being Z_NULL) or the stream state is + inconsistent (for example if deflate has already been called for this stream + or if not at a block boundary for raw deflate). deflateSetDictionary does + not perform any compression: this will be done by deflate(). +*/ + +ZEXTERN int ZEXPORT deflateGetDictionary OF((z_streamp strm, + Bytef *dictionary, + uInt *dictLength)); +/* + Returns the sliding dictionary being maintained by deflate. dictLength is + set to the number of bytes in the dictionary, and that many bytes are copied + to dictionary. dictionary must have enough space, where 32768 bytes is + always enough. If deflateGetDictionary() is called with dictionary equal to + Z_NULL, then only the dictionary length is returned, and nothing is copied. + Similary, if dictLength is Z_NULL, then it is not set. + + deflateGetDictionary() may return a length less than the window size, even + when more than the window size in input has been provided. It may return up + to 258 bytes less in that case, due to how zlib's implementation of deflate + manages the sliding window and lookahead for matches, where matches can be + up to 258 bytes long. If the application needs the last window-size bytes of + input, then that would need to be saved by the application outside of zlib. + + deflateGetDictionary returns Z_OK on success, or Z_STREAM_ERROR if the + stream state is inconsistent. +*/ + +ZEXTERN int ZEXPORT deflateCopy OF((z_streamp dest, + z_streamp source)); +/* + Sets the destination stream as a complete copy of the source stream. + + This function can be useful when several compression strategies will be + tried, for example when there are several ways of pre-processing the input + data with a filter. The streams that will be discarded should then be freed + by calling deflateEnd. Note that deflateCopy duplicates the internal + compression state which can be quite large, so this strategy is slow and can + consume lots of memory. + + deflateCopy returns Z_OK if success, Z_MEM_ERROR if there was not + enough memory, Z_STREAM_ERROR if the source stream state was inconsistent + (such as zalloc being Z_NULL). msg is left unchanged in both source and + destination. +*/ + +ZEXTERN int ZEXPORT deflateReset OF((z_streamp strm)); +/* + This function is equivalent to deflateEnd followed by deflateInit, but + does not free and reallocate the internal compression state. The stream + will leave the compression level and any other attributes that may have been + set unchanged. + + deflateReset returns Z_OK if success, or Z_STREAM_ERROR if the source + stream state was inconsistent (such as zalloc or state being Z_NULL). +*/ + +ZEXTERN int ZEXPORT deflateParams OF((z_streamp strm, + int level, + int strategy)); +/* + Dynamically update the compression level and compression strategy. The + interpretation of level and strategy is as in deflateInit2(). This can be + used to switch between compression and straight copy of the input data, or + to switch to a different kind of input data requiring a different strategy. + If the compression approach (which is a function of the level) or the + strategy is changed, and if any input has been consumed in a previous + deflate() call, then the input available so far is compressed with the old + level and strategy using deflate(strm, Z_BLOCK). There are three approaches + for the compression levels 0, 1..3, and 4..9 respectively. The new level + and strategy will take effect at the next call of deflate(). + + If a deflate(strm, Z_BLOCK) is performed by deflateParams(), and it does + not have enough output space to complete, then the parameter change will not + take effect. In this case, deflateParams() can be called again with the + same parameters and more output space to try again. + + In order to assure a change in the parameters on the first try, the + deflate stream should be flushed using deflate() with Z_BLOCK or other flush + request until strm.avail_out is not zero, before calling deflateParams(). + Then no more input data should be provided before the deflateParams() call. + If this is done, the old level and strategy will be applied to the data + compressed before deflateParams(), and the new level and strategy will be + applied to the the data compressed after deflateParams(). + + deflateParams returns Z_OK on success, Z_STREAM_ERROR if the source stream + state was inconsistent or if a parameter was invalid, or Z_BUF_ERROR if + there was not enough output space to complete the compression of the + available input data before a change in the strategy or approach. Note that + in the case of a Z_BUF_ERROR, the parameters are not changed. A return + value of Z_BUF_ERROR is not fatal, in which case deflateParams() can be + retried with more output space. +*/ + +ZEXTERN int ZEXPORT deflateTune OF((z_streamp strm, + int good_length, + int max_lazy, + int nice_length, + int max_chain)); +/* + Fine tune deflate's internal compression parameters. This should only be + used by someone who understands the algorithm used by zlib's deflate for + searching for the best matching string, and even then only by the most + fanatic optimizer trying to squeeze out the last compressed bit for their + specific input data. Read the deflate.c source code for the meaning of the + max_lazy, good_length, nice_length, and max_chain parameters. + + deflateTune() can be called after deflateInit() or deflateInit2(), and + returns Z_OK on success, or Z_STREAM_ERROR for an invalid deflate stream. + */ + +ZEXTERN uLong ZEXPORT deflateBound OF((z_streamp strm, + uLong sourceLen)); +/* + deflateBound() returns an upper bound on the compressed size after + deflation of sourceLen bytes. It must be called after deflateInit() or + deflateInit2(), and after deflateSetHeader(), if used. This would be used + to allocate an output buffer for deflation in a single pass, and so would be + called before deflate(). If that first deflate() call is provided the + sourceLen input bytes, an output buffer allocated to the size returned by + deflateBound(), and the flush value Z_FINISH, then deflate() is guaranteed + to return Z_STREAM_END. Note that it is possible for the compressed size to + be larger than the value returned by deflateBound() if flush options other + than Z_FINISH or Z_NO_FLUSH are used. +*/ + +ZEXTERN int ZEXPORT deflatePending OF((z_streamp strm, + unsigned *pending, + int *bits)); +/* + deflatePending() returns the number of bytes and bits of output that have + been generated, but not yet provided in the available output. The bytes not + provided would be due to the available output space having being consumed. + The number of bits of output not provided are between 0 and 7, where they + await more bits to join them in order to fill out a full byte. If pending + or bits are Z_NULL, then those values are not set. + + deflatePending returns Z_OK if success, or Z_STREAM_ERROR if the source + stream state was inconsistent. + */ + +ZEXTERN int ZEXPORT deflatePrime OF((z_streamp strm, + int bits, + int value)); +/* + deflatePrime() inserts bits in the deflate output stream. The intent + is that this function is used to start off the deflate output with the bits + leftover from a previous deflate stream when appending to it. As such, this + function can only be used for raw deflate, and must be used before the first + deflate() call after a deflateInit2() or deflateReset(). bits must be less + than or equal to 16, and that many of the least significant bits of value + will be inserted in the output. + + deflatePrime returns Z_OK if success, Z_BUF_ERROR if there was not enough + room in the internal buffer to insert the bits, or Z_STREAM_ERROR if the + source stream state was inconsistent. +*/ + +ZEXTERN int ZEXPORT deflateSetHeader OF((z_streamp strm, + gz_headerp head)); +/* + deflateSetHeader() provides gzip header information for when a gzip + stream is requested by deflateInit2(). deflateSetHeader() may be called + after deflateInit2() or deflateReset() and before the first call of + deflate(). The text, time, os, extra field, name, and comment information + in the provided gz_header structure are written to the gzip header (xflag is + ignored -- the extra flags are set according to the compression level). The + caller must assure that, if not Z_NULL, name and comment are terminated with + a zero byte, and that if extra is not Z_NULL, that extra_len bytes are + available there. If hcrc is true, a gzip header crc is included. Note that + the current versions of the command-line version of gzip (up through version + 1.3.x) do not support header crc's, and will report that it is a "multi-part + gzip file" and give up. + + If deflateSetHeader is not used, the default gzip header has text false, + the time set to zero, and os set to 255, with no extra, name, or comment + fields. The gzip header is returned to the default state by deflateReset(). + + deflateSetHeader returns Z_OK if success, or Z_STREAM_ERROR if the source + stream state was inconsistent. +*/ + +/* +ZEXTERN int ZEXPORT inflateInit2 OF((z_streamp strm, + int windowBits)); + + This is another version of inflateInit with an extra parameter. The + fields next_in, avail_in, zalloc, zfree and opaque must be initialized + before by the caller. + + The windowBits parameter is the base two logarithm of the maximum window + size (the size of the history buffer). It should be in the range 8..15 for + this version of the library. The default value is 15 if inflateInit is used + instead. windowBits must be greater than or equal to the windowBits value + provided to deflateInit2() while compressing, or it must be equal to 15 if + deflateInit2() was not used. If a compressed stream with a larger window + size is given as input, inflate() will return with the error code + Z_DATA_ERROR instead of trying to allocate a larger window. + + windowBits can also be zero to request that inflate use the window size in + the zlib header of the compressed stream. + + windowBits can also be -8..-15 for raw inflate. In this case, -windowBits + determines the window size. inflate() will then process raw deflate data, + not looking for a zlib or gzip header, not generating a check value, and not + looking for any check values for comparison at the end of the stream. This + is for use with other formats that use the deflate compressed data format + such as zip. Those formats provide their own check values. If a custom + format is developed using the raw deflate format for compressed data, it is + recommended that a check value such as an Adler-32 or a CRC-32 be applied to + the uncompressed data as is done in the zlib, gzip, and zip formats. For + most applications, the zlib format should be used as is. Note that comments + above on the use in deflateInit2() applies to the magnitude of windowBits. + + windowBits can also be greater than 15 for optional gzip decoding. Add + 32 to windowBits to enable zlib and gzip decoding with automatic header + detection, or add 16 to decode only the gzip format (the zlib format will + return a Z_DATA_ERROR). If a gzip stream is being decoded, strm->adler is a + CRC-32 instead of an Adler-32. Unlike the gunzip utility and gzread() (see + below), inflate() will not automatically decode concatenated gzip streams. + inflate() will return Z_STREAM_END at the end of the gzip stream. The state + would need to be reset to continue decoding a subsequent gzip stream. + + inflateInit2 returns Z_OK if success, Z_MEM_ERROR if there was not enough + memory, Z_VERSION_ERROR if the zlib library version is incompatible with the + version assumed by the caller, or Z_STREAM_ERROR if the parameters are + invalid, such as a null pointer to the structure. msg is set to null if + there is no error message. inflateInit2 does not perform any decompression + apart from possibly reading the zlib header if present: actual decompression + will be done by inflate(). (So next_in and avail_in may be modified, but + next_out and avail_out are unused and unchanged.) The current implementation + of inflateInit2() does not process any header information -- that is + deferred until inflate() is called. +*/ + +ZEXTERN int ZEXPORT inflateSetDictionary OF((z_streamp strm, + const Bytef *dictionary, + uInt dictLength)); +/* + Initializes the decompression dictionary from the given uncompressed byte + sequence. This function must be called immediately after a call of inflate, + if that call returned Z_NEED_DICT. The dictionary chosen by the compressor + can be determined from the Adler-32 value returned by that call of inflate. + The compressor and decompressor must use exactly the same dictionary (see + deflateSetDictionary). For raw inflate, this function can be called at any + time to set the dictionary. If the provided dictionary is smaller than the + window and there is already data in the window, then the provided dictionary + will amend what's there. The application must insure that the dictionary + that was used for compression is provided. + + inflateSetDictionary returns Z_OK if success, Z_STREAM_ERROR if a + parameter is invalid (e.g. dictionary being Z_NULL) or the stream state is + inconsistent, Z_DATA_ERROR if the given dictionary doesn't match the + expected one (incorrect Adler-32 value). inflateSetDictionary does not + perform any decompression: this will be done by subsequent calls of + inflate(). +*/ + +ZEXTERN int ZEXPORT inflateGetDictionary OF((z_streamp strm, + Bytef *dictionary, + uInt *dictLength)); +/* + Returns the sliding dictionary being maintained by inflate. dictLength is + set to the number of bytes in the dictionary, and that many bytes are copied + to dictionary. dictionary must have enough space, where 32768 bytes is + always enough. If inflateGetDictionary() is called with dictionary equal to + Z_NULL, then only the dictionary length is returned, and nothing is copied. + Similary, if dictLength is Z_NULL, then it is not set. + + inflateGetDictionary returns Z_OK on success, or Z_STREAM_ERROR if the + stream state is inconsistent. +*/ + +ZEXTERN int ZEXPORT inflateSync OF((z_streamp strm)); +/* + Skips invalid compressed data until a possible full flush point (see above + for the description of deflate with Z_FULL_FLUSH) can be found, or until all + available input is skipped. No output is provided. + + inflateSync searches for a 00 00 FF FF pattern in the compressed data. + All full flush points have this pattern, but not all occurrences of this + pattern are full flush points. + + inflateSync returns Z_OK if a possible full flush point has been found, + Z_BUF_ERROR if no more input was provided, Z_DATA_ERROR if no flush point + has been found, or Z_STREAM_ERROR if the stream structure was inconsistent. + In the success case, the application may save the current current value of + total_in which indicates where valid compressed data was found. In the + error case, the application may repeatedly call inflateSync, providing more + input each time, until success or end of the input data. +*/ + +ZEXTERN int ZEXPORT inflateCopy OF((z_streamp dest, + z_streamp source)); +/* + Sets the destination stream as a complete copy of the source stream. + + This function can be useful when randomly accessing a large stream. The + first pass through the stream can periodically record the inflate state, + allowing restarting inflate at those points when randomly accessing the + stream. + + inflateCopy returns Z_OK if success, Z_MEM_ERROR if there was not + enough memory, Z_STREAM_ERROR if the source stream state was inconsistent + (such as zalloc being Z_NULL). msg is left unchanged in both source and + destination. +*/ + +ZEXTERN int ZEXPORT inflateReset OF((z_streamp strm)); +/* + This function is equivalent to inflateEnd followed by inflateInit, + but does not free and reallocate the internal decompression state. The + stream will keep attributes that may have been set by inflateInit2. + + inflateReset returns Z_OK if success, or Z_STREAM_ERROR if the source + stream state was inconsistent (such as zalloc or state being Z_NULL). +*/ + +ZEXTERN int ZEXPORT inflateReset2 OF((z_streamp strm, + int windowBits)); +/* + This function is the same as inflateReset, but it also permits changing + the wrap and window size requests. The windowBits parameter is interpreted + the same as it is for inflateInit2. If the window size is changed, then the + memory allocated for the window is freed, and the window will be reallocated + by inflate() if needed. + + inflateReset2 returns Z_OK if success, or Z_STREAM_ERROR if the source + stream state was inconsistent (such as zalloc or state being Z_NULL), or if + the windowBits parameter is invalid. +*/ + +ZEXTERN int ZEXPORT inflatePrime OF((z_streamp strm, + int bits, + int value)); +/* + This function inserts bits in the inflate input stream. The intent is + that this function is used to start inflating at a bit position in the + middle of a byte. The provided bits will be used before any bytes are used + from next_in. This function should only be used with raw inflate, and + should be used before the first inflate() call after inflateInit2() or + inflateReset(). bits must be less than or equal to 16, and that many of the + least significant bits of value will be inserted in the input. + + If bits is negative, then the input stream bit buffer is emptied. Then + inflatePrime() can be called again to put bits in the buffer. This is used + to clear out bits leftover after feeding inflate a block description prior + to feeding inflate codes. + + inflatePrime returns Z_OK if success, or Z_STREAM_ERROR if the source + stream state was inconsistent. +*/ + +ZEXTERN long ZEXPORT inflateMark OF((z_streamp strm)); +/* + This function returns two values, one in the lower 16 bits of the return + value, and the other in the remaining upper bits, obtained by shifting the + return value down 16 bits. If the upper value is -1 and the lower value is + zero, then inflate() is currently decoding information outside of a block. + If the upper value is -1 and the lower value is non-zero, then inflate is in + the middle of a stored block, with the lower value equaling the number of + bytes from the input remaining to copy. If the upper value is not -1, then + it is the number of bits back from the current bit position in the input of + the code (literal or length/distance pair) currently being processed. In + that case the lower value is the number of bytes already emitted for that + code. + + A code is being processed if inflate is waiting for more input to complete + decoding of the code, or if it has completed decoding but is waiting for + more output space to write the literal or match data. + + inflateMark() is used to mark locations in the input data for random + access, which may be at bit positions, and to note those cases where the + output of a code may span boundaries of random access blocks. The current + location in the input stream can be determined from avail_in and data_type + as noted in the description for the Z_BLOCK flush parameter for inflate. + + inflateMark returns the value noted above, or -65536 if the provided + source stream state was inconsistent. +*/ + +ZEXTERN int ZEXPORT inflateGetHeader OF((z_streamp strm, + gz_headerp head)); +/* + inflateGetHeader() requests that gzip header information be stored in the + provided gz_header structure. inflateGetHeader() may be called after + inflateInit2() or inflateReset(), and before the first call of inflate(). + As inflate() processes the gzip stream, head->done is zero until the header + is completed, at which time head->done is set to one. If a zlib stream is + being decoded, then head->done is set to -1 to indicate that there will be + no gzip header information forthcoming. Note that Z_BLOCK or Z_TREES can be + used to force inflate() to return immediately after header processing is + complete and before any actual data is decompressed. + + The text, time, xflags, and os fields are filled in with the gzip header + contents. hcrc is set to true if there is a header CRC. (The header CRC + was valid if done is set to one.) If extra is not Z_NULL, then extra_max + contains the maximum number of bytes to write to extra. Once done is true, + extra_len contains the actual extra field length, and extra contains the + extra field, or that field truncated if extra_max is less than extra_len. + If name is not Z_NULL, then up to name_max characters are written there, + terminated with a zero unless the length is greater than name_max. If + comment is not Z_NULL, then up to comm_max characters are written there, + terminated with a zero unless the length is greater than comm_max. When any + of extra, name, or comment are not Z_NULL and the respective field is not + present in the header, then that field is set to Z_NULL to signal its + absence. This allows the use of deflateSetHeader() with the returned + structure to duplicate the header. However if those fields are set to + allocated memory, then the application will need to save those pointers + elsewhere so that they can be eventually freed. + + If inflateGetHeader is not used, then the header information is simply + discarded. The header is always checked for validity, including the header + CRC if present. inflateReset() will reset the process to discard the header + information. The application would need to call inflateGetHeader() again to + retrieve the header from the next gzip stream. + + inflateGetHeader returns Z_OK if success, or Z_STREAM_ERROR if the source + stream state was inconsistent. +*/ + +/* +ZEXTERN int ZEXPORT inflateBackInit OF((z_streamp strm, int windowBits, + unsigned char FAR *window)); + + Initialize the internal stream state for decompression using inflateBack() + calls. The fields zalloc, zfree and opaque in strm must be initialized + before the call. If zalloc and zfree are Z_NULL, then the default library- + derived memory allocation routines are used. windowBits is the base two + logarithm of the window size, in the range 8..15. window is a caller + supplied buffer of that size. Except for special applications where it is + assured that deflate was used with small window sizes, windowBits must be 15 + and a 32K byte window must be supplied to be able to decompress general + deflate streams. + + See inflateBack() for the usage of these routines. + + inflateBackInit will return Z_OK on success, Z_STREAM_ERROR if any of + the parameters are invalid, Z_MEM_ERROR if the internal state could not be + allocated, or Z_VERSION_ERROR if the version of the library does not match + the version of the header file. +*/ + +typedef unsigned (*in_func) OF((void FAR *, + z_const unsigned char FAR * FAR *)); +typedef int (*out_func) OF((void FAR *, unsigned char FAR *, unsigned)); + +ZEXTERN int ZEXPORT inflateBack OF((z_streamp strm, + in_func in, void FAR *in_desc, + out_func out, void FAR *out_desc)); +/* + inflateBack() does a raw inflate with a single call using a call-back + interface for input and output. This is potentially more efficient than + inflate() for file i/o applications, in that it avoids copying between the + output and the sliding window by simply making the window itself the output + buffer. inflate() can be faster on modern CPUs when used with large + buffers. inflateBack() trusts the application to not change the output + buffer passed by the output function, at least until inflateBack() returns. + + inflateBackInit() must be called first to allocate the internal state + and to initialize the state with the user-provided window buffer. + inflateBack() may then be used multiple times to inflate a complete, raw + deflate stream with each call. inflateBackEnd() is then called to free the + allocated state. + + A raw deflate stream is one with no zlib or gzip header or trailer. + This routine would normally be used in a utility that reads zip or gzip + files and writes out uncompressed files. The utility would decode the + header and process the trailer on its own, hence this routine expects only + the raw deflate stream to decompress. This is different from the default + behavior of inflate(), which expects a zlib header and trailer around the + deflate stream. + + inflateBack() uses two subroutines supplied by the caller that are then + called by inflateBack() for input and output. inflateBack() calls those + routines until it reads a complete deflate stream and writes out all of the + uncompressed data, or until it encounters an error. The function's + parameters and return types are defined above in the in_func and out_func + typedefs. inflateBack() will call in(in_desc, &buf) which should return the + number of bytes of provided input, and a pointer to that input in buf. If + there is no input available, in() must return zero -- buf is ignored in that + case -- and inflateBack() will return a buffer error. inflateBack() will + call out(out_desc, buf, len) to write the uncompressed data buf[0..len-1]. + out() should return zero on success, or non-zero on failure. If out() + returns non-zero, inflateBack() will return with an error. Neither in() nor + out() are permitted to change the contents of the window provided to + inflateBackInit(), which is also the buffer that out() uses to write from. + The length written by out() will be at most the window size. Any non-zero + amount of input may be provided by in(). + + For convenience, inflateBack() can be provided input on the first call by + setting strm->next_in and strm->avail_in. If that input is exhausted, then + in() will be called. Therefore strm->next_in must be initialized before + calling inflateBack(). If strm->next_in is Z_NULL, then in() will be called + immediately for input. If strm->next_in is not Z_NULL, then strm->avail_in + must also be initialized, and then if strm->avail_in is not zero, input will + initially be taken from strm->next_in[0 .. strm->avail_in - 1]. + + The in_desc and out_desc parameters of inflateBack() is passed as the + first parameter of in() and out() respectively when they are called. These + descriptors can be optionally used to pass any information that the caller- + supplied in() and out() functions need to do their job. + + On return, inflateBack() will set strm->next_in and strm->avail_in to + pass back any unused input that was provided by the last in() call. The + return values of inflateBack() can be Z_STREAM_END on success, Z_BUF_ERROR + if in() or out() returned an error, Z_DATA_ERROR if there was a format error + in the deflate stream (in which case strm->msg is set to indicate the nature + of the error), or Z_STREAM_ERROR if the stream was not properly initialized. + In the case of Z_BUF_ERROR, an input or output error can be distinguished + using strm->next_in which will be Z_NULL only if in() returned an error. If + strm->next_in is not Z_NULL, then the Z_BUF_ERROR was due to out() returning + non-zero. (in() will always be called before out(), so strm->next_in is + assured to be defined if out() returns non-zero.) Note that inflateBack() + cannot return Z_OK. +*/ + +ZEXTERN int ZEXPORT inflateBackEnd OF((z_streamp strm)); +/* + All memory allocated by inflateBackInit() is freed. + + inflateBackEnd() returns Z_OK on success, or Z_STREAM_ERROR if the stream + state was inconsistent. +*/ + +ZEXTERN uLong ZEXPORT zlibCompileFlags OF((void)); +/* Return flags indicating compile-time options. + + Type sizes, two bits each, 00 = 16 bits, 01 = 32, 10 = 64, 11 = other: + 1.0: size of uInt + 3.2: size of uLong + 5.4: size of voidpf (pointer) + 7.6: size of z_off_t + + Compiler, assembler, and debug options: + 8: ZLIB_DEBUG + 9: ASMV or ASMINF -- use ASM code + 10: ZLIB_WINAPI -- exported functions use the WINAPI calling convention + 11: 0 (reserved) + + One-time table building (smaller code, but not thread-safe if true): + 12: BUILDFIXED -- build static block decoding tables when needed + 13: DYNAMIC_CRC_TABLE -- build CRC calculation tables when needed + 14,15: 0 (reserved) + + Library content (indicates missing functionality): + 16: NO_GZCOMPRESS -- gz* functions cannot compress (to avoid linking + deflate code when not needed) + 17: NO_GZIP -- deflate can't write gzip streams, and inflate can't detect + and decode gzip streams (to avoid linking crc code) + 18-19: 0 (reserved) + + Operation variations (changes in library functionality): + 20: PKZIP_BUG_WORKAROUND -- slightly more permissive inflate + 21: FASTEST -- deflate algorithm with only one, lowest compression level + 22,23: 0 (reserved) + + The sprintf variant used by gzprintf (zero is best): + 24: 0 = vs*, 1 = s* -- 1 means limited to 20 arguments after the format + 25: 0 = *nprintf, 1 = *printf -- 1 means gzprintf() not secure! + 26: 0 = returns value, 1 = void -- 1 means inferred string length returned + + Remainder: + 27-31: 0 (reserved) + */ + +#ifndef Z_SOLO + + /* utility functions */ + +/* + The following utility functions are implemented on top of the basic + stream-oriented functions. To simplify the interface, some default options + are assumed (compression level and memory usage, standard memory allocation + functions). The source code of these utility functions can be modified if + you need special options. +*/ + +ZEXTERN int ZEXPORT compress OF((Bytef *dest, uLongf *destLen, + const Bytef *source, uLong sourceLen)); +/* + Compresses the source buffer into the destination buffer. sourceLen is + the byte length of the source buffer. Upon entry, destLen is the total size + of the destination buffer, which must be at least the value returned by + compressBound(sourceLen). Upon exit, destLen is the actual size of the + compressed data. compress() is equivalent to compress2() with a level + parameter of Z_DEFAULT_COMPRESSION. + + compress returns Z_OK if success, Z_MEM_ERROR if there was not + enough memory, Z_BUF_ERROR if there was not enough room in the output + buffer. +*/ + +ZEXTERN int ZEXPORT compress2 OF((Bytef *dest, uLongf *destLen, + const Bytef *source, uLong sourceLen, + int level)); +/* + Compresses the source buffer into the destination buffer. The level + parameter has the same meaning as in deflateInit. sourceLen is the byte + length of the source buffer. Upon entry, destLen is the total size of the + destination buffer, which must be at least the value returned by + compressBound(sourceLen). Upon exit, destLen is the actual size of the + compressed data. + + compress2 returns Z_OK if success, Z_MEM_ERROR if there was not enough + memory, Z_BUF_ERROR if there was not enough room in the output buffer, + Z_STREAM_ERROR if the level parameter is invalid. +*/ + +ZEXTERN uLong ZEXPORT compressBound OF((uLong sourceLen)); +/* + compressBound() returns an upper bound on the compressed size after + compress() or compress2() on sourceLen bytes. It would be used before a + compress() or compress2() call to allocate the destination buffer. +*/ + +ZEXTERN int ZEXPORT uncompress OF((Bytef *dest, uLongf *destLen, + const Bytef *source, uLong sourceLen)); +/* + Decompresses the source buffer into the destination buffer. sourceLen is + the byte length of the source buffer. Upon entry, destLen is the total size + of the destination buffer, which must be large enough to hold the entire + uncompressed data. (The size of the uncompressed data must have been saved + previously by the compressor and transmitted to the decompressor by some + mechanism outside the scope of this compression library.) Upon exit, destLen + is the actual size of the uncompressed data. + + uncompress returns Z_OK if success, Z_MEM_ERROR if there was not + enough memory, Z_BUF_ERROR if there was not enough room in the output + buffer, or Z_DATA_ERROR if the input data was corrupted or incomplete. In + the case where there is not enough room, uncompress() will fill the output + buffer with the uncompressed data up to that point. +*/ + +ZEXTERN int ZEXPORT uncompress2 OF((Bytef *dest, uLongf *destLen, + const Bytef *source, uLong *sourceLen)); +/* + Same as uncompress, except that sourceLen is a pointer, where the + length of the source is *sourceLen. On return, *sourceLen is the number of + source bytes consumed. +*/ + + /* gzip file access functions */ + +/* + This library supports reading and writing files in gzip (.gz) format with + an interface similar to that of stdio, using the functions that start with + "gz". The gzip format is different from the zlib format. gzip is a gzip + wrapper, documented in RFC 1952, wrapped around a deflate stream. +*/ + +typedef struct gzFile_s *gzFile; /* semi-opaque gzip file descriptor */ + +/* +ZEXTERN gzFile ZEXPORT gzopen OF((const char *path, const char *mode)); + + Opens a gzip (.gz) file for reading or writing. The mode parameter is as + in fopen ("rb" or "wb") but can also include a compression level ("wb9") or + a strategy: 'f' for filtered data as in "wb6f", 'h' for Huffman-only + compression as in "wb1h", 'R' for run-length encoding as in "wb1R", or 'F' + for fixed code compression as in "wb9F". (See the description of + deflateInit2 for more information about the strategy parameter.) 'T' will + request transparent writing or appending with no compression and not using + the gzip format. + + "a" can be used instead of "w" to request that the gzip stream that will + be written be appended to the file. "+" will result in an error, since + reading and writing to the same gzip file is not supported. The addition of + "x" when writing will create the file exclusively, which fails if the file + already exists. On systems that support it, the addition of "e" when + reading or writing will set the flag to close the file on an execve() call. + + These functions, as well as gzip, will read and decode a sequence of gzip + streams in a file. The append function of gzopen() can be used to create + such a file. (Also see gzflush() for another way to do this.) When + appending, gzopen does not test whether the file begins with a gzip stream, + nor does it look for the end of the gzip streams to begin appending. gzopen + will simply append a gzip stream to the existing file. + + gzopen can be used to read a file which is not in gzip format; in this + case gzread will directly read from the file without decompression. When + reading, this will be detected automatically by looking for the magic two- + byte gzip header. + + gzopen returns NULL if the file could not be opened, if there was + insufficient memory to allocate the gzFile state, or if an invalid mode was + specified (an 'r', 'w', or 'a' was not provided, or '+' was provided). + errno can be checked to determine if the reason gzopen failed was that the + file could not be opened. +*/ + +ZEXTERN gzFile ZEXPORT gzdopen OF((int fd, const char *mode)); +/* + gzdopen associates a gzFile with the file descriptor fd. File descriptors + are obtained from calls like open, dup, creat, pipe or fileno (if the file + has been previously opened with fopen). The mode parameter is as in gzopen. + + The next call of gzclose on the returned gzFile will also close the file + descriptor fd, just like fclose(fdopen(fd, mode)) closes the file descriptor + fd. If you want to keep fd open, use fd = dup(fd_keep); gz = gzdopen(fd, + mode);. The duplicated descriptor should be saved to avoid a leak, since + gzdopen does not close fd if it fails. If you are using fileno() to get the + file descriptor from a FILE *, then you will have to use dup() to avoid + double-close()ing the file descriptor. Both gzclose() and fclose() will + close the associated file descriptor, so they need to have different file + descriptors. + + gzdopen returns NULL if there was insufficient memory to allocate the + gzFile state, if an invalid mode was specified (an 'r', 'w', or 'a' was not + provided, or '+' was provided), or if fd is -1. The file descriptor is not + used until the next gz* read, write, seek, or close operation, so gzdopen + will not detect if fd is invalid (unless fd is -1). +*/ + +ZEXTERN int ZEXPORT gzbuffer OF((gzFile file, unsigned size)); +/* + Set the internal buffer size used by this library's functions. The + default buffer size is 8192 bytes. This function must be called after + gzopen() or gzdopen(), and before any other calls that read or write the + file. The buffer memory allocation is always deferred to the first read or + write. Three times that size in buffer space is allocated. A larger buffer + size of, for example, 64K or 128K bytes will noticeably increase the speed + of decompression (reading). + + The new buffer size also affects the maximum length for gzprintf(). + + gzbuffer() returns 0 on success, or -1 on failure, such as being called + too late. +*/ + +ZEXTERN int ZEXPORT gzsetparams OF((gzFile file, int level, int strategy)); +/* + Dynamically update the compression level or strategy. See the description + of deflateInit2 for the meaning of these parameters. Previously provided + data is flushed before the parameter change. + + gzsetparams returns Z_OK if success, Z_STREAM_ERROR if the file was not + opened for writing, Z_ERRNO if there is an error writing the flushed data, + or Z_MEM_ERROR if there is a memory allocation error. +*/ + +ZEXTERN int ZEXPORT gzread OF((gzFile file, voidp buf, unsigned len)); +/* + Reads the given number of uncompressed bytes from the compressed file. If + the input file is not in gzip format, gzread copies the given number of + bytes into the buffer directly from the file. + + After reaching the end of a gzip stream in the input, gzread will continue + to read, looking for another gzip stream. Any number of gzip streams may be + concatenated in the input file, and will all be decompressed by gzread(). + If something other than a gzip stream is encountered after a gzip stream, + that remaining trailing garbage is ignored (and no error is returned). + + gzread can be used to read a gzip file that is being concurrently written. + Upon reaching the end of the input, gzread will return with the available + data. If the error code returned by gzerror is Z_OK or Z_BUF_ERROR, then + gzclearerr can be used to clear the end of file indicator in order to permit + gzread to be tried again. Z_OK indicates that a gzip stream was completed + on the last gzread. Z_BUF_ERROR indicates that the input file ended in the + middle of a gzip stream. Note that gzread does not return -1 in the event + of an incomplete gzip stream. This error is deferred until gzclose(), which + will return Z_BUF_ERROR if the last gzread ended in the middle of a gzip + stream. Alternatively, gzerror can be used before gzclose to detect this + case. + + gzread returns the number of uncompressed bytes actually read, less than + len for end of file, or -1 for error. If len is too large to fit in an int, + then nothing is read, -1 is returned, and the error state is set to + Z_STREAM_ERROR. +*/ + +ZEXTERN z_size_t ZEXPORT gzfread OF((voidp buf, z_size_t size, z_size_t nitems, + gzFile file)); +/* + Read up to nitems items of size size from file to buf, otherwise operating + as gzread() does. This duplicates the interface of stdio's fread(), with + size_t request and return types. If the library defines size_t, then + z_size_t is identical to size_t. If not, then z_size_t is an unsigned + integer type that can contain a pointer. + + gzfread() returns the number of full items read of size size, or zero if + the end of the file was reached and a full item could not be read, or if + there was an error. gzerror() must be consulted if zero is returned in + order to determine if there was an error. If the multiplication of size and + nitems overflows, i.e. the product does not fit in a z_size_t, then nothing + is read, zero is returned, and the error state is set to Z_STREAM_ERROR. + + In the event that the end of file is reached and only a partial item is + available at the end, i.e. the remaining uncompressed data length is not a + multiple of size, then the final partial item is nevetheless read into buf + and the end-of-file flag is set. The length of the partial item read is not + provided, but could be inferred from the result of gztell(). This behavior + is the same as the behavior of fread() implementations in common libraries, + but it prevents the direct use of gzfread() to read a concurrently written + file, reseting and retrying on end-of-file, when size is not 1. +*/ + +ZEXTERN int ZEXPORT gzwrite OF((gzFile file, + voidpc buf, unsigned len)); +/* + Writes the given number of uncompressed bytes into the compressed file. + gzwrite returns the number of uncompressed bytes written or 0 in case of + error. +*/ + +ZEXTERN z_size_t ZEXPORT gzfwrite OF((voidpc buf, z_size_t size, + z_size_t nitems, gzFile file)); +/* + gzfwrite() writes nitems items of size size from buf to file, duplicating + the interface of stdio's fwrite(), with size_t request and return types. If + the library defines size_t, then z_size_t is identical to size_t. If not, + then z_size_t is an unsigned integer type that can contain a pointer. + + gzfwrite() returns the number of full items written of size size, or zero + if there was an error. If the multiplication of size and nitems overflows, + i.e. the product does not fit in a z_size_t, then nothing is written, zero + is returned, and the error state is set to Z_STREAM_ERROR. +*/ + +ZEXTERN int ZEXPORTVA gzprintf Z_ARG((gzFile file, const char *format, ...)); +/* + Converts, formats, and writes the arguments to the compressed file under + control of the format string, as in fprintf. gzprintf returns the number of + uncompressed bytes actually written, or a negative zlib error code in case + of error. The number of uncompressed bytes written is limited to 8191, or + one less than the buffer size given to gzbuffer(). The caller should assure + that this limit is not exceeded. If it is exceeded, then gzprintf() will + return an error (0) with nothing written. In this case, there may also be a + buffer overflow with unpredictable consequences, which is possible only if + zlib was compiled with the insecure functions sprintf() or vsprintf() + because the secure snprintf() or vsnprintf() functions were not available. + This can be determined using zlibCompileFlags(). +*/ + +ZEXTERN int ZEXPORT gzputs OF((gzFile file, const char *s)); +/* + Writes the given null-terminated string to the compressed file, excluding + the terminating null character. + + gzputs returns the number of characters written, or -1 in case of error. +*/ + +ZEXTERN char * ZEXPORT gzgets OF((gzFile file, char *buf, int len)); +/* + Reads bytes from the compressed file until len-1 characters are read, or a + newline character is read and transferred to buf, or an end-of-file + condition is encountered. If any characters are read or if len == 1, the + string is terminated with a null character. If no characters are read due + to an end-of-file or len < 1, then the buffer is left untouched. + + gzgets returns buf which is a null-terminated string, or it returns NULL + for end-of-file or in case of error. If there was an error, the contents at + buf are indeterminate. +*/ + +ZEXTERN int ZEXPORT gzputc OF((gzFile file, int c)); +/* + Writes c, converted to an unsigned char, into the compressed file. gzputc + returns the value that was written, or -1 in case of error. +*/ + +ZEXTERN int ZEXPORT gzgetc OF((gzFile file)); +/* + Reads one byte from the compressed file. gzgetc returns this byte or -1 + in case of end of file or error. This is implemented as a macro for speed. + As such, it does not do all of the checking the other functions do. I.e. + it does not check to see if file is NULL, nor whether the structure file + points to has been clobbered or not. +*/ + +ZEXTERN int ZEXPORT gzungetc OF((int c, gzFile file)); +/* + Push one character back onto the stream to be read as the first character + on the next read. At least one character of push-back is allowed. + gzungetc() returns the character pushed, or -1 on failure. gzungetc() will + fail if c is -1, and may fail if a character has been pushed but not read + yet. If gzungetc is used immediately after gzopen or gzdopen, at least the + output buffer size of pushed characters is allowed. (See gzbuffer above.) + The pushed character will be discarded if the stream is repositioned with + gzseek() or gzrewind(). +*/ + +ZEXTERN int ZEXPORT gzflush OF((gzFile file, int flush)); +/* + Flushes all pending output into the compressed file. The parameter flush + is as in the deflate() function. The return value is the zlib error number + (see function gzerror below). gzflush is only permitted when writing. + + If the flush parameter is Z_FINISH, the remaining data is written and the + gzip stream is completed in the output. If gzwrite() is called again, a new + gzip stream will be started in the output. gzread() is able to read such + concatenated gzip streams. + + gzflush should be called only when strictly necessary because it will + degrade compression if called too often. +*/ + +/* +ZEXTERN z_off_t ZEXPORT gzseek OF((gzFile file, + z_off_t offset, int whence)); + + Sets the starting position for the next gzread or gzwrite on the given + compressed file. The offset represents a number of bytes in the + uncompressed data stream. The whence parameter is defined as in lseek(2); + the value SEEK_END is not supported. + + If the file is opened for reading, this function is emulated but can be + extremely slow. If the file is opened for writing, only forward seeks are + supported; gzseek then compresses a sequence of zeroes up to the new + starting position. + + gzseek returns the resulting offset location as measured in bytes from + the beginning of the uncompressed stream, or -1 in case of error, in + particular if the file is opened for writing and the new starting position + would be before the current position. +*/ + +ZEXTERN int ZEXPORT gzrewind OF((gzFile file)); +/* + Rewinds the given file. This function is supported only for reading. + + gzrewind(file) is equivalent to (int)gzseek(file, 0L, SEEK_SET) +*/ + +/* +ZEXTERN z_off_t ZEXPORT gztell OF((gzFile file)); + + Returns the starting position for the next gzread or gzwrite on the given + compressed file. This position represents a number of bytes in the + uncompressed data stream, and is zero when starting, even if appending or + reading a gzip stream from the middle of a file using gzdopen(). + + gztell(file) is equivalent to gzseek(file, 0L, SEEK_CUR) +*/ + +/* +ZEXTERN z_off_t ZEXPORT gzoffset OF((gzFile file)); + + Returns the current offset in the file being read or written. This offset + includes the count of bytes that precede the gzip stream, for example when + appending or when using gzdopen() for reading. When reading, the offset + does not include as yet unused buffered input. This information can be used + for a progress indicator. On error, gzoffset() returns -1. +*/ + +ZEXTERN int ZEXPORT gzeof OF((gzFile file)); +/* + Returns true (1) if the end-of-file indicator has been set while reading, + false (0) otherwise. Note that the end-of-file indicator is set only if the + read tried to go past the end of the input, but came up short. Therefore, + just like feof(), gzeof() may return false even if there is no more data to + read, in the event that the last read request was for the exact number of + bytes remaining in the input file. This will happen if the input file size + is an exact multiple of the buffer size. + + If gzeof() returns true, then the read functions will return no more data, + unless the end-of-file indicator is reset by gzclearerr() and the input file + has grown since the previous end of file was detected. +*/ + +ZEXTERN int ZEXPORT gzdirect OF((gzFile file)); +/* + Returns true (1) if file is being copied directly while reading, or false + (0) if file is a gzip stream being decompressed. + + If the input file is empty, gzdirect() will return true, since the input + does not contain a gzip stream. + + If gzdirect() is used immediately after gzopen() or gzdopen() it will + cause buffers to be allocated to allow reading the file to determine if it + is a gzip file. Therefore if gzbuffer() is used, it should be called before + gzdirect(). + + When writing, gzdirect() returns true (1) if transparent writing was + requested ("wT" for the gzopen() mode), or false (0) otherwise. (Note: + gzdirect() is not needed when writing. Transparent writing must be + explicitly requested, so the application already knows the answer. When + linking statically, using gzdirect() will include all of the zlib code for + gzip file reading and decompression, which may not be desired.) +*/ + +ZEXTERN int ZEXPORT gzclose OF((gzFile file)); +/* + Flushes all pending output if necessary, closes the compressed file and + deallocates the (de)compression state. Note that once file is closed, you + cannot call gzerror with file, since its structures have been deallocated. + gzclose must not be called more than once on the same file, just as free + must not be called more than once on the same allocation. + + gzclose will return Z_STREAM_ERROR if file is not valid, Z_ERRNO on a + file operation error, Z_MEM_ERROR if out of memory, Z_BUF_ERROR if the + last read ended in the middle of a gzip stream, or Z_OK on success. +*/ + +ZEXTERN int ZEXPORT gzclose_r OF((gzFile file)); +ZEXTERN int ZEXPORT gzclose_w OF((gzFile file)); +/* + Same as gzclose(), but gzclose_r() is only for use when reading, and + gzclose_w() is only for use when writing or appending. The advantage to + using these instead of gzclose() is that they avoid linking in zlib + compression or decompression code that is not used when only reading or only + writing respectively. If gzclose() is used, then both compression and + decompression code will be included the application when linking to a static + zlib library. +*/ + +ZEXTERN const char * ZEXPORT gzerror OF((gzFile file, int *errnum)); +/* + Returns the error message for the last error which occurred on the given + compressed file. errnum is set to zlib error number. If an error occurred + in the file system and not in the compression library, errnum is set to + Z_ERRNO and the application may consult errno to get the exact error code. + + The application must not modify the returned string. Future calls to + this function may invalidate the previously returned string. If file is + closed, then the string previously returned by gzerror will no longer be + available. + + gzerror() should be used to distinguish errors from end-of-file for those + functions above that do not distinguish those cases in their return values. +*/ + +ZEXTERN void ZEXPORT gzclearerr OF((gzFile file)); +/* + Clears the error and end-of-file flags for file. This is analogous to the + clearerr() function in stdio. This is useful for continuing to read a gzip + file that is being written concurrently. +*/ + +#endif /* !Z_SOLO */ + + /* checksum functions */ + +/* + These functions are not related to compression but are exported + anyway because they might be useful in applications using the compression + library. +*/ + +ZEXTERN uLong ZEXPORT adler32 OF((uLong adler, const Bytef *buf, uInt len)); +/* + Update a running Adler-32 checksum with the bytes buf[0..len-1] and + return the updated checksum. If buf is Z_NULL, this function returns the + required initial value for the checksum. + + An Adler-32 checksum is almost as reliable as a CRC-32 but can be computed + much faster. + + Usage example: + + uLong adler = adler32(0L, Z_NULL, 0); + + while (read_buffer(buffer, length) != EOF) { + adler = adler32(adler, buffer, length); + } + if (adler != original_adler) error(); +*/ + +ZEXTERN uLong ZEXPORT adler32_z OF((uLong adler, const Bytef *buf, + z_size_t len)); +/* + Same as adler32(), but with a size_t length. +*/ + +/* +ZEXTERN uLong ZEXPORT adler32_combine OF((uLong adler1, uLong adler2, + z_off_t len2)); + + Combine two Adler-32 checksums into one. For two sequences of bytes, seq1 + and seq2 with lengths len1 and len2, Adler-32 checksums were calculated for + each, adler1 and adler2. adler32_combine() returns the Adler-32 checksum of + seq1 and seq2 concatenated, requiring only adler1, adler2, and len2. Note + that the z_off_t type (like off_t) is a signed integer. If len2 is + negative, the result has no meaning or utility. +*/ + +ZEXTERN uLong ZEXPORT crc32 OF((uLong crc, const Bytef *buf, uInt len)); +/* + Update a running CRC-32 with the bytes buf[0..len-1] and return the + updated CRC-32. If buf is Z_NULL, this function returns the required + initial value for the crc. Pre- and post-conditioning (one's complement) is + performed within this function so it shouldn't be done by the application. + + Usage example: + + uLong crc = crc32(0L, Z_NULL, 0); + + while (read_buffer(buffer, length) != EOF) { + crc = crc32(crc, buffer, length); + } + if (crc != original_crc) error(); +*/ + +ZEXTERN uLong ZEXPORT crc32_z OF((uLong adler, const Bytef *buf, + z_size_t len)); +/* + Same as crc32(), but with a size_t length. +*/ + +/* +ZEXTERN uLong ZEXPORT crc32_combine OF((uLong crc1, uLong crc2, z_off_t len2)); + + Combine two CRC-32 check values into one. For two sequences of bytes, + seq1 and seq2 with lengths len1 and len2, CRC-32 check values were + calculated for each, crc1 and crc2. crc32_combine() returns the CRC-32 + check value of seq1 and seq2 concatenated, requiring only crc1, crc2, and + len2. +*/ + + + /* various hacks, don't look :) */ + +/* deflateInit and inflateInit are macros to allow checking the zlib version + * and the compiler's view of z_stream: + */ +ZEXTERN int ZEXPORT deflateInit_ OF((z_streamp strm, int level, + const char *version, int stream_size)); +ZEXTERN int ZEXPORT inflateInit_ OF((z_streamp strm, + const char *version, int stream_size)); +ZEXTERN int ZEXPORT deflateInit2_ OF((z_streamp strm, int level, int method, + int windowBits, int memLevel, + int strategy, const char *version, + int stream_size)); +ZEXTERN int ZEXPORT inflateInit2_ OF((z_streamp strm, int windowBits, + const char *version, int stream_size)); +ZEXTERN int ZEXPORT inflateBackInit_ OF((z_streamp strm, int windowBits, + unsigned char FAR *window, + const char *version, + int stream_size)); +#ifdef Z_PREFIX_SET +# define z_deflateInit(strm, level) \ + deflateInit_((strm), (level), ZLIB_VERSION, (int)sizeof(z_stream)) +# define z_inflateInit(strm) \ + inflateInit_((strm), ZLIB_VERSION, (int)sizeof(z_stream)) +# define z_deflateInit2(strm, level, method, windowBits, memLevel, strategy) \ + deflateInit2_((strm),(level),(method),(windowBits),(memLevel),\ + (strategy), ZLIB_VERSION, (int)sizeof(z_stream)) +# define z_inflateInit2(strm, windowBits) \ + inflateInit2_((strm), (windowBits), ZLIB_VERSION, \ + (int)sizeof(z_stream)) +# define z_inflateBackInit(strm, windowBits, window) \ + inflateBackInit_((strm), (windowBits), (window), \ + ZLIB_VERSION, (int)sizeof(z_stream)) +#else +# define deflateInit(strm, level) \ + deflateInit_((strm), (level), ZLIB_VERSION, (int)sizeof(z_stream)) +# define inflateInit(strm) \ + inflateInit_((strm), ZLIB_VERSION, (int)sizeof(z_stream)) +# define deflateInit2(strm, level, method, windowBits, memLevel, strategy) \ + deflateInit2_((strm),(level),(method),(windowBits),(memLevel),\ + (strategy), ZLIB_VERSION, (int)sizeof(z_stream)) +# define inflateInit2(strm, windowBits) \ + inflateInit2_((strm), (windowBits), ZLIB_VERSION, \ + (int)sizeof(z_stream)) +# define inflateBackInit(strm, windowBits, window) \ + inflateBackInit_((strm), (windowBits), (window), \ + ZLIB_VERSION, (int)sizeof(z_stream)) +#endif + +#ifndef Z_SOLO + +/* gzgetc() macro and its supporting function and exposed data structure. Note + * that the real internal state is much larger than the exposed structure. + * This abbreviated structure exposes just enough for the gzgetc() macro. The + * user should not mess with these exposed elements, since their names or + * behavior could change in the future, perhaps even capriciously. They can + * only be used by the gzgetc() macro. You have been warned. + */ +struct gzFile_s { + unsigned have; + unsigned char *next; + z_off64_t pos; +}; +ZEXTERN int ZEXPORT gzgetc_ OF((gzFile file)); /* backward compatibility */ +#ifdef Z_PREFIX_SET +# undef z_gzgetc +# define z_gzgetc(g) \ + ((g)->have ? ((g)->have--, (g)->pos++, *((g)->next)++) : (gzgetc)(g)) +#else +# define gzgetc(g) \ + ((g)->have ? ((g)->have--, (g)->pos++, *((g)->next)++) : (gzgetc)(g)) +#endif + +/* provide 64-bit offset functions if _LARGEFILE64_SOURCE defined, and/or + * change the regular functions to 64 bits if _FILE_OFFSET_BITS is 64 (if + * both are true, the application gets the *64 functions, and the regular + * functions are changed to 64 bits) -- in case these are set on systems + * without large file support, _LFS64_LARGEFILE must also be true + */ +#ifdef Z_LARGE64 + ZEXTERN gzFile ZEXPORT gzopen64 OF((const char *, const char *)); + ZEXTERN z_off64_t ZEXPORT gzseek64 OF((gzFile, z_off64_t, int)); + ZEXTERN z_off64_t ZEXPORT gztell64 OF((gzFile)); + ZEXTERN z_off64_t ZEXPORT gzoffset64 OF((gzFile)); + ZEXTERN uLong ZEXPORT adler32_combine64 OF((uLong, uLong, z_off64_t)); + ZEXTERN uLong ZEXPORT crc32_combine64 OF((uLong, uLong, z_off64_t)); +#endif + +#if !defined(ZLIB_INTERNAL) && defined(Z_WANT64) +# ifdef Z_PREFIX_SET +# define z_gzopen z_gzopen64 +# define z_gzseek z_gzseek64 +# define z_gztell z_gztell64 +# define z_gzoffset z_gzoffset64 +# define z_adler32_combine z_adler32_combine64 +# define z_crc32_combine z_crc32_combine64 +# else +# define gzopen gzopen64 +# define gzseek gzseek64 +# define gztell gztell64 +# define gzoffset gzoffset64 +# define adler32_combine adler32_combine64 +# define crc32_combine crc32_combine64 +# endif +# ifndef Z_LARGE64 + ZEXTERN gzFile ZEXPORT gzopen64 OF((const char *, const char *)); + ZEXTERN z_off_t ZEXPORT gzseek64 OF((gzFile, z_off_t, int)); + ZEXTERN z_off_t ZEXPORT gztell64 OF((gzFile)); + ZEXTERN z_off_t ZEXPORT gzoffset64 OF((gzFile)); + ZEXTERN uLong ZEXPORT adler32_combine64 OF((uLong, uLong, z_off_t)); + ZEXTERN uLong ZEXPORT crc32_combine64 OF((uLong, uLong, z_off_t)); +# endif +#else + ZEXTERN gzFile ZEXPORT gzopen OF((const char *, const char *)); + ZEXTERN z_off_t ZEXPORT gzseek OF((gzFile, z_off_t, int)); + ZEXTERN z_off_t ZEXPORT gztell OF((gzFile)); + ZEXTERN z_off_t ZEXPORT gzoffset OF((gzFile)); + ZEXTERN uLong ZEXPORT adler32_combine OF((uLong, uLong, z_off_t)); + ZEXTERN uLong ZEXPORT crc32_combine OF((uLong, uLong, z_off_t)); +#endif + +#else /* Z_SOLO */ + + ZEXTERN uLong ZEXPORT adler32_combine OF((uLong, uLong, z_off_t)); + ZEXTERN uLong ZEXPORT crc32_combine OF((uLong, uLong, z_off_t)); + +#endif /* !Z_SOLO */ + +/* undocumented functions */ +ZEXTERN const char * ZEXPORT zError OF((int)); +ZEXTERN int ZEXPORT inflateSyncPoint OF((z_streamp)); +ZEXTERN const z_crc_t FAR * ZEXPORT get_crc_table OF((void)); +ZEXTERN int ZEXPORT inflateUndermine OF((z_streamp, int)); +ZEXTERN int ZEXPORT inflateValidate OF((z_streamp, int)); +ZEXTERN unsigned long ZEXPORT inflateCodesUsed OF ((z_streamp)); +ZEXTERN int ZEXPORT inflateResetKeep OF((z_streamp)); +ZEXTERN int ZEXPORT deflateResetKeep OF((z_streamp)); +#if (defined(_WIN32) || defined(__CYGWIN__)) && !defined(Z_SOLO) +ZEXTERN gzFile ZEXPORT gzopen_w OF((const wchar_t *path, + const char *mode)); +#endif +#if defined(STDC) || defined(Z_HAVE_STDARG_H) +# ifndef Z_SOLO +ZEXTERN int ZEXPORTVA gzvprintf Z_ARG((gzFile file, + const char *format, + va_list va)); +# endif +#endif + +#ifdef __cplusplus +} +#endif + +#endif /* ZLIB_H */ +-------------------------------------------------------------------------------- +END LICENSE FOR external/zlib_archive/zlib.h +-------------------------------------------------------------------------------- +-------------------------------------------------------------------------------- +BEGIN LICENSE FOR external/grpc/LICENSE +-------------------------------------------------------------------------------- + + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +-------------------------------------------------------------------------------- +END LICENSE FOR external/grpc/LICENSE +-------------------------------------------------------------------------------- +-------------------------------------------------------------------------------- +BEGIN LICENSE FOR external/grpc/third_party/nanopb/LICENSE.txt +-------------------------------------------------------------------------------- +Copyright (c) 2011 Petteri Aimonen + +This software is provided 'as-is', without any express or +implied warranty. In no event will the authors be held liable +for any damages arising from the use of this software. + +Permission is granted to anyone to use this software for any +purpose, including commercial applications, and to alter it and +redistribute it freely, subject to the following restrictions: + +1. The origin of this software must not be misrepresented; you + must not claim that you wrote the original software. If you use + this software in a product, an acknowledgment in the product + documentation would be appreciated but is not required. + +2. Altered source versions must be plainly marked as such, and + must not be misrepresented as being the original software. + +3. This notice may not be removed or altered from any source + distribution. +-------------------------------------------------------------------------------- +END LICENSE FOR external/grpc/third_party/nanopb/LICENSE.txt +-------------------------------------------------------------------------------- +-------------------------------------------------------------------------------- +BEGIN LICENSE FOR external/grpc/third_party/address_sorting/LICENSE +-------------------------------------------------------------------------------- +Copyright (C) 1995, 1996, 1997, and 1998 WIDE Project. +All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions +are met: +1. Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. +2. Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in the + documentation and/or other materials provided with the distribution. +3. Neither the name of the project nor the names of its contributors + may be used to endorse or promote products derived from this software + without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE PROJECT AND CONTRIBUTORS ``AS IS'' AND +ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +ARE DISCLAIMED. IN NO EVENT SHALL THE PROJECT OR CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +SUCH DAMAGE. +-------------------------------------------------------------------------------- +END LICENSE FOR external/grpc/third_party/address_sorting/LICENSE +-------------------------------------------------------------------------------- +-------------------------------------------------------------------------------- +BEGIN LICENSE FOR external/aws/LICENSE +-------------------------------------------------------------------------------- + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "{}" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright {yyyy} {name of copyright owner} + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +-------------------------------------------------------------------------------- +END LICENSE FOR external/aws/LICENSE +-------------------------------------------------------------------------------- +-------------------------------------------------------------------------------- +BEGIN LICENSE FOR external/com_github_googlecloudplatform_google_cloud_cpp/LICENSE +-------------------------------------------------------------------------------- + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +-------------------------------------------------------------------------------- +END LICENSE FOR external/com_github_googlecloudplatform_google_cloud_cpp/LICENSE +-------------------------------------------------------------------------------- diff --git a/linux/include/tensorflow/c/c_api.h b/linux/include/tensorflow/c/c_api.h new file mode 100644 index 000000000..850f6ecd6 --- /dev/null +++ b/linux/include/tensorflow/c/c_api.h @@ -0,0 +1,1669 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_C_C_API_H_ +#define TENSORFLOW_C_C_API_H_ + +#include +#include + +// -------------------------------------------------------------------------- +// C API for TensorFlow. +// +// The API leans towards simplicity and uniformity instead of convenience +// since most usage will be by language specific wrappers. +// +// Conventions: +// * We use the prefix TF_ for everything in the API. +// * Objects are always passed around as pointers to opaque structs +// and these structs are allocated/deallocated via the API. +// * TF_Status holds error information. It is an object type +// and therefore is passed around as a pointer to an opaque +// struct as mentioned above. +// * Every call that has a TF_Status* argument clears it on success +// and fills it with error info on failure. +// * unsigned char is used for booleans (instead of the 'bool' type). +// In C++ bool is a keyword while in C99 bool is a macro defined +// in stdbool.h. It is possible for the two to be inconsistent. +// For example, neither the C99 nor the C++11 standard force a byte +// size on the bool type, so the macro defined in stdbool.h could +// be inconsistent with the bool keyword in C++. Thus, the use +// of stdbool.h is avoided and unsigned char is used instead. +// * size_t is used to represent byte sizes of objects that are +// materialized in the address space of the calling process. +// * int is used as an index into arrays. +// * Deletion functions are safe to call on nullptr. +// +// Questions left to address: +// * Might at some point need a way for callers to provide their own Env. +// * Maybe add TF_TensorShape that encapsulates dimension info. +// +// Design decisions made: +// * Backing store for tensor memory has an associated deallocation +// function. This deallocation function will point to client code +// for tensors populated by the client. So the client can do things +// like shadowing a numpy array. +// * We do not provide TF_OK since it is not strictly necessary and we +// are not optimizing for convenience. +// * We make assumption that one session has one graph. This should be +// fine since we have the ability to run sub-graphs. +// * We could allow NULL for some arguments (e.g., NULL options arg). +// However since convenience is not a primary goal, we don't do this. +// * Devices are not in this API. Instead, they are created/used internally +// and the API just provides high level controls over the number of +// devices of each type. + +// Macro to control visibility of exported symbols in the shared library (.so, +// .dylib, .dll). +// This duplicates the TF_EXPORT macro definition in +// tensorflow/core/platform/macros.h in order to keep this .h file independent +// of any other includes.$a +#ifdef SWIG +#define TF_CAPI_EXPORT +#else +#if defined(_WIN32) +#ifdef TF_COMPILE_LIBRARY +#define TF_CAPI_EXPORT __declspec(dllexport) +#else +#define TF_CAPI_EXPORT __declspec(dllimport) +#endif // TF_COMPILE_LIBRARY +#else +#define TF_CAPI_EXPORT __attribute__((visibility("default"))) +#endif // _WIN32 +#endif // SWIG + +#ifdef __cplusplus +extern "C" { +#endif + +// -------------------------------------------------------------------------- +// TF_Version returns a string describing version information of the +// TensorFlow library. TensorFlow using semantic versioning. +TF_CAPI_EXPORT extern const char* TF_Version(); + +// -------------------------------------------------------------------------- +// TF_DataType holds the type for a scalar value. E.g., one slot in a tensor. +// The enum values here are identical to corresponding values in types.proto. +typedef enum TF_DataType { + TF_FLOAT = 1, + TF_DOUBLE = 2, + TF_INT32 = 3, // Int32 tensors are always in 'host' memory. + TF_UINT8 = 4, + TF_INT16 = 5, + TF_INT8 = 6, + TF_STRING = 7, + TF_COMPLEX64 = 8, // Single-precision complex + TF_COMPLEX = 8, // Old identifier kept for API backwards compatibility + TF_INT64 = 9, + TF_BOOL = 10, + TF_QINT8 = 11, // Quantized int8 + TF_QUINT8 = 12, // Quantized uint8 + TF_QINT32 = 13, // Quantized int32 + TF_BFLOAT16 = 14, // Float32 truncated to 16 bits. Only for cast ops. + TF_QINT16 = 15, // Quantized int16 + TF_QUINT16 = 16, // Quantized uint16 + TF_UINT16 = 17, + TF_COMPLEX128 = 18, // Double-precision complex + TF_HALF = 19, + TF_RESOURCE = 20, + TF_VARIANT = 21, + TF_UINT32 = 22, + TF_UINT64 = 23, +} TF_DataType; + +// TF_DataTypeSize returns the sizeof() for the underlying type corresponding +// to the given TF_DataType enum value. Returns 0 for variable length types +// (eg. TF_STRING) or on failure. +TF_CAPI_EXPORT extern size_t TF_DataTypeSize(TF_DataType dt); + +// -------------------------------------------------------------------------- +// TF_Code holds an error code. The enum values here are identical to +// corresponding values in error_codes.proto. +typedef enum TF_Code { + TF_OK = 0, + TF_CANCELLED = 1, + TF_UNKNOWN = 2, + TF_INVALID_ARGUMENT = 3, + TF_DEADLINE_EXCEEDED = 4, + TF_NOT_FOUND = 5, + TF_ALREADY_EXISTS = 6, + TF_PERMISSION_DENIED = 7, + TF_UNAUTHENTICATED = 16, + TF_RESOURCE_EXHAUSTED = 8, + TF_FAILED_PRECONDITION = 9, + TF_ABORTED = 10, + TF_OUT_OF_RANGE = 11, + TF_UNIMPLEMENTED = 12, + TF_INTERNAL = 13, + TF_UNAVAILABLE = 14, + TF_DATA_LOSS = 15, +} TF_Code; + +// -------------------------------------------------------------------------- +// TF_Status holds error information. It either has an OK code, or +// else an error code with an associated error message. +typedef struct TF_Status TF_Status; + +// Return a new status object. +TF_CAPI_EXPORT extern TF_Status* TF_NewStatus(); + +// Delete a previously created status object. +TF_CAPI_EXPORT extern void TF_DeleteStatus(TF_Status*); + +// Record in *s. Any previous information is lost. +// A common use is to clear a status: TF_SetStatus(s, TF_OK, ""); +TF_CAPI_EXPORT extern void TF_SetStatus(TF_Status* s, TF_Code code, + const char* msg); + +// Return the code record in *s. +TF_CAPI_EXPORT extern TF_Code TF_GetCode(const TF_Status* s); + +// Return a pointer to the (null-terminated) error message in *s. The +// return value points to memory that is only usable until the next +// mutation to *s. Always returns an empty string if TF_GetCode(s) is +// TF_OK. +TF_CAPI_EXPORT extern const char* TF_Message(const TF_Status* s); + +// -------------------------------------------------------------------------- +// TF_Buffer holds a pointer to a block of data and its associated length. +// Typically, the data consists of a serialized protocol buffer, but other data +// may also be held in a buffer. +// +// By default, TF_Buffer itself does not do any memory management of the +// pointed-to block. If need be, users of this struct should specify how to +// deallocate the block by setting the `data_deallocator` function pointer. +typedef struct TF_Buffer { + const void* data; + size_t length; + void (*data_deallocator)(void* data, size_t length); +} TF_Buffer; + +// Makes a copy of the input and sets an appropriate deallocator. Useful for +// passing in read-only, input protobufs. +TF_CAPI_EXPORT extern TF_Buffer* TF_NewBufferFromString(const void* proto, + size_t proto_len); + +// Useful for passing *out* a protobuf. +TF_CAPI_EXPORT extern TF_Buffer* TF_NewBuffer(); + +TF_CAPI_EXPORT extern void TF_DeleteBuffer(TF_Buffer*); + +TF_CAPI_EXPORT extern TF_Buffer TF_GetBuffer(TF_Buffer* buffer); + +// -------------------------------------------------------------------------- +// TF_Tensor holds a multi-dimensional array of elements of a single data type. +// For all types other than TF_STRING, the data buffer stores elements +// in row major order. E.g. if data is treated as a vector of TF_DataType: +// +// element 0: index (0, ..., 0) +// element 1: index (0, ..., 1) +// ... +// +// The format for TF_STRING tensors is: +// start_offset: array[uint64] +// data: byte[...] +// +// The string length (as a varint), followed by the contents of the string +// is encoded at data[start_offset[i]]]. TF_StringEncode and TF_StringDecode +// facilitate this encoding. + +typedef struct TF_Tensor TF_Tensor; + +// Return a new tensor that holds the bytes data[0,len-1]. +// +// The data will be deallocated by a subsequent call to TF_DeleteTensor via: +// (*deallocator)(data, len, deallocator_arg) +// Clients must provide a custom deallocator function so they can pass in +// memory managed by something like numpy. +// +// May return NULL (and invoke the deallocator) if the provided data buffer +// (data, len) is inconsistent with a tensor of the given TF_DataType +// and the shape specified by (dima, num_dims). +TF_CAPI_EXPORT extern TF_Tensor* TF_NewTensor( + TF_DataType, const int64_t* dims, int num_dims, void* data, size_t len, + void (*deallocator)(void* data, size_t len, void* arg), + void* deallocator_arg); + +// Allocate and return a new Tensor. +// +// This function is an alternative to TF_NewTensor and should be used when +// memory is allocated to pass the Tensor to the C API. The allocated memory +// satisfies TensorFlow's memory alignment preferences and should be preferred +// over calling malloc and free. +// +// The caller must set the Tensor values by writing them to the pointer returned +// by TF_TensorData with length TF_TensorByteSize. +TF_CAPI_EXPORT extern TF_Tensor* TF_AllocateTensor(TF_DataType, + const int64_t* dims, + int num_dims, size_t len); + +// Deletes `tensor` and returns a new TF_Tensor with the same content if +// possible. Returns nullptr and leaves `tensor` untouched if not. +TF_CAPI_EXPORT extern TF_Tensor* TF_TensorMaybeMove(TF_Tensor* tensor); + +// Destroy a tensor. +TF_CAPI_EXPORT extern void TF_DeleteTensor(TF_Tensor*); + +// Return the type of a tensor element. +TF_CAPI_EXPORT extern TF_DataType TF_TensorType(const TF_Tensor*); + +// Return the number of dimensions that the tensor has. +TF_CAPI_EXPORT extern int TF_NumDims(const TF_Tensor*); + +// Return the length of the tensor in the "dim_index" dimension. +// REQUIRES: 0 <= dim_index < TF_NumDims(tensor) +TF_CAPI_EXPORT extern int64_t TF_Dim(const TF_Tensor* tensor, int dim_index); + +// Return the size of the underlying data in bytes. +TF_CAPI_EXPORT extern size_t TF_TensorByteSize(const TF_Tensor*); + +// Return a pointer to the underlying data buffer. +TF_CAPI_EXPORT extern void* TF_TensorData(const TF_Tensor*); + +// -------------------------------------------------------------------------- +// Encode the string `src` (`src_len` bytes long) into `dst` in the format +// required by TF_STRING tensors. Does not write to memory more than `dst_len` +// bytes beyond `*dst`. `dst_len` should be at least +// TF_StringEncodedSize(src_len). +// +// On success returns the size in bytes of the encoded string. +// Returns an error into `status` otherwise. +TF_CAPI_EXPORT extern size_t TF_StringEncode(const char* src, size_t src_len, + char* dst, size_t dst_len, + TF_Status* status); + +// Decode a string encoded using TF_StringEncode. +// +// On success, sets `*dst` to the start of the decoded string and `*dst_len` to +// its length. Returns the number of bytes starting at `src` consumed while +// decoding. `*dst` points to memory within the encoded buffer. On failure, +// `*dst` and `*dst_len` are undefined and an error is set in `status`. +// +// Does not read memory more than `src_len` bytes beyond `src`. +TF_CAPI_EXPORT extern size_t TF_StringDecode(const char* src, size_t src_len, + const char** dst, size_t* dst_len, + TF_Status* status); + +// Return the size in bytes required to encode a string `len` bytes long into a +// TF_STRING tensor. +TF_CAPI_EXPORT extern size_t TF_StringEncodedSize(size_t len); + +// -------------------------------------------------------------------------- +// TF_SessionOptions holds options that can be passed during session creation. +typedef struct TF_SessionOptions TF_SessionOptions; + +// Return a new options object. +TF_CAPI_EXPORT extern TF_SessionOptions* TF_NewSessionOptions(); + +// Set the target in TF_SessionOptions.options. +// target can be empty, a single entry, or a comma separated list of entries. +// Each entry is in one of the following formats : +// "local" +// ip:port +// host:port +TF_CAPI_EXPORT extern void TF_SetTarget(TF_SessionOptions* options, + const char* target); + +// Set the config in TF_SessionOptions.options. +// config should be a serialized tensorflow.ConfigProto proto. +// If config was not parsed successfully as a ConfigProto, record the +// error information in *status. +TF_CAPI_EXPORT extern void TF_SetConfig(TF_SessionOptions* options, + const void* proto, size_t proto_len, + TF_Status* status); + +// Destroy an options object. +TF_CAPI_EXPORT extern void TF_DeleteSessionOptions(TF_SessionOptions*); + +// TODO(jeff,sanjay): +// - export functions to set Config fields + +// -------------------------------------------------------------------------- +// The new graph construction API, still under development. + +// Represents a computation graph. Graphs may be shared between sessions. +// Graphs are thread-safe when used as directed below. +typedef struct TF_Graph TF_Graph; + +// Return a new graph object. +TF_CAPI_EXPORT extern TF_Graph* TF_NewGraph(); + +// Destroy an options object. Graph will be deleted once no more +// TFSession's are referencing it. +TF_CAPI_EXPORT extern void TF_DeleteGraph(TF_Graph*); + +// Operation being built. The underlying graph must outlive this. +typedef struct TF_OperationDescription TF_OperationDescription; + +// Operation that has been added to the graph. Valid until the graph is +// deleted -- in particular adding a new operation to the graph does not +// invalidate old TF_Operation* pointers. +typedef struct TF_Operation TF_Operation; + +// Represents a specific input of an operation. +typedef struct TF_Input { + TF_Operation* oper; + int index; // The index of the input within oper. +} TF_Input; + +// Represents a specific output of an operation. +typedef struct TF_Output { + TF_Operation* oper; + int index; // The index of the output within oper. +} TF_Output; + +// TF_Function is a grouping of operations with defined inputs and outputs. +// Once created and added to graphs, functions can be invoked by creating an +// operation whose operation type matches the function name. +typedef struct TF_Function TF_Function; + +// Function definition options. TODO(iga): Define and implement +typedef struct TF_FunctionOptions TF_FunctionOptions; + +// Sets the shape of the Tensor referenced by `output` in `graph` to +// the shape described by `dims` and `num_dims`. +// +// If the number of dimensions is unknown, `num_dims` must be set to +// -1 and `dims` can be null. If a dimension is unknown, the +// corresponding entry in the `dims` array must be -1. +// +// This does not overwrite the existing shape associated with `output`, +// but merges the input shape with the existing shape. For example, +// setting a shape of [-1, 2] with an existing shape [2, -1] would set +// a final shape of [2, 2] based on shape merging semantics. +// +// Returns an error into `status` if: +// * `output` is not in `graph`. +// * An invalid shape is being set (e.g., the shape being set +// is incompatible with the existing shape). +TF_CAPI_EXPORT extern void TF_GraphSetTensorShape(TF_Graph* graph, + TF_Output output, + const int64_t* dims, + const int num_dims, + TF_Status* status); + +// Returns the number of dimensions of the Tensor referenced by `output` +// in `graph`. +// +// If the number of dimensions in the shape is unknown, returns -1. +// +// Returns an error into `status` if: +// * `output` is not in `graph`. +TF_CAPI_EXPORT extern int TF_GraphGetTensorNumDims(TF_Graph* graph, + TF_Output output, + TF_Status* status); + +// Returns the shape of the Tensor referenced by `output` in `graph` +// into `dims`. `dims` must be an array large enough to hold `num_dims` +// entries (e.g., the return value of TF_GraphGetTensorNumDims). +// +// If the number of dimensions in the shape is unknown or the shape is +// a scalar, `dims` will remain untouched. Otherwise, each element of +// `dims` will be set corresponding to the size of the dimension. An +// unknown dimension is represented by `-1`. +// +// Returns an error into `status` if: +// * `output` is not in `graph`. +// * `num_dims` does not match the actual number of dimensions. +TF_CAPI_EXPORT extern void TF_GraphGetTensorShape(TF_Graph* graph, + TF_Output output, + int64_t* dims, int num_dims, + TF_Status* status); + +// Operation will only be added to *graph when TF_FinishOperation() is +// called (assuming TF_FinishOperation() does not return an error). +// *graph must not be deleted until after TF_FinishOperation() is +// called. +TF_CAPI_EXPORT extern TF_OperationDescription* TF_NewOperation( + TF_Graph* graph, const char* op_type, const char* oper_name); + +// Specify the device for `desc`. Defaults to empty, meaning unconstrained. +TF_CAPI_EXPORT extern void TF_SetDevice(TF_OperationDescription* desc, + const char* device); + +// The calls to TF_AddInput and TF_AddInputList must match (in number, +// order, and type) the op declaration. For example, the "Concat" op +// has registration: +// REGISTER_OP("Concat") +// .Input("concat_dim: int32") +// .Input("values: N * T") +// .Output("output: T") +// .Attr("N: int >= 2") +// .Attr("T: type"); +// that defines two inputs, "concat_dim" and "values" (in that order). +// You must use TF_AddInput() for the first input (since it takes a +// single tensor), and TF_AddInputList() for the second input (since +// it takes a list, even if you were to pass a list with a single +// tensor), as in: +// TF_OperationDescription* desc = TF_NewOperation(graph, "Concat", "c"); +// TF_Output concat_dim_input = {...}; +// TF_AddInput(desc, concat_dim_input); +// TF_Output values_inputs[5] = {{...}, ..., {...}}; +// TF_AddInputList(desc, values_inputs, 5); + +// For inputs that take a single tensor. +TF_CAPI_EXPORT extern void TF_AddInput(TF_OperationDescription* desc, + TF_Output input); + +// For inputs that take a list of tensors. +// inputs must point to TF_Output[num_inputs]. +TF_CAPI_EXPORT extern void TF_AddInputList(TF_OperationDescription* desc, + const TF_Output* inputs, + int num_inputs); + +// Call once per control input to `desc`. +TF_CAPI_EXPORT extern void TF_AddControlInput(TF_OperationDescription* desc, + TF_Operation* input); + +// Request that `desc` be co-located on the device where `op` +// is placed. +// +// Use of this is discouraged since the implementation of device placement is +// subject to change. Primarily intended for internal libraries +TF_CAPI_EXPORT extern void TF_ColocateWith(TF_OperationDescription* desc, + TF_Operation* op); + +// Call some TF_SetAttr*() function for every attr that is not +// inferred from an input and doesn't have a default value you wish to +// keep. + +// `value` must point to a string of length `length` bytes. +TF_CAPI_EXPORT extern void TF_SetAttrString(TF_OperationDescription* desc, + const char* attr_name, + const void* value, size_t length); +// `values` and `lengths` each must have lengths `num_values`. +// `values[i]` must point to a string of length `lengths[i]` bytes. +TF_CAPI_EXPORT extern void TF_SetAttrStringList(TF_OperationDescription* desc, + const char* attr_name, + const void* const* values, + const size_t* lengths, + int num_values); +TF_CAPI_EXPORT extern void TF_SetAttrInt(TF_OperationDescription* desc, + const char* attr_name, int64_t value); +TF_CAPI_EXPORT extern void TF_SetAttrIntList(TF_OperationDescription* desc, + const char* attr_name, + const int64_t* values, + int num_values); +TF_CAPI_EXPORT extern void TF_SetAttrFloat(TF_OperationDescription* desc, + const char* attr_name, float value); +TF_CAPI_EXPORT extern void TF_SetAttrFloatList(TF_OperationDescription* desc, + const char* attr_name, + const float* values, + int num_values); +TF_CAPI_EXPORT extern void TF_SetAttrBool(TF_OperationDescription* desc, + const char* attr_name, + unsigned char value); +TF_CAPI_EXPORT extern void TF_SetAttrBoolList(TF_OperationDescription* desc, + const char* attr_name, + const unsigned char* values, + int num_values); +TF_CAPI_EXPORT extern void TF_SetAttrType(TF_OperationDescription* desc, + const char* attr_name, + TF_DataType value); +TF_CAPI_EXPORT extern void TF_SetAttrTypeList(TF_OperationDescription* desc, + const char* attr_name, + const TF_DataType* values, + int num_values); +// Set a 'func' attribute to the specified name. +// `value` must point to a string of length `length` bytes. +TF_CAPI_EXPORT extern void TF_SetAttrFuncName(TF_OperationDescription* desc, + const char* attr_name, + const char* value, size_t length); + +// Set `num_dims` to -1 to represent "unknown rank". Otherwise, +// `dims` points to an array of length `num_dims`. `dims[i]` must be +// >= -1, with -1 meaning "unknown dimension". +TF_CAPI_EXPORT extern void TF_SetAttrShape(TF_OperationDescription* desc, + const char* attr_name, + const int64_t* dims, int num_dims); +// `dims` and `num_dims` must point to arrays of length `num_shapes`. +// Set `num_dims[i]` to -1 to represent "unknown rank". Otherwise, +// `dims[i]` points to an array of length `num_dims[i]`. `dims[i][j]` +// must be >= -1, with -1 meaning "unknown dimension". +TF_CAPI_EXPORT extern void TF_SetAttrShapeList(TF_OperationDescription* desc, + const char* attr_name, + const int64_t* const* dims, + const int* num_dims, + int num_shapes); +// `proto` must point to an array of `proto_len` bytes representing a +// binary-serialized TensorShapeProto. +TF_CAPI_EXPORT extern void TF_SetAttrTensorShapeProto( + TF_OperationDescription* desc, const char* attr_name, const void* proto, + size_t proto_len, TF_Status* status); +// `protos` and `proto_lens` must point to arrays of length `num_shapes`. +// `protos[i]` must point to an array of `proto_lens[i]` bytes +// representing a binary-serialized TensorShapeProto. +TF_CAPI_EXPORT extern void TF_SetAttrTensorShapeProtoList( + TF_OperationDescription* desc, const char* attr_name, + const void* const* protos, const size_t* proto_lens, int num_shapes, + TF_Status* status); + +TF_CAPI_EXPORT extern void TF_SetAttrTensor(TF_OperationDescription* desc, + const char* attr_name, + TF_Tensor* value, + TF_Status* status); +TF_CAPI_EXPORT extern void TF_SetAttrTensorList(TF_OperationDescription* desc, + const char* attr_name, + TF_Tensor* const* values, + int num_values, + TF_Status* status); + +// `proto` should point to a sequence of bytes of length `proto_len` +// representing a binary serialization of an AttrValue protocol +// buffer. +TF_CAPI_EXPORT extern void TF_SetAttrValueProto(TF_OperationDescription* desc, + const char* attr_name, + const void* proto, + size_t proto_len, + TF_Status* status); + +// If this function succeeds: +// * *status is set to an OK value, +// * a TF_Operation is added to the graph, +// * a non-null value pointing to the added operation is returned -- +// this value is valid until the underlying graph is deleted. +// Otherwise: +// * *status is set to a non-OK value, +// * the graph is not modified, +// * a null value is returned. +// In either case, it deletes `desc`. +TF_CAPI_EXPORT extern TF_Operation* TF_FinishOperation( + TF_OperationDescription* desc, TF_Status* status); + +// TF_Operation functions. Operations are immutable once created, so +// these are all query functions. + +TF_CAPI_EXPORT extern const char* TF_OperationName(TF_Operation* oper); +TF_CAPI_EXPORT extern const char* TF_OperationOpType(TF_Operation* oper); +TF_CAPI_EXPORT extern const char* TF_OperationDevice(TF_Operation* oper); + +TF_CAPI_EXPORT extern int TF_OperationNumOutputs(TF_Operation* oper); +TF_CAPI_EXPORT extern TF_DataType TF_OperationOutputType(TF_Output oper_out); +TF_CAPI_EXPORT extern int TF_OperationOutputListLength(TF_Operation* oper, + const char* arg_name, + TF_Status* status); + +TF_CAPI_EXPORT extern int TF_OperationNumInputs(TF_Operation* oper); +TF_CAPI_EXPORT extern TF_DataType TF_OperationInputType(TF_Input oper_in); +TF_CAPI_EXPORT extern int TF_OperationInputListLength(TF_Operation* oper, + const char* arg_name, + TF_Status* status); + +// In this code: +// TF_Output producer = TF_OperationInput(consumer); +// There is an edge from producer.oper's output (given by +// producer.index) to consumer.oper's input (given by consumer.index). +TF_CAPI_EXPORT extern TF_Output TF_OperationInput(TF_Input oper_in); + +// Get the number of current consumers of a specific output of an +// operation. Note that this number can change when new operations +// are added to the graph. +TF_CAPI_EXPORT extern int TF_OperationOutputNumConsumers(TF_Output oper_out); + +// Get list of all current consumers of a specific output of an +// operation. `consumers` must point to an array of length at least +// `max_consumers` (ideally set to +// TF_OperationOutputNumConsumers(oper_out)). Beware that a concurrent +// modification of the graph can increase the number of consumers of +// an operation. Returns the number of output consumers (should match +// TF_OperationOutputNumConsumers(oper_out)). +TF_CAPI_EXPORT extern int TF_OperationOutputConsumers(TF_Output oper_out, + TF_Input* consumers, + int max_consumers); + +// Get the number of control inputs to an operation. +TF_CAPI_EXPORT extern int TF_OperationNumControlInputs(TF_Operation* oper); + +// Get list of all control inputs to an operation. `control_inputs` must +// point to an array of length `max_control_inputs` (ideally set to +// TF_OperationNumControlInputs(oper)). Returns the number of control +// inputs (should match TF_OperationNumControlInputs(oper)). +TF_CAPI_EXPORT extern int TF_OperationGetControlInputs( + TF_Operation* oper, TF_Operation** control_inputs, int max_control_inputs); + +// Get the number of operations that have `*oper` as a control input. +// Note that this number can change when new operations are added to +// the graph. +TF_CAPI_EXPORT extern int TF_OperationNumControlOutputs(TF_Operation* oper); + +// Get the list of operations that have `*oper` as a control input. +// `control_outputs` must point to an array of length at least +// `max_control_outputs` (ideally set to +// TF_OperationNumControlOutputs(oper)). Beware that a concurrent +// modification of the graph can increase the number of control +// outputs. Returns the number of control outputs (should match +// TF_OperationNumControlOutputs(oper)). +TF_CAPI_EXPORT extern int TF_OperationGetControlOutputs( + TF_Operation* oper, TF_Operation** control_outputs, + int max_control_outputs); + +// TF_AttrType describes the type of the value of an attribute on an operation. +typedef enum TF_AttrType { + TF_ATTR_STRING = 0, + TF_ATTR_INT = 1, + TF_ATTR_FLOAT = 2, + TF_ATTR_BOOL = 3, + TF_ATTR_TYPE = 4, + TF_ATTR_SHAPE = 5, + TF_ATTR_TENSOR = 6, + TF_ATTR_PLACEHOLDER = 7, + TF_ATTR_FUNC = 8, +} TF_AttrType; + +// TF_AttrMetadata describes the value of an attribute on an operation. +typedef struct TF_AttrMetadata { + // A boolean: 1 if the attribute value is a list, 0 otherwise. + unsigned char is_list; + + // Length of the list if is_list is true. Undefined otherwise. + int64_t list_size; + + // Type of elements of the list if is_list != 0. + // Type of the single value stored in the attribute if is_list == 0. + TF_AttrType type; + + // Total size the attribute value. + // The units of total_size depend on is_list and type. + // (1) If type == TF_ATTR_STRING and is_list == 0 + // then total_size is the byte size of the string + // valued attribute. + // (2) If type == TF_ATTR_STRING and is_list == 1 + // then total_size is the cumulative byte size + // of all the strings in the list. + // (3) If type == TF_ATTR_SHAPE and is_list == 0 + // then total_size is the number of dimensions + // of the shape valued attribute, or -1 + // if its rank is unknown. + // (4) If type == TF_ATTR_SHAPE and is_list == 1 + // then total_size is the cumulative number + // of dimensions of all shapes in the list. + // (5) Otherwise, total_size is undefined. + int64_t total_size; +} TF_AttrMetadata; + +// Returns metadata about the value of the attribute `attr_name` of `oper`. +TF_CAPI_EXPORT extern TF_AttrMetadata TF_OperationGetAttrMetadata( + TF_Operation* oper, const char* attr_name, TF_Status* status); + +// Fills in `value` with the value of the attribute `attr_name`. `value` must +// point to an array of length at least `max_length` (ideally set to +// TF_AttrMetadata.total_size from TF_OperationGetAttrMetadata(oper, +// attr_name)). +TF_CAPI_EXPORT extern void TF_OperationGetAttrString(TF_Operation* oper, + const char* attr_name, + void* value, + size_t max_length, + TF_Status* status); + +// Get the list of strings in the value of the attribute `attr_name`. Fills in +// `values` and `lengths`, each of which must point to an array of length at +// least `max_values`. +// +// The elements of values will point to addresses in `storage` which must be at +// least `storage_size` bytes in length. Ideally, max_values would be set to +// TF_AttrMetadata.list_size and `storage` would be at least +// TF_AttrMetadata.total_size, obtained from TF_OperationGetAttrMetadata(oper, +// attr_name). +// +// Fails if storage_size is too small to hold the requested number of strings. +TF_CAPI_EXPORT extern void TF_OperationGetAttrStringList( + TF_Operation* oper, const char* attr_name, void** values, size_t* lengths, + int max_values, void* storage, size_t storage_size, TF_Status* status); + +TF_CAPI_EXPORT extern void TF_OperationGetAttrInt(TF_Operation* oper, + const char* attr_name, + int64_t* value, + TF_Status* status); + +// Fills in `values` with the value of the attribute `attr_name` of `oper`. +// `values` must point to an array of length at least `max_values` (ideally set +// TF_AttrMetadata.list_size from TF_OperationGetAttrMetadata(oper, +// attr_name)). +TF_CAPI_EXPORT extern void TF_OperationGetAttrIntList(TF_Operation* oper, + const char* attr_name, + int64_t* values, + int max_values, + TF_Status* status); + +TF_CAPI_EXPORT extern void TF_OperationGetAttrFloat(TF_Operation* oper, + const char* attr_name, + float* value, + TF_Status* status); + +// Fills in `values` with the value of the attribute `attr_name` of `oper`. +// `values` must point to an array of length at least `max_values` (ideally set +// to TF_AttrMetadata.list_size from TF_OperationGetAttrMetadata(oper, +// attr_name)). +TF_CAPI_EXPORT extern void TF_OperationGetAttrFloatList(TF_Operation* oper, + const char* attr_name, + float* values, + int max_values, + TF_Status* status); + +TF_CAPI_EXPORT extern void TF_OperationGetAttrBool(TF_Operation* oper, + const char* attr_name, + unsigned char* value, + TF_Status* status); + +// Fills in `values` with the value of the attribute `attr_name` of `oper`. +// `values` must point to an array of length at least `max_values` (ideally set +// to TF_AttrMetadata.list_size from TF_OperationGetAttrMetadata(oper, +// attr_name)). +TF_CAPI_EXPORT extern void TF_OperationGetAttrBoolList(TF_Operation* oper, + const char* attr_name, + unsigned char* values, + int max_values, + TF_Status* status); + +TF_CAPI_EXPORT extern void TF_OperationGetAttrType(TF_Operation* oper, + const char* attr_name, + TF_DataType* value, + TF_Status* status); + +// Fills in `values` with the value of the attribute `attr_name` of `oper`. +// `values` must point to an array of length at least `max_values` (ideally set +// to TF_AttrMetadata.list_size from TF_OperationGetAttrMetadata(oper, +// attr_name)). +TF_CAPI_EXPORT extern void TF_OperationGetAttrTypeList(TF_Operation* oper, + const char* attr_name, + TF_DataType* values, + int max_values, + TF_Status* status); + +// Fills in `value` with the value of the attribute `attr_name` of `oper`. +// `values` must point to an array of length at least `num_dims` (ideally set to +// TF_Attr_Meta.size from TF_OperationGetAttrMetadata(oper, attr_name)). +TF_CAPI_EXPORT extern void TF_OperationGetAttrShape(TF_Operation* oper, + const char* attr_name, + int64_t* value, + int num_dims, + TF_Status* status); + +// Fills in `dims` with the list of shapes in the attribute `attr_name` of +// `oper` and `num_dims` with the corresponding number of dimensions. On return, +// for every i where `num_dims[i]` > 0, `dims[i]` will be an array of +// `num_dims[i]` elements. A value of -1 for `num_dims[i]` indicates that the +// i-th shape in the list is unknown. +// +// The elements of `dims` will point to addresses in `storage` which must be +// large enough to hold at least `storage_size` int64_ts. Ideally, `num_shapes` +// would be set to TF_AttrMetadata.list_size and `storage_size` would be set to +// TF_AttrMetadata.total_size from TF_OperationGetAttrMetadata(oper, +// attr_name). +// +// Fails if storage_size is insufficient to hold the requested shapes. +TF_CAPI_EXPORT extern void TF_OperationGetAttrShapeList( + TF_Operation* oper, const char* attr_name, int64_t** dims, int* num_dims, + int num_shapes, int64_t* storage, int storage_size, TF_Status* status); + +// Sets `value` to the binary-serialized TensorShapeProto of the value of +// `attr_name` attribute of `oper`'. +TF_CAPI_EXPORT extern void TF_OperationGetAttrTensorShapeProto( + TF_Operation* oper, const char* attr_name, TF_Buffer* value, + TF_Status* status); + +// Fills in `values` with binary-serialized TensorShapeProto values of the +// attribute `attr_name` of `oper`. `values` must point to an array of length at +// least `num_values` (ideally set to TF_AttrMetadata.list_size from +// TF_OperationGetAttrMetadata(oper, attr_name)). +TF_CAPI_EXPORT extern void TF_OperationGetAttrTensorShapeProtoList( + TF_Operation* oper, const char* attr_name, TF_Buffer** values, + int max_values, TF_Status* status); + +// Gets the TF_Tensor valued attribute of `attr_name` of `oper`. +// +// Allocates a new TF_Tensor which the caller is expected to take +// ownership of (and can deallocate using TF_DeleteTensor). +TF_CAPI_EXPORT extern void TF_OperationGetAttrTensor(TF_Operation* oper, + const char* attr_name, + TF_Tensor** value, + TF_Status* status); + +// Fills in `values` with the TF_Tensor values of the attribute `attr_name` of +// `oper`. `values` must point to an array of TF_Tensor* of length at least +// `max_values` (ideally set to TF_AttrMetadata.list_size from +// TF_OperationGetAttrMetadata(oper, attr_name)). +// +// The caller takes ownership of all the non-null TF_Tensor* entries in `values` +// (which can be deleted using TF_DeleteTensor(values[i])). +TF_CAPI_EXPORT extern void TF_OperationGetAttrTensorList(TF_Operation* oper, + const char* attr_name, + TF_Tensor** values, + int max_values, + TF_Status* status); + +// Sets `output_attr_value` to the binary-serialized AttrValue proto +// representation of the value of the `attr_name` attr of `oper`. +TF_CAPI_EXPORT extern void TF_OperationGetAttrValueProto( + TF_Operation* oper, const char* attr_name, TF_Buffer* output_attr_value, + TF_Status* status); + +// Returns the operation in the graph with `oper_name`. Returns nullptr if +// no operation found. +TF_CAPI_EXPORT extern TF_Operation* TF_GraphOperationByName( + TF_Graph* graph, const char* oper_name); + +// Iterate through the operations of a graph. To use: +// size_t pos = 0; +// TF_Operation* oper; +// while ((oper = TF_GraphNextOperation(graph, &pos)) != nullptr) { +// DoSomethingWithOperation(oper); +// } +TF_CAPI_EXPORT extern TF_Operation* TF_GraphNextOperation(TF_Graph* graph, + size_t* pos); + +// Write out a serialized representation of `graph` (as a GraphDef protocol +// message) to `output_graph_def` (allocated by TF_NewBuffer()). +// `output_graph_def`'s underlying buffer will be freed when TF_DeleteBuffer() +// is called. +// +// May fail on very large graphs in the future. +TF_CAPI_EXPORT extern void TF_GraphToGraphDef(TF_Graph* graph, + TF_Buffer* output_graph_def, + TF_Status* status); + +// Returns the serialized OpDef proto with name `op_name`, or a bad status if no +// such op exists. This can return OpDefs of functions copied into the graph. +TF_CAPI_EXPORT extern void TF_GraphGetOpDef(TF_Graph* graph, + const char* op_name, + TF_Buffer* output_op_def, + TF_Status* status); + +// Returns the serialized VersionDef proto for this graph. +TF_CAPI_EXPORT extern void TF_GraphVersions(TF_Graph* graph, + TF_Buffer* output_version_def, + TF_Status* status); + +// TF_ImportGraphDefOptions holds options that can be passed to +// TF_GraphImportGraphDef. +typedef struct TF_ImportGraphDefOptions TF_ImportGraphDefOptions; + +TF_CAPI_EXPORT extern TF_ImportGraphDefOptions* TF_NewImportGraphDefOptions(); +TF_CAPI_EXPORT extern void TF_DeleteImportGraphDefOptions( + TF_ImportGraphDefOptions* opts); + +// Set the prefix to be prepended to the names of nodes in `graph_def` that will +// be imported into `graph`. `prefix` is copied and has no lifetime +// requirements. +TF_CAPI_EXPORT extern void TF_ImportGraphDefOptionsSetPrefix( + TF_ImportGraphDefOptions* opts, const char* prefix); + +// Set whether to uniquify imported operation names. If true, imported operation +// names will be modified if their name already exists in the graph. If false, +// conflicting names will be treated as an error. Note that this option has no +// effect if a prefix is set, since the prefix will guarantee all names are +// unique. Defaults to false. +TF_CAPI_EXPORT extern void TF_ImportGraphDefOptionsSetUniquifyNames( + TF_ImportGraphDefOptions* opts, unsigned char uniquify_names); + +// If true, the specified prefix will be modified if it already exists as an +// operation name or prefix in the graph. If false, a conflicting prefix will be +// treated as an error. This option has no effect if no prefix is specified. +TF_CAPI_EXPORT extern void TF_ImportGraphDefOptionsSetUniquifyPrefix( + TF_ImportGraphDefOptions* opts, unsigned char uniquify_prefix); + +// Set any imported nodes with input `src_name:src_index` to have that input +// replaced with `dst`. `src_name` refers to a node in the graph to be imported, +// `dst` references a node already existing in the graph being imported into. +// `src_name` is copied and has no lifetime requirements. +TF_CAPI_EXPORT extern void TF_ImportGraphDefOptionsAddInputMapping( + TF_ImportGraphDefOptions* opts, const char* src_name, int src_index, + TF_Output dst); + +// Set any imported nodes with control input `src_name` to have that input +// replaced with `dst`. `src_name` refers to a node in the graph to be imported, +// `dst` references an operation already existing in the graph being imported +// into. `src_name` is copied and has no lifetime requirements. +TF_CAPI_EXPORT extern void TF_ImportGraphDefOptionsRemapControlDependency( + TF_ImportGraphDefOptions* opts, const char* src_name, TF_Operation* dst); + +// Cause the imported graph to have a control dependency on `oper`. `oper` +// should exist in the graph being imported into. +TF_CAPI_EXPORT extern void TF_ImportGraphDefOptionsAddControlDependency( + TF_ImportGraphDefOptions* opts, TF_Operation* oper); + +// Add an output in `graph_def` to be returned via the `return_outputs` output +// parameter of TF_GraphImportGraphDef(). If the output is remapped via an input +// mapping, the corresponding existing tensor in `graph` will be returned. +// `oper_name` is copied and has no lifetime requirements. +TF_CAPI_EXPORT extern void TF_ImportGraphDefOptionsAddReturnOutput( + TF_ImportGraphDefOptions* opts, const char* oper_name, int index); + +// Returns the number of return outputs added via +// TF_ImportGraphDefOptionsAddReturnOutput(). +TF_CAPI_EXPORT extern int TF_ImportGraphDefOptionsNumReturnOutputs( + const TF_ImportGraphDefOptions* opts); + +// Add an operation in `graph_def` to be returned via the `return_opers` output +// parameter of TF_GraphImportGraphDef(). `oper_name` is copied and has no +// lifetime requirements. +TF_CAPI_EXPORT extern void TF_ImportGraphDefOptionsAddReturnOperation( + TF_ImportGraphDefOptions* opts, const char* oper_name); + +// Returns the number of return operations added via +// TF_ImportGraphDefOptionsAddReturnOperation(). +TF_CAPI_EXPORT extern int TF_ImportGraphDefOptionsNumReturnOperations( + const TF_ImportGraphDefOptions* opts); + +// TF_ImportGraphDefResults holds results that are generated by +// TF_GraphImportGraphDefWithResults(). +typedef struct TF_ImportGraphDefResults TF_ImportGraphDefResults; + +// Fetches the return outputs requested via +// TF_ImportGraphDefOptionsAddReturnOutput(). The number of fetched outputs is +// returned in `num_outputs`. The array of return outputs is returned in +// `outputs`. `*outputs` is owned by and has the lifetime of `results`. +TF_CAPI_EXPORT extern void TF_ImportGraphDefResultsReturnOutputs( + TF_ImportGraphDefResults* results, int* num_outputs, TF_Output** outputs); + +// Fetches the return operations requested via +// TF_ImportGraphDefOptionsAddReturnOperation(). The number of fetched +// operations is returned in `num_opers`. The array of return operations is +// returned in `opers`. `*opers` is owned by and has the lifetime of `results`. +TF_CAPI_EXPORT extern void TF_ImportGraphDefResultsReturnOperations( + TF_ImportGraphDefResults* results, int* num_opers, TF_Operation*** opers); + +// Fetches any input mappings requested via +// TF_ImportGraphDefOptionsAddInputMapping() that didn't appear in the GraphDef +// and weren't used as input to any node in the imported graph def. The number +// of fetched mappings is returned in `num_missing_unused_input_mappings`. The +// array of each mapping's source node name is returned in `src_names`, and the +// array of each mapping's source index is returned in `src_indexes`. +// +// `*src_names`, `*src_indexes`, and the memory backing each string in +// `src_names` are owned by and have the lifetime of `results`. +TF_CAPI_EXPORT extern void TF_ImportGraphDefResultsMissingUnusedInputMappings( + TF_ImportGraphDefResults* results, int* num_missing_unused_input_mappings, + const char*** src_names, int** src_indexes); + +// Deletes a results object returned by TF_GraphImportGraphDefWithResults(). +TF_CAPI_EXPORT extern void TF_DeleteImportGraphDefResults( + TF_ImportGraphDefResults* results); + +// Import the graph serialized in `graph_def` into `graph`. Returns nullptr and +// a bad status on error. Otherwise, returns a populated +// TF_ImportGraphDefResults instance. The returned instance must be deleted via +// TF_DeleteImportGraphDefResults(). +TF_CAPI_EXPORT extern TF_ImportGraphDefResults* +TF_GraphImportGraphDefWithResults(TF_Graph* graph, const TF_Buffer* graph_def, + const TF_ImportGraphDefOptions* options, + TF_Status* status); + +// Import the graph serialized in `graph_def` into `graph`. +// Convenience function for when only return outputs are needed. +// +// `num_return_outputs` must be the number of return outputs added (i.e. the +// result of TF_ImportGraphDefOptionsNumReturnOutputs()). If +// `num_return_outputs` is non-zero, `return_outputs` must be of length +// `num_return_outputs`. Otherwise it can be null. +TF_CAPI_EXPORT extern void TF_GraphImportGraphDefWithReturnOutputs( + TF_Graph* graph, const TF_Buffer* graph_def, + const TF_ImportGraphDefOptions* options, TF_Output* return_outputs, + int num_return_outputs, TF_Status* status); + +// Import the graph serialized in `graph_def` into `graph`. +// Convenience function for when no results are needed. +TF_CAPI_EXPORT extern void TF_GraphImportGraphDef( + TF_Graph* graph, const TF_Buffer* graph_def, + const TF_ImportGraphDefOptions* options, TF_Status* status); + +// Adds a copy of function `func` and optionally its gradient function `grad` +// to `g`. Once `func`/`grad` is added to `g`, it can be called by creating +// an operation using the function's name. +// Any changes to `func`/`grad` (including deleting it) done after this method +// returns, won't affect the copy of `func`/`grad` in `g`. +// If `func` or `grad` are already in `g`, TF_GraphCopyFunction has no +// effect on them, but can establish the function->gradient relationship +// between them if `func` does not already have a gradient. If `func` already +// has a gradient different from `grad`, an error is returned. +// +// `func` must not be null. +// If `grad` is null and `func` is not in `g`, `func` is added without a +// gradient. +// If `grad` is null and `func` is in `g`, TF_GraphCopyFunction is a noop. +// `grad` must have appropriate signature as described in the doc of +// GradientDef in tensorflow/core/framework/function.proto. +// +// If successful, status is set to OK and `func` and `grad` are added to `g`. +// Otherwise, status is set to the encountered error and `g` is unmodified. +TF_CAPI_EXPORT extern void TF_GraphCopyFunction(TF_Graph* g, + const TF_Function* func, + const TF_Function* grad, + TF_Status* status); + +// Returns the number of TF_Functions registered in `g`. +TF_CAPI_EXPORT extern int TF_GraphNumFunctions(TF_Graph* g); + +// Fills in `funcs` with the TF_Function* registered in `g`. +// `funcs` must point to an array of TF_Function* of length at least +// `max_func`. In usual usage, max_func should be set to the result of +// TF_GraphNumFunctions(g). In this case, all the functions registered in +// `g` will be returned. Else, an unspecified subset. +// +// If successful, returns the number of TF_Function* successfully set in +// `funcs` and sets status to OK. The caller takes ownership of +// all the returned TF_Functions. They must be deleted with TF_DeleteFunction. +// On error, returns 0, sets status to the encountered error, and the contents +// of funcs will be undefined. +TF_CAPI_EXPORT extern int TF_GraphGetFunctions(TF_Graph* g, TF_Function** funcs, + int max_func, TF_Status* status); + +// Note: The following function may fail on very large protos in the future. + +TF_CAPI_EXPORT extern void TF_OperationToNodeDef(TF_Operation* oper, + TF_Buffer* output_node_def, + TF_Status* status); + +typedef struct TF_WhileParams { + // The number of inputs to the while loop, i.e. the number of loop variables. + // This is the size of cond_inputs, body_inputs, and body_outputs. + const int ninputs; + + // The while condition graph. The inputs are the current values of the loop + // variables. The output should be a scalar boolean. + TF_Graph* const cond_graph; + const TF_Output* const cond_inputs; + TF_Output cond_output; + + // The loop body graph. The inputs are the current values of the loop + // variables. The outputs are the updated values of the loop variables. + TF_Graph* const body_graph; + const TF_Output* const body_inputs; + TF_Output* const body_outputs; + + // Unique null-terminated name for this while loop. This is used as a prefix + // for created operations. + const char* name; +} TF_WhileParams; + +// Creates a TF_WhileParams for creating a while loop in `g`. `inputs` are +// outputs that already exist in `g` used as initial values for the loop +// variables. +// +// The returned TF_WhileParams will have all fields initialized except +// `cond_output`, `body_outputs`, and `name`. The `body_outputs` buffer will be +// allocated to size `ninputs`. The caller should build `cond_graph` and +// `body_graph` starting from the inputs, and store the final outputs in +// `cond_output` and `body_outputs`. +// +// If `status` is OK, the caller must call either TF_FinishWhile or +// TF_AbortWhile on the returned TF_WhileParams. If `status` isn't OK, the +// returned TF_WhileParams is not valid, and the caller should not call +// TF_FinishWhile() or TF_AbortWhile(). +// +// Missing functionality (TODO): +// - Gradients +// - Reference-type inputs +// - Directly referencing external tensors from the cond/body graphs (this is +// possible in the Python API) +TF_CAPI_EXPORT extern TF_WhileParams TF_NewWhile(TF_Graph* g, TF_Output* inputs, + int ninputs, + TF_Status* status); + +// Builds the while loop specified by `params` and returns the output tensors of +// the while loop in `outputs`. `outputs` should be allocated to size +// `params.ninputs`. +// +// `params` is no longer valid once this returns. +// +// Either this or TF_AbortWhile() must be called after a successful +// TF_NewWhile() call. +TF_CAPI_EXPORT extern void TF_FinishWhile(const TF_WhileParams* params, + TF_Status* status, + TF_Output* outputs); + +// Frees `params`s resources without building a while loop. `params` is no +// longer valid after this returns. Either this or TF_FinishWhile() must be +// called after a successful TF_NewWhile() call. +TF_CAPI_EXPORT extern void TF_AbortWhile(const TF_WhileParams* params); + +// Adds operations to compute the partial derivatives of sum of `y`s w.r.t `x`s, +// i.e., d(y_1 + y_2 + ...)/dx_1, d(y_1 + y_2 + ...)/dx_2... +// +// `dx` are used as initial gradients (which represent the symbolic partial +// derivatives of some loss function `L` w.r.t. `y`). +// `dx` must be nullptr or have size `ny`. +// If `dx` is nullptr, the implementation will use dx of `OnesLike` for all +// shapes in `y`. +// The partial derivatives are returned in `dy`. `dy` should be allocated to +// size `nx`. +// +// Gradient nodes are automatically named under the "gradients/" prefix. To +// guarantee name uniqueness, subsequent calls to the same graph will +// append an incremental tag to the prefix: "gradients_1/", "gradients_2/", ... +// See TF_AddGradientsWithPrefix, which provides a means to specify a custom +// name prefix for operations added to a graph to compute the gradients. +// +// WARNING: This function does not yet support all the gradients that python +// supports. See +// https://www.tensorflow.org/code/tensorflow/cc/gradients/README.md +// for instructions on how to add C++ more gradients. +TF_CAPI_EXPORT void TF_AddGradients(TF_Graph* g, TF_Output* y, int ny, + TF_Output* x, int nx, TF_Output* dx, + TF_Status* status, TF_Output* dy); + +// Adds operations to compute the partial derivatives of sum of `y`s w.r.t `x`s, +// i.e., d(y_1 + y_2 + ...)/dx_1, d(y_1 + y_2 + ...)/dx_2... +// This is a variant of TF_AddGradients that allows to caller to pass a custom +// name prefix to the operations added to a graph to compute the gradients. +// +// `dx` are used as initial gradients (which represent the symbolic partial +// derivatives of some loss function `L` w.r.t. `y`). +// `dx` must be nullptr or have size `ny`. +// If `dx` is nullptr, the implementation will use dx of `OnesLike` for all +// shapes in `y`. +// The partial derivatives are returned in `dy`. `dy` should be allocated to +// size `nx`. +// `prefix` names the scope into which all gradients operations are being added. +// `prefix` must be unique within the provided graph otherwise this operation +// will fail. If `prefix` is nullptr, the default prefixing behaviour takes +// place, see TF_AddGradients for more details. +// +// WARNING: This function does not yet support all the gradients that python +// supports. See +// https://www.tensorflow.org/code/tensorflow/cc/gradients/README.md +// for instructions on how to add C++ more gradients. +TF_CAPI_EXPORT void TF_AddGradientsWithPrefix(TF_Graph* g, const char* prefix, + TF_Output* y, int ny, + TF_Output* x, int nx, + TF_Output* dx, TF_Status* status, + TF_Output* dy); + +// Create a TF_Function from a TF_Graph +// +// Params: +// fn_body - the graph whose operations (or subset of whose operations) will be +// converted to TF_Function. +// fn_name - the name of the new TF_Function. Should match the operation +// name (OpDef.name) regexp [A-Z][A-Za-z0-9_.\\-/]*. +// If `append_hash_to_fn_name` is false, `fn_name` must be distinct +// from other function and operation names (at least those +// registered in graphs where this function will be used). +// append_hash_to_fn_name - Must be 0 or 1. If set to 1, the actual name +// of the function will be `fn_name` appended with +// '_'. +// If set to 0, the function's name will be `fn_name`. +// num_opers - `num_opers` contains the number of elements in the `opers` array +// or a special value of -1 meaning that no array is given. +// The distinction between an empty array of operations and no +// array of operations is necessary to distinguish the case of +// creating a function with no body (e.g. identity or permutation) +// and the case of creating a function whose body contains all +// the nodes in the graph (except for the automatic skipping, see +// below). +// opers - Array of operations to become the body of the function or null. +// - If no array is given (`num_opers` = -1), all the +// operations in `fn_body` will become part of the function +// except operations referenced in `inputs`. These operations +// must have a single output (these operations are typically +// placeholders created for the sole purpose of representing +// an input. We can relax this constraint if there are +// compelling use cases). +// - If an array is given (`num_opers` >= 0), all operations +// in it will become part of the function. In particular, no +// automatic skipping of dummy input operations is performed. +// ninputs - number of elements in `inputs` array +// inputs - array of TF_Outputs that specify the inputs to the function. +// If `ninputs` is zero (the function takes no inputs), `inputs` +// can be null. The names used for function inputs are normalized +// names of the operations (usually placeholders) pointed to by +// `inputs`. These operation names should start with a letter. +// Normalization will convert all letters to lowercase and +// non-alphanumeric characters to '_' to make resulting names match +// the "[a-z][a-z0-9_]*" pattern for operation argument names. +// `inputs` cannot contain the same tensor twice. +// noutputs - number of elements in `outputs` array +// outputs - array of TF_Outputs that specify the outputs of the function. +// If `noutputs` is zero (the function returns no outputs), `outputs` +// can be null. `outputs` can contain the same tensor more than once. +// output_names - The names of the function's outputs. `output_names` array +// must either have the same length as `outputs` +// (i.e. `noutputs`) or be null. In the former case, +// the names should match the regular expression for ArgDef +// names - "[a-z][a-z0-9_]*". In the latter case, +// names for outputs will be generated automatically. +// opts - various options for the function, e.g. XLA's inlining control. +// description - optional human-readable description of this function. +// status - Set to OK on success and an appropriate error on failure. +// +// Note that when the same TF_Output is listed as both an input and an output, +// the corresponding function's output will equal to this input, +// instead of the original node's output. +// +// Callers must also satisfy the following constraints: +// - `inputs` cannot refer to TF_Outputs within a control flow context. For +// example, one cannot use the output of "switch" node as input. +// - `inputs` and `outputs` cannot have reference types. Reference types are +// not exposed through C API and are being replaced with Resources. We support +// reference types inside function's body to support legacy code. Do not +// use them in new code. +// - Every node in the function's body must have all of its inputs (including +// control inputs). In other words, for every node in the body, each input +// must be either listed in `inputs` or must come from another node in +// the body. In particular, it is an error to have a control edge going from +// a node outside of the body into a node in the body. This applies to control +// edges going from nodes referenced in `inputs` to nodes in the body when +// the former nodes are not in the body (automatically skipped or not +// included in explicitly specified body). +// +// Returns: +// On success, a newly created TF_Function instance. It must be deleted by +// calling TF_DeleteFunction. +// +// On failure, null. +TF_CAPI_EXPORT extern TF_Function* TF_GraphToFunction( + const TF_Graph* fn_body, const char* fn_name, + unsigned char append_hash_to_fn_name, int num_opers, + const TF_Operation* const* opers, int ninputs, const TF_Output* inputs, + int noutputs, const TF_Output* outputs, const char* const* output_names, + const TF_FunctionOptions* opts, const char* description, TF_Status* status); + +// Returns the name of the graph function. +// The return value points to memory that is only usable until the next +// mutation to *func. +TF_CAPI_EXPORT extern const char* TF_FunctionName(TF_Function* func); + +// Write out a serialized representation of `func` (as a FunctionDef protocol +// message) to `output_func_def` (allocated by TF_NewBuffer()). +// `output_func_def`'s underlying buffer will be freed when TF_DeleteBuffer() +// is called. +// +// May fail on very large graphs in the future. +TF_CAPI_EXPORT extern void TF_FunctionToFunctionDef(TF_Function* func, + TF_Buffer* output_func_def, + TF_Status* status); + +// Construct and return the function whose FunctionDef representation is +// serialized in `proto`. `proto_len` must equal the number of bytes +// pointed to by `proto`. +// Returns: +// On success, a newly created TF_Function instance. It must be deleted by +// calling TF_DeleteFunction. +// +// On failure, null. +TF_CAPI_EXPORT extern TF_Function* TF_FunctionImportFunctionDef( + const void* proto, size_t proto_len, TF_Status* status); + +// Sets function attribute named `attr_name` to value stored in `proto`. +// If this attribute is already set to another value, it is overridden. +// `proto` should point to a sequence of bytes of length `proto_len` +// representing a binary serialization of an AttrValue protocol +// buffer. +TF_CAPI_EXPORT extern void TF_FunctionSetAttrValueProto(TF_Function* func, + const char* attr_name, + const void* proto, + size_t proto_len, + TF_Status* status); + +// Sets `output_attr_value` to the binary-serialized AttrValue proto +// representation of the value of the `attr_name` attr of `func`. +// If `attr_name` attribute is not present, status is set to an error. +TF_CAPI_EXPORT extern void TF_FunctionGetAttrValueProto( + TF_Function* func, const char* attr_name, TF_Buffer* output_attr_value, + TF_Status* status); + +// Frees the memory used by the `func` struct. +// TF_DeleteFunction is a noop if `func` is null. +// Deleting a function does not remove it from any graphs it was copied to. +TF_CAPI_EXPORT extern void TF_DeleteFunction(TF_Function* func); + +// Attempts to evaluate `output`. This will only be possible if `output` doesn't +// depend on any graph inputs (this function is safe to call if this isn't the +// case though). +// +// If the evaluation is successful, this function returns true and `output`s +// value is returned in `result`. Otherwise returns false. An error status is +// returned if something is wrong with the graph or input. Note that this may +// return false even if no error status is set. +TF_CAPI_EXPORT extern unsigned char TF_TryEvaluateConstant(TF_Graph* graph, + TF_Output output, + TF_Tensor** result, + TF_Status* status); + +// TODO(josh11b): Register OpDef, available to all operations added +// to this graph. + +// -------------------------------------------------------------------------- +// API for driving Graph execution. + +typedef struct TF_Session TF_Session; + +// Return a new execution session with the associated graph, or NULL on +// error. Does not take ownership of any input parameters. +// +// *`graph` must be a valid graph (not deleted or nullptr). `graph` will be be +// kept alive for the lifetime of the returned TF_Session. New nodes can still +// be added to `graph` after this call. +TF_CAPI_EXPORT extern TF_Session* TF_NewSession(TF_Graph* graph, + const TF_SessionOptions* opts, + TF_Status* status); + +// This function creates a new TF_Session (which is created on success) using +// `session_options`, and then initializes state (restoring tensors and other +// assets) using `run_options`. +// +// Any NULL and non-NULL value combinations for (`run_options, `meta_graph_def`) +// are valid. +// +// - `export_dir` must be set to the path of the exported SavedModel. +// - `tags` must include the set of tags used to identify one MetaGraphDef in +// the SavedModel. +// - `graph` must be a graph newly allocated with TF_NewGraph(). +// +// If successful, populates `graph` with the contents of the Graph and +// `meta_graph_def` with the MetaGraphDef of the loaded model. +TF_CAPI_EXPORT extern TF_Session* TF_LoadSessionFromSavedModel( + const TF_SessionOptions* session_options, const TF_Buffer* run_options, + const char* export_dir, const char* const* tags, int tags_len, + TF_Graph* graph, TF_Buffer* meta_graph_def, TF_Status* status); + +// Close a session. +// +// Contacts any other processes associated with the session, if applicable. +// May not be called after TF_DeleteSession(). +TF_CAPI_EXPORT extern void TF_CloseSession(TF_Session*, TF_Status* status); + +// Destroy a session object. +// +// Even if error information is recorded in *status, this call discards all +// local resources associated with the session. The session may not be used +// during or after this call (and the session drops its reference to the +// corresponding graph). +TF_CAPI_EXPORT extern void TF_DeleteSession(TF_Session*, TF_Status* status); + +// Run the graph associated with the session starting with the supplied inputs +// (inputs[0,ninputs-1] with corresponding values in input_values[0,ninputs-1]). +// +// Any NULL and non-NULL value combinations for (`run_options`, +// `run_metadata`) are valid. +// +// - `run_options` may be NULL, in which case it will be ignored; or +// non-NULL, in which case it must point to a `TF_Buffer` containing the +// serialized representation of a `RunOptions` protocol buffer. +// - `run_metadata` may be NULL, in which case it will be ignored; or +// non-NULL, in which case it must point to an empty, freshly allocated +// `TF_Buffer` that may be updated to contain the serialized representation +// of a `RunMetadata` protocol buffer. +// +// The caller retains ownership of `input_values` (which can be deleted using +// TF_DeleteTensor). The caller also retains ownership of `run_options` and/or +// `run_metadata` (when not NULL) and should manually call TF_DeleteBuffer on +// them. +// +// On success, the tensors corresponding to outputs[0,noutputs-1] are placed in +// output_values[]. Ownership of the elements of output_values[] is transferred +// to the caller, which must eventually call TF_DeleteTensor on them. +// +// On failure, output_values[] contains NULLs. +TF_CAPI_EXPORT extern void TF_SessionRun( + TF_Session* session, + // RunOptions + const TF_Buffer* run_options, + // Input tensors + const TF_Output* inputs, TF_Tensor* const* input_values, int ninputs, + // Output tensors + const TF_Output* outputs, TF_Tensor** output_values, int noutputs, + // Target operations + const TF_Operation* const* target_opers, int ntargets, + // RunMetadata + TF_Buffer* run_metadata, + // Output status + TF_Status*); + +// Set up the graph with the intended feeds (inputs) and fetches (outputs) for a +// sequence of partial run calls. +// +// On success, returns a handle that is used for subsequent PRun calls. The +// handle should be deleted with TF_DeletePRunHandle when it is no longer +// needed. +// +// On failure, out_status contains a tensorflow::Status with an error +// message. *handle is set to nullptr. +TF_CAPI_EXPORT extern void TF_SessionPRunSetup( + TF_Session*, + // Input names + const TF_Output* inputs, int ninputs, + // Output names + const TF_Output* outputs, int noutputs, + // Target operations + const TF_Operation* const* target_opers, int ntargets, + // Output handle + const char** handle, + // Output status + TF_Status*); + +// Continue to run the graph with additional feeds and fetches. The +// execution state is uniquely identified by the handle. +TF_CAPI_EXPORT extern void TF_SessionPRun( + TF_Session*, const char* handle, + // Input tensors + const TF_Output* inputs, TF_Tensor* const* input_values, int ninputs, + // Output tensors + const TF_Output* outputs, TF_Tensor** output_values, int noutputs, + // Target operations + const TF_Operation* const* target_opers, int ntargets, + // Output status + TF_Status*); + +// Deletes a handle allocated by TF_SessionPRunSetup. +// Once called, no more calls to TF_SessionPRun should be made. +TF_CAPI_EXPORT extern void TF_DeletePRunHandle(const char* handle); + +// -------------------------------------------------------------------------- +// The deprecated session API. Please switch to the above instead of +// TF_ExtendGraph(). This deprecated API can be removed at any time without +// notice. + +typedef struct TF_DeprecatedSession TF_DeprecatedSession; + +TF_CAPI_EXPORT extern TF_DeprecatedSession* TF_NewDeprecatedSession( + const TF_SessionOptions*, TF_Status* status); +TF_CAPI_EXPORT extern void TF_CloseDeprecatedSession(TF_DeprecatedSession*, + TF_Status* status); +TF_CAPI_EXPORT extern void TF_DeleteDeprecatedSession(TF_DeprecatedSession*, + TF_Status* status); +TF_CAPI_EXPORT extern void TF_Reset(const TF_SessionOptions* opt, + const char** containers, int ncontainers, + TF_Status* status); +// Treat the bytes proto[0,proto_len-1] as a serialized GraphDef and +// add the nodes in that GraphDef to the graph for the session. +// +// Prefer use of TF_Session and TF_GraphImportGraphDef over this. +TF_CAPI_EXPORT extern void TF_ExtendGraph(TF_DeprecatedSession*, + const void* proto, size_t proto_len, + TF_Status*); + +// See TF_SessionRun() above. +TF_CAPI_EXPORT extern void TF_Run(TF_DeprecatedSession*, + const TF_Buffer* run_options, + const char** input_names, TF_Tensor** inputs, + int ninputs, const char** output_names, + TF_Tensor** outputs, int noutputs, + const char** target_oper_names, int ntargets, + TF_Buffer* run_metadata, TF_Status*); + +// See TF_SessionPRunSetup() above. +TF_CAPI_EXPORT extern void TF_PRunSetup(TF_DeprecatedSession*, + const char** input_names, int ninputs, + const char** output_names, int noutputs, + const char** target_oper_names, + int ntargets, const char** handle, + TF_Status*); + +// See TF_SessionPRun above. +TF_CAPI_EXPORT extern void TF_PRun(TF_DeprecatedSession*, const char* handle, + const char** input_names, TF_Tensor** inputs, + int ninputs, const char** output_names, + TF_Tensor** outputs, int noutputs, + const char** target_oper_names, int ntargets, + TF_Status*); + +typedef struct TF_DeviceList TF_DeviceList; + +// Lists all devices in a TF_Session. +// +// Caller takes ownership of the returned TF_DeviceList* which must eventually +// be freed with a call to TF_DeleteDeviceList. +TF_CAPI_EXPORT extern TF_DeviceList* TF_SessionListDevices(TF_Session* session, + TF_Status* status); + +// Lists all devices in a TF_Session. +// +// Caller takes ownership of the returned TF_DeviceList* which must eventually +// be freed with a call to TF_DeleteDeviceList. +TF_CAPI_EXPORT extern TF_DeviceList* TF_DeprecatedSessionListDevices( + TF_DeprecatedSession* session, TF_Status* status); + +// Deallocates the device list. +TF_CAPI_EXPORT extern void TF_DeleteDeviceList(TF_DeviceList* list); + +// Counts the number of elements in the device list. +TF_CAPI_EXPORT extern int TF_DeviceListCount(const TF_DeviceList* list); + +// Retrieves the full name of the device (e.g. /job:worker/replica:0/...) +// The return value will be a pointer to a null terminated string. The caller +// must not modify or delete the string. It will be deallocated upon a call to +// TF_DeleteDeviceList. +// +// If index is out of bounds, an error code will be set in the status object, +// and a null pointer will be returned. +TF_CAPI_EXPORT extern const char* TF_DeviceListName(const TF_DeviceList* list, + int index, + TF_Status* status); + +// Retrieves the type of the device at the given index. +// +// The caller must not modify or delete the string. It will be deallocated upon +// a call to TF_DeleteDeviceList. +// +// If index is out of bounds, an error code will be set in the status object, +// and a null pointer will be returned. +TF_CAPI_EXPORT extern const char* TF_DeviceListType(const TF_DeviceList* list, + int index, + TF_Status* status); + +// Retrieve the amount of memory associated with a given device. +// +// If index is out of bounds, an error code will be set in the status object, +// and -1 will be returned. +TF_CAPI_EXPORT extern int64_t TF_DeviceListMemoryBytes( + const TF_DeviceList* list, int index, TF_Status* status); + +// Retrieve the incarnation number of a given device. +// +// If index is out of bounds, an error code will be set in the status object, +// and 0 will be returned. +TF_CAPI_EXPORT extern uint64_t TF_DeviceListIncarnation( + const TF_DeviceList* list, int index, TF_Status* status); + +// -------------------------------------------------------------------------- +// Load plugins containing custom ops and kernels + +// TF_Library holds information about dynamically loaded TensorFlow plugins. +typedef struct TF_Library TF_Library; + +// Load the library specified by library_filename and register the ops and +// kernels present in that library. +// +// Pass "library_filename" to a platform-specific mechanism for dynamically +// loading a library. The rules for determining the exact location of the +// library are platform-specific and are not documented here. +// +// On success, place OK in status and return the newly created library handle. +// The caller owns the library handle. +// +// On failure, place an error status in status and return NULL. +TF_CAPI_EXPORT extern TF_Library* TF_LoadLibrary(const char* library_filename, + TF_Status* status); + +// Get the OpList of OpDefs defined in the library pointed by lib_handle. +// +// Returns a TF_Buffer. The memory pointed to by the result is owned by +// lib_handle. The data in the buffer will be the serialized OpList proto for +// ops defined in the library. +TF_CAPI_EXPORT extern TF_Buffer TF_GetOpList(TF_Library* lib_handle); + +// Frees the memory associated with the library handle. +// Does NOT unload the library. +TF_CAPI_EXPORT extern void TF_DeleteLibraryHandle(TF_Library* lib_handle); + +// Get the OpList of all OpDefs defined in this address space. +// Returns a TF_Buffer, ownership of which is transferred to the caller +// (and can be freed using TF_DeleteBuffer). +// +// The data in the buffer will be the serialized OpList proto for ops registered +// in this address space. +TF_CAPI_EXPORT extern TF_Buffer* TF_GetAllOpList(); + +// TF_ApiDefMap encapsulates a collection of API definitions for an operation. +// +// This object maps the name of a TensorFlow operation to a description of the +// API to generate for it, as defined by the ApiDef protocol buffer ( +// https://www.tensorflow.org/code/tensorflow/core/framework/api_def.proto) +// +// The ApiDef messages are typically used to generate convenience wrapper +// functions for TensorFlow operations in various language bindings. +typedef struct TF_ApiDefMap TF_ApiDefMap; + +// Creates a new TF_ApiDefMap instance. +// +// Params: +// op_list_buffer - TF_Buffer instance containing serialized OpList +// protocol buffer. (See +// https://www.tensorflow.org/code/tensorflow/core/framework/op_def.proto +// for the OpList proto definition). +// status - Set to OK on success and an appropriate error on failure. +TF_CAPI_EXPORT extern TF_ApiDefMap* TF_NewApiDefMap(TF_Buffer* op_list_buffer, + TF_Status* status); + +// Deallocates a TF_ApiDefMap. +TF_CAPI_EXPORT extern void TF_DeleteApiDefMap(TF_ApiDefMap* apimap); + +// Add ApiDefs to the map. +// +// `text` corresponds to a text representation of an ApiDefs protocol message. +// (https://www.tensorflow.org/code/tensorflow/core/framework/api_def.proto). +// +// The provided ApiDefs will be merged with existing ones in the map, with +// precedence given to the newly added version in case of conflicts with +// previous calls to TF_ApiDefMapPut. +TF_CAPI_EXPORT extern void TF_ApiDefMapPut(TF_ApiDefMap* api_def_map, + const char* text, size_t text_len, + TF_Status* status); + +// Returns a serialized ApiDef protocol buffer for the TensorFlow operation +// named `name`. +TF_CAPI_EXPORT extern TF_Buffer* TF_ApiDefMapGet(TF_ApiDefMap* api_def_map, + const char* name, + size_t name_len, + TF_Status* status); + +// -------------------------------------------------------------------------- +// Kernel definition information. + +// Returns a serialized KernelList protocol buffer containing KernelDefs for all +// registered kernels. +TF_CAPI_EXPORT extern TF_Buffer* TF_GetAllRegisteredKernels(TF_Status* status); + +// Returns a serialized KernelList protocol buffer containing KernelDefs for all +// kernels registered for the operation named `name`. +TF_CAPI_EXPORT extern TF_Buffer* TF_GetRegisteredKernelsForOp( + const char* name, TF_Status* status); + +#ifdef __cplusplus +} /* end extern "C" */ +#endif + +#endif // TENSORFLOW_C_C_API_H_ diff --git a/linux/include/tensorflow/c/c_api_experimental.h b/linux/include/tensorflow/c/c_api_experimental.h new file mode 100644 index 000000000..d98d532e3 --- /dev/null +++ b/linux/include/tensorflow/c/c_api_experimental.h @@ -0,0 +1,190 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_C_C_API_EXPERIMENTAL_H_ +#define TENSORFLOW_C_C_API_EXPERIMENTAL_H_ + +#include +#include + +#include "tensorflow/c/c_api.h" +#include "tensorflow/c/eager/c_api.h" + +// -------------------------------------------------------------------------- +// Experimental C API for TensorFlow. +// +// The API here is subject to changes in the future. +// -------------------------------------------------------------------------- + +// Macro to control visibility of exported symbols in the shared library (.so, +// .dylib, .dll). +// This duplicates the TF_EXPORT macro definition in +// tensorflow/core/platform/macros.h in order to keep this .h file independent +// of any other includes.$a +#ifdef SWIG +#define TF_CAPI_EXPORT +#else +#if defined(_WIN32) +#ifdef TF_COMPILE_LIBRARY +#define TF_CAPI_EXPORT __declspec(dllexport) +#else +#define TF_CAPI_EXPORT __declspec(dllimport) +#endif // TF_COMPILE_LIBRARY +#else +#define TF_CAPI_EXPORT __attribute__((visibility("default"))) +#endif // _WIN32 +#endif // SWIG + +#ifdef __cplusplus +extern "C" { +#endif + +// When `enable` is true, set +// tensorflow.ConfigProto.OptimizerOptions.global_jit_level to ON_1, and also +// set XLA flag values to prepare for XLA compilation. Otherwise set +// global_jit_level to OFF. +// +// This and the next API are syntax sugar over TF_SetConfig(), and is used by +// clients that cannot read/write the tensorflow.ConfigProto proto. +// TODO: Migrate to TF_CreateConfig() below. +TF_CAPI_EXPORT extern void TF_EnableXLACompilation(TF_SessionOptions* options, + unsigned char enable); + +// Create a serialized tensorflow.ConfigProto proto, where: +// +// a) ConfigProto.optimizer_options.global_jit_level is set to to ON_1 if +// `enable_xla_compilation` is non-zero, and OFF otherwise. +// b) ConfigProto.gpu_options.allow_growth is set to `gpu_memory_allow_growth`. +TF_CAPI_EXPORT extern TF_Buffer* TF_CreateConfig( + unsigned char enable_xla_compilation, + unsigned char gpu_memory_allow_growth); + +// Create a serialized tensorflow.RunOptions proto, where RunOptions.trace_level +// is set to FULL_TRACE if `enable_full_trace` is non-zero, and NO_TRACE +// otherwise. +TF_CAPI_EXPORT extern TF_Buffer* TF_CreateRunOptions( + unsigned char enable_full_trace); + +// Returns the graph content in a human-readable format, with length set in +// `len`. The format is subject to change in the future. +// The returned string is heap-allocated, and caller should call free() on it. +TF_CAPI_EXPORT extern const char* TF_GraphDebugString(TF_Graph* graph, + size_t* len); + +// Creates a stack of data set + iterator nodes, currently hard-coded to return +// a sequence of 3 float values <42.0, 43.0, 44.0> over 3 calls. On success, +// returns the IteratorGetNext node, which caller can run or feed into an node. +// +// TODO(hongm): Extend the API to allow customization of the nodes created. +TF_CAPI_EXPORT extern TF_Operation* TF_MakeFakeIteratorGetNextWithDatasets( + TF_Graph* graph, TF_Status* status); + +// Similar to the above API, except that the returned iterator reads the +// file based dataset from `file_path`. +// If `is_mnist` is 0, the dataset corresponds to ImageNet. +// The iterators outputs 2 tensors: +// - A float tensor of shape `batch_size` X 784 when `is_mnist` is non-zero, or +// `batch_size` X 224 X 224 X 3 otherwise. +// - An int32 tensor of shape `batch_size` +// TODO(hongm): Extend the API to allow customization of the nodes created. +TF_CAPI_EXPORT extern TF_Operation* TF_MakeFileBasedIteratorGetNextWithDatasets( + TF_Graph* graph, const char* file_path, int batch_size, + unsigned char is_mnist, TF_Status* status); + +// On success, dequeues a tensor from a TF-managed FifoQueue given by +// `tensor_id`, associated with `session`. There must be a graph node named +// "fifo_queue_dequeue_", to be executed by this API call. + +// Caller must call TF_DeleteTensor() over the returned tensor. If the queue is +// empty, this call is blocked. +// +// Tensors are enqueued via the corresponding TF enqueue op. +// TODO(hongm): Add support for `timeout_ms`. +TF_CAPI_EXPORT extern TF_Tensor* TF_DequeueNamedTensor(TF_Session* session, + int tensor_id, + TF_Status* status); + +// On success, enqueues `tensor` into a TF-managed FifoQueue given by +// `tensor_id`, associated with `session`. There must be a graph node named +// "fifo_queue_enqueue_", to be executed by this API call. It reads +// from a placeholder node "arg_tensor_enqueue_". +// +// `tensor` is still owned by the caller. This call will be blocked if the queue +// has reached its capacity, and will be unblocked when the queued tensors again +// drop below the capacity due to dequeuing. +// +// Tensors are dequeued via the corresponding TF dequeue op. +// TODO(hongm): Add support for `timeout_ms`. +TF_CAPI_EXPORT extern void TF_EnqueueNamedTensor(TF_Session* session, + int tensor_id, + TF_Tensor* tensor, + TF_Status* status); +// Create a serialized tensorflow.ServerDef proto. +TF_Buffer* TFE_GetServerDef(const char* text_proto, TF_Status* status); + +// TODO: remove this API in favor of the next one. +TF_CAPI_EXPORT extern TFE_Context* TFE_NewContextFromSession( + const TFE_ContextOptions* opts, TF_Session* sess, TF_Status* status); + +// Creates from `session` a new eager context to run a graph function or +// sends/recvs, so that these concurrent TFE executions can share (via +// `session` and its associated device mgr) the same set of fifo queue resource +// ops, used for host<->TF tensor transfers. This way the sends/recvs calls and +// graph function execution can access the same fifo queue resource handles +// (associated with devices managed by the device manager, which can be obtained +// from `session`). +// +// TODO: Remove this function once we migrate away from using session. +TF_CAPI_EXPORT extern TFE_Context* TFE_CreateContextFromSession( + TF_Session* session, TF_Status* status); + +// TODO: Retire this API in favor of the next one. +TF_CAPI_EXPORT extern TFE_TensorHandle* TFE_DequeueNamedTensor( + TF_Session* session, int tensor_id, TF_DataType inputType, + TF_Status* status); + +TF_CAPI_EXPORT extern TFE_TensorHandle* TFE_DequeueNamedTensorFromCtx( + TFE_Context* ctx, int tensor_id, TF_DataType inputType, TF_Status* status); + +TF_CAPI_EXPORT extern void TFE_EnqueueNamedTensor(TF_Session* session, + int tensor_id, + TFE_TensorHandle* tensor, + TF_Status* status); + +TF_CAPI_EXPORT extern void TFE_EnqueueNamedTensorFromCtx( + TFE_Context* ctx, int tensor_id, TFE_TensorHandle* tensor, + TF_Status* status); + +// TODO: consider folding the 2 APIs below into the ones above. +TF_CAPI_EXPORT extern void TFE_EnqueueVariantTensor(TF_Session* session, + int tensor_id, + TFE_TensorHandle* tensor, + TF_Status* status); + +TF_CAPI_EXPORT extern TFE_TensorHandle* TFE_DequeueVariantTensor( + TF_Session* session, int tensor_id, TF_Status* status); + +// Prints `handle` in a human readable format to standard output for debugging. +TF_CAPI_EXPORT extern void TFE_TensorHandlePrintDebugString( + TFE_TensorHandle* handle); + +TF_CAPI_EXPORT extern void TF_MakeInternalErrorStatus(TF_Status* status, + const char* errMsg); + +#ifdef __cplusplus +} /* end extern "C" */ +#endif + +#endif // TENSORFLOW_C_C_API_EXPERIMENTAL_H_ diff --git a/linux/include/tensorflow/c/c_api_internal.h b/linux/include/tensorflow/c/c_api_internal.h new file mode 100644 index 000000000..95652a113 --- /dev/null +++ b/linux/include/tensorflow/c/c_api_internal.h @@ -0,0 +1,223 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_C_C_API_INTERNAL_H_ +#define TENSORFLOW_C_C_API_INTERNAL_H_ + +#include "tensorflow/c/c_api.h" + +#include +#include +#include +#include +#include + +#ifndef __ANDROID__ +#include "tensorflow/core/framework/op_gen_lib.h" +#endif +#include "tensorflow/core/common_runtime/shape_refiner.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/graph/graph.h" +#include "tensorflow/core/graph/graph_constructor.h" +#include "tensorflow/core/graph/node_builder.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/platform/mutex.h" +#include "tensorflow/core/platform/types.h" +#include "tensorflow/core/public/session.h" + +namespace tensorflow { +class Device; +class DeviceMgr; +} // namespace tensorflow + +// Internal structures used by the C API. These are likely to change and should +// not be depended on. + +struct TF_Status { + tensorflow::Status status; +}; + +struct TF_Tensor { + ~TF_Tensor(); + + TF_DataType dtype; + tensorflow::TensorShape shape; + tensorflow::TensorBuffer* buffer; +}; + +struct TF_SessionOptions { + tensorflow::SessionOptions options; +}; + +struct TF_DeprecatedSession { + tensorflow::Session* session; +}; + +struct TF_Library { + void* lib_handle; + TF_Buffer op_list; +}; + +struct TF_Graph { + TF_Graph(); + + tensorflow::mutex mu; + tensorflow::Graph graph GUARDED_BY(mu); + + // Runs shape inference. + tensorflow::ShapeRefiner refiner GUARDED_BY(mu); + + // Maps from name of an operation to the Node* in 'graph'. + std::unordered_map name_map + GUARDED_BY(mu); + + // The keys of this map are all the active sessions using this graph. Each + // value records whether the graph has been mutated since the corresponding + // session has been run (this is detected in RecordMutation function). If the + // string is empty, no mutation has occurred. Otherwise the string is a + // description of the mutation suitable for returning to the user. + // + // Sessions are added to this map in TF_NewSession, and removed in + // TF_DeleteSession. + // TF_Graph may only / must be deleted when + // sessions.size() == 0 && delete_requested == true + // + // TODO(b/74949947): mutations currently trigger a warning instead of a bad + // status, this should be reverted when possible. + tensorflow::gtl::FlatMap sessions + GUARDED_BY(mu); + bool delete_requested GUARDED_BY(mu); // set true by TF_DeleteGraph + + // Used to link graphs contained in TF_WhileParams to the parent graph that + // will eventually contain the full while loop. + TF_Graph* parent; + TF_Output* parent_inputs; +}; + +struct TF_OperationDescription { + TF_OperationDescription(TF_Graph* g, const char* op_type, + const char* node_name) + : node_builder(node_name, op_type, g->graph.op_registry()), graph(g) {} + + tensorflow::NodeBuilder node_builder; + TF_Graph* graph; + std::set colocation_constraints; +}; + +struct TF_Operation { + tensorflow::Node node; +}; + +struct TF_Session { + TF_Session(tensorflow::Session* s, TF_Graph* g); + + tensorflow::Session* session; + TF_Graph* const graph; + + tensorflow::mutex mu ACQUIRED_AFTER(TF_Graph::mu); + int last_num_graph_nodes; + + // If true, TF_SessionRun and similar methods will call + // ExtendSessionGraphHelper before running the graph (this is the default + // public behavior). Can be set to false if the caller needs to call + // ExtendSessionGraphHelper manually. + std::atomic extend_before_run; +}; + +struct TF_ImportGraphDefOptions { + tensorflow::ImportGraphDefOptions opts; + + // Backing memory for TensorId fields in opts. + // TODO(skyewm): it'd be better if ImportGraphDefOptions owned this. + std::list tensor_id_data; +}; + +struct TF_ImportGraphDefResults { + std::vector return_tensors; + std::vector return_nodes; + std::vector missing_unused_key_names; + std::vector missing_unused_key_indexes; + + // Backing memory for missing_unused_key_names values. + std::list missing_unused_key_names_data; +}; + +struct TF_DeviceList { + std::vector response; +}; + +struct TF_Function { + tensorflow::FunctionDef fdef; +}; + +struct TF_ApiDefMap { + explicit TF_ApiDefMap(const tensorflow::OpList& op_list) + : +#ifndef __ANDROID__ + api_def_map(op_list), +#endif + update_docs_called(false) { + } + +#ifndef __ANDROID__ + tensorflow::ApiDefMap api_def_map GUARDED_BY(lock); +#endif + bool update_docs_called GUARDED_BY(lock); + tensorflow::mutex lock; +}; + +namespace tensorflow { + +class TensorCApi { + public: + static TensorBuffer* Buffer(const Tensor& tensor) { return tensor.buf_; } + static Tensor MakeTensor(TF_DataType type, const TensorShape& shape, + TensorBuffer* buf) { + return Tensor(static_cast(type), shape, buf); + } +}; + +Status TF_TensorToTensor(const TF_Tensor* src, Tensor* dst); + +TF_Tensor* TF_TensorFromTensor(const Tensor& src, TF_Status* status); + +Status MessageToBuffer(const tensorflow::protobuf::Message& in, TF_Buffer* out); + +// Set the shapes and types of the output's handle. +// +// The lengths of the arrays pointed to by `shapes`, `ranks`, and `types` must +// all be equal to `num_shapes_and_types`. If `ranks[i] != -1`, (i.e., if the +// rank is known), then it must be equal to the length of `shapes[i]`; if +// `ranks[i] == 1`, then `shapes[i]` may be nullptr. +// +// TODO(akshayka): Implement a corresponding getter method. +void TF_GraphSetOutputHandleShapesAndTypes(TF_Graph* graph, TF_Output output, + int num_shapes_and_types, + const int64_t** shapes, + const int* ranks, + const TF_DataType* types, + TF_Status* status); + +void RecordMutation(TF_Graph* graph, const TF_Operation& op, + const char* mutation_type) + EXCLUSIVE_LOCKS_REQUIRED(graph->mu); + +bool ExtendSessionGraphHelper(TF_Session* session, TF_Status* status) + LOCKS_EXCLUDED(session->graph->mu, session->mu); + +} // end namespace tensorflow + +#endif // TENSORFLOW_C_C_API_INTERNAL_H_ diff --git a/linux/include/tensorflow/c/checkpoint_reader.h b/linux/include/tensorflow/c/checkpoint_reader.h new file mode 100644 index 000000000..91654c8d4 --- /dev/null +++ b/linux/include/tensorflow/c/checkpoint_reader.h @@ -0,0 +1,82 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_C_CHECKPOINT_READER_H_ +#define TENSORFLOW_C_CHECKPOINT_READER_H_ + +#include +#include + +#include "tensorflow/c/tf_status_helper.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/platform/types.h" +#include "tensorflow/core/util/tensor_bundle/tensor_bundle.h" +#include "tensorflow/core/util/tensor_slice_reader.h" + +namespace tensorflow { +namespace checkpoint { + +class TensorSliceReader; + +// A wrapper around BundleReader (for V2 checkpoints) and +// checkpoint::TensorSliceReader (for V1), that is more easily SWIG wrapped for +// other languages. +// +// The class currently only interacts with single-slice (i.e., non-partitioned) +// variables. +class CheckpointReader { + public: + CheckpointReader(const string& filepattern, TF_Status* out_status); + + bool HasTensor(const string& name) const; + const string DebugString() const; + + // Returns a map from variable names to their shapes. Slices of a partitioned + // tensor are combined into a single entry. + const TensorSliceReader::VarToShapeMap& GetVariableToShapeMap() const; + + // Returns a map from variable names to their data types. Slices of a + // partitioned tensor are combined into a single entry. + const TensorSliceReader::VarToDataTypeMap& GetVariableToDataTypeMap() const; + + // Attempts to look up the tensor named "name" and stores the found result in + // "out_tensor". + void GetTensor(const string& name, + std::unique_ptr* out_tensor, + TF_Status* out_status) const; + + private: + // Uses "v2_reader_" to build "var name -> shape" and "var name -> data type" + // maps; both owned by caller. + // REQUIRES: "v2_reader_ != nullptr && v2_reader_.status().ok()". + std::pair, + std::unique_ptr > + BuildV2VarMaps(); + + // Invariant: exactly one of "reader_" and "v2_reader_" is non-null. + std::unique_ptr reader_; + std::unique_ptr v2_reader_; + + std::unique_ptr var_to_shape_map_; + std::unique_ptr var_to_data_type_map_; + + TF_DISALLOW_COPY_AND_ASSIGN(CheckpointReader); +}; + +} // namespace checkpoint +} // namespace tensorflow + +#endif // TENSORFLOW_C_CHECKPOINT_READER_H_ diff --git a/linux/include/tensorflow/c/eager/c_api.h b/linux/include/tensorflow/c/eager/c_api.h new file mode 100644 index 000000000..b2454d872 --- /dev/null +++ b/linux/include/tensorflow/c/eager/c_api.h @@ -0,0 +1,429 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_C_EAGER_C_API_H_ +#define TENSORFLOW_C_EAGER_C_API_H_ + +// C API extensions to experiment with eager execution of kernels. +// WARNING: Unlike tensorflow/c/c_api.h, the API here is not guaranteed to be +// stable and can change without notice. + +#include "tensorflow/c/c_api.h" + +// Macro to control visibility of exported symbols in the shared library (.so, +// .dylib, .dll). +// This duplicates the TF_EXPORT macro definition in +// tensorflow/core/platform/macros.h in order to keep this .h file independent +// of any other includes.$a +#ifdef SWIG +#define TF_CAPI_EXPORT +#else +#if defined(_WIN32) +#ifdef TF_COMPILE_LIBRARY +#define TF_CAPI_EXPORT __declspec(dllexport) +#else +#define TF_CAPI_EXPORT __declspec(dllimport) +#endif // TF_COMPILE_LIBRARY +#else +#define TF_CAPI_EXPORT __attribute__((visibility("default"))) +#endif // _WIN32 +#endif // SWIG + +#ifdef __cplusplus +extern "C" { +#endif + +typedef struct TFE_ContextOptions TFE_ContextOptions; + +// Return a new options object. +TF_CAPI_EXPORT extern TFE_ContextOptions* TFE_NewContextOptions(); + +// Set the config in TF_ContextOptions.options. +// config should be a serialized tensorflow.ConfigProto proto. +// If config was not parsed successfully as a ConfigProto, record the +// error information in *status. +TF_CAPI_EXPORT extern void TFE_ContextOptionsSetConfig( + TFE_ContextOptions* options, const void* proto, size_t proto_len, + TF_Status* status); + +// Controls how to act when we try to run an operation on a given device but +// some input tensors are not on that device. +typedef enum TFE_ContextDevicePlacementPolicy { + // Running operations with input tensors on the wrong device will fail. + TFE_DEVICE_PLACEMENT_EXPLICIT = 0, + // Copy the tensor to the right device but log a warning. + TFE_DEVICE_PLACEMENT_WARN = 1, + // Silently copy the tensor, which has a performance cost since the operation + // will be blocked till the copy completes. This is the default placement + // policy. + TFE_DEVICE_PLACEMENT_SILENT = 2, + // Placement policy which silently copies int32 tensors but not other dtypes. + TFE_DEVICE_PLACEMENT_SILENT_FOR_INT32 = 3, +} TFE_ContextDevicePlacementPolicy; + +// Sets the default execution mode (sync/async). Note that this can be +// overridden per thread using TFE_ContextSetAsyncForThread. +TF_CAPI_EXPORT extern void TFE_ContextOptionsSetAsync(TFE_ContextOptions*, + unsigned char enable); + +TF_CAPI_EXPORT extern void TFE_ContextOptionsSetDevicePlacementPolicy( + TFE_ContextOptions*, TFE_ContextDevicePlacementPolicy); + +// Destroy an options object. +TF_CAPI_EXPORT extern void TFE_DeleteContextOptions(TFE_ContextOptions*); + +// "Context" under which operations/functions are executed. It encapsulates +// things like the available devices, resource manager etc. +// +// TODO(ashankar): Merge with TF_Session? +typedef struct TFE_Context TFE_Context; + +TF_CAPI_EXPORT extern TFE_Context* TFE_NewContext( + const TFE_ContextOptions* opts, TF_Status* status); +TF_CAPI_EXPORT extern void TFE_DeleteContext(TFE_Context* ctx); +TF_CAPI_EXPORT extern TF_DeviceList* TFE_ContextListDevices(TFE_Context* ctx, + TF_Status* status); + +// Clears the internal caches in the TFE context. Useful when reseeding random +// ops. +TF_CAPI_EXPORT extern void TFE_ContextClearCaches(TFE_Context* ctx); + +// Sets a thread-local device placement policy. After this call, other calls to +// TFE_Execute in the same thread will use the device policy specified here +// instead of the device policy used to construct the context. This has no +// effect on the device policy used by other program threads. +TF_CAPI_EXPORT extern void TFE_ContextSetThreadLocalDevicePlacementPolicy( + TFE_Context*, TFE_ContextDevicePlacementPolicy); + +// Returns the device placement policy to be used by this context in the current +// thread. +TF_CAPI_EXPORT extern TFE_ContextDevicePlacementPolicy +TFE_ContextGetDevicePlacementPolicy(TFE_Context*); + +// Overrides the execution mode (sync/async) for the current thread. +TF_CAPI_EXPORT extern void TFE_ContextSetAsyncForThread(TFE_Context*, + unsigned char enable, + TF_Status* status); + +// A tensorflow.ServerDef specifies remote workers (in addition to the current +// workers name). Operations created on this context can then be executed on +// any of these remote workers by setting an appropriate device. +// +// If the following is set, all servers identified by the +// ServerDef must be up when the context is created. +TF_CAPI_EXPORT extern void TFE_ContextSetServerDef(TFE_Context* ctx, + int keep_alive_secs, + const void* proto, + size_t proto_len, + TF_Status* status); + +// Causes the calling thread to block till all ops dispatched in async mode +// have been executed. Note that "execution" here refers to kernel execution / +// scheduling of copies, etc. Similar to sync execution, it doesn't guarantee +// that lower level device queues (like GPU streams) have been flushed. +// +// This call may not block for execution of ops enqueued concurrently with this +// call. +TF_CAPI_EXPORT extern void TFE_ContextAsyncWait(TFE_Context*, + TF_Status* status); + +// When an error happens, any pending operations are discarded and newly issued +// ops return an error. This call clears the error state and re-enables +// execution of newly issued ops. +// +// Note that outputs of discarded ops remain in a corrupt state and should not +// be used for future calls. +// TODO(agarwal): mark the affected handles and raise errors if they are used. +TF_CAPI_EXPORT extern void TFE_ContextAsyncClearError(TFE_Context*); + +// A handle to a tensor on a device. +// +// Like a TF_Tensor, a TFE_TensorHandle refers to a tensor with a value, shape, +// type etc. Unlike a TF_Tensor, a TFE_TensorHandle may refer to such tensors +// placed in memory of different devices or remote address spaces. +typedef struct TFE_TensorHandle TFE_TensorHandle; + +TF_CAPI_EXPORT extern TFE_TensorHandle* TFE_NewTensorHandle(TF_Tensor* t, + TF_Status* status); +// Indicates that the caller will not be using `h` any more. +TF_CAPI_EXPORT extern void TFE_DeleteTensorHandle(TFE_TensorHandle* h); +TF_CAPI_EXPORT extern TF_DataType TFE_TensorHandleDataType(TFE_TensorHandle* h); +// This function will block till the operation that produces `h` has completed. +TF_CAPI_EXPORT extern int TFE_TensorHandleNumDims(TFE_TensorHandle* h, + TF_Status* status); +TF_CAPI_EXPORT extern int64_t TFE_TensorHandleNumElements(TFE_TensorHandle* h, + TF_Status* status); +// This function will block till the operation that produces `h` has completed. +TF_CAPI_EXPORT extern int64_t TFE_TensorHandleDim(TFE_TensorHandle* h, + int dim_index, + TF_Status* status); +// This function will block till the operation that produces `h` has completed. +TF_CAPI_EXPORT extern const char* TFE_TensorHandleDeviceName( + TFE_TensorHandle* h, TF_Status* status); + +// Return a pointer to a new TFE_TensorHandle that shares the underlying tensor +// with `h`. On success, `status` is set to OK. On failure, `status` reflects +// the error and a nullptr is returned. +TF_CAPI_EXPORT extern TFE_TensorHandle* TFE_TensorHandleCopySharingTensor( + TFE_TensorHandle* h, TF_Status* status); + +// This function will block till the operation that produces `h` has +// completed. The memory returned might alias the internal memory used by +// TensorFlow. Hence, callers should not mutate this memory (for example by +// modifying the memory region pointed to by TF_TensorData() on the returned +// TF_Tensor). +TF_CAPI_EXPORT extern TF_Tensor* TFE_TensorHandleResolve(TFE_TensorHandle* h, + TF_Status* status); + +// Create a new TFE_TensorHandle with the same contents as 'h' but placed +// in the memory of the device name 'device_name'. +// If source and destination are the same device, then this creates a new handle +// that shares the underlying buffer. Otherwise, it currently requires at least +// one of the source or destination devices to be CPU (i.e., for the source or +// destination tensor to be placed in host memory). +// If async execution is enabled, the copy may be enqueued and the call will +// return "non-ready" handle. Else, this function returns after the copy has +// been done. +TF_CAPI_EXPORT extern TFE_TensorHandle* TFE_TensorHandleCopyToDevice( + TFE_TensorHandle* h, TFE_Context* ctx, const char* device_name, + TF_Status* status); + +// Debugging/Profiling information for TFE_TensorHandle +// +// TFE_TensorDebugInfo contains information useful for debugging and +// profiling tensors. +typedef struct TFE_TensorDebugInfo TFE_TensorDebugInfo; + +// Retrieves TFE_TensorDebugInfo for `handle`. +// If TFE_TensorHandleTensorDebugInfo succeeds, `status` is set to OK and caller +// is responsible for deleting returned TFE_TensorDebugInfo. +// If TFE_TensorHandleTensorDebugInfo fails, `status` is set to appropriate +// error and nullptr is returned. This function can block till the operation +// that produces `handle` has completed. +TF_CAPI_EXPORT extern TFE_TensorDebugInfo* TFE_TensorHandleTensorDebugInfo( + TFE_TensorHandle* handle, TF_Status* status); + +// Deletes `debug_info`. +TF_CAPI_EXPORT extern void TFE_DeleteTensorDebugInfo( + TFE_TensorDebugInfo* debug_info); + +// Returns the number of dimensions used to represent the tensor on its device. +// The number of dimensions used to reprensent the tensor on device can be +// different from the number returned by TFE_TensorHandleNumDims. +// The return value was current at the time of TFE_TensorDebugInfo creation. +TF_CAPI_EXPORT extern int TFE_TensorDebugInfoOnDeviceNumDims( + TFE_TensorDebugInfo* debug_info); + +// Returns the number of elements in dimension `dim_index`. +// Tensor representation on device can be transposed from its representation +// on host. The data contained in dimension `dim_index` on device +// can correspond to the data contained in another dimension in on-host +// representation. The dimensions are indexed using the standard TensorFlow +// major-to-minor order (slowest varying dimension first), +// not the XLA's minor-to-major order. +// On-device dimensions can be padded. TFE_TensorDebugInfoOnDeviceDim returns +// the number of elements in a dimension after padding. +// The return value was current at the time of TFE_TensorDebugInfo creation. +TF_CAPI_EXPORT extern int64_t TFE_TensorDebugInfoOnDeviceDim( + TFE_TensorDebugInfo* debug_info, int dim_index); + +// Description of the TensorFlow op to execute. +// +// Assumes that the provided 'ctx' outlives the returned TFE_Op, i.e., +// TFE_DeleteOp() is called before TFE_DeleteContext(). +// +// Very similar to TF_OperationDescription with some differences: +// (1) TF_Output or TFE_TensorHandle* as arguments to TF_AddInput, +// TF_AddInputList +// (2) TF_ColocateWith, TF_AddControlInput etc. do not make sense. +// (3) Implementation detail: Avoid use of NodeBuilder/NodeDefBuilder since +// the additional sanity checks there seem unnecessary; +typedef struct TFE_Op TFE_Op; + +TF_CAPI_EXPORT extern TFE_Op* TFE_NewOp(TFE_Context* ctx, + const char* op_or_function_name, + TF_Status* status); + +TF_CAPI_EXPORT extern void TFE_DeleteOp(TFE_Op* op); + +TF_CAPI_EXPORT extern void TFE_OpSetDevice(TFE_Op* op, const char* device_name, + TF_Status* status); +// The returned string remains valid throughout the lifetime of 'op'. +TF_CAPI_EXPORT extern const char* TFE_OpGetDevice(TFE_Op* op, + TF_Status* status); + +// When 'enable' is set to 1, and if TensorFlow library is built with XLA +// support, a subsequent TFE_Execute() call on `op` will run the op via XLA. +// +// If the library is not built with XLA support, this call would be a no-op. +TF_CAPI_EXPORT extern void TFE_OpSetXLACompilation(TFE_Op* op, + unsigned char enable); + +TF_CAPI_EXPORT extern void TFE_OpAddInput(TFE_Op* op, TFE_TensorHandle* h, + TF_Status* status); + +TF_CAPI_EXPORT extern TF_AttrType TFE_OpGetAttrType(TFE_Op* op, + const char* attr_name, + unsigned char* is_list, + TF_Status* status); +// Get an attribute type given an op name; a fusion of TFE_NewOp and +// TFE_OpGetAttrType for use from Python without the overhead of the individual +// calls and memory management of TFE_Op. +TF_CAPI_EXPORT extern TF_AttrType TFE_OpNameGetAttrType( + TFE_Context* ctx, const char* op_or_function_name, const char* attr_name, + unsigned char* is_list, TF_Status* status); + +TF_CAPI_EXPORT extern void TFE_OpSetAttrString(TFE_Op* op, + const char* attr_name, + const void* value, + size_t length); +TF_CAPI_EXPORT extern void TFE_OpSetAttrInt(TFE_Op* op, const char* attr_name, + int64_t value); +TF_CAPI_EXPORT extern void TFE_OpSetAttrFloat(TFE_Op* op, const char* attr_name, + float value); +TF_CAPI_EXPORT extern void TFE_OpSetAttrBool(TFE_Op* op, const char* attr_name, + unsigned char value); +TF_CAPI_EXPORT extern void TFE_OpSetAttrType(TFE_Op* op, const char* attr_name, + TF_DataType value); +// If the number of dimensions is unknown, `num_dims` must be set to +// -1 and `dims` can be null. If a dimension is unknown, the +// corresponding entry in the `dims` array must be -1. +TF_CAPI_EXPORT extern void TFE_OpSetAttrShape(TFE_Op* op, const char* attr_name, + const int64_t* dims, + const int num_dims, + TF_Status* out_status); + +// Sets the attribute attr_name to be a function specified by 'function'. +// +// TODO(ashankar,iga): Add this functionality to the C API for graph +// construction. Perhaps we want an AttrValueMap equivalent in the C API? +TF_CAPI_EXPORT extern void TFE_OpSetAttrFunction(TFE_Op* op, + const char* attr_name, + const TFE_Op* value); + +TF_CAPI_EXPORT void TFE_OpSetAttrFunctionName(TFE_Op* op, const char* attr_name, + const char* data, size_t length); + +TF_CAPI_EXPORT extern void TFE_OpSetAttrTensor(TFE_Op* op, + const char* attr_name, + TF_Tensor* tensor, + TF_Status* status); + +TF_CAPI_EXPORT extern void TFE_OpSetAttrStringList(TFE_Op* op, + const char* attr_name, + const void* const* values, + const size_t* lengths, + int num_values); +TF_CAPI_EXPORT extern void TFE_OpSetAttrIntList(TFE_Op* op, + const char* attr_name, + const int64_t* values, + int num_values); +TF_CAPI_EXPORT extern void TFE_OpSetAttrFloatList(TFE_Op* op, + const char* attr_name, + const float* values, + int num_values); +TF_CAPI_EXPORT extern void TFE_OpSetAttrBoolList(TFE_Op* op, + const char* attr_name, + const unsigned char* values, + int num_values); +TF_CAPI_EXPORT extern void TFE_OpSetAttrTypeList(TFE_Op* op, + const char* attr_name, + const TF_DataType* values, + int num_values); +TF_CAPI_EXPORT extern void TFE_OpSetAttrShapeList( + TFE_Op* op, const char* attr_name, const int64_t** dims, + const int* num_dims, int num_values, TF_Status* out_status); +TF_CAPI_EXPORT extern void TFE_OpSetAttrFunctionList(TFE_Op* op, + const char* attr_name, + const TFE_Op** value, + int num_values); + +// Execute the operation defined by 'op' and return handles to computed +// tensors in `retvals`. +// +// 'retvals' must point to a pre-allocated array of TFE_TensorHandle* and +// '*num_retvals' should be set to the size of this array. It is an error if +// the size of 'retvals' is less than the number of outputs. This call sets +// *num_retvals to the number of outputs. +// +// If async execution is enabled, the call may simply enqueue the execution +// and return "non-ready" handles in `retvals`. Note that any handles contained +// in 'op' should not be mutated till the kernel execution actually finishes. +// +// For sync execution, if any of the inputs to `op` are not ready, this call +// will block till they become ready and then return when the kernel execution +// is done. +// TODO(agarwal): change num_retvals to int from int*. +TF_CAPI_EXPORT extern void TFE_Execute(TFE_Op* op, TFE_TensorHandle** retvals, + int* num_retvals, TF_Status* status); + +// Add a function (serialized FunctionDef protocol buffer) to ctx so +// that it can be invoked using TFE_Execute. +TF_CAPI_EXPORT extern void TFE_ContextAddFunctionDef( + TFE_Context* ctx, const char* serialized_function_def, size_t size, + TF_Status* status); + +// Adds a function (created from TF_GraphToFunction or +// TF_FunctionImportFunctionDef) to the context, allowing it to be executed with +// TFE_Execute by creating an op with the same name as the function. +TF_CAPI_EXPORT extern void TFE_ContextAddFunction(TFE_Context* ctx, + TF_Function* function, + TF_Status* status); + +// Enables tracing of RunMetadata on the ops executed from this context. +TF_CAPI_EXPORT extern void TFE_ContextEnableRunMetadata(TFE_Context* ctx); + +// Disables tracing of RunMetadata on the ops executed from this context. +TF_CAPI_EXPORT extern void TFE_ContextDisableRunMetadata(TFE_Context* ctx); + +// Populates the passed-in buffer with a serialized RunMetadata protocol buffer +// containing any run metadata information accumulated so far and clears this +// information. +// If async mode is enabled, this call blocks till all currently pending ops are +// done. +TF_CAPI_EXPORT extern void TFE_ContextExportRunMetadata(TFE_Context* ctx, + TF_Buffer* buf, + TF_Status* status); + +// Some TF ops need a step container to be set to limit the lifetime of some +// resources (mostly TensorArray and Stack, used in while loop gradients in +// graph mode). Calling this on a context tells it to start a step. +TF_CAPI_EXPORT extern void TFE_ContextStartStep(TFE_Context* ctx); + +// Ends a step. When there is no active step (that is, every started step has +// been ended) step containers will be cleared. Note: it is not safe to call +// TFE_ContextEndStep while ops which rely on the step container may be running. +TF_CAPI_EXPORT extern void TFE_ContextEndStep(TFE_Context* ctx); + +#ifdef __cplusplus +} /* end extern "C" */ +#endif + +#ifdef __cplusplus +// A workaround to ease conversion to and from numpy objects and +// TFE_TensorHandle's. +// +// TODO(ashankar): Figure out an alternative scheme that precludes the need for +// these API-boundary breaking methods. +namespace tensorflow { +class Tensor; +} // namespace tensorflow + +const tensorflow::Tensor* TFE_TensorHandleUnderlyingTensorInHostMemory( + TFE_TensorHandle* h, TF_Status* status); +TFE_TensorHandle* TFE_NewTensorHandle(const tensorflow::Tensor& t); +#endif + +#endif // TENSORFLOW_C_EAGER_C_API_H_ diff --git a/linux/include/tensorflow/c/tf_status_helper.h b/linux/include/tensorflow/c/tf_status_helper.h new file mode 100644 index 000000000..7661a01de --- /dev/null +++ b/linux/include/tensorflow/c/tf_status_helper.h @@ -0,0 +1,32 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_C_TF_STATUS_HELPER_H_ +#define TENSORFLOW_C_TF_STATUS_HELPER_H_ + +#include "tensorflow/c/c_api.h" +#include "tensorflow/core/lib/core/status.h" + +namespace tensorflow { + +// Set the attribute of "tf_status" from the attributes of "status". +void Set_TF_Status_from_Status(TF_Status* tf_status, const Status& status); + +// Returns a "status" from "tf_status". +Status StatusFromTF_Status(const TF_Status* tf_status); + +} // namespace tensorflow + +#endif // TENSORFLOW_C_TF_STATUS_HELPER_H_ diff --git a/linux/include/tensorflow/cc/client/client_session.h b/linux/include/tensorflow/cc/client/client_session.h new file mode 100644 index 000000000..7dd653eec --- /dev/null +++ b/linux/include/tensorflow/cc/client/client_session.h @@ -0,0 +1,129 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CC_CLIENT_CLIENT_SESSION_H_ +#define TENSORFLOW_CC_CLIENT_CLIENT_SESSION_H_ + +#include +#include +#include +#include + +#include "tensorflow/cc/framework/ops.h" +#include "tensorflow/cc/framework/scope.h" +#include "tensorflow/core/public/session_options.h" + +namespace tensorflow { + +/// @addtogroup core +/// @{ + +/// A `ClientSession` object lets the caller drive the evaluation of the +/// TensorFlow graph constructed with the C++ API. +/// +/// Example: +/// +/// Scope root = Scope::NewRootScope(); +/// auto a = Placeholder(root, DT_INT32); +/// auto c = Add(root, a, {41}); +/// +/// ClientSession session(root); +/// std::vector outputs; +/// +/// Status s = session.Run({ {a, {1}} }, {c}, &outputs); +/// if (!s.ok()) { ... } +class ClientSession { + public: + /// A data type to represent feeds to a Run call. + /// + /// This is a map of `Output` objects returned by op-constructors to the value + /// to feed them with. See `Input::Initializer` for details on what can be + /// used as feed values. + typedef std::unordered_map FeedType; + + /// Create a new session to evaluate the graph contained in `scope` by + /// connecting to the TensorFlow runtime specified by `target`. + ClientSession(const Scope& scope, const string& target); + + /// Same as above, but use the empty string ("") as the target specification. + ClientSession(const Scope& scope); + + /// Create a new session, configuring it with `session_options`. + ClientSession(const Scope& scope, const SessionOptions& session_options); + + ~ClientSession(); + + /// Evaluate the tensors in `fetch_outputs`. The values are returned as + /// `Tensor` objects in `outputs`. The number and order of `outputs` will + /// match `fetch_outputs`. + Status Run(const std::vector& fetch_outputs, + std::vector* outputs) const; + + /// Same as above, but use the mapping in `inputs` as feeds. + Status Run(const FeedType& inputs, const std::vector& fetch_outputs, + std::vector* outputs) const; + + /// Same as above. Additionally runs the operations ins `run_outputs`. + Status Run(const FeedType& inputs, const std::vector& fetch_outputs, + const std::vector& run_outputs, + std::vector* outputs) const; + + /// Use `run_options` to turn on performance profiling. `run_metadata`, if not + /// null, is filled in with the profiling results. + Status Run(const RunOptions& run_options, const FeedType& inputs, + const std::vector& fetch_outputs, + const std::vector& run_outputs, + std::vector* outputs, RunMetadata* run_metadata) const; + + /// \brief A handle to a subgraph, created with + /// `ClientSession::MakeCallable()`. + typedef int64 CallableHandle; + + /// \brief Creates a `handle` for invoking the subgraph defined by + /// `callable_options`. + /// NOTE: This API is still experimental and may change. + Status MakeCallable(const CallableOptions& callable_options, + CallableHandle* out_handle); + + /// \brief Invokes the subgraph named by `handle` with the given options and + /// input tensors. + /// + /// The order of tensors in `feed_tensors` must match the order of names in + /// `CallableOptions::feed()` and the order of tensors in `fetch_tensors` will + /// match the order of names in `CallableOptions::fetch()` when this subgraph + /// was created. + /// NOTE: This API is still experimental and may change. + Status RunCallable(CallableHandle handle, + const std::vector& feed_tensors, + std::vector* fetch_tensors, + RunMetadata* run_metadata); + + /// \brief Releases resources associated with the given `handle` in this + /// session. + /// NOTE: This API is still experimental and may change. + Status ReleaseCallable(CallableHandle handle); + + private: + class Impl; + std::unique_ptr impl_; + Impl* impl() { return impl_.get(); } + const Impl* impl() const { return impl_.get(); } +}; + +/// @} + +} // end namespace tensorflow + +#endif // TENSORFLOW_CC_CLIENT_CLIENT_SESSION_H_ diff --git a/linux/include/tensorflow/cc/framework/grad_op_registry.h b/linux/include/tensorflow/cc/framework/grad_op_registry.h new file mode 100644 index 000000000..0fc5abb20 --- /dev/null +++ b/linux/include/tensorflow/cc/framework/grad_op_registry.h @@ -0,0 +1,75 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CC_FRAMEWORK_GRAD_OP_REGISTRY_H_ +#define TENSORFLOW_CC_FRAMEWORK_GRAD_OP_REGISTRY_H_ + +#include + +#include "tensorflow/cc/framework/ops.h" +#include "tensorflow/cc/framework/scope.h" + +namespace tensorflow { +namespace ops { + +/// GradFunc is the signature for all gradient functions in GradOpRegistry. +/// Implementations should add operations to compute the gradient outputs of +/// 'op' (returned in 'grad_outputs') using 'scope' and 'grad_inputs'. +typedef Status (*GradFunc)(const Scope& scope, const Operation& op, + const std::vector& grad_inputs, + std::vector* grad_outputs); + +/// GradOpRegistry maintains a static registry of gradient functions. +/// Gradient functions are indexed in the registry by the forward op name (i.e. +/// "MatMul" -> MatMulGrad func). +class GradOpRegistry { + public: + /// Registers 'func' as the gradient function for 'op'. + /// Returns true if registration was successful, check fails otherwise. + bool Register(const string& op, GradFunc func); + + /// Sets 'func' to the gradient function for 'op' and returns Status OK if + /// the gradient function for 'op' exists in the registry. + /// Note that 'func' can be null for ops that have registered no-gradient with + /// the registry. + /// Returns error status otherwise. + Status Lookup(const string& op, GradFunc* func) const; + + /// Returns a pointer to the global gradient function registry. + static GradOpRegistry* Global(); + + private: + std::unordered_map registry_; +}; + +} // namespace ops + +// Macros used to define gradient functions for ops. +#define REGISTER_GRADIENT_OP(name, fn) \ + REGISTER_GRADIENT_OP_UNIQ_HELPER(__COUNTER__, name, fn) + +#define REGISTER_NO_GRADIENT_OP(name) \ + REGISTER_GRADIENT_OP_UNIQ_HELPER(__COUNTER__, name, nullptr) + +#define REGISTER_GRADIENT_OP_UNIQ_HELPER(ctr, name, fn) \ + REGISTER_GRADIENT_OP_UNIQ(ctr, name, fn) + +#define REGISTER_GRADIENT_OP_UNIQ(ctr, name, fn) \ + static bool unused_ret_val_##ctr = \ + ::tensorflow::ops::GradOpRegistry::Global()->Register(name, fn) + +} // namespace tensorflow + +#endif // TENSORFLOW_CC_FRAMEWORK_GRAD_OP_REGISTRY_H_ diff --git a/linux/include/tensorflow/cc/framework/gradient_checker.h b/linux/include/tensorflow/cc/framework/gradient_checker.h new file mode 100644 index 000000000..1aa215a90 --- /dev/null +++ b/linux/include/tensorflow/cc/framework/gradient_checker.h @@ -0,0 +1,63 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CC_FRAMEWORK_GRADIENT_CHECKER_H_ +#define TENSORFLOW_CC_FRAMEWORK_GRADIENT_CHECKER_H_ + +#include "tensorflow/cc/framework/ops.h" +#include "tensorflow/cc/framework/scope.h" +#include "tensorflow/core/framework/tensor.h" + +namespace tensorflow { + +/// Returns in 'max_error' the maximum element-wise error for dy/dx between the +/// computed and numeric Jacobian matrices where 'xs' and 'ys' are tensors. +/// X_T and Y_T are the c++ types for the x and y tensors, and JAC_T is a +/// real-valued type to store the Jacobian derivatives dy/dx. +/// This function adds operations to the graph associated with 'scope'. +/// +/// Examples: +/// if y = Square(x), where x (and so y) are DT_FLOAT, +/// should be +/// +/// if y = Square(x), where x (and so y) are DT_DOUBLE, +/// should be +/// +/// if y = Square(x), where x (and so y) are DT_COMPLEX64, +/// should be +/// Note that JAC_T is always real-valued, and should be an appropriate +/// precision to host the partial derivatives for dy/dx +/// +/// if y = ComplexAbs(x) where x is DT_COMPLEX64 (so y is DT_FLOAT) +/// should be +/// +/// if y = Complex(x, x) where x is DT_FLOAT (so y is DT_COMPLEX64) +/// should be +template +Status ComputeGradientError(const Scope& scope, const OutputList& xs, + const std::vector& x_shapes, + const OutputList& ys, + const std::vector& y_shapes, + JAC_T* max_error); + +/// Overload of ComputeGradientError which takes an initial value for 'x'. +template +Status ComputeGradientError(const Scope& scope, const Output& x, + const Tensor& x_init_value, const Output& y, + const TensorShape& y_shape, JAC_T* max_error); + +} // namespace tensorflow + +#endif // TENSORFLOW_CC_FRAMEWORK_GRADIENT_CHECKER_H_ diff --git a/linux/include/tensorflow/cc/framework/gradients.h b/linux/include/tensorflow/cc/framework/gradients.h new file mode 100644 index 000000000..0a377ad56 --- /dev/null +++ b/linux/include/tensorflow/cc/framework/gradients.h @@ -0,0 +1,52 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CC_FRAMEWORK_GRADIENTS_H_ +#define TENSORFLOW_CC_FRAMEWORK_GRADIENTS_H_ + +#include "tensorflow/cc/framework/ops.h" +#include "tensorflow/cc/framework/scope.h" + +namespace tensorflow { + +/// NOTE: This API is a work in progress and will likely be changing frequently. +/// +/// Given initial gradients 'grad_inputs' (which represent the symbolic partial +/// derivatives of some loss function 'L' w.r.t 'outputs'), adds gradient nodes +/// to the graph associated with 'scope', which compute (and return in +/// 'grad_outputs') the symbolic partial derivatives of 'L' w.r.t 'inputs'. +Status AddSymbolicGradients(const Scope& scope, + const std::vector& outputs, + const std::vector& inputs, + const std::vector& grad_inputs, + std::vector* grad_outputs); + +// Same as above, but uses 'OnesLike' for all shapes in +// 'outputs' as grad_inputs. +Status AddSymbolicGradients(const Scope& scope, + const std::vector& outputs, + const std::vector& inputs, + std::vector* grad_outputs); + +/// Returns a sentinel Output that represents 'no gradient' (i.e. no gradient +/// flows along some graph edge during backpropagation). +/// Can be returned in 'grad_outputs' by an invocation of 'AddSymbolicGradients' +/// (note that gradient flow through an Output can be stopped through the use of +/// the StopGradient node). +Output NoGradient(); + +} // namespace tensorflow + +#endif // TENSORFLOW_CC_FRAMEWORK_GRADIENTS_H_ diff --git a/linux/include/tensorflow/cc/framework/ops.h b/linux/include/tensorflow/cc/framework/ops.h new file mode 100644 index 000000000..0717e7dd4 --- /dev/null +++ b/linux/include/tensorflow/cc/framework/ops.h @@ -0,0 +1,299 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CC_FRAMEWORK_OPS_H_ +#define TENSORFLOW_CC_FRAMEWORK_OPS_H_ + +#include + +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor.pb.h" +#include "tensorflow/core/graph/graph.h" +#include "tensorflow/core/lib/hash/hash.h" +#include "tensorflow/core/lib/strings/strcat.h" + +namespace tensorflow { + +/// @defgroup core Core Tensorflow API + +class Output; + +/// @addtogroup core +/// @{ + +/// Represents a node in the computation graph. +class Operation { + public: + Operation() : node_(nullptr) {} + explicit Operation(Node* n); + + int32 num_inputs() const { return node_->num_inputs(); } + DataType input_type(int32 o) const { return node_->input_type(o); } + Output input(int32 i) const; + + int32 num_outputs() const { return node_->num_outputs(); } + DataType output_type(int32 o) const { return node_->output_type(o); } + Output output(int32 i) const; + + Node* node() const { return node_; } + + uint64 hash(int32 index) const; + + bool operator==(const Operation& other) const { return node_ == other.node_; } + + private: + typedef std::vector> Inputs; + static Inputs GetInputs(Node* node); + + Inputs inputs_; + Node* node_; +}; + +/// Represents a tensor value produced by an Operation. +class Output { + public: + Output() = default; + explicit Output(Node* n) : op_(n) {} + Output(Node* n, int32 index) : op_(n), index_(index) {} + Output(const Operation& op, int32 index) : op_(op), index_(index) {} + + Operation op() const { return op_; } + Node* node() const { return op().node(); } + int32 index() const { return index_; } + DataType type() const { return op_.output_type(index_); } + string name() const { return strings::StrCat(node()->name(), ":", index()); } + bool operator==(const Output& other) const { + return op_ == other.op_ && index_ == other.index_; + } + + uint64 hash() const { return op_.hash(index_); } + + private: + Operation op_ = Operation(nullptr); + int32 index_ = 0; +}; + +/// Hash class that can be used for e.g. storing Outputs in an unordered_map +struct OutputHash { + std::size_t operator()(const Output& output) const { + return Hash64Combine(std::hash()(output.node()), + std::hash()(output.index())); + } +}; + +/// Represents a tensor value that can be used as an operand to an Operation. +class Input { + public: + /// Initializer enables constructing an Input object from various kinds of C++ + /// constants such as simple primitive constants and nested initializer lists + /// representing a multi-dimensional array. Initializer constructors are all + /// templates, so the aforementioned kinds of C++ constants can be used to + /// construct an Initializer. Initializer stores the value it got constructed + /// with in a Tensor object. + struct Initializer { + /// Construct from a scalar value of an arithmetic type or a type that can + /// be converted to a string (eg. a string literal). + template ::value || + std::is_convertible::value>::type> + Initializer(const T& v) { // NOLINT(runtime/explicit) + typedef typename RealType::type RealT; + Tensor t(DataTypeToEnum::v(), TensorShape()); + t.flat()(0) = RealT(v); + tensor = t; + } + + Initializer(const Tensor& t) : tensor(t) {} // NOLINT(runtime/explicit) + + /// Construct from a scalar value and an explicit shape + template ::value || + std::is_convertible::value>::type> + Initializer(const T& v, const TensorShape& shape) { + typedef typename RealType::type RealT; + Tensor t(DataTypeToEnum::v(), shape); + for (int64 i = 0; i < t.NumElements(); ++i) { + t.flat()(i) = RealT(v); + } + tensor = t; + } + + /// Construct from a initializer list of scalars (a one-dimensional tensor). + template ::value || + std::is_convertible::value>::type> + Initializer( + const std::initializer_list& v) { // NOLINT(runtime/explicit) + typedef typename RealType::type RealT; + Tensor t(DataTypeToEnum::v(), + TensorShape{static_cast(v.size())}); + std::copy_n(v.begin(), v.size(), t.flat().data()); + tensor = t; + } + + /// Construct from a initializer list of scalars and an explicit shape. + template ::value || + std::is_convertible::value>::type> + Initializer(const std::initializer_list& v, const TensorShape& shape) { + typedef typename RealType::type RealT; + Tensor t(DataTypeToEnum::v(), shape); + if (t.NumElements() != static_cast(v.size())) { + status = errors::InvalidArgument( + "Cannot construct a tensor with ", t.NumElements(), + " from an initializer list with ", v.size(), " elements"); + return; + } + std::copy_n(v.begin(), v.size(), t.flat().data()); + tensor = t; + } + + /// Construct a multi-dimensional tensor from a nested initializer + /// list. Note that C++ syntax allows nesting of arbitrarily typed + /// initializer lists, so such invalid initializers cannot be disallowed at + /// compile time. This function performs checks to make sure that the nested + /// initializer list is indeed a valid multi-dimensional tensor. + Initializer(const std::initializer_list& v); + + // START_SKIP_DOXYGEN + template ::value> + struct RealType { + typedef string type; + }; + + template + struct RealType { + typedef T type; + }; + // END_SKIP_DOXYGEN + + TensorProto AsTensorProto() { + TensorProto tensor_proto; + if (tensor.NumElements() > 1) { + tensor.AsProtoTensorContent(&tensor_proto); + } else { + tensor.AsProtoField(&tensor_proto); + } + return tensor_proto; + } + + Status status; + Tensor tensor; + }; + + /// All of Input's constructors are implicit. Input can be implicitly + /// constructed from the following objects : + /// * Output: This is so that the output of an Operation can be directly used + /// as the input to a op wrapper, which takes Inputs. + /// * A scalar, or a multi-dimensional tensor specified as a recursive + /// initializer list. This enables directly passing constants as + /// inputs to op wrappers. + /// * A Tensor object. + Input(const Output& o) : output_(o) {} // NOLINT(runtime/explicit) + + template ::value || + std::is_convertible::value>::type> + Input(const T& v) // NOLINT(runtime/explicit) + : Input(Initializer(v)) {} + + Input(const Initializer& init) // NOLINT(runtime/explicit) + : status_(init.status), + tensor_(init.tensor) {} + + Input(const Tensor& t) // NOLINT(runtime/explicit) + : status_(Status::OK()), + tensor_(t) {} + + Input(const std::initializer_list& + init) { // NOLINT(runtime/explicit) + for (const auto& i : init) { + if (!i.status.ok()) { + status_ = i.status; + return; + } + } + tensor_ = Initializer(init).tensor; + } + + /// Constructor specifying a node name, index and datatype. This should only + /// be used for specifying a backward edge, needed by control flow. + Input(const string& name, int32 i, DataType dt) + : node_name_(name), index_(i), data_type_(dt) {} + + Node* node() const { return output_.node(); } + string node_name() const { return node_name_; } + int32 index() const { return node_name_.empty() ? output_.index() : index_; } + DataType data_type() const { return data_type_; } + Status status() const { return status_; } + const Tensor& tensor() const { return tensor_; } + + private: + Status status_; + Output output_ = Output(Operation(nullptr), 0); + Tensor tensor_; + const string node_name_ = ""; + int32 index_ = 0; + DataType data_type_ = DT_INVALID; +}; + +/// A type for representing the output of ops that produce more than one output, +/// or a list of tensors. +typedef std::vector OutputList; + +/// A type for representing the input to ops that require a list of tensors. +class InputList { + public: + /// Implicitly convert a list of outputs to a list of inputs. This is useful + /// to write code such as ops::Concat(ops::Split(x, 4)). + InputList(const OutputList& out) { // NOLINT(runtime/explicit) + for (auto const& x : out) { + inputs_.push_back(x); + } + } + + InputList( + const std::initializer_list& inputs) // NOLINT(runtime/explicit) + : inputs_(inputs.begin(), inputs.end()) {} + + InputList(const tensorflow::gtl::ArraySlice& + inputs) // NOLINT(runtime/explicit) + : inputs_(inputs.begin(), inputs.end()) {} + + InputList( + const std::initializer_list& out) { // NOLINT(runtime/explicit) + for (auto const& x : out) { + inputs_.push_back(x); + } + } + + typename std::vector::iterator begin() { return inputs_.begin(); } + typename std::vector::iterator end() { return inputs_.end(); } + typename std::vector::const_iterator begin() const { + return inputs_.begin(); + } + typename std::vector::const_iterator end() const { + return inputs_.end(); + } + + private: + std::vector inputs_; +}; + +/// @} + +} // namespace tensorflow + +#endif // TENSORFLOW_CC_FRAMEWORK_OPS_H_ diff --git a/linux/include/tensorflow/cc/framework/scope.h b/linux/include/tensorflow/cc/framework/scope.h new file mode 100644 index 000000000..e307d8989 --- /dev/null +++ b/linux/include/tensorflow/cc/framework/scope.h @@ -0,0 +1,249 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CC_FRAMEWORK_SCOPE_H_ +#define TENSORFLOW_CC_FRAMEWORK_SCOPE_H_ + +#include +#include +#include +#include +#include + +#include "tensorflow/cc/framework/ops.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/lib/gtl/array_slice.h" + +namespace tensorflow { + +class Graph; +class GraphDef; +class NodeBuilder; +struct CompositeOpScopes; + +/// @addtogroup core +/// @{ + +/// A `Scope` object represents a set of related TensorFlow ops that have the +/// same properties such as a common name prefix. +/// +/// A Scope object is a container for TensorFlow Op properties. Op constructors +/// get a Scope object as a mandatory first argument and the constructed op +/// acquires the properties in the object. +/// +/// A simple example: +/// +/// using namespace ops; +/// Scope root = Scope::NewRootScope(); +/// auto c1 = Const(root, { {1, 1} }); +/// auto m = MatMul(root, c1, { {41}, {1} }); +/// GraphDef gdef; +/// Status s = root.ToGraphDef(&gdef); +/// if (!s.ok()) { ... } +/// +/// Scope hierarchy: +/// +/// The Scope class provides various With<> functions that create a new scope. +/// The new scope typically has one property changed while other properties are +/// inherited from the parent scope. +/// NewSubScope(name) method appends `name` to the prefix of names for ops +/// created within the scope, and WithOpName() changes the suffix which +/// otherwise defaults to the type of the op. +/// +/// Name examples: +/// +/// Scope root = Scope::NewRootScope(); +/// Scope linear = root.NewSubScope("linear"); +/// // W will be named "linear/W" +/// auto W = Variable(linear.WithOpName("W"), +/// {2, 2}, DT_FLOAT); +/// // b will be named "linear/b" +/// auto b = Variable(linear.WithOpName("b"), +/// {2}, DT_FLOAT); +/// auto x = Const(linear, {...}); // name: "linear/Const" +/// auto m = MatMul(linear, x, W); // name: "linear/MatMul" +/// auto r = BiasAdd(linear, m, b); // name: "linear/BiasAdd" +/// +/// Scope lifetime: +/// +/// A new scope is created by calling Scope::NewRootScope. This creates some +/// resources that are shared by all the child scopes that inherit from this +/// scope, directly or transitively. For instance, a new scope creates a new +/// Graph object to which operations are added when the new scope or its +/// children are used by an Op constructor. The new scope also has a Status +/// object which will be used to indicate errors by Op-constructor functions +/// called on any child scope. The Op-constructor functions have to check the +/// scope's status by calling the ok() method before proceeding to construct the +/// op. +/// +/// Thread safety: +/// +/// A `Scope` object is NOT thread-safe. Threads cannot concurrently call +/// op-constructor functions on the same `Scope` object. +class Scope { + public: + Scope(const Scope& other); + ~Scope(); + Scope& operator=(const Scope& other); + + // The following functions are for users making graphs. They return brand new + // scopes, or scopes derived from an existing scope object. + + /// Return a new scope. + /// This creates a new graph and all operations constructed in this graph + /// should use the returned object as the "root" scope. + static Scope NewRootScope(); + + /// Return a new scope. Ops created with this scope will have + /// `name/child_scope_name` as the prefix. The actual name will be unique + /// in the current scope. All other properties are inherited from the current + /// scope. If `child_scope_name` is empty, the `/` is elided. + Scope NewSubScope(const string& child_scope_name) const; + + /// Return a new scope. All ops created within the returned scope will have + /// names of the form `name/op_name[_suffix]`. + Scope WithOpName(const string& op_name) const; + + /// Return a new scope. All ops created within the returned scope will have as + /// control dependencies the union of operations in the control_deps vector + /// and the control dependencies of the current scope. + Scope WithControlDependencies( + const gtl::ArraySlice& control_deps) const; + /// Same as above, but convenient to add control dependency on the operation + /// producing the control_dep output. + Scope WithControlDependencies(const Output& control_dep) const; + + /// Return a new scope. All ops created within the returned scope will have no + /// control dependencies on other operations. + Scope WithNoControlDependencies() const; + + /// Return a new scope. All ops created within the returned scope will have + /// the device field set to 'device'. + Scope WithDevice(const string& device) const; + + /// Returns a new scope. All ops created within the returned scope will have + /// their assigned device set to `assigned_device`. + Scope WithAssignedDevice(const string& assigned_device) const; + + /// Return a new scope. All ops created within the returned scope will be + /// co-located on the device where op is placed. + /// NOTE: This function is intended to be use internal libraries only for + /// controlling placement of ops on to devices. Public use is not encouraged + /// because the implementation of device placement is subject to change. + Scope ColocateWith(const Operation& op) const; + /// Convenience function for above. + Scope ColocateWith(const Output& out) const { return ColocateWith(out.op()); } + /// Clear all colocation constraints. + Scope ClearColocation() const; + + /// Return a new scope. The op-constructor functions taking the returned scope + /// as the scope argument will exit as soon as an error is detected, instead + /// of setting the status on the scope. + Scope ExitOnError() const; + + /// Return a new scope. All ops created with the new scope will have + /// kernel_label as the value for their '_kernel' attribute; + Scope WithKernelLabel(const string& kernel_label) const; + + // The following functions are for scope object consumers. + + /// Return a unique name, using default_name if an op name has not been + /// specified. + string GetUniqueNameForOp(const string& default_name) const; + + /// Update the status on this scope. + /// Note: The status object is shared between all children of this scope. + /// If the resulting status is not Status::OK() and exit_on_error_ is set on + /// this scope, this function exits by calling LOG(FATAL). + void UpdateStatus(const Status s) const; + + // START_SKIP_DOXYGEN + + /// Update the builder with properties accumulated in this scope. Does not set + /// status(). + // TODO(skyewm): NodeBuilder is not part of public API + void UpdateBuilder(NodeBuilder* builder) const; + // END_SKIP_DOXYGEN + + CompositeOpScopes GetCompositeOpScopes(const string& composite_op_name) const; + + bool ok() const; + + // TODO(skyewm): Graph is not part of public API + Graph* graph() const; + + // TODO(skyewm): Graph is not part of public API + std::shared_ptr graph_as_shared_ptr() const; + + Status status() const; + + /// If status() is Status::OK(), convert the Graph object stored in this scope + /// to a GraphDef proto and return Status::OK(). Otherwise, return the error + /// status as is without performing GraphDef conversion. + Status ToGraphDef(GraphDef* gdef) const; + + // START_SKIP_DOXYGEN + + /// If status() is Status::OK(), construct a Graph object using the default + /// GraphConstructorOptions, and return Status::OK if graph construction was + /// successful. Otherwise, return the error status. + // TODO(josh11b, keveman): Make this faster; right now it converts + // Graph->GraphDef->Graph. This cleans up the graph (e.g. adds + // edges from the source and to the sink node, resolves back edges + // by name), and makes sure the resulting graph is valid. + Status ToGraph(Graph* g) const; + + // Calls AddNode() using this scope's ShapeRefiner. This exists in the public + // API to prevent custom op wrappers from needing access to shape_refiner.h or + // scope_internal.h. + // TODO(skyewm): remove this from public API + Status DoShapeInference(Node* node) const; + + // Creates a new root scope that causes all DoShapeInference() calls to return + // Status::OK() (on the returned scope and any subscopes). Used for testing. + // TODO(skyewm): fix tests that still require this and eventually remove, or + // at least remove from public API + static Scope DisabledShapeInferenceScope(); + // END_SKIP_DOXYGEN + + const std::vector& control_deps() const; + + // START_SKIP_DOXYGEN + class Impl; + Impl* impl() { return impl_.get(); } + const Impl* impl() const { return impl_.get(); } + // END_SKIP_DOXYGEN + + private: + friend class InternalScope; + std::unique_ptr impl_; + explicit Scope(Impl*); +}; + +/// A helper struct to hold the scopes that would be used by a function +/// constructing a composite op. +struct CompositeOpScopes { + /// Scope to be used for creating the local ops (primitive or other composite + /// ops). + Scope child; + /// Scope to be used for creating the last op. + Scope last; +}; + +/// @} + +} // namespace tensorflow + +#endif // TENSORFLOW_CC_FRAMEWORK_SCOPE_H_ diff --git a/linux/include/tensorflow/cc/framework/scope_internal.h b/linux/include/tensorflow/cc/framework/scope_internal.h new file mode 100644 index 000000000..514e02e84 --- /dev/null +++ b/linux/include/tensorflow/cc/framework/scope_internal.h @@ -0,0 +1,124 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CC_FRAMEWORK_SCOPE_INTERNAL_H_ +#define TENSORFLOW_CC_FRAMEWORK_SCOPE_INTERNAL_H_ + +#include "tensorflow/cc/framework/scope.h" + +namespace tensorflow { + +class ShapeRefiner; + +// NewInternalScope returns a new scope which doesn't take ownership of +// graph, status, name_map, and refiner. +// This is intended to enable the C API (which are used by other language +// bindings) to create a Scope and access C++ functionality (i.e. gradients). +// +// Shape inference is disabled if `refiner` is nullptr. +Scope NewInternalScope(Graph* graph, Status* status, ShapeRefiner* refiner); + +class Scope::Impl { + public: + // A NameMap is used to keep track of suffixes for names used in a scope. A + // name that has not been used so far in a scope will get no suffix. Later + // uses of the same name will get suffixes _1, _2, _3, etc. Multiple scopes + // can share the same NameMap. For instance, a new scope created using + // WithControlDependencies() would share the same NameMap with the parent. + typedef std::unordered_map NameMap; + + Impl(const std::shared_ptr& graph, + const std::shared_ptr& status, + const std::shared_ptr& name_map, + const std::shared_ptr& refiner); + + const string& name() const { return name_; } + const std::vector& control_deps() const { return control_deps_; } + + private: + friend class Scope; + + // Tag types to choose the constructor to dispatch. + struct Tags { + enum class ScopeName; + enum class OpName; + enum class ControlDeps; + enum class Device; + enum class SingleUseScope; + enum class ExitOnError; + enum class KernelLabel; + enum class Colocate; + enum class AssignedDevice; + }; + + Impl(Graph* graph, Status* status, NameMap* name_map, ShapeRefiner* refiner, + bool disable_shape_inference); + Impl(const Scope& other, Tags::ScopeName, const string& name, + bool copy_names); + Impl(const Scope& other, Tags::OpName, const string& name, + const string& op_name); + Impl(const Scope& other, Tags::ControlDeps, + std::vector control_deps, bool clear_control_deps); + Impl(const Scope& other, Tags::Device, const string& device); + Impl(const Scope& other, Tags::SingleUseScope, const string& op_name); + Impl(const Scope& other, Tags::ExitOnError); + Impl(const Scope& other, Tags::KernelLabel, const string& kernel_label); + Impl(const Scope& other, Tags::Colocate, const Operation& colocate_with_op, + bool clear_colocations); + Impl(const Scope& other, Tags::AssignedDevice, const string& assigned_device); + + std::unordered_set GetColocationConstraints( + const Operation& colocate_with_op) const; + + // Helper functions to get a unique names. + string GetUniqueName(const string& prefix, bool check_single_use) const; + string GetNameForOp(const string& default_name) const; + + bool single_use_scope() const { return scope_used_ != nullptr; } + + // The graph, status, and name maps are shared by all child scopes + // created from a single 'root' scope. A root scope is created by calling the + // Scope::NewRootScope function, which creates a new graph, a new status and + // the name maps. + std::shared_ptr graph_ = nullptr; + std::shared_ptr status_ = nullptr; + std::shared_ptr name_map_ = nullptr; + std::shared_ptr refiner_ = nullptr; + + // If scope_used_ is not nullptr, op_name_ should be empty and + // GetUniqueNameForOp can only be called once on this scope. More calls to + // GetUniqueNameForOp will cause an error status to be set on this scope. + std::shared_ptr scope_used_ = nullptr; + + const std::vector control_deps_; + + // The fully-qualified name of this scope (i.e. includes any parent scope + // names). + const string name_ = ""; + const string op_name_ = ""; + const bool exit_on_error_ = false; + const string kernel_label_ = ""; + const string device_ = ""; + const string assigned_device_ = ""; + const std::unordered_set colocation_constraints_; + + // If true, Scope::DoShapeInference() always returns Status:OK(). + // TODO(skyewm): remove this when possible + const bool disable_shape_inference_; +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CC_FRAMEWORK_SCOPE_INTERNAL_H_ diff --git a/linux/include/tensorflow/cc/framework/while_gradients.h b/linux/include/tensorflow/cc/framework/while_gradients.h new file mode 100644 index 000000000..cb4e579c8 --- /dev/null +++ b/linux/include/tensorflow/cc/framework/while_gradients.h @@ -0,0 +1,40 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CC_FRAMEWORK_WHILE_GRADIENTS_H_ +#define TENSORFLOW_CC_FRAMEWORK_WHILE_GRADIENTS_H_ + +#include "tensorflow/cc/framework/ops.h" +#include "tensorflow/cc/framework/scope.h" +#include "tensorflow/core/graph/while_context.h" + +// Utility functions for constructing while loop gradients + +namespace tensorflow { + +// Adds the gradient computation for the while loop associated with +// `while_ctx`. `grad_inputs` are the partial derivatives w.r.t. the loop +// outputs, i.e. the exit nodes. The partial derivatives w.r.t. the loop +// inputs, i.e. the input loop vars, are returned in `grad_outputs`. +// `grad_inputs` and `grad_outputs` are both in loop-variable order, as defined +// by the original inputs to BuildWhileLoop(). +// TODO(skyewm): maybe comment on NoGradient once it's supported +Status AddWhileLoopGradient(WhileContext* while_ctx, const Scope& scope, + const std::vector& grad_inputs, + std::vector* grad_outputs); + +} // namespace tensorflow + +#endif // TENSORFLOW_CC_FRAMEWORK_WHILE_GRADIENTS_H_ diff --git a/linux/include/tensorflow/cc/ops/array_ops.h b/linux/include/tensorflow/cc/ops/array_ops.h new file mode 100644 index 000000000..af1f749d2 --- /dev/null +++ b/linux/include/tensorflow/cc/ops/array_ops.h @@ -0,0 +1,5447 @@ +// This file is MACHINE GENERATED! Do not edit. + +#ifndef TENSORFLOW_CC_OPS_ARRAY_OPS_H_ +#define TENSORFLOW_CC_OPS_ARRAY_OPS_H_ + +// This file is MACHINE GENERATED! Do not edit. + +#include "tensorflow/cc/framework/ops.h" +#include "tensorflow/cc/framework/scope.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/gtl/array_slice.h" + +namespace tensorflow { +namespace ops { + +/// @defgroup array_ops Array Ops +/// @{ + +/// BatchToSpace for 4-D tensors of type T. +/// +/// This is a legacy version of the more general BatchToSpaceND. +/// +/// Rearranges (permutes) data from batch into blocks of spatial data, followed by +/// cropping. This is the reverse transformation of SpaceToBatch. More specifically, +/// this op outputs a copy of the input tensor where values from the `batch` +/// dimension are moved in spatial blocks to the `height` and `width` dimensions, +/// followed by cropping along the `height` and `width` dimensions. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: 4-D tensor with shape +/// `[batch*block_size*block_size, height_pad/block_size, width_pad/block_size, +/// depth]`. Note that the batch size of the input tensor must be divisible by +/// `block_size * block_size`. +/// * crops: 2-D tensor of non-negative integers with shape `[2, 2]`. It specifies +/// how many elements to crop from the intermediate result across the spatial +/// dimensions as follows: +/// +/// crops = [[crop_top, crop_bottom], [crop_left, crop_right]] +/// +/// Returns: +/// * `Output`: 4-D with shape `[batch, height, width, depth]`, where: +/// +/// height = height_pad - crop_top - crop_bottom +/// width = width_pad - crop_left - crop_right +/// +/// The attr `block_size` must be greater than one. It indicates the block size. +/// +/// Some examples: +/// +/// (1) For the following input of shape `[4, 1, 1, 1]` and block_size of 2: +/// +/// ``` +/// [[[[1]]], [[[2]]], [[[3]]], [[[4]]]] +/// ``` +/// +/// The output tensor has shape `[1, 2, 2, 1]` and value: +/// +/// ``` +/// x = [[[[1], [2]], [[3], [4]]]] +/// ``` +/// +/// (2) For the following input of shape `[4, 1, 1, 3]` and block_size of 2: +/// +/// ``` +/// [[[1, 2, 3]], [[4, 5, 6]], [[7, 8, 9]], [[10, 11, 12]]] +/// ``` +/// +/// The output tensor has shape `[1, 2, 2, 3]` and value: +/// +/// ``` +/// x = [[[[1, 2, 3], [4, 5, 6]], +/// [[7, 8, 9], [10, 11, 12]]]] +/// ``` +/// +/// (3) For the following input of shape `[4, 2, 2, 1]` and block_size of 2: +/// +/// ``` +/// x = [[[[1], [3]], [[9], [11]]], +/// [[[2], [4]], [[10], [12]]], +/// [[[5], [7]], [[13], [15]]], +/// [[[6], [8]], [[14], [16]]]] +/// ``` +/// +/// The output tensor has shape `[1, 4, 4, 1]` and value: +/// +/// ``` +/// x = [[[1], [2], [3], [4]], +/// [[5], [6], [7], [8]], +/// [[9], [10], [11], [12]], +/// [[13], [14], [15], [16]]] +/// ``` +/// +/// (4) For the following input of shape `[8, 1, 2, 1]` and block_size of 2: +/// +/// ``` +/// x = [[[[1], [3]]], [[[9], [11]]], [[[2], [4]]], [[[10], [12]]], +/// [[[5], [7]]], [[[13], [15]]], [[[6], [8]]], [[[14], [16]]]] +/// ``` +/// +/// The output tensor has shape `[2, 2, 4, 1]` and value: +/// +/// ``` +/// x = [[[[1], [3]], [[5], [7]]], +/// [[[2], [4]], [[10], [12]]], +/// [[[5], [7]], [[13], [15]]], +/// [[[6], [8]], [[14], [16]]]] +/// ``` +class BatchToSpace { + public: + BatchToSpace(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input crops, int64 block_size); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// BatchToSpace for N-D tensors of type T. +/// +/// This operation reshapes the "batch" dimension 0 into `M + 1` dimensions of shape +/// `block_shape + [batch]`, interleaves these blocks back into the grid defined by +/// the spatial dimensions `[1, ..., M]`, to obtain a result with the same rank as +/// the input. The spatial dimensions of this intermediate result are then +/// optionally cropped according to `crops` to produce the output. This is the +/// reverse of SpaceToBatch. See below for a precise description. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: N-D with shape `input_shape = [batch] + spatial_shape + remaining_shape`, +/// where spatial_shape has M dimensions. +/// * block_shape: 1-D with shape `[M]`, all values must be >= 1. +/// * crops: 2-D with shape `[M, 2]`, all values must be >= 0. +/// `crops[i] = [crop_start, crop_end]` specifies the amount to crop from input +/// dimension `i + 1`, which corresponds to spatial dimension `i`. It is +/// required that +/// `crop_start[i] + crop_end[i] <= block_shape[i] * input_shape[i + 1]`. +/// +/// This operation is equivalent to the following steps: +/// +/// 1. Reshape `input` to `reshaped` of shape: +/// [block_shape[0], ..., block_shape[M-1], +/// batch / prod(block_shape), +/// input_shape[1], ..., input_shape[N-1]] +/// +/// 2. Permute dimensions of `reshaped` to produce `permuted` of shape +/// [batch / prod(block_shape), +/// +/// input_shape[1], block_shape[0], +/// ..., +/// input_shape[M], block_shape[M-1], +/// +/// input_shape[M+1], ..., input_shape[N-1]] +/// +/// 3. Reshape `permuted` to produce `reshaped_permuted` of shape +/// [batch / prod(block_shape), +/// +/// input_shape[1] * block_shape[0], +/// ..., +/// input_shape[M] * block_shape[M-1], +/// +/// input_shape[M+1], +/// ..., +/// input_shape[N-1]] +/// +/// 4. Crop the start and end of dimensions `[1, ..., M]` of +/// `reshaped_permuted` according to `crops` to produce the output of shape: +/// [batch / prod(block_shape), +/// +/// input_shape[1] * block_shape[0] - crops[0,0] - crops[0,1], +/// ..., +/// input_shape[M] * block_shape[M-1] - crops[M-1,0] - crops[M-1,1], +/// +/// input_shape[M+1], ..., input_shape[N-1]] +/// +/// Some examples: +/// +/// (1) For the following input of shape `[4, 1, 1, 1]`, `block_shape = [2, 2]`, and +/// `crops = [[0, 0], [0, 0]]`: +/// +/// ``` +/// [[[[1]]], [[[2]]], [[[3]]], [[[4]]]] +/// ``` +/// +/// The output tensor has shape `[1, 2, 2, 1]` and value: +/// +/// ``` +/// x = [[[[1], [2]], [[3], [4]]]] +/// ``` +/// +/// (2) For the following input of shape `[4, 1, 1, 3]`, `block_shape = [2, 2]`, and +/// `crops = [[0, 0], [0, 0]]`: +/// +/// ``` +/// [[[1, 2, 3]], [[4, 5, 6]], [[7, 8, 9]], [[10, 11, 12]]] +/// ``` +/// +/// The output tensor has shape `[1, 2, 2, 3]` and value: +/// +/// ``` +/// x = [[[[1, 2, 3], [4, 5, 6]], +/// [[7, 8, 9], [10, 11, 12]]]] +/// ``` +/// +/// (3) For the following input of shape `[4, 2, 2, 1]`, `block_shape = [2, 2]`, and +/// `crops = [[0, 0], [0, 0]]`: +/// +/// ``` +/// x = [[[[1], [3]], [[9], [11]]], +/// [[[2], [4]], [[10], [12]]], +/// [[[5], [7]], [[13], [15]]], +/// [[[6], [8]], [[14], [16]]]] +/// ``` +/// +/// The output tensor has shape `[1, 4, 4, 1]` and value: +/// +/// ``` +/// x = [[[1], [2], [3], [4]], +/// [[5], [6], [7], [8]], +/// [[9], [10], [11], [12]], +/// [[13], [14], [15], [16]]] +/// ``` +/// +/// (4) For the following input of shape `[8, 1, 3, 1]`, `block_shape = [2, 2]`, and +/// `crops = [[0, 0], [2, 0]]`: +/// +/// ``` +/// x = [[[[0], [1], [3]]], [[[0], [9], [11]]], +/// [[[0], [2], [4]]], [[[0], [10], [12]]], +/// [[[0], [5], [7]]], [[[0], [13], [15]]], +/// [[[0], [6], [8]]], [[[0], [14], [16]]]] +/// ``` +/// +/// The output tensor has shape `[2, 2, 4, 1]` and value: +/// +/// ``` +/// x = [[[[1], [2], [3], [4]], +/// [[5], [6], [7], [8]]], +/// [[[9], [10], [11], [12]], +/// [[13], [14], [15], [16]]]] +/// ``` +/// +/// Returns: +/// * `Output`: The output tensor. +class BatchToSpaceND { + public: + BatchToSpaceND(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input block_shape, ::tensorflow::Input crops); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Bitcasts a tensor from one type to another without copying data. +/// +/// Given a tensor `input`, this operation returns a tensor that has the same buffer +/// data as `input` with datatype `type`. +/// +/// If the input datatype `T` is larger than the output datatype `type` then the +/// shape changes from [...] to [..., sizeof(`T`)/sizeof(`type`)]. +/// +/// If `T` is smaller than `type`, the operator requires that the rightmost +/// dimension be equal to sizeof(`type`)/sizeof(`T`). The shape then goes from +/// [..., sizeof(`type`)/sizeof(`T`)] to [...]. +/// +/// *NOTE*: Bitcast is implemented as a low-level cast, so machines with different +/// endian orderings will give different results. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The output tensor. +class Bitcast { + public: + Bitcast(const ::tensorflow::Scope& scope, ::tensorflow::Input input, DataType + type); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Return the shape of s0 op s1 with broadcast. +/// +/// Given `s0` and `s1`, tensors that represent shapes, compute `r0`, the +/// broadcasted shape. `s0`, `s1` and `r0` are all integer vectors. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The r0 tensor. +class BroadcastDynamicShape { + public: + BroadcastDynamicShape(const ::tensorflow::Scope& scope, ::tensorflow::Input s0, + ::tensorflow::Input s1); + operator ::tensorflow::Output() const { return r0; } + operator ::tensorflow::Input() const { return r0; } + ::tensorflow::Node* node() const { return r0.node(); } + + Operation operation; + ::tensorflow::Output r0; +}; + +/// Broadcast an array for a compatible shape. +/// +/// Broadcasting is the process of making arrays to have compatible shapes +/// for arithmetic operations. Two shapes are compatible if for each +/// dimension pair they are either equal or one of them is one. When trying +/// to broadcast a Tensor to a shape, it starts with the trailing dimensions, +/// and works its way forward. +/// +/// For example, +/// ``` +/// >>> x = tf.constant([1, 2, 3]) +/// >>> y = tf.broadcast_to(x, [3, 3]) +/// >>> sess.run(y) +/// array([[1, 2, 3], +/// [1, 2, 3], +/// [1, 2, 3]], dtype=int32) +/// ``` +/// In the above example, the input Tensor with the shape of `[1, 3]` +/// is broadcasted to output Tensor with shape of `[3, 3]`. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: A Tensor to broadcast. +/// * shape: An 1-D `int` Tensor. The shape of the desired output. +/// +/// Returns: +/// * `Output`: A Tensor. +class BroadcastTo { + public: + BroadcastTo(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input shape); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Checks a tensor for NaN and Inf values. +/// +/// When run, reports an `InvalidArgument` error if `tensor` has any values +/// that are not a number (NaN) or infinity (Inf). Otherwise, passes `tensor` as-is. +/// +/// Arguments: +/// * scope: A Scope object +/// * message: Prefix of the error message. +/// +/// Returns: +/// * `Output`: The output tensor. +class CheckNumerics { + public: + CheckNumerics(const ::tensorflow::Scope& scope, ::tensorflow::Input tensor, + StringPiece message); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Concatenates tensors along one dimension. +/// +/// Arguments: +/// * scope: A Scope object +/// * values: List of `N` Tensors to concatenate. Their ranks and types must match, +/// and their sizes must match in all dimensions except `concat_dim`. +/// * axis: 0-D. The dimension along which to concatenate. Must be in the +/// range [-rank(values), rank(values)). +/// +/// Returns: +/// * `Output`: A `Tensor` with the concatenation of values stacked along the +/// `concat_dim` dimension. This tensor's shape matches that of `values` except +/// in `concat_dim` where it has the sum of the sizes. +class Concat { + public: + Concat(const ::tensorflow::Scope& scope, ::tensorflow::InputList values, + ::tensorflow::Input axis); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Shuffle dimensions of x according to a permutation and conjugate the result. +/// +/// The output `y` has the same rank as `x`. The shapes of `x` and `y` satisfy: +/// `y.shape[i] == x.shape[perm[i]] for i in [0, 1, ..., rank(x) - 1]` +/// `y[i,j,k,...,s,t,u] == conj(x[perm[i], perm[j], perm[k],...,perm[s], perm[t], perm[u]])` +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The y tensor. +class ConjugateTranspose { + public: + ConjugateTranspose(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input perm); + operator ::tensorflow::Output() const { return y; } + operator ::tensorflow::Input() const { return y; } + ::tensorflow::Node* node() const { return y.node(); } + + Operation operation; + ::tensorflow::Output y; +}; + +/// Identity op for gradient debugging. +/// +/// This op is hidden from public in Python. It is used by TensorFlow Debugger to +/// register gradient tensors for gradient debugging. +/// This op operates on non-reference-type tensors. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The output tensor. +class DebugGradientIdentity { + public: + DebugGradientIdentity(const ::tensorflow::Scope& scope, ::tensorflow::Input + input); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Identity op for gradient debugging. +/// +/// This op is hidden from public in Python. It is used by TensorFlow Debugger to +/// register gradient tensors for gradient debugging. +/// This op operates on reference-type tensors. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The output tensor. +class DebugGradientRefIdentity { + public: + DebugGradientRefIdentity(const ::tensorflow::Scope& scope, ::tensorflow::Input + input); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Makes a copy of `x`. +/// +/// Arguments: +/// * scope: A Scope object +/// * x: The source tensor of type `T`. +/// +/// Returns: +/// * `Output`: y: A `Tensor` of type `T`. A copy of `x`. Guaranteed that `y` +/// is not an alias of `x`. +class DeepCopy { + public: + DeepCopy(const ::tensorflow::Scope& scope, ::tensorflow::Input x); + operator ::tensorflow::Output() const { return y; } + operator ::tensorflow::Input() const { return y; } + ::tensorflow::Node* node() const { return y.node(); } + + Operation operation; + ::tensorflow::Output y; +}; + +/// DepthToSpace for tensors of type T. +/// +/// Rearranges data from depth into blocks of spatial data. +/// This is the reverse transformation of SpaceToDepth. More specifically, +/// this op outputs a copy of the input tensor where values from the `depth` +/// dimension are moved in spatial blocks to the `height` and `width` dimensions. +/// The attr `block_size` indicates the input block size and how the data is moved. +/// +/// * Chunks of data of size `block_size * block_size` from depth are rearranged +/// into non-overlapping blocks of size `block_size x block_size` +/// * The width the output tensor is `input_depth * block_size`, whereas the +/// height is `input_height * block_size`. +/// * The Y, X coordinates within each block of the output image are determined +/// by the high order component of the input channel index. +/// * The depth of the input tensor must be divisible by +/// `block_size * block_size`. +/// +/// The `data_format` attr specifies the layout of the input and output tensors +/// with the following options: +/// "NHWC": `[ batch, height, width, channels ]` +/// "NCHW": `[ batch, channels, height, width ]` +/// "NCHW_VECT_C": +/// `qint8 [ batch, channels / 4, height, width, 4 ]` +/// +/// It is useful to consider the operation as transforming a 6-D Tensor. +/// e.g. for data_format = NHWC, +/// Each element in the input tensor can be specified via 6 coordinates, +/// ordered by decreasing memory layout significance as: +/// n,iY,iX,bY,bX,oC (where n=batch index, iX, iY means X or Y coordinates +/// within the input image, bX, bY means coordinates +/// within the output block, oC means output channels). +/// The output would be the input transposed to the following layout: +/// n,iY,bY,iX,bX,oC +/// +/// This operation is useful for resizing the activations between convolutions +/// (but keeping all data), e.g. instead of pooling. It is also useful for training +/// purely convolutional models. +/// +/// For example, given an input of shape `[1, 1, 1, 4]`, data_format = "NHWC" and +/// block_size = 2: +/// +/// ``` +/// x = [[[[1, 2, 3, 4]]]] +/// +/// ``` +/// +/// This operation will output a tensor of shape `[1, 2, 2, 1]`: +/// +/// ``` +/// [[[[1], [2]], +/// [[3], [4]]]] +/// ``` +/// +/// Here, the input has a batch of 1 and each batch element has shape `[1, 1, 4]`, +/// the corresponding output will have 2x2 elements and will have a depth of +/// 1 channel (1 = `4 / (block_size * block_size)`). +/// The output element shape is `[2, 2, 1]`. +/// +/// For an input tensor with larger depth, here of shape `[1, 1, 1, 12]`, e.g. +/// +/// ``` +/// x = [[[[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]]]] +/// ``` +/// +/// This operation, for block size of 2, will return the following tensor of shape +/// `[1, 2, 2, 3]` +/// +/// ``` +/// [[[[1, 2, 3], [4, 5, 6]], +/// [[7, 8, 9], [10, 11, 12]]]] +/// +/// ``` +/// +/// Similarly, for the following input of shape `[1 2 2 4]`, and a block size of 2: +/// +/// ``` +/// x = [[[[1, 2, 3, 4], +/// [5, 6, 7, 8]], +/// [[9, 10, 11, 12], +/// [13, 14, 15, 16]]]] +/// ``` +/// +/// the operator will return the following tensor of shape `[1 4 4 1]`: +/// +/// ``` +/// x = [[[ [1], [2], [5], [6]], +/// [ [3], [4], [7], [8]], +/// [ [9], [10], [13], [14]], +/// [ [11], [12], [15], [16]]]] +/// +/// ``` +/// +/// Arguments: +/// * scope: A Scope object +/// * block_size: The size of the spatial block, same as in Space2Depth. +/// +/// Returns: +/// * `Output`: The output tensor. +class DepthToSpace { + public: + /// Optional attribute setters for DepthToSpace + struct Attrs { + /// Defaults to "NHWC" + TF_MUST_USE_RESULT Attrs DataFormat(StringPiece x) { + Attrs ret = *this; + ret.data_format_ = x; + return ret; + } + + StringPiece data_format_ = "NHWC"; + }; + DepthToSpace(const ::tensorflow::Scope& scope, ::tensorflow::Input input, int64 + block_size); + DepthToSpace(const ::tensorflow::Scope& scope, ::tensorflow::Input input, int64 + block_size, const DepthToSpace::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs DataFormat(StringPiece x) { + return Attrs().DataFormat(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Dequantize the 'input' tensor into a float Tensor. +/// +/// [min_range, max_range] are scalar floats that specify the range for +/// the 'input' data. The 'mode' attribute controls exactly which calculations are +/// used to convert the float values to their quantized equivalents. +/// +/// In 'MIN_COMBINED' mode, each value of the tensor will undergo the following: +/// +/// ``` +/// if T == qint8, in[i] += (range(T) + 1)/ 2.0 +/// out[i] = min_range + (in[i]* (max_range - min_range) / range(T)) +/// ``` +/// here `range(T) = numeric_limits::max() - numeric_limits::min()` +/// +/// *MIN_COMBINED Mode Example* +/// +/// If the input comes from a QuantizedRelu6, the output type is +/// quint8 (range of 0-255) but the possible range of QuantizedRelu6 is +/// 0-6. The min_range and max_range values are therefore 0.0 and 6.0. +/// Dequantize on quint8 will take each value, cast to float, and multiply +/// by 6 / 255. +/// Note that if quantizedtype is qint8, the operation will additionally add +/// each value by 128 prior to casting. +/// +/// If the mode is 'MIN_FIRST', then this approach is used: +/// +/// ```c++ +/// num_discrete_values = 1 << (# of bits in T) +/// range_adjust = num_discrete_values / (num_discrete_values - 1) +/// range = (range_max - range_min) * range_adjust +/// range_scale = range / num_discrete_values +/// const double offset_input = static_cast(input) - lowest_quantized; +/// result = range_min + ((input - numeric_limits::min()) * range_scale) +/// ``` +/// +/// *SCALED mode Example* +/// +/// `SCALED` mode matches the quantization approach used in +/// `QuantizeAndDequantize{V2|V3}`. +/// +/// If the mode is `SCALED`, we do not use the full range of the output type, +/// choosing to elide the lowest possible value for symmetry (e.g., output range is +/// -127 to 127, not -128 to 127 for signed 8 bit quantization), so that 0.0 maps to +/// 0. +/// +/// We first find the range of values in our tensor. The +/// range we use is always centered on 0, so we find m such that +/// ```c++ +/// m = max(abs(input_min), abs(input_max)) +/// ``` +/// +/// Our input tensor range is then `[-m, m]`. +/// +/// Next, we choose our fixed-point quantization buckets, `[min_fixed, max_fixed]`. +/// If T is signed, this is +/// ``` +/// num_bits = sizeof(T) * 8 +/// [min_fixed, max_fixed] = +/// [-(1 << (num_bits - 1) - 1), (1 << (num_bits - 1)) - 1] +/// ``` +/// +/// Otherwise, if T is unsigned, the fixed-point range is +/// ``` +/// [min_fixed, max_fixed] = [0, (1 << num_bits) - 1] +/// ``` +/// +/// From this we compute our scaling factor, s: +/// ```c++ +/// s = (2 * m) / (max_fixed - min_fixed) +/// ``` +/// +/// Now we can dequantize the elements of our tensor: +/// ```c++ +/// result = input * s +/// ``` +/// +/// Arguments: +/// * scope: A Scope object +/// * min_range: The minimum scalar value possibly produced for the input. +/// * max_range: The maximum scalar value possibly produced for the input. +/// +/// Returns: +/// * `Output`: The output tensor. +class Dequantize { + public: + /// Optional attribute setters for Dequantize + struct Attrs { + /// Defaults to "MIN_COMBINED" + TF_MUST_USE_RESULT Attrs Mode(StringPiece x) { + Attrs ret = *this; + ret.mode_ = x; + return ret; + } + + StringPiece mode_ = "MIN_COMBINED"; + }; + Dequantize(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input min_range, ::tensorflow::Input max_range); + Dequantize(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input min_range, ::tensorflow::Input max_range, const + Dequantize::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs Mode(StringPiece x) { + return Attrs().Mode(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Returns a diagonal tensor with a given diagonal values. +/// +/// Given a `diagonal`, this operation returns a tensor with the `diagonal` and +/// everything else padded with zeros. The diagonal is computed as follows: +/// +/// Assume `diagonal` has dimensions [D1,..., Dk], then the output is a tensor of +/// rank 2k with dimensions [D1,..., Dk, D1,..., Dk] where: +/// +/// `output[i1,..., ik, i1,..., ik] = diagonal[i1, ..., ik]` and 0 everywhere else. +/// +/// For example: +/// +/// ``` +/// # 'diagonal' is [1, 2, 3, 4] +/// tf.diag(diagonal) ==> [[1, 0, 0, 0] +/// [0, 2, 0, 0] +/// [0, 0, 3, 0] +/// [0, 0, 0, 4]] +/// ``` +/// +/// Arguments: +/// * scope: A Scope object +/// * diagonal: Rank k tensor where k is at most 1. +/// +/// Returns: +/// * `Output`: The output tensor. +class Diag { + public: + Diag(const ::tensorflow::Scope& scope, ::tensorflow::Input diagonal); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Returns the diagonal part of the tensor. +/// +/// This operation returns a tensor with the `diagonal` part +/// of the `input`. The `diagonal` part is computed as follows: +/// +/// Assume `input` has dimensions `[D1,..., Dk, D1,..., Dk]`, then the output is a +/// tensor of rank `k` with dimensions `[D1,..., Dk]` where: +/// +/// `diagonal[i1,..., ik] = input[i1, ..., ik, i1,..., ik]`. +/// +/// For example: +/// +/// ``` +/// # 'input' is [[1, 0, 0, 0] +/// [0, 2, 0, 0] +/// [0, 0, 3, 0] +/// [0, 0, 0, 4]] +/// +/// tf.diag_part(input) ==> [1, 2, 3, 4] +/// ``` +/// +/// Arguments: +/// * scope: A Scope object +/// * input: Rank k tensor where k is even and not zero. +/// +/// Returns: +/// * `Output`: The extracted diagonal. +class DiagPart { + public: + DiagPart(const ::tensorflow::Scope& scope, ::tensorflow::Input input); + operator ::tensorflow::Output() const { return diagonal; } + operator ::tensorflow::Input() const { return diagonal; } + ::tensorflow::Node* node() const { return diagonal.node(); } + + Operation operation; + ::tensorflow::Output diagonal; +}; + +/// Computes the (possibly normalized) Levenshtein Edit Distance. +/// +/// The inputs are variable-length sequences provided by SparseTensors +/// (hypothesis_indices, hypothesis_values, hypothesis_shape) +/// and +/// (truth_indices, truth_values, truth_shape). +/// +/// The inputs are: +/// +/// Arguments: +/// * scope: A Scope object +/// * hypothesis_indices: The indices of the hypothesis list SparseTensor. +/// This is an N x R int64 matrix. +/// * hypothesis_values: The values of the hypothesis list SparseTensor. +/// This is an N-length vector. +/// * hypothesis_shape: The shape of the hypothesis list SparseTensor. +/// This is an R-length vector. +/// * truth_indices: The indices of the truth list SparseTensor. +/// This is an M x R int64 matrix. +/// * truth_values: The values of the truth list SparseTensor. +/// This is an M-length vector. +/// * truth_shape: truth indices, vector. +/// +/// Optional attributes (see `Attrs`): +/// * normalize: boolean (if true, edit distances are normalized by length of truth). +/// +/// The output is: +/// +/// Returns: +/// * `Output`: A dense float tensor with rank R - 1. +/// +/// For the example input: +/// +/// // hypothesis represents a 2x1 matrix with variable-length values: +/// // (0,0) = ["a"] +/// // (1,0) = ["b"] +/// hypothesis_indices = [[0, 0, 0], +/// [1, 0, 0]] +/// hypothesis_values = ["a", "b"] +/// hypothesis_shape = [2, 1, 1] +/// +/// // truth represents a 2x2 matrix with variable-length values: +/// // (0,0) = [] +/// // (0,1) = ["a"] +/// // (1,0) = ["b", "c"] +/// // (1,1) = ["a"] +/// truth_indices = [[0, 1, 0], +/// [1, 0, 0], +/// [1, 0, 1], +/// [1, 1, 0]] +/// truth_values = ["a", "b", "c", "a"] +/// truth_shape = [2, 2, 2] +/// normalize = true +/// +/// The output will be: +/// +/// // output is a 2x2 matrix with edit distances normalized by truth lengths. +/// output = [[inf, 1.0], // (0,0): no truth, (0,1): no hypothesis +/// [0.5, 1.0]] // (1,0): addition, (1,1): no hypothesis +class EditDistance { + public: + /// Optional attribute setters for EditDistance + struct Attrs { + /// boolean (if true, edit distances are normalized by length of truth). + /// + /// The output is: + /// + /// Defaults to true + TF_MUST_USE_RESULT Attrs Normalize(bool x) { + Attrs ret = *this; + ret.normalize_ = x; + return ret; + } + + bool normalize_ = true; + }; + EditDistance(const ::tensorflow::Scope& scope, ::tensorflow::Input + hypothesis_indices, ::tensorflow::Input hypothesis_values, + ::tensorflow::Input hypothesis_shape, ::tensorflow::Input + truth_indices, ::tensorflow::Input truth_values, + ::tensorflow::Input truth_shape); + EditDistance(const ::tensorflow::Scope& scope, ::tensorflow::Input + hypothesis_indices, ::tensorflow::Input hypothesis_values, + ::tensorflow::Input hypothesis_shape, ::tensorflow::Input + truth_indices, ::tensorflow::Input truth_values, + ::tensorflow::Input truth_shape, const EditDistance::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs Normalize(bool x) { + return Attrs().Normalize(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Creates a tensor with the given shape. +/// +/// This operation creates a tensor of `shape` and `dtype`. +/// +/// Arguments: +/// * scope: A Scope object +/// * shape: 1-D. Represents the shape of the output tensor. +/// +/// Optional attributes (see `Attrs`): +/// * init: If True, initialize the returned tensor with the default value of dtype. Otherwise, the implementation is free not to initializethe tensor's content. +/// +/// Returns: +/// * `Output`: A `Tensor` of type `T`. +class Empty { + public: + /// Optional attribute setters for Empty + struct Attrs { + /// If True, initialize the returned tensor with the default value of dtype. Otherwise, the implementation is free not to initializethe tensor's content. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs Init(bool x) { + Attrs ret = *this; + ret.init_ = x; + return ret; + } + + bool init_ = false; + }; + Empty(const ::tensorflow::Scope& scope, ::tensorflow::Input shape, DataType + dtype); + Empty(const ::tensorflow::Scope& scope, ::tensorflow::Input shape, DataType + dtype, const Empty::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs Init(bool x) { + return Attrs().Init(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Ensures that the tensor's shape matches the expected shape. +/// +/// Raises an error if the input tensor's shape does not match the specified shape. +/// Returns the input tensor otherwise. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: A tensor, whose shape is to be validated. +/// * shape: The expected (possibly partially specified) shape of the input tensor. +/// +/// Returns: +/// * `Output`: A tensor with the same shape and contents as the input tensor or value. +class EnsureShape { + public: + EnsureShape(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + PartialTensorShape shape); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Inserts a dimension of 1 into a tensor's shape. +/// +/// Given a tensor `input`, this operation inserts a dimension of 1 at the +/// dimension index `axis` of `input`'s shape. The dimension index `axis` starts at +/// zero; if you specify a negative number for `axis` it is counted backward from +/// the end. +/// +/// This operation is useful if you want to add a batch dimension to a single +/// element. For example, if you have a single image of shape `[height, width, +/// channels]`, you can make it a batch of 1 image with `expand_dims(image, 0)`, +/// which will make the shape `[1, height, width, channels]`. +/// +/// Other examples: +/// +/// ``` +/// # 't' is a tensor of shape [2] +/// shape(expand_dims(t, 0)) ==> [1, 2] +/// shape(expand_dims(t, 1)) ==> [2, 1] +/// shape(expand_dims(t, -1)) ==> [2, 1] +/// +/// # 't2' is a tensor of shape [2, 3, 5] +/// shape(expand_dims(t2, 0)) ==> [1, 2, 3, 5] +/// shape(expand_dims(t2, 2)) ==> [2, 3, 1, 5] +/// shape(expand_dims(t2, 3)) ==> [2, 3, 5, 1] +/// ``` +/// +/// This operation requires that: +/// +/// `-1-input.dims() <= dim <= input.dims()` +/// +/// This operation is related to `squeeze()`, which removes dimensions of +/// size 1. +/// +/// Arguments: +/// * scope: A Scope object +/// * axis: 0-D (scalar). Specifies the dimension index at which to +/// expand the shape of `input`. Must be in the range +/// `[-rank(input) - 1, rank(input)]`. +/// +/// Returns: +/// * `Output`: Contains the same data as `input`, but its shape has an additional +/// dimension of size 1 added. +class ExpandDims { + public: + ExpandDims(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input axis); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Extract `patches` from `images` and put them in the "depth" output dimension. +/// +/// Arguments: +/// * scope: A Scope object +/// * images: 4-D Tensor with shape `[batch, in_rows, in_cols, depth]`. +/// * ksizes: The size of the sliding window for each dimension of `images`. +/// * strides: 1-D of length 4. How far the centers of two consecutive patches are in +/// the images. Must be: `[1, stride_rows, stride_cols, 1]`. +/// * rates: 1-D of length 4. Must be: `[1, rate_rows, rate_cols, 1]`. This is the +/// input stride, specifying how far two consecutive patch samples are in the +/// input. Equivalent to extracting patches with +/// `patch_sizes_eff = patch_sizes + (patch_sizes - 1) * (rates - 1)`, followed by +/// subsampling them spatially by a factor of `rates`. This is equivalent to +/// `rate` in dilated (a.k.a. Atrous) convolutions. +/// * padding: The type of padding algorithm to use. +/// +/// We specify the size-related attributes as: +/// +/// ```python +/// ksizes = [1, ksize_rows, ksize_cols, 1] +/// strides = [1, strides_rows, strides_cols, 1] +/// rates = [1, rates_rows, rates_cols, 1] +/// ``` +/// +/// Returns: +/// * `Output`: 4-D Tensor with shape `[batch, out_rows, out_cols, ksize_rows * +/// ksize_cols * depth]` containing image patches with size +/// `ksize_rows x ksize_cols x depth` vectorized in the "depth" dimension. Note +/// `out_rows` and `out_cols` are the dimensions of the output patches. +class ExtractImagePatches { + public: + ExtractImagePatches(const ::tensorflow::Scope& scope, ::tensorflow::Input + images, const gtl::ArraySlice& ksizes, const + gtl::ArraySlice& strides, const gtl::ArraySlice& + rates, StringPiece padding); + operator ::tensorflow::Output() const { return patches; } + operator ::tensorflow::Input() const { return patches; } + ::tensorflow::Node* node() const { return patches.node(); } + + Operation operation; + ::tensorflow::Output patches; +}; + +/// Extract `patches` from `input` and put them in the "depth" output +/// dimension. 3D extension of `extract_image_patches`. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: 5-D Tensor with shape `[batch, in_planes, in_rows, in_cols, depth]`. +/// * ksizes: The size of the sliding window for each dimension of `input`. +/// * strides: 1-D of length 5. How far the centers of two consecutive patches are in +/// `input`. Must be: `[1, stride_planes, stride_rows, stride_cols, 1]`. +/// * padding: The type of padding algorithm to use. +/// +/// We specify the size-related attributes as: +/// +/// ```python +/// ksizes = [1, ksize_planes, ksize_rows, ksize_cols, 1] +/// strides = [1, stride_planes, strides_rows, strides_cols, 1] +/// ``` +/// +/// Returns: +/// * `Output`: 5-D Tensor with shape `[batch, out_planes, out_rows, out_cols, +/// ksize_planes * ksize_rows * ksize_cols * depth]` containing patches +/// with size `ksize_planes x ksize_rows x ksize_cols x depth` vectorized +/// in the "depth" dimension. Note `out_planes`, `out_rows` and `out_cols` +/// are the dimensions of the output patches. +class ExtractVolumePatches { + public: + ExtractVolumePatches(const ::tensorflow::Scope& scope, ::tensorflow::Input + input, const gtl::ArraySlice& ksizes, const + gtl::ArraySlice& strides, StringPiece padding); + operator ::tensorflow::Output() const { return patches; } + operator ::tensorflow::Input() const { return patches; } + ::tensorflow::Node* node() const { return patches.node(); } + + Operation operation; + ::tensorflow::Output patches; +}; + +/// Fake-quantize the 'inputs' tensor, type float to 'outputs' tensor of same type. +/// +/// Attributes `[min; max]` define the clamping range for the `inputs` data. +/// `inputs` values are quantized into the quantization range (`[0; 2^num_bits - 1]` +/// when `narrow_range` is false and `[1; 2^num_bits - 1]` when it is true) and +/// then de-quantized and output as floats in `[min; max]` interval. +/// `num_bits` is the bitwidth of the quantization; between 2 and 16, inclusive. +/// +/// Quantization is called fake since the output is still in floating point. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The outputs tensor. +class FakeQuantWithMinMaxArgs { + public: + /// Optional attribute setters for FakeQuantWithMinMaxArgs + struct Attrs { + /// Defaults to -6 + TF_MUST_USE_RESULT Attrs Min(float x) { + Attrs ret = *this; + ret.min_ = x; + return ret; + } + + /// Defaults to 6 + TF_MUST_USE_RESULT Attrs Max(float x) { + Attrs ret = *this; + ret.max_ = x; + return ret; + } + + /// Defaults to 8 + TF_MUST_USE_RESULT Attrs NumBits(int64 x) { + Attrs ret = *this; + ret.num_bits_ = x; + return ret; + } + + /// Defaults to false + TF_MUST_USE_RESULT Attrs NarrowRange(bool x) { + Attrs ret = *this; + ret.narrow_range_ = x; + return ret; + } + + float min_ = -6.0f; + float max_ = 6.0f; + int64 num_bits_ = 8; + bool narrow_range_ = false; + }; + FakeQuantWithMinMaxArgs(const ::tensorflow::Scope& scope, ::tensorflow::Input + inputs); + FakeQuantWithMinMaxArgs(const ::tensorflow::Scope& scope, ::tensorflow::Input + inputs, const FakeQuantWithMinMaxArgs::Attrs& attrs); + operator ::tensorflow::Output() const { return outputs; } + operator ::tensorflow::Input() const { return outputs; } + ::tensorflow::Node* node() const { return outputs.node(); } + + static Attrs Min(float x) { + return Attrs().Min(x); + } + static Attrs Max(float x) { + return Attrs().Max(x); + } + static Attrs NumBits(int64 x) { + return Attrs().NumBits(x); + } + static Attrs NarrowRange(bool x) { + return Attrs().NarrowRange(x); + } + + Operation operation; + ::tensorflow::Output outputs; +}; + +/// Compute gradients for a FakeQuantWithMinMaxArgs operation. +/// +/// Arguments: +/// * scope: A Scope object +/// * gradients: Backpropagated gradients above the FakeQuantWithMinMaxArgs operation. +/// * inputs: Values passed as inputs to the FakeQuantWithMinMaxArgs operation. +/// +/// Returns: +/// * `Output`: Backpropagated gradients below the FakeQuantWithMinMaxArgs operation: +/// `gradients * (inputs >= min && inputs <= max)`. +class FakeQuantWithMinMaxArgsGradient { + public: + /// Optional attribute setters for FakeQuantWithMinMaxArgsGradient + struct Attrs { + /// Defaults to -6 + TF_MUST_USE_RESULT Attrs Min(float x) { + Attrs ret = *this; + ret.min_ = x; + return ret; + } + + /// Defaults to 6 + TF_MUST_USE_RESULT Attrs Max(float x) { + Attrs ret = *this; + ret.max_ = x; + return ret; + } + + /// Defaults to 8 + TF_MUST_USE_RESULT Attrs NumBits(int64 x) { + Attrs ret = *this; + ret.num_bits_ = x; + return ret; + } + + /// Defaults to false + TF_MUST_USE_RESULT Attrs NarrowRange(bool x) { + Attrs ret = *this; + ret.narrow_range_ = x; + return ret; + } + + float min_ = -6.0f; + float max_ = 6.0f; + int64 num_bits_ = 8; + bool narrow_range_ = false; + }; + FakeQuantWithMinMaxArgsGradient(const ::tensorflow::Scope& scope, + ::tensorflow::Input gradients, + ::tensorflow::Input inputs); + FakeQuantWithMinMaxArgsGradient(const ::tensorflow::Scope& scope, + ::tensorflow::Input gradients, + ::tensorflow::Input inputs, const + FakeQuantWithMinMaxArgsGradient::Attrs& attrs); + operator ::tensorflow::Output() const { return backprops; } + operator ::tensorflow::Input() const { return backprops; } + ::tensorflow::Node* node() const { return backprops.node(); } + + static Attrs Min(float x) { + return Attrs().Min(x); + } + static Attrs Max(float x) { + return Attrs().Max(x); + } + static Attrs NumBits(int64 x) { + return Attrs().NumBits(x); + } + static Attrs NarrowRange(bool x) { + return Attrs().NarrowRange(x); + } + + Operation operation; + ::tensorflow::Output backprops; +}; + +/// Fake-quantize the 'inputs' tensor of type float via global float scalars `min` +/// +/// and `max` to 'outputs' tensor of same shape as `inputs`. +/// +/// `[min; max]` define the clamping range for the `inputs` data. +/// `inputs` values are quantized into the quantization range (`[0; 2^num_bits - 1]` +/// when `narrow_range` is false and `[1; 2^num_bits - 1]` when it is true) and +/// then de-quantized and output as floats in `[min; max]` interval. +/// `num_bits` is the bitwidth of the quantization; between 2 and 16, inclusive. +/// +/// This operation has a gradient and thus allows for training `min` and `max` +/// values. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The outputs tensor. +class FakeQuantWithMinMaxVars { + public: + /// Optional attribute setters for FakeQuantWithMinMaxVars + struct Attrs { + /// Defaults to 8 + TF_MUST_USE_RESULT Attrs NumBits(int64 x) { + Attrs ret = *this; + ret.num_bits_ = x; + return ret; + } + + /// Defaults to false + TF_MUST_USE_RESULT Attrs NarrowRange(bool x) { + Attrs ret = *this; + ret.narrow_range_ = x; + return ret; + } + + int64 num_bits_ = 8; + bool narrow_range_ = false; + }; + FakeQuantWithMinMaxVars(const ::tensorflow::Scope& scope, ::tensorflow::Input + inputs, ::tensorflow::Input min, ::tensorflow::Input + max); + FakeQuantWithMinMaxVars(const ::tensorflow::Scope& scope, ::tensorflow::Input + inputs, ::tensorflow::Input min, ::tensorflow::Input + max, const FakeQuantWithMinMaxVars::Attrs& attrs); + operator ::tensorflow::Output() const { return outputs; } + operator ::tensorflow::Input() const { return outputs; } + ::tensorflow::Node* node() const { return outputs.node(); } + + static Attrs NumBits(int64 x) { + return Attrs().NumBits(x); + } + static Attrs NarrowRange(bool x) { + return Attrs().NarrowRange(x); + } + + Operation operation; + ::tensorflow::Output outputs; +}; + +/// Compute gradients for a FakeQuantWithMinMaxVars operation. +/// +/// Arguments: +/// * scope: A Scope object +/// * gradients: Backpropagated gradients above the FakeQuantWithMinMaxVars operation. +/// * inputs: Values passed as inputs to the FakeQuantWithMinMaxVars operation. +/// min, max: Quantization interval, scalar floats. +/// +/// Optional attributes (see `Attrs`): +/// * num_bits: The bitwidth of the quantization; between 2 and 8, inclusive. +/// * narrow_range: Whether to quantize into 2^num_bits - 1 distinct values. +/// +/// Returns: +/// * `Output` backprops_wrt_input: Backpropagated gradients w.r.t. inputs: +/// `gradients * (inputs >= min && inputs <= max)`. +/// * `Output` backprop_wrt_min: Backpropagated gradients w.r.t. min parameter: +/// `sum(gradients * (inputs < min))`. +/// * `Output` backprop_wrt_max: Backpropagated gradients w.r.t. max parameter: +/// `sum(gradients * (inputs > max))`. +class FakeQuantWithMinMaxVarsGradient { + public: + /// Optional attribute setters for FakeQuantWithMinMaxVarsGradient + struct Attrs { + /// The bitwidth of the quantization; between 2 and 8, inclusive. + /// + /// Defaults to 8 + TF_MUST_USE_RESULT Attrs NumBits(int64 x) { + Attrs ret = *this; + ret.num_bits_ = x; + return ret; + } + + /// Whether to quantize into 2^num_bits - 1 distinct values. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs NarrowRange(bool x) { + Attrs ret = *this; + ret.narrow_range_ = x; + return ret; + } + + int64 num_bits_ = 8; + bool narrow_range_ = false; + }; + FakeQuantWithMinMaxVarsGradient(const ::tensorflow::Scope& scope, + ::tensorflow::Input gradients, + ::tensorflow::Input inputs, ::tensorflow::Input + min, ::tensorflow::Input max); + FakeQuantWithMinMaxVarsGradient(const ::tensorflow::Scope& scope, + ::tensorflow::Input gradients, + ::tensorflow::Input inputs, ::tensorflow::Input + min, ::tensorflow::Input max, const + FakeQuantWithMinMaxVarsGradient::Attrs& attrs); + + static Attrs NumBits(int64 x) { + return Attrs().NumBits(x); + } + static Attrs NarrowRange(bool x) { + return Attrs().NarrowRange(x); + } + + Operation operation; + ::tensorflow::Output backprops_wrt_input; + ::tensorflow::Output backprop_wrt_min; + ::tensorflow::Output backprop_wrt_max; +}; + +/// Fake-quantize the 'inputs' tensor of type float and one of the shapes: `[d]`, +/// +/// `[b, d]` `[b, h, w, d]` via per-channel floats `min` and `max` of shape `[d]` +/// to 'outputs' tensor of same shape as `inputs`. +/// +/// `[min; max]` define the clamping range for the `inputs` data. +/// `inputs` values are quantized into the quantization range (`[0; 2^num_bits - 1]` +/// when `narrow_range` is false and `[1; 2^num_bits - 1]` when it is true) and +/// then de-quantized and output as floats in `[min; max]` interval. +/// `num_bits` is the bitwidth of the quantization; between 2 and 16, inclusive. +/// +/// This operation has a gradient and thus allows for training `min` and `max` +/// values. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The outputs tensor. +class FakeQuantWithMinMaxVarsPerChannel { + public: + /// Optional attribute setters for FakeQuantWithMinMaxVarsPerChannel + struct Attrs { + /// Defaults to 8 + TF_MUST_USE_RESULT Attrs NumBits(int64 x) { + Attrs ret = *this; + ret.num_bits_ = x; + return ret; + } + + /// Defaults to false + TF_MUST_USE_RESULT Attrs NarrowRange(bool x) { + Attrs ret = *this; + ret.narrow_range_ = x; + return ret; + } + + int64 num_bits_ = 8; + bool narrow_range_ = false; + }; + FakeQuantWithMinMaxVarsPerChannel(const ::tensorflow::Scope& scope, + ::tensorflow::Input inputs, + ::tensorflow::Input min, ::tensorflow::Input + max); + FakeQuantWithMinMaxVarsPerChannel(const ::tensorflow::Scope& scope, + ::tensorflow::Input inputs, + ::tensorflow::Input min, ::tensorflow::Input + max, const + FakeQuantWithMinMaxVarsPerChannel::Attrs& + attrs); + operator ::tensorflow::Output() const { return outputs; } + operator ::tensorflow::Input() const { return outputs; } + ::tensorflow::Node* node() const { return outputs.node(); } + + static Attrs NumBits(int64 x) { + return Attrs().NumBits(x); + } + static Attrs NarrowRange(bool x) { + return Attrs().NarrowRange(x); + } + + Operation operation; + ::tensorflow::Output outputs; +}; + +/// Compute gradients for a FakeQuantWithMinMaxVarsPerChannel operation. +/// +/// Arguments: +/// * scope: A Scope object +/// * gradients: Backpropagated gradients above the FakeQuantWithMinMaxVars operation, +/// shape one of: `[d]`, `[b, d]`, `[b, h, w, d]`. +/// * inputs: Values passed as inputs to the FakeQuantWithMinMaxVars operation, shape +/// same as `gradients`. +/// min, max: Quantization interval, floats of shape `[d]`. +/// +/// Optional attributes (see `Attrs`): +/// * num_bits: The bitwidth of the quantization; between 2 and 16, inclusive. +/// * narrow_range: Whether to quantize into 2^num_bits - 1 distinct values. +/// +/// Returns: +/// * `Output` backprops_wrt_input: Backpropagated gradients w.r.t. inputs, shape same as +/// `inputs`: +/// `gradients * (inputs >= min && inputs <= max)`. +/// * `Output` backprop_wrt_min: Backpropagated gradients w.r.t. min parameter, shape `[d]`: +/// `sum_per_d(gradients * (inputs < min))`. +/// * `Output` backprop_wrt_max: Backpropagated gradients w.r.t. max parameter, shape `[d]`: +/// `sum_per_d(gradients * (inputs > max))`. +class FakeQuantWithMinMaxVarsPerChannelGradient { + public: + /// Optional attribute setters for FakeQuantWithMinMaxVarsPerChannelGradient + struct Attrs { + /// The bitwidth of the quantization; between 2 and 16, inclusive. + /// + /// Defaults to 8 + TF_MUST_USE_RESULT Attrs NumBits(int64 x) { + Attrs ret = *this; + ret.num_bits_ = x; + return ret; + } + + /// Whether to quantize into 2^num_bits - 1 distinct values. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs NarrowRange(bool x) { + Attrs ret = *this; + ret.narrow_range_ = x; + return ret; + } + + int64 num_bits_ = 8; + bool narrow_range_ = false; + }; + FakeQuantWithMinMaxVarsPerChannelGradient(const ::tensorflow::Scope& scope, + ::tensorflow::Input gradients, + ::tensorflow::Input inputs, + ::tensorflow::Input min, + ::tensorflow::Input max); + FakeQuantWithMinMaxVarsPerChannelGradient(const ::tensorflow::Scope& scope, + ::tensorflow::Input gradients, + ::tensorflow::Input inputs, + ::tensorflow::Input min, + ::tensorflow::Input max, const + FakeQuantWithMinMaxVarsPerChannelGradient::Attrs& + attrs); + + static Attrs NumBits(int64 x) { + return Attrs().NumBits(x); + } + static Attrs NarrowRange(bool x) { + return Attrs().NarrowRange(x); + } + + Operation operation; + ::tensorflow::Output backprops_wrt_input; + ::tensorflow::Output backprop_wrt_min; + ::tensorflow::Output backprop_wrt_max; +}; + +/// Creates a tensor filled with a scalar value. +/// +/// This operation creates a tensor of shape `dims` and fills it with `value`. +/// +/// For example: +/// +/// ``` +/// # Output tensor has shape [2, 3]. +/// fill([2, 3], 9) ==> [[9, 9, 9] +/// [9, 9, 9]] +/// ``` +/// +/// `tf.fill` differs from `tf.constant` in a few ways: +/// +/// * `tf.fill` only supports scalar contents, whereas `tf.constant` supports +/// Tensor values. +/// * `tf.fill` creates an Op in the computation graph that constructs the actual +/// Tensor value at runtime. This is in contrast to `tf.constant` which embeds +/// the entire Tensor into the graph with a `Const` node. +/// * Because `tf.fill` evaluates at graph runtime, it supports dynamic shapes +/// based on other runtime Tensors, unlike `tf.constant`. +/// +/// Arguments: +/// * scope: A Scope object +/// * dims: 1-D. Represents the shape of the output tensor. +/// * value: 0-D (scalar). Value to fill the returned tensor. +/// +/// @compatibility(numpy) +/// Equivalent to np.full +/// @end_compatibility +/// +/// Returns: +/// * `Output`: The output tensor. +class Fill { + public: + Fill(const ::tensorflow::Scope& scope, ::tensorflow::Input dims, + ::tensorflow::Input value); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Gather slices from `params` according to `indices`. +/// +/// `indices` must be an integer tensor of any dimension (usually 0-D or 1-D). +/// Produces an output tensor with shape `indices.shape + params.shape[1:]` where: +/// +/// ```python +/// # Scalar indices +/// output[:, ..., :] = params[indices, :, ... :] +/// +/// # Vector indices +/// output[i, :, ..., :] = params[indices[i], :, ... :] +/// +/// # Higher rank indices +/// output[i, ..., j, :, ... :] = params[indices[i, ..., j], :, ..., :] +/// ``` +/// +/// If `indices` is a permutation and `len(indices) == params.shape[0]` then +/// this operation will permute `params` accordingly. +/// +/// `validate_indices`: DEPRECATED. If this operation is assigned to CPU, values in +/// `indices` are always validated to be within range. If assigned to GPU, +/// out-of-bound indices result in safe but unspecified behavior, which may include +/// raising an error. +/// +///
+/// +///
+/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The output tensor. +class Gather { + public: + /// Optional attribute setters for Gather + struct Attrs { + /// Defaults to true + TF_MUST_USE_RESULT Attrs ValidateIndices(bool x) { + Attrs ret = *this; + ret.validate_indices_ = x; + return ret; + } + + bool validate_indices_ = true; + }; + Gather(const ::tensorflow::Scope& scope, ::tensorflow::Input params, + ::tensorflow::Input indices); + Gather(const ::tensorflow::Scope& scope, ::tensorflow::Input params, + ::tensorflow::Input indices, const Gather::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs ValidateIndices(bool x) { + return Attrs().ValidateIndices(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Gather slices from `params` into a Tensor with shape specified by `indices`. +/// +/// `indices` is an K-dimensional integer tensor, best thought of as a +/// (K-1)-dimensional tensor of indices into `params`, where each element defines a +/// slice of `params`: +/// +/// output[\\(i_0, ..., i_{K-2}\\)] = params[indices[\\(i_0, ..., i_{K-2}\\)]] +/// +/// Whereas in `tf.gather` `indices` defines slices into the first +/// dimension of `params`, in `tf.gather_nd`, `indices` defines slices into the +/// first `N` dimensions of `params`, where `N = indices.shape[-1]`. +/// +/// The last dimension of `indices` can be at most the rank of +/// `params`: +/// +/// indices.shape[-1] <= params.rank +/// +/// The last dimension of `indices` corresponds to elements +/// (if `indices.shape[-1] == params.rank`) or slices +/// (if `indices.shape[-1] < params.rank`) along dimension `indices.shape[-1]` +/// of `params`. The output tensor has shape +/// +/// indices.shape[:-1] + params.shape[indices.shape[-1]:] +/// +/// Note that on CPU, if an out of bound index is found, an error is returned. +/// On GPU, if an out of bound index is found, a 0 is stored in the +/// corresponding output value. +/// +/// Some examples below. +/// +/// Simple indexing into a matrix: +/// +/// ```python +/// indices = [[0, 0], [1, 1]] +/// params = [['a', 'b'], ['c', 'd']] +/// output = ['a', 'd'] +/// ``` +/// +/// Slice indexing into a matrix: +/// +/// ```python +/// indices = [[1], [0]] +/// params = [['a', 'b'], ['c', 'd']] +/// output = [['c', 'd'], ['a', 'b']] +/// ``` +/// +/// Indexing into a 3-tensor: +/// +/// ```python +/// indices = [[1]] +/// params = [[['a0', 'b0'], ['c0', 'd0']], +/// [['a1', 'b1'], ['c1', 'd1']]] +/// output = [[['a1', 'b1'], ['c1', 'd1']]] +/// +/// +/// indices = [[0, 1], [1, 0]] +/// params = [[['a0', 'b0'], ['c0', 'd0']], +/// [['a1', 'b1'], ['c1', 'd1']]] +/// output = [['c0', 'd0'], ['a1', 'b1']] +/// +/// +/// indices = [[0, 0, 1], [1, 0, 1]] +/// params = [[['a0', 'b0'], ['c0', 'd0']], +/// [['a1', 'b1'], ['c1', 'd1']]] +/// output = ['b0', 'b1'] +/// ``` +/// +/// Batched indexing into a matrix: +/// +/// ```python +/// indices = [[[0, 0]], [[0, 1]]] +/// params = [['a', 'b'], ['c', 'd']] +/// output = [['a'], ['b']] +/// ``` +/// +/// Batched slice indexing into a matrix: +/// +/// ```python +/// indices = [[[1]], [[0]]] +/// params = [['a', 'b'], ['c', 'd']] +/// output = [[['c', 'd']], [['a', 'b']]] +/// ``` +/// +/// Batched indexing into a 3-tensor: +/// +/// ```python +/// indices = [[[1]], [[0]]] +/// params = [[['a0', 'b0'], ['c0', 'd0']], +/// [['a1', 'b1'], ['c1', 'd1']]] +/// output = [[[['a1', 'b1'], ['c1', 'd1']]], +/// [[['a0', 'b0'], ['c0', 'd0']]]] +/// +/// indices = [[[0, 1], [1, 0]], [[0, 0], [1, 1]]] +/// params = [[['a0', 'b0'], ['c0', 'd0']], +/// [['a1', 'b1'], ['c1', 'd1']]] +/// output = [[['c0', 'd0'], ['a1', 'b1']], +/// [['a0', 'b0'], ['c1', 'd1']]] +/// +/// +/// indices = [[[0, 0, 1], [1, 0, 1]], [[0, 1, 1], [1, 1, 0]]] +/// params = [[['a0', 'b0'], ['c0', 'd0']], +/// [['a1', 'b1'], ['c1', 'd1']]] +/// output = [['b0', 'b1'], ['d0', 'c1']] +/// ``` +/// +/// See also `tf.gather` and `tf.batch_gather`. +/// +/// Arguments: +/// * scope: A Scope object +/// * params: The tensor from which to gather values. +/// * indices: Index tensor. +/// +/// Returns: +/// * `Output`: Values from `params` gathered from indices given by `indices`, with +/// shape `indices.shape[:-1] + params.shape[indices.shape[-1]:]`. +class GatherNd { + public: + GatherNd(const ::tensorflow::Scope& scope, ::tensorflow::Input params, + ::tensorflow::Input indices); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Gather slices from `params` axis `axis` according to `indices`. +/// +/// `indices` must be an integer tensor of any dimension (usually 0-D or 1-D). +/// Produces an output tensor with shape `params.shape[:axis] + indices.shape + +/// params.shape[axis + 1:]` where: +/// +/// ```python +/// # Scalar indices (output is rank(params) - 1). +/// output[a_0, ..., a_n, b_0, ..., b_n] = +/// params[a_0, ..., a_n, indices, b_0, ..., b_n] +/// +/// # Vector indices (output is rank(params)). +/// output[a_0, ..., a_n, i, b_0, ..., b_n] = +/// params[a_0, ..., a_n, indices[i], b_0, ..., b_n] +/// +/// # Higher rank indices (output is rank(params) + rank(indices) - 1). +/// output[a_0, ..., a_n, i, ..., j, b_0, ... b_n] = +/// params[a_0, ..., a_n, indices[i, ..., j], b_0, ..., b_n] +/// ``` +/// +///
+/// +///
+/// +/// Note that on CPU, if an out of bound index is found, an error is returned. +/// On GPU, if an out of bound index is found, a 0 is stored in the +/// corresponding output value. +/// +/// See also `tf.batch_gather` and `tf.gather_nd`. +/// +/// Arguments: +/// * scope: A Scope object +/// * params: The tensor from which to gather values. Must be at least rank +/// `axis + 1`. +/// * indices: Index tensor. Must be in range `[0, params.shape[axis])`. +/// * axis: The axis in `params` to gather `indices` from. Defaults to the first +/// dimension. Supports negative indexes. +/// +/// Returns: +/// * `Output`: Values from `params` gathered from indices given by `indices`, with +/// shape `params.shape[:axis] + indices.shape + params.shape[axis + 1:]`. +class GatherV2 { + public: + GatherV2(const ::tensorflow::Scope& scope, ::tensorflow::Input params, + ::tensorflow::Input indices, ::tensorflow::Input axis); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Gives a guarantee to the TF runtime that the input tensor is a constant. +/// +/// The runtime is then free to make optimizations based on this. +/// +/// Only accepts value typed tensors as inputs and rejects resource variable handles +/// as input. +/// +/// Returns the input tensor without modification. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The output tensor. +class GuaranteeConst { + public: + GuaranteeConst(const ::tensorflow::Scope& scope, ::tensorflow::Input input); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Return a tensor with the same shape and contents as the input tensor or value. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The output tensor. +class Identity { + public: + Identity(const ::tensorflow::Scope& scope, ::tensorflow::Input input); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Returns a list of tensors with the same shapes and contents as the input +/// +/// tensors. +/// +/// This op can be used to override the gradient for complicated functions. For +/// example, suppose y = f(x) and we wish to apply a custom function g for backprop +/// such that dx = g(dy). In Python, +/// +/// ```python +/// with tf.get_default_graph().gradient_override_map( +/// {'IdentityN': 'OverrideGradientWithG'}): +/// y, _ = identity_n([f(x), x]) +/// +/// @tf.RegisterGradient('OverrideGradientWithG') +/// def ApplyG(op, dy, _): +/// return [None, g(dy)] # Do not backprop to f(x). +/// ``` +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `OutputList`: The output tensor. +class IdentityN { + public: + IdentityN(const ::tensorflow::Scope& scope, ::tensorflow::InputList input); + ::tensorflow::Output operator[](size_t index) const { return output[index]; } + + + Operation operation; + ::tensorflow::OutputList output; +}; + +/// Returns immutable tensor from memory region. +/// +/// The current implementation memmaps the tensor from a file. +/// +/// Arguments: +/// * scope: A Scope object +/// * dtype: Type of the returned tensor. +/// * shape: Shape of the returned tensor. +/// * memory_region_name: Name of readonly memory region used by the tensor, see +/// NewReadOnlyMemoryRegionFromFile in tensorflow::Env. +/// +/// Returns: +/// * `Output`: The tensor tensor. +class ImmutableConst { + public: + ImmutableConst(const ::tensorflow::Scope& scope, DataType dtype, + PartialTensorShape shape, StringPiece memory_region_name); + operator ::tensorflow::Output() const { return tensor; } + operator ::tensorflow::Input() const { return tensor; } + ::tensorflow::Node* node() const { return tensor.node(); } + + Operation operation; + ::tensorflow::Output tensor; +}; + +/// Adds v into specified rows of x. +/// +/// Computes y = x; y[i, :] += v; return y. +/// +/// Arguments: +/// * scope: A Scope object +/// * x: A `Tensor` of type T. +/// * i: A vector. Indices into the left-most dimension of `x`. +/// * v: A `Tensor` of type T. Same dimension sizes as x except the first dimension, which must be the same as i's size. +/// +/// Returns: +/// * `Output`: A `Tensor` of type T. An alias of `x`. The content of `y` is undefined if there are duplicates in `i`. +class InplaceAdd { + public: + InplaceAdd(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input i, ::tensorflow::Input v); + operator ::tensorflow::Output() const { return y; } + operator ::tensorflow::Input() const { return y; } + ::tensorflow::Node* node() const { return y.node(); } + + Operation operation; + ::tensorflow::Output y; +}; + +/// Subtracts `v` into specified rows of `x`. +/// +/// Computes y = x; y[i, :] -= v; return y. +/// +/// Arguments: +/// * scope: A Scope object +/// * x: A `Tensor` of type T. +/// * i: A vector. Indices into the left-most dimension of `x`. +/// * v: A `Tensor` of type T. Same dimension sizes as x except the first dimension, which must be the same as i's size. +/// +/// Returns: +/// * `Output`: A `Tensor` of type T. An alias of `x`. The content of `y` is undefined if there are duplicates in `i`. +class InplaceSub { + public: + InplaceSub(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input i, ::tensorflow::Input v); + operator ::tensorflow::Output() const { return y; } + operator ::tensorflow::Input() const { return y; } + ::tensorflow::Node* node() const { return y.node(); } + + Operation operation; + ::tensorflow::Output y; +}; + +/// Updates specified rows with values in `v`. +/// +/// Computes `x[i, :] = v; return x`. +/// +/// Arguments: +/// * scope: A Scope object +/// * x: A tensor of type `T`. +/// * i: A vector. Indices into the left-most dimension of `x`. +/// * v: A `Tensor` of type T. Same dimension sizes as x except the first dimension, which must be the same as i's size. +/// +/// Returns: +/// * `Output`: A `Tensor` of type T. An alias of `x`. The content of `y` is undefined if there are duplicates in `i`. +class InplaceUpdate { + public: + InplaceUpdate(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input i, ::tensorflow::Input v); + operator ::tensorflow::Output() const { return y; } + operator ::tensorflow::Input() const { return y; } + ::tensorflow::Node* node() const { return y.node(); } + + Operation operation; + ::tensorflow::Output y; +}; + +/// Computes the inverse permutation of a tensor. +/// +/// This operation computes the inverse of an index permutation. It takes a 1-D +/// integer tensor `x`, which represents the indices of a zero-based array, and +/// swaps each value with its index position. In other words, for an output tensor +/// `y` and an input tensor `x`, this operation computes the following: +/// +/// `y[x[i]] = i for i in [0, 1, ..., len(x) - 1]` +/// +/// The values must include 0. There can be no duplicate values or negative values. +/// +/// For example: +/// +/// ``` +/// # tensor `x` is [3, 4, 0, 2, 1] +/// invert_permutation(x) ==> [2, 4, 3, 0, 1] +/// ``` +/// +/// Arguments: +/// * scope: A Scope object +/// * x: 1-D. +/// +/// Returns: +/// * `Output`: 1-D. +class InvertPermutation { + public: + InvertPermutation(const ::tensorflow::Scope& scope, ::tensorflow::Input x); + operator ::tensorflow::Output() const { return y; } + operator ::tensorflow::Input() const { return y; } + ::tensorflow::Node* node() const { return y.node(); } + + Operation operation; + ::tensorflow::Output y; +}; + +/// Computes the difference between two lists of numbers or strings. +/// +/// Given a list `x` and a list `y`, this operation returns a list `out` that +/// represents all values that are in `x` but not in `y`. The returned list `out` +/// is sorted in the same order that the numbers appear in `x` (duplicates are +/// preserved). This operation also returns a list `idx` that represents the +/// position of each `out` element in `x`. In other words: +/// +/// `out[i] = x[idx[i]] for i in [0, 1, ..., len(out) - 1]` +/// +/// For example, given this input: +/// +/// ``` +/// x = [1, 2, 3, 4, 5, 6] +/// y = [1, 3, 5] +/// ``` +/// +/// This operation would return: +/// +/// ``` +/// out ==> [2, 4, 6] +/// idx ==> [1, 3, 5] +/// ``` +/// +/// Arguments: +/// * scope: A Scope object +/// * x: 1-D. Values to keep. +/// * y: 1-D. Values to remove. +/// +/// Returns: +/// * `Output` out: 1-D. Values present in `x` but not in `y`. +/// * `Output` idx: 1-D. Positions of `x` values preserved in `out`. +class SetDiff1D { + public: + /// Optional attribute setters for SetDiff1D + struct Attrs { + /// Defaults to DT_INT32 + TF_MUST_USE_RESULT Attrs OutIdx(DataType x) { + Attrs ret = *this; + ret.out_idx_ = x; + return ret; + } + + DataType out_idx_ = DT_INT32; + }; + SetDiff1D(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input y); + SetDiff1D(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input y, const SetDiff1D::Attrs& attrs); + + static Attrs OutIdx(DataType x) { + return Attrs().OutIdx(x); + } + + Operation operation; + ::tensorflow::Output out; + ::tensorflow::Output idx; +}; + +/// Copy a tensor setting everything outside a central band in each innermost matrix +/// +/// to zero. +/// +/// The `band` part is computed as follows: +/// Assume `input` has `k` dimensions `[I, J, K, ..., M, N]`, then the output is a +/// tensor with the same shape where +/// +/// `band[i, j, k, ..., m, n] = in_band(m, n) * input[i, j, k, ..., m, n]`. +/// +/// The indicator function +/// +/// `in_band(m, n) = (num_lower < 0 || (m-n) <= num_lower)) && +/// (num_upper < 0 || (n-m) <= num_upper)`. +/// +/// For example: +/// +/// ``` +/// # if 'input' is [[ 0, 1, 2, 3] +/// [-1, 0, 1, 2] +/// [-2, -1, 0, 1] +/// [-3, -2, -1, 0]], +/// +/// tf.matrix_band_part(input, 1, -1) ==> [[ 0, 1, 2, 3] +/// [-1, 0, 1, 2] +/// [ 0, -1, 0, 1] +/// [ 0, 0, -1, 0]], +/// +/// tf.matrix_band_part(input, 2, 1) ==> [[ 0, 1, 0, 0] +/// [-1, 0, 1, 0] +/// [-2, -1, 0, 1] +/// [ 0, -2, -1, 0]] +/// ``` +/// +/// Useful special cases: +/// +/// ``` +/// tf.matrix_band_part(input, 0, -1) ==> Upper triangular part. +/// tf.matrix_band_part(input, -1, 0) ==> Lower triangular part. +/// tf.matrix_band_part(input, 0, 0) ==> Diagonal. +/// ``` +/// +/// Arguments: +/// * scope: A Scope object +/// * input: Rank `k` tensor. +/// * num_lower: 0-D tensor. Number of subdiagonals to keep. If negative, keep entire +/// lower triangle. +/// * num_upper: 0-D tensor. Number of superdiagonals to keep. If negative, keep +/// entire upper triangle. +/// +/// Returns: +/// * `Output`: Rank `k` tensor of the same shape as input. The extracted banded tensor. +class MatrixBandPart { + public: + MatrixBandPart(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input num_lower, ::tensorflow::Input num_upper); + operator ::tensorflow::Output() const { return band; } + operator ::tensorflow::Input() const { return band; } + ::tensorflow::Node* node() const { return band.node(); } + + Operation operation; + ::tensorflow::Output band; +}; + +/// Returns a batched diagonal tensor with a given batched diagonal values. +/// +/// Given a `diagonal`, this operation returns a tensor with the `diagonal` and +/// everything else padded with zeros. The diagonal is computed as follows: +/// +/// Assume `diagonal` has `k` dimensions `[I, J, K, ..., N]`, then the output is a +/// tensor of rank `k+1` with dimensions [I, J, K, ..., N, N]` where: +/// +/// `output[i, j, k, ..., m, n] = 1{m=n} * diagonal[i, j, k, ..., n]`. +/// +/// For example: +/// +/// ``` +/// # 'diagonal' is [[1, 2, 3, 4], [5, 6, 7, 8]] +/// +/// and diagonal.shape = (2, 4) +/// +/// tf.matrix_diag(diagonal) ==> [[[1, 0, 0, 0] +/// [0, 2, 0, 0] +/// [0, 0, 3, 0] +/// [0, 0, 0, 4]], +/// [[5, 0, 0, 0] +/// [0, 6, 0, 0] +/// [0, 0, 7, 0] +/// [0, 0, 0, 8]]] +/// +/// which has shape (2, 4, 4) +/// ``` +/// +/// Arguments: +/// * scope: A Scope object +/// * diagonal: Rank `k`, where `k >= 1`. +/// +/// Returns: +/// * `Output`: Rank `k+1`, with `output.shape = diagonal.shape + [diagonal.shape[-1]]`. +class MatrixDiag { + public: + MatrixDiag(const ::tensorflow::Scope& scope, ::tensorflow::Input diagonal); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Returns the batched diagonal part of a batched tensor. +/// +/// This operation returns a tensor with the `diagonal` part +/// of the batched `input`. The `diagonal` part is computed as follows: +/// +/// Assume `input` has `k` dimensions `[I, J, K, ..., M, N]`, then the output is a +/// tensor of rank `k - 1` with dimensions `[I, J, K, ..., min(M, N)]` where: +/// +/// `diagonal[i, j, k, ..., n] = input[i, j, k, ..., n, n]`. +/// +/// The input must be at least a matrix. +/// +/// For example: +/// +/// ``` +/// # 'input' is [[[1, 0, 0, 0] +/// [0, 2, 0, 0] +/// [0, 0, 3, 0] +/// [0, 0, 0, 4]], +/// [[5, 0, 0, 0] +/// [0, 6, 0, 0] +/// [0, 0, 7, 0] +/// [0, 0, 0, 8]]] +/// +/// and input.shape = (2, 4, 4) +/// +/// tf.matrix_diag_part(input) ==> [[1, 2, 3, 4], [5, 6, 7, 8]] +/// +/// which has shape (2, 4) +/// ``` +/// +/// Arguments: +/// * scope: A Scope object +/// * input: Rank `k` tensor where `k >= 2`. +/// +/// Returns: +/// * `Output`: The extracted diagonal(s) having shape +/// `diagonal.shape = input.shape[:-2] + [min(input.shape[-2:])]`. +class MatrixDiagPart { + public: + MatrixDiagPart(const ::tensorflow::Scope& scope, ::tensorflow::Input input); + operator ::tensorflow::Output() const { return diagonal; } + operator ::tensorflow::Input() const { return diagonal; } + ::tensorflow::Node* node() const { return diagonal.node(); } + + Operation operation; + ::tensorflow::Output diagonal; +}; + +/// Returns a batched matrix tensor with new batched diagonal values. +/// +/// Given `input` and `diagonal`, this operation returns a tensor with the +/// same shape and values as `input`, except for the main diagonal of the +/// innermost matrices. These will be overwritten by the values in `diagonal`. +/// +/// The output is computed as follows: +/// +/// Assume `input` has `k+1` dimensions `[I, J, K, ..., M, N]` and `diagonal` has +/// `k` dimensions `[I, J, K, ..., min(M, N)]`. Then the output is a +/// tensor of rank `k+1` with dimensions `[I, J, K, ..., M, N]` where: +/// +/// * `output[i, j, k, ..., m, n] = diagonal[i, j, k, ..., n]` for `m == n`. +/// * `output[i, j, k, ..., m, n] = input[i, j, k, ..., m, n]` for `m != n`. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: Rank `k+1`, where `k >= 1`. +/// * diagonal: Rank `k`, where `k >= 1`. +/// +/// Returns: +/// * `Output`: Rank `k+1`, with `output.shape = input.shape`. +class MatrixSetDiag { + public: + MatrixSetDiag(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input diagonal); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Pads a tensor with mirrored values. +/// +/// This operation pads a `input` with mirrored values according to the `paddings` +/// you specify. `paddings` is an integer tensor with shape `[n, 2]`, where n is +/// the rank of `input`. For each dimension D of `input`, `paddings[D, 0]` indicates +/// how many values to add before the contents of `input` in that dimension, and +/// `paddings[D, 1]` indicates how many values to add after the contents of `input` +/// in that dimension. Both `paddings[D, 0]` and `paddings[D, 1]` must be no greater +/// than `input.dim_size(D)` (or `input.dim_size(D) - 1`) if `copy_border` is true +/// (if false, respectively). +/// +/// The padded size of each dimension D of the output is: +/// +/// `paddings(D, 0) + input.dim_size(D) + paddings(D, 1)` +/// +/// For example: +/// +/// ``` +/// # 't' is [[1, 2, 3], [4, 5, 6]]. +/// # 'paddings' is [[1, 1]], [2, 2]]. +/// # 'mode' is SYMMETRIC. +/// # rank of 't' is 2. +/// pad(t, paddings) ==> [[2, 1, 1, 2, 3, 3, 2] +/// [2, 1, 1, 2, 3, 3, 2] +/// [5, 4, 4, 5, 6, 6, 5] +/// [5, 4, 4, 5, 6, 6, 5]] +/// ``` +/// +/// Arguments: +/// * scope: A Scope object +/// * input: The input tensor to be padded. +/// * paddings: A two-column matrix specifying the padding sizes. The number of +/// rows must be the same as the rank of `input`. +/// * mode: Either `REFLECT` or `SYMMETRIC`. In reflect mode the padded regions +/// do not include the borders, while in symmetric mode the padded regions +/// do include the borders. For example, if `input` is `[1, 2, 3]` and `paddings` +/// is `[0, 2]`, then the output is `[1, 2, 3, 2, 1]` in reflect mode, and +/// it is `[1, 2, 3, 3, 2]` in symmetric mode. +/// +/// Returns: +/// * `Output`: The padded tensor. +class MirrorPad { + public: + MirrorPad(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input paddings, StringPiece mode); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Returns a one-hot tensor. +/// +/// The locations represented by indices in `indices` take value `on_value`, +/// while all other locations take value `off_value`. +/// +/// If the input `indices` is rank `N`, the output will have rank `N+1`, +/// The new axis is created at dimension `axis` (default: the new axis is +/// appended at the end). +/// +/// If `indices` is a scalar the output shape will be a vector of length `depth`. +/// +/// If `indices` is a vector of length `features`, the output shape will be: +/// ``` +/// features x depth if axis == -1 +/// depth x features if axis == 0 +/// ``` +/// +/// If `indices` is a matrix (batch) with shape `[batch, features]`, +/// the output shape will be: +/// ``` +/// batch x features x depth if axis == -1 +/// batch x depth x features if axis == 1 +/// depth x batch x features if axis == 0 +/// ``` +/// +/// +/// Examples +/// ========= +/// +/// Suppose that +/// +/// ``` +/// indices = [0, 2, -1, 1] +/// depth = 3 +/// on_value = 5.0 +/// off_value = 0.0 +/// axis = -1 +/// ``` +/// +/// Then output is `[4 x 3]`: +/// +/// ```output = +/// [5.0 0.0 0.0] // one_hot(0) +/// [0.0 0.0 5.0] // one_hot(2) +/// [0.0 0.0 0.0] // one_hot(-1) +/// [0.0 5.0 0.0] // one_hot(1) +/// ``` +/// +/// Suppose that +/// +/// ``` +/// indices = [0, 2, -1, 1] +/// depth = 3 +/// on_value = 0.0 +/// off_value = 3.0 +/// axis = 0 +/// ``` +/// +/// Then output is `[3 x 4]`: +/// +/// ```output = +/// [0.0 3.0 3.0 3.0] +/// [3.0 3.0 3.0 0.0] +/// [3.0 3.0 3.0 3.0] +/// [3.0 0.0 3.0 3.0] +/// // ^ one_hot(0) +/// // ^ one_hot(2) +/// // ^ one_hot(-1) +/// // ^ one_hot(1) +/// ``` +/// Suppose that +/// +/// ``` +/// indices = [[0, 2], [1, -1]] +/// depth = 3 +/// on_value = 1.0 +/// off_value = 0.0 +/// axis = -1 +/// ``` +/// +/// Then output is `[2 x 2 x 3]`: +/// +/// ```output = +/// [ +/// [1.0, 0.0, 0.0] // one_hot(0) +/// [0.0, 0.0, 1.0] // one_hot(2) +/// ][ +/// [0.0, 1.0, 0.0] // one_hot(1) +/// [0.0, 0.0, 0.0] // one_hot(-1) +/// ]``` +/// +/// Arguments: +/// * scope: A Scope object +/// * indices: A tensor of indices. +/// * depth: A scalar defining the depth of the one hot dimension. +/// * on_value: A scalar defining the value to fill in output when `indices[j] = i`. +/// * off_value: A scalar defining the value to fill in output when `indices[j] != i`. +/// +/// Optional attributes (see `Attrs`): +/// * axis: The axis to fill (default: -1, a new inner-most axis). +/// +/// Returns: +/// * `Output`: The one-hot tensor. +class OneHot { + public: + /// Optional attribute setters for OneHot + struct Attrs { + /// The axis to fill (default: -1, a new inner-most axis). + /// + /// Defaults to -1 + TF_MUST_USE_RESULT Attrs Axis(int64 x) { + Attrs ret = *this; + ret.axis_ = x; + return ret; + } + + int64 axis_ = -1; + }; + OneHot(const ::tensorflow::Scope& scope, ::tensorflow::Input indices, + ::tensorflow::Input depth, ::tensorflow::Input on_value, + ::tensorflow::Input off_value); + OneHot(const ::tensorflow::Scope& scope, ::tensorflow::Input indices, + ::tensorflow::Input depth, ::tensorflow::Input on_value, + ::tensorflow::Input off_value, const OneHot::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs Axis(int64 x) { + return Attrs().Axis(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Returns a tensor of ones with the same shape and type as x. +/// +/// Arguments: +/// * scope: A Scope object +/// * x: a tensor of type T. +/// +/// Returns: +/// * `Output`: a tensor of the same shape and type as x but filled with ones. +class OnesLike { + public: + OnesLike(const ::tensorflow::Scope& scope, ::tensorflow::Input x); + operator ::tensorflow::Output() const { return y; } + operator ::tensorflow::Input() const { return y; } + ::tensorflow::Node* node() const { return y.node(); } + + Operation operation; + ::tensorflow::Output y; +}; + +/// Packs a list of `N` rank-`R` tensors into one rank-`(R+1)` tensor. +/// +/// Packs the `N` tensors in `values` into a tensor with rank one higher than each +/// tensor in `values`, by packing them along the `axis` dimension. +/// Given a list of tensors of shape `(A, B, C)`; +/// +/// if `axis == 0` then the `output` tensor will have the shape `(N, A, B, C)`. +/// if `axis == 1` then the `output` tensor will have the shape `(A, N, B, C)`. +/// Etc. +/// +/// For example: +/// +/// ``` +/// # 'x' is [1, 4] +/// # 'y' is [2, 5] +/// # 'z' is [3, 6] +/// pack([x, y, z]) => [[1, 4], [2, 5], [3, 6]] # Pack along first dim. +/// pack([x, y, z], axis=1) => [[1, 2, 3], [4, 5, 6]] +/// ``` +/// +/// This is the opposite of `unpack`. +/// +/// Arguments: +/// * scope: A Scope object +/// * values: Must be of same shape and type. +/// +/// Optional attributes (see `Attrs`): +/// * axis: Dimension along which to pack. Negative values wrap around, so the +/// valid range is `[-(R+1), R+1)`. +/// +/// Returns: +/// * `Output`: The packed tensor. +class Stack { + public: + /// Optional attribute setters for Stack + struct Attrs { + /// Dimension along which to pack. Negative values wrap around, so the + /// valid range is `[-(R+1), R+1)`. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Axis(int64 x) { + Attrs ret = *this; + ret.axis_ = x; + return ret; + } + + int64 axis_ = 0; + }; + Stack(const ::tensorflow::Scope& scope, ::tensorflow::InputList values); + Stack(const ::tensorflow::Scope& scope, ::tensorflow::InputList values, const + Stack::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs Axis(int64 x) { + return Attrs().Axis(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Pads a tensor with zeros. +/// +/// This operation pads a `input` with zeros according to the `paddings` you +/// specify. `paddings` is an integer tensor with shape `[Dn, 2]`, where n is the +/// rank of `input`. For each dimension D of `input`, `paddings[D, 0]` indicates +/// how many zeros to add before the contents of `input` in that dimension, and +/// `paddings[D, 1]` indicates how many zeros to add after the contents of `input` +/// in that dimension. +/// +/// The padded size of each dimension D of the output is: +/// +/// `paddings(D, 0) + input.dim_size(D) + paddings(D, 1)` +/// +/// For example: +/// +/// ``` +/// # 't' is [[1, 1], [2, 2]] +/// # 'paddings' is [[1, 1], [2, 2]] +/// # rank of 't' is 2 +/// pad(t, paddings) ==> [[0, 0, 0, 0, 0, 0] +/// [0, 0, 1, 1, 0, 0] +/// [0, 0, 2, 2, 0, 0] +/// [0, 0, 0, 0, 0, 0]] +/// ``` +/// +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The output tensor. +class Pad { + public: + Pad(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input paddings); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Pads a tensor. +/// +/// This operation pads `input` according to the `paddings` and `constant_values` +/// you specify. `paddings` is an integer tensor with shape `[Dn, 2]`, where n is +/// the rank of `input`. For each dimension D of `input`, `paddings[D, 0]` indicates +/// how many padding values to add before the contents of `input` in that dimension, +/// and `paddings[D, 1]` indicates how many padding values to add after the contents +/// of `input` in that dimension. `constant_values` is a scalar tensor of the same +/// type as `input` that indicates the value to use for padding `input`. +/// +/// The padded size of each dimension D of the output is: +/// +/// `paddings(D, 0) + input.dim_size(D) + paddings(D, 1)` +/// +/// For example: +/// +/// ``` +/// # 't' is [[1, 1], [2, 2]] +/// # 'paddings' is [[1, 1], [2, 2]] +/// # 'constant_values' is 0 +/// # rank of 't' is 2 +/// pad(t, paddings) ==> [[0, 0, 0, 0, 0, 0] +/// [0, 0, 1, 1, 0, 0] +/// [0, 0, 2, 2, 0, 0] +/// [0, 0, 0, 0, 0, 0]] +/// ``` +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The output tensor. +class PadV2 { + public: + PadV2(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input paddings, ::tensorflow::Input constant_values); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Concatenates a list of `N` tensors along the first dimension. +/// +/// The input tensors are all required to have size 1 in the first dimension. +/// +/// For example: +/// +/// ``` +/// # 'x' is [[1, 4]] +/// # 'y' is [[2, 5]] +/// # 'z' is [[3, 6]] +/// parallel_concat([x, y, z]) => [[1, 4], [2, 5], [3, 6]] # Pack along first dim. +/// ``` +/// +/// The difference between concat and parallel_concat is that concat requires all +/// of the inputs be computed before the operation will begin but doesn't require +/// that the input shapes be known during graph construction. Parallel concat +/// will copy pieces of the input into the output as they become available, in +/// some situations this can provide a performance benefit. +/// +/// Arguments: +/// * scope: A Scope object +/// * values: Tensors to be concatenated. All must have size 1 in the first dimension +/// and same shape. +/// * shape: the final shape of the result; should be equal to the shapes of any input +/// but with the number of input values in the first dimension. +/// +/// Returns: +/// * `Output`: The concatenated tensor. +class ParallelConcat { + public: + ParallelConcat(const ::tensorflow::Scope& scope, ::tensorflow::InputList + values, PartialTensorShape shape); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// A placeholder op for a value that will be fed into the computation. +/// +/// N.B. This operation will fail with an error if it is executed. It is +/// intended as a way to represent a value that will always be fed, and to +/// provide attrs that enable the fed value to be checked at runtime. +/// +/// Arguments: +/// * scope: A Scope object +/// * dtype: The type of elements in the tensor. +/// +/// Optional attributes (see `Attrs`): +/// * shape: (Optional) The shape of the tensor. If the shape has 0 dimensions, the +/// shape is unconstrained. +/// +/// Returns: +/// * `Output`: A placeholder tensor that must be replaced using the feed mechanism. +class Placeholder { + public: + /// Optional attribute setters for Placeholder + struct Attrs { + /// (Optional) The shape of the tensor. If the shape has 0 dimensions, the + /// shape is unconstrained. + /// + /// Defaults to + TF_MUST_USE_RESULT Attrs Shape(PartialTensorShape x) { + Attrs ret = *this; + ret.shape_ = x; + return ret; + } + + PartialTensorShape shape_ = ::tensorflow::PartialTensorShape() /* unknown */; + }; + Placeholder(const ::tensorflow::Scope& scope, DataType dtype); + Placeholder(const ::tensorflow::Scope& scope, DataType dtype, const + Placeholder::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs Shape(PartialTensorShape x) { + return Attrs().Shape(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// A placeholder op that passes through `input` when its output is not fed. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: The default value to produce when `output` is not fed. +/// * shape: The (possibly partial) shape of the tensor. +/// +/// Returns: +/// * `Output`: A placeholder tensor that defaults to `input` if it is not fed. +class PlaceholderWithDefault { + public: + PlaceholderWithDefault(const ::tensorflow::Scope& scope, ::tensorflow::Input + input, PartialTensorShape shape); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// An identity op that triggers an error if a gradient is requested. +/// +/// When executed in a graph, this op outputs its input tensor as-is. +/// +/// When building ops to compute gradients, the TensorFlow gradient system +/// will return an error when trying to lookup the gradient of this op, +/// because no gradient must ever be registered for this function. This +/// op exists to prevent subtle bugs from silently returning unimplemented +/// gradients in some corner cases. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: any tensor. +/// +/// Optional attributes (see `Attrs`): +/// * message: Will be printed in the error when anyone tries to differentiate +/// this operation. +/// +/// Returns: +/// * `Output`: the same input tensor. +class PreventGradient { + public: + /// Optional attribute setters for PreventGradient + struct Attrs { + /// Will be printed in the error when anyone tries to differentiate + /// this operation. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs Message(StringPiece x) { + Attrs ret = *this; + ret.message_ = x; + return ret; + } + + StringPiece message_ = ""; + }; + PreventGradient(const ::tensorflow::Scope& scope, ::tensorflow::Input input); + PreventGradient(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + const PreventGradient::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs Message(StringPiece x) { + return Attrs().Message(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Quantizes then dequantizes a tensor. +/// +/// This op simulates the precision loss from the quantized forward pass by: +/// +/// 1. Quantizing the tensor to fixed point numbers, which should match the target +/// quantization method when it is used in inference. +/// 2. Dequantizing it back to floating point numbers for the following ops, most +/// likely matmul. +/// +/// There are different ways to quantize. This version uses only scaling, so 0.0 +/// maps to 0. +/// +/// From the specified 'num_bits' in the quantized output type, it determines +/// minimum and maximum representable quantized values. +/// +/// e.g. +/// +/// * [-128, 127] for signed, num_bits = 8, or +/// * [0, 255] for unsigned, num_bits = 8. +/// +/// If range_given == False, the initial input_min, input_max will be determined +/// automatically as the minimum and maximum values in the input tensor, otherwise +/// the specified values of input_min, input_max are used. +/// +/// Note: If the input_min, input_max are specified, they do not need to equal the +/// actual minimum and maximum values in the tensor. e.g. in some cases it may be +/// beneficial to specify these values such that the low probability extremes of the +/// input distribution are clipped. +/// +/// This op determines the maximum scale_factor that would map the initial +/// [input_min, input_max] range to a range that lies within the representable +/// quantized range. +/// +/// It determines the scale from one of input_min and input_max, then updates the +/// other one to maximize the respresentable range. +/// +/// e.g. +/// +/// * if the output is signed, num_bits = 8, [input_min, input_max] = [-10.0, +/// 5.0]: it would use a scale_factor of -128 / -10.0 = 12.8 In this case, it +/// would update input_max to be 127 / 12.8 = 9.921875 +/// * if the output is signed, num_bits = 8, [input_min, input_max] = [-10.0, +/// 10.0]: it would use a scale_factor of 127 / 10.0 = 12.7 In this case, it +/// would update input_min to be 128.0 / 12.7 = -10.07874 +/// * if the output is unsigned, input_min is forced to be 0, and only the +/// specified input_max is used. +/// +/// After determining the scale_factor and updating the input range, it applies the +/// following to each value in the 'input' tensor. +/// +/// output = round(clamp(value, input_min, input_max) * scale_factor) / scale_factor. +/// +/// +/// Arguments: +/// * scope: A Scope object +/// * input: Tensor to quantize and then dequantize. +/// * input_min: If `range_given == True`, this specifies the minimum input value that needs to +/// be represented, otherwise it is determined from the min value of the `input` +/// tensor. +/// * input_max: If `range_given == True`, this specifies the maximum input value that needs to +/// be represented, otherwise it is determined from the max value of the `input` +/// tensor. +/// +/// Optional attributes (see `Attrs`): +/// * signed_input: Whether the quantization is signed or unsigned. (actually this parameter should +/// have been called `signed_output`) +/// * num_bits: The bitwidth of the quantization. +/// * range_given: Whether the range is given or should be determined from the `input` tensor. +/// +/// Returns: +/// * `Output`: The output tensor. +class QuantizeAndDequantizeV2 { + public: + /// Optional attribute setters for QuantizeAndDequantizeV2 + struct Attrs { + /// Whether the quantization is signed or unsigned. (actually this parameter should + /// have been called `signed_output`) + /// + /// Defaults to true + TF_MUST_USE_RESULT Attrs SignedInput(bool x) { + Attrs ret = *this; + ret.signed_input_ = x; + return ret; + } + + /// The bitwidth of the quantization. + /// + /// Defaults to 8 + TF_MUST_USE_RESULT Attrs NumBits(int64 x) { + Attrs ret = *this; + ret.num_bits_ = x; + return ret; + } + + /// Whether the range is given or should be determined from the `input` tensor. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs RangeGiven(bool x) { + Attrs ret = *this; + ret.range_given_ = x; + return ret; + } + + bool signed_input_ = true; + int64 num_bits_ = 8; + bool range_given_ = false; + }; + QuantizeAndDequantizeV2(const ::tensorflow::Scope& scope, ::tensorflow::Input + input, ::tensorflow::Input input_min, + ::tensorflow::Input input_max); + QuantizeAndDequantizeV2(const ::tensorflow::Scope& scope, ::tensorflow::Input + input, ::tensorflow::Input input_min, + ::tensorflow::Input input_max, const + QuantizeAndDequantizeV2::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs SignedInput(bool x) { + return Attrs().SignedInput(x); + } + static Attrs NumBits(int64 x) { + return Attrs().NumBits(x); + } + static Attrs RangeGiven(bool x) { + return Attrs().RangeGiven(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Quantizes then dequantizes a tensor. +/// +/// This is almost identical to QuantizeAndDequantizeV2, except that num_bits is a +/// tensor, so its value can change during training. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The output tensor. +class QuantizeAndDequantizeV3 { + public: + /// Optional attribute setters for QuantizeAndDequantizeV3 + struct Attrs { + /// Defaults to true + TF_MUST_USE_RESULT Attrs SignedInput(bool x) { + Attrs ret = *this; + ret.signed_input_ = x; + return ret; + } + + /// Defaults to true + TF_MUST_USE_RESULT Attrs RangeGiven(bool x) { + Attrs ret = *this; + ret.range_given_ = x; + return ret; + } + + bool signed_input_ = true; + bool range_given_ = true; + }; + QuantizeAndDequantizeV3(const ::tensorflow::Scope& scope, ::tensorflow::Input + input, ::tensorflow::Input input_min, + ::tensorflow::Input input_max, ::tensorflow::Input + num_bits); + QuantizeAndDequantizeV3(const ::tensorflow::Scope& scope, ::tensorflow::Input + input, ::tensorflow::Input input_min, + ::tensorflow::Input input_max, ::tensorflow::Input + num_bits, const QuantizeAndDequantizeV3::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs SignedInput(bool x) { + return Attrs().SignedInput(x); + } + static Attrs RangeGiven(bool x) { + return Attrs().RangeGiven(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Quantize the 'input' tensor of type float to 'output' tensor of type 'T'. +/// +/// [min_range, max_range] are scalar floats that specify the range for +/// the 'input' data. The 'mode' attribute controls exactly which calculations are +/// used to convert the float values to their quantized equivalents. The +/// 'round_mode' attribute controls which rounding tie-breaking algorithm is used +/// when rounding float values to their quantized equivalents. +/// +/// In 'MIN_COMBINED' mode, each value of the tensor will undergo the following: +/// +/// ``` +/// out[i] = (in[i] - min_range) * range(T) / (max_range - min_range) +/// if T == qint8, out[i] -= (range(T) + 1) / 2.0 +/// ``` +/// +/// here `range(T) = numeric_limits::max() - numeric_limits::min()` +/// +/// *MIN_COMBINED Mode Example* +/// +/// Assume the input is type float and has a possible range of [0.0, 6.0] and the +/// output type is quint8 ([0, 255]). The min_range and max_range values should be +/// specified as 0.0 and 6.0. Quantizing from float to quint8 will multiply each +/// value of the input by 255/6 and cast to quint8. +/// +/// If the output type was qint8 ([-128, 127]), the operation will additionally +/// subtract each value by 128 prior to casting, so that the range of values aligns +/// with the range of qint8. +/// +/// If the mode is 'MIN_FIRST', then this approach is used: +/// +/// ``` +/// num_discrete_values = 1 << (# of bits in T) +/// range_adjust = num_discrete_values / (num_discrete_values - 1) +/// range = (range_max - range_min) * range_adjust +/// range_scale = num_discrete_values / range +/// quantized = round(input * range_scale) - round(range_min * range_scale) + +/// numeric_limits::min() +/// quantized = max(quantized, numeric_limits::min()) +/// quantized = min(quantized, numeric_limits::max()) +/// ``` +/// +/// The biggest difference between this and MIN_COMBINED is that the minimum range +/// is rounded first, before it's subtracted from the rounded value. With +/// MIN_COMBINED, a small bias is introduced where repeated iterations of quantizing +/// and dequantizing will introduce a larger and larger error. +/// +/// *SCALED mode Example* +/// +/// `SCALED` mode matches the quantization approach used in +/// `QuantizeAndDequantize{V2|V3}`. +/// +/// If the mode is `SCALED`, we do not use the full range of the output type, +/// choosing to elide the lowest possible value for symmetry (e.g., output range is +/// -127 to 127, not -128 to 127 for signed 8 bit quantization), so that 0.0 maps to +/// 0. +/// +/// We first find the range of values in our tensor. The +/// range we use is always centered on 0, so we find m such that +/// +/// ```c++ +/// m = max(abs(input_min), abs(input_max)) +/// ``` +/// +/// Our input tensor range is then `[-m, m]`. +/// +/// Next, we choose our fixed-point quantization buckets, `[min_fixed, max_fixed]`. +/// If T is signed, this is +/// +/// ``` +/// num_bits = sizeof(T) * 8 +/// [min_fixed, max_fixed] = +/// [-(1 << (num_bits - 1) - 1), (1 << (num_bits - 1)) - 1] +/// ``` +/// +/// Otherwise, if T is unsigned, the fixed-point range is +/// +/// ``` +/// [min_fixed, max_fixed] = [0, (1 << num_bits) - 1] +/// ``` +/// +/// From this we compute our scaling factor, s: +/// +/// ```c++ +/// s = (max_fixed - min_fixed) / (2 * m) +/// ``` +/// +/// Now we can quantize the elements of our tensor: +/// +/// ```c++ +/// result = round(input * s) +/// ``` +/// +/// One thing to watch out for is that the operator may choose to adjust the +/// requested minimum and maximum values slightly during the quantization process, +/// so you should always use the output ports as the range for further calculations. +/// For example, if the requested minimum and maximum values are close to equal, +/// they will be separated by a small epsilon value to prevent ill-formed quantized +/// buffers from being created. Otherwise, you can end up with buffers where all the +/// quantized values map to the same float value, which causes problems for +/// operations that have to perform further calculations on them. +/// +/// Arguments: +/// * scope: A Scope object +/// * min_range: The minimum scalar value possibly produced for the input. +/// * max_range: The maximum scalar value possibly produced for the input. +/// +/// Returns: +/// * `Output` output: The quantized data produced from the float input. +/// * `Output` output_min: The actual minimum scalar value used for the output. +/// * `Output` output_max: The actual maximum scalar value used for the output. +class QuantizeV2 { + public: + /// Optional attribute setters for QuantizeV2 + struct Attrs { + /// Defaults to "MIN_COMBINED" + TF_MUST_USE_RESULT Attrs Mode(StringPiece x) { + Attrs ret = *this; + ret.mode_ = x; + return ret; + } + + /// Defaults to "HALF_AWAY_FROM_ZERO" + TF_MUST_USE_RESULT Attrs RoundMode(StringPiece x) { + Attrs ret = *this; + ret.round_mode_ = x; + return ret; + } + + StringPiece mode_ = "MIN_COMBINED"; + StringPiece round_mode_ = "HALF_AWAY_FROM_ZERO"; + }; + QuantizeV2(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input min_range, ::tensorflow::Input max_range, + DataType T); + QuantizeV2(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input min_range, ::tensorflow::Input max_range, + DataType T, const QuantizeV2::Attrs& attrs); + + static Attrs Mode(StringPiece x) { + return Attrs().Mode(x); + } + static Attrs RoundMode(StringPiece x) { + return Attrs().RoundMode(x); + } + + Operation operation; + ::tensorflow::Output output; + ::tensorflow::Output output_min; + ::tensorflow::Output output_max; +}; + +/// Concatenates quantized tensors along one dimension. +/// +/// Arguments: +/// * scope: A Scope object +/// * concat_dim: 0-D. The dimension along which to concatenate. Must be in the +/// range [0, rank(values)). +/// * values: The `N` Tensors to concatenate. Their ranks and types must match, +/// and their sizes must match in all dimensions except `concat_dim`. +/// * input_mins: The minimum scalar values for each of the input tensors. +/// * input_maxes: The maximum scalar values for each of the input tensors. +/// +/// Returns: +/// * `Output` output: A `Tensor` with the concatenation of values stacked along the +/// `concat_dim` dimension. This tensor's shape matches that of `values` except +/// in `concat_dim` where it has the sum of the sizes. +/// * `Output` output_min: The float value that the minimum quantized output value represents. +/// * `Output` output_max: The float value that the maximum quantized output value represents. +class QuantizedConcat { + public: + QuantizedConcat(const ::tensorflow::Scope& scope, ::tensorflow::Input + concat_dim, ::tensorflow::InputList values, + ::tensorflow::InputList input_mins, ::tensorflow::InputList + input_maxes); + + Operation operation; + ::tensorflow::Output output; + ::tensorflow::Output output_min; + ::tensorflow::Output output_max; +}; + +/// Quantized Instance normalization. +/// +/// Arguments: +/// * scope: A Scope object +/// * x: A 4D input Tensor. +/// * x_min: The value represented by the lowest quantized input. +/// * x_max: The value represented by the highest quantized input. +/// +/// Optional attributes (see `Attrs`): +/// * output_range_given: If True, `given_y_min` and `given_y_min` +/// and `given_y_max` are used as the output range. Otherwise, +/// the implementation computes the output range. +/// * given_y_min: Output in `y_min` if `output_range_given` is True. +/// * given_y_max: Output in `y_max` if `output_range_given` is True. +/// * variance_epsilon: A small float number to avoid dividing by 0. +/// * min_separation: Minimum value of `y_max - y_min` +/// +/// Returns: +/// * `Output` y: A 4D Tensor. +/// * `Output` y_min: The value represented by the lowest quantized output. +/// * `Output` y_max: The value represented by the highest quantized output. +class QuantizedInstanceNorm { + public: + /// Optional attribute setters for QuantizedInstanceNorm + struct Attrs { + /// If True, `given_y_min` and `given_y_min` + /// and `given_y_max` are used as the output range. Otherwise, + /// the implementation computes the output range. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs OutputRangeGiven(bool x) { + Attrs ret = *this; + ret.output_range_given_ = x; + return ret; + } + + /// Output in `y_min` if `output_range_given` is True. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs GivenYMin(float x) { + Attrs ret = *this; + ret.given_y_min_ = x; + return ret; + } + + /// Output in `y_max` if `output_range_given` is True. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs GivenYMax(float x) { + Attrs ret = *this; + ret.given_y_max_ = x; + return ret; + } + + /// A small float number to avoid dividing by 0. + /// + /// Defaults to 1e-05 + TF_MUST_USE_RESULT Attrs VarianceEpsilon(float x) { + Attrs ret = *this; + ret.variance_epsilon_ = x; + return ret; + } + + /// Minimum value of `y_max - y_min` + /// + /// Defaults to 0.001 + TF_MUST_USE_RESULT Attrs MinSeparation(float x) { + Attrs ret = *this; + ret.min_separation_ = x; + return ret; + } + + bool output_range_given_ = false; + float given_y_min_ = 0.0f; + float given_y_max_ = 0.0f; + float variance_epsilon_ = 1e-05f; + float min_separation_ = 0.001f; + }; + QuantizedInstanceNorm(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input x_min, ::tensorflow::Input x_max); + QuantizedInstanceNorm(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input x_min, ::tensorflow::Input x_max, + const QuantizedInstanceNorm::Attrs& attrs); + + static Attrs OutputRangeGiven(bool x) { + return Attrs().OutputRangeGiven(x); + } + static Attrs GivenYMin(float x) { + return Attrs().GivenYMin(x); + } + static Attrs GivenYMax(float x) { + return Attrs().GivenYMax(x); + } + static Attrs VarianceEpsilon(float x) { + return Attrs().VarianceEpsilon(x); + } + static Attrs MinSeparation(float x) { + return Attrs().MinSeparation(x); + } + + Operation operation; + ::tensorflow::Output y; + ::tensorflow::Output y_min; + ::tensorflow::Output y_max; +}; + +/// Reshapes a quantized tensor as per the Reshape op. +/// +/// ``` +/// +/// Arguments: +/// * scope: A Scope object +/// * shape: Defines the shape of the output tensor. +/// * input_min: The minimum value of the input. +/// * input_max: The maximum value of the input. +/// +/// Returns: +/// * `Output` output +/// * `Output` output_min: This value is copied from input_min. +/// * `Output` output_max: This value is copied from input_max. +class QuantizedReshape { + public: + QuantizedReshape(const ::tensorflow::Scope& scope, ::tensorflow::Input tensor, + ::tensorflow::Input shape, ::tensorflow::Input input_min, + ::tensorflow::Input input_max); + + Operation operation; + ::tensorflow::Output output; + ::tensorflow::Output output_min; + ::tensorflow::Output output_max; +}; + +/// Returns the rank of a tensor. +/// +/// This operation returns an integer representing the rank of `input`. +/// +/// For example: +/// +/// ``` +/// # 't' is [[[1, 1, 1], [2, 2, 2]], [[3, 3, 3], [4, 4, 4]]] +/// # shape of tensor 't' is [2, 2, 3] +/// rank(t) ==> 3 +/// ``` +/// +/// **Note**: The rank of a tensor is not the same as the rank of a matrix. The rank +/// of a tensor is the number of indices required to uniquely select each element +/// of the tensor. Rank is also known as "order", "degree", or "ndims." +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The output tensor. +class Rank { + public: + Rank(const ::tensorflow::Scope& scope, ::tensorflow::Input input); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Reshapes a tensor. +/// +/// Given `tensor`, this operation returns a tensor that has the same values +/// as `tensor` with shape `shape`. +/// +/// If one component of `shape` is the special value -1, the size of that dimension +/// is computed so that the total size remains constant. In particular, a `shape` +/// of `[-1]` flattens into 1-D. At most one component of `shape` can be -1. +/// +/// If `shape` is 1-D or higher, then the operation returns a tensor with shape +/// `shape` filled with the values of `tensor`. In this case, the number of elements +/// implied by `shape` must be the same as the number of elements in `tensor`. +/// +/// For example: +/// +/// ``` +/// # tensor 't' is [1, 2, 3, 4, 5, 6, 7, 8, 9] +/// # tensor 't' has shape [9] +/// reshape(t, [3, 3]) ==> [[1, 2, 3], +/// [4, 5, 6], +/// [7, 8, 9]] +/// +/// # tensor 't' is [[[1, 1], [2, 2]], +/// # [[3, 3], [4, 4]]] +/// # tensor 't' has shape [2, 2, 2] +/// reshape(t, [2, 4]) ==> [[1, 1, 2, 2], +/// [3, 3, 4, 4]] +/// +/// # tensor 't' is [[[1, 1, 1], +/// # [2, 2, 2]], +/// # [[3, 3, 3], +/// # [4, 4, 4]], +/// # [[5, 5, 5], +/// # [6, 6, 6]]] +/// # tensor 't' has shape [3, 2, 3] +/// # pass '[-1]' to flatten 't' +/// reshape(t, [-1]) ==> [1, 1, 1, 2, 2, 2, 3, 3, 3, 4, 4, 4, 5, 5, 5, 6, 6, 6] +/// +/// # -1 can also be used to infer the shape +/// +/// # -1 is inferred to be 9: +/// reshape(t, [2, -1]) ==> [[1, 1, 1, 2, 2, 2, 3, 3, 3], +/// [4, 4, 4, 5, 5, 5, 6, 6, 6]] +/// # -1 is inferred to be 2: +/// reshape(t, [-1, 9]) ==> [[1, 1, 1, 2, 2, 2, 3, 3, 3], +/// [4, 4, 4, 5, 5, 5, 6, 6, 6]] +/// # -1 is inferred to be 3: +/// reshape(t, [ 2, -1, 3]) ==> [[[1, 1, 1], +/// [2, 2, 2], +/// [3, 3, 3]], +/// [[4, 4, 4], +/// [5, 5, 5], +/// [6, 6, 6]]] +/// +/// # tensor 't' is [7] +/// # shape `[]` reshapes to a scalar +/// reshape(t, []) ==> 7 +/// ``` +/// +/// Arguments: +/// * scope: A Scope object +/// * shape: Defines the shape of the output tensor. +/// +/// Returns: +/// * `Output`: The output tensor. +class Reshape { + public: + Reshape(const ::tensorflow::Scope& scope, ::tensorflow::Input tensor, + ::tensorflow::Input shape); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Assign `value` to the sliced l-value reference of `ref`. +/// +/// The values of `value` are assigned to the positions in the variable +/// `ref` that are selected by the slice parameters. The slice parameters +/// `begin, `end`, `strides`, etc. work exactly as in `StridedSlice`. +/// +/// NOTE this op currently does not support broadcasting and so `value`'s +/// shape must be exactly the shape produced by the slice of `ref`. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * the created `Operation` +class ResourceStridedSliceAssign { + public: + /// Optional attribute setters for ResourceStridedSliceAssign + struct Attrs { + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs BeginMask(int64 x) { + Attrs ret = *this; + ret.begin_mask_ = x; + return ret; + } + + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs EndMask(int64 x) { + Attrs ret = *this; + ret.end_mask_ = x; + return ret; + } + + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs EllipsisMask(int64 x) { + Attrs ret = *this; + ret.ellipsis_mask_ = x; + return ret; + } + + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs NewAxisMask(int64 x) { + Attrs ret = *this; + ret.new_axis_mask_ = x; + return ret; + } + + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs ShrinkAxisMask(int64 x) { + Attrs ret = *this; + ret.shrink_axis_mask_ = x; + return ret; + } + + int64 begin_mask_ = 0; + int64 end_mask_ = 0; + int64 ellipsis_mask_ = 0; + int64 new_axis_mask_ = 0; + int64 shrink_axis_mask_ = 0; + }; + ResourceStridedSliceAssign(const ::tensorflow::Scope& scope, + ::tensorflow::Input ref, ::tensorflow::Input begin, + ::tensorflow::Input end, ::tensorflow::Input + strides, ::tensorflow::Input value); + ResourceStridedSliceAssign(const ::tensorflow::Scope& scope, + ::tensorflow::Input ref, ::tensorflow::Input begin, + ::tensorflow::Input end, ::tensorflow::Input + strides, ::tensorflow::Input value, const + ResourceStridedSliceAssign::Attrs& attrs); + operator ::tensorflow::Operation() const { return operation; } + + static Attrs BeginMask(int64 x) { + return Attrs().BeginMask(x); + } + static Attrs EndMask(int64 x) { + return Attrs().EndMask(x); + } + static Attrs EllipsisMask(int64 x) { + return Attrs().EllipsisMask(x); + } + static Attrs NewAxisMask(int64 x) { + return Attrs().NewAxisMask(x); + } + static Attrs ShrinkAxisMask(int64 x) { + return Attrs().ShrinkAxisMask(x); + } + + Operation operation; +}; + +/// Reverses variable length slices. +/// +/// This op first slices `input` along the dimension `batch_dim`, and for each +/// slice `i`, reverses the first `seq_lengths[i]` elements along +/// the dimension `seq_dim`. +/// +/// The elements of `seq_lengths` must obey `seq_lengths[i] <= input.dims[seq_dim]`, +/// and `seq_lengths` must be a vector of length `input.dims[batch_dim]`. +/// +/// The output slice `i` along dimension `batch_dim` is then given by input +/// slice `i`, with the first `seq_lengths[i]` slices along dimension +/// `seq_dim` reversed. +/// +/// For example: +/// +/// ``` +/// # Given this: +/// batch_dim = 0 +/// seq_dim = 1 +/// input.dims = (4, 8, ...) +/// seq_lengths = [7, 2, 3, 5] +/// +/// # then slices of input are reversed on seq_dim, but only up to seq_lengths: +/// output[0, 0:7, :, ...] = input[0, 7:0:-1, :, ...] +/// output[1, 0:2, :, ...] = input[1, 2:0:-1, :, ...] +/// output[2, 0:3, :, ...] = input[2, 3:0:-1, :, ...] +/// output[3, 0:5, :, ...] = input[3, 5:0:-1, :, ...] +/// +/// # while entries past seq_lens are copied through: +/// output[0, 7:, :, ...] = input[0, 7:, :, ...] +/// output[1, 2:, :, ...] = input[1, 2:, :, ...] +/// output[2, 3:, :, ...] = input[2, 3:, :, ...] +/// output[3, 2:, :, ...] = input[3, 2:, :, ...] +/// ``` +/// +/// In contrast, if: +/// +/// ``` +/// # Given this: +/// batch_dim = 2 +/// seq_dim = 0 +/// input.dims = (8, ?, 4, ...) +/// seq_lengths = [7, 2, 3, 5] +/// +/// # then slices of input are reversed on seq_dim, but only up to seq_lengths: +/// output[0:7, :, 0, :, ...] = input[7:0:-1, :, 0, :, ...] +/// output[0:2, :, 1, :, ...] = input[2:0:-1, :, 1, :, ...] +/// output[0:3, :, 2, :, ...] = input[3:0:-1, :, 2, :, ...] +/// output[0:5, :, 3, :, ...] = input[5:0:-1, :, 3, :, ...] +/// +/// # while entries past seq_lens are copied through: +/// output[7:, :, 0, :, ...] = input[7:, :, 0, :, ...] +/// output[2:, :, 1, :, ...] = input[2:, :, 1, :, ...] +/// output[3:, :, 2, :, ...] = input[3:, :, 2, :, ...] +/// output[2:, :, 3, :, ...] = input[2:, :, 3, :, ...] +/// ``` +/// +/// Arguments: +/// * scope: A Scope object +/// * input: The input to reverse. +/// * seq_lengths: 1-D with length `input.dims(batch_dim)` and +/// `max(seq_lengths) <= input.dims(seq_dim)` +/// * seq_dim: The dimension which is partially reversed. +/// +/// Optional attributes (see `Attrs`): +/// * batch_dim: The dimension along which reversal is performed. +/// +/// Returns: +/// * `Output`: The partially reversed input. It has the same shape as `input`. +class ReverseSequence { + public: + /// Optional attribute setters for ReverseSequence + struct Attrs { + /// The dimension along which reversal is performed. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs BatchDim(int64 x) { + Attrs ret = *this; + ret.batch_dim_ = x; + return ret; + } + + int64 batch_dim_ = 0; + }; + ReverseSequence(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input seq_lengths, int64 seq_dim); + ReverseSequence(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input seq_lengths, int64 seq_dim, const + ReverseSequence::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs BatchDim(int64 x) { + return Attrs().BatchDim(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Reverses specific dimensions of a tensor. +/// +/// NOTE `tf.reverse` has now changed behavior in preparation for 1.0. +/// `tf.reverse_v2` is currently an alias that will be deprecated before TF 1.0. +/// +/// Given a `tensor`, and a `int32` tensor `axis` representing the set of +/// dimensions of `tensor` to reverse. This operation reverses each dimension +/// `i` for which there exists `j` s.t. `axis[j] == i`. +/// +/// `tensor` can have up to 8 dimensions. The number of dimensions specified +/// in `axis` may be 0 or more entries. If an index is specified more than +/// once, a InvalidArgument error is raised. +/// +/// For example: +/// +/// ``` +/// # tensor 't' is [[[[ 0, 1, 2, 3], +/// # [ 4, 5, 6, 7], +/// # [ 8, 9, 10, 11]], +/// # [[12, 13, 14, 15], +/// # [16, 17, 18, 19], +/// # [20, 21, 22, 23]]]] +/// # tensor 't' shape is [1, 2, 3, 4] +/// +/// # 'dims' is [3] or 'dims' is [-1] +/// reverse(t, dims) ==> [[[[ 3, 2, 1, 0], +/// [ 7, 6, 5, 4], +/// [ 11, 10, 9, 8]], +/// [[15, 14, 13, 12], +/// [19, 18, 17, 16], +/// [23, 22, 21, 20]]]] +/// +/// # 'dims' is '[1]' (or 'dims' is '[-3]') +/// reverse(t, dims) ==> [[[[12, 13, 14, 15], +/// [16, 17, 18, 19], +/// [20, 21, 22, 23] +/// [[ 0, 1, 2, 3], +/// [ 4, 5, 6, 7], +/// [ 8, 9, 10, 11]]]] +/// +/// # 'dims' is '[2]' (or 'dims' is '[-2]') +/// reverse(t, dims) ==> [[[[8, 9, 10, 11], +/// [4, 5, 6, 7], +/// [0, 1, 2, 3]] +/// [[20, 21, 22, 23], +/// [16, 17, 18, 19], +/// [12, 13, 14, 15]]]] +/// ``` +/// +/// Arguments: +/// * scope: A Scope object +/// * tensor: Up to 8-D. +/// * axis: 1-D. The indices of the dimensions to reverse. Must be in the range +/// `[-rank(tensor), rank(tensor))`. +/// +/// Returns: +/// * `Output`: The same shape as `tensor`. +class Reverse { + public: + Reverse(const ::tensorflow::Scope& scope, ::tensorflow::Input tensor, + ::tensorflow::Input axis); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Scatter `updates` into a new tensor according to `indices`. +/// +/// Creates a new tensor by applying sparse `updates` to individual values or +/// slices within a tensor (initially zero for numeric, empty for string) of +/// the given `shape` according to indices. This operator is the inverse of the +/// `tf.gather_nd` operator which extracts values or slices from a given tensor. +/// +/// If `indices` contains duplicates, then their updates are accumulated (summed). +/// +/// **WARNING**: The order in which updates are applied is nondeterministic, so the +/// output will be nondeterministic if `indices` contains duplicates -- because +/// of some numerical approximation issues, numbers summed in different order +/// may yield different results. +/// +/// `indices` is an integer tensor containing indices into a new tensor of shape +/// `shape`. The last dimension of `indices` can be at most the rank of `shape`: +/// +/// indices.shape[-1] <= shape.rank +/// +/// The last dimension of `indices` corresponds to indices into elements +/// (if `indices.shape[-1] = shape.rank`) or slices +/// (if `indices.shape[-1] < shape.rank`) along dimension `indices.shape[-1]` of +/// `shape`. `updates` is a tensor with shape +/// +/// indices.shape[:-1] + shape[indices.shape[-1]:] +/// +/// The simplest form of scatter is to insert individual elements in a tensor by +/// index. For example, say we want to insert 4 scattered elements in a rank-1 +/// tensor with 8 elements. +/// +///
+/// +///
+/// +/// In Python, this scatter operation would look like this: +/// +/// ```python +/// indices = tf.constant([[4], [3], [1], [7]]) +/// updates = tf.constant([9, 10, 11, 12]) +/// shape = tf.constant([8]) +/// scatter = tf.scatter_nd(indices, updates, shape) +/// with tf.Session() as sess: +/// print(sess.run(scatter)) +/// ``` +/// +/// The resulting tensor would look like this: +/// +/// [0, 11, 0, 10, 9, 0, 0, 12] +/// +/// We can also, insert entire slices of a higher rank tensor all at once. For +/// example, if we wanted to insert two slices in the first dimension of a +/// rank-3 tensor with two matrices of new values. +/// +///
+/// +///
+/// +/// In Python, this scatter operation would look like this: +/// +/// ```python +/// indices = tf.constant([[0], [2]]) +/// updates = tf.constant([[[5, 5, 5, 5], [6, 6, 6, 6], +/// [7, 7, 7, 7], [8, 8, 8, 8]], +/// [[5, 5, 5, 5], [6, 6, 6, 6], +/// [7, 7, 7, 7], [8, 8, 8, 8]]]) +/// shape = tf.constant([4, 4, 4]) +/// scatter = tf.scatter_nd(indices, updates, shape) +/// with tf.Session() as sess: +/// print(sess.run(scatter)) +/// ``` +/// +/// The resulting tensor would look like this: +/// +/// [[[5, 5, 5, 5], [6, 6, 6, 6], [7, 7, 7, 7], [8, 8, 8, 8]], +/// [[0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0]], +/// [[5, 5, 5, 5], [6, 6, 6, 6], [7, 7, 7, 7], [8, 8, 8, 8]], +/// [[0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0]]] +/// +/// Note that on CPU, if an out of bound index is found, an error is returned. +/// On GPU, if an out of bound index is found, the index is ignored. +/// +/// Arguments: +/// * scope: A Scope object +/// * indices: Index tensor. +/// * updates: Updates to scatter into output. +/// * shape: 1-D. The shape of the resulting tensor. +/// +/// Returns: +/// * `Output`: A new tensor with the given shape and updates applied according +/// to the indices. +class ScatterNd { + public: + ScatterNd(const ::tensorflow::Scope& scope, ::tensorflow::Input indices, + ::tensorflow::Input updates, ::tensorflow::Input shape); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Applies sparse addition to `input` using individual values or slices +/// +/// from `updates` according to indices `indices`. The updates are non-aliasing: +/// `input` is only modified in-place if no other operations will use it. +/// Otherwise, a copy of `input` is made. This operation has a gradient with +/// respect to both `input` and `updates`. +/// +/// `input` is a `Tensor` with rank `P` and `indices` is a `Tensor` of rank `Q`. +/// +/// `indices` must be integer tensor, containing indices into `input`. +/// It must be shape \\([d_0, ..., d_{Q-2}, K]\\) where `0 < K <= P`. +/// +/// The innermost dimension of `indices` (with length `K`) corresponds to +/// indices into elements (if `K = P`) or `(P-K)`-dimensional slices +/// (if `K < P`) along the `K`th dimension of `input`. +/// +/// `updates` is `Tensor` of rank `Q-1+P-K` with shape: +/// +/// $$[d_0, ..., d_{Q-2}, input.shape[K], ..., input.shape[P-1]].$$ +/// +/// For example, say we want to add 4 scattered elements to a rank-1 tensor to 8 +/// elements. In Python, that addition would look like this: +/// +/// input = tf.constant([1, 2, 3, 4, 5, 6, 7, 8]) +/// indices = tf.constant([[4], [3], [1], [7]]) +/// updates = tf.constant([9, 10, 11, 12]) +/// output = tf.scatter_nd_non_aliasing_add(input, indices, updates) +/// with tf.Session() as sess: +/// print(sess.run(output)) +/// +/// The resulting value `output` would look like this: +/// +/// [1, 13, 3, 14, 14, 6, 7, 20] +/// +/// See `tf.scatter_nd` for more details about how to make updates to slices. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: A Tensor. +/// * indices: A Tensor. Must be one of the following types: `int32`, `int64`. +/// A tensor of indices into `input`. +/// * updates: A Tensor. Must have the same type as ref. A tensor of updated values +/// to add to `input`. +/// +/// Returns: +/// * `Output`: A `Tensor` with the same shape as `input`, containing values of `input` +/// updated with `updates`. +class ScatterNdNonAliasingAdd { + public: + ScatterNdNonAliasingAdd(const ::tensorflow::Scope& scope, ::tensorflow::Input + input, ::tensorflow::Input indices, ::tensorflow::Input + updates); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Returns the shape of a tensor. +/// +/// This operation returns a 1-D integer tensor representing the shape of `input`. +/// +/// For example: +/// +/// ``` +/// # 't' is [[[1, 1, 1], [2, 2, 2]], [[3, 3, 3], [4, 4, 4]]] +/// shape(t) ==> [2, 2, 3] +/// ``` +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The output tensor. +class Shape { + public: + /// Optional attribute setters for Shape + struct Attrs { + /// Defaults to DT_INT32 + TF_MUST_USE_RESULT Attrs OutType(DataType x) { + Attrs ret = *this; + ret.out_type_ = x; + return ret; + } + + DataType out_type_ = DT_INT32; + }; + Shape(const ::tensorflow::Scope& scope, ::tensorflow::Input input); + Shape(const ::tensorflow::Scope& scope, ::tensorflow::Input input, const + Shape::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs OutType(DataType x) { + return Attrs().OutType(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Returns shape of tensors. +/// +/// This operation returns N 1-D integer tensors representing shape of `input[i]s`. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `OutputList`: The output tensor. +class ShapeN { + public: + /// Optional attribute setters for ShapeN + struct Attrs { + /// Defaults to DT_INT32 + TF_MUST_USE_RESULT Attrs OutType(DataType x) { + Attrs ret = *this; + ret.out_type_ = x; + return ret; + } + + DataType out_type_ = DT_INT32; + }; + ShapeN(const ::tensorflow::Scope& scope, ::tensorflow::InputList input); + ShapeN(const ::tensorflow::Scope& scope, ::tensorflow::InputList input, const + ShapeN::Attrs& attrs); + ::tensorflow::Output operator[](size_t index) const { return output[index]; } + + + static Attrs OutType(DataType x) { + return Attrs().OutType(x); + } + + Operation operation; + ::tensorflow::OutputList output; +}; + +/// Returns the size of a tensor. +/// +/// This operation returns an integer representing the number of elements in +/// `input`. +/// +/// For example: +/// +/// ``` +/// # 't' is [[[1, 1,, 1], [2, 2, 2]], [[3, 3, 3], [4, 4, 4]]]] +/// size(t) ==> 12 +/// ``` +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The output tensor. +class Size { + public: + /// Optional attribute setters for Size + struct Attrs { + /// Defaults to DT_INT32 + TF_MUST_USE_RESULT Attrs OutType(DataType x) { + Attrs ret = *this; + ret.out_type_ = x; + return ret; + } + + DataType out_type_ = DT_INT32; + }; + Size(const ::tensorflow::Scope& scope, ::tensorflow::Input input); + Size(const ::tensorflow::Scope& scope, ::tensorflow::Input input, const + Size::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs OutType(DataType x) { + return Attrs().OutType(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Return a slice from 'input'. +/// +/// The output tensor is a tensor with dimensions described by 'size' +/// whose values are extracted from 'input' starting at the offsets in +/// 'begin'. +/// +/// *Requirements*: +/// 0 <= begin[i] <= begin[i] + size[i] <= Di for i in [0, n) +/// +/// Arguments: +/// * scope: A Scope object +/// * begin: begin[i] specifies the offset into the 'i'th dimension of +/// 'input' to slice from. +/// * size: size[i] specifies the number of elements of the 'i'th dimension +/// of 'input' to slice. If size[i] is -1, all remaining elements in dimension +/// i are included in the slice (i.e. this is equivalent to setting +/// size[i] = input.dim_size(i) - begin[i]). +/// +/// Returns: +/// * `Output`: The output tensor. +class Slice { + public: + Slice(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input begin, ::tensorflow::Input size); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Returns a copy of the input tensor. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The output tensor. +class Snapshot { + public: + Snapshot(const ::tensorflow::Scope& scope, ::tensorflow::Input input); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// SpaceToBatch for 4-D tensors of type T. +/// +/// This is a legacy version of the more general SpaceToBatchND. +/// +/// Zero-pads and then rearranges (permutes) blocks of spatial data into batch. +/// More specifically, this op outputs a copy of the input tensor where values from +/// the `height` and `width` dimensions are moved to the `batch` dimension. After +/// the zero-padding, both `height` and `width` of the input must be divisible by the +/// block size. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: 4-D with shape `[batch, height, width, depth]`. +/// * paddings: 2-D tensor of non-negative integers with shape `[2, 2]`. It specifies +/// the padding of the input with zeros across the spatial dimensions as follows: +/// +/// paddings = [[pad_top, pad_bottom], [pad_left, pad_right]] +/// +/// The effective spatial dimensions of the zero-padded input tensor will be: +/// +/// height_pad = pad_top + height + pad_bottom +/// width_pad = pad_left + width + pad_right +/// +/// The attr `block_size` must be greater than one. It indicates the block size. +/// +/// * Non-overlapping blocks of size `block_size x block size` in the height and +/// width dimensions are rearranged into the batch dimension at each location. +/// * The batch of the output tensor is `batch * block_size * block_size`. +/// * Both height_pad and width_pad must be divisible by block_size. +/// +/// The shape of the output will be: +/// +/// [batch*block_size*block_size, height_pad/block_size, width_pad/block_size, +/// depth] +/// +/// Some examples: +/// +/// (1) For the following input of shape `[1, 2, 2, 1]` and block_size of 2: +/// +/// ``` +/// x = [[[[1], [2]], [[3], [4]]]] +/// ``` +/// +/// The output tensor has shape `[4, 1, 1, 1]` and value: +/// +/// ``` +/// [[[[1]]], [[[2]]], [[[3]]], [[[4]]]] +/// ``` +/// +/// (2) For the following input of shape `[1, 2, 2, 3]` and block_size of 2: +/// +/// ``` +/// x = [[[[1, 2, 3], [4, 5, 6]], +/// [[7, 8, 9], [10, 11, 12]]]] +/// ``` +/// +/// The output tensor has shape `[4, 1, 1, 3]` and value: +/// +/// ``` +/// [[[1, 2, 3]], [[4, 5, 6]], [[7, 8, 9]], [[10, 11, 12]]] +/// ``` +/// +/// (3) For the following input of shape `[1, 4, 4, 1]` and block_size of 2: +/// +/// ``` +/// x = [[[[1], [2], [3], [4]], +/// [[5], [6], [7], [8]], +/// [[9], [10], [11], [12]], +/// [[13], [14], [15], [16]]]] +/// ``` +/// +/// The output tensor has shape `[4, 2, 2, 1]` and value: +/// +/// ``` +/// x = [[[[1], [3]], [[9], [11]]], +/// [[[2], [4]], [[10], [12]]], +/// [[[5], [7]], [[13], [15]]], +/// [[[6], [8]], [[14], [16]]]] +/// ``` +/// +/// (4) For the following input of shape `[2, 2, 4, 1]` and block_size of 2: +/// +/// ``` +/// x = [[[[1], [2], [3], [4]], +/// [[5], [6], [7], [8]]], +/// [[[9], [10], [11], [12]], +/// [[13], [14], [15], [16]]]] +/// ``` +/// +/// The output tensor has shape `[8, 1, 2, 1]` and value: +/// +/// ``` +/// x = [[[[1], [3]]], [[[9], [11]]], [[[2], [4]]], [[[10], [12]]], +/// [[[5], [7]]], [[[13], [15]]], [[[6], [8]]], [[[14], [16]]]] +/// ``` +/// +/// Among others, this operation is useful for reducing atrous convolution into +/// regular convolution. +/// +/// Returns: +/// * `Output`: The output tensor. +class SpaceToBatch { + public: + SpaceToBatch(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input paddings, int64 block_size); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// SpaceToBatch for N-D tensors of type T. +/// +/// This operation divides "spatial" dimensions `[1, ..., M]` of the input into a +/// grid of blocks of shape `block_shape`, and interleaves these blocks with the +/// "batch" dimension (0) such that in the output, the spatial dimensions +/// `[1, ..., M]` correspond to the position within the grid, and the batch +/// dimension combines both the position within a spatial block and the original +/// batch position. Prior to division into blocks, the spatial dimensions of the +/// input are optionally zero padded according to `paddings`. See below for a +/// precise description. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: N-D with shape `input_shape = [batch] + spatial_shape + remaining_shape`, +/// where spatial_shape has `M` dimensions. +/// * block_shape: 1-D with shape `[M]`, all values must be >= 1. +/// * paddings: 2-D with shape `[M, 2]`, all values must be >= 0. +/// `paddings[i] = [pad_start, pad_end]` specifies the padding for input dimension +/// `i + 1`, which corresponds to spatial dimension `i`. It is required that +/// `block_shape[i]` divides `input_shape[i + 1] + pad_start + pad_end`. +/// +/// This operation is equivalent to the following steps: +/// +/// 1. Zero-pad the start and end of dimensions `[1, ..., M]` of the +/// input according to `paddings` to produce `padded` of shape `padded_shape`. +/// +/// 2. Reshape `padded` to `reshaped_padded` of shape: +/// +/// [batch] + +/// [padded_shape[1] / block_shape[0], +/// block_shape[0], +/// ..., +/// padded_shape[M] / block_shape[M-1], +/// block_shape[M-1]] + +/// remaining_shape +/// +/// 3. Permute dimensions of `reshaped_padded` to produce +/// `permuted_reshaped_padded` of shape: +/// +/// block_shape + +/// [batch] + +/// [padded_shape[1] / block_shape[0], +/// ..., +/// padded_shape[M] / block_shape[M-1]] + +/// remaining_shape +/// +/// 4. Reshape `permuted_reshaped_padded` to flatten `block_shape` into the batch +/// dimension, producing an output tensor of shape: +/// +/// [batch * prod(block_shape)] + +/// [padded_shape[1] / block_shape[0], +/// ..., +/// padded_shape[M] / block_shape[M-1]] + +/// remaining_shape +/// +/// Some examples: +/// +/// (1) For the following input of shape `[1, 2, 2, 1]`, `block_shape = [2, 2]`, and +/// `paddings = [[0, 0], [0, 0]]`: +/// +/// ``` +/// x = [[[[1], [2]], [[3], [4]]]] +/// ``` +/// +/// The output tensor has shape `[4, 1, 1, 1]` and value: +/// +/// ``` +/// [[[[1]]], [[[2]]], [[[3]]], [[[4]]]] +/// ``` +/// +/// (2) For the following input of shape `[1, 2, 2, 3]`, `block_shape = [2, 2]`, and +/// `paddings = [[0, 0], [0, 0]]`: +/// +/// ``` +/// x = [[[[1, 2, 3], [4, 5, 6]], +/// [[7, 8, 9], [10, 11, 12]]]] +/// ``` +/// +/// The output tensor has shape `[4, 1, 1, 3]` and value: +/// +/// ``` +/// [[[1, 2, 3]], [[4, 5, 6]], [[7, 8, 9]], [[10, 11, 12]]] +/// ``` +/// +/// (3) For the following input of shape `[1, 4, 4, 1]`, `block_shape = [2, 2]`, and +/// `paddings = [[0, 0], [0, 0]]`: +/// +/// ``` +/// x = [[[[1], [2], [3], [4]], +/// [[5], [6], [7], [8]], +/// [[9], [10], [11], [12]], +/// [[13], [14], [15], [16]]]] +/// ``` +/// +/// The output tensor has shape `[4, 2, 2, 1]` and value: +/// +/// ``` +/// x = [[[[1], [3]], [[9], [11]]], +/// [[[2], [4]], [[10], [12]]], +/// [[[5], [7]], [[13], [15]]], +/// [[[6], [8]], [[14], [16]]]] +/// ``` +/// +/// (4) For the following input of shape `[2, 2, 4, 1]`, block_shape = `[2, 2]`, and +/// paddings = `[[0, 0], [2, 0]]`: +/// +/// ``` +/// x = [[[[1], [2], [3], [4]], +/// [[5], [6], [7], [8]]], +/// [[[9], [10], [11], [12]], +/// [[13], [14], [15], [16]]]] +/// ``` +/// +/// The output tensor has shape `[8, 1, 3, 1]` and value: +/// +/// ``` +/// x = [[[[0], [1], [3]]], [[[0], [9], [11]]], +/// [[[0], [2], [4]]], [[[0], [10], [12]]], +/// [[[0], [5], [7]]], [[[0], [13], [15]]], +/// [[[0], [6], [8]]], [[[0], [14], [16]]]] +/// ``` +/// +/// Among others, this operation is useful for reducing atrous convolution into +/// regular convolution. +/// +/// Returns: +/// * `Output`: The output tensor. +class SpaceToBatchND { + public: + SpaceToBatchND(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input block_shape, ::tensorflow::Input paddings); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// SpaceToDepth for tensors of type T. +/// +/// Rearranges blocks of spatial data, into depth. More specifically, +/// this op outputs a copy of the input tensor where values from the `height` +/// and `width` dimensions are moved to the `depth` dimension. +/// The attr `block_size` indicates the input block size. +/// +/// * Non-overlapping blocks of size `block_size x block size` are rearranged +/// into depth at each location. +/// * The depth of the output tensor is `block_size * block_size * input_depth`. +/// * The Y, X coordinates within each block of the input become the high order +/// component of the output channel index. +/// * The input tensor's height and width must be divisible by block_size. +/// +/// The `data_format` attr specifies the layout of the input and output tensors +/// with the following options: +/// "NHWC": `[ batch, height, width, channels ]` +/// "NCHW": `[ batch, channels, height, width ]` +/// "NCHW_VECT_C": +/// `qint8 [ batch, channels / 4, height, width, 4 ]` +/// +/// It is useful to consider the operation as transforming a 6-D Tensor. +/// e.g. for data_format = NHWC, +/// Each element in the input tensor can be specified via 6 coordinates, +/// ordered by decreasing memory layout significance as: +/// n,oY,bY,oX,bX,iC (where n=batch index, oX, oY means X or Y coordinates +/// within the output image, bX, bY means coordinates +/// within the input block, iC means input channels). +/// The output would be a transpose to the following layout: +/// n,oY,oX,bY,bX,iC +/// +/// This operation is useful for resizing the activations between convolutions +/// (but keeping all data), e.g. instead of pooling. It is also useful for training +/// purely convolutional models. +/// +/// For example, given an input of shape `[1, 2, 2, 1]`, data_format = "NHWC" and +/// block_size = 2: +/// +/// ``` +/// x = [[[[1], [2]], +/// [[3], [4]]]] +/// ``` +/// +/// This operation will output a tensor of shape `[1, 1, 1, 4]`: +/// +/// ``` +/// [[[[1, 2, 3, 4]]]] +/// ``` +/// +/// Here, the input has a batch of 1 and each batch element has shape `[2, 2, 1]`, +/// the corresponding output will have a single element (i.e. width and height are +/// both 1) and will have a depth of 4 channels (1 * block_size * block_size). +/// The output element shape is `[1, 1, 4]`. +/// +/// For an input tensor with larger depth, here of shape `[1, 2, 2, 3]`, e.g. +/// +/// ``` +/// x = [[[[1, 2, 3], [4, 5, 6]], +/// [[7, 8, 9], [10, 11, 12]]]] +/// ``` +/// +/// This operation, for block_size of 2, will return the following tensor of shape +/// `[1, 1, 1, 12]` +/// +/// ``` +/// [[[[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]]]] +/// ``` +/// +/// Similarly, for the following input of shape `[1 4 4 1]`, and a block size of 2: +/// +/// ``` +/// x = [[[[1], [2], [5], [6]], +/// [[3], [4], [7], [8]], +/// [[9], [10], [13], [14]], +/// [[11], [12], [15], [16]]]] +/// ``` +/// +/// the operator will return the following tensor of shape `[1 2 2 4]`: +/// +/// ``` +/// x = [[[[1, 2, 3, 4], +/// [5, 6, 7, 8]], +/// [[9, 10, 11, 12], +/// [13, 14, 15, 16]]]] +/// ``` +/// +/// Arguments: +/// * scope: A Scope object +/// * block_size: The size of the spatial block. +/// +/// Returns: +/// * `Output`: The output tensor. +class SpaceToDepth { + public: + /// Optional attribute setters for SpaceToDepth + struct Attrs { + /// Defaults to "NHWC" + TF_MUST_USE_RESULT Attrs DataFormat(StringPiece x) { + Attrs ret = *this; + ret.data_format_ = x; + return ret; + } + + StringPiece data_format_ = "NHWC"; + }; + SpaceToDepth(const ::tensorflow::Scope& scope, ::tensorflow::Input input, int64 + block_size); + SpaceToDepth(const ::tensorflow::Scope& scope, ::tensorflow::Input input, int64 + block_size, const SpaceToDepth::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs DataFormat(StringPiece x) { + return Attrs().DataFormat(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Splits a tensor into `num_split` tensors along one dimension. +/// +/// Arguments: +/// * scope: A Scope object +/// * axis: 0-D. The dimension along which to split. Must be in the range +/// `[-rank(value), rank(value))`. +/// * value: The tensor to split. +/// * num_split: The number of ways to split. Must evenly divide +/// `value.shape[split_dim]`. +/// +/// Returns: +/// * `OutputList`: They are identically shaped tensors, whose shape matches that of `value` +/// except along `axis`, where their sizes are +/// `values.shape[split_dim] / num_split`. +class Split { + public: + Split(const ::tensorflow::Scope& scope, ::tensorflow::Input axis, + ::tensorflow::Input value, int64 num_split); + ::tensorflow::Output operator[](size_t index) const { return output[index]; } + + + Operation operation; + ::tensorflow::OutputList output; +}; + +/// Splits a tensor into `num_split` tensors along one dimension. +/// +/// Arguments: +/// * scope: A Scope object +/// * value: The tensor to split. +/// * size_splits: list containing the sizes of each output tensor along the split +/// dimension. Must sum to the dimension of value along split_dim. +/// Can contain one -1 indicating that dimension is to be inferred. +/// * axis: 0-D. The dimension along which to split. Must be in the range +/// `[-rank(value), rank(value))`. +/// +/// Returns: +/// * `OutputList`: Tensors whose shape matches that of `value` +/// except along `axis`, where their sizes are +/// `size_splits[i]`. +class SplitV { + public: + SplitV(const ::tensorflow::Scope& scope, ::tensorflow::Input value, + ::tensorflow::Input size_splits, ::tensorflow::Input axis, int64 + num_split); + ::tensorflow::Output operator[](size_t index) const { return output[index]; } + + + Operation operation; + ::tensorflow::OutputList output; +}; + +/// Removes dimensions of size 1 from the shape of a tensor. +/// +/// Given a tensor `input`, this operation returns a tensor of the same type with +/// all dimensions of size 1 removed. If you don't want to remove all size 1 +/// dimensions, you can remove specific size 1 dimensions by specifying +/// `axis`. +/// +/// For example: +/// +/// ``` +/// # 't' is a tensor of shape [1, 2, 1, 3, 1, 1] +/// shape(squeeze(t)) ==> [2, 3] +/// ``` +/// +/// Or, to remove specific size 1 dimensions: +/// +/// ``` +/// # 't' is a tensor of shape [1, 2, 1, 3, 1, 1] +/// shape(squeeze(t, [2, 4])) ==> [1, 2, 3, 1] +/// ``` +/// +/// Arguments: +/// * scope: A Scope object +/// * input: The `input` to squeeze. +/// +/// Optional attributes (see `Attrs`): +/// * axis: If specified, only squeezes the dimensions listed. The dimension +/// index starts at 0. It is an error to squeeze a dimension that is not 1. Must +/// be in the range `[-rank(input), rank(input))`. +/// +/// Returns: +/// * `Output`: Contains the same data as `input`, but has one or more dimensions of +/// size 1 removed. +class Squeeze { + public: + /// Optional attribute setters for Squeeze + struct Attrs { + /// If specified, only squeezes the dimensions listed. The dimension + /// index starts at 0. It is an error to squeeze a dimension that is not 1. Must + /// be in the range `[-rank(input), rank(input))`. + /// + /// Defaults to [] + TF_MUST_USE_RESULT Attrs Axis(const gtl::ArraySlice& x) { + Attrs ret = *this; + ret.axis_ = x; + return ret; + } + + gtl::ArraySlice axis_ = {}; + }; + Squeeze(const ::tensorflow::Scope& scope, ::tensorflow::Input input); + Squeeze(const ::tensorflow::Scope& scope, ::tensorflow::Input input, const + Squeeze::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs Axis(const gtl::ArraySlice& x) { + return Attrs().Axis(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Stops gradient computation. +/// +/// When executed in a graph, this op outputs its input tensor as-is. +/// +/// When building ops to compute gradients, this op prevents the contribution of +/// its inputs to be taken into account. Normally, the gradient generator adds ops +/// to a graph to compute the derivatives of a specified 'loss' by recursively +/// finding out inputs that contributed to its computation. If you insert this op +/// in the graph it inputs are masked from the gradient generator. They are not +/// taken into account for computing gradients. +/// +/// This is useful any time you want to compute a value with TensorFlow but need +/// to pretend that the value was a constant. Some examples include: +/// +/// * The *EM* algorithm where the *M-step* should not involve backpropagation +/// through the output of the *E-step*. +/// * Contrastive divergence training of Boltzmann machines where, when +/// differentiating the energy function, the training must not backpropagate +/// through the graph that generated the samples from the model. +/// * Adversarial training, where no backprop should happen through the adversarial +/// example generation process. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The output tensor. +class StopGradient { + public: + StopGradient(const ::tensorflow::Scope& scope, ::tensorflow::Input input); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Return a strided slice from `input`. +/// +/// Note, most python users will want to use the Python `Tensor.__getitem__` +/// or `Variable.__getitem__` rather than this op directly. +/// +/// The goal of this op is to produce a new tensor with a subset of +/// the elements from the `n` dimensional `input` tensor. The subset is chosen using +/// a sequence of `m` sparse range specifications encoded into the arguments +/// of this function. Note, in some cases +/// `m` could be equal to `n`, but this need not be the case. Each +/// range specification entry can be one of the following: +/// +/// - An ellipsis (...). Ellipses are used to imply zero or more +/// dimensions of full-dimension selection and are produced using +/// `ellipsis_mask`. For example, `foo[...]` is the identity slice. +/// +/// - A new axis. This is used to insert a new shape=1 dimension and is +/// produced using `new_axis_mask`. For example, `foo[:, ...]` where +/// `foo` is shape `(3, 4)` produces a `(1, 3, 4)` tensor. +/// +/// +/// - A range `begin:end:stride`. This is used to specify how much to choose from +/// a given dimension. `stride` can be any integer but 0. `begin` is an integer +/// which represents the index of the first value to select while `end` represents +/// the index of the last value to select. The number of values selected in each +/// dimension is `end - begin` if `stride > 0` and `begin - end` if `stride < 0`. +/// `begin` and `end` can be negative where `-1` is the last element, `-2` is +/// the second to last. `begin_mask` controls whether to replace the explicitly +/// given `begin` with an implicit effective value of `0` if `stride > 0` and +/// `-1` if `stride < 0`. `end_mask` is analogous but produces the number +/// required to create the largest open interval. For example, given a shape +/// `(3,)` tensor `foo[:]`, the effective `begin` and `end` are `0` and `3`. Do +/// not assume this is equivalent to `foo[0:-1]` which has an effective `begin` +/// and `end` of `0` and `2`. Another example is `foo[-2::-1]` which reverses the +/// first dimension of a tensor while dropping the last two (in the original +/// order elements). For example `foo = [1,2,3,4]; foo[-2::-1]` is `[4,3]`. +/// +/// - A single index. This is used to keep only elements that have a given +/// index. For example (`foo[2, :]` on a shape `(5,6)` tensor produces a +/// shape `(6,)` tensor. This is encoded in `begin` and `end` and +/// `shrink_axis_mask`. +/// +/// Each conceptual range specification is encoded in the op's argument. This +/// encoding is best understand by considering a non-trivial example. In +/// particular, +/// `foo[1, 2:4, None, ..., :-3:-1, :]` will be encoded as +/// +/// ``` +/// begin = [1, 2, x, x, 0, x] # x denotes don't care (usually 0) +/// end = [2, 4, x, x, -3, x] +/// strides = [1, 1, x, x, -1, 1] +/// begin_mask = 1<<4 | 1 << 5 = 48 +/// end_mask = 1<<5 = 32 +/// ellipsis_mask = 1<<3 = 8 +/// new_axis_mask = 1<<2 4 +/// shrink_axis_mask = 1<<0 +/// ``` +/// +/// In this case if `foo.shape` is (5, 5, 5, 5, 5, 5) the final shape of +/// the slice becomes (2, 1, 5, 5, 2, 5). +/// Let us walk step by step through each argument specification. +/// +/// 1. The first argument in the example slice is turned into `begin = 1` and +/// `end = begin + 1 = 2`. To disambiguate from the original spec `2:4` we +/// also set the appropriate bit in `shrink_axis_mask`. +/// +/// 2. `2:4` is contributes 2, 4, 1 to begin, end, and stride. All masks have +/// zero bits contributed. +/// +/// 3. None is a synonym for `tf.newaxis`. This means insert a dimension of size 1 +/// dimension in the final shape. Dummy values are contributed to begin, +/// end and stride, while the new_axis_mask bit is set. +/// +/// 4. `...` grab the full ranges from as many dimensions as needed to +/// fully specify a slice for every dimension of the input shape. +/// +/// 5. `:-3:-1` shows the use of negative indices. A negative index `i` associated +/// with a dimension that has shape `s` is converted to a positive index +/// `s + i`. So `-1` becomes `s-1` (i.e. the last element). This conversion +/// is done internally so begin, end and strides receive x, -3, and -1. +/// The appropriate begin_mask bit is set to indicate the start range is the +/// full range (ignoring the x). +/// +/// 6. `:` indicates that the entire contents of the corresponding dimension +/// is selected. This is equivalent to `::` or `0::1`. begin, end, and strides +/// receive 0, 0, and 1, respectively. The appropriate bits in `begin_mask` and +/// `end_mask` are also set. +/// +/// *Requirements*: +/// `0 != strides[i] for i in [0, m)` +/// `ellipsis_mask must be a power of two (only one ellipsis)` +/// +/// Arguments: +/// * scope: A Scope object +/// * begin: `begin[k]` specifies the offset into the `k`th range specification. +/// The exact dimension this corresponds to will be determined by context. +/// Out-of-bounds values will be silently clamped. If the `k`th bit of +/// `begin_mask` then `begin[k]` is ignored and the full range of the +/// appropriate dimension is used instead. Negative values causes indexing +/// to start from the highest element e.g. If `foo==[1,2,3]` then `foo[-1]==3`. +/// * end: `end[i]` is like `begin` with the exception that `end_mask` is +/// used to determine full ranges. +/// * strides: `strides[i]` specifies the increment in the `i`th specification +/// after extracting a given element. Negative indices will reverse +/// the original order. Out or range values are +/// clamped to `[0,dim[i]) if slice[i]>0` or `[-1,dim[i]-1] if slice[i] < 0` +/// +/// Optional attributes (see `Attrs`): +/// * begin_mask: a bitmask where a bit i being 1 means to ignore the begin +/// value and instead use the largest interval possible. At runtime +/// begin[i] will be replaced with `[0, n-1)` if `stride[i] > 0` or +/// `[-1, n-1]` if `stride[i] < 0` +/// * end_mask: analogous to `begin_mask` +/// * ellipsis_mask: a bitmask where bit `i` being 1 means the `i`th +/// position is actually an ellipsis. One bit at most can be 1. +/// If `ellipsis_mask == 0`, then an implicit ellipsis mask of `1 << (m+1)` +/// is provided. This means that `foo[3:5] == foo[3:5, ...]`. An ellipsis +/// implicitly creates as many range specifications as necessary to fully +/// specify the sliced range for every dimension. For example for a 4-dimensional +/// tensor `foo` the slice `foo[2, ..., 5:8]` implies `foo[2, :, :, 5:8]`. +/// * new_axis_mask: a bitmask where bit `i` being 1 means the `i`th +/// specification creates a new shape 1 dimension. For example +/// `foo[:4, tf.newaxis, :2]` would produce a shape `(4, 1, 2)` tensor. +/// * shrink_axis_mask: a bitmask where bit `i` implies that the `i`th +/// specification should shrink the dimensionality. begin and end +/// must imply a slice of size 1 in the dimension. For example in +/// python one might do `foo[:, 3, :]` which would result in +/// `shrink_axis_mask` being 2. +/// +/// Returns: +/// * `Output`: The output tensor. +class StridedSlice { + public: + /// Optional attribute setters for StridedSlice + struct Attrs { + /// a bitmask where a bit i being 1 means to ignore the begin + /// value and instead use the largest interval possible. At runtime + /// begin[i] will be replaced with `[0, n-1)` if `stride[i] > 0` or + /// `[-1, n-1]` if `stride[i] < 0` + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs BeginMask(int64 x) { + Attrs ret = *this; + ret.begin_mask_ = x; + return ret; + } + + /// analogous to `begin_mask` + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs EndMask(int64 x) { + Attrs ret = *this; + ret.end_mask_ = x; + return ret; + } + + /// a bitmask where bit `i` being 1 means the `i`th + /// position is actually an ellipsis. One bit at most can be 1. + /// If `ellipsis_mask == 0`, then an implicit ellipsis mask of `1 << (m+1)` + /// is provided. This means that `foo[3:5] == foo[3:5, ...]`. An ellipsis + /// implicitly creates as many range specifications as necessary to fully + /// specify the sliced range for every dimension. For example for a 4-dimensional + /// tensor `foo` the slice `foo[2, ..., 5:8]` implies `foo[2, :, :, 5:8]`. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs EllipsisMask(int64 x) { + Attrs ret = *this; + ret.ellipsis_mask_ = x; + return ret; + } + + /// a bitmask where bit `i` being 1 means the `i`th + /// specification creates a new shape 1 dimension. For example + /// `foo[:4, tf.newaxis, :2]` would produce a shape `(4, 1, 2)` tensor. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs NewAxisMask(int64 x) { + Attrs ret = *this; + ret.new_axis_mask_ = x; + return ret; + } + + /// a bitmask where bit `i` implies that the `i`th + /// specification should shrink the dimensionality. begin and end + /// must imply a slice of size 1 in the dimension. For example in + /// python one might do `foo[:, 3, :]` which would result in + /// `shrink_axis_mask` being 2. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs ShrinkAxisMask(int64 x) { + Attrs ret = *this; + ret.shrink_axis_mask_ = x; + return ret; + } + + int64 begin_mask_ = 0; + int64 end_mask_ = 0; + int64 ellipsis_mask_ = 0; + int64 new_axis_mask_ = 0; + int64 shrink_axis_mask_ = 0; + }; + StridedSlice(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input begin, ::tensorflow::Input end, + ::tensorflow::Input strides); + StridedSlice(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input begin, ::tensorflow::Input end, + ::tensorflow::Input strides, const StridedSlice::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs BeginMask(int64 x) { + return Attrs().BeginMask(x); + } + static Attrs EndMask(int64 x) { + return Attrs().EndMask(x); + } + static Attrs EllipsisMask(int64 x) { + return Attrs().EllipsisMask(x); + } + static Attrs NewAxisMask(int64 x) { + return Attrs().NewAxisMask(x); + } + static Attrs ShrinkAxisMask(int64 x) { + return Attrs().ShrinkAxisMask(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Assign `value` to the sliced l-value reference of `ref`. +/// +/// The values of `value` are assigned to the positions in the variable +/// `ref` that are selected by the slice parameters. The slice parameters +/// `begin, `end`, `strides`, etc. work exactly as in `StridedSlice`. +/// +/// NOTE this op currently does not support broadcasting and so `value`'s +/// shape must be exactly the shape produced by the slice of `ref`. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The output_ref tensor. +class StridedSliceAssign { + public: + /// Optional attribute setters for StridedSliceAssign + struct Attrs { + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs BeginMask(int64 x) { + Attrs ret = *this; + ret.begin_mask_ = x; + return ret; + } + + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs EndMask(int64 x) { + Attrs ret = *this; + ret.end_mask_ = x; + return ret; + } + + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs EllipsisMask(int64 x) { + Attrs ret = *this; + ret.ellipsis_mask_ = x; + return ret; + } + + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs NewAxisMask(int64 x) { + Attrs ret = *this; + ret.new_axis_mask_ = x; + return ret; + } + + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs ShrinkAxisMask(int64 x) { + Attrs ret = *this; + ret.shrink_axis_mask_ = x; + return ret; + } + + int64 begin_mask_ = 0; + int64 end_mask_ = 0; + int64 ellipsis_mask_ = 0; + int64 new_axis_mask_ = 0; + int64 shrink_axis_mask_ = 0; + }; + StridedSliceAssign(const ::tensorflow::Scope& scope, ::tensorflow::Input ref, + ::tensorflow::Input begin, ::tensorflow::Input end, + ::tensorflow::Input strides, ::tensorflow::Input value); + StridedSliceAssign(const ::tensorflow::Scope& scope, ::tensorflow::Input ref, + ::tensorflow::Input begin, ::tensorflow::Input end, + ::tensorflow::Input strides, ::tensorflow::Input value, + const StridedSliceAssign::Attrs& attrs); + operator ::tensorflow::Output() const { return output_ref; } + operator ::tensorflow::Input() const { return output_ref; } + ::tensorflow::Node* node() const { return output_ref.node(); } + + static Attrs BeginMask(int64 x) { + return Attrs().BeginMask(x); + } + static Attrs EndMask(int64 x) { + return Attrs().EndMask(x); + } + static Attrs EllipsisMask(int64 x) { + return Attrs().EllipsisMask(x); + } + static Attrs NewAxisMask(int64 x) { + return Attrs().NewAxisMask(x); + } + static Attrs ShrinkAxisMask(int64 x) { + return Attrs().ShrinkAxisMask(x); + } + + Operation operation; + ::tensorflow::Output output_ref; +}; + +/// Returns the gradient of `StridedSlice`. +/// +/// Since `StridedSlice` cuts out pieces of its `input` which is size +/// `shape`, its gradient will have the same shape (which is passed here +/// as `shape`). The gradient will be zero in any element that the slice +/// does not select. +/// +/// Arguments are the same as StridedSliceGrad with the exception that +/// `dy` is the input gradient to be propagated and `shape` is the +/// shape of `StridedSlice`'s `input`. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The output tensor. +class StridedSliceGrad { + public: + /// Optional attribute setters for StridedSliceGrad + struct Attrs { + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs BeginMask(int64 x) { + Attrs ret = *this; + ret.begin_mask_ = x; + return ret; + } + + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs EndMask(int64 x) { + Attrs ret = *this; + ret.end_mask_ = x; + return ret; + } + + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs EllipsisMask(int64 x) { + Attrs ret = *this; + ret.ellipsis_mask_ = x; + return ret; + } + + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs NewAxisMask(int64 x) { + Attrs ret = *this; + ret.new_axis_mask_ = x; + return ret; + } + + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs ShrinkAxisMask(int64 x) { + Attrs ret = *this; + ret.shrink_axis_mask_ = x; + return ret; + } + + int64 begin_mask_ = 0; + int64 end_mask_ = 0; + int64 ellipsis_mask_ = 0; + int64 new_axis_mask_ = 0; + int64 shrink_axis_mask_ = 0; + }; + StridedSliceGrad(const ::tensorflow::Scope& scope, ::tensorflow::Input shape, + ::tensorflow::Input begin, ::tensorflow::Input end, + ::tensorflow::Input strides, ::tensorflow::Input dy); + StridedSliceGrad(const ::tensorflow::Scope& scope, ::tensorflow::Input shape, + ::tensorflow::Input begin, ::tensorflow::Input end, + ::tensorflow::Input strides, ::tensorflow::Input dy, const + StridedSliceGrad::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs BeginMask(int64 x) { + return Attrs().BeginMask(x); + } + static Attrs EndMask(int64 x) { + return Attrs().EndMask(x); + } + static Attrs EllipsisMask(int64 x) { + return Attrs().EllipsisMask(x); + } + static Attrs NewAxisMask(int64 x) { + return Attrs().NewAxisMask(x); + } + static Attrs ShrinkAxisMask(int64 x) { + return Attrs().ShrinkAxisMask(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Constructs a tensor by tiling a given tensor. +/// +/// This operation creates a new tensor by replicating `input` `multiples` times. +/// The output tensor's i'th dimension has `input.dims(i) * multiples[i]` elements, +/// and the values of `input` are replicated `multiples[i]` times along the 'i'th +/// dimension. For example, tiling `[a b c d]` by `[2]` produces +/// `[a b c d a b c d]`. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: 1-D or higher. +/// * multiples: 1-D. Length must be the same as the number of dimensions in `input` +/// +/// Returns: +/// * `Output`: The output tensor. +class Tile { + public: + Tile(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input multiples); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Shuffle dimensions of x according to a permutation. +/// +/// The output `y` has the same rank as `x`. The shapes of `x` and `y` satisfy: +/// `y.shape[i] == x.shape[perm[i]] for i in [0, 1, ..., rank(x) - 1]` +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The y tensor. +class Transpose { + public: + Transpose(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input perm); + operator ::tensorflow::Output() const { return y; } + operator ::tensorflow::Input() const { return y; } + ::tensorflow::Node* node() const { return y.node(); } + + Operation operation; + ::tensorflow::Output y; +}; + +/// Finds unique elements in a 1-D tensor. +/// +/// This operation returns a tensor `y` containing all of the unique elements of `x` +/// sorted in the same order that they occur in `x`. This operation also returns a +/// tensor `idx` the same size as `x` that contains the index of each value of `x` +/// in the unique output `y`. In other words: +/// +/// `y[idx[i]] = x[i] for i in [0, 1,...,rank(x) - 1]` +/// +/// For example: +/// +/// ``` +/// # tensor 'x' is [1, 1, 2, 4, 4, 4, 7, 8, 8] +/// y, idx = unique(x) +/// y ==> [1, 2, 4, 7, 8] +/// idx ==> [0, 0, 1, 2, 2, 2, 3, 4, 4] +/// ``` +/// +/// Arguments: +/// * scope: A Scope object +/// * x: 1-D. +/// +/// Returns: +/// * `Output` y: 1-D. +/// * `Output` idx: 1-D. +class Unique { + public: + /// Optional attribute setters for Unique + struct Attrs { + /// Defaults to DT_INT32 + TF_MUST_USE_RESULT Attrs OutIdx(DataType x) { + Attrs ret = *this; + ret.out_idx_ = x; + return ret; + } + + DataType out_idx_ = DT_INT32; + }; + Unique(const ::tensorflow::Scope& scope, ::tensorflow::Input x); + Unique(const ::tensorflow::Scope& scope, ::tensorflow::Input x, const + Unique::Attrs& attrs); + + static Attrs OutIdx(DataType x) { + return Attrs().OutIdx(x); + } + + Operation operation; + ::tensorflow::Output y; + ::tensorflow::Output idx; +}; + +/// Finds unique elements along an axis of a tensor. +/// +/// This operation either returns a tensor `y` containing unique elements +/// along the `axis` of a tensor. The returned unique elements is sorted +/// in the same order as they occur along `axis` in `x`. +/// This operation also returns a tensor `idx` that is the same size as +/// the number of the elements in `x` along the `axis` dimension. It +/// contains the index in the unique output `y`. +/// In other words, for an `1-D` tensor `x` with `axis = None: +/// +/// `y[idx[i]] = x[i] for i in [0, 1,...,rank(x) - 1]` +/// +/// For example: +/// +/// ``` +/// # tensor 'x' is [1, 1, 2, 4, 4, 4, 7, 8, 8] +/// y, idx = unique(x) +/// y ==> [1, 2, 4, 7, 8] +/// idx ==> [0, 0, 1, 2, 2, 2, 3, 4, 4] +/// ``` +/// +/// For an `2-D` tensor `x` with `axis = 0`: +/// +/// ``` +/// # tensor 'x' is [[1, 0, 0], +/// # [1, 0, 0], +/// # [2, 0, 0]] +/// y, idx = unique(x, axis=0) +/// y ==> [[1, 0, 0], +/// [2, 0, 0]] +/// idx ==> [0, 0, 1] +/// ``` +/// +/// For an `2-D` tensor `x` with `axis = 1`: +/// +/// ``` +/// # tensor 'x' is [[1, 0, 0], +/// # [1, 0, 0], +/// # [2, 0, 0]] +/// y, idx = unique(x, axis=1) +/// y ==> [[1, 0], +/// [1, 0], +/// [2, 0]] +/// idx ==> [0, 1, 1] +/// ``` +/// +/// Arguments: +/// * scope: A Scope object +/// * x: A `Tensor`. +/// * axis: A `Tensor` of type `int32` (default: None). The axis of the Tensor to +/// find the unique elements. +/// +/// Returns: +/// * `Output` y: A `Tensor`. Unique elements along the `axis` of `Tensor` x. +/// * `Output` idx: A 1-D Tensor. Has the same type as x that contains the index of each +/// value of x in the output y. +class UniqueV2 { + public: + /// Optional attribute setters for UniqueV2 + struct Attrs { + /// Defaults to DT_INT32 + TF_MUST_USE_RESULT Attrs OutIdx(DataType x) { + Attrs ret = *this; + ret.out_idx_ = x; + return ret; + } + + DataType out_idx_ = DT_INT32; + }; + UniqueV2(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input axis); + UniqueV2(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input axis, const UniqueV2::Attrs& attrs); + + static Attrs OutIdx(DataType x) { + return Attrs().OutIdx(x); + } + + Operation operation; + ::tensorflow::Output y; + ::tensorflow::Output idx; +}; + +/// Finds unique elements in a 1-D tensor. +/// +/// This operation returns a tensor `y` containing all of the unique elements of `x` +/// sorted in the same order that they occur in `x`. This operation also returns a +/// tensor `idx` the same size as `x` that contains the index of each value of `x` +/// in the unique output `y`. Finally, it returns a third tensor `count` that +/// contains the count of each element of `y` in `x`. In other words: +/// +/// `y[idx[i]] = x[i] for i in [0, 1,...,rank(x) - 1]` +/// +/// For example: +/// +/// ``` +/// # tensor 'x' is [1, 1, 2, 4, 4, 4, 7, 8, 8] +/// y, idx, count = unique_with_counts(x) +/// y ==> [1, 2, 4, 7, 8] +/// idx ==> [0, 0, 1, 2, 2, 2, 3, 4, 4] +/// count ==> [2, 1, 3, 1, 2] +/// ``` +/// +/// Arguments: +/// * scope: A Scope object +/// * x: 1-D. +/// +/// Returns: +/// * `Output` y: 1-D. +/// * `Output` idx: 1-D. +/// * `Output` count: 1-D. +class UniqueWithCounts { + public: + /// Optional attribute setters for UniqueWithCounts + struct Attrs { + /// Defaults to DT_INT32 + TF_MUST_USE_RESULT Attrs OutIdx(DataType x) { + Attrs ret = *this; + ret.out_idx_ = x; + return ret; + } + + DataType out_idx_ = DT_INT32; + }; + UniqueWithCounts(const ::tensorflow::Scope& scope, ::tensorflow::Input x); + UniqueWithCounts(const ::tensorflow::Scope& scope, ::tensorflow::Input x, const + UniqueWithCounts::Attrs& attrs); + + static Attrs OutIdx(DataType x) { + return Attrs().OutIdx(x); + } + + Operation operation; + ::tensorflow::Output y; + ::tensorflow::Output idx; + ::tensorflow::Output count; +}; + +/// Finds unique elements along an axis of a tensor. +/// +/// This operation either returns a tensor `y` containing unique elements +/// along the `axis` of a tensor. The returned unique elements is sorted +/// in the same order as they occur along `axis` in `x`. +/// This operation also returns a tensor `idx` and a tensor `count` +/// that are the same size as the number of the elements in `x` along the +/// `axis` dimension. The `idx` contains the index in the unique output `y` +/// and the `count` contains the count in the unique output `y`. +/// In other words, for an `1-D` tensor `x` with `axis = None: +/// +/// `y[idx[i]] = x[i] for i in [0, 1,...,rank(x) - 1]` +/// +/// For example: +/// +/// ``` +/// # tensor 'x' is [1, 1, 2, 4, 4, 4, 7, 8, 8] +/// y, idx, count = unique_with_counts(x) +/// y ==> [1, 2, 4, 7, 8] +/// idx ==> [0, 0, 1, 2, 2, 2, 3, 4, 4] +/// count ==> [2, 1, 3, 1, 2] +/// ``` +/// +/// For an `2-D` tensor `x` with `axis = 0`: +/// +/// ``` +/// # tensor 'x' is [[1, 0, 0], +/// # [1, 0, 0], +/// # [2, 0, 0]] +/// y, idx, count = unique_with_counts(x, axis=0) +/// y ==> [[1, 0, 0], +/// [2, 0, 0]] +/// idx ==> [0, 0, 1] +/// count ==> [2, 1] +/// ``` +/// +/// For an `2-D` tensor `x` with `axis = 1`: +/// +/// ``` +/// # tensor 'x' is [[1, 0, 0], +/// # [1, 0, 0], +/// # [2, 0, 0]] +/// y, idx, count = unique_with_counts(x, axis=1) +/// y ==> [[1, 0], +/// [1, 0], +/// [2, 0]] +/// idx ==> [0, 1, 1] +/// count ==> [1, 2] +/// ``` +/// +/// Arguments: +/// * scope: A Scope object +/// * x: A `Tensor`. +/// * axis: A `Tensor` of type `int32` (default: None). The axis of the Tensor to +/// find the unique elements. +/// +/// Returns: +/// * `Output` y: A `Tensor`. Unique elements along the `axis` of `Tensor` x. +/// * `Output` idx: A 1-D Tensor. Has the same type as x that contains the index of each +/// value of x in the output y. +/// * `Output` count: A 1-D Tensor. The count of each value of x in the output y. +class UniqueWithCountsV2 { + public: + /// Optional attribute setters for UniqueWithCountsV2 + struct Attrs { + /// Defaults to DT_INT32 + TF_MUST_USE_RESULT Attrs OutIdx(DataType x) { + Attrs ret = *this; + ret.out_idx_ = x; + return ret; + } + + DataType out_idx_ = DT_INT32; + }; + UniqueWithCountsV2(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input axis); + UniqueWithCountsV2(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input axis, const UniqueWithCountsV2::Attrs& + attrs); + + static Attrs OutIdx(DataType x) { + return Attrs().OutIdx(x); + } + + Operation operation; + ::tensorflow::Output y; + ::tensorflow::Output idx; + ::tensorflow::Output count; +}; + +/// Unpacks a given dimension of a rank-`R` tensor into `num` rank-`(R-1)` tensors. +/// +/// Unpacks `num` tensors from `value` by chipping it along the `axis` dimension. +/// For example, given a tensor of shape `(A, B, C, D)`; +/// +/// If `axis == 0` then the i'th tensor in `output` is the slice `value[i, :, :, :]` +/// and each tensor in `output` will have shape `(B, C, D)`. (Note that the +/// dimension unpacked along is gone, unlike `split`). +/// +/// If `axis == 1` then the i'th tensor in `output` is the slice `value[:, i, :, :]` +/// and each tensor in `output` will have shape `(A, C, D)`. +/// Etc. +/// +/// This is the opposite of `pack`. +/// +/// Arguments: +/// * scope: A Scope object +/// * value: 1-D or higher, with `axis` dimension size equal to `num`. +/// +/// Optional attributes (see `Attrs`): +/// * axis: Dimension along which to unpack. Negative values wrap around, so the +/// valid range is `[-R, R)`. +/// +/// Returns: +/// * `OutputList`: The list of tensors unpacked from `value`. +class Unstack { + public: + /// Optional attribute setters for Unstack + struct Attrs { + /// Dimension along which to unpack. Negative values wrap around, so the + /// valid range is `[-R, R)`. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Axis(int64 x) { + Attrs ret = *this; + ret.axis_ = x; + return ret; + } + + int64 axis_ = 0; + }; + Unstack(const ::tensorflow::Scope& scope, ::tensorflow::Input value, int64 num); + Unstack(const ::tensorflow::Scope& scope, ::tensorflow::Input value, int64 num, + const Unstack::Attrs& attrs); + ::tensorflow::Output operator[](size_t index) const { return output[index]; } + + + static Attrs Axis(int64 x) { + return Attrs().Axis(x); + } + + Operation operation; + ::tensorflow::OutputList output; +}; + +/// Converts a flat index or array of flat indices into a tuple of +/// +/// coordinate arrays. +/// +/// @compatibility(numpy) +/// Equivalent to np.unravel_index +/// @end_compatibility +/// +/// Arguments: +/// * scope: A Scope object +/// * indices: An 0-D or 1-D `int` Tensor whose elements are indices into the +/// flattened version of an array of dimensions dims. +/// * dims: An 1-D `int` Tensor. The shape of the array to use for unraveling +/// indices. +/// +/// Returns: +/// * `Output`: An 2-D (or 1-D if indices is 0-D) tensor where each row has the +/// same shape as the indices array. +class UnravelIndex { + public: + UnravelIndex(const ::tensorflow::Scope& scope, ::tensorflow::Input indices, + ::tensorflow::Input dims); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Returns locations of nonzero / true values in a tensor. +/// +/// This operation returns the coordinates of true elements in `condition`. The +/// coordinates are returned in a 2-D tensor where the first dimension (rows) +/// represents the number of true elements, and the second dimension (columns) +/// represents the coordinates of the true elements. Keep in mind, the shape of +/// the output tensor can vary depending on how many true values there are in +/// `condition`. Indices are output in row-major order. +/// +/// For example: +/// +/// ``` +/// # 'input' tensor is [[True, False] +/// # [True, False]] +/// # 'input' has two true values, so output has two coordinates. +/// # 'input' has rank of 2, so coordinates have two indices. +/// where(input) ==> [[0, 0], +/// [1, 0]] +/// +/// # `condition` tensor is [[[True, False] +/// # [True, False]] +/// # [[False, True] +/// # [False, True]] +/// # [[False, False] +/// # [False, True]]] +/// # 'input' has 5 true values, so output has 5 coordinates. +/// # 'input' has rank of 3, so coordinates have three indices. +/// where(input) ==> [[0, 0, 0], +/// [0, 1, 0], +/// [1, 0, 1], +/// [1, 1, 1], +/// [2, 1, 1]] +/// +/// # `condition` tensor is [[[1.5, 0.0] +/// # [-0.5, 0.0]] +/// # [[0.0, 0.25] +/// # [0.0, 0.75]] +/// # [[0.0, 0.0] +/// # [0.0, 0.01]]] +/// # 'input' has 5 nonzero values, so output has 5 coordinates. +/// # 'input' has rank of 3, so coordinates have three indices. +/// where(input) ==> [[0, 0, 0], +/// [0, 1, 0], +/// [1, 0, 1], +/// [1, 1, 1], +/// [2, 1, 1]] +/// +/// # `condition` tensor is [[[1.5 + 0.0j, 0.0 + 0.0j] +/// # [0.0 + 0.5j, 0.0 + 0.0j]] +/// # [[0.0 + 0.0j, 0.25 + 1.5j] +/// # [0.0 + 0.0j, 0.75 + 0.0j]] +/// # [[0.0 + 0.0j, 0.0 + 0.0j] +/// # [0.0 + 0.0j, 0.01 + 0.0j]]] +/// # 'input' has 5 nonzero magnitude values, so output has 5 coordinates. +/// # 'input' has rank of 3, so coordinates have three indices. +/// where(input) ==> [[0, 0, 0], +/// [0, 1, 0], +/// [1, 0, 1], +/// [1, 1, 1], +/// [2, 1, 1]] +/// ``` +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The index tensor. +class Where { + public: + Where(const ::tensorflow::Scope& scope, ::tensorflow::Input condition); + operator ::tensorflow::Output() const { return index; } + operator ::tensorflow::Input() const { return index; } + ::tensorflow::Node* node() const { return index.node(); } + + Operation operation; + ::tensorflow::Output index; +}; + +/// Returns a tensor of zeros with the same shape and type as x. +/// +/// Arguments: +/// * scope: A Scope object +/// * x: a tensor of type T. +/// +/// Returns: +/// * `Output`: a tensor of the same shape and type as x but filled with zeros. +class ZerosLike { + public: + ZerosLike(const ::tensorflow::Scope& scope, ::tensorflow::Input x); + operator ::tensorflow::Output() const { return y; } + operator ::tensorflow::Input() const { return y; } + ::tensorflow::Node* node() const { return y.node(); } + + Operation operation; + ::tensorflow::Output y; +}; + +/// @} + +} // namespace ops +} // namespace tensorflow + +#endif // TENSORFLOW_CC_OPS_ARRAY_OPS_H_ diff --git a/linux/include/tensorflow/cc/ops/array_ops_internal.h b/linux/include/tensorflow/cc/ops/array_ops_internal.h new file mode 100644 index 000000000..ada44088c --- /dev/null +++ b/linux/include/tensorflow/cc/ops/array_ops_internal.h @@ -0,0 +1,226 @@ +// This file is MACHINE GENERATED! Do not edit. + +#ifndef TENSORFLOW_CC_OPS_ARRAY_OPS_INTERNAL_H_ +#define TENSORFLOW_CC_OPS_ARRAY_OPS_INTERNAL_H_ + +// This file is MACHINE GENERATED! Do not edit. + +#include "tensorflow/cc/framework/ops.h" +#include "tensorflow/cc/framework/scope.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/gtl/array_slice.h" + +namespace tensorflow { +namespace ops { +namespace internal { +// NOTE: This namespace has internal TensorFlow details that +// are not part of TensorFlow's public API. + +/// @defgroup array_ops_internal Array Ops Internal +/// @{ + +/// Return the reduction indices for computing gradients of s0 op s1 with broadcast. +/// +/// This is typically used by gradient computations for a broadcasting operation. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output` r0 +/// * `Output` r1 +class BroadcastGradientArgs { + public: + BroadcastGradientArgs(const ::tensorflow::Scope& scope, ::tensorflow::Input s0, + ::tensorflow::Input s1); + + Operation operation; + ::tensorflow::Output r0; + ::tensorflow::Output r1; +}; + +/// Applies lower_bound(sorted_search_values, values) along each row. +/// +/// Each set of rows with the same index in (sorted_inputs, values) is treated +/// independently. The resulting row is the equivalent of calling +/// `np.searchsorted(sorted_inputs, values, side='left')`. +/// +/// The result is not a global index to the entire +/// `Tensor`, but rather just the index in the last dimension. +/// +/// A 2-D example: +/// sorted_sequence = [[0, 3, 9, 9, 10], +/// [1, 2, 3, 4, 5]] +/// values = [[2, 4, 9], +/// [0, 2, 6]] +/// +/// result = LowerBound(sorted_sequence, values) +/// +/// result == [[1, 2, 2], +/// [0, 1, 5]] +/// +/// Arguments: +/// * scope: A Scope object +/// * sorted_inputs: 2-D Tensor where each row is ordered. +/// * values: 2-D Tensor with the same numbers of rows as `sorted_search_values`. Contains +/// the values that will be searched for in `sorted_search_values`. +/// +/// Returns: +/// * `Output`: A `Tensor` with the same shape as `values`. It contains the first scalar index +/// into the last dimension where values can be inserted without changing the +/// ordered property. +class LowerBound { + public: + /// Optional attribute setters for LowerBound + struct Attrs { + /// Defaults to DT_INT32 + TF_MUST_USE_RESULT Attrs OutType(DataType x) { + Attrs ret = *this; + ret.out_type_ = x; + return ret; + } + + DataType out_type_ = DT_INT32; + }; + LowerBound(const ::tensorflow::Scope& scope, ::tensorflow::Input sorted_inputs, + ::tensorflow::Input values); + LowerBound(const ::tensorflow::Scope& scope, ::tensorflow::Input sorted_inputs, + ::tensorflow::Input values, const LowerBound::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs OutType(DataType x) { + return Attrs().OutType(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Gradient op for `MirrorPad` op. This op folds a mirror-padded tensor. +/// +/// This operation folds the padded areas of `input` by `MirrorPad` according to the +/// `paddings` you specify. `paddings` must be the same as `paddings` argument +/// given to the corresponding `MirrorPad` op. +/// +/// The folded size of each dimension D of the output is: +/// +/// `input.dim_size(D) - paddings(D, 0) - paddings(D, 1)` +/// +/// For example: +/// +/// ``` +/// # 't' is [[1, 2, 3], [4, 5, 6], [7, 8, 9]]. +/// # 'paddings' is [[0, 1]], [0, 1]]. +/// # 'mode' is SYMMETRIC. +/// # rank of 't' is 2. +/// pad(t, paddings) ==> [[ 1, 5] +/// [11, 28]] +/// ``` +/// +/// Arguments: +/// * scope: A Scope object +/// * input: The input tensor to be folded. +/// * paddings: A two-column matrix specifying the padding sizes. The number of +/// rows must be the same as the rank of `input`. +/// * mode: The mode used in the `MirrorPad` op. +/// +/// Returns: +/// * `Output`: The folded tensor. +class MirrorPadGrad { + public: + MirrorPadGrad(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input paddings, StringPiece mode); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Return the same ref tensor as the input ref tensor. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The output tensor. +class RefIdentity { + public: + RefIdentity(const ::tensorflow::Scope& scope, ::tensorflow::Input input); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Applies upper_bound(sorted_search_values, values) along each row. +/// +/// Each set of rows with the same index in (sorted_inputs, values) is treated +/// independently. The resulting row is the equivalent of calling +/// `np.searchsorted(sorted_inputs, values, side='right')`. +/// +/// The result is not a global index to the entire +/// `Tensor`, but rather just the index in the last dimension. +/// +/// A 2-D example: +/// sorted_sequence = [[0, 3, 9, 9, 10], +/// [1, 2, 3, 4, 5]] +/// values = [[2, 4, 9], +/// [0, 2, 6]] +/// +/// result = UpperBound(sorted_sequence, values) +/// +/// result == [[1, 2, 4], +/// [0, 2, 5]] +/// +/// Arguments: +/// * scope: A Scope object +/// * sorted_inputs: 2-D Tensor where each row is ordered. +/// * values: 2-D Tensor with the same numbers of rows as `sorted_search_values`. Contains +/// the values that will be searched for in `sorted_search_values`. +/// +/// Returns: +/// * `Output`: A `Tensor` with the same shape as `values`. It contains the last scalar index +/// into the last dimension where values can be inserted without changing the +/// ordered property. +class UpperBound { + public: + /// Optional attribute setters for UpperBound + struct Attrs { + /// Defaults to DT_INT32 + TF_MUST_USE_RESULT Attrs OutType(DataType x) { + Attrs ret = *this; + ret.out_type_ = x; + return ret; + } + + DataType out_type_ = DT_INT32; + }; + UpperBound(const ::tensorflow::Scope& scope, ::tensorflow::Input sorted_inputs, + ::tensorflow::Input values); + UpperBound(const ::tensorflow::Scope& scope, ::tensorflow::Input sorted_inputs, + ::tensorflow::Input values, const UpperBound::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs OutType(DataType x) { + return Attrs().OutType(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +} // namespace internal +} // namespace ops +} // namespace tensorflow + +#endif // TENSORFLOW_CC_OPS_ARRAY_OPS_INTERNAL_H_ diff --git a/linux/include/tensorflow/cc/ops/audio_ops.h b/linux/include/tensorflow/cc/ops/audio_ops.h new file mode 100644 index 000000000..ccf626a85 --- /dev/null +++ b/linux/include/tensorflow/cc/ops/audio_ops.h @@ -0,0 +1,295 @@ +// This file is MACHINE GENERATED! Do not edit. + +#ifndef TENSORFLOW_CC_OPS_AUDIO_OPS_H_ +#define TENSORFLOW_CC_OPS_AUDIO_OPS_H_ + +// This file is MACHINE GENERATED! Do not edit. + +#include "tensorflow/cc/framework/ops.h" +#include "tensorflow/cc/framework/scope.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/gtl/array_slice.h" + +namespace tensorflow { +namespace ops { + +/// @defgroup audio_ops Audio Ops +/// @{ + +/// Produces a visualization of audio data over time. +/// +/// Spectrograms are a standard way of representing audio information as a series of +/// slices of frequency information, one slice for each window of time. By joining +/// these together into a sequence, they form a distinctive fingerprint of the sound +/// over time. +/// +/// This op expects to receive audio data as an input, stored as floats in the range +/// -1 to 1, together with a window width in samples, and a stride specifying how +/// far to move the window between slices. From this it generates a three +/// dimensional output. The lowest dimension has an amplitude value for each +/// frequency during that time slice. The next dimension is time, with successive +/// frequency slices. The final dimension is for the channels in the input, so a +/// stereo audio input would have two here for example. +/// +/// This means the layout when converted and saved as an image is rotated 90 degrees +/// clockwise from a typical spectrogram. Time is descending down the Y axis, and +/// the frequency decreases from left to right. +/// +/// Each value in the result represents the square root of the sum of the real and +/// imaginary parts of an FFT on the current window of samples. In this way, the +/// lowest dimension represents the power of each frequency in the current window, +/// and adjacent windows are concatenated in the next dimension. +/// +/// To get a more intuitive and visual look at what this operation does, you can run +/// tensorflow/examples/wav_to_spectrogram to read in an audio file and save out the +/// resulting spectrogram as a PNG image. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: Float representation of audio data. +/// * window_size: How wide the input window is in samples. For the highest efficiency +/// this should be a power of two, but other values are accepted. +/// * stride: How widely apart the center of adjacent sample windows should be. +/// +/// Optional attributes (see `Attrs`): +/// * magnitude_squared: Whether to return the squared magnitude or just the +/// magnitude. Using squared magnitude can avoid extra calculations. +/// +/// Returns: +/// * `Output`: 3D representation of the audio frequencies as an image. +class AudioSpectrogram { + public: + /// Optional attribute setters for AudioSpectrogram + struct Attrs { + /// Whether to return the squared magnitude or just the + /// magnitude. Using squared magnitude can avoid extra calculations. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs MagnitudeSquared(bool x) { + Attrs ret = *this; + ret.magnitude_squared_ = x; + return ret; + } + + bool magnitude_squared_ = false; + }; + AudioSpectrogram(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + int64 window_size, int64 stride); + AudioSpectrogram(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + int64 window_size, int64 stride, const + AudioSpectrogram::Attrs& attrs); + operator ::tensorflow::Output() const { return spectrogram; } + operator ::tensorflow::Input() const { return spectrogram; } + ::tensorflow::Node* node() const { return spectrogram.node(); } + + static Attrs MagnitudeSquared(bool x) { + return Attrs().MagnitudeSquared(x); + } + + Operation operation; + ::tensorflow::Output spectrogram; +}; + +/// Decode a 16-bit PCM WAV file to a float tensor. +/// +/// The -32768 to 32767 signed 16-bit values will be scaled to -1.0 to 1.0 in float. +/// +/// When desired_channels is set, if the input contains fewer channels than this +/// then the last channel will be duplicated to give the requested number, else if +/// the input has more channels than requested then the additional channels will be +/// ignored. +/// +/// If desired_samples is set, then the audio will be cropped or padded with zeroes +/// to the requested length. +/// +/// The first output contains a Tensor with the content of the audio samples. The +/// lowest dimension will be the number of channels, and the second will be the +/// number of samples. For example, a ten-sample-long stereo WAV file should give an +/// output shape of [10, 2]. +/// +/// Arguments: +/// * scope: A Scope object +/// * contents: The WAV-encoded audio, usually from a file. +/// +/// Optional attributes (see `Attrs`): +/// * desired_channels: Number of sample channels wanted. +/// * desired_samples: Length of audio requested. +/// +/// Returns: +/// * `Output` audio: 2-D with shape `[length, channels]`. +/// * `Output` sample_rate: Scalar holding the sample rate found in the WAV header. +class DecodeWav { + public: + /// Optional attribute setters for DecodeWav + struct Attrs { + /// Number of sample channels wanted. + /// + /// Defaults to -1 + TF_MUST_USE_RESULT Attrs DesiredChannels(int64 x) { + Attrs ret = *this; + ret.desired_channels_ = x; + return ret; + } + + /// Length of audio requested. + /// + /// Defaults to -1 + TF_MUST_USE_RESULT Attrs DesiredSamples(int64 x) { + Attrs ret = *this; + ret.desired_samples_ = x; + return ret; + } + + int64 desired_channels_ = -1; + int64 desired_samples_ = -1; + }; + DecodeWav(const ::tensorflow::Scope& scope, ::tensorflow::Input contents); + DecodeWav(const ::tensorflow::Scope& scope, ::tensorflow::Input contents, const + DecodeWav::Attrs& attrs); + + static Attrs DesiredChannels(int64 x) { + return Attrs().DesiredChannels(x); + } + static Attrs DesiredSamples(int64 x) { + return Attrs().DesiredSamples(x); + } + + Operation operation; + ::tensorflow::Output audio; + ::tensorflow::Output sample_rate; +}; + +/// Encode audio data using the WAV file format. +/// +/// This operation will generate a string suitable to be saved out to create a .wav +/// audio file. It will be encoded in the 16-bit PCM format. It takes in float +/// values in the range -1.0f to 1.0f, and any outside that value will be clamped to +/// that range. +/// +/// `audio` is a 2-D float Tensor of shape `[length, channels]`. +/// `sample_rate` is a scalar Tensor holding the rate to use (e.g. 44100). +/// +/// Arguments: +/// * scope: A Scope object +/// * audio: 2-D with shape `[length, channels]`. +/// * sample_rate: Scalar containing the sample frequency. +/// +/// Returns: +/// * `Output`: 0-D. WAV-encoded file contents. +class EncodeWav { + public: + EncodeWav(const ::tensorflow::Scope& scope, ::tensorflow::Input audio, + ::tensorflow::Input sample_rate); + operator ::tensorflow::Output() const { return contents; } + operator ::tensorflow::Input() const { return contents; } + ::tensorflow::Node* node() const { return contents.node(); } + + Operation operation; + ::tensorflow::Output contents; +}; + +/// Transforms a spectrogram into a form that's useful for speech recognition. +/// +/// Mel Frequency Cepstral Coefficients are a way of representing audio data that's +/// been effective as an input feature for machine learning. They are created by +/// taking the spectrum of a spectrogram (a 'cepstrum'), and discarding some of the +/// higher frequencies that are less significant to the human ear. They have a long +/// history in the speech recognition world, and https://en.wikipedia.org/wiki/Mel-frequency_cepstrum +/// is a good resource to learn more. +/// +/// Arguments: +/// * scope: A Scope object +/// * spectrogram: Typically produced by the Spectrogram op, with magnitude_squared +/// set to true. +/// * sample_rate: How many samples per second the source audio used. +/// +/// Optional attributes (see `Attrs`): +/// * upper_frequency_limit: The highest frequency to use when calculating the +/// ceptstrum. +/// * lower_frequency_limit: The lowest frequency to use when calculating the +/// ceptstrum. +/// * filterbank_channel_count: Resolution of the Mel bank used internally. +/// * dct_coefficient_count: How many output channels to produce per time slice. +/// +/// Returns: +/// * `Output`: The output tensor. +class Mfcc { + public: + /// Optional attribute setters for Mfcc + struct Attrs { + /// The highest frequency to use when calculating the + /// ceptstrum. + /// + /// Defaults to 4000 + TF_MUST_USE_RESULT Attrs UpperFrequencyLimit(float x) { + Attrs ret = *this; + ret.upper_frequency_limit_ = x; + return ret; + } + + /// The lowest frequency to use when calculating the + /// ceptstrum. + /// + /// Defaults to 20 + TF_MUST_USE_RESULT Attrs LowerFrequencyLimit(float x) { + Attrs ret = *this; + ret.lower_frequency_limit_ = x; + return ret; + } + + /// Resolution of the Mel bank used internally. + /// + /// Defaults to 40 + TF_MUST_USE_RESULT Attrs FilterbankChannelCount(int64 x) { + Attrs ret = *this; + ret.filterbank_channel_count_ = x; + return ret; + } + + /// How many output channels to produce per time slice. + /// + /// Defaults to 13 + TF_MUST_USE_RESULT Attrs DctCoefficientCount(int64 x) { + Attrs ret = *this; + ret.dct_coefficient_count_ = x; + return ret; + } + + float upper_frequency_limit_ = 4000.0f; + float lower_frequency_limit_ = 20.0f; + int64 filterbank_channel_count_ = 40; + int64 dct_coefficient_count_ = 13; + }; + Mfcc(const ::tensorflow::Scope& scope, ::tensorflow::Input spectrogram, + ::tensorflow::Input sample_rate); + Mfcc(const ::tensorflow::Scope& scope, ::tensorflow::Input spectrogram, + ::tensorflow::Input sample_rate, const Mfcc::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs UpperFrequencyLimit(float x) { + return Attrs().UpperFrequencyLimit(x); + } + static Attrs LowerFrequencyLimit(float x) { + return Attrs().LowerFrequencyLimit(x); + } + static Attrs FilterbankChannelCount(int64 x) { + return Attrs().FilterbankChannelCount(x); + } + static Attrs DctCoefficientCount(int64 x) { + return Attrs().DctCoefficientCount(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// @} + +} // namespace ops +} // namespace tensorflow + +#endif // TENSORFLOW_CC_OPS_AUDIO_OPS_H_ diff --git a/linux/include/tensorflow/cc/ops/audio_ops_internal.h b/linux/include/tensorflow/cc/ops/audio_ops_internal.h new file mode 100644 index 000000000..7b0ac2632 --- /dev/null +++ b/linux/include/tensorflow/cc/ops/audio_ops_internal.h @@ -0,0 +1,28 @@ +// This file is MACHINE GENERATED! Do not edit. + +#ifndef TENSORFLOW_CC_OPS_AUDIO_OPS_INTERNAL_H_ +#define TENSORFLOW_CC_OPS_AUDIO_OPS_INTERNAL_H_ + +// This file is MACHINE GENERATED! Do not edit. + +#include "tensorflow/cc/framework/ops.h" +#include "tensorflow/cc/framework/scope.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/gtl/array_slice.h" + +namespace tensorflow { +namespace ops { +namespace internal { +// NOTE: This namespace has internal TensorFlow details that +// are not part of TensorFlow's public API. + +/// @defgroup audio_ops_internal Audio Ops Internal +/// @{ + +} // namespace internal +} // namespace ops +} // namespace tensorflow + +#endif // TENSORFLOW_CC_OPS_AUDIO_OPS_INTERNAL_H_ diff --git a/linux/include/tensorflow/cc/ops/candidate_sampling_ops.h b/linux/include/tensorflow/cc/ops/candidate_sampling_ops.h new file mode 100644 index 000000000..480b0ee7b --- /dev/null +++ b/linux/include/tensorflow/cc/ops/candidate_sampling_ops.h @@ -0,0 +1,657 @@ +// This file is MACHINE GENERATED! Do not edit. + +#ifndef TENSORFLOW_CC_OPS_CANDIDATE_SAMPLING_OPS_H_ +#define TENSORFLOW_CC_OPS_CANDIDATE_SAMPLING_OPS_H_ + +// This file is MACHINE GENERATED! Do not edit. + +#include "tensorflow/cc/framework/ops.h" +#include "tensorflow/cc/framework/scope.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/gtl/array_slice.h" + +namespace tensorflow { +namespace ops { + +/// @defgroup candidate_sampling_ops Candidate Sampling Ops +/// @{ + +/// Generates labels for candidate sampling with a learned unigram distribution. +/// +/// See explanations of candidate sampling and the data formats at +/// go/candidate-sampling. +/// +/// For each batch, this op picks a single set of sampled candidate labels. +/// +/// The advantages of sampling candidates per-batch are simplicity and the +/// possibility of efficient dense matrix multiplication. The disadvantage is that +/// the sampled candidates must be chosen independently of the context and of the +/// true labels. +/// +/// Arguments: +/// * scope: A Scope object +/// * true_classes: A batch_size * num_true matrix, in which each row contains the +/// IDs of the num_true target_classes in the corresponding original label. +/// * num_true: Number of true labels per context. +/// * num_sampled: Number of candidates to produce. +/// * unique: If unique is true, we sample with rejection, so that all sampled +/// candidates in a batch are unique. This requires some approximation to +/// estimate the post-rejection sampling probabilities. +/// +/// Optional attributes (see `Attrs`): +/// * seed: If either seed or seed2 are set to be non-zero, the random number +/// generator is seeded by the given seed. Otherwise, it is seeded by a +/// random seed. +/// * seed2: An second seed to avoid seed collision. +/// +/// Returns: +/// * `Output` sampled_candidates: A vector of length num_sampled, in which each element is +/// the ID of a sampled candidate. +/// * `Output` true_expected_count: A batch_size * num_true matrix, representing +/// the number of times each candidate is expected to occur in a batch +/// of sampled candidates. If unique=true, then this is a probability. +/// * `Output` sampled_expected_count: A vector of length num_sampled, for each sampled +/// candidate representing the number of times the candidate is expected +/// to occur in a batch of sampled candidates. If unique=true, then this is a +/// probability. +class AllCandidateSampler { + public: + /// Optional attribute setters for AllCandidateSampler + struct Attrs { + /// If either seed or seed2 are set to be non-zero, the random number + /// generator is seeded by the given seed. Otherwise, it is seeded by a + /// random seed. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Seed(int64 x) { + Attrs ret = *this; + ret.seed_ = x; + return ret; + } + + /// An second seed to avoid seed collision. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Seed2(int64 x) { + Attrs ret = *this; + ret.seed2_ = x; + return ret; + } + + int64 seed_ = 0; + int64 seed2_ = 0; + }; + AllCandidateSampler(const ::tensorflow::Scope& scope, ::tensorflow::Input + true_classes, int64 num_true, int64 num_sampled, bool + unique); + AllCandidateSampler(const ::tensorflow::Scope& scope, ::tensorflow::Input + true_classes, int64 num_true, int64 num_sampled, bool + unique, const AllCandidateSampler::Attrs& attrs); + + static Attrs Seed(int64 x) { + return Attrs().Seed(x); + } + static Attrs Seed2(int64 x) { + return Attrs().Seed2(x); + } + + Operation operation; + ::tensorflow::Output sampled_candidates; + ::tensorflow::Output true_expected_count; + ::tensorflow::Output sampled_expected_count; +}; + +/// Computes the ids of the positions in sampled_candidates that match true_labels. +/// +/// When doing log-odds NCE, the result of this op should be passed through a +/// SparseToDense op, then added to the logits of the sampled candidates. This has +/// the effect of 'removing' the sampled labels that match the true labels by +/// making the classifier sure that they are sampled labels. +/// +/// Arguments: +/// * scope: A Scope object +/// * true_classes: The true_classes output of UnpackSparseLabels. +/// * sampled_candidates: The sampled_candidates output of CandidateSampler. +/// * num_true: Number of true labels per context. +/// +/// Optional attributes (see `Attrs`): +/// * seed: If either seed or seed2 are set to be non-zero, the random number +/// generator is seeded by the given seed. Otherwise, it is seeded by a +/// random seed. +/// * seed2: An second seed to avoid seed collision. +/// +/// Returns: +/// * `Output` indices: A vector of indices corresponding to rows of true_candidates. +/// * `Output` ids: A vector of IDs of positions in sampled_candidates that match a true_label +/// for the row with the corresponding index in indices. +/// * `Output` weights: A vector of the same length as indices and ids, in which each element +/// is -FLOAT_MAX. +class ComputeAccidentalHits { + public: + /// Optional attribute setters for ComputeAccidentalHits + struct Attrs { + /// If either seed or seed2 are set to be non-zero, the random number + /// generator is seeded by the given seed. Otherwise, it is seeded by a + /// random seed. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Seed(int64 x) { + Attrs ret = *this; + ret.seed_ = x; + return ret; + } + + /// An second seed to avoid seed collision. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Seed2(int64 x) { + Attrs ret = *this; + ret.seed2_ = x; + return ret; + } + + int64 seed_ = 0; + int64 seed2_ = 0; + }; + ComputeAccidentalHits(const ::tensorflow::Scope& scope, ::tensorflow::Input + true_classes, ::tensorflow::Input sampled_candidates, + int64 num_true); + ComputeAccidentalHits(const ::tensorflow::Scope& scope, ::tensorflow::Input + true_classes, ::tensorflow::Input sampled_candidates, + int64 num_true, const ComputeAccidentalHits::Attrs& + attrs); + + static Attrs Seed(int64 x) { + return Attrs().Seed(x); + } + static Attrs Seed2(int64 x) { + return Attrs().Seed2(x); + } + + Operation operation; + ::tensorflow::Output indices; + ::tensorflow::Output ids; + ::tensorflow::Output weights; +}; + +/// Generates labels for candidate sampling with a learned unigram distribution. +/// +/// A unigram sampler could use a fixed unigram distribution read from a +/// file or passed in as an in-memory array instead of building up the distribution +/// from data on the fly. There is also an option to skew the distribution by +/// applying a distortion power to the weights. +/// +/// The vocabulary file should be in CSV-like format, with the last field +/// being the weight associated with the word. +/// +/// For each batch, this op picks a single set of sampled candidate labels. +/// +/// The advantages of sampling candidates per-batch are simplicity and the +/// possibility of efficient dense matrix multiplication. The disadvantage is that +/// the sampled candidates must be chosen independently of the context and of the +/// true labels. +/// +/// Arguments: +/// * scope: A Scope object +/// * true_classes: A batch_size * num_true matrix, in which each row contains the +/// IDs of the num_true target_classes in the corresponding original label. +/// * num_true: Number of true labels per context. +/// * num_sampled: Number of candidates to randomly sample. +/// * unique: If unique is true, we sample with rejection, so that all sampled +/// candidates in a batch are unique. This requires some approximation to +/// estimate the post-rejection sampling probabilities. +/// * range_max: The sampler will sample integers from the interval [0, range_max). +/// +/// Optional attributes (see `Attrs`): +/// * vocab_file: Each valid line in this file (which should have a CSV-like format) +/// corresponds to a valid word ID. IDs are in sequential order, starting from +/// num_reserved_ids. The last entry in each line is expected to be a value +/// corresponding to the count or relative probability. Exactly one of vocab_file +/// and unigrams needs to be passed to this op. +/// * distortion: The distortion is used to skew the unigram probability distribution. +/// Each weight is first raised to the distortion's power before adding to the +/// internal unigram distribution. As a result, distortion = 1.0 gives regular +/// unigram sampling (as defined by the vocab file), and distortion = 0.0 gives +/// a uniform distribution. +/// * num_reserved_ids: Optionally some reserved IDs can be added in the range [0, +/// ..., num_reserved_ids) by the users. One use case is that a special unknown +/// word token is used as ID 0. These IDs will have a sampling probability of 0. +/// * num_shards: A sampler can be used to sample from a subset of the original range +/// in order to speed up the whole computation through parallelism. This parameter +/// (together with 'shard') indicates the number of partitions that are being +/// used in the overall computation. +/// * shard: A sampler can be used to sample from a subset of the original range +/// in order to speed up the whole computation through parallelism. This parameter +/// (together with 'num_shards') indicates the particular partition number of a +/// sampler op, when partitioning is being used. +/// * unigrams: A list of unigram counts or probabilities, one per ID in sequential +/// order. Exactly one of vocab_file and unigrams should be passed to this op. +/// * seed: If either seed or seed2 are set to be non-zero, the random number +/// generator is seeded by the given seed. Otherwise, it is seeded by a +/// random seed. +/// * seed2: An second seed to avoid seed collision. +/// +/// Returns: +/// * `Output` sampled_candidates: A vector of length num_sampled, in which each element is +/// the ID of a sampled candidate. +/// * `Output` true_expected_count: A batch_size * num_true matrix, representing +/// the number of times each candidate is expected to occur in a batch +/// of sampled candidates. If unique=true, then this is a probability. +/// * `Output` sampled_expected_count: A vector of length num_sampled, for each sampled +/// candidate representing the number of times the candidate is expected +/// to occur in a batch of sampled candidates. If unique=true, then this is a +/// probability. +class FixedUnigramCandidateSampler { + public: + /// Optional attribute setters for FixedUnigramCandidateSampler + struct Attrs { + /// Each valid line in this file (which should have a CSV-like format) + /// corresponds to a valid word ID. IDs are in sequential order, starting from + /// num_reserved_ids. The last entry in each line is expected to be a value + /// corresponding to the count or relative probability. Exactly one of vocab_file + /// and unigrams needs to be passed to this op. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs VocabFile(StringPiece x) { + Attrs ret = *this; + ret.vocab_file_ = x; + return ret; + } + + /// The distortion is used to skew the unigram probability distribution. + /// Each weight is first raised to the distortion's power before adding to the + /// internal unigram distribution. As a result, distortion = 1.0 gives regular + /// unigram sampling (as defined by the vocab file), and distortion = 0.0 gives + /// a uniform distribution. + /// + /// Defaults to 1 + TF_MUST_USE_RESULT Attrs Distortion(float x) { + Attrs ret = *this; + ret.distortion_ = x; + return ret; + } + + /// Optionally some reserved IDs can be added in the range [0, + /// ..., num_reserved_ids) by the users. One use case is that a special unknown + /// word token is used as ID 0. These IDs will have a sampling probability of 0. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs NumReservedIds(int64 x) { + Attrs ret = *this; + ret.num_reserved_ids_ = x; + return ret; + } + + /// A sampler can be used to sample from a subset of the original range + /// in order to speed up the whole computation through parallelism. This parameter + /// (together with 'shard') indicates the number of partitions that are being + /// used in the overall computation. + /// + /// Defaults to 1 + TF_MUST_USE_RESULT Attrs NumShards(int64 x) { + Attrs ret = *this; + ret.num_shards_ = x; + return ret; + } + + /// A sampler can be used to sample from a subset of the original range + /// in order to speed up the whole computation through parallelism. This parameter + /// (together with 'num_shards') indicates the particular partition number of a + /// sampler op, when partitioning is being used. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Shard(int64 x) { + Attrs ret = *this; + ret.shard_ = x; + return ret; + } + + /// A list of unigram counts or probabilities, one per ID in sequential + /// order. Exactly one of vocab_file and unigrams should be passed to this op. + /// + /// Defaults to [] + TF_MUST_USE_RESULT Attrs Unigrams(const gtl::ArraySlice& x) { + Attrs ret = *this; + ret.unigrams_ = x; + return ret; + } + + /// If either seed or seed2 are set to be non-zero, the random number + /// generator is seeded by the given seed. Otherwise, it is seeded by a + /// random seed. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Seed(int64 x) { + Attrs ret = *this; + ret.seed_ = x; + return ret; + } + + /// An second seed to avoid seed collision. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Seed2(int64 x) { + Attrs ret = *this; + ret.seed2_ = x; + return ret; + } + + StringPiece vocab_file_ = ""; + float distortion_ = 1.0f; + int64 num_reserved_ids_ = 0; + int64 num_shards_ = 1; + int64 shard_ = 0; + gtl::ArraySlice unigrams_ = {}; + int64 seed_ = 0; + int64 seed2_ = 0; + }; + FixedUnigramCandidateSampler(const ::tensorflow::Scope& scope, + ::tensorflow::Input true_classes, int64 num_true, + int64 num_sampled, bool unique, int64 range_max); + FixedUnigramCandidateSampler(const ::tensorflow::Scope& scope, + ::tensorflow::Input true_classes, int64 num_true, + int64 num_sampled, bool unique, int64 range_max, + const FixedUnigramCandidateSampler::Attrs& attrs); + + static Attrs VocabFile(StringPiece x) { + return Attrs().VocabFile(x); + } + static Attrs Distortion(float x) { + return Attrs().Distortion(x); + } + static Attrs NumReservedIds(int64 x) { + return Attrs().NumReservedIds(x); + } + static Attrs NumShards(int64 x) { + return Attrs().NumShards(x); + } + static Attrs Shard(int64 x) { + return Attrs().Shard(x); + } + static Attrs Unigrams(const gtl::ArraySlice& x) { + return Attrs().Unigrams(x); + } + static Attrs Seed(int64 x) { + return Attrs().Seed(x); + } + static Attrs Seed2(int64 x) { + return Attrs().Seed2(x); + } + + Operation operation; + ::tensorflow::Output sampled_candidates; + ::tensorflow::Output true_expected_count; + ::tensorflow::Output sampled_expected_count; +}; + +/// Generates labels for candidate sampling with a learned unigram distribution. +/// +/// See explanations of candidate sampling and the data formats at +/// go/candidate-sampling. +/// +/// For each batch, this op picks a single set of sampled candidate labels. +/// +/// The advantages of sampling candidates per-batch are simplicity and the +/// possibility of efficient dense matrix multiplication. The disadvantage is that +/// the sampled candidates must be chosen independently of the context and of the +/// true labels. +/// +/// Arguments: +/// * scope: A Scope object +/// * true_classes: A batch_size * num_true matrix, in which each row contains the +/// IDs of the num_true target_classes in the corresponding original label. +/// * num_true: Number of true labels per context. +/// * num_sampled: Number of candidates to randomly sample. +/// * unique: If unique is true, we sample with rejection, so that all sampled +/// candidates in a batch are unique. This requires some approximation to +/// estimate the post-rejection sampling probabilities. +/// * range_max: The sampler will sample integers from the interval [0, range_max). +/// +/// Optional attributes (see `Attrs`): +/// * seed: If either seed or seed2 are set to be non-zero, the random number +/// generator is seeded by the given seed. Otherwise, it is seeded by a +/// random seed. +/// * seed2: An second seed to avoid seed collision. +/// +/// Returns: +/// * `Output` sampled_candidates: A vector of length num_sampled, in which each element is +/// the ID of a sampled candidate. +/// * `Output` true_expected_count: A batch_size * num_true matrix, representing +/// the number of times each candidate is expected to occur in a batch +/// of sampled candidates. If unique=true, then this is a probability. +/// * `Output` sampled_expected_count: A vector of length num_sampled, for each sampled +/// candidate representing the number of times the candidate is expected +/// to occur in a batch of sampled candidates. If unique=true, then this is a +/// probability. +class LearnedUnigramCandidateSampler { + public: + /// Optional attribute setters for LearnedUnigramCandidateSampler + struct Attrs { + /// If either seed or seed2 are set to be non-zero, the random number + /// generator is seeded by the given seed. Otherwise, it is seeded by a + /// random seed. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Seed(int64 x) { + Attrs ret = *this; + ret.seed_ = x; + return ret; + } + + /// An second seed to avoid seed collision. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Seed2(int64 x) { + Attrs ret = *this; + ret.seed2_ = x; + return ret; + } + + int64 seed_ = 0; + int64 seed2_ = 0; + }; + LearnedUnigramCandidateSampler(const ::tensorflow::Scope& scope, + ::tensorflow::Input true_classes, int64 + num_true, int64 num_sampled, bool unique, int64 + range_max); + LearnedUnigramCandidateSampler(const ::tensorflow::Scope& scope, + ::tensorflow::Input true_classes, int64 + num_true, int64 num_sampled, bool unique, int64 + range_max, const + LearnedUnigramCandidateSampler::Attrs& attrs); + + static Attrs Seed(int64 x) { + return Attrs().Seed(x); + } + static Attrs Seed2(int64 x) { + return Attrs().Seed2(x); + } + + Operation operation; + ::tensorflow::Output sampled_candidates; + ::tensorflow::Output true_expected_count; + ::tensorflow::Output sampled_expected_count; +}; + +/// Generates labels for candidate sampling with a log-uniform distribution. +/// +/// See explanations of candidate sampling and the data formats at +/// go/candidate-sampling. +/// +/// For each batch, this op picks a single set of sampled candidate labels. +/// +/// The advantages of sampling candidates per-batch are simplicity and the +/// possibility of efficient dense matrix multiplication. The disadvantage is that +/// the sampled candidates must be chosen independently of the context and of the +/// true labels. +/// +/// Arguments: +/// * scope: A Scope object +/// * true_classes: A batch_size * num_true matrix, in which each row contains the +/// IDs of the num_true target_classes in the corresponding original label. +/// * num_true: Number of true labels per context. +/// * num_sampled: Number of candidates to randomly sample. +/// * unique: If unique is true, we sample with rejection, so that all sampled +/// candidates in a batch are unique. This requires some approximation to +/// estimate the post-rejection sampling probabilities. +/// * range_max: The sampler will sample integers from the interval [0, range_max). +/// +/// Optional attributes (see `Attrs`): +/// * seed: If either seed or seed2 are set to be non-zero, the random number +/// generator is seeded by the given seed. Otherwise, it is seeded by a +/// random seed. +/// * seed2: An second seed to avoid seed collision. +/// +/// Returns: +/// * `Output` sampled_candidates: A vector of length num_sampled, in which each element is +/// the ID of a sampled candidate. +/// * `Output` true_expected_count: A batch_size * num_true matrix, representing +/// the number of times each candidate is expected to occur in a batch +/// of sampled candidates. If unique=true, then this is a probability. +/// * `Output` sampled_expected_count: A vector of length num_sampled, for each sampled +/// candidate representing the number of times the candidate is expected +/// to occur in a batch of sampled candidates. If unique=true, then this is a +/// probability. +class LogUniformCandidateSampler { + public: + /// Optional attribute setters for LogUniformCandidateSampler + struct Attrs { + /// If either seed or seed2 are set to be non-zero, the random number + /// generator is seeded by the given seed. Otherwise, it is seeded by a + /// random seed. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Seed(int64 x) { + Attrs ret = *this; + ret.seed_ = x; + return ret; + } + + /// An second seed to avoid seed collision. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Seed2(int64 x) { + Attrs ret = *this; + ret.seed2_ = x; + return ret; + } + + int64 seed_ = 0; + int64 seed2_ = 0; + }; + LogUniformCandidateSampler(const ::tensorflow::Scope& scope, + ::tensorflow::Input true_classes, int64 num_true, + int64 num_sampled, bool unique, int64 range_max); + LogUniformCandidateSampler(const ::tensorflow::Scope& scope, + ::tensorflow::Input true_classes, int64 num_true, + int64 num_sampled, bool unique, int64 range_max, + const LogUniformCandidateSampler::Attrs& attrs); + + static Attrs Seed(int64 x) { + return Attrs().Seed(x); + } + static Attrs Seed2(int64 x) { + return Attrs().Seed2(x); + } + + Operation operation; + ::tensorflow::Output sampled_candidates; + ::tensorflow::Output true_expected_count; + ::tensorflow::Output sampled_expected_count; +}; + +/// Generates labels for candidate sampling with a uniform distribution. +/// +/// See explanations of candidate sampling and the data formats at +/// go/candidate-sampling. +/// +/// For each batch, this op picks a single set of sampled candidate labels. +/// +/// The advantages of sampling candidates per-batch are simplicity and the +/// possibility of efficient dense matrix multiplication. The disadvantage is that +/// the sampled candidates must be chosen independently of the context and of the +/// true labels. +/// +/// Arguments: +/// * scope: A Scope object +/// * true_classes: A batch_size * num_true matrix, in which each row contains the +/// IDs of the num_true target_classes in the corresponding original label. +/// * num_true: Number of true labels per context. +/// * num_sampled: Number of candidates to randomly sample. +/// * unique: If unique is true, we sample with rejection, so that all sampled +/// candidates in a batch are unique. This requires some approximation to +/// estimate the post-rejection sampling probabilities. +/// * range_max: The sampler will sample integers from the interval [0, range_max). +/// +/// Optional attributes (see `Attrs`): +/// * seed: If either seed or seed2 are set to be non-zero, the random number +/// generator is seeded by the given seed. Otherwise, it is seeded by a +/// random seed. +/// * seed2: An second seed to avoid seed collision. +/// +/// Returns: +/// * `Output` sampled_candidates: A vector of length num_sampled, in which each element is +/// the ID of a sampled candidate. +/// * `Output` true_expected_count: A batch_size * num_true matrix, representing +/// the number of times each candidate is expected to occur in a batch +/// of sampled candidates. If unique=true, then this is a probability. +/// * `Output` sampled_expected_count: A vector of length num_sampled, for each sampled +/// candidate representing the number of times the candidate is expected +/// to occur in a batch of sampled candidates. If unique=true, then this is a +/// probability. +class UniformCandidateSampler { + public: + /// Optional attribute setters for UniformCandidateSampler + struct Attrs { + /// If either seed or seed2 are set to be non-zero, the random number + /// generator is seeded by the given seed. Otherwise, it is seeded by a + /// random seed. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Seed(int64 x) { + Attrs ret = *this; + ret.seed_ = x; + return ret; + } + + /// An second seed to avoid seed collision. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Seed2(int64 x) { + Attrs ret = *this; + ret.seed2_ = x; + return ret; + } + + int64 seed_ = 0; + int64 seed2_ = 0; + }; + UniformCandidateSampler(const ::tensorflow::Scope& scope, ::tensorflow::Input + true_classes, int64 num_true, int64 num_sampled, bool + unique, int64 range_max); + UniformCandidateSampler(const ::tensorflow::Scope& scope, ::tensorflow::Input + true_classes, int64 num_true, int64 num_sampled, bool + unique, int64 range_max, const + UniformCandidateSampler::Attrs& attrs); + + static Attrs Seed(int64 x) { + return Attrs().Seed(x); + } + static Attrs Seed2(int64 x) { + return Attrs().Seed2(x); + } + + Operation operation; + ::tensorflow::Output sampled_candidates; + ::tensorflow::Output true_expected_count; + ::tensorflow::Output sampled_expected_count; +}; + +/// @} + +} // namespace ops +} // namespace tensorflow + +#endif // TENSORFLOW_CC_OPS_CANDIDATE_SAMPLING_OPS_H_ diff --git a/linux/include/tensorflow/cc/ops/candidate_sampling_ops_internal.h b/linux/include/tensorflow/cc/ops/candidate_sampling_ops_internal.h new file mode 100644 index 000000000..e458b8688 --- /dev/null +++ b/linux/include/tensorflow/cc/ops/candidate_sampling_ops_internal.h @@ -0,0 +1,28 @@ +// This file is MACHINE GENERATED! Do not edit. + +#ifndef TENSORFLOW_CC_OPS_CANDIDATE_SAMPLING_OPS_INTERNAL_H_ +#define TENSORFLOW_CC_OPS_CANDIDATE_SAMPLING_OPS_INTERNAL_H_ + +// This file is MACHINE GENERATED! Do not edit. + +#include "tensorflow/cc/framework/ops.h" +#include "tensorflow/cc/framework/scope.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/gtl/array_slice.h" + +namespace tensorflow { +namespace ops { +namespace internal { +// NOTE: This namespace has internal TensorFlow details that +// are not part of TensorFlow's public API. + +/// @defgroup candidate_sampling_ops_internal Candidate Sampling Ops Internal +/// @{ + +} // namespace internal +} // namespace ops +} // namespace tensorflow + +#endif // TENSORFLOW_CC_OPS_CANDIDATE_SAMPLING_OPS_INTERNAL_H_ diff --git a/linux/include/tensorflow/cc/ops/const_op.h b/linux/include/tensorflow/cc/ops/const_op.h new file mode 100644 index 000000000..424a68366 --- /dev/null +++ b/linux/include/tensorflow/cc/ops/const_op.h @@ -0,0 +1,85 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CC_OPS_CONST_OP_H_ +#define TENSORFLOW_CC_OPS_CONST_OP_H_ + +#include "tensorflow/cc/framework/ops.h" +#include "tensorflow/cc/framework/scope.h" +#include "tensorflow/core/graph/node_builder.h" + +namespace tensorflow { +namespace ops { + +/// @defgroup const_op Const Op +/// @{ + +Output Const(const Scope& scope, const Input::Initializer& val); + +Output ConstFromProto(const Scope& scope, const TensorProto& proto); + +NodeBuilder::NodeOut AsNodeOut(const Scope& scope, const Input& inp); + +template +Output Const(const Scope& scope, const Input::Initializer& val) { + auto orig_const_output = Const(scope, val); + if (!scope.ok()) return Output(); + + typedef typename Input::Initializer::RealType::type DstT; + + if (val.tensor.dtype() == DataTypeToEnum::v()) { + return orig_const_output; + } + if (val.tensor.NumElements() == 0) { + Tensor t(DataTypeToEnum::v(), val.tensor.shape()); + return Const(scope, Input::Initializer(t)); + } + + // TODO(keveman): Refactor Cast op's kernel implementation such that the code + // can be directly called here instead of adding the Cast op to the graph. + auto orig_const = AsNodeOut(scope, orig_const_output); + const auto cast_op_name = scope.GetUniqueNameForOp("Cast"); + + auto cast_builder = NodeBuilder(cast_op_name, "Cast") + .Input(orig_const) + .Attr("DstT", DataTypeToEnum::v()); + scope.UpdateBuilder(&cast_builder); + Node* ret; + scope.UpdateStatus(cast_builder.Finalize(scope.graph(), &ret)); + if (!scope.ok()) return Output(); + scope.UpdateStatus(scope.DoShapeInference(ret)); + return Output(ret, 0); +} + +template +Output Const(const Scope& scope, const T& v, const TensorShape shape) { + return Const(scope, Input::Initializer(v, shape)); +} + +template +Output Const(const Scope& scope, const std::initializer_list& v, + const TensorShape shape) { + return Const(scope, Input::Initializer(v, shape)); +} + +std::vector AsNodeOutList(const Scope& scope, + const InputList& inp); + +/// }@ + +} // namespace ops +} // namespace tensorflow + +#endif // TENSORFLOW_CC_OPS_CONST_OP_H_ diff --git a/linux/include/tensorflow/cc/ops/control_flow_ops.h b/linux/include/tensorflow/cc/ops/control_flow_ops.h new file mode 100644 index 000000000..772a71557 --- /dev/null +++ b/linux/include/tensorflow/cc/ops/control_flow_ops.h @@ -0,0 +1,250 @@ +// This file is MACHINE GENERATED! Do not edit. + +#ifndef TENSORFLOW_CC_OPS_CONTROL_FLOW_OPS_H_ +#define TENSORFLOW_CC_OPS_CONTROL_FLOW_OPS_H_ + +// This file is MACHINE GENERATED! Do not edit. + +#include "tensorflow/cc/framework/ops.h" +#include "tensorflow/cc/framework/scope.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/gtl/array_slice.h" + +namespace tensorflow { +namespace ops { + +/// @defgroup control_flow_ops Control Flow Ops +/// @{ + +/// Raise a exception to abort the process when called. +/// +/// If exit_without_error is true, the process will exit normally, +/// otherwise it will exit with a SIGABORT signal. +/// +/// Returns nothing but an exception. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Optional attributes (see `Attrs`): +/// * error_msg: A string which is the message associated with the exception. +/// +/// Returns: +/// * the created `Operation` +class Abort { + public: + /// Optional attribute setters for Abort + struct Attrs { + /// A string which is the message associated with the exception. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs ErrorMsg(StringPiece x) { + Attrs ret = *this; + ret.error_msg_ = x; + return ret; + } + + /// Defaults to false + TF_MUST_USE_RESULT Attrs ExitWithoutError(bool x) { + Attrs ret = *this; + ret.exit_without_error_ = x; + return ret; + } + + StringPiece error_msg_ = ""; + bool exit_without_error_ = false; + }; + Abort(const ::tensorflow::Scope& scope); + Abort(const ::tensorflow::Scope& scope, const Abort::Attrs& attrs); + operator ::tensorflow::Operation() const { return operation; } + + static Attrs ErrorMsg(StringPiece x) { + return Attrs().ErrorMsg(x); + } + static Attrs ExitWithoutError(bool x) { + return Attrs().ExitWithoutError(x); + } + + Operation operation; +}; + +/// Does nothing. Serves as a control trigger for scheduling. +/// +/// Only useful as a placeholder for control edges. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * the created `Operation` +class ControlTrigger { + public: + ControlTrigger(const ::tensorflow::Scope& scope); + operator ::tensorflow::Operation() const { return operation; } + + Operation operation; +}; + +/// Forwards the input to the output. +/// +/// This operator represents the loop termination condition used by the +/// "pivot" switches of a loop. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: A boolean scalar, representing the branch predicate of the Switch op. +/// +/// Returns: +/// * `Output`: The same tensor as `input`. +class LoopCond { + public: + LoopCond(const ::tensorflow::Scope& scope, ::tensorflow::Input input); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Forwards the value of an available tensor from `inputs` to `output`. +/// +/// `Merge` waits for at least one of the tensors in `inputs` to become available. +/// It is usually combined with `Switch` to implement branching. +/// +/// `Merge` forwards the first tensor to become available to `output`, and sets +/// `value_index` to its index in `inputs`. +/// +/// Arguments: +/// * scope: A Scope object +/// * inputs: The input tensors, exactly one of which will become available. +/// +/// Returns: +/// * `Output` output: Will be set to the available input tensor. +/// * `Output` value_index: The index of the chosen input tensor in `inputs`. +class Merge { + public: + Merge(const ::tensorflow::Scope& scope, ::tensorflow::InputList inputs); + + Operation operation; + ::tensorflow::Output output; + ::tensorflow::Output value_index; +}; + +/// Makes its input available to the next iteration. +/// +/// Arguments: +/// * scope: A Scope object +/// * data: The tensor to be made available to the next iteration. +/// +/// Returns: +/// * `Output`: The same tensor as `data`. +class NextIteration { + public: + NextIteration(const ::tensorflow::Scope& scope, ::tensorflow::Input data); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Makes its input available to the next iteration. +/// +/// Arguments: +/// * scope: A Scope object +/// * data: The tensor to be made available to the next iteration. +/// +/// Returns: +/// * `Output`: The same tensor as `data`. +class RefNextIteration { + public: + RefNextIteration(const ::tensorflow::Scope& scope, ::tensorflow::Input data); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Forwards the `index`th element of `inputs` to `output`. +/// +/// Arguments: +/// * scope: A Scope object +/// * index: A scalar that determines the input that gets selected. +/// * inputs: A list of ref tensors, one of which will be forwarded to `output`. +/// +/// Returns: +/// * `Output`: The forwarded tensor. +class RefSelect { + public: + RefSelect(const ::tensorflow::Scope& scope, ::tensorflow::Input index, + ::tensorflow::InputList inputs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Forwards the ref tensor `data` to the output port determined by `pred`. +/// +/// If `pred` is true, the `data` input is forwarded to `output_true`. Otherwise, +/// the data goes to `output_false`. +/// +/// See also `Switch` and `Merge`. +/// +/// Arguments: +/// * scope: A Scope object +/// * data: The ref tensor to be forwarded to the appropriate output. +/// * pred: A scalar that specifies which output port will receive data. +/// +/// Returns: +/// * `Output` output_false: If `pred` is false, data will be forwarded to this output. +/// * `Output` output_true: If `pred` is true, data will be forwarded to this output. +class RefSwitch { + public: + RefSwitch(const ::tensorflow::Scope& scope, ::tensorflow::Input data, + ::tensorflow::Input pred); + + Operation operation; + ::tensorflow::Output output_false; + ::tensorflow::Output output_true; +}; + +/// Forwards `data` to the output port determined by `pred`. +/// +/// If `pred` is true, the `data` input is forwarded to `output_true`. Otherwise, +/// the data goes to `output_false`. +/// +/// See also `RefSwitch` and `Merge`. +/// +/// Arguments: +/// * scope: A Scope object +/// * data: The tensor to be forwarded to the appropriate output. +/// * pred: A scalar that specifies which output port will receive data. +/// +/// Returns: +/// * `Output` output_false: If `pred` is false, data will be forwarded to this output. +/// * `Output` output_true: If `pred` is true, data will be forwarded to this output. +class Switch { + public: + Switch(const ::tensorflow::Scope& scope, ::tensorflow::Input data, + ::tensorflow::Input pred); + + Operation operation; + ::tensorflow::Output output_false; + ::tensorflow::Output output_true; +}; + +/// @} + +} // namespace ops +} // namespace tensorflow + +#endif // TENSORFLOW_CC_OPS_CONTROL_FLOW_OPS_H_ diff --git a/linux/include/tensorflow/cc/ops/control_flow_ops_internal.h b/linux/include/tensorflow/cc/ops/control_flow_ops_internal.h new file mode 100644 index 000000000..59548de54 --- /dev/null +++ b/linux/include/tensorflow/cc/ops/control_flow_ops_internal.h @@ -0,0 +1,219 @@ +// This file is MACHINE GENERATED! Do not edit. + +#ifndef TENSORFLOW_CC_OPS_CONTROL_FLOW_OPS_INTERNAL_H_ +#define TENSORFLOW_CC_OPS_CONTROL_FLOW_OPS_INTERNAL_H_ + +// This file is MACHINE GENERATED! Do not edit. + +#include "tensorflow/cc/framework/ops.h" +#include "tensorflow/cc/framework/scope.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/gtl/array_slice.h" + +namespace tensorflow { +namespace ops { +namespace internal { +// NOTE: This namespace has internal TensorFlow details that +// are not part of TensorFlow's public API. + +/// @defgroup control_flow_ops_internal Control Flow Ops Internal +/// @{ + +/// Creates or finds a child frame, and makes `data` available to the child frame. +/// +/// This op is used together with `Exit` to create loops in the graph. +/// The unique `frame_name` is used by the `Executor` to identify frames. If +/// `is_constant` is true, `output` is a constant in the child frame; otherwise +/// it may be changed in the child frame. At most `parallel_iterations` iterations +/// are run in parallel in the child frame. +/// +/// Arguments: +/// * scope: A Scope object +/// * data: The tensor to be made available to the child frame. +/// * frame_name: The name of the child frame. +/// +/// Optional attributes (see `Attrs`): +/// * is_constant: If true, the output is constant within the child frame. +/// * parallel_iterations: The number of iterations allowed to run in parallel. +/// +/// Returns: +/// * `Output`: The same tensor as `data`. +class Enter { + public: + /// Optional attribute setters for Enter + struct Attrs { + /// If true, the output is constant within the child frame. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs IsConstant(bool x) { + Attrs ret = *this; + ret.is_constant_ = x; + return ret; + } + + /// The number of iterations allowed to run in parallel. + /// + /// Defaults to 10 + TF_MUST_USE_RESULT Attrs ParallelIterations(int64 x) { + Attrs ret = *this; + ret.parallel_iterations_ = x; + return ret; + } + + bool is_constant_ = false; + int64 parallel_iterations_ = 10; + }; + Enter(const ::tensorflow::Scope& scope, ::tensorflow::Input data, StringPiece + frame_name); + Enter(const ::tensorflow::Scope& scope, ::tensorflow::Input data, StringPiece + frame_name, const Enter::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs IsConstant(bool x) { + return Attrs().IsConstant(x); + } + static Attrs ParallelIterations(int64 x) { + return Attrs().ParallelIterations(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Exits the current frame to its parent frame. +/// +/// Exit makes its input `data` available to the parent frame. +/// +/// Arguments: +/// * scope: A Scope object +/// * data: The tensor to be made available to the parent frame. +/// +/// Returns: +/// * `Output`: The same tensor as `data`. +class Exit { + public: + Exit(const ::tensorflow::Scope& scope, ::tensorflow::Input data); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Creates or finds a child frame, and makes `data` available to the child frame. +/// +/// The unique `frame_name` is used by the `Executor` to identify frames. If +/// `is_constant` is true, `output` is a constant in the child frame; otherwise +/// it may be changed in the child frame. At most `parallel_iterations` iterations +/// are run in parallel in the child frame. +/// +/// Arguments: +/// * scope: A Scope object +/// * data: The tensor to be made available to the child frame. +/// * frame_name: The name of the child frame. +/// +/// Optional attributes (see `Attrs`): +/// * is_constant: If true, the output is constant within the child frame. +/// * parallel_iterations: The number of iterations allowed to run in parallel. +/// +/// Returns: +/// * `Output`: The same tensor as `data`. +class RefEnter { + public: + /// Optional attribute setters for RefEnter + struct Attrs { + /// If true, the output is constant within the child frame. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs IsConstant(bool x) { + Attrs ret = *this; + ret.is_constant_ = x; + return ret; + } + + /// The number of iterations allowed to run in parallel. + /// + /// Defaults to 10 + TF_MUST_USE_RESULT Attrs ParallelIterations(int64 x) { + Attrs ret = *this; + ret.parallel_iterations_ = x; + return ret; + } + + bool is_constant_ = false; + int64 parallel_iterations_ = 10; + }; + RefEnter(const ::tensorflow::Scope& scope, ::tensorflow::Input data, + StringPiece frame_name); + RefEnter(const ::tensorflow::Scope& scope, ::tensorflow::Input data, + StringPiece frame_name, const RefEnter::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs IsConstant(bool x) { + return Attrs().IsConstant(x); + } + static Attrs ParallelIterations(int64 x) { + return Attrs().ParallelIterations(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Exits the current frame to its parent frame. +/// +/// Exit makes its input `data` available to the parent frame. +/// +/// Arguments: +/// * scope: A Scope object +/// * data: The tensor to be made available to the parent frame. +/// +/// Returns: +/// * `Output`: The same tensor as `data`. +class RefExit { + public: + RefExit(const ::tensorflow::Scope& scope, ::tensorflow::Input data); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Forwards the value of an available tensor from `inputs` to `output`. +/// +/// `Merge` waits for at least one of the tensors in `inputs` to become available. +/// It is usually combined with `Switch` to implement branching. +/// +/// `Merge` forwards the first tensor for become available to `output`, and sets +/// `value_index` to its index in `inputs`. +/// +/// Arguments: +/// * scope: A Scope object +/// * inputs: The input tensors, exactly one of which will become available. +/// +/// Returns: +/// * `Output` output: Will be set to the available input tensor. +/// * `Output` value_index: The index of the chosen input tensor in `inputs`. +class RefMerge { + public: + RefMerge(const ::tensorflow::Scope& scope, ::tensorflow::InputList inputs); + + Operation operation; + ::tensorflow::Output output; + ::tensorflow::Output value_index; +}; + +} // namespace internal +} // namespace ops +} // namespace tensorflow + +#endif // TENSORFLOW_CC_OPS_CONTROL_FLOW_OPS_INTERNAL_H_ diff --git a/linux/include/tensorflow/cc/ops/data_flow_ops.h b/linux/include/tensorflow/cc/ops/data_flow_ops.h new file mode 100644 index 000000000..3d6bc7451 --- /dev/null +++ b/linux/include/tensorflow/cc/ops/data_flow_ops.h @@ -0,0 +1,3780 @@ +// This file is MACHINE GENERATED! Do not edit. + +#ifndef TENSORFLOW_CC_OPS_DATA_FLOW_OPS_H_ +#define TENSORFLOW_CC_OPS_DATA_FLOW_OPS_H_ + +// This file is MACHINE GENERATED! Do not edit. + +#include "tensorflow/cc/framework/ops.h" +#include "tensorflow/cc/framework/scope.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/gtl/array_slice.h" + +namespace tensorflow { +namespace ops { + +/// @defgroup data_flow_ops Data Flow Ops +/// @{ + +/// Applies a gradient to a given accumulator. +/// +/// Does not add if local_step is lesser than the accumulator's global_step. +/// +/// Arguments: +/// * scope: A Scope object +/// * handle: The handle to a accumulator. +/// * local_step: The local_step value at which the gradient was computed. +/// * gradient: A tensor of the gradient to be accumulated. +/// +/// Returns: +/// * the created `Operation` +class AccumulatorApplyGradient { + public: + AccumulatorApplyGradient(const ::tensorflow::Scope& scope, ::tensorflow::Input + handle, ::tensorflow::Input local_step, + ::tensorflow::Input gradient); + operator ::tensorflow::Operation() const { return operation; } + + Operation operation; +}; + +/// Returns the number of gradients aggregated in the given accumulators. +/// +/// Arguments: +/// * scope: A Scope object +/// * handle: The handle to an accumulator. +/// +/// Returns: +/// * `Output`: The number of gradients aggregated in the given accumulator. +class AccumulatorNumAccumulated { + public: + AccumulatorNumAccumulated(const ::tensorflow::Scope& scope, ::tensorflow::Input + handle); + operator ::tensorflow::Output() const { return num_accumulated; } + operator ::tensorflow::Input() const { return num_accumulated; } + ::tensorflow::Node* node() const { return num_accumulated.node(); } + + Operation operation; + ::tensorflow::Output num_accumulated; +}; + +/// Updates the accumulator with a new value for global_step. +/// +/// Logs warning if the accumulator's value is already higher than +/// new_global_step. +/// +/// Arguments: +/// * scope: A Scope object +/// * handle: The handle to an accumulator. +/// * new_global_step: The new global_step value to set. +/// +/// Returns: +/// * the created `Operation` +class AccumulatorSetGlobalStep { + public: + AccumulatorSetGlobalStep(const ::tensorflow::Scope& scope, ::tensorflow::Input + handle, ::tensorflow::Input new_global_step); + operator ::tensorflow::Operation() const { return operation; } + + Operation operation; +}; + +/// Extracts the average gradient in the given ConditionalAccumulator. +/// +/// The op blocks until sufficient (i.e., more than num_required) +/// gradients have been accumulated. If the accumulator has already +/// aggregated more than num_required gradients, it returns the average of +/// the accumulated gradients. Also automatically increments the recorded +/// global_step in the accumulator by 1, and resets the aggregate to 0. +/// +/// Arguments: +/// * scope: A Scope object +/// * handle: The handle to an accumulator. +/// * num_required: Number of gradients required before we return an aggregate. +/// * dtype: The data type of accumulated gradients. Needs to correspond to the type +/// of the accumulator. +/// +/// Returns: +/// * `Output`: The average of the accumulated gradients. +class AccumulatorTakeGradient { + public: + AccumulatorTakeGradient(const ::tensorflow::Scope& scope, ::tensorflow::Input + handle, ::tensorflow::Input num_required, DataType + dtype); + operator ::tensorflow::Output() const { return average; } + operator ::tensorflow::Input() const { return average; } + ::tensorflow::Node* node() const { return average.node(); } + + Operation operation; + ::tensorflow::Output average; +}; + +/// Defines a barrier that persists across different graph executions. +/// +/// A barrier represents a key-value map, where each key is a string, and +/// each value is a tuple of tensors. +/// +/// At runtime, the barrier contains 'complete' and 'incomplete' +/// elements. A complete element has defined tensors for all components of +/// its value tuple, and may be accessed using BarrierTakeMany. An +/// incomplete element has some undefined components in its value tuple, +/// and may be updated using BarrierInsertMany. +/// +/// Arguments: +/// * scope: A Scope object +/// * component_types: The type of each component in a value. +/// +/// Optional attributes (see `Attrs`): +/// * shapes: The shape of each component in a value. Each shape must be 1 in the +/// first dimension. The length of this attr must be the same as the length of +/// component_types. +/// * capacity: The capacity of the barrier. The default capacity is MAX_INT32, +/// which is the largest capacity of the underlying queue. +/// * container: If non-empty, this barrier is placed in the given container. +/// Otherwise, a default container is used. +/// * shared_name: If non-empty, this barrier will be shared under the given name +/// across multiple sessions. +/// +/// Returns: +/// * `Output`: The handle to the barrier. +class Barrier { + public: + /// Optional attribute setters for Barrier + struct Attrs { + /// The shape of each component in a value. Each shape must be 1 in the + /// first dimension. The length of this attr must be the same as the length of + /// component_types. + /// + /// Defaults to [] + TF_MUST_USE_RESULT Attrs Shapes(const gtl::ArraySlice& x) { + Attrs ret = *this; + ret.shapes_ = x; + return ret; + } + + /// The capacity of the barrier. The default capacity is MAX_INT32, + /// which is the largest capacity of the underlying queue. + /// + /// Defaults to -1 + TF_MUST_USE_RESULT Attrs Capacity(int64 x) { + Attrs ret = *this; + ret.capacity_ = x; + return ret; + } + + /// If non-empty, this barrier is placed in the given container. + /// Otherwise, a default container is used. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs Container(StringPiece x) { + Attrs ret = *this; + ret.container_ = x; + return ret; + } + + /// If non-empty, this barrier will be shared under the given name + /// across multiple sessions. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs SharedName(StringPiece x) { + Attrs ret = *this; + ret.shared_name_ = x; + return ret; + } + + gtl::ArraySlice shapes_ = {}; + int64 capacity_ = -1; + StringPiece container_ = ""; + StringPiece shared_name_ = ""; + }; + Barrier(const ::tensorflow::Scope& scope, const DataTypeSlice& component_types); + Barrier(const ::tensorflow::Scope& scope, const DataTypeSlice& component_types, + const Barrier::Attrs& attrs); + operator ::tensorflow::Output() const { return handle; } + operator ::tensorflow::Input() const { return handle; } + ::tensorflow::Node* node() const { return handle.node(); } + + static Attrs Shapes(const gtl::ArraySlice& x) { + return Attrs().Shapes(x); + } + static Attrs Capacity(int64 x) { + return Attrs().Capacity(x); + } + static Attrs Container(StringPiece x) { + return Attrs().Container(x); + } + static Attrs SharedName(StringPiece x) { + return Attrs().SharedName(x); + } + + Operation operation; + ::tensorflow::Output handle; +}; + +/// Closes the given barrier. +/// +/// This operation signals that no more new elements will be inserted in the +/// given barrier. Subsequent InsertMany that try to introduce a new key will fail. +/// Subsequent InsertMany operations that just add missing components to already +/// existing elements will continue to succeed. Subsequent TakeMany operations will +/// continue to succeed if sufficient completed elements remain in the barrier. +/// Subsequent TakeMany operations that would block will fail immediately. +/// +/// Arguments: +/// * scope: A Scope object +/// * handle: The handle to a barrier. +/// +/// Optional attributes (see `Attrs`): +/// * cancel_pending_enqueues: If true, all pending enqueue requests that are +/// blocked on the barrier's queue will be canceled. InsertMany will fail, even +/// if no new key is introduced. +/// +/// Returns: +/// * the created `Operation` +class BarrierClose { + public: + /// Optional attribute setters for BarrierClose + struct Attrs { + /// If true, all pending enqueue requests that are + /// blocked on the barrier's queue will be canceled. InsertMany will fail, even + /// if no new key is introduced. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs CancelPendingEnqueues(bool x) { + Attrs ret = *this; + ret.cancel_pending_enqueues_ = x; + return ret; + } + + bool cancel_pending_enqueues_ = false; + }; + BarrierClose(const ::tensorflow::Scope& scope, ::tensorflow::Input handle); + BarrierClose(const ::tensorflow::Scope& scope, ::tensorflow::Input handle, + const BarrierClose::Attrs& attrs); + operator ::tensorflow::Operation() const { return operation; } + + static Attrs CancelPendingEnqueues(bool x) { + return Attrs().CancelPendingEnqueues(x); + } + + Operation operation; +}; + +/// Computes the number of incomplete elements in the given barrier. +/// +/// Arguments: +/// * scope: A Scope object +/// * handle: The handle to a barrier. +/// +/// Returns: +/// * `Output`: The number of incomplete elements (i.e. those with some of their value +/// components not set) in the barrier. +class BarrierIncompleteSize { + public: + BarrierIncompleteSize(const ::tensorflow::Scope& scope, ::tensorflow::Input + handle); + operator ::tensorflow::Output() const { return size; } + operator ::tensorflow::Input() const { return size; } + ::tensorflow::Node* node() const { return size.node(); } + + Operation operation; + ::tensorflow::Output size; +}; + +/// For each key, assigns the respective value to the specified component. +/// +/// If a key is not found in the barrier, this operation will create a new +/// incomplete element. If a key is found in the barrier, and the element +/// already has a value at component_index, this operation will fail with +/// INVALID_ARGUMENT, and leave the barrier in an undefined state. +/// +/// Arguments: +/// * scope: A Scope object +/// * handle: The handle to a barrier. +/// * keys: A one-dimensional tensor of keys, with length n. +/// * values: An any-dimensional tensor of values, which are associated with the +/// respective keys. The 0th dimension must have length n. +/// * component_index: The component of the barrier elements that is being assigned. +/// +/// Returns: +/// * the created `Operation` +class BarrierInsertMany { + public: + BarrierInsertMany(const ::tensorflow::Scope& scope, ::tensorflow::Input handle, + ::tensorflow::Input keys, ::tensorflow::Input values, int64 + component_index); + operator ::tensorflow::Operation() const { return operation; } + + Operation operation; +}; + +/// Computes the number of complete elements in the given barrier. +/// +/// Arguments: +/// * scope: A Scope object +/// * handle: The handle to a barrier. +/// +/// Returns: +/// * `Output`: The number of complete elements (i.e. those with all of their value +/// components set) in the barrier. +class BarrierReadySize { + public: + BarrierReadySize(const ::tensorflow::Scope& scope, ::tensorflow::Input handle); + operator ::tensorflow::Output() const { return size; } + operator ::tensorflow::Input() const { return size; } + ::tensorflow::Node* node() const { return size.node(); } + + Operation operation; + ::tensorflow::Output size; +}; + +/// Takes the given number of completed elements from a barrier. +/// +/// This operation concatenates completed-element component tensors along +/// the 0th dimension to make a single component tensor. +/// +/// Elements come out of the barrier when they are complete, and in the order +/// in which they were placed into the barrier. The indices output provides +/// information about the batch in which each element was originally inserted +/// into the barrier. +/// +/// Arguments: +/// * scope: A Scope object +/// * handle: The handle to a barrier. +/// * num_elements: A single-element tensor containing the number of elements to +/// take. +/// * component_types: The type of each component in a value. +/// +/// Optional attributes (see `Attrs`): +/// * allow_small_batch: Allow to return less than num_elements items if barrier is +/// already closed. +/// * timeout_ms: If the queue is empty, this operation will block for up to +/// timeout_ms milliseconds. +/// Note: This option is not supported yet. +/// +/// Returns: +/// * `Output` indices: A one-dimensional tensor of indices, with length num_elems. +/// These indices refer to the batch in which the values were placed into the +/// barrier (starting with MIN_LONG and increasing with each BarrierInsertMany). +/// * `Output` keys: A one-dimensional tensor of keys, with length num_elements. +/// * `OutputList` values: One any-dimensional tensor per component in a barrier element. All +/// values have length num_elements in the 0th dimension. +class BarrierTakeMany { + public: + /// Optional attribute setters for BarrierTakeMany + struct Attrs { + /// Allow to return less than num_elements items if barrier is + /// already closed. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs AllowSmallBatch(bool x) { + Attrs ret = *this; + ret.allow_small_batch_ = x; + return ret; + } + + /// Defaults to false + TF_MUST_USE_RESULT Attrs WaitForIncomplete(bool x) { + Attrs ret = *this; + ret.wait_for_incomplete_ = x; + return ret; + } + + /// If the queue is empty, this operation will block for up to + /// timeout_ms milliseconds. + /// Note: This option is not supported yet. + /// + /// Defaults to -1 + TF_MUST_USE_RESULT Attrs TimeoutMs(int64 x) { + Attrs ret = *this; + ret.timeout_ms_ = x; + return ret; + } + + bool allow_small_batch_ = false; + bool wait_for_incomplete_ = false; + int64 timeout_ms_ = -1; + }; + BarrierTakeMany(const ::tensorflow::Scope& scope, ::tensorflow::Input handle, + ::tensorflow::Input num_elements, const DataTypeSlice& + component_types); + BarrierTakeMany(const ::tensorflow::Scope& scope, ::tensorflow::Input handle, + ::tensorflow::Input num_elements, const DataTypeSlice& + component_types, const BarrierTakeMany::Attrs& attrs); + + static Attrs AllowSmallBatch(bool x) { + return Attrs().AllowSmallBatch(x); + } + static Attrs WaitForIncomplete(bool x) { + return Attrs().WaitForIncomplete(x); + } + static Attrs TimeoutMs(int64 x) { + return Attrs().TimeoutMs(x); + } + + Operation operation; + ::tensorflow::Output indices; + ::tensorflow::Output keys; + ::tensorflow::OutputList values; +}; + +/// A conditional accumulator for aggregating gradients. +/// +/// The accumulator accepts gradients marked with local_step greater or +/// equal to the most recent global_step known to the accumulator. The +/// average can be extracted from the accumulator, provided sufficient +/// gradients have been accumulated. Extracting the average automatically +/// resets the aggregate to 0, and increments the global_step recorded by +/// the accumulator. +/// +/// Arguments: +/// * scope: A Scope object +/// * dtype: The type of the value being accumulated. +/// * shape: The shape of the values, can be [], in which case shape is unknown. +/// +/// Optional attributes (see `Attrs`): +/// * container: If non-empty, this accumulator is placed in the given container. +/// Otherwise, a default container is used. +/// * shared_name: If non-empty, this accumulator will be shared under the +/// given name across multiple sessions. +/// +/// Returns: +/// * `Output`: The handle to the accumulator. +class ConditionalAccumulator { + public: + /// Optional attribute setters for ConditionalAccumulator + struct Attrs { + /// If non-empty, this accumulator is placed in the given container. + /// Otherwise, a default container is used. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs Container(StringPiece x) { + Attrs ret = *this; + ret.container_ = x; + return ret; + } + + /// If non-empty, this accumulator will be shared under the + /// given name across multiple sessions. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs SharedName(StringPiece x) { + Attrs ret = *this; + ret.shared_name_ = x; + return ret; + } + + /// Defaults to "MEAN" + TF_MUST_USE_RESULT Attrs ReductionType(StringPiece x) { + Attrs ret = *this; + ret.reduction_type_ = x; + return ret; + } + + StringPiece container_ = ""; + StringPiece shared_name_ = ""; + StringPiece reduction_type_ = "MEAN"; + }; + ConditionalAccumulator(const ::tensorflow::Scope& scope, DataType dtype, + PartialTensorShape shape); + ConditionalAccumulator(const ::tensorflow::Scope& scope, DataType dtype, + PartialTensorShape shape, const + ConditionalAccumulator::Attrs& attrs); + operator ::tensorflow::Output() const { return handle; } + operator ::tensorflow::Input() const { return handle; } + ::tensorflow::Node* node() const { return handle.node(); } + + static Attrs Container(StringPiece x) { + return Attrs().Container(x); + } + static Attrs SharedName(StringPiece x) { + return Attrs().SharedName(x); + } + static Attrs ReductionType(StringPiece x) { + return Attrs().ReductionType(x); + } + + Operation operation; + ::tensorflow::Output handle; +}; + +/// Delete the tensor specified by its handle in the session. +/// +/// Arguments: +/// * scope: A Scope object +/// * handle: The handle for a tensor stored in the session state. +/// +/// Returns: +/// * the created `Operation` +class DeleteSessionTensor { + public: + DeleteSessionTensor(const ::tensorflow::Scope& scope, ::tensorflow::Input + handle); + operator ::tensorflow::Operation() const { return operation; } + + Operation operation; +}; + +/// Partitions `data` into `num_partitions` tensors using indices from `partitions`. +/// +/// For each index tuple `js` of size `partitions.ndim`, the slice `data[js, ...]` +/// becomes part of `outputs[partitions[js]]`. The slices with `partitions[js] = i` +/// are placed in `outputs[i]` in lexicographic order of `js`, and the first +/// dimension of `outputs[i]` is the number of entries in `partitions` equal to `i`. +/// In detail, +/// +/// ```python +/// outputs[i].shape = [sum(partitions == i)] + data.shape[partitions.ndim:] +/// +/// outputs[i] = pack([data[js, ...] for js if partitions[js] == i]) +/// ``` +/// +/// `data.shape` must start with `partitions.shape`. +/// +/// For example: +/// +/// ```python +/// # Scalar partitions. +/// partitions = 1 +/// num_partitions = 2 +/// data = [10, 20] +/// outputs[0] = [] # Empty with shape [0, 2] +/// outputs[1] = [[10, 20]] +/// +/// # Vector partitions. +/// partitions = [0, 0, 1, 1, 0] +/// num_partitions = 2 +/// data = [10, 20, 30, 40, 50] +/// outputs[0] = [10, 20, 50] +/// outputs[1] = [30, 40] +/// ``` +/// +/// See `dynamic_stitch` for an example on how to merge partitions back. +/// +///
+/// +///
+/// +/// Arguments: +/// * scope: A Scope object +/// * partitions: Any shape. Indices in the range `[0, num_partitions)`. +/// * num_partitions: The number of partitions to output. +/// +/// Returns: +/// * `OutputList`: The outputs tensor. +class DynamicPartition { + public: + DynamicPartition(const ::tensorflow::Scope& scope, ::tensorflow::Input data, + ::tensorflow::Input partitions, int64 num_partitions); + ::tensorflow::Output operator[](size_t index) const { return outputs[index]; } + + + Operation operation; + ::tensorflow::OutputList outputs; +}; + +/// Interleave the values from the `data` tensors into a single tensor. +/// +/// Builds a merged tensor such that +/// +/// ```python +/// merged[indices[m][i, ..., j], ...] = data[m][i, ..., j, ...] +/// ``` +/// +/// For example, if each `indices[m]` is scalar or vector, we have +/// +/// ```python +/// # Scalar indices: +/// merged[indices[m], ...] = data[m][...] +/// +/// # Vector indices: +/// merged[indices[m][i], ...] = data[m][i, ...] +/// ``` +/// +/// Each `data[i].shape` must start with the corresponding `indices[i].shape`, +/// and the rest of `data[i].shape` must be constant w.r.t. `i`. That is, we +/// must have `data[i].shape = indices[i].shape + constant`. In terms of this +/// `constant`, the output shape is +/// +/// merged.shape = [max(indices)] + constant +/// +/// Values are merged in order, so if an index appears in both `indices[m][i]` and +/// `indices[n][j]` for `(m,i) < (n,j)` the slice `data[n][j]` will appear in the +/// merged result. If you do not need this guarantee, ParallelDynamicStitch might +/// perform better on some devices. +/// +/// For example: +/// +/// ```python +/// indices[0] = 6 +/// indices[1] = [4, 1] +/// indices[2] = [[5, 2], [0, 3]] +/// data[0] = [61, 62] +/// data[1] = [[41, 42], [11, 12]] +/// data[2] = [[[51, 52], [21, 22]], [[1, 2], [31, 32]]] +/// merged = [[1, 2], [11, 12], [21, 22], [31, 32], [41, 42], +/// [51, 52], [61, 62]] +/// ``` +/// +/// This method can be used to merge partitions created by `dynamic_partition` +/// as illustrated on the following example: +/// +/// ```python +/// # Apply function (increments x_i) on elements for which a certain condition +/// # apply (x_i != -1 in this example). +/// x=tf.constant([0.1, -1., 5.2, 4.3, -1., 7.4]) +/// condition_mask=tf.not_equal(x,tf.constant(-1.)) +/// partitioned_data = tf.dynamic_partition( +/// x, tf.cast(condition_mask, tf.int32) , 2) +/// partitioned_data[1] = partitioned_data[1] + 1.0 +/// condition_indices = tf.dynamic_partition( +/// tf.range(tf.shape(x)[0]), tf.cast(condition_mask, tf.int32) , 2) +/// x = tf.dynamic_stitch(condition_indices, partitioned_data) +/// # Here x=[1.1, -1., 6.2, 5.3, -1, 8.4], the -1. values remain +/// # unchanged. +/// ``` +/// +///
+/// +///
+/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The merged tensor. +class DynamicStitch { + public: + DynamicStitch(const ::tensorflow::Scope& scope, ::tensorflow::InputList + indices, ::tensorflow::InputList data); + operator ::tensorflow::Output() const { return merged; } + operator ::tensorflow::Input() const { return merged; } + ::tensorflow::Node* node() const { return merged.node(); } + + Operation operation; + ::tensorflow::Output merged; +}; + +/// A queue that produces elements in first-in first-out order. +/// +/// Arguments: +/// * scope: A Scope object +/// * component_types: The type of each component in a value. +/// +/// Optional attributes (see `Attrs`): +/// * shapes: The shape of each component in a value. The length of this attr must +/// be either 0 or the same as the length of component_types. If the length of +/// this attr is 0, the shapes of queue elements are not constrained, and +/// only one element may be dequeued at a time. +/// * capacity: The upper bound on the number of elements in this queue. +/// Negative numbers mean no limit. +/// * container: If non-empty, this queue is placed in the given container. +/// Otherwise, a default container is used. +/// * shared_name: If non-empty, this queue will be shared under the given name +/// across multiple sessions. +/// +/// Returns: +/// * `Output`: The handle to the queue. +class FIFOQueue { + public: + /// Optional attribute setters for FIFOQueue + struct Attrs { + /// The shape of each component in a value. The length of this attr must + /// be either 0 or the same as the length of component_types. If the length of + /// this attr is 0, the shapes of queue elements are not constrained, and + /// only one element may be dequeued at a time. + /// + /// Defaults to [] + TF_MUST_USE_RESULT Attrs Shapes(const gtl::ArraySlice& x) { + Attrs ret = *this; + ret.shapes_ = x; + return ret; + } + + /// The upper bound on the number of elements in this queue. + /// Negative numbers mean no limit. + /// + /// Defaults to -1 + TF_MUST_USE_RESULT Attrs Capacity(int64 x) { + Attrs ret = *this; + ret.capacity_ = x; + return ret; + } + + /// If non-empty, this queue is placed in the given container. + /// Otherwise, a default container is used. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs Container(StringPiece x) { + Attrs ret = *this; + ret.container_ = x; + return ret; + } + + /// If non-empty, this queue will be shared under the given name + /// across multiple sessions. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs SharedName(StringPiece x) { + Attrs ret = *this; + ret.shared_name_ = x; + return ret; + } + + gtl::ArraySlice shapes_ = {}; + int64 capacity_ = -1; + StringPiece container_ = ""; + StringPiece shared_name_ = ""; + }; + FIFOQueue(const ::tensorflow::Scope& scope, const DataTypeSlice& + component_types); + FIFOQueue(const ::tensorflow::Scope& scope, const DataTypeSlice& + component_types, const FIFOQueue::Attrs& attrs); + operator ::tensorflow::Output() const { return handle; } + operator ::tensorflow::Input() const { return handle; } + ::tensorflow::Node* node() const { return handle.node(); } + + static Attrs Shapes(const gtl::ArraySlice& x) { + return Attrs().Shapes(x); + } + static Attrs Capacity(int64 x) { + return Attrs().Capacity(x); + } + static Attrs Container(StringPiece x) { + return Attrs().Container(x); + } + static Attrs SharedName(StringPiece x) { + return Attrs().SharedName(x); + } + + Operation operation; + ::tensorflow::Output handle; +}; + +/// Store the input tensor in the state of the current session. +/// +/// Arguments: +/// * scope: A Scope object +/// * value: The tensor to be stored. +/// +/// Returns: +/// * `Output`: The handle for the tensor stored in the session state, represented +/// as a string. +class GetSessionHandle { + public: + GetSessionHandle(const ::tensorflow::Scope& scope, ::tensorflow::Input value); + operator ::tensorflow::Output() const { return handle; } + operator ::tensorflow::Input() const { return handle; } + ::tensorflow::Node* node() const { return handle.node(); } + + Operation operation; + ::tensorflow::Output handle; +}; + +/// Store the input tensor in the state of the current session. +/// +/// Arguments: +/// * scope: A Scope object +/// * value: The tensor to be stored. +/// +/// Returns: +/// * `Output`: The handle for the tensor stored in the session state, represented +/// as a ResourceHandle object. +class GetSessionHandleV2 { + public: + GetSessionHandleV2(const ::tensorflow::Scope& scope, ::tensorflow::Input value); + operator ::tensorflow::Output() const { return handle; } + operator ::tensorflow::Input() const { return handle; } + ::tensorflow::Node* node() const { return handle.node(); } + + Operation operation; + ::tensorflow::Output handle; +}; + +/// Get the value of the tensor specified by its handle. +/// +/// Arguments: +/// * scope: A Scope object +/// * handle: The handle for a tensor stored in the session state. +/// * dtype: The type of the output value. +/// +/// Returns: +/// * `Output`: The tensor for the given handle. +class GetSessionTensor { + public: + GetSessionTensor(const ::tensorflow::Scope& scope, ::tensorflow::Input handle, + DataType dtype); + operator ::tensorflow::Output() const { return value; } + operator ::tensorflow::Input() const { return value; } + ::tensorflow::Node* node() const { return value.node(); } + + Operation operation; + ::tensorflow::Output value; +}; + +/// Op removes all elements in the underlying container. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * the created `Operation` +class MapClear { + public: + /// Optional attribute setters for MapClear + struct Attrs { + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Capacity(int64 x) { + Attrs ret = *this; + ret.capacity_ = x; + return ret; + } + + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs MemoryLimit(int64 x) { + Attrs ret = *this; + ret.memory_limit_ = x; + return ret; + } + + /// Defaults to "" + TF_MUST_USE_RESULT Attrs Container(StringPiece x) { + Attrs ret = *this; + ret.container_ = x; + return ret; + } + + /// Defaults to "" + TF_MUST_USE_RESULT Attrs SharedName(StringPiece x) { + Attrs ret = *this; + ret.shared_name_ = x; + return ret; + } + + int64 capacity_ = 0; + int64 memory_limit_ = 0; + StringPiece container_ = ""; + StringPiece shared_name_ = ""; + }; + MapClear(const ::tensorflow::Scope& scope, const DataTypeSlice& dtypes); + MapClear(const ::tensorflow::Scope& scope, const DataTypeSlice& dtypes, const + MapClear::Attrs& attrs); + operator ::tensorflow::Operation() const { return operation; } + + static Attrs Capacity(int64 x) { + return Attrs().Capacity(x); + } + static Attrs MemoryLimit(int64 x) { + return Attrs().MemoryLimit(x); + } + static Attrs Container(StringPiece x) { + return Attrs().Container(x); + } + static Attrs SharedName(StringPiece x) { + return Attrs().SharedName(x); + } + + Operation operation; +}; + +/// Op returns the number of incomplete elements in the underlying container. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The size tensor. +class MapIncompleteSize { + public: + /// Optional attribute setters for MapIncompleteSize + struct Attrs { + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Capacity(int64 x) { + Attrs ret = *this; + ret.capacity_ = x; + return ret; + } + + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs MemoryLimit(int64 x) { + Attrs ret = *this; + ret.memory_limit_ = x; + return ret; + } + + /// Defaults to "" + TF_MUST_USE_RESULT Attrs Container(StringPiece x) { + Attrs ret = *this; + ret.container_ = x; + return ret; + } + + /// Defaults to "" + TF_MUST_USE_RESULT Attrs SharedName(StringPiece x) { + Attrs ret = *this; + ret.shared_name_ = x; + return ret; + } + + int64 capacity_ = 0; + int64 memory_limit_ = 0; + StringPiece container_ = ""; + StringPiece shared_name_ = ""; + }; + MapIncompleteSize(const ::tensorflow::Scope& scope, const DataTypeSlice& + dtypes); + MapIncompleteSize(const ::tensorflow::Scope& scope, const DataTypeSlice& + dtypes, const MapIncompleteSize::Attrs& attrs); + operator ::tensorflow::Output() const { return size; } + operator ::tensorflow::Input() const { return size; } + ::tensorflow::Node* node() const { return size.node(); } + + static Attrs Capacity(int64 x) { + return Attrs().Capacity(x); + } + static Attrs MemoryLimit(int64 x) { + return Attrs().MemoryLimit(x); + } + static Attrs Container(StringPiece x) { + return Attrs().Container(x); + } + static Attrs SharedName(StringPiece x) { + return Attrs().SharedName(x); + } + + Operation operation; + ::tensorflow::Output size; +}; + +/// Op peeks at the values at the specified key. If the +/// +/// underlying container does not contain this key +/// this op will block until it does. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `OutputList`: The values tensor. +class MapPeek { + public: + /// Optional attribute setters for MapPeek + struct Attrs { + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Capacity(int64 x) { + Attrs ret = *this; + ret.capacity_ = x; + return ret; + } + + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs MemoryLimit(int64 x) { + Attrs ret = *this; + ret.memory_limit_ = x; + return ret; + } + + /// Defaults to "" + TF_MUST_USE_RESULT Attrs Container(StringPiece x) { + Attrs ret = *this; + ret.container_ = x; + return ret; + } + + /// Defaults to "" + TF_MUST_USE_RESULT Attrs SharedName(StringPiece x) { + Attrs ret = *this; + ret.shared_name_ = x; + return ret; + } + + int64 capacity_ = 0; + int64 memory_limit_ = 0; + StringPiece container_ = ""; + StringPiece shared_name_ = ""; + }; + MapPeek(const ::tensorflow::Scope& scope, ::tensorflow::Input key, + ::tensorflow::Input indices, const DataTypeSlice& dtypes); + MapPeek(const ::tensorflow::Scope& scope, ::tensorflow::Input key, + ::tensorflow::Input indices, const DataTypeSlice& dtypes, const + MapPeek::Attrs& attrs); + ::tensorflow::Output operator[](size_t index) const { return values[index]; } + + + static Attrs Capacity(int64 x) { + return Attrs().Capacity(x); + } + static Attrs MemoryLimit(int64 x) { + return Attrs().MemoryLimit(x); + } + static Attrs Container(StringPiece x) { + return Attrs().Container(x); + } + static Attrs SharedName(StringPiece x) { + return Attrs().SharedName(x); + } + + Operation operation; + ::tensorflow::OutputList values; +}; + +/// Op returns the number of elements in the underlying container. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The size tensor. +class MapSize { + public: + /// Optional attribute setters for MapSize + struct Attrs { + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Capacity(int64 x) { + Attrs ret = *this; + ret.capacity_ = x; + return ret; + } + + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs MemoryLimit(int64 x) { + Attrs ret = *this; + ret.memory_limit_ = x; + return ret; + } + + /// Defaults to "" + TF_MUST_USE_RESULT Attrs Container(StringPiece x) { + Attrs ret = *this; + ret.container_ = x; + return ret; + } + + /// Defaults to "" + TF_MUST_USE_RESULT Attrs SharedName(StringPiece x) { + Attrs ret = *this; + ret.shared_name_ = x; + return ret; + } + + int64 capacity_ = 0; + int64 memory_limit_ = 0; + StringPiece container_ = ""; + StringPiece shared_name_ = ""; + }; + MapSize(const ::tensorflow::Scope& scope, const DataTypeSlice& dtypes); + MapSize(const ::tensorflow::Scope& scope, const DataTypeSlice& dtypes, const + MapSize::Attrs& attrs); + operator ::tensorflow::Output() const { return size; } + operator ::tensorflow::Input() const { return size; } + ::tensorflow::Node* node() const { return size.node(); } + + static Attrs Capacity(int64 x) { + return Attrs().Capacity(x); + } + static Attrs MemoryLimit(int64 x) { + return Attrs().MemoryLimit(x); + } + static Attrs Container(StringPiece x) { + return Attrs().Container(x); + } + static Attrs SharedName(StringPiece x) { + return Attrs().SharedName(x); + } + + Operation operation; + ::tensorflow::Output size; +}; + +/// Stage (key, values) in the underlying container which behaves like a hashtable. +/// +/// Arguments: +/// * scope: A Scope object +/// * key: int64 +/// * values: a list of tensors +/// dtypes A list of data types that inserted values should adhere to. +/// +/// Optional attributes (see `Attrs`): +/// * capacity: Maximum number of elements in the Staging Area. If > 0, inserts +/// on the container will block when the capacity is reached. +/// * container: If non-empty, this queue is placed in the given container. Otherwise, +/// a default container is used. +/// * shared_name: It is necessary to match this name to the matching Unstage Op. +/// +/// Returns: +/// * the created `Operation` +class MapStage { + public: + /// Optional attribute setters for MapStage + struct Attrs { + /// Maximum number of elements in the Staging Area. If > 0, inserts + /// on the container will block when the capacity is reached. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Capacity(int64 x) { + Attrs ret = *this; + ret.capacity_ = x; + return ret; + } + + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs MemoryLimit(int64 x) { + Attrs ret = *this; + ret.memory_limit_ = x; + return ret; + } + + /// If non-empty, this queue is placed in the given container. Otherwise, + /// a default container is used. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs Container(StringPiece x) { + Attrs ret = *this; + ret.container_ = x; + return ret; + } + + /// It is necessary to match this name to the matching Unstage Op. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs SharedName(StringPiece x) { + Attrs ret = *this; + ret.shared_name_ = x; + return ret; + } + + int64 capacity_ = 0; + int64 memory_limit_ = 0; + StringPiece container_ = ""; + StringPiece shared_name_ = ""; + }; + MapStage(const ::tensorflow::Scope& scope, ::tensorflow::Input key, + ::tensorflow::Input indices, ::tensorflow::InputList values, const + DataTypeSlice& dtypes); + MapStage(const ::tensorflow::Scope& scope, ::tensorflow::Input key, + ::tensorflow::Input indices, ::tensorflow::InputList values, const + DataTypeSlice& dtypes, const MapStage::Attrs& attrs); + operator ::tensorflow::Operation() const { return operation; } + + static Attrs Capacity(int64 x) { + return Attrs().Capacity(x); + } + static Attrs MemoryLimit(int64 x) { + return Attrs().MemoryLimit(x); + } + static Attrs Container(StringPiece x) { + return Attrs().Container(x); + } + static Attrs SharedName(StringPiece x) { + return Attrs().SharedName(x); + } + + Operation operation; +}; + +/// Op removes and returns the values associated with the key +/// +/// from the underlying container. If the underlying container +/// does not contain this key, the op will block until it does. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `OutputList`: The values tensor. +class MapUnstage { + public: + /// Optional attribute setters for MapUnstage + struct Attrs { + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Capacity(int64 x) { + Attrs ret = *this; + ret.capacity_ = x; + return ret; + } + + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs MemoryLimit(int64 x) { + Attrs ret = *this; + ret.memory_limit_ = x; + return ret; + } + + /// Defaults to "" + TF_MUST_USE_RESULT Attrs Container(StringPiece x) { + Attrs ret = *this; + ret.container_ = x; + return ret; + } + + /// Defaults to "" + TF_MUST_USE_RESULT Attrs SharedName(StringPiece x) { + Attrs ret = *this; + ret.shared_name_ = x; + return ret; + } + + int64 capacity_ = 0; + int64 memory_limit_ = 0; + StringPiece container_ = ""; + StringPiece shared_name_ = ""; + }; + MapUnstage(const ::tensorflow::Scope& scope, ::tensorflow::Input key, + ::tensorflow::Input indices, const DataTypeSlice& dtypes); + MapUnstage(const ::tensorflow::Scope& scope, ::tensorflow::Input key, + ::tensorflow::Input indices, const DataTypeSlice& dtypes, const + MapUnstage::Attrs& attrs); + ::tensorflow::Output operator[](size_t index) const { return values[index]; } + + + static Attrs Capacity(int64 x) { + return Attrs().Capacity(x); + } + static Attrs MemoryLimit(int64 x) { + return Attrs().MemoryLimit(x); + } + static Attrs Container(StringPiece x) { + return Attrs().Container(x); + } + static Attrs SharedName(StringPiece x) { + return Attrs().SharedName(x); + } + + Operation operation; + ::tensorflow::OutputList values; +}; + +/// Op removes and returns a random (key, value) +/// +/// from the underlying container. If the underlying container +/// does not contain elements, the op will block until it does. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output` key +/// * `OutputList` values +class MapUnstageNoKey { + public: + /// Optional attribute setters for MapUnstageNoKey + struct Attrs { + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Capacity(int64 x) { + Attrs ret = *this; + ret.capacity_ = x; + return ret; + } + + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs MemoryLimit(int64 x) { + Attrs ret = *this; + ret.memory_limit_ = x; + return ret; + } + + /// Defaults to "" + TF_MUST_USE_RESULT Attrs Container(StringPiece x) { + Attrs ret = *this; + ret.container_ = x; + return ret; + } + + /// Defaults to "" + TF_MUST_USE_RESULT Attrs SharedName(StringPiece x) { + Attrs ret = *this; + ret.shared_name_ = x; + return ret; + } + + int64 capacity_ = 0; + int64 memory_limit_ = 0; + StringPiece container_ = ""; + StringPiece shared_name_ = ""; + }; + MapUnstageNoKey(const ::tensorflow::Scope& scope, ::tensorflow::Input indices, + const DataTypeSlice& dtypes); + MapUnstageNoKey(const ::tensorflow::Scope& scope, ::tensorflow::Input indices, + const DataTypeSlice& dtypes, const MapUnstageNoKey::Attrs& + attrs); + + static Attrs Capacity(int64 x) { + return Attrs().Capacity(x); + } + static Attrs MemoryLimit(int64 x) { + return Attrs().MemoryLimit(x); + } + static Attrs Container(StringPiece x) { + return Attrs().Container(x); + } + static Attrs SharedName(StringPiece x) { + return Attrs().SharedName(x); + } + + Operation operation; + ::tensorflow::Output key; + ::tensorflow::OutputList values; +}; + +/// Op removes all elements in the underlying container. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * the created `Operation` +class OrderedMapClear { + public: + /// Optional attribute setters for OrderedMapClear + struct Attrs { + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Capacity(int64 x) { + Attrs ret = *this; + ret.capacity_ = x; + return ret; + } + + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs MemoryLimit(int64 x) { + Attrs ret = *this; + ret.memory_limit_ = x; + return ret; + } + + /// Defaults to "" + TF_MUST_USE_RESULT Attrs Container(StringPiece x) { + Attrs ret = *this; + ret.container_ = x; + return ret; + } + + /// Defaults to "" + TF_MUST_USE_RESULT Attrs SharedName(StringPiece x) { + Attrs ret = *this; + ret.shared_name_ = x; + return ret; + } + + int64 capacity_ = 0; + int64 memory_limit_ = 0; + StringPiece container_ = ""; + StringPiece shared_name_ = ""; + }; + OrderedMapClear(const ::tensorflow::Scope& scope, const DataTypeSlice& dtypes); + OrderedMapClear(const ::tensorflow::Scope& scope, const DataTypeSlice& dtypes, + const OrderedMapClear::Attrs& attrs); + operator ::tensorflow::Operation() const { return operation; } + + static Attrs Capacity(int64 x) { + return Attrs().Capacity(x); + } + static Attrs MemoryLimit(int64 x) { + return Attrs().MemoryLimit(x); + } + static Attrs Container(StringPiece x) { + return Attrs().Container(x); + } + static Attrs SharedName(StringPiece x) { + return Attrs().SharedName(x); + } + + Operation operation; +}; + +/// Op returns the number of incomplete elements in the underlying container. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The size tensor. +class OrderedMapIncompleteSize { + public: + /// Optional attribute setters for OrderedMapIncompleteSize + struct Attrs { + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Capacity(int64 x) { + Attrs ret = *this; + ret.capacity_ = x; + return ret; + } + + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs MemoryLimit(int64 x) { + Attrs ret = *this; + ret.memory_limit_ = x; + return ret; + } + + /// Defaults to "" + TF_MUST_USE_RESULT Attrs Container(StringPiece x) { + Attrs ret = *this; + ret.container_ = x; + return ret; + } + + /// Defaults to "" + TF_MUST_USE_RESULT Attrs SharedName(StringPiece x) { + Attrs ret = *this; + ret.shared_name_ = x; + return ret; + } + + int64 capacity_ = 0; + int64 memory_limit_ = 0; + StringPiece container_ = ""; + StringPiece shared_name_ = ""; + }; + OrderedMapIncompleteSize(const ::tensorflow::Scope& scope, const DataTypeSlice& + dtypes); + OrderedMapIncompleteSize(const ::tensorflow::Scope& scope, const DataTypeSlice& + dtypes, const OrderedMapIncompleteSize::Attrs& attrs); + operator ::tensorflow::Output() const { return size; } + operator ::tensorflow::Input() const { return size; } + ::tensorflow::Node* node() const { return size.node(); } + + static Attrs Capacity(int64 x) { + return Attrs().Capacity(x); + } + static Attrs MemoryLimit(int64 x) { + return Attrs().MemoryLimit(x); + } + static Attrs Container(StringPiece x) { + return Attrs().Container(x); + } + static Attrs SharedName(StringPiece x) { + return Attrs().SharedName(x); + } + + Operation operation; + ::tensorflow::Output size; +}; + +/// Op peeks at the values at the specified key. If the +/// +/// underlying container does not contain this key +/// this op will block until it does. This Op is optimized for +/// performance. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `OutputList`: The values tensor. +class OrderedMapPeek { + public: + /// Optional attribute setters for OrderedMapPeek + struct Attrs { + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Capacity(int64 x) { + Attrs ret = *this; + ret.capacity_ = x; + return ret; + } + + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs MemoryLimit(int64 x) { + Attrs ret = *this; + ret.memory_limit_ = x; + return ret; + } + + /// Defaults to "" + TF_MUST_USE_RESULT Attrs Container(StringPiece x) { + Attrs ret = *this; + ret.container_ = x; + return ret; + } + + /// Defaults to "" + TF_MUST_USE_RESULT Attrs SharedName(StringPiece x) { + Attrs ret = *this; + ret.shared_name_ = x; + return ret; + } + + int64 capacity_ = 0; + int64 memory_limit_ = 0; + StringPiece container_ = ""; + StringPiece shared_name_ = ""; + }; + OrderedMapPeek(const ::tensorflow::Scope& scope, ::tensorflow::Input key, + ::tensorflow::Input indices, const DataTypeSlice& dtypes); + OrderedMapPeek(const ::tensorflow::Scope& scope, ::tensorflow::Input key, + ::tensorflow::Input indices, const DataTypeSlice& dtypes, const + OrderedMapPeek::Attrs& attrs); + ::tensorflow::Output operator[](size_t index) const { return values[index]; } + + + static Attrs Capacity(int64 x) { + return Attrs().Capacity(x); + } + static Attrs MemoryLimit(int64 x) { + return Attrs().MemoryLimit(x); + } + static Attrs Container(StringPiece x) { + return Attrs().Container(x); + } + static Attrs SharedName(StringPiece x) { + return Attrs().SharedName(x); + } + + Operation operation; + ::tensorflow::OutputList values; +}; + +/// Op returns the number of elements in the underlying container. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The size tensor. +class OrderedMapSize { + public: + /// Optional attribute setters for OrderedMapSize + struct Attrs { + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Capacity(int64 x) { + Attrs ret = *this; + ret.capacity_ = x; + return ret; + } + + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs MemoryLimit(int64 x) { + Attrs ret = *this; + ret.memory_limit_ = x; + return ret; + } + + /// Defaults to "" + TF_MUST_USE_RESULT Attrs Container(StringPiece x) { + Attrs ret = *this; + ret.container_ = x; + return ret; + } + + /// Defaults to "" + TF_MUST_USE_RESULT Attrs SharedName(StringPiece x) { + Attrs ret = *this; + ret.shared_name_ = x; + return ret; + } + + int64 capacity_ = 0; + int64 memory_limit_ = 0; + StringPiece container_ = ""; + StringPiece shared_name_ = ""; + }; + OrderedMapSize(const ::tensorflow::Scope& scope, const DataTypeSlice& dtypes); + OrderedMapSize(const ::tensorflow::Scope& scope, const DataTypeSlice& dtypes, + const OrderedMapSize::Attrs& attrs); + operator ::tensorflow::Output() const { return size; } + operator ::tensorflow::Input() const { return size; } + ::tensorflow::Node* node() const { return size.node(); } + + static Attrs Capacity(int64 x) { + return Attrs().Capacity(x); + } + static Attrs MemoryLimit(int64 x) { + return Attrs().MemoryLimit(x); + } + static Attrs Container(StringPiece x) { + return Attrs().Container(x); + } + static Attrs SharedName(StringPiece x) { + return Attrs().SharedName(x); + } + + Operation operation; + ::tensorflow::Output size; +}; + +/// Stage (key, values) in the underlying container which behaves like a ordered +/// +/// associative container. Elements are ordered by key. +/// +/// Arguments: +/// * scope: A Scope object +/// * key: int64 +/// * values: a list of tensors +/// dtypes A list of data types that inserted values should adhere to. +/// +/// Optional attributes (see `Attrs`): +/// * capacity: Maximum number of elements in the Staging Area. If > 0, inserts +/// on the container will block when the capacity is reached. +/// * container: If non-empty, this queue is placed in the given container. Otherwise, +/// a default container is used. +/// * shared_name: It is necessary to match this name to the matching Unstage Op. +/// +/// Returns: +/// * the created `Operation` +class OrderedMapStage { + public: + /// Optional attribute setters for OrderedMapStage + struct Attrs { + /// Maximum number of elements in the Staging Area. If > 0, inserts + /// on the container will block when the capacity is reached. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Capacity(int64 x) { + Attrs ret = *this; + ret.capacity_ = x; + return ret; + } + + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs MemoryLimit(int64 x) { + Attrs ret = *this; + ret.memory_limit_ = x; + return ret; + } + + /// If non-empty, this queue is placed in the given container. Otherwise, + /// a default container is used. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs Container(StringPiece x) { + Attrs ret = *this; + ret.container_ = x; + return ret; + } + + /// It is necessary to match this name to the matching Unstage Op. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs SharedName(StringPiece x) { + Attrs ret = *this; + ret.shared_name_ = x; + return ret; + } + + int64 capacity_ = 0; + int64 memory_limit_ = 0; + StringPiece container_ = ""; + StringPiece shared_name_ = ""; + }; + OrderedMapStage(const ::tensorflow::Scope& scope, ::tensorflow::Input key, + ::tensorflow::Input indices, ::tensorflow::InputList values, + const DataTypeSlice& dtypes); + OrderedMapStage(const ::tensorflow::Scope& scope, ::tensorflow::Input key, + ::tensorflow::Input indices, ::tensorflow::InputList values, + const DataTypeSlice& dtypes, const OrderedMapStage::Attrs& + attrs); + operator ::tensorflow::Operation() const { return operation; } + + static Attrs Capacity(int64 x) { + return Attrs().Capacity(x); + } + static Attrs MemoryLimit(int64 x) { + return Attrs().MemoryLimit(x); + } + static Attrs Container(StringPiece x) { + return Attrs().Container(x); + } + static Attrs SharedName(StringPiece x) { + return Attrs().SharedName(x); + } + + Operation operation; +}; + +/// Op removes and returns the values associated with the key +/// +/// from the underlying container. If the underlying container +/// does not contain this key, the op will block until it does. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `OutputList`: The values tensor. +class OrderedMapUnstage { + public: + /// Optional attribute setters for OrderedMapUnstage + struct Attrs { + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Capacity(int64 x) { + Attrs ret = *this; + ret.capacity_ = x; + return ret; + } + + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs MemoryLimit(int64 x) { + Attrs ret = *this; + ret.memory_limit_ = x; + return ret; + } + + /// Defaults to "" + TF_MUST_USE_RESULT Attrs Container(StringPiece x) { + Attrs ret = *this; + ret.container_ = x; + return ret; + } + + /// Defaults to "" + TF_MUST_USE_RESULT Attrs SharedName(StringPiece x) { + Attrs ret = *this; + ret.shared_name_ = x; + return ret; + } + + int64 capacity_ = 0; + int64 memory_limit_ = 0; + StringPiece container_ = ""; + StringPiece shared_name_ = ""; + }; + OrderedMapUnstage(const ::tensorflow::Scope& scope, ::tensorflow::Input key, + ::tensorflow::Input indices, const DataTypeSlice& dtypes); + OrderedMapUnstage(const ::tensorflow::Scope& scope, ::tensorflow::Input key, + ::tensorflow::Input indices, const DataTypeSlice& dtypes, + const OrderedMapUnstage::Attrs& attrs); + ::tensorflow::Output operator[](size_t index) const { return values[index]; } + + + static Attrs Capacity(int64 x) { + return Attrs().Capacity(x); + } + static Attrs MemoryLimit(int64 x) { + return Attrs().MemoryLimit(x); + } + static Attrs Container(StringPiece x) { + return Attrs().Container(x); + } + static Attrs SharedName(StringPiece x) { + return Attrs().SharedName(x); + } + + Operation operation; + ::tensorflow::OutputList values; +}; + +/// Op removes and returns the (key, value) element with the smallest +/// +/// key from the underlying container. If the underlying container +/// does not contain elements, the op will block until it does. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output` key +/// * `OutputList` values +class OrderedMapUnstageNoKey { + public: + /// Optional attribute setters for OrderedMapUnstageNoKey + struct Attrs { + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Capacity(int64 x) { + Attrs ret = *this; + ret.capacity_ = x; + return ret; + } + + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs MemoryLimit(int64 x) { + Attrs ret = *this; + ret.memory_limit_ = x; + return ret; + } + + /// Defaults to "" + TF_MUST_USE_RESULT Attrs Container(StringPiece x) { + Attrs ret = *this; + ret.container_ = x; + return ret; + } + + /// Defaults to "" + TF_MUST_USE_RESULT Attrs SharedName(StringPiece x) { + Attrs ret = *this; + ret.shared_name_ = x; + return ret; + } + + int64 capacity_ = 0; + int64 memory_limit_ = 0; + StringPiece container_ = ""; + StringPiece shared_name_ = ""; + }; + OrderedMapUnstageNoKey(const ::tensorflow::Scope& scope, ::tensorflow::Input + indices, const DataTypeSlice& dtypes); + OrderedMapUnstageNoKey(const ::tensorflow::Scope& scope, ::tensorflow::Input + indices, const DataTypeSlice& dtypes, const + OrderedMapUnstageNoKey::Attrs& attrs); + + static Attrs Capacity(int64 x) { + return Attrs().Capacity(x); + } + static Attrs MemoryLimit(int64 x) { + return Attrs().MemoryLimit(x); + } + static Attrs Container(StringPiece x) { + return Attrs().Container(x); + } + static Attrs SharedName(StringPiece x) { + return Attrs().SharedName(x); + } + + Operation operation; + ::tensorflow::Output key; + ::tensorflow::OutputList values; +}; + +/// A queue that produces elements in first-in first-out order. +/// +/// Variable-size shapes are allowed by setting the corresponding shape dimensions +/// to 0 in the shape attr. In this case DequeueMany will pad up to the maximum +/// size of any given element in the minibatch. See below for details. +/// +/// Arguments: +/// * scope: A Scope object +/// * component_types: The type of each component in a value. +/// +/// Optional attributes (see `Attrs`): +/// * shapes: The shape of each component in a value. The length of this attr must +/// be either 0 or the same as the length of component_types. +/// Shapes of fixed rank but variable size are allowed by setting +/// any shape dimension to -1. In this case, the inputs' shape may vary along +/// the given dimension, and DequeueMany will pad the given dimension with +/// zeros up to the maximum shape of all elements in the given batch. +/// If the length of this attr is 0, different queue elements may have +/// different ranks and shapes, but only one element may be dequeued at a time. +/// * capacity: The upper bound on the number of elements in this queue. +/// Negative numbers mean no limit. +/// * container: If non-empty, this queue is placed in the given container. +/// Otherwise, a default container is used. +/// * shared_name: If non-empty, this queue will be shared under the given name +/// across multiple sessions. +/// +/// Returns: +/// * `Output`: The handle to the queue. +class PaddingFIFOQueue { + public: + /// Optional attribute setters for PaddingFIFOQueue + struct Attrs { + /// The shape of each component in a value. The length of this attr must + /// be either 0 or the same as the length of component_types. + /// Shapes of fixed rank but variable size are allowed by setting + /// any shape dimension to -1. In this case, the inputs' shape may vary along + /// the given dimension, and DequeueMany will pad the given dimension with + /// zeros up to the maximum shape of all elements in the given batch. + /// If the length of this attr is 0, different queue elements may have + /// different ranks and shapes, but only one element may be dequeued at a time. + /// + /// Defaults to [] + TF_MUST_USE_RESULT Attrs Shapes(const gtl::ArraySlice& x) { + Attrs ret = *this; + ret.shapes_ = x; + return ret; + } + + /// The upper bound on the number of elements in this queue. + /// Negative numbers mean no limit. + /// + /// Defaults to -1 + TF_MUST_USE_RESULT Attrs Capacity(int64 x) { + Attrs ret = *this; + ret.capacity_ = x; + return ret; + } + + /// If non-empty, this queue is placed in the given container. + /// Otherwise, a default container is used. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs Container(StringPiece x) { + Attrs ret = *this; + ret.container_ = x; + return ret; + } + + /// If non-empty, this queue will be shared under the given name + /// across multiple sessions. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs SharedName(StringPiece x) { + Attrs ret = *this; + ret.shared_name_ = x; + return ret; + } + + gtl::ArraySlice shapes_ = {}; + int64 capacity_ = -1; + StringPiece container_ = ""; + StringPiece shared_name_ = ""; + }; + PaddingFIFOQueue(const ::tensorflow::Scope& scope, const DataTypeSlice& + component_types); + PaddingFIFOQueue(const ::tensorflow::Scope& scope, const DataTypeSlice& + component_types, const PaddingFIFOQueue::Attrs& attrs); + operator ::tensorflow::Output() const { return handle; } + operator ::tensorflow::Input() const { return handle; } + ::tensorflow::Node* node() const { return handle.node(); } + + static Attrs Shapes(const gtl::ArraySlice& x) { + return Attrs().Shapes(x); + } + static Attrs Capacity(int64 x) { + return Attrs().Capacity(x); + } + static Attrs Container(StringPiece x) { + return Attrs().Container(x); + } + static Attrs SharedName(StringPiece x) { + return Attrs().SharedName(x); + } + + Operation operation; + ::tensorflow::Output handle; +}; + +/// Interleave the values from the `data` tensors into a single tensor. +/// +/// Builds a merged tensor such that +/// +/// ```python +/// merged[indices[m][i, ..., j], ...] = data[m][i, ..., j, ...] +/// ``` +/// +/// For example, if each `indices[m]` is scalar or vector, we have +/// +/// ```python +/// # Scalar indices: +/// merged[indices[m], ...] = data[m][...] +/// +/// # Vector indices: +/// merged[indices[m][i], ...] = data[m][i, ...] +/// ``` +/// +/// Each `data[i].shape` must start with the corresponding `indices[i].shape`, +/// and the rest of `data[i].shape` must be constant w.r.t. `i`. That is, we +/// must have `data[i].shape = indices[i].shape + constant`. In terms of this +/// `constant`, the output shape is +/// +/// merged.shape = [max(indices)] + constant +/// +/// Values may be merged in parallel, so if an index appears in both `indices[m][i]` +/// and `indices[n][j]`, the result may be invalid. This differs from the normal +/// DynamicStitch operator that defines the behavior in that case. +/// +/// For example: +/// +/// ```python +/// indices[0] = 6 +/// indices[1] = [4, 1] +/// indices[2] = [[5, 2], [0, 3]] +/// data[0] = [61, 62] +/// data[1] = [[41, 42], [11, 12]] +/// data[2] = [[[51, 52], [21, 22]], [[1, 2], [31, 32]]] +/// merged = [[1, 2], [11, 12], [21, 22], [31, 32], [41, 42], +/// [51, 52], [61, 62]] +/// ``` +/// +/// This method can be used to merge partitions created by `dynamic_partition` +/// as illustrated on the following example: +/// +/// ```python +/// # Apply function (increments x_i) on elements for which a certain condition +/// # apply (x_i != -1 in this example). +/// x=tf.constant([0.1, -1., 5.2, 4.3, -1., 7.4]) +/// condition_mask=tf.not_equal(x,tf.constant(-1.)) +/// partitioned_data = tf.dynamic_partition( +/// x, tf.cast(condition_mask, tf.int32) , 2) +/// partitioned_data[1] = partitioned_data[1] + 1.0 +/// condition_indices = tf.dynamic_partition( +/// tf.range(tf.shape(x)[0]), tf.cast(condition_mask, tf.int32) , 2) +/// x = tf.dynamic_stitch(condition_indices, partitioned_data) +/// # Here x=[1.1, -1., 6.2, 5.3, -1, 8.4], the -1. values remain +/// # unchanged. +/// ``` +/// +///
+/// +///
+/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The merged tensor. +class ParallelDynamicStitch { + public: + ParallelDynamicStitch(const ::tensorflow::Scope& scope, ::tensorflow::InputList + indices, ::tensorflow::InputList data); + operator ::tensorflow::Output() const { return merged; } + operator ::tensorflow::Input() const { return merged; } + ::tensorflow::Node* node() const { return merged.node(); } + + Operation operation; + ::tensorflow::Output merged; +}; + +/// A queue that produces elements sorted by the first component value. +/// +/// Note that the PriorityQueue requires the first component of any element +/// to be a scalar int64, in addition to the other elements declared by +/// component_types. Therefore calls to Enqueue and EnqueueMany (resp. Dequeue +/// and DequeueMany) on a PriorityQueue will all require (resp. output) one extra +/// entry in their input (resp. output) lists. +/// +/// Arguments: +/// * scope: A Scope object +/// * shapes: The shape of each component in a value. The length of this attr must +/// be either 0 or the same as the length of component_types. If the length of +/// this attr is 0, the shapes of queue elements are not constrained, and +/// only one element may be dequeued at a time. +/// +/// Optional attributes (see `Attrs`): +/// * component_types: The type of each component in a value. +/// * capacity: The upper bound on the number of elements in this queue. +/// Negative numbers mean no limit. +/// * container: If non-empty, this queue is placed in the given container. +/// Otherwise, a default container is used. +/// * shared_name: If non-empty, this queue will be shared under the given name +/// across multiple sessions. +/// +/// Returns: +/// * `Output`: The handle to the queue. +class PriorityQueue { + public: + /// Optional attribute setters for PriorityQueue + struct Attrs { + /// The type of each component in a value. + /// + /// Defaults to [] + TF_MUST_USE_RESULT Attrs ComponentTypes(const DataTypeSlice& x) { + Attrs ret = *this; + ret.component_types_ = x; + return ret; + } + + /// The upper bound on the number of elements in this queue. + /// Negative numbers mean no limit. + /// + /// Defaults to -1 + TF_MUST_USE_RESULT Attrs Capacity(int64 x) { + Attrs ret = *this; + ret.capacity_ = x; + return ret; + } + + /// If non-empty, this queue is placed in the given container. + /// Otherwise, a default container is used. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs Container(StringPiece x) { + Attrs ret = *this; + ret.container_ = x; + return ret; + } + + /// If non-empty, this queue will be shared under the given name + /// across multiple sessions. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs SharedName(StringPiece x) { + Attrs ret = *this; + ret.shared_name_ = x; + return ret; + } + + DataTypeSlice component_types_ = {}; + int64 capacity_ = -1; + StringPiece container_ = ""; + StringPiece shared_name_ = ""; + }; + PriorityQueue(const ::tensorflow::Scope& scope, const + gtl::ArraySlice& shapes); + PriorityQueue(const ::tensorflow::Scope& scope, const + gtl::ArraySlice& shapes, const + PriorityQueue::Attrs& attrs); + operator ::tensorflow::Output() const { return handle; } + operator ::tensorflow::Input() const { return handle; } + ::tensorflow::Node* node() const { return handle.node(); } + + static Attrs ComponentTypes(const DataTypeSlice& x) { + return Attrs().ComponentTypes(x); + } + static Attrs Capacity(int64 x) { + return Attrs().Capacity(x); + } + static Attrs Container(StringPiece x) { + return Attrs().Container(x); + } + static Attrs SharedName(StringPiece x) { + return Attrs().SharedName(x); + } + + Operation operation; + ::tensorflow::Output handle; +}; + +/// Closes the given queue. +/// +/// This operation signals that no more elements will be enqueued in the +/// given queue. Subsequent Enqueue(Many) operations will fail. +/// Subsequent Dequeue(Many) operations will continue to succeed if +/// sufficient elements remain in the queue. Subsequent Dequeue(Many) +/// operations that would block will fail immediately. +/// +/// Arguments: +/// * scope: A Scope object +/// * handle: The handle to a queue. +/// +/// Optional attributes (see `Attrs`): +/// * cancel_pending_enqueues: If true, all pending enqueue requests that are +/// blocked on the given queue will be canceled. +/// +/// Returns: +/// * the created `Operation` +class QueueClose { + public: + /// Optional attribute setters for QueueClose + struct Attrs { + /// If true, all pending enqueue requests that are + /// blocked on the given queue will be canceled. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs CancelPendingEnqueues(bool x) { + Attrs ret = *this; + ret.cancel_pending_enqueues_ = x; + return ret; + } + + bool cancel_pending_enqueues_ = false; + }; + QueueClose(const ::tensorflow::Scope& scope, ::tensorflow::Input handle); + QueueClose(const ::tensorflow::Scope& scope, ::tensorflow::Input handle, const + QueueClose::Attrs& attrs); + operator ::tensorflow::Operation() const { return operation; } + + static Attrs CancelPendingEnqueues(bool x) { + return Attrs().CancelPendingEnqueues(x); + } + + Operation operation; +}; + +/// Dequeues `n` tuples of one or more tensors from the given queue. +/// +/// If the queue is closed and there are fewer than `n` elements, then an +/// OutOfRange error is returned. +/// +/// This operation concatenates queue-element component tensors along the +/// 0th dimension to make a single component tensor. All of the components +/// in the dequeued tuple will have size `n` in the 0th dimension. +/// +/// This operation has `k` outputs, where `k` is the number of components in +/// the tuples stored in the given queue, and output `i` is the ith +/// component of the dequeued tuple. +/// +/// N.B. If the queue is empty, this operation will block until `n` elements +/// have been dequeued (or 'timeout_ms' elapses, if specified). +/// +/// Arguments: +/// * scope: A Scope object +/// * handle: The handle to a queue. +/// * n: The number of tuples to dequeue. +/// * component_types: The type of each component in a tuple. +/// +/// Optional attributes (see `Attrs`): +/// * timeout_ms: If the queue has fewer than n elements, this operation +/// will block for up to timeout_ms milliseconds. +/// Note: This option is not supported yet. +/// +/// Returns: +/// * `OutputList`: One or more tensors that were dequeued as a tuple. +class QueueDequeueMany { + public: + /// Optional attribute setters for QueueDequeueMany + struct Attrs { + /// If the queue has fewer than n elements, this operation + /// will block for up to timeout_ms milliseconds. + /// Note: This option is not supported yet. + /// + /// Defaults to -1 + TF_MUST_USE_RESULT Attrs TimeoutMs(int64 x) { + Attrs ret = *this; + ret.timeout_ms_ = x; + return ret; + } + + int64 timeout_ms_ = -1; + }; + QueueDequeueMany(const ::tensorflow::Scope& scope, ::tensorflow::Input handle, + ::tensorflow::Input n, const DataTypeSlice& component_types); + QueueDequeueMany(const ::tensorflow::Scope& scope, ::tensorflow::Input handle, + ::tensorflow::Input n, const DataTypeSlice& component_types, + const QueueDequeueMany::Attrs& attrs); + ::tensorflow::Output operator[](size_t index) const { return components[index]; } + + + static Attrs TimeoutMs(int64 x) { + return Attrs().TimeoutMs(x); + } + + Operation operation; + ::tensorflow::OutputList components; +}; + +/// Dequeues `n` tuples of one or more tensors from the given queue. +/// +/// This operation is not supported by all queues. If a queue does not support +/// DequeueUpTo, then an Unimplemented error is returned. +/// +/// If the queue is closed and there are more than 0 but less than `n` +/// elements remaining, then instead of returning an OutOfRange error like +/// QueueDequeueMany, less than `n` elements are returned immediately. If +/// the queue is closed and there are 0 elements left in the queue, then +/// an OutOfRange error is returned just like in QueueDequeueMany. +/// Otherwise the behavior is identical to QueueDequeueMany: +/// +/// This operation concatenates queue-element component tensors along the +/// 0th dimension to make a single component tensor. All of the components +/// in the dequeued tuple will have size n in the 0th dimension. +/// +/// This operation has `k` outputs, where `k` is the number of components in +/// the tuples stored in the given queue, and output `i` is the ith +/// component of the dequeued tuple. +/// +/// Arguments: +/// * scope: A Scope object +/// * handle: The handle to a queue. +/// * n: The number of tuples to dequeue. +/// * component_types: The type of each component in a tuple. +/// +/// Optional attributes (see `Attrs`): +/// * timeout_ms: If the queue has fewer than n elements, this operation +/// will block for up to timeout_ms milliseconds. +/// Note: This option is not supported yet. +/// +/// Returns: +/// * `OutputList`: One or more tensors that were dequeued as a tuple. +class QueueDequeueUpTo { + public: + /// Optional attribute setters for QueueDequeueUpTo + struct Attrs { + /// If the queue has fewer than n elements, this operation + /// will block for up to timeout_ms milliseconds. + /// Note: This option is not supported yet. + /// + /// Defaults to -1 + TF_MUST_USE_RESULT Attrs TimeoutMs(int64 x) { + Attrs ret = *this; + ret.timeout_ms_ = x; + return ret; + } + + int64 timeout_ms_ = -1; + }; + QueueDequeueUpTo(const ::tensorflow::Scope& scope, ::tensorflow::Input handle, + ::tensorflow::Input n, const DataTypeSlice& component_types); + QueueDequeueUpTo(const ::tensorflow::Scope& scope, ::tensorflow::Input handle, + ::tensorflow::Input n, const DataTypeSlice& component_types, + const QueueDequeueUpTo::Attrs& attrs); + ::tensorflow::Output operator[](size_t index) const { return components[index]; } + + + static Attrs TimeoutMs(int64 x) { + return Attrs().TimeoutMs(x); + } + + Operation operation; + ::tensorflow::OutputList components; +}; + +/// Dequeues a tuple of one or more tensors from the given queue. +/// +/// This operation has k outputs, where k is the number of components +/// in the tuples stored in the given queue, and output i is the ith +/// component of the dequeued tuple. +/// +/// N.B. If the queue is empty, this operation will block until an element +/// has been dequeued (or 'timeout_ms' elapses, if specified). +/// +/// Arguments: +/// * scope: A Scope object +/// * handle: The handle to a queue. +/// * component_types: The type of each component in a tuple. +/// +/// Optional attributes (see `Attrs`): +/// * timeout_ms: If the queue is empty, this operation will block for up to +/// timeout_ms milliseconds. +/// Note: This option is not supported yet. +/// +/// Returns: +/// * `OutputList`: One or more tensors that were dequeued as a tuple. +class QueueDequeue { + public: + /// Optional attribute setters for QueueDequeue + struct Attrs { + /// If the queue is empty, this operation will block for up to + /// timeout_ms milliseconds. + /// Note: This option is not supported yet. + /// + /// Defaults to -1 + TF_MUST_USE_RESULT Attrs TimeoutMs(int64 x) { + Attrs ret = *this; + ret.timeout_ms_ = x; + return ret; + } + + int64 timeout_ms_ = -1; + }; + QueueDequeue(const ::tensorflow::Scope& scope, ::tensorflow::Input handle, + const DataTypeSlice& component_types); + QueueDequeue(const ::tensorflow::Scope& scope, ::tensorflow::Input handle, + const DataTypeSlice& component_types, const QueueDequeue::Attrs& + attrs); + ::tensorflow::Output operator[](size_t index) const { return components[index]; } + + + static Attrs TimeoutMs(int64 x) { + return Attrs().TimeoutMs(x); + } + + Operation operation; + ::tensorflow::OutputList components; +}; + +/// Enqueues zero or more tuples of one or more tensors in the given queue. +/// +/// This operation slices each component tensor along the 0th dimension to +/// make multiple queue elements. All of the tuple components must have the +/// same size in the 0th dimension. +/// +/// The components input has k elements, which correspond to the components of +/// tuples stored in the given queue. +/// +/// N.B. If the queue is full, this operation will block until the given +/// elements have been enqueued (or 'timeout_ms' elapses, if specified). +/// +/// Arguments: +/// * scope: A Scope object +/// * handle: The handle to a queue. +/// * components: One or more tensors from which the enqueued tensors should +/// be taken. +/// +/// Optional attributes (see `Attrs`): +/// * timeout_ms: If the queue is too full, this operation will block for up +/// to timeout_ms milliseconds. +/// Note: This option is not supported yet. +/// +/// Returns: +/// * the created `Operation` +class QueueEnqueueMany { + public: + /// Optional attribute setters for QueueEnqueueMany + struct Attrs { + /// If the queue is too full, this operation will block for up + /// to timeout_ms milliseconds. + /// Note: This option is not supported yet. + /// + /// Defaults to -1 + TF_MUST_USE_RESULT Attrs TimeoutMs(int64 x) { + Attrs ret = *this; + ret.timeout_ms_ = x; + return ret; + } + + int64 timeout_ms_ = -1; + }; + QueueEnqueueMany(const ::tensorflow::Scope& scope, ::tensorflow::Input handle, + ::tensorflow::InputList components); + QueueEnqueueMany(const ::tensorflow::Scope& scope, ::tensorflow::Input handle, + ::tensorflow::InputList components, const + QueueEnqueueMany::Attrs& attrs); + operator ::tensorflow::Operation() const { return operation; } + + static Attrs TimeoutMs(int64 x) { + return Attrs().TimeoutMs(x); + } + + Operation operation; +}; + +/// Enqueues a tuple of one or more tensors in the given queue. +/// +/// The components input has k elements, which correspond to the components of +/// tuples stored in the given queue. +/// +/// N.B. If the queue is full, this operation will block until the given +/// element has been enqueued (or 'timeout_ms' elapses, if specified). +/// +/// Arguments: +/// * scope: A Scope object +/// * handle: The handle to a queue. +/// * components: One or more tensors from which the enqueued tensors should be taken. +/// +/// Optional attributes (see `Attrs`): +/// * timeout_ms: If the queue is full, this operation will block for up to +/// timeout_ms milliseconds. +/// Note: This option is not supported yet. +/// +/// Returns: +/// * the created `Operation` +class QueueEnqueue { + public: + /// Optional attribute setters for QueueEnqueue + struct Attrs { + /// If the queue is full, this operation will block for up to + /// timeout_ms milliseconds. + /// Note: This option is not supported yet. + /// + /// Defaults to -1 + TF_MUST_USE_RESULT Attrs TimeoutMs(int64 x) { + Attrs ret = *this; + ret.timeout_ms_ = x; + return ret; + } + + int64 timeout_ms_ = -1; + }; + QueueEnqueue(const ::tensorflow::Scope& scope, ::tensorflow::Input handle, + ::tensorflow::InputList components); + QueueEnqueue(const ::tensorflow::Scope& scope, ::tensorflow::Input handle, + ::tensorflow::InputList components, const QueueEnqueue::Attrs& + attrs); + operator ::tensorflow::Operation() const { return operation; } + + static Attrs TimeoutMs(int64 x) { + return Attrs().TimeoutMs(x); + } + + Operation operation; +}; + +/// Returns true if queue is closed. +/// +/// This operation returns true if the queue is closed and false if the queue +/// is open. +/// +/// Arguments: +/// * scope: A Scope object +/// * handle: The handle to a queue. +/// +/// Returns: +/// * `Output`: The is_closed tensor. +class QueueIsClosed { + public: + QueueIsClosed(const ::tensorflow::Scope& scope, ::tensorflow::Input handle); + operator ::tensorflow::Output() const { return is_closed; } + operator ::tensorflow::Input() const { return is_closed; } + ::tensorflow::Node* node() const { return is_closed.node(); } + + Operation operation; + ::tensorflow::Output is_closed; +}; + +/// Returns true if queue is closed. +/// +/// This operation returns true if the queue is closed and false if the queue +/// is open. +/// +/// Arguments: +/// * scope: A Scope object +/// * handle: The handle to a queue. +/// +/// Returns: +/// * `Output`: The is_closed tensor. +class QueueIsClosedV2 { + public: + QueueIsClosedV2(const ::tensorflow::Scope& scope, ::tensorflow::Input handle); + operator ::tensorflow::Output() const { return is_closed; } + operator ::tensorflow::Input() const { return is_closed; } + ::tensorflow::Node* node() const { return is_closed.node(); } + + Operation operation; + ::tensorflow::Output is_closed; +}; + +/// Computes the number of elements in the given queue. +/// +/// Arguments: +/// * scope: A Scope object +/// * handle: The handle to a queue. +/// +/// Returns: +/// * `Output`: The number of elements in the given queue. +class QueueSize { + public: + QueueSize(const ::tensorflow::Scope& scope, ::tensorflow::Input handle); + operator ::tensorflow::Output() const { return size; } + operator ::tensorflow::Input() const { return size; } + ::tensorflow::Node* node() const { return size.node(); } + + Operation operation; + ::tensorflow::Output size; +}; + +/// A queue that randomizes the order of elements. +/// +/// Arguments: +/// * scope: A Scope object +/// * component_types: The type of each component in a value. +/// +/// Optional attributes (see `Attrs`): +/// * shapes: The shape of each component in a value. The length of this attr must +/// be either 0 or the same as the length of component_types. If the length of +/// this attr is 0, the shapes of queue elements are not constrained, and +/// only one element may be dequeued at a time. +/// * capacity: The upper bound on the number of elements in this queue. +/// Negative numbers mean no limit. +/// * min_after_dequeue: Dequeue will block unless there would be this +/// many elements after the dequeue or the queue is closed. This +/// ensures a minimum level of mixing of elements. +/// * seed: If either seed or seed2 is set to be non-zero, the random number +/// generator is seeded by the given seed. Otherwise, a random seed is used. +/// * seed2: A second seed to avoid seed collision. +/// * container: If non-empty, this queue is placed in the given container. +/// Otherwise, a default container is used. +/// * shared_name: If non-empty, this queue will be shared under the given name +/// across multiple sessions. +/// +/// Returns: +/// * `Output`: The handle to the queue. +class RandomShuffleQueue { + public: + /// Optional attribute setters for RandomShuffleQueue + struct Attrs { + /// The shape of each component in a value. The length of this attr must + /// be either 0 or the same as the length of component_types. If the length of + /// this attr is 0, the shapes of queue elements are not constrained, and + /// only one element may be dequeued at a time. + /// + /// Defaults to [] + TF_MUST_USE_RESULT Attrs Shapes(const gtl::ArraySlice& x) { + Attrs ret = *this; + ret.shapes_ = x; + return ret; + } + + /// The upper bound on the number of elements in this queue. + /// Negative numbers mean no limit. + /// + /// Defaults to -1 + TF_MUST_USE_RESULT Attrs Capacity(int64 x) { + Attrs ret = *this; + ret.capacity_ = x; + return ret; + } + + /// Dequeue will block unless there would be this + /// many elements after the dequeue or the queue is closed. This + /// ensures a minimum level of mixing of elements. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs MinAfterDequeue(int64 x) { + Attrs ret = *this; + ret.min_after_dequeue_ = x; + return ret; + } + + /// If either seed or seed2 is set to be non-zero, the random number + /// generator is seeded by the given seed. Otherwise, a random seed is used. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Seed(int64 x) { + Attrs ret = *this; + ret.seed_ = x; + return ret; + } + + /// A second seed to avoid seed collision. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Seed2(int64 x) { + Attrs ret = *this; + ret.seed2_ = x; + return ret; + } + + /// If non-empty, this queue is placed in the given container. + /// Otherwise, a default container is used. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs Container(StringPiece x) { + Attrs ret = *this; + ret.container_ = x; + return ret; + } + + /// If non-empty, this queue will be shared under the given name + /// across multiple sessions. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs SharedName(StringPiece x) { + Attrs ret = *this; + ret.shared_name_ = x; + return ret; + } + + gtl::ArraySlice shapes_ = {}; + int64 capacity_ = -1; + int64 min_after_dequeue_ = 0; + int64 seed_ = 0; + int64 seed2_ = 0; + StringPiece container_ = ""; + StringPiece shared_name_ = ""; + }; + RandomShuffleQueue(const ::tensorflow::Scope& scope, const DataTypeSlice& + component_types); + RandomShuffleQueue(const ::tensorflow::Scope& scope, const DataTypeSlice& + component_types, const RandomShuffleQueue::Attrs& attrs); + operator ::tensorflow::Output() const { return handle; } + operator ::tensorflow::Input() const { return handle; } + ::tensorflow::Node* node() const { return handle.node(); } + + static Attrs Shapes(const gtl::ArraySlice& x) { + return Attrs().Shapes(x); + } + static Attrs Capacity(int64 x) { + return Attrs().Capacity(x); + } + static Attrs MinAfterDequeue(int64 x) { + return Attrs().MinAfterDequeue(x); + } + static Attrs Seed(int64 x) { + return Attrs().Seed(x); + } + static Attrs Seed2(int64 x) { + return Attrs().Seed2(x); + } + static Attrs Container(StringPiece x) { + return Attrs().Container(x); + } + static Attrs SharedName(StringPiece x) { + return Attrs().SharedName(x); + } + + Operation operation; + ::tensorflow::Output handle; +}; + +/// Emits randomized records. +/// +/// Arguments: +/// * scope: A Scope object +/// * file_pattern: Glob pattern for the data files. +/// +/// Optional attributes (see `Attrs`): +/// * file_random_seed: Random seeds used to produce randomized records. +/// * file_shuffle_shift_ratio: Shifts the list of files after the list is randomly +/// shuffled. +/// * file_buffer_size: The randomization shuffling buffer. +/// * file_parallelism: How many sstables are opened and concurrently iterated over. +/// * batch_size: The batch size. +/// * compression_type: The type of compression for the file. Currently ZLIB and +/// GZIP are supported. Defaults to none. +/// +/// Returns: +/// * `Output`: A tensor of shape [batch_size]. +class RecordInput { + public: + /// Optional attribute setters for RecordInput + struct Attrs { + /// Random seeds used to produce randomized records. + /// + /// Defaults to 301 + TF_MUST_USE_RESULT Attrs FileRandomSeed(int64 x) { + Attrs ret = *this; + ret.file_random_seed_ = x; + return ret; + } + + /// Shifts the list of files after the list is randomly + /// shuffled. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs FileShuffleShiftRatio(float x) { + Attrs ret = *this; + ret.file_shuffle_shift_ratio_ = x; + return ret; + } + + /// The randomization shuffling buffer. + /// + /// Defaults to 10000 + TF_MUST_USE_RESULT Attrs FileBufferSize(int64 x) { + Attrs ret = *this; + ret.file_buffer_size_ = x; + return ret; + } + + /// How many sstables are opened and concurrently iterated over. + /// + /// Defaults to 16 + TF_MUST_USE_RESULT Attrs FileParallelism(int64 x) { + Attrs ret = *this; + ret.file_parallelism_ = x; + return ret; + } + + /// The batch size. + /// + /// Defaults to 32 + TF_MUST_USE_RESULT Attrs BatchSize(int64 x) { + Attrs ret = *this; + ret.batch_size_ = x; + return ret; + } + + /// The type of compression for the file. Currently ZLIB and + /// GZIP are supported. Defaults to none. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs CompressionType(StringPiece x) { + Attrs ret = *this; + ret.compression_type_ = x; + return ret; + } + + int64 file_random_seed_ = 301; + float file_shuffle_shift_ratio_ = 0.0f; + int64 file_buffer_size_ = 10000; + int64 file_parallelism_ = 16; + int64 batch_size_ = 32; + StringPiece compression_type_ = ""; + }; + RecordInput(const ::tensorflow::Scope& scope, StringPiece file_pattern); + RecordInput(const ::tensorflow::Scope& scope, StringPiece file_pattern, const + RecordInput::Attrs& attrs); + operator ::tensorflow::Output() const { return records; } + operator ::tensorflow::Input() const { return records; } + ::tensorflow::Node* node() const { return records.node(); } + + static Attrs FileRandomSeed(int64 x) { + return Attrs().FileRandomSeed(x); + } + static Attrs FileShuffleShiftRatio(float x) { + return Attrs().FileShuffleShiftRatio(x); + } + static Attrs FileBufferSize(int64 x) { + return Attrs().FileBufferSize(x); + } + static Attrs FileParallelism(int64 x) { + return Attrs().FileParallelism(x); + } + static Attrs BatchSize(int64 x) { + return Attrs().BatchSize(x); + } + static Attrs CompressionType(StringPiece x) { + return Attrs().CompressionType(x); + } + + Operation operation; + ::tensorflow::Output records; +}; + +/// Applies a sparse gradient to a given accumulator. +/// +/// Does not add if local_step is smaller than the accumulator's +/// global_step. +/// +/// Arguments: +/// * scope: A Scope object +/// * handle: The handle to a accumulator. +/// * local_step: The local_step value at which the sparse gradient was computed. +/// * gradient_indices: Indices of the sparse gradient to be accumulated. Must be a +/// vector. +/// * gradient_values: Values are the non-zero slices of the gradient, and must have +/// the same first dimension as indices, i.e., the nnz represented by indices and +/// values must be consistent. +/// * gradient_shape: Shape of the sparse gradient to be accumulated. +/// * has_known_shape: Boolean indicating whether gradient_shape is unknown, in which +/// case the input is ignored during validation. +/// +/// Returns: +/// * the created `Operation` +class SparseAccumulatorApplyGradient { + public: + SparseAccumulatorApplyGradient(const ::tensorflow::Scope& scope, + ::tensorflow::Input handle, ::tensorflow::Input + local_step, ::tensorflow::Input + gradient_indices, ::tensorflow::Input + gradient_values, ::tensorflow::Input + gradient_shape, bool has_known_shape); + operator ::tensorflow::Operation() const { return operation; } + + Operation operation; +}; + +/// Extracts the average sparse gradient in a SparseConditionalAccumulator. +/// +/// The op will blocks until sufficient (i.e., more than num_required) +/// gradients have been accumulated. If the accumulator has already +/// aggregated more than num_required gradients, it will return its +/// average of the accumulated gradients. Also automatically increments +/// the recorded global_step in the accumulator by 1, and resets the +/// aggregate to 0. +/// +/// Arguments: +/// * scope: A Scope object +/// * handle: The handle to a SparseConditionalAccumulator. +/// * num_required: Number of gradients required before we return an aggregate. +/// * dtype: The data type of accumulated gradients. Needs to correspond to the type +/// of the accumulator. +/// +/// Returns: +/// * `Output` indices: Indices of the average of the accumulated sparse gradients. +/// * `Output` values: Values of the average of the accumulated sparse gradients. +/// * `Output` shape: Shape of the average of the accumulated sparse gradients. +class SparseAccumulatorTakeGradient { + public: + SparseAccumulatorTakeGradient(const ::tensorflow::Scope& scope, + ::tensorflow::Input handle, ::tensorflow::Input + num_required, DataType dtype); + + Operation operation; + ::tensorflow::Output indices; + ::tensorflow::Output values; + ::tensorflow::Output shape; +}; + +/// A conditional accumulator for aggregating sparse gradients. +/// +/// The accumulator accepts gradients marked with local_step greater or +/// equal to the most recent global_step known to the accumulator. The +/// average can be extracted from the accumulator, provided sufficient +/// gradients have been accumulated. Extracting the average automatically +/// resets the aggregate to 0, and increments the global_step recorded by +/// the accumulator. +/// +/// Arguments: +/// * scope: A Scope object +/// * dtype: The type of the value being accumulated. +/// * shape: The shape of the values. +/// +/// Optional attributes (see `Attrs`): +/// * container: If non-empty, this accumulator is placed in the given container. +/// Otherwise, a default container is used. +/// * shared_name: If non-empty, this accumulator will be shared under the given name +/// across multiple sessions. +/// +/// Returns: +/// * `Output`: The handle to the accumulator. +class SparseConditionalAccumulator { + public: + /// Optional attribute setters for SparseConditionalAccumulator + struct Attrs { + /// If non-empty, this accumulator is placed in the given container. + /// Otherwise, a default container is used. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs Container(StringPiece x) { + Attrs ret = *this; + ret.container_ = x; + return ret; + } + + /// If non-empty, this accumulator will be shared under the given name + /// across multiple sessions. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs SharedName(StringPiece x) { + Attrs ret = *this; + ret.shared_name_ = x; + return ret; + } + + /// Defaults to "MEAN" + TF_MUST_USE_RESULT Attrs ReductionType(StringPiece x) { + Attrs ret = *this; + ret.reduction_type_ = x; + return ret; + } + + StringPiece container_ = ""; + StringPiece shared_name_ = ""; + StringPiece reduction_type_ = "MEAN"; + }; + SparseConditionalAccumulator(const ::tensorflow::Scope& scope, DataType dtype, + PartialTensorShape shape); + SparseConditionalAccumulator(const ::tensorflow::Scope& scope, DataType dtype, + PartialTensorShape shape, const + SparseConditionalAccumulator::Attrs& attrs); + operator ::tensorflow::Output() const { return handle; } + operator ::tensorflow::Input() const { return handle; } + ::tensorflow::Node* node() const { return handle.node(); } + + static Attrs Container(StringPiece x) { + return Attrs().Container(x); + } + static Attrs SharedName(StringPiece x) { + return Attrs().SharedName(x); + } + static Attrs ReductionType(StringPiece x) { + return Attrs().ReductionType(x); + } + + Operation operation; + ::tensorflow::Output handle; +}; + +/// Stage values similar to a lightweight Enqueue. +/// +/// The basic functionality of this Op is similar to a queue with many +/// fewer capabilities and options. This Op is optimized for performance. +/// +/// Arguments: +/// * scope: A Scope object +/// * values: a list of tensors +/// dtypes A list of data types that inserted values should adhere to. +/// +/// Optional attributes (see `Attrs`): +/// * capacity: Maximum number of elements in the Staging Area. If > 0, inserts +/// on the container will block when the capacity is reached. +/// * memory_limit: The maximum number of bytes allowed for Tensors in the Staging Area. +/// If > 0, inserts will block until sufficient space is available. +/// * container: If non-empty, this queue is placed in the given container. Otherwise, +/// a default container is used. +/// * shared_name: It is necessary to match this name to the matching Unstage Op. +/// +/// Returns: +/// * the created `Operation` +class Stage { + public: + /// Optional attribute setters for Stage + struct Attrs { + /// Maximum number of elements in the Staging Area. If > 0, inserts + /// on the container will block when the capacity is reached. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Capacity(int64 x) { + Attrs ret = *this; + ret.capacity_ = x; + return ret; + } + + /// The maximum number of bytes allowed for Tensors in the Staging Area. + /// If > 0, inserts will block until sufficient space is available. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs MemoryLimit(int64 x) { + Attrs ret = *this; + ret.memory_limit_ = x; + return ret; + } + + /// If non-empty, this queue is placed in the given container. Otherwise, + /// a default container is used. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs Container(StringPiece x) { + Attrs ret = *this; + ret.container_ = x; + return ret; + } + + /// It is necessary to match this name to the matching Unstage Op. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs SharedName(StringPiece x) { + Attrs ret = *this; + ret.shared_name_ = x; + return ret; + } + + int64 capacity_ = 0; + int64 memory_limit_ = 0; + StringPiece container_ = ""; + StringPiece shared_name_ = ""; + }; + Stage(const ::tensorflow::Scope& scope, ::tensorflow::InputList values); + Stage(const ::tensorflow::Scope& scope, ::tensorflow::InputList values, const + Stage::Attrs& attrs); + operator ::tensorflow::Operation() const { return operation; } + + static Attrs Capacity(int64 x) { + return Attrs().Capacity(x); + } + static Attrs MemoryLimit(int64 x) { + return Attrs().MemoryLimit(x); + } + static Attrs Container(StringPiece x) { + return Attrs().Container(x); + } + static Attrs SharedName(StringPiece x) { + return Attrs().SharedName(x); + } + + Operation operation; +}; + +/// Op removes all elements in the underlying container. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * the created `Operation` +class StageClear { + public: + /// Optional attribute setters for StageClear + struct Attrs { + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Capacity(int64 x) { + Attrs ret = *this; + ret.capacity_ = x; + return ret; + } + + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs MemoryLimit(int64 x) { + Attrs ret = *this; + ret.memory_limit_ = x; + return ret; + } + + /// Defaults to "" + TF_MUST_USE_RESULT Attrs Container(StringPiece x) { + Attrs ret = *this; + ret.container_ = x; + return ret; + } + + /// Defaults to "" + TF_MUST_USE_RESULT Attrs SharedName(StringPiece x) { + Attrs ret = *this; + ret.shared_name_ = x; + return ret; + } + + int64 capacity_ = 0; + int64 memory_limit_ = 0; + StringPiece container_ = ""; + StringPiece shared_name_ = ""; + }; + StageClear(const ::tensorflow::Scope& scope, const DataTypeSlice& dtypes); + StageClear(const ::tensorflow::Scope& scope, const DataTypeSlice& dtypes, const + StageClear::Attrs& attrs); + operator ::tensorflow::Operation() const { return operation; } + + static Attrs Capacity(int64 x) { + return Attrs().Capacity(x); + } + static Attrs MemoryLimit(int64 x) { + return Attrs().MemoryLimit(x); + } + static Attrs Container(StringPiece x) { + return Attrs().Container(x); + } + static Attrs SharedName(StringPiece x) { + return Attrs().SharedName(x); + } + + Operation operation; +}; + +/// Op peeks at the values at the specified index. If the +/// +/// underlying container does not contain sufficient elements +/// this op will block until it does. This Op is optimized for +/// performance. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `OutputList`: The values tensor. +class StagePeek { + public: + /// Optional attribute setters for StagePeek + struct Attrs { + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Capacity(int64 x) { + Attrs ret = *this; + ret.capacity_ = x; + return ret; + } + + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs MemoryLimit(int64 x) { + Attrs ret = *this; + ret.memory_limit_ = x; + return ret; + } + + /// Defaults to "" + TF_MUST_USE_RESULT Attrs Container(StringPiece x) { + Attrs ret = *this; + ret.container_ = x; + return ret; + } + + /// Defaults to "" + TF_MUST_USE_RESULT Attrs SharedName(StringPiece x) { + Attrs ret = *this; + ret.shared_name_ = x; + return ret; + } + + int64 capacity_ = 0; + int64 memory_limit_ = 0; + StringPiece container_ = ""; + StringPiece shared_name_ = ""; + }; + StagePeek(const ::tensorflow::Scope& scope, ::tensorflow::Input index, const + DataTypeSlice& dtypes); + StagePeek(const ::tensorflow::Scope& scope, ::tensorflow::Input index, const + DataTypeSlice& dtypes, const StagePeek::Attrs& attrs); + ::tensorflow::Output operator[](size_t index) const { return values[index]; } + + + static Attrs Capacity(int64 x) { + return Attrs().Capacity(x); + } + static Attrs MemoryLimit(int64 x) { + return Attrs().MemoryLimit(x); + } + static Attrs Container(StringPiece x) { + return Attrs().Container(x); + } + static Attrs SharedName(StringPiece x) { + return Attrs().SharedName(x); + } + + Operation operation; + ::tensorflow::OutputList values; +}; + +/// Op returns the number of elements in the underlying container. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The size tensor. +class StageSize { + public: + /// Optional attribute setters for StageSize + struct Attrs { + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Capacity(int64 x) { + Attrs ret = *this; + ret.capacity_ = x; + return ret; + } + + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs MemoryLimit(int64 x) { + Attrs ret = *this; + ret.memory_limit_ = x; + return ret; + } + + /// Defaults to "" + TF_MUST_USE_RESULT Attrs Container(StringPiece x) { + Attrs ret = *this; + ret.container_ = x; + return ret; + } + + /// Defaults to "" + TF_MUST_USE_RESULT Attrs SharedName(StringPiece x) { + Attrs ret = *this; + ret.shared_name_ = x; + return ret; + } + + int64 capacity_ = 0; + int64 memory_limit_ = 0; + StringPiece container_ = ""; + StringPiece shared_name_ = ""; + }; + StageSize(const ::tensorflow::Scope& scope, const DataTypeSlice& dtypes); + StageSize(const ::tensorflow::Scope& scope, const DataTypeSlice& dtypes, const + StageSize::Attrs& attrs); + operator ::tensorflow::Output() const { return size; } + operator ::tensorflow::Input() const { return size; } + ::tensorflow::Node* node() const { return size.node(); } + + static Attrs Capacity(int64 x) { + return Attrs().Capacity(x); + } + static Attrs MemoryLimit(int64 x) { + return Attrs().MemoryLimit(x); + } + static Attrs Container(StringPiece x) { + return Attrs().Container(x); + } + static Attrs SharedName(StringPiece x) { + return Attrs().SharedName(x); + } + + Operation operation; + ::tensorflow::Output size; +}; + +/// Delete the TensorArray from its resource container. +/// +/// This enables the user to close and release the resource in the middle +/// of a step/run. +/// +/// Arguments: +/// * scope: A Scope object +/// * handle: The handle to a TensorArray (output of TensorArray or TensorArrayGrad). +/// +/// Returns: +/// * the created `Operation` +class TensorArrayClose { + public: + TensorArrayClose(const ::tensorflow::Scope& scope, ::tensorflow::Input handle); + operator ::tensorflow::Operation() const { return operation; } + + Operation operation; +}; + +/// Concat the elements from the TensorArray into value `value`. +/// +/// Takes `T` elements of shapes +/// +/// ``` +/// (n0 x d0 x d1 x ...), (n1 x d0 x d1 x ...), ..., (n(T-1) x d0 x d1 x ...) +/// ``` +/// +/// and concatenates them into a Tensor of shape: +/// +/// ```(n0 + n1 + ... + n(T-1) x d0 x d1 x ...)``` +/// +/// All elements must have the same shape (excepting the first dimension). +/// +/// Arguments: +/// * scope: A Scope object +/// * handle: The handle to a TensorArray. +/// * flow_in: A float scalar that enforces proper chaining of operations. +/// * dtype: The type of the elem that is returned. +/// +/// Optional attributes (see `Attrs`): +/// * element_shape_except0: The expected shape of an element, if known, +/// excluding the first dimension. Used to validate the shapes of +/// TensorArray elements. If this shape is not fully specified, concatenating +/// zero-size TensorArrays is an error. +/// +/// Returns: +/// * `Output` value: All of the elements in the TensorArray, concatenated along the first +/// axis. +/// * `Output` lengths: A vector of the row sizes of the original T elements in the +/// value output. In the example above, this would be the values: +/// `(n1, n2, ..., n(T-1))`. +class TensorArrayConcat { + public: + /// Optional attribute setters for TensorArrayConcat + struct Attrs { + /// The expected shape of an element, if known, + /// excluding the first dimension. Used to validate the shapes of + /// TensorArray elements. If this shape is not fully specified, concatenating + /// zero-size TensorArrays is an error. + /// + /// Defaults to + TF_MUST_USE_RESULT Attrs ElementShapeExcept0(PartialTensorShape x) { + Attrs ret = *this; + ret.element_shape_except0_ = x; + return ret; + } + + PartialTensorShape element_shape_except0_ = ::tensorflow::PartialTensorShape() /* unknown */; + }; + TensorArrayConcat(const ::tensorflow::Scope& scope, ::tensorflow::Input handle, + ::tensorflow::Input flow_in, DataType dtype); + TensorArrayConcat(const ::tensorflow::Scope& scope, ::tensorflow::Input handle, + ::tensorflow::Input flow_in, DataType dtype, const + TensorArrayConcat::Attrs& attrs); + + static Attrs ElementShapeExcept0(PartialTensorShape x) { + return Attrs().ElementShapeExcept0(x); + } + + Operation operation; + ::tensorflow::Output value; + ::tensorflow::Output lengths; +}; + +/// Gather specific elements from the TensorArray into output `value`. +/// +/// All elements selected by `indices` must have the same shape. +/// +/// Arguments: +/// * scope: A Scope object +/// * handle: The handle to a TensorArray. +/// * indices: The locations in the TensorArray from which to read tensor elements. +/// * flow_in: A float scalar that enforces proper chaining of operations. +/// * dtype: The type of the elem that is returned. +/// +/// Optional attributes (see `Attrs`): +/// * element_shape: The expected shape of an element, if known. Used to +/// validate the shapes of TensorArray elements. If this shape is not +/// fully specified, gathering zero-size TensorArrays is an error. +/// +/// Returns: +/// * `Output`: All of the elements in the TensorArray, concatenated along a new +/// axis (the new dimension 0). +class TensorArrayGather { + public: + /// Optional attribute setters for TensorArrayGather + struct Attrs { + /// The expected shape of an element, if known. Used to + /// validate the shapes of TensorArray elements. If this shape is not + /// fully specified, gathering zero-size TensorArrays is an error. + /// + /// Defaults to + TF_MUST_USE_RESULT Attrs ElementShape(PartialTensorShape x) { + Attrs ret = *this; + ret.element_shape_ = x; + return ret; + } + + PartialTensorShape element_shape_ = ::tensorflow::PartialTensorShape() /* unknown */; + }; + TensorArrayGather(const ::tensorflow::Scope& scope, ::tensorflow::Input handle, + ::tensorflow::Input indices, ::tensorflow::Input flow_in, + DataType dtype); + TensorArrayGather(const ::tensorflow::Scope& scope, ::tensorflow::Input handle, + ::tensorflow::Input indices, ::tensorflow::Input flow_in, + DataType dtype, const TensorArrayGather::Attrs& attrs); + operator ::tensorflow::Output() const { return value; } + operator ::tensorflow::Input() const { return value; } + ::tensorflow::Node* node() const { return value.node(); } + + static Attrs ElementShape(PartialTensorShape x) { + return Attrs().ElementShape(x); + } + + Operation operation; + ::tensorflow::Output value; +}; + +/// Creates a TensorArray for storing the gradients of values in the given handle. +/// +/// If the given TensorArray gradient already exists, returns a reference to it. +/// +/// Locks the size of the original TensorArray by disabling its dynamic size flag. +/// +/// **A note about the input flow_in:** +/// +/// The handle flow_in forces the execution of the gradient lookup to occur +/// only after certain other operations have occurred. For example, when +/// the forward TensorArray is dynamically sized, writes to this TensorArray +/// may resize the object. The gradient TensorArray is statically sized based +/// on the size of the forward TensorArray when this operation executes. +/// Furthermore, the size of the forward TensorArray is frozen by this call. +/// As a result, the flow is used to ensure that the call to generate the gradient +/// TensorArray only happens after all writes are executed. +/// +/// In the case of dynamically sized TensorArrays, gradient computation should +/// only be performed on read operations that have themselves been chained via +/// flow to occur only after all writes have executed. That way the final size +/// of the forward TensorArray is known when this operation is called. +/// +/// **A note about the source attribute:** +/// +/// TensorArray gradient calls use an accumulator TensorArray object. If +/// multiple gradients are calculated and run in the same session, the multiple +/// gradient nodes may accidentally flow through the same accumulator TensorArray. +/// This double counts and generally breaks the TensorArray gradient flow. +/// +/// The solution is to identify which gradient call this particular +/// TensorArray gradient is being called in. This is performed by identifying +/// a unique string (e.g. "gradients", "gradients_1", ...) from the input +/// gradient Tensor's name. This string is used as a suffix when creating +/// the TensorArray gradient object here (the attribute `source`). +/// +/// The attribute `source` is added as a suffix to the forward TensorArray's +/// name when performing the creation / lookup, so that each separate gradient +/// calculation gets its own TensorArray accumulator. +/// +/// Arguments: +/// * scope: A Scope object +/// * handle: The handle to the forward TensorArray. +/// * flow_in: A float scalar that enforces proper chaining of operations. +/// * source: The gradient source string, used to decide which gradient TensorArray +/// to return. +/// +/// Returns: +/// * `Output` grad_handle +/// * `Output` flow_out +class TensorArrayGrad { + public: + TensorArrayGrad(const ::tensorflow::Scope& scope, ::tensorflow::Input handle, + ::tensorflow::Input flow_in, StringPiece source); + + Operation operation; + ::tensorflow::Output grad_handle; + ::tensorflow::Output flow_out; +}; + +/// Creates a TensorArray for storing multiple gradients of values in the given handle. +/// +/// Similar to TensorArrayGradV3. However it creates an accumulator with an +/// expanded shape compared to the input TensorArray whose gradient is being +/// computed. This enables multiple gradients for the same TensorArray to be +/// calculated using the same accumulator. +/// +/// Arguments: +/// * scope: A Scope object +/// * handle: The handle to the forward TensorArray. +/// * flow_in: A float scalar that enforces proper chaining of operations. +/// * shape_to_prepend: An int32 vector representing a shape. Elements in the gradient accumulator will +/// have shape which is this shape_to_prepend value concatenated with shape of the +/// elements in the TensorArray corresponding to the input handle. +/// * source: The gradient source string, used to decide which gradient TensorArray +/// to return. +/// +/// Returns: +/// * `Output` grad_handle +/// * `Output` flow_out +class TensorArrayGradWithShape { + public: + TensorArrayGradWithShape(const ::tensorflow::Scope& scope, ::tensorflow::Input + handle, ::tensorflow::Input flow_in, + ::tensorflow::Input shape_to_prepend, StringPiece + source); + + Operation operation; + ::tensorflow::Output grad_handle; + ::tensorflow::Output flow_out; +}; + +/// Read an element from the TensorArray into output `value`. +/// +/// Arguments: +/// * scope: A Scope object +/// * handle: The handle to a TensorArray. +/// * flow_in: A float scalar that enforces proper chaining of operations. +/// * dtype: The type of the elem that is returned. +/// +/// Returns: +/// * `Output`: The tensor that is read from the TensorArray. +class TensorArrayRead { + public: + TensorArrayRead(const ::tensorflow::Scope& scope, ::tensorflow::Input handle, + ::tensorflow::Input index, ::tensorflow::Input flow_in, + DataType dtype); + operator ::tensorflow::Output() const { return value; } + operator ::tensorflow::Input() const { return value; } + ::tensorflow::Node* node() const { return value.node(); } + + Operation operation; + ::tensorflow::Output value; +}; + +/// Scatter the data from the input value into specific TensorArray elements. +/// +/// `indices` must be a vector, its length must match the first dim of `value`. +/// +/// Arguments: +/// * scope: A Scope object +/// * handle: The handle to a TensorArray. +/// * indices: The locations at which to write the tensor elements. +/// * value: The concatenated tensor to write to the TensorArray. +/// * flow_in: A float scalar that enforces proper chaining of operations. +/// +/// Returns: +/// * `Output`: A float scalar that enforces proper chaining of operations. +class TensorArrayScatter { + public: + TensorArrayScatter(const ::tensorflow::Scope& scope, ::tensorflow::Input + handle, ::tensorflow::Input indices, ::tensorflow::Input + value, ::tensorflow::Input flow_in); + operator ::tensorflow::Output() const { return flow_out; } + operator ::tensorflow::Input() const { return flow_out; } + ::tensorflow::Node* node() const { return flow_out.node(); } + + Operation operation; + ::tensorflow::Output flow_out; +}; + +/// Get the current size of the TensorArray. +/// +/// Arguments: +/// * scope: A Scope object +/// * handle: The handle to a TensorArray (output of TensorArray or TensorArrayGrad). +/// * flow_in: A float scalar that enforces proper chaining of operations. +/// +/// Returns: +/// * `Output`: The current size of the TensorArray. +class TensorArraySize { + public: + TensorArraySize(const ::tensorflow::Scope& scope, ::tensorflow::Input handle, + ::tensorflow::Input flow_in); + operator ::tensorflow::Output() const { return size; } + operator ::tensorflow::Input() const { return size; } + ::tensorflow::Node* node() const { return size.node(); } + + Operation operation; + ::tensorflow::Output size; +}; + +/// Split the data from the input value into TensorArray elements. +/// +/// Assuming that `lengths` takes on values +/// +/// ```(n0, n1, ..., n(T-1))``` +/// +/// and that `value` has shape +/// +/// ```(n0 + n1 + ... + n(T-1) x d0 x d1 x ...)```, +/// +/// this splits values into a TensorArray with T tensors. +/// +/// TensorArray index t will be the subtensor of values with starting position +/// +/// ```(n0 + n1 + ... + n(t-1), 0, 0, ...)``` +/// +/// and having size +/// +/// ```nt x d0 x d1 x ...``` +/// +/// Arguments: +/// * scope: A Scope object +/// * handle: The handle to a TensorArray. +/// * value: The concatenated tensor to write to the TensorArray. +/// * lengths: The vector of lengths, how to split the rows of value into the +/// TensorArray. +/// * flow_in: A float scalar that enforces proper chaining of operations. +/// +/// Returns: +/// * `Output`: A float scalar that enforces proper chaining of operations. +class TensorArraySplit { + public: + TensorArraySplit(const ::tensorflow::Scope& scope, ::tensorflow::Input handle, + ::tensorflow::Input value, ::tensorflow::Input lengths, + ::tensorflow::Input flow_in); + operator ::tensorflow::Output() const { return flow_out; } + operator ::tensorflow::Input() const { return flow_out; } + ::tensorflow::Node* node() const { return flow_out.node(); } + + Operation operation; + ::tensorflow::Output flow_out; +}; + +/// An array of Tensors of given size. +/// +/// Write data via Write and read via Read or Pack. +/// +/// Arguments: +/// * scope: A Scope object +/// * size: The size of the array. +/// * dtype: The type of the elements on the tensor_array. +/// +/// Optional attributes (see `Attrs`): +/// * element_shape: The expected shape of an element, if known. Used to +/// validate the shapes of TensorArray elements. If this shape is not +/// fully specified, gathering zero-size TensorArrays is an error. +/// * dynamic_size: A boolean that determines whether writes to the TensorArray +/// are allowed to grow the size. By default, this is not allowed. +/// * clear_after_read: If true (default), Tensors in the TensorArray are cleared +/// after being read. This disables multiple read semantics but allows early +/// release of memory. +/// * identical_element_shapes: If true (default is false), then all +/// elements in the TensorArray will be expected to have have identical shapes. +/// This allows certain behaviors, like dynamically checking for +/// consistent shapes on write, and being able to fill in properly +/// shaped zero tensors on stack -- even if the element_shape attribute +/// is not fully defined. +/// * tensor_array_name: Overrides the name used for the temporary tensor_array +/// resource. Default value is the name of the 'TensorArray' op (which +/// is guaranteed unique). +/// +/// Returns: +/// * `Output` handle: The handle to the TensorArray. +/// * `Output` flow: A scalar used to control gradient flow. +class TensorArray { + public: + /// Optional attribute setters for TensorArray + struct Attrs { + /// The expected shape of an element, if known. Used to + /// validate the shapes of TensorArray elements. If this shape is not + /// fully specified, gathering zero-size TensorArrays is an error. + /// + /// Defaults to + TF_MUST_USE_RESULT Attrs ElementShape(PartialTensorShape x) { + Attrs ret = *this; + ret.element_shape_ = x; + return ret; + } + + /// A boolean that determines whether writes to the TensorArray + /// are allowed to grow the size. By default, this is not allowed. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs DynamicSize(bool x) { + Attrs ret = *this; + ret.dynamic_size_ = x; + return ret; + } + + /// If true (default), Tensors in the TensorArray are cleared + /// after being read. This disables multiple read semantics but allows early + /// release of memory. + /// + /// Defaults to true + TF_MUST_USE_RESULT Attrs ClearAfterRead(bool x) { + Attrs ret = *this; + ret.clear_after_read_ = x; + return ret; + } + + /// If true (default is false), then all + /// elements in the TensorArray will be expected to have have identical shapes. + /// This allows certain behaviors, like dynamically checking for + /// consistent shapes on write, and being able to fill in properly + /// shaped zero tensors on stack -- even if the element_shape attribute + /// is not fully defined. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs IdenticalElementShapes(bool x) { + Attrs ret = *this; + ret.identical_element_shapes_ = x; + return ret; + } + + /// Overrides the name used for the temporary tensor_array + /// resource. Default value is the name of the 'TensorArray' op (which + /// is guaranteed unique). + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs TensorArrayName(StringPiece x) { + Attrs ret = *this; + ret.tensor_array_name_ = x; + return ret; + } + + PartialTensorShape element_shape_ = ::tensorflow::PartialTensorShape() /* unknown */; + bool dynamic_size_ = false; + bool clear_after_read_ = true; + bool identical_element_shapes_ = false; + StringPiece tensor_array_name_ = ""; + }; + TensorArray(const ::tensorflow::Scope& scope, ::tensorflow::Input size, + DataType dtype); + TensorArray(const ::tensorflow::Scope& scope, ::tensorflow::Input size, + DataType dtype, const TensorArray::Attrs& attrs); + + static Attrs ElementShape(PartialTensorShape x) { + return Attrs().ElementShape(x); + } + static Attrs DynamicSize(bool x) { + return Attrs().DynamicSize(x); + } + static Attrs ClearAfterRead(bool x) { + return Attrs().ClearAfterRead(x); + } + static Attrs IdenticalElementShapes(bool x) { + return Attrs().IdenticalElementShapes(x); + } + static Attrs TensorArrayName(StringPiece x) { + return Attrs().TensorArrayName(x); + } + + Operation operation; + ::tensorflow::Output handle; + ::tensorflow::Output flow; +}; + +/// Push an element onto the tensor_array. +/// +/// Arguments: +/// * scope: A Scope object +/// * handle: The handle to a TensorArray. +/// * index: The position to write to inside the TensorArray. +/// * value: The tensor to write to the TensorArray. +/// * flow_in: A float scalar that enforces proper chaining of operations. +/// +/// Returns: +/// * `Output`: A float scalar that enforces proper chaining of operations. +class TensorArrayWrite { + public: + TensorArrayWrite(const ::tensorflow::Scope& scope, ::tensorflow::Input handle, + ::tensorflow::Input index, ::tensorflow::Input value, + ::tensorflow::Input flow_in); + operator ::tensorflow::Output() const { return flow_out; } + operator ::tensorflow::Input() const { return flow_out; } + ::tensorflow::Node* node() const { return flow_out.node(); } + + Operation operation; + ::tensorflow::Output flow_out; +}; + +/// Op is similar to a lightweight Dequeue. +/// +/// The basic functionality is similar to dequeue with many fewer +/// capabilities and options. This Op is optimized for performance. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `OutputList`: The values tensor. +class Unstage { + public: + /// Optional attribute setters for Unstage + struct Attrs { + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Capacity(int64 x) { + Attrs ret = *this; + ret.capacity_ = x; + return ret; + } + + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs MemoryLimit(int64 x) { + Attrs ret = *this; + ret.memory_limit_ = x; + return ret; + } + + /// Defaults to "" + TF_MUST_USE_RESULT Attrs Container(StringPiece x) { + Attrs ret = *this; + ret.container_ = x; + return ret; + } + + /// Defaults to "" + TF_MUST_USE_RESULT Attrs SharedName(StringPiece x) { + Attrs ret = *this; + ret.shared_name_ = x; + return ret; + } + + int64 capacity_ = 0; + int64 memory_limit_ = 0; + StringPiece container_ = ""; + StringPiece shared_name_ = ""; + }; + Unstage(const ::tensorflow::Scope& scope, const DataTypeSlice& dtypes); + Unstage(const ::tensorflow::Scope& scope, const DataTypeSlice& dtypes, const + Unstage::Attrs& attrs); + ::tensorflow::Output operator[](size_t index) const { return values[index]; } + + + static Attrs Capacity(int64 x) { + return Attrs().Capacity(x); + } + static Attrs MemoryLimit(int64 x) { + return Attrs().MemoryLimit(x); + } + static Attrs Container(StringPiece x) { + return Attrs().Container(x); + } + static Attrs SharedName(StringPiece x) { + return Attrs().SharedName(x); + } + + Operation operation; + ::tensorflow::OutputList values; +}; + +/// @} + +} // namespace ops +} // namespace tensorflow + +#endif // TENSORFLOW_CC_OPS_DATA_FLOW_OPS_H_ diff --git a/linux/include/tensorflow/cc/ops/data_flow_ops_internal.h b/linux/include/tensorflow/cc/ops/data_flow_ops_internal.h new file mode 100644 index 000000000..e4ab562cd --- /dev/null +++ b/linux/include/tensorflow/cc/ops/data_flow_ops_internal.h @@ -0,0 +1,28 @@ +// This file is MACHINE GENERATED! Do not edit. + +#ifndef TENSORFLOW_CC_OPS_DATA_FLOW_OPS_INTERNAL_H_ +#define TENSORFLOW_CC_OPS_DATA_FLOW_OPS_INTERNAL_H_ + +// This file is MACHINE GENERATED! Do not edit. + +#include "tensorflow/cc/framework/ops.h" +#include "tensorflow/cc/framework/scope.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/gtl/array_slice.h" + +namespace tensorflow { +namespace ops { +namespace internal { +// NOTE: This namespace has internal TensorFlow details that +// are not part of TensorFlow's public API. + +/// @defgroup data_flow_ops_internal Data Flow Ops Internal +/// @{ + +} // namespace internal +} // namespace ops +} // namespace tensorflow + +#endif // TENSORFLOW_CC_OPS_DATA_FLOW_OPS_INTERNAL_H_ diff --git a/linux/include/tensorflow/cc/ops/image_ops.h b/linux/include/tensorflow/cc/ops/image_ops.h new file mode 100644 index 000000000..8cbea81ab --- /dev/null +++ b/linux/include/tensorflow/cc/ops/image_ops.h @@ -0,0 +1,2171 @@ +// This file is MACHINE GENERATED! Do not edit. + +#ifndef TENSORFLOW_CC_OPS_IMAGE_OPS_H_ +#define TENSORFLOW_CC_OPS_IMAGE_OPS_H_ + +// This file is MACHINE GENERATED! Do not edit. + +#include "tensorflow/cc/framework/ops.h" +#include "tensorflow/cc/framework/scope.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/gtl/array_slice.h" + +namespace tensorflow { +namespace ops { + +/// @defgroup image_ops Image Ops +/// @{ + +/// Adjust the contrast of one or more images. +/// +/// `images` is a tensor of at least 3 dimensions. The last 3 dimensions are +/// interpreted as `[height, width, channels]`. The other dimensions only +/// represent a collection of images, such as `[batch, height, width, channels].` +/// +/// Contrast is adjusted independently for each channel of each image. +/// +/// For each channel, the Op first computes the mean of the image pixels in the +/// channel and then adjusts each component of each pixel to +/// `(x - mean) * contrast_factor + mean`. +/// +/// Arguments: +/// * scope: A Scope object +/// * images: Images to adjust. At least 3-D. +/// * contrast_factor: A float multiplier for adjusting contrast. +/// +/// Returns: +/// * `Output`: The contrast-adjusted image or images. +class AdjustContrast { + public: + AdjustContrast(const ::tensorflow::Scope& scope, ::tensorflow::Input images, + ::tensorflow::Input contrast_factor); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Adjust the hue of one or more images. +/// +/// `images` is a tensor of at least 3 dimensions. The last dimension is +/// interpretted as channels, and must be three. +/// +/// The input image is considered in the RGB colorspace. Conceptually, the RGB +/// colors are first mapped into HSV. A delta is then applied all the hue values, +/// and then remapped back to RGB colorspace. +/// +/// Arguments: +/// * scope: A Scope object +/// * images: Images to adjust. At least 3-D. +/// * delta: A float delta to add to the hue. +/// +/// Returns: +/// * `Output`: The hue-adjusted image or images. +class AdjustHue { + public: + AdjustHue(const ::tensorflow::Scope& scope, ::tensorflow::Input images, + ::tensorflow::Input delta); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Adjust the saturation of one or more images. +/// +/// `images` is a tensor of at least 3 dimensions. The last dimension is +/// interpretted as channels, and must be three. +/// +/// The input image is considered in the RGB colorspace. Conceptually, the RGB +/// colors are first mapped into HSV. A scale is then applied all the saturation +/// values, and then remapped back to RGB colorspace. +/// +/// Arguments: +/// * scope: A Scope object +/// * images: Images to adjust. At least 3-D. +/// * scale: A float scale to add to the saturation. +/// +/// Returns: +/// * `Output`: The hue-adjusted image or images. +class AdjustSaturation { + public: + AdjustSaturation(const ::tensorflow::Scope& scope, ::tensorflow::Input images, + ::tensorflow::Input scale); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Extracts crops from the input image tensor and resizes them. +/// +/// Extracts crops from the input image tensor and resizes them using bilinear +/// sampling or nearest neighbor sampling (possibly with aspect ratio change) to a +/// common output size specified by `crop_size`. This is more general than the +/// `crop_to_bounding_box` op which extracts a fixed size slice from the input image +/// and does not allow resizing or aspect ratio change. +/// +/// Returns a tensor with `crops` from the input `image` at positions defined at the +/// bounding box locations in `boxes`. The cropped boxes are all resized (with +/// bilinear or nearest neighbor interpolation) to a fixed +/// `size = [crop_height, crop_width]`. The result is a 4-D tensor +/// `[num_boxes, crop_height, crop_width, depth]`. The resizing is corner aligned. +/// In particular, if `boxes = [[0, 0, 1, 1]]`, the method will give identical +/// results to using `tf.image.resize_bilinear()` or +/// `tf.image.resize_nearest_neighbor()`(depends on the `method` argument) with +/// `align_corners=True`. +/// +/// Arguments: +/// * scope: A Scope object +/// * image: A 4-D tensor of shape `[batch, image_height, image_width, depth]`. +/// Both `image_height` and `image_width` need to be positive. +/// * boxes: A 2-D tensor of shape `[num_boxes, 4]`. The `i`-th row of the tensor +/// specifies the coordinates of a box in the `box_ind[i]` image and is specified +/// in normalized coordinates `[y1, x1, y2, x2]`. A normalized coordinate value of +/// `y` is mapped to the image coordinate at `y * (image_height - 1)`, so as the +/// `[0, 1]` interval of normalized image height is mapped to +/// `[0, image_height - 1]` in image height coordinates. We do allow `y1` > `y2`, in +/// which case the sampled crop is an up-down flipped version of the original +/// image. The width dimension is treated similarly. Normalized coordinates +/// outside the `[0, 1]` range are allowed, in which case we use +/// `extrapolation_value` to extrapolate the input image values. +/// * box_ind: A 1-D tensor of shape `[num_boxes]` with int32 values in `[0, batch)`. +/// The value of `box_ind[i]` specifies the image that the `i`-th box refers to. +/// * crop_size: A 1-D tensor of 2 elements, `size = [crop_height, crop_width]`. All +/// cropped image patches are resized to this size. The aspect ratio of the image +/// content is not preserved. Both `crop_height` and `crop_width` need to be +/// positive. +/// +/// Optional attributes (see `Attrs`): +/// * method: A string specifying the sampling method for resizing. It can be either +/// `"bilinear"` or `"nearest"` and default to `"bilinear"`. Currently two sampling +/// methods are supported: Bilinear and Nearest Neighbor. +/// * extrapolation_value: Value used for extrapolation, when applicable. +/// +/// Returns: +/// * `Output`: A 4-D tensor of shape `[num_boxes, crop_height, crop_width, depth]`. +class CropAndResize { + public: + /// Optional attribute setters for CropAndResize + struct Attrs { + /// A string specifying the sampling method for resizing. It can be either + /// `"bilinear"` or `"nearest"` and default to `"bilinear"`. Currently two sampling + /// methods are supported: Bilinear and Nearest Neighbor. + /// + /// Defaults to "bilinear" + TF_MUST_USE_RESULT Attrs Method(StringPiece x) { + Attrs ret = *this; + ret.method_ = x; + return ret; + } + + /// Value used for extrapolation, when applicable. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs ExtrapolationValue(float x) { + Attrs ret = *this; + ret.extrapolation_value_ = x; + return ret; + } + + StringPiece method_ = "bilinear"; + float extrapolation_value_ = 0.0f; + }; + CropAndResize(const ::tensorflow::Scope& scope, ::tensorflow::Input image, + ::tensorflow::Input boxes, ::tensorflow::Input box_ind, + ::tensorflow::Input crop_size); + CropAndResize(const ::tensorflow::Scope& scope, ::tensorflow::Input image, + ::tensorflow::Input boxes, ::tensorflow::Input box_ind, + ::tensorflow::Input crop_size, const CropAndResize::Attrs& attrs); + operator ::tensorflow::Output() const { return crops; } + operator ::tensorflow::Input() const { return crops; } + ::tensorflow::Node* node() const { return crops.node(); } + + static Attrs Method(StringPiece x) { + return Attrs().Method(x); + } + static Attrs ExtrapolationValue(float x) { + return Attrs().ExtrapolationValue(x); + } + + Operation operation; + ::tensorflow::Output crops; +}; + +/// Computes the gradient of the crop_and_resize op wrt the input boxes tensor. +/// +/// Arguments: +/// * scope: A Scope object +/// * grads: A 4-D tensor of shape `[num_boxes, crop_height, crop_width, depth]`. +/// * image: A 4-D tensor of shape `[batch, image_height, image_width, depth]`. +/// Both `image_height` and `image_width` need to be positive. +/// * boxes: A 2-D tensor of shape `[num_boxes, 4]`. The `i`-th row of the tensor +/// specifies the coordinates of a box in the `box_ind[i]` image and is specified +/// in normalized coordinates `[y1, x1, y2, x2]`. A normalized coordinate value of +/// `y` is mapped to the image coordinate at `y * (image_height - 1)`, so as the +/// `[0, 1]` interval of normalized image height is mapped to +/// `[0, image_height - 1] in image height coordinates. We do allow y1 > y2, in +/// which case the sampled crop is an up-down flipped version of the original +/// image. The width dimension is treated similarly. Normalized coordinates +/// outside the `[0, 1]` range are allowed, in which case we use +/// `extrapolation_value` to extrapolate the input image values. +/// * box_ind: A 1-D tensor of shape `[num_boxes]` with int32 values in `[0, batch)`. +/// The value of `box_ind[i]` specifies the image that the `i`-th box refers to. +/// +/// Optional attributes (see `Attrs`): +/// * method: A string specifying the interpolation method. Only 'bilinear' is +/// supported for now. +/// +/// Returns: +/// * `Output`: A 2-D tensor of shape `[num_boxes, 4]`. +class CropAndResizeGradBoxes { + public: + /// Optional attribute setters for CropAndResizeGradBoxes + struct Attrs { + /// A string specifying the interpolation method. Only 'bilinear' is + /// supported for now. + /// + /// Defaults to "bilinear" + TF_MUST_USE_RESULT Attrs Method(StringPiece x) { + Attrs ret = *this; + ret.method_ = x; + return ret; + } + + StringPiece method_ = "bilinear"; + }; + CropAndResizeGradBoxes(const ::tensorflow::Scope& scope, ::tensorflow::Input + grads, ::tensorflow::Input image, ::tensorflow::Input + boxes, ::tensorflow::Input box_ind); + CropAndResizeGradBoxes(const ::tensorflow::Scope& scope, ::tensorflow::Input + grads, ::tensorflow::Input image, ::tensorflow::Input + boxes, ::tensorflow::Input box_ind, const + CropAndResizeGradBoxes::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs Method(StringPiece x) { + return Attrs().Method(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Computes the gradient of the crop_and_resize op wrt the input image tensor. +/// +/// Arguments: +/// * scope: A Scope object +/// * grads: A 4-D tensor of shape `[num_boxes, crop_height, crop_width, depth]`. +/// * boxes: A 2-D tensor of shape `[num_boxes, 4]`. The `i`-th row of the tensor +/// specifies the coordinates of a box in the `box_ind[i]` image and is specified +/// in normalized coordinates `[y1, x1, y2, x2]`. A normalized coordinate value of +/// `y` is mapped to the image coordinate at `y * (image_height - 1)`, so as the +/// `[0, 1]` interval of normalized image height is mapped to +/// `[0, image_height - 1] in image height coordinates. We do allow y1 > y2, in +/// which case the sampled crop is an up-down flipped version of the original +/// image. The width dimension is treated similarly. Normalized coordinates +/// outside the `[0, 1]` range are allowed, in which case we use +/// `extrapolation_value` to extrapolate the input image values. +/// * box_ind: A 1-D tensor of shape `[num_boxes]` with int32 values in `[0, batch)`. +/// The value of `box_ind[i]` specifies the image that the `i`-th box refers to. +/// * image_size: A 1-D tensor with value `[batch, image_height, image_width, depth]` +/// containing the original image size. Both `image_height` and `image_width` need +/// to be positive. +/// +/// Optional attributes (see `Attrs`): +/// * method: A string specifying the interpolation method. Only 'bilinear' is +/// supported for now. +/// +/// Returns: +/// * `Output`: A 4-D tensor of shape `[batch, image_height, image_width, depth]`. +class CropAndResizeGradImage { + public: + /// Optional attribute setters for CropAndResizeGradImage + struct Attrs { + /// A string specifying the interpolation method. Only 'bilinear' is + /// supported for now. + /// + /// Defaults to "bilinear" + TF_MUST_USE_RESULT Attrs Method(StringPiece x) { + Attrs ret = *this; + ret.method_ = x; + return ret; + } + + StringPiece method_ = "bilinear"; + }; + CropAndResizeGradImage(const ::tensorflow::Scope& scope, ::tensorflow::Input + grads, ::tensorflow::Input boxes, ::tensorflow::Input + box_ind, ::tensorflow::Input image_size, DataType T); + CropAndResizeGradImage(const ::tensorflow::Scope& scope, ::tensorflow::Input + grads, ::tensorflow::Input boxes, ::tensorflow::Input + box_ind, ::tensorflow::Input image_size, DataType T, + const CropAndResizeGradImage::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs Method(StringPiece x) { + return Attrs().Method(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Decode and Crop a JPEG-encoded image to a uint8 tensor. +/// +/// The attr `channels` indicates the desired number of color channels for the +/// decoded image. +/// +/// Accepted values are: +/// +/// * 0: Use the number of channels in the JPEG-encoded image. +/// * 1: output a grayscale image. +/// * 3: output an RGB image. +/// +/// If needed, the JPEG-encoded image is transformed to match the requested number +/// of color channels. +/// +/// The attr `ratio` allows downscaling the image by an integer factor during +/// decoding. Allowed values are: 1, 2, 4, and 8. This is much faster than +/// downscaling the image later. +/// +/// +/// It is equivalent to a combination of decode and crop, but much faster by only +/// decoding partial jpeg image. +/// +/// Arguments: +/// * scope: A Scope object +/// * contents: 0-D. The JPEG-encoded image. +/// * crop_window: 1-D. The crop window: [crop_y, crop_x, crop_height, crop_width]. +/// +/// Optional attributes (see `Attrs`): +/// * channels: Number of color channels for the decoded image. +/// * ratio: Downscaling ratio. +/// * fancy_upscaling: If true use a slower but nicer upscaling of the +/// chroma planes (yuv420/422 only). +/// * try_recover_truncated: If true try to recover an image from truncated input. +/// * acceptable_fraction: The minimum required fraction of lines before a truncated +/// input is accepted. +/// * dct_method: string specifying a hint about the algorithm used for +/// decompression. Defaults to "" which maps to a system-specific +/// default. Currently valid values are ["INTEGER_FAST", +/// "INTEGER_ACCURATE"]. The hint may be ignored (e.g., the internal +/// jpeg library changes to a version that does not have that specific +/// option.) +/// +/// Returns: +/// * `Output`: 3-D with shape `[height, width, channels]`.. +class DecodeAndCropJpeg { + public: + /// Optional attribute setters for DecodeAndCropJpeg + struct Attrs { + /// Number of color channels for the decoded image. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Channels(int64 x) { + Attrs ret = *this; + ret.channels_ = x; + return ret; + } + + /// Downscaling ratio. + /// + /// Defaults to 1 + TF_MUST_USE_RESULT Attrs Ratio(int64 x) { + Attrs ret = *this; + ret.ratio_ = x; + return ret; + } + + /// If true use a slower but nicer upscaling of the + /// chroma planes (yuv420/422 only). + /// + /// Defaults to true + TF_MUST_USE_RESULT Attrs FancyUpscaling(bool x) { + Attrs ret = *this; + ret.fancy_upscaling_ = x; + return ret; + } + + /// If true try to recover an image from truncated input. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs TryRecoverTruncated(bool x) { + Attrs ret = *this; + ret.try_recover_truncated_ = x; + return ret; + } + + /// The minimum required fraction of lines before a truncated + /// input is accepted. + /// + /// Defaults to 1 + TF_MUST_USE_RESULT Attrs AcceptableFraction(float x) { + Attrs ret = *this; + ret.acceptable_fraction_ = x; + return ret; + } + + /// string specifying a hint about the algorithm used for + /// decompression. Defaults to "" which maps to a system-specific + /// default. Currently valid values are ["INTEGER_FAST", + /// "INTEGER_ACCURATE"]. The hint may be ignored (e.g., the internal + /// jpeg library changes to a version that does not have that specific + /// option.) + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs DctMethod(StringPiece x) { + Attrs ret = *this; + ret.dct_method_ = x; + return ret; + } + + int64 channels_ = 0; + int64 ratio_ = 1; + bool fancy_upscaling_ = true; + bool try_recover_truncated_ = false; + float acceptable_fraction_ = 1.0f; + StringPiece dct_method_ = ""; + }; + DecodeAndCropJpeg(const ::tensorflow::Scope& scope, ::tensorflow::Input + contents, ::tensorflow::Input crop_window); + DecodeAndCropJpeg(const ::tensorflow::Scope& scope, ::tensorflow::Input + contents, ::tensorflow::Input crop_window, const + DecodeAndCropJpeg::Attrs& attrs); + operator ::tensorflow::Output() const { return image; } + operator ::tensorflow::Input() const { return image; } + ::tensorflow::Node* node() const { return image.node(); } + + static Attrs Channels(int64 x) { + return Attrs().Channels(x); + } + static Attrs Ratio(int64 x) { + return Attrs().Ratio(x); + } + static Attrs FancyUpscaling(bool x) { + return Attrs().FancyUpscaling(x); + } + static Attrs TryRecoverTruncated(bool x) { + return Attrs().TryRecoverTruncated(x); + } + static Attrs AcceptableFraction(float x) { + return Attrs().AcceptableFraction(x); + } + static Attrs DctMethod(StringPiece x) { + return Attrs().DctMethod(x); + } + + Operation operation; + ::tensorflow::Output image; +}; + +/// Decode the first frame of a BMP-encoded image to a uint8 tensor. +/// +/// The attr `channels` indicates the desired number of color channels for the +/// decoded image. +/// +/// Accepted values are: +/// +/// * 0: Use the number of channels in the BMP-encoded image. +/// * 3: output an RGB image. +/// * 4: output an RGBA image. +/// +/// Arguments: +/// * scope: A Scope object +/// * contents: 0-D. The BMP-encoded image. +/// +/// Returns: +/// * `Output`: 3-D with shape `[height, width, channels]`. RGB order +class DecodeBmp { + public: + /// Optional attribute setters for DecodeBmp + struct Attrs { + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Channels(int64 x) { + Attrs ret = *this; + ret.channels_ = x; + return ret; + } + + int64 channels_ = 0; + }; + DecodeBmp(const ::tensorflow::Scope& scope, ::tensorflow::Input contents); + DecodeBmp(const ::tensorflow::Scope& scope, ::tensorflow::Input contents, const + DecodeBmp::Attrs& attrs); + operator ::tensorflow::Output() const { return image; } + operator ::tensorflow::Input() const { return image; } + ::tensorflow::Node* node() const { return image.node(); } + + static Attrs Channels(int64 x) { + return Attrs().Channels(x); + } + + Operation operation; + ::tensorflow::Output image; +}; + +/// Decode the first frame of a GIF-encoded image to a uint8 tensor. +/// +/// GIF with frame or transparency compression are not supported +/// convert animated GIF from compressed to uncompressed by: +/// +/// convert $src.gif -coalesce $dst.gif +/// +/// This op also supports decoding JPEGs and PNGs, though it is cleaner to use +/// `tf.image.decode_image`. +/// +/// Arguments: +/// * scope: A Scope object +/// * contents: 0-D. The GIF-encoded image. +/// +/// Returns: +/// * `Output`: 4-D with shape `[num_frames, height, width, 3]`. RGB order +class DecodeGif { + public: + DecodeGif(const ::tensorflow::Scope& scope, ::tensorflow::Input contents); + operator ::tensorflow::Output() const { return image; } + operator ::tensorflow::Input() const { return image; } + ::tensorflow::Node* node() const { return image.node(); } + + Operation operation; + ::tensorflow::Output image; +}; + +/// Decode a JPEG-encoded image to a uint8 tensor. +/// +/// The attr `channels` indicates the desired number of color channels for the +/// decoded image. +/// +/// Accepted values are: +/// +/// * 0: Use the number of channels in the JPEG-encoded image. +/// * 1: output a grayscale image. +/// * 3: output an RGB image. +/// +/// If needed, the JPEG-encoded image is transformed to match the requested number +/// of color channels. +/// +/// The attr `ratio` allows downscaling the image by an integer factor during +/// decoding. Allowed values are: 1, 2, 4, and 8. This is much faster than +/// downscaling the image later. +/// +/// +/// This op also supports decoding PNGs and non-animated GIFs since the interface is +/// the same, though it is cleaner to use `tf.image.decode_image`. +/// +/// Arguments: +/// * scope: A Scope object +/// * contents: 0-D. The JPEG-encoded image. +/// +/// Optional attributes (see `Attrs`): +/// * channels: Number of color channels for the decoded image. +/// * ratio: Downscaling ratio. +/// * fancy_upscaling: If true use a slower but nicer upscaling of the +/// chroma planes (yuv420/422 only). +/// * try_recover_truncated: If true try to recover an image from truncated input. +/// * acceptable_fraction: The minimum required fraction of lines before a truncated +/// input is accepted. +/// * dct_method: string specifying a hint about the algorithm used for +/// decompression. Defaults to "" which maps to a system-specific +/// default. Currently valid values are ["INTEGER_FAST", +/// "INTEGER_ACCURATE"]. The hint may be ignored (e.g., the internal +/// jpeg library changes to a version that does not have that specific +/// option.) +/// +/// Returns: +/// * `Output`: 3-D with shape `[height, width, channels]`.. +class DecodeJpeg { + public: + /// Optional attribute setters for DecodeJpeg + struct Attrs { + /// Number of color channels for the decoded image. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Channels(int64 x) { + Attrs ret = *this; + ret.channels_ = x; + return ret; + } + + /// Downscaling ratio. + /// + /// Defaults to 1 + TF_MUST_USE_RESULT Attrs Ratio(int64 x) { + Attrs ret = *this; + ret.ratio_ = x; + return ret; + } + + /// If true use a slower but nicer upscaling of the + /// chroma planes (yuv420/422 only). + /// + /// Defaults to true + TF_MUST_USE_RESULT Attrs FancyUpscaling(bool x) { + Attrs ret = *this; + ret.fancy_upscaling_ = x; + return ret; + } + + /// If true try to recover an image from truncated input. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs TryRecoverTruncated(bool x) { + Attrs ret = *this; + ret.try_recover_truncated_ = x; + return ret; + } + + /// The minimum required fraction of lines before a truncated + /// input is accepted. + /// + /// Defaults to 1 + TF_MUST_USE_RESULT Attrs AcceptableFraction(float x) { + Attrs ret = *this; + ret.acceptable_fraction_ = x; + return ret; + } + + /// string specifying a hint about the algorithm used for + /// decompression. Defaults to "" which maps to a system-specific + /// default. Currently valid values are ["INTEGER_FAST", + /// "INTEGER_ACCURATE"]. The hint may be ignored (e.g., the internal + /// jpeg library changes to a version that does not have that specific + /// option.) + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs DctMethod(StringPiece x) { + Attrs ret = *this; + ret.dct_method_ = x; + return ret; + } + + int64 channels_ = 0; + int64 ratio_ = 1; + bool fancy_upscaling_ = true; + bool try_recover_truncated_ = false; + float acceptable_fraction_ = 1.0f; + StringPiece dct_method_ = ""; + }; + DecodeJpeg(const ::tensorflow::Scope& scope, ::tensorflow::Input contents); + DecodeJpeg(const ::tensorflow::Scope& scope, ::tensorflow::Input contents, + const DecodeJpeg::Attrs& attrs); + operator ::tensorflow::Output() const { return image; } + operator ::tensorflow::Input() const { return image; } + ::tensorflow::Node* node() const { return image.node(); } + + static Attrs Channels(int64 x) { + return Attrs().Channels(x); + } + static Attrs Ratio(int64 x) { + return Attrs().Ratio(x); + } + static Attrs FancyUpscaling(bool x) { + return Attrs().FancyUpscaling(x); + } + static Attrs TryRecoverTruncated(bool x) { + return Attrs().TryRecoverTruncated(x); + } + static Attrs AcceptableFraction(float x) { + return Attrs().AcceptableFraction(x); + } + static Attrs DctMethod(StringPiece x) { + return Attrs().DctMethod(x); + } + + Operation operation; + ::tensorflow::Output image; +}; + +/// Decode a PNG-encoded image to a uint8 or uint16 tensor. +/// +/// The attr `channels` indicates the desired number of color channels for the +/// decoded image. +/// +/// Accepted values are: +/// +/// * 0: Use the number of channels in the PNG-encoded image. +/// * 1: output a grayscale image. +/// * 3: output an RGB image. +/// * 4: output an RGBA image. +/// +/// If needed, the PNG-encoded image is transformed to match the requested number +/// of color channels. +/// +/// This op also supports decoding JPEGs and non-animated GIFs since the interface +/// is the same, though it is cleaner to use `tf.image.decode_image`. +/// +/// Arguments: +/// * scope: A Scope object +/// * contents: 0-D. The PNG-encoded image. +/// +/// Optional attributes (see `Attrs`): +/// * channels: Number of color channels for the decoded image. +/// +/// Returns: +/// * `Output`: 3-D with shape `[height, width, channels]`. +class DecodePng { + public: + /// Optional attribute setters for DecodePng + struct Attrs { + /// Number of color channels for the decoded image. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Channels(int64 x) { + Attrs ret = *this; + ret.channels_ = x; + return ret; + } + + /// Defaults to DT_UINT8 + TF_MUST_USE_RESULT Attrs Dtype(DataType x) { + Attrs ret = *this; + ret.dtype_ = x; + return ret; + } + + int64 channels_ = 0; + DataType dtype_ = DT_UINT8; + }; + DecodePng(const ::tensorflow::Scope& scope, ::tensorflow::Input contents); + DecodePng(const ::tensorflow::Scope& scope, ::tensorflow::Input contents, const + DecodePng::Attrs& attrs); + operator ::tensorflow::Output() const { return image; } + operator ::tensorflow::Input() const { return image; } + ::tensorflow::Node* node() const { return image.node(); } + + static Attrs Channels(int64 x) { + return Attrs().Channels(x); + } + static Attrs Dtype(DataType x) { + return Attrs().Dtype(x); + } + + Operation operation; + ::tensorflow::Output image; +}; + +/// Draw bounding boxes on a batch of images. +/// +/// Outputs a copy of `images` but draws on top of the pixels zero or more bounding +/// boxes specified by the locations in `boxes`. The coordinates of the each +/// bounding box in `boxes` are encoded as `[y_min, x_min, y_max, x_max]`. The +/// bounding box coordinates are floats in `[0.0, 1.0]` relative to the width and +/// height of the underlying image. +/// +/// For example, if an image is 100 x 200 pixels (height x width) and the bounding +/// box is `[0.1, 0.2, 0.5, 0.9]`, the upper-left and bottom-right coordinates of +/// the bounding box will be `(40, 10)` to `(180, 50)` (in (x,y) coordinates). +/// +/// Parts of the bounding box may fall outside the image. +/// +/// Arguments: +/// * scope: A Scope object +/// * images: 4-D with shape `[batch, height, width, depth]`. A batch of images. +/// * boxes: 3-D with shape `[batch, num_bounding_boxes, 4]` containing bounding +/// boxes. +/// +/// Returns: +/// * `Output`: 4-D with the same shape as `images`. The batch of input images with +/// bounding boxes drawn on the images. +class DrawBoundingBoxes { + public: + DrawBoundingBoxes(const ::tensorflow::Scope& scope, ::tensorflow::Input images, + ::tensorflow::Input boxes); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// JPEG-encode an image. +/// +/// `image` is a 3-D uint8 Tensor of shape `[height, width, channels]`. +/// +/// The attr `format` can be used to override the color format of the encoded +/// output. Values can be: +/// +/// * `''`: Use a default format based on the number of channels in the image. +/// * `grayscale`: Output a grayscale JPEG image. The `channels` dimension +/// of `image` must be 1. +/// * `rgb`: Output an RGB JPEG image. The `channels` dimension +/// of `image` must be 3. +/// +/// If `format` is not specified or is the empty string, a default format is picked +/// in function of the number of channels in `image`: +/// +/// * 1: Output a grayscale image. +/// * 3: Output an RGB image. +/// +/// Arguments: +/// * scope: A Scope object +/// * image: 3-D with shape `[height, width, channels]`. +/// +/// Optional attributes (see `Attrs`): +/// * format: Per pixel image format. +/// * quality: Quality of the compression from 0 to 100 (higher is better and slower). +/// * progressive: If True, create a JPEG that loads progressively (coarse to fine). +/// * optimize_size: If True, spend CPU/RAM to reduce size with no quality change. +/// * chroma_downsampling: See http://en.wikipedia.org/wiki/Chroma_subsampling. +/// * density_unit: Unit used to specify `x_density` and `y_density`: +/// pixels per inch (`'in'`) or centimeter (`'cm'`). +/// * x_density: Horizontal pixels per density unit. +/// * y_density: Vertical pixels per density unit. +/// * xmp_metadata: If not empty, embed this XMP metadata in the image header. +/// +/// Returns: +/// * `Output`: 0-D. JPEG-encoded image. +class EncodeJpeg { + public: + /// Optional attribute setters for EncodeJpeg + struct Attrs { + /// Per pixel image format. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs Format(StringPiece x) { + Attrs ret = *this; + ret.format_ = x; + return ret; + } + + /// Quality of the compression from 0 to 100 (higher is better and slower). + /// + /// Defaults to 95 + TF_MUST_USE_RESULT Attrs Quality(int64 x) { + Attrs ret = *this; + ret.quality_ = x; + return ret; + } + + /// If True, create a JPEG that loads progressively (coarse to fine). + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs Progressive(bool x) { + Attrs ret = *this; + ret.progressive_ = x; + return ret; + } + + /// If True, spend CPU/RAM to reduce size with no quality change. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs OptimizeSize(bool x) { + Attrs ret = *this; + ret.optimize_size_ = x; + return ret; + } + + /// See http://en.wikipedia.org/wiki/Chroma_subsampling. + /// + /// Defaults to true + TF_MUST_USE_RESULT Attrs ChromaDownsampling(bool x) { + Attrs ret = *this; + ret.chroma_downsampling_ = x; + return ret; + } + + /// Unit used to specify `x_density` and `y_density`: + /// pixels per inch (`'in'`) or centimeter (`'cm'`). + /// + /// Defaults to "in" + TF_MUST_USE_RESULT Attrs DensityUnit(StringPiece x) { + Attrs ret = *this; + ret.density_unit_ = x; + return ret; + } + + /// Horizontal pixels per density unit. + /// + /// Defaults to 300 + TF_MUST_USE_RESULT Attrs XDensity(int64 x) { + Attrs ret = *this; + ret.x_density_ = x; + return ret; + } + + /// Vertical pixels per density unit. + /// + /// Defaults to 300 + TF_MUST_USE_RESULT Attrs YDensity(int64 x) { + Attrs ret = *this; + ret.y_density_ = x; + return ret; + } + + /// If not empty, embed this XMP metadata in the image header. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs XmpMetadata(StringPiece x) { + Attrs ret = *this; + ret.xmp_metadata_ = x; + return ret; + } + + StringPiece format_ = ""; + int64 quality_ = 95; + bool progressive_ = false; + bool optimize_size_ = false; + bool chroma_downsampling_ = true; + StringPiece density_unit_ = "in"; + int64 x_density_ = 300; + int64 y_density_ = 300; + StringPiece xmp_metadata_ = ""; + }; + EncodeJpeg(const ::tensorflow::Scope& scope, ::tensorflow::Input image); + EncodeJpeg(const ::tensorflow::Scope& scope, ::tensorflow::Input image, const + EncodeJpeg::Attrs& attrs); + operator ::tensorflow::Output() const { return contents; } + operator ::tensorflow::Input() const { return contents; } + ::tensorflow::Node* node() const { return contents.node(); } + + static Attrs Format(StringPiece x) { + return Attrs().Format(x); + } + static Attrs Quality(int64 x) { + return Attrs().Quality(x); + } + static Attrs Progressive(bool x) { + return Attrs().Progressive(x); + } + static Attrs OptimizeSize(bool x) { + return Attrs().OptimizeSize(x); + } + static Attrs ChromaDownsampling(bool x) { + return Attrs().ChromaDownsampling(x); + } + static Attrs DensityUnit(StringPiece x) { + return Attrs().DensityUnit(x); + } + static Attrs XDensity(int64 x) { + return Attrs().XDensity(x); + } + static Attrs YDensity(int64 x) { + return Attrs().YDensity(x); + } + static Attrs XmpMetadata(StringPiece x) { + return Attrs().XmpMetadata(x); + } + + Operation operation; + ::tensorflow::Output contents; +}; + +/// PNG-encode an image. +/// +/// `image` is a 3-D uint8 or uint16 Tensor of shape `[height, width, channels]` +/// where `channels` is: +/// +/// * 1: for grayscale. +/// * 2: for grayscale + alpha. +/// * 3: for RGB. +/// * 4: for RGBA. +/// +/// The ZLIB compression level, `compression`, can be -1 for the PNG-encoder +/// default or a value from 0 to 9. 9 is the highest compression level, generating +/// the smallest output, but is slower. +/// +/// Arguments: +/// * scope: A Scope object +/// * image: 3-D with shape `[height, width, channels]`. +/// +/// Optional attributes (see `Attrs`): +/// * compression: Compression level. +/// +/// Returns: +/// * `Output`: 0-D. PNG-encoded image. +class EncodePng { + public: + /// Optional attribute setters for EncodePng + struct Attrs { + /// Compression level. + /// + /// Defaults to -1 + TF_MUST_USE_RESULT Attrs Compression(int64 x) { + Attrs ret = *this; + ret.compression_ = x; + return ret; + } + + int64 compression_ = -1; + }; + EncodePng(const ::tensorflow::Scope& scope, ::tensorflow::Input image); + EncodePng(const ::tensorflow::Scope& scope, ::tensorflow::Input image, const + EncodePng::Attrs& attrs); + operator ::tensorflow::Output() const { return contents; } + operator ::tensorflow::Input() const { return contents; } + ::tensorflow::Node* node() const { return contents.node(); } + + static Attrs Compression(int64 x) { + return Attrs().Compression(x); + } + + Operation operation; + ::tensorflow::Output contents; +}; + +/// Extracts a glimpse from the input tensor. +/// +/// Returns a set of windows called glimpses extracted at location +/// `offsets` from the input tensor. If the windows only partially +/// overlaps the inputs, the non overlapping areas will be filled with +/// random noise. +/// +/// The result is a 4-D tensor of shape `[batch_size, glimpse_height, +/// glimpse_width, channels]`. The channels and batch dimensions are the +/// same as that of the input tensor. The height and width of the output +/// windows are specified in the `size` parameter. +/// +/// The argument `normalized` and `centered` controls how the windows are built: +/// +/// * If the coordinates are normalized but not centered, 0.0 and 1.0 +/// correspond to the minimum and maximum of each height and width +/// dimension. +/// * If the coordinates are both normalized and centered, they range from +/// -1.0 to 1.0. The coordinates (-1.0, -1.0) correspond to the upper +/// left corner, the lower right corner is located at (1.0, 1.0) and the +/// center is at (0, 0). +/// * If the coordinates are not normalized they are interpreted as +/// numbers of pixels. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: A 4-D float tensor of shape `[batch_size, height, width, channels]`. +/// * size: A 1-D tensor of 2 elements containing the size of the glimpses +/// to extract. The glimpse height must be specified first, following +/// by the glimpse width. +/// * offsets: A 2-D integer tensor of shape `[batch_size, 2]` containing +/// the y, x locations of the center of each window. +/// +/// Optional attributes (see `Attrs`): +/// * centered: indicates if the offset coordinates are centered relative to +/// the image, in which case the (0, 0) offset is relative to the center +/// of the input images. If false, the (0,0) offset corresponds to the +/// upper left corner of the input images. +/// * normalized: indicates if the offset coordinates are normalized. +/// * uniform_noise: indicates if the noise should be generated using a +/// uniform distribution or a Gaussian distribution. +/// +/// Returns: +/// * `Output`: A tensor representing the glimpses `[batch_size, +/// glimpse_height, glimpse_width, channels]`. +class ExtractGlimpse { + public: + /// Optional attribute setters for ExtractGlimpse + struct Attrs { + /// indicates if the offset coordinates are centered relative to + /// the image, in which case the (0, 0) offset is relative to the center + /// of the input images. If false, the (0,0) offset corresponds to the + /// upper left corner of the input images. + /// + /// Defaults to true + TF_MUST_USE_RESULT Attrs Centered(bool x) { + Attrs ret = *this; + ret.centered_ = x; + return ret; + } + + /// indicates if the offset coordinates are normalized. + /// + /// Defaults to true + TF_MUST_USE_RESULT Attrs Normalized(bool x) { + Attrs ret = *this; + ret.normalized_ = x; + return ret; + } + + /// indicates if the noise should be generated using a + /// uniform distribution or a Gaussian distribution. + /// + /// Defaults to true + TF_MUST_USE_RESULT Attrs UniformNoise(bool x) { + Attrs ret = *this; + ret.uniform_noise_ = x; + return ret; + } + + bool centered_ = true; + bool normalized_ = true; + bool uniform_noise_ = true; + }; + ExtractGlimpse(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input size, ::tensorflow::Input offsets); + ExtractGlimpse(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input size, ::tensorflow::Input offsets, const + ExtractGlimpse::Attrs& attrs); + operator ::tensorflow::Output() const { return glimpse; } + operator ::tensorflow::Input() const { return glimpse; } + ::tensorflow::Node* node() const { return glimpse.node(); } + + static Attrs Centered(bool x) { + return Attrs().Centered(x); + } + static Attrs Normalized(bool x) { + return Attrs().Normalized(x); + } + static Attrs UniformNoise(bool x) { + return Attrs().UniformNoise(x); + } + + Operation operation; + ::tensorflow::Output glimpse; +}; + +/// Extract the shape information of a JPEG-encoded image. +/// +/// This op only parses the image header, so it is much faster than DecodeJpeg. +/// +/// Arguments: +/// * scope: A Scope object +/// * contents: 0-D. The JPEG-encoded image. +/// +/// Optional attributes (see `Attrs`): +/// * output_type: (Optional) The output type of the operation (int32 or int64). +/// Defaults to int32. +/// +/// Returns: +/// * `Output`: 1-D. The image shape with format [height, width, channels]. +class ExtractJpegShape { + public: + /// Optional attribute setters for ExtractJpegShape + struct Attrs { + /// (Optional) The output type of the operation (int32 or int64). + /// Defaults to int32. + /// + /// Defaults to DT_INT32 + TF_MUST_USE_RESULT Attrs OutputType(DataType x) { + Attrs ret = *this; + ret.output_type_ = x; + return ret; + } + + DataType output_type_ = DT_INT32; + }; + ExtractJpegShape(const ::tensorflow::Scope& scope, ::tensorflow::Input + contents); + ExtractJpegShape(const ::tensorflow::Scope& scope, ::tensorflow::Input + contents, const ExtractJpegShape::Attrs& attrs); + operator ::tensorflow::Output() const { return image_shape; } + operator ::tensorflow::Input() const { return image_shape; } + ::tensorflow::Node* node() const { return image_shape.node(); } + + static Attrs OutputType(DataType x) { + return Attrs().OutputType(x); + } + + Operation operation; + ::tensorflow::Output image_shape; +}; + +/// Convert one or more images from HSV to RGB. +/// +/// Outputs a tensor of the same shape as the `images` tensor, containing the RGB +/// value of the pixels. The output is only well defined if the value in `images` +/// are in `[0,1]`. +/// +/// See `rgb_to_hsv` for a description of the HSV encoding. +/// +/// Arguments: +/// * scope: A Scope object +/// * images: 1-D or higher rank. HSV data to convert. Last dimension must be size 3. +/// +/// Returns: +/// * `Output`: `images` converted to RGB. +class HSVToRGB { + public: + HSVToRGB(const ::tensorflow::Scope& scope, ::tensorflow::Input images); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Greedily selects a subset of bounding boxes in descending order of score, +/// +/// pruning away boxes that have high intersection-over-union (IOU) overlap +/// with previously selected boxes. Bounding boxes are supplied as +/// [y1, x1, y2, x2], where (y1, x1) and (y2, x2) are the coordinates of any +/// diagonal pair of box corners and the coordinates can be provided as normalized +/// (i.e., lying in the interval [0, 1]) or absolute. Note that this algorithm +/// is agnostic to where the origin is in the coordinate system. Note that this +/// algorithm is invariant to orthogonal transformations and translations +/// of the coordinate system; thus translating or reflections of the coordinate +/// system result in the same boxes being selected by the algorithm. +/// The output of this operation is a set of integers indexing into the input +/// collection of bounding boxes representing the selected boxes. The bounding +/// box coordinates corresponding to the selected indices can then be obtained +/// using the `tf.gather operation`. For example: +/// selected_indices = tf.image.non_max_suppression( +/// boxes, scores, max_output_size, iou_threshold) +/// selected_boxes = tf.gather(boxes, selected_indices) +/// +/// Arguments: +/// * scope: A Scope object +/// * boxes: A 2-D float tensor of shape `[num_boxes, 4]`. +/// * scores: A 1-D float tensor of shape `[num_boxes]` representing a single +/// score corresponding to each box (each row of boxes). +/// * max_output_size: A scalar integer tensor representing the maximum number of +/// boxes to be selected by non max suppression. +/// +/// Optional attributes (see `Attrs`): +/// * iou_threshold: A float representing the threshold for deciding whether boxes +/// overlap too much with respect to IOU. +/// +/// Returns: +/// * `Output`: A 1-D integer tensor of shape `[M]` representing the selected +/// indices from the boxes tensor, where `M <= max_output_size`. +class NonMaxSuppression { + public: + /// Optional attribute setters for NonMaxSuppression + struct Attrs { + /// A float representing the threshold for deciding whether boxes + /// overlap too much with respect to IOU. + /// + /// Defaults to 0.5 + TF_MUST_USE_RESULT Attrs IouThreshold(float x) { + Attrs ret = *this; + ret.iou_threshold_ = x; + return ret; + } + + float iou_threshold_ = 0.5f; + }; + NonMaxSuppression(const ::tensorflow::Scope& scope, ::tensorflow::Input boxes, + ::tensorflow::Input scores, ::tensorflow::Input + max_output_size); + NonMaxSuppression(const ::tensorflow::Scope& scope, ::tensorflow::Input boxes, + ::tensorflow::Input scores, ::tensorflow::Input + max_output_size, const NonMaxSuppression::Attrs& attrs); + operator ::tensorflow::Output() const { return selected_indices; } + operator ::tensorflow::Input() const { return selected_indices; } + ::tensorflow::Node* node() const { return selected_indices.node(); } + + static Attrs IouThreshold(float x) { + return Attrs().IouThreshold(x); + } + + Operation operation; + ::tensorflow::Output selected_indices; +}; + +/// Greedily selects a subset of bounding boxes in descending order of score, +/// +/// pruning away boxes that have high intersection-over-union (IOU) overlap +/// with previously selected boxes. Bounding boxes are supplied as +/// [y1, x1, y2, x2], where (y1, x1) and (y2, x2) are the coordinates of any +/// diagonal pair of box corners and the coordinates can be provided as normalized +/// (i.e., lying in the interval [0, 1]) or absolute. Note that this algorithm +/// is agnostic to where the origin is in the coordinate system. Note that this +/// algorithm is invariant to orthogonal transformations and translations +/// of the coordinate system; thus translating or reflections of the coordinate +/// system result in the same boxes being selected by the algorithm. +/// +/// The output of this operation is a set of integers indexing into the input +/// collection of bounding boxes representing the selected boxes. The bounding +/// box coordinates corresponding to the selected indices can then be obtained +/// using the `tf.gather operation`. For example: +/// +/// selected_indices = tf.image.non_max_suppression_v2( +/// boxes, scores, max_output_size, iou_threshold) +/// selected_boxes = tf.gather(boxes, selected_indices) +/// +/// Arguments: +/// * scope: A Scope object +/// * boxes: A 2-D float tensor of shape `[num_boxes, 4]`. +/// * scores: A 1-D float tensor of shape `[num_boxes]` representing a single +/// score corresponding to each box (each row of boxes). +/// * max_output_size: A scalar integer tensor representing the maximum number of +/// boxes to be selected by non max suppression. +/// * iou_threshold: A 0-D float tensor representing the threshold for deciding whether +/// boxes overlap too much with respect to IOU. +/// +/// Returns: +/// * `Output`: A 1-D integer tensor of shape `[M]` representing the selected +/// indices from the boxes tensor, where `M <= max_output_size`. +class NonMaxSuppressionV2 { + public: + NonMaxSuppressionV2(const ::tensorflow::Scope& scope, ::tensorflow::Input + boxes, ::tensorflow::Input scores, ::tensorflow::Input + max_output_size, ::tensorflow::Input iou_threshold); + operator ::tensorflow::Output() const { return selected_indices; } + operator ::tensorflow::Input() const { return selected_indices; } + ::tensorflow::Node* node() const { return selected_indices.node(); } + + Operation operation; + ::tensorflow::Output selected_indices; +}; + +/// Greedily selects a subset of bounding boxes in descending order of score, +/// +/// pruning away boxes that have high intersection-over-union (IOU) overlap +/// with previously selected boxes. Bounding boxes with score less than +/// `score_threshold` are removed. Bounding boxes are supplied as +/// [y1, x1, y2, x2], where (y1, x1) and (y2, x2) are the coordinates of any +/// diagonal pair of box corners and the coordinates can be provided as normalized +/// (i.e., lying in the interval [0, 1]) or absolute. Note that this algorithm +/// is agnostic to where the origin is in the coordinate system and more +/// generally is invariant to orthogonal transformations and translations +/// of the coordinate system; thus translating or reflections of the coordinate +/// system result in the same boxes being selected by the algorithm. +/// The output of this operation is a set of integers indexing into the input +/// collection of bounding boxes representing the selected boxes. The bounding +/// box coordinates corresponding to the selected indices can then be obtained +/// using the `tf.gather operation`. For example: +/// selected_indices = tf.image.non_max_suppression_v2( +/// boxes, scores, max_output_size, iou_threshold, score_threshold) +/// selected_boxes = tf.gather(boxes, selected_indices) +/// +/// Arguments: +/// * scope: A Scope object +/// * boxes: A 2-D float tensor of shape `[num_boxes, 4]`. +/// * scores: A 1-D float tensor of shape `[num_boxes]` representing a single +/// score corresponding to each box (each row of boxes). +/// * max_output_size: A scalar integer tensor representing the maximum number of +/// boxes to be selected by non max suppression. +/// * iou_threshold: A 0-D float tensor representing the threshold for deciding whether +/// boxes overlap too much with respect to IOU. +/// * score_threshold: A 0-D float tensor representing the threshold for deciding when to remove +/// boxes based on score. +/// +/// Returns: +/// * `Output`: A 1-D integer tensor of shape `[M]` representing the selected +/// indices from the boxes tensor, where `M <= max_output_size`. +class NonMaxSuppressionV3 { + public: + NonMaxSuppressionV3(const ::tensorflow::Scope& scope, ::tensorflow::Input + boxes, ::tensorflow::Input scores, ::tensorflow::Input + max_output_size, ::tensorflow::Input iou_threshold, + ::tensorflow::Input score_threshold); + operator ::tensorflow::Output() const { return selected_indices; } + operator ::tensorflow::Input() const { return selected_indices; } + ::tensorflow::Node* node() const { return selected_indices.node(); } + + Operation operation; + ::tensorflow::Output selected_indices; +}; + +/// Greedily selects a subset of bounding boxes in descending order of score, +/// +/// pruning away boxes that have high intersection-over-union (IOU) overlap +/// with previously selected boxes. Bounding boxes with score less than +/// `score_threshold` are removed. Bounding boxes are supplied as +/// [y1, x1, y2, x2], where (y1, x1) and (y2, x2) are the coordinates of any +/// diagonal pair of box corners and the coordinates can be provided as normalized +/// (i.e., lying in the interval [0, 1]) or absolute. Note that this algorithm +/// is agnostic to where the origin is in the coordinate system and more +/// generally is invariant to orthogonal transformations and translations +/// of the coordinate system; thus translating or reflections of the coordinate +/// system result in the same boxes being selected by the algorithm. +/// The output of this operation is a set of integers indexing into the input +/// collection of bounding boxes representing the selected boxes. The bounding +/// box coordinates corresponding to the selected indices can then be obtained +/// using the `tf.gather operation`. For example: +/// selected_indices = tf.image.non_max_suppression_v2( +/// boxes, scores, max_output_size, iou_threshold, score_threshold) +/// selected_boxes = tf.gather(boxes, selected_indices) +/// +/// Arguments: +/// * scope: A Scope object +/// * boxes: A 2-D float tensor of shape `[num_boxes, 4]`. +/// * scores: A 1-D float tensor of shape `[num_boxes]` representing a single +/// score corresponding to each box (each row of boxes). +/// * max_output_size: A scalar integer tensor representing the maximum number of +/// boxes to be selected by non max suppression. +/// * iou_threshold: A 0-D float tensor representing the threshold for deciding whether +/// boxes overlap too much with respect to IOU. +/// * score_threshold: A 0-D float tensor representing the threshold for deciding when to remove +/// boxes based on score. +/// +/// Optional attributes (see `Attrs`): +/// * pad_to_max_output_size: If true, the output `selected_indices` is padded to be of length +/// `max_output_size`. Defaults to false. +/// +/// Returns: +/// * `Output` selected_indices: A 1-D integer tensor of shape `[M]` representing the selected +/// indices from the boxes tensor, where `M <= max_output_size`. +/// * `Output` valid_outputs: A 0-D integer tensor representing the number of valid elements in +/// `selected_indices`, with the valid elements appearing first. +class NonMaxSuppressionV4 { + public: + /// Optional attribute setters for NonMaxSuppressionV4 + struct Attrs { + /// If true, the output `selected_indices` is padded to be of length + /// `max_output_size`. Defaults to false. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs PadToMaxOutputSize(bool x) { + Attrs ret = *this; + ret.pad_to_max_output_size_ = x; + return ret; + } + + bool pad_to_max_output_size_ = false; + }; + NonMaxSuppressionV4(const ::tensorflow::Scope& scope, ::tensorflow::Input + boxes, ::tensorflow::Input scores, ::tensorflow::Input + max_output_size, ::tensorflow::Input iou_threshold, + ::tensorflow::Input score_threshold); + NonMaxSuppressionV4(const ::tensorflow::Scope& scope, ::tensorflow::Input + boxes, ::tensorflow::Input scores, ::tensorflow::Input + max_output_size, ::tensorflow::Input iou_threshold, + ::tensorflow::Input score_threshold, const + NonMaxSuppressionV4::Attrs& attrs); + + static Attrs PadToMaxOutputSize(bool x) { + return Attrs().PadToMaxOutputSize(x); + } + + Operation operation; + ::tensorflow::Output selected_indices; + ::tensorflow::Output valid_outputs; +}; + +/// Greedily selects a subset of bounding boxes in descending order of score, +/// +/// pruning away boxes that have high overlaps +/// with previously selected boxes. Bounding boxes with score less than +/// `score_threshold` are removed. N-by-n overlap values are supplied as square matrix, +/// which allows for defining a custom overlap criterium (eg. intersection over union, +/// intersection over area, etc.). +/// +/// The output of this operation is a set of integers indexing into the input +/// collection of bounding boxes representing the selected boxes. The bounding +/// box coordinates corresponding to the selected indices can then be obtained +/// using the `tf.gather operation`. For example: +/// +/// selected_indices = tf.image.non_max_suppression_with_overlaps( +/// overlaps, scores, max_output_size, overlap_threshold, score_threshold) +/// selected_boxes = tf.gather(boxes, selected_indices) +/// +/// Arguments: +/// * scope: A Scope object +/// * overlaps: A 2-D float tensor of shape `[num_boxes, num_boxes]` representing +/// the n-by-n box overlap values. +/// * scores: A 1-D float tensor of shape `[num_boxes]` representing a single +/// score corresponding to each box (each row of boxes). +/// * max_output_size: A scalar integer tensor representing the maximum number of +/// boxes to be selected by non max suppression. +/// * overlap_threshold: A 0-D float tensor representing the threshold for deciding whether +/// boxes overlap too. +/// * score_threshold: A 0-D float tensor representing the threshold for deciding when to remove +/// boxes based on score. +/// +/// Returns: +/// * `Output`: A 1-D integer tensor of shape `[M]` representing the selected +/// indices from the boxes tensor, where `M <= max_output_size`. +class NonMaxSuppressionWithOverlaps { + public: + NonMaxSuppressionWithOverlaps(const ::tensorflow::Scope& scope, + ::tensorflow::Input overlaps, ::tensorflow::Input + scores, ::tensorflow::Input max_output_size, + ::tensorflow::Input overlap_threshold, + ::tensorflow::Input score_threshold); + operator ::tensorflow::Output() const { return selected_indices; } + operator ::tensorflow::Input() const { return selected_indices; } + ::tensorflow::Node* node() const { return selected_indices.node(); } + + Operation operation; + ::tensorflow::Output selected_indices; +}; + +/// Resize quantized `images` to `size` using quantized bilinear interpolation. +/// +/// Input images and output images must be quantized types. +/// +/// Arguments: +/// * scope: A Scope object +/// * images: 4-D with shape `[batch, height, width, channels]`. +/// * size: = A 1-D int32 Tensor of 2 elements: `new_height, new_width`. The +/// new size for the images. +/// +/// Optional attributes (see `Attrs`): +/// * align_corners: If true, the centers of the 4 corner pixels of the input and output tensors are +/// aligned, preserving the values at the corner pixels. Defaults to false. +/// +/// Returns: +/// * `Output` resized_images: 4-D with shape +/// `[batch, new_height, new_width, channels]`. +/// * `Output` out_min +/// * `Output` out_max +class QuantizedResizeBilinear { + public: + /// Optional attribute setters for QuantizedResizeBilinear + struct Attrs { + /// If true, the centers of the 4 corner pixels of the input and output tensors are + /// aligned, preserving the values at the corner pixels. Defaults to false. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs AlignCorners(bool x) { + Attrs ret = *this; + ret.align_corners_ = x; + return ret; + } + + bool align_corners_ = false; + }; + QuantizedResizeBilinear(const ::tensorflow::Scope& scope, ::tensorflow::Input + images, ::tensorflow::Input size, ::tensorflow::Input + min, ::tensorflow::Input max); + QuantizedResizeBilinear(const ::tensorflow::Scope& scope, ::tensorflow::Input + images, ::tensorflow::Input size, ::tensorflow::Input + min, ::tensorflow::Input max, const + QuantizedResizeBilinear::Attrs& attrs); + + static Attrs AlignCorners(bool x) { + return Attrs().AlignCorners(x); + } + + Operation operation; + ::tensorflow::Output resized_images; + ::tensorflow::Output out_min; + ::tensorflow::Output out_max; +}; + +/// Converts one or more images from RGB to HSV. +/// +/// Outputs a tensor of the same shape as the `images` tensor, containing the HSV +/// value of the pixels. The output is only well defined if the value in `images` +/// are in `[0,1]`. +/// +/// `output[..., 0]` contains hue, `output[..., 1]` contains saturation, and +/// `output[..., 2]` contains value. All HSV values are in `[0,1]`. A hue of 0 +/// corresponds to pure red, hue 1/3 is pure green, and 2/3 is pure blue. +/// +/// Arguments: +/// * scope: A Scope object +/// * images: 1-D or higher rank. RGB data to convert. Last dimension must be size 3. +/// +/// Returns: +/// * `Output`: `images` converted to HSV. +class RGBToHSV { + public: + RGBToHSV(const ::tensorflow::Scope& scope, ::tensorflow::Input images); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Resize `images` to `size` using area interpolation. +/// +/// Input images can be of different types but output images are always float. +/// +/// The range of pixel values for the output image might be slightly different +/// from the range for the input image because of limited numerical precision. +/// To guarantee an output range, for example `[0.0, 1.0]`, apply +/// `tf.clip_by_value` to the output. +/// +/// Each output pixel is computed by first transforming the pixel's footprint into +/// the input tensor and then averaging the pixels that intersect the footprint. An +/// input pixel's contribution to the average is weighted by the fraction of its +/// area that intersects the footprint. This is the same as OpenCV's INTER_AREA. +/// +/// Arguments: +/// * scope: A Scope object +/// * images: 4-D with shape `[batch, height, width, channels]`. +/// * size: = A 1-D int32 Tensor of 2 elements: `new_height, new_width`. The +/// new size for the images. +/// +/// Optional attributes (see `Attrs`): +/// * align_corners: If true, the centers of the 4 corner pixels of the input and output tensors are +/// aligned, preserving the values at the corner pixels. Defaults to false. +/// +/// Returns: +/// * `Output`: 4-D with shape +/// `[batch, new_height, new_width, channels]`. +class ResizeArea { + public: + /// Optional attribute setters for ResizeArea + struct Attrs { + /// If true, the centers of the 4 corner pixels of the input and output tensors are + /// aligned, preserving the values at the corner pixels. Defaults to false. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs AlignCorners(bool x) { + Attrs ret = *this; + ret.align_corners_ = x; + return ret; + } + + bool align_corners_ = false; + }; + ResizeArea(const ::tensorflow::Scope& scope, ::tensorflow::Input images, + ::tensorflow::Input size); + ResizeArea(const ::tensorflow::Scope& scope, ::tensorflow::Input images, + ::tensorflow::Input size, const ResizeArea::Attrs& attrs); + operator ::tensorflow::Output() const { return resized_images; } + operator ::tensorflow::Input() const { return resized_images; } + ::tensorflow::Node* node() const { return resized_images.node(); } + + static Attrs AlignCorners(bool x) { + return Attrs().AlignCorners(x); + } + + Operation operation; + ::tensorflow::Output resized_images; +}; + +/// Resize `images` to `size` using bicubic interpolation. +/// +/// Input images can be of different types but output images are always float. +/// +/// Arguments: +/// * scope: A Scope object +/// * images: 4-D with shape `[batch, height, width, channels]`. +/// * size: = A 1-D int32 Tensor of 2 elements: `new_height, new_width`. The +/// new size for the images. +/// +/// Optional attributes (see `Attrs`): +/// * align_corners: If true, the centers of the 4 corner pixels of the input and output tensors are +/// aligned, preserving the values at the corner pixels. Defaults to false. +/// +/// Returns: +/// * `Output`: 4-D with shape +/// `[batch, new_height, new_width, channels]`. +class ResizeBicubic { + public: + /// Optional attribute setters for ResizeBicubic + struct Attrs { + /// If true, the centers of the 4 corner pixels of the input and output tensors are + /// aligned, preserving the values at the corner pixels. Defaults to false. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs AlignCorners(bool x) { + Attrs ret = *this; + ret.align_corners_ = x; + return ret; + } + + bool align_corners_ = false; + }; + ResizeBicubic(const ::tensorflow::Scope& scope, ::tensorflow::Input images, + ::tensorflow::Input size); + ResizeBicubic(const ::tensorflow::Scope& scope, ::tensorflow::Input images, + ::tensorflow::Input size, const ResizeBicubic::Attrs& attrs); + operator ::tensorflow::Output() const { return resized_images; } + operator ::tensorflow::Input() const { return resized_images; } + ::tensorflow::Node* node() const { return resized_images.node(); } + + static Attrs AlignCorners(bool x) { + return Attrs().AlignCorners(x); + } + + Operation operation; + ::tensorflow::Output resized_images; +}; + +/// Resize `images` to `size` using bilinear interpolation. +/// +/// Input images can be of different types but output images are always float. +/// +/// Arguments: +/// * scope: A Scope object +/// * images: 4-D with shape `[batch, height, width, channels]`. +/// * size: = A 1-D int32 Tensor of 2 elements: `new_height, new_width`. The +/// new size for the images. +/// +/// Optional attributes (see `Attrs`): +/// * align_corners: If true, the centers of the 4 corner pixels of the input and output tensors are +/// aligned, preserving the values at the corner pixels. Defaults to false. +/// +/// Returns: +/// * `Output`: 4-D with shape +/// `[batch, new_height, new_width, channels]`. +class ResizeBilinear { + public: + /// Optional attribute setters for ResizeBilinear + struct Attrs { + /// If true, the centers of the 4 corner pixels of the input and output tensors are + /// aligned, preserving the values at the corner pixels. Defaults to false. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs AlignCorners(bool x) { + Attrs ret = *this; + ret.align_corners_ = x; + return ret; + } + + bool align_corners_ = false; + }; + ResizeBilinear(const ::tensorflow::Scope& scope, ::tensorflow::Input images, + ::tensorflow::Input size); + ResizeBilinear(const ::tensorflow::Scope& scope, ::tensorflow::Input images, + ::tensorflow::Input size, const ResizeBilinear::Attrs& attrs); + operator ::tensorflow::Output() const { return resized_images; } + operator ::tensorflow::Input() const { return resized_images; } + ::tensorflow::Node* node() const { return resized_images.node(); } + + static Attrs AlignCorners(bool x) { + return Attrs().AlignCorners(x); + } + + Operation operation; + ::tensorflow::Output resized_images; +}; + +/// Resize `images` to `size` using nearest neighbor interpolation. +/// +/// Arguments: +/// * scope: A Scope object +/// * images: 4-D with shape `[batch, height, width, channels]`. +/// * size: = A 1-D int32 Tensor of 2 elements: `new_height, new_width`. The +/// new size for the images. +/// +/// Optional attributes (see `Attrs`): +/// * align_corners: If true, the centers of the 4 corner pixels of the input and output tensors are +/// aligned, preserving the values at the corner pixels. Defaults to false. +/// +/// Returns: +/// * `Output`: 4-D with shape +/// `[batch, new_height, new_width, channels]`. +class ResizeNearestNeighbor { + public: + /// Optional attribute setters for ResizeNearestNeighbor + struct Attrs { + /// If true, the centers of the 4 corner pixels of the input and output tensors are + /// aligned, preserving the values at the corner pixels. Defaults to false. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs AlignCorners(bool x) { + Attrs ret = *this; + ret.align_corners_ = x; + return ret; + } + + bool align_corners_ = false; + }; + ResizeNearestNeighbor(const ::tensorflow::Scope& scope, ::tensorflow::Input + images, ::tensorflow::Input size); + ResizeNearestNeighbor(const ::tensorflow::Scope& scope, ::tensorflow::Input + images, ::tensorflow::Input size, const + ResizeNearestNeighbor::Attrs& attrs); + operator ::tensorflow::Output() const { return resized_images; } + operator ::tensorflow::Input() const { return resized_images; } + ::tensorflow::Node* node() const { return resized_images.node(); } + + static Attrs AlignCorners(bool x) { + return Attrs().AlignCorners(x); + } + + Operation operation; + ::tensorflow::Output resized_images; +}; + +/// Generate a single randomly distorted bounding box for an image. +/// +/// Bounding box annotations are often supplied in addition to ground-truth labels +/// in image recognition or object localization tasks. A common technique for +/// training such a system is to randomly distort an image while preserving +/// its content, i.e. *data augmentation*. This Op outputs a randomly distorted +/// localization of an object, i.e. bounding box, given an `image_size`, +/// `bounding_boxes` and a series of constraints. +/// +/// The output of this Op is a single bounding box that may be used to crop the +/// original image. The output is returned as 3 tensors: `begin`, `size` and +/// `bboxes`. The first 2 tensors can be fed directly into `tf.slice` to crop the +/// image. The latter may be supplied to `tf.image.draw_bounding_boxes` to visualize +/// what the bounding box looks like. +/// +/// Bounding boxes are supplied and returned as `[y_min, x_min, y_max, x_max]`. The +/// bounding box coordinates are floats in `[0.0, 1.0]` relative to the width and +/// height of the underlying image. +/// +/// For example, +/// +/// ```python +/// # Generate a single distorted bounding box. +/// begin, size, bbox_for_draw = tf.image.sample_distorted_bounding_box( +/// tf.shape(image), +/// bounding_boxes=bounding_boxes) +/// +/// # Draw the bounding box in an image summary. +/// image_with_box = tf.image.draw_bounding_boxes(tf.expand_dims(image, 0), +/// bbox_for_draw) +/// tf.summary.image('images_with_box', image_with_box) +/// +/// # Employ the bounding box to distort the image. +/// distorted_image = tf.slice(image, begin, size) +/// ``` +/// +/// Note that if no bounding box information is available, setting +/// `use_image_if_no_bounding_boxes = true` will assume there is a single implicit +/// bounding box covering the whole image. If `use_image_if_no_bounding_boxes` is +/// false and no bounding boxes are supplied, an error is raised. +/// +/// Arguments: +/// * scope: A Scope object +/// * image_size: 1-D, containing `[height, width, channels]`. +/// * bounding_boxes: 3-D with shape `[batch, N, 4]` describing the N bounding boxes +/// associated with the image. +/// +/// Optional attributes (see `Attrs`): +/// * seed: If either `seed` or `seed2` are set to non-zero, the random number +/// generator is seeded by the given `seed`. Otherwise, it is seeded by a random +/// seed. +/// * seed2: A second seed to avoid seed collision. +/// * min_object_covered: The cropped area of the image must contain at least this +/// fraction of any bounding box supplied. The value of this parameter should be +/// non-negative. In the case of 0, the cropped area does not need to overlap +/// any of the bounding boxes supplied. +/// * aspect_ratio_range: The cropped area of the image must have an aspect ratio = +/// width / height within this range. +/// * area_range: The cropped area of the image must contain a fraction of the +/// supplied image within this range. +/// * max_attempts: Number of attempts at generating a cropped region of the image +/// of the specified constraints. After `max_attempts` failures, return the entire +/// image. +/// * use_image_if_no_bounding_boxes: Controls behavior if no bounding boxes supplied. +/// If true, assume an implicit bounding box covering the whole input. If false, +/// raise an error. +/// +/// Returns: +/// * `Output` begin: 1-D, containing `[offset_height, offset_width, 0]`. Provide as input to +/// `tf.slice`. +/// * `Output` size: 1-D, containing `[target_height, target_width, -1]`. Provide as input to +/// `tf.slice`. +/// * `Output` bboxes: 3-D with shape `[1, 1, 4]` containing the distorted bounding box. +/// Provide as input to `tf.image.draw_bounding_boxes`. +class SampleDistortedBoundingBox { + public: + /// Optional attribute setters for SampleDistortedBoundingBox + struct Attrs { + /// If either `seed` or `seed2` are set to non-zero, the random number + /// generator is seeded by the given `seed`. Otherwise, it is seeded by a random + /// seed. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Seed(int64 x) { + Attrs ret = *this; + ret.seed_ = x; + return ret; + } + + /// A second seed to avoid seed collision. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Seed2(int64 x) { + Attrs ret = *this; + ret.seed2_ = x; + return ret; + } + + /// The cropped area of the image must contain at least this + /// fraction of any bounding box supplied. The value of this parameter should be + /// non-negative. In the case of 0, the cropped area does not need to overlap + /// any of the bounding boxes supplied. + /// + /// Defaults to 0.1 + TF_MUST_USE_RESULT Attrs MinObjectCovered(float x) { + Attrs ret = *this; + ret.min_object_covered_ = x; + return ret; + } + + /// The cropped area of the image must have an aspect ratio = + /// width / height within this range. + /// + /// Defaults to [0.75, 1.33] + TF_MUST_USE_RESULT Attrs AspectRatioRange(const gtl::ArraySlice& x) { + Attrs ret = *this; + ret.aspect_ratio_range_ = x; + return ret; + } + + /// The cropped area of the image must contain a fraction of the + /// supplied image within this range. + /// + /// Defaults to [0.05, 1] + TF_MUST_USE_RESULT Attrs AreaRange(const gtl::ArraySlice& x) { + Attrs ret = *this; + ret.area_range_ = x; + return ret; + } + + /// Number of attempts at generating a cropped region of the image + /// of the specified constraints. After `max_attempts` failures, return the entire + /// image. + /// + /// Defaults to 100 + TF_MUST_USE_RESULT Attrs MaxAttempts(int64 x) { + Attrs ret = *this; + ret.max_attempts_ = x; + return ret; + } + + /// Controls behavior if no bounding boxes supplied. + /// If true, assume an implicit bounding box covering the whole input. If false, + /// raise an error. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseImageIfNoBoundingBoxes(bool x) { + Attrs ret = *this; + ret.use_image_if_no_bounding_boxes_ = x; + return ret; + } + + int64 seed_ = 0; + int64 seed2_ = 0; + float min_object_covered_ = 0.1f; + gtl::ArraySlice aspect_ratio_range_ = Default_aspect_ratio_range(); + gtl::ArraySlice area_range_ = Default_area_range(); + int64 max_attempts_ = 100; + bool use_image_if_no_bounding_boxes_ = false; + private: + static gtl::ArraySlice Default_aspect_ratio_range() { + static const float kStorage[] = {0.75f, 1.33f}; + return gtl::ArraySlice(kStorage); + } + static gtl::ArraySlice Default_area_range() { + static const float kStorage[] = {0.05f, 1.0f}; + return gtl::ArraySlice(kStorage); + } + }; + SampleDistortedBoundingBox(const ::tensorflow::Scope& scope, + ::tensorflow::Input image_size, ::tensorflow::Input + bounding_boxes); + SampleDistortedBoundingBox(const ::tensorflow::Scope& scope, + ::tensorflow::Input image_size, ::tensorflow::Input + bounding_boxes, const + SampleDistortedBoundingBox::Attrs& attrs); + + static Attrs Seed(int64 x) { + return Attrs().Seed(x); + } + static Attrs Seed2(int64 x) { + return Attrs().Seed2(x); + } + static Attrs MinObjectCovered(float x) { + return Attrs().MinObjectCovered(x); + } + static Attrs AspectRatioRange(const gtl::ArraySlice& x) { + return Attrs().AspectRatioRange(x); + } + static Attrs AreaRange(const gtl::ArraySlice& x) { + return Attrs().AreaRange(x); + } + static Attrs MaxAttempts(int64 x) { + return Attrs().MaxAttempts(x); + } + static Attrs UseImageIfNoBoundingBoxes(bool x) { + return Attrs().UseImageIfNoBoundingBoxes(x); + } + + Operation operation; + ::tensorflow::Output begin; + ::tensorflow::Output size; + ::tensorflow::Output bboxes; +}; + +/// Generate a single randomly distorted bounding box for an image. +/// +/// Bounding box annotations are often supplied in addition to ground-truth labels +/// in image recognition or object localization tasks. A common technique for +/// training such a system is to randomly distort an image while preserving +/// its content, i.e. *data augmentation*. This Op outputs a randomly distorted +/// localization of an object, i.e. bounding box, given an `image_size`, +/// `bounding_boxes` and a series of constraints. +/// +/// The output of this Op is a single bounding box that may be used to crop the +/// original image. The output is returned as 3 tensors: `begin`, `size` and +/// `bboxes`. The first 2 tensors can be fed directly into `tf.slice` to crop the +/// image. The latter may be supplied to `tf.image.draw_bounding_boxes` to visualize +/// what the bounding box looks like. +/// +/// Bounding boxes are supplied and returned as `[y_min, x_min, y_max, x_max]`. The +/// bounding box coordinates are floats in `[0.0, 1.0]` relative to the width and +/// height of the underlying image. +/// +/// For example, +/// +/// ```python +/// # Generate a single distorted bounding box. +/// begin, size, bbox_for_draw = tf.image.sample_distorted_bounding_box( +/// tf.shape(image), +/// bounding_boxes=bounding_boxes) +/// +/// # Draw the bounding box in an image summary. +/// image_with_box = tf.image.draw_bounding_boxes(tf.expand_dims(image, 0), +/// bbox_for_draw) +/// tf.summary.image('images_with_box', image_with_box) +/// +/// # Employ the bounding box to distort the image. +/// distorted_image = tf.slice(image, begin, size) +/// ``` +/// +/// Note that if no bounding box information is available, setting +/// `use_image_if_no_bounding_boxes = true` will assume there is a single implicit +/// bounding box covering the whole image. If `use_image_if_no_bounding_boxes` is +/// false and no bounding boxes are supplied, an error is raised. +/// +/// Arguments: +/// * scope: A Scope object +/// * image_size: 1-D, containing `[height, width, channels]`. +/// * bounding_boxes: 3-D with shape `[batch, N, 4]` describing the N bounding boxes +/// associated with the image. +/// * min_object_covered: The cropped area of the image must contain at least this +/// fraction of any bounding box supplied. The value of this parameter should be +/// non-negative. In the case of 0, the cropped area does not need to overlap +/// any of the bounding boxes supplied. +/// +/// Optional attributes (see `Attrs`): +/// * seed: If either `seed` or `seed2` are set to non-zero, the random number +/// generator is seeded by the given `seed`. Otherwise, it is seeded by a random +/// seed. +/// * seed2: A second seed to avoid seed collision. +/// * aspect_ratio_range: The cropped area of the image must have an aspect ratio = +/// width / height within this range. +/// * area_range: The cropped area of the image must contain a fraction of the +/// supplied image within this range. +/// * max_attempts: Number of attempts at generating a cropped region of the image +/// of the specified constraints. After `max_attempts` failures, return the entire +/// image. +/// * use_image_if_no_bounding_boxes: Controls behavior if no bounding boxes supplied. +/// If true, assume an implicit bounding box covering the whole input. If false, +/// raise an error. +/// +/// Returns: +/// * `Output` begin: 1-D, containing `[offset_height, offset_width, 0]`. Provide as input to +/// `tf.slice`. +/// * `Output` size: 1-D, containing `[target_height, target_width, -1]`. Provide as input to +/// `tf.slice`. +/// * `Output` bboxes: 3-D with shape `[1, 1, 4]` containing the distorted bounding box. +/// Provide as input to `tf.image.draw_bounding_boxes`. +class SampleDistortedBoundingBoxV2 { + public: + /// Optional attribute setters for SampleDistortedBoundingBoxV2 + struct Attrs { + /// If either `seed` or `seed2` are set to non-zero, the random number + /// generator is seeded by the given `seed`. Otherwise, it is seeded by a random + /// seed. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Seed(int64 x) { + Attrs ret = *this; + ret.seed_ = x; + return ret; + } + + /// A second seed to avoid seed collision. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Seed2(int64 x) { + Attrs ret = *this; + ret.seed2_ = x; + return ret; + } + + /// The cropped area of the image must have an aspect ratio = + /// width / height within this range. + /// + /// Defaults to [0.75, 1.33] + TF_MUST_USE_RESULT Attrs AspectRatioRange(const gtl::ArraySlice& x) { + Attrs ret = *this; + ret.aspect_ratio_range_ = x; + return ret; + } + + /// The cropped area of the image must contain a fraction of the + /// supplied image within this range. + /// + /// Defaults to [0.05, 1] + TF_MUST_USE_RESULT Attrs AreaRange(const gtl::ArraySlice& x) { + Attrs ret = *this; + ret.area_range_ = x; + return ret; + } + + /// Number of attempts at generating a cropped region of the image + /// of the specified constraints. After `max_attempts` failures, return the entire + /// image. + /// + /// Defaults to 100 + TF_MUST_USE_RESULT Attrs MaxAttempts(int64 x) { + Attrs ret = *this; + ret.max_attempts_ = x; + return ret; + } + + /// Controls behavior if no bounding boxes supplied. + /// If true, assume an implicit bounding box covering the whole input. If false, + /// raise an error. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseImageIfNoBoundingBoxes(bool x) { + Attrs ret = *this; + ret.use_image_if_no_bounding_boxes_ = x; + return ret; + } + + int64 seed_ = 0; + int64 seed2_ = 0; + gtl::ArraySlice aspect_ratio_range_ = Default_aspect_ratio_range(); + gtl::ArraySlice area_range_ = Default_area_range(); + int64 max_attempts_ = 100; + bool use_image_if_no_bounding_boxes_ = false; + private: + static gtl::ArraySlice Default_aspect_ratio_range() { + static const float kStorage[] = {0.75f, 1.33f}; + return gtl::ArraySlice(kStorage); + } + static gtl::ArraySlice Default_area_range() { + static const float kStorage[] = {0.05f, 1.0f}; + return gtl::ArraySlice(kStorage); + } + }; + SampleDistortedBoundingBoxV2(const ::tensorflow::Scope& scope, + ::tensorflow::Input image_size, + ::tensorflow::Input bounding_boxes, + ::tensorflow::Input min_object_covered); + SampleDistortedBoundingBoxV2(const ::tensorflow::Scope& scope, + ::tensorflow::Input image_size, + ::tensorflow::Input bounding_boxes, + ::tensorflow::Input min_object_covered, const + SampleDistortedBoundingBoxV2::Attrs& attrs); + + static Attrs Seed(int64 x) { + return Attrs().Seed(x); + } + static Attrs Seed2(int64 x) { + return Attrs().Seed2(x); + } + static Attrs AspectRatioRange(const gtl::ArraySlice& x) { + return Attrs().AspectRatioRange(x); + } + static Attrs AreaRange(const gtl::ArraySlice& x) { + return Attrs().AreaRange(x); + } + static Attrs MaxAttempts(int64 x) { + return Attrs().MaxAttempts(x); + } + static Attrs UseImageIfNoBoundingBoxes(bool x) { + return Attrs().UseImageIfNoBoundingBoxes(x); + } + + Operation operation; + ::tensorflow::Output begin; + ::tensorflow::Output size; + ::tensorflow::Output bboxes; +}; + +/// @} + +} // namespace ops +} // namespace tensorflow + +#endif // TENSORFLOW_CC_OPS_IMAGE_OPS_H_ diff --git a/linux/include/tensorflow/cc/ops/image_ops_internal.h b/linux/include/tensorflow/cc/ops/image_ops_internal.h new file mode 100644 index 000000000..9523df48a --- /dev/null +++ b/linux/include/tensorflow/cc/ops/image_ops_internal.h @@ -0,0 +1,174 @@ +// This file is MACHINE GENERATED! Do not edit. + +#ifndef TENSORFLOW_CC_OPS_IMAGE_OPS_INTERNAL_H_ +#define TENSORFLOW_CC_OPS_IMAGE_OPS_INTERNAL_H_ + +// This file is MACHINE GENERATED! Do not edit. + +#include "tensorflow/cc/framework/ops.h" +#include "tensorflow/cc/framework/scope.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/gtl/array_slice.h" + +namespace tensorflow { +namespace ops { +namespace internal { +// NOTE: This namespace has internal TensorFlow details that +// are not part of TensorFlow's public API. + +/// @defgroup image_ops_internal Image Ops Internal +/// @{ + +/// Computes the gradient of bicubic interpolation. +/// +/// Arguments: +/// * scope: A Scope object +/// * grads: 4-D with shape `[batch, height, width, channels]`. +/// * original_image: 4-D with shape `[batch, orig_height, orig_width, channels]`, +/// The image tensor that was resized. +/// +/// Optional attributes (see `Attrs`): +/// * align_corners: If true, the centers of the 4 corner pixels of the input and grad tensors are +/// aligned. Defaults to false. +/// +/// Returns: +/// * `Output`: 4-D with shape `[batch, orig_height, orig_width, channels]`. +/// Gradients with respect to the input image. Input image must have been +/// float or double. +class ResizeBicubicGrad { + public: + /// Optional attribute setters for ResizeBicubicGrad + struct Attrs { + /// If true, the centers of the 4 corner pixels of the input and grad tensors are + /// aligned. Defaults to false. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs AlignCorners(bool x) { + Attrs ret = *this; + ret.align_corners_ = x; + return ret; + } + + bool align_corners_ = false; + }; + ResizeBicubicGrad(const ::tensorflow::Scope& scope, ::tensorflow::Input grads, + ::tensorflow::Input original_image); + ResizeBicubicGrad(const ::tensorflow::Scope& scope, ::tensorflow::Input grads, + ::tensorflow::Input original_image, const + ResizeBicubicGrad::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs AlignCorners(bool x) { + return Attrs().AlignCorners(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Computes the gradient of bilinear interpolation. +/// +/// Arguments: +/// * scope: A Scope object +/// * grads: 4-D with shape `[batch, height, width, channels]`. +/// * original_image: 4-D with shape `[batch, orig_height, orig_width, channels]`, +/// The image tensor that was resized. +/// +/// Optional attributes (see `Attrs`): +/// * align_corners: If true, the centers of the 4 corner pixels of the input and grad tensors are +/// aligned. Defaults to false. +/// +/// Returns: +/// * `Output`: 4-D with shape `[batch, orig_height, orig_width, channels]`. +/// Gradients with respect to the input image. Input image must have been +/// float or double. +class ResizeBilinearGrad { + public: + /// Optional attribute setters for ResizeBilinearGrad + struct Attrs { + /// If true, the centers of the 4 corner pixels of the input and grad tensors are + /// aligned. Defaults to false. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs AlignCorners(bool x) { + Attrs ret = *this; + ret.align_corners_ = x; + return ret; + } + + bool align_corners_ = false; + }; + ResizeBilinearGrad(const ::tensorflow::Scope& scope, ::tensorflow::Input grads, + ::tensorflow::Input original_image); + ResizeBilinearGrad(const ::tensorflow::Scope& scope, ::tensorflow::Input grads, + ::tensorflow::Input original_image, const + ResizeBilinearGrad::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs AlignCorners(bool x) { + return Attrs().AlignCorners(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Computes the gradient of nearest neighbor interpolation. +/// +/// Arguments: +/// * scope: A Scope object +/// * grads: 4-D with shape `[batch, height, width, channels]`. +/// * size: = A 1-D int32 Tensor of 2 elements: `orig_height, orig_width`. The +/// original input size. +/// +/// Optional attributes (see `Attrs`): +/// * align_corners: If true, the centers of the 4 corner pixels of the input and grad tensors are +/// aligned. Defaults to false. +/// +/// Returns: +/// * `Output`: 4-D with shape `[batch, orig_height, orig_width, channels]`. Gradients +/// with respect to the input image. +class ResizeNearestNeighborGrad { + public: + /// Optional attribute setters for ResizeNearestNeighborGrad + struct Attrs { + /// If true, the centers of the 4 corner pixels of the input and grad tensors are + /// aligned. Defaults to false. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs AlignCorners(bool x) { + Attrs ret = *this; + ret.align_corners_ = x; + return ret; + } + + bool align_corners_ = false; + }; + ResizeNearestNeighborGrad(const ::tensorflow::Scope& scope, ::tensorflow::Input + grads, ::tensorflow::Input size); + ResizeNearestNeighborGrad(const ::tensorflow::Scope& scope, ::tensorflow::Input + grads, ::tensorflow::Input size, const + ResizeNearestNeighborGrad::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs AlignCorners(bool x) { + return Attrs().AlignCorners(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +} // namespace internal +} // namespace ops +} // namespace tensorflow + +#endif // TENSORFLOW_CC_OPS_IMAGE_OPS_INTERNAL_H_ diff --git a/linux/include/tensorflow/cc/ops/io_ops.h b/linux/include/tensorflow/cc/ops/io_ops.h new file mode 100644 index 000000000..3e2533190 --- /dev/null +++ b/linux/include/tensorflow/cc/ops/io_ops.h @@ -0,0 +1,1035 @@ +// This file is MACHINE GENERATED! Do not edit. + +#ifndef TENSORFLOW_CC_OPS_IO_OPS_H_ +#define TENSORFLOW_CC_OPS_IO_OPS_H_ + +// This file is MACHINE GENERATED! Do not edit. + +#include "tensorflow/cc/framework/ops.h" +#include "tensorflow/cc/framework/scope.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/gtl/array_slice.h" + +namespace tensorflow { +namespace ops { + +/// @defgroup io_ops Io Ops +/// @{ + +/// A Reader that outputs fixed-length records from a file. +/// +/// Arguments: +/// * scope: A Scope object +/// * record_bytes: Number of bytes in the record. +/// +/// Optional attributes (see `Attrs`): +/// * header_bytes: Number of bytes in the header, defaults to 0. +/// * footer_bytes: Number of bytes in the footer, defaults to 0. +/// * hop_bytes: Number of bytes to hop before each read. Default of 0 means using +/// record_bytes. +/// * container: If non-empty, this reader is placed in the given container. +/// Otherwise, a default container is used. +/// * shared_name: If non-empty, this reader is named in the given bucket +/// with this shared_name. Otherwise, the node name is used instead. +/// * encoding: The type of encoding for the file. Currently ZLIB and GZIP +/// are supported. Defaults to none. +/// +/// Returns: +/// * `Output`: The handle to reference the Reader. +class FixedLengthRecordReader { + public: + /// Optional attribute setters for FixedLengthRecordReader + struct Attrs { + /// Number of bytes in the header, defaults to 0. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs HeaderBytes(int64 x) { + Attrs ret = *this; + ret.header_bytes_ = x; + return ret; + } + + /// Number of bytes in the footer, defaults to 0. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs FooterBytes(int64 x) { + Attrs ret = *this; + ret.footer_bytes_ = x; + return ret; + } + + /// Number of bytes to hop before each read. Default of 0 means using + /// record_bytes. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs HopBytes(int64 x) { + Attrs ret = *this; + ret.hop_bytes_ = x; + return ret; + } + + /// If non-empty, this reader is placed in the given container. + /// Otherwise, a default container is used. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs Container(StringPiece x) { + Attrs ret = *this; + ret.container_ = x; + return ret; + } + + /// If non-empty, this reader is named in the given bucket + /// with this shared_name. Otherwise, the node name is used instead. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs SharedName(StringPiece x) { + Attrs ret = *this; + ret.shared_name_ = x; + return ret; + } + + /// The type of encoding for the file. Currently ZLIB and GZIP + /// are supported. Defaults to none. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs Encoding(StringPiece x) { + Attrs ret = *this; + ret.encoding_ = x; + return ret; + } + + int64 header_bytes_ = 0; + int64 footer_bytes_ = 0; + int64 hop_bytes_ = 0; + StringPiece container_ = ""; + StringPiece shared_name_ = ""; + StringPiece encoding_ = ""; + }; + FixedLengthRecordReader(const ::tensorflow::Scope& scope, int64 record_bytes); + FixedLengthRecordReader(const ::tensorflow::Scope& scope, int64 record_bytes, + const FixedLengthRecordReader::Attrs& attrs); + operator ::tensorflow::Output() const { return reader_handle; } + operator ::tensorflow::Input() const { return reader_handle; } + ::tensorflow::Node* node() const { return reader_handle.node(); } + + static Attrs HeaderBytes(int64 x) { + return Attrs().HeaderBytes(x); + } + static Attrs FooterBytes(int64 x) { + return Attrs().FooterBytes(x); + } + static Attrs HopBytes(int64 x) { + return Attrs().HopBytes(x); + } + static Attrs Container(StringPiece x) { + return Attrs().Container(x); + } + static Attrs SharedName(StringPiece x) { + return Attrs().SharedName(x); + } + static Attrs Encoding(StringPiece x) { + return Attrs().Encoding(x); + } + + Operation operation; + ::tensorflow::Output reader_handle; +}; + +/// A Reader that outputs the queued work as both the key and value. +/// +/// To use, enqueue strings in a Queue. ReaderRead will take the front +/// work string and output (work, work). +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Optional attributes (see `Attrs`): +/// * container: If non-empty, this reader is placed in the given container. +/// Otherwise, a default container is used. +/// * shared_name: If non-empty, this reader is named in the given bucket +/// with this shared_name. Otherwise, the node name is used instead. +/// +/// Returns: +/// * `Output`: The handle to reference the Reader. +class IdentityReader { + public: + /// Optional attribute setters for IdentityReader + struct Attrs { + /// If non-empty, this reader is placed in the given container. + /// Otherwise, a default container is used. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs Container(StringPiece x) { + Attrs ret = *this; + ret.container_ = x; + return ret; + } + + /// If non-empty, this reader is named in the given bucket + /// with this shared_name. Otherwise, the node name is used instead. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs SharedName(StringPiece x) { + Attrs ret = *this; + ret.shared_name_ = x; + return ret; + } + + StringPiece container_ = ""; + StringPiece shared_name_ = ""; + }; + IdentityReader(const ::tensorflow::Scope& scope); + IdentityReader(const ::tensorflow::Scope& scope, const IdentityReader::Attrs& + attrs); + operator ::tensorflow::Output() const { return reader_handle; } + operator ::tensorflow::Input() const { return reader_handle; } + ::tensorflow::Node* node() const { return reader_handle.node(); } + + static Attrs Container(StringPiece x) { + return Attrs().Container(x); + } + static Attrs SharedName(StringPiece x) { + return Attrs().SharedName(x); + } + + Operation operation; + ::tensorflow::Output reader_handle; +}; + +/// A Reader that outputs the records from a LMDB file. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Optional attributes (see `Attrs`): +/// * container: If non-empty, this reader is placed in the given container. +/// Otherwise, a default container is used. +/// * shared_name: If non-empty, this reader is named in the given bucket +/// with this shared_name. Otherwise, the node name is used instead. +/// +/// Returns: +/// * `Output`: The handle to reference the Reader. +class LMDBReader { + public: + /// Optional attribute setters for LMDBReader + struct Attrs { + /// If non-empty, this reader is placed in the given container. + /// Otherwise, a default container is used. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs Container(StringPiece x) { + Attrs ret = *this; + ret.container_ = x; + return ret; + } + + /// If non-empty, this reader is named in the given bucket + /// with this shared_name. Otherwise, the node name is used instead. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs SharedName(StringPiece x) { + Attrs ret = *this; + ret.shared_name_ = x; + return ret; + } + + StringPiece container_ = ""; + StringPiece shared_name_ = ""; + }; + LMDBReader(const ::tensorflow::Scope& scope); + LMDBReader(const ::tensorflow::Scope& scope, const LMDBReader::Attrs& attrs); + operator ::tensorflow::Output() const { return reader_handle; } + operator ::tensorflow::Input() const { return reader_handle; } + ::tensorflow::Node* node() const { return reader_handle.node(); } + + static Attrs Container(StringPiece x) { + return Attrs().Container(x); + } + static Attrs SharedName(StringPiece x) { + return Attrs().SharedName(x); + } + + Operation operation; + ::tensorflow::Output reader_handle; +}; + +/// Returns the set of files matching one or more glob patterns. +/// +/// Note that this routine only supports wildcard characters in the +/// basename portion of the pattern, not in the directory portion. +/// Note also that the order of filenames returned can be non-deterministic. +/// +/// Arguments: +/// * scope: A Scope object +/// * pattern: Shell wildcard pattern(s). Scalar or vector of type string. +/// +/// Returns: +/// * `Output`: A vector of matching filenames. +class MatchingFiles { + public: + MatchingFiles(const ::tensorflow::Scope& scope, ::tensorflow::Input pattern); + operator ::tensorflow::Output() const { return filenames; } + operator ::tensorflow::Input() const { return filenames; } + ::tensorflow::Node* node() const { return filenames.node(); } + + Operation operation; + ::tensorflow::Output filenames; +}; + +/// V2 format specific: merges the metadata files of sharded checkpoints. The +/// +/// result is one logical checkpoint, with one physical metadata file and renamed +/// data files. +/// +/// Intended for "grouping" multiple checkpoints in a sharded checkpoint setup. +/// +/// If delete_old_dirs is true, attempts to delete recursively the dirname of each +/// path in the input checkpoint_prefixes. This is useful when those paths are non +/// user-facing temporary locations. +/// +/// Arguments: +/// * scope: A Scope object +/// * checkpoint_prefixes: prefixes of V2 checkpoints to merge. +/// * destination_prefix: scalar. The desired final prefix. Allowed to be the same +/// as one of the checkpoint_prefixes. +/// +/// Optional attributes (see `Attrs`): +/// * delete_old_dirs: see above. +/// +/// Returns: +/// * the created `Operation` +class MergeV2Checkpoints { + public: + /// Optional attribute setters for MergeV2Checkpoints + struct Attrs { + /// see above. + /// + /// Defaults to true + TF_MUST_USE_RESULT Attrs DeleteOldDirs(bool x) { + Attrs ret = *this; + ret.delete_old_dirs_ = x; + return ret; + } + + bool delete_old_dirs_ = true; + }; + MergeV2Checkpoints(const ::tensorflow::Scope& scope, ::tensorflow::Input + checkpoint_prefixes, ::tensorflow::Input destination_prefix); + MergeV2Checkpoints(const ::tensorflow::Scope& scope, ::tensorflow::Input + checkpoint_prefixes, ::tensorflow::Input destination_prefix, + const MergeV2Checkpoints::Attrs& attrs); + operator ::tensorflow::Operation() const { return operation; } + + static Attrs DeleteOldDirs(bool x) { + return Attrs().DeleteOldDirs(x); + } + + Operation operation; +}; + +/// Reads and outputs the entire contents of the input filename. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The contents tensor. +class ReadFile { + public: + ReadFile(const ::tensorflow::Scope& scope, ::tensorflow::Input filename); + operator ::tensorflow::Output() const { return contents; } + operator ::tensorflow::Input() const { return contents; } + ::tensorflow::Node* node() const { return contents.node(); } + + Operation operation; + ::tensorflow::Output contents; +}; + +/// Returns the number of records this Reader has produced. +/// +/// This is the same as the number of ReaderRead executions that have +/// succeeded. +/// +/// Arguments: +/// * scope: A Scope object +/// * reader_handle: Handle to a Reader. +/// +/// Returns: +/// * `Output`: The records_produced tensor. +class ReaderNumRecordsProduced { + public: + ReaderNumRecordsProduced(const ::tensorflow::Scope& scope, ::tensorflow::Input + reader_handle); + operator ::tensorflow::Output() const { return records_produced; } + operator ::tensorflow::Input() const { return records_produced; } + ::tensorflow::Node* node() const { return records_produced.node(); } + + Operation operation; + ::tensorflow::Output records_produced; +}; + +/// Returns the number of work units this Reader has finished processing. +/// +/// Arguments: +/// * scope: A Scope object +/// * reader_handle: Handle to a Reader. +/// +/// Returns: +/// * `Output`: The units_completed tensor. +class ReaderNumWorkUnitsCompleted { + public: + ReaderNumWorkUnitsCompleted(const ::tensorflow::Scope& scope, + ::tensorflow::Input reader_handle); + operator ::tensorflow::Output() const { return units_completed; } + operator ::tensorflow::Input() const { return units_completed; } + ::tensorflow::Node* node() const { return units_completed.node(); } + + Operation operation; + ::tensorflow::Output units_completed; +}; + +/// Returns up to `num_records` (key, value) pairs produced by a Reader. +/// +/// Will dequeue from the input queue if necessary (e.g. when the +/// Reader needs to start reading from a new file since it has finished +/// with the previous file). +/// It may return less than `num_records` even before the last batch. +/// +/// Arguments: +/// * scope: A Scope object +/// * reader_handle: Handle to a `Reader`. +/// * queue_handle: Handle to a `Queue`, with string work items. +/// * num_records: number of records to read from `Reader`. +/// +/// Returns: +/// * `Output` keys: A 1-D tensor. +/// * `Output` values: A 1-D tensor. +class ReaderReadUpTo { + public: + ReaderReadUpTo(const ::tensorflow::Scope& scope, ::tensorflow::Input + reader_handle, ::tensorflow::Input queue_handle, + ::tensorflow::Input num_records); + + Operation operation; + ::tensorflow::Output keys; + ::tensorflow::Output values; +}; + +/// Returns the next record (key, value pair) produced by a Reader. +/// +/// Will dequeue from the input queue if necessary (e.g. when the +/// Reader needs to start reading from a new file since it has finished +/// with the previous file). +/// +/// Arguments: +/// * scope: A Scope object +/// * reader_handle: Handle to a Reader. +/// * queue_handle: Handle to a Queue, with string work items. +/// +/// Returns: +/// * `Output` key: A scalar. +/// * `Output` value: A scalar. +class ReaderRead { + public: + ReaderRead(const ::tensorflow::Scope& scope, ::tensorflow::Input reader_handle, + ::tensorflow::Input queue_handle); + + Operation operation; + ::tensorflow::Output key; + ::tensorflow::Output value; +}; + +/// Restore a Reader to its initial clean state. +/// +/// Arguments: +/// * scope: A Scope object +/// * reader_handle: Handle to a Reader. +/// +/// Returns: +/// * the created `Operation` +class ReaderReset { + public: + ReaderReset(const ::tensorflow::Scope& scope, ::tensorflow::Input + reader_handle); + operator ::tensorflow::Operation() const { return operation; } + + Operation operation; +}; + +/// Restore a reader to a previously saved state. +/// +/// Not all Readers support being restored, so this can produce an +/// Unimplemented error. +/// +/// Arguments: +/// * scope: A Scope object +/// * reader_handle: Handle to a Reader. +/// * state: Result of a ReaderSerializeState of a Reader with type +/// matching reader_handle. +/// +/// Returns: +/// * the created `Operation` +class ReaderRestoreState { + public: + ReaderRestoreState(const ::tensorflow::Scope& scope, ::tensorflow::Input + reader_handle, ::tensorflow::Input state); + operator ::tensorflow::Operation() const { return operation; } + + Operation operation; +}; + +/// Produce a string tensor that encodes the state of a Reader. +/// +/// Not all Readers support being serialized, so this can produce an +/// Unimplemented error. +/// +/// Arguments: +/// * scope: A Scope object +/// * reader_handle: Handle to a Reader. +/// +/// Returns: +/// * `Output`: The state tensor. +class ReaderSerializeState { + public: + ReaderSerializeState(const ::tensorflow::Scope& scope, ::tensorflow::Input + reader_handle); + operator ::tensorflow::Output() const { return state; } + operator ::tensorflow::Input() const { return state; } + ::tensorflow::Node* node() const { return state.node(); } + + Operation operation; + ::tensorflow::Output state; +}; + +/// Restores a tensor from checkpoint files. +/// +/// Reads a tensor stored in one or several files. If there are several files (for +/// instance because a tensor was saved as slices), `file_pattern` may contain +/// wildcard symbols (`*` and `?`) in the filename portion only, not in the +/// directory portion. +/// +/// If a `file_pattern` matches several files, `preferred_shard` can be used to hint +/// in which file the requested tensor is likely to be found. This op will first +/// open the file at index `preferred_shard` in the list of matching files and try +/// to restore tensors from that file. Only if some tensors or tensor slices are +/// not found in that first file, then the Op opens all the files. Setting +/// `preferred_shard` to match the value passed as the `shard` input +/// of a matching `Save` Op may speed up Restore. This attribute only affects +/// performance, not correctness. The default value -1 means files are processed in +/// order. +/// +/// See also `RestoreSlice`. +/// +/// Arguments: +/// * scope: A Scope object +/// * file_pattern: Must have a single element. The pattern of the files from +/// which we read the tensor. +/// * tensor_name: Must have a single element. The name of the tensor to be +/// restored. +/// * dt: The type of the tensor to be restored. +/// +/// Optional attributes (see `Attrs`): +/// * preferred_shard: Index of file to open first if multiple files match +/// `file_pattern`. +/// +/// Returns: +/// * `Output`: The restored tensor. +class Restore { + public: + /// Optional attribute setters for Restore + struct Attrs { + /// Index of file to open first if multiple files match + /// `file_pattern`. + /// + /// Defaults to -1 + TF_MUST_USE_RESULT Attrs PreferredShard(int64 x) { + Attrs ret = *this; + ret.preferred_shard_ = x; + return ret; + } + + int64 preferred_shard_ = -1; + }; + Restore(const ::tensorflow::Scope& scope, ::tensorflow::Input file_pattern, + ::tensorflow::Input tensor_name, DataType dt); + Restore(const ::tensorflow::Scope& scope, ::tensorflow::Input file_pattern, + ::tensorflow::Input tensor_name, DataType dt, const Restore::Attrs& + attrs); + operator ::tensorflow::Output() const { return tensor; } + operator ::tensorflow::Input() const { return tensor; } + ::tensorflow::Node* node() const { return tensor.node(); } + + static Attrs PreferredShard(int64 x) { + return Attrs().PreferredShard(x); + } + + Operation operation; + ::tensorflow::Output tensor; +}; + +/// Restores a tensor from checkpoint files. +/// +/// This is like `Restore` except that restored tensor can be listed as filling +/// only a slice of a larger tensor. `shape_and_slice` specifies the shape of the +/// larger tensor and the slice that the restored tensor covers. +/// +/// The `shape_and_slice` input has the same format as the +/// elements of the `shapes_and_slices` input of the `SaveSlices` op. +/// +/// Arguments: +/// * scope: A Scope object +/// * file_pattern: Must have a single element. The pattern of the files from +/// which we read the tensor. +/// * tensor_name: Must have a single element. The name of the tensor to be +/// restored. +/// * shape_and_slice: Scalar. The shapes and slice specifications to use when +/// restoring a tensors. +/// * dt: The type of the tensor to be restored. +/// +/// Optional attributes (see `Attrs`): +/// * preferred_shard: Index of file to open first if multiple files match +/// `file_pattern`. See the documentation for `Restore`. +/// +/// Returns: +/// * `Output`: The restored tensor. +class RestoreSlice { + public: + /// Optional attribute setters for RestoreSlice + struct Attrs { + /// Index of file to open first if multiple files match + /// `file_pattern`. See the documentation for `Restore`. + /// + /// Defaults to -1 + TF_MUST_USE_RESULT Attrs PreferredShard(int64 x) { + Attrs ret = *this; + ret.preferred_shard_ = x; + return ret; + } + + int64 preferred_shard_ = -1; + }; + RestoreSlice(const ::tensorflow::Scope& scope, ::tensorflow::Input + file_pattern, ::tensorflow::Input tensor_name, ::tensorflow::Input + shape_and_slice, DataType dt); + RestoreSlice(const ::tensorflow::Scope& scope, ::tensorflow::Input + file_pattern, ::tensorflow::Input tensor_name, ::tensorflow::Input + shape_and_slice, DataType dt, const RestoreSlice::Attrs& attrs); + operator ::tensorflow::Output() const { return tensor; } + operator ::tensorflow::Input() const { return tensor; } + ::tensorflow::Node* node() const { return tensor.node(); } + + static Attrs PreferredShard(int64 x) { + return Attrs().PreferredShard(x); + } + + Operation operation; + ::tensorflow::Output tensor; +}; + +/// Restores tensors from a V2 checkpoint. +/// +/// For backward compatibility with the V1 format, this Op currently allows +/// restoring from a V1 checkpoint as well: +/// - This Op first attempts to find the V2 index file pointed to by "prefix", and +/// if found proceed to read it as a V2 checkpoint; +/// - Otherwise the V1 read path is invoked. +/// Relying on this behavior is not recommended, as the ability to fall back to read +/// V1 might be deprecated and eventually removed. +/// +/// By default, restores the named tensors in full. If the caller wishes to restore +/// specific slices of stored tensors, "shape_and_slices" should be non-empty +/// strings and correspondingly well-formed. +/// +/// Callers must ensure all the named tensors are indeed stored in the checkpoint. +/// +/// Arguments: +/// * scope: A Scope object +/// * prefix: Must have a single element. The prefix of a V2 checkpoint. +/// * tensor_names: shape {N}. The names of the tensors to be restored. +/// * shape_and_slices: shape {N}. The slice specs of the tensors to be restored. +/// Empty strings indicate that they are non-partitioned tensors. +/// * dtypes: shape {N}. The list of expected dtype for the tensors. Must match +/// those stored in the checkpoint. +/// +/// Returns: +/// * `OutputList`: shape {N}. The restored tensors, whose shapes are read from the +/// checkpoint directly. +class RestoreV2 { + public: + RestoreV2(const ::tensorflow::Scope& scope, ::tensorflow::Input prefix, + ::tensorflow::Input tensor_names, ::tensorflow::Input + shape_and_slices, const DataTypeSlice& dtypes); + ::tensorflow::Output operator[](size_t index) const { return tensors[index]; } + + + Operation operation; + ::tensorflow::OutputList tensors; +}; + +/// Saves the input tensors to disk. +/// +/// The size of `tensor_names` must match the number of tensors in `data`. `data[i]` +/// is written to `filename` with name `tensor_names[i]`. +/// +/// See also `SaveSlices`. +/// +/// Arguments: +/// * scope: A Scope object +/// * filename: Must have a single element. The name of the file to which we write +/// the tensor. +/// * tensor_names: Shape `[N]`. The names of the tensors to be saved. +/// * data: `N` tensors to save. +/// +/// Returns: +/// * the created `Operation` +class Save { + public: + Save(const ::tensorflow::Scope& scope, ::tensorflow::Input filename, + ::tensorflow::Input tensor_names, ::tensorflow::InputList data); + operator ::tensorflow::Operation() const { return operation; } + + Operation operation; +}; + +/// Saves input tensors slices to disk. +/// +/// This is like `Save` except that tensors can be listed in the saved file as being +/// a slice of a larger tensor. `shapes_and_slices` specifies the shape of the +/// larger tensor and the slice that this tensor covers. `shapes_and_slices` must +/// have as many elements as `tensor_names`. +/// +/// Elements of the `shapes_and_slices` input must either be: +/// +/// * The empty string, in which case the corresponding tensor is +/// saved normally. +/// * A string of the form `dim0 dim1 ... dimN-1 slice-spec` where the +/// `dimI` are the dimensions of the larger tensor and `slice-spec` +/// specifies what part is covered by the tensor to save. +/// +/// `slice-spec` itself is a `:`-separated list: `slice0:slice1:...:sliceN-1` +/// where each `sliceI` is either: +/// +/// * The string `-` meaning that the slice covers all indices of this dimension +/// * `start,length` where `start` and `length` are integers. In that +/// case the slice covers `length` indices starting at `start`. +/// +/// See also `Save`. +/// +/// Arguments: +/// * scope: A Scope object +/// * filename: Must have a single element. The name of the file to which we write the +/// tensor. +/// * tensor_names: Shape `[N]`. The names of the tensors to be saved. +/// * shapes_and_slices: Shape `[N]`. The shapes and slice specifications to use when +/// saving the tensors. +/// * data: `N` tensors to save. +/// +/// Returns: +/// * the created `Operation` +class SaveSlices { + public: + SaveSlices(const ::tensorflow::Scope& scope, ::tensorflow::Input filename, + ::tensorflow::Input tensor_names, ::tensorflow::Input + shapes_and_slices, ::tensorflow::InputList data); + operator ::tensorflow::Operation() const { return operation; } + + Operation operation; +}; + +/// Saves tensors in V2 checkpoint format. +/// +/// By default, saves the named tensors in full. If the caller wishes to save +/// specific slices of full tensors, "shape_and_slices" should be non-empty strings +/// and correspondingly well-formed. +/// +/// Arguments: +/// * scope: A Scope object +/// * prefix: Must have a single element. The prefix of the V2 checkpoint to which we +/// write the tensors. +/// * tensor_names: shape {N}. The names of the tensors to be saved. +/// * shape_and_slices: shape {N}. The slice specs of the tensors to be saved. +/// Empty strings indicate that they are non-partitioned tensors. +/// * tensors: `N` tensors to save. +/// +/// Returns: +/// * the created `Operation` +class SaveV2 { + public: + SaveV2(const ::tensorflow::Scope& scope, ::tensorflow::Input prefix, + ::tensorflow::Input tensor_names, ::tensorflow::Input shape_and_slices, + ::tensorflow::InputList tensors); + operator ::tensorflow::Operation() const { return operation; } + + Operation operation; +}; + +/// Generate a sharded filename. The filename is printf formatted as +/// +/// %s-%05d-of-%05d, basename, shard, num_shards. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The filename tensor. +class ShardedFilename { + public: + ShardedFilename(const ::tensorflow::Scope& scope, ::tensorflow::Input basename, + ::tensorflow::Input shard, ::tensorflow::Input num_shards); + operator ::tensorflow::Output() const { return filename; } + operator ::tensorflow::Input() const { return filename; } + ::tensorflow::Node* node() const { return filename.node(); } + + Operation operation; + ::tensorflow::Output filename; +}; + +/// Generate a glob pattern matching all sharded file names. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The filename tensor. +class ShardedFilespec { + public: + ShardedFilespec(const ::tensorflow::Scope& scope, ::tensorflow::Input basename, + ::tensorflow::Input num_shards); + operator ::tensorflow::Output() const { return filename; } + operator ::tensorflow::Input() const { return filename; } + ::tensorflow::Node* node() const { return filename.node(); } + + Operation operation; + ::tensorflow::Output filename; +}; + +/// A Reader that outputs the records from a TensorFlow Records file. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Optional attributes (see `Attrs`): +/// * container: If non-empty, this reader is placed in the given container. +/// Otherwise, a default container is used. +/// * shared_name: If non-empty, this reader is named in the given bucket +/// with this shared_name. Otherwise, the node name is used instead. +/// +/// Returns: +/// * `Output`: The handle to reference the Reader. +class TFRecordReader { + public: + /// Optional attribute setters for TFRecordReader + struct Attrs { + /// If non-empty, this reader is placed in the given container. + /// Otherwise, a default container is used. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs Container(StringPiece x) { + Attrs ret = *this; + ret.container_ = x; + return ret; + } + + /// If non-empty, this reader is named in the given bucket + /// with this shared_name. Otherwise, the node name is used instead. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs SharedName(StringPiece x) { + Attrs ret = *this; + ret.shared_name_ = x; + return ret; + } + + /// Defaults to "" + TF_MUST_USE_RESULT Attrs CompressionType(StringPiece x) { + Attrs ret = *this; + ret.compression_type_ = x; + return ret; + } + + StringPiece container_ = ""; + StringPiece shared_name_ = ""; + StringPiece compression_type_ = ""; + }; + TFRecordReader(const ::tensorflow::Scope& scope); + TFRecordReader(const ::tensorflow::Scope& scope, const TFRecordReader::Attrs& + attrs); + operator ::tensorflow::Output() const { return reader_handle; } + operator ::tensorflow::Input() const { return reader_handle; } + ::tensorflow::Node* node() const { return reader_handle.node(); } + + static Attrs Container(StringPiece x) { + return Attrs().Container(x); + } + static Attrs SharedName(StringPiece x) { + return Attrs().SharedName(x); + } + static Attrs CompressionType(StringPiece x) { + return Attrs().CompressionType(x); + } + + Operation operation; + ::tensorflow::Output reader_handle; +}; + +/// A Reader that outputs the lines of a file delimited by '\n'. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Optional attributes (see `Attrs`): +/// * skip_header_lines: Number of lines to skip from the beginning of every file. +/// * container: If non-empty, this reader is placed in the given container. +/// Otherwise, a default container is used. +/// * shared_name: If non-empty, this reader is named in the given bucket +/// with this shared_name. Otherwise, the node name is used instead. +/// +/// Returns: +/// * `Output`: The handle to reference the Reader. +class TextLineReader { + public: + /// Optional attribute setters for TextLineReader + struct Attrs { + /// Number of lines to skip from the beginning of every file. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs SkipHeaderLines(int64 x) { + Attrs ret = *this; + ret.skip_header_lines_ = x; + return ret; + } + + /// If non-empty, this reader is placed in the given container. + /// Otherwise, a default container is used. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs Container(StringPiece x) { + Attrs ret = *this; + ret.container_ = x; + return ret; + } + + /// If non-empty, this reader is named in the given bucket + /// with this shared_name. Otherwise, the node name is used instead. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs SharedName(StringPiece x) { + Attrs ret = *this; + ret.shared_name_ = x; + return ret; + } + + int64 skip_header_lines_ = 0; + StringPiece container_ = ""; + StringPiece shared_name_ = ""; + }; + TextLineReader(const ::tensorflow::Scope& scope); + TextLineReader(const ::tensorflow::Scope& scope, const TextLineReader::Attrs& + attrs); + operator ::tensorflow::Output() const { return reader_handle; } + operator ::tensorflow::Input() const { return reader_handle; } + ::tensorflow::Node* node() const { return reader_handle.node(); } + + static Attrs SkipHeaderLines(int64 x) { + return Attrs().SkipHeaderLines(x); + } + static Attrs Container(StringPiece x) { + return Attrs().Container(x); + } + static Attrs SharedName(StringPiece x) { + return Attrs().SharedName(x); + } + + Operation operation; + ::tensorflow::Output reader_handle; +}; + +/// A Reader that outputs the entire contents of a file as a value. +/// +/// To use, enqueue filenames in a Queue. The output of ReaderRead will +/// be a filename (key) and the contents of that file (value). +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Optional attributes (see `Attrs`): +/// * container: If non-empty, this reader is placed in the given container. +/// Otherwise, a default container is used. +/// * shared_name: If non-empty, this reader is named in the given bucket +/// with this shared_name. Otherwise, the node name is used instead. +/// +/// Returns: +/// * `Output`: The handle to reference the Reader. +class WholeFileReader { + public: + /// Optional attribute setters for WholeFileReader + struct Attrs { + /// If non-empty, this reader is placed in the given container. + /// Otherwise, a default container is used. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs Container(StringPiece x) { + Attrs ret = *this; + ret.container_ = x; + return ret; + } + + /// If non-empty, this reader is named in the given bucket + /// with this shared_name. Otherwise, the node name is used instead. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs SharedName(StringPiece x) { + Attrs ret = *this; + ret.shared_name_ = x; + return ret; + } + + StringPiece container_ = ""; + StringPiece shared_name_ = ""; + }; + WholeFileReader(const ::tensorflow::Scope& scope); + WholeFileReader(const ::tensorflow::Scope& scope, const WholeFileReader::Attrs& + attrs); + operator ::tensorflow::Output() const { return reader_handle; } + operator ::tensorflow::Input() const { return reader_handle; } + ::tensorflow::Node* node() const { return reader_handle.node(); } + + static Attrs Container(StringPiece x) { + return Attrs().Container(x); + } + static Attrs SharedName(StringPiece x) { + return Attrs().SharedName(x); + } + + Operation operation; + ::tensorflow::Output reader_handle; +}; + +/// Writes contents to the file at input filename. Creates file and recursively +/// +/// creates directory if not existing. +/// +/// Arguments: +/// * scope: A Scope object +/// * filename: scalar. The name of the file to which we write the contents. +/// * contents: scalar. The content to be written to the output file. +/// +/// Returns: +/// * the created `Operation` +class WriteFile { + public: + WriteFile(const ::tensorflow::Scope& scope, ::tensorflow::Input filename, + ::tensorflow::Input contents); + operator ::tensorflow::Operation() const { return operation; } + + Operation operation; +}; + +/// @} + +} // namespace ops +} // namespace tensorflow + +#endif // TENSORFLOW_CC_OPS_IO_OPS_H_ diff --git a/linux/include/tensorflow/cc/ops/io_ops_internal.h b/linux/include/tensorflow/cc/ops/io_ops_internal.h new file mode 100644 index 000000000..dcd3590cb --- /dev/null +++ b/linux/include/tensorflow/cc/ops/io_ops_internal.h @@ -0,0 +1,28 @@ +// This file is MACHINE GENERATED! Do not edit. + +#ifndef TENSORFLOW_CC_OPS_IO_OPS_INTERNAL_H_ +#define TENSORFLOW_CC_OPS_IO_OPS_INTERNAL_H_ + +// This file is MACHINE GENERATED! Do not edit. + +#include "tensorflow/cc/framework/ops.h" +#include "tensorflow/cc/framework/scope.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/gtl/array_slice.h" + +namespace tensorflow { +namespace ops { +namespace internal { +// NOTE: This namespace has internal TensorFlow details that +// are not part of TensorFlow's public API. + +/// @defgroup io_ops_internal Io Ops Internal +/// @{ + +} // namespace internal +} // namespace ops +} // namespace tensorflow + +#endif // TENSORFLOW_CC_OPS_IO_OPS_INTERNAL_H_ diff --git a/linux/include/tensorflow/cc/ops/linalg_ops.h b/linux/include/tensorflow/cc/ops/linalg_ops.h new file mode 100644 index 000000000..446f24d8f --- /dev/null +++ b/linux/include/tensorflow/cc/ops/linalg_ops.h @@ -0,0 +1,601 @@ +// This file is MACHINE GENERATED! Do not edit. + +#ifndef TENSORFLOW_CC_OPS_LINALG_OPS_H_ +#define TENSORFLOW_CC_OPS_LINALG_OPS_H_ + +// This file is MACHINE GENERATED! Do not edit. + +#include "tensorflow/cc/framework/ops.h" +#include "tensorflow/cc/framework/scope.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/gtl/array_slice.h" + +namespace tensorflow { +namespace ops { + +/// @defgroup linalg_ops Linalg Ops +/// @{ + +/// Computes the Cholesky decomposition of one or more square matrices. +/// +/// The input is a tensor of shape `[..., M, M]` whose inner-most 2 dimensions +/// form square matrices. +/// +/// The input has to be symmetric and positive definite. Only the lower-triangular +/// part of the input will be used for this operation. The upper-triangular part +/// will not be read. +/// +/// The output is a tensor of the same shape as the input +/// containing the Cholesky decompositions for all input submatrices `[..., :, :]`. +/// +/// **Note**: The gradient computation on GPU is faster for large matrices but +/// not for large batch dimensions when the submatrices are small. In this +/// case it might be faster to use the CPU. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: Shape is `[..., M, M]`. +/// +/// Returns: +/// * `Output`: Shape is `[..., M, M]`. +class Cholesky { + public: + Cholesky(const ::tensorflow::Scope& scope, ::tensorflow::Input input); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Computes the reverse mode backpropagated gradient of the Cholesky algorithm. +/// +/// For an explanation see "Differentiation of the Cholesky algorithm" by +/// Iain Murray http://arxiv.org/abs/1602.07527. +/// +/// Arguments: +/// * scope: A Scope object +/// * l: Output of batch Cholesky algorithm l = cholesky(A). Shape is `[..., M, M]`. +/// Algorithm depends only on lower triangular part of the innermost matrices of +/// this tensor. +/// * grad: df/dl where f is some scalar function. Shape is `[..., M, M]`. +/// Algorithm depends only on lower triangular part of the innermost matrices of +/// this tensor. +/// +/// Returns: +/// * `Output`: Symmetrized version of df/dA . Shape is `[..., M, M]` +class CholeskyGrad { + public: + CholeskyGrad(const ::tensorflow::Scope& scope, ::tensorflow::Input l, + ::tensorflow::Input grad); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Computes the sign and the log of the absolute value of the determinant of +/// +/// one or more square matrices. +/// +/// The input is a tensor of shape `[N, M, M]` whose inner-most 2 dimensions +/// form square matrices. The outputs are two tensors containing the signs and +/// absolute values of the log determinants for all N input submatrices +/// `[..., :, :]` such that the determinant = sign*exp(log_abs_determinant). +/// The log_abs_determinant is computed as det(P)*sum(log(diag(LU))) where LU +/// is the LU decomposition of the input and P is the corresponding +/// permutation matrix. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: Shape is `[N, M, M]`. +/// +/// Returns: +/// * `Output` sign: The signs of the log determinants of the inputs. Shape is `[N]`. +/// * `Output` log_abs_determinant: The logs of the absolute values of the determinants +/// of the N input matrices. Shape is `[N]`. +class LogMatrixDeterminant { + public: + LogMatrixDeterminant(const ::tensorflow::Scope& scope, ::tensorflow::Input + input); + + Operation operation; + ::tensorflow::Output sign; + ::tensorflow::Output log_abs_determinant; +}; + +/// Computes the determinant of one or more square matrices. +/// +/// The input is a tensor of shape `[..., M, M]` whose inner-most 2 dimensions +/// form square matrices. The output is a tensor containing the determinants +/// for all input submatrices `[..., :, :]`. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: Shape is `[..., M, M]`. +/// +/// Returns: +/// * `Output`: Shape is `[...]`. +class MatrixDeterminant { + public: + MatrixDeterminant(const ::tensorflow::Scope& scope, ::tensorflow::Input input); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Computes the inverse of one or more square invertible matrices or their +/// +/// adjoints (conjugate transposes). +/// +/// The input is a tensor of shape `[..., M, M]` whose inner-most 2 dimensions +/// form square matrices. The output is a tensor of the same shape as the input +/// containing the inverse for all input submatrices `[..., :, :]`. +/// +/// The op uses LU decomposition with partial pivoting to compute the inverses. +/// +/// If a matrix is not invertible there is no guarantee what the op does. It +/// may detect the condition and raise an exception or it may simply return a +/// garbage result. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: Shape is `[..., M, M]`. +/// +/// Returns: +/// * `Output`: Shape is `[..., M, M]`. +/// +/// @compatibility(numpy) +/// Equivalent to np.linalg.inv +/// @end_compatibility +class MatrixInverse { + public: + /// Optional attribute setters for MatrixInverse + struct Attrs { + /// Defaults to false + TF_MUST_USE_RESULT Attrs Adjoint(bool x) { + Attrs ret = *this; + ret.adjoint_ = x; + return ret; + } + + bool adjoint_ = false; + }; + MatrixInverse(const ::tensorflow::Scope& scope, ::tensorflow::Input input); + MatrixInverse(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + const MatrixInverse::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs Adjoint(bool x) { + return Attrs().Adjoint(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Solves systems of linear equations. +/// +/// `Matrix` is a tensor of shape `[..., M, M]` whose inner-most 2 dimensions +/// form square matrices. `Rhs` is a tensor of shape `[..., M, K]`. The `output` is +/// a tensor shape `[..., M, K]`. If `adjoint` is `False` then each output matrix +/// satisfies `matrix[..., :, :] * output[..., :, :] = rhs[..., :, :]`. +/// If `adjoint` is `True` then each output matrix satisfies +/// `adjoint(matrix[..., :, :]) * output[..., :, :] = rhs[..., :, :]`. +/// +/// Arguments: +/// * scope: A Scope object +/// * matrix: Shape is `[..., M, M]`. +/// * rhs: Shape is `[..., M, K]`. +/// +/// Optional attributes (see `Attrs`): +/// * adjoint: Boolean indicating whether to solve with `matrix` or its (block-wise) +/// adjoint. +/// +/// Returns: +/// * `Output`: Shape is `[..., M, K]`. +class MatrixSolve { + public: + /// Optional attribute setters for MatrixSolve + struct Attrs { + /// Boolean indicating whether to solve with `matrix` or its (block-wise) + /// adjoint. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs Adjoint(bool x) { + Attrs ret = *this; + ret.adjoint_ = x; + return ret; + } + + bool adjoint_ = false; + }; + MatrixSolve(const ::tensorflow::Scope& scope, ::tensorflow::Input matrix, + ::tensorflow::Input rhs); + MatrixSolve(const ::tensorflow::Scope& scope, ::tensorflow::Input matrix, + ::tensorflow::Input rhs, const MatrixSolve::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs Adjoint(bool x) { + return Attrs().Adjoint(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Solves one or more linear least-squares problems. +/// +/// `matrix` is a tensor of shape `[..., M, N]` whose inner-most 2 dimensions +/// form real or complex matrices of size `[M, N]`. `Rhs` is a tensor of the same +/// type as `matrix` and shape `[..., M, K]`. +/// The output is a tensor shape `[..., N, K]` where each output matrix solves +/// each of the equations +/// `matrix[..., :, :]` * `output[..., :, :]` = `rhs[..., :, :]` +/// in the least squares sense. +/// +/// We use the following notation for (complex) matrix and right-hand sides +/// in the batch: +/// +/// `matrix`=\\(A \in \mathbb{C}^{m \times n}\\), +/// `rhs`=\\(B \in \mathbb{C}^{m \times k}\\), +/// `output`=\\(X \in \mathbb{C}^{n \times k}\\), +/// `l2_regularizer`=\\(\lambda \in \mathbb{R}\\). +/// +/// If `fast` is `True`, then the solution is computed by solving the normal +/// equations using Cholesky decomposition. Specifically, if \\(m \ge n\\) then +/// \\(X = (A^H A + \lambda I)^{-1} A^H B\\), which solves the least-squares +/// problem \\(X = \mathrm{argmin}_{Z \in \Re^{n \times k} } ||A Z - B||_F^2 + \lambda ||Z||_F^2\\). +/// If \\(m \lt n\\) then `output` is computed as +/// \\(X = A^H (A A^H + \lambda I)^{-1} B\\), which (for \\(\lambda = 0\\)) is the +/// minimum-norm solution to the under-determined linear system, i.e. +/// \\(X = \mathrm{argmin}_{Z \in \mathbb{C}^{n \times k} } ||Z||_F^2 \\), +/// subject to \\(A Z = B\\). Notice that the fast path is only numerically stable +/// when \\(A\\) is numerically full rank and has a condition number +/// \\(\mathrm{cond}(A) \lt \frac{1}{\sqrt{\epsilon_{mach} } }\\) or \\(\lambda\\) is +/// sufficiently large. +/// +/// If `fast` is `False` an algorithm based on the numerically robust complete +/// orthogonal decomposition is used. This computes the minimum-norm +/// least-squares solution, even when \\(A\\) is rank deficient. This path is +/// typically 6-7 times slower than the fast path. If `fast` is `False` then +/// `l2_regularizer` is ignored. +/// +/// Arguments: +/// * scope: A Scope object +/// * matrix: Shape is `[..., M, N]`. +/// * rhs: Shape is `[..., M, K]`. +/// * l2_regularizer: Scalar tensor. +/// +/// @compatibility(numpy) +/// Equivalent to np.linalg.lstsq +/// @end_compatibility +/// +/// Returns: +/// * `Output`: Shape is `[..., N, K]`. +class MatrixSolveLs { + public: + /// Optional attribute setters for MatrixSolveLs + struct Attrs { + /// Defaults to true + TF_MUST_USE_RESULT Attrs Fast(bool x) { + Attrs ret = *this; + ret.fast_ = x; + return ret; + } + + bool fast_ = true; + }; + MatrixSolveLs(const ::tensorflow::Scope& scope, ::tensorflow::Input matrix, + ::tensorflow::Input rhs, ::tensorflow::Input l2_regularizer); + MatrixSolveLs(const ::tensorflow::Scope& scope, ::tensorflow::Input matrix, + ::tensorflow::Input rhs, ::tensorflow::Input l2_regularizer, + const MatrixSolveLs::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs Fast(bool x) { + return Attrs().Fast(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Solves systems of linear equations with upper or lower triangular matrices by +/// +/// backsubstitution. +/// +/// `matrix` is a tensor of shape `[..., M, M]` whose inner-most 2 dimensions form +/// square matrices. If `lower` is `True` then the strictly upper triangular part +/// of each inner-most matrix is assumed to be zero and not accessed. +/// If `lower` is False then the strictly lower triangular part of each inner-most +/// matrix is assumed to be zero and not accessed. +/// `rhs` is a tensor of shape `[..., M, K]`. +/// +/// The output is a tensor of shape `[..., M, K]`. If `adjoint` is +/// `True` then the innermost matrices in `output` satisfy matrix equations +/// `matrix[..., :, :] * output[..., :, :] = rhs[..., :, :]`. +/// If `adjoint` is `False` then the strictly then the innermost matrices in +/// `output` satisfy matrix equations +/// `adjoint(matrix[..., i, k]) * output[..., k, j] = rhs[..., i, j]`. +/// +/// Arguments: +/// * scope: A Scope object +/// * matrix: Shape is `[..., M, M]`. +/// * rhs: Shape is `[..., M, K]`. +/// +/// Optional attributes (see `Attrs`): +/// * lower: Boolean indicating whether the innermost matrices in `matrix` are +/// lower or upper triangular. +/// * adjoint: Boolean indicating whether to solve with `matrix` or its (block-wise) +/// adjoint. +/// +/// @compatibility(numpy) +/// Equivalent to scipy.linalg.solve_triangular +/// @end_compatibility +/// +/// Returns: +/// * `Output`: Shape is `[..., M, K]`. +class MatrixTriangularSolve { + public: + /// Optional attribute setters for MatrixTriangularSolve + struct Attrs { + /// Boolean indicating whether the innermost matrices in `matrix` are + /// lower or upper triangular. + /// + /// Defaults to true + TF_MUST_USE_RESULT Attrs Lower(bool x) { + Attrs ret = *this; + ret.lower_ = x; + return ret; + } + + /// Boolean indicating whether to solve with `matrix` or its (block-wise) + /// adjoint. + /// + /// @compatibility(numpy) + /// Equivalent to scipy.linalg.solve_triangular + /// @end_compatibility + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs Adjoint(bool x) { + Attrs ret = *this; + ret.adjoint_ = x; + return ret; + } + + bool lower_ = true; + bool adjoint_ = false; + }; + MatrixTriangularSolve(const ::tensorflow::Scope& scope, ::tensorflow::Input + matrix, ::tensorflow::Input rhs); + MatrixTriangularSolve(const ::tensorflow::Scope& scope, ::tensorflow::Input + matrix, ::tensorflow::Input rhs, const + MatrixTriangularSolve::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs Lower(bool x) { + return Attrs().Lower(x); + } + static Attrs Adjoint(bool x) { + return Attrs().Adjoint(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Computes the QR decompositions of one or more matrices. +/// +/// Computes the QR decomposition of each inner matrix in `tensor` such that +/// `tensor[..., :, :] = q[..., :, :] * r[..., :,:])` +/// +/// ```python +/// # a is a tensor. +/// # q is a tensor of orthonormal matrices. +/// # r is a tensor of upper triangular matrices. +/// q, r = qr(a) +/// q_full, r_full = qr(a, full_matrices=True) +/// ``` +/// +/// Arguments: +/// * scope: A Scope object +/// * input: A tensor of shape `[..., M, N]` whose inner-most 2 dimensions +/// form matrices of size `[M, N]`. Let `P` be the minimum of `M` and `N`. +/// +/// Optional attributes (see `Attrs`): +/// * full_matrices: If true, compute full-sized `q` and `r`. If false +/// (the default), compute only the leading `P` columns of `q`. +/// +/// Returns: +/// * `Output` q: Orthonormal basis for range of `a`. If `full_matrices` is `False` then +/// shape is `[..., M, P]`; if `full_matrices` is `True` then shape is +/// `[..., M, M]`. +/// * `Output` r: Triangular factor. If `full_matrices` is `False` then shape is +/// `[..., P, N]`. If `full_matrices` is `True` then shape is `[..., M, N]`. +class Qr { + public: + /// Optional attribute setters for Qr + struct Attrs { + /// If true, compute full-sized `q` and `r`. If false + /// (the default), compute only the leading `P` columns of `q`. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs FullMatrices(bool x) { + Attrs ret = *this; + ret.full_matrices_ = x; + return ret; + } + + bool full_matrices_ = false; + }; + Qr(const ::tensorflow::Scope& scope, ::tensorflow::Input input); + Qr(const ::tensorflow::Scope& scope, ::tensorflow::Input input, const + Qr::Attrs& attrs); + + static Attrs FullMatrices(bool x) { + return Attrs().FullMatrices(x); + } + + Operation operation; + ::tensorflow::Output q; + ::tensorflow::Output r; +}; + +/// Computes the eigen decomposition of one or more square self-adjoint matrices. +/// +/// Computes the eigenvalues and (optionally) eigenvectors of each inner matrix in +/// `input` such that `input[..., :, :] = v[..., :, :] * diag(e[..., :])`. The eigenvalues +/// are sorted in non-decreasing order. +/// +/// ```python +/// # a is a tensor. +/// # e is a tensor of eigenvalues. +/// # v is a tensor of eigenvectors. +/// e, v = self_adjoint_eig(a) +/// e = self_adjoint_eig(a, compute_v=False) +/// ``` +/// +/// Arguments: +/// * scope: A Scope object +/// * input: `Tensor` input of shape `[N, N]`. +/// +/// Optional attributes (see `Attrs`): +/// * compute_v: If `True` then eigenvectors will be computed and returned in `v`. +/// Otherwise, only the eigenvalues will be computed. +/// +/// Returns: +/// * `Output` e: Eigenvalues. Shape is `[N]`. +/// * `Output` v: Eigenvectors. Shape is `[N, N]`. +class SelfAdjointEig { + public: + /// Optional attribute setters for SelfAdjointEig + struct Attrs { + /// If `True` then eigenvectors will be computed and returned in `v`. + /// Otherwise, only the eigenvalues will be computed. + /// + /// Defaults to true + TF_MUST_USE_RESULT Attrs ComputeV(bool x) { + Attrs ret = *this; + ret.compute_v_ = x; + return ret; + } + + bool compute_v_ = true; + }; + SelfAdjointEig(const ::tensorflow::Scope& scope, ::tensorflow::Input input); + SelfAdjointEig(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + const SelfAdjointEig::Attrs& attrs); + + static Attrs ComputeV(bool x) { + return Attrs().ComputeV(x); + } + + Operation operation; + ::tensorflow::Output e; + ::tensorflow::Output v; +}; + +/// Computes the singular value decompositions of one or more matrices. +/// +/// Computes the SVD of each inner matrix in `input` such that +/// `input[..., :, :] = u[..., :, :] * diag(s[..., :, :]) * transpose(v[..., :, :])` +/// +/// ```python +/// # a is a tensor containing a batch of matrices. +/// # s is a tensor of singular values for each matrix. +/// # u is the tensor containing of left singular vectors for each matrix. +/// # v is the tensor containing of right singular vectors for each matrix. +/// s, u, v = svd(a) +/// s, _, _ = svd(a, compute_uv=False) +/// ``` +/// +/// Arguments: +/// * scope: A Scope object +/// * input: A tensor of shape `[..., M, N]` whose inner-most 2 dimensions +/// form matrices of size `[M, N]`. Let `P` be the minimum of `M` and `N`. +/// +/// Optional attributes (see `Attrs`): +/// * compute_uv: If true, left and right singular vectors will be +/// computed and returned in `u` and `v`, respectively. +/// If false, `u` and `v` are not set and should never referenced. +/// * full_matrices: If true, compute full-sized `u` and `v`. If false +/// (the default), compute only the leading `P` singular vectors. +/// Ignored if `compute_uv` is `False`. +/// +/// Returns: +/// * `Output` s: Singular values. Shape is `[..., P]`. +/// * `Output` u: Left singular vectors. If `full_matrices` is `False` then shape is +/// `[..., M, P]`; if `full_matrices` is `True` then shape is +/// `[..., M, M]`. Undefined if `compute_uv` is `False`. +/// * `Output` v: Left singular vectors. If `full_matrices` is `False` then shape is +/// `[..., N, P]`. If `full_matrices` is `True` then shape is `[..., N, N]`. +/// Undefined if `compute_uv` is false. +class Svd { + public: + /// Optional attribute setters for Svd + struct Attrs { + /// If true, left and right singular vectors will be + /// computed and returned in `u` and `v`, respectively. + /// If false, `u` and `v` are not set and should never referenced. + /// + /// Defaults to true + TF_MUST_USE_RESULT Attrs ComputeUv(bool x) { + Attrs ret = *this; + ret.compute_uv_ = x; + return ret; + } + + /// If true, compute full-sized `u` and `v`. If false + /// (the default), compute only the leading `P` singular vectors. + /// Ignored if `compute_uv` is `False`. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs FullMatrices(bool x) { + Attrs ret = *this; + ret.full_matrices_ = x; + return ret; + } + + bool compute_uv_ = true; + bool full_matrices_ = false; + }; + Svd(const ::tensorflow::Scope& scope, ::tensorflow::Input input); + Svd(const ::tensorflow::Scope& scope, ::tensorflow::Input input, const + Svd::Attrs& attrs); + + static Attrs ComputeUv(bool x) { + return Attrs().ComputeUv(x); + } + static Attrs FullMatrices(bool x) { + return Attrs().FullMatrices(x); + } + + Operation operation; + ::tensorflow::Output s; + ::tensorflow::Output u; + ::tensorflow::Output v; +}; + +/// @} + +} // namespace ops +} // namespace tensorflow + +#endif // TENSORFLOW_CC_OPS_LINALG_OPS_H_ diff --git a/linux/include/tensorflow/cc/ops/linalg_ops_internal.h b/linux/include/tensorflow/cc/ops/linalg_ops_internal.h new file mode 100644 index 000000000..d169f881b --- /dev/null +++ b/linux/include/tensorflow/cc/ops/linalg_ops_internal.h @@ -0,0 +1,67 @@ +// This file is MACHINE GENERATED! Do not edit. + +#ifndef TENSORFLOW_CC_OPS_LINALG_OPS_INTERNAL_H_ +#define TENSORFLOW_CC_OPS_LINALG_OPS_INTERNAL_H_ + +// This file is MACHINE GENERATED! Do not edit. + +#include "tensorflow/cc/framework/ops.h" +#include "tensorflow/cc/framework/scope.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/gtl/array_slice.h" + +namespace tensorflow { +namespace ops { +namespace internal { +// NOTE: This namespace has internal TensorFlow details that +// are not part of TensorFlow's public API. + +/// @defgroup linalg_ops_internal Linalg Ops Internal +/// @{ + +/// Computes the matrix logarithm of one or more square matrices: +/// +/// +/// \\(log(exp(A)) = A\\) +/// +/// This op is only defined for complex matrices. If A is positive-definite and +/// real, then casting to a complex matrix, taking the logarithm and casting back +/// to a real matrix will give the correct result. +/// +/// This function computes the matrix logarithm using the Schur-Parlett algorithm. +/// Details of the algorithm can be found in Section 11.6.2 of: +/// Nicholas J. Higham, Functions of Matrices: Theory and Computation, SIAM 2008. +/// ISBN 978-0-898716-46-7. +/// +/// The input is a tensor of shape `[..., M, M]` whose inner-most 2 dimensions +/// form square matrices. The output is a tensor of the same shape as the input +/// containing the exponential for all input submatrices `[..., :, :]`. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: Shape is `[..., M, M]`. +/// +/// Returns: +/// * `Output`: Shape is `[..., M, M]`. +/// +/// @compatibility(scipy) +/// Equivalent to scipy.linalg.logm +/// @end_compatibility +class MatrixLogarithm { + public: + MatrixLogarithm(const ::tensorflow::Scope& scope, ::tensorflow::Input input); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +} // namespace internal +} // namespace ops +} // namespace tensorflow + +#endif // TENSORFLOW_CC_OPS_LINALG_OPS_INTERNAL_H_ diff --git a/linux/include/tensorflow/cc/ops/logging_ops.h b/linux/include/tensorflow/cc/ops/logging_ops.h new file mode 100644 index 000000000..876776506 --- /dev/null +++ b/linux/include/tensorflow/cc/ops/logging_ops.h @@ -0,0 +1,530 @@ +// This file is MACHINE GENERATED! Do not edit. + +#ifndef TENSORFLOW_CC_OPS_LOGGING_OPS_H_ +#define TENSORFLOW_CC_OPS_LOGGING_OPS_H_ + +// This file is MACHINE GENERATED! Do not edit. + +#include "tensorflow/cc/framework/ops.h" +#include "tensorflow/cc/framework/scope.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/gtl/array_slice.h" + +namespace tensorflow { +namespace ops { + +/// @defgroup logging_ops Logging Ops +/// @{ + +/// Asserts that the given condition is true. +/// +/// If `condition` evaluates to false, print the list of tensors in `data`. +/// `summarize` determines how many entries of the tensors to print. +/// +/// Arguments: +/// * scope: A Scope object +/// * condition: The condition to evaluate. +/// * data: The tensors to print out when condition is false. +/// +/// Optional attributes (see `Attrs`): +/// * summarize: Print this many entries of each tensor. +/// +/// Returns: +/// * the created `Operation` +class Assert { + public: + /// Optional attribute setters for Assert + struct Attrs { + /// Print this many entries of each tensor. + /// + /// Defaults to 3 + TF_MUST_USE_RESULT Attrs Summarize(int64 x) { + Attrs ret = *this; + ret.summarize_ = x; + return ret; + } + + int64 summarize_ = 3; + }; + Assert(const ::tensorflow::Scope& scope, ::tensorflow::Input condition, + ::tensorflow::InputList data); + Assert(const ::tensorflow::Scope& scope, ::tensorflow::Input condition, + ::tensorflow::InputList data, const Assert::Attrs& attrs); + operator ::tensorflow::Operation() const { return operation; } + + static Attrs Summarize(int64 x) { + return Attrs().Summarize(x); + } + + Operation operation; +}; + +/// Outputs a `Summary` protocol buffer with audio. +/// +/// The summary has up to `max_outputs` summary values containing audio. The +/// audio is built from `tensor` which must be 3-D with shape `[batch_size, +/// frames, channels]` or 2-D with shape `[batch_size, frames]`. The values are +/// assumed to be in the range of `[-1.0, 1.0]` with a sample rate of `sample_rate`. +/// +/// The `tag` argument is a scalar `Tensor` of type `string`. It is used to +/// build the `tag` of the summary values: +/// +/// * If `max_outputs` is 1, the summary value tag is '*tag*/audio'. +/// * If `max_outputs` is greater than 1, the summary value tags are +/// generated sequentially as '*tag*/audio/0', '*tag*/audio/1', etc. +/// +/// Arguments: +/// * scope: A Scope object +/// * tag: Scalar. Used to build the `tag` attribute of the summary values. +/// * tensor: 2-D of shape `[batch_size, frames]`. +/// * sample_rate: The sample rate of the signal in hertz. +/// +/// Optional attributes (see `Attrs`): +/// * max_outputs: Max number of batch elements to generate audio for. +/// +/// Returns: +/// * `Output`: Scalar. Serialized `Summary` protocol buffer. +class AudioSummary { + public: + /// Optional attribute setters for AudioSummary + struct Attrs { + /// Max number of batch elements to generate audio for. + /// + /// Defaults to 3 + TF_MUST_USE_RESULT Attrs MaxOutputs(int64 x) { + Attrs ret = *this; + ret.max_outputs_ = x; + return ret; + } + + int64 max_outputs_ = 3; + }; + AudioSummary(const ::tensorflow::Scope& scope, ::tensorflow::Input tag, + ::tensorflow::Input tensor, ::tensorflow::Input sample_rate); + AudioSummary(const ::tensorflow::Scope& scope, ::tensorflow::Input tag, + ::tensorflow::Input tensor, ::tensorflow::Input sample_rate, const + AudioSummary::Attrs& attrs); + operator ::tensorflow::Output() const { return summary; } + operator ::tensorflow::Input() const { return summary; } + ::tensorflow::Node* node() const { return summary.node(); } + + static Attrs MaxOutputs(int64 x) { + return Attrs().MaxOutputs(x); + } + + Operation operation; + ::tensorflow::Output summary; +}; + +/// Outputs a `Summary` protocol buffer with a histogram. +/// +/// The generated +/// [`Summary`](https://www.tensorflow.org/code/tensorflow/core/framework/summary.proto) +/// has one summary value containing a histogram for `values`. +/// +/// This op reports an `InvalidArgument` error if any value is not finite. +/// +/// Arguments: +/// * scope: A Scope object +/// * tag: Scalar. Tag to use for the `Summary.Value`. +/// * values: Any shape. Values to use to build the histogram. +/// +/// Returns: +/// * `Output`: Scalar. Serialized `Summary` protocol buffer. +class HistogramSummary { + public: + HistogramSummary(const ::tensorflow::Scope& scope, ::tensorflow::Input tag, + ::tensorflow::Input values); + operator ::tensorflow::Output() const { return summary; } + operator ::tensorflow::Input() const { return summary; } + ::tensorflow::Node* node() const { return summary.node(); } + + Operation operation; + ::tensorflow::Output summary; +}; + +/// Outputs a `Summary` protocol buffer with images. +/// +/// The summary has up to `max_images` summary values containing images. The +/// images are built from `tensor` which must be 4-D with shape `[batch_size, +/// height, width, channels]` and where `channels` can be: +/// +/// * 1: `tensor` is interpreted as Grayscale. +/// * 3: `tensor` is interpreted as RGB. +/// * 4: `tensor` is interpreted as RGBA. +/// +/// The images have the same number of channels as the input tensor. For float +/// input, the values are normalized one image at a time to fit in the range +/// `[0, 255]`. `uint8` values are unchanged. The op uses two different +/// normalization algorithms: +/// +/// * If the input values are all positive, they are rescaled so the largest one +/// is 255. +/// +/// * If any input value is negative, the values are shifted so input value 0.0 +/// is at 127. They are then rescaled so that either the smallest value is 0, +/// or the largest one is 255. +/// +/// The `tag` argument is a scalar `Tensor` of type `string`. It is used to +/// build the `tag` of the summary values: +/// +/// * If `max_images` is 1, the summary value tag is '*tag*/image'. +/// * If `max_images` is greater than 1, the summary value tags are +/// generated sequentially as '*tag*/image/0', '*tag*/image/1', etc. +/// +/// The `bad_color` argument is the color to use in the generated images for +/// non-finite input values. It is a `uint8` 1-D tensor of length `channels`. +/// Each element must be in the range `[0, 255]` (It represents the value of a +/// pixel in the output image). Non-finite values in the input tensor are +/// replaced by this tensor in the output image. The default value is the color +/// red. +/// +/// Arguments: +/// * scope: A Scope object +/// * tag: Scalar. Used to build the `tag` attribute of the summary values. +/// * tensor: 4-D of shape `[batch_size, height, width, channels]` where +/// `channels` is 1, 3, or 4. +/// +/// Optional attributes (see `Attrs`): +/// * max_images: Max number of batch elements to generate images for. +/// * bad_color: Color to use for pixels with non-finite values. +/// +/// Returns: +/// * `Output`: Scalar. Serialized `Summary` protocol buffer. +class ImageSummary { + public: + /// Optional attribute setters for ImageSummary + struct Attrs { + /// Max number of batch elements to generate images for. + /// + /// Defaults to 3 + TF_MUST_USE_RESULT Attrs MaxImages(int64 x) { + Attrs ret = *this; + ret.max_images_ = x; + return ret; + } + + /// Color to use for pixels with non-finite values. + /// + /// Defaults to Tensor + TF_MUST_USE_RESULT Attrs BadColor(const TensorProto& x) { + Attrs ret = *this; + ret.bad_color_ = x; + return ret; + } + + int64 max_images_ = 3; + TensorProto bad_color_ = Input::Initializer({255, 0, 0, 255}, {4}).AsTensorProto(); + }; + ImageSummary(const ::tensorflow::Scope& scope, ::tensorflow::Input tag, + ::tensorflow::Input tensor); + ImageSummary(const ::tensorflow::Scope& scope, ::tensorflow::Input tag, + ::tensorflow::Input tensor, const ImageSummary::Attrs& attrs); + operator ::tensorflow::Output() const { return summary; } + operator ::tensorflow::Input() const { return summary; } + ::tensorflow::Node* node() const { return summary.node(); } + + static Attrs MaxImages(int64 x) { + return Attrs().MaxImages(x); + } + static Attrs BadColor(const TensorProto& x) { + return Attrs().BadColor(x); + } + + Operation operation; + ::tensorflow::Output summary; +}; + +/// Merges summaries. +/// +/// This op creates a +/// [`Summary`](https://www.tensorflow.org/code/tensorflow/core/framework/summary.proto) +/// protocol buffer that contains the union of all the values in the input +/// summaries. +/// +/// When the Op is run, it reports an `InvalidArgument` error if multiple values +/// in the summaries to merge use the same tag. +/// +/// Arguments: +/// * scope: A Scope object +/// * inputs: Can be of any shape. Each must contain serialized `Summary` protocol +/// buffers. +/// +/// Returns: +/// * `Output`: Scalar. Serialized `Summary` protocol buffer. +class MergeSummary { + public: + MergeSummary(const ::tensorflow::Scope& scope, ::tensorflow::InputList inputs); + operator ::tensorflow::Output() const { return summary; } + operator ::tensorflow::Input() const { return summary; } + ::tensorflow::Node* node() const { return summary.node(); } + + Operation operation; + ::tensorflow::Output summary; +}; + +/// Prints a list of tensors. +/// +/// Passes `input` through to `output` and prints `data` when evaluating. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: The tensor passed to `output` +/// * data: A list of tensors to print out when op is evaluated. +/// +/// Optional attributes (see `Attrs`): +/// * message: A string, prefix of the error message. +/// * first_n: Only log `first_n` number of times. -1 disables logging. +/// * summarize: Only print this many entries of each tensor. +/// +/// Returns: +/// * `Output`: = The unmodified `input` tensor +class Print { + public: + /// Optional attribute setters for Print + struct Attrs { + /// A string, prefix of the error message. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs Message(StringPiece x) { + Attrs ret = *this; + ret.message_ = x; + return ret; + } + + /// Only log `first_n` number of times. -1 disables logging. + /// + /// Defaults to -1 + TF_MUST_USE_RESULT Attrs FirstN(int64 x) { + Attrs ret = *this; + ret.first_n_ = x; + return ret; + } + + /// Only print this many entries of each tensor. + /// + /// Defaults to 3 + TF_MUST_USE_RESULT Attrs Summarize(int64 x) { + Attrs ret = *this; + ret.summarize_ = x; + return ret; + } + + StringPiece message_ = ""; + int64 first_n_ = -1; + int64 summarize_ = 3; + }; + Print(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::InputList data); + Print(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::InputList data, const Print::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs Message(StringPiece x) { + return Attrs().Message(x); + } + static Attrs FirstN(int64 x) { + return Attrs().FirstN(x); + } + static Attrs Summarize(int64 x) { + return Attrs().Summarize(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Prints a string scalar. +/// +/// Prints a string scalar to the desired output_stream. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: The string scalar to print. +/// +/// Optional attributes (see `Attrs`): +/// * output_stream: A string specifying the output stream or logging level to print to. +/// +/// Returns: +/// * the created `Operation` +class PrintV2 { + public: + /// Optional attribute setters for PrintV2 + struct Attrs { + /// A string specifying the output stream or logging level to print to. + /// + /// Defaults to "stderr" + TF_MUST_USE_RESULT Attrs OutputStream(StringPiece x) { + Attrs ret = *this; + ret.output_stream_ = x; + return ret; + } + + StringPiece output_stream_ = "stderr"; + }; + PrintV2(const ::tensorflow::Scope& scope, ::tensorflow::Input input); + PrintV2(const ::tensorflow::Scope& scope, ::tensorflow::Input input, const + PrintV2::Attrs& attrs); + operator ::tensorflow::Operation() const { return operation; } + + static Attrs OutputStream(StringPiece x) { + return Attrs().OutputStream(x); + } + + Operation operation; +}; + +/// Outputs a `Summary` protocol buffer with scalar values. +/// +/// The input `tags` and `values` must have the same shape. The generated summary +/// has a summary value for each tag-value pair in `tags` and `values`. +/// +/// Arguments: +/// * scope: A Scope object +/// * tags: Tags for the summary. +/// * values: Same shape as `tags. Values for the summary. +/// +/// Returns: +/// * `Output`: Scalar. Serialized `Summary` protocol buffer. +class ScalarSummary { + public: + ScalarSummary(const ::tensorflow::Scope& scope, ::tensorflow::Input tags, + ::tensorflow::Input values); + operator ::tensorflow::Output() const { return summary; } + operator ::tensorflow::Input() const { return summary; } + ::tensorflow::Node* node() const { return summary.node(); } + + Operation operation; + ::tensorflow::Output summary; +}; + +/// Outputs a `Summary` protocol buffer with a tensor. +/// +/// This op is being phased out in favor of TensorSummaryV2, which lets callers pass +/// a tag as well as a serialized SummaryMetadata proto string that contains +/// plugin-specific data. We will keep this op to maintain backwards compatibility. +/// +/// Arguments: +/// * scope: A Scope object +/// * tensor: A tensor to serialize. +/// +/// Optional attributes (see `Attrs`): +/// * description: A json-encoded SummaryDescription proto. +/// * labels: An unused list of strings. +/// * display_name: An unused string. +/// +/// Returns: +/// * `Output`: The summary tensor. +class TensorSummary { + public: + /// Optional attribute setters for TensorSummary + struct Attrs { + /// A json-encoded SummaryDescription proto. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs Description(StringPiece x) { + Attrs ret = *this; + ret.description_ = x; + return ret; + } + + /// An unused list of strings. + /// + /// Defaults to [] + TF_MUST_USE_RESULT Attrs Labels(const gtl::ArraySlice& x) { + Attrs ret = *this; + ret.labels_ = x; + return ret; + } + + /// An unused string. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs DisplayName(StringPiece x) { + Attrs ret = *this; + ret.display_name_ = x; + return ret; + } + + StringPiece description_ = ""; + gtl::ArraySlice labels_ = {}; + StringPiece display_name_ = ""; + }; + TensorSummary(const ::tensorflow::Scope& scope, ::tensorflow::Input tensor); + TensorSummary(const ::tensorflow::Scope& scope, ::tensorflow::Input tensor, + const TensorSummary::Attrs& attrs); + operator ::tensorflow::Output() const { return summary; } + operator ::tensorflow::Input() const { return summary; } + ::tensorflow::Node* node() const { return summary.node(); } + + static Attrs Description(StringPiece x) { + return Attrs().Description(x); + } + static Attrs Labels(const gtl::ArraySlice& x) { + return Attrs().Labels(x); + } + static Attrs DisplayName(StringPiece x) { + return Attrs().DisplayName(x); + } + + Operation operation; + ::tensorflow::Output summary; +}; + +/// Outputs a `Summary` protocol buffer with a tensor and per-plugin data. +/// +/// Arguments: +/// * scope: A Scope object +/// * tag: A string attached to this summary. Used for organization in TensorBoard. +/// * tensor: A tensor to serialize. +/// * serialized_summary_metadata: A serialized SummaryMetadata proto. Contains plugin +/// data. +/// +/// Returns: +/// * `Output`: The summary tensor. +class TensorSummaryV2 { + public: + TensorSummaryV2(const ::tensorflow::Scope& scope, ::tensorflow::Input tag, + ::tensorflow::Input tensor, ::tensorflow::Input + serialized_summary_metadata); + operator ::tensorflow::Output() const { return summary; } + operator ::tensorflow::Input() const { return summary; } + ::tensorflow::Node* node() const { return summary.node(); } + + Operation operation; + ::tensorflow::Output summary; +}; + +/// Provides the time since epoch in seconds. +/// +/// Returns the timestamp as a `float64` for seconds since the Unix epoch. +/// +/// Note: the timestamp is computed when the op is executed, not when it is added +/// to the graph. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The ts tensor. +class Timestamp { + public: + Timestamp(const ::tensorflow::Scope& scope); + operator ::tensorflow::Output() const { return ts; } + operator ::tensorflow::Input() const { return ts; } + ::tensorflow::Node* node() const { return ts.node(); } + + Operation operation; + ::tensorflow::Output ts; +}; + +/// @} + +} // namespace ops +} // namespace tensorflow + +#endif // TENSORFLOW_CC_OPS_LOGGING_OPS_H_ diff --git a/linux/include/tensorflow/cc/ops/logging_ops_internal.h b/linux/include/tensorflow/cc/ops/logging_ops_internal.h new file mode 100644 index 000000000..d49835bbc --- /dev/null +++ b/linux/include/tensorflow/cc/ops/logging_ops_internal.h @@ -0,0 +1,28 @@ +// This file is MACHINE GENERATED! Do not edit. + +#ifndef TENSORFLOW_CC_OPS_LOGGING_OPS_INTERNAL_H_ +#define TENSORFLOW_CC_OPS_LOGGING_OPS_INTERNAL_H_ + +// This file is MACHINE GENERATED! Do not edit. + +#include "tensorflow/cc/framework/ops.h" +#include "tensorflow/cc/framework/scope.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/gtl/array_slice.h" + +namespace tensorflow { +namespace ops { +namespace internal { +// NOTE: This namespace has internal TensorFlow details that +// are not part of TensorFlow's public API. + +/// @defgroup logging_ops_internal Logging Ops Internal +/// @{ + +} // namespace internal +} // namespace ops +} // namespace tensorflow + +#endif // TENSORFLOW_CC_OPS_LOGGING_OPS_INTERNAL_H_ diff --git a/linux/include/tensorflow/cc/ops/lookup_ops.h b/linux/include/tensorflow/cc/ops/lookup_ops.h new file mode 100644 index 000000000..8d60704fa --- /dev/null +++ b/linux/include/tensorflow/cc/ops/lookup_ops.h @@ -0,0 +1,610 @@ +// This file is MACHINE GENERATED! Do not edit. + +#ifndef TENSORFLOW_CC_OPS_LOOKUP_OPS_H_ +#define TENSORFLOW_CC_OPS_LOOKUP_OPS_H_ + +// This file is MACHINE GENERATED! Do not edit. + +#include "tensorflow/cc/framework/ops.h" +#include "tensorflow/cc/framework/scope.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/gtl/array_slice.h" + +namespace tensorflow { +namespace ops { + +/// @defgroup lookup_ops Lookup Ops +/// @{ + +/// Creates a non-initialized hash table. +/// +/// This op creates a hash table, specifying the type of its keys and values. +/// Before using the table you will have to initialize it. After initialization the +/// table will be immutable. +/// +/// Arguments: +/// * scope: A Scope object +/// * key_dtype: Type of the table keys. +/// * value_dtype: Type of the table values. +/// +/// Optional attributes (see `Attrs`): +/// * container: If non-empty, this table is placed in the given container. +/// Otherwise, a default container is used. +/// * shared_name: If non-empty, this table is shared under the given name across +/// multiple sessions. +/// * use_node_name_sharing: If true and shared_name is empty, the table is shared +/// using the node name. +/// +/// Returns: +/// * `Output`: Handle to a table. +class HashTable { + public: + /// Optional attribute setters for HashTable + struct Attrs { + /// If non-empty, this table is placed in the given container. + /// Otherwise, a default container is used. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs Container(StringPiece x) { + Attrs ret = *this; + ret.container_ = x; + return ret; + } + + /// If non-empty, this table is shared under the given name across + /// multiple sessions. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs SharedName(StringPiece x) { + Attrs ret = *this; + ret.shared_name_ = x; + return ret; + } + + /// If true and shared_name is empty, the table is shared + /// using the node name. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseNodeNameSharing(bool x) { + Attrs ret = *this; + ret.use_node_name_sharing_ = x; + return ret; + } + + StringPiece container_ = ""; + StringPiece shared_name_ = ""; + bool use_node_name_sharing_ = false; + }; + HashTable(const ::tensorflow::Scope& scope, DataType key_dtype, DataType + value_dtype); + HashTable(const ::tensorflow::Scope& scope, DataType key_dtype, DataType + value_dtype, const HashTable::Attrs& attrs); + operator ::tensorflow::Output() const { return table_handle; } + operator ::tensorflow::Input() const { return table_handle; } + ::tensorflow::Node* node() const { return table_handle.node(); } + + static Attrs Container(StringPiece x) { + return Attrs().Container(x); + } + static Attrs SharedName(StringPiece x) { + return Attrs().SharedName(x); + } + static Attrs UseNodeNameSharing(bool x) { + return Attrs().UseNodeNameSharing(x); + } + + Operation operation; + ::tensorflow::Output table_handle; +}; + +/// Initializes a table from a text file. +/// +/// It inserts one key-value pair into the table for each line of the file. +/// The key and value is extracted from the whole line content, elements from the +/// split line based on `delimiter` or the line number (starting from zero). +/// Where to extract the key and value from a line is specified by `key_index` and +/// `value_index`. +/// +/// - A value of -1 means use the line number(starting from zero), expects `int64`. +/// - A value of -2 means use the whole line content, expects `string`. +/// - A value >= 0 means use the index (starting at zero) of the split line based +/// on `delimiter`. +/// +/// Arguments: +/// * scope: A Scope object +/// * table_handle: Handle to a table which will be initialized. +/// * filename: Filename of a vocabulary text file. +/// * key_index: Column index in a line to get the table `key` values from. +/// * value_index: Column index that represents information of a line to get the table +/// `value` values from. +/// +/// Optional attributes (see `Attrs`): +/// * vocab_size: Number of elements of the file, use -1 if unknown. +/// * delimiter: Delimiter to separate fields in a line. +/// +/// Returns: +/// * the created `Operation` +class InitializeTableFromTextFile { + public: + /// Optional attribute setters for InitializeTableFromTextFile + struct Attrs { + /// Number of elements of the file, use -1 if unknown. + /// + /// Defaults to -1 + TF_MUST_USE_RESULT Attrs VocabSize(int64 x) { + Attrs ret = *this; + ret.vocab_size_ = x; + return ret; + } + + /// Delimiter to separate fields in a line. + /// + /// Defaults to "\t" + TF_MUST_USE_RESULT Attrs Delimiter(StringPiece x) { + Attrs ret = *this; + ret.delimiter_ = x; + return ret; + } + + int64 vocab_size_ = -1; + StringPiece delimiter_ = "\t"; + }; + InitializeTableFromTextFile(const ::tensorflow::Scope& scope, + ::tensorflow::Input table_handle, + ::tensorflow::Input filename, int64 key_index, + int64 value_index); + InitializeTableFromTextFile(const ::tensorflow::Scope& scope, + ::tensorflow::Input table_handle, + ::tensorflow::Input filename, int64 key_index, + int64 value_index, const + InitializeTableFromTextFile::Attrs& attrs); + operator ::tensorflow::Operation() const { return operation; } + + static Attrs VocabSize(int64 x) { + return Attrs().VocabSize(x); + } + static Attrs Delimiter(StringPiece x) { + return Attrs().Delimiter(x); + } + + Operation operation; +}; + +/// Table initializer that takes two tensors for keys and values respectively. +/// +/// Arguments: +/// * scope: A Scope object +/// * table_handle: Handle to a table which will be initialized. +/// * keys: Keys of type Tkey. +/// * values: Values of type Tval. +/// +/// Returns: +/// * the created `Operation` +class InitializeTable { + public: + InitializeTable(const ::tensorflow::Scope& scope, ::tensorflow::Input + table_handle, ::tensorflow::Input keys, ::tensorflow::Input + values); + operator ::tensorflow::Operation() const { return operation; } + + Operation operation; +}; + +/// Outputs all keys and values in the table. +/// +/// Arguments: +/// * scope: A Scope object +/// * table_handle: Handle to the table. +/// +/// Returns: +/// * `Output` keys: Vector of all keys present in the table. +/// * `Output` values: Tensor of all values in the table. Indexed in parallel with `keys`. +class LookupTableExport { + public: + LookupTableExport(const ::tensorflow::Scope& scope, ::tensorflow::Input + table_handle, DataType Tkeys, DataType Tvalues); + + Operation operation; + ::tensorflow::Output keys; + ::tensorflow::Output values; +}; + +/// Looks up keys in a table, outputs the corresponding values. +/// +/// The tensor `keys` must of the same type as the keys of the table. +/// The output `values` is of the type of the table values. +/// +/// The scalar `default_value` is the value output for keys not present in the +/// table. It must also be of the same type as the table values. +/// +/// Arguments: +/// * scope: A Scope object +/// * table_handle: Handle to the table. +/// * keys: Any shape. Keys to look up. +/// +/// Returns: +/// * `Output`: Same shape as `keys`. Values found in the table, or `default_values` +/// for missing keys. +class LookupTableFind { + public: + LookupTableFind(const ::tensorflow::Scope& scope, ::tensorflow::Input + table_handle, ::tensorflow::Input keys, ::tensorflow::Input + default_value); + operator ::tensorflow::Output() const { return values; } + operator ::tensorflow::Input() const { return values; } + ::tensorflow::Node* node() const { return values.node(); } + + Operation operation; + ::tensorflow::Output values; +}; + +/// Replaces the contents of the table with the specified keys and values. +/// +/// The tensor `keys` must be of the same type as the keys of the table. +/// The tensor `values` must be of the type of the table values. +/// +/// Arguments: +/// * scope: A Scope object +/// * table_handle: Handle to the table. +/// * keys: Any shape. Keys to look up. +/// * values: Values to associate with keys. +/// +/// Returns: +/// * the created `Operation` +class LookupTableImport { + public: + LookupTableImport(const ::tensorflow::Scope& scope, ::tensorflow::Input + table_handle, ::tensorflow::Input keys, ::tensorflow::Input + values); + operator ::tensorflow::Operation() const { return operation; } + + Operation operation; +}; + +/// Updates the table to associates keys with values. +/// +/// The tensor `keys` must be of the same type as the keys of the table. +/// The tensor `values` must be of the type of the table values. +/// +/// Arguments: +/// * scope: A Scope object +/// * table_handle: Handle to the table. +/// * keys: Any shape. Keys to look up. +/// * values: Values to associate with keys. +/// +/// Returns: +/// * the created `Operation` +class LookupTableInsert { + public: + LookupTableInsert(const ::tensorflow::Scope& scope, ::tensorflow::Input + table_handle, ::tensorflow::Input keys, ::tensorflow::Input + values); + operator ::tensorflow::Operation() const { return operation; } + + Operation operation; +}; + +/// Computes the number of elements in the given table. +/// +/// Arguments: +/// * scope: A Scope object +/// * table_handle: Handle to the table. +/// +/// Returns: +/// * `Output`: Scalar that contains number of elements in the table. +class LookupTableSize { + public: + LookupTableSize(const ::tensorflow::Scope& scope, ::tensorflow::Input + table_handle); + operator ::tensorflow::Output() const { return size; } + operator ::tensorflow::Input() const { return size; } + ::tensorflow::Node* node() const { return size.node(); } + + Operation operation; + ::tensorflow::Output size; +}; + +/// Creates an empty hash table that uses tensors as the backing store. +/// +/// It uses "open addressing" with quadratic reprobing to resolve +/// collisions. +/// +/// This op creates a mutable hash table, specifying the type of its keys and +/// values. Each value must be a scalar. Data can be inserted into the table using +/// the insert operations. It does not support the initialization operation. +/// +/// Arguments: +/// * scope: A Scope object +/// * empty_key: The key used to represent empty key buckets internally. Must not +/// be used in insert or lookup operations. +/// * value_dtype: Type of the table values. +/// +/// Optional attributes (see `Attrs`): +/// * container: If non-empty, this table is placed in the given container. +/// Otherwise, a default container is used. +/// * shared_name: If non-empty, this table is shared under the given name across +/// multiple sessions. +/// * value_shape: The shape of each value. +/// * initial_num_buckets: The initial number of hash table buckets. Must be a power +/// to 2. +/// * max_load_factor: The maximum ratio between number of entries and number of +/// buckets before growing the table. Must be between 0 and 1. +/// +/// Returns: +/// * `Output`: Handle to a table. +class MutableDenseHashTable { + public: + /// Optional attribute setters for MutableDenseHashTable + struct Attrs { + /// If non-empty, this table is placed in the given container. + /// Otherwise, a default container is used. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs Container(StringPiece x) { + Attrs ret = *this; + ret.container_ = x; + return ret; + } + + /// If non-empty, this table is shared under the given name across + /// multiple sessions. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs SharedName(StringPiece x) { + Attrs ret = *this; + ret.shared_name_ = x; + return ret; + } + + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseNodeNameSharing(bool x) { + Attrs ret = *this; + ret.use_node_name_sharing_ = x; + return ret; + } + + /// The shape of each value. + /// + /// Defaults to [] + TF_MUST_USE_RESULT Attrs ValueShape(PartialTensorShape x) { + Attrs ret = *this; + ret.value_shape_ = x; + return ret; + } + + /// The initial number of hash table buckets. Must be a power + /// to 2. + /// + /// Defaults to 131072 + TF_MUST_USE_RESULT Attrs InitialNumBuckets(int64 x) { + Attrs ret = *this; + ret.initial_num_buckets_ = x; + return ret; + } + + /// The maximum ratio between number of entries and number of + /// buckets before growing the table. Must be between 0 and 1. + /// + /// Defaults to 0.8 + TF_MUST_USE_RESULT Attrs MaxLoadFactor(float x) { + Attrs ret = *this; + ret.max_load_factor_ = x; + return ret; + } + + StringPiece container_ = ""; + StringPiece shared_name_ = ""; + bool use_node_name_sharing_ = false; + PartialTensorShape value_shape_ = {}; + int64 initial_num_buckets_ = 131072; + float max_load_factor_ = 0.8f; + }; + MutableDenseHashTable(const ::tensorflow::Scope& scope, ::tensorflow::Input + empty_key, DataType value_dtype); + MutableDenseHashTable(const ::tensorflow::Scope& scope, ::tensorflow::Input + empty_key, DataType value_dtype, const + MutableDenseHashTable::Attrs& attrs); + operator ::tensorflow::Output() const { return table_handle; } + operator ::tensorflow::Input() const { return table_handle; } + ::tensorflow::Node* node() const { return table_handle.node(); } + + static Attrs Container(StringPiece x) { + return Attrs().Container(x); + } + static Attrs SharedName(StringPiece x) { + return Attrs().SharedName(x); + } + static Attrs UseNodeNameSharing(bool x) { + return Attrs().UseNodeNameSharing(x); + } + static Attrs ValueShape(PartialTensorShape x) { + return Attrs().ValueShape(x); + } + static Attrs InitialNumBuckets(int64 x) { + return Attrs().InitialNumBuckets(x); + } + static Attrs MaxLoadFactor(float x) { + return Attrs().MaxLoadFactor(x); + } + + Operation operation; + ::tensorflow::Output table_handle; +}; + +/// Creates an empty hash table. +/// +/// This op creates a mutable hash table, specifying the type of its keys and +/// values. Each value must be a vector. Data can be inserted into the table using +/// the insert operations. It does not support the initialization operation. +/// +/// Arguments: +/// * scope: A Scope object +/// * key_dtype: Type of the table keys. +/// * value_dtype: Type of the table values. +/// +/// Optional attributes (see `Attrs`): +/// * container: If non-empty, this table is placed in the given container. +/// Otherwise, a default container is used. +/// * shared_name: If non-empty, this table is shared under the given name across +/// multiple sessions. +/// +/// Returns: +/// * `Output`: Handle to a table. +class MutableHashTableOfTensors { + public: + /// Optional attribute setters for MutableHashTableOfTensors + struct Attrs { + /// If non-empty, this table is placed in the given container. + /// Otherwise, a default container is used. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs Container(StringPiece x) { + Attrs ret = *this; + ret.container_ = x; + return ret; + } + + /// If non-empty, this table is shared under the given name across + /// multiple sessions. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs SharedName(StringPiece x) { + Attrs ret = *this; + ret.shared_name_ = x; + return ret; + } + + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseNodeNameSharing(bool x) { + Attrs ret = *this; + ret.use_node_name_sharing_ = x; + return ret; + } + + /// Defaults to [] + TF_MUST_USE_RESULT Attrs ValueShape(PartialTensorShape x) { + Attrs ret = *this; + ret.value_shape_ = x; + return ret; + } + + StringPiece container_ = ""; + StringPiece shared_name_ = ""; + bool use_node_name_sharing_ = false; + PartialTensorShape value_shape_ = {}; + }; + MutableHashTableOfTensors(const ::tensorflow::Scope& scope, DataType key_dtype, + DataType value_dtype); + MutableHashTableOfTensors(const ::tensorflow::Scope& scope, DataType key_dtype, + DataType value_dtype, const + MutableHashTableOfTensors::Attrs& attrs); + operator ::tensorflow::Output() const { return table_handle; } + operator ::tensorflow::Input() const { return table_handle; } + ::tensorflow::Node* node() const { return table_handle.node(); } + + static Attrs Container(StringPiece x) { + return Attrs().Container(x); + } + static Attrs SharedName(StringPiece x) { + return Attrs().SharedName(x); + } + static Attrs UseNodeNameSharing(bool x) { + return Attrs().UseNodeNameSharing(x); + } + static Attrs ValueShape(PartialTensorShape x) { + return Attrs().ValueShape(x); + } + + Operation operation; + ::tensorflow::Output table_handle; +}; + +/// Creates an empty hash table. +/// +/// This op creates a mutable hash table, specifying the type of its keys and +/// values. Each value must be a scalar. Data can be inserted into the table using +/// the insert operations. It does not support the initialization operation. +/// +/// Arguments: +/// * scope: A Scope object +/// * key_dtype: Type of the table keys. +/// * value_dtype: Type of the table values. +/// +/// Optional attributes (see `Attrs`): +/// * container: If non-empty, this table is placed in the given container. +/// Otherwise, a default container is used. +/// * shared_name: If non-empty, this table is shared under the given name across +/// multiple sessions. +/// * use_node_name_sharing: If true and shared_name is empty, the table is shared +/// using the node name. +/// +/// Returns: +/// * `Output`: Handle to a table. +class MutableHashTable { + public: + /// Optional attribute setters for MutableHashTable + struct Attrs { + /// If non-empty, this table is placed in the given container. + /// Otherwise, a default container is used. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs Container(StringPiece x) { + Attrs ret = *this; + ret.container_ = x; + return ret; + } + + /// If non-empty, this table is shared under the given name across + /// multiple sessions. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs SharedName(StringPiece x) { + Attrs ret = *this; + ret.shared_name_ = x; + return ret; + } + + /// If true and shared_name is empty, the table is shared + /// using the node name. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseNodeNameSharing(bool x) { + Attrs ret = *this; + ret.use_node_name_sharing_ = x; + return ret; + } + + StringPiece container_ = ""; + StringPiece shared_name_ = ""; + bool use_node_name_sharing_ = false; + }; + MutableHashTable(const ::tensorflow::Scope& scope, DataType key_dtype, DataType + value_dtype); + MutableHashTable(const ::tensorflow::Scope& scope, DataType key_dtype, DataType + value_dtype, const MutableHashTable::Attrs& attrs); + operator ::tensorflow::Output() const { return table_handle; } + operator ::tensorflow::Input() const { return table_handle; } + ::tensorflow::Node* node() const { return table_handle.node(); } + + static Attrs Container(StringPiece x) { + return Attrs().Container(x); + } + static Attrs SharedName(StringPiece x) { + return Attrs().SharedName(x); + } + static Attrs UseNodeNameSharing(bool x) { + return Attrs().UseNodeNameSharing(x); + } + + Operation operation; + ::tensorflow::Output table_handle; +}; + +/// @} + +} // namespace ops +} // namespace tensorflow + +#endif // TENSORFLOW_CC_OPS_LOOKUP_OPS_H_ diff --git a/linux/include/tensorflow/cc/ops/lookup_ops_internal.h b/linux/include/tensorflow/cc/ops/lookup_ops_internal.h new file mode 100644 index 000000000..c83c91431 --- /dev/null +++ b/linux/include/tensorflow/cc/ops/lookup_ops_internal.h @@ -0,0 +1,28 @@ +// This file is MACHINE GENERATED! Do not edit. + +#ifndef TENSORFLOW_CC_OPS_LOOKUP_OPS_INTERNAL_H_ +#define TENSORFLOW_CC_OPS_LOOKUP_OPS_INTERNAL_H_ + +// This file is MACHINE GENERATED! Do not edit. + +#include "tensorflow/cc/framework/ops.h" +#include "tensorflow/cc/framework/scope.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/gtl/array_slice.h" + +namespace tensorflow { +namespace ops { +namespace internal { +// NOTE: This namespace has internal TensorFlow details that +// are not part of TensorFlow's public API. + +/// @defgroup lookup_ops_internal Lookup Ops Internal +/// @{ + +} // namespace internal +} // namespace ops +} // namespace tensorflow + +#endif // TENSORFLOW_CC_OPS_LOOKUP_OPS_INTERNAL_H_ diff --git a/linux/include/tensorflow/cc/ops/manip_ops.h b/linux/include/tensorflow/cc/ops/manip_ops.h new file mode 100644 index 000000000..564d5c0f7 --- /dev/null +++ b/linux/include/tensorflow/cc/ops/manip_ops.h @@ -0,0 +1,76 @@ +// This file is MACHINE GENERATED! Do not edit. + +#ifndef TENSORFLOW_CC_OPS_MANIP_OPS_H_ +#define TENSORFLOW_CC_OPS_MANIP_OPS_H_ + +// This file is MACHINE GENERATED! Do not edit. + +#include "tensorflow/cc/framework/ops.h" +#include "tensorflow/cc/framework/scope.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/gtl/array_slice.h" + +namespace tensorflow { +namespace ops { + +/// @defgroup manip_ops Manip Ops +/// @{ + +/// Rolls the elements of a tensor along an axis. +/// +/// The elements are shifted positively (towards larger indices) by the offset of +/// `shift` along the dimension of `axis`. Negative `shift` values will shift +/// elements in the opposite direction. Elements that roll passed the last position +/// will wrap around to the first and vice versa. Multiple shifts along multiple +/// axes may be specified. +/// +/// For example: +/// +/// ``` +/// # 't' is [0, 1, 2, 3, 4] +/// roll(t, shift=2, axis=0) ==> [3, 4, 0, 1, 2] +/// +/// # shifting along multiple dimensions +/// # 't' is [[0, 1, 2, 3, 4], [5, 6, 7, 8, 9]] +/// roll(t, shift=[1, -2], axis=[0, 1]) ==> [[7, 8, 9, 5, 6], [2, 3, 4, 0, 1]] +/// +/// # shifting along the same axis multiple times +/// # 't' is [[0, 1, 2, 3, 4], [5, 6, 7, 8, 9]] +/// roll(t, shift=[2, -3], axis=[1, 1]) ==> [[1, 2, 3, 4, 0], [6, 7, 8, 9, 5]] +/// ``` +/// +/// Arguments: +/// * scope: A Scope object +/// * shift: Dimension must be 0-D or 1-D. `shift[i]` specifies the number of places by which +/// elements are shifted positively (towards larger indices) along the dimension +/// specified by `axis[i]`. Negative shifts will roll the elements in the opposite +/// direction. +/// * axis: Dimension must be 0-D or 1-D. `axis[i]` specifies the dimension that the shift +/// `shift[i]` should occur. If the same axis is referenced more than once, the +/// total shift for that axis will be the sum of all the shifts that belong to that +/// axis. +/// +/// Returns: +/// * `Output`: Has the same shape and size as the input. The elements are shifted +/// positively (towards larger indices) by the offsets of `shift` along the +/// dimensions of `axis`. +class Roll { + public: + Roll(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input shift, ::tensorflow::Input axis); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// @} + +} // namespace ops +} // namespace tensorflow + +#endif // TENSORFLOW_CC_OPS_MANIP_OPS_H_ diff --git a/linux/include/tensorflow/cc/ops/manip_ops_internal.h b/linux/include/tensorflow/cc/ops/manip_ops_internal.h new file mode 100644 index 000000000..cdf273e5c --- /dev/null +++ b/linux/include/tensorflow/cc/ops/manip_ops_internal.h @@ -0,0 +1,28 @@ +// This file is MACHINE GENERATED! Do not edit. + +#ifndef TENSORFLOW_CC_OPS_MANIP_OPS_INTERNAL_H_ +#define TENSORFLOW_CC_OPS_MANIP_OPS_INTERNAL_H_ + +// This file is MACHINE GENERATED! Do not edit. + +#include "tensorflow/cc/framework/ops.h" +#include "tensorflow/cc/framework/scope.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/gtl/array_slice.h" + +namespace tensorflow { +namespace ops { +namespace internal { +// NOTE: This namespace has internal TensorFlow details that +// are not part of TensorFlow's public API. + +/// @defgroup manip_ops_internal Manip Ops Internal +/// @{ + +} // namespace internal +} // namespace ops +} // namespace tensorflow + +#endif // TENSORFLOW_CC_OPS_MANIP_OPS_INTERNAL_H_ diff --git a/linux/include/tensorflow/cc/ops/math_ops.h b/linux/include/tensorflow/cc/ops/math_ops.h new file mode 100644 index 000000000..e78f10c29 --- /dev/null +++ b/linux/include/tensorflow/cc/ops/math_ops.h @@ -0,0 +1,4102 @@ +// This file is MACHINE GENERATED! Do not edit. + +#ifndef TENSORFLOW_CC_OPS_MATH_OPS_H_ +#define TENSORFLOW_CC_OPS_MATH_OPS_H_ + +// This file is MACHINE GENERATED! Do not edit. + +#include "tensorflow/cc/framework/ops.h" +#include "tensorflow/cc/framework/scope.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/gtl/array_slice.h" + +namespace tensorflow { +namespace ops { + +/// @defgroup math_ops Math Ops +/// @{ + +/// Computes the absolute value of a tensor. +/// +/// Given a tensor `x`, this operation returns a tensor containing the absolute +/// value of each element in `x`. For example, if x is an input element and y is +/// an output element, this operation computes \\(y = |x|\\). +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The y tensor. +class Abs { + public: + Abs(const ::tensorflow::Scope& scope, ::tensorflow::Input x); + operator ::tensorflow::Output() const { return y; } + operator ::tensorflow::Input() const { return y; } + ::tensorflow::Node* node() const { return y.node(); } + + Operation operation; + ::tensorflow::Output y; +}; + +/// Returns the element-wise sum of a list of tensors. +/// +/// `tf.accumulate_n_v2` performs the same operation as `tf.add_n`, but does not +/// wait for all of its inputs to be ready before beginning to sum. This can +/// save memory if inputs are ready at different times, since minimum temporary +/// storage is proportional to the output size rather than the inputs size. +/// +/// Unlike the original `accumulate_n`, `accumulate_n_v2` is differentiable. +/// +/// Returns a `Tensor` of same shape and type as the elements of `inputs`. +/// +/// Arguments: +/// * scope: A Scope object +/// * inputs: A list of `Tensor` objects, each with same shape and type. +/// * shape: Shape of elements of `inputs`. +/// +/// Returns: +/// * `Output`: The sum tensor. +class AccumulateNV2 { + public: + AccumulateNV2(const ::tensorflow::Scope& scope, ::tensorflow::InputList inputs, + PartialTensorShape shape); + operator ::tensorflow::Output() const { return sum; } + operator ::tensorflow::Input() const { return sum; } + ::tensorflow::Node* node() const { return sum.node(); } + + Operation operation; + ::tensorflow::Output sum; +}; + +/// Computes acos of x element-wise. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The y tensor. +class Acos { + public: + Acos(const ::tensorflow::Scope& scope, ::tensorflow::Input x); + operator ::tensorflow::Output() const { return y; } + operator ::tensorflow::Input() const { return y; } + ::tensorflow::Node* node() const { return y.node(); } + + Operation operation; + ::tensorflow::Output y; +}; + +/// Computes inverse hyperbolic cosine of x element-wise. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The y tensor. +class Acosh { + public: + Acosh(const ::tensorflow::Scope& scope, ::tensorflow::Input x); + operator ::tensorflow::Output() const { return y; } + operator ::tensorflow::Input() const { return y; } + ::tensorflow::Node* node() const { return y.node(); } + + Operation operation; + ::tensorflow::Output y; +}; + +/// Returns x + y element-wise. +/// +/// *NOTE*: `Add` supports broadcasting. `AddN` does not. More about broadcasting +/// [here](http://docs.scipy.org/doc/numpy/user/basics.broadcasting.html) +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The z tensor. +class Add { + public: + Add(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input y); + operator ::tensorflow::Output() const { return z; } + operator ::tensorflow::Input() const { return z; } + ::tensorflow::Node* node() const { return z.node(); } + + Operation operation; + ::tensorflow::Output z; +}; + +/// Add all input tensors element wise. +/// +/// Arguments: +/// * scope: A Scope object +/// * inputs: Must all be the same size and shape. +/// +/// Returns: +/// * `Output`: The sum tensor. +class AddN { + public: + AddN(const ::tensorflow::Scope& scope, ::tensorflow::InputList inputs); + operator ::tensorflow::Output() const { return sum; } + operator ::tensorflow::Input() const { return sum; } + ::tensorflow::Node* node() const { return sum.node(); } + + Operation operation; + ::tensorflow::Output sum; +}; + +/// Returns x + y element-wise. +/// +/// *NOTE*: `Add` supports broadcasting. `AddN` does not. More about broadcasting +/// [here](http://docs.scipy.org/doc/numpy/user/basics.broadcasting.html) +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The z tensor. +class AddV2 { + public: + AddV2(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input y); + operator ::tensorflow::Output() const { return z; } + operator ::tensorflow::Input() const { return z; } + ::tensorflow::Node* node() const { return z.node(); } + + Operation operation; + ::tensorflow::Output z; +}; + +/// Computes the "logical and" of elements across dimensions of a tensor. +/// +/// Reduces `input` along the dimensions given in `axis`. Unless +/// `keep_dims` is true, the rank of the tensor is reduced by 1 for each entry in +/// `axis`. If `keep_dims` is true, the reduced dimensions are +/// retained with length 1. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: The tensor to reduce. +/// * axis: The dimensions to reduce. Must be in the range +/// `[-rank(input), rank(input))`. +/// +/// Optional attributes (see `Attrs`): +/// * keep_dims: If true, retain reduced dimensions with length 1. +/// +/// Returns: +/// * `Output`: The reduced tensor. +/// +/// Aliases: +/// * ReduceAll +class All { + public: + /// Optional attribute setters for All + struct Attrs { + /// If true, retain reduced dimensions with length 1. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs KeepDims(bool x) { + Attrs ret = *this; + ret.keep_dims_ = x; + return ret; + } + + bool keep_dims_ = false; + }; + All(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input axis); + All(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input axis, const All::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs KeepDims(bool x) { + return Attrs().KeepDims(x); + } + + Operation operation; + ::tensorflow::Output output; +}; +typedef All ReduceAll; + +/// Returns the argument of a complex number. +/// +/// Given a tensor `input` of complex numbers, this operation returns a tensor of +/// type `float` that is the argument of each element in `input`. All elements in +/// `input` must be complex numbers of the form \\(a + bj\\), where *a* +/// is the real part and *b* is the imaginary part. +/// +/// The argument returned by this operation is of the form \\(atan2(b, a)\\). +/// +/// For example: +/// +/// ``` +/// # tensor 'input' is [-2.25 + 4.75j, 3.25 + 5.75j] +/// tf.angle(input) ==> [2.0132, 1.056] +/// ``` +/// +/// @compatibility(numpy) +/// Equivalent to np.angle. +/// @end_compatibility +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The output tensor. +class Angle { + public: + /// Optional attribute setters for Angle + struct Attrs { + /// Defaults to DT_FLOAT + TF_MUST_USE_RESULT Attrs Tout(DataType x) { + Attrs ret = *this; + ret.Tout_ = x; + return ret; + } + + DataType Tout_ = DT_FLOAT; + }; + Angle(const ::tensorflow::Scope& scope, ::tensorflow::Input input); + Angle(const ::tensorflow::Scope& scope, ::tensorflow::Input input, const + Angle::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs Tout(DataType x) { + return Attrs().Tout(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Computes the "logical or" of elements across dimensions of a tensor. +/// +/// Reduces `input` along the dimensions given in `axis`. Unless +/// `keep_dims` is true, the rank of the tensor is reduced by 1 for each entry in +/// `axis`. If `keep_dims` is true, the reduced dimensions are +/// retained with length 1. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: The tensor to reduce. +/// * axis: The dimensions to reduce. Must be in the range +/// `[-rank(input), rank(input))`. +/// +/// Optional attributes (see `Attrs`): +/// * keep_dims: If true, retain reduced dimensions with length 1. +/// +/// Returns: +/// * `Output`: The reduced tensor. +/// +/// Aliases: +/// * ReduceAny +class Any { + public: + /// Optional attribute setters for Any + struct Attrs { + /// If true, retain reduced dimensions with length 1. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs KeepDims(bool x) { + Attrs ret = *this; + ret.keep_dims_ = x; + return ret; + } + + bool keep_dims_ = false; + }; + Any(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input axis); + Any(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input axis, const Any::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs KeepDims(bool x) { + return Attrs().KeepDims(x); + } + + Operation operation; + ::tensorflow::Output output; +}; +typedef Any ReduceAny; + +/// Returns the truth value of abs(x-y) < tolerance element-wise. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The z tensor. +class ApproximateEqual { + public: + /// Optional attribute setters for ApproximateEqual + struct Attrs { + /// Defaults to 1e-05 + TF_MUST_USE_RESULT Attrs Tolerance(float x) { + Attrs ret = *this; + ret.tolerance_ = x; + return ret; + } + + float tolerance_ = 1e-05f; + }; + ApproximateEqual(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input y); + ApproximateEqual(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input y, const ApproximateEqual::Attrs& attrs); + operator ::tensorflow::Output() const { return z; } + operator ::tensorflow::Input() const { return z; } + ::tensorflow::Node* node() const { return z.node(); } + + static Attrs Tolerance(float x) { + return Attrs().Tolerance(x); + } + + Operation operation; + ::tensorflow::Output z; +}; + +/// Returns the index with the largest value across dimensions of a tensor. +/// +/// Note that in case of ties the identity of the return value is not guaranteed. +/// +/// Arguments: +/// * scope: A Scope object +/// * dimension: int32 or int64, must be in the range `[-rank(input), rank(input))`. +/// Describes which dimension of the input Tensor to reduce across. For vectors, +/// use dimension = 0. +/// +/// Returns: +/// * `Output`: The output tensor. +class ArgMax { + public: + /// Optional attribute setters for ArgMax + struct Attrs { + /// Defaults to DT_INT64 + TF_MUST_USE_RESULT Attrs OutputType(DataType x) { + Attrs ret = *this; + ret.output_type_ = x; + return ret; + } + + DataType output_type_ = DT_INT64; + }; + ArgMax(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input dimension); + ArgMax(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input dimension, const ArgMax::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs OutputType(DataType x) { + return Attrs().OutputType(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Returns the index with the smallest value across dimensions of a tensor. +/// +/// Note that in case of ties the identity of the return value is not guaranteed. +/// +/// Arguments: +/// * scope: A Scope object +/// * dimension: int32 or int64, must be in the range `[-rank(input), rank(input))`. +/// Describes which dimension of the input Tensor to reduce across. For vectors, +/// use dimension = 0. +/// +/// Returns: +/// * `Output`: The output tensor. +class ArgMin { + public: + /// Optional attribute setters for ArgMin + struct Attrs { + /// Defaults to DT_INT64 + TF_MUST_USE_RESULT Attrs OutputType(DataType x) { + Attrs ret = *this; + ret.output_type_ = x; + return ret; + } + + DataType output_type_ = DT_INT64; + }; + ArgMin(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input dimension); + ArgMin(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input dimension, const ArgMin::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs OutputType(DataType x) { + return Attrs().OutputType(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Computes asin of x element-wise. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The y tensor. +class Asin { + public: + Asin(const ::tensorflow::Scope& scope, ::tensorflow::Input x); + operator ::tensorflow::Output() const { return y; } + operator ::tensorflow::Input() const { return y; } + ::tensorflow::Node* node() const { return y.node(); } + + Operation operation; + ::tensorflow::Output y; +}; + +/// Computes inverse hyperbolic sine of x element-wise. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The y tensor. +class Asinh { + public: + Asinh(const ::tensorflow::Scope& scope, ::tensorflow::Input x); + operator ::tensorflow::Output() const { return y; } + operator ::tensorflow::Input() const { return y; } + ::tensorflow::Node* node() const { return y.node(); } + + Operation operation; + ::tensorflow::Output y; +}; + +/// Computes atan of x element-wise. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The y tensor. +class Atan { + public: + Atan(const ::tensorflow::Scope& scope, ::tensorflow::Input x); + operator ::tensorflow::Output() const { return y; } + operator ::tensorflow::Input() const { return y; } + ::tensorflow::Node* node() const { return y.node(); } + + Operation operation; + ::tensorflow::Output y; +}; + +/// Computes arctangent of `y/x` element-wise, respecting signs of the arguments. +/// +/// This is the angle \( \theta \in [-\pi, \pi] \) such that +/// \[ x = r \cos(\theta) \] +/// and +/// \[ y = r \sin(\theta) \] +/// where \(r = \sqrt(x^2 + y^2) \). +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The z tensor. +class Atan2 { + public: + Atan2(const ::tensorflow::Scope& scope, ::tensorflow::Input y, + ::tensorflow::Input x); + operator ::tensorflow::Output() const { return z; } + operator ::tensorflow::Input() const { return z; } + ::tensorflow::Node* node() const { return z.node(); } + + Operation operation; + ::tensorflow::Output z; +}; + +/// Computes inverse hyperbolic tangent of x element-wise. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The y tensor. +class Atanh { + public: + Atanh(const ::tensorflow::Scope& scope, ::tensorflow::Input x); + operator ::tensorflow::Output() const { return y; } + operator ::tensorflow::Input() const { return y; } + ::tensorflow::Node* node() const { return y.node(); } + + Operation operation; + ::tensorflow::Output y; +}; + +/// Multiplies slices of two tensors in batches. +/// +/// Multiplies all slices of `Tensor` `x` and `y` (each slice can be +/// viewed as an element of a batch), and arranges the individual results +/// in a single output tensor of the same batch size. Each of the +/// individual slices can optionally be adjointed (to adjoint a matrix +/// means to transpose and conjugate it) before multiplication by setting +/// the `adj_x` or `adj_y` flag to `True`, which are by default `False`. +/// +/// The input tensors `x` and `y` are 2-D or higher with shape `[..., r_x, c_x]` +/// and `[..., r_y, c_y]`. +/// +/// The output tensor is 2-D or higher with shape `[..., r_o, c_o]`, where: +/// +/// r_o = c_x if adj_x else r_x +/// c_o = r_y if adj_y else c_y +/// +/// It is computed as: +/// +/// output[..., :, :] = matrix(x[..., :, :]) * matrix(y[..., :, :]) +/// +/// Arguments: +/// * scope: A Scope object +/// * x: 2-D or higher with shape `[..., r_x, c_x]`. +/// * y: 2-D or higher with shape `[..., r_y, c_y]`. +/// +/// Optional attributes (see `Attrs`): +/// * adj_x: If `True`, adjoint the slices of `x`. Defaults to `False`. +/// * adj_y: If `True`, adjoint the slices of `y`. Defaults to `False`. +/// +/// Returns: +/// * `Output`: 3-D or higher with shape `[..., r_o, c_o]` +class BatchMatMul { + public: + /// Optional attribute setters for BatchMatMul + struct Attrs { + /// If `True`, adjoint the slices of `x`. Defaults to `False`. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs AdjX(bool x) { + Attrs ret = *this; + ret.adj_x_ = x; + return ret; + } + + /// If `True`, adjoint the slices of `y`. Defaults to `False`. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs AdjY(bool x) { + Attrs ret = *this; + ret.adj_y_ = x; + return ret; + } + + bool adj_x_ = false; + bool adj_y_ = false; + }; + BatchMatMul(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input y); + BatchMatMul(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input y, const BatchMatMul::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs AdjX(bool x) { + return Attrs().AdjX(x); + } + static Attrs AdjY(bool x) { + return Attrs().AdjY(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Computes the Bessel i0e function of `x` element-wise. +/// +/// Exponentially scaled modified Bessel function of order 0 defined as +/// `bessel_i0e(x) = exp(-abs(x)) bessel_i0(x)`. +/// +/// This function is faster and numerically stabler than `bessel_i0(x)`. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The y tensor. +class BesselI0e { + public: + BesselI0e(const ::tensorflow::Scope& scope, ::tensorflow::Input x); + operator ::tensorflow::Output() const { return y; } + operator ::tensorflow::Input() const { return y; } + ::tensorflow::Node* node() const { return y.node(); } + + Operation operation; + ::tensorflow::Output y; +}; + +/// Computes the Bessel i1e function of `x` element-wise. +/// +/// Exponentially scaled modified Bessel function of order 0 defined as +/// `bessel_i1e(x) = exp(-abs(x)) bessel_i1(x)`. +/// +/// This function is faster and numerically stabler than `bessel_i1(x)`. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The y tensor. +class BesselI1e { + public: + BesselI1e(const ::tensorflow::Scope& scope, ::tensorflow::Input x); + operator ::tensorflow::Output() const { return y; } + operator ::tensorflow::Input() const { return y; } + ::tensorflow::Node* node() const { return y.node(); } + + Operation operation; + ::tensorflow::Output y; +}; + +/// Compute the regularized incomplete beta integral \\(I_x(a, b)\\). +/// +/// The regularized incomplete beta integral is defined as: +/// +/// +/// \\(I_x(a, b) = \frac{B(x; a, b)}{B(a, b)}\\) +/// +/// where +/// +/// +/// \\(B(x; a, b) = \int_0^x t^{a-1} (1 - t)^{b-1} dt\\) +/// +/// +/// is the incomplete beta function and \\(B(a, b)\\) is the *complete* +/// beta function. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The z tensor. +class Betainc { + public: + Betainc(const ::tensorflow::Scope& scope, ::tensorflow::Input a, + ::tensorflow::Input b, ::tensorflow::Input x); + operator ::tensorflow::Output() const { return z; } + operator ::tensorflow::Input() const { return z; } + ::tensorflow::Node* node() const { return z.node(); } + + Operation operation; + ::tensorflow::Output z; +}; + +/// Counts the number of occurrences of each value in an integer array. +/// +/// Outputs a vector with length `size` and the same dtype as `weights`. If +/// `weights` are empty, then index `i` stores the number of times the value `i` is +/// counted in `arr`. If `weights` are non-empty, then index `i` stores the sum of +/// the value in `weights` at each index where the corresponding value in `arr` is +/// `i`. +/// +/// Values in `arr` outside of the range [0, size) are ignored. +/// +/// Arguments: +/// * scope: A Scope object +/// * arr: int32 `Tensor`. +/// * size: non-negative int32 scalar `Tensor`. +/// * weights: is an int32, int64, float32, or float64 `Tensor` with the same +/// shape as `arr`, or a length-0 `Tensor`, in which case it acts as all weights +/// equal to 1. +/// +/// Returns: +/// * `Output`: 1D `Tensor` with length equal to `size`. The counts or summed weights for +/// each value in the range [0, size). +class Bincount { + public: + Bincount(const ::tensorflow::Scope& scope, ::tensorflow::Input arr, + ::tensorflow::Input size, ::tensorflow::Input weights); + operator ::tensorflow::Output() const { return bins; } + operator ::tensorflow::Input() const { return bins; } + ::tensorflow::Node* node() const { return bins.node(); } + + Operation operation; + ::tensorflow::Output bins; +}; + +/// Bucketizes 'input' based on 'boundaries'. +/// +/// For example, if the inputs are +/// boundaries = [0, 10, 100] +/// input = [[-5, 10000] +/// [150, 10] +/// [5, 100]] +/// +/// then the output will be +/// output = [[0, 3] +/// [3, 2] +/// [1, 3]] +/// +/// Arguments: +/// * scope: A Scope object +/// * input: Any shape of Tensor contains with int or float type. +/// * boundaries: A sorted list of floats gives the boundary of the buckets. +/// +/// Returns: +/// * `Output`: Same shape with 'input', each value of input replaced with bucket index. +/// +/// @compatibility(numpy) +/// Equivalent to np.digitize. +/// @end_compatibility +class Bucketize { + public: + Bucketize(const ::tensorflow::Scope& scope, ::tensorflow::Input input, const + gtl::ArraySlice& boundaries); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Cast x of type SrcT to y of DstT. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The y tensor. +class Cast { + public: + /// Optional attribute setters for Cast + struct Attrs { + /// Defaults to false + TF_MUST_USE_RESULT Attrs Truncate(bool x) { + Attrs ret = *this; + ret.Truncate_ = x; + return ret; + } + + bool Truncate_ = false; + }; + Cast(const ::tensorflow::Scope& scope, ::tensorflow::Input x, DataType DstT); + Cast(const ::tensorflow::Scope& scope, ::tensorflow::Input x, DataType DstT, + const Cast::Attrs& attrs); + operator ::tensorflow::Output() const { return y; } + operator ::tensorflow::Input() const { return y; } + ::tensorflow::Node* node() const { return y.node(); } + + static Attrs Truncate(bool x) { + return Attrs().Truncate(x); + } + + Operation operation; + ::tensorflow::Output y; +}; + +/// Returns element-wise smallest integer not less than x. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The y tensor. +class Ceil { + public: + Ceil(const ::tensorflow::Scope& scope, ::tensorflow::Input x); + operator ::tensorflow::Output() const { return y; } + operator ::tensorflow::Input() const { return y; } + ::tensorflow::Node* node() const { return y.node(); } + + Operation operation; + ::tensorflow::Output y; +}; + +/// Clips tensor values to a specified min and max. +/// +/// Given a tensor `t`, this operation returns a tensor of the same type and +/// shape as `t` with its values clipped to `clip_value_min` and `clip_value_max`. +/// Any values less than `clip_value_min` are set to `clip_value_min`. Any values +/// greater than `clip_value_max` are set to `clip_value_max`. +/// +/// Arguments: +/// * scope: A Scope object +/// * t: A `Tensor`. +/// * clip_value_min: A 0-D (scalar) `Tensor`, or a `Tensor` with the same shape +/// as `t`. The minimum value to clip by. +/// * clip_value_max: A 0-D (scalar) `Tensor`, or a `Tensor` with the same shape +/// as `t`. The maximum value to clip by. +/// +/// Returns: +/// * `Output`: A clipped `Tensor` with the same shape as input 't'. +class ClipByValue { + public: + ClipByValue(const ::tensorflow::Scope& scope, ::tensorflow::Input t, + ::tensorflow::Input clip_value_min, ::tensorflow::Input + clip_value_max); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Compare values of `input` to `threshold` and pack resulting bits into a `uint8`. +/// +/// Each comparison returns a boolean `true` (if `input_value > threshold`) +/// or and `false` otherwise. +/// +/// This operation is useful for Locality-Sensitive-Hashing (LSH) and other +/// algorithms that use hashing approximations of cosine and `L2` distances; +/// codes can be generated from an input via: +/// +/// ```python +/// codebook_size = 50 +/// codebook_bits = codebook_size * 32 +/// codebook = tf.get_variable('codebook', [x.shape[-1].value, codebook_bits], +/// dtype=x.dtype, +/// initializer=tf.orthogonal_initializer()) +/// codes = compare_and_threshold(tf.matmul(x, codebook), threshold=0.) +/// codes = tf.bitcast(codes, tf.int32) # go from uint8 to int32 +/// # now codes has shape x.shape[:-1] + [codebook_size] +/// ``` +/// +/// **NOTE**: Currently, the innermost dimension of the tensor must be divisible +/// by 8. +/// +/// Given an `input` shaped `[s0, s1, ..., s_n]`, the output is +/// a `uint8` tensor shaped `[s0, s1, ..., s_n / 8]`. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: Values to compare against `threshold` and bitpack. +/// * threshold: Threshold to compare against. +/// +/// Returns: +/// * `Output`: The bitpacked comparisons. +class CompareAndBitpack { + public: + CompareAndBitpack(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input threshold); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Converts two real numbers to a complex number. +/// +/// Given a tensor `real` representing the real part of a complex number, and a +/// tensor `imag` representing the imaginary part of a complex number, this +/// operation returns complex numbers elementwise of the form \\(a + bj\\), where +/// *a* represents the `real` part and *b* represents the `imag` part. +/// +/// The input tensors `real` and `imag` must have the same shape. +/// +/// For example: +/// +/// ``` +/// # tensor 'real' is [2.25, 3.25] +/// # tensor `imag` is [4.75, 5.75] +/// tf.complex(real, imag) ==> [[2.25 + 4.75j], [3.25 + 5.75j]] +/// ``` +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The out tensor. +class Complex { + public: + /// Optional attribute setters for Complex + struct Attrs { + /// Defaults to DT_COMPLEX64 + TF_MUST_USE_RESULT Attrs Tout(DataType x) { + Attrs ret = *this; + ret.Tout_ = x; + return ret; + } + + DataType Tout_ = DT_COMPLEX64; + }; + Complex(const ::tensorflow::Scope& scope, ::tensorflow::Input real, + ::tensorflow::Input imag); + Complex(const ::tensorflow::Scope& scope, ::tensorflow::Input real, + ::tensorflow::Input imag, const Complex::Attrs& attrs); + operator ::tensorflow::Output() const { return out; } + operator ::tensorflow::Input() const { return out; } + ::tensorflow::Node* node() const { return out.node(); } + + static Attrs Tout(DataType x) { + return Attrs().Tout(x); + } + + Operation operation; + ::tensorflow::Output out; +}; + +/// Computes the complex absolute value of a tensor. +/// +/// Given a tensor `x` of complex numbers, this operation returns a tensor of type +/// `float` or `double` that is the absolute value of each element in `x`. All +/// elements in `x` must be complex numbers of the form \\(a + bj\\). The absolute +/// value is computed as \\( \sqrt{a^2 + b^2}\\). +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The y tensor. +class ComplexAbs { + public: + /// Optional attribute setters for ComplexAbs + struct Attrs { + /// Defaults to DT_FLOAT + TF_MUST_USE_RESULT Attrs Tout(DataType x) { + Attrs ret = *this; + ret.Tout_ = x; + return ret; + } + + DataType Tout_ = DT_FLOAT; + }; + ComplexAbs(const ::tensorflow::Scope& scope, ::tensorflow::Input x); + ComplexAbs(const ::tensorflow::Scope& scope, ::tensorflow::Input x, const + ComplexAbs::Attrs& attrs); + operator ::tensorflow::Output() const { return y; } + operator ::tensorflow::Input() const { return y; } + ::tensorflow::Node* node() const { return y.node(); } + + static Attrs Tout(DataType x) { + return Attrs().Tout(x); + } + + Operation operation; + ::tensorflow::Output y; +}; + +/// Returns the complex conjugate of a complex number. +/// +/// Given a tensor `input` of complex numbers, this operation returns a tensor of +/// complex numbers that are the complex conjugate of each element in `input`. The +/// complex numbers in `input` must be of the form \\(a + bj\\), where *a* is the +/// real part and *b* is the imaginary part. +/// +/// The complex conjugate returned by this operation is of the form \\(a - bj\\). +/// +/// For example: +/// +/// ``` +/// # tensor 'input' is [-2.25 + 4.75j, 3.25 + 5.75j] +/// tf.conj(input) ==> [-2.25 - 4.75j, 3.25 - 5.75j] +/// ``` +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The output tensor. +class Conj { + public: + Conj(const ::tensorflow::Scope& scope, ::tensorflow::Input input); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Computes cos of x element-wise. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The y tensor. +class Cos { + public: + Cos(const ::tensorflow::Scope& scope, ::tensorflow::Input x); + operator ::tensorflow::Output() const { return y; } + operator ::tensorflow::Input() const { return y; } + ::tensorflow::Node* node() const { return y.node(); } + + Operation operation; + ::tensorflow::Output y; +}; + +/// Computes hyperbolic cosine of x element-wise. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The y tensor. +class Cosh { + public: + Cosh(const ::tensorflow::Scope& scope, ::tensorflow::Input x); + operator ::tensorflow::Output() const { return y; } + operator ::tensorflow::Input() const { return y; } + ::tensorflow::Node* node() const { return y.node(); } + + Operation operation; + ::tensorflow::Output y; +}; + +/// Compute the pairwise cross product. +/// +/// `a` and `b` must be the same shape; they can either be simple 3-element vectors, +/// or any shape where the innermost dimension is 3. In the latter case, each pair +/// of corresponding 3-element vectors is cross-multiplied independently. +/// +/// Arguments: +/// * scope: A Scope object +/// * a: A tensor containing 3-element vectors. +/// * b: Another tensor, of same type and shape as `a`. +/// +/// Returns: +/// * `Output`: Pairwise cross product of the vectors in `a` and `b`. +class Cross { + public: + Cross(const ::tensorflow::Scope& scope, ::tensorflow::Input a, + ::tensorflow::Input b); + operator ::tensorflow::Output() const { return product; } + operator ::tensorflow::Input() const { return product; } + ::tensorflow::Node* node() const { return product.node(); } + + Operation operation; + ::tensorflow::Output product; +}; + +/// Compute the cumulative product of the tensor `x` along `axis`. +/// +/// By default, this op performs an inclusive cumprod, which means that the first +/// element of the input is identical to the first element of the output: +/// +/// ```python +/// tf.cumprod([a, b, c]) # => [a, a * b, a * b * c] +/// ``` +/// +/// By setting the `exclusive` kwarg to `True`, an exclusive cumprod is +/// performed instead: +/// +/// ```python +/// tf.cumprod([a, b, c], exclusive=True) # => [1, a, a * b] +/// ``` +/// +/// By setting the `reverse` kwarg to `True`, the cumprod is performed in the +/// opposite direction: +/// +/// ```python +/// tf.cumprod([a, b, c], reverse=True) # => [a * b * c, b * c, c] +/// ``` +/// +/// This is more efficient than using separate `tf.reverse` ops. +/// +/// The `reverse` and `exclusive` kwargs can also be combined: +/// +/// ```python +/// tf.cumprod([a, b, c], exclusive=True, reverse=True) # => [b * c, c, 1] +/// ``` +/// +/// Arguments: +/// * scope: A Scope object +/// * x: A `Tensor`. Must be one of the following types: `float32`, `float64`, +/// `int64`, `int32`, `uint8`, `uint16`, `int16`, `int8`, `complex64`, +/// `complex128`, `qint8`, `quint8`, `qint32`, `half`. +/// * axis: A `Tensor` of type `int32` (default: 0). Must be in the range +/// `[-rank(x), rank(x))`. +/// +/// Optional attributes (see `Attrs`): +/// * exclusive: If `True`, perform exclusive cumprod. +/// * reverse: A `bool` (default: False). +/// +/// Returns: +/// * `Output`: The out tensor. +class Cumprod { + public: + /// Optional attribute setters for Cumprod + struct Attrs { + /// If `True`, perform exclusive cumprod. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs Exclusive(bool x) { + Attrs ret = *this; + ret.exclusive_ = x; + return ret; + } + + /// A `bool` (default: False). + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs Reverse(bool x) { + Attrs ret = *this; + ret.reverse_ = x; + return ret; + } + + bool exclusive_ = false; + bool reverse_ = false; + }; + Cumprod(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input axis); + Cumprod(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input axis, const Cumprod::Attrs& attrs); + operator ::tensorflow::Output() const { return out; } + operator ::tensorflow::Input() const { return out; } + ::tensorflow::Node* node() const { return out.node(); } + + static Attrs Exclusive(bool x) { + return Attrs().Exclusive(x); + } + static Attrs Reverse(bool x) { + return Attrs().Reverse(x); + } + + Operation operation; + ::tensorflow::Output out; +}; + +/// Compute the cumulative sum of the tensor `x` along `axis`. +/// +/// By default, this op performs an inclusive cumsum, which means that the first +/// element of the input is identical to the first element of the output: +/// +/// ```python +/// tf.cumsum([a, b, c]) # => [a, a + b, a + b + c] +/// ``` +/// +/// By setting the `exclusive` kwarg to `True`, an exclusive cumsum is +/// performed instead: +/// +/// ```python +/// tf.cumsum([a, b, c], exclusive=True) # => [0, a, a + b] +/// ``` +/// +/// By setting the `reverse` kwarg to `True`, the cumsum is performed in the +/// opposite direction: +/// +/// ```python +/// tf.cumsum([a, b, c], reverse=True) # => [a + b + c, b + c, c] +/// ``` +/// +/// This is more efficient than using separate `tf.reverse` ops. +/// +/// The `reverse` and `exclusive` kwargs can also be combined: +/// +/// ```python +/// tf.cumsum([a, b, c], exclusive=True, reverse=True) # => [b + c, c, 0] +/// ``` +/// +/// Arguments: +/// * scope: A Scope object +/// * x: A `Tensor`. Must be one of the following types: `float32`, `float64`, +/// `int64`, `int32`, `uint8`, `uint16`, `int16`, `int8`, `complex64`, +/// `complex128`, `qint8`, `quint8`, `qint32`, `half`. +/// * axis: A `Tensor` of type `int32` (default: 0). Must be in the range +/// `[-rank(x), rank(x))`. +/// +/// Optional attributes (see `Attrs`): +/// * exclusive: If `True`, perform exclusive cumsum. +/// * reverse: A `bool` (default: False). +/// +/// Returns: +/// * `Output`: The out tensor. +class Cumsum { + public: + /// Optional attribute setters for Cumsum + struct Attrs { + /// If `True`, perform exclusive cumsum. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs Exclusive(bool x) { + Attrs ret = *this; + ret.exclusive_ = x; + return ret; + } + + /// A `bool` (default: False). + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs Reverse(bool x) { + Attrs ret = *this; + ret.reverse_ = x; + return ret; + } + + bool exclusive_ = false; + bool reverse_ = false; + }; + Cumsum(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input axis); + Cumsum(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input axis, const Cumsum::Attrs& attrs); + operator ::tensorflow::Output() const { return out; } + operator ::tensorflow::Input() const { return out; } + ::tensorflow::Node* node() const { return out.node(); } + + static Attrs Exclusive(bool x) { + return Attrs().Exclusive(x); + } + static Attrs Reverse(bool x) { + return Attrs().Reverse(x); + } + + Operation operation; + ::tensorflow::Output out; +}; + +/// Computes Psi, the derivative of Lgamma (the log of the absolute value of +/// +/// `Gamma(x)`), element-wise. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The y tensor. +class Digamma { + public: + Digamma(const ::tensorflow::Scope& scope, ::tensorflow::Input x); + operator ::tensorflow::Output() const { return y; } + operator ::tensorflow::Input() const { return y; } + ::tensorflow::Node* node() const { return y.node(); } + + Operation operation; + ::tensorflow::Output y; +}; + +/// Returns x / y element-wise. +/// +/// *NOTE*: `Div` supports broadcasting. More about broadcasting +/// [here](http://docs.scipy.org/doc/numpy/user/basics.broadcasting.html) +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The z tensor. +class Div { + public: + Div(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input y); + operator ::tensorflow::Output() const { return z; } + operator ::tensorflow::Input() const { return z; } + ::tensorflow::Node* node() const { return z.node(); } + + Operation operation; + ::tensorflow::Output z; +}; + +/// Returns 0 if the denominator is zero. +/// +/// +/// *NOTE*: `DivNoNan` supports broadcasting. More about broadcasting +/// [here](http://docs.scipy.org/doc/numpy/user/basics.broadcasting.html) +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The z tensor. +class DivNoNan { + public: + DivNoNan(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input y); + operator ::tensorflow::Output() const { return z; } + operator ::tensorflow::Input() const { return z; } + ::tensorflow::Node* node() const { return z.node(); } + + Operation operation; + ::tensorflow::Output z; +}; + +/// Returns the truth value of (x == y) element-wise. +/// +/// *NOTE*: `Equal` supports broadcasting. More about broadcasting +/// [here](http://docs.scipy.org/doc/numpy/user/basics.broadcasting.html) +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The z tensor. +class Equal { + public: + Equal(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input y); + operator ::tensorflow::Output() const { return z; } + operator ::tensorflow::Input() const { return z; } + ::tensorflow::Node* node() const { return z.node(); } + + Operation operation; + ::tensorflow::Output z; +}; + +/// Computes the Gauss error function of `x` element-wise. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The y tensor. +class Erf { + public: + Erf(const ::tensorflow::Scope& scope, ::tensorflow::Input x); + operator ::tensorflow::Output() const { return y; } + operator ::tensorflow::Input() const { return y; } + ::tensorflow::Node* node() const { return y.node(); } + + Operation operation; + ::tensorflow::Output y; +}; + +/// Computes the complementary error function of `x` element-wise. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The y tensor. +class Erfc { + public: + Erfc(const ::tensorflow::Scope& scope, ::tensorflow::Input x); + operator ::tensorflow::Output() const { return y; } + operator ::tensorflow::Input() const { return y; } + ::tensorflow::Node* node() const { return y.node(); } + + Operation operation; + ::tensorflow::Output y; +}; + +/// Computes exponential of x element-wise. \\(y = e^x\\). +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The y tensor. +class Exp { + public: + Exp(const ::tensorflow::Scope& scope, ::tensorflow::Input x); + operator ::tensorflow::Output() const { return y; } + operator ::tensorflow::Input() const { return y; } + ::tensorflow::Node* node() const { return y.node(); } + + Operation operation; + ::tensorflow::Output y; +}; + +/// Computes exponential of x - 1 element-wise. +/// +/// I.e., \\(y = (\exp x) - 1\\). +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The y tensor. +class Expm1 { + public: + Expm1(const ::tensorflow::Scope& scope, ::tensorflow::Input x); + operator ::tensorflow::Output() const { return y; } + operator ::tensorflow::Input() const { return y; } + ::tensorflow::Node* node() const { return y.node(); } + + Operation operation; + ::tensorflow::Output y; +}; + +/// Returns element-wise largest integer not greater than x. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The y tensor. +class Floor { + public: + Floor(const ::tensorflow::Scope& scope, ::tensorflow::Input x); + operator ::tensorflow::Output() const { return y; } + operator ::tensorflow::Input() const { return y; } + ::tensorflow::Node* node() const { return y.node(); } + + Operation operation; + ::tensorflow::Output y; +}; + +/// Returns x // y element-wise. +/// +/// *NOTE*: `FloorDiv` supports broadcasting. More about broadcasting +/// [here](http://docs.scipy.org/doc/numpy/user/basics.broadcasting.html) +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The z tensor. +class FloorDiv { + public: + FloorDiv(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input y); + operator ::tensorflow::Output() const { return z; } + operator ::tensorflow::Input() const { return z; } + ::tensorflow::Node* node() const { return z.node(); } + + Operation operation; + ::tensorflow::Output z; +}; + +/// Returns element-wise remainder of division. When `x < 0` xor `y < 0` is +/// +/// true, this follows Python semantics in that the result here is consistent +/// with a flooring divide. E.g. `floor(x / y) * y + mod(x, y) = x`. +/// +/// *NOTE*: `FloorMod` supports broadcasting. More about broadcasting +/// [here](http://docs.scipy.org/doc/numpy/user/basics.broadcasting.html) +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The z tensor. +class FloorMod { + public: + FloorMod(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input y); + operator ::tensorflow::Output() const { return z; } + operator ::tensorflow::Input() const { return z; } + ::tensorflow::Node* node() const { return z.node(); } + + Operation operation; + ::tensorflow::Output z; +}; + +/// Returns the truth value of (x > y) element-wise. +/// +/// *NOTE*: `Greater` supports broadcasting. More about broadcasting +/// [here](http://docs.scipy.org/doc/numpy/user/basics.broadcasting.html) +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The z tensor. +class Greater { + public: + Greater(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input y); + operator ::tensorflow::Output() const { return z; } + operator ::tensorflow::Input() const { return z; } + ::tensorflow::Node* node() const { return z.node(); } + + Operation operation; + ::tensorflow::Output z; +}; + +/// Returns the truth value of (x >= y) element-wise. +/// +/// *NOTE*: `GreaterEqual` supports broadcasting. More about broadcasting +/// [here](http://docs.scipy.org/doc/numpy/user/basics.broadcasting.html) +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The z tensor. +class GreaterEqual { + public: + GreaterEqual(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input y); + operator ::tensorflow::Output() const { return z; } + operator ::tensorflow::Input() const { return z; } + ::tensorflow::Node* node() const { return z.node(); } + + Operation operation; + ::tensorflow::Output z; +}; + +/// Return histogram of values. +/// +/// Given the tensor `values`, this operation returns a rank 1 histogram counting +/// the number of entries in `values` that fall into every bin. The bins are +/// equal width and determined by the arguments `value_range` and `nbins`. +/// +/// ```python +/// # Bins will be: (-inf, 1), [1, 2), [2, 3), [3, 4), [4, inf) +/// nbins = 5 +/// value_range = [0.0, 5.0] +/// new_values = [-1.0, 0.0, 1.5, 2.0, 5.0, 15] +/// +/// with tf.get_default_session() as sess: +/// hist = tf.histogram_fixed_width(new_values, value_range, nbins=5) +/// variables.global_variables_initializer().run() +/// sess.run(hist) => [2, 1, 1, 0, 2] +/// ``` +/// +/// Arguments: +/// * scope: A Scope object +/// * values: Numeric `Tensor`. +/// * value_range: Shape [2] `Tensor` of same `dtype` as `values`. +/// values <= value_range[0] will be mapped to hist[0], +/// values >= value_range[1] will be mapped to hist[-1]. +/// * nbins: Scalar `int32 Tensor`. Number of histogram bins. +/// +/// Returns: +/// * `Output`: A 1-D `Tensor` holding histogram of values. +class HistogramFixedWidth { + public: + /// Optional attribute setters for HistogramFixedWidth + struct Attrs { + /// Defaults to DT_INT32 + TF_MUST_USE_RESULT Attrs Dtype(DataType x) { + Attrs ret = *this; + ret.dtype_ = x; + return ret; + } + + DataType dtype_ = DT_INT32; + }; + HistogramFixedWidth(const ::tensorflow::Scope& scope, ::tensorflow::Input + values, ::tensorflow::Input value_range, + ::tensorflow::Input nbins); + HistogramFixedWidth(const ::tensorflow::Scope& scope, ::tensorflow::Input + values, ::tensorflow::Input value_range, + ::tensorflow::Input nbins, const + HistogramFixedWidth::Attrs& attrs); + operator ::tensorflow::Output() const { return out; } + operator ::tensorflow::Input() const { return out; } + ::tensorflow::Node* node() const { return out.node(); } + + static Attrs Dtype(DataType x) { + return Attrs().Dtype(x); + } + + Operation operation; + ::tensorflow::Output out; +}; + +/// Compute the lower regularized incomplete Gamma function `P(a, x)`. +/// +/// The lower regularized incomplete Gamma function is defined as: +/// +/// +/// \\(P(a, x) = gamma(a, x) / Gamma(a) = 1 - Q(a, x)\\) +/// +/// where +/// +/// \\(gamma(a, x) = \\int_{0}^{x} t^{a-1} exp(-t) dt\\) +/// +/// is the lower incomplete Gamma function. +/// +/// Note, above `Q(a, x)` (`Igammac`) is the upper regularized complete +/// Gamma function. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The z tensor. +class Igamma { + public: + Igamma(const ::tensorflow::Scope& scope, ::tensorflow::Input a, + ::tensorflow::Input x); + operator ::tensorflow::Output() const { return z; } + operator ::tensorflow::Input() const { return z; } + ::tensorflow::Node* node() const { return z.node(); } + + Operation operation; + ::tensorflow::Output z; +}; + +/// Compute the upper regularized incomplete Gamma function `Q(a, x)`. +/// +/// The upper regularized incomplete Gamma function is defined as: +/// +/// \\(Q(a, x) = Gamma(a, x) / Gamma(a) = 1 - P(a, x)\\) +/// +/// where +/// +/// \\(Gamma(a, x) = int_{x}^{\infty} t^{a-1} exp(-t) dt\\) +/// +/// is the upper incomplete Gama function. +/// +/// Note, above `P(a, x)` (`Igamma`) is the lower regularized complete +/// Gamma function. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The z tensor. +class Igammac { + public: + Igammac(const ::tensorflow::Scope& scope, ::tensorflow::Input a, + ::tensorflow::Input x); + operator ::tensorflow::Output() const { return z; } + operator ::tensorflow::Input() const { return z; } + ::tensorflow::Node* node() const { return z.node(); } + + Operation operation; + ::tensorflow::Output z; +}; + +/// Returns the imaginary part of a complex number. +/// +/// Given a tensor `input` of complex numbers, this operation returns a tensor of +/// type `float` that is the imaginary part of each element in `input`. All +/// elements in `input` must be complex numbers of the form \\(a + bj\\), where *a* +/// is the real part and *b* is the imaginary part returned by this operation. +/// +/// For example: +/// +/// ``` +/// # tensor 'input' is [-2.25 + 4.75j, 3.25 + 5.75j] +/// tf.imag(input) ==> [4.75, 5.75] +/// ``` +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The output tensor. +class Imag { + public: + /// Optional attribute setters for Imag + struct Attrs { + /// Defaults to DT_FLOAT + TF_MUST_USE_RESULT Attrs Tout(DataType x) { + Attrs ret = *this; + ret.Tout_ = x; + return ret; + } + + DataType Tout_ = DT_FLOAT; + }; + Imag(const ::tensorflow::Scope& scope, ::tensorflow::Input input); + Imag(const ::tensorflow::Scope& scope, ::tensorflow::Input input, const + Imag::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs Tout(DataType x) { + return Attrs().Tout(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Computes the reciprocal of x element-wise. +/// +/// I.e., \\(y = 1 / x\\). +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The y tensor. +class Inv { + public: + Inv(const ::tensorflow::Scope& scope, ::tensorflow::Input x); + operator ::tensorflow::Output() const { return y; } + operator ::tensorflow::Input() const { return y; } + ::tensorflow::Node* node() const { return y.node(); } + + Operation operation; + ::tensorflow::Output y; +}; + +/// Returns which elements of x are finite. +/// +/// @compatibility(numpy) +/// Equivalent to np.isfinite +/// @end_compatibility +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The y tensor. +class IsFinite { + public: + IsFinite(const ::tensorflow::Scope& scope, ::tensorflow::Input x); + operator ::tensorflow::Output() const { return y; } + operator ::tensorflow::Input() const { return y; } + ::tensorflow::Node* node() const { return y.node(); } + + Operation operation; + ::tensorflow::Output y; +}; + +/// Returns which elements of x are Inf. +/// +/// @compatibility(numpy) +/// Equivalent to np.isinf +/// @end_compatibility +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The y tensor. +class IsInf { + public: + IsInf(const ::tensorflow::Scope& scope, ::tensorflow::Input x); + operator ::tensorflow::Output() const { return y; } + operator ::tensorflow::Input() const { return y; } + ::tensorflow::Node* node() const { return y.node(); } + + Operation operation; + ::tensorflow::Output y; +}; + +/// Returns which elements of x are NaN. +/// +/// @compatibility(numpy) +/// Equivalent to np.isnan +/// @end_compatibility +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The y tensor. +class IsNan { + public: + IsNan(const ::tensorflow::Scope& scope, ::tensorflow::Input x); + operator ::tensorflow::Output() const { return y; } + operator ::tensorflow::Input() const { return y; } + ::tensorflow::Node* node() const { return y.node(); } + + Operation operation; + ::tensorflow::Output y; +}; + +/// Returns the truth value of (x < y) element-wise. +/// +/// *NOTE*: `Less` supports broadcasting. More about broadcasting +/// [here](http://docs.scipy.org/doc/numpy/user/basics.broadcasting.html) +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The z tensor. +class Less { + public: + Less(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input y); + operator ::tensorflow::Output() const { return z; } + operator ::tensorflow::Input() const { return z; } + ::tensorflow::Node* node() const { return z.node(); } + + Operation operation; + ::tensorflow::Output z; +}; + +/// Returns the truth value of (x <= y) element-wise. +/// +/// *NOTE*: `LessEqual` supports broadcasting. More about broadcasting +/// [here](http://docs.scipy.org/doc/numpy/user/basics.broadcasting.html) +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The z tensor. +class LessEqual { + public: + LessEqual(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input y); + operator ::tensorflow::Output() const { return z; } + operator ::tensorflow::Input() const { return z; } + ::tensorflow::Node* node() const { return z.node(); } + + Operation operation; + ::tensorflow::Output z; +}; + +/// Computes the log of the absolute value of `Gamma(x)` element-wise. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The y tensor. +class Lgamma { + public: + Lgamma(const ::tensorflow::Scope& scope, ::tensorflow::Input x); + operator ::tensorflow::Output() const { return y; } + operator ::tensorflow::Input() const { return y; } + ::tensorflow::Node* node() const { return y.node(); } + + Operation operation; + ::tensorflow::Output y; +}; + +/// Generates values in an interval. +/// +/// A sequence of `num` evenly-spaced values are generated beginning at `start`. +/// If `num > 1`, the values in the sequence increase by `stop - start / num - 1`, +/// so that the last one is exactly `stop`. +/// +/// For example: +/// +/// ``` +/// tf.linspace(10.0, 12.0, 3, name="linspace") => [ 10.0 11.0 12.0] +/// ``` +/// +/// Arguments: +/// * scope: A Scope object +/// * start: 0-D tensor. First entry in the range. +/// * stop: 0-D tensor. Last entry in the range. +/// * num: 0-D tensor. Number of values to generate. +/// +/// Returns: +/// * `Output`: 1-D. The generated values. +class LinSpace { + public: + LinSpace(const ::tensorflow::Scope& scope, ::tensorflow::Input start, + ::tensorflow::Input stop, ::tensorflow::Input num); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Computes natural logarithm of x element-wise. +/// +/// I.e., \\(y = \log_e x\\). +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The y tensor. +class Log { + public: + Log(const ::tensorflow::Scope& scope, ::tensorflow::Input x); + operator ::tensorflow::Output() const { return y; } + operator ::tensorflow::Input() const { return y; } + ::tensorflow::Node* node() const { return y.node(); } + + Operation operation; + ::tensorflow::Output y; +}; + +/// Computes natural logarithm of (1 + x) element-wise. +/// +/// I.e., \\(y = \log_e (1 + x)\\). +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The y tensor. +class Log1p { + public: + Log1p(const ::tensorflow::Scope& scope, ::tensorflow::Input x); + operator ::tensorflow::Output() const { return y; } + operator ::tensorflow::Input() const { return y; } + ::tensorflow::Node* node() const { return y.node(); } + + Operation operation; + ::tensorflow::Output y; +}; + +/// Returns the truth value of x AND y element-wise. +/// +/// *NOTE*: `LogicalAnd` supports broadcasting. More about broadcasting +/// [here](http://docs.scipy.org/doc/numpy/user/basics.broadcasting.html) +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The z tensor. +class LogicalAnd { + public: + LogicalAnd(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input y); + operator ::tensorflow::Output() const { return z; } + operator ::tensorflow::Input() const { return z; } + ::tensorflow::Node* node() const { return z.node(); } + + Operation operation; + ::tensorflow::Output z; +}; + +/// Returns the truth value of NOT x element-wise. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The y tensor. +class LogicalNot { + public: + LogicalNot(const ::tensorflow::Scope& scope, ::tensorflow::Input x); + operator ::tensorflow::Output() const { return y; } + operator ::tensorflow::Input() const { return y; } + ::tensorflow::Node* node() const { return y.node(); } + + Operation operation; + ::tensorflow::Output y; +}; + +/// Returns the truth value of x OR y element-wise. +/// +/// *NOTE*: `LogicalOr` supports broadcasting. More about broadcasting +/// [here](http://docs.scipy.org/doc/numpy/user/basics.broadcasting.html) +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The z tensor. +class LogicalOr { + public: + LogicalOr(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input y); + operator ::tensorflow::Output() const { return z; } + operator ::tensorflow::Input() const { return z; } + ::tensorflow::Node* node() const { return z.node(); } + + Operation operation; + ::tensorflow::Output z; +}; + +/// Multiply the matrix "a" by the matrix "b". +/// +/// The inputs must be two-dimensional matrices and the inner dimension of +/// "a" (after being transposed if transpose_a is true) must match the +/// outer dimension of "b" (after being transposed if transposed_b is +/// true). +/// +/// *Note*: The default kernel implementation for MatMul on GPUs uses +/// cublas. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Optional attributes (see `Attrs`): +/// * transpose_a: If true, "a" is transposed before multiplication. +/// * transpose_b: If true, "b" is transposed before multiplication. +/// +/// Returns: +/// * `Output`: The product tensor. +class MatMul { + public: + /// Optional attribute setters for MatMul + struct Attrs { + /// If true, "a" is transposed before multiplication. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs TransposeA(bool x) { + Attrs ret = *this; + ret.transpose_a_ = x; + return ret; + } + + /// If true, "b" is transposed before multiplication. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs TransposeB(bool x) { + Attrs ret = *this; + ret.transpose_b_ = x; + return ret; + } + + bool transpose_a_ = false; + bool transpose_b_ = false; + }; + MatMul(const ::tensorflow::Scope& scope, ::tensorflow::Input a, + ::tensorflow::Input b); + MatMul(const ::tensorflow::Scope& scope, ::tensorflow::Input a, + ::tensorflow::Input b, const MatMul::Attrs& attrs); + operator ::tensorflow::Output() const { return product; } + operator ::tensorflow::Input() const { return product; } + ::tensorflow::Node* node() const { return product.node(); } + + static Attrs TransposeA(bool x) { + return Attrs().TransposeA(x); + } + static Attrs TransposeB(bool x) { + return Attrs().TransposeB(x); + } + + Operation operation; + ::tensorflow::Output product; +}; + +/// Computes the maximum of elements across dimensions of a tensor. +/// +/// Reduces `input` along the dimensions given in `axis`. Unless +/// `keep_dims` is true, the rank of the tensor is reduced by 1 for each entry in +/// `axis`. If `keep_dims` is true, the reduced dimensions are +/// retained with length 1. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: The tensor to reduce. +/// * axis: The dimensions to reduce. Must be in the range +/// `[-rank(input), rank(input))`. +/// +/// Optional attributes (see `Attrs`): +/// * keep_dims: If true, retain reduced dimensions with length 1. +/// +/// Returns: +/// * `Output`: The reduced tensor. +/// +/// Aliases: +/// * ReduceMax +class Max { + public: + /// Optional attribute setters for Max + struct Attrs { + /// If true, retain reduced dimensions with length 1. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs KeepDims(bool x) { + Attrs ret = *this; + ret.keep_dims_ = x; + return ret; + } + + bool keep_dims_ = false; + }; + Max(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input axis); + Max(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input axis, const Max::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs KeepDims(bool x) { + return Attrs().KeepDims(x); + } + + Operation operation; + ::tensorflow::Output output; +}; +typedef Max ReduceMax; + +/// Returns the max of x and y (i.e. x > y ? x : y) element-wise. +/// +/// *NOTE*: `Maximum` supports broadcasting. More about broadcasting +/// [here](http://docs.scipy.org/doc/numpy/user/basics.broadcasting.html) +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The z tensor. +class Maximum { + public: + Maximum(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input y); + operator ::tensorflow::Output() const { return z; } + operator ::tensorflow::Input() const { return z; } + ::tensorflow::Node* node() const { return z.node(); } + + Operation operation; + ::tensorflow::Output z; +}; + +/// Computes the mean of elements across dimensions of a tensor. +/// +/// Reduces `input` along the dimensions given in `axis`. Unless +/// `keep_dims` is true, the rank of the tensor is reduced by 1 for each entry in +/// `axis`. If `keep_dims` is true, the reduced dimensions are +/// retained with length 1. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: The tensor to reduce. +/// * axis: The dimensions to reduce. Must be in the range +/// `[-rank(input), rank(input))`. +/// +/// Optional attributes (see `Attrs`): +/// * keep_dims: If true, retain reduced dimensions with length 1. +/// +/// Returns: +/// * `Output`: The reduced tensor. +/// +/// Aliases: +/// * ReduceMean +class Mean { + public: + /// Optional attribute setters for Mean + struct Attrs { + /// If true, retain reduced dimensions with length 1. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs KeepDims(bool x) { + Attrs ret = *this; + ret.keep_dims_ = x; + return ret; + } + + bool keep_dims_ = false; + }; + Mean(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input axis); + Mean(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input axis, const Mean::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs KeepDims(bool x) { + return Attrs().KeepDims(x); + } + + Operation operation; + ::tensorflow::Output output; +}; +typedef Mean ReduceMean; + +/// Computes the minimum of elements across dimensions of a tensor. +/// +/// Reduces `input` along the dimensions given in `axis`. Unless +/// `keep_dims` is true, the rank of the tensor is reduced by 1 for each entry in +/// `axis`. If `keep_dims` is true, the reduced dimensions are +/// retained with length 1. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: The tensor to reduce. +/// * axis: The dimensions to reduce. Must be in the range +/// `[-rank(input), rank(input))`. +/// +/// Optional attributes (see `Attrs`): +/// * keep_dims: If true, retain reduced dimensions with length 1. +/// +/// Returns: +/// * `Output`: The reduced tensor. +/// +/// Aliases: +/// * ReduceMin +class Min { + public: + /// Optional attribute setters for Min + struct Attrs { + /// If true, retain reduced dimensions with length 1. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs KeepDims(bool x) { + Attrs ret = *this; + ret.keep_dims_ = x; + return ret; + } + + bool keep_dims_ = false; + }; + Min(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input axis); + Min(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input axis, const Min::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs KeepDims(bool x) { + return Attrs().KeepDims(x); + } + + Operation operation; + ::tensorflow::Output output; +}; +typedef Min ReduceMin; + +/// Returns the min of x and y (i.e. x < y ? x : y) element-wise. +/// +/// *NOTE*: `Minimum` supports broadcasting. More about broadcasting +/// [here](http://docs.scipy.org/doc/numpy/user/basics.broadcasting.html) +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The z tensor. +class Minimum { + public: + Minimum(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input y); + operator ::tensorflow::Output() const { return z; } + operator ::tensorflow::Input() const { return z; } + ::tensorflow::Node* node() const { return z.node(); } + + Operation operation; + ::tensorflow::Output z; +}; + +/// Returns element-wise remainder of division. This emulates C semantics in that +/// +/// the result here is consistent with a truncating divide. E.g. +/// `tf.truncatediv(x, y) * y + truncate_mod(x, y) = x`. +/// +/// *NOTE*: `Mod` supports broadcasting. More about broadcasting +/// [here](http://docs.scipy.org/doc/numpy/user/basics.broadcasting.html) +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The z tensor. +class Mod { + public: + Mod(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input y); + operator ::tensorflow::Output() const { return z; } + operator ::tensorflow::Input() const { return z; } + ::tensorflow::Node* node() const { return z.node(); } + + Operation operation; + ::tensorflow::Output z; +}; + +/// Returns x * y element-wise. +/// +/// *NOTE*: `Multiply` supports broadcasting. More about broadcasting +/// [here](http://docs.scipy.org/doc/numpy/user/basics.broadcasting.html) +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The z tensor. +/// +/// Aliases: +/// * Mul +class Multiply { + public: + Multiply(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input y); + operator ::tensorflow::Output() const { return z; } + operator ::tensorflow::Input() const { return z; } + ::tensorflow::Node* node() const { return z.node(); } + + Operation operation; + ::tensorflow::Output z; +}; +typedef Multiply Mul; + +/// Computes numerical negative value element-wise. +/// +/// I.e., \\(y = -x\\). +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The y tensor. +/// +/// Aliases: +/// * Neg +class Negate { + public: + Negate(const ::tensorflow::Scope& scope, ::tensorflow::Input x); + operator ::tensorflow::Output() const { return y; } + operator ::tensorflow::Input() const { return y; } + ::tensorflow::Node* node() const { return y.node(); } + + Operation operation; + ::tensorflow::Output y; +}; +typedef Negate Neg; + +/// Returns the truth value of (x != y) element-wise. +/// +/// *NOTE*: `NotEqual` supports broadcasting. More about broadcasting +/// [here](http://docs.scipy.org/doc/numpy/user/basics.broadcasting.html) +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The z tensor. +class NotEqual { + public: + NotEqual(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input y); + operator ::tensorflow::Output() const { return z; } + operator ::tensorflow::Input() const { return z; } + ::tensorflow::Node* node() const { return z.node(); } + + Operation operation; + ::tensorflow::Output z; +}; + +/// Compute the polygamma function \\(\psi^{(n)}(x)\\). +/// +/// The polygamma function is defined as: +/// +/// +/// \\(\psi^{(n)}(x) = \frac{d^n}{dx^n} \psi(x)\\) +/// +/// where \\(\psi(x)\\) is the digamma function. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The z tensor. +class Polygamma { + public: + Polygamma(const ::tensorflow::Scope& scope, ::tensorflow::Input a, + ::tensorflow::Input x); + operator ::tensorflow::Output() const { return z; } + operator ::tensorflow::Input() const { return z; } + ::tensorflow::Node* node() const { return z.node(); } + + Operation operation; + ::tensorflow::Output z; +}; + +/// Computes the power of one value to another. +/// +/// Given a tensor `x` and a tensor `y`, this operation computes \\(x^y\\) for +/// corresponding elements in `x` and `y`. For example: +/// +/// ``` +/// # tensor 'x' is [[2, 2]], [3, 3]] +/// # tensor 'y' is [[8, 16], [2, 3]] +/// tf.pow(x, y) ==> [[256, 65536], [9, 27]] +/// ``` +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The z tensor. +class Pow { + public: + Pow(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input y); + operator ::tensorflow::Output() const { return z; } + operator ::tensorflow::Input() const { return z; } + ::tensorflow::Node* node() const { return z.node(); } + + Operation operation; + ::tensorflow::Output z; +}; + +/// Computes the product of elements across dimensions of a tensor. +/// +/// Reduces `input` along the dimensions given in `axis`. Unless +/// `keep_dims` is true, the rank of the tensor is reduced by 1 for each entry in +/// `axis`. If `keep_dims` is true, the reduced dimensions are +/// retained with length 1. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: The tensor to reduce. +/// * axis: The dimensions to reduce. Must be in the range +/// `[-rank(input), rank(input))`. +/// +/// Optional attributes (see `Attrs`): +/// * keep_dims: If true, retain reduced dimensions with length 1. +/// +/// Returns: +/// * `Output`: The reduced tensor. +/// +/// Aliases: +/// * ReduceProd +class Prod { + public: + /// Optional attribute setters for Prod + struct Attrs { + /// If true, retain reduced dimensions with length 1. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs KeepDims(bool x) { + Attrs ret = *this; + ret.keep_dims_ = x; + return ret; + } + + bool keep_dims_ = false; + }; + Prod(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input axis); + Prod(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input axis, const Prod::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs KeepDims(bool x) { + return Attrs().KeepDims(x); + } + + Operation operation; + ::tensorflow::Output output; +}; +typedef Prod ReduceProd; + +/// Convert the quantized 'input' tensor into a lower-precision 'output', using the +/// +/// actual distribution of the values to maximize the usage of the lower bit depth +/// and adjusting the output min and max ranges accordingly. +/// +/// [input_min, input_max] are scalar floats that specify the range for the float +/// interpretation of the 'input' data. For example, if input_min is -1.0f and +/// input_max is 1.0f, and we are dealing with quint16 quantized data, then a 0 +/// value in the 16-bit data should be interpreted as -1.0f, and a 65535 means 1.0f. +/// +/// This operator tries to squeeze as much precision as possible into an output with +/// a lower bit depth by calculating the actual min and max values found in the +/// data. For example, maybe that quint16 input has no values lower than 16,384 and +/// none higher than 49,152. That means only half the range is actually needed, all +/// the float interpretations are between -0.5f and 0.5f, so if we want to compress +/// the data into a quint8 output, we can use that range rather than the theoretical +/// -1.0f to 1.0f that is suggested by the input min and max. +/// +/// In practice, this is most useful for taking output from operations like +/// QuantizedMatMul that can produce higher bit-depth outputs than their inputs and +/// may have large potential output ranges, but in practice have a distribution of +/// input values that only uses a small fraction of the possible range. By feeding +/// that output into this operator, we can reduce it from 32 bits down to 8 with +/// minimal loss of accuracy. +/// +/// Arguments: +/// * scope: A Scope object +/// * input_min: The float value that the minimum quantized input value represents. +/// * input_max: The float value that the maximum quantized input value represents. +/// * out_type: The type of the output. Should be a lower bit depth than Tinput. +/// +/// Returns: +/// * `Output` output +/// * `Output` output_min: The float value that the minimum quantized output value represents. +/// * `Output` output_max: The float value that the maximum quantized output value represents. +class QuantizeDownAndShrinkRange { + public: + QuantizeDownAndShrinkRange(const ::tensorflow::Scope& scope, + ::tensorflow::Input input, ::tensorflow::Input + input_min, ::tensorflow::Input input_max, DataType + out_type); + + Operation operation; + ::tensorflow::Output output; + ::tensorflow::Output output_min; + ::tensorflow::Output output_max; +}; + +/// Returns x + y element-wise, working on quantized buffers. +/// +/// Arguments: +/// * scope: A Scope object +/// * min_x: The float value that the lowest quantized `x` value represents. +/// * max_x: The float value that the highest quantized `x` value represents. +/// * min_y: The float value that the lowest quantized `y` value represents. +/// * max_y: The float value that the highest quantized `y` value represents. +/// +/// Returns: +/// * `Output` z +/// * `Output` min_z: The float value that the lowest quantized output value represents. +/// * `Output` max_z: The float value that the highest quantized output value represents. +/// +/// *NOTE*: `QuantizedAdd` supports limited forms of broadcasting. More about +/// broadcasting [here](http://docs.scipy.org/doc/numpy/user/basics.broadcasting.html) +class QuantizedAdd { + public: + /// Optional attribute setters for QuantizedAdd + struct Attrs { + /// Defaults to DT_QINT32 + TF_MUST_USE_RESULT Attrs Toutput(DataType x) { + Attrs ret = *this; + ret.Toutput_ = x; + return ret; + } + + DataType Toutput_ = DT_QINT32; + }; + QuantizedAdd(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input y, ::tensorflow::Input min_x, + ::tensorflow::Input max_x, ::tensorflow::Input min_y, + ::tensorflow::Input max_y); + QuantizedAdd(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input y, ::tensorflow::Input min_x, + ::tensorflow::Input max_x, ::tensorflow::Input min_y, + ::tensorflow::Input max_y, const QuantizedAdd::Attrs& attrs); + + static Attrs Toutput(DataType x) { + return Attrs().Toutput(x); + } + + Operation operation; + ::tensorflow::Output z; + ::tensorflow::Output min_z; + ::tensorflow::Output max_z; +}; + +/// Perform a quantized matrix multiplication of `a` by the matrix `b`. +/// +/// The inputs must be two-dimensional matrices and the inner dimension of +/// `a` (after being transposed if `transpose_a` is non-zero) must match the +/// outer dimension of `b` (after being transposed if `transposed_b` is +/// non-zero). +/// +/// Arguments: +/// * scope: A Scope object +/// * a: Must be a two-dimensional tensor. +/// * b: Must be a two-dimensional tensor. +/// * min_a: The float value that the lowest quantized `a` value represents. +/// * max_a: The float value that the highest quantized `a` value represents. +/// * min_b: The float value that the lowest quantized `b` value represents. +/// * max_b: The float value that the highest quantized `b` value represents. +/// +/// Optional attributes (see `Attrs`): +/// * transpose_a: If true, `a` is transposed before multiplication. +/// * transpose_b: If true, `b` is transposed before multiplication. +/// * Tactivation: The type of output produced by activation function +/// following this operation. +/// +/// Returns: +/// * `Output` out +/// * `Output` min_out: The float value that the lowest quantized output value represents. +/// * `Output` max_out: The float value that the highest quantized output value represents. +class QuantizedMatMul { + public: + /// Optional attribute setters for QuantizedMatMul + struct Attrs { + /// Defaults to DT_QINT32 + TF_MUST_USE_RESULT Attrs Toutput(DataType x) { + Attrs ret = *this; + ret.Toutput_ = x; + return ret; + } + + /// If true, `a` is transposed before multiplication. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs TransposeA(bool x) { + Attrs ret = *this; + ret.transpose_a_ = x; + return ret; + } + + /// If true, `b` is transposed before multiplication. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs TransposeB(bool x) { + Attrs ret = *this; + ret.transpose_b_ = x; + return ret; + } + + /// The type of output produced by activation function + /// following this operation. + /// + /// Defaults to DT_QUINT8 + TF_MUST_USE_RESULT Attrs Tactivation(DataType x) { + Attrs ret = *this; + ret.Tactivation_ = x; + return ret; + } + + DataType Toutput_ = DT_QINT32; + bool transpose_a_ = false; + bool transpose_b_ = false; + DataType Tactivation_ = DT_QUINT8; + }; + QuantizedMatMul(const ::tensorflow::Scope& scope, ::tensorflow::Input a, + ::tensorflow::Input b, ::tensorflow::Input min_a, + ::tensorflow::Input max_a, ::tensorflow::Input min_b, + ::tensorflow::Input max_b); + QuantizedMatMul(const ::tensorflow::Scope& scope, ::tensorflow::Input a, + ::tensorflow::Input b, ::tensorflow::Input min_a, + ::tensorflow::Input max_a, ::tensorflow::Input min_b, + ::tensorflow::Input max_b, const QuantizedMatMul::Attrs& attrs); + + static Attrs Toutput(DataType x) { + return Attrs().Toutput(x); + } + static Attrs TransposeA(bool x) { + return Attrs().TransposeA(x); + } + static Attrs TransposeB(bool x) { + return Attrs().TransposeB(x); + } + static Attrs Tactivation(DataType x) { + return Attrs().Tactivation(x); + } + + Operation operation; + ::tensorflow::Output out; + ::tensorflow::Output min_out; + ::tensorflow::Output max_out; +}; + +/// Returns x * y element-wise, working on quantized buffers. +/// +/// Arguments: +/// * scope: A Scope object +/// * min_x: The float value that the lowest quantized `x` value represents. +/// * max_x: The float value that the highest quantized `x` value represents. +/// * min_y: The float value that the lowest quantized `y` value represents. +/// * max_y: The float value that the highest quantized `y` value represents. +/// +/// Returns: +/// * `Output` z +/// * `Output` min_z: The float value that the lowest quantized output value represents. +/// * `Output` max_z: The float value that the highest quantized output value represents. +/// +/// *NOTE*: `QuantizedMul` supports limited forms of broadcasting. More about +/// broadcasting [here](http://docs.scipy.org/doc/numpy/user/basics.broadcasting.html) +class QuantizedMul { + public: + /// Optional attribute setters for QuantizedMul + struct Attrs { + /// Defaults to DT_QINT32 + TF_MUST_USE_RESULT Attrs Toutput(DataType x) { + Attrs ret = *this; + ret.Toutput_ = x; + return ret; + } + + DataType Toutput_ = DT_QINT32; + }; + QuantizedMul(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input y, ::tensorflow::Input min_x, + ::tensorflow::Input max_x, ::tensorflow::Input min_y, + ::tensorflow::Input max_y); + QuantizedMul(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input y, ::tensorflow::Input min_x, + ::tensorflow::Input max_x, ::tensorflow::Input min_y, + ::tensorflow::Input max_y, const QuantizedMul::Attrs& attrs); + + static Attrs Toutput(DataType x) { + return Attrs().Toutput(x); + } + + Operation operation; + ::tensorflow::Output z; + ::tensorflow::Output min_z; + ::tensorflow::Output max_z; +}; + +/// Creates a sequence of numbers. +/// +/// This operation creates a sequence of numbers that begins at `start` and +/// extends by increments of `delta` up to but not including `limit`. +/// +/// For example: +/// +/// ``` +/// # 'start' is 3 +/// # 'limit' is 18 +/// # 'delta' is 3 +/// tf.range(start, limit, delta) ==> [3, 6, 9, 12, 15] +/// ``` +/// +/// Arguments: +/// * scope: A Scope object +/// * start: 0-D (scalar). First entry in the sequence. +/// * limit: 0-D (scalar). Upper limit of sequence, exclusive. +/// * delta: 0-D (scalar). Optional. Default is 1. Number that increments `start`. +/// +/// Returns: +/// * `Output`: 1-D. +class Range { + public: + Range(const ::tensorflow::Scope& scope, ::tensorflow::Input start, + ::tensorflow::Input limit, ::tensorflow::Input delta); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Returns the real part of a complex number. +/// +/// Given a tensor `input` of complex numbers, this operation returns a tensor of +/// type `float` that is the real part of each element in `input`. All elements in +/// `input` must be complex numbers of the form \\(a + bj\\), where *a* is the real +/// part returned by this operation and *b* is the imaginary part. +/// +/// For example: +/// +/// ``` +/// # tensor 'input' is [-2.25 + 4.75j, 3.25 + 5.75j] +/// tf.real(input) ==> [-2.25, 3.25] +/// ``` +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The output tensor. +class Real { + public: + /// Optional attribute setters for Real + struct Attrs { + /// Defaults to DT_FLOAT + TF_MUST_USE_RESULT Attrs Tout(DataType x) { + Attrs ret = *this; + ret.Tout_ = x; + return ret; + } + + DataType Tout_ = DT_FLOAT; + }; + Real(const ::tensorflow::Scope& scope, ::tensorflow::Input input); + Real(const ::tensorflow::Scope& scope, ::tensorflow::Input input, const + Real::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs Tout(DataType x) { + return Attrs().Tout(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Returns x / y element-wise for real types. +/// +/// If `x` and `y` are reals, this will return the floating-point division. +/// +/// *NOTE*: `Div` supports broadcasting. More about broadcasting +/// [here](http://docs.scipy.org/doc/numpy/user/basics.broadcasting.html) +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The z tensor. +class RealDiv { + public: + RealDiv(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input y); + operator ::tensorflow::Output() const { return z; } + operator ::tensorflow::Input() const { return z; } + ::tensorflow::Node* node() const { return z.node(); } + + Operation operation; + ::tensorflow::Output z; +}; + +/// Computes the reciprocal of x element-wise. +/// +/// I.e., \\(y = 1 / x\\). +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The y tensor. +class Reciprocal { + public: + Reciprocal(const ::tensorflow::Scope& scope, ::tensorflow::Input x); + operator ::tensorflow::Output() const { return y; } + operator ::tensorflow::Input() const { return y; } + ::tensorflow::Node* node() const { return y.node(); } + + Operation operation; + ::tensorflow::Output y; +}; + +/// Given a quantized tensor described by (input, input_min, input_max), outputs a +/// +/// range that covers the actual values present in that tensor. This op is +/// typically used to produce the requested_output_min and requested_output_max for +/// Requantize. +/// +/// Arguments: +/// * scope: A Scope object +/// * input_min: The float value that the minimum quantized input value represents. +/// * input_max: The float value that the maximum quantized input value represents. +/// +/// Returns: +/// * `Output` output_min: The computed min output. +/// * `Output` output_max: the computed max output. +class RequantizationRange { + public: + RequantizationRange(const ::tensorflow::Scope& scope, ::tensorflow::Input + input, ::tensorflow::Input input_min, ::tensorflow::Input + input_max); + + Operation operation; + ::tensorflow::Output output_min; + ::tensorflow::Output output_max; +}; + +/// Convert the quantized 'input' tensor into a lower-precision 'output', using the +/// +/// output range specified with 'requested_output_min' and 'requested_output_max'. +/// +/// [input_min, input_max] are scalar floats that specify the range for the float +/// interpretation of the 'input' data. For example, if input_min is -1.0f and +/// input_max is 1.0f, and we are dealing with quint16 quantized data, then a 0 +/// value in the 16-bit data should be interpreted as -1.0f, and a 65535 means 1.0f. +/// +/// Arguments: +/// * scope: A Scope object +/// * input_min: The float value that the minimum quantized input value represents. +/// * input_max: The float value that the maximum quantized input value represents. +/// * requested_output_min: The float value that the minimum quantized output value represents. +/// * requested_output_max: The float value that the maximum quantized output value represents. +/// * out_type: The type of the output. Should be a lower bit depth than Tinput. +/// +/// Returns: +/// * `Output` output +/// * `Output` output_min: The requested_output_min value is copied into this output. +/// * `Output` output_max: The requested_output_max value is copied into this output. +class Requantize { + public: + Requantize(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input input_min, ::tensorflow::Input input_max, + ::tensorflow::Input requested_output_min, ::tensorflow::Input + requested_output_max, DataType out_type); + + Operation operation; + ::tensorflow::Output output; + ::tensorflow::Output output_min; + ::tensorflow::Output output_max; +}; + +/// Returns element-wise integer closest to x. +/// +/// If the result is midway between two representable values, +/// the even representable is chosen. +/// For example: +/// +/// ``` +/// rint(-1.5) ==> -2.0 +/// rint(0.5000001) ==> 1.0 +/// rint([-1.7, -1.5, -0.2, 0.2, 1.5, 1.7, 2.0]) ==> [-2., -2., -0., 0., 2., 2., 2.] +/// ``` +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The y tensor. +class Rint { + public: + Rint(const ::tensorflow::Scope& scope, ::tensorflow::Input x); + operator ::tensorflow::Output() const { return y; } + operator ::tensorflow::Input() const { return y; } + ::tensorflow::Node* node() const { return y.node(); } + + Operation operation; + ::tensorflow::Output y; +}; + +/// Rounds the values of a tensor to the nearest integer, element-wise. +/// +/// Rounds half to even. Also known as bankers rounding. If you want to round +/// according to the current system rounding mode use std::cint. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The y tensor. +class Round { + public: + Round(const ::tensorflow::Scope& scope, ::tensorflow::Input x); + operator ::tensorflow::Output() const { return y; } + operator ::tensorflow::Input() const { return y; } + ::tensorflow::Node* node() const { return y.node(); } + + Operation operation; + ::tensorflow::Output y; +}; + +/// Computes reciprocal of square root of x element-wise. +/// +/// I.e., \\(y = 1 / \sqrt{x}\\). +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The y tensor. +class Rsqrt { + public: + Rsqrt(const ::tensorflow::Scope& scope, ::tensorflow::Input x); + operator ::tensorflow::Output() const { return y; } + operator ::tensorflow::Input() const { return y; } + ::tensorflow::Node* node() const { return y.node(); } + + Operation operation; + ::tensorflow::Output y; +}; + +/// Computes the maximum along segments of a tensor. +/// +/// Read +/// [the section on segmentation](https://tensorflow.org/api_guides/python/math_ops#Segmentation) +/// for an explanation of segments. +/// +/// Computes a tensor such that +/// \\(output_i = \max_j(data_j)\\) where `max` is over `j` such +/// that `segment_ids[j] == i`. +/// +/// If the max is empty for a given segment ID `i`, `output[i] = 0`. +/// +///
+/// +///
+/// +/// Arguments: +/// * scope: A Scope object +/// * segment_ids: A 1-D tensor whose size is equal to the size of `data`'s +/// first dimension. Values should be sorted and can be repeated. +/// +/// Returns: +/// * `Output`: Has same shape as data, except for dimension 0 which +/// has size `k`, the number of segments. +class SegmentMax { + public: + SegmentMax(const ::tensorflow::Scope& scope, ::tensorflow::Input data, + ::tensorflow::Input segment_ids); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Computes the mean along segments of a tensor. +/// +/// Read +/// [the section on segmentation](https://tensorflow.org/api_guides/python/math_ops#Segmentation) +/// for an explanation of segments. +/// +/// Computes a tensor such that +/// \\(output_i = \frac{\sum_j data_j}{N}\\) where `mean` is +/// over `j` such that `segment_ids[j] == i` and `N` is the total number of +/// values summed. +/// +/// If the mean is empty for a given segment ID `i`, `output[i] = 0`. +/// +///
+/// +///
+/// +/// Arguments: +/// * scope: A Scope object +/// * segment_ids: A 1-D tensor whose size is equal to the size of `data`'s +/// first dimension. Values should be sorted and can be repeated. +/// +/// Returns: +/// * `Output`: Has same shape as data, except for dimension 0 which +/// has size `k`, the number of segments. +class SegmentMean { + public: + SegmentMean(const ::tensorflow::Scope& scope, ::tensorflow::Input data, + ::tensorflow::Input segment_ids); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Computes the minimum along segments of a tensor. +/// +/// Read +/// [the section on segmentation](https://tensorflow.org/api_guides/python/math_ops#Segmentation) +/// for an explanation of segments. +/// +/// Computes a tensor such that +/// \\(output_i = \min_j(data_j)\\) where `min` is over `j` such +/// that `segment_ids[j] == i`. +/// +/// If the min is empty for a given segment ID `i`, `output[i] = 0`. +/// +///
+/// +///
+/// +/// Arguments: +/// * scope: A Scope object +/// * segment_ids: A 1-D tensor whose size is equal to the size of `data`'s +/// first dimension. Values should be sorted and can be repeated. +/// +/// Returns: +/// * `Output`: Has same shape as data, except for dimension 0 which +/// has size `k`, the number of segments. +class SegmentMin { + public: + SegmentMin(const ::tensorflow::Scope& scope, ::tensorflow::Input data, + ::tensorflow::Input segment_ids); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Computes the product along segments of a tensor. +/// +/// Read +/// [the section on segmentation](https://tensorflow.org/api_guides/python/math_ops#Segmentation) +/// for an explanation of segments. +/// +/// Computes a tensor such that +/// \\(output_i = \prod_j data_j\\) where the product is over `j` such +/// that `segment_ids[j] == i`. +/// +/// If the product is empty for a given segment ID `i`, `output[i] = 1`. +/// +///
+/// +///
+/// +/// Arguments: +/// * scope: A Scope object +/// * segment_ids: A 1-D tensor whose size is equal to the size of `data`'s +/// first dimension. Values should be sorted and can be repeated. +/// +/// Returns: +/// * `Output`: Has same shape as data, except for dimension 0 which +/// has size `k`, the number of segments. +class SegmentProd { + public: + SegmentProd(const ::tensorflow::Scope& scope, ::tensorflow::Input data, + ::tensorflow::Input segment_ids); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Computes the sum along segments of a tensor. +/// +/// Read +/// [the section on segmentation](https://tensorflow.org/api_guides/python/math_ops#Segmentation) +/// for an explanation of segments. +/// +/// Computes a tensor such that +/// \\(output_i = \sum_j data_j\\) where sum is over `j` such +/// that `segment_ids[j] == i`. +/// +/// If the sum is empty for a given segment ID `i`, `output[i] = 0`. +/// +///
+/// +///
+/// +/// Arguments: +/// * scope: A Scope object +/// * segment_ids: A 1-D tensor whose size is equal to the size of `data`'s +/// first dimension. Values should be sorted and can be repeated. +/// +/// Returns: +/// * `Output`: Has same shape as data, except for dimension 0 which +/// has size `k`, the number of segments. +class SegmentSum { + public: + SegmentSum(const ::tensorflow::Scope& scope, ::tensorflow::Input data, + ::tensorflow::Input segment_ids); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Selects elements from `x` or `y`, depending on `condition`. +/// +/// The `x`, and `y` tensors must all have the same shape, and the +/// output will also have that shape. +/// +/// The `condition` tensor must be a scalar if `x` and `y` are scalars. +/// If `x` and `y` are vectors or higher rank, then `condition` must be either a +/// scalar, a vector with size matching the first dimension of `x`, or must have +/// the same shape as `x`. +/// +/// The `condition` tensor acts as a mask that chooses, based on the value at each +/// element, whether the corresponding element / row in the output should be +/// taken from `x` (if true) or `y` (if false). +/// +/// If `condition` is a vector and `x` and `y` are higher rank matrices, then +/// it chooses which row (outer dimension) to copy from `x` and `y`. +/// If `condition` has the same shape as `x` and `y`, then it chooses which +/// element to copy from `x` and `y`. +/// +/// For example: +/// +/// ```python +/// # 'condition' tensor is [[True, False] +/// # [False, True]] +/// # 't' is [[1, 2], +/// # [3, 4]] +/// # 'e' is [[5, 6], +/// # [7, 8]] +/// select(condition, t, e) # => [[1, 6], [7, 4]] +/// +/// +/// # 'condition' tensor is [True, False] +/// # 't' is [[1, 2], +/// # [3, 4]] +/// # 'e' is [[5, 6], +/// # [7, 8]] +/// select(condition, t, e) ==> [[1, 2], +/// [7, 8]] +/// +/// ``` +/// +/// Arguments: +/// * scope: A Scope object +/// * x: = A `Tensor` which may have the same shape as `condition`. +/// If `condition` is rank 1, `x` may have higher rank, +/// but its first dimension must match the size of `condition`. +/// * y: = A `Tensor` with the same type and shape as `x`. +/// +/// Returns: +/// * `Output`: = A `Tensor` with the same type and shape as `x` and `y`. +class Where3 { + public: + Where3(const ::tensorflow::Scope& scope, ::tensorflow::Input condition, + ::tensorflow::Input x, ::tensorflow::Input y); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Computes sigmoid of `x` element-wise. +/// +/// Specifically, `y = 1 / (1 + exp(-x))`. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The y tensor. +class Sigmoid { + public: + Sigmoid(const ::tensorflow::Scope& scope, ::tensorflow::Input x); + operator ::tensorflow::Output() const { return y; } + operator ::tensorflow::Input() const { return y; } + ::tensorflow::Node* node() const { return y.node(); } + + Operation operation; + ::tensorflow::Output y; +}; + +/// Returns an element-wise indication of the sign of a number. +/// +/// `y = sign(x) = -1` if `x < 0`; 0 if `x == 0`; 1 if `x > 0`. +/// +/// For complex numbers, `y = sign(x) = x / |x|` if `x != 0`, otherwise `y = 0`. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The y tensor. +class Sign { + public: + Sign(const ::tensorflow::Scope& scope, ::tensorflow::Input x); + operator ::tensorflow::Output() const { return y; } + operator ::tensorflow::Input() const { return y; } + ::tensorflow::Node* node() const { return y.node(); } + + Operation operation; + ::tensorflow::Output y; +}; + +/// Computes sin of x element-wise. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The y tensor. +class Sin { + public: + Sin(const ::tensorflow::Scope& scope, ::tensorflow::Input x); + operator ::tensorflow::Output() const { return y; } + operator ::tensorflow::Input() const { return y; } + ::tensorflow::Node* node() const { return y.node(); } + + Operation operation; + ::tensorflow::Output y; +}; + +/// Computes hyperbolic sine of x element-wise. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The y tensor. +class Sinh { + public: + Sinh(const ::tensorflow::Scope& scope, ::tensorflow::Input x); + operator ::tensorflow::Output() const { return y; } + operator ::tensorflow::Input() const { return y; } + ::tensorflow::Node* node() const { return y.node(); } + + Operation operation; + ::tensorflow::Output y; +}; + +/// Multiply matrix "a" by matrix "b". +/// +/// The inputs must be two-dimensional matrices and the inner dimension of "a" must +/// match the outer dimension of "b". Both "a" and "b" must be `Tensor`s not +/// `SparseTensor`s. This op is optimized for the case where at least one of "a" or +/// "b" is sparse, in the sense that they have a large proportion of zero values. +/// The breakeven for using this versus a dense matrix multiply on one platform was +/// 30% zero values in the sparse matrix. +/// +/// The gradient computation of this operation will only take advantage of sparsity +/// in the input gradient when that gradient comes from a Relu. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The product tensor. +class SparseMatMul { + public: + /// Optional attribute setters for SparseMatMul + struct Attrs { + /// Defaults to false + TF_MUST_USE_RESULT Attrs TransposeA(bool x) { + Attrs ret = *this; + ret.transpose_a_ = x; + return ret; + } + + /// Defaults to false + TF_MUST_USE_RESULT Attrs TransposeB(bool x) { + Attrs ret = *this; + ret.transpose_b_ = x; + return ret; + } + + /// Defaults to false + TF_MUST_USE_RESULT Attrs AIsSparse(bool x) { + Attrs ret = *this; + ret.a_is_sparse_ = x; + return ret; + } + + /// Defaults to false + TF_MUST_USE_RESULT Attrs BIsSparse(bool x) { + Attrs ret = *this; + ret.b_is_sparse_ = x; + return ret; + } + + bool transpose_a_ = false; + bool transpose_b_ = false; + bool a_is_sparse_ = false; + bool b_is_sparse_ = false; + }; + SparseMatMul(const ::tensorflow::Scope& scope, ::tensorflow::Input a, + ::tensorflow::Input b); + SparseMatMul(const ::tensorflow::Scope& scope, ::tensorflow::Input a, + ::tensorflow::Input b, const SparseMatMul::Attrs& attrs); + operator ::tensorflow::Output() const { return product; } + operator ::tensorflow::Input() const { return product; } + ::tensorflow::Node* node() const { return product.node(); } + + static Attrs TransposeA(bool x) { + return Attrs().TransposeA(x); + } + static Attrs TransposeB(bool x) { + return Attrs().TransposeB(x); + } + static Attrs AIsSparse(bool x) { + return Attrs().AIsSparse(x); + } + static Attrs BIsSparse(bool x) { + return Attrs().BIsSparse(x); + } + + Operation operation; + ::tensorflow::Output product; +}; + +/// Computes the mean along sparse segments of a tensor. +/// +/// Read +/// [the section on segmentation](https://tensorflow.org/api_guides/python/math_ops#Segmentation) +/// for an explanation of segments. +/// +/// Like `SegmentMean`, but `segment_ids` can have rank less than `data`'s first +/// dimension, selecting a subset of dimension 0, specified by `indices`. +/// +/// Arguments: +/// * scope: A Scope object +/// * indices: A 1-D tensor. Has same rank as `segment_ids`. +/// * segment_ids: A 1-D tensor. Values should be sorted and can be repeated. +/// +/// Returns: +/// * `Output`: Has same shape as data, except for dimension 0 which +/// has size `k`, the number of segments. +class SparseSegmentMean { + public: + SparseSegmentMean(const ::tensorflow::Scope& scope, ::tensorflow::Input data, + ::tensorflow::Input indices, ::tensorflow::Input segment_ids); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Computes gradients for SparseSegmentMean. +/// +/// Returns tensor "output" with same shape as grad, except for dimension 0 whose +/// value is output_dim0. +/// +/// Arguments: +/// * scope: A Scope object +/// * grad: gradient propagated to the SparseSegmentMean op. +/// * indices: indices passed to the corresponding SparseSegmentMean op. +/// * segment_ids: segment_ids passed to the corresponding SparseSegmentMean op. +/// * output_dim0: dimension 0 of "data" passed to SparseSegmentMean op. +/// +/// Returns: +/// * `Output`: The output tensor. +class SparseSegmentMeanGrad { + public: + SparseSegmentMeanGrad(const ::tensorflow::Scope& scope, ::tensorflow::Input + grad, ::tensorflow::Input indices, ::tensorflow::Input + segment_ids, ::tensorflow::Input output_dim0); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Computes the mean along sparse segments of a tensor. +/// +/// Like `SparseSegmentMean`, but allows missing ids in `segment_ids`. If an id is +/// misisng, the `output` tensor at that position will be zeroed. +/// +/// Read +/// [the section on segmentation](https://tensorflow.org/api_guides/python/math_ops#Segmentation) +/// for an explanation of segments. +/// +/// Arguments: +/// * scope: A Scope object +/// * indices: A 1-D tensor. Has same rank as `segment_ids`. +/// * segment_ids: A 1-D tensor. Values should be sorted and can be repeated. +/// * num_segments: Should equal the number of distinct segment IDs. +/// +/// Returns: +/// * `Output`: Has same shape as data, except for dimension 0 which has size +/// `num_segments`. +class SparseSegmentMeanWithNumSegments { + public: + SparseSegmentMeanWithNumSegments(const ::tensorflow::Scope& scope, + ::tensorflow::Input data, ::tensorflow::Input + indices, ::tensorflow::Input segment_ids, + ::tensorflow::Input num_segments); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Computes the sum along sparse segments of a tensor divided by the sqrt of N. +/// +/// N is the size of the segment being reduced. +/// +/// Read +/// [the section on segmentation](https://tensorflow.org/api_guides/python/math_ops#Segmentation) +/// for an explanation of segments. +/// +/// Arguments: +/// * scope: A Scope object +/// * indices: A 1-D tensor. Has same rank as `segment_ids`. +/// * segment_ids: A 1-D tensor. Values should be sorted and can be repeated. +/// +/// Returns: +/// * `Output`: Has same shape as data, except for dimension 0 which +/// has size `k`, the number of segments. +class SparseSegmentSqrtN { + public: + SparseSegmentSqrtN(const ::tensorflow::Scope& scope, ::tensorflow::Input data, + ::tensorflow::Input indices, ::tensorflow::Input + segment_ids); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Computes gradients for SparseSegmentSqrtN. +/// +/// Returns tensor "output" with same shape as grad, except for dimension 0 whose +/// value is output_dim0. +/// +/// Arguments: +/// * scope: A Scope object +/// * grad: gradient propagated to the SparseSegmentSqrtN op. +/// * indices: indices passed to the corresponding SparseSegmentSqrtN op. +/// * segment_ids: segment_ids passed to the corresponding SparseSegmentSqrtN op. +/// * output_dim0: dimension 0 of "data" passed to SparseSegmentSqrtN op. +/// +/// Returns: +/// * `Output`: The output tensor. +class SparseSegmentSqrtNGrad { + public: + SparseSegmentSqrtNGrad(const ::tensorflow::Scope& scope, ::tensorflow::Input + grad, ::tensorflow::Input indices, ::tensorflow::Input + segment_ids, ::tensorflow::Input output_dim0); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Computes the sum along sparse segments of a tensor divided by the sqrt of N. +/// +/// N is the size of the segment being reduced. +/// +/// Like `SparseSegmentSqrtN`, but allows missing ids in `segment_ids`. If an id is +/// misisng, the `output` tensor at that position will be zeroed. +/// +/// Read +/// [the section on segmentation](https://tensorflow.org/api_guides/python/math_ops#Segmentation) +/// for an explanation of segments. +/// +/// Arguments: +/// * scope: A Scope object +/// * indices: A 1-D tensor. Has same rank as `segment_ids`. +/// * segment_ids: A 1-D tensor. Values should be sorted and can be repeated. +/// * num_segments: Should equal the number of distinct segment IDs. +/// +/// Returns: +/// * `Output`: Has same shape as data, except for dimension 0 which +/// has size `k`, the number of segments. +class SparseSegmentSqrtNWithNumSegments { + public: + SparseSegmentSqrtNWithNumSegments(const ::tensorflow::Scope& scope, + ::tensorflow::Input data, ::tensorflow::Input + indices, ::tensorflow::Input segment_ids, + ::tensorflow::Input num_segments); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Computes the sum along sparse segments of a tensor. +/// +/// Read +/// [the section on segmentation](https://tensorflow.org/api_guides/python/math_ops#Segmentation) +/// for an explanation of segments. +/// +/// Like `SegmentSum`, but `segment_ids` can have rank less than `data`'s first +/// dimension, selecting a subset of dimension 0, specified by `indices`. +/// +/// For example: +/// +/// ```python +/// c = tf.constant([[1,2,3,4], [-1,-2,-3,-4], [5,6,7,8]]) +/// +/// # Select two rows, one segment. +/// tf.sparse_segment_sum(c, tf.constant([0, 1]), tf.constant([0, 0])) +/// # => [[0 0 0 0]] +/// +/// # Select two rows, two segment. +/// tf.sparse_segment_sum(c, tf.constant([0, 1]), tf.constant([0, 1])) +/// # => [[ 1 2 3 4] +/// # [-1 -2 -3 -4]] +/// +/// # Select all rows, two segments. +/// tf.sparse_segment_sum(c, tf.constant([0, 1, 2]), tf.constant([0, 0, 1])) +/// # => [[0 0 0 0] +/// # [5 6 7 8]] +/// +/// # Which is equivalent to: +/// tf.segment_sum(c, tf.constant([0, 0, 1])) +/// ``` +/// +/// Arguments: +/// * scope: A Scope object +/// * indices: A 1-D tensor. Has same rank as `segment_ids`. +/// * segment_ids: A 1-D tensor. Values should be sorted and can be repeated. +/// +/// Returns: +/// * `Output`: Has same shape as data, except for dimension 0 which +/// has size `k`, the number of segments. +class SparseSegmentSum { + public: + SparseSegmentSum(const ::tensorflow::Scope& scope, ::tensorflow::Input data, + ::tensorflow::Input indices, ::tensorflow::Input segment_ids); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Computes the sum along sparse segments of a tensor. +/// +/// Like `SparseSegmentSum`, but allows missing ids in `segment_ids`. If an id is +/// misisng, the `output` tensor at that position will be zeroed. +/// +/// Read +/// [the section on segmentation](https://tensorflow.org/api_guides/python/math_ops#Segmentation) +/// for an explanation of segments. +/// +/// For example: +/// +/// ```python +/// c = tf.constant([[1,2,3,4], [-1,-2,-3,-4], [5,6,7,8]]) +/// +/// tf.sparse_segment_sum_with_num_segments( +/// c, tf.constant([0, 1]), tf.constant([0, 0]), num_segments=3) +/// # => [[0 0 0 0] +/// # [0 0 0 0] +/// # [0 0 0 0]] +/// +/// tf.sparse_segment_sum_with_num_segments(c, +/// tf.constant([0, 1]), +/// tf.constant([0, 2], +/// num_segments=4)) +/// # => [[ 1 2 3 4] +/// # [ 0 0 0 0] +/// # [-1 -2 -3 -4] +/// # [ 0 0 0 0]] +/// ``` +/// +/// Arguments: +/// * scope: A Scope object +/// * indices: A 1-D tensor. Has same rank as `segment_ids`. +/// * segment_ids: A 1-D tensor. Values should be sorted and can be repeated. +/// * num_segments: Should equal the number of distinct segment IDs. +/// +/// Returns: +/// * `Output`: Has same shape as data, except for dimension 0 which +/// has size `num_segments`. +class SparseSegmentSumWithNumSegments { + public: + SparseSegmentSumWithNumSegments(const ::tensorflow::Scope& scope, + ::tensorflow::Input data, ::tensorflow::Input + indices, ::tensorflow::Input segment_ids, + ::tensorflow::Input num_segments); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Computes square root of x element-wise. +/// +/// I.e., \\(y = \sqrt{x} = x^{1/2}\\). +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The y tensor. +class Sqrt { + public: + Sqrt(const ::tensorflow::Scope& scope, ::tensorflow::Input x); + operator ::tensorflow::Output() const { return y; } + operator ::tensorflow::Input() const { return y; } + ::tensorflow::Node* node() const { return y.node(); } + + Operation operation; + ::tensorflow::Output y; +}; + +/// Computes square of x element-wise. +/// +/// I.e., \\(y = x * x = x^2\\). +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The y tensor. +class Square { + public: + Square(const ::tensorflow::Scope& scope, ::tensorflow::Input x); + operator ::tensorflow::Output() const { return y; } + operator ::tensorflow::Input() const { return y; } + ::tensorflow::Node* node() const { return y.node(); } + + Operation operation; + ::tensorflow::Output y; +}; + +/// Returns (x - y)(x - y) element-wise. +/// +/// *NOTE*: `SquaredDifference` supports broadcasting. More about broadcasting +/// [here](http://docs.scipy.org/doc/numpy/user/basics.broadcasting.html) +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The z tensor. +class SquaredDifference { + public: + SquaredDifference(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input y); + operator ::tensorflow::Output() const { return z; } + operator ::tensorflow::Input() const { return z; } + ::tensorflow::Node* node() const { return z.node(); } + + Operation operation; + ::tensorflow::Output z; +}; + +/// Returns x - y element-wise. +/// +/// *NOTE*: `Subtract` supports broadcasting. More about broadcasting +/// [here](http://docs.scipy.org/doc/numpy/user/basics.broadcasting.html) +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The z tensor. +/// +/// Aliases: +/// * Sub +class Subtract { + public: + Subtract(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input y); + operator ::tensorflow::Output() const { return z; } + operator ::tensorflow::Input() const { return z; } + ::tensorflow::Node* node() const { return z.node(); } + + Operation operation; + ::tensorflow::Output z; +}; +typedef Subtract Sub; + +/// Computes the sum of elements across dimensions of a tensor. +/// +/// Reduces `input` along the dimensions given in `axis`. Unless +/// `keep_dims` is true, the rank of the tensor is reduced by 1 for each entry in +/// `axis`. If `keep_dims` is true, the reduced dimensions are +/// retained with length 1. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: The tensor to reduce. +/// * axis: The dimensions to reduce. Must be in the range +/// `[-rank(input), rank(input))`. +/// +/// Optional attributes (see `Attrs`): +/// * keep_dims: If true, retain reduced dimensions with length 1. +/// +/// Returns: +/// * `Output`: The reduced tensor. +/// +/// Aliases: +/// * ReduceSum +class Sum { + public: + /// Optional attribute setters for Sum + struct Attrs { + /// If true, retain reduced dimensions with length 1. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs KeepDims(bool x) { + Attrs ret = *this; + ret.keep_dims_ = x; + return ret; + } + + bool keep_dims_ = false; + }; + Sum(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input axis); + Sum(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input axis, const Sum::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs KeepDims(bool x) { + return Attrs().KeepDims(x); + } + + Operation operation; + ::tensorflow::Output output; +}; +typedef Sum ReduceSum; + +/// Computes tan of x element-wise. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The y tensor. +class Tan { + public: + Tan(const ::tensorflow::Scope& scope, ::tensorflow::Input x); + operator ::tensorflow::Output() const { return y; } + operator ::tensorflow::Input() const { return y; } + ::tensorflow::Node* node() const { return y.node(); } + + Operation operation; + ::tensorflow::Output y; +}; + +/// Computes hyperbolic tangent of `x` element-wise. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The y tensor. +class Tanh { + public: + Tanh(const ::tensorflow::Scope& scope, ::tensorflow::Input x); + operator ::tensorflow::Output() const { return y; } + operator ::tensorflow::Input() const { return y; } + ::tensorflow::Node* node() const { return y.node(); } + + Operation operation; + ::tensorflow::Output y; +}; + +/// Returns x / y element-wise for integer types. +/// +/// Truncation designates that negative numbers will round fractional quantities +/// toward zero. I.e. -7 / 5 = -1. This matches C semantics but it is different +/// than Python semantics. See `FloorDiv` for a division function that matches +/// Python Semantics. +/// +/// *NOTE*: `TruncateDiv` supports broadcasting. More about broadcasting +/// [here](http://docs.scipy.org/doc/numpy/user/basics.broadcasting.html) +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The z tensor. +class TruncateDiv { + public: + TruncateDiv(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input y); + operator ::tensorflow::Output() const { return z; } + operator ::tensorflow::Input() const { return z; } + ::tensorflow::Node* node() const { return z.node(); } + + Operation operation; + ::tensorflow::Output z; +}; + +/// Returns element-wise remainder of division. This emulates C semantics in that +/// +/// the result here is consistent with a truncating divide. E.g. `truncate(x / y) * +/// y + truncate_mod(x, y) = x`. +/// +/// *NOTE*: `TruncateMod` supports broadcasting. More about broadcasting +/// [here](http://docs.scipy.org/doc/numpy/user/basics.broadcasting.html) +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The z tensor. +class TruncateMod { + public: + TruncateMod(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input y); + operator ::tensorflow::Output() const { return z; } + operator ::tensorflow::Input() const { return z; } + ::tensorflow::Node* node() const { return z.node(); } + + Operation operation; + ::tensorflow::Output z; +}; + +/// Computes the maximum along segments of a tensor. +/// +/// Read +/// [the section on segmentation](https://tensorflow.org/api_guides/python/math_ops#Segmentation) +/// for an explanation of segments. +/// +/// This operator is similar to the unsorted segment sum operator found +/// [(here)](../../../api_docs/python/math_ops.md#UnsortedSegmentSum). +/// Instead of computing the sum over segments, it computes the maximum such that: +/// +/// \\(output_i = \max_{j...} data[j...]\\) where max is over tuples `j...` such +/// that `segment_ids[j...] == i`. +/// +/// If the maximum is empty for a given segment ID `i`, it outputs the smallest +/// possible value for the specific numeric type, +/// `output[i] = numeric_limits::lowest()`. +/// +/// If the given segment ID `i` is negative, then the corresponding value is +/// dropped, and will not be included in the result. +/// +///
+/// +///
+/// +/// Arguments: +/// * scope: A Scope object +/// * segment_ids: A tensor whose shape is a prefix of `data.shape`.END +/// } +/// out_arg { +/// name: "output" +/// description: <::max()`. +/// +/// If the given segment ID `i` is negative, then the corresponding value is +/// dropped, and will not be included in the result. +/// +/// Arguments: +/// * scope: A Scope object +/// * segment_ids: A tensor whose shape is a prefix of `data.shape`. +/// +/// Returns: +/// * `Output`: Has same shape as data, except for the first `segment_ids.rank` +/// dimensions, which are replaced with a single dimension which has size +/// `num_segments`. +class UnsortedSegmentMin { + public: + UnsortedSegmentMin(const ::tensorflow::Scope& scope, ::tensorflow::Input data, + ::tensorflow::Input segment_ids, ::tensorflow::Input + num_segments); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Computes the product along segments of a tensor. +/// +/// Read +/// [the section on segmentation](https://tensorflow.org/api_guides/python/math_ops#segmentation) +/// for an explanation of segments. +/// +/// This operator is similar to the unsorted segment sum operator found +/// [(here)](../../../api_docs/python/math_ops.md#UnsortedSegmentSum). +/// Instead of computing the sum over segments, it computes the product of all +/// entries belonging to a segment such that: +/// +/// \\(output_i = \prod_{j...} data[j...]\\) where the product is over tuples +/// `j...` such that `segment_ids[j...] == i`. +/// +/// If there is no entry for a given segment ID `i`, it outputs 1. +/// +/// If the given segment ID `i` is negative, then the corresponding value is +/// dropped, and will not be included in the result. +/// +/// Arguments: +/// * scope: A Scope object +/// * segment_ids: A tensor whose shape is a prefix of `data.shape`. +/// +/// Returns: +/// * `Output`: Has same shape as data, except for the first `segment_ids.rank` +/// dimensions, which are replaced with a single dimension which has size +/// `num_segments`. +class UnsortedSegmentProd { + public: + UnsortedSegmentProd(const ::tensorflow::Scope& scope, ::tensorflow::Input data, + ::tensorflow::Input segment_ids, ::tensorflow::Input + num_segments); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Computes the sum along segments of a tensor. +/// +/// Read +/// [the section on segmentation](https://tensorflow.org/api_guides/python/math_ops#Segmentation) +/// for an explanation of segments. +/// +/// Computes a tensor such that +/// \\(output[i] = \sum_{j...} data[j...]\\) where the sum is over tuples `j...` such +/// that `segment_ids[j...] == i`. Unlike `SegmentSum`, `segment_ids` +/// need not be sorted and need not cover all values in the full +/// range of valid values. +/// +/// If the sum is empty for a given segment ID `i`, `output[i] = 0`. +/// If the given segment ID `i` is negative, the value is dropped and will not be +/// added to the sum of the segment. +/// +/// `num_segments` should equal the number of distinct segment IDs. +/// +///
+/// +///
+/// +/// Arguments: +/// * scope: A Scope object +/// * segment_ids: A tensor whose shape is a prefix of `data.shape`. +/// +/// Returns: +/// * `Output`: Has same shape as data, except for the first `segment_ids.rank` +/// dimensions, which are replaced with a single dimension which has size +/// `num_segments`. +class UnsortedSegmentSum { + public: + UnsortedSegmentSum(const ::tensorflow::Scope& scope, ::tensorflow::Input data, + ::tensorflow::Input segment_ids, ::tensorflow::Input + num_segments); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Returns 0 if x == 0, and x / y otherwise, elementwise. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The z tensor. +class Xdivy { + public: + Xdivy(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input y); + operator ::tensorflow::Output() const { return z; } + operator ::tensorflow::Input() const { return z; } + ::tensorflow::Node* node() const { return z.node(); } + + Operation operation; + ::tensorflow::Output z; +}; + +/// Returns 0 if x == 0, and x * log(y) otherwise, elementwise. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The z tensor. +class Xlogy { + public: + Xlogy(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input y); + operator ::tensorflow::Output() const { return z; } + operator ::tensorflow::Input() const { return z; } + ::tensorflow::Node* node() const { return z.node(); } + + Operation operation; + ::tensorflow::Output z; +}; + +/// Compute the Hurwitz zeta function \\(\zeta(x, q)\\). +/// +/// The Hurwitz zeta function is defined as: +/// +/// +/// \\(\zeta(x, q) = \sum_{n=0}^{\infty} (q + n)^{-x}\\) +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The z tensor. +class Zeta { + public: + Zeta(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input q); + operator ::tensorflow::Output() const { return z; } + operator ::tensorflow::Input() const { return z; } + ::tensorflow::Node* node() const { return z.node(); } + + Operation operation; + ::tensorflow::Output z; +}; + +/// @} + +} // namespace ops +} // namespace tensorflow + +#endif // TENSORFLOW_CC_OPS_MATH_OPS_H_ diff --git a/linux/include/tensorflow/cc/ops/math_ops_internal.h b/linux/include/tensorflow/cc/ops/math_ops_internal.h new file mode 100644 index 000000000..939b70ab5 --- /dev/null +++ b/linux/include/tensorflow/cc/ops/math_ops_internal.h @@ -0,0 +1,179 @@ +// This file is MACHINE GENERATED! Do not edit. + +#ifndef TENSORFLOW_CC_OPS_MATH_OPS_INTERNAL_H_ +#define TENSORFLOW_CC_OPS_MATH_OPS_INTERNAL_H_ + +// This file is MACHINE GENERATED! Do not edit. + +#include "tensorflow/cc/framework/ops.h" +#include "tensorflow/cc/framework/scope.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/gtl/array_slice.h" + +namespace tensorflow { +namespace ops { +namespace internal { +// NOTE: This namespace has internal TensorFlow details that +// are not part of TensorFlow's public API. + +/// @defgroup math_ops_internal Math Ops Internal +/// @{ + +/// Computes the gradient of `igamma(a, x)` wrt `a`. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The z tensor. +class IgammaGradA { + public: + IgammaGradA(const ::tensorflow::Scope& scope, ::tensorflow::Input a, + ::tensorflow::Input x); + operator ::tensorflow::Output() const { return z; } + operator ::tensorflow::Input() const { return z; } + ::tensorflow::Node* node() const { return z.node(); } + + Operation operation; + ::tensorflow::Output z; +}; + +/// Computes the gradient for the inverse of `x` wrt its input. +/// +/// Specifically, `grad = -dy * y*y`, where `y = 1/x`, and `dy` +/// is the corresponding input gradient. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The z tensor. +class InvGrad { + public: + InvGrad(const ::tensorflow::Scope& scope, ::tensorflow::Input y, + ::tensorflow::Input dy); + operator ::tensorflow::Output() const { return z; } + operator ::tensorflow::Input() const { return z; } + ::tensorflow::Node* node() const { return z.node(); } + + Operation operation; + ::tensorflow::Output z; +}; + +/// Computes the gradient for the inverse of `x` wrt its input. +/// +/// Specifically, `grad = -dy * y*y`, where `y = 1/x`, and `dy` +/// is the corresponding input gradient. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The z tensor. +class ReciprocalGrad { + public: + ReciprocalGrad(const ::tensorflow::Scope& scope, ::tensorflow::Input y, + ::tensorflow::Input dy); + operator ::tensorflow::Output() const { return z; } + operator ::tensorflow::Input() const { return z; } + ::tensorflow::Node* node() const { return z.node(); } + + Operation operation; + ::tensorflow::Output z; +}; + +/// Computes the gradient for the rsqrt of `x` wrt its input. +/// +/// Specifically, `grad = dy * -0.5 * y^3`, where `y = rsqrt(x)`, and `dy` +/// is the corresponding input gradient. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The z tensor. +class RsqrtGrad { + public: + RsqrtGrad(const ::tensorflow::Scope& scope, ::tensorflow::Input y, + ::tensorflow::Input dy); + operator ::tensorflow::Output() const { return z; } + operator ::tensorflow::Input() const { return z; } + ::tensorflow::Node* node() const { return z.node(); } + + Operation operation; + ::tensorflow::Output z; +}; + +/// Computes the gradient of the sigmoid of `x` wrt its input. +/// +/// Specifically, `grad = dy * y * (1 - y)`, where `y = sigmoid(x)`, and +/// `dy` is the corresponding input gradient. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The z tensor. +class SigmoidGrad { + public: + SigmoidGrad(const ::tensorflow::Scope& scope, ::tensorflow::Input y, + ::tensorflow::Input dy); + operator ::tensorflow::Output() const { return z; } + operator ::tensorflow::Input() const { return z; } + ::tensorflow::Node* node() const { return z.node(); } + + Operation operation; + ::tensorflow::Output z; +}; + +/// Computes the gradient for the sqrt of `x` wrt its input. +/// +/// Specifically, `grad = dy * 0.5 / y`, where `y = sqrt(x)`, and `dy` +/// is the corresponding input gradient. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The z tensor. +class SqrtGrad { + public: + SqrtGrad(const ::tensorflow::Scope& scope, ::tensorflow::Input y, + ::tensorflow::Input dy); + operator ::tensorflow::Output() const { return z; } + operator ::tensorflow::Input() const { return z; } + ::tensorflow::Node* node() const { return z.node(); } + + Operation operation; + ::tensorflow::Output z; +}; + +/// Computes the gradient for the tanh of `x` wrt its input. +/// +/// Specifically, `grad = dy * (1 - y*y)`, where `y = tanh(x)`, and `dy` +/// is the corresponding input gradient. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The z tensor. +class TanhGrad { + public: + TanhGrad(const ::tensorflow::Scope& scope, ::tensorflow::Input y, + ::tensorflow::Input dy); + operator ::tensorflow::Output() const { return z; } + operator ::tensorflow::Input() const { return z; } + ::tensorflow::Node* node() const { return z.node(); } + + Operation operation; + ::tensorflow::Output z; +}; + +} // namespace internal +} // namespace ops +} // namespace tensorflow + +#endif // TENSORFLOW_CC_OPS_MATH_OPS_INTERNAL_H_ diff --git a/linux/include/tensorflow/cc/ops/nn_ops.h b/linux/include/tensorflow/cc/ops/nn_ops.h new file mode 100644 index 000000000..ce6b58f48 --- /dev/null +++ b/linux/include/tensorflow/cc/ops/nn_ops.h @@ -0,0 +1,3684 @@ +// This file is MACHINE GENERATED! Do not edit. + +#ifndef TENSORFLOW_CC_OPS_NN_OPS_H_ +#define TENSORFLOW_CC_OPS_NN_OPS_H_ + +// This file is MACHINE GENERATED! Do not edit. + +#include "tensorflow/cc/framework/ops.h" +#include "tensorflow/cc/framework/scope.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/gtl/array_slice.h" + +namespace tensorflow { +namespace ops { + +/// @defgroup nn_ops Nn Ops +/// @{ + +/// Performs average pooling on the input. +/// +/// Each entry in `output` is the mean of the corresponding size `ksize` +/// window in `value`. +/// +/// Arguments: +/// * scope: A Scope object +/// * value: 4-D with shape `[batch, height, width, channels]`. +/// * ksize: The size of the sliding window for each dimension of `value`. +/// * strides: The stride of the sliding window for each dimension of `value`. +/// * padding: The type of padding algorithm to use. +/// +/// Optional attributes (see `Attrs`): +/// * data_format: Specify the data format of the input and output data. With the +/// default format "NHWC", the data is stored in the order of: +/// [batch, in_height, in_width, in_channels]. +/// Alternatively, the format could be "NCHW", the data storage order of: +/// [batch, in_channels, in_height, in_width]. +/// +/// Returns: +/// * `Output`: The average pooled output tensor. +class AvgPool { + public: + /// Optional attribute setters for AvgPool + struct Attrs { + /// Specify the data format of the input and output data. With the + /// default format "NHWC", the data is stored in the order of: + /// [batch, in_height, in_width, in_channels]. + /// Alternatively, the format could be "NCHW", the data storage order of: + /// [batch, in_channels, in_height, in_width]. + /// + /// Defaults to "NHWC" + TF_MUST_USE_RESULT Attrs DataFormat(StringPiece x) { + Attrs ret = *this; + ret.data_format_ = x; + return ret; + } + + StringPiece data_format_ = "NHWC"; + }; + AvgPool(const ::tensorflow::Scope& scope, ::tensorflow::Input value, const + gtl::ArraySlice& ksize, const gtl::ArraySlice& strides, + StringPiece padding); + AvgPool(const ::tensorflow::Scope& scope, ::tensorflow::Input value, const + gtl::ArraySlice& ksize, const gtl::ArraySlice& strides, + StringPiece padding, const AvgPool::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs DataFormat(StringPiece x) { + return Attrs().DataFormat(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Performs 3D average pooling on the input. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: Shape `[batch, depth, rows, cols, channels]` tensor to pool over. +/// * ksize: 1-D tensor of length 5. The size of the window for each dimension of +/// the input tensor. Must have `ksize[0] = ksize[4] = 1`. +/// * strides: 1-D tensor of length 5. The stride of the sliding window for each +/// dimension of `input`. Must have `strides[0] = strides[4] = 1`. +/// * padding: The type of padding algorithm to use. +/// +/// Optional attributes (see `Attrs`): +/// * data_format: The data format of the input and output data. With the +/// default format "NDHWC", the data is stored in the order of: +/// [batch, in_depth, in_height, in_width, in_channels]. +/// Alternatively, the format could be "NCDHW", the data storage order is: +/// [batch, in_channels, in_depth, in_height, in_width]. +/// +/// Returns: +/// * `Output`: The average pooled output tensor. +class AvgPool3D { + public: + /// Optional attribute setters for AvgPool3D + struct Attrs { + /// The data format of the input and output data. With the + /// default format "NDHWC", the data is stored in the order of: + /// [batch, in_depth, in_height, in_width, in_channels]. + /// Alternatively, the format could be "NCDHW", the data storage order is: + /// [batch, in_channels, in_depth, in_height, in_width]. + /// + /// Defaults to "NDHWC" + TF_MUST_USE_RESULT Attrs DataFormat(StringPiece x) { + Attrs ret = *this; + ret.data_format_ = x; + return ret; + } + + StringPiece data_format_ = "NDHWC"; + }; + AvgPool3D(const ::tensorflow::Scope& scope, ::tensorflow::Input input, const + gtl::ArraySlice& ksize, const gtl::ArraySlice& strides, + StringPiece padding); + AvgPool3D(const ::tensorflow::Scope& scope, ::tensorflow::Input input, const + gtl::ArraySlice& ksize, const gtl::ArraySlice& strides, + StringPiece padding, const AvgPool3D::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs DataFormat(StringPiece x) { + return Attrs().DataFormat(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Computes gradients of average pooling function. +/// +/// Arguments: +/// * scope: A Scope object +/// * orig_input_shape: The original input dimensions. +/// * grad: Output backprop of shape `[batch, depth, rows, cols, channels]`. +/// * ksize: 1-D tensor of length 5. The size of the window for each dimension of +/// the input tensor. Must have `ksize[0] = ksize[4] = 1`. +/// * strides: 1-D tensor of length 5. The stride of the sliding window for each +/// dimension of `input`. Must have `strides[0] = strides[4] = 1`. +/// * padding: The type of padding algorithm to use. +/// +/// Optional attributes (see `Attrs`): +/// * data_format: The data format of the input and output data. With the +/// default format "NDHWC", the data is stored in the order of: +/// [batch, in_depth, in_height, in_width, in_channels]. +/// Alternatively, the format could be "NCDHW", the data storage order is: +/// [batch, in_channels, in_depth, in_height, in_width]. +/// +/// Returns: +/// * `Output`: The backprop for input. +class AvgPool3DGrad { + public: + /// Optional attribute setters for AvgPool3DGrad + struct Attrs { + /// The data format of the input and output data. With the + /// default format "NDHWC", the data is stored in the order of: + /// [batch, in_depth, in_height, in_width, in_channels]. + /// Alternatively, the format could be "NCDHW", the data storage order is: + /// [batch, in_channels, in_depth, in_height, in_width]. + /// + /// Defaults to "NDHWC" + TF_MUST_USE_RESULT Attrs DataFormat(StringPiece x) { + Attrs ret = *this; + ret.data_format_ = x; + return ret; + } + + StringPiece data_format_ = "NDHWC"; + }; + AvgPool3DGrad(const ::tensorflow::Scope& scope, ::tensorflow::Input + orig_input_shape, ::tensorflow::Input grad, const + gtl::ArraySlice& ksize, const gtl::ArraySlice& strides, + StringPiece padding); + AvgPool3DGrad(const ::tensorflow::Scope& scope, ::tensorflow::Input + orig_input_shape, ::tensorflow::Input grad, const + gtl::ArraySlice& ksize, const gtl::ArraySlice& strides, + StringPiece padding, const AvgPool3DGrad::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs DataFormat(StringPiece x) { + return Attrs().DataFormat(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Adds `bias` to `value`. +/// +/// This is a special case of `tf.add` where `bias` is restricted to be 1-D. +/// Broadcasting is supported, so `value` may have any number of dimensions. +/// +/// Arguments: +/// * scope: A Scope object +/// * value: Any number of dimensions. +/// * bias: 1-D with size the last dimension of `value`. +/// +/// Optional attributes (see `Attrs`): +/// * data_format: Specify the data format of the input and output data. With the +/// default format "NHWC", the bias tensor will be added to the last dimension +/// of the value tensor. +/// Alternatively, the format could be "NCHW", the data storage order of: +/// [batch, in_channels, in_height, in_width]. +/// The tensor will be added to "in_channels", the third-to-the-last +/// dimension. +/// +/// Returns: +/// * `Output`: Broadcasted sum of `value` and `bias`. +class BiasAdd { + public: + /// Optional attribute setters for BiasAdd + struct Attrs { + /// Specify the data format of the input and output data. With the + /// default format "NHWC", the bias tensor will be added to the last dimension + /// of the value tensor. + /// Alternatively, the format could be "NCHW", the data storage order of: + /// [batch, in_channels, in_height, in_width]. + /// The tensor will be added to "in_channels", the third-to-the-last + /// dimension. + /// + /// Defaults to "NHWC" + TF_MUST_USE_RESULT Attrs DataFormat(StringPiece x) { + Attrs ret = *this; + ret.data_format_ = x; + return ret; + } + + StringPiece data_format_ = "NHWC"; + }; + BiasAdd(const ::tensorflow::Scope& scope, ::tensorflow::Input value, + ::tensorflow::Input bias); + BiasAdd(const ::tensorflow::Scope& scope, ::tensorflow::Input value, + ::tensorflow::Input bias, const BiasAdd::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs DataFormat(StringPiece x) { + return Attrs().DataFormat(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// The backward operation for "BiasAdd" on the "bias" tensor. +/// +/// It accumulates all the values from out_backprop into the feature dimension. +/// For NHWC data format, the feature dimension is the last. For NCHW data format, +/// the feature dimension is the third-to-last. +/// +/// Arguments: +/// * scope: A Scope object +/// * out_backprop: Any number of dimensions. +/// +/// Optional attributes (see `Attrs`): +/// * data_format: Specify the data format of the input and output data. With the +/// default format "NHWC", the bias tensor will be added to the last dimension +/// of the value tensor. +/// Alternatively, the format could be "NCHW", the data storage order of: +/// [batch, in_channels, in_height, in_width]. +/// The tensor will be added to "in_channels", the third-to-the-last +/// dimension. +/// +/// Returns: +/// * `Output`: 1-D with size the feature dimension of `out_backprop`. +class BiasAddGrad { + public: + /// Optional attribute setters for BiasAddGrad + struct Attrs { + /// Specify the data format of the input and output data. With the + /// default format "NHWC", the bias tensor will be added to the last dimension + /// of the value tensor. + /// Alternatively, the format could be "NCHW", the data storage order of: + /// [batch, in_channels, in_height, in_width]. + /// The tensor will be added to "in_channels", the third-to-the-last + /// dimension. + /// + /// Defaults to "NHWC" + TF_MUST_USE_RESULT Attrs DataFormat(StringPiece x) { + Attrs ret = *this; + ret.data_format_ = x; + return ret; + } + + StringPiece data_format_ = "NHWC"; + }; + BiasAddGrad(const ::tensorflow::Scope& scope, ::tensorflow::Input out_backprop); + BiasAddGrad(const ::tensorflow::Scope& scope, ::tensorflow::Input out_backprop, + const BiasAddGrad::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs DataFormat(StringPiece x) { + return Attrs().DataFormat(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Computes a 2-D convolution given 4-D `input` and `filter` tensors. +/// +/// Given an input tensor of shape `[batch, in_height, in_width, in_channels]` +/// and a filter / kernel tensor of shape +/// `[filter_height, filter_width, in_channels, out_channels]`, this op +/// performs the following: +/// +/// 1. Flattens the filter to a 2-D matrix with shape +/// `[filter_height * filter_width * in_channels, output_channels]`. +/// 2. Extracts image patches from the input tensor to form a *virtual* +/// tensor of shape `[batch, out_height, out_width, +/// filter_height * filter_width * in_channels]`. +/// 3. For each patch, right-multiplies the filter matrix and the image patch +/// vector. +/// +/// In detail, with the default NHWC format, +/// +/// output[b, i, j, k] = +/// sum_{di, dj, q} input[b, strides[1] * i + di, strides[2] * j + dj, q] * +/// filter[di, dj, q, k] +/// +/// Must have `strides[0] = strides[3] = 1`. For the most common case of the same +/// horizontal and vertices strides, `strides = [1, stride, stride, 1]`. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: A 4-D tensor. The dimension order is interpreted according to the value +/// of `data_format`, see below for details. +/// * filter: A 4-D tensor of shape +/// `[filter_height, filter_width, in_channels, out_channels]` +/// * strides: 1-D tensor of length 4. The stride of the sliding window for each +/// dimension of `input`. The dimension order is determined by the value of +/// `data_format`, see below for details. +/// * padding: The type of padding algorithm to use. +/// +/// Optional attributes (see `Attrs`): +/// * data_format: Specify the data format of the input and output data. With the +/// default format "NHWC", the data is stored in the order of: +/// [batch, height, width, channels]. +/// Alternatively, the format could be "NCHW", the data storage order of: +/// [batch, channels, height, width]. +/// * dilations: 1-D tensor of length 4. The dilation factor for each dimension of +/// `input`. If set to k > 1, there will be k-1 skipped cells between each +/// filter element on that dimension. The dimension order is determined by the +/// value of `data_format`, see above for details. Dilations in the batch and +/// depth dimensions must be 1. +/// +/// Returns: +/// * `Output`: A 4-D tensor. The dimension order is determined by the value of +/// `data_format`, see below for details. +class Conv2D { + public: + /// Optional attribute setters for Conv2D + struct Attrs { + /// Defaults to true + TF_MUST_USE_RESULT Attrs UseCudnnOnGpu(bool x) { + Attrs ret = *this; + ret.use_cudnn_on_gpu_ = x; + return ret; + } + + /// Specify the data format of the input and output data. With the + /// default format "NHWC", the data is stored in the order of: + /// [batch, height, width, channels]. + /// Alternatively, the format could be "NCHW", the data storage order of: + /// [batch, channels, height, width]. + /// + /// Defaults to "NHWC" + TF_MUST_USE_RESULT Attrs DataFormat(StringPiece x) { + Attrs ret = *this; + ret.data_format_ = x; + return ret; + } + + /// 1-D tensor of length 4. The dilation factor for each dimension of + /// `input`. If set to k > 1, there will be k-1 skipped cells between each + /// filter element on that dimension. The dimension order is determined by the + /// value of `data_format`, see above for details. Dilations in the batch and + /// depth dimensions must be 1. + /// + /// Defaults to [1, 1, 1, 1] + TF_MUST_USE_RESULT Attrs Dilations(const gtl::ArraySlice& x) { + Attrs ret = *this; + ret.dilations_ = x; + return ret; + } + + bool use_cudnn_on_gpu_ = true; + StringPiece data_format_ = "NHWC"; + gtl::ArraySlice dilations_ = Default_dilations(); + private: + static gtl::ArraySlice Default_dilations() { + static const int kStorage[] = {1, 1, 1, 1}; + return gtl::ArraySlice(kStorage); + } + }; + Conv2D(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input filter, const gtl::ArraySlice& strides, + StringPiece padding); + Conv2D(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input filter, const gtl::ArraySlice& strides, + StringPiece padding, const Conv2D::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs UseCudnnOnGpu(bool x) { + return Attrs().UseCudnnOnGpu(x); + } + static Attrs DataFormat(StringPiece x) { + return Attrs().DataFormat(x); + } + static Attrs Dilations(const gtl::ArraySlice& x) { + return Attrs().Dilations(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Computes the gradients of convolution with respect to the filter. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: 4-D with shape `[batch, in_height, in_width, in_channels]`. +/// * filter_sizes: An integer vector representing the tensor shape of `filter`, +/// where `filter` is a 4-D +/// `[filter_height, filter_width, in_channels, out_channels]` tensor. +/// * out_backprop: 4-D with shape `[batch, out_height, out_width, out_channels]`. +/// Gradients w.r.t. the output of the convolution. +/// * strides: The stride of the sliding window for each dimension of the input +/// of the convolution. Must be in the same order as the dimension specified with +/// format. +/// * padding: The type of padding algorithm to use. +/// +/// Optional attributes (see `Attrs`): +/// * data_format: Specify the data format of the input and output data. With the +/// default format "NHWC", the data is stored in the order of: +/// [batch, in_height, in_width, in_channels]. +/// Alternatively, the format could be "NCHW", the data storage order of: +/// [batch, in_channels, in_height, in_width]. +/// * dilations: 1-D tensor of length 4. The dilation factor for each dimension of +/// `input`. If set to k > 1, there will be k-1 skipped cells between each filter +/// element on that dimension. The dimension order is determined by the value of +/// `data_format`, see above for details. Dilations in the batch and depth +/// dimensions must be 1. +/// +/// Returns: +/// * `Output`: 4-D with shape +/// `[filter_height, filter_width, in_channels, out_channels]`. Gradient w.r.t. +/// the `filter` input of the convolution. +class Conv2DBackpropFilter { + public: + /// Optional attribute setters for Conv2DBackpropFilter + struct Attrs { + /// Defaults to true + TF_MUST_USE_RESULT Attrs UseCudnnOnGpu(bool x) { + Attrs ret = *this; + ret.use_cudnn_on_gpu_ = x; + return ret; + } + + /// Specify the data format of the input and output data. With the + /// default format "NHWC", the data is stored in the order of: + /// [batch, in_height, in_width, in_channels]. + /// Alternatively, the format could be "NCHW", the data storage order of: + /// [batch, in_channels, in_height, in_width]. + /// + /// Defaults to "NHWC" + TF_MUST_USE_RESULT Attrs DataFormat(StringPiece x) { + Attrs ret = *this; + ret.data_format_ = x; + return ret; + } + + /// 1-D tensor of length 4. The dilation factor for each dimension of + /// `input`. If set to k > 1, there will be k-1 skipped cells between each filter + /// element on that dimension. The dimension order is determined by the value of + /// `data_format`, see above for details. Dilations in the batch and depth + /// dimensions must be 1. + /// + /// Defaults to [1, 1, 1, 1] + TF_MUST_USE_RESULT Attrs Dilations(const gtl::ArraySlice& x) { + Attrs ret = *this; + ret.dilations_ = x; + return ret; + } + + bool use_cudnn_on_gpu_ = true; + StringPiece data_format_ = "NHWC"; + gtl::ArraySlice dilations_ = Default_dilations(); + private: + static gtl::ArraySlice Default_dilations() { + static const int kStorage[] = {1, 1, 1, 1}; + return gtl::ArraySlice(kStorage); + } + }; + Conv2DBackpropFilter(const ::tensorflow::Scope& scope, ::tensorflow::Input + input, ::tensorflow::Input filter_sizes, + ::tensorflow::Input out_backprop, const + gtl::ArraySlice& strides, StringPiece padding); + Conv2DBackpropFilter(const ::tensorflow::Scope& scope, ::tensorflow::Input + input, ::tensorflow::Input filter_sizes, + ::tensorflow::Input out_backprop, const + gtl::ArraySlice& strides, StringPiece padding, const + Conv2DBackpropFilter::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs UseCudnnOnGpu(bool x) { + return Attrs().UseCudnnOnGpu(x); + } + static Attrs DataFormat(StringPiece x) { + return Attrs().DataFormat(x); + } + static Attrs Dilations(const gtl::ArraySlice& x) { + return Attrs().Dilations(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Computes the gradients of convolution with respect to the input. +/// +/// Arguments: +/// * scope: A Scope object +/// * input_sizes: An integer vector representing the shape of `input`, +/// where `input` is a 4-D `[batch, height, width, channels]` tensor. +/// * filter: 4-D with shape +/// `[filter_height, filter_width, in_channels, out_channels]`. +/// * out_backprop: 4-D with shape `[batch, out_height, out_width, out_channels]`. +/// Gradients w.r.t. the output of the convolution. +/// * strides: The stride of the sliding window for each dimension of the input +/// of the convolution. Must be in the same order as the dimension specified with +/// format. +/// * padding: The type of padding algorithm to use. +/// +/// Optional attributes (see `Attrs`): +/// * data_format: Specify the data format of the input and output data. With the +/// default format "NHWC", the data is stored in the order of: +/// [batch, in_height, in_width, in_channels]. +/// Alternatively, the format could be "NCHW", the data storage order of: +/// [batch, in_channels, in_height, in_width]. +/// * dilations: 1-D tensor of length 4. The dilation factor for each dimension of +/// `input`. If set to k > 1, there will be k-1 skipped cells between each filter +/// element on that dimension. The dimension order is determined by the value of +/// `data_format`, see above for details. Dilations in the batch and depth +/// dimensions must be 1. +/// +/// Returns: +/// * `Output`: 4-D with shape `[batch, in_height, in_width, in_channels]`. Gradient +/// w.r.t. the input of the convolution. +class Conv2DBackpropInput { + public: + /// Optional attribute setters for Conv2DBackpropInput + struct Attrs { + /// Defaults to true + TF_MUST_USE_RESULT Attrs UseCudnnOnGpu(bool x) { + Attrs ret = *this; + ret.use_cudnn_on_gpu_ = x; + return ret; + } + + /// Specify the data format of the input and output data. With the + /// default format "NHWC", the data is stored in the order of: + /// [batch, in_height, in_width, in_channels]. + /// Alternatively, the format could be "NCHW", the data storage order of: + /// [batch, in_channels, in_height, in_width]. + /// + /// Defaults to "NHWC" + TF_MUST_USE_RESULT Attrs DataFormat(StringPiece x) { + Attrs ret = *this; + ret.data_format_ = x; + return ret; + } + + /// 1-D tensor of length 4. The dilation factor for each dimension of + /// `input`. If set to k > 1, there will be k-1 skipped cells between each filter + /// element on that dimension. The dimension order is determined by the value of + /// `data_format`, see above for details. Dilations in the batch and depth + /// dimensions must be 1. + /// + /// Defaults to [1, 1, 1, 1] + TF_MUST_USE_RESULT Attrs Dilations(const gtl::ArraySlice& x) { + Attrs ret = *this; + ret.dilations_ = x; + return ret; + } + + bool use_cudnn_on_gpu_ = true; + StringPiece data_format_ = "NHWC"; + gtl::ArraySlice dilations_ = Default_dilations(); + private: + static gtl::ArraySlice Default_dilations() { + static const int kStorage[] = {1, 1, 1, 1}; + return gtl::ArraySlice(kStorage); + } + }; + Conv2DBackpropInput(const ::tensorflow::Scope& scope, ::tensorflow::Input + input_sizes, ::tensorflow::Input filter, + ::tensorflow::Input out_backprop, const + gtl::ArraySlice& strides, StringPiece padding); + Conv2DBackpropInput(const ::tensorflow::Scope& scope, ::tensorflow::Input + input_sizes, ::tensorflow::Input filter, + ::tensorflow::Input out_backprop, const + gtl::ArraySlice& strides, StringPiece padding, const + Conv2DBackpropInput::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs UseCudnnOnGpu(bool x) { + return Attrs().UseCudnnOnGpu(x); + } + static Attrs DataFormat(StringPiece x) { + return Attrs().DataFormat(x); + } + static Attrs Dilations(const gtl::ArraySlice& x) { + return Attrs().Dilations(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Computes a 3-D convolution given 5-D `input` and `filter` tensors. +/// +/// In signal processing, cross-correlation is a measure of similarity of +/// two waveforms as a function of a time-lag applied to one of them. This +/// is also known as a sliding dot product or sliding inner-product. +/// +/// Our Conv3D implements a form of cross-correlation. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: Shape `[batch, in_depth, in_height, in_width, in_channels]`. +/// * filter: Shape `[filter_depth, filter_height, filter_width, in_channels, +/// out_channels]`. `in_channels` must match between `input` and `filter`. +/// * strides: 1-D tensor of length 5. The stride of the sliding window for each +/// dimension of `input`. Must have `strides[0] = strides[4] = 1`. +/// * padding: The type of padding algorithm to use. +/// +/// Optional attributes (see `Attrs`): +/// * data_format: The data format of the input and output data. With the +/// default format "NDHWC", the data is stored in the order of: +/// [batch, in_depth, in_height, in_width, in_channels]. +/// Alternatively, the format could be "NCDHW", the data storage order is: +/// [batch, in_channels, in_depth, in_height, in_width]. +/// * dilations: 1-D tensor of length 5. The dilation factor for each dimension of +/// `input`. If set to k > 1, there will be k-1 skipped cells between each +/// filter element on that dimension. The dimension order is determined by the +/// value of `data_format`, see above for details. Dilations in the batch and +/// depth dimensions must be 1. +/// +/// Returns: +/// * `Output`: The output tensor. +class Conv3D { + public: + /// Optional attribute setters for Conv3D + struct Attrs { + /// The data format of the input and output data. With the + /// default format "NDHWC", the data is stored in the order of: + /// [batch, in_depth, in_height, in_width, in_channels]. + /// Alternatively, the format could be "NCDHW", the data storage order is: + /// [batch, in_channels, in_depth, in_height, in_width]. + /// + /// Defaults to "NDHWC" + TF_MUST_USE_RESULT Attrs DataFormat(StringPiece x) { + Attrs ret = *this; + ret.data_format_ = x; + return ret; + } + + /// 1-D tensor of length 5. The dilation factor for each dimension of + /// `input`. If set to k > 1, there will be k-1 skipped cells between each + /// filter element on that dimension. The dimension order is determined by the + /// value of `data_format`, see above for details. Dilations in the batch and + /// depth dimensions must be 1. + /// + /// Defaults to [1, 1, 1, 1, 1] + TF_MUST_USE_RESULT Attrs Dilations(const gtl::ArraySlice& x) { + Attrs ret = *this; + ret.dilations_ = x; + return ret; + } + + StringPiece data_format_ = "NDHWC"; + gtl::ArraySlice dilations_ = Default_dilations(); + private: + static gtl::ArraySlice Default_dilations() { + static const int kStorage[] = {1, 1, 1, 1, 1}; + return gtl::ArraySlice(kStorage); + } + }; + Conv3D(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input filter, const gtl::ArraySlice& strides, + StringPiece padding); + Conv3D(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input filter, const gtl::ArraySlice& strides, + StringPiece padding, const Conv3D::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs DataFormat(StringPiece x) { + return Attrs().DataFormat(x); + } + static Attrs Dilations(const gtl::ArraySlice& x) { + return Attrs().Dilations(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Computes the gradients of 3-D convolution with respect to the filter. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: Shape `[batch, depth, rows, cols, in_channels]`. +/// * filter_sizes: An integer vector representing the tensor shape of `filter`, +/// where `filter` is a 5-D +/// `[filter_depth, filter_height, filter_width, in_channels, out_channels]` +/// tensor. +/// * out_backprop: Backprop signal of shape `[batch, out_depth, out_rows, out_cols, +/// out_channels]`. +/// * strides: 1-D tensor of length 5. The stride of the sliding window for each +/// dimension of `input`. Must have `strides[0] = strides[4] = 1`. +/// * padding: The type of padding algorithm to use. +/// +/// Optional attributes (see `Attrs`): +/// * data_format: The data format of the input and output data. With the +/// default format "NDHWC", the data is stored in the order of: +/// [batch, in_depth, in_height, in_width, in_channels]. +/// Alternatively, the format could be "NCDHW", the data storage order is: +/// [batch, in_channels, in_depth, in_height, in_width]. +/// * dilations: 1-D tensor of length 5. The dilation factor for each dimension of +/// `input`. If set to k > 1, there will be k-1 skipped cells between each +/// filter element on that dimension. The dimension order is determined by the +/// value of `data_format`, see above for details. Dilations in the batch and +/// depth dimensions must be 1. +/// +/// Returns: +/// * `Output`: The output tensor. +class Conv3DBackpropFilterV2 { + public: + /// Optional attribute setters for Conv3DBackpropFilterV2 + struct Attrs { + /// The data format of the input and output data. With the + /// default format "NDHWC", the data is stored in the order of: + /// [batch, in_depth, in_height, in_width, in_channels]. + /// Alternatively, the format could be "NCDHW", the data storage order is: + /// [batch, in_channels, in_depth, in_height, in_width]. + /// + /// Defaults to "NDHWC" + TF_MUST_USE_RESULT Attrs DataFormat(StringPiece x) { + Attrs ret = *this; + ret.data_format_ = x; + return ret; + } + + /// 1-D tensor of length 5. The dilation factor for each dimension of + /// `input`. If set to k > 1, there will be k-1 skipped cells between each + /// filter element on that dimension. The dimension order is determined by the + /// value of `data_format`, see above for details. Dilations in the batch and + /// depth dimensions must be 1. + /// + /// Defaults to [1, 1, 1, 1, 1] + TF_MUST_USE_RESULT Attrs Dilations(const gtl::ArraySlice& x) { + Attrs ret = *this; + ret.dilations_ = x; + return ret; + } + + StringPiece data_format_ = "NDHWC"; + gtl::ArraySlice dilations_ = Default_dilations(); + private: + static gtl::ArraySlice Default_dilations() { + static const int kStorage[] = {1, 1, 1, 1, 1}; + return gtl::ArraySlice(kStorage); + } + }; + Conv3DBackpropFilterV2(const ::tensorflow::Scope& scope, ::tensorflow::Input + input, ::tensorflow::Input filter_sizes, + ::tensorflow::Input out_backprop, const + gtl::ArraySlice& strides, StringPiece padding); + Conv3DBackpropFilterV2(const ::tensorflow::Scope& scope, ::tensorflow::Input + input, ::tensorflow::Input filter_sizes, + ::tensorflow::Input out_backprop, const + gtl::ArraySlice& strides, StringPiece padding, + const Conv3DBackpropFilterV2::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs DataFormat(StringPiece x) { + return Attrs().DataFormat(x); + } + static Attrs Dilations(const gtl::ArraySlice& x) { + return Attrs().Dilations(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Computes the gradients of 3-D convolution with respect to the input. +/// +/// Arguments: +/// * scope: A Scope object +/// * input_sizes: An integer vector representing the tensor shape of `input`, +/// where `input` is a 5-D +/// `[batch, depth, rows, cols, in_channels]` tensor. +/// * filter: Shape `[depth, rows, cols, in_channels, out_channels]`. +/// `in_channels` must match between `input` and `filter`. +/// * out_backprop: Backprop signal of shape `[batch, out_depth, out_rows, out_cols, +/// out_channels]`. +/// * strides: 1-D tensor of length 5. The stride of the sliding window for each +/// dimension of `input`. Must have `strides[0] = strides[4] = 1`. +/// * padding: The type of padding algorithm to use. +/// +/// Optional attributes (see `Attrs`): +/// * data_format: The data format of the input and output data. With the +/// default format "NDHWC", the data is stored in the order of: +/// [batch, in_depth, in_height, in_width, in_channels]. +/// Alternatively, the format could be "NCDHW", the data storage order is: +/// [batch, in_channels, in_depth, in_height, in_width]. +/// * dilations: 1-D tensor of length 5. The dilation factor for each dimension of +/// `input`. If set to k > 1, there will be k-1 skipped cells between each +/// filter element on that dimension. The dimension order is determined by the +/// value of `data_format`, see above for details. Dilations in the batch and +/// depth dimensions must be 1. +/// +/// Returns: +/// * `Output`: The output tensor. +class Conv3DBackpropInputV2 { + public: + /// Optional attribute setters for Conv3DBackpropInputV2 + struct Attrs { + /// The data format of the input and output data. With the + /// default format "NDHWC", the data is stored in the order of: + /// [batch, in_depth, in_height, in_width, in_channels]. + /// Alternatively, the format could be "NCDHW", the data storage order is: + /// [batch, in_channels, in_depth, in_height, in_width]. + /// + /// Defaults to "NDHWC" + TF_MUST_USE_RESULT Attrs DataFormat(StringPiece x) { + Attrs ret = *this; + ret.data_format_ = x; + return ret; + } + + /// 1-D tensor of length 5. The dilation factor for each dimension of + /// `input`. If set to k > 1, there will be k-1 skipped cells between each + /// filter element on that dimension. The dimension order is determined by the + /// value of `data_format`, see above for details. Dilations in the batch and + /// depth dimensions must be 1. + /// + /// Defaults to [1, 1, 1, 1, 1] + TF_MUST_USE_RESULT Attrs Dilations(const gtl::ArraySlice& x) { + Attrs ret = *this; + ret.dilations_ = x; + return ret; + } + + StringPiece data_format_ = "NDHWC"; + gtl::ArraySlice dilations_ = Default_dilations(); + private: + static gtl::ArraySlice Default_dilations() { + static const int kStorage[] = {1, 1, 1, 1, 1}; + return gtl::ArraySlice(kStorage); + } + }; + Conv3DBackpropInputV2(const ::tensorflow::Scope& scope, ::tensorflow::Input + input_sizes, ::tensorflow::Input filter, + ::tensorflow::Input out_backprop, const + gtl::ArraySlice& strides, StringPiece padding); + Conv3DBackpropInputV2(const ::tensorflow::Scope& scope, ::tensorflow::Input + input_sizes, ::tensorflow::Input filter, + ::tensorflow::Input out_backprop, const + gtl::ArraySlice& strides, StringPiece padding, const + Conv3DBackpropInputV2::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs DataFormat(StringPiece x) { + return Attrs().DataFormat(x); + } + static Attrs Dilations(const gtl::ArraySlice& x) { + return Attrs().Dilations(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Returns the dimension index in the destination data format given the one in +/// +/// the source data format. +/// +/// Arguments: +/// * scope: A Scope object +/// * x: A Tensor with each element as a dimension index in source data format. +/// Must be in the range [-4, 4). +/// +/// Optional attributes (see `Attrs`): +/// * src_format: source data format. +/// * dst_format: destination data format. +/// +/// Returns: +/// * `Output`: A Tensor with each element as a dimension index in destination data format. +class DataFormatDimMap { + public: + /// Optional attribute setters for DataFormatDimMap + struct Attrs { + /// source data format. + /// + /// Defaults to "NHWC" + TF_MUST_USE_RESULT Attrs SrcFormat(StringPiece x) { + Attrs ret = *this; + ret.src_format_ = x; + return ret; + } + + /// destination data format. + /// + /// Defaults to "NCHW" + TF_MUST_USE_RESULT Attrs DstFormat(StringPiece x) { + Attrs ret = *this; + ret.dst_format_ = x; + return ret; + } + + StringPiece src_format_ = "NHWC"; + StringPiece dst_format_ = "NCHW"; + }; + DataFormatDimMap(const ::tensorflow::Scope& scope, ::tensorflow::Input x); + DataFormatDimMap(const ::tensorflow::Scope& scope, ::tensorflow::Input x, const + DataFormatDimMap::Attrs& attrs); + operator ::tensorflow::Output() const { return y; } + operator ::tensorflow::Input() const { return y; } + ::tensorflow::Node* node() const { return y.node(); } + + static Attrs SrcFormat(StringPiece x) { + return Attrs().SrcFormat(x); + } + static Attrs DstFormat(StringPiece x) { + return Attrs().DstFormat(x); + } + + Operation operation; + ::tensorflow::Output y; +}; + +/// Returns the permuted vector/tensor in the destination data format given the +/// +/// one in the source data format. +/// +/// Arguments: +/// * scope: A Scope object +/// * x: Vector of size 4 or Tensor of shape (4, 2) in source data format. +/// +/// Optional attributes (see `Attrs`): +/// * src_format: source data format. +/// * dst_format: destination data format. +/// +/// Returns: +/// * `Output`: Vector of size 4 or Tensor of shape (4, 2) in destination data format. +class DataFormatVecPermute { + public: + /// Optional attribute setters for DataFormatVecPermute + struct Attrs { + /// source data format. + /// + /// Defaults to "NHWC" + TF_MUST_USE_RESULT Attrs SrcFormat(StringPiece x) { + Attrs ret = *this; + ret.src_format_ = x; + return ret; + } + + /// destination data format. + /// + /// Defaults to "NCHW" + TF_MUST_USE_RESULT Attrs DstFormat(StringPiece x) { + Attrs ret = *this; + ret.dst_format_ = x; + return ret; + } + + StringPiece src_format_ = "NHWC"; + StringPiece dst_format_ = "NCHW"; + }; + DataFormatVecPermute(const ::tensorflow::Scope& scope, ::tensorflow::Input x); + DataFormatVecPermute(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + const DataFormatVecPermute::Attrs& attrs); + operator ::tensorflow::Output() const { return y; } + operator ::tensorflow::Input() const { return y; } + ::tensorflow::Node* node() const { return y.node(); } + + static Attrs SrcFormat(StringPiece x) { + return Attrs().SrcFormat(x); + } + static Attrs DstFormat(StringPiece x) { + return Attrs().DstFormat(x); + } + + Operation operation; + ::tensorflow::Output y; +}; + +/// Computes a 2-D depthwise convolution given 4-D `input` and `filter` tensors. +/// +/// Given an input tensor of shape `[batch, in_height, in_width, in_channels]` +/// and a filter / kernel tensor of shape +/// `[filter_height, filter_width, in_channels, channel_multiplier]`, containing +/// `in_channels` convolutional filters of depth 1, `depthwise_conv2d` applies +/// a different filter to each input channel (expanding from 1 channel to +/// `channel_multiplier` channels for each), then concatenates the results +/// together. Thus, the output has `in_channels * channel_multiplier` channels. +/// +/// ``` +/// for k in 0..in_channels-1 +/// for q in 0..channel_multiplier-1 +/// output[b, i, j, k * channel_multiplier + q] = +/// sum_{di, dj} input[b, strides[1] * i + di, strides[2] * j + dj, k] * +/// filter[di, dj, k, q] +/// ``` +/// +/// Must have `strides[0] = strides[3] = 1`. For the most common case of the same +/// horizontal and vertices strides, `strides = [1, stride, stride, 1]`. +/// +/// Arguments: +/// * scope: A Scope object +/// * strides: 1-D of length 4. The stride of the sliding window for each dimension +/// of `input`. +/// * padding: The type of padding algorithm to use. +/// +/// Optional attributes (see `Attrs`): +/// * data_format: Specify the data format of the input and output data. With the +/// default format "NHWC", the data is stored in the order of: +/// [batch, height, width, channels]. +/// Alternatively, the format could be "NCHW", the data storage order of: +/// [batch, channels, height, width]. +/// * dilations: 1-D tensor of length 4. The dilation factor for each dimension of +/// `input`. If set to k > 1, there will be k-1 skipped cells between each filter +/// element on that dimension. The dimension order is determined by the value of +/// `data_format`, see above for details. Dilations in the batch and depth +/// dimensions must be 1. +/// +/// Returns: +/// * `Output`: The output tensor. +class DepthwiseConv2dNative { + public: + /// Optional attribute setters for DepthwiseConv2dNative + struct Attrs { + /// Specify the data format of the input and output data. With the + /// default format "NHWC", the data is stored in the order of: + /// [batch, height, width, channels]. + /// Alternatively, the format could be "NCHW", the data storage order of: + /// [batch, channels, height, width]. + /// + /// Defaults to "NHWC" + TF_MUST_USE_RESULT Attrs DataFormat(StringPiece x) { + Attrs ret = *this; + ret.data_format_ = x; + return ret; + } + + /// 1-D tensor of length 4. The dilation factor for each dimension of + /// `input`. If set to k > 1, there will be k-1 skipped cells between each filter + /// element on that dimension. The dimension order is determined by the value of + /// `data_format`, see above for details. Dilations in the batch and depth + /// dimensions must be 1. + /// + /// Defaults to [1, 1, 1, 1] + TF_MUST_USE_RESULT Attrs Dilations(const gtl::ArraySlice& x) { + Attrs ret = *this; + ret.dilations_ = x; + return ret; + } + + StringPiece data_format_ = "NHWC"; + gtl::ArraySlice dilations_ = Default_dilations(); + private: + static gtl::ArraySlice Default_dilations() { + static const int kStorage[] = {1, 1, 1, 1}; + return gtl::ArraySlice(kStorage); + } + }; + DepthwiseConv2dNative(const ::tensorflow::Scope& scope, ::tensorflow::Input + input, ::tensorflow::Input filter, const + gtl::ArraySlice& strides, StringPiece padding); + DepthwiseConv2dNative(const ::tensorflow::Scope& scope, ::tensorflow::Input + input, ::tensorflow::Input filter, const + gtl::ArraySlice& strides, StringPiece padding, const + DepthwiseConv2dNative::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs DataFormat(StringPiece x) { + return Attrs().DataFormat(x); + } + static Attrs Dilations(const gtl::ArraySlice& x) { + return Attrs().Dilations(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Computes the gradients of depthwise convolution with respect to the filter. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: 4-D with shape based on `data_format`. For example, if +/// `data_format` is 'NHWC' then `input` is a 4-D `[batch, in_height, +/// in_width, in_channels]` tensor. +/// * filter_sizes: An integer vector representing the tensor shape of `filter`, +/// where `filter` is a 4-D +/// `[filter_height, filter_width, in_channels, depthwise_multiplier]` tensor. +/// * out_backprop: 4-D with shape based on `data_format`. +/// For example, if `data_format` is 'NHWC' then +/// out_backprop shape is `[batch, out_height, out_width, out_channels]`. +/// Gradients w.r.t. the output of the convolution. +/// * strides: The stride of the sliding window for each dimension of the input +/// of the convolution. +/// * padding: The type of padding algorithm to use. +/// +/// Optional attributes (see `Attrs`): +/// * data_format: Specify the data format of the input and output data. With the +/// default format "NHWC", the data is stored in the order of: +/// [batch, height, width, channels]. +/// Alternatively, the format could be "NCHW", the data storage order of: +/// [batch, channels, height, width]. +/// * dilations: 1-D tensor of length 4. The dilation factor for each dimension of +/// `input`. If set to k > 1, there will be k-1 skipped cells between each filter +/// element on that dimension. The dimension order is determined by the value of +/// `data_format`, see above for details. Dilations in the batch and depth +/// dimensions must be 1. +/// +/// Returns: +/// * `Output`: 4-D with shape +/// `[filter_height, filter_width, in_channels, out_channels]`. Gradient w.r.t. +/// the `filter` input of the convolution. +class DepthwiseConv2dNativeBackpropFilter { + public: + /// Optional attribute setters for DepthwiseConv2dNativeBackpropFilter + struct Attrs { + /// Specify the data format of the input and output data. With the + /// default format "NHWC", the data is stored in the order of: + /// [batch, height, width, channels]. + /// Alternatively, the format could be "NCHW", the data storage order of: + /// [batch, channels, height, width]. + /// + /// Defaults to "NHWC" + TF_MUST_USE_RESULT Attrs DataFormat(StringPiece x) { + Attrs ret = *this; + ret.data_format_ = x; + return ret; + } + + /// 1-D tensor of length 4. The dilation factor for each dimension of + /// `input`. If set to k > 1, there will be k-1 skipped cells between each filter + /// element on that dimension. The dimension order is determined by the value of + /// `data_format`, see above for details. Dilations in the batch and depth + /// dimensions must be 1. + /// + /// Defaults to [1, 1, 1, 1] + TF_MUST_USE_RESULT Attrs Dilations(const gtl::ArraySlice& x) { + Attrs ret = *this; + ret.dilations_ = x; + return ret; + } + + StringPiece data_format_ = "NHWC"; + gtl::ArraySlice dilations_ = Default_dilations(); + private: + static gtl::ArraySlice Default_dilations() { + static const int kStorage[] = {1, 1, 1, 1}; + return gtl::ArraySlice(kStorage); + } + }; + DepthwiseConv2dNativeBackpropFilter(const ::tensorflow::Scope& scope, + ::tensorflow::Input input, + ::tensorflow::Input filter_sizes, + ::tensorflow::Input out_backprop, const + gtl::ArraySlice& strides, StringPiece + padding); + DepthwiseConv2dNativeBackpropFilter(const ::tensorflow::Scope& scope, + ::tensorflow::Input input, + ::tensorflow::Input filter_sizes, + ::tensorflow::Input out_backprop, const + gtl::ArraySlice& strides, StringPiece + padding, const + DepthwiseConv2dNativeBackpropFilter::Attrs& + attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs DataFormat(StringPiece x) { + return Attrs().DataFormat(x); + } + static Attrs Dilations(const gtl::ArraySlice& x) { + return Attrs().Dilations(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Computes the gradients of depthwise convolution with respect to the input. +/// +/// Arguments: +/// * scope: A Scope object +/// * input_sizes: An integer vector representing the shape of `input`, based +/// on `data_format`. For example, if `data_format` is 'NHWC' then +/// `input` is a 4-D `[batch, height, width, channels]` tensor. +/// * filter: 4-D with shape +/// `[filter_height, filter_width, in_channels, depthwise_multiplier]`. +/// * out_backprop: 4-D with shape based on `data_format`. +/// For example, if `data_format` is 'NHWC' then +/// out_backprop shape is `[batch, out_height, out_width, out_channels]`. +/// Gradients w.r.t. the output of the convolution. +/// * strides: The stride of the sliding window for each dimension of the input +/// of the convolution. +/// * padding: The type of padding algorithm to use. +/// +/// Optional attributes (see `Attrs`): +/// * data_format: Specify the data format of the input and output data. With the +/// default format "NHWC", the data is stored in the order of: +/// [batch, height, width, channels]. +/// Alternatively, the format could be "NCHW", the data storage order of: +/// [batch, channels, height, width]. +/// * dilations: 1-D tensor of length 4. The dilation factor for each dimension of +/// `input`. If set to k > 1, there will be k-1 skipped cells between each filter +/// element on that dimension. The dimension order is determined by the value of +/// `data_format`, see above for details. Dilations in the batch and depth +/// dimensions must be 1. +/// +/// Returns: +/// * `Output`: 4-D with shape according to `data_format`. For example, if +/// `data_format` is 'NHWC', output shape is `[batch, in_height, +/// in_width, in_channels]`. Gradient w.r.t. the input of the +/// convolution. +class DepthwiseConv2dNativeBackpropInput { + public: + /// Optional attribute setters for DepthwiseConv2dNativeBackpropInput + struct Attrs { + /// Specify the data format of the input and output data. With the + /// default format "NHWC", the data is stored in the order of: + /// [batch, height, width, channels]. + /// Alternatively, the format could be "NCHW", the data storage order of: + /// [batch, channels, height, width]. + /// + /// Defaults to "NHWC" + TF_MUST_USE_RESULT Attrs DataFormat(StringPiece x) { + Attrs ret = *this; + ret.data_format_ = x; + return ret; + } + + /// 1-D tensor of length 4. The dilation factor for each dimension of + /// `input`. If set to k > 1, there will be k-1 skipped cells between each filter + /// element on that dimension. The dimension order is determined by the value of + /// `data_format`, see above for details. Dilations in the batch and depth + /// dimensions must be 1. + /// + /// Defaults to [1, 1, 1, 1] + TF_MUST_USE_RESULT Attrs Dilations(const gtl::ArraySlice& x) { + Attrs ret = *this; + ret.dilations_ = x; + return ret; + } + + StringPiece data_format_ = "NHWC"; + gtl::ArraySlice dilations_ = Default_dilations(); + private: + static gtl::ArraySlice Default_dilations() { + static const int kStorage[] = {1, 1, 1, 1}; + return gtl::ArraySlice(kStorage); + } + }; + DepthwiseConv2dNativeBackpropInput(const ::tensorflow::Scope& scope, + ::tensorflow::Input input_sizes, + ::tensorflow::Input filter, + ::tensorflow::Input out_backprop, const + gtl::ArraySlice& strides, StringPiece + padding); + DepthwiseConv2dNativeBackpropInput(const ::tensorflow::Scope& scope, + ::tensorflow::Input input_sizes, + ::tensorflow::Input filter, + ::tensorflow::Input out_backprop, const + gtl::ArraySlice& strides, StringPiece + padding, const + DepthwiseConv2dNativeBackpropInput::Attrs& + attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs DataFormat(StringPiece x) { + return Attrs().DataFormat(x); + } + static Attrs Dilations(const gtl::ArraySlice& x) { + return Attrs().Dilations(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Computes the grayscale dilation of 4-D `input` and 3-D `filter` tensors. +/// +/// The `input` tensor has shape `[batch, in_height, in_width, depth]` and the +/// `filter` tensor has shape `[filter_height, filter_width, depth]`, i.e., each +/// input channel is processed independently of the others with its own structuring +/// function. The `output` tensor has shape +/// `[batch, out_height, out_width, depth]`. The spatial dimensions of the output +/// tensor depend on the `padding` algorithm. We currently only support the default +/// "NHWC" `data_format`. +/// +/// In detail, the grayscale morphological 2-D dilation is the max-sum correlation +/// (for consistency with `conv2d`, we use unmirrored filters): +/// +/// output[b, y, x, c] = +/// max_{dy, dx} input[b, +/// strides[1] * y + rates[1] * dy, +/// strides[2] * x + rates[2] * dx, +/// c] + +/// filter[dy, dx, c] +/// +/// Max-pooling is a special case when the filter has size equal to the pooling +/// kernel size and contains all zeros. +/// +/// Note on duality: The dilation of `input` by the `filter` is equal to the +/// negation of the erosion of `-input` by the reflected `filter`. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: 4-D with shape `[batch, in_height, in_width, depth]`. +/// * filter: 3-D with shape `[filter_height, filter_width, depth]`. +/// * strides: The stride of the sliding window for each dimension of the input +/// tensor. Must be: `[1, stride_height, stride_width, 1]`. +/// * rates: The input stride for atrous morphological dilation. Must be: +/// `[1, rate_height, rate_width, 1]`. +/// * padding: The type of padding algorithm to use. +/// +/// Returns: +/// * `Output`: 4-D with shape `[batch, out_height, out_width, depth]`. +class Dilation2D { + public: + Dilation2D(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input filter, const gtl::ArraySlice& strides, + const gtl::ArraySlice& rates, StringPiece padding); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Computes the gradient of morphological 2-D dilation with respect to the filter. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: 4-D with shape `[batch, in_height, in_width, depth]`. +/// * filter: 3-D with shape `[filter_height, filter_width, depth]`. +/// * out_backprop: 4-D with shape `[batch, out_height, out_width, depth]`. +/// * strides: 1-D of length 4. The stride of the sliding window for each dimension of +/// the input tensor. Must be: `[1, stride_height, stride_width, 1]`. +/// * rates: 1-D of length 4. The input stride for atrous morphological dilation. +/// Must be: `[1, rate_height, rate_width, 1]`. +/// * padding: The type of padding algorithm to use. +/// +/// Returns: +/// * `Output`: 3-D with shape `[filter_height, filter_width, depth]`. +class Dilation2DBackpropFilter { + public: + Dilation2DBackpropFilter(const ::tensorflow::Scope& scope, ::tensorflow::Input + input, ::tensorflow::Input filter, ::tensorflow::Input + out_backprop, const gtl::ArraySlice& strides, + const gtl::ArraySlice& rates, StringPiece + padding); + operator ::tensorflow::Output() const { return filter_backprop; } + operator ::tensorflow::Input() const { return filter_backprop; } + ::tensorflow::Node* node() const { return filter_backprop.node(); } + + Operation operation; + ::tensorflow::Output filter_backprop; +}; + +/// Computes the gradient of morphological 2-D dilation with respect to the input. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: 4-D with shape `[batch, in_height, in_width, depth]`. +/// * filter: 3-D with shape `[filter_height, filter_width, depth]`. +/// * out_backprop: 4-D with shape `[batch, out_height, out_width, depth]`. +/// * strides: 1-D of length 4. The stride of the sliding window for each dimension of +/// the input tensor. Must be: `[1, stride_height, stride_width, 1]`. +/// * rates: 1-D of length 4. The input stride for atrous morphological dilation. +/// Must be: `[1, rate_height, rate_width, 1]`. +/// * padding: The type of padding algorithm to use. +/// +/// Returns: +/// * `Output`: 4-D with shape `[batch, in_height, in_width, depth]`. +class Dilation2DBackpropInput { + public: + Dilation2DBackpropInput(const ::tensorflow::Scope& scope, ::tensorflow::Input + input, ::tensorflow::Input filter, ::tensorflow::Input + out_backprop, const gtl::ArraySlice& strides, + const gtl::ArraySlice& rates, StringPiece padding); + operator ::tensorflow::Output() const { return in_backprop; } + operator ::tensorflow::Input() const { return in_backprop; } + ::tensorflow::Node* node() const { return in_backprop.node(); } + + Operation operation; + ::tensorflow::Output in_backprop; +}; + +/// Computes exponential linear: `exp(features) - 1` if < 0, `features` otherwise. +/// +/// See [Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs) +/// ](http://arxiv.org/abs/1511.07289) +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The activations tensor. +class Elu { + public: + Elu(const ::tensorflow::Scope& scope, ::tensorflow::Input features); + operator ::tensorflow::Output() const { return activations; } + operator ::tensorflow::Input() const { return activations; } + ::tensorflow::Node* node() const { return activations.node(); } + + Operation operation; + ::tensorflow::Output activations; +}; + +/// Performs fractional average pooling on the input. +/// +/// Fractional average pooling is similar to Fractional max pooling in the pooling +/// region generation step. The only difference is that after pooling regions are +/// generated, a mean operation is performed instead of a max operation in each +/// pooling region. +/// +/// Arguments: +/// * scope: A Scope object +/// * value: 4-D with shape `[batch, height, width, channels]`. +/// * pooling_ratio: Pooling ratio for each dimension of `value`, currently only +/// supports row and col dimension and should be >= 1.0. For example, a valid +/// pooling ratio looks like [1.0, 1.44, 1.73, 1.0]. The first and last elements +/// must be 1.0 because we don't allow pooling on batch and channels +/// dimensions. 1.44 and 1.73 are pooling ratio on height and width dimensions +/// respectively. +/// +/// Optional attributes (see `Attrs`): +/// * pseudo_random: When set to True, generates the pooling sequence in a +/// pseudorandom fashion, otherwise, in a random fashion. Check paper [Benjamin +/// Graham, Fractional Max-Pooling](http://arxiv.org/abs/1412.6071) for +/// difference between pseudorandom and random. +/// * overlapping: When set to True, it means when pooling, the values at the boundary +/// of adjacent pooling cells are used by both cells. For example: +/// +/// `index 0 1 2 3 4` +/// +/// `value 20 5 16 3 7` +/// +/// If the pooling sequence is [0, 2, 4], then 16, at index 2 will be used twice. +/// The result would be [41/3, 26/3] for fractional avg pooling. +/// * deterministic: When set to True, a fixed pooling region will be used when +/// iterating over a FractionalAvgPool node in the computation graph. Mainly used +/// in unit test to make FractionalAvgPool deterministic. +/// * seed: If either seed or seed2 are set to be non-zero, the random number +/// generator is seeded by the given seed. Otherwise, it is seeded by a +/// random seed. +/// * seed2: An second seed to avoid seed collision. +/// +/// Returns: +/// * `Output` output: output tensor after fractional avg pooling. +/// * `Output` row_pooling_sequence: row pooling sequence, needed to calculate gradient. +/// * `Output` col_pooling_sequence: column pooling sequence, needed to calculate gradient. +class FractionalAvgPool { + public: + /// Optional attribute setters for FractionalAvgPool + struct Attrs { + /// When set to True, generates the pooling sequence in a + /// pseudorandom fashion, otherwise, in a random fashion. Check paper [Benjamin + /// Graham, Fractional Max-Pooling](http://arxiv.org/abs/1412.6071) for + /// difference between pseudorandom and random. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs PseudoRandom(bool x) { + Attrs ret = *this; + ret.pseudo_random_ = x; + return ret; + } + + /// When set to True, it means when pooling, the values at the boundary + /// of adjacent pooling cells are used by both cells. For example: + /// + /// `index 0 1 2 3 4` + /// + /// `value 20 5 16 3 7` + /// + /// If the pooling sequence is [0, 2, 4], then 16, at index 2 will be used twice. + /// The result would be [41/3, 26/3] for fractional avg pooling. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs Overlapping(bool x) { + Attrs ret = *this; + ret.overlapping_ = x; + return ret; + } + + /// When set to True, a fixed pooling region will be used when + /// iterating over a FractionalAvgPool node in the computation graph. Mainly used + /// in unit test to make FractionalAvgPool deterministic. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs Deterministic(bool x) { + Attrs ret = *this; + ret.deterministic_ = x; + return ret; + } + + /// If either seed or seed2 are set to be non-zero, the random number + /// generator is seeded by the given seed. Otherwise, it is seeded by a + /// random seed. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Seed(int64 x) { + Attrs ret = *this; + ret.seed_ = x; + return ret; + } + + /// An second seed to avoid seed collision. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Seed2(int64 x) { + Attrs ret = *this; + ret.seed2_ = x; + return ret; + } + + bool pseudo_random_ = false; + bool overlapping_ = false; + bool deterministic_ = false; + int64 seed_ = 0; + int64 seed2_ = 0; + }; + FractionalAvgPool(const ::tensorflow::Scope& scope, ::tensorflow::Input value, + const gtl::ArraySlice& pooling_ratio); + FractionalAvgPool(const ::tensorflow::Scope& scope, ::tensorflow::Input value, + const gtl::ArraySlice& pooling_ratio, const + FractionalAvgPool::Attrs& attrs); + + static Attrs PseudoRandom(bool x) { + return Attrs().PseudoRandom(x); + } + static Attrs Overlapping(bool x) { + return Attrs().Overlapping(x); + } + static Attrs Deterministic(bool x) { + return Attrs().Deterministic(x); + } + static Attrs Seed(int64 x) { + return Attrs().Seed(x); + } + static Attrs Seed2(int64 x) { + return Attrs().Seed2(x); + } + + Operation operation; + ::tensorflow::Output output; + ::tensorflow::Output row_pooling_sequence; + ::tensorflow::Output col_pooling_sequence; +}; + +/// Performs fractional max pooling on the input. +/// +/// Fractional max pooling is slightly different than regular max pooling. In +/// regular max pooling, you downsize an input set by taking the maximum value of +/// smaller N x N subsections of the set (often 2x2), and try to reduce the set by +/// a factor of N, where N is an integer. Fractional max pooling, as you might +/// expect from the word "fractional", means that the overall reduction ratio N +/// does not have to be an integer. +/// +/// The sizes of the pooling regions are generated randomly but are fairly uniform. +/// For example, let's look at the height dimension, and the constraints on the +/// list of rows that will be pool boundaries. +/// +/// First we define the following: +/// +/// 1. input_row_length : the number of rows from the input set +/// 2. output_row_length : which will be smaller than the input +/// 3. alpha = input_row_length / output_row_length : our reduction ratio +/// 4. K = floor(alpha) +/// 5. row_pooling_sequence : this is the result list of pool boundary rows +/// +/// Then, row_pooling_sequence should satisfy: +/// +/// 1. a[0] = 0 : the first value of the sequence is 0 +/// 2. a[end] = input_row_length : the last value of the sequence is the size +/// 3. K <= (a[i+1] - a[i]) <= K+1 : all intervals are K or K+1 size +/// 4. length(row_pooling_sequence) = output_row_length+1 +/// +/// For more details on fractional max pooling, see this paper: +/// [Benjamin Graham, Fractional Max-Pooling](http://arxiv.org/abs/1412.6071) +/// +/// Arguments: +/// * scope: A Scope object +/// * value: 4-D with shape `[batch, height, width, channels]`. +/// * pooling_ratio: Pooling ratio for each dimension of `value`, currently only +/// supports row and col dimension and should be >= 1.0. For example, a valid +/// pooling ratio looks like [1.0, 1.44, 1.73, 1.0]. The first and last elements +/// must be 1.0 because we don't allow pooling on batch and channels +/// dimensions. 1.44 and 1.73 are pooling ratio on height and width dimensions +/// respectively. +/// +/// Optional attributes (see `Attrs`): +/// * pseudo_random: When set to True, generates the pooling sequence in a +/// pseudorandom fashion, otherwise, in a random fashion. Check paper [Benjamin +/// Graham, Fractional Max-Pooling](http://arxiv.org/abs/1412.6071) for +/// difference between pseudorandom and random. +/// * overlapping: When set to True, it means when pooling, the values at the boundary +/// of adjacent pooling cells are used by both cells. For example: +/// +/// `index 0 1 2 3 4` +/// +/// `value 20 5 16 3 7` +/// +/// If the pooling sequence is [0, 2, 4], then 16, at index 2 will be used twice. +/// The result would be [20, 16] for fractional max pooling. +/// * deterministic: When set to True, a fixed pooling region will be used when +/// iterating over a FractionalMaxPool node in the computation graph. Mainly used +/// in unit test to make FractionalMaxPool deterministic. +/// * seed: If either seed or seed2 are set to be non-zero, the random number +/// generator is seeded by the given seed. Otherwise, it is seeded by a +/// random seed. +/// * seed2: An second seed to avoid seed collision. +/// +/// Returns: +/// * `Output` output: output tensor after fractional max pooling. +/// * `Output` row_pooling_sequence: row pooling sequence, needed to calculate gradient. +/// * `Output` col_pooling_sequence: column pooling sequence, needed to calculate gradient. +class FractionalMaxPool { + public: + /// Optional attribute setters for FractionalMaxPool + struct Attrs { + /// When set to True, generates the pooling sequence in a + /// pseudorandom fashion, otherwise, in a random fashion. Check paper [Benjamin + /// Graham, Fractional Max-Pooling](http://arxiv.org/abs/1412.6071) for + /// difference between pseudorandom and random. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs PseudoRandom(bool x) { + Attrs ret = *this; + ret.pseudo_random_ = x; + return ret; + } + + /// When set to True, it means when pooling, the values at the boundary + /// of adjacent pooling cells are used by both cells. For example: + /// + /// `index 0 1 2 3 4` + /// + /// `value 20 5 16 3 7` + /// + /// If the pooling sequence is [0, 2, 4], then 16, at index 2 will be used twice. + /// The result would be [20, 16] for fractional max pooling. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs Overlapping(bool x) { + Attrs ret = *this; + ret.overlapping_ = x; + return ret; + } + + /// When set to True, a fixed pooling region will be used when + /// iterating over a FractionalMaxPool node in the computation graph. Mainly used + /// in unit test to make FractionalMaxPool deterministic. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs Deterministic(bool x) { + Attrs ret = *this; + ret.deterministic_ = x; + return ret; + } + + /// If either seed or seed2 are set to be non-zero, the random number + /// generator is seeded by the given seed. Otherwise, it is seeded by a + /// random seed. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Seed(int64 x) { + Attrs ret = *this; + ret.seed_ = x; + return ret; + } + + /// An second seed to avoid seed collision. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Seed2(int64 x) { + Attrs ret = *this; + ret.seed2_ = x; + return ret; + } + + bool pseudo_random_ = false; + bool overlapping_ = false; + bool deterministic_ = false; + int64 seed_ = 0; + int64 seed2_ = 0; + }; + FractionalMaxPool(const ::tensorflow::Scope& scope, ::tensorflow::Input value, + const gtl::ArraySlice& pooling_ratio); + FractionalMaxPool(const ::tensorflow::Scope& scope, ::tensorflow::Input value, + const gtl::ArraySlice& pooling_ratio, const + FractionalMaxPool::Attrs& attrs); + + static Attrs PseudoRandom(bool x) { + return Attrs().PseudoRandom(x); + } + static Attrs Overlapping(bool x) { + return Attrs().Overlapping(x); + } + static Attrs Deterministic(bool x) { + return Attrs().Deterministic(x); + } + static Attrs Seed(int64 x) { + return Attrs().Seed(x); + } + static Attrs Seed2(int64 x) { + return Attrs().Seed2(x); + } + + Operation operation; + ::tensorflow::Output output; + ::tensorflow::Output row_pooling_sequence; + ::tensorflow::Output col_pooling_sequence; +}; + +/// Batch normalization. +/// +/// Note that the size of 4D Tensors are defined by either "NHWC" or "NCHW". +/// The size of 1D Tensors matches the dimension C of the 4D Tensors. +/// +/// Arguments: +/// * scope: A Scope object +/// * x: A 4D Tensor for input data. +/// * scale: A 1D Tensor for scaling factor, to scale the normalized x. +/// * offset: A 1D Tensor for offset, to shift to the normalized x. +/// * mean: A 1D Tensor for population mean. Used for inference only; +/// must be empty for training. +/// * variance: A 1D Tensor for population variance. Used for inference only; +/// must be empty for training. +/// +/// Optional attributes (see `Attrs`): +/// * epsilon: A small float number added to the variance of x. +/// * data_format: The data format for x and y. Either "NHWC" (default) or "NCHW". +/// * is_training: A bool value to indicate the operation is for training (default) +/// or inference. +/// +/// Returns: +/// * `Output` y: A 4D Tensor for output data. +/// * `Output` batch_mean: A 1D Tensor for the computed batch mean, to be used by TensorFlow +/// to compute the running mean. +/// * `Output` batch_variance: A 1D Tensor for the computed batch variance, to be used by +/// TensorFlow to compute the running variance. +/// * `Output` reserve_space_1: A 1D Tensor for the computed batch mean, to be reused +/// in the gradient computation. +/// * `Output` reserve_space_2: A 1D Tensor for the computed batch variance (inverted variance +/// in the cuDNN case), to be reused in the gradient computation. +class FusedBatchNorm { + public: + /// Optional attribute setters for FusedBatchNorm + struct Attrs { + /// A small float number added to the variance of x. + /// + /// Defaults to 0.0001 + TF_MUST_USE_RESULT Attrs Epsilon(float x) { + Attrs ret = *this; + ret.epsilon_ = x; + return ret; + } + + /// The data format for x and y. Either "NHWC" (default) or "NCHW". + /// + /// Defaults to "NHWC" + TF_MUST_USE_RESULT Attrs DataFormat(StringPiece x) { + Attrs ret = *this; + ret.data_format_ = x; + return ret; + } + + /// A bool value to indicate the operation is for training (default) + /// or inference. + /// + /// Defaults to true + TF_MUST_USE_RESULT Attrs IsTraining(bool x) { + Attrs ret = *this; + ret.is_training_ = x; + return ret; + } + + float epsilon_ = 0.0001f; + StringPiece data_format_ = "NHWC"; + bool is_training_ = true; + }; + FusedBatchNorm(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input scale, ::tensorflow::Input offset, + ::tensorflow::Input mean, ::tensorflow::Input variance); + FusedBatchNorm(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input scale, ::tensorflow::Input offset, + ::tensorflow::Input mean, ::tensorflow::Input variance, const + FusedBatchNorm::Attrs& attrs); + + static Attrs Epsilon(float x) { + return Attrs().Epsilon(x); + } + static Attrs DataFormat(StringPiece x) { + return Attrs().DataFormat(x); + } + static Attrs IsTraining(bool x) { + return Attrs().IsTraining(x); + } + + Operation operation; + ::tensorflow::Output y; + ::tensorflow::Output batch_mean; + ::tensorflow::Output batch_variance; + ::tensorflow::Output reserve_space_1; + ::tensorflow::Output reserve_space_2; +}; + +/// Gradient for batch normalization. +/// +/// Note that the size of 4D Tensors are defined by either "NHWC" or "NCHW". +/// The size of 1D Tensors matches the dimension C of the 4D Tensors. +/// +/// Arguments: +/// * scope: A Scope object +/// * y_backprop: A 4D Tensor for the gradient with respect to y. +/// * x: A 4D Tensor for input data. +/// * scale: A 1D Tensor for scaling factor, to scale the normalized x. +/// * reserve_space_1: When is_training is True, a 1D Tensor for the computed batch +/// mean to be reused in gradient computation. When is_training is +/// False, a 1D Tensor for the population mean to be reused in both +/// 1st and 2nd order gradient computation. +/// * reserve_space_2: When is_training is True, a 1D Tensor for the computed batch +/// variance (inverted variance in the cuDNN case) to be reused in +/// gradient computation. When is_training is False, a 1D Tensor +/// for the population variance to be reused in both 1st and 2nd +/// order gradient computation. +/// +/// Optional attributes (see `Attrs`): +/// * epsilon: A small float number added to the variance of x. +/// * data_format: The data format for y_backprop, x, x_backprop. +/// Either "NHWC" (default) or "NCHW". +/// * is_training: A bool value to indicate the operation is for training (default) +/// or inference. +/// +/// Returns: +/// * `Output` x_backprop: A 4D Tensor for the gradient with respect to x. +/// * `Output` scale_backprop: A 1D Tensor for the gradient with respect to scale. +/// * `Output` offset_backprop: A 1D Tensor for the gradient with respect to offset. +/// * `Output` reserve_space_3: Unused placeholder to match the mean input in FusedBatchNorm. +/// * `Output` reserve_space_4: Unused placeholder to match the variance input +/// in FusedBatchNorm. +class FusedBatchNormGrad { + public: + /// Optional attribute setters for FusedBatchNormGrad + struct Attrs { + /// A small float number added to the variance of x. + /// + /// Defaults to 0.0001 + TF_MUST_USE_RESULT Attrs Epsilon(float x) { + Attrs ret = *this; + ret.epsilon_ = x; + return ret; + } + + /// The data format for y_backprop, x, x_backprop. + /// Either "NHWC" (default) or "NCHW". + /// + /// Defaults to "NHWC" + TF_MUST_USE_RESULT Attrs DataFormat(StringPiece x) { + Attrs ret = *this; + ret.data_format_ = x; + return ret; + } + + /// A bool value to indicate the operation is for training (default) + /// or inference. + /// + /// Defaults to true + TF_MUST_USE_RESULT Attrs IsTraining(bool x) { + Attrs ret = *this; + ret.is_training_ = x; + return ret; + } + + float epsilon_ = 0.0001f; + StringPiece data_format_ = "NHWC"; + bool is_training_ = true; + }; + FusedBatchNormGrad(const ::tensorflow::Scope& scope, ::tensorflow::Input + y_backprop, ::tensorflow::Input x, ::tensorflow::Input + scale, ::tensorflow::Input reserve_space_1, + ::tensorflow::Input reserve_space_2); + FusedBatchNormGrad(const ::tensorflow::Scope& scope, ::tensorflow::Input + y_backprop, ::tensorflow::Input x, ::tensorflow::Input + scale, ::tensorflow::Input reserve_space_1, + ::tensorflow::Input reserve_space_2, const + FusedBatchNormGrad::Attrs& attrs); + + static Attrs Epsilon(float x) { + return Attrs().Epsilon(x); + } + static Attrs DataFormat(StringPiece x) { + return Attrs().DataFormat(x); + } + static Attrs IsTraining(bool x) { + return Attrs().IsTraining(x); + } + + Operation operation; + ::tensorflow::Output x_backprop; + ::tensorflow::Output scale_backprop; + ::tensorflow::Output offset_backprop; + ::tensorflow::Output reserve_space_3; + ::tensorflow::Output reserve_space_4; +}; + +/// Gradient for batch normalization. +/// +/// Note that the size of 4D Tensors are defined by either "NHWC" or "NCHW". +/// The size of 1D Tensors matches the dimension C of the 4D Tensors. +/// +/// Arguments: +/// * scope: A Scope object +/// * y_backprop: A 4D Tensor for the gradient with respect to y. +/// * x: A 4D Tensor for input data. +/// * scale: A 1D Tensor for scaling factor, to scale the normalized x. +/// * reserve_space_1: When is_training is True, a 1D Tensor for the computed batch +/// mean to be reused in gradient computation. When is_training is +/// False, a 1D Tensor for the population mean to be reused in both +/// 1st and 2nd order gradient computation. +/// * reserve_space_2: When is_training is True, a 1D Tensor for the computed batch +/// variance (inverted variance in the cuDNN case) to be reused in +/// gradient computation. When is_training is False, a 1D Tensor +/// for the population variance to be reused in both 1st and 2nd +/// order gradient computation. +/// +/// Optional attributes (see `Attrs`): +/// * epsilon: A small float number added to the variance of x. +/// * data_format: The data format for y_backprop, x, x_backprop. +/// Either "NHWC" (default) or "NCHW". +/// * is_training: A bool value to indicate the operation is for training (default) +/// or inference. +/// +/// Returns: +/// * `Output` x_backprop: A 4D Tensor for the gradient with respect to x. +/// * `Output` scale_backprop: A 1D Tensor for the gradient with respect to scale. +/// * `Output` offset_backprop: A 1D Tensor for the gradient with respect to offset. +/// * `Output` reserve_space_3: Unused placeholder to match the mean input in FusedBatchNorm. +/// * `Output` reserve_space_4: Unused placeholder to match the variance input +/// in FusedBatchNorm. +class FusedBatchNormGradV2 { + public: + /// Optional attribute setters for FusedBatchNormGradV2 + struct Attrs { + /// A small float number added to the variance of x. + /// + /// Defaults to 0.0001 + TF_MUST_USE_RESULT Attrs Epsilon(float x) { + Attrs ret = *this; + ret.epsilon_ = x; + return ret; + } + + /// The data format for y_backprop, x, x_backprop. + /// Either "NHWC" (default) or "NCHW". + /// + /// Defaults to "NHWC" + TF_MUST_USE_RESULT Attrs DataFormat(StringPiece x) { + Attrs ret = *this; + ret.data_format_ = x; + return ret; + } + + /// A bool value to indicate the operation is for training (default) + /// or inference. + /// + /// Defaults to true + TF_MUST_USE_RESULT Attrs IsTraining(bool x) { + Attrs ret = *this; + ret.is_training_ = x; + return ret; + } + + float epsilon_ = 0.0001f; + StringPiece data_format_ = "NHWC"; + bool is_training_ = true; + }; + FusedBatchNormGradV2(const ::tensorflow::Scope& scope, ::tensorflow::Input + y_backprop, ::tensorflow::Input x, ::tensorflow::Input + scale, ::tensorflow::Input reserve_space_1, + ::tensorflow::Input reserve_space_2); + FusedBatchNormGradV2(const ::tensorflow::Scope& scope, ::tensorflow::Input + y_backprop, ::tensorflow::Input x, ::tensorflow::Input + scale, ::tensorflow::Input reserve_space_1, + ::tensorflow::Input reserve_space_2, const + FusedBatchNormGradV2::Attrs& attrs); + + static Attrs Epsilon(float x) { + return Attrs().Epsilon(x); + } + static Attrs DataFormat(StringPiece x) { + return Attrs().DataFormat(x); + } + static Attrs IsTraining(bool x) { + return Attrs().IsTraining(x); + } + + Operation operation; + ::tensorflow::Output x_backprop; + ::tensorflow::Output scale_backprop; + ::tensorflow::Output offset_backprop; + ::tensorflow::Output reserve_space_3; + ::tensorflow::Output reserve_space_4; +}; + +/// Batch normalization. +/// +/// Note that the size of 4D Tensors are defined by either "NHWC" or "NCHW". +/// The size of 1D Tensors matches the dimension C of the 4D Tensors. +/// +/// Arguments: +/// * scope: A Scope object +/// * x: A 4D Tensor for input data. +/// * scale: A 1D Tensor for scaling factor, to scale the normalized x. +/// * offset: A 1D Tensor for offset, to shift to the normalized x. +/// * mean: A 1D Tensor for population mean. Used for inference only; +/// must be empty for training. +/// * variance: A 1D Tensor for population variance. Used for inference only; +/// must be empty for training. +/// +/// Optional attributes (see `Attrs`): +/// * epsilon: A small float number added to the variance of x. +/// * data_format: The data format for x and y. Either "NHWC" (default) or "NCHW". +/// * is_training: A bool value to indicate the operation is for training (default) +/// or inference. +/// +/// Returns: +/// * `Output` y: A 4D Tensor for output data. +/// * `Output` batch_mean: A 1D Tensor for the computed batch mean, to be used by TensorFlow +/// to compute the running mean. +/// * `Output` batch_variance: A 1D Tensor for the computed batch variance, to be used by +/// TensorFlow to compute the running variance. +/// * `Output` reserve_space_1: A 1D Tensor for the computed batch mean, to be reused +/// in the gradient computation. +/// * `Output` reserve_space_2: A 1D Tensor for the computed batch variance (inverted variance +/// in the cuDNN case), to be reused in the gradient computation. +class FusedBatchNormV2 { + public: + /// Optional attribute setters for FusedBatchNormV2 + struct Attrs { + /// A small float number added to the variance of x. + /// + /// Defaults to 0.0001 + TF_MUST_USE_RESULT Attrs Epsilon(float x) { + Attrs ret = *this; + ret.epsilon_ = x; + return ret; + } + + /// The data format for x and y. Either "NHWC" (default) or "NCHW". + /// + /// Defaults to "NHWC" + TF_MUST_USE_RESULT Attrs DataFormat(StringPiece x) { + Attrs ret = *this; + ret.data_format_ = x; + return ret; + } + + /// A bool value to indicate the operation is for training (default) + /// or inference. + /// + /// Defaults to true + TF_MUST_USE_RESULT Attrs IsTraining(bool x) { + Attrs ret = *this; + ret.is_training_ = x; + return ret; + } + + float epsilon_ = 0.0001f; + StringPiece data_format_ = "NHWC"; + bool is_training_ = true; + }; + FusedBatchNormV2(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input scale, ::tensorflow::Input offset, + ::tensorflow::Input mean, ::tensorflow::Input variance); + FusedBatchNormV2(const ::tensorflow::Scope& scope, ::tensorflow::Input x, + ::tensorflow::Input scale, ::tensorflow::Input offset, + ::tensorflow::Input mean, ::tensorflow::Input variance, const + FusedBatchNormV2::Attrs& attrs); + + static Attrs Epsilon(float x) { + return Attrs().Epsilon(x); + } + static Attrs DataFormat(StringPiece x) { + return Attrs().DataFormat(x); + } + static Attrs IsTraining(bool x) { + return Attrs().IsTraining(x); + } + + Operation operation; + ::tensorflow::Output y; + ::tensorflow::Output batch_mean; + ::tensorflow::Output batch_variance; + ::tensorflow::Output reserve_space_1; + ::tensorflow::Output reserve_space_2; +}; + +/// Performs a padding as a preprocess during a convolution. +/// +/// Similar to FusedResizeAndPadConv2d, this op allows for an optimized +/// implementation where the spatial padding transformation stage is fused with the +/// im2col lookup, but in this case without the bilinear filtering required for +/// resizing. Fusing the padding prevents the need to write out the intermediate +/// results as whole tensors, reducing memory pressure, and we can get some latency +/// gains by merging the transformation calculations. +/// The data_format attribute for Conv2D isn't supported by this op, and 'NHWC' +/// order is used instead. +/// Internally this op uses a single per-graph scratch buffer, which means that it +/// will block if multiple versions are being run in parallel. This is because this +/// operator is primarily an optimization to minimize memory usage. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: 4-D with shape `[batch, in_height, in_width, in_channels]`. +/// * paddings: A two-column matrix specifying the padding sizes. The number of +/// rows must be the same as the rank of `input`. +/// * filter: 4-D with shape +/// `[filter_height, filter_width, in_channels, out_channels]`. +/// * strides: 1-D of length 4. The stride of the sliding window for each dimension +/// of `input`. Must be in the same order as the dimension specified with format. +/// * padding: The type of padding algorithm to use. +/// +/// Returns: +/// * `Output`: The output tensor. +class FusedPadConv2D { + public: + FusedPadConv2D(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input paddings, ::tensorflow::Input filter, + StringPiece mode, const gtl::ArraySlice& strides, + StringPiece padding); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Performs a resize and padding as a preprocess during a convolution. +/// +/// It's often possible to do spatial transformations more efficiently as part of +/// the packing stage of a convolution, so this op allows for an optimized +/// implementation where these stages are fused together. This prevents the need to +/// write out the intermediate results as whole tensors, reducing memory pressure, +/// and we can get some latency gains by merging the transformation calculations. +/// The data_format attribute for Conv2D isn't supported by this op, and defaults to +/// 'NHWC' order. +/// Internally this op uses a single per-graph scratch buffer, which means that it +/// will block if multiple versions are being run in parallel. This is because this +/// operator is primarily an optimization to minimize memory usage. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: 4-D with shape `[batch, in_height, in_width, in_channels]`. +/// * size: A 1-D int32 Tensor of 2 elements: `new_height, new_width`. The +/// new size for the images. +/// * paddings: A two-column matrix specifying the padding sizes. The number of +/// rows must be the same as the rank of `input`. +/// * filter: 4-D with shape +/// `[filter_height, filter_width, in_channels, out_channels]`. +/// * strides: 1-D of length 4. The stride of the sliding window for each dimension +/// of `input`. Must be in the same order as the dimension specified with format. +/// * padding: The type of padding algorithm to use. +/// +/// Optional attributes (see `Attrs`): +/// * resize_align_corners: If true, the centers of the 4 corner pixels of the input and output tensors are +/// aligned, preserving the values at the corner pixels. Defaults to false. +/// +/// Returns: +/// * `Output`: The output tensor. +class FusedResizeAndPadConv2D { + public: + /// Optional attribute setters for FusedResizeAndPadConv2D + struct Attrs { + /// If true, the centers of the 4 corner pixels of the input and output tensors are + /// aligned, preserving the values at the corner pixels. Defaults to false. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs ResizeAlignCorners(bool x) { + Attrs ret = *this; + ret.resize_align_corners_ = x; + return ret; + } + + bool resize_align_corners_ = false; + }; + FusedResizeAndPadConv2D(const ::tensorflow::Scope& scope, ::tensorflow::Input + input, ::tensorflow::Input size, ::tensorflow::Input + paddings, ::tensorflow::Input filter, StringPiece mode, + const gtl::ArraySlice& strides, StringPiece + padding); + FusedResizeAndPadConv2D(const ::tensorflow::Scope& scope, ::tensorflow::Input + input, ::tensorflow::Input size, ::tensorflow::Input + paddings, ::tensorflow::Input filter, StringPiece mode, + const gtl::ArraySlice& strides, StringPiece + padding, const FusedResizeAndPadConv2D::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs ResizeAlignCorners(bool x) { + return Attrs().ResizeAlignCorners(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Says whether the targets are in the top `K` predictions. +/// +/// This outputs a `batch_size` bool array, an entry `out[i]` is `true` if the +/// prediction for the target class is among the top `k` predictions among +/// all predictions for example `i`. Note that the behavior of `InTopK` differs +/// from the `TopK` op in its handling of ties; if multiple classes have the +/// same prediction value and straddle the top-`k` boundary, all of those +/// classes are considered to be in the top `k`. +/// +/// More formally, let +/// +/// \\(predictions_i\\) be the predictions for all classes for example `i`, +/// \\(targets_i\\) be the target class for example `i`, +/// \\(out_i\\) be the output for example `i`, +/// +/// $$out_i = predictions_{i, targets_i} \in TopKIncludingTies(predictions_i)$$ +/// +/// Arguments: +/// * scope: A Scope object +/// * predictions: A `batch_size` x `classes` tensor. +/// * targets: A `batch_size` vector of class ids. +/// * k: Number of top elements to look at for computing precision. +/// +/// Returns: +/// * `Output`: Computed Precision at `k` as a `bool Tensor`. +class InTopK { + public: + InTopK(const ::tensorflow::Scope& scope, ::tensorflow::Input predictions, + ::tensorflow::Input targets, int64 k); + operator ::tensorflow::Output() const { return precision; } + operator ::tensorflow::Input() const { return precision; } + ::tensorflow::Node* node() const { return precision.node(); } + + Operation operation; + ::tensorflow::Output precision; +}; + +/// Says whether the targets are in the top `K` predictions. +/// +/// This outputs a `batch_size` bool array, an entry `out[i]` is `true` if the +/// prediction for the target class is among the top `k` predictions among +/// all predictions for example `i`. Note that the behavior of `InTopK` differs +/// from the `TopK` op in its handling of ties; if multiple classes have the +/// same prediction value and straddle the top-`k` boundary, all of those +/// classes are considered to be in the top `k`. +/// +/// More formally, let +/// +/// \\(predictions_i\\) be the predictions for all classes for example `i`, +/// \\(targets_i\\) be the target class for example `i`, +/// \\(out_i\\) be the output for example `i`, +/// +/// $$out_i = predictions_{i, targets_i} \in TopKIncludingTies(predictions_i)$$ +/// +/// Arguments: +/// * scope: A Scope object +/// * predictions: A `batch_size` x `classes` tensor. +/// * targets: A `batch_size` vector of class ids. +/// * k: Number of top elements to look at for computing precision. +/// +/// Returns: +/// * `Output`: Computed precision at `k` as a `bool Tensor`. +class InTopKV2 { + public: + InTopKV2(const ::tensorflow::Scope& scope, ::tensorflow::Input predictions, + ::tensorflow::Input targets, ::tensorflow::Input k); + operator ::tensorflow::Output() const { return precision; } + operator ::tensorflow::Input() const { return precision; } + ::tensorflow::Node* node() const { return precision.node(); } + + Operation operation; + ::tensorflow::Output precision; +}; + +/// L2 Loss. +/// +/// Computes half the L2 norm of a tensor without the `sqrt`: +/// +/// output = sum(t ** 2) / 2 +/// +/// Arguments: +/// * scope: A Scope object +/// * t: Typically 2-D, but may have any dimensions. +/// +/// Returns: +/// * `Output`: 0-D. +class L2Loss { + public: + L2Loss(const ::tensorflow::Scope& scope, ::tensorflow::Input t); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Local Response Normalization. +/// +/// The 4-D `input` tensor is treated as a 3-D array of 1-D vectors (along the last +/// dimension), and each vector is normalized independently. Within a given vector, +/// each component is divided by the weighted, squared sum of inputs within +/// `depth_radius`. In detail, +/// +/// sqr_sum[a, b, c, d] = +/// sum(input[a, b, c, d - depth_radius : d + depth_radius + 1] ** 2) +/// output = input / (bias + alpha * sqr_sum) ** beta +/// +/// For details, see [Krizhevsky et al., ImageNet classification with deep +/// convolutional neural networks (NIPS 2012)](http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks). +/// +/// Arguments: +/// * scope: A Scope object +/// * input: 4-D. +/// +/// Optional attributes (see `Attrs`): +/// * depth_radius: 0-D. Half-width of the 1-D normalization window. +/// * bias: An offset (usually positive to avoid dividing by 0). +/// * alpha: A scale factor, usually positive. +/// * beta: An exponent. +/// +/// Returns: +/// * `Output`: The output tensor. +class LRN { + public: + /// Optional attribute setters for LRN + struct Attrs { + /// 0-D. Half-width of the 1-D normalization window. + /// + /// Defaults to 5 + TF_MUST_USE_RESULT Attrs DepthRadius(int64 x) { + Attrs ret = *this; + ret.depth_radius_ = x; + return ret; + } + + /// An offset (usually positive to avoid dividing by 0). + /// + /// Defaults to 1 + TF_MUST_USE_RESULT Attrs Bias(float x) { + Attrs ret = *this; + ret.bias_ = x; + return ret; + } + + /// A scale factor, usually positive. + /// + /// Defaults to 1 + TF_MUST_USE_RESULT Attrs Alpha(float x) { + Attrs ret = *this; + ret.alpha_ = x; + return ret; + } + + /// An exponent. + /// + /// Defaults to 0.5 + TF_MUST_USE_RESULT Attrs Beta(float x) { + Attrs ret = *this; + ret.beta_ = x; + return ret; + } + + int64 depth_radius_ = 5; + float bias_ = 1.0f; + float alpha_ = 1.0f; + float beta_ = 0.5f; + }; + LRN(const ::tensorflow::Scope& scope, ::tensorflow::Input input); + LRN(const ::tensorflow::Scope& scope, ::tensorflow::Input input, const + LRN::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs DepthRadius(int64 x) { + return Attrs().DepthRadius(x); + } + static Attrs Bias(float x) { + return Attrs().Bias(x); + } + static Attrs Alpha(float x) { + return Attrs().Alpha(x); + } + static Attrs Beta(float x) { + return Attrs().Beta(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Computes log softmax activations. +/// +/// For each batch `i` and class `j` we have +/// +/// logsoftmax[i, j] = logits[i, j] - log(sum(exp(logits[i]))) +/// +/// Arguments: +/// * scope: A Scope object +/// * logits: 2-D with shape `[batch_size, num_classes]`. +/// +/// Returns: +/// * `Output`: Same shape as `logits`. +class LogSoftmax { + public: + LogSoftmax(const ::tensorflow::Scope& scope, ::tensorflow::Input logits); + operator ::tensorflow::Output() const { return logsoftmax; } + operator ::tensorflow::Input() const { return logsoftmax; } + ::tensorflow::Node* node() const { return logsoftmax.node(); } + + Operation operation; + ::tensorflow::Output logsoftmax; +}; + +/// Performs max pooling on the input. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: 4-D input to pool over. +/// * ksize: The size of the window for each dimension of the input tensor. +/// * strides: The stride of the sliding window for each dimension of the +/// input tensor. +/// * padding: The type of padding algorithm to use. +/// +/// Optional attributes (see `Attrs`): +/// * data_format: Specify the data format of the input and output data. With the +/// default format "NHWC", the data is stored in the order of: +/// [batch, in_height, in_width, in_channels]. +/// Alternatively, the format could be "NCHW", the data storage order of: +/// [batch, in_channels, in_height, in_width]. +/// +/// Returns: +/// * `Output`: The max pooled output tensor. +class MaxPool { + public: + /// Optional attribute setters for MaxPool + struct Attrs { + /// Specify the data format of the input and output data. With the + /// default format "NHWC", the data is stored in the order of: + /// [batch, in_height, in_width, in_channels]. + /// Alternatively, the format could be "NCHW", the data storage order of: + /// [batch, in_channels, in_height, in_width]. + /// + /// Defaults to "NHWC" + TF_MUST_USE_RESULT Attrs DataFormat(StringPiece x) { + Attrs ret = *this; + ret.data_format_ = x; + return ret; + } + + StringPiece data_format_ = "NHWC"; + }; + MaxPool(const ::tensorflow::Scope& scope, ::tensorflow::Input input, const + gtl::ArraySlice& ksize, const gtl::ArraySlice& strides, + StringPiece padding); + MaxPool(const ::tensorflow::Scope& scope, ::tensorflow::Input input, const + gtl::ArraySlice& ksize, const gtl::ArraySlice& strides, + StringPiece padding, const MaxPool::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs DataFormat(StringPiece x) { + return Attrs().DataFormat(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Performs 3D max pooling on the input. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: Shape `[batch, depth, rows, cols, channels]` tensor to pool over. +/// * ksize: 1-D tensor of length 5. The size of the window for each dimension of +/// the input tensor. Must have `ksize[0] = ksize[4] = 1`. +/// * strides: 1-D tensor of length 5. The stride of the sliding window for each +/// dimension of `input`. Must have `strides[0] = strides[4] = 1`. +/// * padding: The type of padding algorithm to use. +/// +/// Optional attributes (see `Attrs`): +/// * data_format: The data format of the input and output data. With the +/// default format "NDHWC", the data is stored in the order of: +/// [batch, in_depth, in_height, in_width, in_channels]. +/// Alternatively, the format could be "NCDHW", the data storage order is: +/// [batch, in_channels, in_depth, in_height, in_width]. +/// +/// Returns: +/// * `Output`: The max pooled output tensor. +class MaxPool3D { + public: + /// Optional attribute setters for MaxPool3D + struct Attrs { + /// The data format of the input and output data. With the + /// default format "NDHWC", the data is stored in the order of: + /// [batch, in_depth, in_height, in_width, in_channels]. + /// Alternatively, the format could be "NCDHW", the data storage order is: + /// [batch, in_channels, in_depth, in_height, in_width]. + /// + /// Defaults to "NDHWC" + TF_MUST_USE_RESULT Attrs DataFormat(StringPiece x) { + Attrs ret = *this; + ret.data_format_ = x; + return ret; + } + + StringPiece data_format_ = "NDHWC"; + }; + MaxPool3D(const ::tensorflow::Scope& scope, ::tensorflow::Input input, const + gtl::ArraySlice& ksize, const gtl::ArraySlice& strides, + StringPiece padding); + MaxPool3D(const ::tensorflow::Scope& scope, ::tensorflow::Input input, const + gtl::ArraySlice& ksize, const gtl::ArraySlice& strides, + StringPiece padding, const MaxPool3D::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs DataFormat(StringPiece x) { + return Attrs().DataFormat(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Computes gradients of max pooling function. +/// +/// Arguments: +/// * scope: A Scope object +/// * orig_input: The original input tensor. +/// * orig_output: The original output tensor. +/// * grad: Output backprop of shape `[batch, depth, rows, cols, channels]`. +/// * ksize: 1-D tensor of length 5. The size of the window for each dimension of +/// the input tensor. Must have `ksize[0] = ksize[4] = 1`. +/// * strides: 1-D tensor of length 5. The stride of the sliding window for each +/// dimension of `input`. Must have `strides[0] = strides[4] = 1`. +/// * padding: The type of padding algorithm to use. +/// +/// Optional attributes (see `Attrs`): +/// * data_format: The data format of the input and output data. With the +/// default format "NDHWC", the data is stored in the order of: +/// [batch, in_depth, in_height, in_width, in_channels]. +/// Alternatively, the format could be "NCDHW", the data storage order is: +/// [batch, in_channels, in_depth, in_height, in_width]. +/// +/// Returns: +/// * `Output`: The output tensor. +class MaxPool3DGrad { + public: + /// Optional attribute setters for MaxPool3DGrad + struct Attrs { + /// The data format of the input and output data. With the + /// default format "NDHWC", the data is stored in the order of: + /// [batch, in_depth, in_height, in_width, in_channels]. + /// Alternatively, the format could be "NCDHW", the data storage order is: + /// [batch, in_channels, in_depth, in_height, in_width]. + /// + /// Defaults to "NDHWC" + TF_MUST_USE_RESULT Attrs DataFormat(StringPiece x) { + Attrs ret = *this; + ret.data_format_ = x; + return ret; + } + + StringPiece data_format_ = "NDHWC"; + }; + MaxPool3DGrad(const ::tensorflow::Scope& scope, ::tensorflow::Input orig_input, + ::tensorflow::Input orig_output, ::tensorflow::Input grad, const + gtl::ArraySlice& ksize, const gtl::ArraySlice& strides, + StringPiece padding); + MaxPool3DGrad(const ::tensorflow::Scope& scope, ::tensorflow::Input orig_input, + ::tensorflow::Input orig_output, ::tensorflow::Input grad, const + gtl::ArraySlice& ksize, const gtl::ArraySlice& strides, + StringPiece padding, const MaxPool3DGrad::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs DataFormat(StringPiece x) { + return Attrs().DataFormat(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Computes second-order gradients of the maxpooling function. +/// +/// Arguments: +/// * scope: A Scope object +/// * orig_input: The original input tensor. +/// * orig_output: The original output tensor. +/// * grad: Output backprop of shape `[batch, depth, rows, cols, channels]`. +/// * ksize: 1-D tensor of length 5. The size of the window for each dimension of +/// the input tensor. Must have `ksize[0] = ksize[4] = 1`. +/// * strides: 1-D tensor of length 5. The stride of the sliding window for each +/// dimension of `input`. Must have `strides[0] = strides[4] = 1`. +/// * padding: The type of padding algorithm to use. +/// +/// Optional attributes (see `Attrs`): +/// * data_format: The data format of the input and output data. With the +/// default format "NDHWC", the data is stored in the order of: +/// [batch, in_depth, in_height, in_width, in_channels]. +/// Alternatively, the format could be "NCDHW", the data storage order is: +/// [batch, in_channels, in_depth, in_height, in_width]. +/// +/// Returns: +/// * `Output`: Gradients of gradients w.r.t. the input to `max_pool`. +class MaxPool3DGradGrad { + public: + /// Optional attribute setters for MaxPool3DGradGrad + struct Attrs { + /// The data format of the input and output data. With the + /// default format "NDHWC", the data is stored in the order of: + /// [batch, in_depth, in_height, in_width, in_channels]. + /// Alternatively, the format could be "NCDHW", the data storage order is: + /// [batch, in_channels, in_depth, in_height, in_width]. + /// + /// Defaults to "NDHWC" + TF_MUST_USE_RESULT Attrs DataFormat(StringPiece x) { + Attrs ret = *this; + ret.data_format_ = x; + return ret; + } + + StringPiece data_format_ = "NDHWC"; + }; + MaxPool3DGradGrad(const ::tensorflow::Scope& scope, ::tensorflow::Input + orig_input, ::tensorflow::Input orig_output, + ::tensorflow::Input grad, const gtl::ArraySlice& ksize, + const gtl::ArraySlice& strides, StringPiece padding); + MaxPool3DGradGrad(const ::tensorflow::Scope& scope, ::tensorflow::Input + orig_input, ::tensorflow::Input orig_output, + ::tensorflow::Input grad, const gtl::ArraySlice& ksize, + const gtl::ArraySlice& strides, StringPiece padding, + const MaxPool3DGradGrad::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs DataFormat(StringPiece x) { + return Attrs().DataFormat(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Computes second-order gradients of the maxpooling function. +/// +/// Arguments: +/// * scope: A Scope object +/// * orig_input: The original input tensor. +/// * orig_output: The original output tensor. +/// * grad: 4-D. Gradients of gradients w.r.t. the input of `max_pool`. +/// * ksize: The size of the window for each dimension of the input tensor. +/// * strides: The stride of the sliding window for each dimension of the +/// input tensor. +/// * padding: The type of padding algorithm to use. +/// +/// Optional attributes (see `Attrs`): +/// * data_format: Specify the data format of the input and output data. With the +/// default format "NHWC", the data is stored in the order of: +/// [batch, in_height, in_width, in_channels]. +/// Alternatively, the format could be "NCHW", the data storage order of: +/// [batch, in_channels, in_height, in_width]. +/// +/// Returns: +/// * `Output`: Gradients of gradients w.r.t. the input to `max_pool`. +class MaxPoolGradGrad { + public: + /// Optional attribute setters for MaxPoolGradGrad + struct Attrs { + /// Specify the data format of the input and output data. With the + /// default format "NHWC", the data is stored in the order of: + /// [batch, in_height, in_width, in_channels]. + /// Alternatively, the format could be "NCHW", the data storage order of: + /// [batch, in_channels, in_height, in_width]. + /// + /// Defaults to "NHWC" + TF_MUST_USE_RESULT Attrs DataFormat(StringPiece x) { + Attrs ret = *this; + ret.data_format_ = x; + return ret; + } + + StringPiece data_format_ = "NHWC"; + }; + MaxPoolGradGrad(const ::tensorflow::Scope& scope, ::tensorflow::Input + orig_input, ::tensorflow::Input orig_output, + ::tensorflow::Input grad, const gtl::ArraySlice& ksize, + const gtl::ArraySlice& strides, StringPiece padding); + MaxPoolGradGrad(const ::tensorflow::Scope& scope, ::tensorflow::Input + orig_input, ::tensorflow::Input orig_output, + ::tensorflow::Input grad, const gtl::ArraySlice& ksize, + const gtl::ArraySlice& strides, StringPiece padding, const + MaxPoolGradGrad::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs DataFormat(StringPiece x) { + return Attrs().DataFormat(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Computes second-order gradients of the maxpooling function. +/// +/// Arguments: +/// * scope: A Scope object +/// * orig_input: The original input tensor. +/// * orig_output: The original output tensor. +/// * grad: 4-D. Gradients of gradients w.r.t. the input of `max_pool`. +/// * ksize: The size of the window for each dimension of the input tensor. +/// * strides: The stride of the sliding window for each dimension of the +/// input tensor. +/// * padding: The type of padding algorithm to use. +/// +/// Optional attributes (see `Attrs`): +/// * data_format: Specify the data format of the input and output data. With the +/// default format "NHWC", the data is stored in the order of: +/// [batch, in_height, in_width, in_channels]. +/// Alternatively, the format could be "NCHW", the data storage order of: +/// [batch, in_channels, in_height, in_width]. +/// +/// Returns: +/// * `Output`: Gradients of gradients w.r.t. the input to `max_pool`. +class MaxPoolGradGradV2 { + public: + /// Optional attribute setters for MaxPoolGradGradV2 + struct Attrs { + /// Specify the data format of the input and output data. With the + /// default format "NHWC", the data is stored in the order of: + /// [batch, in_height, in_width, in_channels]. + /// Alternatively, the format could be "NCHW", the data storage order of: + /// [batch, in_channels, in_height, in_width]. + /// + /// Defaults to "NHWC" + TF_MUST_USE_RESULT Attrs DataFormat(StringPiece x) { + Attrs ret = *this; + ret.data_format_ = x; + return ret; + } + + StringPiece data_format_ = "NHWC"; + }; + MaxPoolGradGradV2(const ::tensorflow::Scope& scope, ::tensorflow::Input + orig_input, ::tensorflow::Input orig_output, + ::tensorflow::Input grad, ::tensorflow::Input ksize, + ::tensorflow::Input strides, StringPiece padding); + MaxPoolGradGradV2(const ::tensorflow::Scope& scope, ::tensorflow::Input + orig_input, ::tensorflow::Input orig_output, + ::tensorflow::Input grad, ::tensorflow::Input ksize, + ::tensorflow::Input strides, StringPiece padding, const + MaxPoolGradGradV2::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs DataFormat(StringPiece x) { + return Attrs().DataFormat(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Computes second-order gradients of the maxpooling function. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: The original input. +/// * grad: 4-D with shape `[batch, height, width, channels]`. Gradients w.r.t. the +/// input of `max_pool`. +/// * argmax: The indices of the maximum values chosen for each output of `max_pool`. +/// * ksize: The size of the window for each dimension of the input tensor. +/// * strides: The stride of the sliding window for each dimension of the +/// input tensor. +/// * padding: The type of padding algorithm to use. +/// +/// Returns: +/// * `Output`: Gradients of gradients w.r.t. the input of `max_pool`. +class MaxPoolGradGradWithArgmax { + public: + MaxPoolGradGradWithArgmax(const ::tensorflow::Scope& scope, ::tensorflow::Input + input, ::tensorflow::Input grad, ::tensorflow::Input + argmax, const gtl::ArraySlice& ksize, const + gtl::ArraySlice& strides, StringPiece padding); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Computes gradients of the maxpooling function. +/// +/// Arguments: +/// * scope: A Scope object +/// * orig_input: The original input tensor. +/// * orig_output: The original output tensor. +/// * grad: 4-D. Gradients w.r.t. the output of `max_pool`. +/// * ksize: The size of the window for each dimension of the input tensor. +/// * strides: The stride of the sliding window for each dimension of the +/// input tensor. +/// * padding: The type of padding algorithm to use. +/// +/// Optional attributes (see `Attrs`): +/// * data_format: Specify the data format of the input and output data. With the +/// default format "NHWC", the data is stored in the order of: +/// [batch, in_height, in_width, in_channels]. +/// Alternatively, the format could be "NCHW", the data storage order of: +/// [batch, in_channels, in_height, in_width]. +/// +/// Returns: +/// * `Output`: Gradients w.r.t. the input to `max_pool`. +class MaxPoolGradV2 { + public: + /// Optional attribute setters for MaxPoolGradV2 + struct Attrs { + /// Specify the data format of the input and output data. With the + /// default format "NHWC", the data is stored in the order of: + /// [batch, in_height, in_width, in_channels]. + /// Alternatively, the format could be "NCHW", the data storage order of: + /// [batch, in_channels, in_height, in_width]. + /// + /// Defaults to "NHWC" + TF_MUST_USE_RESULT Attrs DataFormat(StringPiece x) { + Attrs ret = *this; + ret.data_format_ = x; + return ret; + } + + StringPiece data_format_ = "NHWC"; + }; + MaxPoolGradV2(const ::tensorflow::Scope& scope, ::tensorflow::Input orig_input, + ::tensorflow::Input orig_output, ::tensorflow::Input grad, + ::tensorflow::Input ksize, ::tensorflow::Input strides, + StringPiece padding); + MaxPoolGradV2(const ::tensorflow::Scope& scope, ::tensorflow::Input orig_input, + ::tensorflow::Input orig_output, ::tensorflow::Input grad, + ::tensorflow::Input ksize, ::tensorflow::Input strides, + StringPiece padding, const MaxPoolGradV2::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs DataFormat(StringPiece x) { + return Attrs().DataFormat(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Performs max pooling on the input. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: 4-D input to pool over. +/// * ksize: The size of the window for each dimension of the input tensor. +/// * strides: The stride of the sliding window for each dimension of the +/// input tensor. +/// * padding: The type of padding algorithm to use. +/// +/// Optional attributes (see `Attrs`): +/// * data_format: Specify the data format of the input and output data. With the +/// default format "NHWC", the data is stored in the order of: +/// [batch, in_height, in_width, in_channels]. +/// Alternatively, the format could be "NCHW", the data storage order of: +/// [batch, in_channels, in_height, in_width]. +/// +/// Returns: +/// * `Output`: The max pooled output tensor. +class MaxPoolV2 { + public: + /// Optional attribute setters for MaxPoolV2 + struct Attrs { + /// Specify the data format of the input and output data. With the + /// default format "NHWC", the data is stored in the order of: + /// [batch, in_height, in_width, in_channels]. + /// Alternatively, the format could be "NCHW", the data storage order of: + /// [batch, in_channels, in_height, in_width]. + /// + /// Defaults to "NHWC" + TF_MUST_USE_RESULT Attrs DataFormat(StringPiece x) { + Attrs ret = *this; + ret.data_format_ = x; + return ret; + } + + StringPiece data_format_ = "NHWC"; + }; + MaxPoolV2(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input ksize, ::tensorflow::Input strides, StringPiece + padding); + MaxPoolV2(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input ksize, ::tensorflow::Input strides, StringPiece + padding, const MaxPoolV2::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs DataFormat(StringPiece x) { + return Attrs().DataFormat(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Performs max pooling on the input and outputs both max values and indices. +/// +/// The indices in `argmax` are flattened, so that a maximum value at position +/// `[b, y, x, c]` becomes flattened index +/// `((b * height + y) * width + x) * channels + c`. +/// +/// The indices returned are always in `[0, height) x [0, width)` before flattening, +/// even if padding is involved and the mathematically correct answer is outside +/// (either negative or too large). This is a bug, but fixing it is difficult to do +/// in a safe backwards compatible way, especially due to flattening. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: 4-D with shape `[batch, height, width, channels]`. Input to pool over. +/// * ksize: The size of the window for each dimension of the input tensor. +/// * strides: The stride of the sliding window for each dimension of the +/// input tensor. +/// * padding: The type of padding algorithm to use. +/// +/// Returns: +/// * `Output` output: The max pooled output tensor. +/// * `Output` argmax: 4-D. The flattened indices of the max values chosen for each output. +class MaxPoolWithArgmax { + public: + /// Optional attribute setters for MaxPoolWithArgmax + struct Attrs { + /// Defaults to DT_INT64 + TF_MUST_USE_RESULT Attrs Targmax(DataType x) { + Attrs ret = *this; + ret.Targmax_ = x; + return ret; + } + + DataType Targmax_ = DT_INT64; + }; + MaxPoolWithArgmax(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + const gtl::ArraySlice& ksize, const + gtl::ArraySlice& strides, StringPiece padding); + MaxPoolWithArgmax(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + const gtl::ArraySlice& ksize, const + gtl::ArraySlice& strides, StringPiece padding, const + MaxPoolWithArgmax::Attrs& attrs); + + static Attrs Targmax(DataType x) { + return Attrs().Targmax(x); + } + + Operation operation; + ::tensorflow::Output output; + ::tensorflow::Output argmax; +}; + +/// Finds values of the `n`-th order statistic for the last dimension. +/// +/// If the input is a vector (rank-1), finds the entries which is the nth-smallest +/// value in the vector and outputs their values as scalar tensor. +/// +/// For matrices (resp. higher rank input), computes the entries which is the +/// nth-smallest value in each row (resp. vector along the last dimension). Thus, +/// +/// values.shape = input.shape[:-1] +/// +/// Arguments: +/// * scope: A Scope object +/// * input: 1-D or higher with last dimension at least `n+1`. +/// * n: 0-D. Position of sorted vector to select along the last dimension (along +/// each row for matrices). Valid range of n is `[0, input.shape[:-1])` +/// +/// Optional attributes (see `Attrs`): +/// * reverse: When set to True, find the nth-largest value in the vector and vice +/// versa. +/// +/// Returns: +/// * `Output`: The `n`-th order statistic along each last dimensional slice. +class NthElement { + public: + /// Optional attribute setters for NthElement + struct Attrs { + /// When set to True, find the nth-largest value in the vector and vice + /// versa. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs Reverse(bool x) { + Attrs ret = *this; + ret.reverse_ = x; + return ret; + } + + bool reverse_ = false; + }; + NthElement(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input n); + NthElement(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input n, const NthElement::Attrs& attrs); + operator ::tensorflow::Output() const { return values; } + operator ::tensorflow::Input() const { return values; } + ::tensorflow::Node* node() const { return values.node(); } + + static Attrs Reverse(bool x) { + return Attrs().Reverse(x); + } + + Operation operation; + ::tensorflow::Output values; +}; + +/// Produces the average pool of the input tensor for quantized types. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: 4-D with shape `[batch, height, width, channels]`. +/// * min_input: The float value that the lowest quantized input value represents. +/// * max_input: The float value that the highest quantized input value represents. +/// * ksize: The size of the window for each dimension of the input tensor. +/// The length must be 4 to match the number of dimensions of the input. +/// * strides: The stride of the sliding window for each dimension of the input +/// tensor. The length must be 4 to match the number of dimensions of the input. +/// * padding: The type of padding algorithm to use. +/// +/// Returns: +/// * `Output` output +/// * `Output` min_output: The float value that the lowest quantized output value represents. +/// * `Output` max_output: The float value that the highest quantized output value represents. +class QuantizedAvgPool { + public: + QuantizedAvgPool(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input min_input, ::tensorflow::Input max_input, + const gtl::ArraySlice& ksize, const gtl::ArraySlice& + strides, StringPiece padding); + + Operation operation; + ::tensorflow::Output output; + ::tensorflow::Output min_output; + ::tensorflow::Output max_output; +}; + +/// Quantized Batch normalization. +/// +/// This op is deprecated and will be removed in the future. Prefer +/// `tf.nn.batch_normalization`. +/// +/// Arguments: +/// * scope: A Scope object +/// * t: A 4D input Tensor. +/// * t_min: The value represented by the lowest quantized input. +/// * t_max: The value represented by the highest quantized input. +/// * m: A 1D mean Tensor with size matching the last dimension of t. +/// This is the first output from tf.nn.moments, +/// or a saved moving average thereof. +/// * m_min: The value represented by the lowest quantized mean. +/// * m_max: The value represented by the highest quantized mean. +/// * v: A 1D variance Tensor with size matching the last dimension of t. +/// This is the second output from tf.nn.moments, +/// or a saved moving average thereof. +/// * v_min: The value represented by the lowest quantized variance. +/// * v_max: The value represented by the highest quantized variance. +/// * beta: A 1D beta Tensor with size matching the last dimension of t. +/// An offset to be added to the normalized tensor. +/// * beta_min: The value represented by the lowest quantized offset. +/// * beta_max: The value represented by the highest quantized offset. +/// * gamma: A 1D gamma Tensor with size matching the last dimension of t. +/// If "scale_after_normalization" is true, this tensor will be multiplied +/// with the normalized tensor. +/// * gamma_min: The value represented by the lowest quantized gamma. +/// * gamma_max: The value represented by the highest quantized gamma. +/// * variance_epsilon: A small float number to avoid dividing by 0. +/// * scale_after_normalization: A bool indicating whether the resulted tensor +/// needs to be multiplied with gamma. +/// +/// Returns: +/// * `Output` result +/// * `Output` result_min +/// * `Output` result_max +class QuantizedBatchNormWithGlobalNormalization { + public: + QuantizedBatchNormWithGlobalNormalization(const ::tensorflow::Scope& scope, + ::tensorflow::Input t, + ::tensorflow::Input t_min, + ::tensorflow::Input t_max, + ::tensorflow::Input m, + ::tensorflow::Input m_min, + ::tensorflow::Input m_max, + ::tensorflow::Input v, + ::tensorflow::Input v_min, + ::tensorflow::Input v_max, + ::tensorflow::Input beta, + ::tensorflow::Input beta_min, + ::tensorflow::Input beta_max, + ::tensorflow::Input gamma, + ::tensorflow::Input gamma_min, + ::tensorflow::Input gamma_max, + DataType out_type, float + variance_epsilon, bool + scale_after_normalization); + + Operation operation; + ::tensorflow::Output result; + ::tensorflow::Output result_min; + ::tensorflow::Output result_max; +}; + +/// Adds Tensor 'bias' to Tensor 'input' for Quantized types. +/// +/// Broadcasts the values of bias on dimensions 0..N-2 of 'input'. +/// +/// Arguments: +/// * scope: A Scope object +/// * bias: A 1D bias Tensor with size matching the last dimension of 'input'. +/// * min_input: The float value that the lowest quantized input value represents. +/// * max_input: The float value that the highest quantized input value represents. +/// * min_bias: The float value that the lowest quantized bias value represents. +/// * max_bias: The float value that the highest quantized bias value represents. +/// +/// Returns: +/// * `Output` output +/// * `Output` min_out: The float value that the lowest quantized output value represents. +/// * `Output` max_out: The float value that the highest quantized output value represents. +class QuantizedBiasAdd { + public: + QuantizedBiasAdd(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input bias, ::tensorflow::Input min_input, + ::tensorflow::Input max_input, ::tensorflow::Input min_bias, + ::tensorflow::Input max_bias, DataType out_type); + + Operation operation; + ::tensorflow::Output output; + ::tensorflow::Output min_out; + ::tensorflow::Output max_out; +}; + +/// Computes a 2D convolution given quantized 4D input and filter tensors. +/// +/// The inputs are quantized tensors where the lowest value represents the real +/// number of the associated minimum, and the highest represents the maximum. +/// This means that you can only interpret the quantized output in the same way, by +/// taking the returned minimum and maximum values into account. +/// +/// Arguments: +/// * scope: A Scope object +/// * filter: filter's input_depth dimension must match input's depth dimensions. +/// * min_input: The float value that the lowest quantized input value represents. +/// * max_input: The float value that the highest quantized input value represents. +/// * min_filter: The float value that the lowest quantized filter value represents. +/// * max_filter: The float value that the highest quantized filter value represents. +/// * strides: The stride of the sliding window for each dimension of the input +/// tensor. +/// * padding: The type of padding algorithm to use. +/// +/// Optional attributes (see `Attrs`): +/// * dilations: 1-D tensor of length 4. The dilation factor for each dimension of +/// `input`. If set to k > 1, there will be k-1 skipped cells between each +/// filter element on that dimension. The dimension order is determined by the +/// value of `data_format`, see above for details. Dilations in the batch and +/// depth dimensions must be 1. +/// +/// Returns: +/// * `Output` output +/// * `Output` min_output: The float value that the lowest quantized output value represents. +/// * `Output` max_output: The float value that the highest quantized output value represents. +class QuantizedConv2D { + public: + /// Optional attribute setters for QuantizedConv2D + struct Attrs { + /// Defaults to DT_QINT32 + TF_MUST_USE_RESULT Attrs OutType(DataType x) { + Attrs ret = *this; + ret.out_type_ = x; + return ret; + } + + /// 1-D tensor of length 4. The dilation factor for each dimension of + /// `input`. If set to k > 1, there will be k-1 skipped cells between each + /// filter element on that dimension. The dimension order is determined by the + /// value of `data_format`, see above for details. Dilations in the batch and + /// depth dimensions must be 1. + /// + /// Defaults to [1, 1, 1, 1] + TF_MUST_USE_RESULT Attrs Dilations(const gtl::ArraySlice& x) { + Attrs ret = *this; + ret.dilations_ = x; + return ret; + } + + DataType out_type_ = DT_QINT32; + gtl::ArraySlice dilations_ = Default_dilations(); + private: + static gtl::ArraySlice Default_dilations() { + static const int kStorage[] = {1, 1, 1, 1}; + return gtl::ArraySlice(kStorage); + } + }; + QuantizedConv2D(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input filter, ::tensorflow::Input min_input, + ::tensorflow::Input max_input, ::tensorflow::Input min_filter, + ::tensorflow::Input max_filter, const gtl::ArraySlice& + strides, StringPiece padding); + QuantizedConv2D(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input filter, ::tensorflow::Input min_input, + ::tensorflow::Input max_input, ::tensorflow::Input min_filter, + ::tensorflow::Input max_filter, const gtl::ArraySlice& + strides, StringPiece padding, const QuantizedConv2D::Attrs& + attrs); + + static Attrs OutType(DataType x) { + return Attrs().OutType(x); + } + static Attrs Dilations(const gtl::ArraySlice& x) { + return Attrs().Dilations(x); + } + + Operation operation; + ::tensorflow::Output output; + ::tensorflow::Output min_output; + ::tensorflow::Output max_output; +}; + +/// Produces the max pool of the input tensor for quantized types. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: The 4D (batch x rows x cols x depth) Tensor to MaxReduce over. +/// * min_input: The float value that the lowest quantized input value represents. +/// * max_input: The float value that the highest quantized input value represents. +/// * ksize: The size of the window for each dimension of the input tensor. +/// The length must be 4 to match the number of dimensions of the input. +/// * strides: The stride of the sliding window for each dimension of the input +/// tensor. The length must be 4 to match the number of dimensions of the input. +/// * padding: The type of padding algorithm to use. +/// +/// Returns: +/// * `Output` output +/// * `Output` min_output: The float value that the lowest quantized output value represents. +/// * `Output` max_output: The float value that the highest quantized output value represents. +class QuantizedMaxPool { + public: + QuantizedMaxPool(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input min_input, ::tensorflow::Input max_input, + const gtl::ArraySlice& ksize, const gtl::ArraySlice& + strides, StringPiece padding); + + Operation operation; + ::tensorflow::Output output; + ::tensorflow::Output min_output; + ::tensorflow::Output max_output; +}; + +/// Computes Quantized Rectified Linear: `max(features, 0)` +/// +/// Arguments: +/// * scope: A Scope object +/// * min_features: The float value that the lowest quantized value represents. +/// * max_features: The float value that the highest quantized value represents. +/// +/// Returns: +/// * `Output` activations: Has the same output shape as "features". +/// * `Output` min_activations: The float value that the lowest quantized value represents. +/// * `Output` max_activations: The float value that the highest quantized value represents. +class QuantizedRelu { + public: + /// Optional attribute setters for QuantizedRelu + struct Attrs { + /// Defaults to DT_QUINT8 + TF_MUST_USE_RESULT Attrs OutType(DataType x) { + Attrs ret = *this; + ret.out_type_ = x; + return ret; + } + + DataType out_type_ = DT_QUINT8; + }; + QuantizedRelu(const ::tensorflow::Scope& scope, ::tensorflow::Input features, + ::tensorflow::Input min_features, ::tensorflow::Input + max_features); + QuantizedRelu(const ::tensorflow::Scope& scope, ::tensorflow::Input features, + ::tensorflow::Input min_features, ::tensorflow::Input + max_features, const QuantizedRelu::Attrs& attrs); + + static Attrs OutType(DataType x) { + return Attrs().OutType(x); + } + + Operation operation; + ::tensorflow::Output activations; + ::tensorflow::Output min_activations; + ::tensorflow::Output max_activations; +}; + +/// Computes Quantized Rectified Linear 6: `min(max(features, 0), 6)` +/// +/// Arguments: +/// * scope: A Scope object +/// * min_features: The float value that the lowest quantized value represents. +/// * max_features: The float value that the highest quantized value represents. +/// +/// Returns: +/// * `Output` activations: Has the same output shape as "features". +/// * `Output` min_activations: The float value that the lowest quantized value represents. +/// * `Output` max_activations: The float value that the highest quantized value represents. +class QuantizedRelu6 { + public: + /// Optional attribute setters for QuantizedRelu6 + struct Attrs { + /// Defaults to DT_QUINT8 + TF_MUST_USE_RESULT Attrs OutType(DataType x) { + Attrs ret = *this; + ret.out_type_ = x; + return ret; + } + + DataType out_type_ = DT_QUINT8; + }; + QuantizedRelu6(const ::tensorflow::Scope& scope, ::tensorflow::Input features, + ::tensorflow::Input min_features, ::tensorflow::Input + max_features); + QuantizedRelu6(const ::tensorflow::Scope& scope, ::tensorflow::Input features, + ::tensorflow::Input min_features, ::tensorflow::Input + max_features, const QuantizedRelu6::Attrs& attrs); + + static Attrs OutType(DataType x) { + return Attrs().OutType(x); + } + + Operation operation; + ::tensorflow::Output activations; + ::tensorflow::Output min_activations; + ::tensorflow::Output max_activations; +}; + +/// Computes Quantized Rectified Linear X: `min(max(features, 0), max_value)` +/// +/// Arguments: +/// * scope: A Scope object +/// * min_features: The float value that the lowest quantized value represents. +/// * max_features: The float value that the highest quantized value represents. +/// +/// Returns: +/// * `Output` activations: Has the same output shape as "features". +/// * `Output` min_activations: The float value that the lowest quantized value represents. +/// * `Output` max_activations: The float value that the highest quantized value represents. +class QuantizedReluX { + public: + /// Optional attribute setters for QuantizedReluX + struct Attrs { + /// Defaults to DT_QUINT8 + TF_MUST_USE_RESULT Attrs OutType(DataType x) { + Attrs ret = *this; + ret.out_type_ = x; + return ret; + } + + DataType out_type_ = DT_QUINT8; + }; + QuantizedReluX(const ::tensorflow::Scope& scope, ::tensorflow::Input features, + ::tensorflow::Input max_value, ::tensorflow::Input min_features, + ::tensorflow::Input max_features); + QuantizedReluX(const ::tensorflow::Scope& scope, ::tensorflow::Input features, + ::tensorflow::Input max_value, ::tensorflow::Input min_features, + ::tensorflow::Input max_features, const QuantizedReluX::Attrs& + attrs); + + static Attrs OutType(DataType x) { + return Attrs().OutType(x); + } + + Operation operation; + ::tensorflow::Output activations; + ::tensorflow::Output min_activations; + ::tensorflow::Output max_activations; +}; + +/// Computes rectified linear: `max(features, 0)`. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The activations tensor. +class Relu { + public: + Relu(const ::tensorflow::Scope& scope, ::tensorflow::Input features); + operator ::tensorflow::Output() const { return activations; } + operator ::tensorflow::Input() const { return activations; } + ::tensorflow::Node* node() const { return activations.node(); } + + Operation operation; + ::tensorflow::Output activations; +}; + +/// Computes rectified linear 6: `min(max(features, 0), 6)`. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The activations tensor. +class Relu6 { + public: + Relu6(const ::tensorflow::Scope& scope, ::tensorflow::Input features); + operator ::tensorflow::Output() const { return activations; } + operator ::tensorflow::Input() const { return activations; } + ::tensorflow::Node* node() const { return activations.node(); } + + Operation operation; + ::tensorflow::Output activations; +}; + +/// Computes scaled exponential linear: `scale * alpha * (exp(features) - 1)` +/// +/// if < 0, `scale * features` otherwise. +/// +/// To be used together with +/// `initializer = tf.variance_scaling_initializer(factor=1.0, mode='FAN_IN')`. +/// For correct dropout, use `tf.contrib.nn.alpha_dropout`. +/// +/// See [Self-Normalizing Neural Networks](https://arxiv.org/abs/1706.02515) +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The activations tensor. +class Selu { + public: + Selu(const ::tensorflow::Scope& scope, ::tensorflow::Input features); + operator ::tensorflow::Output() const { return activations; } + operator ::tensorflow::Input() const { return activations; } + ::tensorflow::Node* node() const { return activations.node(); } + + Operation operation; + ::tensorflow::Output activations; +}; + +/// Computes softmax activations. +/// +/// For each batch `i` and class `j` we have +/// +/// $$softmax[i, j] = exp(logits[i, j]) / sum_j(exp(logits[i, j]))$$ +/// +/// Arguments: +/// * scope: A Scope object +/// * logits: 2-D with shape `[batch_size, num_classes]`. +/// +/// Returns: +/// * `Output`: Same shape as `logits`. +class Softmax { + public: + Softmax(const ::tensorflow::Scope& scope, ::tensorflow::Input logits); + operator ::tensorflow::Output() const { return softmax; } + operator ::tensorflow::Input() const { return softmax; } + ::tensorflow::Node* node() const { return softmax.node(); } + + Operation operation; + ::tensorflow::Output softmax; +}; + +/// Computes softmax cross entropy cost and gradients to backpropagate. +/// +/// Inputs are the logits, not probabilities. +/// +/// Arguments: +/// * scope: A Scope object +/// * features: batch_size x num_classes matrix +/// * labels: batch_size x num_classes matrix +/// The caller must ensure that each batch of labels represents a valid +/// probability distribution. +/// +/// Returns: +/// * `Output` loss: Per example loss (batch_size vector). +/// * `Output` backprop: backpropagated gradients (batch_size x num_classes matrix). +class SoftmaxCrossEntropyWithLogits { + public: + SoftmaxCrossEntropyWithLogits(const ::tensorflow::Scope& scope, + ::tensorflow::Input features, ::tensorflow::Input + labels); + + Operation operation; + ::tensorflow::Output loss; + ::tensorflow::Output backprop; +}; + +/// Computes softplus: `log(exp(features) + 1)`. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The activations tensor. +class Softplus { + public: + Softplus(const ::tensorflow::Scope& scope, ::tensorflow::Input features); + operator ::tensorflow::Output() const { return activations; } + operator ::tensorflow::Input() const { return activations; } + ::tensorflow::Node* node() const { return activations.node(); } + + Operation operation; + ::tensorflow::Output activations; +}; + +/// Computes softsign: `features / (abs(features) + 1)`. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The activations tensor. +class Softsign { + public: + Softsign(const ::tensorflow::Scope& scope, ::tensorflow::Input features); + operator ::tensorflow::Output() const { return activations; } + operator ::tensorflow::Input() const { return activations; } + ::tensorflow::Node* node() const { return activations.node(); } + + Operation operation; + ::tensorflow::Output activations; +}; + +/// Computes softmax cross entropy cost and gradients to backpropagate. +/// +/// Unlike `SoftmaxCrossEntropyWithLogits`, this operation does not accept +/// a matrix of label probabilities, but rather a single label per row +/// of features. This label is considered to have probability 1.0 for the +/// given row. +/// +/// Inputs are the logits, not probabilities. +/// +/// Arguments: +/// * scope: A Scope object +/// * features: batch_size x num_classes matrix +/// * labels: batch_size vector with values in [0, num_classes). +/// This is the label for the given minibatch entry. +/// +/// Returns: +/// * `Output` loss: Per example loss (batch_size vector). +/// * `Output` backprop: backpropagated gradients (batch_size x num_classes matrix). +class SparseSoftmaxCrossEntropyWithLogits { + public: + SparseSoftmaxCrossEntropyWithLogits(const ::tensorflow::Scope& scope, + ::tensorflow::Input features, + ::tensorflow::Input labels); + + Operation operation; + ::tensorflow::Output loss; + ::tensorflow::Output backprop; +}; + +/// Finds values and indices of the `k` largest elements for the last dimension. +/// +/// If the input is a vector (rank-1), finds the `k` largest entries in the vector +/// and outputs their values and indices as vectors. Thus `values[j]` is the +/// `j`-th largest entry in `input`, and its index is `indices[j]`. +/// +/// For matrices (resp. higher rank input), computes the top `k` entries in each +/// row (resp. vector along the last dimension). Thus, +/// +/// values.shape = indices.shape = input.shape[:-1] + [k] +/// +/// If two elements are equal, the lower-index element appears first. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: 1-D or higher with last dimension at least `k`. +/// * k: 0-D. Number of top elements to look for along the last dimension (along each +/// row for matrices). +/// +/// Optional attributes (see `Attrs`): +/// * sorted: If true the resulting `k` elements will be sorted by the values in +/// descending order. +/// +/// Returns: +/// * `Output` values: The `k` largest elements along each last dimensional slice. +/// * `Output` indices: The indices of `values` within the last dimension of `input`. +class TopK { + public: + /// Optional attribute setters for TopK + struct Attrs { + /// If true the resulting `k` elements will be sorted by the values in + /// descending order. + /// + /// Defaults to true + TF_MUST_USE_RESULT Attrs Sorted(bool x) { + Attrs ret = *this; + ret.sorted_ = x; + return ret; + } + + bool sorted_ = true; + }; + TopK(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input k); + TopK(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input k, const TopK::Attrs& attrs); + + static Attrs Sorted(bool x) { + return Attrs().Sorted(x); + } + + Operation operation; + ::tensorflow::Output values; + ::tensorflow::Output indices; +}; + +/// @} + +} // namespace ops +} // namespace tensorflow + +#endif // TENSORFLOW_CC_OPS_NN_OPS_H_ diff --git a/linux/include/tensorflow/cc/ops/nn_ops_internal.h b/linux/include/tensorflow/cc/ops/nn_ops_internal.h new file mode 100644 index 000000000..7f7d87ca6 --- /dev/null +++ b/linux/include/tensorflow/cc/ops/nn_ops_internal.h @@ -0,0 +1,541 @@ +// This file is MACHINE GENERATED! Do not edit. + +#ifndef TENSORFLOW_CC_OPS_NN_OPS_INTERNAL_H_ +#define TENSORFLOW_CC_OPS_NN_OPS_INTERNAL_H_ + +// This file is MACHINE GENERATED! Do not edit. + +#include "tensorflow/cc/framework/ops.h" +#include "tensorflow/cc/framework/scope.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/gtl/array_slice.h" + +namespace tensorflow { +namespace ops { +namespace internal { +// NOTE: This namespace has internal TensorFlow details that +// are not part of TensorFlow's public API. + +/// @defgroup nn_ops_internal Nn Ops Internal +/// @{ + +/// Computes gradients of the average pooling function. +/// +/// Arguments: +/// * scope: A Scope object +/// * orig_input_shape: 1-D. Shape of the original input to `avg_pool`. +/// * grad: 4-D with shape `[batch, height, width, channels]`. Gradients w.r.t. +/// the output of `avg_pool`. +/// * ksize: The size of the sliding window for each dimension of the input. +/// * strides: The stride of the sliding window for each dimension of the input. +/// * padding: The type of padding algorithm to use. +/// +/// Optional attributes (see `Attrs`): +/// * data_format: Specify the data format of the input and output data. With the +/// default format "NHWC", the data is stored in the order of: +/// [batch, in_height, in_width, in_channels]. +/// Alternatively, the format could be "NCHW", the data storage order of: +/// [batch, in_channels, in_height, in_width]. +/// +/// Returns: +/// * `Output`: 4-D. Gradients w.r.t. the input of `avg_pool`. +class AvgPoolGrad { + public: + /// Optional attribute setters for AvgPoolGrad + struct Attrs { + /// Specify the data format of the input and output data. With the + /// default format "NHWC", the data is stored in the order of: + /// [batch, in_height, in_width, in_channels]. + /// Alternatively, the format could be "NCHW", the data storage order of: + /// [batch, in_channels, in_height, in_width]. + /// + /// Defaults to "NHWC" + TF_MUST_USE_RESULT Attrs DataFormat(StringPiece x) { + Attrs ret = *this; + ret.data_format_ = x; + return ret; + } + + StringPiece data_format_ = "NHWC"; + }; + AvgPoolGrad(const ::tensorflow::Scope& scope, ::tensorflow::Input + orig_input_shape, ::tensorflow::Input grad, const + gtl::ArraySlice& ksize, const gtl::ArraySlice& strides, + StringPiece padding); + AvgPoolGrad(const ::tensorflow::Scope& scope, ::tensorflow::Input + orig_input_shape, ::tensorflow::Input grad, const + gtl::ArraySlice& ksize, const gtl::ArraySlice& strides, + StringPiece padding, const AvgPoolGrad::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs DataFormat(StringPiece x) { + return Attrs().DataFormat(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Computes gradients for the exponential linear (Elu) operation. +/// +/// Arguments: +/// * scope: A Scope object +/// * gradients: The backpropagated gradients to the corresponding Elu operation. +/// * outputs: The outputs of the corresponding Elu operation. +/// +/// Returns: +/// * `Output`: The gradients: `gradients * (outputs + 1)` if outputs < 0, +/// `gradients` otherwise. +class EluGrad { + public: + EluGrad(const ::tensorflow::Scope& scope, ::tensorflow::Input gradients, + ::tensorflow::Input outputs); + operator ::tensorflow::Output() const { return backprops; } + operator ::tensorflow::Input() const { return backprops; } + ::tensorflow::Node* node() const { return backprops.node(); } + + Operation operation; + ::tensorflow::Output backprops; +}; + +/// Computes gradient of the FractionalAvgPool function. +/// +/// Unlike FractionalMaxPoolGrad, we don't need to find arg_max for +/// FractionalAvgPoolGrad, we just need to evenly back-propagate each element of +/// out_backprop to those indices that form the same pooling cell. Therefore, we +/// just need to know the shape of original input tensor, instead of the whole +/// tensor. +/// +/// Arguments: +/// * scope: A Scope object +/// * orig_input_tensor_shape: Original input tensor shape for `fractional_avg_pool` +/// * out_backprop: 4-D with shape `[batch, height, width, channels]`. Gradients +/// w.r.t. the output of `fractional_avg_pool`. +/// * row_pooling_sequence: row pooling sequence, form pooling region with +/// col_pooling_sequence. +/// * col_pooling_sequence: column pooling sequence, form pooling region with +/// row_pooling sequence. +/// +/// Optional attributes (see `Attrs`): +/// * overlapping: When set to True, it means when pooling, the values at the boundary +/// of adjacent pooling cells are used by both cells. For example: +/// +/// `index 0 1 2 3 4` +/// +/// `value 20 5 16 3 7` +/// +/// If the pooling sequence is [0, 2, 4], then 16, at index 2 will be used twice. +/// The result would be [41/3, 26/3] for fractional avg pooling. +/// +/// Returns: +/// * `Output`: 4-D. Gradients w.r.t. the input of `fractional_avg_pool`. +class FractionalAvgPoolGrad { + public: + /// Optional attribute setters for FractionalAvgPoolGrad + struct Attrs { + /// When set to True, it means when pooling, the values at the boundary + /// of adjacent pooling cells are used by both cells. For example: + /// + /// `index 0 1 2 3 4` + /// + /// `value 20 5 16 3 7` + /// + /// If the pooling sequence is [0, 2, 4], then 16, at index 2 will be used twice. + /// The result would be [41/3, 26/3] for fractional avg pooling. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs Overlapping(bool x) { + Attrs ret = *this; + ret.overlapping_ = x; + return ret; + } + + bool overlapping_ = false; + }; + FractionalAvgPoolGrad(const ::tensorflow::Scope& scope, ::tensorflow::Input + orig_input_tensor_shape, ::tensorflow::Input + out_backprop, ::tensorflow::Input row_pooling_sequence, + ::tensorflow::Input col_pooling_sequence); + FractionalAvgPoolGrad(const ::tensorflow::Scope& scope, ::tensorflow::Input + orig_input_tensor_shape, ::tensorflow::Input + out_backprop, ::tensorflow::Input row_pooling_sequence, + ::tensorflow::Input col_pooling_sequence, const + FractionalAvgPoolGrad::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs Overlapping(bool x) { + return Attrs().Overlapping(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Computes gradient of the FractionalMaxPool function. +/// +/// Arguments: +/// * scope: A Scope object +/// * orig_input: Original input for `fractional_max_pool` +/// * orig_output: Original output for `fractional_max_pool` +/// * out_backprop: 4-D with shape `[batch, height, width, channels]`. Gradients +/// w.r.t. the output of `fractional_max_pool`. +/// * row_pooling_sequence: row pooling sequence, form pooling region with +/// col_pooling_sequence. +/// * col_pooling_sequence: column pooling sequence, form pooling region with +/// row_pooling sequence. +/// +/// Optional attributes (see `Attrs`): +/// * overlapping: When set to True, it means when pooling, the values at the boundary +/// of adjacent pooling cells are used by both cells. For example: +/// +/// `index 0 1 2 3 4` +/// +/// `value 20 5 16 3 7` +/// +/// If the pooling sequence is [0, 2, 4], then 16, at index 2 will be used twice. +/// The result would be [20, 16] for fractional max pooling. +/// +/// Returns: +/// * `Output`: 4-D. Gradients w.r.t. the input of `fractional_max_pool`. +class FractionalMaxPoolGrad { + public: + /// Optional attribute setters for FractionalMaxPoolGrad + struct Attrs { + /// When set to True, it means when pooling, the values at the boundary + /// of adjacent pooling cells are used by both cells. For example: + /// + /// `index 0 1 2 3 4` + /// + /// `value 20 5 16 3 7` + /// + /// If the pooling sequence is [0, 2, 4], then 16, at index 2 will be used twice. + /// The result would be [20, 16] for fractional max pooling. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs Overlapping(bool x) { + Attrs ret = *this; + ret.overlapping_ = x; + return ret; + } + + bool overlapping_ = false; + }; + FractionalMaxPoolGrad(const ::tensorflow::Scope& scope, ::tensorflow::Input + orig_input, ::tensorflow::Input orig_output, + ::tensorflow::Input out_backprop, ::tensorflow::Input + row_pooling_sequence, ::tensorflow::Input + col_pooling_sequence); + FractionalMaxPoolGrad(const ::tensorflow::Scope& scope, ::tensorflow::Input + orig_input, ::tensorflow::Input orig_output, + ::tensorflow::Input out_backprop, ::tensorflow::Input + row_pooling_sequence, ::tensorflow::Input + col_pooling_sequence, const FractionalMaxPoolGrad::Attrs& + attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs Overlapping(bool x) { + return Attrs().Overlapping(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Gradients for Local Response Normalization. +/// +/// Arguments: +/// * scope: A Scope object +/// * input_grads: 4-D with shape `[batch, height, width, channels]`. +/// * input_image: 4-D with shape `[batch, height, width, channels]`. +/// * output_image: 4-D with shape `[batch, height, width, channels]`. +/// +/// Optional attributes (see `Attrs`): +/// * depth_radius: A depth radius. +/// * bias: An offset (usually > 0 to avoid dividing by 0). +/// * alpha: A scale factor, usually positive. +/// * beta: An exponent. +/// +/// Returns: +/// * `Output`: The gradients for LRN. +class LRNGrad { + public: + /// Optional attribute setters for LRNGrad + struct Attrs { + /// A depth radius. + /// + /// Defaults to 5 + TF_MUST_USE_RESULT Attrs DepthRadius(int64 x) { + Attrs ret = *this; + ret.depth_radius_ = x; + return ret; + } + + /// An offset (usually > 0 to avoid dividing by 0). + /// + /// Defaults to 1 + TF_MUST_USE_RESULT Attrs Bias(float x) { + Attrs ret = *this; + ret.bias_ = x; + return ret; + } + + /// A scale factor, usually positive. + /// + /// Defaults to 1 + TF_MUST_USE_RESULT Attrs Alpha(float x) { + Attrs ret = *this; + ret.alpha_ = x; + return ret; + } + + /// An exponent. + /// + /// Defaults to 0.5 + TF_MUST_USE_RESULT Attrs Beta(float x) { + Attrs ret = *this; + ret.beta_ = x; + return ret; + } + + int64 depth_radius_ = 5; + float bias_ = 1.0f; + float alpha_ = 1.0f; + float beta_ = 0.5f; + }; + LRNGrad(const ::tensorflow::Scope& scope, ::tensorflow::Input input_grads, + ::tensorflow::Input input_image, ::tensorflow::Input output_image); + LRNGrad(const ::tensorflow::Scope& scope, ::tensorflow::Input input_grads, + ::tensorflow::Input input_image, ::tensorflow::Input output_image, + const LRNGrad::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs DepthRadius(int64 x) { + return Attrs().DepthRadius(x); + } + static Attrs Bias(float x) { + return Attrs().Bias(x); + } + static Attrs Alpha(float x) { + return Attrs().Alpha(x); + } + static Attrs Beta(float x) { + return Attrs().Beta(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Computes gradients of the maxpooling function. +/// +/// Arguments: +/// * scope: A Scope object +/// * orig_input: The original input tensor. +/// * orig_output: The original output tensor. +/// * grad: 4-D. Gradients w.r.t. the output of `max_pool`. +/// * ksize: The size of the window for each dimension of the input tensor. +/// * strides: The stride of the sliding window for each dimension of the +/// input tensor. +/// * padding: The type of padding algorithm to use. +/// +/// Optional attributes (see `Attrs`): +/// * data_format: Specify the data format of the input and output data. With the +/// default format "NHWC", the data is stored in the order of: +/// [batch, in_height, in_width, in_channels]. +/// Alternatively, the format could be "NCHW", the data storage order of: +/// [batch, in_channels, in_height, in_width]. +/// +/// Returns: +/// * `Output`: Gradients w.r.t. the input to `max_pool`. +class MaxPoolGrad { + public: + /// Optional attribute setters for MaxPoolGrad + struct Attrs { + /// Specify the data format of the input and output data. With the + /// default format "NHWC", the data is stored in the order of: + /// [batch, in_height, in_width, in_channels]. + /// Alternatively, the format could be "NCHW", the data storage order of: + /// [batch, in_channels, in_height, in_width]. + /// + /// Defaults to "NHWC" + TF_MUST_USE_RESULT Attrs DataFormat(StringPiece x) { + Attrs ret = *this; + ret.data_format_ = x; + return ret; + } + + StringPiece data_format_ = "NHWC"; + }; + MaxPoolGrad(const ::tensorflow::Scope& scope, ::tensorflow::Input orig_input, + ::tensorflow::Input orig_output, ::tensorflow::Input grad, const + gtl::ArraySlice& ksize, const gtl::ArraySlice& strides, + StringPiece padding); + MaxPoolGrad(const ::tensorflow::Scope& scope, ::tensorflow::Input orig_input, + ::tensorflow::Input orig_output, ::tensorflow::Input grad, const + gtl::ArraySlice& ksize, const gtl::ArraySlice& strides, + StringPiece padding, const MaxPoolGrad::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs DataFormat(StringPiece x) { + return Attrs().DataFormat(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Computes gradients of the maxpooling function. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: The original input. +/// * grad: 4-D with shape `[batch, height, width, channels]`. Gradients w.r.t. the +/// output of `max_pool`. +/// * argmax: The indices of the maximum values chosen for each output of `max_pool`. +/// * ksize: The size of the window for each dimension of the input tensor. +/// * strides: The stride of the sliding window for each dimension of the +/// input tensor. +/// * padding: The type of padding algorithm to use. +/// +/// Returns: +/// * `Output`: Gradients w.r.t. the input of `max_pool`. +class MaxPoolGradWithArgmax { + public: + MaxPoolGradWithArgmax(const ::tensorflow::Scope& scope, ::tensorflow::Input + input, ::tensorflow::Input grad, ::tensorflow::Input + argmax, const gtl::ArraySlice& ksize, const + gtl::ArraySlice& strides, StringPiece padding); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Computes rectified linear 6 gradients for a Relu6 operation. +/// +/// Arguments: +/// * scope: A Scope object +/// * gradients: The backpropagated gradients to the corresponding Relu6 operation. +/// * features: The features passed as input to the corresponding Relu6 operation, or +/// its output; using either one produces the same result. +/// +/// Returns: +/// * `Output`: The gradients: +/// `gradients * (features > 0) * (features < 6)`. +class Relu6Grad { + public: + Relu6Grad(const ::tensorflow::Scope& scope, ::tensorflow::Input gradients, + ::tensorflow::Input features); + operator ::tensorflow::Output() const { return backprops; } + operator ::tensorflow::Input() const { return backprops; } + ::tensorflow::Node* node() const { return backprops.node(); } + + Operation operation; + ::tensorflow::Output backprops; +}; + +/// Computes rectified linear gradients for a Relu operation. +/// +/// Arguments: +/// * scope: A Scope object +/// * gradients: The backpropagated gradients to the corresponding Relu operation. +/// * features: The features passed as input to the corresponding Relu operation, OR +/// the outputs of that operation (both work equivalently). +/// +/// Returns: +/// * `Output`: `gradients * (features > 0)`. +class ReluGrad { + public: + ReluGrad(const ::tensorflow::Scope& scope, ::tensorflow::Input gradients, + ::tensorflow::Input features); + operator ::tensorflow::Output() const { return backprops; } + operator ::tensorflow::Input() const { return backprops; } + ::tensorflow::Node* node() const { return backprops.node(); } + + Operation operation; + ::tensorflow::Output backprops; +}; + +/// Computes gradients for the scaled exponential linear (Selu) operation. +/// +/// Arguments: +/// * scope: A Scope object +/// * gradients: The backpropagated gradients to the corresponding Selu operation. +/// * outputs: The outputs of the corresponding Selu operation. +/// +/// Returns: +/// * `Output`: The gradients: `gradients * (outputs + scale * alpha)` +/// if outputs < 0, `scale * gradients` otherwise. +class SeluGrad { + public: + SeluGrad(const ::tensorflow::Scope& scope, ::tensorflow::Input gradients, + ::tensorflow::Input outputs); + operator ::tensorflow::Output() const { return backprops; } + operator ::tensorflow::Input() const { return backprops; } + ::tensorflow::Node* node() const { return backprops.node(); } + + Operation operation; + ::tensorflow::Output backprops; +}; + +/// Computes softplus gradients for a softplus operation. +/// +/// Arguments: +/// * scope: A Scope object +/// * gradients: The backpropagated gradients to the corresponding softplus operation. +/// * features: The features passed as input to the corresponding softplus operation. +/// +/// Returns: +/// * `Output`: The gradients: `gradients / (1 + exp(-features))`. +class SoftplusGrad { + public: + SoftplusGrad(const ::tensorflow::Scope& scope, ::tensorflow::Input gradients, + ::tensorflow::Input features); + operator ::tensorflow::Output() const { return backprops; } + operator ::tensorflow::Input() const { return backprops; } + ::tensorflow::Node* node() const { return backprops.node(); } + + Operation operation; + ::tensorflow::Output backprops; +}; + +/// Computes softsign gradients for a softsign operation. +/// +/// Arguments: +/// * scope: A Scope object +/// * gradients: The backpropagated gradients to the corresponding softsign operation. +/// * features: The features passed as input to the corresponding softsign operation. +/// +/// Returns: +/// * `Output`: The gradients: `gradients / (1 + abs(features)) ** 2`. +class SoftsignGrad { + public: + SoftsignGrad(const ::tensorflow::Scope& scope, ::tensorflow::Input gradients, + ::tensorflow::Input features); + operator ::tensorflow::Output() const { return backprops; } + operator ::tensorflow::Input() const { return backprops; } + ::tensorflow::Node* node() const { return backprops.node(); } + + Operation operation; + ::tensorflow::Output backprops; +}; + +} // namespace internal +} // namespace ops +} // namespace tensorflow + +#endif // TENSORFLOW_CC_OPS_NN_OPS_INTERNAL_H_ diff --git a/linux/include/tensorflow/cc/ops/no_op.h b/linux/include/tensorflow/cc/ops/no_op.h new file mode 100644 index 000000000..99298ee4b --- /dev/null +++ b/linux/include/tensorflow/cc/ops/no_op.h @@ -0,0 +1,41 @@ +// This file is MACHINE GENERATED! Do not edit. + +#ifndef TENSORFLOW_CC_OPS_NO_OP_H_ +#define TENSORFLOW_CC_OPS_NO_OP_H_ + +// This file is MACHINE GENERATED! Do not edit. + +#include "tensorflow/cc/framework/ops.h" +#include "tensorflow/cc/framework/scope.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/gtl/array_slice.h" + +namespace tensorflow { +namespace ops { + +/// @defgroup no_op No Op +/// @{ + +/// Does nothing. Only useful as a placeholder for control edges. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * the created `Operation` +class NoOp { + public: + NoOp(const ::tensorflow::Scope& scope); + operator ::tensorflow::Operation() const { return operation; } + + Operation operation; +}; + +/// @} + +} // namespace ops +} // namespace tensorflow + +#endif // TENSORFLOW_CC_OPS_NO_OP_H_ diff --git a/linux/include/tensorflow/cc/ops/no_op_internal.h b/linux/include/tensorflow/cc/ops/no_op_internal.h new file mode 100644 index 000000000..1a3a6854e --- /dev/null +++ b/linux/include/tensorflow/cc/ops/no_op_internal.h @@ -0,0 +1,28 @@ +// This file is MACHINE GENERATED! Do not edit. + +#ifndef TENSORFLOW_CC_OPS_NO_OP_INTERNAL_H_ +#define TENSORFLOW_CC_OPS_NO_OP_INTERNAL_H_ + +// This file is MACHINE GENERATED! Do not edit. + +#include "tensorflow/cc/framework/ops.h" +#include "tensorflow/cc/framework/scope.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/gtl/array_slice.h" + +namespace tensorflow { +namespace ops { +namespace internal { +// NOTE: This namespace has internal TensorFlow details that +// are not part of TensorFlow's public API. + +/// @defgroup no_op_internal No Op Internal +/// @{ + +} // namespace internal +} // namespace ops +} // namespace tensorflow + +#endif // TENSORFLOW_CC_OPS_NO_OP_INTERNAL_H_ diff --git a/linux/include/tensorflow/cc/ops/parsing_ops.h b/linux/include/tensorflow/cc/ops/parsing_ops.h new file mode 100644 index 000000000..b6d3b831b --- /dev/null +++ b/linux/include/tensorflow/cc/ops/parsing_ops.h @@ -0,0 +1,854 @@ +// This file is MACHINE GENERATED! Do not edit. + +#ifndef TENSORFLOW_CC_OPS_PARSING_OPS_H_ +#define TENSORFLOW_CC_OPS_PARSING_OPS_H_ + +// This file is MACHINE GENERATED! Do not edit. + +#include "tensorflow/cc/framework/ops.h" +#include "tensorflow/cc/framework/scope.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/gtl/array_slice.h" + +namespace tensorflow { +namespace ops { + +/// @defgroup parsing_ops Parsing Ops +/// @{ + +/// Convert CSV records to tensors. Each column maps to one tensor. +/// +/// RFC 4180 format is expected for the CSV records. +/// (https://tools.ietf.org/html/rfc4180) +/// Note that we allow leading and trailing spaces with int or float field. +/// +/// Arguments: +/// * scope: A Scope object +/// * records: Each string is a record/row in the csv and all records should have +/// the same format. +/// * record_defaults: One tensor per column of the input record, with either a +/// scalar default value for that column or an empty vector if the column is +/// required. +/// +/// Optional attributes (see `Attrs`): +/// * field_delim: char delimiter to separate fields in a record. +/// * use_quote_delim: If false, treats double quotation marks as regular +/// characters inside of the string fields (ignoring RFC 4180, Section 2, +/// Bullet 5). +/// * na_value: Additional string to recognize as NA/NaN. +/// +/// Returns: +/// * `OutputList`: Each tensor will have the same shape as records. +class DecodeCSV { + public: + /// Optional attribute setters for DecodeCSV + struct Attrs { + /// char delimiter to separate fields in a record. + /// + /// Defaults to "," + TF_MUST_USE_RESULT Attrs FieldDelim(StringPiece x) { + Attrs ret = *this; + ret.field_delim_ = x; + return ret; + } + + /// If false, treats double quotation marks as regular + /// characters inside of the string fields (ignoring RFC 4180, Section 2, + /// Bullet 5). + /// + /// Defaults to true + TF_MUST_USE_RESULT Attrs UseQuoteDelim(bool x) { + Attrs ret = *this; + ret.use_quote_delim_ = x; + return ret; + } + + /// Additional string to recognize as NA/NaN. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs NaValue(StringPiece x) { + Attrs ret = *this; + ret.na_value_ = x; + return ret; + } + + /// Defaults to [] + TF_MUST_USE_RESULT Attrs SelectCols(const gtl::ArraySlice& x) { + Attrs ret = *this; + ret.select_cols_ = x; + return ret; + } + + StringPiece field_delim_ = ","; + bool use_quote_delim_ = true; + StringPiece na_value_ = ""; + gtl::ArraySlice select_cols_ = {}; + }; + DecodeCSV(const ::tensorflow::Scope& scope, ::tensorflow::Input records, + ::tensorflow::InputList record_defaults); + DecodeCSV(const ::tensorflow::Scope& scope, ::tensorflow::Input records, + ::tensorflow::InputList record_defaults, const DecodeCSV::Attrs& + attrs); + ::tensorflow::Output operator[](size_t index) const { return output[index]; } + + + static Attrs FieldDelim(StringPiece x) { + return Attrs().FieldDelim(x); + } + static Attrs UseQuoteDelim(bool x) { + return Attrs().UseQuoteDelim(x); + } + static Attrs NaValue(StringPiece x) { + return Attrs().NaValue(x); + } + static Attrs SelectCols(const gtl::ArraySlice& x) { + return Attrs().SelectCols(x); + } + + Operation operation; + ::tensorflow::OutputList output; +}; + +/// Decompress strings. +/// +/// This op decompresses each element of the `bytes` input `Tensor`, which +/// is assumed to be compressed using the given `compression_type`. +/// +/// The `output` is a string `Tensor` of the same shape as `bytes`, +/// each element containing the decompressed data from the corresponding +/// element in `bytes`. +/// +/// Arguments: +/// * scope: A Scope object +/// * bytes: A Tensor of string which is compressed. +/// +/// Optional attributes (see `Attrs`): +/// * compression_type: A scalar containing either (i) the empty string (no +/// compression), (ii) "ZLIB", or (iii) "GZIP". +/// +/// Returns: +/// * `Output`: A Tensor with the same shape as input `bytes`, uncompressed +/// from bytes. +class DecodeCompressed { + public: + /// Optional attribute setters for DecodeCompressed + struct Attrs { + /// A scalar containing either (i) the empty string (no + /// compression), (ii) "ZLIB", or (iii) "GZIP". + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs CompressionType(StringPiece x) { + Attrs ret = *this; + ret.compression_type_ = x; + return ret; + } + + StringPiece compression_type_ = ""; + }; + DecodeCompressed(const ::tensorflow::Scope& scope, ::tensorflow::Input bytes); + DecodeCompressed(const ::tensorflow::Scope& scope, ::tensorflow::Input bytes, + const DecodeCompressed::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs CompressionType(StringPiece x) { + return Attrs().CompressionType(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Convert JSON-encoded Example records to binary protocol buffer strings. +/// +/// This op translates a tensor containing Example records, encoded using +/// the [standard JSON +/// mapping](https://developers.google.com/protocol-buffers/docs/proto3#json), +/// into a tensor containing the same records encoded as binary protocol +/// buffers. The resulting tensor can then be fed to any of the other +/// Example-parsing ops. +/// +/// Arguments: +/// * scope: A Scope object +/// * json_examples: Each string is a JSON object serialized according to the JSON +/// mapping of the Example proto. +/// +/// Returns: +/// * `Output`: Each string is a binary Example protocol buffer corresponding +/// to the respective element of `json_examples`. +class DecodeJSONExample { + public: + DecodeJSONExample(const ::tensorflow::Scope& scope, ::tensorflow::Input + json_examples); + operator ::tensorflow::Output() const { return binary_examples; } + operator ::tensorflow::Input() const { return binary_examples; } + ::tensorflow::Node* node() const { return binary_examples.node(); } + + Operation operation; + ::tensorflow::Output binary_examples; +}; + +/// Reinterpret the bytes of a string as a vector of numbers. +/// +/// Arguments: +/// * scope: A Scope object +/// * bytes: All the elements must have the same length. +/// +/// Optional attributes (see `Attrs`): +/// * little_endian: Whether the input `bytes` are in little-endian order. +/// Ignored for `out_type` values that are stored in a single byte like +/// `uint8`. +/// +/// Returns: +/// * `Output`: A Tensor with one more dimension than the input `bytes`. The +/// added dimension will have size equal to the length of the elements +/// of `bytes` divided by the number of bytes to represent `out_type`. +class DecodeRaw { + public: + /// Optional attribute setters for DecodeRaw + struct Attrs { + /// Whether the input `bytes` are in little-endian order. + /// Ignored for `out_type` values that are stored in a single byte like + /// `uint8`. + /// + /// Defaults to true + TF_MUST_USE_RESULT Attrs LittleEndian(bool x) { + Attrs ret = *this; + ret.little_endian_ = x; + return ret; + } + + bool little_endian_ = true; + }; + DecodeRaw(const ::tensorflow::Scope& scope, ::tensorflow::Input bytes, DataType + out_type); + DecodeRaw(const ::tensorflow::Scope& scope, ::tensorflow::Input bytes, DataType + out_type, const DecodeRaw::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs LittleEndian(bool x) { + return Attrs().LittleEndian(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Transforms a vector of brain.Example protos (as strings) into typed tensors. +/// +/// Arguments: +/// * scope: A Scope object +/// * serialized: A vector containing a batch of binary serialized Example protos. +/// * names: A vector containing the names of the serialized protos. +/// May contain, for example, table key (descriptive) names for the +/// corresponding serialized protos. These are purely useful for debugging +/// purposes, and the presence of values here has no effect on the output. +/// May also be an empty vector if no names are available. +/// If non-empty, this vector must be the same length as "serialized". +/// * sparse_keys: A list of Nsparse string Tensors (scalars). +/// The keys expected in the Examples' features associated with sparse values. +/// * dense_keys: A list of Ndense string Tensors (scalars). +/// The keys expected in the Examples' features associated with dense values. +/// * dense_defaults: A list of Ndense Tensors (some may be empty). +/// dense_defaults[j] provides default values +/// when the example's feature_map lacks dense_key[j]. If an empty Tensor is +/// provided for dense_defaults[j], then the Feature dense_keys[j] is required. +/// The input type is inferred from dense_defaults[j], even when it's empty. +/// If dense_defaults[j] is not empty, and dense_shapes[j] is fully defined, +/// then the shape of dense_defaults[j] must match that of dense_shapes[j]. +/// If dense_shapes[j] has an undefined major dimension (variable strides dense +/// feature), dense_defaults[j] must contain a single element: +/// the padding element. +/// * sparse_types: A list of Nsparse types; the data types of data in each Feature +/// given in sparse_keys. +/// Currently the ParseExample supports DT_FLOAT (FloatList), +/// DT_INT64 (Int64List), and DT_STRING (BytesList). +/// * dense_shapes: A list of Ndense shapes; the shapes of data in each Feature +/// given in dense_keys. +/// The number of elements in the Feature corresponding to dense_key[j] +/// must always equal dense_shapes[j].NumEntries(). +/// If dense_shapes[j] == (D0, D1, ..., DN) then the shape of output +/// Tensor dense_values[j] will be (|serialized|, D0, D1, ..., DN): +/// The dense outputs are just the inputs row-stacked by batch. +/// This works for dense_shapes[j] = (-1, D1, ..., DN). In this case +/// the shape of the output Tensor dense_values[j] will be +/// (|serialized|, M, D1, .., DN), where M is the maximum number of blocks +/// of elements of length D1 * .... * DN, across all minibatch entries +/// in the input. Any minibatch entry with less than M blocks of elements of +/// length D1 * ... * DN will be padded with the corresponding default_value +/// scalar element along the second dimension. +/// +/// Returns: +/// * `OutputList` sparse_indices +/// * `OutputList` sparse_values +/// * `OutputList` sparse_shapes +/// * `OutputList` dense_values +class ParseExample { + public: + ParseExample(const ::tensorflow::Scope& scope, ::tensorflow::Input serialized, + ::tensorflow::Input names, ::tensorflow::InputList sparse_keys, + ::tensorflow::InputList dense_keys, ::tensorflow::InputList + dense_defaults, const DataTypeSlice& sparse_types, const + gtl::ArraySlice& dense_shapes); + + Operation operation; + ::tensorflow::OutputList sparse_indices; + ::tensorflow::OutputList sparse_values; + ::tensorflow::OutputList sparse_shapes; + ::tensorflow::OutputList dense_values; +}; + +/// Transforms a vector of brain.SequenceExample protos (as strings) into typed tensors. +/// +/// Arguments: +/// * scope: A Scope object +/// * serialized: A vector containing binary serialized SequenceExample protos. +/// * debug_name: A vector containing the names of the serialized protos. +/// May contain, for example, table key (descriptive) name for the +/// corresponding serialized proto. This is purely useful for debugging +/// purposes, and the presence of values here has no effect on the output. +/// May also be an empty vector if no name is available. +/// * context_dense_defaults: A list of Ncontext_dense Tensors (some may be empty). +/// context_dense_defaults[j] provides default values +/// when the SequenceExample's context map lacks context_dense_key[j]. +/// If an empty Tensor is provided for context_dense_defaults[j], +/// then the Feature context_dense_keys[j] is required. +/// The input type is inferred from context_dense_defaults[j], even when it's +/// empty. If context_dense_defaults[j] is not empty, its shape must match +/// context_dense_shapes[j]. +/// * feature_list_dense_missing_assumed_empty: A vector listing the +/// FeatureList keys which may be missing from the SequenceExamples. If the +/// associated FeatureList is missing, it is treated as empty. By default, +/// any FeatureList not listed in this vector must exist in the SequenceExamples. +/// * context_sparse_keys: A list of Ncontext_sparse string Tensors (scalars). +/// The keys expected in the Examples' features associated with context_sparse +/// values. +/// * context_dense_keys: A list of Ncontext_dense string Tensors (scalars). +/// The keys expected in the SequenceExamples' context features associated with +/// dense values. +/// * feature_list_sparse_keys: A list of Nfeature_list_sparse string Tensors +/// (scalars). The keys expected in the FeatureLists associated with sparse +/// values. +/// * feature_list_dense_keys: A list of Nfeature_list_dense string Tensors (scalars). +/// The keys expected in the SequenceExamples' feature_lists associated +/// with lists of dense values. +/// +/// Optional attributes (see `Attrs`): +/// * context_sparse_types: A list of Ncontext_sparse types; the data types of data in +/// each context Feature given in context_sparse_keys. +/// Currently the ParseSingleSequenceExample supports DT_FLOAT (FloatList), +/// DT_INT64 (Int64List), and DT_STRING (BytesList). +/// * context_dense_shapes: A list of Ncontext_dense shapes; the shapes of data in +/// each context Feature given in context_dense_keys. +/// The number of elements in the Feature corresponding to context_dense_key[j] +/// must always equal context_dense_shapes[j].NumEntries(). +/// The shape of context_dense_values[j] will match context_dense_shapes[j]. +/// * feature_list_sparse_types: A list of Nfeature_list_sparse types; the data types +/// of data in each FeatureList given in feature_list_sparse_keys. +/// Currently the ParseSingleSequenceExample supports DT_FLOAT (FloatList), +/// DT_INT64 (Int64List), and DT_STRING (BytesList). +/// * feature_list_dense_shapes: A list of Nfeature_list_dense shapes; the shapes of +/// data in each FeatureList given in feature_list_dense_keys. +/// The shape of each Feature in the FeatureList corresponding to +/// feature_list_dense_key[j] must always equal +/// feature_list_dense_shapes[j].NumEntries(). +/// +/// Returns: +/// * `OutputList` context_sparse_indices +/// * `OutputList` context_sparse_values +/// * `OutputList` context_sparse_shapes +/// * `OutputList` context_dense_values +/// * `OutputList` feature_list_sparse_indices +/// * `OutputList` feature_list_sparse_values +/// * `OutputList` feature_list_sparse_shapes +/// * `OutputList` feature_list_dense_values +/// * `OutputList` feature_list_dense_lengths +class ParseSequenceExample { + public: + /// Optional attribute setters for ParseSequenceExample + struct Attrs { + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs NcontextSparse(int64 x) { + Attrs ret = *this; + ret.Ncontext_sparse_ = x; + return ret; + } + + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs NcontextDense(int64 x) { + Attrs ret = *this; + ret.Ncontext_dense_ = x; + return ret; + } + + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs NfeatureListSparse(int64 x) { + Attrs ret = *this; + ret.Nfeature_list_sparse_ = x; + return ret; + } + + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs NfeatureListDense(int64 x) { + Attrs ret = *this; + ret.Nfeature_list_dense_ = x; + return ret; + } + + /// A list of Ncontext_sparse types; the data types of data in + /// each context Feature given in context_sparse_keys. + /// Currently the ParseSingleSequenceExample supports DT_FLOAT (FloatList), + /// DT_INT64 (Int64List), and DT_STRING (BytesList). + /// + /// Defaults to [] + TF_MUST_USE_RESULT Attrs ContextSparseTypes(const DataTypeSlice& x) { + Attrs ret = *this; + ret.context_sparse_types_ = x; + return ret; + } + + /// Defaults to [] + TF_MUST_USE_RESULT Attrs FeatureListDenseTypes(const DataTypeSlice& x) { + Attrs ret = *this; + ret.feature_list_dense_types_ = x; + return ret; + } + + /// A list of Ncontext_dense shapes; the shapes of data in + /// each context Feature given in context_dense_keys. + /// The number of elements in the Feature corresponding to context_dense_key[j] + /// must always equal context_dense_shapes[j].NumEntries(). + /// The shape of context_dense_values[j] will match context_dense_shapes[j]. + /// + /// Defaults to [] + TF_MUST_USE_RESULT Attrs ContextDenseShapes(const gtl::ArraySlice& x) { + Attrs ret = *this; + ret.context_dense_shapes_ = x; + return ret; + } + + /// A list of Nfeature_list_sparse types; the data types + /// of data in each FeatureList given in feature_list_sparse_keys. + /// Currently the ParseSingleSequenceExample supports DT_FLOAT (FloatList), + /// DT_INT64 (Int64List), and DT_STRING (BytesList). + /// + /// Defaults to [] + TF_MUST_USE_RESULT Attrs FeatureListSparseTypes(const DataTypeSlice& x) { + Attrs ret = *this; + ret.feature_list_sparse_types_ = x; + return ret; + } + + /// A list of Nfeature_list_dense shapes; the shapes of + /// data in each FeatureList given in feature_list_dense_keys. + /// The shape of each Feature in the FeatureList corresponding to + /// feature_list_dense_key[j] must always equal + /// feature_list_dense_shapes[j].NumEntries(). + /// + /// Defaults to [] + TF_MUST_USE_RESULT Attrs FeatureListDenseShapes(const gtl::ArraySlice& x) { + Attrs ret = *this; + ret.feature_list_dense_shapes_ = x; + return ret; + } + + int64 Ncontext_sparse_ = 0; + int64 Ncontext_dense_ = 0; + int64 Nfeature_list_sparse_ = 0; + int64 Nfeature_list_dense_ = 0; + DataTypeSlice context_sparse_types_ = {}; + DataTypeSlice feature_list_dense_types_ = {}; + gtl::ArraySlice context_dense_shapes_ = {}; + DataTypeSlice feature_list_sparse_types_ = {}; + gtl::ArraySlice feature_list_dense_shapes_ = {}; + }; + ParseSequenceExample(const ::tensorflow::Scope& scope, ::tensorflow::Input + serialized, ::tensorflow::Input debug_name, + ::tensorflow::InputList context_dense_defaults, const + gtl::ArraySlice& + feature_list_dense_missing_assumed_empty, const + gtl::ArraySlice& context_sparse_keys, const + gtl::ArraySlice& context_dense_keys, const + gtl::ArraySlice& feature_list_sparse_keys, const + gtl::ArraySlice& feature_list_dense_keys); + ParseSequenceExample(const ::tensorflow::Scope& scope, ::tensorflow::Input + serialized, ::tensorflow::Input debug_name, + ::tensorflow::InputList context_dense_defaults, const + gtl::ArraySlice& + feature_list_dense_missing_assumed_empty, const + gtl::ArraySlice& context_sparse_keys, const + gtl::ArraySlice& context_dense_keys, const + gtl::ArraySlice& feature_list_sparse_keys, const + gtl::ArraySlice& feature_list_dense_keys, const + ParseSequenceExample::Attrs& attrs); + + static Attrs NcontextSparse(int64 x) { + return Attrs().NcontextSparse(x); + } + static Attrs NcontextDense(int64 x) { + return Attrs().NcontextDense(x); + } + static Attrs NfeatureListSparse(int64 x) { + return Attrs().NfeatureListSparse(x); + } + static Attrs NfeatureListDense(int64 x) { + return Attrs().NfeatureListDense(x); + } + static Attrs ContextSparseTypes(const DataTypeSlice& x) { + return Attrs().ContextSparseTypes(x); + } + static Attrs FeatureListDenseTypes(const DataTypeSlice& x) { + return Attrs().FeatureListDenseTypes(x); + } + static Attrs ContextDenseShapes(const gtl::ArraySlice& x) { + return Attrs().ContextDenseShapes(x); + } + static Attrs FeatureListSparseTypes(const DataTypeSlice& x) { + return Attrs().FeatureListSparseTypes(x); + } + static Attrs FeatureListDenseShapes(const gtl::ArraySlice& x) { + return Attrs().FeatureListDenseShapes(x); + } + + Operation operation; + ::tensorflow::OutputList context_sparse_indices; + ::tensorflow::OutputList context_sparse_values; + ::tensorflow::OutputList context_sparse_shapes; + ::tensorflow::OutputList context_dense_values; + ::tensorflow::OutputList feature_list_sparse_indices; + ::tensorflow::OutputList feature_list_sparse_values; + ::tensorflow::OutputList feature_list_sparse_shapes; + ::tensorflow::OutputList feature_list_dense_values; + ::tensorflow::OutputList feature_list_dense_lengths; +}; + +/// Transforms a tf.Example proto (as a string) into typed tensors. +/// +/// Arguments: +/// * scope: A Scope object +/// * serialized: A vector containing a batch of binary serialized Example protos. +/// * dense_defaults: A list of Tensors (some may be empty), whose length matches +/// the length of `dense_keys`. dense_defaults[j] provides default values +/// when the example's feature_map lacks dense_key[j]. If an empty Tensor is +/// provided for dense_defaults[j], then the Feature dense_keys[j] is required. +/// The input type is inferred from dense_defaults[j], even when it's empty. +/// If dense_defaults[j] is not empty, and dense_shapes[j] is fully defined, +/// then the shape of dense_defaults[j] must match that of dense_shapes[j]. +/// If dense_shapes[j] has an undefined major dimension (variable strides dense +/// feature), dense_defaults[j] must contain a single element: +/// the padding element. +/// * num_sparse: The number of sparse features to be parsed from the example. This +/// must match the lengths of `sparse_keys` and `sparse_types`. +/// * sparse_keys: A list of `num_sparse` strings. +/// The keys expected in the Examples' features associated with sparse values. +/// * dense_keys: The keys expected in the Examples' features associated with dense +/// values. +/// * sparse_types: A list of `num_sparse` types; the data types of data in each +/// Feature given in sparse_keys. +/// Currently the ParseSingleExample op supports DT_FLOAT (FloatList), +/// DT_INT64 (Int64List), and DT_STRING (BytesList). +/// * dense_shapes: The shapes of data in each Feature given in dense_keys. +/// The length of this list must match the length of `dense_keys`. The +/// number of elements in the Feature corresponding to dense_key[j] must +/// always equal dense_shapes[j].NumEntries(). If dense_shapes[j] == +/// (D0, D1, ..., DN) then the shape of output Tensor dense_values[j] +/// will be (D0, D1, ..., DN): In the case dense_shapes[j] = (-1, D1, +/// ..., DN), the shape of the output Tensor dense_values[j] will be (M, +/// D1, .., DN), where M is the number of blocks of elements of length +/// D1 * .... * DN, in the input. +/// +/// Returns: +/// * `OutputList` sparse_indices +/// * `OutputList` sparse_values +/// * `OutputList` sparse_shapes +/// * `OutputList` dense_values +class ParseSingleExample { + public: + ParseSingleExample(const ::tensorflow::Scope& scope, ::tensorflow::Input + serialized, ::tensorflow::InputList dense_defaults, int64 + num_sparse, const gtl::ArraySlice& sparse_keys, + const gtl::ArraySlice& dense_keys, const + DataTypeSlice& sparse_types, const + gtl::ArraySlice& dense_shapes); + + Operation operation; + ::tensorflow::OutputList sparse_indices; + ::tensorflow::OutputList sparse_values; + ::tensorflow::OutputList sparse_shapes; + ::tensorflow::OutputList dense_values; +}; + +/// Transforms a scalar brain.SequenceExample proto (as strings) into typed tensors. +/// +/// Arguments: +/// * scope: A Scope object +/// * serialized: A scalar containing a binary serialized SequenceExample proto. +/// * feature_list_dense_missing_assumed_empty: A vector listing the +/// FeatureList keys which may be missing from the SequenceExample. If the +/// associated FeatureList is missing, it is treated as empty. By default, +/// any FeatureList not listed in this vector must exist in the SequenceExample. +/// * context_sparse_keys: A list of Ncontext_sparse string Tensors (scalars). +/// The keys expected in the Examples' features associated with context_sparse +/// values. +/// * context_dense_keys: A list of Ncontext_dense string Tensors (scalars). +/// The keys expected in the SequenceExamples' context features associated with +/// dense values. +/// * feature_list_sparse_keys: A list of Nfeature_list_sparse string Tensors +/// (scalars). The keys expected in the FeatureLists associated with sparse +/// values. +/// * feature_list_dense_keys: A list of Nfeature_list_dense string Tensors (scalars). +/// The keys expected in the SequenceExamples' feature_lists associated +/// with lists of dense values. +/// * context_dense_defaults: A list of Ncontext_dense Tensors (some may be empty). +/// context_dense_defaults[j] provides default values +/// when the SequenceExample's context map lacks context_dense_key[j]. +/// If an empty Tensor is provided for context_dense_defaults[j], +/// then the Feature context_dense_keys[j] is required. +/// The input type is inferred from context_dense_defaults[j], even when it's +/// empty. If context_dense_defaults[j] is not empty, its shape must match +/// context_dense_shapes[j]. +/// * debug_name: A scalar containing the name of the serialized proto. +/// May contain, for example, table key (descriptive) name for the +/// corresponding serialized proto. This is purely useful for debugging +/// purposes, and the presence of values here has no effect on the output. +/// May also be an empty scalar if no name is available. +/// +/// Optional attributes (see `Attrs`): +/// * context_sparse_types: A list of Ncontext_sparse types; the data types of data in +/// each context Feature given in context_sparse_keys. +/// Currently the ParseSingleSequenceExample supports DT_FLOAT (FloatList), +/// DT_INT64 (Int64List), and DT_STRING (BytesList). +/// * context_dense_shapes: A list of Ncontext_dense shapes; the shapes of data in +/// each context Feature given in context_dense_keys. +/// The number of elements in the Feature corresponding to context_dense_key[j] +/// must always equal context_dense_shapes[j].NumEntries(). +/// The shape of context_dense_values[j] will match context_dense_shapes[j]. +/// * feature_list_sparse_types: A list of Nfeature_list_sparse types; the data types +/// of data in each FeatureList given in feature_list_sparse_keys. +/// Currently the ParseSingleSequenceExample supports DT_FLOAT (FloatList), +/// DT_INT64 (Int64List), and DT_STRING (BytesList). +/// * feature_list_dense_shapes: A list of Nfeature_list_dense shapes; the shapes of +/// data in each FeatureList given in feature_list_dense_keys. +/// The shape of each Feature in the FeatureList corresponding to +/// feature_list_dense_key[j] must always equal +/// feature_list_dense_shapes[j].NumEntries(). +/// +/// Returns: +/// * `OutputList` context_sparse_indices +/// * `OutputList` context_sparse_values +/// * `OutputList` context_sparse_shapes +/// * `OutputList` context_dense_values +/// * `OutputList` feature_list_sparse_indices +/// * `OutputList` feature_list_sparse_values +/// * `OutputList` feature_list_sparse_shapes +/// * `OutputList` feature_list_dense_values +class ParseSingleSequenceExample { + public: + /// Optional attribute setters for ParseSingleSequenceExample + struct Attrs { + /// A list of Ncontext_sparse types; the data types of data in + /// each context Feature given in context_sparse_keys. + /// Currently the ParseSingleSequenceExample supports DT_FLOAT (FloatList), + /// DT_INT64 (Int64List), and DT_STRING (BytesList). + /// + /// Defaults to [] + TF_MUST_USE_RESULT Attrs ContextSparseTypes(const DataTypeSlice& x) { + Attrs ret = *this; + ret.context_sparse_types_ = x; + return ret; + } + + /// Defaults to [] + TF_MUST_USE_RESULT Attrs FeatureListDenseTypes(const DataTypeSlice& x) { + Attrs ret = *this; + ret.feature_list_dense_types_ = x; + return ret; + } + + /// A list of Ncontext_dense shapes; the shapes of data in + /// each context Feature given in context_dense_keys. + /// The number of elements in the Feature corresponding to context_dense_key[j] + /// must always equal context_dense_shapes[j].NumEntries(). + /// The shape of context_dense_values[j] will match context_dense_shapes[j]. + /// + /// Defaults to [] + TF_MUST_USE_RESULT Attrs ContextDenseShapes(const gtl::ArraySlice& x) { + Attrs ret = *this; + ret.context_dense_shapes_ = x; + return ret; + } + + /// A list of Nfeature_list_sparse types; the data types + /// of data in each FeatureList given in feature_list_sparse_keys. + /// Currently the ParseSingleSequenceExample supports DT_FLOAT (FloatList), + /// DT_INT64 (Int64List), and DT_STRING (BytesList). + /// + /// Defaults to [] + TF_MUST_USE_RESULT Attrs FeatureListSparseTypes(const DataTypeSlice& x) { + Attrs ret = *this; + ret.feature_list_sparse_types_ = x; + return ret; + } + + /// A list of Nfeature_list_dense shapes; the shapes of + /// data in each FeatureList given in feature_list_dense_keys. + /// The shape of each Feature in the FeatureList corresponding to + /// feature_list_dense_key[j] must always equal + /// feature_list_dense_shapes[j].NumEntries(). + /// + /// Defaults to [] + TF_MUST_USE_RESULT Attrs FeatureListDenseShapes(const gtl::ArraySlice& x) { + Attrs ret = *this; + ret.feature_list_dense_shapes_ = x; + return ret; + } + + DataTypeSlice context_sparse_types_ = {}; + DataTypeSlice feature_list_dense_types_ = {}; + gtl::ArraySlice context_dense_shapes_ = {}; + DataTypeSlice feature_list_sparse_types_ = {}; + gtl::ArraySlice feature_list_dense_shapes_ = {}; + }; + ParseSingleSequenceExample(const ::tensorflow::Scope& scope, + ::tensorflow::Input serialized, ::tensorflow::Input + feature_list_dense_missing_assumed_empty, + ::tensorflow::InputList context_sparse_keys, + ::tensorflow::InputList context_dense_keys, + ::tensorflow::InputList feature_list_sparse_keys, + ::tensorflow::InputList feature_list_dense_keys, + ::tensorflow::InputList context_dense_defaults, + ::tensorflow::Input debug_name); + ParseSingleSequenceExample(const ::tensorflow::Scope& scope, + ::tensorflow::Input serialized, ::tensorflow::Input + feature_list_dense_missing_assumed_empty, + ::tensorflow::InputList context_sparse_keys, + ::tensorflow::InputList context_dense_keys, + ::tensorflow::InputList feature_list_sparse_keys, + ::tensorflow::InputList feature_list_dense_keys, + ::tensorflow::InputList context_dense_defaults, + ::tensorflow::Input debug_name, const + ParseSingleSequenceExample::Attrs& attrs); + + static Attrs ContextSparseTypes(const DataTypeSlice& x) { + return Attrs().ContextSparseTypes(x); + } + static Attrs FeatureListDenseTypes(const DataTypeSlice& x) { + return Attrs().FeatureListDenseTypes(x); + } + static Attrs ContextDenseShapes(const gtl::ArraySlice& x) { + return Attrs().ContextDenseShapes(x); + } + static Attrs FeatureListSparseTypes(const DataTypeSlice& x) { + return Attrs().FeatureListSparseTypes(x); + } + static Attrs FeatureListDenseShapes(const gtl::ArraySlice& x) { + return Attrs().FeatureListDenseShapes(x); + } + + Operation operation; + ::tensorflow::OutputList context_sparse_indices; + ::tensorflow::OutputList context_sparse_values; + ::tensorflow::OutputList context_sparse_shapes; + ::tensorflow::OutputList context_dense_values; + ::tensorflow::OutputList feature_list_sparse_indices; + ::tensorflow::OutputList feature_list_sparse_values; + ::tensorflow::OutputList feature_list_sparse_shapes; + ::tensorflow::OutputList feature_list_dense_values; +}; + +/// Transforms a serialized tensorflow.TensorProto proto into a Tensor. +/// +/// Arguments: +/// * scope: A Scope object +/// * serialized: A scalar string containing a serialized TensorProto proto. +/// * out_type: The type of the serialized tensor. The provided type must match the +/// type of the serialized tensor and no implicit conversion will take place. +/// +/// Returns: +/// * `Output`: A Tensor of type `out_type`. +class ParseTensor { + public: + ParseTensor(const ::tensorflow::Scope& scope, ::tensorflow::Input serialized, + DataType out_type); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Transforms a Tensor into a serialized TensorProto proto. +/// +/// Arguments: +/// * scope: A Scope object +/// * tensor: A Tensor of type `T`. +/// +/// Returns: +/// * `Output`: A serialized TensorProto proto of the input tensor. +class SerializeTensor { + public: + SerializeTensor(const ::tensorflow::Scope& scope, ::tensorflow::Input tensor); + operator ::tensorflow::Output() const { return serialized; } + operator ::tensorflow::Input() const { return serialized; } + ::tensorflow::Node* node() const { return serialized.node(); } + + Operation operation; + ::tensorflow::Output serialized; +}; + +/// Converts each string in the input Tensor to the specified numeric type. +/// +/// (Note that int32 overflow results in an error while float overflow +/// results in a rounded value.) +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Optional attributes (see `Attrs`): +/// * out_type: The numeric type to interpret each string in `string_tensor` as. +/// +/// Returns: +/// * `Output`: A Tensor of the same shape as the input `string_tensor`. +class StringToNumber { + public: + /// Optional attribute setters for StringToNumber + struct Attrs { + /// The numeric type to interpret each string in `string_tensor` as. + /// + /// Defaults to DT_FLOAT + TF_MUST_USE_RESULT Attrs OutType(DataType x) { + Attrs ret = *this; + ret.out_type_ = x; + return ret; + } + + DataType out_type_ = DT_FLOAT; + }; + StringToNumber(const ::tensorflow::Scope& scope, ::tensorflow::Input + string_tensor); + StringToNumber(const ::tensorflow::Scope& scope, ::tensorflow::Input + string_tensor, const StringToNumber::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs OutType(DataType x) { + return Attrs().OutType(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// @} + +} // namespace ops +} // namespace tensorflow + +#endif // TENSORFLOW_CC_OPS_PARSING_OPS_H_ diff --git a/linux/include/tensorflow/cc/ops/parsing_ops_internal.h b/linux/include/tensorflow/cc/ops/parsing_ops_internal.h new file mode 100644 index 000000000..c400b81c5 --- /dev/null +++ b/linux/include/tensorflow/cc/ops/parsing_ops_internal.h @@ -0,0 +1,28 @@ +// This file is MACHINE GENERATED! Do not edit. + +#ifndef TENSORFLOW_CC_OPS_PARSING_OPS_INTERNAL_H_ +#define TENSORFLOW_CC_OPS_PARSING_OPS_INTERNAL_H_ + +// This file is MACHINE GENERATED! Do not edit. + +#include "tensorflow/cc/framework/ops.h" +#include "tensorflow/cc/framework/scope.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/gtl/array_slice.h" + +namespace tensorflow { +namespace ops { +namespace internal { +// NOTE: This namespace has internal TensorFlow details that +// are not part of TensorFlow's public API. + +/// @defgroup parsing_ops_internal Parsing Ops Internal +/// @{ + +} // namespace internal +} // namespace ops +} // namespace tensorflow + +#endif // TENSORFLOW_CC_OPS_PARSING_OPS_INTERNAL_H_ diff --git a/linux/include/tensorflow/cc/ops/random_ops.h b/linux/include/tensorflow/cc/ops/random_ops.h new file mode 100644 index 000000000..3e0f62119 --- /dev/null +++ b/linux/include/tensorflow/cc/ops/random_ops.h @@ -0,0 +1,663 @@ +// This file is MACHINE GENERATED! Do not edit. + +#ifndef TENSORFLOW_CC_OPS_RANDOM_OPS_H_ +#define TENSORFLOW_CC_OPS_RANDOM_OPS_H_ + +// This file is MACHINE GENERATED! Do not edit. + +#include "tensorflow/cc/framework/ops.h" +#include "tensorflow/cc/framework/scope.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/gtl/array_slice.h" + +namespace tensorflow { +namespace ops { + +/// @defgroup random_ops Random Ops +/// @{ + +/// Draws samples from a multinomial distribution. +/// +/// Arguments: +/// * scope: A Scope object +/// * logits: 2-D Tensor with shape `[batch_size, num_classes]`. Each slice `[i, :]` +/// represents the unnormalized log probabilities for all classes. +/// * num_samples: 0-D. Number of independent samples to draw for each row slice. +/// +/// Optional attributes (see `Attrs`): +/// * seed: If either seed or seed2 is set to be non-zero, the internal random number +/// generator is seeded by the given seed. Otherwise, a random seed is used. +/// * seed2: A second seed to avoid seed collision. +/// +/// Returns: +/// * `Output`: 2-D Tensor with shape `[batch_size, num_samples]`. Each slice `[i, :]` +/// contains the drawn class labels with range `[0, num_classes)`. +class Multinomial { + public: + /// Optional attribute setters for Multinomial + struct Attrs { + /// If either seed or seed2 is set to be non-zero, the internal random number + /// generator is seeded by the given seed. Otherwise, a random seed is used. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Seed(int64 x) { + Attrs ret = *this; + ret.seed_ = x; + return ret; + } + + /// A second seed to avoid seed collision. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Seed2(int64 x) { + Attrs ret = *this; + ret.seed2_ = x; + return ret; + } + + /// Defaults to DT_INT64 + TF_MUST_USE_RESULT Attrs OutputDtype(DataType x) { + Attrs ret = *this; + ret.output_dtype_ = x; + return ret; + } + + int64 seed_ = 0; + int64 seed2_ = 0; + DataType output_dtype_ = DT_INT64; + }; + Multinomial(const ::tensorflow::Scope& scope, ::tensorflow::Input logits, + ::tensorflow::Input num_samples); + Multinomial(const ::tensorflow::Scope& scope, ::tensorflow::Input logits, + ::tensorflow::Input num_samples, const Multinomial::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs Seed(int64 x) { + return Attrs().Seed(x); + } + static Attrs Seed2(int64 x) { + return Attrs().Seed2(x); + } + static Attrs OutputDtype(DataType x) { + return Attrs().OutputDtype(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Outputs random values from a normal distribution. The parameters may each be a +/// +/// scalar which applies to the entire output, or a vector of length shape[0] which +/// stores the parameters for each batch. +/// +/// Arguments: +/// * scope: A Scope object +/// * shape: The shape of the output tensor. Batches are indexed by the 0th dimension. +/// * means: The mean parameter of each batch. +/// * stdevs: The standard deviation parameter of each batch. Must be greater than 0. +/// * minvals: The minimum cutoff. May be -infinity. +/// * maxvals: The maximum cutoff. May be +infinity, and must be more than the minval +/// for each batch. +/// +/// Optional attributes (see `Attrs`): +/// * seed: If either `seed` or `seed2` are set to be non-zero, the random number +/// generator is seeded by the given seed. Otherwise, it is seeded by a +/// random seed. +/// * seed2: A second seed to avoid seed collision. +/// +/// Returns: +/// * `Output`: A matrix of shape num_batches x samples_per_batch, filled with random +/// truncated normal values using the parameters for each row. +class ParameterizedTruncatedNormal { + public: + /// Optional attribute setters for ParameterizedTruncatedNormal + struct Attrs { + /// If either `seed` or `seed2` are set to be non-zero, the random number + /// generator is seeded by the given seed. Otherwise, it is seeded by a + /// random seed. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Seed(int64 x) { + Attrs ret = *this; + ret.seed_ = x; + return ret; + } + + /// A second seed to avoid seed collision. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Seed2(int64 x) { + Attrs ret = *this; + ret.seed2_ = x; + return ret; + } + + int64 seed_ = 0; + int64 seed2_ = 0; + }; + ParameterizedTruncatedNormal(const ::tensorflow::Scope& scope, + ::tensorflow::Input shape, ::tensorflow::Input + means, ::tensorflow::Input stdevs, + ::tensorflow::Input minvals, ::tensorflow::Input + maxvals); + ParameterizedTruncatedNormal(const ::tensorflow::Scope& scope, + ::tensorflow::Input shape, ::tensorflow::Input + means, ::tensorflow::Input stdevs, + ::tensorflow::Input minvals, ::tensorflow::Input + maxvals, const + ParameterizedTruncatedNormal::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs Seed(int64 x) { + return Attrs().Seed(x); + } + static Attrs Seed2(int64 x) { + return Attrs().Seed2(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Outputs random values from the Gamma distribution(s) described by alpha. +/// +/// This op uses the algorithm by Marsaglia et al. to acquire samples via +/// transformation-rejection from pairs of uniform and normal random variables. +/// See http://dl.acm.org/citation.cfm?id=358414 +/// +/// Arguments: +/// * scope: A Scope object +/// * shape: 1-D integer tensor. Shape of independent samples to draw from each +/// distribution described by the shape parameters given in alpha. +/// * alpha: A tensor in which each scalar is a "shape" parameter describing the +/// associated gamma distribution. +/// +/// Optional attributes (see `Attrs`): +/// * seed: If either `seed` or `seed2` are set to be non-zero, the random number +/// generator is seeded by the given seed. Otherwise, it is seeded by a +/// random seed. +/// * seed2: A second seed to avoid seed collision. +/// +/// Returns: +/// * `Output`: A tensor with shape `shape + shape(alpha)`. Each slice +/// `[:, ..., :, i0, i1, ...iN]` contains the samples drawn for +/// `alpha[i0, i1, ...iN]`. The dtype of the output matches the dtype of alpha. +class RandomGamma { + public: + /// Optional attribute setters for RandomGamma + struct Attrs { + /// If either `seed` or `seed2` are set to be non-zero, the random number + /// generator is seeded by the given seed. Otherwise, it is seeded by a + /// random seed. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Seed(int64 x) { + Attrs ret = *this; + ret.seed_ = x; + return ret; + } + + /// A second seed to avoid seed collision. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Seed2(int64 x) { + Attrs ret = *this; + ret.seed2_ = x; + return ret; + } + + int64 seed_ = 0; + int64 seed2_ = 0; + }; + RandomGamma(const ::tensorflow::Scope& scope, ::tensorflow::Input shape, + ::tensorflow::Input alpha); + RandomGamma(const ::tensorflow::Scope& scope, ::tensorflow::Input shape, + ::tensorflow::Input alpha, const RandomGamma::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs Seed(int64 x) { + return Attrs().Seed(x); + } + static Attrs Seed2(int64 x) { + return Attrs().Seed2(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Outputs random values from the Poisson distribution(s) described by rate. +/// +/// This op uses two algorithms, depending on rate. If rate >= 10, then +/// the algorithm by Hormann is used to acquire samples via +/// transformation-rejection. +/// See http://www.sciencedirect.com/science/article/pii/0167668793909974. +/// +/// Otherwise, Knuth's algorithm is used to acquire samples via multiplying uniform +/// random variables. +/// See Donald E. Knuth (1969). Seminumerical Algorithms. The Art of Computer +/// Programming, Volume 2. Addison Wesley +/// +/// Arguments: +/// * scope: A Scope object +/// * shape: 1-D integer tensor. Shape of independent samples to draw from each +/// distribution described by the shape parameters given in rate. +/// * rate: A tensor in which each scalar is a "rate" parameter describing the +/// associated poisson distribution. +/// +/// Optional attributes (see `Attrs`): +/// * seed: If either `seed` or `seed2` are set to be non-zero, the random number +/// generator is seeded by the given seed. Otherwise, it is seeded by a +/// random seed. +/// * seed2: A second seed to avoid seed collision. +/// +/// Returns: +/// * `Output`: A tensor with shape `shape + shape(rate)`. Each slice +/// `[:, ..., :, i0, i1, ...iN]` contains the samples drawn for +/// `rate[i0, i1, ...iN]`. +class RandomPoissonV2 { + public: + /// Optional attribute setters for RandomPoissonV2 + struct Attrs { + /// If either `seed` or `seed2` are set to be non-zero, the random number + /// generator is seeded by the given seed. Otherwise, it is seeded by a + /// random seed. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Seed(int64 x) { + Attrs ret = *this; + ret.seed_ = x; + return ret; + } + + /// A second seed to avoid seed collision. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Seed2(int64 x) { + Attrs ret = *this; + ret.seed2_ = x; + return ret; + } + + /// Defaults to DT_INT64 + TF_MUST_USE_RESULT Attrs Dtype(DataType x) { + Attrs ret = *this; + ret.dtype_ = x; + return ret; + } + + int64 seed_ = 0; + int64 seed2_ = 0; + DataType dtype_ = DT_INT64; + }; + RandomPoissonV2(const ::tensorflow::Scope& scope, ::tensorflow::Input shape, + ::tensorflow::Input rate); + RandomPoissonV2(const ::tensorflow::Scope& scope, ::tensorflow::Input shape, + ::tensorflow::Input rate, const RandomPoissonV2::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs Seed(int64 x) { + return Attrs().Seed(x); + } + static Attrs Seed2(int64 x) { + return Attrs().Seed2(x); + } + static Attrs Dtype(DataType x) { + return Attrs().Dtype(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Randomly shuffles a tensor along its first dimension. +/// +/// The tensor is shuffled along dimension 0, such that each `value[j]` is mapped +/// to one and only one `output[i]`. For example, a mapping that might occur for a +/// 3x2 tensor is: +/// +/// ``` +/// [[1, 2], [[5, 6], +/// [3, 4], ==> [1, 2], +/// [5, 6]] [3, 4]] +/// ``` +/// +/// Arguments: +/// * scope: A Scope object +/// * value: The tensor to be shuffled. +/// +/// Optional attributes (see `Attrs`): +/// * seed: If either `seed` or `seed2` are set to be non-zero, the random number +/// generator is seeded by the given seed. Otherwise, it is seeded by a +/// random seed. +/// * seed2: A second seed to avoid seed collision. +/// +/// Returns: +/// * `Output`: A tensor of same shape and type as `value`, shuffled along its first +/// dimension. +class RandomShuffle { + public: + /// Optional attribute setters for RandomShuffle + struct Attrs { + /// If either `seed` or `seed2` are set to be non-zero, the random number + /// generator is seeded by the given seed. Otherwise, it is seeded by a + /// random seed. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Seed(int64 x) { + Attrs ret = *this; + ret.seed_ = x; + return ret; + } + + /// A second seed to avoid seed collision. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Seed2(int64 x) { + Attrs ret = *this; + ret.seed2_ = x; + return ret; + } + + int64 seed_ = 0; + int64 seed2_ = 0; + }; + RandomShuffle(const ::tensorflow::Scope& scope, ::tensorflow::Input value); + RandomShuffle(const ::tensorflow::Scope& scope, ::tensorflow::Input value, + const RandomShuffle::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs Seed(int64 x) { + return Attrs().Seed(x); + } + static Attrs Seed2(int64 x) { + return Attrs().Seed2(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Outputs random values from a normal distribution. +/// +/// The generated values will have mean 0 and standard deviation 1. +/// +/// Arguments: +/// * scope: A Scope object +/// * shape: The shape of the output tensor. +/// * dtype: The type of the output. +/// +/// Optional attributes (see `Attrs`): +/// * seed: If either `seed` or `seed2` are set to be non-zero, the random number +/// generator is seeded by the given seed. Otherwise, it is seeded by a +/// random seed. +/// * seed2: A second seed to avoid seed collision. +/// +/// Returns: +/// * `Output`: A tensor of the specified shape filled with random normal values. +class RandomNormal { + public: + /// Optional attribute setters for RandomNormal + struct Attrs { + /// If either `seed` or `seed2` are set to be non-zero, the random number + /// generator is seeded by the given seed. Otherwise, it is seeded by a + /// random seed. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Seed(int64 x) { + Attrs ret = *this; + ret.seed_ = x; + return ret; + } + + /// A second seed to avoid seed collision. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Seed2(int64 x) { + Attrs ret = *this; + ret.seed2_ = x; + return ret; + } + + int64 seed_ = 0; + int64 seed2_ = 0; + }; + RandomNormal(const ::tensorflow::Scope& scope, ::tensorflow::Input shape, + DataType dtype); + RandomNormal(const ::tensorflow::Scope& scope, ::tensorflow::Input shape, + DataType dtype, const RandomNormal::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs Seed(int64 x) { + return Attrs().Seed(x); + } + static Attrs Seed2(int64 x) { + return Attrs().Seed2(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Outputs random values from a uniform distribution. +/// +/// The generated values follow a uniform distribution in the range `[0, 1)`. The +/// lower bound 0 is included in the range, while the upper bound 1 is excluded. +/// +/// Arguments: +/// * scope: A Scope object +/// * shape: The shape of the output tensor. +/// * dtype: The type of the output. +/// +/// Optional attributes (see `Attrs`): +/// * seed: If either `seed` or `seed2` are set to be non-zero, the random number +/// generator is seeded by the given seed. Otherwise, it is seeded by a +/// random seed. +/// * seed2: A second seed to avoid seed collision. +/// +/// Returns: +/// * `Output`: A tensor of the specified shape filled with uniform random values. +class RandomUniform { + public: + /// Optional attribute setters for RandomUniform + struct Attrs { + /// If either `seed` or `seed2` are set to be non-zero, the random number + /// generator is seeded by the given seed. Otherwise, it is seeded by a + /// random seed. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Seed(int64 x) { + Attrs ret = *this; + ret.seed_ = x; + return ret; + } + + /// A second seed to avoid seed collision. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Seed2(int64 x) { + Attrs ret = *this; + ret.seed2_ = x; + return ret; + } + + int64 seed_ = 0; + int64 seed2_ = 0; + }; + RandomUniform(const ::tensorflow::Scope& scope, ::tensorflow::Input shape, + DataType dtype); + RandomUniform(const ::tensorflow::Scope& scope, ::tensorflow::Input shape, + DataType dtype, const RandomUniform::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs Seed(int64 x) { + return Attrs().Seed(x); + } + static Attrs Seed2(int64 x) { + return Attrs().Seed2(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Outputs random integers from a uniform distribution. +/// +/// The generated values are uniform integers in the range `[minval, maxval)`. +/// The lower bound `minval` is included in the range, while the upper bound +/// `maxval` is excluded. +/// +/// The random integers are slightly biased unless `maxval - minval` is an exact +/// power of two. The bias is small for values of `maxval - minval` significantly +/// smaller than the range of the output (either `2^32` or `2^64`). +/// +/// Arguments: +/// * scope: A Scope object +/// * shape: The shape of the output tensor. +/// * minval: 0-D. Inclusive lower bound on the generated integers. +/// * maxval: 0-D. Exclusive upper bound on the generated integers. +/// +/// Optional attributes (see `Attrs`): +/// * seed: If either `seed` or `seed2` are set to be non-zero, the random number +/// generator is seeded by the given seed. Otherwise, it is seeded by a +/// random seed. +/// * seed2: A second seed to avoid seed collision. +/// +/// Returns: +/// * `Output`: A tensor of the specified shape filled with uniform random integers. +class RandomUniformInt { + public: + /// Optional attribute setters for RandomUniformInt + struct Attrs { + /// If either `seed` or `seed2` are set to be non-zero, the random number + /// generator is seeded by the given seed. Otherwise, it is seeded by a + /// random seed. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Seed(int64 x) { + Attrs ret = *this; + ret.seed_ = x; + return ret; + } + + /// A second seed to avoid seed collision. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Seed2(int64 x) { + Attrs ret = *this; + ret.seed2_ = x; + return ret; + } + + int64 seed_ = 0; + int64 seed2_ = 0; + }; + RandomUniformInt(const ::tensorflow::Scope& scope, ::tensorflow::Input shape, + ::tensorflow::Input minval, ::tensorflow::Input maxval); + RandomUniformInt(const ::tensorflow::Scope& scope, ::tensorflow::Input shape, + ::tensorflow::Input minval, ::tensorflow::Input maxval, const + RandomUniformInt::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs Seed(int64 x) { + return Attrs().Seed(x); + } + static Attrs Seed2(int64 x) { + return Attrs().Seed2(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Outputs random values from a truncated normal distribution. +/// +/// The generated values follow a normal distribution with mean 0 and standard +/// deviation 1, except that values whose magnitude is more than 2 standard +/// deviations from the mean are dropped and re-picked. +/// +/// Arguments: +/// * scope: A Scope object +/// * shape: The shape of the output tensor. +/// * dtype: The type of the output. +/// +/// Optional attributes (see `Attrs`): +/// * seed: If either `seed` or `seed2` are set to be non-zero, the random number +/// generator is seeded by the given seed. Otherwise, it is seeded by a +/// random seed. +/// * seed2: A second seed to avoid seed collision. +/// +/// Returns: +/// * `Output`: A tensor of the specified shape filled with random truncated normal +/// values. +class TruncatedNormal { + public: + /// Optional attribute setters for TruncatedNormal + struct Attrs { + /// If either `seed` or `seed2` are set to be non-zero, the random number + /// generator is seeded by the given seed. Otherwise, it is seeded by a + /// random seed. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Seed(int64 x) { + Attrs ret = *this; + ret.seed_ = x; + return ret; + } + + /// A second seed to avoid seed collision. + /// + /// Defaults to 0 + TF_MUST_USE_RESULT Attrs Seed2(int64 x) { + Attrs ret = *this; + ret.seed2_ = x; + return ret; + } + + int64 seed_ = 0; + int64 seed2_ = 0; + }; + TruncatedNormal(const ::tensorflow::Scope& scope, ::tensorflow::Input shape, + DataType dtype); + TruncatedNormal(const ::tensorflow::Scope& scope, ::tensorflow::Input shape, + DataType dtype, const TruncatedNormal::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs Seed(int64 x) { + return Attrs().Seed(x); + } + static Attrs Seed2(int64 x) { + return Attrs().Seed2(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// @} + +} // namespace ops +} // namespace tensorflow + +#endif // TENSORFLOW_CC_OPS_RANDOM_OPS_H_ diff --git a/linux/include/tensorflow/cc/ops/random_ops_internal.h b/linux/include/tensorflow/cc/ops/random_ops_internal.h new file mode 100644 index 000000000..31a15e64d --- /dev/null +++ b/linux/include/tensorflow/cc/ops/random_ops_internal.h @@ -0,0 +1,47 @@ +// This file is MACHINE GENERATED! Do not edit. + +#ifndef TENSORFLOW_CC_OPS_RANDOM_OPS_INTERNAL_H_ +#define TENSORFLOW_CC_OPS_RANDOM_OPS_INTERNAL_H_ + +// This file is MACHINE GENERATED! Do not edit. + +#include "tensorflow/cc/framework/ops.h" +#include "tensorflow/cc/framework/scope.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/gtl/array_slice.h" + +namespace tensorflow { +namespace ops { +namespace internal { +// NOTE: This namespace has internal TensorFlow details that +// are not part of TensorFlow's public API. + +/// @defgroup random_ops_internal Random Ops Internal +/// @{ + +/// Computes the derivative of a Gamma random sample w.r.t. `alpha`. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The output tensor. +class RandomGammaGrad { + public: + RandomGammaGrad(const ::tensorflow::Scope& scope, ::tensorflow::Input alpha, + ::tensorflow::Input sample); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +} // namespace internal +} // namespace ops +} // namespace tensorflow + +#endif // TENSORFLOW_CC_OPS_RANDOM_OPS_INTERNAL_H_ diff --git a/linux/include/tensorflow/cc/ops/remote_fused_graph_ops.h b/linux/include/tensorflow/cc/ops/remote_fused_graph_ops.h new file mode 100644 index 000000000..b38b9a765 --- /dev/null +++ b/linux/include/tensorflow/cc/ops/remote_fused_graph_ops.h @@ -0,0 +1,46 @@ +// This file is MACHINE GENERATED! Do not edit. + +#ifndef TENSORFLOW_CC_OPS_REMOTE_FUSED_GRAPH_OPS_H_ +#define TENSORFLOW_CC_OPS_REMOTE_FUSED_GRAPH_OPS_H_ + +// This file is MACHINE GENERATED! Do not edit. + +#include "tensorflow/cc/framework/ops.h" +#include "tensorflow/cc/framework/scope.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/gtl/array_slice.h" + +namespace tensorflow { +namespace ops { + +/// @defgroup remote_fused_graph_ops Remote Fused Graph Ops +/// @{ + +/// TODO: add doc. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `OutputList`: The outputs tensor. +class RemoteFusedGraphExecute { + public: + RemoteFusedGraphExecute(const ::tensorflow::Scope& scope, + ::tensorflow::InputList inputs, const DataTypeSlice& + Toutputs, StringPiece + serialized_remote_fused_graph_execute_info); + ::tensorflow::Output operator[](size_t index) const { return outputs[index]; } + + + Operation operation; + ::tensorflow::OutputList outputs; +}; + +/// @} + +} // namespace ops +} // namespace tensorflow + +#endif // TENSORFLOW_CC_OPS_REMOTE_FUSED_GRAPH_OPS_H_ diff --git a/linux/include/tensorflow/cc/ops/sparse_ops.h b/linux/include/tensorflow/cc/ops/sparse_ops.h new file mode 100644 index 000000000..5e0c749d2 --- /dev/null +++ b/linux/include/tensorflow/cc/ops/sparse_ops.h @@ -0,0 +1,1680 @@ +// This file is MACHINE GENERATED! Do not edit. + +#ifndef TENSORFLOW_CC_OPS_SPARSE_OPS_H_ +#define TENSORFLOW_CC_OPS_SPARSE_OPS_H_ + +// This file is MACHINE GENERATED! Do not edit. + +#include "tensorflow/cc/framework/ops.h" +#include "tensorflow/cc/framework/scope.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/gtl/array_slice.h" + +namespace tensorflow { +namespace ops { + +/// @defgroup sparse_ops Sparse Ops +/// @{ + +/// Add an `N`-minibatch `SparseTensor` to a `SparseTensorsMap`, return `N` handles. +/// +/// A `SparseTensor` of rank `R` is represented by three tensors: `sparse_indices`, +/// `sparse_values`, and `sparse_shape`, where +/// +/// ```sparse_indices.shape[1] == sparse_shape.shape[0] == R``` +/// +/// An `N`-minibatch of `SparseTensor` objects is represented as a `SparseTensor` +/// having a first `sparse_indices` column taking values between `[0, N)`, where +/// the minibatch size `N == sparse_shape[0]`. +/// +/// The input `SparseTensor` must have rank `R` greater than 1, and the first +/// dimension is treated as the minibatch dimension. Elements of the `SparseTensor` +/// must be sorted in increasing order of this first dimension. The stored +/// `SparseTensor` objects pointed to by each row of the output `sparse_handles` +/// will have rank `R-1`. +/// +/// The `SparseTensor` values can then be read out as part of a minibatch by passing +/// the given keys as vector elements to `TakeManySparseFromTensorsMap`. To ensure +/// the correct `SparseTensorsMap` is accessed, ensure that the same +/// `container` and `shared_name` are passed to that Op. If no `shared_name` +/// is provided here, instead use the *name* of the Operation created by calling +/// `AddManySparseToTensorsMap` as the `shared_name` passed to +/// `TakeManySparseFromTensorsMap`. Ensure the Operations are colocated. +/// +/// Arguments: +/// * scope: A Scope object +/// * sparse_indices: 2-D. The `indices` of the minibatch `SparseTensor`. +/// `sparse_indices[:, 0]` must be ordered values in `[0, N)`. +/// * sparse_values: 1-D. The `values` of the minibatch `SparseTensor`. +/// * sparse_shape: 1-D. The `shape` of the minibatch `SparseTensor`. +/// The minibatch size `N == sparse_shape[0]`. +/// +/// Optional attributes (see `Attrs`): +/// * container: The container name for the `SparseTensorsMap` created by this op. +/// * shared_name: The shared name for the `SparseTensorsMap` created by this op. +/// If blank, the new Operation's unique name is used. +/// +/// Returns: +/// * `Output`: 1-D. The handles of the `SparseTensor` now stored in the +/// `SparseTensorsMap`. Shape: `[N]`. +class AddManySparseToTensorsMap { + public: + /// Optional attribute setters for AddManySparseToTensorsMap + struct Attrs { + /// The container name for the `SparseTensorsMap` created by this op. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs Container(StringPiece x) { + Attrs ret = *this; + ret.container_ = x; + return ret; + } + + /// The shared name for the `SparseTensorsMap` created by this op. + /// If blank, the new Operation's unique name is used. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs SharedName(StringPiece x) { + Attrs ret = *this; + ret.shared_name_ = x; + return ret; + } + + StringPiece container_ = ""; + StringPiece shared_name_ = ""; + }; + AddManySparseToTensorsMap(const ::tensorflow::Scope& scope, ::tensorflow::Input + sparse_indices, ::tensorflow::Input sparse_values, + ::tensorflow::Input sparse_shape); + AddManySparseToTensorsMap(const ::tensorflow::Scope& scope, ::tensorflow::Input + sparse_indices, ::tensorflow::Input sparse_values, + ::tensorflow::Input sparse_shape, const + AddManySparseToTensorsMap::Attrs& attrs); + operator ::tensorflow::Output() const { return sparse_handles; } + operator ::tensorflow::Input() const { return sparse_handles; } + ::tensorflow::Node* node() const { return sparse_handles.node(); } + + static Attrs Container(StringPiece x) { + return Attrs().Container(x); + } + static Attrs SharedName(StringPiece x) { + return Attrs().SharedName(x); + } + + Operation operation; + ::tensorflow::Output sparse_handles; +}; + +/// Add a `SparseTensor` to a `SparseTensorsMap` return its handle. +/// +/// A `SparseTensor` is represented by three tensors: `sparse_indices`, +/// `sparse_values`, and `sparse_shape`. +/// +/// This operator takes the given `SparseTensor` and adds it to a container +/// object (a `SparseTensorsMap`). A unique key within this container is generated +/// in the form of an `int64`, and this is the value that is returned. +/// +/// The `SparseTensor` can then be read out as part of a minibatch by passing +/// the key as a vector element to `TakeManySparseFromTensorsMap`. To ensure +/// the correct `SparseTensorsMap` is accessed, ensure that the same +/// `container` and `shared_name` are passed to that Op. If no `shared_name` +/// is provided here, instead use the *name* of the Operation created by calling +/// `AddSparseToTensorsMap` as the `shared_name` passed to +/// `TakeManySparseFromTensorsMap`. Ensure the Operations are colocated. +/// +/// Arguments: +/// * scope: A Scope object +/// * sparse_indices: 2-D. The `indices` of the `SparseTensor`. +/// * sparse_values: 1-D. The `values` of the `SparseTensor`. +/// * sparse_shape: 1-D. The `shape` of the `SparseTensor`. +/// +/// Optional attributes (see `Attrs`): +/// * container: The container name for the `SparseTensorsMap` created by this op. +/// * shared_name: The shared name for the `SparseTensorsMap` created by this op. +/// If blank, the new Operation's unique name is used. +/// +/// Returns: +/// * `Output`: 0-D. The handle of the `SparseTensor` now stored in the +/// `SparseTensorsMap`. +class AddSparseToTensorsMap { + public: + /// Optional attribute setters for AddSparseToTensorsMap + struct Attrs { + /// The container name for the `SparseTensorsMap` created by this op. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs Container(StringPiece x) { + Attrs ret = *this; + ret.container_ = x; + return ret; + } + + /// The shared name for the `SparseTensorsMap` created by this op. + /// If blank, the new Operation's unique name is used. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs SharedName(StringPiece x) { + Attrs ret = *this; + ret.shared_name_ = x; + return ret; + } + + StringPiece container_ = ""; + StringPiece shared_name_ = ""; + }; + AddSparseToTensorsMap(const ::tensorflow::Scope& scope, ::tensorflow::Input + sparse_indices, ::tensorflow::Input sparse_values, + ::tensorflow::Input sparse_shape); + AddSparseToTensorsMap(const ::tensorflow::Scope& scope, ::tensorflow::Input + sparse_indices, ::tensorflow::Input sparse_values, + ::tensorflow::Input sparse_shape, const + AddSparseToTensorsMap::Attrs& attrs); + operator ::tensorflow::Output() const { return sparse_handle; } + operator ::tensorflow::Input() const { return sparse_handle; } + ::tensorflow::Node* node() const { return sparse_handle.node(); } + + static Attrs Container(StringPiece x) { + return Attrs().Container(x); + } + static Attrs SharedName(StringPiece x) { + return Attrs().SharedName(x); + } + + Operation operation; + ::tensorflow::Output sparse_handle; +}; + +/// Deserialize and concatenate `SparseTensors` from a serialized minibatch. +/// +/// The input `serialized_sparse` must be a string matrix of shape `[N x 3]` where +/// `N` is the minibatch size and the rows correspond to packed outputs of +/// `SerializeSparse`. The ranks of the original `SparseTensor` objects +/// must all match. When the final `SparseTensor` is created, it has rank one +/// higher than the ranks of the incoming `SparseTensor` objects +/// (they have been concatenated along a new row dimension). +/// +/// The output `SparseTensor` object's shape values for all dimensions but the +/// first are the max across the input `SparseTensor` objects' shape values +/// for the corresponding dimensions. Its first shape value is `N`, the minibatch +/// size. +/// +/// The input `SparseTensor` objects' indices are assumed ordered in +/// standard lexicographic order. If this is not the case, after this +/// step run `SparseReorder` to restore index ordering. +/// +/// For example, if the serialized input is a `[2 x 3]` matrix representing two +/// original `SparseTensor` objects: +/// +/// index = [ 0] +/// [10] +/// [20] +/// values = [1, 2, 3] +/// shape = [50] +/// +/// and +/// +/// index = [ 2] +/// [10] +/// values = [4, 5] +/// shape = [30] +/// +/// then the final deserialized `SparseTensor` will be: +/// +/// index = [0 0] +/// [0 10] +/// [0 20] +/// [1 2] +/// [1 10] +/// values = [1, 2, 3, 4, 5] +/// shape = [2 50] +/// +/// Arguments: +/// * scope: A Scope object +/// * serialized_sparse: 2-D, The `N` serialized `SparseTensor` objects. +/// Must have 3 columns. +/// * dtype: The `dtype` of the serialized `SparseTensor` objects. +/// +/// Returns: +/// * `Output` sparse_indices +/// * `Output` sparse_values +/// * `Output` sparse_shape +class DeserializeManySparse { + public: + DeserializeManySparse(const ::tensorflow::Scope& scope, ::tensorflow::Input + serialized_sparse, DataType dtype); + + Operation operation; + ::tensorflow::Output sparse_indices; + ::tensorflow::Output sparse_values; + ::tensorflow::Output sparse_shape; +}; + +/// Deserialize `SparseTensor` objects. +/// +/// The input `serialized_sparse` must have the shape `[?, ?, ..., ?, 3]` where +/// the last dimension stores serialized `SparseTensor` objects and the other N +/// dimensions (N >= 0) correspond to a batch. The ranks of the original +/// `SparseTensor` objects must all match. When the final `SparseTensor` is +/// created, its rank is the rank of the incoming `SparseTensor` objects plus N; +/// the sparse tensors have been concatenated along new dimensions, one for each +/// batch. +/// +/// The output `SparseTensor` object's shape values for the original dimensions +/// are the max across the input `SparseTensor` objects' shape values for the +/// corresponding dimensions. The new dimensions match the size of the batch. +/// +/// The input `SparseTensor` objects' indices are assumed ordered in +/// standard lexicographic order. If this is not the case, after this +/// step run `SparseReorder` to restore index ordering. +/// +/// For example, if the serialized input is a `[2 x 3]` matrix representing two +/// original `SparseTensor` objects: +/// +/// index = [ 0] +/// [10] +/// [20] +/// values = [1, 2, 3] +/// shape = [50] +/// +/// and +/// +/// index = [ 2] +/// [10] +/// values = [4, 5] +/// shape = [30] +/// +/// then the final deserialized `SparseTensor` will be: +/// +/// index = [0 0] +/// [0 10] +/// [0 20] +/// [1 2] +/// [1 10] +/// values = [1, 2, 3, 4, 5] +/// shape = [2 50] +/// +/// Arguments: +/// * scope: A Scope object +/// * serialized_sparse: The serialized `SparseTensor` objects. The last dimension +/// must have 3 columns. +/// * dtype: The `dtype` of the serialized `SparseTensor` objects. +/// +/// Returns: +/// * `Output` sparse_indices +/// * `Output` sparse_values +/// * `Output` sparse_shape +class DeserializeSparse { + public: + DeserializeSparse(const ::tensorflow::Scope& scope, ::tensorflow::Input + serialized_sparse, DataType dtype); + + Operation operation; + ::tensorflow::Output sparse_indices; + ::tensorflow::Output sparse_values; + ::tensorflow::Output sparse_shape; +}; + +/// Serialize an `N`-minibatch `SparseTensor` into an `[N, 3]` `Tensor` object. +/// +/// The `SparseTensor` must have rank `R` greater than 1, and the first dimension +/// is treated as the minibatch dimension. Elements of the `SparseTensor` +/// must be sorted in increasing order of this first dimension. The serialized +/// `SparseTensor` objects going into each row of `serialized_sparse` will have +/// rank `R-1`. +/// +/// The minibatch size `N` is extracted from `sparse_shape[0]`. +/// +/// Arguments: +/// * scope: A Scope object +/// * sparse_indices: 2-D. The `indices` of the minibatch `SparseTensor`. +/// * sparse_values: 1-D. The `values` of the minibatch `SparseTensor`. +/// * sparse_shape: 1-D. The `shape` of the minibatch `SparseTensor`. +/// +/// Optional attributes (see `Attrs`): +/// * out_type: The `dtype` to use for serialization; the supported types are `string` +/// (default) and `variant`. +/// +/// Returns: +/// * `Output`: The serialized_sparse tensor. +class SerializeManySparse { + public: + /// Optional attribute setters for SerializeManySparse + struct Attrs { + /// The `dtype` to use for serialization; the supported types are `string` + /// (default) and `variant`. + /// + /// Defaults to DT_STRING + TF_MUST_USE_RESULT Attrs OutType(DataType x) { + Attrs ret = *this; + ret.out_type_ = x; + return ret; + } + + DataType out_type_ = DT_STRING; + }; + SerializeManySparse(const ::tensorflow::Scope& scope, ::tensorflow::Input + sparse_indices, ::tensorflow::Input sparse_values, + ::tensorflow::Input sparse_shape); + SerializeManySparse(const ::tensorflow::Scope& scope, ::tensorflow::Input + sparse_indices, ::tensorflow::Input sparse_values, + ::tensorflow::Input sparse_shape, const + SerializeManySparse::Attrs& attrs); + operator ::tensorflow::Output() const { return serialized_sparse; } + operator ::tensorflow::Input() const { return serialized_sparse; } + ::tensorflow::Node* node() const { return serialized_sparse.node(); } + + static Attrs OutType(DataType x) { + return Attrs().OutType(x); + } + + Operation operation; + ::tensorflow::Output serialized_sparse; +}; + +/// Serialize a `SparseTensor` into a `[3]` `Tensor` object. +/// +/// Arguments: +/// * scope: A Scope object +/// * sparse_indices: 2-D. The `indices` of the `SparseTensor`. +/// * sparse_values: 1-D. The `values` of the `SparseTensor`. +/// * sparse_shape: 1-D. The `shape` of the `SparseTensor`. +/// +/// Optional attributes (see `Attrs`): +/// * out_type: The `dtype` to use for serialization; the supported types are `string` +/// (default) and `variant`. +/// +/// Returns: +/// * `Output`: The serialized_sparse tensor. +class SerializeSparse { + public: + /// Optional attribute setters for SerializeSparse + struct Attrs { + /// The `dtype` to use for serialization; the supported types are `string` + /// (default) and `variant`. + /// + /// Defaults to DT_STRING + TF_MUST_USE_RESULT Attrs OutType(DataType x) { + Attrs ret = *this; + ret.out_type_ = x; + return ret; + } + + DataType out_type_ = DT_STRING; + }; + SerializeSparse(const ::tensorflow::Scope& scope, ::tensorflow::Input + sparse_indices, ::tensorflow::Input sparse_values, + ::tensorflow::Input sparse_shape); + SerializeSparse(const ::tensorflow::Scope& scope, ::tensorflow::Input + sparse_indices, ::tensorflow::Input sparse_values, + ::tensorflow::Input sparse_shape, const SerializeSparse::Attrs& + attrs); + operator ::tensorflow::Output() const { return serialized_sparse; } + operator ::tensorflow::Input() const { return serialized_sparse; } + ::tensorflow::Node* node() const { return serialized_sparse.node(); } + + static Attrs OutType(DataType x) { + return Attrs().OutType(x); + } + + Operation operation; + ::tensorflow::Output serialized_sparse; +}; + +/// Adds two `SparseTensor` objects to produce another `SparseTensor`. +/// +/// The input `SparseTensor` objects' indices are assumed ordered in standard +/// lexicographic order. If this is not the case, before this step run +/// `SparseReorder` to restore index ordering. +/// +/// By default, if two values sum to zero at some index, the output `SparseTensor` +/// would still include that particular location in its index, storing a zero in the +/// corresponding value slot. To override this, callers can specify `thresh`, +/// indicating that if the sum has a magnitude strictly smaller than `thresh`, its +/// corresponding value and index would then not be included. In particular, +/// `thresh == 0` (default) means everything is kept and actual thresholding happens +/// only for a positive value. +/// +/// In the following shapes, `nnz` is the count after taking `thresh` into account. +/// +/// Arguments: +/// * scope: A Scope object +/// * a_indices: 2-D. The `indices` of the first `SparseTensor`, size `[nnz, ndims]` Matrix. +/// * a_values: 1-D. The `values` of the first `SparseTensor`, size `[nnz]` Vector. +/// * a_shape: 1-D. The `shape` of the first `SparseTensor`, size `[ndims]` Vector. +/// * b_indices: 2-D. The `indices` of the second `SparseTensor`, size `[nnz, ndims]` Matrix. +/// * b_values: 1-D. The `values` of the second `SparseTensor`, size `[nnz]` Vector. +/// * b_shape: 1-D. The `shape` of the second `SparseTensor`, size `[ndims]` Vector. +/// * thresh: 0-D. The magnitude threshold that determines if an output value/index +/// pair takes space. +/// +/// Returns: +/// * `Output` sum_indices +/// * `Output` sum_values +/// * `Output` sum_shape +class SparseAdd { + public: + SparseAdd(const ::tensorflow::Scope& scope, ::tensorflow::Input a_indices, + ::tensorflow::Input a_values, ::tensorflow::Input a_shape, + ::tensorflow::Input b_indices, ::tensorflow::Input b_values, + ::tensorflow::Input b_shape, ::tensorflow::Input thresh); + + Operation operation; + ::tensorflow::Output sum_indices; + ::tensorflow::Output sum_values; + ::tensorflow::Output sum_shape; +}; + +/// The gradient operator for the SparseAdd op. +/// +/// The SparseAdd op calculates A + B, where A, B, and the sum are all represented +/// as `SparseTensor` objects. This op takes in the upstream gradient w.r.t. +/// non-empty values of the sum, and outputs the gradients w.r.t. the non-empty +/// values of A and B. +/// +/// Arguments: +/// * scope: A Scope object +/// * backprop_val_grad: 1-D with shape `[nnz(sum)]`. The gradient with respect to +/// the non-empty values of the sum. +/// * a_indices: 2-D. The `indices` of the `SparseTensor` A, size `[nnz(A), ndims]`. +/// * b_indices: 2-D. The `indices` of the `SparseTensor` B, size `[nnz(B), ndims]`. +/// * sum_indices: 2-D. The `indices` of the sum `SparseTensor`, size +/// `[nnz(sum), ndims]`. +/// +/// Returns: +/// * `Output` a_val_grad: 1-D with shape `[nnz(A)]`. The gradient with respect to the +/// non-empty values of A. +/// * `Output` b_val_grad: 1-D with shape `[nnz(B)]`. The gradient with respect to the +/// non-empty values of B. +class SparseAddGrad { + public: + SparseAddGrad(const ::tensorflow::Scope& scope, ::tensorflow::Input + backprop_val_grad, ::tensorflow::Input a_indices, + ::tensorflow::Input b_indices, ::tensorflow::Input sum_indices); + + Operation operation; + ::tensorflow::Output a_val_grad; + ::tensorflow::Output b_val_grad; +}; + +/// Concatenates a list of `SparseTensor` along the specified dimension. +/// +/// Concatenation is with respect to the dense versions of these sparse tensors. +/// It is assumed that each input is a `SparseTensor` whose elements are ordered +/// along increasing dimension number. +/// +/// All inputs' shapes must match, except for the concat dimension. The +/// `indices`, `values`, and `shapes` lists must have the same length. +/// +/// The output shape is identical to the inputs', except along the concat +/// dimension, where it is the sum of the inputs' sizes along that dimension. +/// +/// The output elements will be resorted to preserve the sort order along +/// increasing dimension number. +/// +/// This op runs in `O(M log M)` time, where `M` is the total number of non-empty +/// values across all inputs. This is due to the need for an internal sort in +/// order to concatenate efficiently across an arbitrary dimension. +/// +/// For example, if `concat_dim = 1` and the inputs are +/// +/// sp_inputs[0]: shape = [2, 3] +/// [0, 2]: "a" +/// [1, 0]: "b" +/// [1, 1]: "c" +/// +/// sp_inputs[1]: shape = [2, 4] +/// [0, 1]: "d" +/// [0, 2]: "e" +/// +/// then the output will be +/// +/// shape = [2, 7] +/// [0, 2]: "a" +/// [0, 4]: "d" +/// [0, 5]: "e" +/// [1, 0]: "b" +/// [1, 1]: "c" +/// +/// Graphically this is equivalent to doing +/// +/// [ a] concat [ d e ] = [ a d e ] +/// [b c ] [ ] [b c ] +/// +/// Arguments: +/// * scope: A Scope object +/// * indices: 2-D. Indices of each input `SparseTensor`. +/// * values: 1-D. Non-empty values of each `SparseTensor`. +/// * shapes: 1-D. Shapes of each `SparseTensor`. +/// * concat_dim: Dimension to concatenate along. Must be in range [-rank, rank), +/// where rank is the number of dimensions in each input `SparseTensor`. +/// +/// Returns: +/// * `Output` output_indices: 2-D. Indices of the concatenated `SparseTensor`. +/// * `Output` output_values: 1-D. Non-empty values of the concatenated `SparseTensor`. +/// * `Output` output_shape: 1-D. Shape of the concatenated `SparseTensor`. +class SparseConcat { + public: + SparseConcat(const ::tensorflow::Scope& scope, ::tensorflow::InputList indices, + ::tensorflow::InputList values, ::tensorflow::InputList shapes, + int64 concat_dim); + + Operation operation; + ::tensorflow::Output output_indices; + ::tensorflow::Output output_values; + ::tensorflow::Output output_shape; +}; + +/// Generates sparse cross from a list of sparse and dense tensors. +/// +/// The op takes two lists, one of 2D `SparseTensor` and one of 2D `Tensor`, each +/// representing features of one feature column. It outputs a 2D `SparseTensor` with +/// the batchwise crosses of these features. +/// +/// For example, if the inputs are +/// +/// inputs[0]: SparseTensor with shape = [2, 2] +/// [0, 0]: "a" +/// [1, 0]: "b" +/// [1, 1]: "c" +/// +/// inputs[1]: SparseTensor with shape = [2, 1] +/// [0, 0]: "d" +/// [1, 0]: "e" +/// +/// inputs[2]: Tensor [["f"], ["g"]] +/// +/// then the output will be +/// +/// shape = [2, 2] +/// [0, 0]: "a_X_d_X_f" +/// [1, 0]: "b_X_e_X_g" +/// [1, 1]: "c_X_e_X_g" +/// +/// if hashed_output=true then the output will be +/// +/// shape = [2, 2] +/// [0, 0]: FingerprintCat64( +/// Fingerprint64("f"), FingerprintCat64( +/// Fingerprint64("d"), Fingerprint64("a"))) +/// [1, 0]: FingerprintCat64( +/// Fingerprint64("g"), FingerprintCat64( +/// Fingerprint64("e"), Fingerprint64("b"))) +/// [1, 1]: FingerprintCat64( +/// Fingerprint64("g"), FingerprintCat64( +/// Fingerprint64("e"), Fingerprint64("c"))) +/// +/// Arguments: +/// * scope: A Scope object +/// * indices: 2-D. Indices of each input `SparseTensor`. +/// * values: 1-D. values of each `SparseTensor`. +/// * shapes: 1-D. Shapes of each `SparseTensor`. +/// * dense_inputs: 2-D. Columns represented by dense `Tensor`. +/// * hashed_output: If true, returns the hash of the cross instead of the string. +/// This will allow us avoiding string manipulations. +/// * num_buckets: It is used if hashed_output is true. +/// output = hashed_value%num_buckets if num_buckets > 0 else hashed_value. +/// * hash_key: Specify the hash_key that will be used by the `FingerprintCat64` +/// function to combine the crosses fingerprints. +/// +/// Returns: +/// * `Output` output_indices: 2-D. Indices of the concatenated `SparseTensor`. +/// * `Output` output_values: 1-D. Non-empty values of the concatenated or hashed +/// `SparseTensor`. +/// * `Output` output_shape: 1-D. Shape of the concatenated `SparseTensor`. +class SparseCross { + public: + SparseCross(const ::tensorflow::Scope& scope, ::tensorflow::InputList indices, + ::tensorflow::InputList values, ::tensorflow::InputList shapes, + ::tensorflow::InputList dense_inputs, bool hashed_output, int64 + num_buckets, int64 hash_key, DataType out_type, DataType + internal_type); + + Operation operation; + ::tensorflow::Output output_indices; + ::tensorflow::Output output_values; + ::tensorflow::Output output_shape; +}; + +/// Adds up a SparseTensor and a dense Tensor, using these special rules: +/// +/// (1) Broadcasts the dense side to have the same shape as the sparse side, if +/// eligible; +/// (2) Then, only the dense values pointed to by the indices of the SparseTensor +/// participate in the cwise addition. +/// +/// By these rules, the result is a logical SparseTensor with exactly the same +/// indices and shape, but possibly with different non-zero values. The output of +/// this Op is the resultant non-zero values. +/// +/// Arguments: +/// * scope: A Scope object +/// * sp_indices: 2-D. `N x R` matrix with the indices of non-empty values in a +/// SparseTensor, possibly not in canonical ordering. +/// * sp_values: 1-D. `N` non-empty values corresponding to `sp_indices`. +/// * sp_shape: 1-D. Shape of the input SparseTensor. +/// * dense: `R`-D. The dense Tensor operand. +/// +/// Returns: +/// * `Output`: 1-D. The `N` values that are operated on. +class SparseDenseCwiseAdd { + public: + SparseDenseCwiseAdd(const ::tensorflow::Scope& scope, ::tensorflow::Input + sp_indices, ::tensorflow::Input sp_values, + ::tensorflow::Input sp_shape, ::tensorflow::Input dense); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Component-wise divides a SparseTensor by a dense Tensor. +/// +/// *Limitation*: this Op only broadcasts the dense side to the sparse side, but not +/// the other direction. +/// +/// Arguments: +/// * scope: A Scope object +/// * sp_indices: 2-D. `N x R` matrix with the indices of non-empty values in a +/// SparseTensor, possibly not in canonical ordering. +/// * sp_values: 1-D. `N` non-empty values corresponding to `sp_indices`. +/// * sp_shape: 1-D. Shape of the input SparseTensor. +/// * dense: `R`-D. The dense Tensor operand. +/// +/// Returns: +/// * `Output`: 1-D. The `N` values that are operated on. +class SparseDenseCwiseDiv { + public: + SparseDenseCwiseDiv(const ::tensorflow::Scope& scope, ::tensorflow::Input + sp_indices, ::tensorflow::Input sp_values, + ::tensorflow::Input sp_shape, ::tensorflow::Input dense); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Component-wise multiplies a SparseTensor by a dense Tensor. +/// +/// The output locations corresponding to the implicitly zero elements in the sparse +/// tensor will be zero (i.e., will not take up storage space), regardless of the +/// contents of the dense tensor (even if it's +/-INF and that INF*0 == NaN). +/// +/// *Limitation*: this Op only broadcasts the dense side to the sparse side, but not +/// the other direction. +/// +/// Arguments: +/// * scope: A Scope object +/// * sp_indices: 2-D. `N x R` matrix with the indices of non-empty values in a +/// SparseTensor, possibly not in canonical ordering. +/// * sp_values: 1-D. `N` non-empty values corresponding to `sp_indices`. +/// * sp_shape: 1-D. Shape of the input SparseTensor. +/// * dense: `R`-D. The dense Tensor operand. +/// +/// Returns: +/// * `Output`: 1-D. The `N` values that are operated on. +class SparseDenseCwiseMul { + public: + SparseDenseCwiseMul(const ::tensorflow::Scope& scope, ::tensorflow::Input + sp_indices, ::tensorflow::Input sp_values, + ::tensorflow::Input sp_shape, ::tensorflow::Input dense); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Fills empty rows in the input 2-D `SparseTensor` with a default value. +/// +/// The input `SparseTensor` is represented via the tuple of inputs +/// (`indices`, `values`, `dense_shape`). The output `SparseTensor` has the +/// same `dense_shape` but with indices `output_indices` and values +/// `output_values`. +/// +/// This op inserts a single entry for every row that doesn't have any values. +/// The index is created as `[row, 0, ..., 0]` and the inserted value +/// is `default_value`. +/// +/// For example, suppose `sp_input` has shape `[5, 6]` and non-empty values: +/// +/// [0, 1]: a +/// [0, 3]: b +/// [2, 0]: c +/// [3, 1]: d +/// +/// Rows 1 and 4 are empty, so the output will be of shape `[5, 6]` with values: +/// +/// [0, 1]: a +/// [0, 3]: b +/// [1, 0]: default_value +/// [2, 0]: c +/// [3, 1]: d +/// [4, 0]: default_value +/// +/// The output `SparseTensor` will be in row-major order and will have the +/// same shape as the input. +/// +/// This op also returns an indicator vector shaped `[dense_shape[0]]` such that +/// +/// empty_row_indicator[i] = True iff row i was an empty row. +/// +/// And a reverse index map vector shaped `[indices.shape[0]]` that is used during +/// backpropagation, +/// +/// reverse_index_map[j] = out_j s.t. indices[j, :] == output_indices[out_j, :] +/// +/// Arguments: +/// * scope: A Scope object +/// * indices: 2-D. the indices of the sparse tensor. +/// * values: 1-D. the values of the sparse tensor. +/// * dense_shape: 1-D. the shape of the sparse tensor. +/// * default_value: 0-D. default value to insert into location `[row, 0, ..., 0]` +/// for rows missing from the input sparse tensor. +/// output indices: 2-D. the indices of the filled sparse tensor. +/// +/// Returns: +/// * `Output` output_indices +/// * `Output` output_values: 1-D. the values of the filled sparse tensor. +/// * `Output` empty_row_indicator: 1-D. whether the dense row was missing in the +/// input sparse tensor. +/// * `Output` reverse_index_map: 1-D. a map from the input indices to the output indices. +class SparseFillEmptyRows { + public: + SparseFillEmptyRows(const ::tensorflow::Scope& scope, ::tensorflow::Input + indices, ::tensorflow::Input values, ::tensorflow::Input + dense_shape, ::tensorflow::Input default_value); + + Operation operation; + ::tensorflow::Output output_indices; + ::tensorflow::Output output_values; + ::tensorflow::Output empty_row_indicator; + ::tensorflow::Output reverse_index_map; +}; + +/// The gradient of SparseFillEmptyRows. +/// +/// Takes vectors reverse_index_map, shaped `[N]`, and grad_values, +/// shaped `[N_full]`, where `N_full >= N` and copies data into either +/// `d_values` or `d_default_value`. Here `d_values` is shaped `[N]` and +/// `d_default_value` is a scalar. +/// +/// d_values[j] = grad_values[reverse_index_map[j]] +/// d_default_value = sum_{k : 0 .. N_full - 1} ( +/// grad_values[k] * 1{k not in reverse_index_map}) +/// +/// Arguments: +/// * scope: A Scope object +/// * reverse_index_map: 1-D. The reverse index map from SparseFillEmptyRows. +/// * grad_values: 1-D. The gradients from backprop. +/// +/// Returns: +/// * `Output` d_values: 1-D. The backprop into values. +/// * `Output` d_default_value: 0-D. The backprop into default_value. +class SparseFillEmptyRowsGrad { + public: + SparseFillEmptyRowsGrad(const ::tensorflow::Scope& scope, ::tensorflow::Input + reverse_index_map, ::tensorflow::Input grad_values); + + Operation operation; + ::tensorflow::Output d_values; + ::tensorflow::Output d_default_value; +}; + +/// Computes the max of elements across dimensions of a SparseTensor. +/// +/// This Op takes a SparseTensor and is the sparse counterpart to +/// `tf.reduce_max()`. In particular, this Op also returns a dense `Tensor` +/// instead of a sparse one. +/// +/// Reduces `sp_input` along the dimensions given in `reduction_axes`. Unless +/// `keep_dims` is true, the rank of the tensor is reduced by 1 for each entry in +/// `reduction_axes`. If `keep_dims` is true, the reduced dimensions are retained +/// with length 1. +/// +/// If `reduction_axes` has no entries, all dimensions are reduced, and a tensor +/// with a single element is returned. Additionally, the axes can be negative, +/// which are interpreted according to the indexing rules in Python. +/// +/// Arguments: +/// * scope: A Scope object +/// * input_indices: 2-D. `N x R` matrix with the indices of non-empty values in a +/// SparseTensor, possibly not in canonical ordering. +/// * input_values: 1-D. `N` non-empty values corresponding to `input_indices`. +/// * input_shape: 1-D. Shape of the input SparseTensor. +/// * reduction_axes: 1-D. Length-`K` vector containing the reduction axes. +/// +/// Optional attributes (see `Attrs`): +/// * keep_dims: If true, retain reduced dimensions with length 1. +/// +/// Returns: +/// * `Output`: `R-K`-D. The reduced Tensor. +class SparseReduceMax { + public: + /// Optional attribute setters for SparseReduceMax + struct Attrs { + /// If true, retain reduced dimensions with length 1. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs KeepDims(bool x) { + Attrs ret = *this; + ret.keep_dims_ = x; + return ret; + } + + bool keep_dims_ = false; + }; + SparseReduceMax(const ::tensorflow::Scope& scope, ::tensorflow::Input + input_indices, ::tensorflow::Input input_values, + ::tensorflow::Input input_shape, ::tensorflow::Input + reduction_axes); + SparseReduceMax(const ::tensorflow::Scope& scope, ::tensorflow::Input + input_indices, ::tensorflow::Input input_values, + ::tensorflow::Input input_shape, ::tensorflow::Input + reduction_axes, const SparseReduceMax::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs KeepDims(bool x) { + return Attrs().KeepDims(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Computes the max of elements across dimensions of a SparseTensor. +/// +/// This Op takes a SparseTensor and is the sparse counterpart to +/// `tf.reduce_max()`. In contrast to SparseReduceMax, this Op returns a +/// SparseTensor. +/// +/// Reduces `sp_input` along the dimensions given in `reduction_axes`. Unless +/// `keep_dims` is true, the rank of the tensor is reduced by 1 for each entry in +/// `reduction_axes`. If `keep_dims` is true, the reduced dimensions are retained +/// with length 1. +/// +/// If `reduction_axes` has no entries, all dimensions are reduced, and a tensor +/// with a single element is returned. Additionally, the axes can be negative, +/// which are interpreted according to the indexing rules in Python. +/// +/// Arguments: +/// * scope: A Scope object +/// * input_indices: 2-D. `N x R` matrix with the indices of non-empty values in a +/// SparseTensor, possibly not in canonical ordering. +/// * input_values: 1-D. `N` non-empty values corresponding to `input_indices`. +/// * input_shape: 1-D. Shape of the input SparseTensor. +/// * reduction_axes: 1-D. Length-`K` vector containing the reduction axes. +/// +/// Optional attributes (see `Attrs`): +/// * keep_dims: If true, retain reduced dimensions with length 1. +/// +/// Returns: +/// * `Output` output_indices +/// * `Output` output_values +/// * `Output` output_shape +class SparseReduceMaxSparse { + public: + /// Optional attribute setters for SparseReduceMaxSparse + struct Attrs { + /// If true, retain reduced dimensions with length 1. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs KeepDims(bool x) { + Attrs ret = *this; + ret.keep_dims_ = x; + return ret; + } + + bool keep_dims_ = false; + }; + SparseReduceMaxSparse(const ::tensorflow::Scope& scope, ::tensorflow::Input + input_indices, ::tensorflow::Input input_values, + ::tensorflow::Input input_shape, ::tensorflow::Input + reduction_axes); + SparseReduceMaxSparse(const ::tensorflow::Scope& scope, ::tensorflow::Input + input_indices, ::tensorflow::Input input_values, + ::tensorflow::Input input_shape, ::tensorflow::Input + reduction_axes, const SparseReduceMaxSparse::Attrs& + attrs); + + static Attrs KeepDims(bool x) { + return Attrs().KeepDims(x); + } + + Operation operation; + ::tensorflow::Output output_indices; + ::tensorflow::Output output_values; + ::tensorflow::Output output_shape; +}; + +/// Computes the sum of elements across dimensions of a SparseTensor. +/// +/// This Op takes a SparseTensor and is the sparse counterpart to +/// `tf.reduce_sum()`. In particular, this Op also returns a dense `Tensor` +/// instead of a sparse one. +/// +/// Reduces `sp_input` along the dimensions given in `reduction_axes`. Unless +/// `keep_dims` is true, the rank of the tensor is reduced by 1 for each entry in +/// `reduction_axes`. If `keep_dims` is true, the reduced dimensions are retained +/// with length 1. +/// +/// If `reduction_axes` has no entries, all dimensions are reduced, and a tensor +/// with a single element is returned. Additionally, the axes can be negative, +/// which are interpreted according to the indexing rules in Python. +/// +/// Arguments: +/// * scope: A Scope object +/// * input_indices: 2-D. `N x R` matrix with the indices of non-empty values in a +/// SparseTensor, possibly not in canonical ordering. +/// * input_values: 1-D. `N` non-empty values corresponding to `input_indices`. +/// * input_shape: 1-D. Shape of the input SparseTensor. +/// * reduction_axes: 1-D. Length-`K` vector containing the reduction axes. +/// +/// Optional attributes (see `Attrs`): +/// * keep_dims: If true, retain reduced dimensions with length 1. +/// +/// Returns: +/// * `Output`: `R-K`-D. The reduced Tensor. +class SparseReduceSum { + public: + /// Optional attribute setters for SparseReduceSum + struct Attrs { + /// If true, retain reduced dimensions with length 1. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs KeepDims(bool x) { + Attrs ret = *this; + ret.keep_dims_ = x; + return ret; + } + + bool keep_dims_ = false; + }; + SparseReduceSum(const ::tensorflow::Scope& scope, ::tensorflow::Input + input_indices, ::tensorflow::Input input_values, + ::tensorflow::Input input_shape, ::tensorflow::Input + reduction_axes); + SparseReduceSum(const ::tensorflow::Scope& scope, ::tensorflow::Input + input_indices, ::tensorflow::Input input_values, + ::tensorflow::Input input_shape, ::tensorflow::Input + reduction_axes, const SparseReduceSum::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs KeepDims(bool x) { + return Attrs().KeepDims(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Computes the sum of elements across dimensions of a SparseTensor. +/// +/// This Op takes a SparseTensor and is the sparse counterpart to +/// `tf.reduce_sum()`. In contrast to SparseReduceSum, this Op returns a +/// SparseTensor. +/// +/// Reduces `sp_input` along the dimensions given in `reduction_axes`. Unless +/// `keep_dims` is true, the rank of the tensor is reduced by 1 for each entry in +/// `reduction_axes`. If `keep_dims` is true, the reduced dimensions are retained +/// with length 1. +/// +/// If `reduction_axes` has no entries, all dimensions are reduced, and a tensor +/// with a single element is returned. Additionally, the axes can be negative, +/// which are interpreted according to the indexing rules in Python. +/// +/// Arguments: +/// * scope: A Scope object +/// * input_indices: 2-D. `N x R` matrix with the indices of non-empty values in a +/// SparseTensor, possibly not in canonical ordering. +/// * input_values: 1-D. `N` non-empty values corresponding to `input_indices`. +/// * input_shape: 1-D. Shape of the input SparseTensor. +/// * reduction_axes: 1-D. Length-`K` vector containing the reduction axes. +/// +/// Optional attributes (see `Attrs`): +/// * keep_dims: If true, retain reduced dimensions with length 1. +/// +/// Returns: +/// * `Output` output_indices +/// * `Output` output_values +/// * `Output` output_shape +class SparseReduceSumSparse { + public: + /// Optional attribute setters for SparseReduceSumSparse + struct Attrs { + /// If true, retain reduced dimensions with length 1. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs KeepDims(bool x) { + Attrs ret = *this; + ret.keep_dims_ = x; + return ret; + } + + bool keep_dims_ = false; + }; + SparseReduceSumSparse(const ::tensorflow::Scope& scope, ::tensorflow::Input + input_indices, ::tensorflow::Input input_values, + ::tensorflow::Input input_shape, ::tensorflow::Input + reduction_axes); + SparseReduceSumSparse(const ::tensorflow::Scope& scope, ::tensorflow::Input + input_indices, ::tensorflow::Input input_values, + ::tensorflow::Input input_shape, ::tensorflow::Input + reduction_axes, const SparseReduceSumSparse::Attrs& + attrs); + + static Attrs KeepDims(bool x) { + return Attrs().KeepDims(x); + } + + Operation operation; + ::tensorflow::Output output_indices; + ::tensorflow::Output output_values; + ::tensorflow::Output output_shape; +}; + +/// Reorders a SparseTensor into the canonical, row-major ordering. +/// +/// Note that by convention, all sparse ops preserve the canonical ordering along +/// increasing dimension number. The only time ordering can be violated is during +/// manual manipulation of the indices and values vectors to add entries. +/// +/// Reordering does not affect the shape of the SparseTensor. +/// +/// If the tensor has rank `R` and `N` non-empty values, `input_indices` has +/// shape `[N, R]`, input_values has length `N`, and input_shape has length `R`. +/// +/// Arguments: +/// * scope: A Scope object +/// * input_indices: 2-D. `N x R` matrix with the indices of non-empty values in a +/// SparseTensor, possibly not in canonical ordering. +/// * input_values: 1-D. `N` non-empty values corresponding to `input_indices`. +/// * input_shape: 1-D. Shape of the input SparseTensor. +/// +/// Returns: +/// * `Output` output_indices: 2-D. `N x R` matrix with the same indices as input_indices, but +/// in canonical row-major ordering. +/// * `Output` output_values: 1-D. `N` non-empty values corresponding to `output_indices`. +class SparseReorder { + public: + SparseReorder(const ::tensorflow::Scope& scope, ::tensorflow::Input + input_indices, ::tensorflow::Input input_values, + ::tensorflow::Input input_shape); + + Operation operation; + ::tensorflow::Output output_indices; + ::tensorflow::Output output_values; +}; + +/// Reshapes a SparseTensor to represent values in a new dense shape. +/// +/// This operation has the same semantics as reshape on the represented dense +/// tensor. The `input_indices` are recomputed based on the requested `new_shape`. +/// +/// If one component of `new_shape` is the special value -1, the size of that +/// dimension is computed so that the total dense size remains constant. At +/// most one component of `new_shape` can be -1. The number of dense elements +/// implied by `new_shape` must be the same as the number of dense elements +/// originally implied by `input_shape`. +/// +/// Reshaping does not affect the order of values in the SparseTensor. +/// +/// If the input tensor has rank `R_in` and `N` non-empty values, and `new_shape` +/// has length `R_out`, then `input_indices` has shape `[N, R_in]`, +/// `input_shape` has length `R_in`, `output_indices` has shape `[N, R_out]`, and +/// `output_shape` has length `R_out`. +/// +/// Arguments: +/// * scope: A Scope object +/// * input_indices: 2-D. `N x R_in` matrix with the indices of non-empty values in a +/// SparseTensor. +/// * input_shape: 1-D. `R_in` vector with the input SparseTensor's dense shape. +/// * new_shape: 1-D. `R_out` vector with the requested new dense shape. +/// +/// Returns: +/// * `Output` output_indices: 2-D. `N x R_out` matrix with the updated indices of non-empty +/// values in the output SparseTensor. +/// * `Output` output_shape: 1-D. `R_out` vector with the full dense shape of the output +/// SparseTensor. This is the same as `new_shape` but with any -1 dimensions +/// filled in. +class SparseReshape { + public: + SparseReshape(const ::tensorflow::Scope& scope, ::tensorflow::Input + input_indices, ::tensorflow::Input input_shape, + ::tensorflow::Input new_shape); + + Operation operation; + ::tensorflow::Output output_indices; + ::tensorflow::Output output_shape; +}; + +/// Slice a `SparseTensor` based on the `start` and `size`. +/// +/// For example, if the input is +/// +/// input_tensor = shape = [2, 7] +/// [ a d e ] +/// [b c ] +/// +/// Graphically the output tensors are: +/// +/// sparse_slice([0, 0], [2, 4]) = shape = [2, 4] +/// [ a ] +/// [b c ] +/// +/// sparse_slice([0, 4], [2, 3]) = shape = [2, 3] +/// [ d e ] +/// [ ] +/// +/// Arguments: +/// * scope: A Scope object +/// * indices: 2-D tensor represents the indices of the sparse tensor. +/// * values: 1-D tensor represents the values of the sparse tensor. +/// * shape: 1-D. tensor represents the shape of the sparse tensor. +/// * start: 1-D. tensor represents the start of the slice. +/// * size: 1-D. tensor represents the size of the slice. +/// output indices: A list of 1-D tensors represents the indices of the output +/// sparse tensors. +/// +/// Returns: +/// * `Output` output_indices +/// * `Output` output_values: A list of 1-D tensors represents the values of the output sparse +/// tensors. +/// * `Output` output_shape: A list of 1-D tensors represents the shape of the output sparse +/// tensors. +class SparseSlice { + public: + SparseSlice(const ::tensorflow::Scope& scope, ::tensorflow::Input indices, + ::tensorflow::Input values, ::tensorflow::Input shape, + ::tensorflow::Input start, ::tensorflow::Input size); + + Operation operation; + ::tensorflow::Output output_indices; + ::tensorflow::Output output_values; + ::tensorflow::Output output_shape; +}; + +/// The gradient operator for the SparseSlice op. +/// +/// This op takes in the upstream gradient w.r.t. non-empty values of +/// the sliced `SparseTensor`, and outputs the gradients w.r.t. +/// the non-empty values of input `SparseTensor`. +/// +/// Arguments: +/// * scope: A Scope object +/// * backprop_val_grad: 1-D. The gradient with respect to +/// the non-empty values of the sliced `SparseTensor`. +/// * input_indices: 2-D. The `indices` of the input `SparseTensor`. +/// * input_start: 1-D. tensor represents the start of the slice. +/// * output_indices: 2-D. The `indices` of the sliced `SparseTensor`. +/// +/// Returns: +/// * `Output`: 1-D. The gradient with respect to the non-empty values of input `SparseTensor`. +class SparseSliceGrad { + public: + SparseSliceGrad(const ::tensorflow::Scope& scope, ::tensorflow::Input + backprop_val_grad, ::tensorflow::Input input_indices, + ::tensorflow::Input input_start, ::tensorflow::Input + output_indices); + operator ::tensorflow::Output() const { return val_grad; } + operator ::tensorflow::Input() const { return val_grad; } + ::tensorflow::Node* node() const { return val_grad.node(); } + + Operation operation; + ::tensorflow::Output val_grad; +}; + +/// Applies softmax to a batched N-D `SparseTensor`. +/// +/// The inputs represent an N-D SparseTensor with logical shape `[..., B, C]` +/// (where `N >= 2`), and with indices sorted in the canonical lexicographic order. +/// +/// This op is equivalent to applying the normal `tf.nn.softmax()` to each innermost +/// logical submatrix with shape `[B, C]`, but with the catch that *the implicitly +/// zero elements do not participate*. Specifically, the algorithm is equivalent +/// to the following: +/// +/// (1) Applies `tf.nn.softmax()` to a densified view of each innermost submatrix +/// with shape `[B, C]`, along the size-C dimension; +/// (2) Masks out the original implicitly-zero locations; +/// (3) Renormalizes the remaining elements. +/// +/// Hence, the `SparseTensor` result has exactly the same non-zero indices and +/// shape. +/// +/// Arguments: +/// * scope: A Scope object +/// * sp_indices: 2-D. `NNZ x R` matrix with the indices of non-empty values in a +/// SparseTensor, in canonical ordering. +/// * sp_values: 1-D. `NNZ` non-empty values corresponding to `sp_indices`. +/// * sp_shape: 1-D. Shape of the input SparseTensor. +/// +/// Returns: +/// * `Output`: 1-D. The `NNZ` values for the result `SparseTensor`. +class SparseSoftmax { + public: + SparseSoftmax(const ::tensorflow::Scope& scope, ::tensorflow::Input sp_indices, + ::tensorflow::Input sp_values, ::tensorflow::Input sp_shape); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Returns the element-wise max of two SparseTensors. +/// +/// Assumes the two SparseTensors have the same shape, i.e., no broadcasting. +/// +/// Arguments: +/// * scope: A Scope object +/// * a_indices: 2-D. `N x R` matrix with the indices of non-empty values in a +/// SparseTensor, in the canonical lexicographic ordering. +/// * a_values: 1-D. `N` non-empty values corresponding to `a_indices`. +/// * a_shape: 1-D. Shape of the input SparseTensor. +/// * b_indices: counterpart to `a_indices` for the other operand. +/// * b_values: counterpart to `a_values` for the other operand; must be of the same dtype. +/// * b_shape: counterpart to `a_shape` for the other operand; the two shapes must be equal. +/// +/// Returns: +/// * `Output` output_indices: 2-D. The indices of the output SparseTensor. +/// * `Output` output_values: 1-D. The values of the output SparseTensor. +class SparseSparseMaximum { + public: + SparseSparseMaximum(const ::tensorflow::Scope& scope, ::tensorflow::Input + a_indices, ::tensorflow::Input a_values, + ::tensorflow::Input a_shape, ::tensorflow::Input b_indices, + ::tensorflow::Input b_values, ::tensorflow::Input b_shape); + + Operation operation; + ::tensorflow::Output output_indices; + ::tensorflow::Output output_values; +}; + +/// Returns the element-wise min of two SparseTensors. +/// +/// Assumes the two SparseTensors have the same shape, i.e., no broadcasting. +/// +/// Arguments: +/// * scope: A Scope object +/// * a_indices: 2-D. `N x R` matrix with the indices of non-empty values in a +/// SparseTensor, in the canonical lexicographic ordering. +/// * a_values: 1-D. `N` non-empty values corresponding to `a_indices`. +/// * a_shape: 1-D. Shape of the input SparseTensor. +/// * b_indices: counterpart to `a_indices` for the other operand. +/// * b_values: counterpart to `a_values` for the other operand; must be of the same dtype. +/// * b_shape: counterpart to `a_shape` for the other operand; the two shapes must be equal. +/// +/// Returns: +/// * `Output` output_indices: 2-D. The indices of the output SparseTensor. +/// * `Output` output_values: 1-D. The values of the output SparseTensor. +class SparseSparseMinimum { + public: + SparseSparseMinimum(const ::tensorflow::Scope& scope, ::tensorflow::Input + a_indices, ::tensorflow::Input a_values, + ::tensorflow::Input a_shape, ::tensorflow::Input b_indices, + ::tensorflow::Input b_values, ::tensorflow::Input b_shape); + + Operation operation; + ::tensorflow::Output output_indices; + ::tensorflow::Output output_values; +}; + +/// Split a `SparseTensor` into `num_split` tensors along one dimension. +/// +/// If the `shape[split_dim]` is not an integer multiple of `num_split`. Slices +/// `[0 : shape[split_dim] % num_split]` gets one extra dimension. +/// For example, if `split_dim = 1` and `num_split = 2` and the input is +/// +/// input_tensor = shape = [2, 7] +/// [ a d e ] +/// [b c ] +/// +/// Graphically the output tensors are: +/// +/// output_tensor[0] = shape = [2, 4] +/// [ a ] +/// [b c ] +/// +/// output_tensor[1] = shape = [2, 3] +/// [ d e ] +/// [ ] +/// +/// Arguments: +/// * scope: A Scope object +/// * split_dim: 0-D. The dimension along which to split. Must be in the range +/// `[0, rank(shape))`. +/// * indices: 2-D tensor represents the indices of the sparse tensor. +/// * values: 1-D tensor represents the values of the sparse tensor. +/// * shape: 1-D. tensor represents the shape of the sparse tensor. +/// output indices: A list of 1-D tensors represents the indices of the output +/// sparse tensors. +/// * num_split: The number of ways to split. +/// +/// Returns: +/// * `OutputList` output_indices +/// * `OutputList` output_values: A list of 1-D tensors represents the values of the output sparse +/// tensors. +/// * `OutputList` output_shape: A list of 1-D tensors represents the shape of the output sparse +/// tensors. +class SparseSplit { + public: + SparseSplit(const ::tensorflow::Scope& scope, ::tensorflow::Input split_dim, + ::tensorflow::Input indices, ::tensorflow::Input values, + ::tensorflow::Input shape, int64 num_split); + + Operation operation; + ::tensorflow::OutputList output_indices; + ::tensorflow::OutputList output_values; + ::tensorflow::OutputList output_shape; +}; + +/// Adds up a `SparseTensor` and a dense `Tensor`, producing a dense `Tensor`. +/// +/// This Op does not require `a_indices` be sorted in standard lexicographic order. +/// +/// Arguments: +/// * scope: A Scope object +/// * a_indices: 2-D. The `indices` of the `SparseTensor`, with shape `[nnz, ndims]`. +/// * a_values: 1-D. The `values` of the `SparseTensor`, with shape `[nnz]`. +/// * a_shape: 1-D. The `shape` of the `SparseTensor`, with shape `[ndims]`. +/// * b: `ndims`-D Tensor. With shape `a_shape`. +/// +/// Returns: +/// * `Output`: The output tensor. +class SparseTensorDenseAdd { + public: + SparseTensorDenseAdd(const ::tensorflow::Scope& scope, ::tensorflow::Input + a_indices, ::tensorflow::Input a_values, + ::tensorflow::Input a_shape, ::tensorflow::Input b); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Multiply SparseTensor (of rank 2) "A" by dense matrix "B". +/// +/// No validity checking is performed on the indices of A. However, the following +/// input format is recommended for optimal behavior: +/// +/// if adjoint_a == false: +/// A should be sorted in lexicographically increasing order. Use SparseReorder +/// if you're not sure. +/// if adjoint_a == true: +/// A should be sorted in order of increasing dimension 1 (i.e., "column major" +/// order instead of "row major" order). +/// +/// Arguments: +/// * scope: A Scope object +/// * a_indices: 2-D. The `indices` of the `SparseTensor`, size `[nnz, 2]` Matrix. +/// * a_values: 1-D. The `values` of the `SparseTensor`, size `[nnz]` Vector. +/// * a_shape: 1-D. The `shape` of the `SparseTensor`, size `[2]` Vector. +/// * b: 2-D. A dense Matrix. +/// +/// Optional attributes (see `Attrs`): +/// * adjoint_a: Use the adjoint of A in the matrix multiply. If A is complex, this +/// is transpose(conj(A)). Otherwise it's transpose(A). +/// * adjoint_b: Use the adjoint of B in the matrix multiply. If B is complex, this +/// is transpose(conj(B)). Otherwise it's transpose(B). +/// +/// Returns: +/// * `Output`: The product tensor. +class SparseTensorDenseMatMul { + public: + /// Optional attribute setters for SparseTensorDenseMatMul + struct Attrs { + /// Use the adjoint of A in the matrix multiply. If A is complex, this + /// is transpose(conj(A)). Otherwise it's transpose(A). + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs AdjointA(bool x) { + Attrs ret = *this; + ret.adjoint_a_ = x; + return ret; + } + + /// Use the adjoint of B in the matrix multiply. If B is complex, this + /// is transpose(conj(B)). Otherwise it's transpose(B). + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs AdjointB(bool x) { + Attrs ret = *this; + ret.adjoint_b_ = x; + return ret; + } + + bool adjoint_a_ = false; + bool adjoint_b_ = false; + }; + SparseTensorDenseMatMul(const ::tensorflow::Scope& scope, ::tensorflow::Input + a_indices, ::tensorflow::Input a_values, + ::tensorflow::Input a_shape, ::tensorflow::Input b); + SparseTensorDenseMatMul(const ::tensorflow::Scope& scope, ::tensorflow::Input + a_indices, ::tensorflow::Input a_values, + ::tensorflow::Input a_shape, ::tensorflow::Input b, + const SparseTensorDenseMatMul::Attrs& attrs); + operator ::tensorflow::Output() const { return product; } + operator ::tensorflow::Input() const { return product; } + ::tensorflow::Node* node() const { return product.node(); } + + static Attrs AdjointA(bool x) { + return Attrs().AdjointA(x); + } + static Attrs AdjointB(bool x) { + return Attrs().AdjointB(x); + } + + Operation operation; + ::tensorflow::Output product; +}; + +/// Converts a sparse representation into a dense tensor. +/// +/// Builds an array `dense` with shape `output_shape` such that +/// +/// ``` +/// # If sparse_indices is scalar +/// dense[i] = (i == sparse_indices ? sparse_values : default_value) +/// +/// # If sparse_indices is a vector, then for each i +/// dense[sparse_indices[i]] = sparse_values[i] +/// +/// # If sparse_indices is an n by d matrix, then for each i in [0, n) +/// dense[sparse_indices[i][0], ..., sparse_indices[i][d-1]] = sparse_values[i] +/// ``` +/// +/// All other values in `dense` are set to `default_value`. If `sparse_values` is a +/// scalar, all sparse indices are set to this single value. +/// +/// Indices should be sorted in lexicographic order, and indices must not +/// contain any repeats. If `validate_indices` is true, these properties +/// are checked during execution. +/// +/// Arguments: +/// * scope: A Scope object +/// * sparse_indices: 0-D, 1-D, or 2-D. `sparse_indices[i]` contains the complete +/// index where `sparse_values[i]` will be placed. +/// * output_shape: 1-D. Shape of the dense output tensor. +/// * sparse_values: 1-D. Values corresponding to each row of `sparse_indices`, +/// or a scalar value to be used for all sparse indices. +/// * default_value: Scalar value to set for indices not specified in +/// `sparse_indices`. +/// +/// Optional attributes (see `Attrs`): +/// * validate_indices: If true, indices are checked to make sure they are sorted in +/// lexicographic order and that there are no repeats. +/// +/// Returns: +/// * `Output`: Dense output tensor of shape `output_shape`. +class SparseToDense { + public: + /// Optional attribute setters for SparseToDense + struct Attrs { + /// If true, indices are checked to make sure they are sorted in + /// lexicographic order and that there are no repeats. + /// + /// Defaults to true + TF_MUST_USE_RESULT Attrs ValidateIndices(bool x) { + Attrs ret = *this; + ret.validate_indices_ = x; + return ret; + } + + bool validate_indices_ = true; + }; + SparseToDense(const ::tensorflow::Scope& scope, ::tensorflow::Input + sparse_indices, ::tensorflow::Input output_shape, + ::tensorflow::Input sparse_values, ::tensorflow::Input + default_value); + SparseToDense(const ::tensorflow::Scope& scope, ::tensorflow::Input + sparse_indices, ::tensorflow::Input output_shape, + ::tensorflow::Input sparse_values, ::tensorflow::Input + default_value, const SparseToDense::Attrs& attrs); + operator ::tensorflow::Output() const { return dense; } + operator ::tensorflow::Input() const { return dense; } + ::tensorflow::Node* node() const { return dense.node(); } + + static Attrs ValidateIndices(bool x) { + return Attrs().ValidateIndices(x); + } + + Operation operation; + ::tensorflow::Output dense; +}; + +/// Read `SparseTensors` from a `SparseTensorsMap` and concatenate them. +/// +/// The input `sparse_handles` must be an `int64` matrix of shape `[N, 1]` where +/// `N` is the minibatch size and the rows correspond to the output handles of +/// `AddSparseToTensorsMap` or `AddManySparseToTensorsMap`. The ranks of the +/// original `SparseTensor` objects that went into the given input ops must all +/// match. When the final `SparseTensor` is created, it has rank one +/// higher than the ranks of the incoming `SparseTensor` objects +/// (they have been concatenated along a new row dimension on the left). +/// +/// The output `SparseTensor` object's shape values for all dimensions but the +/// first are the max across the input `SparseTensor` objects' shape values +/// for the corresponding dimensions. Its first shape value is `N`, the minibatch +/// size. +/// +/// The input `SparseTensor` objects' indices are assumed ordered in +/// standard lexicographic order. If this is not the case, after this +/// step run `SparseReorder` to restore index ordering. +/// +/// For example, if the handles represent an input, which is a `[2, 3]` matrix +/// representing two original `SparseTensor` objects: +/// +/// ``` +/// index = [ 0] +/// [10] +/// [20] +/// values = [1, 2, 3] +/// shape = [50] +/// ``` +/// +/// and +/// +/// ``` +/// index = [ 2] +/// [10] +/// values = [4, 5] +/// shape = [30] +/// ``` +/// +/// then the final `SparseTensor` will be: +/// +/// ``` +/// index = [0 0] +/// [0 10] +/// [0 20] +/// [1 2] +/// [1 10] +/// values = [1, 2, 3, 4, 5] +/// shape = [2 50] +/// ``` +/// +/// Arguments: +/// * scope: A Scope object +/// * sparse_handles: 1-D, The `N` serialized `SparseTensor` objects. +/// Shape: `[N]`. +/// * dtype: The `dtype` of the `SparseTensor` objects stored in the +/// `SparseTensorsMap`. +/// +/// Optional attributes (see `Attrs`): +/// * container: The container name for the `SparseTensorsMap` read by this op. +/// * shared_name: The shared name for the `SparseTensorsMap` read by this op. +/// It should not be blank; rather the `shared_name` or unique Operation name +/// of the Op that created the original `SparseTensorsMap` should be used. +/// +/// Returns: +/// * `Output` sparse_indices: 2-D. The `indices` of the minibatch `SparseTensor`. +/// * `Output` sparse_values: 1-D. The `values` of the minibatch `SparseTensor`. +/// * `Output` sparse_shape: 1-D. The `shape` of the minibatch `SparseTensor`. +class TakeManySparseFromTensorsMap { + public: + /// Optional attribute setters for TakeManySparseFromTensorsMap + struct Attrs { + /// The container name for the `SparseTensorsMap` read by this op. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs Container(StringPiece x) { + Attrs ret = *this; + ret.container_ = x; + return ret; + } + + /// The shared name for the `SparseTensorsMap` read by this op. + /// It should not be blank; rather the `shared_name` or unique Operation name + /// of the Op that created the original `SparseTensorsMap` should be used. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs SharedName(StringPiece x) { + Attrs ret = *this; + ret.shared_name_ = x; + return ret; + } + + StringPiece container_ = ""; + StringPiece shared_name_ = ""; + }; + TakeManySparseFromTensorsMap(const ::tensorflow::Scope& scope, + ::tensorflow::Input sparse_handles, DataType + dtype); + TakeManySparseFromTensorsMap(const ::tensorflow::Scope& scope, + ::tensorflow::Input sparse_handles, DataType + dtype, const TakeManySparseFromTensorsMap::Attrs& + attrs); + + static Attrs Container(StringPiece x) { + return Attrs().Container(x); + } + static Attrs SharedName(StringPiece x) { + return Attrs().SharedName(x); + } + + Operation operation; + ::tensorflow::Output sparse_indices; + ::tensorflow::Output sparse_values; + ::tensorflow::Output sparse_shape; +}; + +/// @} + +} // namespace ops +} // namespace tensorflow + +#endif // TENSORFLOW_CC_OPS_SPARSE_OPS_H_ diff --git a/linux/include/tensorflow/cc/ops/sparse_ops_internal.h b/linux/include/tensorflow/cc/ops/sparse_ops_internal.h new file mode 100644 index 000000000..0015a85c5 --- /dev/null +++ b/linux/include/tensorflow/cc/ops/sparse_ops_internal.h @@ -0,0 +1,28 @@ +// This file is MACHINE GENERATED! Do not edit. + +#ifndef TENSORFLOW_CC_OPS_SPARSE_OPS_INTERNAL_H_ +#define TENSORFLOW_CC_OPS_SPARSE_OPS_INTERNAL_H_ + +// This file is MACHINE GENERATED! Do not edit. + +#include "tensorflow/cc/framework/ops.h" +#include "tensorflow/cc/framework/scope.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/gtl/array_slice.h" + +namespace tensorflow { +namespace ops { +namespace internal { +// NOTE: This namespace has internal TensorFlow details that +// are not part of TensorFlow's public API. + +/// @defgroup sparse_ops_internal Sparse Ops Internal +/// @{ + +} // namespace internal +} // namespace ops +} // namespace tensorflow + +#endif // TENSORFLOW_CC_OPS_SPARSE_OPS_INTERNAL_H_ diff --git a/linux/include/tensorflow/cc/ops/standard_ops.h b/linux/include/tensorflow/cc/ops/standard_ops.h new file mode 100644 index 000000000..98f53010e --- /dev/null +++ b/linux/include/tensorflow/cc/ops/standard_ops.h @@ -0,0 +1,40 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CC_OPS_STANDARD_OPS_H_ +#define TENSORFLOW_CC_OPS_STANDARD_OPS_H_ + +#include "tensorflow/cc/ops/array_ops.h" +#include "tensorflow/cc/ops/candidate_sampling_ops.h" +#include "tensorflow/cc/ops/const_op.h" +#include "tensorflow/cc/ops/control_flow_ops.h" +#include "tensorflow/cc/ops/data_flow_ops.h" +#include "tensorflow/cc/ops/image_ops.h" +#include "tensorflow/cc/ops/io_ops.h" +#include "tensorflow/cc/ops/linalg_ops.h" +#include "tensorflow/cc/ops/logging_ops.h" +#include "tensorflow/cc/ops/lookup_ops.h" +#include "tensorflow/cc/ops/math_ops.h" +#include "tensorflow/cc/ops/nn_ops.h" +#include "tensorflow/cc/ops/no_op.h" +#include "tensorflow/cc/ops/parsing_ops.h" +#include "tensorflow/cc/ops/random_ops.h" +#include "tensorflow/cc/ops/sparse_ops.h" +#include "tensorflow/cc/ops/state_ops.h" +#include "tensorflow/cc/ops/string_ops.h" +#include "tensorflow/cc/ops/training_ops.h" +#include "tensorflow/cc/ops/user_ops.h" + +#endif // TENSORFLOW_CC_OPS_STANDARD_OPS_H_ diff --git a/linux/include/tensorflow/cc/ops/state_ops.h b/linux/include/tensorflow/cc/ops/state_ops.h new file mode 100644 index 000000000..4f7746a9c --- /dev/null +++ b/linux/include/tensorflow/cc/ops/state_ops.h @@ -0,0 +1,1337 @@ +// This file is MACHINE GENERATED! Do not edit. + +#ifndef TENSORFLOW_CC_OPS_STATE_OPS_H_ +#define TENSORFLOW_CC_OPS_STATE_OPS_H_ + +// This file is MACHINE GENERATED! Do not edit. + +#include "tensorflow/cc/framework/ops.h" +#include "tensorflow/cc/framework/scope.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/gtl/array_slice.h" + +namespace tensorflow { +namespace ops { + +/// @defgroup state_ops State Ops +/// @{ + +/// Update 'ref' by assigning 'value' to it. +/// +/// This operation outputs "ref" after the assignment is done. +/// This makes it easier to chain operations that need to use the reset value. +/// +/// Arguments: +/// * scope: A Scope object +/// * ref: Should be from a `Variable` node. May be uninitialized. +/// * value: The value to be assigned to the variable. +/// +/// Optional attributes (see `Attrs`): +/// * validate_shape: If true, the operation will validate that the shape +/// of 'value' matches the shape of the Tensor being assigned to. If false, +/// 'ref' will take on the shape of 'value'. +/// * use_locking: If True, the assignment will be protected by a lock; +/// otherwise the behavior is undefined, but may exhibit less contention. +/// +/// Returns: +/// * `Output`: = Same as "ref". Returned as a convenience for operations that want +/// to use the new value after the variable has been reset. +class Assign { + public: + /// Optional attribute setters for Assign + struct Attrs { + /// If true, the operation will validate that the shape + /// of 'value' matches the shape of the Tensor being assigned to. If false, + /// 'ref' will take on the shape of 'value'. + /// + /// Defaults to true + TF_MUST_USE_RESULT Attrs ValidateShape(bool x) { + Attrs ret = *this; + ret.validate_shape_ = x; + return ret; + } + + /// If True, the assignment will be protected by a lock; + /// otherwise the behavior is undefined, but may exhibit less contention. + /// + /// Defaults to true + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool validate_shape_ = true; + bool use_locking_ = true; + }; + Assign(const ::tensorflow::Scope& scope, ::tensorflow::Input ref, + ::tensorflow::Input value); + Assign(const ::tensorflow::Scope& scope, ::tensorflow::Input ref, + ::tensorflow::Input value, const Assign::Attrs& attrs); + operator ::tensorflow::Output() const { return output_ref; } + operator ::tensorflow::Input() const { return output_ref; } + ::tensorflow::Node* node() const { return output_ref.node(); } + + static Attrs ValidateShape(bool x) { + return Attrs().ValidateShape(x); + } + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; + ::tensorflow::Output output_ref; +}; + +/// Update 'ref' by adding 'value' to it. +/// +/// This operation outputs "ref" after the update is done. +/// This makes it easier to chain operations that need to use the reset value. +/// +/// Arguments: +/// * scope: A Scope object +/// * ref: Should be from a `Variable` node. +/// * value: The value to be added to the variable. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If True, the addition will be protected by a lock; +/// otherwise the behavior is undefined, but may exhibit less contention. +/// +/// Returns: +/// * `Output`: = Same as "ref". Returned as a convenience for operations that want +/// to use the new value after the variable has been updated. +class AssignAdd { + public: + /// Optional attribute setters for AssignAdd + struct Attrs { + /// If True, the addition will be protected by a lock; + /// otherwise the behavior is undefined, but may exhibit less contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = false; + }; + AssignAdd(const ::tensorflow::Scope& scope, ::tensorflow::Input ref, + ::tensorflow::Input value); + AssignAdd(const ::tensorflow::Scope& scope, ::tensorflow::Input ref, + ::tensorflow::Input value, const AssignAdd::Attrs& attrs); + operator ::tensorflow::Output() const { return output_ref; } + operator ::tensorflow::Input() const { return output_ref; } + ::tensorflow::Node* node() const { return output_ref.node(); } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; + ::tensorflow::Output output_ref; +}; + +/// Update 'ref' by subtracting 'value' from it. +/// +/// This operation outputs "ref" after the update is done. +/// This makes it easier to chain operations that need to use the reset value. +/// +/// Arguments: +/// * scope: A Scope object +/// * ref: Should be from a `Variable` node. +/// * value: The value to be subtracted to the variable. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If True, the subtraction will be protected by a lock; +/// otherwise the behavior is undefined, but may exhibit less contention. +/// +/// Returns: +/// * `Output`: = Same as "ref". Returned as a convenience for operations that want +/// to use the new value after the variable has been updated. +class AssignSub { + public: + /// Optional attribute setters for AssignSub + struct Attrs { + /// If True, the subtraction will be protected by a lock; + /// otherwise the behavior is undefined, but may exhibit less contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = false; + }; + AssignSub(const ::tensorflow::Scope& scope, ::tensorflow::Input ref, + ::tensorflow::Input value); + AssignSub(const ::tensorflow::Scope& scope, ::tensorflow::Input ref, + ::tensorflow::Input value, const AssignSub::Attrs& attrs); + operator ::tensorflow::Output() const { return output_ref; } + operator ::tensorflow::Input() const { return output_ref; } + ::tensorflow::Node* node() const { return output_ref.node(); } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; + ::tensorflow::Output output_ref; +}; + +/// Increments 'ref' until it reaches 'limit'. +/// +/// Arguments: +/// * scope: A Scope object +/// * ref: Should be from a scalar `Variable` node. +/// * limit: If incrementing ref would bring it above limit, instead generates an +/// 'OutOfRange' error. +/// +/// Returns: +/// * `Output`: A copy of the input before increment. If nothing else modifies the +/// input, the values produced will all be distinct. +class CountUpTo { + public: + CountUpTo(const ::tensorflow::Scope& scope, ::tensorflow::Input ref, int64 + limit); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Destroys the temporary variable and returns its final value. +/// +/// Sets output to the value of the Tensor pointed to by 'ref', then destroys +/// the temporary variable called 'var_name'. +/// All other uses of 'ref' *must* have executed before this op. +/// This is typically achieved by chaining the ref through each assign op, or by +/// using control dependencies. +/// +/// Outputs the final value of the tensor pointed to by 'ref'. +/// +/// Arguments: +/// * scope: A Scope object +/// * ref: A reference to the temporary variable tensor. +/// * var_name: Name of the temporary variable, usually the name of the matching +/// 'TemporaryVariable' op. +/// +/// Returns: +/// * `Output`: The value tensor. +class DestroyTemporaryVariable { + public: + DestroyTemporaryVariable(const ::tensorflow::Scope& scope, ::tensorflow::Input + ref, StringPiece var_name); + operator ::tensorflow::Output() const { return value; } + operator ::tensorflow::Input() const { return value; } + ::tensorflow::Node* node() const { return value.node(); } + + Operation operation; + ::tensorflow::Output value; +}; + +/// Checks whether a tensor has been initialized. +/// +/// Outputs boolean scalar indicating whether the tensor has been initialized. +/// +/// Arguments: +/// * scope: A Scope object +/// * ref: Should be from a `Variable` node. May be uninitialized. +/// +/// Returns: +/// * `Output`: The is_initialized tensor. +class IsVariableInitialized { + public: + IsVariableInitialized(const ::tensorflow::Scope& scope, ::tensorflow::Input + ref); + operator ::tensorflow::Output() const { return is_initialized; } + operator ::tensorflow::Input() const { return is_initialized; } + ::tensorflow::Node* node() const { return is_initialized.node(); } + + Operation operation; + ::tensorflow::Output is_initialized; +}; + +/// Increments variable pointed to by 'resource' until it reaches 'limit'. +/// +/// Arguments: +/// * scope: A Scope object +/// * resource: Should be from a scalar `Variable` node. +/// * limit: If incrementing ref would bring it above limit, instead generates an +/// 'OutOfRange' error. +/// +/// Returns: +/// * `Output`: A copy of the input before increment. If nothing else modifies the +/// input, the values produced will all be distinct. +class ResourceCountUpTo { + public: + ResourceCountUpTo(const ::tensorflow::Scope& scope, ::tensorflow::Input + resource, int64 limit, DataType T); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Adds sparse `updates` to individual values or slices within a given +/// +/// variable according to `indices`. +/// +/// `ref` is a `Tensor` with rank `P` and `indices` is a `Tensor` of rank `Q`. +/// +/// `indices` must be integer tensor, containing indices into `ref`. +/// It must be shape `[d_0, ..., d_{Q-2}, K]` where `0 < K <= P`. +/// +/// The innermost dimension of `indices` (with length `K`) corresponds to +/// indices into elements (if `K = P`) or slices (if `K < P`) along the `K`th +/// dimension of `ref`. +/// +/// `updates` is `Tensor` of rank `Q-1+P-K` with shape: +/// +/// ``` +/// [d_0, ..., d_{Q-2}, ref.shape[K], ..., ref.shape[P-1]]. +/// ``` +/// +/// For example, say we want to update 4 scattered elements to a rank-1 tensor to +/// 8 elements. In Python, that update would look like this: +/// +/// ```python +/// ref = tf.Variable([1, 2, 3, 4, 5, 6, 7, 8], use_resource=True) +/// indices = tf.constant([[4], [3], [1] ,[7]]) +/// updates = tf.constant([9, 10, 11, 12]) +/// update = tf.scatter_nd_add(ref, indices, updates) +/// with tf.Session() as sess: +/// print sess.run(update) +/// ``` +/// +/// The resulting update to ref would look like this: +/// +/// [1, 12, 3, 14, 14, 6, 7, 20] +/// +/// See `tf.scatter_nd` for more details about how to make updates to +/// slices. +/// +/// Arguments: +/// * scope: A Scope object +/// * ref: A resource handle. Must be from a VarHandleOp. +/// * indices: A Tensor. Must be one of the following types: int32, int64. +/// A tensor of indices into ref. +/// * updates: A Tensor. Must have the same type as ref. A tensor of +/// values to add to ref. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: An optional bool. Defaults to True. If True, the assignment will +/// be protected by a lock; otherwise the behavior is undefined, +/// but may exhibit less contention. +/// +/// Returns: +/// * the created `Operation` +class ResourceScatterNdAdd { + public: + /// Optional attribute setters for ResourceScatterNdAdd + struct Attrs { + /// An optional bool. Defaults to True. If True, the assignment will + /// be protected by a lock; otherwise the behavior is undefined, + /// but may exhibit less contention. + /// + /// Defaults to true + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = true; + }; + ResourceScatterNdAdd(const ::tensorflow::Scope& scope, ::tensorflow::Input ref, + ::tensorflow::Input indices, ::tensorflow::Input updates); + ResourceScatterNdAdd(const ::tensorflow::Scope& scope, ::tensorflow::Input ref, + ::tensorflow::Input indices, ::tensorflow::Input updates, + const ResourceScatterNdAdd::Attrs& attrs); + operator ::tensorflow::Operation() const { return operation; } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; +}; + +/// Applies sparse `updates` to individual values or slices within a given +/// +/// variable according to `indices`. +/// +/// `ref` is a `Tensor` with rank `P` and `indices` is a `Tensor` of rank `Q`. +/// +/// `indices` must be integer tensor, containing indices into `ref`. +/// It must be shape `[d_0, ..., d_{Q-2}, K]` where `0 < K <= P`. +/// +/// The innermost dimension of `indices` (with length `K`) corresponds to +/// indices into elements (if `K = P`) or slices (if `K < P`) along the `K`th +/// dimension of `ref`. +/// +/// `updates` is `Tensor` of rank `Q-1+P-K` with shape: +/// +/// ``` +/// [d_0, ..., d_{Q-2}, ref.shape[K], ..., ref.shape[P-1]]. +/// ``` +/// +/// For example, say we want to update 4 scattered elements to a rank-1 tensor to +/// 8 elements. In Python, that update would look like this: +/// +/// ```python +/// ref = tf.Variable([1, 2, 3, 4, 5, 6, 7, 8]) +/// indices = tf.constant([[4], [3], [1] ,[7]]) +/// updates = tf.constant([9, 10, 11, 12]) +/// update = tf.scatter_nd_update(ref, indices, updates) +/// with tf.Session() as sess: +/// print sess.run(update) +/// ``` +/// +/// The resulting update to ref would look like this: +/// +/// [1, 11, 3, 10, 9, 6, 7, 12] +/// +/// See `tf.scatter_nd` for more details about how to make updates to +/// slices. +/// +/// Arguments: +/// * scope: A Scope object +/// * ref: A resource handle. Must be from a VarHandleOp. +/// * indices: A Tensor. Must be one of the following types: int32, int64. +/// A tensor of indices into ref. +/// * updates: A Tensor. Must have the same type as ref. A tensor of updated +/// values to add to ref. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: An optional bool. Defaults to True. If True, the assignment will +/// be protected by a lock; otherwise the behavior is undefined, +/// but may exhibit less contention. +/// +/// Returns: +/// * the created `Operation` +class ResourceScatterNdUpdate { + public: + /// Optional attribute setters for ResourceScatterNdUpdate + struct Attrs { + /// An optional bool. Defaults to True. If True, the assignment will + /// be protected by a lock; otherwise the behavior is undefined, + /// but may exhibit less contention. + /// + /// Defaults to true + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = true; + }; + ResourceScatterNdUpdate(const ::tensorflow::Scope& scope, ::tensorflow::Input + ref, ::tensorflow::Input indices, ::tensorflow::Input + updates); + ResourceScatterNdUpdate(const ::tensorflow::Scope& scope, ::tensorflow::Input + ref, ::tensorflow::Input indices, ::tensorflow::Input + updates, const ResourceScatterNdUpdate::Attrs& attrs); + operator ::tensorflow::Operation() const { return operation; } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; +}; + +/// Adds sparse updates to a variable reference. +/// +/// This operation computes +/// +/// # Scalar indices +/// ref[indices, ...] += updates[...] +/// +/// # Vector indices (for each i) +/// ref[indices[i], ...] += updates[i, ...] +/// +/// # High rank indices (for each i, ..., j) +/// ref[indices[i, ..., j], ...] += updates[i, ..., j, ...] +/// +/// This operation outputs `ref` after the update is done. +/// This makes it easier to chain operations that need to use the reset value. +/// +/// Duplicate entries are handled correctly: if multiple `indices` reference +/// the same location, their contributions add. +/// +/// Requires `updates.shape = indices.shape + ref.shape[1:]` or `updates.shape = []`. +/// +///
+/// +///
+/// +/// Arguments: +/// * scope: A Scope object +/// * ref: Should be from a `Variable` node. +/// * indices: A tensor of indices into the first dimension of `ref`. +/// * updates: A tensor of updated values to add to `ref`. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If True, the addition will be protected by a lock; +/// otherwise the behavior is undefined, but may exhibit less contention. +/// +/// Returns: +/// * `Output`: = Same as `ref`. Returned as a convenience for operations that want +/// to use the updated values after the update is done. +class ScatterAdd { + public: + /// Optional attribute setters for ScatterAdd + struct Attrs { + /// If True, the addition will be protected by a lock; + /// otherwise the behavior is undefined, but may exhibit less contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = false; + }; + ScatterAdd(const ::tensorflow::Scope& scope, ::tensorflow::Input ref, + ::tensorflow::Input indices, ::tensorflow::Input updates); + ScatterAdd(const ::tensorflow::Scope& scope, ::tensorflow::Input ref, + ::tensorflow::Input indices, ::tensorflow::Input updates, const + ScatterAdd::Attrs& attrs); + operator ::tensorflow::Output() const { return output_ref; } + operator ::tensorflow::Input() const { return output_ref; } + ::tensorflow::Node* node() const { return output_ref.node(); } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; + ::tensorflow::Output output_ref; +}; + +/// Divides a variable reference by sparse updates. +/// +/// This operation computes +/// +/// ```python +/// # Scalar indices +/// ref[indices, ...] /= updates[...] +/// +/// # Vector indices (for each i) +/// ref[indices[i], ...] /= updates[i, ...] +/// +/// # High rank indices (for each i, ..., j) +/// ref[indices[i, ..., j], ...] /= updates[i, ..., j, ...] +/// ``` +/// +/// This operation outputs `ref` after the update is done. +/// This makes it easier to chain operations that need to use the reset value. +/// +/// Duplicate entries are handled correctly: if multiple `indices` reference +/// the same location, their contributions divide. +/// +/// Requires `updates.shape = indices.shape + ref.shape[1:]` or `updates.shape = []`. +/// +/// Arguments: +/// * scope: A Scope object +/// * ref: Should be from a `Variable` node. +/// * indices: A tensor of indices into the first dimension of `ref`. +/// * updates: A tensor of values that `ref` is divided by. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If True, the operation will be protected by a lock; +/// otherwise the behavior is undefined, but may exhibit less contention. +/// +/// Returns: +/// * `Output`: = Same as `ref`. Returned as a convenience for operations that want +/// to use the updated values after the update is done. +class ScatterDiv { + public: + /// Optional attribute setters for ScatterDiv + struct Attrs { + /// If True, the operation will be protected by a lock; + /// otherwise the behavior is undefined, but may exhibit less contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = false; + }; + ScatterDiv(const ::tensorflow::Scope& scope, ::tensorflow::Input ref, + ::tensorflow::Input indices, ::tensorflow::Input updates); + ScatterDiv(const ::tensorflow::Scope& scope, ::tensorflow::Input ref, + ::tensorflow::Input indices, ::tensorflow::Input updates, const + ScatterDiv::Attrs& attrs); + operator ::tensorflow::Output() const { return output_ref; } + operator ::tensorflow::Input() const { return output_ref; } + ::tensorflow::Node* node() const { return output_ref.node(); } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; + ::tensorflow::Output output_ref; +}; + +/// Reduces sparse updates into a variable reference using the `max` operation. +/// +/// This operation computes +/// +/// # Scalar indices +/// ref[indices, ...] = max(ref[indices, ...], updates[...]) +/// +/// # Vector indices (for each i) +/// ref[indices[i], ...] = max(ref[indices[i], ...], updates[i, ...]) +/// +/// # High rank indices (for each i, ..., j) +/// ref[indices[i, ..., j], ...] = max(ref[indices[i, ..., j], ...], updates[i, ..., j, ...]) +/// +/// This operation outputs `ref` after the update is done. +/// This makes it easier to chain operations that need to use the reset value. +/// +/// Duplicate entries are handled correctly: if multiple `indices` reference +/// the same location, their contributions combine. +/// +/// Requires `updates.shape = indices.shape + ref.shape[1:]` or `updates.shape = []`. +/// +///
+/// +///
+/// +/// Arguments: +/// * scope: A Scope object +/// * ref: Should be from a `Variable` node. +/// * indices: A tensor of indices into the first dimension of `ref`. +/// * updates: A tensor of updated values to reduce into `ref`. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If True, the update will be protected by a lock; +/// otherwise the behavior is undefined, but may exhibit less contention. +/// +/// Returns: +/// * `Output`: = Same as `ref`. Returned as a convenience for operations that want +/// to use the updated values after the update is done. +class ScatterMax { + public: + /// Optional attribute setters for ScatterMax + struct Attrs { + /// If True, the update will be protected by a lock; + /// otherwise the behavior is undefined, but may exhibit less contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = false; + }; + ScatterMax(const ::tensorflow::Scope& scope, ::tensorflow::Input ref, + ::tensorflow::Input indices, ::tensorflow::Input updates); + ScatterMax(const ::tensorflow::Scope& scope, ::tensorflow::Input ref, + ::tensorflow::Input indices, ::tensorflow::Input updates, const + ScatterMax::Attrs& attrs); + operator ::tensorflow::Output() const { return output_ref; } + operator ::tensorflow::Input() const { return output_ref; } + ::tensorflow::Node* node() const { return output_ref.node(); } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; + ::tensorflow::Output output_ref; +}; + +/// Reduces sparse updates into a variable reference using the `min` operation. +/// +/// This operation computes +/// +/// # Scalar indices +/// ref[indices, ...] = min(ref[indices, ...], updates[...]) +/// +/// # Vector indices (for each i) +/// ref[indices[i], ...] = min(ref[indices[i], ...], updates[i, ...]) +/// +/// # High rank indices (for each i, ..., j) +/// ref[indices[i, ..., j], ...] = min(ref[indices[i, ..., j], ...], updates[i, ..., j, ...]) +/// +/// This operation outputs `ref` after the update is done. +/// This makes it easier to chain operations that need to use the reset value. +/// +/// Duplicate entries are handled correctly: if multiple `indices` reference +/// the same location, their contributions combine. +/// +/// Requires `updates.shape = indices.shape + ref.shape[1:]` or `updates.shape = []`. +/// +///
+/// +///
+/// +/// Arguments: +/// * scope: A Scope object +/// * ref: Should be from a `Variable` node. +/// * indices: A tensor of indices into the first dimension of `ref`. +/// * updates: A tensor of updated values to reduce into `ref`. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If True, the update will be protected by a lock; +/// otherwise the behavior is undefined, but may exhibit less contention. +/// +/// Returns: +/// * `Output`: = Same as `ref`. Returned as a convenience for operations that want +/// to use the updated values after the update is done. +class ScatterMin { + public: + /// Optional attribute setters for ScatterMin + struct Attrs { + /// If True, the update will be protected by a lock; + /// otherwise the behavior is undefined, but may exhibit less contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = false; + }; + ScatterMin(const ::tensorflow::Scope& scope, ::tensorflow::Input ref, + ::tensorflow::Input indices, ::tensorflow::Input updates); + ScatterMin(const ::tensorflow::Scope& scope, ::tensorflow::Input ref, + ::tensorflow::Input indices, ::tensorflow::Input updates, const + ScatterMin::Attrs& attrs); + operator ::tensorflow::Output() const { return output_ref; } + operator ::tensorflow::Input() const { return output_ref; } + ::tensorflow::Node* node() const { return output_ref.node(); } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; + ::tensorflow::Output output_ref; +}; + +/// Multiplies sparse updates into a variable reference. +/// +/// This operation computes +/// +/// ```python +/// # Scalar indices +/// ref[indices, ...] *= updates[...] +/// +/// # Vector indices (for each i) +/// ref[indices[i], ...] *= updates[i, ...] +/// +/// # High rank indices (for each i, ..., j) +/// ref[indices[i, ..., j], ...] *= updates[i, ..., j, ...] +/// ``` +/// +/// This operation outputs `ref` after the update is done. +/// This makes it easier to chain operations that need to use the reset value. +/// +/// Duplicate entries are handled correctly: if multiple `indices` reference +/// the same location, their contributions multiply. +/// +/// Requires `updates.shape = indices.shape + ref.shape[1:]` or `updates.shape = []`. +/// +/// Arguments: +/// * scope: A Scope object +/// * ref: Should be from a `Variable` node. +/// * indices: A tensor of indices into the first dimension of `ref`. +/// * updates: A tensor of updated values to multiply to `ref`. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If True, the operation will be protected by a lock; +/// otherwise the behavior is undefined, but may exhibit less contention. +/// +/// Returns: +/// * `Output`: = Same as `ref`. Returned as a convenience for operations that want +/// to use the updated values after the update is done. +class ScatterMul { + public: + /// Optional attribute setters for ScatterMul + struct Attrs { + /// If True, the operation will be protected by a lock; + /// otherwise the behavior is undefined, but may exhibit less contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = false; + }; + ScatterMul(const ::tensorflow::Scope& scope, ::tensorflow::Input ref, + ::tensorflow::Input indices, ::tensorflow::Input updates); + ScatterMul(const ::tensorflow::Scope& scope, ::tensorflow::Input ref, + ::tensorflow::Input indices, ::tensorflow::Input updates, const + ScatterMul::Attrs& attrs); + operator ::tensorflow::Output() const { return output_ref; } + operator ::tensorflow::Input() const { return output_ref; } + ::tensorflow::Node* node() const { return output_ref.node(); } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; + ::tensorflow::Output output_ref; +}; + +/// Applies sparse addition between `updates` and individual values or slices +/// +/// within a given variable according to `indices`. +/// +/// `ref` is a `Tensor` with rank `P` and `indices` is a `Tensor` of rank `Q`. +/// +/// `indices` must be integer tensor, containing indices into `ref`. +/// It must be shape `\\([d_0, ..., d_{Q-2}, K]\\)` where `0 < K <= P`. +/// +/// The innermost dimension of `indices` (with length `K`) corresponds to +/// indices into elements (if `K = P`) or slices (if `K < P`) along the `K`th +/// dimension of `ref`. +/// +/// `updates` is `Tensor` of rank `Q-1+P-K` with shape: +/// +/// $$[d_0, ..., d_{Q-2}, ref.shape[K], ..., ref.shape[P-1]].$$ +/// +/// For example, say we want to add 4 scattered elements to a rank-1 tensor to 8 +/// elements. In Python, that addition would look like this: +/// +/// ref = tf.Variable([1, 2, 3, 4, 5, 6, 7, 8]) +/// indices = tf.constant([[4], [3], [1], [7]]) +/// updates = tf.constant([9, 10, 11, 12]) +/// add = tf.scatter_nd_add(ref, indices, updates) +/// with tf.Session() as sess: +/// print sess.run(add) +/// +/// The resulting update to ref would look like this: +/// +/// [1, 13, 3, 14, 14, 6, 7, 20] +/// +/// See `tf.scatter_nd` for more details about how to make updates to +/// slices. +/// +/// Arguments: +/// * scope: A Scope object +/// * ref: A mutable Tensor. Should be from a Variable node. +/// * indices: A Tensor. Must be one of the following types: int32, int64. +/// A tensor of indices into ref. +/// * updates: A Tensor. Must have the same type as ref. A tensor of updated values +/// to add to ref. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: An optional bool. Defaults to True. If True, the assignment will +/// be protected by a lock; otherwise the behavior is undefined, +/// but may exhibit less contention. +/// +/// Returns: +/// * `Output`: Same as ref. Returned as a convenience for operations that want +/// to use the updated values after the update is done. +class ScatterNdAdd { + public: + /// Optional attribute setters for ScatterNdAdd + struct Attrs { + /// An optional bool. Defaults to True. If True, the assignment will + /// be protected by a lock; otherwise the behavior is undefined, + /// but may exhibit less contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = false; + }; + ScatterNdAdd(const ::tensorflow::Scope& scope, ::tensorflow::Input ref, + ::tensorflow::Input indices, ::tensorflow::Input updates); + ScatterNdAdd(const ::tensorflow::Scope& scope, ::tensorflow::Input ref, + ::tensorflow::Input indices, ::tensorflow::Input updates, const + ScatterNdAdd::Attrs& attrs); + operator ::tensorflow::Output() const { return output_ref; } + operator ::tensorflow::Input() const { return output_ref; } + ::tensorflow::Node* node() const { return output_ref.node(); } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; + ::tensorflow::Output output_ref; +}; + +/// Applies sparse subtraction between `updates` and individual values or slices +/// +/// within a given variable according to `indices`. +/// +/// `ref` is a `Tensor` with rank `P` and `indices` is a `Tensor` of rank `Q`. +/// +/// `indices` must be integer tensor, containing indices into `ref`. +/// It must be shape \\([d_0, ..., d_{Q-2}, K]\\) where `0 < K <= P`. +/// +/// The innermost dimension of `indices` (with length `K`) corresponds to +/// indices into elements (if `K = P`) or slices (if `K < P`) along the `K`th +/// dimension of `ref`. +/// +/// `updates` is `Tensor` of rank `Q-1+P-K` with shape: +/// +/// $$[d_0, ..., d_{Q-2}, ref.shape[K], ..., ref.shape[P-1]].$$ +/// +/// For example, say we want to subtract 4 scattered elements from a rank-1 tensor +/// with 8 elements. In Python, that subtraction would look like this: +/// +/// ref = tf.Variable([1, 2, 3, 4, 5, 6, 7, 8]) +/// indices = tf.constant([[4], [3], [1], [7]]) +/// updates = tf.constant([9, 10, 11, 12]) +/// sub = tf.scatter_nd_sub(ref, indices, updates) +/// with tf.Session() as sess: +/// print sess.run(sub) +/// +/// The resulting update to ref would look like this: +/// +/// [1, -9, 3, -6, -4, 6, 7, -4] +/// +/// See `tf.scatter_nd` for more details about how to make updates to +/// slices. +/// +/// Arguments: +/// * scope: A Scope object +/// * ref: A mutable Tensor. Should be from a Variable node. +/// * indices: A Tensor. Must be one of the following types: int32, int64. +/// A tensor of indices into ref. +/// * updates: A Tensor. Must have the same type as ref. A tensor of updated values +/// to subtract from ref. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: An optional bool. Defaults to True. If True, the assignment will +/// be protected by a lock; otherwise the behavior is undefined, +/// but may exhibit less contention. +/// +/// Returns: +/// * `Output`: Same as ref. Returned as a convenience for operations that want +/// to use the updated values after the update is done. +class ScatterNdSub { + public: + /// Optional attribute setters for ScatterNdSub + struct Attrs { + /// An optional bool. Defaults to True. If True, the assignment will + /// be protected by a lock; otherwise the behavior is undefined, + /// but may exhibit less contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = false; + }; + ScatterNdSub(const ::tensorflow::Scope& scope, ::tensorflow::Input ref, + ::tensorflow::Input indices, ::tensorflow::Input updates); + ScatterNdSub(const ::tensorflow::Scope& scope, ::tensorflow::Input ref, + ::tensorflow::Input indices, ::tensorflow::Input updates, const + ScatterNdSub::Attrs& attrs); + operator ::tensorflow::Output() const { return output_ref; } + operator ::tensorflow::Input() const { return output_ref; } + ::tensorflow::Node* node() const { return output_ref.node(); } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; + ::tensorflow::Output output_ref; +}; + +/// Applies sparse `updates` to individual values or slices within a given +/// +/// variable according to `indices`. +/// +/// `ref` is a `Tensor` with rank `P` and `indices` is a `Tensor` of rank `Q`. +/// +/// `indices` must be integer tensor, containing indices into `ref`. +/// It must be shape \\([d_0, ..., d_{Q-2}, K]\\) where `0 < K <= P`. +/// +/// The innermost dimension of `indices` (with length `K`) corresponds to +/// indices into elements (if `K = P`) or slices (if `K < P`) along the `K`th +/// dimension of `ref`. +/// +/// `updates` is `Tensor` of rank `Q-1+P-K` with shape: +/// +/// $$[d_0, ..., d_{Q-2}, ref.shape[K], ..., ref.shape[P-1]].$$ +/// +/// For example, say we want to update 4 scattered elements to a rank-1 tensor to +/// 8 elements. In Python, that update would look like this: +/// +/// ```python +/// ref = tf.Variable([1, 2, 3, 4, 5, 6, 7, 8]) +/// indices = tf.constant([[4], [3], [1] ,[7]]) +/// updates = tf.constant([9, 10, 11, 12]) +/// update = tf.scatter_nd_update(ref, indices, updates) +/// with tf.Session() as sess: +/// print sess.run(update) +/// ``` +/// +/// The resulting update to ref would look like this: +/// +/// [1, 11, 3, 10, 9, 6, 7, 12] +/// +/// See `tf.scatter_nd` for more details about how to make updates to +/// slices. +/// +/// See also `tf.scatter_update` and `tf.batch_scatter_update`. +/// +/// Arguments: +/// * scope: A Scope object +/// * ref: A mutable Tensor. Should be from a Variable node. +/// * indices: A Tensor. Must be one of the following types: int32, int64. +/// A tensor of indices into ref. +/// * updates: A Tensor. Must have the same type as ref. A tensor of updated +/// values to add to ref. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: An optional bool. Defaults to True. If True, the assignment will +/// be protected by a lock; otherwise the behavior is undefined, +/// but may exhibit less contention. +/// +/// Returns: +/// * `Output`: Same as ref. Returned as a convenience for operations that want to +/// use the updated values after the update is done. +class ScatterNdUpdate { + public: + /// Optional attribute setters for ScatterNdUpdate + struct Attrs { + /// An optional bool. Defaults to True. If True, the assignment will + /// be protected by a lock; otherwise the behavior is undefined, + /// but may exhibit less contention. + /// + /// Defaults to true + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = true; + }; + ScatterNdUpdate(const ::tensorflow::Scope& scope, ::tensorflow::Input ref, + ::tensorflow::Input indices, ::tensorflow::Input updates); + ScatterNdUpdate(const ::tensorflow::Scope& scope, ::tensorflow::Input ref, + ::tensorflow::Input indices, ::tensorflow::Input updates, const + ScatterNdUpdate::Attrs& attrs); + operator ::tensorflow::Output() const { return output_ref; } + operator ::tensorflow::Input() const { return output_ref; } + ::tensorflow::Node* node() const { return output_ref.node(); } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; + ::tensorflow::Output output_ref; +}; + +/// Subtracts sparse updates to a variable reference. +/// +/// ```python +/// # Scalar indices +/// ref[indices, ...] -= updates[...] +/// +/// # Vector indices (for each i) +/// ref[indices[i], ...] -= updates[i, ...] +/// +/// # High rank indices (for each i, ..., j) +/// ref[indices[i, ..., j], ...] -= updates[i, ..., j, ...] +/// ``` +/// +/// This operation outputs `ref` after the update is done. +/// This makes it easier to chain operations that need to use the reset value. +/// +/// Duplicate entries are handled correctly: if multiple `indices` reference +/// the same location, their (negated) contributions add. +/// +/// Requires `updates.shape = indices.shape + ref.shape[1:]` or `updates.shape = []`. +/// +///
+/// +///
+/// +/// Arguments: +/// * scope: A Scope object +/// * ref: Should be from a `Variable` node. +/// * indices: A tensor of indices into the first dimension of `ref`. +/// * updates: A tensor of updated values to subtract from `ref`. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If True, the subtraction will be protected by a lock; +/// otherwise the behavior is undefined, but may exhibit less contention. +/// +/// Returns: +/// * `Output`: = Same as `ref`. Returned as a convenience for operations that want +/// to use the updated values after the update is done. +class ScatterSub { + public: + /// Optional attribute setters for ScatterSub + struct Attrs { + /// If True, the subtraction will be protected by a lock; + /// otherwise the behavior is undefined, but may exhibit less contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = false; + }; + ScatterSub(const ::tensorflow::Scope& scope, ::tensorflow::Input ref, + ::tensorflow::Input indices, ::tensorflow::Input updates); + ScatterSub(const ::tensorflow::Scope& scope, ::tensorflow::Input ref, + ::tensorflow::Input indices, ::tensorflow::Input updates, const + ScatterSub::Attrs& attrs); + operator ::tensorflow::Output() const { return output_ref; } + operator ::tensorflow::Input() const { return output_ref; } + ::tensorflow::Node* node() const { return output_ref.node(); } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; + ::tensorflow::Output output_ref; +}; + +/// Applies sparse updates to a variable reference. +/// +/// This operation computes +/// +/// ```python +/// # Scalar indices +/// ref[indices, ...] = updates[...] +/// +/// # Vector indices (for each i) +/// ref[indices[i], ...] = updates[i, ...] +/// +/// # High rank indices (for each i, ..., j) +/// ref[indices[i, ..., j], ...] = updates[i, ..., j, ...] +/// ``` +/// +/// This operation outputs `ref` after the update is done. +/// This makes it easier to chain operations that need to use the reset value. +/// +/// If values in `ref` is to be updated more than once, because there are +/// duplicate entries in `indices`, the order at which the updates happen +/// for each value is undefined. +/// +/// Requires `updates.shape = indices.shape + ref.shape[1:]` or `updates.shape = []`. +/// +///
+/// +///
+/// +/// See also `tf.batch_scatter_update` and `tf.scatter_nd_update`. +/// +/// Arguments: +/// * scope: A Scope object +/// * ref: Should be from a `Variable` node. +/// * indices: A tensor of indices into the first dimension of `ref`. +/// * updates: A tensor of updated values to store in `ref`. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If True, the assignment will be protected by a lock; +/// otherwise the behavior is undefined, but may exhibit less contention. +/// +/// Returns: +/// * `Output`: = Same as `ref`. Returned as a convenience for operations that want +/// to use the updated values after the update is done. +class ScatterUpdate { + public: + /// Optional attribute setters for ScatterUpdate + struct Attrs { + /// If True, the assignment will be protected by a lock; + /// otherwise the behavior is undefined, but may exhibit less contention. + /// + /// Defaults to true + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = true; + }; + ScatterUpdate(const ::tensorflow::Scope& scope, ::tensorflow::Input ref, + ::tensorflow::Input indices, ::tensorflow::Input updates); + ScatterUpdate(const ::tensorflow::Scope& scope, ::tensorflow::Input ref, + ::tensorflow::Input indices, ::tensorflow::Input updates, const + ScatterUpdate::Attrs& attrs); + operator ::tensorflow::Output() const { return output_ref; } + operator ::tensorflow::Input() const { return output_ref; } + ::tensorflow::Node* node() const { return output_ref.node(); } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; + ::tensorflow::Output output_ref; +}; + +/// Returns a tensor that may be mutated, but only persists within a single step. +/// +/// This is an experimental op for internal use only and it is possible to use this +/// op in unsafe ways. DO NOT USE unless you fully understand the risks. +/// +/// It is the caller's responsibility to ensure that 'ref' is eventually passed to a +/// matching 'DestroyTemporaryVariable' op after all other uses have completed. +/// +/// Outputs a ref to the tensor state so it may be read or modified. +/// +/// E.g. +/// var = state_ops._temporary_variable([1, 2], types.float_) +/// var_name = var.op.name +/// var = state_ops.assign(var, [[4.0, 5.0]]) +/// var = state_ops.assign_add(var, [[6.0, 7.0]]) +/// final = state_ops._destroy_temporary_variable(var, var_name=var_name) +/// +/// Arguments: +/// * scope: A Scope object +/// * shape: The shape of the variable tensor. +/// * dtype: The type of elements in the variable tensor. +/// +/// Optional attributes (see `Attrs`): +/// * var_name: Overrides the name used for the temporary variable resource. Default +/// value is the name of the 'TemporaryVariable' op (which is guaranteed unique). +/// +/// Returns: +/// * `Output`: A reference to the variable tensor. +class TemporaryVariable { + public: + /// Optional attribute setters for TemporaryVariable + struct Attrs { + /// Overrides the name used for the temporary variable resource. Default + /// value is the name of the 'TemporaryVariable' op (which is guaranteed unique). + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs VarName(StringPiece x) { + Attrs ret = *this; + ret.var_name_ = x; + return ret; + } + + StringPiece var_name_ = ""; + }; + TemporaryVariable(const ::tensorflow::Scope& scope, PartialTensorShape shape, + DataType dtype); + TemporaryVariable(const ::tensorflow::Scope& scope, PartialTensorShape shape, + DataType dtype, const TemporaryVariable::Attrs& attrs); + operator ::tensorflow::Output() const { return ref; } + operator ::tensorflow::Input() const { return ref; } + ::tensorflow::Node* node() const { return ref.node(); } + + static Attrs VarName(StringPiece x) { + return Attrs().VarName(x); + } + + Operation operation; + ::tensorflow::Output ref; +}; + +/// Holds state in the form of a tensor that persists across steps. +/// +/// Outputs a ref to the tensor state so it may be read or modified. +/// TODO(zhifengc/mrry): Adds a pointer to a more detail document +/// about sharing states in tensorflow. +/// +/// Arguments: +/// * scope: A Scope object +/// * shape: The shape of the variable tensor. +/// * dtype: The type of elements in the variable tensor. +/// +/// Optional attributes (see `Attrs`): +/// * container: If non-empty, this variable is placed in the given container. +/// Otherwise, a default container is used. +/// * shared_name: If non-empty, this variable is named in the given bucket +/// with this shared_name. Otherwise, the node name is used instead. +/// +/// Returns: +/// * `Output`: A reference to the variable tensor. +class Variable { + public: + /// Optional attribute setters for Variable + struct Attrs { + /// If non-empty, this variable is placed in the given container. + /// Otherwise, a default container is used. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs Container(StringPiece x) { + Attrs ret = *this; + ret.container_ = x; + return ret; + } + + /// If non-empty, this variable is named in the given bucket + /// with this shared_name. Otherwise, the node name is used instead. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs SharedName(StringPiece x) { + Attrs ret = *this; + ret.shared_name_ = x; + return ret; + } + + StringPiece container_ = ""; + StringPiece shared_name_ = ""; + }; + Variable(const ::tensorflow::Scope& scope, PartialTensorShape shape, DataType + dtype); + Variable(const ::tensorflow::Scope& scope, PartialTensorShape shape, DataType + dtype, const Variable::Attrs& attrs); + operator ::tensorflow::Output() const { return ref; } + operator ::tensorflow::Input() const { return ref; } + ::tensorflow::Node* node() const { return ref.node(); } + + static Attrs Container(StringPiece x) { + return Attrs().Container(x); + } + static Attrs SharedName(StringPiece x) { + return Attrs().SharedName(x); + } + + Operation operation; + ::tensorflow::Output ref; +}; + +/// @} + +} // namespace ops +} // namespace tensorflow + +#endif // TENSORFLOW_CC_OPS_STATE_OPS_H_ diff --git a/linux/include/tensorflow/cc/ops/state_ops_internal.h b/linux/include/tensorflow/cc/ops/state_ops_internal.h new file mode 100644 index 000000000..f9482a10a --- /dev/null +++ b/linux/include/tensorflow/cc/ops/state_ops_internal.h @@ -0,0 +1,28 @@ +// This file is MACHINE GENERATED! Do not edit. + +#ifndef TENSORFLOW_CC_OPS_STATE_OPS_INTERNAL_H_ +#define TENSORFLOW_CC_OPS_STATE_OPS_INTERNAL_H_ + +// This file is MACHINE GENERATED! Do not edit. + +#include "tensorflow/cc/framework/ops.h" +#include "tensorflow/cc/framework/scope.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/gtl/array_slice.h" + +namespace tensorflow { +namespace ops { +namespace internal { +// NOTE: This namespace has internal TensorFlow details that +// are not part of TensorFlow's public API. + +/// @defgroup state_ops_internal State Ops Internal +/// @{ + +} // namespace internal +} // namespace ops +} // namespace tensorflow + +#endif // TENSORFLOW_CC_OPS_STATE_OPS_INTERNAL_H_ diff --git a/linux/include/tensorflow/cc/ops/string_ops.h b/linux/include/tensorflow/cc/ops/string_ops.h new file mode 100644 index 000000000..105301acb --- /dev/null +++ b/linux/include/tensorflow/cc/ops/string_ops.h @@ -0,0 +1,902 @@ +// This file is MACHINE GENERATED! Do not edit. + +#ifndef TENSORFLOW_CC_OPS_STRING_OPS_H_ +#define TENSORFLOW_CC_OPS_STRING_OPS_H_ + +// This file is MACHINE GENERATED! Do not edit. + +#include "tensorflow/cc/framework/ops.h" +#include "tensorflow/cc/framework/scope.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/gtl/array_slice.h" + +namespace tensorflow { +namespace ops { + +/// @defgroup string_ops String Ops +/// @{ + +/// Converts each entry in the given tensor to strings. Supports many numeric +/// +/// types and boolean. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Optional attributes (see `Attrs`): +/// * precision: The post-decimal precision to use for floating point numbers. +/// Only used if precision > -1. +/// * scientific: Use scientific notation for floating point numbers. +/// * shortest: Use shortest representation (either scientific or standard) for +/// floating point numbers. +/// * width: Pad pre-decimal numbers to this width. +/// Applies to both floating point and integer numbers. +/// Only used if width > -1. +/// * fill: The value to pad if width > -1. If empty, pads with spaces. +/// Another typical value is '0'. String cannot be longer than 1 character. +/// +/// Returns: +/// * `Output`: The output tensor. +class AsString { + public: + /// Optional attribute setters for AsString + struct Attrs { + /// The post-decimal precision to use for floating point numbers. + /// Only used if precision > -1. + /// + /// Defaults to -1 + TF_MUST_USE_RESULT Attrs Precision(int64 x) { + Attrs ret = *this; + ret.precision_ = x; + return ret; + } + + /// Use scientific notation for floating point numbers. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs Scientific(bool x) { + Attrs ret = *this; + ret.scientific_ = x; + return ret; + } + + /// Use shortest representation (either scientific or standard) for + /// floating point numbers. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs Shortest(bool x) { + Attrs ret = *this; + ret.shortest_ = x; + return ret; + } + + /// Pad pre-decimal numbers to this width. + /// Applies to both floating point and integer numbers. + /// Only used if width > -1. + /// + /// Defaults to -1 + TF_MUST_USE_RESULT Attrs Width(int64 x) { + Attrs ret = *this; + ret.width_ = x; + return ret; + } + + /// The value to pad if width > -1. If empty, pads with spaces. + /// Another typical value is '0'. String cannot be longer than 1 character. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs Fill(StringPiece x) { + Attrs ret = *this; + ret.fill_ = x; + return ret; + } + + int64 precision_ = -1; + bool scientific_ = false; + bool shortest_ = false; + int64 width_ = -1; + StringPiece fill_ = ""; + }; + AsString(const ::tensorflow::Scope& scope, ::tensorflow::Input input); + AsString(const ::tensorflow::Scope& scope, ::tensorflow::Input input, const + AsString::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs Precision(int64 x) { + return Attrs().Precision(x); + } + static Attrs Scientific(bool x) { + return Attrs().Scientific(x); + } + static Attrs Shortest(bool x) { + return Attrs().Shortest(x); + } + static Attrs Width(int64 x) { + return Attrs().Width(x); + } + static Attrs Fill(StringPiece x) { + return Attrs().Fill(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Decode web-safe base64-encoded strings. +/// +/// Input may or may not have padding at the end. See EncodeBase64 for padding. +/// Web-safe means that input must use - and _ instead of + and /. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: Base64 strings to decode. +/// +/// Returns: +/// * `Output`: Decoded strings. +class DecodeBase64 { + public: + DecodeBase64(const ::tensorflow::Scope& scope, ::tensorflow::Input input); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Encode strings into web-safe base64 format. +/// +/// Refer to the following article for more information on base64 format: +/// en.wikipedia.org/wiki/Base64. Base64 strings may have padding with '=' at the +/// end so that the encoded has length multiple of 4. See Padding section of the +/// link above. +/// +/// Web-safe means that the encoder uses - and _ instead of + and /. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: Strings to be encoded. +/// +/// Optional attributes (see `Attrs`): +/// * pad: Bool whether padding is applied at the ends. +/// +/// Returns: +/// * `Output`: Input strings encoded in base64. +class EncodeBase64 { + public: + /// Optional attribute setters for EncodeBase64 + struct Attrs { + /// Bool whether padding is applied at the ends. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs Pad(bool x) { + Attrs ret = *this; + ret.pad_ = x; + return ret; + } + + bool pad_ = false; + }; + EncodeBase64(const ::tensorflow::Scope& scope, ::tensorflow::Input input); + EncodeBase64(const ::tensorflow::Scope& scope, ::tensorflow::Input input, const + EncodeBase64::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs Pad(bool x) { + return Attrs().Pad(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Joins a string Tensor across the given dimensions. +/// +/// Computes the string join across dimensions in the given string Tensor of shape +/// `[\\(d_0, d_1, ..., d_{n-1}\\)]`. Returns a new Tensor created by joining the input +/// strings with the given separator (default: empty string). Negative indices are +/// counted backwards from the end, with `-1` being equivalent to `n - 1`. If +/// indices are not specified, joins across all dimensions beginning from `n - 1` +/// through `0`. +/// +/// For example: +/// +/// ```python +/// # tensor `a` is [["a", "b"], ["c", "d"]] +/// tf.reduce_join(a, 0) ==> ["ac", "bd"] +/// tf.reduce_join(a, 1) ==> ["ab", "cd"] +/// tf.reduce_join(a, -2) = tf.reduce_join(a, 0) ==> ["ac", "bd"] +/// tf.reduce_join(a, -1) = tf.reduce_join(a, 1) ==> ["ab", "cd"] +/// tf.reduce_join(a, 0, keep_dims=True) ==> [["ac", "bd"]] +/// tf.reduce_join(a, 1, keep_dims=True) ==> [["ab"], ["cd"]] +/// tf.reduce_join(a, 0, separator=".") ==> ["a.c", "b.d"] +/// tf.reduce_join(a, [0, 1]) ==> "acbd" +/// tf.reduce_join(a, [1, 0]) ==> "abcd" +/// tf.reduce_join(a, []) ==> [["a", "b"], ["c", "d"]] +/// tf.reduce_join(a) = tf.reduce_join(a, [1, 0]) ==> "abcd" +/// ``` +/// +/// Arguments: +/// * scope: A Scope object +/// * inputs: The input to be joined. All reduced indices must have non-zero size. +/// * reduction_indices: The dimensions to reduce over. Dimensions are reduced in the +/// order specified. Omitting `reduction_indices` is equivalent to passing +/// `[n-1, n-2, ..., 0]`. Negative indices from `-n` to `-1` are supported. +/// +/// Optional attributes (see `Attrs`): +/// * keep_dims: If `True`, retain reduced dimensions with length `1`. +/// * separator: The separator to use when joining. +/// +/// Returns: +/// * `Output`: Has shape equal to that of the input with reduced dimensions removed or +/// set to `1` depending on `keep_dims`. +class ReduceJoin { + public: + /// Optional attribute setters for ReduceJoin + struct Attrs { + /// If `True`, retain reduced dimensions with length `1`. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs KeepDims(bool x) { + Attrs ret = *this; + ret.keep_dims_ = x; + return ret; + } + + /// The separator to use when joining. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs Separator(StringPiece x) { + Attrs ret = *this; + ret.separator_ = x; + return ret; + } + + bool keep_dims_ = false; + StringPiece separator_ = ""; + }; + ReduceJoin(const ::tensorflow::Scope& scope, ::tensorflow::Input inputs, + ::tensorflow::Input reduction_indices); + ReduceJoin(const ::tensorflow::Scope& scope, ::tensorflow::Input inputs, + ::tensorflow::Input reduction_indices, const ReduceJoin::Attrs& + attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs KeepDims(bool x) { + return Attrs().KeepDims(x); + } + static Attrs Separator(StringPiece x) { + return Attrs().Separator(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Check if the input matches the regex pattern. +/// +/// The input is a string tensor of any shape. The pattern is a scalar +/// string tensor which is applied to every element of the input tensor. +/// The boolean values (True or False) of the output tensor indicate +/// if the input matches the regex pattern provided. +/// +/// The pattern follows the re2 syntax (https://github.com/google/re2/wiki/Syntax) +/// +/// Arguments: +/// * scope: A Scope object +/// * input: A string tensor of the text to be processed. +/// * pattern: A scalar string tensor containing the regular expression to match the input. +/// +/// Returns: +/// * `Output`: A bool tensor with the same shape as `input`. +class RegexFullMatch { + public: + RegexFullMatch(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input pattern); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Replaces the match of pattern in input with rewrite. +/// +/// It follows the re2 syntax (https://github.com/google/re2/wiki/Syntax) +/// +/// Arguments: +/// * scope: A Scope object +/// * input: The text to be processed. +/// * pattern: The regular expression to match the input. +/// * rewrite: The rewrite to be applied to the matched expresion. +/// +/// Optional attributes (see `Attrs`): +/// * replace_global: If True, the replacement is global, otherwise the replacement +/// is done only on the first match. +/// +/// Returns: +/// * `Output`: The text after applying pattern and rewrite. +class RegexReplace { + public: + /// Optional attribute setters for RegexReplace + struct Attrs { + /// If True, the replacement is global, otherwise the replacement + /// is done only on the first match. + /// + /// Defaults to true + TF_MUST_USE_RESULT Attrs ReplaceGlobal(bool x) { + Attrs ret = *this; + ret.replace_global_ = x; + return ret; + } + + bool replace_global_ = true; + }; + RegexReplace(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input pattern, ::tensorflow::Input rewrite); + RegexReplace(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input pattern, ::tensorflow::Input rewrite, const + RegexReplace::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs ReplaceGlobal(bool x) { + return Attrs().ReplaceGlobal(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Formats a string template using a list of tensors. +/// +/// Formats a string template using a list of tensors, pretty-printing tensor summaries. +/// +/// Arguments: +/// * scope: A Scope object +/// * inputs: The list of tensors to format into the placeholder string. +/// +/// Optional attributes (see `Attrs`): +/// * template_: A string, the template to format tensor summaries into. +/// * placeholder: A string, at each placeholder in the template a subsequent tensor summary will be inserted. +/// * summarize: When formatting the tensor summaries print the first and last summarize entries of each tensor dimension. +/// +/// Returns: +/// * `Output`: = The resulting string scalar. +class StringFormat { + public: + /// Optional attribute setters for StringFormat + struct Attrs { + /// A string, the template to format tensor summaries into. + /// + /// Defaults to "%s" + TF_MUST_USE_RESULT Attrs Template(StringPiece x) { + Attrs ret = *this; + ret.template_ = x; + return ret; + } + + /// A string, at each placeholder in the template a subsequent tensor summary will be inserted. + /// + /// Defaults to "%s" + TF_MUST_USE_RESULT Attrs Placeholder(StringPiece x) { + Attrs ret = *this; + ret.placeholder_ = x; + return ret; + } + + /// When formatting the tensor summaries print the first and last summarize entries of each tensor dimension. + /// + /// Defaults to 3 + TF_MUST_USE_RESULT Attrs Summarize(int64 x) { + Attrs ret = *this; + ret.summarize_ = x; + return ret; + } + + StringPiece template_ = "%s"; + StringPiece placeholder_ = "%s"; + int64 summarize_ = 3; + }; + StringFormat(const ::tensorflow::Scope& scope, ::tensorflow::InputList inputs); + StringFormat(const ::tensorflow::Scope& scope, ::tensorflow::InputList inputs, + const StringFormat::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs Template(StringPiece x) { + return Attrs().Template(x); + } + static Attrs Placeholder(StringPiece x) { + return Attrs().Placeholder(x); + } + static Attrs Summarize(int64 x) { + return Attrs().Summarize(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Joins the strings in the given list of string tensors into one tensor; +/// +/// with the given separator (default is an empty separator). +/// +/// Arguments: +/// * scope: A Scope object +/// * inputs: A list of string tensors. The tensors must all have the same shape, +/// or be scalars. Scalars may be mixed in; these will be broadcast to the shape +/// of non-scalar inputs. +/// +/// Optional attributes (see `Attrs`): +/// * separator: string, an optional join separator. +/// +/// Returns: +/// * `Output`: The output tensor. +class StringJoin { + public: + /// Optional attribute setters for StringJoin + struct Attrs { + /// string, an optional join separator. + /// + /// Defaults to "" + TF_MUST_USE_RESULT Attrs Separator(StringPiece x) { + Attrs ret = *this; + ret.separator_ = x; + return ret; + } + + StringPiece separator_ = ""; + }; + StringJoin(const ::tensorflow::Scope& scope, ::tensorflow::InputList inputs); + StringJoin(const ::tensorflow::Scope& scope, ::tensorflow::InputList inputs, + const StringJoin::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs Separator(StringPiece x) { + return Attrs().Separator(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// String lengths of `input`. +/// +/// Computes the length of each string given in the input tensor. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: The string for which to compute the length. +/// +/// Optional attributes (see `Attrs`): +/// * unit: The unit that is counted to compute string length. One of: `"BYTE"` (for +/// the number of bytes in each string) or `"UTF8_CHAR"` (for the number of UTF-8 +/// encoded Unicode code points in each string). Results are undefined +/// if `unit=UTF8_CHAR` and the `input` strings do not contain structurally +/// valid UTF-8. +/// +/// Returns: +/// * `Output`: Integer tensor that has the same shape as `input`. The output contains the +/// element-wise string lengths of `input`. +class StringLength { + public: + /// Optional attribute setters for StringLength + struct Attrs { + /// The unit that is counted to compute string length. One of: `"BYTE"` (for + /// the number of bytes in each string) or `"UTF8_CHAR"` (for the number of UTF-8 + /// encoded Unicode code points in each string). Results are undefined + /// if `unit=UTF8_CHAR` and the `input` strings do not contain structurally + /// valid UTF-8. + /// + /// Defaults to "BYTE" + TF_MUST_USE_RESULT Attrs Unit(StringPiece x) { + Attrs ret = *this; + ret.unit_ = x; + return ret; + } + + StringPiece unit_ = "BYTE"; + }; + StringLength(const ::tensorflow::Scope& scope, ::tensorflow::Input input); + StringLength(const ::tensorflow::Scope& scope, ::tensorflow::Input input, const + StringLength::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs Unit(StringPiece x) { + return Attrs().Unit(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Split elements of `input` based on `delimiter` into a `SparseTensor`. +/// +/// Let N be the size of source (typically N will be the batch size). Split each +/// element of `input` based on `delimiter` and return a `SparseTensor` +/// containing the splitted tokens. Empty tokens are ignored. +/// +/// `delimiter` can be empty, or a string of split characters. If `delimiter` is an +/// empty string, each element of `input` is split into individual single-byte +/// character strings, including splitting of UTF-8 multibyte sequences. Otherwise +/// every character of `delimiter` is a potential split point. +/// +/// For example: +/// N = 2, input[0] is 'hello world' and input[1] is 'a b c', then the output +/// will be +/// +/// indices = [0, 0; +/// 0, 1; +/// 1, 0; +/// 1, 1; +/// 1, 2] +/// shape = [2, 3] +/// values = ['hello', 'world', 'a', 'b', 'c'] +/// +/// Arguments: +/// * scope: A Scope object +/// * input: 1-D. Strings to split. +/// * delimiter: 0-D. Delimiter characters (bytes), or empty string. +/// +/// Optional attributes (see `Attrs`): +/// * skip_empty: A `bool`. If `True`, skip the empty strings from the result. +/// +/// Returns: +/// * `Output` indices: A dense matrix of int64 representing the indices of the sparse tensor. +/// * `Output` values: A vector of strings corresponding to the splited values. +/// * `Output` shape: a length-2 vector of int64 representing the shape of the sparse +/// tensor, where the first value is N and the second value is the maximum number +/// of tokens in a single input entry. +class StringSplit { + public: + /// Optional attribute setters for StringSplit + struct Attrs { + /// A `bool`. If `True`, skip the empty strings from the result. + /// + /// Defaults to true + TF_MUST_USE_RESULT Attrs SkipEmpty(bool x) { + Attrs ret = *this; + ret.skip_empty_ = x; + return ret; + } + + bool skip_empty_ = true; + }; + StringSplit(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input delimiter); + StringSplit(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input delimiter, const StringSplit::Attrs& attrs); + + static Attrs SkipEmpty(bool x) { + return Attrs().SkipEmpty(x); + } + + Operation operation; + ::tensorflow::Output indices; + ::tensorflow::Output values; + ::tensorflow::Output shape; +}; + +/// Split elements of `source` based on `sep` into a `SparseTensor`. +/// +/// Let N be the size of source (typically N will be the batch size). Split each +/// element of `source` based on `sep` and return a `SparseTensor` +/// containing the split tokens. Empty tokens are ignored. +/// +/// For example, N = 2, source[0] is 'hello world' and source[1] is 'a b c', +/// then the output will be +/// ``` +/// st.indices = [0, 0; +/// 0, 1; +/// 1, 0; +/// 1, 1; +/// 1, 2] +/// st.shape = [2, 3] +/// st.values = ['hello', 'world', 'a', 'b', 'c'] +/// ``` +/// +/// If `sep` is given, consecutive delimiters are not grouped together and are +/// deemed to delimit empty strings. For example, source of `"1<>2<><>3"` and +/// sep of `"<>"` returns `["1", "2", "", "3"]`. If `sep` is None or an empty +/// string, consecutive whitespace are regarded as a single separator, and the +/// result will contain no empty strings at the startor end if the string has +/// leading or trailing whitespace. +/// +/// Note that the above mentioned behavior matches python's str.split. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: `1-D` string `Tensor`, the strings to split. +/// * sep: `0-D` string `Tensor`, the delimiter character. +/// +/// Optional attributes (see `Attrs`): +/// * maxsplit: An `int`. If `maxsplit > 0`, limit of the split of the result. +/// +/// Returns: +/// * `Output` indices +/// * `Output` values +/// * `Output` shape +class StringSplitV2 { + public: + /// Optional attribute setters for StringSplitV2 + struct Attrs { + /// An `int`. If `maxsplit > 0`, limit of the split of the result. + /// + /// Defaults to -1 + TF_MUST_USE_RESULT Attrs Maxsplit(int64 x) { + Attrs ret = *this; + ret.maxsplit_ = x; + return ret; + } + + int64 maxsplit_ = -1; + }; + StringSplitV2(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input sep); + StringSplitV2(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input sep, const StringSplitV2::Attrs& attrs); + + static Attrs Maxsplit(int64 x) { + return Attrs().Maxsplit(x); + } + + Operation operation; + ::tensorflow::Output indices; + ::tensorflow::Output values; + ::tensorflow::Output shape; +}; + +/// Strip leading and trailing whitespaces from the Tensor. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: A string `Tensor` of any shape. +/// +/// Returns: +/// * `Output`: A string `Tensor` of the same shape as the input. +class StringStrip { + public: + StringStrip(const ::tensorflow::Scope& scope, ::tensorflow::Input input); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Converts each string in the input Tensor to its hash mod by a number of buckets. +/// +/// The hash function is deterministic on the content of the string within the +/// process. +/// +/// Note that the hash function may change from time to time. +/// This functionality will be deprecated and it's recommended to use +/// `tf.string_to_hash_bucket_fast()` or `tf.string_to_hash_bucket_strong()`. +/// +/// Arguments: +/// * scope: A Scope object +/// * num_buckets: The number of buckets. +/// +/// Returns: +/// * `Output`: A Tensor of the same shape as the input `string_tensor`. +class StringToHashBucket { + public: + StringToHashBucket(const ::tensorflow::Scope& scope, ::tensorflow::Input + string_tensor, int64 num_buckets); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Converts each string in the input Tensor to its hash mod by a number of buckets. +/// +/// The hash function is deterministic on the content of the string within the +/// process and will never change. However, it is not suitable for cryptography. +/// This function may be used when CPU time is scarce and inputs are trusted or +/// unimportant. There is a risk of adversaries constructing inputs that all hash +/// to the same bucket. To prevent this problem, use a strong hash function with +/// `tf.string_to_hash_bucket_strong`. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: The strings to assign a hash bucket. +/// * num_buckets: The number of buckets. +/// +/// Returns: +/// * `Output`: A Tensor of the same shape as the input `string_tensor`. +class StringToHashBucketFast { + public: + StringToHashBucketFast(const ::tensorflow::Scope& scope, ::tensorflow::Input + input, int64 num_buckets); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Converts each string in the input Tensor to its hash mod by a number of buckets. +/// +/// The hash function is deterministic on the content of the string within the +/// process. The hash function is a keyed hash function, where attribute `key` +/// defines the key of the hash function. `key` is an array of 2 elements. +/// +/// A strong hash is important when inputs may be malicious, e.g. URLs with +/// additional components. Adversaries could try to make their inputs hash to the +/// same bucket for a denial-of-service attack or to skew the results. A strong +/// hash prevents this by making it difficult, if not infeasible, to compute inputs +/// that hash to the same bucket. This comes at a cost of roughly 4x higher compute +/// time than `tf.string_to_hash_bucket_fast`. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: The strings to assign a hash bucket. +/// * num_buckets: The number of buckets. +/// * key: The key for the keyed hash function passed as a list of two uint64 +/// elements. +/// +/// Returns: +/// * `Output`: A Tensor of the same shape as the input `string_tensor`. +class StringToHashBucketStrong { + public: + StringToHashBucketStrong(const ::tensorflow::Scope& scope, ::tensorflow::Input + input, int64 num_buckets, const gtl::ArraySlice& + key); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Return substrings from `Tensor` of strings. +/// +/// For each string in the input `Tensor`, creates a substring starting at index +/// `pos` with a total length of `len`. +/// +/// If `len` defines a substring that would extend beyond the length of the input +/// string, then as many characters as possible are used. +/// +/// A negative `pos` indicates distance within the string backwards from the end. +/// +/// If `pos` specifies an index which is out of range for any of the input strings, +/// then an `InvalidArgumentError` is thrown. +/// +/// `pos` and `len` must have the same shape, otherwise a `ValueError` is thrown on +/// Op creation. +/// +/// *NOTE*: `Substr` supports broadcasting up to two dimensions. More about +/// broadcasting +/// [here](http://docs.scipy.org/doc/numpy/user/basics.broadcasting.html) +/// +/// --- +/// +/// Examples +/// +/// Using scalar `pos` and `len`: +/// +/// ```python +/// input = [b'Hello', b'World'] +/// position = 1 +/// length = 3 +/// +/// output = [b'ell', b'orl'] +/// ``` +/// +/// Using `pos` and `len` with same shape as `input`: +/// +/// ```python +/// input = [[b'ten', b'eleven', b'twelve'], +/// [b'thirteen', b'fourteen', b'fifteen'], +/// [b'sixteen', b'seventeen', b'eighteen']] +/// position = [[1, 2, 3], +/// [1, 2, 3], +/// [1, 2, 3]] +/// length = [[2, 3, 4], +/// [4, 3, 2], +/// [5, 5, 5]] +/// +/// output = [[b'en', b'eve', b'lve'], +/// [b'hirt', b'urt', b'te'], +/// [b'ixtee', b'vente', b'hteen']] +/// ``` +/// +/// Broadcasting `pos` and `len` onto `input`: +/// +/// ``` +/// input = [[b'ten', b'eleven', b'twelve'], +/// [b'thirteen', b'fourteen', b'fifteen'], +/// [b'sixteen', b'seventeen', b'eighteen'], +/// [b'nineteen', b'twenty', b'twentyone']] +/// position = [1, 2, 3] +/// length = [1, 2, 3] +/// +/// output = [[b'e', b'ev', b'lve'], +/// [b'h', b'ur', b'tee'], +/// [b'i', b've', b'hte'], +/// [b'i', b'en', b'nty']] +/// ``` +/// +/// Broadcasting `input` onto `pos` and `len`: +/// +/// ``` +/// input = b'thirteen' +/// position = [1, 5, 7] +/// length = [3, 2, 1] +/// +/// output = [b'hir', b'ee', b'n'] +/// ``` +/// +/// Arguments: +/// * scope: A Scope object +/// * input: Tensor of strings +/// * pos: Scalar defining the position of first character in each substring +/// * len: Scalar defining the number of characters to include in each substring +/// +/// Returns: +/// * `Output`: Tensor of substrings +class Substr { + public: + Substr(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + ::tensorflow::Input pos, ::tensorflow::Input len); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Determine the script codes of a given tensor of Unicode integer code points. +/// +/// This operation converts Unicode code points to script codes corresponding to +/// each code point. Script codes correspond to International Components for +/// Unicode (ICU) UScriptCode values. See http://icu-project.org/apiref/icu4c/uscript_8h.html. +/// Returns -1 (USCRIPT_INVALID_CODE) for invalid codepoints. Output shape will +/// match input shape. +/// +/// Arguments: +/// * scope: A Scope object +/// * input: A Tensor of int32 Unicode code points. +/// +/// Returns: +/// * `Output`: A Tensor of int32 script codes corresponding to each input code point. +class UnicodeScript { + public: + UnicodeScript(const ::tensorflow::Scope& scope, ::tensorflow::Input input); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// @} + +} // namespace ops +} // namespace tensorflow + +#endif // TENSORFLOW_CC_OPS_STRING_OPS_H_ diff --git a/linux/include/tensorflow/cc/ops/string_ops_internal.h b/linux/include/tensorflow/cc/ops/string_ops_internal.h new file mode 100644 index 000000000..aa3341db7 --- /dev/null +++ b/linux/include/tensorflow/cc/ops/string_ops_internal.h @@ -0,0 +1,105 @@ +// This file is MACHINE GENERATED! Do not edit. + +#ifndef TENSORFLOW_CC_OPS_STRING_OPS_INTERNAL_H_ +#define TENSORFLOW_CC_OPS_STRING_OPS_INTERNAL_H_ + +// This file is MACHINE GENERATED! Do not edit. + +#include "tensorflow/cc/framework/ops.h" +#include "tensorflow/cc/framework/scope.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/gtl/array_slice.h" + +namespace tensorflow { +namespace ops { +namespace internal { +// NOTE: This namespace has internal TensorFlow details that +// are not part of TensorFlow's public API. + +/// @defgroup string_ops_internal String Ops Internal +/// @{ + +/// Check if the input matches the regex pattern. +/// +/// The input is a string tensor of any shape. The pattern is the +/// regular expression to be matched with every element of the input tensor. +/// The boolean values (True or False) of the output tensor indicate +/// if the input matches the regex pattern provided. +/// +/// The pattern follows the re2 syntax (https://github.com/google/re2/wiki/Syntax) +/// +/// Arguments: +/// * scope: A Scope object +/// * input: A string tensor of the text to be processed. +/// * pattern: The regular expression to match the input. +/// +/// Returns: +/// * `Output`: A bool tensor with the same shape as `input`. +class StaticRegexFullMatch { + public: + StaticRegexFullMatch(const ::tensorflow::Scope& scope, ::tensorflow::Input + input, StringPiece pattern); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + Operation operation; + ::tensorflow::Output output; +}; + +/// Replaces the match of pattern in input with rewrite. +/// +/// It follows the re2 syntax (https://github.com/google/re2/wiki/Syntax) +/// +/// Arguments: +/// * scope: A Scope object +/// * input: The text to be processed. +/// * pattern: The regular expression to match the input. +/// * rewrite: The rewrite to be applied to the matched expresion. +/// +/// Optional attributes (see `Attrs`): +/// * replace_global: If True, the replacement is global, otherwise the replacement +/// is done only on the first match. +/// +/// Returns: +/// * `Output`: The text after applying pattern and rewrite. +class StaticRegexReplace { + public: + /// Optional attribute setters for StaticRegexReplace + struct Attrs { + /// If True, the replacement is global, otherwise the replacement + /// is done only on the first match. + /// + /// Defaults to true + TF_MUST_USE_RESULT Attrs ReplaceGlobal(bool x) { + Attrs ret = *this; + ret.replace_global_ = x; + return ret; + } + + bool replace_global_ = true; + }; + StaticRegexReplace(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + StringPiece pattern, StringPiece rewrite); + StaticRegexReplace(const ::tensorflow::Scope& scope, ::tensorflow::Input input, + StringPiece pattern, StringPiece rewrite, const + StaticRegexReplace::Attrs& attrs); + operator ::tensorflow::Output() const { return output; } + operator ::tensorflow::Input() const { return output; } + ::tensorflow::Node* node() const { return output.node(); } + + static Attrs ReplaceGlobal(bool x) { + return Attrs().ReplaceGlobal(x); + } + + Operation operation; + ::tensorflow::Output output; +}; + +} // namespace internal +} // namespace ops +} // namespace tensorflow + +#endif // TENSORFLOW_CC_OPS_STRING_OPS_INTERNAL_H_ diff --git a/linux/include/tensorflow/cc/ops/training_ops.h b/linux/include/tensorflow/cc/ops/training_ops.h new file mode 100644 index 000000000..a8ebb385d --- /dev/null +++ b/linux/include/tensorflow/cc/ops/training_ops.h @@ -0,0 +1,3177 @@ +// This file is MACHINE GENERATED! Do not edit. + +#ifndef TENSORFLOW_CC_OPS_TRAINING_OPS_H_ +#define TENSORFLOW_CC_OPS_TRAINING_OPS_H_ + +// This file is MACHINE GENERATED! Do not edit. + +#include "tensorflow/cc/framework/ops.h" +#include "tensorflow/cc/framework/scope.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/gtl/array_slice.h" + +namespace tensorflow { +namespace ops { + +/// @defgroup training_ops Training Ops +/// @{ + +/// Update '*var' according to the adadelta scheme. +/// +/// accum = rho() * accum + (1 - rho()) * grad.square(); +/// update = (update_accum + epsilon).sqrt() * (accum + epsilon()).rsqrt() * grad; +/// update_accum = rho() * update_accum + (1 - rho()) * update.square(); +/// var -= update; +/// +/// Arguments: +/// * scope: A Scope object +/// * var: Should be from a Variable(). +/// * accum: Should be from a Variable(). +/// * accum_update: Should be from a Variable(). +/// * lr: Scaling factor. Must be a scalar. +/// * rho: Decay factor. Must be a scalar. +/// * epsilon: Constant factor. Must be a scalar. +/// * grad: The gradient. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If True, updating of the var, accum and update_accum tensors will be protected by +/// a lock; otherwise the behavior is undefined, but may exhibit less contention. +/// +/// Returns: +/// * `Output`: Same as "var". +class ApplyAdadelta { + public: + /// Optional attribute setters for ApplyAdadelta + struct Attrs { + /// If True, updating of the var, accum and update_accum tensors will be protected by + /// a lock; otherwise the behavior is undefined, but may exhibit less contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = false; + }; + ApplyAdadelta(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input accum, ::tensorflow::Input accum_update, + ::tensorflow::Input lr, ::tensorflow::Input rho, + ::tensorflow::Input epsilon, ::tensorflow::Input grad); + ApplyAdadelta(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input accum, ::tensorflow::Input accum_update, + ::tensorflow::Input lr, ::tensorflow::Input rho, + ::tensorflow::Input epsilon, ::tensorflow::Input grad, const + ApplyAdadelta::Attrs& attrs); + operator ::tensorflow::Output() const { return out; } + operator ::tensorflow::Input() const { return out; } + ::tensorflow::Node* node() const { return out.node(); } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; + ::tensorflow::Output out; +}; + +/// Update '*var' according to the adagrad scheme. +/// +/// accum += grad * grad +/// var -= lr * grad * (1 / sqrt(accum)) +/// +/// Arguments: +/// * scope: A Scope object +/// * var: Should be from a Variable(). +/// * accum: Should be from a Variable(). +/// * lr: Scaling factor. Must be a scalar. +/// * grad: The gradient. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If `True`, updating of the var and accum tensors will be protected +/// by a lock; otherwise the behavior is undefined, but may exhibit less +/// contention. +/// +/// Returns: +/// * `Output`: Same as "var". +class ApplyAdagrad { + public: + /// Optional attribute setters for ApplyAdagrad + struct Attrs { + /// If `True`, updating of the var and accum tensors will be protected + /// by a lock; otherwise the behavior is undefined, but may exhibit less + /// contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + /// Defaults to true + TF_MUST_USE_RESULT Attrs UpdateSlots(bool x) { + Attrs ret = *this; + ret.update_slots_ = x; + return ret; + } + + bool use_locking_ = false; + bool update_slots_ = true; + }; + ApplyAdagrad(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input accum, ::tensorflow::Input lr, + ::tensorflow::Input grad); + ApplyAdagrad(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input accum, ::tensorflow::Input lr, + ::tensorflow::Input grad, const ApplyAdagrad::Attrs& attrs); + operator ::tensorflow::Output() const { return out; } + operator ::tensorflow::Input() const { return out; } + ::tensorflow::Node* node() const { return out.node(); } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + static Attrs UpdateSlots(bool x) { + return Attrs().UpdateSlots(x); + } + + Operation operation; + ::tensorflow::Output out; +}; + +/// Update '*var' according to the proximal adagrad scheme. +/// +/// Arguments: +/// * scope: A Scope object +/// * var: Should be from a Variable(). +/// * gradient_accumulator: Should be from a Variable(). +/// * gradient_squared_accumulator: Should be from a Variable(). +/// * grad: The gradient. +/// * lr: Scaling factor. Must be a scalar. +/// * l1: L1 regularization. Must be a scalar. +/// * l2: L2 regularization. Must be a scalar. +/// * global_step: Training step number. Must be a scalar. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If True, updating of the var and accum tensors will be protected by +/// a lock; otherwise the behavior is undefined, but may exhibit less contention. +/// +/// Returns: +/// * `Output`: Same as "var". +class ApplyAdagradDA { + public: + /// Optional attribute setters for ApplyAdagradDA + struct Attrs { + /// If True, updating of the var and accum tensors will be protected by + /// a lock; otherwise the behavior is undefined, but may exhibit less contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = false; + }; + ApplyAdagradDA(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input gradient_accumulator, ::tensorflow::Input + gradient_squared_accumulator, ::tensorflow::Input grad, + ::tensorflow::Input lr, ::tensorflow::Input l1, + ::tensorflow::Input l2, ::tensorflow::Input global_step); + ApplyAdagradDA(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input gradient_accumulator, ::tensorflow::Input + gradient_squared_accumulator, ::tensorflow::Input grad, + ::tensorflow::Input lr, ::tensorflow::Input l1, + ::tensorflow::Input l2, ::tensorflow::Input global_step, const + ApplyAdagradDA::Attrs& attrs); + operator ::tensorflow::Output() const { return out; } + operator ::tensorflow::Input() const { return out; } + ::tensorflow::Node* node() const { return out.node(); } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; + ::tensorflow::Output out; +}; + +/// Update '*var' according to the Adam algorithm. +/// +/// $$lr_t := \text{learning\_rate} * \sqrt{1 - beta_2^t} / (1 - beta_1^t)$$ +/// $$m_t := beta_1 * m_{t-1} + (1 - beta_1) * g$$ +/// $$v_t := beta_2 * v_{t-1} + (1 - beta_2) * g * g$$ +/// $$variable := variable - lr_t * m_t / (\sqrt{v_t} + \epsilon)$$ +/// +/// Arguments: +/// * scope: A Scope object +/// * var: Should be from a Variable(). +/// * m: Should be from a Variable(). +/// * v: Should be from a Variable(). +/// * beta1_power: Must be a scalar. +/// * beta2_power: Must be a scalar. +/// * lr: Scaling factor. Must be a scalar. +/// * beta1: Momentum factor. Must be a scalar. +/// * beta2: Momentum factor. Must be a scalar. +/// * epsilon: Ridge term. Must be a scalar. +/// * grad: The gradient. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If `True`, updating of the var, m, and v tensors will be protected +/// by a lock; otherwise the behavior is undefined, but may exhibit less +/// contention. +/// * use_nesterov: If `True`, uses the nesterov update. +/// +/// Returns: +/// * `Output`: Same as "var". +class ApplyAdam { + public: + /// Optional attribute setters for ApplyAdam + struct Attrs { + /// If `True`, updating of the var, m, and v tensors will be protected + /// by a lock; otherwise the behavior is undefined, but may exhibit less + /// contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + /// If `True`, uses the nesterov update. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseNesterov(bool x) { + Attrs ret = *this; + ret.use_nesterov_ = x; + return ret; + } + + bool use_locking_ = false; + bool use_nesterov_ = false; + }; + ApplyAdam(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input m, ::tensorflow::Input v, ::tensorflow::Input + beta1_power, ::tensorflow::Input beta2_power, ::tensorflow::Input lr, + ::tensorflow::Input beta1, ::tensorflow::Input beta2, + ::tensorflow::Input epsilon, ::tensorflow::Input grad); + ApplyAdam(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input m, ::tensorflow::Input v, ::tensorflow::Input + beta1_power, ::tensorflow::Input beta2_power, ::tensorflow::Input lr, + ::tensorflow::Input beta1, ::tensorflow::Input beta2, + ::tensorflow::Input epsilon, ::tensorflow::Input grad, const + ApplyAdam::Attrs& attrs); + operator ::tensorflow::Output() const { return out; } + operator ::tensorflow::Input() const { return out; } + ::tensorflow::Node* node() const { return out.node(); } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + static Attrs UseNesterov(bool x) { + return Attrs().UseNesterov(x); + } + + Operation operation; + ::tensorflow::Output out; +}; + +/// Update '*var' according to the AddSign update. +/// +/// m_t <- beta1 * m_{t-1} + (1 - beta1) * g +/// update <- (alpha + sign_decay * sign(g) *sign(m)) * g +/// variable <- variable - lr_t * update +/// +/// Arguments: +/// * scope: A Scope object +/// * var: Should be from a Variable(). +/// * m: Should be from a Variable(). +/// * lr: Scaling factor. Must be a scalar. +/// * alpha: Must be a scalar. +/// * sign_decay: Must be a scalar. +/// * beta: Must be a scalar. +/// * grad: The gradient. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If `True`, updating of the var and m tensors is +/// protected by a lock; otherwise the behavior is undefined, but may exhibit less +/// contention. +/// +/// Returns: +/// * `Output`: Same as "var". +class ApplyAddSign { + public: + /// Optional attribute setters for ApplyAddSign + struct Attrs { + /// If `True`, updating of the var and m tensors is + /// protected by a lock; otherwise the behavior is undefined, but may exhibit less + /// contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = false; + }; + ApplyAddSign(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input m, ::tensorflow::Input lr, ::tensorflow::Input + alpha, ::tensorflow::Input sign_decay, ::tensorflow::Input beta, + ::tensorflow::Input grad); + ApplyAddSign(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input m, ::tensorflow::Input lr, ::tensorflow::Input + alpha, ::tensorflow::Input sign_decay, ::tensorflow::Input beta, + ::tensorflow::Input grad, const ApplyAddSign::Attrs& attrs); + operator ::tensorflow::Output() const { return out; } + operator ::tensorflow::Input() const { return out; } + ::tensorflow::Node* node() const { return out.node(); } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; + ::tensorflow::Output out; +}; + +/// Update '*var' according to the centered RMSProp algorithm. +/// +/// The centered RMSProp algorithm uses an estimate of the centered second moment +/// (i.e., the variance) for normalization, as opposed to regular RMSProp, which +/// uses the (uncentered) second moment. This often helps with training, but is +/// slightly more expensive in terms of computation and memory. +/// +/// Note that in dense implementation of this algorithm, mg, ms, and mom will +/// update even if the grad is zero, but in this sparse implementation, mg, ms, +/// and mom will not update in iterations during which the grad is zero. +/// +/// mean_square = decay * mean_square + (1-decay) * gradient ** 2 +/// mean_grad = decay * mean_grad + (1-decay) * gradient +/// +/// Delta = learning_rate * gradient / sqrt(mean_square + epsilon - mean_grad ** 2) +/// +/// mg <- rho * mg_{t-1} + (1-rho) * grad +/// ms <- rho * ms_{t-1} + (1-rho) * grad * grad +/// mom <- momentum * mom_{t-1} + lr * grad / sqrt(ms - mg * mg + epsilon) +/// var <- var - mom +/// +/// Arguments: +/// * scope: A Scope object +/// * var: Should be from a Variable(). +/// * mg: Should be from a Variable(). +/// * ms: Should be from a Variable(). +/// * mom: Should be from a Variable(). +/// * lr: Scaling factor. Must be a scalar. +/// * rho: Decay rate. Must be a scalar. +/// * epsilon: Ridge term. Must be a scalar. +/// * grad: The gradient. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If `True`, updating of the var, mg, ms, and mom tensors is +/// protected by a lock; otherwise the behavior is undefined, but may exhibit less +/// contention. +/// +/// Returns: +/// * `Output`: Same as "var". +class ApplyCenteredRMSProp { + public: + /// Optional attribute setters for ApplyCenteredRMSProp + struct Attrs { + /// If `True`, updating of the var, mg, ms, and mom tensors is + /// protected by a lock; otherwise the behavior is undefined, but may exhibit less + /// contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = false; + }; + ApplyCenteredRMSProp(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input mg, ::tensorflow::Input ms, + ::tensorflow::Input mom, ::tensorflow::Input lr, + ::tensorflow::Input rho, ::tensorflow::Input momentum, + ::tensorflow::Input epsilon, ::tensorflow::Input grad); + ApplyCenteredRMSProp(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input mg, ::tensorflow::Input ms, + ::tensorflow::Input mom, ::tensorflow::Input lr, + ::tensorflow::Input rho, ::tensorflow::Input momentum, + ::tensorflow::Input epsilon, ::tensorflow::Input grad, + const ApplyCenteredRMSProp::Attrs& attrs); + operator ::tensorflow::Output() const { return out; } + operator ::tensorflow::Input() const { return out; } + ::tensorflow::Node* node() const { return out.node(); } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; + ::tensorflow::Output out; +}; + +/// Update '*var' according to the Ftrl-proximal scheme. +/// +/// accum_new = accum + grad * grad +/// linear += grad + (accum_new^(-lr_power) - accum^(-lr_power)) / lr * var +/// quadratic = 1.0 / (accum_new^(lr_power) * lr) + 2 * l2 +/// var = (sign(linear) * l1 - linear) / quadratic if |linear| > l1 else 0.0 +/// accum = accum_new +/// +/// Arguments: +/// * scope: A Scope object +/// * var: Should be from a Variable(). +/// * accum: Should be from a Variable(). +/// * linear: Should be from a Variable(). +/// * grad: The gradient. +/// * lr: Scaling factor. Must be a scalar. +/// * l1: L1 regulariation. Must be a scalar. +/// * l2: L2 regulariation. Must be a scalar. +/// * lr_power: Scaling factor. Must be a scalar. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If `True`, updating of the var and accum tensors will be protected +/// by a lock; otherwise the behavior is undefined, but may exhibit less +/// contention. +/// +/// Returns: +/// * `Output`: Same as "var". +class ApplyFtrl { + public: + /// Optional attribute setters for ApplyFtrl + struct Attrs { + /// If `True`, updating of the var and accum tensors will be protected + /// by a lock; otherwise the behavior is undefined, but may exhibit less + /// contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = false; + }; + ApplyFtrl(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input accum, ::tensorflow::Input linear, + ::tensorflow::Input grad, ::tensorflow::Input lr, ::tensorflow::Input + l1, ::tensorflow::Input l2, ::tensorflow::Input lr_power); + ApplyFtrl(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input accum, ::tensorflow::Input linear, + ::tensorflow::Input grad, ::tensorflow::Input lr, ::tensorflow::Input + l1, ::tensorflow::Input l2, ::tensorflow::Input lr_power, const + ApplyFtrl::Attrs& attrs); + operator ::tensorflow::Output() const { return out; } + operator ::tensorflow::Input() const { return out; } + ::tensorflow::Node* node() const { return out.node(); } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; + ::tensorflow::Output out; +}; + +/// Update '*var' according to the Ftrl-proximal scheme. +/// +/// grad_with_shrinkage = grad + 2 * l2_shrinkage * var +/// accum_new = accum + grad_with_shrinkage * grad_with_shrinkage +/// linear += grad_with_shrinkage + +/// (accum_new^(-lr_power) - accum^(-lr_power)) / lr * var +/// quadratic = 1.0 / (accum_new^(lr_power) * lr) + 2 * l2 +/// var = (sign(linear) * l1 - linear) / quadratic if |linear| > l1 else 0.0 +/// accum = accum_new +/// +/// Arguments: +/// * scope: A Scope object +/// * var: Should be from a Variable(). +/// * accum: Should be from a Variable(). +/// * linear: Should be from a Variable(). +/// * grad: The gradient. +/// * lr: Scaling factor. Must be a scalar. +/// * l1: L1 regulariation. Must be a scalar. +/// * l2: L2 shrinkage regulariation. Must be a scalar. +/// * lr_power: Scaling factor. Must be a scalar. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If `True`, updating of the var and accum tensors will be protected +/// by a lock; otherwise the behavior is undefined, but may exhibit less +/// contention. +/// +/// Returns: +/// * `Output`: Same as "var". +class ApplyFtrlV2 { + public: + /// Optional attribute setters for ApplyFtrlV2 + struct Attrs { + /// If `True`, updating of the var and accum tensors will be protected + /// by a lock; otherwise the behavior is undefined, but may exhibit less + /// contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = false; + }; + ApplyFtrlV2(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input accum, ::tensorflow::Input linear, + ::tensorflow::Input grad, ::tensorflow::Input lr, + ::tensorflow::Input l1, ::tensorflow::Input l2, ::tensorflow::Input + l2_shrinkage, ::tensorflow::Input lr_power); + ApplyFtrlV2(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input accum, ::tensorflow::Input linear, + ::tensorflow::Input grad, ::tensorflow::Input lr, + ::tensorflow::Input l1, ::tensorflow::Input l2, ::tensorflow::Input + l2_shrinkage, ::tensorflow::Input lr_power, const + ApplyFtrlV2::Attrs& attrs); + operator ::tensorflow::Output() const { return out; } + operator ::tensorflow::Input() const { return out; } + ::tensorflow::Node* node() const { return out.node(); } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; + ::tensorflow::Output out; +}; + +/// Update '*var' by subtracting 'alpha' * 'delta' from it. +/// +/// Arguments: +/// * scope: A Scope object +/// * var: Should be from a Variable(). +/// * alpha: Scaling factor. Must be a scalar. +/// * delta: The change. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If `True`, the subtraction will be protected by a lock; +/// otherwise the behavior is undefined, but may exhibit less contention. +/// +/// Returns: +/// * `Output`: Same as "var". +class ApplyGradientDescent { + public: + /// Optional attribute setters for ApplyGradientDescent + struct Attrs { + /// If `True`, the subtraction will be protected by a lock; + /// otherwise the behavior is undefined, but may exhibit less contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = false; + }; + ApplyGradientDescent(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input alpha, ::tensorflow::Input delta); + ApplyGradientDescent(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input alpha, ::tensorflow::Input delta, + const ApplyGradientDescent::Attrs& attrs); + operator ::tensorflow::Output() const { return out; } + operator ::tensorflow::Input() const { return out; } + ::tensorflow::Node* node() const { return out.node(); } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; + ::tensorflow::Output out; +}; + +/// Update '*var' according to the momentum scheme. Set use_nesterov = True if you +/// +/// want to use Nesterov momentum. +/// +/// accum = accum * momentum + grad +/// var -= lr * accum +/// +/// Arguments: +/// * scope: A Scope object +/// * var: Should be from a Variable(). +/// * accum: Should be from a Variable(). +/// * lr: Scaling factor. Must be a scalar. +/// * grad: The gradient. +/// * momentum: Momentum. Must be a scalar. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If `True`, updating of the var and accum tensors will be protected +/// by a lock; otherwise the behavior is undefined, but may exhibit less +/// contention. +/// * use_nesterov: If `True`, the tensor passed to compute grad will be +/// var - lr * momentum * accum, so in the end, the var you get is actually +/// var - lr * momentum * accum. +/// +/// Returns: +/// * `Output`: Same as "var". +class ApplyMomentum { + public: + /// Optional attribute setters for ApplyMomentum + struct Attrs { + /// If `True`, updating of the var and accum tensors will be protected + /// by a lock; otherwise the behavior is undefined, but may exhibit less + /// contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + /// If `True`, the tensor passed to compute grad will be + /// var - lr * momentum * accum, so in the end, the var you get is actually + /// var - lr * momentum * accum. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseNesterov(bool x) { + Attrs ret = *this; + ret.use_nesterov_ = x; + return ret; + } + + bool use_locking_ = false; + bool use_nesterov_ = false; + }; + ApplyMomentum(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input accum, ::tensorflow::Input lr, + ::tensorflow::Input grad, ::tensorflow::Input momentum); + ApplyMomentum(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input accum, ::tensorflow::Input lr, + ::tensorflow::Input grad, ::tensorflow::Input momentum, const + ApplyMomentum::Attrs& attrs); + operator ::tensorflow::Output() const { return out; } + operator ::tensorflow::Input() const { return out; } + ::tensorflow::Node* node() const { return out.node(); } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + static Attrs UseNesterov(bool x) { + return Attrs().UseNesterov(x); + } + + Operation operation; + ::tensorflow::Output out; +}; + +/// Update '*var' according to the AddSign update. +/// +/// m_t <- beta1 * m_{t-1} + (1 - beta1) * g +/// update <- exp(logbase * sign_decay * sign(g) * sign(m_t)) * g +/// variable <- variable - lr_t * update +/// +/// Arguments: +/// * scope: A Scope object +/// * var: Should be from a Variable(). +/// * m: Should be from a Variable(). +/// * lr: Scaling factor. Must be a scalar. +/// * logbase: Must be a scalar. +/// * sign_decay: Must be a scalar. +/// * beta: Must be a scalar. +/// * grad: The gradient. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If `True`, updating of the var and m tensors is +/// protected by a lock; otherwise the behavior is undefined, but may exhibit less +/// contention. +/// +/// Returns: +/// * `Output`: Same as "var". +class ApplyPowerSign { + public: + /// Optional attribute setters for ApplyPowerSign + struct Attrs { + /// If `True`, updating of the var and m tensors is + /// protected by a lock; otherwise the behavior is undefined, but may exhibit less + /// contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = false; + }; + ApplyPowerSign(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input m, ::tensorflow::Input lr, + ::tensorflow::Input logbase, ::tensorflow::Input sign_decay, + ::tensorflow::Input beta, ::tensorflow::Input grad); + ApplyPowerSign(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input m, ::tensorflow::Input lr, + ::tensorflow::Input logbase, ::tensorflow::Input sign_decay, + ::tensorflow::Input beta, ::tensorflow::Input grad, const + ApplyPowerSign::Attrs& attrs); + operator ::tensorflow::Output() const { return out; } + operator ::tensorflow::Input() const { return out; } + ::tensorflow::Node* node() const { return out.node(); } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; + ::tensorflow::Output out; +}; + +/// Update '*var' and '*accum' according to FOBOS with Adagrad learning rate. +/// +/// accum += grad * grad +/// prox_v = var - lr * grad * (1 / sqrt(accum)) +/// var = sign(prox_v)/(1+lr*l2) * max{|prox_v|-lr*l1,0} +/// +/// Arguments: +/// * scope: A Scope object +/// * var: Should be from a Variable(). +/// * accum: Should be from a Variable(). +/// * lr: Scaling factor. Must be a scalar. +/// * l1: L1 regularization. Must be a scalar. +/// * l2: L2 regularization. Must be a scalar. +/// * grad: The gradient. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If True, updating of the var and accum tensors will be protected by +/// a lock; otherwise the behavior is undefined, but may exhibit less contention. +/// +/// Returns: +/// * `Output`: Same as "var". +class ApplyProximalAdagrad { + public: + /// Optional attribute setters for ApplyProximalAdagrad + struct Attrs { + /// If True, updating of the var and accum tensors will be protected by + /// a lock; otherwise the behavior is undefined, but may exhibit less contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = false; + }; + ApplyProximalAdagrad(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input accum, ::tensorflow::Input lr, + ::tensorflow::Input l1, ::tensorflow::Input l2, + ::tensorflow::Input grad); + ApplyProximalAdagrad(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input accum, ::tensorflow::Input lr, + ::tensorflow::Input l1, ::tensorflow::Input l2, + ::tensorflow::Input grad, const + ApplyProximalAdagrad::Attrs& attrs); + operator ::tensorflow::Output() const { return out; } + operator ::tensorflow::Input() const { return out; } + ::tensorflow::Node* node() const { return out.node(); } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; + ::tensorflow::Output out; +}; + +/// Update '*var' as FOBOS algorithm with fixed learning rate. +/// +/// prox_v = var - alpha * delta +/// var = sign(prox_v)/(1+alpha*l2) * max{|prox_v|-alpha*l1,0} +/// +/// Arguments: +/// * scope: A Scope object +/// * var: Should be from a Variable(). +/// * alpha: Scaling factor. Must be a scalar. +/// * l1: L1 regularization. Must be a scalar. +/// * l2: L2 regularization. Must be a scalar. +/// * delta: The change. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If True, the subtraction will be protected by a lock; +/// otherwise the behavior is undefined, but may exhibit less contention. +/// +/// Returns: +/// * `Output`: Same as "var". +class ApplyProximalGradientDescent { + public: + /// Optional attribute setters for ApplyProximalGradientDescent + struct Attrs { + /// If True, the subtraction will be protected by a lock; + /// otherwise the behavior is undefined, but may exhibit less contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = false; + }; + ApplyProximalGradientDescent(const ::tensorflow::Scope& scope, + ::tensorflow::Input var, ::tensorflow::Input + alpha, ::tensorflow::Input l1, ::tensorflow::Input + l2, ::tensorflow::Input delta); + ApplyProximalGradientDescent(const ::tensorflow::Scope& scope, + ::tensorflow::Input var, ::tensorflow::Input + alpha, ::tensorflow::Input l1, ::tensorflow::Input + l2, ::tensorflow::Input delta, const + ApplyProximalGradientDescent::Attrs& attrs); + operator ::tensorflow::Output() const { return out; } + operator ::tensorflow::Input() const { return out; } + ::tensorflow::Node* node() const { return out.node(); } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; + ::tensorflow::Output out; +}; + +/// Update '*var' according to the RMSProp algorithm. +/// +/// Note that in dense implementation of this algorithm, ms and mom will +/// update even if the grad is zero, but in this sparse implementation, ms +/// and mom will not update in iterations during which the grad is zero. +/// +/// mean_square = decay * mean_square + (1-decay) * gradient ** 2 +/// Delta = learning_rate * gradient / sqrt(mean_square + epsilon) +/// +/// ms <- rho * ms_{t-1} + (1-rho) * grad * grad +/// mom <- momentum * mom_{t-1} + lr * grad / sqrt(ms + epsilon) +/// var <- var - mom +/// +/// Arguments: +/// * scope: A Scope object +/// * var: Should be from a Variable(). +/// * ms: Should be from a Variable(). +/// * mom: Should be from a Variable(). +/// * lr: Scaling factor. Must be a scalar. +/// * rho: Decay rate. Must be a scalar. +/// * epsilon: Ridge term. Must be a scalar. +/// * grad: The gradient. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If `True`, updating of the var, ms, and mom tensors is protected +/// by a lock; otherwise the behavior is undefined, but may exhibit less +/// contention. +/// +/// Returns: +/// * `Output`: Same as "var". +class ApplyRMSProp { + public: + /// Optional attribute setters for ApplyRMSProp + struct Attrs { + /// If `True`, updating of the var, ms, and mom tensors is protected + /// by a lock; otherwise the behavior is undefined, but may exhibit less + /// contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = false; + }; + ApplyRMSProp(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input ms, ::tensorflow::Input mom, + ::tensorflow::Input lr, ::tensorflow::Input rho, + ::tensorflow::Input momentum, ::tensorflow::Input epsilon, + ::tensorflow::Input grad); + ApplyRMSProp(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input ms, ::tensorflow::Input mom, + ::tensorflow::Input lr, ::tensorflow::Input rho, + ::tensorflow::Input momentum, ::tensorflow::Input epsilon, + ::tensorflow::Input grad, const ApplyRMSProp::Attrs& attrs); + operator ::tensorflow::Output() const { return out; } + operator ::tensorflow::Input() const { return out; } + ::tensorflow::Node* node() const { return out.node(); } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; + ::tensorflow::Output out; +}; + +/// Update '*var' according to the adadelta scheme. +/// +/// accum = rho() * accum + (1 - rho()) * grad.square(); +/// update = (update_accum + epsilon).sqrt() * (accum + epsilon()).rsqrt() * grad; +/// update_accum = rho() * update_accum + (1 - rho()) * update.square(); +/// var -= update; +/// +/// Arguments: +/// * scope: A Scope object +/// * var: Should be from a Variable(). +/// * accum: Should be from a Variable(). +/// * accum_update: Should be from a Variable(). +/// * lr: Scaling factor. Must be a scalar. +/// * rho: Decay factor. Must be a scalar. +/// * epsilon: Constant factor. Must be a scalar. +/// * grad: The gradient. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If True, updating of the var, accum and update_accum tensors will be protected by +/// a lock; otherwise the behavior is undefined, but may exhibit less contention. +/// +/// Returns: +/// * the created `Operation` +class ResourceApplyAdadelta { + public: + /// Optional attribute setters for ResourceApplyAdadelta + struct Attrs { + /// If True, updating of the var, accum and update_accum tensors will be protected by + /// a lock; otherwise the behavior is undefined, but may exhibit less contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = false; + }; + ResourceApplyAdadelta(const ::tensorflow::Scope& scope, ::tensorflow::Input + var, ::tensorflow::Input accum, ::tensorflow::Input + accum_update, ::tensorflow::Input lr, ::tensorflow::Input + rho, ::tensorflow::Input epsilon, ::tensorflow::Input + grad); + ResourceApplyAdadelta(const ::tensorflow::Scope& scope, ::tensorflow::Input + var, ::tensorflow::Input accum, ::tensorflow::Input + accum_update, ::tensorflow::Input lr, ::tensorflow::Input + rho, ::tensorflow::Input epsilon, ::tensorflow::Input + grad, const ResourceApplyAdadelta::Attrs& attrs); + operator ::tensorflow::Operation() const { return operation; } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; +}; + +/// Update '*var' according to the adagrad scheme. +/// +/// accum += grad * grad +/// var -= lr * grad * (1 / sqrt(accum)) +/// +/// Arguments: +/// * scope: A Scope object +/// * var: Should be from a Variable(). +/// * accum: Should be from a Variable(). +/// * lr: Scaling factor. Must be a scalar. +/// * grad: The gradient. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If `True`, updating of the var and accum tensors will be protected +/// by a lock; otherwise the behavior is undefined, but may exhibit less +/// contention. +/// +/// Returns: +/// * the created `Operation` +class ResourceApplyAdagrad { + public: + /// Optional attribute setters for ResourceApplyAdagrad + struct Attrs { + /// If `True`, updating of the var and accum tensors will be protected + /// by a lock; otherwise the behavior is undefined, but may exhibit less + /// contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + /// Defaults to true + TF_MUST_USE_RESULT Attrs UpdateSlots(bool x) { + Attrs ret = *this; + ret.update_slots_ = x; + return ret; + } + + bool use_locking_ = false; + bool update_slots_ = true; + }; + ResourceApplyAdagrad(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input accum, ::tensorflow::Input lr, + ::tensorflow::Input grad); + ResourceApplyAdagrad(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input accum, ::tensorflow::Input lr, + ::tensorflow::Input grad, const + ResourceApplyAdagrad::Attrs& attrs); + operator ::tensorflow::Operation() const { return operation; } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + static Attrs UpdateSlots(bool x) { + return Attrs().UpdateSlots(x); + } + + Operation operation; +}; + +/// Update '*var' according to the proximal adagrad scheme. +/// +/// Arguments: +/// * scope: A Scope object +/// * var: Should be from a Variable(). +/// * gradient_accumulator: Should be from a Variable(). +/// * gradient_squared_accumulator: Should be from a Variable(). +/// * grad: The gradient. +/// * lr: Scaling factor. Must be a scalar. +/// * l1: L1 regularization. Must be a scalar. +/// * l2: L2 regularization. Must be a scalar. +/// * global_step: Training step number. Must be a scalar. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If True, updating of the var and accum tensors will be protected by +/// a lock; otherwise the behavior is undefined, but may exhibit less contention. +/// +/// Returns: +/// * the created `Operation` +class ResourceApplyAdagradDA { + public: + /// Optional attribute setters for ResourceApplyAdagradDA + struct Attrs { + /// If True, updating of the var and accum tensors will be protected by + /// a lock; otherwise the behavior is undefined, but may exhibit less contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = false; + }; + ResourceApplyAdagradDA(const ::tensorflow::Scope& scope, ::tensorflow::Input + var, ::tensorflow::Input gradient_accumulator, + ::tensorflow::Input gradient_squared_accumulator, + ::tensorflow::Input grad, ::tensorflow::Input lr, + ::tensorflow::Input l1, ::tensorflow::Input l2, + ::tensorflow::Input global_step); + ResourceApplyAdagradDA(const ::tensorflow::Scope& scope, ::tensorflow::Input + var, ::tensorflow::Input gradient_accumulator, + ::tensorflow::Input gradient_squared_accumulator, + ::tensorflow::Input grad, ::tensorflow::Input lr, + ::tensorflow::Input l1, ::tensorflow::Input l2, + ::tensorflow::Input global_step, const + ResourceApplyAdagradDA::Attrs& attrs); + operator ::tensorflow::Operation() const { return operation; } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; +}; + +/// Update '*var' according to the Adam algorithm. +/// +/// $$lr_t := \text{learning\_rate} * \sqrt{1 - beta_2^t} / (1 - beta_1^t)$$ +/// $$m_t := beta_1 * m_{t-1} + (1 - beta_1) * g$$ +/// $$v_t := beta_2 * v_{t-1} + (1 - beta_2) * g * g$$ +/// $$variable := variable - lr_t * m_t / (\sqrt{v_t} + \epsilon)$$ +/// +/// Arguments: +/// * scope: A Scope object +/// * var: Should be from a Variable(). +/// * m: Should be from a Variable(). +/// * v: Should be from a Variable(). +/// * beta1_power: Must be a scalar. +/// * beta2_power: Must be a scalar. +/// * lr: Scaling factor. Must be a scalar. +/// * beta1: Momentum factor. Must be a scalar. +/// * beta2: Momentum factor. Must be a scalar. +/// * epsilon: Ridge term. Must be a scalar. +/// * grad: The gradient. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If `True`, updating of the var, m, and v tensors will be protected +/// by a lock; otherwise the behavior is undefined, but may exhibit less +/// contention. +/// * use_nesterov: If `True`, uses the nesterov update. +/// +/// Returns: +/// * the created `Operation` +class ResourceApplyAdam { + public: + /// Optional attribute setters for ResourceApplyAdam + struct Attrs { + /// If `True`, updating of the var, m, and v tensors will be protected + /// by a lock; otherwise the behavior is undefined, but may exhibit less + /// contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + /// If `True`, uses the nesterov update. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseNesterov(bool x) { + Attrs ret = *this; + ret.use_nesterov_ = x; + return ret; + } + + bool use_locking_ = false; + bool use_nesterov_ = false; + }; + ResourceApplyAdam(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input m, ::tensorflow::Input v, + ::tensorflow::Input beta1_power, ::tensorflow::Input + beta2_power, ::tensorflow::Input lr, ::tensorflow::Input + beta1, ::tensorflow::Input beta2, ::tensorflow::Input + epsilon, ::tensorflow::Input grad); + ResourceApplyAdam(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input m, ::tensorflow::Input v, + ::tensorflow::Input beta1_power, ::tensorflow::Input + beta2_power, ::tensorflow::Input lr, ::tensorflow::Input + beta1, ::tensorflow::Input beta2, ::tensorflow::Input + epsilon, ::tensorflow::Input grad, const + ResourceApplyAdam::Attrs& attrs); + operator ::tensorflow::Operation() const { return operation; } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + static Attrs UseNesterov(bool x) { + return Attrs().UseNesterov(x); + } + + Operation operation; +}; + +/// Update '*var' according to the AddSign update. +/// +/// m_t <- beta1 * m_{t-1} + (1 - beta1) * g +/// update <- (alpha + sign_decay * sign(g) *sign(m)) * g +/// variable <- variable - lr_t * update +/// +/// Arguments: +/// * scope: A Scope object +/// * var: Should be from a Variable(). +/// * m: Should be from a Variable(). +/// * lr: Scaling factor. Must be a scalar. +/// * alpha: Must be a scalar. +/// * sign_decay: Must be a scalar. +/// * beta: Must be a scalar. +/// * grad: The gradient. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If `True`, updating of the var and m tensors is +/// protected by a lock; otherwise the behavior is undefined, but may exhibit less +/// contention. +/// +/// Returns: +/// * the created `Operation` +class ResourceApplyAddSign { + public: + /// Optional attribute setters for ResourceApplyAddSign + struct Attrs { + /// If `True`, updating of the var and m tensors is + /// protected by a lock; otherwise the behavior is undefined, but may exhibit less + /// contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = false; + }; + ResourceApplyAddSign(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input m, ::tensorflow::Input lr, + ::tensorflow::Input alpha, ::tensorflow::Input sign_decay, + ::tensorflow::Input beta, ::tensorflow::Input grad); + ResourceApplyAddSign(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input m, ::tensorflow::Input lr, + ::tensorflow::Input alpha, ::tensorflow::Input sign_decay, + ::tensorflow::Input beta, ::tensorflow::Input grad, const + ResourceApplyAddSign::Attrs& attrs); + operator ::tensorflow::Operation() const { return operation; } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; +}; + +/// Update '*var' according to the centered RMSProp algorithm. +/// +/// The centered RMSProp algorithm uses an estimate of the centered second moment +/// (i.e., the variance) for normalization, as opposed to regular RMSProp, which +/// uses the (uncentered) second moment. This often helps with training, but is +/// slightly more expensive in terms of computation and memory. +/// +/// Note that in dense implementation of this algorithm, mg, ms, and mom will +/// update even if the grad is zero, but in this sparse implementation, mg, ms, +/// and mom will not update in iterations during which the grad is zero. +/// +/// mean_square = decay * mean_square + (1-decay) * gradient ** 2 +/// mean_grad = decay * mean_grad + (1-decay) * gradient +/// +/// Delta = learning_rate * gradient / sqrt(mean_square + epsilon - mean_grad ** 2) +/// +/// mg <- rho * mg_{t-1} + (1-rho) * grad +/// ms <- rho * ms_{t-1} + (1-rho) * grad * grad +/// mom <- momentum * mom_{t-1} + lr * grad / sqrt(ms - mg * mg + epsilon) +/// var <- var - mom +/// +/// Arguments: +/// * scope: A Scope object +/// * var: Should be from a Variable(). +/// * mg: Should be from a Variable(). +/// * ms: Should be from a Variable(). +/// * mom: Should be from a Variable(). +/// * lr: Scaling factor. Must be a scalar. +/// * rho: Decay rate. Must be a scalar. +/// * epsilon: Ridge term. Must be a scalar. +/// * grad: The gradient. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If `True`, updating of the var, mg, ms, and mom tensors is +/// protected by a lock; otherwise the behavior is undefined, but may exhibit less +/// contention. +/// +/// Returns: +/// * the created `Operation` +class ResourceApplyCenteredRMSProp { + public: + /// Optional attribute setters for ResourceApplyCenteredRMSProp + struct Attrs { + /// If `True`, updating of the var, mg, ms, and mom tensors is + /// protected by a lock; otherwise the behavior is undefined, but may exhibit less + /// contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = false; + }; + ResourceApplyCenteredRMSProp(const ::tensorflow::Scope& scope, + ::tensorflow::Input var, ::tensorflow::Input mg, + ::tensorflow::Input ms, ::tensorflow::Input mom, + ::tensorflow::Input lr, ::tensorflow::Input rho, + ::tensorflow::Input momentum, ::tensorflow::Input + epsilon, ::tensorflow::Input grad); + ResourceApplyCenteredRMSProp(const ::tensorflow::Scope& scope, + ::tensorflow::Input var, ::tensorflow::Input mg, + ::tensorflow::Input ms, ::tensorflow::Input mom, + ::tensorflow::Input lr, ::tensorflow::Input rho, + ::tensorflow::Input momentum, ::tensorflow::Input + epsilon, ::tensorflow::Input grad, const + ResourceApplyCenteredRMSProp::Attrs& attrs); + operator ::tensorflow::Operation() const { return operation; } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; +}; + +/// Update '*var' according to the Ftrl-proximal scheme. +/// +/// accum_new = accum + grad * grad +/// linear += grad - (accum_new^(-lr_power) - accum^(-lr_power)) / lr * var +/// quadratic = 1.0 / (accum_new^(lr_power) * lr) + 2 * l2 +/// var = (sign(linear) * l1 - linear) / quadratic if |linear| > l1 else 0.0 +/// accum = accum_new +/// +/// Arguments: +/// * scope: A Scope object +/// * var: Should be from a Variable(). +/// * accum: Should be from a Variable(). +/// * linear: Should be from a Variable(). +/// * grad: The gradient. +/// * lr: Scaling factor. Must be a scalar. +/// * l1: L1 regulariation. Must be a scalar. +/// * l2: L2 regulariation. Must be a scalar. +/// * lr_power: Scaling factor. Must be a scalar. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If `True`, updating of the var and accum tensors will be protected +/// by a lock; otherwise the behavior is undefined, but may exhibit less +/// contention. +/// +/// Returns: +/// * the created `Operation` +class ResourceApplyFtrl { + public: + /// Optional attribute setters for ResourceApplyFtrl + struct Attrs { + /// If `True`, updating of the var and accum tensors will be protected + /// by a lock; otherwise the behavior is undefined, but may exhibit less + /// contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = false; + }; + ResourceApplyFtrl(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input accum, ::tensorflow::Input linear, + ::tensorflow::Input grad, ::tensorflow::Input lr, + ::tensorflow::Input l1, ::tensorflow::Input l2, + ::tensorflow::Input lr_power); + ResourceApplyFtrl(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input accum, ::tensorflow::Input linear, + ::tensorflow::Input grad, ::tensorflow::Input lr, + ::tensorflow::Input l1, ::tensorflow::Input l2, + ::tensorflow::Input lr_power, const ResourceApplyFtrl::Attrs& + attrs); + operator ::tensorflow::Operation() const { return operation; } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; +}; + +/// Update '*var' according to the Ftrl-proximal scheme. +/// +/// grad_with_shrinkage = grad + 2 * l2_shrinkage * var +/// accum_new = accum + grad_with_shrinkage * grad_with_shrinkage +/// linear += grad_with_shrinkage + +/// (accum_new^(-lr_power) - accum^(-lr_power)) / lr * var +/// quadratic = 1.0 / (accum_new^(lr_power) * lr) + 2 * l2 +/// var = (sign(linear) * l1 - linear) / quadratic if |linear| > l1 else 0.0 +/// accum = accum_new +/// +/// Arguments: +/// * scope: A Scope object +/// * var: Should be from a Variable(). +/// * accum: Should be from a Variable(). +/// * linear: Should be from a Variable(). +/// * grad: The gradient. +/// * lr: Scaling factor. Must be a scalar. +/// * l1: L1 regulariation. Must be a scalar. +/// * l2: L2 shrinkage regulariation. Must be a scalar. +/// * lr_power: Scaling factor. Must be a scalar. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If `True`, updating of the var and accum tensors will be protected +/// by a lock; otherwise the behavior is undefined, but may exhibit less +/// contention. +/// +/// Returns: +/// * the created `Operation` +class ResourceApplyFtrlV2 { + public: + /// Optional attribute setters for ResourceApplyFtrlV2 + struct Attrs { + /// If `True`, updating of the var and accum tensors will be protected + /// by a lock; otherwise the behavior is undefined, but may exhibit less + /// contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = false; + }; + ResourceApplyFtrlV2(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input accum, ::tensorflow::Input linear, + ::tensorflow::Input grad, ::tensorflow::Input lr, + ::tensorflow::Input l1, ::tensorflow::Input l2, + ::tensorflow::Input l2_shrinkage, ::tensorflow::Input + lr_power); + ResourceApplyFtrlV2(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input accum, ::tensorflow::Input linear, + ::tensorflow::Input grad, ::tensorflow::Input lr, + ::tensorflow::Input l1, ::tensorflow::Input l2, + ::tensorflow::Input l2_shrinkage, ::tensorflow::Input + lr_power, const ResourceApplyFtrlV2::Attrs& attrs); + operator ::tensorflow::Operation() const { return operation; } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; +}; + +/// Update '*var' by subtracting 'alpha' * 'delta' from it. +/// +/// Arguments: +/// * scope: A Scope object +/// * var: Should be from a Variable(). +/// * alpha: Scaling factor. Must be a scalar. +/// * delta: The change. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If `True`, the subtraction will be protected by a lock; +/// otherwise the behavior is undefined, but may exhibit less contention. +/// +/// Returns: +/// * the created `Operation` +class ResourceApplyGradientDescent { + public: + /// Optional attribute setters for ResourceApplyGradientDescent + struct Attrs { + /// If `True`, the subtraction will be protected by a lock; + /// otherwise the behavior is undefined, but may exhibit less contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = false; + }; + ResourceApplyGradientDescent(const ::tensorflow::Scope& scope, + ::tensorflow::Input var, ::tensorflow::Input + alpha, ::tensorflow::Input delta); + ResourceApplyGradientDescent(const ::tensorflow::Scope& scope, + ::tensorflow::Input var, ::tensorflow::Input + alpha, ::tensorflow::Input delta, const + ResourceApplyGradientDescent::Attrs& attrs); + operator ::tensorflow::Operation() const { return operation; } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; +}; + +/// Update '*var' according to the momentum scheme. Set use_nesterov = True if you +/// +/// want to use Nesterov momentum. +/// +/// accum = accum * momentum + grad +/// var -= lr * accum +/// +/// Arguments: +/// * scope: A Scope object +/// * var: Should be from a Variable(). +/// * accum: Should be from a Variable(). +/// * lr: Scaling factor. Must be a scalar. +/// * grad: The gradient. +/// * momentum: Momentum. Must be a scalar. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If `True`, updating of the var and accum tensors will be protected +/// by a lock; otherwise the behavior is undefined, but may exhibit less +/// contention. +/// * use_nesterov: If `True`, the tensor passed to compute grad will be +/// var - lr * momentum * accum, so in the end, the var you get is actually +/// var - lr * momentum * accum. +/// +/// Returns: +/// * the created `Operation` +class ResourceApplyMomentum { + public: + /// Optional attribute setters for ResourceApplyMomentum + struct Attrs { + /// If `True`, updating of the var and accum tensors will be protected + /// by a lock; otherwise the behavior is undefined, but may exhibit less + /// contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + /// If `True`, the tensor passed to compute grad will be + /// var - lr * momentum * accum, so in the end, the var you get is actually + /// var - lr * momentum * accum. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseNesterov(bool x) { + Attrs ret = *this; + ret.use_nesterov_ = x; + return ret; + } + + bool use_locking_ = false; + bool use_nesterov_ = false; + }; + ResourceApplyMomentum(const ::tensorflow::Scope& scope, ::tensorflow::Input + var, ::tensorflow::Input accum, ::tensorflow::Input lr, + ::tensorflow::Input grad, ::tensorflow::Input momentum); + ResourceApplyMomentum(const ::tensorflow::Scope& scope, ::tensorflow::Input + var, ::tensorflow::Input accum, ::tensorflow::Input lr, + ::tensorflow::Input grad, ::tensorflow::Input momentum, + const ResourceApplyMomentum::Attrs& attrs); + operator ::tensorflow::Operation() const { return operation; } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + static Attrs UseNesterov(bool x) { + return Attrs().UseNesterov(x); + } + + Operation operation; +}; + +/// Update '*var' according to the AddSign update. +/// +/// m_t <- beta1 * m_{t-1} + (1 - beta1) * g +/// update <- exp(logbase * sign_decay * sign(g) * sign(m_t)) * g +/// variable <- variable - lr_t * update +/// +/// Arguments: +/// * scope: A Scope object +/// * var: Should be from a Variable(). +/// * m: Should be from a Variable(). +/// * lr: Scaling factor. Must be a scalar. +/// * logbase: Must be a scalar. +/// * sign_decay: Must be a scalar. +/// * beta: Must be a scalar. +/// * grad: The gradient. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If `True`, updating of the var and m tensors is +/// protected by a lock; otherwise the behavior is undefined, but may exhibit less +/// contention. +/// +/// Returns: +/// * the created `Operation` +class ResourceApplyPowerSign { + public: + /// Optional attribute setters for ResourceApplyPowerSign + struct Attrs { + /// If `True`, updating of the var and m tensors is + /// protected by a lock; otherwise the behavior is undefined, but may exhibit less + /// contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = false; + }; + ResourceApplyPowerSign(const ::tensorflow::Scope& scope, ::tensorflow::Input + var, ::tensorflow::Input m, ::tensorflow::Input lr, + ::tensorflow::Input logbase, ::tensorflow::Input + sign_decay, ::tensorflow::Input beta, + ::tensorflow::Input grad); + ResourceApplyPowerSign(const ::tensorflow::Scope& scope, ::tensorflow::Input + var, ::tensorflow::Input m, ::tensorflow::Input lr, + ::tensorflow::Input logbase, ::tensorflow::Input + sign_decay, ::tensorflow::Input beta, + ::tensorflow::Input grad, const + ResourceApplyPowerSign::Attrs& attrs); + operator ::tensorflow::Operation() const { return operation; } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; +}; + +/// Update '*var' and '*accum' according to FOBOS with Adagrad learning rate. +/// +/// accum += grad * grad +/// prox_v = var - lr * grad * (1 / sqrt(accum)) +/// var = sign(prox_v)/(1+lr*l2) * max{|prox_v|-lr*l1,0} +/// +/// Arguments: +/// * scope: A Scope object +/// * var: Should be from a Variable(). +/// * accum: Should be from a Variable(). +/// * lr: Scaling factor. Must be a scalar. +/// * l1: L1 regularization. Must be a scalar. +/// * l2: L2 regularization. Must be a scalar. +/// * grad: The gradient. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If True, updating of the var and accum tensors will be protected by +/// a lock; otherwise the behavior is undefined, but may exhibit less contention. +/// +/// Returns: +/// * the created `Operation` +class ResourceApplyProximalAdagrad { + public: + /// Optional attribute setters for ResourceApplyProximalAdagrad + struct Attrs { + /// If True, updating of the var and accum tensors will be protected by + /// a lock; otherwise the behavior is undefined, but may exhibit less contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = false; + }; + ResourceApplyProximalAdagrad(const ::tensorflow::Scope& scope, + ::tensorflow::Input var, ::tensorflow::Input + accum, ::tensorflow::Input lr, ::tensorflow::Input + l1, ::tensorflow::Input l2, ::tensorflow::Input + grad); + ResourceApplyProximalAdagrad(const ::tensorflow::Scope& scope, + ::tensorflow::Input var, ::tensorflow::Input + accum, ::tensorflow::Input lr, ::tensorflow::Input + l1, ::tensorflow::Input l2, ::tensorflow::Input + grad, const ResourceApplyProximalAdagrad::Attrs& + attrs); + operator ::tensorflow::Operation() const { return operation; } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; +}; + +/// Update '*var' as FOBOS algorithm with fixed learning rate. +/// +/// prox_v = var - alpha * delta +/// var = sign(prox_v)/(1+alpha*l2) * max{|prox_v|-alpha*l1,0} +/// +/// Arguments: +/// * scope: A Scope object +/// * var: Should be from a Variable(). +/// * alpha: Scaling factor. Must be a scalar. +/// * l1: L1 regularization. Must be a scalar. +/// * l2: L2 regularization. Must be a scalar. +/// * delta: The change. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If True, the subtraction will be protected by a lock; +/// otherwise the behavior is undefined, but may exhibit less contention. +/// +/// Returns: +/// * the created `Operation` +class ResourceApplyProximalGradientDescent { + public: + /// Optional attribute setters for ResourceApplyProximalGradientDescent + struct Attrs { + /// If True, the subtraction will be protected by a lock; + /// otherwise the behavior is undefined, but may exhibit less contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = false; + }; + ResourceApplyProximalGradientDescent(const ::tensorflow::Scope& scope, + ::tensorflow::Input var, + ::tensorflow::Input alpha, + ::tensorflow::Input l1, + ::tensorflow::Input l2, + ::tensorflow::Input delta); + ResourceApplyProximalGradientDescent(const ::tensorflow::Scope& scope, + ::tensorflow::Input var, + ::tensorflow::Input alpha, + ::tensorflow::Input l1, + ::tensorflow::Input l2, + ::tensorflow::Input delta, const + ResourceApplyProximalGradientDescent::Attrs& + attrs); + operator ::tensorflow::Operation() const { return operation; } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; +}; + +/// Update '*var' according to the RMSProp algorithm. +/// +/// Note that in dense implementation of this algorithm, ms and mom will +/// update even if the grad is zero, but in this sparse implementation, ms +/// and mom will not update in iterations during which the grad is zero. +/// +/// mean_square = decay * mean_square + (1-decay) * gradient ** 2 +/// Delta = learning_rate * gradient / sqrt(mean_square + epsilon) +/// +/// ms <- rho * ms_{t-1} + (1-rho) * grad * grad +/// mom <- momentum * mom_{t-1} + lr * grad / sqrt(ms + epsilon) +/// var <- var - mom +/// +/// Arguments: +/// * scope: A Scope object +/// * var: Should be from a Variable(). +/// * ms: Should be from a Variable(). +/// * mom: Should be from a Variable(). +/// * lr: Scaling factor. Must be a scalar. +/// * rho: Decay rate. Must be a scalar. +/// * epsilon: Ridge term. Must be a scalar. +/// * grad: The gradient. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If `True`, updating of the var, ms, and mom tensors is protected +/// by a lock; otherwise the behavior is undefined, but may exhibit less +/// contention. +/// +/// Returns: +/// * the created `Operation` +class ResourceApplyRMSProp { + public: + /// Optional attribute setters for ResourceApplyRMSProp + struct Attrs { + /// If `True`, updating of the var, ms, and mom tensors is protected + /// by a lock; otherwise the behavior is undefined, but may exhibit less + /// contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = false; + }; + ResourceApplyRMSProp(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input ms, ::tensorflow::Input mom, + ::tensorflow::Input lr, ::tensorflow::Input rho, + ::tensorflow::Input momentum, ::tensorflow::Input epsilon, + ::tensorflow::Input grad); + ResourceApplyRMSProp(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input ms, ::tensorflow::Input mom, + ::tensorflow::Input lr, ::tensorflow::Input rho, + ::tensorflow::Input momentum, ::tensorflow::Input epsilon, + ::tensorflow::Input grad, const + ResourceApplyRMSProp::Attrs& attrs); + operator ::tensorflow::Operation() const { return operation; } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; +}; + +/// var: Should be from a Variable(). +/// +/// Arguments: +/// * scope: A Scope object +/// * accum: Should be from a Variable(). +/// * accum_update: : Should be from a Variable(). +/// * lr: Learning rate. Must be a scalar. +/// * rho: Decay factor. Must be a scalar. +/// * epsilon: Constant factor. Must be a scalar. +/// * grad: The gradient. +/// * indices: A vector of indices into the first dimension of var and accum. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If True, updating of the var and accum tensors will be protected by +/// a lock; otherwise the behavior is undefined, but may exhibit less contention. +/// +/// Returns: +/// * the created `Operation` +class ResourceSparseApplyAdadelta { + public: + /// Optional attribute setters for ResourceSparseApplyAdadelta + struct Attrs { + /// If True, updating of the var and accum tensors will be protected by + /// a lock; otherwise the behavior is undefined, but may exhibit less contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = false; + }; + ResourceSparseApplyAdadelta(const ::tensorflow::Scope& scope, + ::tensorflow::Input var, ::tensorflow::Input accum, + ::tensorflow::Input accum_update, + ::tensorflow::Input lr, ::tensorflow::Input rho, + ::tensorflow::Input epsilon, ::tensorflow::Input + grad, ::tensorflow::Input indices); + ResourceSparseApplyAdadelta(const ::tensorflow::Scope& scope, + ::tensorflow::Input var, ::tensorflow::Input accum, + ::tensorflow::Input accum_update, + ::tensorflow::Input lr, ::tensorflow::Input rho, + ::tensorflow::Input epsilon, ::tensorflow::Input + grad, ::tensorflow::Input indices, const + ResourceSparseApplyAdadelta::Attrs& attrs); + operator ::tensorflow::Operation() const { return operation; } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; +}; + +/// Update relevant entries in '*var' and '*accum' according to the adagrad scheme. +/// +/// That is for rows we have grad for, we update var and accum as follows: +/// accum += grad * grad +/// var -= lr * grad * (1 / sqrt(accum)) +/// +/// Arguments: +/// * scope: A Scope object +/// * var: Should be from a Variable(). +/// * accum: Should be from a Variable(). +/// * lr: Learning rate. Must be a scalar. +/// * grad: The gradient. +/// * indices: A vector of indices into the first dimension of var and accum. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If `True`, updating of the var and accum tensors will be protected +/// by a lock; otherwise the behavior is undefined, but may exhibit less +/// contention. +/// +/// Returns: +/// * the created `Operation` +class ResourceSparseApplyAdagrad { + public: + /// Optional attribute setters for ResourceSparseApplyAdagrad + struct Attrs { + /// If `True`, updating of the var and accum tensors will be protected + /// by a lock; otherwise the behavior is undefined, but may exhibit less + /// contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + /// Defaults to true + TF_MUST_USE_RESULT Attrs UpdateSlots(bool x) { + Attrs ret = *this; + ret.update_slots_ = x; + return ret; + } + + bool use_locking_ = false; + bool update_slots_ = true; + }; + ResourceSparseApplyAdagrad(const ::tensorflow::Scope& scope, + ::tensorflow::Input var, ::tensorflow::Input accum, + ::tensorflow::Input lr, ::tensorflow::Input grad, + ::tensorflow::Input indices); + ResourceSparseApplyAdagrad(const ::tensorflow::Scope& scope, + ::tensorflow::Input var, ::tensorflow::Input accum, + ::tensorflow::Input lr, ::tensorflow::Input grad, + ::tensorflow::Input indices, const + ResourceSparseApplyAdagrad::Attrs& attrs); + operator ::tensorflow::Operation() const { return operation; } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + static Attrs UpdateSlots(bool x) { + return Attrs().UpdateSlots(x); + } + + Operation operation; +}; + +/// Update entries in '*var' and '*accum' according to the proximal adagrad scheme. +/// +/// Arguments: +/// * scope: A Scope object +/// * var: Should be from a Variable(). +/// * gradient_accumulator: Should be from a Variable(). +/// * gradient_squared_accumulator: Should be from a Variable(). +/// * grad: The gradient. +/// * indices: A vector of indices into the first dimension of var and accum. +/// * lr: Learning rate. Must be a scalar. +/// * l1: L1 regularization. Must be a scalar. +/// * l2: L2 regularization. Must be a scalar. +/// * global_step: Training step number. Must be a scalar. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If True, updating of the var and accum tensors will be protected by +/// a lock; otherwise the behavior is undefined, but may exhibit less contention. +/// +/// Returns: +/// * the created `Operation` +class ResourceSparseApplyAdagradDA { + public: + /// Optional attribute setters for ResourceSparseApplyAdagradDA + struct Attrs { + /// If True, updating of the var and accum tensors will be protected by + /// a lock; otherwise the behavior is undefined, but may exhibit less contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = false; + }; + ResourceSparseApplyAdagradDA(const ::tensorflow::Scope& scope, + ::tensorflow::Input var, ::tensorflow::Input + gradient_accumulator, ::tensorflow::Input + gradient_squared_accumulator, ::tensorflow::Input + grad, ::tensorflow::Input indices, + ::tensorflow::Input lr, ::tensorflow::Input l1, + ::tensorflow::Input l2, ::tensorflow::Input + global_step); + ResourceSparseApplyAdagradDA(const ::tensorflow::Scope& scope, + ::tensorflow::Input var, ::tensorflow::Input + gradient_accumulator, ::tensorflow::Input + gradient_squared_accumulator, ::tensorflow::Input + grad, ::tensorflow::Input indices, + ::tensorflow::Input lr, ::tensorflow::Input l1, + ::tensorflow::Input l2, ::tensorflow::Input + global_step, const + ResourceSparseApplyAdagradDA::Attrs& attrs); + operator ::tensorflow::Operation() const { return operation; } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; +}; + +/// Update '*var' according to the centered RMSProp algorithm. +/// +/// The centered RMSProp algorithm uses an estimate of the centered second moment +/// (i.e., the variance) for normalization, as opposed to regular RMSProp, which +/// uses the (uncentered) second moment. This often helps with training, but is +/// slightly more expensive in terms of computation and memory. +/// +/// Note that in dense implementation of this algorithm, mg, ms, and mom will +/// update even if the grad is zero, but in this sparse implementation, mg, ms, +/// and mom will not update in iterations during which the grad is zero. +/// +/// mean_square = decay * mean_square + (1-decay) * gradient ** 2 +/// mean_grad = decay * mean_grad + (1-decay) * gradient +/// Delta = learning_rate * gradient / sqrt(mean_square + epsilon - mean_grad ** 2) +/// +/// ms <- rho * ms_{t-1} + (1-rho) * grad * grad +/// mom <- momentum * mom_{t-1} + lr * grad / sqrt(ms + epsilon) +/// var <- var - mom +/// +/// Arguments: +/// * scope: A Scope object +/// * var: Should be from a Variable(). +/// * mg: Should be from a Variable(). +/// * ms: Should be from a Variable(). +/// * mom: Should be from a Variable(). +/// * lr: Scaling factor. Must be a scalar. +/// * rho: Decay rate. Must be a scalar. +/// * epsilon: Ridge term. Must be a scalar. +/// * grad: The gradient. +/// * indices: A vector of indices into the first dimension of var, ms and mom. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If `True`, updating of the var, mg, ms, and mom tensors is +/// protected by a lock; otherwise the behavior is undefined, but may exhibit less +/// contention. +/// +/// Returns: +/// * the created `Operation` +class ResourceSparseApplyCenteredRMSProp { + public: + /// Optional attribute setters for ResourceSparseApplyCenteredRMSProp + struct Attrs { + /// If `True`, updating of the var, mg, ms, and mom tensors is + /// protected by a lock; otherwise the behavior is undefined, but may exhibit less + /// contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = false; + }; + ResourceSparseApplyCenteredRMSProp(const ::tensorflow::Scope& scope, + ::tensorflow::Input var, ::tensorflow::Input + mg, ::tensorflow::Input ms, + ::tensorflow::Input mom, ::tensorflow::Input + lr, ::tensorflow::Input rho, + ::tensorflow::Input momentum, + ::tensorflow::Input epsilon, + ::tensorflow::Input grad, + ::tensorflow::Input indices); + ResourceSparseApplyCenteredRMSProp(const ::tensorflow::Scope& scope, + ::tensorflow::Input var, ::tensorflow::Input + mg, ::tensorflow::Input ms, + ::tensorflow::Input mom, ::tensorflow::Input + lr, ::tensorflow::Input rho, + ::tensorflow::Input momentum, + ::tensorflow::Input epsilon, + ::tensorflow::Input grad, + ::tensorflow::Input indices, const + ResourceSparseApplyCenteredRMSProp::Attrs& + attrs); + operator ::tensorflow::Operation() const { return operation; } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; +}; + +/// Update relevant entries in '*var' according to the Ftrl-proximal scheme. +/// +/// That is for rows we have grad for, we update var, accum and linear as follows: +/// accum_new = accum + grad * grad +/// linear += grad + (accum_new^(-lr_power) - accum^(-lr_power)) / lr * var +/// quadratic = 1.0 / (accum_new^(lr_power) * lr) + 2 * l2 +/// var = (sign(linear) * l1 - linear) / quadratic if |linear| > l1 else 0.0 +/// accum = accum_new +/// +/// Arguments: +/// * scope: A Scope object +/// * var: Should be from a Variable(). +/// * accum: Should be from a Variable(). +/// * linear: Should be from a Variable(). +/// * grad: The gradient. +/// * indices: A vector of indices into the first dimension of var and accum. +/// * lr: Scaling factor. Must be a scalar. +/// * l1: L1 regularization. Must be a scalar. +/// * l2: L2 regularization. Must be a scalar. +/// * lr_power: Scaling factor. Must be a scalar. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If `True`, updating of the var and accum tensors will be protected +/// by a lock; otherwise the behavior is undefined, but may exhibit less +/// contention. +/// +/// Returns: +/// * the created `Operation` +class ResourceSparseApplyFtrl { + public: + /// Optional attribute setters for ResourceSparseApplyFtrl + struct Attrs { + /// If `True`, updating of the var and accum tensors will be protected + /// by a lock; otherwise the behavior is undefined, but may exhibit less + /// contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = false; + }; + ResourceSparseApplyFtrl(const ::tensorflow::Scope& scope, ::tensorflow::Input + var, ::tensorflow::Input accum, ::tensorflow::Input + linear, ::tensorflow::Input grad, ::tensorflow::Input + indices, ::tensorflow::Input lr, ::tensorflow::Input + l1, ::tensorflow::Input l2, ::tensorflow::Input + lr_power); + ResourceSparseApplyFtrl(const ::tensorflow::Scope& scope, ::tensorflow::Input + var, ::tensorflow::Input accum, ::tensorflow::Input + linear, ::tensorflow::Input grad, ::tensorflow::Input + indices, ::tensorflow::Input lr, ::tensorflow::Input + l1, ::tensorflow::Input l2, ::tensorflow::Input + lr_power, const ResourceSparseApplyFtrl::Attrs& attrs); + operator ::tensorflow::Operation() const { return operation; } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; +}; + +/// Update relevant entries in '*var' according to the Ftrl-proximal scheme. +/// +/// That is for rows we have grad for, we update var, accum and linear as follows: +/// grad_with_shrinkage = grad + 2 * l2_shrinkage * var +/// accum_new = accum + grad_with_shrinkage * grad_with_shrinkage +/// linear += grad_with_shrinkage + +/// (accum_new^(-lr_power) - accum^(-lr_power)) / lr * var +/// quadratic = 1.0 / (accum_new^(lr_power) * lr) + 2 * l2 +/// var = (sign(linear) * l1 - linear) / quadratic if |linear| > l1 else 0.0 +/// accum = accum_new +/// +/// Arguments: +/// * scope: A Scope object +/// * var: Should be from a Variable(). +/// * accum: Should be from a Variable(). +/// * linear: Should be from a Variable(). +/// * grad: The gradient. +/// * indices: A vector of indices into the first dimension of var and accum. +/// * lr: Scaling factor. Must be a scalar. +/// * l1: L1 regularization. Must be a scalar. +/// * l2: L2 shrinkage regulariation. Must be a scalar. +/// * lr_power: Scaling factor. Must be a scalar. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If `True`, updating of the var and accum tensors will be protected +/// by a lock; otherwise the behavior is undefined, but may exhibit less +/// contention. +/// +/// Returns: +/// * the created `Operation` +class ResourceSparseApplyFtrlV2 { + public: + /// Optional attribute setters for ResourceSparseApplyFtrlV2 + struct Attrs { + /// If `True`, updating of the var and accum tensors will be protected + /// by a lock; otherwise the behavior is undefined, but may exhibit less + /// contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = false; + }; + ResourceSparseApplyFtrlV2(const ::tensorflow::Scope& scope, ::tensorflow::Input + var, ::tensorflow::Input accum, ::tensorflow::Input + linear, ::tensorflow::Input grad, ::tensorflow::Input + indices, ::tensorflow::Input lr, ::tensorflow::Input + l1, ::tensorflow::Input l2, ::tensorflow::Input + l2_shrinkage, ::tensorflow::Input lr_power); + ResourceSparseApplyFtrlV2(const ::tensorflow::Scope& scope, ::tensorflow::Input + var, ::tensorflow::Input accum, ::tensorflow::Input + linear, ::tensorflow::Input grad, ::tensorflow::Input + indices, ::tensorflow::Input lr, ::tensorflow::Input + l1, ::tensorflow::Input l2, ::tensorflow::Input + l2_shrinkage, ::tensorflow::Input lr_power, const + ResourceSparseApplyFtrlV2::Attrs& attrs); + operator ::tensorflow::Operation() const { return operation; } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; +}; + +/// Update relevant entries in '*var' and '*accum' according to the momentum scheme. +/// +/// Set use_nesterov = True if you want to use Nesterov momentum. +/// +/// That is for rows we have grad for, we update var and accum as follows: +/// +/// accum = accum * momentum + grad +/// var -= lr * accum +/// +/// Arguments: +/// * scope: A Scope object +/// * var: Should be from a Variable(). +/// * accum: Should be from a Variable(). +/// * lr: Learning rate. Must be a scalar. +/// * grad: The gradient. +/// * indices: A vector of indices into the first dimension of var and accum. +/// * momentum: Momentum. Must be a scalar. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If `True`, updating of the var and accum tensors will be protected +/// by a lock; otherwise the behavior is undefined, but may exhibit less +/// contention. +/// * use_nesterov: If `True`, the tensor passed to compute grad will be +/// var - lr * momentum * accum, so in the end, the var you get is actually +/// var - lr * momentum * accum. +/// +/// Returns: +/// * the created `Operation` +class ResourceSparseApplyMomentum { + public: + /// Optional attribute setters for ResourceSparseApplyMomentum + struct Attrs { + /// If `True`, updating of the var and accum tensors will be protected + /// by a lock; otherwise the behavior is undefined, but may exhibit less + /// contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + /// If `True`, the tensor passed to compute grad will be + /// var - lr * momentum * accum, so in the end, the var you get is actually + /// var - lr * momentum * accum. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseNesterov(bool x) { + Attrs ret = *this; + ret.use_nesterov_ = x; + return ret; + } + + bool use_locking_ = false; + bool use_nesterov_ = false; + }; + ResourceSparseApplyMomentum(const ::tensorflow::Scope& scope, + ::tensorflow::Input var, ::tensorflow::Input accum, + ::tensorflow::Input lr, ::tensorflow::Input grad, + ::tensorflow::Input indices, ::tensorflow::Input + momentum); + ResourceSparseApplyMomentum(const ::tensorflow::Scope& scope, + ::tensorflow::Input var, ::tensorflow::Input accum, + ::tensorflow::Input lr, ::tensorflow::Input grad, + ::tensorflow::Input indices, ::tensorflow::Input + momentum, const ResourceSparseApplyMomentum::Attrs& + attrs); + operator ::tensorflow::Operation() const { return operation; } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + static Attrs UseNesterov(bool x) { + return Attrs().UseNesterov(x); + } + + Operation operation; +}; + +/// Sparse update entries in '*var' and '*accum' according to FOBOS algorithm. +/// +/// That is for rows we have grad for, we update var and accum as follows: +/// accum += grad * grad +/// prox_v = var +/// prox_v -= lr * grad * (1 / sqrt(accum)) +/// var = sign(prox_v)/(1+lr*l2) * max{|prox_v|-lr*l1,0} +/// +/// Arguments: +/// * scope: A Scope object +/// * var: Should be from a Variable(). +/// * accum: Should be from a Variable(). +/// * lr: Learning rate. Must be a scalar. +/// * l1: L1 regularization. Must be a scalar. +/// * l2: L2 regularization. Must be a scalar. +/// * grad: The gradient. +/// * indices: A vector of indices into the first dimension of var and accum. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If True, updating of the var and accum tensors will be protected by +/// a lock; otherwise the behavior is undefined, but may exhibit less contention. +/// +/// Returns: +/// * the created `Operation` +class ResourceSparseApplyProximalAdagrad { + public: + /// Optional attribute setters for ResourceSparseApplyProximalAdagrad + struct Attrs { + /// If True, updating of the var and accum tensors will be protected by + /// a lock; otherwise the behavior is undefined, but may exhibit less contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = false; + }; + ResourceSparseApplyProximalAdagrad(const ::tensorflow::Scope& scope, + ::tensorflow::Input var, ::tensorflow::Input + accum, ::tensorflow::Input lr, + ::tensorflow::Input l1, ::tensorflow::Input + l2, ::tensorflow::Input grad, + ::tensorflow::Input indices); + ResourceSparseApplyProximalAdagrad(const ::tensorflow::Scope& scope, + ::tensorflow::Input var, ::tensorflow::Input + accum, ::tensorflow::Input lr, + ::tensorflow::Input l1, ::tensorflow::Input + l2, ::tensorflow::Input grad, + ::tensorflow::Input indices, const + ResourceSparseApplyProximalAdagrad::Attrs& + attrs); + operator ::tensorflow::Operation() const { return operation; } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; +}; + +/// Sparse update '*var' as FOBOS algorithm with fixed learning rate. +/// +/// That is for rows we have grad for, we update var as follows: +/// prox_v = var - alpha * grad +/// var = sign(prox_v)/(1+alpha*l2) * max{|prox_v|-alpha*l1,0} +/// +/// Arguments: +/// * scope: A Scope object +/// * var: Should be from a Variable(). +/// * alpha: Scaling factor. Must be a scalar. +/// * l1: L1 regularization. Must be a scalar. +/// * l2: L2 regularization. Must be a scalar. +/// * grad: The gradient. +/// * indices: A vector of indices into the first dimension of var and accum. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If True, the subtraction will be protected by a lock; +/// otherwise the behavior is undefined, but may exhibit less contention. +/// +/// Returns: +/// * the created `Operation` +class ResourceSparseApplyProximalGradientDescent { + public: + /// Optional attribute setters for ResourceSparseApplyProximalGradientDescent + struct Attrs { + /// If True, the subtraction will be protected by a lock; + /// otherwise the behavior is undefined, but may exhibit less contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = false; + }; + ResourceSparseApplyProximalGradientDescent(const ::tensorflow::Scope& scope, + ::tensorflow::Input var, + ::tensorflow::Input alpha, + ::tensorflow::Input l1, + ::tensorflow::Input l2, + ::tensorflow::Input grad, + ::tensorflow::Input indices); + ResourceSparseApplyProximalGradientDescent(const ::tensorflow::Scope& scope, + ::tensorflow::Input var, + ::tensorflow::Input alpha, + ::tensorflow::Input l1, + ::tensorflow::Input l2, + ::tensorflow::Input grad, + ::tensorflow::Input indices, const + ResourceSparseApplyProximalGradientDescent::Attrs& + attrs); + operator ::tensorflow::Operation() const { return operation; } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; +}; + +/// Update '*var' according to the RMSProp algorithm. +/// +/// Note that in dense implementation of this algorithm, ms and mom will +/// update even if the grad is zero, but in this sparse implementation, ms +/// and mom will not update in iterations during which the grad is zero. +/// +/// mean_square = decay * mean_square + (1-decay) * gradient ** 2 +/// Delta = learning_rate * gradient / sqrt(mean_square + epsilon) +/// +/// ms <- rho * ms_{t-1} + (1-rho) * grad * grad +/// mom <- momentum * mom_{t-1} + lr * grad / sqrt(ms + epsilon) +/// var <- var - mom +/// +/// Arguments: +/// * scope: A Scope object +/// * var: Should be from a Variable(). +/// * ms: Should be from a Variable(). +/// * mom: Should be from a Variable(). +/// * lr: Scaling factor. Must be a scalar. +/// * rho: Decay rate. Must be a scalar. +/// * epsilon: Ridge term. Must be a scalar. +/// * grad: The gradient. +/// * indices: A vector of indices into the first dimension of var, ms and mom. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If `True`, updating of the var, ms, and mom tensors is protected +/// by a lock; otherwise the behavior is undefined, but may exhibit less +/// contention. +/// +/// Returns: +/// * the created `Operation` +class ResourceSparseApplyRMSProp { + public: + /// Optional attribute setters for ResourceSparseApplyRMSProp + struct Attrs { + /// If `True`, updating of the var, ms, and mom tensors is protected + /// by a lock; otherwise the behavior is undefined, but may exhibit less + /// contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = false; + }; + ResourceSparseApplyRMSProp(const ::tensorflow::Scope& scope, + ::tensorflow::Input var, ::tensorflow::Input ms, + ::tensorflow::Input mom, ::tensorflow::Input lr, + ::tensorflow::Input rho, ::tensorflow::Input + momentum, ::tensorflow::Input epsilon, + ::tensorflow::Input grad, ::tensorflow::Input + indices); + ResourceSparseApplyRMSProp(const ::tensorflow::Scope& scope, + ::tensorflow::Input var, ::tensorflow::Input ms, + ::tensorflow::Input mom, ::tensorflow::Input lr, + ::tensorflow::Input rho, ::tensorflow::Input + momentum, ::tensorflow::Input epsilon, + ::tensorflow::Input grad, ::tensorflow::Input + indices, const ResourceSparseApplyRMSProp::Attrs& + attrs); + operator ::tensorflow::Operation() const { return operation; } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; +}; + +/// var: Should be from a Variable(). +/// +/// Arguments: +/// * scope: A Scope object +/// * accum: Should be from a Variable(). +/// * accum_update: : Should be from a Variable(). +/// * lr: Learning rate. Must be a scalar. +/// * rho: Decay factor. Must be a scalar. +/// * epsilon: Constant factor. Must be a scalar. +/// * grad: The gradient. +/// * indices: A vector of indices into the first dimension of var and accum. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If True, updating of the var and accum tensors will be protected by +/// a lock; otherwise the behavior is undefined, but may exhibit less contention. +/// +/// Returns: +/// * `Output`: Same as "var". +class SparseApplyAdadelta { + public: + /// Optional attribute setters for SparseApplyAdadelta + struct Attrs { + /// If True, updating of the var and accum tensors will be protected by + /// a lock; otherwise the behavior is undefined, but may exhibit less contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = false; + }; + SparseApplyAdadelta(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input accum, ::tensorflow::Input + accum_update, ::tensorflow::Input lr, ::tensorflow::Input + rho, ::tensorflow::Input epsilon, ::tensorflow::Input grad, + ::tensorflow::Input indices); + SparseApplyAdadelta(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input accum, ::tensorflow::Input + accum_update, ::tensorflow::Input lr, ::tensorflow::Input + rho, ::tensorflow::Input epsilon, ::tensorflow::Input grad, + ::tensorflow::Input indices, const + SparseApplyAdadelta::Attrs& attrs); + operator ::tensorflow::Output() const { return out; } + operator ::tensorflow::Input() const { return out; } + ::tensorflow::Node* node() const { return out.node(); } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; + ::tensorflow::Output out; +}; + +/// Update relevant entries in '*var' and '*accum' according to the adagrad scheme. +/// +/// That is for rows we have grad for, we update var and accum as follows: +/// $$accum += grad * grad$$ +/// $$var -= lr * grad * (1 / sqrt(accum))$$ +/// +/// Arguments: +/// * scope: A Scope object +/// * var: Should be from a Variable(). +/// * accum: Should be from a Variable(). +/// * lr: Learning rate. Must be a scalar. +/// * grad: The gradient. +/// * indices: A vector of indices into the first dimension of var and accum. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If `True`, updating of the var and accum tensors will be protected +/// by a lock; otherwise the behavior is undefined, but may exhibit less +/// contention. +/// +/// Returns: +/// * `Output`: Same as "var". +class SparseApplyAdagrad { + public: + /// Optional attribute setters for SparseApplyAdagrad + struct Attrs { + /// If `True`, updating of the var and accum tensors will be protected + /// by a lock; otherwise the behavior is undefined, but may exhibit less + /// contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + /// Defaults to true + TF_MUST_USE_RESULT Attrs UpdateSlots(bool x) { + Attrs ret = *this; + ret.update_slots_ = x; + return ret; + } + + bool use_locking_ = false; + bool update_slots_ = true; + }; + SparseApplyAdagrad(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input accum, ::tensorflow::Input lr, + ::tensorflow::Input grad, ::tensorflow::Input indices); + SparseApplyAdagrad(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input accum, ::tensorflow::Input lr, + ::tensorflow::Input grad, ::tensorflow::Input indices, const + SparseApplyAdagrad::Attrs& attrs); + operator ::tensorflow::Output() const { return out; } + operator ::tensorflow::Input() const { return out; } + ::tensorflow::Node* node() const { return out.node(); } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + static Attrs UpdateSlots(bool x) { + return Attrs().UpdateSlots(x); + } + + Operation operation; + ::tensorflow::Output out; +}; + +/// Update entries in '*var' and '*accum' according to the proximal adagrad scheme. +/// +/// Arguments: +/// * scope: A Scope object +/// * var: Should be from a Variable(). +/// * gradient_accumulator: Should be from a Variable(). +/// * gradient_squared_accumulator: Should be from a Variable(). +/// * grad: The gradient. +/// * indices: A vector of indices into the first dimension of var and accum. +/// * lr: Learning rate. Must be a scalar. +/// * l1: L1 regularization. Must be a scalar. +/// * l2: L2 regularization. Must be a scalar. +/// * global_step: Training step number. Must be a scalar. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If True, updating of the var and accum tensors will be protected by +/// a lock; otherwise the behavior is undefined, but may exhibit less contention. +/// +/// Returns: +/// * `Output`: Same as "var". +class SparseApplyAdagradDA { + public: + /// Optional attribute setters for SparseApplyAdagradDA + struct Attrs { + /// If True, updating of the var and accum tensors will be protected by + /// a lock; otherwise the behavior is undefined, but may exhibit less contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = false; + }; + SparseApplyAdagradDA(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input gradient_accumulator, + ::tensorflow::Input gradient_squared_accumulator, + ::tensorflow::Input grad, ::tensorflow::Input indices, + ::tensorflow::Input lr, ::tensorflow::Input l1, + ::tensorflow::Input l2, ::tensorflow::Input global_step); + SparseApplyAdagradDA(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input gradient_accumulator, + ::tensorflow::Input gradient_squared_accumulator, + ::tensorflow::Input grad, ::tensorflow::Input indices, + ::tensorflow::Input lr, ::tensorflow::Input l1, + ::tensorflow::Input l2, ::tensorflow::Input global_step, + const SparseApplyAdagradDA::Attrs& attrs); + operator ::tensorflow::Output() const { return out; } + operator ::tensorflow::Input() const { return out; } + ::tensorflow::Node* node() const { return out.node(); } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; + ::tensorflow::Output out; +}; + +/// Update '*var' according to the centered RMSProp algorithm. +/// +/// The centered RMSProp algorithm uses an estimate of the centered second moment +/// (i.e., the variance) for normalization, as opposed to regular RMSProp, which +/// uses the (uncentered) second moment. This often helps with training, but is +/// slightly more expensive in terms of computation and memory. +/// +/// Note that in dense implementation of this algorithm, mg, ms, and mom will +/// update even if the grad is zero, but in this sparse implementation, mg, ms, +/// and mom will not update in iterations during which the grad is zero. +/// +/// mean_square = decay * mean_square + (1-decay) * gradient ** 2 +/// mean_grad = decay * mean_grad + (1-decay) * gradient +/// Delta = learning_rate * gradient / sqrt(mean_square + epsilon - mean_grad ** 2) +/// +/// $$ms <- rho * ms_{t-1} + (1-rho) * grad * grad$$ +/// $$mom <- momentum * mom_{t-1} + lr * grad / sqrt(ms + epsilon)$$ +/// $$var <- var - mom$$ +/// +/// Arguments: +/// * scope: A Scope object +/// * var: Should be from a Variable(). +/// * mg: Should be from a Variable(). +/// * ms: Should be from a Variable(). +/// * mom: Should be from a Variable(). +/// * lr: Scaling factor. Must be a scalar. +/// * rho: Decay rate. Must be a scalar. +/// * epsilon: Ridge term. Must be a scalar. +/// * grad: The gradient. +/// * indices: A vector of indices into the first dimension of var, ms and mom. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If `True`, updating of the var, mg, ms, and mom tensors is +/// protected by a lock; otherwise the behavior is undefined, but may exhibit less +/// contention. +/// +/// Returns: +/// * `Output`: Same as "var". +class SparseApplyCenteredRMSProp { + public: + /// Optional attribute setters for SparseApplyCenteredRMSProp + struct Attrs { + /// If `True`, updating of the var, mg, ms, and mom tensors is + /// protected by a lock; otherwise the behavior is undefined, but may exhibit less + /// contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = false; + }; + SparseApplyCenteredRMSProp(const ::tensorflow::Scope& scope, + ::tensorflow::Input var, ::tensorflow::Input mg, + ::tensorflow::Input ms, ::tensorflow::Input mom, + ::tensorflow::Input lr, ::tensorflow::Input rho, + ::tensorflow::Input momentum, ::tensorflow::Input + epsilon, ::tensorflow::Input grad, + ::tensorflow::Input indices); + SparseApplyCenteredRMSProp(const ::tensorflow::Scope& scope, + ::tensorflow::Input var, ::tensorflow::Input mg, + ::tensorflow::Input ms, ::tensorflow::Input mom, + ::tensorflow::Input lr, ::tensorflow::Input rho, + ::tensorflow::Input momentum, ::tensorflow::Input + epsilon, ::tensorflow::Input grad, + ::tensorflow::Input indices, const + SparseApplyCenteredRMSProp::Attrs& attrs); + operator ::tensorflow::Output() const { return out; } + operator ::tensorflow::Input() const { return out; } + ::tensorflow::Node* node() const { return out.node(); } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; + ::tensorflow::Output out; +}; + +/// Update relevant entries in '*var' according to the Ftrl-proximal scheme. +/// +/// That is for rows we have grad for, we update var, accum and linear as follows: +/// $$accum_new = accum + grad * grad$$ +/// $$linear += grad + (accum_{new}^{-lr_{power}} - accum^{-lr_{power}} / lr * var$$ +/// $$quadratic = 1.0 / (accum_{new}^{lr_{power}} * lr) + 2 * l2$$ +/// $$var = (sign(linear) * l1 - linear) / quadratic\ if\ |linear| > l1\ else\ 0.0$$ +/// $$accum = accum_{new}$$ +/// +/// Arguments: +/// * scope: A Scope object +/// * var: Should be from a Variable(). +/// * accum: Should be from a Variable(). +/// * linear: Should be from a Variable(). +/// * grad: The gradient. +/// * indices: A vector of indices into the first dimension of var and accum. +/// * lr: Scaling factor. Must be a scalar. +/// * l1: L1 regularization. Must be a scalar. +/// * l2: L2 regularization. Must be a scalar. +/// * lr_power: Scaling factor. Must be a scalar. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If `True`, updating of the var and accum tensors will be protected +/// by a lock; otherwise the behavior is undefined, but may exhibit less +/// contention. +/// +/// Returns: +/// * `Output`: Same as "var". +class SparseApplyFtrl { + public: + /// Optional attribute setters for SparseApplyFtrl + struct Attrs { + /// If `True`, updating of the var and accum tensors will be protected + /// by a lock; otherwise the behavior is undefined, but may exhibit less + /// contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = false; + }; + SparseApplyFtrl(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input accum, ::tensorflow::Input linear, + ::tensorflow::Input grad, ::tensorflow::Input indices, + ::tensorflow::Input lr, ::tensorflow::Input l1, + ::tensorflow::Input l2, ::tensorflow::Input lr_power); + SparseApplyFtrl(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input accum, ::tensorflow::Input linear, + ::tensorflow::Input grad, ::tensorflow::Input indices, + ::tensorflow::Input lr, ::tensorflow::Input l1, + ::tensorflow::Input l2, ::tensorflow::Input lr_power, const + SparseApplyFtrl::Attrs& attrs); + operator ::tensorflow::Output() const { return out; } + operator ::tensorflow::Input() const { return out; } + ::tensorflow::Node* node() const { return out.node(); } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; + ::tensorflow::Output out; +}; + +/// Update relevant entries in '*var' according to the Ftrl-proximal scheme. +/// +/// That is for rows we have grad for, we update var, accum and linear as follows: +/// grad_with_shrinkage = grad + 2 * l2_shrinkage * var +/// accum_new = accum + grad_with_shrinkage * grad_with_shrinkage +/// linear += grad_with_shrinkage + +/// (accum_new^(-lr_power) - accum^(-lr_power)) / lr * var +/// quadratic = 1.0 / (accum_new^(lr_power) * lr) + 2 * l2 +/// var = (sign(linear) * l1 - linear) / quadratic if |linear| > l1 else 0.0 +/// accum = accum_new +/// +/// Arguments: +/// * scope: A Scope object +/// * var: Should be from a Variable(). +/// * accum: Should be from a Variable(). +/// * linear: Should be from a Variable(). +/// * grad: The gradient. +/// * indices: A vector of indices into the first dimension of var and accum. +/// * lr: Scaling factor. Must be a scalar. +/// * l1: L1 regularization. Must be a scalar. +/// * l2: L2 shrinkage regulariation. Must be a scalar. +/// * lr_power: Scaling factor. Must be a scalar. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If `True`, updating of the var and accum tensors will be protected +/// by a lock; otherwise the behavior is undefined, but may exhibit less +/// contention. +/// +/// Returns: +/// * `Output`: Same as "var". +class SparseApplyFtrlV2 { + public: + /// Optional attribute setters for SparseApplyFtrlV2 + struct Attrs { + /// If `True`, updating of the var and accum tensors will be protected + /// by a lock; otherwise the behavior is undefined, but may exhibit less + /// contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = false; + }; + SparseApplyFtrlV2(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input accum, ::tensorflow::Input linear, + ::tensorflow::Input grad, ::tensorflow::Input indices, + ::tensorflow::Input lr, ::tensorflow::Input l1, + ::tensorflow::Input l2, ::tensorflow::Input l2_shrinkage, + ::tensorflow::Input lr_power); + SparseApplyFtrlV2(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input accum, ::tensorflow::Input linear, + ::tensorflow::Input grad, ::tensorflow::Input indices, + ::tensorflow::Input lr, ::tensorflow::Input l1, + ::tensorflow::Input l2, ::tensorflow::Input l2_shrinkage, + ::tensorflow::Input lr_power, const SparseApplyFtrlV2::Attrs& + attrs); + operator ::tensorflow::Output() const { return out; } + operator ::tensorflow::Input() const { return out; } + ::tensorflow::Node* node() const { return out.node(); } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; + ::tensorflow::Output out; +}; + +/// Update relevant entries in '*var' and '*accum' according to the momentum scheme. +/// +/// Set use_nesterov = True if you want to use Nesterov momentum. +/// +/// That is for rows we have grad for, we update var and accum as follows: +/// +/// $$accum = accum * momentum + grad$$ +/// $$var -= lr * accum$$ +/// +/// Arguments: +/// * scope: A Scope object +/// * var: Should be from a Variable(). +/// * accum: Should be from a Variable(). +/// * lr: Learning rate. Must be a scalar. +/// * grad: The gradient. +/// * indices: A vector of indices into the first dimension of var and accum. +/// * momentum: Momentum. Must be a scalar. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If `True`, updating of the var and accum tensors will be protected +/// by a lock; otherwise the behavior is undefined, but may exhibit less +/// contention. +/// * use_nesterov: If `True`, the tensor passed to compute grad will be +/// var - lr * momentum * accum, so in the end, the var you get is actually +/// var - lr * momentum * accum. +/// +/// Returns: +/// * `Output`: Same as "var". +class SparseApplyMomentum { + public: + /// Optional attribute setters for SparseApplyMomentum + struct Attrs { + /// If `True`, updating of the var and accum tensors will be protected + /// by a lock; otherwise the behavior is undefined, but may exhibit less + /// contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + /// If `True`, the tensor passed to compute grad will be + /// var - lr * momentum * accum, so in the end, the var you get is actually + /// var - lr * momentum * accum. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseNesterov(bool x) { + Attrs ret = *this; + ret.use_nesterov_ = x; + return ret; + } + + bool use_locking_ = false; + bool use_nesterov_ = false; + }; + SparseApplyMomentum(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input accum, ::tensorflow::Input lr, + ::tensorflow::Input grad, ::tensorflow::Input indices, + ::tensorflow::Input momentum); + SparseApplyMomentum(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input accum, ::tensorflow::Input lr, + ::tensorflow::Input grad, ::tensorflow::Input indices, + ::tensorflow::Input momentum, const + SparseApplyMomentum::Attrs& attrs); + operator ::tensorflow::Output() const { return out; } + operator ::tensorflow::Input() const { return out; } + ::tensorflow::Node* node() const { return out.node(); } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + static Attrs UseNesterov(bool x) { + return Attrs().UseNesterov(x); + } + + Operation operation; + ::tensorflow::Output out; +}; + +/// Sparse update entries in '*var' and '*accum' according to FOBOS algorithm. +/// +/// That is for rows we have grad for, we update var and accum as follows: +/// $$accum += grad * grad$$ +/// $$prox_v = var$$ +/// $$prox_v -= lr * grad * (1 / sqrt(accum))$$ +/// $$var = sign(prox_v)/(1+lr*l2) * max{|prox_v|-lr*l1,0}$$ +/// +/// Arguments: +/// * scope: A Scope object +/// * var: Should be from a Variable(). +/// * accum: Should be from a Variable(). +/// * lr: Learning rate. Must be a scalar. +/// * l1: L1 regularization. Must be a scalar. +/// * l2: L2 regularization. Must be a scalar. +/// * grad: The gradient. +/// * indices: A vector of indices into the first dimension of var and accum. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If True, updating of the var and accum tensors will be protected by +/// a lock; otherwise the behavior is undefined, but may exhibit less contention. +/// +/// Returns: +/// * `Output`: Same as "var". +class SparseApplyProximalAdagrad { + public: + /// Optional attribute setters for SparseApplyProximalAdagrad + struct Attrs { + /// If True, updating of the var and accum tensors will be protected by + /// a lock; otherwise the behavior is undefined, but may exhibit less contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = false; + }; + SparseApplyProximalAdagrad(const ::tensorflow::Scope& scope, + ::tensorflow::Input var, ::tensorflow::Input accum, + ::tensorflow::Input lr, ::tensorflow::Input l1, + ::tensorflow::Input l2, ::tensorflow::Input grad, + ::tensorflow::Input indices); + SparseApplyProximalAdagrad(const ::tensorflow::Scope& scope, + ::tensorflow::Input var, ::tensorflow::Input accum, + ::tensorflow::Input lr, ::tensorflow::Input l1, + ::tensorflow::Input l2, ::tensorflow::Input grad, + ::tensorflow::Input indices, const + SparseApplyProximalAdagrad::Attrs& attrs); + operator ::tensorflow::Output() const { return out; } + operator ::tensorflow::Input() const { return out; } + ::tensorflow::Node* node() const { return out.node(); } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; + ::tensorflow::Output out; +}; + +/// Sparse update '*var' as FOBOS algorithm with fixed learning rate. +/// +/// That is for rows we have grad for, we update var as follows: +/// $$prox_v = var - alpha * grad$$ +/// $$var = sign(prox_v)/(1+alpha*l2) * max{|prox_v|-alpha*l1,0}$$ +/// +/// Arguments: +/// * scope: A Scope object +/// * var: Should be from a Variable(). +/// * alpha: Scaling factor. Must be a scalar. +/// * l1: L1 regularization. Must be a scalar. +/// * l2: L2 regularization. Must be a scalar. +/// * grad: The gradient. +/// * indices: A vector of indices into the first dimension of var and accum. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If True, the subtraction will be protected by a lock; +/// otherwise the behavior is undefined, but may exhibit less contention. +/// +/// Returns: +/// * `Output`: Same as "var". +class SparseApplyProximalGradientDescent { + public: + /// Optional attribute setters for SparseApplyProximalGradientDescent + struct Attrs { + /// If True, the subtraction will be protected by a lock; + /// otherwise the behavior is undefined, but may exhibit less contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = false; + }; + SparseApplyProximalGradientDescent(const ::tensorflow::Scope& scope, + ::tensorflow::Input var, ::tensorflow::Input + alpha, ::tensorflow::Input l1, + ::tensorflow::Input l2, ::tensorflow::Input + grad, ::tensorflow::Input indices); + SparseApplyProximalGradientDescent(const ::tensorflow::Scope& scope, + ::tensorflow::Input var, ::tensorflow::Input + alpha, ::tensorflow::Input l1, + ::tensorflow::Input l2, ::tensorflow::Input + grad, ::tensorflow::Input indices, const + SparseApplyProximalGradientDescent::Attrs& + attrs); + operator ::tensorflow::Output() const { return out; } + operator ::tensorflow::Input() const { return out; } + ::tensorflow::Node* node() const { return out.node(); } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; + ::tensorflow::Output out; +}; + +/// Update '*var' according to the RMSProp algorithm. +/// +/// Note that in dense implementation of this algorithm, ms and mom will +/// update even if the grad is zero, but in this sparse implementation, ms +/// and mom will not update in iterations during which the grad is zero. +/// +/// mean_square = decay * mean_square + (1-decay) * gradient ** 2 +/// Delta = learning_rate * gradient / sqrt(mean_square + epsilon) +/// +/// $$ms <- rho * ms_{t-1} + (1-rho) * grad * grad$$ +/// $$mom <- momentum * mom_{t-1} + lr * grad / sqrt(ms + epsilon)$$ +/// $$var <- var - mom$$ +/// +/// Arguments: +/// * scope: A Scope object +/// * var: Should be from a Variable(). +/// * ms: Should be from a Variable(). +/// * mom: Should be from a Variable(). +/// * lr: Scaling factor. Must be a scalar. +/// * rho: Decay rate. Must be a scalar. +/// * epsilon: Ridge term. Must be a scalar. +/// * grad: The gradient. +/// * indices: A vector of indices into the first dimension of var, ms and mom. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If `True`, updating of the var, ms, and mom tensors is protected +/// by a lock; otherwise the behavior is undefined, but may exhibit less +/// contention. +/// +/// Returns: +/// * `Output`: Same as "var". +class SparseApplyRMSProp { + public: + /// Optional attribute setters for SparseApplyRMSProp + struct Attrs { + /// If `True`, updating of the var, ms, and mom tensors is protected + /// by a lock; otherwise the behavior is undefined, but may exhibit less + /// contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = false; + }; + SparseApplyRMSProp(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input ms, ::tensorflow::Input mom, + ::tensorflow::Input lr, ::tensorflow::Input rho, + ::tensorflow::Input momentum, ::tensorflow::Input epsilon, + ::tensorflow::Input grad, ::tensorflow::Input indices); + SparseApplyRMSProp(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input ms, ::tensorflow::Input mom, + ::tensorflow::Input lr, ::tensorflow::Input rho, + ::tensorflow::Input momentum, ::tensorflow::Input epsilon, + ::tensorflow::Input grad, ::tensorflow::Input indices, const + SparseApplyRMSProp::Attrs& attrs); + operator ::tensorflow::Output() const { return out; } + operator ::tensorflow::Input() const { return out; } + ::tensorflow::Node* node() const { return out.node(); } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; + ::tensorflow::Output out; +}; + +/// @} + +} // namespace ops +} // namespace tensorflow + +#endif // TENSORFLOW_CC_OPS_TRAINING_OPS_H_ diff --git a/linux/include/tensorflow/cc/ops/training_ops_internal.h b/linux/include/tensorflow/cc/ops/training_ops_internal.h new file mode 100644 index 000000000..4c53e4441 --- /dev/null +++ b/linux/include/tensorflow/cc/ops/training_ops_internal.h @@ -0,0 +1,154 @@ +// This file is MACHINE GENERATED! Do not edit. + +#ifndef TENSORFLOW_CC_OPS_TRAINING_OPS_INTERNAL_H_ +#define TENSORFLOW_CC_OPS_TRAINING_OPS_INTERNAL_H_ + +// This file is MACHINE GENERATED! Do not edit. + +#include "tensorflow/cc/framework/ops.h" +#include "tensorflow/cc/framework/scope.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/gtl/array_slice.h" + +namespace tensorflow { +namespace ops { +namespace internal { +// NOTE: This namespace has internal TensorFlow details that +// are not part of TensorFlow's public API. + +/// @defgroup training_ops_internal Training Ops Internal +/// @{ + +/// Update '*var' according to the AdaMax algorithm. +/// +/// m_t <- beta1 * m_{t-1} + (1 - beta1) * g +/// v_t <- max(beta2 * v_{t-1}, abs(g)) +/// variable <- variable - learning_rate / (1 - beta1^t) * m_t / (v_t + epsilon) +/// +/// Arguments: +/// * scope: A Scope object +/// * var: Should be from a Variable(). +/// * m: Should be from a Variable(). +/// * v: Should be from a Variable(). +/// * beta1_power: Must be a scalar. +/// * lr: Scaling factor. Must be a scalar. +/// * beta1: Momentum factor. Must be a scalar. +/// * beta2: Momentum factor. Must be a scalar. +/// * epsilon: Ridge term. Must be a scalar. +/// * grad: The gradient. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If `True`, updating of the var, m, and v tensors will be protected +/// by a lock; otherwise the behavior is undefined, but may exhibit less +/// contention. +/// +/// Returns: +/// * `Output`: Same as "var". +class ApplyAdaMax { + public: + /// Optional attribute setters for ApplyAdaMax + struct Attrs { + /// If `True`, updating of the var, m, and v tensors will be protected + /// by a lock; otherwise the behavior is undefined, but may exhibit less + /// contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = false; + }; + ApplyAdaMax(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input m, ::tensorflow::Input v, ::tensorflow::Input + beta1_power, ::tensorflow::Input lr, ::tensorflow::Input beta1, + ::tensorflow::Input beta2, ::tensorflow::Input epsilon, + ::tensorflow::Input grad); + ApplyAdaMax(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input m, ::tensorflow::Input v, ::tensorflow::Input + beta1_power, ::tensorflow::Input lr, ::tensorflow::Input beta1, + ::tensorflow::Input beta2, ::tensorflow::Input epsilon, + ::tensorflow::Input grad, const ApplyAdaMax::Attrs& attrs); + operator ::tensorflow::Output() const { return out; } + operator ::tensorflow::Input() const { return out; } + ::tensorflow::Node* node() const { return out.node(); } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; + ::tensorflow::Output out; +}; + +/// Update '*var' according to the AdaMax algorithm. +/// +/// m_t <- beta1 * m_{t-1} + (1 - beta1) * g +/// v_t <- max(beta2 * v_{t-1}, abs(g)) +/// variable <- variable - learning_rate / (1 - beta1^t) * m_t / (v_t + epsilon) +/// +/// Arguments: +/// * scope: A Scope object +/// * var: Should be from a Variable(). +/// * m: Should be from a Variable(). +/// * v: Should be from a Variable(). +/// * beta1_power: Must be a scalar. +/// * lr: Scaling factor. Must be a scalar. +/// * beta1: Momentum factor. Must be a scalar. +/// * beta2: Momentum factor. Must be a scalar. +/// * epsilon: Ridge term. Must be a scalar. +/// * grad: The gradient. +/// +/// Optional attributes (see `Attrs`): +/// * use_locking: If `True`, updating of the var, m, and v tensors will be protected +/// by a lock; otherwise the behavior is undefined, but may exhibit less +/// contention. +/// +/// Returns: +/// * the created `Operation` +class ResourceApplyAdaMax { + public: + /// Optional attribute setters for ResourceApplyAdaMax + struct Attrs { + /// If `True`, updating of the var, m, and v tensors will be protected + /// by a lock; otherwise the behavior is undefined, but may exhibit less + /// contention. + /// + /// Defaults to false + TF_MUST_USE_RESULT Attrs UseLocking(bool x) { + Attrs ret = *this; + ret.use_locking_ = x; + return ret; + } + + bool use_locking_ = false; + }; + ResourceApplyAdaMax(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input m, ::tensorflow::Input v, + ::tensorflow::Input beta1_power, ::tensorflow::Input lr, + ::tensorflow::Input beta1, ::tensorflow::Input beta2, + ::tensorflow::Input epsilon, ::tensorflow::Input grad); + ResourceApplyAdaMax(const ::tensorflow::Scope& scope, ::tensorflow::Input var, + ::tensorflow::Input m, ::tensorflow::Input v, + ::tensorflow::Input beta1_power, ::tensorflow::Input lr, + ::tensorflow::Input beta1, ::tensorflow::Input beta2, + ::tensorflow::Input epsilon, ::tensorflow::Input grad, + const ResourceApplyAdaMax::Attrs& attrs); + operator ::tensorflow::Operation() const { return operation; } + + static Attrs UseLocking(bool x) { + return Attrs().UseLocking(x); + } + + Operation operation; +}; + +} // namespace internal +} // namespace ops +} // namespace tensorflow + +#endif // TENSORFLOW_CC_OPS_TRAINING_OPS_INTERNAL_H_ diff --git a/linux/include/tensorflow/cc/ops/user_ops.h b/linux/include/tensorflow/cc/ops/user_ops.h new file mode 100644 index 000000000..f7e6811a8 --- /dev/null +++ b/linux/include/tensorflow/cc/ops/user_ops.h @@ -0,0 +1,44 @@ +// This file is MACHINE GENERATED! Do not edit. + +#ifndef TENSORFLOW_CC_OPS_USER_OPS_H_ +#define TENSORFLOW_CC_OPS_USER_OPS_H_ + +// This file is MACHINE GENERATED! Do not edit. + +#include "tensorflow/cc/framework/ops.h" +#include "tensorflow/cc/framework/scope.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/gtl/array_slice.h" + +namespace tensorflow { +namespace ops { + +/// @defgroup user_ops User Ops +/// @{ + +/// Output a fact about factorials. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The fact tensor. +class Fact { + public: + Fact(const ::tensorflow::Scope& scope); + operator ::tensorflow::Output() const { return fact; } + operator ::tensorflow::Input() const { return fact; } + ::tensorflow::Node* node() const { return fact.node(); } + + Operation operation; + ::tensorflow::Output fact; +}; + +/// @} + +} // namespace ops +} // namespace tensorflow + +#endif // TENSORFLOW_CC_OPS_USER_OPS_H_ diff --git a/linux/include/tensorflow/cc/ops/user_ops_internal.h b/linux/include/tensorflow/cc/ops/user_ops_internal.h new file mode 100644 index 000000000..784254dd4 --- /dev/null +++ b/linux/include/tensorflow/cc/ops/user_ops_internal.h @@ -0,0 +1,28 @@ +// This file is MACHINE GENERATED! Do not edit. + +#ifndef TENSORFLOW_CC_OPS_USER_OPS_INTERNAL_H_ +#define TENSORFLOW_CC_OPS_USER_OPS_INTERNAL_H_ + +// This file is MACHINE GENERATED! Do not edit. + +#include "tensorflow/cc/framework/ops.h" +#include "tensorflow/cc/framework/scope.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/gtl/array_slice.h" + +namespace tensorflow { +namespace ops { +namespace internal { +// NOTE: This namespace has internal TensorFlow details that +// are not part of TensorFlow's public API. + +/// @defgroup user_ops_internal User Ops Internal +/// @{ + +} // namespace internal +} // namespace ops +} // namespace tensorflow + +#endif // TENSORFLOW_CC_OPS_USER_OPS_INTERNAL_H_ diff --git a/linux/include/tensorflow/cc/ops/while_loop.h b/linux/include/tensorflow/cc/ops/while_loop.h new file mode 100644 index 000000000..727237b5c --- /dev/null +++ b/linux/include/tensorflow/cc/ops/while_loop.h @@ -0,0 +1,74 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CC_OPS_WHILE_LOOP_H_ +#define TENSORFLOW_CC_OPS_WHILE_LOOP_H_ + +#include "tensorflow/cc/framework/ops.h" +#include "tensorflow/cc/framework/scope.h" + +namespace tensorflow { +namespace ops { + +// Function that takes cond graph inputs and returns cond graph boolean output. +// 'output' need not be set if an error is returned. +typedef std::function& inputs, + Output* output)> + CondGraphBuilderFn; + +// Function that takes body graph inputs and returns body graph outputs. +// 'outputs' need not be populated if an error is returned. +typedef std::function& inputs, + std::vector* outputs)> + BodyGraphBuilderFn; + +// Constructs a while loop. +// +// Arguments: +// * scope: used to construct the while loop. +// * inputs: the initial values of the loop variables. Must be non-empty. +// * cond: a function that builds the condition graph of the loop. Takes the +// current loop variables as inputs and returns a scalar boolean Output +// indicating whether the loop should continue. +// * body: a function that builds the body graph of the loop. Takes the current +// loop variables as inputs and returns the updated loop variables. +// * frame_name: the frame name to use for this while loop. This should be a +// unique name. This will be used as a prefix for created operations. +// * outputs: output param that returns final loop variable outputs in non-error +// case. Must be non-null and empty. +// * create_while_ctx: if true, a WhileContext is created and populated for this +// loop. See core/graph/while_context.h for more details on +// WhileContexts. This is set to false for loops used as part of gradient +// computations, since they're part of the gradient for a loop in the +// forward-pass. +// TODO(skyewm): revisit this. Should we create WhileContexts for all loops, +// even if we don't need them? +// * cond_output: if non-null, the output of the predicate is returned. This +// will always be a LoopCond node. +// +// Returns an error if the while loop could not be fully constructed. +// +// TODO(skyewm): clean up partially-constructed loop in error case +// TODO(skyewm): create public interface to this method +Status BuildWhileLoop(const Scope& scope, const std::vector& inputs, + const CondGraphBuilderFn& cond, + const BodyGraphBuilderFn& body, const string& frame_name, + OutputList* outputs, bool create_while_ctx = true, + Output* cond_output = nullptr); + +} // namespace ops +} // namespace tensorflow + +#endif // TENSORFLOW_CC_OPS_WHILE_LOOP_H_ diff --git a/linux/include/tensorflow/cc/profiler/profiler.h b/linux/include/tensorflow/cc/profiler/profiler.h new file mode 100644 index 000000000..64edbb576 --- /dev/null +++ b/linux/include/tensorflow/cc/profiler/profiler.h @@ -0,0 +1,97 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CC_PROFILER_PROFILER_H_ +#define TENSORFLOW_CC_PROFILER_PROFILER_H_ + +#include "tensorflow/core/framework/graph.pb.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/profiler/internal/tfprof_stats.h" +#include "tensorflow/core/profiler/tfprof_options.h" +#include "tensorflow/core/profiler/tfprof_output.pb.h" + +namespace tensorflow { +namespace tfprof { + +/// @addtogroup core +/// @{ + +/// A `Profiler` object lets the caller profile the execution of a graph. +/// +/// Example: +/// // First build a graph and run tracing. +/// Scope root = Scope::NewRootScope(); +/// auto a = Placeholder(root, DT_INT32); +/// auto c = Add(root, a, {41}); +/// +/// ClientSession session(root); +/// std::vector outputs; +/// RunOptions run_options; +/// run_options.set_trace_level(RunOptions::FULL_TRACE); +/// RunMetadata run_meta; +/// Status s = session.Run(run_options, { {a, {1}} }, {c}, &outputs, +/// &run_meta); +/// if (!s.ok()) { ... } +/// +/// // Then create profiler to do profiling. +/// GraphDef graph; +/// root.ToGraphDef(&graph); +/// Profiler profiler(graph); +/// profiler.AddStep(0, run_meta); +/// Options opts = ... // TODO(xpan): Support option building API. +/// MultiGraphNodeProto r = profiler.ProfileOperations(opts); +/// +class Profiler { + public: + /// `graph` is the model's GraphDef. + Profiler(const GraphDef& graph); + + /// Adds tracing information `run_meta` to profiler. A `run_meta` is + /// generated by a TensorFlow session run call. `step` is the key + /// to the `run_meta`. When calling ProfileXXX methods, caller can specify + /// `step` in `options` to selectively profile the corresponding `run_meta`. + /// Multiple different `run_meta` can be keyed by the same `step` in order + /// to group them together. + void AddStep(int64 step, const RunMetadata& run_meta); + + /// Profiles the model by organizing nodes in graph structure. + /// Each node is an op and the nodes are connected by the op inputs/outputs. + GraphNodeProto ProfileGraph(const Options& options); + + /// Profiles the model by organizing nodes in name scope structure. + /// Each node is an op, and nodes are organized by the ops' name + /// scope, similar to a file system tree. + /// E.g. /foo is the root of operation /foo/matmul_1 and foo/conv_2. + GraphNodeProto ProfileNameScope(const Options& options); + + /// Profiles the model by organizing nodes by operation types. + /// Each node is an operation type (e.g. Conv2D or MatMul), containing all + /// ops belonging to that type in the model. + MultiGraphNodeProto ProfileOperations(const Options& options); + + /// Serialize the profile content (ProfileProto) into a binary string, + /// User can write the string to file for offline analysis by + /// tfprof command-line tools or graphical user interface. + Status SerializeToString(string* content); + + private: + std::unique_ptr stats_; +}; +/// @} + +} // namespace tfprof +} // namespace tensorflow + +#endif // TENSORFLOW_CC_PROFILER_PROFILER_H_ diff --git a/linux/include/tensorflow/cc/saved_model/constants.h b/linux/include/tensorflow/cc/saved_model/constants.h new file mode 100644 index 000000000..645a3f101 --- /dev/null +++ b/linux/include/tensorflow/cc/saved_model/constants.h @@ -0,0 +1,50 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CC_SAVED_MODEL_CONSTANTS_H_ +#define TENSORFLOW_CC_SAVED_MODEL_CONSTANTS_H_ + +namespace tensorflow { + +/// SavedModel assets directory. +constexpr char kSavedModelAssetsDirectory[] = "assets"; + +/// SavedModel assets.extra directory. +constexpr char kSavedModelAssetsExtraDirectory[] = "assets.extra"; + +/// SavedModel assets key for graph collection-def. +constexpr char kSavedModelAssetsKey[] = "saved_model_assets"; + +/// SavedModel proto filename. +constexpr char kSavedModelFilenamePb[] = "saved_model.pb"; + +/// SavedModel text format proto filename. +constexpr char kSavedModelFilenamePbTxt[] = "saved_model.pbtxt"; + +/// SavedModel legacy init op key. +constexpr char kSavedModelLegacyInitOpKey[] = "legacy_init_op"; + +/// SavedModel main op key. +constexpr char kSavedModelMainOpKey[] = "saved_model_main_op"; + +/// Directory in which to save the SavedModel variables. +constexpr char kSavedModelVariablesDirectory[] = "variables"; + +/// SavedModel variables filename. +constexpr char kSavedModelVariablesFilename[] = "variables"; + +} // namespace tensorflow + +#endif // TENSORFLOW_CC_SAVED_MODEL_CONSTANTS_H_ diff --git a/linux/include/tensorflow/cc/saved_model/loader.h b/linux/include/tensorflow/cc/saved_model/loader.h new file mode 100644 index 000000000..a8e098fa5 --- /dev/null +++ b/linux/include/tensorflow/cc/saved_model/loader.h @@ -0,0 +1,64 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +/// SavedModel loading functions and SavedModelBundle struct. + +#ifndef TENSORFLOW_CC_SAVED_MODEL_LOADER_H_ +#define TENSORFLOW_CC_SAVED_MODEL_LOADER_H_ + +#include +#include + +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/protobuf/meta_graph.pb.h" +#include "tensorflow/core/public/session.h" + +namespace tensorflow { + +/// SavedModel representation once the SavedModel is loaded from storage. +struct SavedModelBundle { + std::unique_ptr session; + MetaGraphDef meta_graph_def; + + /// A TensorFlow Session does not Close itself on destruction. To avoid + /// resource leaks, we explicitly call Close on Sessions that we create. + ~SavedModelBundle() { + if (session) { + session->Close().IgnoreError(); + } + } + + SavedModelBundle() = default; +}; + +/// Loads a SavedModel from the specified export directory. The meta graph def +/// to be loaded is identified by the supplied tags, corresponding exactly to +/// the set of tags used at SavedModel build time. Returns a SavedModel bundle +/// with a session and the requested meta graph def, if found. +Status LoadSavedModel(const SessionOptions& session_options, + const RunOptions& run_options, const string& export_dir, + const std::unordered_set& tags, + SavedModelBundle* const bundle); + +/// Checks whether the provided directory could contain a SavedModel. Note that +/// the method does not load any data by itself. If the method returns `false`, +/// the export directory definitely does not contain a SavedModel. If the method +/// returns `true`, the export directory may contain a SavedModel but provides +/// no guarantee that it can be loaded. +bool MaybeSavedModelDirectory(const string& export_dir); + +} // namespace tensorflow + +#endif // TENSORFLOW_CC_SAVED_MODEL_LOADER_H_ diff --git a/linux/include/tensorflow/cc/saved_model/reader.h b/linux/include/tensorflow/cc/saved_model/reader.h new file mode 100644 index 000000000..5815108df --- /dev/null +++ b/linux/include/tensorflow/cc/saved_model/reader.h @@ -0,0 +1,39 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +/// Functions to read the SavedModel proto, or parts of it. + +#ifndef TENSORFLOW_CC_SAVED_MODEL_READER_H_ +#define TENSORFLOW_CC_SAVED_MODEL_READER_H_ + +#include +#include + +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/protobuf/meta_graph.pb.h" + +namespace tensorflow { + +// Reads the SavedModel proto from saved_model.pb(txt) in the given directory, +// finds the MetaGraphDef that matches the given set of tags and writes it to +// the `meta_graph_def` parameter. Returns a failure status when the SavedModel +// file does not exist or no MetaGraphDef matches the tags. +Status ReadMetaGraphDefFromSavedModel(const string& export_dir, + const std::unordered_set& tags, + MetaGraphDef* const meta_graph_def); + +} // namespace tensorflow + +#endif // TENSORFLOW_CC_SAVED_MODEL_READER_H_ diff --git a/linux/include/tensorflow/cc/saved_model/signature_constants.h b/linux/include/tensorflow/cc/saved_model/signature_constants.h new file mode 100644 index 000000000..7d8c07f5c --- /dev/null +++ b/linux/include/tensorflow/cc/saved_model/signature_constants.h @@ -0,0 +1,69 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CC_SAVED_MODEL_SIGNATURE_CONSTANTS_H_ +#define TENSORFLOW_CC_SAVED_MODEL_SIGNATURE_CONSTANTS_H_ + +namespace tensorflow { + +/// Key in the signature def map for `default` serving signatures. The default +/// signature is used in inference requests where a specific signature was not +/// specified. +static constexpr char kDefaultServingSignatureDefKey[] = "serving_default"; + +//////////////////////////////////////////////////////////////////////////////// +/// Classification API constants. + +/// Classification inputs. +static constexpr char kClassifyInputs[] = "inputs"; + +/// Classification method name used in a SignatureDef. +static constexpr char kClassifyMethodName[] = "tensorflow/serving/classify"; + +/// Classification classes output. +static constexpr char kClassifyOutputClasses[] = "classes"; + +/// Classification scores output. +static constexpr char kClassifyOutputScores[] = "scores"; + +//////////////////////////////////////////////////////////////////////////////// +/// Predict API constants. + +/// Predict inputs. +static constexpr char kPredictInputs[] = "inputs"; + +/// Predict method name used in a SignatureDef. +static constexpr char kPredictMethodName[] = "tensorflow/serving/predict"; + +/// Predict outputs. +static constexpr char kPredictOutputs[] = "outputs"; + +//////////////////////////////////////////////////////////////////////////////// +/// Regression API constants. + +/// Regression inputs. +static constexpr char kRegressInputs[] = "inputs"; + +/// Regression method name used in a SignatureDef. +static constexpr char kRegressMethodName[] = "tensorflow/serving/regress"; + +/// Regression outputs. +static constexpr char kRegressOutputs[] = "outputs"; + +//////////////////////////////////////////////////////////////////////////////// + +} // namespace tensorflow + +#endif // TENSORFLOW_CC_SAVED_MODEL_SIGNATURE_CONSTANTS_H_ diff --git a/linux/include/tensorflow/cc/saved_model/tag_constants.h b/linux/include/tensorflow/cc/saved_model/tag_constants.h new file mode 100644 index 000000000..68a090e0c --- /dev/null +++ b/linux/include/tensorflow/cc/saved_model/tag_constants.h @@ -0,0 +1,35 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CC_SAVED_MODEL_TAG_CONSTANTS_H_ +#define TENSORFLOW_CC_SAVED_MODEL_TAG_CONSTANTS_H_ + +namespace tensorflow { + +/// Tag for the `gpu` graph. +constexpr char kSavedModelTagGpu[] = "gpu"; + +/// Tag for the `tpu` graph. +constexpr char kSavedModelTagTpu[] = "tpu"; + +/// Tag for the `serving` graph. +constexpr char kSavedModelTagServe[] = "serve"; + +/// Tag for the `training` graph. +constexpr char kSavedModelTagTrain[] = "train"; + +} // namespace tensorflow + +#endif // TENSORFLOW_CC_SAVED_MODEL_TAG_CONSTANTS_H_ diff --git a/linux/include/tensorflow/cc/tools/freeze_saved_model.h b/linux/include/tensorflow/cc/tools/freeze_saved_model.h new file mode 100644 index 000000000..b10f29805 --- /dev/null +++ b/linux/include/tensorflow/cc/tools/freeze_saved_model.h @@ -0,0 +1,43 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CC_TOOLS_FREEZE_SAVED_MODEL_H_ +#define TENSORFLOW_CC_TOOLS_FREEZE_SAVED_MODEL_H_ + +#include + +#include "tensorflow/cc/saved_model/loader.h" +#include "tensorflow/core/framework/graph.pb.h" +#include "tensorflow/core/lib/core/status.h" + +namespace tensorflow { + +// Returns a frozen GraphDef, input tensors, and output tensors from the loaded +// SavedModelBundle. +// `inputs` and `outputs` consist of the union of all inputs and outputs in the +// SignatureDefs in the SavedModelBundle. +// FreezeSavedModel sets `frozen_graph_def` to a GraphDef of all nodes needed by +// `outputs`. All variables in the supplied SavedModelBundle are converted to +// constants, set to the value of the variables, by running the restored Session +// in the SavedModelBundle. +// WARNING: Only the variable checkpoints will be reflected in the frozen +// graph_def. All saved_model assets will be ignored. +Status FreezeSavedModel(const SavedModelBundle& saved_model_bundle, + GraphDef* frozen_graph_def, + std::unordered_set* inputs, + std::unordered_set* outputs); + +} // namespace tensorflow + +#endif // TENSORFLOW_CC_TOOLS_FREEZE_SAVED_MODEL_H_ diff --git a/linux/include/tensorflow/cc/training/coordinator.h b/linux/include/tensorflow/cc/training/coordinator.h new file mode 100644 index 000000000..7168b7752 --- /dev/null +++ b/linux/include/tensorflow/cc/training/coordinator.h @@ -0,0 +1,131 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CC_TRAINING_COORDINATOR_H_ +#define TENSORFLOW_CC_TRAINING_COORDINATOR_H_ + +#include +#include +#include +#include + +#include "tensorflow/core/framework/cost_graph.pb.h" +#include "tensorflow/core/lib/core/error_codes.pb.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/mutex.h" +#include "tensorflow/core/protobuf/config.pb.h" + +namespace tensorflow { + +/// The abstract interface for runners which must implement the Join and the +/// IsRunning function. +class RunnerInterface { + public: + virtual ~RunnerInterface() {} + virtual Status Join() = 0; + virtual Status ExportCostGraph(CostGraphDef* cost_graph) const { + return Status(error::INVALID_ARGUMENT, "No cost model to export."); + } + /// Returns true iff the runner is running, i.e. if it is trying to populate + /// its queue. + virtual bool IsRunning() const = 0; +}; + +/// Coordinator class manages the termination of a collection of QueueRunners. +/// Without a coordinator, QueueRunners have to be joined in a specific order; +/// otherwise the QueueRunner::Join() could sometimes hang. The +/// Coordinator::RequestStop() plays the key role which notifies all running +/// threads under a coordinator to stop. This function could be called by any +/// thread or any client. +/// Usage, in the client: +/// Coordinator coord; +/// std::unique_ptr qr(&coord, ...); +/// qr.Start(session); +/// coord.RegisterRunner(std::move(qr)); +/// /// do some work +/// TF_CHECK_OK(coord.Join()); +/// In each thread of QueueRunner, the coordinator needs to be used as: +/// void Run() { +/// while (!coord->ShouldStop()) { +/// /// do some work +/// if (error) { +/// coord->RequestStop(); +/// coord->ReportStatus(error_status); +/// } +/// } +/// } +class Coordinator { + public: + Coordinator(); + + /// Constructor with a list of error codes which would not be taken as errors + /// in status reporting. + Coordinator(const std::vector& clean_stop_errors); + + /// In the destructor, RequestStop() and Join() would be called. + ~Coordinator(); + + /// Registers a runner, i.e. a unit of running threads which is usually a + /// QueueRunner. It takes the ownership of runner to avoid lifecycle-related + /// problems. Note, the coordinator would not start these threads; they are + /// supposed to be in running state when they are registered here. + Status RegisterRunner(std::unique_ptr runner); + + /// Returns true iff all the registered runners have been stopped. + bool AllRunnersStopped(); + + /// Requests all running threads to stop. + Status RequestStop(); + + /// Returns true if its RequestStop() has been called. + bool ShouldStop(); + + /// Joins all threads, returns OK or the first reported and unexpected status. + Status Join(); + + /// Reports status to the coordinator. This is usually called by threads. + void ReportStatus(const Status& status); + + /// Returns the latest status. + Status GetStatus(); + + /// Returns immediately if the coordinator is stopped or blocks until + /// RequestStop() is called. + void WaitForStop(); + + // Returns the cost graph from stored run metadata in registered runners. + Status ExportCostGraph(CostGraphDef* cost_graph) const; + + private: + std::unordered_set clean_stop_errors_; + condition_variable wait_for_stop_; + + mutex mu_; + bool should_stop_ GUARDED_BY(mu_); + + mutex status_lock_; + Status status_ GUARDED_BY(status_lock_); + + mutable mutex runners_lock_; + std::vector> runners_ + GUARDED_BY(runners_lock_); + + TF_DISALLOW_COPY_AND_ASSIGN(Coordinator); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CC_TRAINING_COORDINATOR_H_ diff --git a/linux/include/tensorflow/cc/training/queue_runner.h b/linux/include/tensorflow/cc/training/queue_runner.h new file mode 100644 index 000000000..21189b4b0 --- /dev/null +++ b/linux/include/tensorflow/cc/training/queue_runner.h @@ -0,0 +1,140 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CC_TRAINING_QUEUE_RUNNER_H_ +#define TENSORFLOW_CC_TRAINING_QUEUE_RUNNER_H_ + +#include +#include +#include +#include + +#include "tensorflow/cc/training/coordinator.h" +#include "tensorflow/core/lib/core/blocking_counter.h" +#include "tensorflow/core/lib/core/error_codes.pb.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/lib/core/threadpool.h" +#include "tensorflow/core/platform/mutex.h" +#include "tensorflow/core/protobuf/config.pb.h" +#include "tensorflow/core/protobuf/queue_runner.pb.h" +#include "tensorflow/core/public/session.h" + +namespace tensorflow { + +/// QueueRunner class imitates the behavior of the python version of QueueRunner +/// which creates a thread for each enqueue op, runs close op on completion. +class QueueRunner : public RunnerInterface { + public: + /// Creates a new QueueRunner from proto. + // TODO(yuefengz): we may want to initialize from queues and ops in the + // future. + static Status New(const QueueRunnerDef& queue_runner_def, + std::unique_ptr* result); + + /// Creates a new QueueRunner with a coordinator, see coordinator.h for usage. + static Status New(const QueueRunnerDef& queue_runner_def, Coordinator* coord, + std::unique_ptr* result); + + /// Adds a callback that the queue runner will call when it detects an error. + void AddErrorCallback(const std::function& cb); + + /// Delete the previously registered callbacks. + void ClearErrorCallbacks(); + + /// The destructor would join all the threads. + ~QueueRunner(); + + /// Starts the queue runner with the given session. + Status Start(Session* sess); + + /// Starts the queue runner with the given session and sets the run arguments + /// for sess->Run. It also collects and stores the cost model. + Status StartAndCollectCostGraph(Session* sess, + const RunOptions& run_options = RunOptions()); + + /// Starts the queue runner with the given session, and wait for up to the + /// specified time (in milliseconds) for the queues to start to fill up. + Status Start(Session* sess, int wait_for_ms); + Status StartAndCollectCostGraph(Session* session, int wait_for_ms, + const RunOptions& run_options = RunOptions()); + + /// Requests to stop and runs the cancel op. It would be called in a separate + /// thread when coordinator is set. If there is no coordinator it should be + /// called before calling Join. + void Stop(Session* sess); + + /// Joins all the threads. Returns okay if all threads run successfully; + /// otherwise returns the first captured failure status. + Status Join() final; + + /// Returns the latest status. + Status GetStatus(); + + // Returns the stored cost model. + Status ExportCostGraph(CostGraphDef* cost_graph) const override; + + private: + QueueRunner() : coord_(nullptr), stopped_(false), cg_mu_(nullptr) {} + + // Initializes the instance with the QueueRunnerDef proto. + Status Init(const QueueRunnerDef& queue_runner_def); + + // The Run function for each thread. + void Run(Session* sess, const string& enqueue_op); + + // Updates the internal status; it only keeps OK or the first unexpected error + // status. + void UpdateStatus(const Status& status); + + bool IsQueueClosed(Status status) const { + return queue_closed_exception_types_.count( + static_cast(status.code())) > 0; + } + + bool IsRunning() const override { return !stopped_; } + + void SetRunArgumentsAndCostGraph(const RunOptions& run_options); + + Status RealRun(Session* sess, const string& op, bool update_costs); + + string queue_name_; + std::vector enqueue_op_names_; + string close_op_name_; + string cancel_op_name_; + // code::Code casted to int to avoid a hash function. + std::unordered_set queue_closed_exception_types_; + + std::unique_ptr thread_pool_; + mutex mu_; + int runs_ = 0; + Status status_ GUARDED_BY(mu_); + Status enqueue_status_ GUARDED_BY(mu_); + std::unique_ptr counter_; + + Coordinator* coord_; + + std::atomic stopped_; + + mutex cb_mu_; + std::vector> callbacks_; + + mutable std::unique_ptr cg_mu_; + std::unique_ptr cost_graph_ GUARDED_BY(cg_mu_); + RunOptions run_options_; +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CC_TRAINING_QUEUE_RUNNER_H_ diff --git a/linux/include/tensorflow/compiler/jit/build_xla_ops_pass.h b/linux/include/tensorflow/compiler/jit/build_xla_ops_pass.h new file mode 100644 index 000000000..1dd38fa95 --- /dev/null +++ b/linux/include/tensorflow/compiler/jit/build_xla_ops_pass.h @@ -0,0 +1,33 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_JIT_BUILD_XLA_OPS_PASS_H_ +#define TENSORFLOW_COMPILER_JIT_BUILD_XLA_OPS_PASS_H_ + +#include "tensorflow/core/common_runtime/optimization_registry.h" +#include "tensorflow/core/lib/core/status.h" + +namespace tensorflow { + +// Adds _XlaCompile and _XlaRun operations to the TF graph that compiles and +// executes (using XLA) TF function calls marked with "_XlaCompiledKernel". +class BuildXlaOpsPass : public GraphOptimizationPass { + public: + Status Run(const GraphOptimizationPassOptions& options) override; +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_JIT_BUILD_XLA_OPS_PASS_H_ diff --git a/linux/include/tensorflow/compiler/jit/deadness_analysis.h b/linux/include/tensorflow/compiler/jit/deadness_analysis.h new file mode 100644 index 000000000..6e7ab4116 --- /dev/null +++ b/linux/include/tensorflow/compiler/jit/deadness_analysis.h @@ -0,0 +1,68 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_JIT_DEADNESS_ANALYSIS_H_ +#define TENSORFLOW_COMPILER_JIT_DEADNESS_ANALYSIS_H_ + +#include "tensorflow/core/graph/graph.h" + +namespace tensorflow { + +// This analyzes a TensorFlow graph to identify nodes which may have partially +// dead inputs (i.e. these nodes may have some dead inputs and some alive +// inputs). +// +// For example, the ADD node in the following graph +// +// V0 PRED0 V1 PRED1 +// | | | | +// v v v v +// SWITCH SWITCH +// | | +// +---+ + ---+ +// | | +// v v +// ADD +// +// can have its inputs independently dead or alive based on the runtime values +// of PRED0 and PRED1. +// +// It is tempting to call this a liveness analysis but I avoided that because +// "liveness" already has other connotations. +class DeadnessAnalysis { + public: + // Returns true if `node` may have some live inputs and some dead inputs. + // + // This is a conservatively correct routine -- if it returns false then `node` + // is guaranteed to not have inputs with mismatching liveness, but not the + // converse. + // + // REQUIRES: node is not a Merge operation. + virtual bool HasInputsWithMismatchingDeadness(const Node& node) = 0; + + // Prints out the internal state of this instance. For debugging purposes + // only. + virtual void Print() const = 0; + virtual ~DeadnessAnalysis(); + + // Run the deadness analysis over `graph` and returns an error or a populated + // instance of DeadnessAnalysis in `result`. + static Status Run(const Graph& graph, + std::unique_ptr* result); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_JIT_DEADNESS_ANALYSIS_H_ diff --git a/linux/include/tensorflow/compiler/jit/deadness_analysis_internal.h b/linux/include/tensorflow/compiler/jit/deadness_analysis_internal.h new file mode 100644 index 000000000..354782374 --- /dev/null +++ b/linux/include/tensorflow/compiler/jit/deadness_analysis_internal.h @@ -0,0 +1,40 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_JIT_DEADNESS_ANALYSIS_INTERNAL_H_ +#define TENSORFLOW_COMPILER_JIT_DEADNESS_ANALYSIS_INTERNAL_H_ + +#include "absl/container/flat_hash_map.h" +#include "tensorflow/core/graph/tensor_id.h" + +namespace tensorflow { +namespace deadness_analysis_internal { + +// Returns a map describing the predicate each Tensor was mapped to. For +// testing purposes only. +using PredicateMapTy = absl::flat_hash_map; +Status ComputePredicates(const Graph& graph, PredicateMapTy* out_predicate_map); + +// Returns a map describing the predicate each Tensor was mapped to. For +// testing purposes only. Makes deadness analysis visit the graph in the order +// specified in `reverse_post_order` which must be a valid RPO for the graph +// minus NextIteration->Merge edges. +Status ComputePredicates(const Graph& graph, + absl::Span reverse_post_order, + PredicateMapTy* out_predicate_map); +} // namespace deadness_analysis_internal +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_JIT_DEADNESS_ANALYSIS_INTERNAL_H_ diff --git a/linux/include/tensorflow/compiler/jit/defs.h b/linux/include/tensorflow/compiler/jit/defs.h new file mode 100644 index 000000000..a3aabc949 --- /dev/null +++ b/linux/include/tensorflow/compiler/jit/defs.h @@ -0,0 +1,30 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// Provides definitions needed for use of the TensorFlow XLA +// device. + +#ifndef TENSORFLOW_COMPILER_JIT_DEFS_H_ +#define TENSORFLOW_COMPILER_JIT_DEFS_H_ + +namespace tensorflow { + +// Name of attribute used to tag operators for compilation with XLA +extern const char* const kXlaCompileAttr; // "_XlaCompile" +extern const char* const kXlaScopeAttr; // "_XlaScope" + +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_JIT_DEFS_H_ diff --git a/linux/include/tensorflow/compiler/jit/encapsulate_subgraphs_pass.h b/linux/include/tensorflow/compiler/jit/encapsulate_subgraphs_pass.h new file mode 100644 index 000000000..90354a801 --- /dev/null +++ b/linux/include/tensorflow/compiler/jit/encapsulate_subgraphs_pass.h @@ -0,0 +1,118 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// An optimization pass that groups nodes marked with a common +// kXlaClusterAttr into functions, and replaces the original nodes by +// calls. The calls are annotated with kXlaCompiledKernelAttr. + +#ifndef TENSORFLOW_COMPILER_JIT_ENCAPSULATE_SUBGRAPHS_PASS_H_ +#define TENSORFLOW_COMPILER_JIT_ENCAPSULATE_SUBGRAPHS_PASS_H_ + +#include "tensorflow/core/common_runtime/optimization_registry.h" +#include "tensorflow/core/framework/function.h" +#include "tensorflow/core/graph/graph.h" +#include "tensorflow/core/lib/core/status.h" + +namespace tensorflow { + +// A rewriting function to apply to each subgraph during encapsulation. +// 'arg_source_tensors' are the tensors corresponding to the arguments in the +// original source graph (*not* 'graph'). +// +// 'graph' is the subgraph. The rewriting may renumber the inputs and outputs; +// 'input_permutation' is a mapping from old argument numbers to new argument +// numbers, whereas 'output_permutation' is the same for outputs. Both +// 'input_permutation' and 'output_permutation' are initialized to the identity +// permutation. 'nodedef' is the NodeDef for the call to the function under +// construction, provided to allow additional attributes to be set. +// The rewrite may also change the NodeDef's operator name, and that +// name will be used as the name of the generated function. +typedef std::function& arg_source_tensors, + std::unique_ptr* graph, std::vector* input_permutation, + std::vector* output_permutation, NodeDef* node_def)> + RewriteSubgraphFn; + +// Transformation that finds subgraphs whose nodes are marked with +// 'group_attribute', splits those subgraphs into functions, and replaces +// the originals with function calls. +// +// 'group_attribute' must be a string valued-attribute that names the new +// functions to introduce. +// +// 'outside_compilation_attribute' must be a string-valued attribute that is +// used to tag nodes within a subgraph to be part of an 'outside_compilation' +// cluster within the subgraph. A cluster is formed from the set of nodes with +// the same value of outside_compilation_subgraph and group_attribute. The nodes +// in an outside_compilation cluster are left in the original graph. Edges +// crossing from the subgraph to an outside_compilation cluster nested in the +// subgraph are lifted into a SendToHost/RecvAtHost pair of nodes, and edges +// crossing from an outside_compilation cluster into its enclosing subgraph are +// lifted into a SendFromHost/RecvFromHost pair of nodes. +// +// If 'rewrite_subgraph_fn' is set, it is applied to each subgraph before +// function conversion. +// +// If 'reuse_existing_functions' is set, use an existing function with the +// same name, if any. +// +// TODO(phawkins): currently, some information in control edges +// is not preserved. Suppose you have A and B in the main +// graph, C and D in a subgraph. B and C have control deps from A, D has control +// dep from B. Originally D must run after C, post-transformation this +// dependency is lost. +Status EncapsulateSubgraphsInFunctions( + string group_attribute, string outside_compilation_attribute, + const Graph& graph_in, const RewriteSubgraphFn& rewrite_subgraph_fn, + bool reuse_existing_functions, std::unique_ptr* graph_out, + FunctionLibraryDefinition* library); + +// The attribute that marks function calls produced by the encapsulate +// subgraphs pass and that should in turn be compiled via XlaLaunch operators. +extern const char* const kXlaCompiledKernelAttr; + +// Does `node` have the kXlaCompiledKernelAttr attribute? +bool IsXlaCompiledKernel(const Node& node); + +// Functions produced by the EncapsulateSubgraphs pass have their arguments in +// the order: +// 1) compile-time constant arguments, in host memory, +// 2) other arguments, in device memory. +// 3) resource variable arguments, in host memory. Note that only the resource +// Tensor itself is in host memory; the underlying value may be in device +// memory. +// The functions are annotated with the following attributes that describe how +// many constant and resource arguments there are: + +// Name of the attribute containing the number of constant arguments. +extern const char* const kXlaNumConstantArgsAttr; + +// Name of the attribute containing the number of resource variable arguments. +extern const char* const kXlaNumResourceArgsAttr; + +// Sorts each node's control inputs by their names. This guarantees that for two +// structually equivalent GraphDefs, we get the same traversal ordering on +// node's control input fields. +// TODO(hpucha): Move the utilities to a more appropriate place. +void SortControlInputs(GraphDef* gdef); + +class EncapsulateSubgraphsPass : public GraphOptimizationPass { + public: + Status Run(const GraphOptimizationPassOptions& options) override; +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_JIT_ENCAPSULATE_SUBGRAPHS_PASS_H_ diff --git a/linux/include/tensorflow/compiler/jit/encapsulate_xla_computations_pass.h b/linux/include/tensorflow/compiler/jit/encapsulate_xla_computations_pass.h new file mode 100644 index 000000000..99e9dfd59 --- /dev/null +++ b/linux/include/tensorflow/compiler/jit/encapsulate_xla_computations_pass.h @@ -0,0 +1,60 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + ==============================================================================*/ +// Rewrites computations generated by the xla.compile() Python code into +// XlaLaunch nodes. +// +// xla.compile() does two main things: +// a) marks operators that make up an XLA computation with the attribute +// _xla_compile_id=XYZ, where XYZ is a unique key. +// b) adds XlaClusterOutput nodes to represent outputs of the computation. +// These nodes are not marked with the _xla_compile_id attribute. + +#ifndef TENSORFLOW_COMPILER_JIT_ENCAPSULATE_XLA_COMPUTATIONS_PASS_H_ +#define TENSORFLOW_COMPILER_JIT_ENCAPSULATE_XLA_COMPUTATIONS_PASS_H_ + +#include "tensorflow/core/common_runtime/optimization_registry.h" +#include "tensorflow/core/graph/graph.h" +#include "tensorflow/core/platform/env.h" + + namespace tensorflow { + +// Encapsulates nodes marked with the _xla_compile_id attribute into +// XlaLaunch operators. +class EncapsulateXlaComputationsPass : public GraphOptimizationPass { + public: + static const char* const kXlaClusterAttr; // _xla_compile_id + + Status Run(const GraphOptimizationPassOptions& options) override; + + // The following methods are public only for unit tests. + + // This pass has two stages: + // a) first, we call EncapsulateSubgraphsPass to encapsulate all nodes + // marked with the same _xla_compile_id attribute into functions. These + // functions contain the computations to be passed to XlaLaunch. During + // encapsulation, we sort the arguments into the order expected by + // XlaLaunch. + static Status Encapsulate(std::unique_ptr* graph, + FunctionLibraryDefinition* flib_def); + + // b) we rewrite the function calls generated in phase (a) into XlaLaunch + // operators. We also convert the XlaClusterOutput output nodes of the + // function call into the outputs of the XlaLaunch operator. + static Status BuildXlaLaunchOps(Graph* graph); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_JIT_ENCAPSULATE_XLA_COMPUTATIONS_PASS_H_ diff --git a/linux/include/tensorflow/compiler/jit/graphcycles/graphcycles.h b/linux/include/tensorflow/compiler/jit/graphcycles/graphcycles.h new file mode 100644 index 000000000..44448fa3d --- /dev/null +++ b/linux/include/tensorflow/compiler/jit/graphcycles/graphcycles.h @@ -0,0 +1,132 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_JIT_GRAPHCYCLES_GRAPHCYCLES_H_ +#define TENSORFLOW_COMPILER_JIT_GRAPHCYCLES_GRAPHCYCLES_H_ + +// GraphCycles detects the introduction of a cycle into a directed +// graph that is being built up incrementally. +// +// Nodes are identified by small integers. It is not possible to +// record multiple edges with the same (source, destination) pair; +// requests to add an edge where one already exists are silently +// ignored. +// +// It is also not possible to introduce a cycle; an attempt to insert +// an edge that would introduce a cycle fails and returns false. +// +// GraphCycles uses no internal locking; calls into it should be +// serialized externally. + +// Performance considerations: +// Works well on sparse graphs, poorly on dense graphs. +// Extra information is maintained incrementally to detect cycles quickly. +// InsertEdge() is very fast when the edge already exists, and reasonably fast +// otherwise. +// FindPath() is linear in the size of the graph. +// The current implementation uses O(|V|+|E|) space. + +#include + +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +// NOTE!!! +// For now a copy of this is forked to net/plaque. If you +// find a bug or add a feature, please inform the owners of the +// net/plaque copy in case it should be integrated. +// NOTE!!! +class GraphCycles { + public: + GraphCycles(); + ~GraphCycles(); + + // Allocate an unused node id and return it. + // The new node has a null pointer for its node data. + // All node identifiers passed to other routines in this interface + // must have been allocated by NewNode() and not yet deallocated + // by RemoveNode(). + int32 NewNode(); + + // Remove "node" from the graph, deleting all edges to and from it. + // After this call the identifier "node" it may no longer be used + // as an argument to any routine until it has been reallocated with + // NewNode(). + void RemoveNode(int32 node); + + // Attempt to insert an edge from source_node to dest_node. If the + // edge would introduce a cycle, return false without making any + // changes. Otherwise add the edge and return true. + bool InsertEdge(int32 source_node, int32 dest_node); + + // Remove any edge that exists from source_node to dest_node. + void RemoveEdge(int32 source_node, int32 dest_node); + + // Return whether there is an edge directly from source_node to dest_node. + bool HasEdge(int32 source_node, int32 dest_node) const; + + // Contracts the edge from 'a' to node 'b', merging nodes 'a' and 'b'. 'b' is + // removed from the graph, and edges to/from 'b' are replaced with edges + // to/from 'a'. If contracting the edge would create a cycle, does nothing + // and returns false. + bool ContractEdge(int32 a, int32 b); + + // Return true if can contract edge, otherwise return false. + bool CanContractEdge(int32 a, int32 b); + + // Return whether dest_node is reachable from source_node + // by following edges. + bool IsReachable(int32 source_node, int32 dest_node) const; + + // A faster non-thread-safe version of IsReachable. + bool IsReachableNonConst(int32 source_node, int32 dest_node); + + // Return or set the node data for a node. This data is unused + // by the implementation. + void *GetNodeData(int32 node) const; + void SetNodeData(int32 node, void *data); + + // Find a path from "source" to "dest". If such a path exists, place the + // node IDs of the nodes on the path in the array path[], and return the + // number of nodes on the path. If the path is longer than max_path_len + // nodes, only the first max_path_len nodes are placed in path[]. The client + // should compare the return value with max_path_len" to see when this + // occurs. If no path exists, return 0. Any valid path stored in path[] + // will start with "source" and end with "dest". There is no guarantee that + // the path is the shortest, but no node will appear twice in the path, + // except the source and destination node if they are identical; therefore, + // the return value is at most one greater than the number of nodes in the + // graph. + int FindPath(int32 source, int32 dest, int max_path_len, int32 path[]) const; + + // Check internal invariants. Crashes on failure, returns true on success. + // Expensive: should only be called from graphcycles_test.cc. + bool CheckInvariants() const; + + std::unordered_set Successors(int32 node); + std::unordered_set Predecessors(int32 node); + + // ---------------------------------------------------- + struct Rep; + + private: + Rep *rep_; // opaque representation + TF_DISALLOW_COPY_AND_ASSIGN(GraphCycles); +}; + +} // namespace tensorflow +#endif // TENSORFLOW_COMPILER_JIT_GRAPHCYCLES_GRAPHCYCLES_H_ diff --git a/linux/include/tensorflow/compiler/jit/kernels/xla_ops.h b/linux/include/tensorflow/compiler/jit/kernels/xla_ops.h new file mode 100644 index 000000000..489d26eb3 --- /dev/null +++ b/linux/include/tensorflow/compiler/jit/kernels/xla_ops.h @@ -0,0 +1,168 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_JIT_KERNELS_XLA_OPS_H_ +#define TENSORFLOW_COMPILER_JIT_KERNELS_XLA_OPS_H_ + +#include "tensorflow/compiler/jit/xla_compilation_cache.h" +#include "tensorflow/compiler/jit/xla_device.h" +#include "tensorflow/compiler/jit/xla_launch_util.h" +#include "tensorflow/core/framework/allocator.h" +#include "tensorflow/core/framework/op.h" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/util/stream_executor_util.h" + +namespace tensorflow { + +// Holds some information about the platform on which an +// XlaLaunch/_XlaCompile/_XlaRun op must run on. +class XlaPlatformInfo { + public: + XlaPlatformInfo() : device_type_("") {} + explicit XlaPlatformInfo(const DeviceType device_type, + se::Platform::Id platform_id, + const XlaDevice::Metadata* xla_device_metadata, + std::unique_ptr xla_allocator, + xla::DeviceMemoryAllocator* device_allocator) + : device_type_(device_type), + platform_id_(platform_id), + xla_device_metadata_(xla_device_metadata), + xla_allocator_(std::move(xla_allocator)), + device_allocator_(device_allocator) { + CHECK((device_allocator_ != nullptr) ^ (xla_allocator_.get() != nullptr)); + } + + XlaPlatformInfo& operator=(XlaPlatformInfo&& other) = default; + + bool UseMultipleStreams() const { + return xla_device_metadata_ && xla_device_metadata_->UseMultipleStreams(); + } + + xla::DeviceMemoryAllocator* allocator() const { + return device_allocator_ ? device_allocator_ : xla_allocator_.get(); + } + DeviceType device_type() const { return device_type_; } + + // This is equal to xla_device_metadata()->platform()->id() if + // xla_device_metadata() is not nullptr. + se::Platform::Id platform_id() const { return platform_id_; } + + // This may be null if the op this XlaPlatformInfo is for was not placed on an + // XLA device. + const XlaDevice::Metadata* xla_device_metadata() const { + return xla_device_metadata_; + } + bool is_on_xla_device() const { return xla_device_metadata() != nullptr; } + + private: + DeviceType device_type_; + se::Platform::Id platform_id_; + + // xla_device_metadata_ lives in the tensorflow::DeviceBase in which the + // XlaLaunch/_XlaCompile/_XlaRun op is placed and thus does not die before the + // XlaLaunch/_XlaCompile/_XlaRun OpKernel. + const XlaDevice::Metadata* xla_device_metadata_; + + // If the op associated with this XlaPlatformInfo is placed on an XLA device + // then device_allocator_ is the xla::Backend's memory allocator and + // xla_allocator_ is null. If the op is placed on a regular CPU or GPU device + // then device_allocator_ is null and xla_allocator_ points to an appropriate + // XlaAllocator instance. + std::unique_ptr xla_allocator_; + xla::DeviceMemoryAllocator* device_allocator_; + + TF_DISALLOW_COPY_AND_ASSIGN(XlaPlatformInfo); +}; + +// XlaLocalLaunchBase is almost the same as XlaLocalLaunchOp. +// The only difference is that it does not require arguments to follow +// the "constants, then regular args, then resources" order. +// It takes vectors of constant and resource arguments explicitly. +// It does not have corresponding OpDef because it is never present +// in the GraphDef. +// Currently, it is used by eager runtime. FunctionLibraryRuntime creates +// this kernel when asked to create a kernel for an XLA-compiled function. +class XlaLocalLaunchBase : public OpKernel { + public: + XlaLocalLaunchBase(OpKernelConstruction* ctx, + const std::vector& constants, + const std::vector& resources, + const NameAttrList& function); + XlaLocalLaunchBase(const XlaLocalLaunchBase&) = delete; + XlaLocalLaunchBase& operator=(const XlaLocalLaunchBase&) = delete; + ~XlaLocalLaunchBase() override = default; + + void Compute(OpKernelContext* ctx) override; + + protected: + // Indexes of compile-time constant inputs + std::vector constants_; + // Indexes of resource inputs + std::vector resources_; + + NameAttrList function_; + XlaPlatformInfo platform_info_; +}; + +// XlaLocalLaunchOp is used to replace a region of the TensorFlow graph +// which will be compiled and executed using XLA. The XlaLocalLaunchOp is +// responsible for handling interactions with the TensorFlow executor. +// Once all inputs are present, and their shapes are known, the op can +// use a 'XlaCompilationCache' to compile and execute code which is specific +// to the shapes of input Tensors. +// XlaLocalLaunchOp uses xla::LocalClient::Compile() and +// xla::LocalExecutable::Run(), and passes arguments into/out of XLA in device +// memory. +class XlaLocalLaunchOp : public XlaLocalLaunchBase { + public: + explicit XlaLocalLaunchOp(OpKernelConstruction* ctx); + ~XlaLocalLaunchOp() override; + + private: + TF_DISALLOW_COPY_AND_ASSIGN(XlaLocalLaunchOp); +}; + +class XlaCompileOp : public OpKernel { + public: + explicit XlaCompileOp(OpKernelConstruction* ctx); + + void Compute(OpKernelContext* ctx) override; + + private: + // Indexes of compile-time constant inputs + std::vector constants_; + // Indexes of resource inputs + std::vector resources_; + + NameAttrList function_; + + XlaPlatformInfo platform_info_; +}; + +class XlaRunOp : public OpKernel { + public: + explicit XlaRunOp(OpKernelConstruction* ctx); + + void Compute(OpKernelContext* ctx) override; + + private: + XlaPlatformInfo platform_info_; +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_JIT_KERNELS_XLA_LAUNCH_OP_H_ diff --git a/linux/include/tensorflow/compiler/jit/legacy_flags/mark_for_compilation_pass_flags.h b/linux/include/tensorflow/compiler/jit/legacy_flags/mark_for_compilation_pass_flags.h new file mode 100644 index 000000000..2affda6ab --- /dev/null +++ b/linux/include/tensorflow/compiler/jit/legacy_flags/mark_for_compilation_pass_flags.h @@ -0,0 +1,68 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_JIT_LEGACY_FLAGS_MARK_FOR_COMPILATION_PASS_FLAGS_H_ +#define TENSORFLOW_COMPILER_JIT_LEGACY_FLAGS_MARK_FOR_COMPILATION_PASS_FLAGS_H_ + +// Legacy flags for the XLA bridge's mark_for_compilation_pass module. + +#include + +#include "tensorflow/core/platform/types.h" +#include "tensorflow/core/util/command_line_flags.h" + +namespace tensorflow { +namespace legacy_flags { + +// Append to *flag_list flag definitions associated with the XLA bridge's +// mark_for_compilation_pass module. +void AppendMarkForCompilationPassFlags( + std::vector* flag_list); + +// The values of flags associated with the XLA bridge's +// mark_for_compilation_pass module. +typedef struct { + int32 tf_xla_auto_jit; // Control compilation of operators into XLA + // computations on CPU and GPU devices. 0 = use + // ConfigProto setting; -1 = off; 1 = on for things + // very likely to be improved; 2 = on for everything. + // Experimental. + int32 tf_xla_min_cluster_size; // Minimum number of operators in an XLA + // compilation. Ignored for operators placed + // on an XLA device or operators explicitly + // marked for compilation. + int32 tf_xla_max_cluster_size; // Maximum number of operators in an XLA + // compilation. + bool tf_xla_clustering_debug; // Dump graphs during XLA compilation. + bool tf_xla_cpu_global_jit; // Enables global JIT compilation for CPU + // via SessionOptions. + int64 tf_xla_clustering_fuel; // "Compiler fuel" for clustering. Only this + // many ops will be marked as eligible for + // clustering. + bool tf_xla_fusion_only; // This flag is effective only when global_jit_level + // is set to ON* and overrides its behavior. If + // true, enable fusion of element-wise operations + // only using XLA. +} MarkForCompilationPassFlags; + +// Return a pointer to the MarkForCompilationPassFlags struct; +// repeated calls return the same pointer. +// This should be called only after Flags::Parse() has returned. +MarkForCompilationPassFlags* GetMarkForCompilationPassFlags(); + +} // namespace legacy_flags +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_JIT_LEGACY_FLAGS_MARK_FOR_COMPILATION_PASS_FLAGS_H_ diff --git a/linux/include/tensorflow/compiler/jit/legacy_flags/xla_device_flags.h b/linux/include/tensorflow/compiler/jit/legacy_flags/xla_device_flags.h new file mode 100644 index 000000000..27b22121a --- /dev/null +++ b/linux/include/tensorflow/compiler/jit/legacy_flags/xla_device_flags.h @@ -0,0 +1,47 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_JIT_LEGACY_FLAGS_XLA_DEVICE_FLAGS_H_ +#define TENSORFLOW_COMPILER_JIT_LEGACY_FLAGS_XLA_DEVICE_FLAGS_H_ + +// Legacy flags for the XLA bridge's xla_device module. + +#include + +#include "tensorflow/core/platform/types.h" +#include "tensorflow/core/util/command_line_flags.h" + +namespace tensorflow { +namespace legacy_flags { + +// The values of flags associated with the XLA bridge's +// xla_device module. +typedef struct { + // Switch the CPU device into "on-demand" mode, where instead of + // autoclustering ops are compiled one by one just-in-time. + // Enabling this mode by a legacy flag is a temporary mechanism. When this + // feature is battle-tested, we will switch this to be a session option. + bool tf_xla_compile_on_demand; +} XlaDeviceFlags; + +// Return a pointer to the XlaDeviceFlags struct; +// repeated calls return the same pointer. +// This should be called only after Flags::Parse() has returned. +XlaDeviceFlags* GetXlaDeviceFlags(); + +} // namespace legacy_flags +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_JIT_LEGACY_FLAGS_XLA_DEVICE_FLAGS_H_ diff --git a/linux/include/tensorflow/compiler/jit/mark_for_compilation_pass.h b/linux/include/tensorflow/compiler/jit/mark_for_compilation_pass.h new file mode 100644 index 000000000..f1137af3c --- /dev/null +++ b/linux/include/tensorflow/compiler/jit/mark_for_compilation_pass.h @@ -0,0 +1,57 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// An optimization passes that marks nodes that are to be compiled with +// attribute kXlaClusterAttr. Nodes with the same cluster ID will be compiled +// together. + +#ifndef TENSORFLOW_COMPILER_JIT_MARK_FOR_COMPILATION_PASS_H_ +#define TENSORFLOW_COMPILER_JIT_MARK_FOR_COMPILATION_PASS_H_ + +#include "tensorflow/core/common_runtime/optimization_registry.h" + +namespace tensorflow { + +// The attribute that marks nodes to be grouped into functions by the +// encapsulate subgraphs pass. +extern const char* const kXlaClusterAttr; + +// The attribute that marks nodes in a cluster to be placed outside the xla +// compilation by the encapsulate subgraphs pass. +extern const char* const kXlaOutsideCompilationAttr; + +// Pass that marks a subset of operators in the graph with attribute +// _XlaCluster so they are compiled by the EncapsulateSubgraphsPass. +class MarkForCompilationPass : public GraphOptimizationPass { + public: + MarkForCompilationPass() = default; + + Status Run(const GraphOptimizationPassOptions& options) override; + + private: + Status RunImpl(const GraphOptimizationPassOptions& options, + const std::function& + is_compilable_fn = {}); + + friend class MarkForCompilationPassTestHelper; +}; + +// Returns true iff 'ndef' is a call to a function that is compilable. A +// function is compilable iff every operator in the function body is +// compilable. +bool IsCompilable(FunctionLibraryRuntime* flr, const NodeDef& ndef); +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_JIT_MARK_FOR_COMPILATION_PASS_H_ diff --git a/linux/include/tensorflow/compiler/jit/mark_for_compilation_pass_test_helper.h b/linux/include/tensorflow/compiler/jit/mark_for_compilation_pass_test_helper.h new file mode 100644 index 000000000..216baaf93 --- /dev/null +++ b/linux/include/tensorflow/compiler/jit/mark_for_compilation_pass_test_helper.h @@ -0,0 +1,40 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_JIT_MARK_FOR_COMPILATION_PASS_TEST_HELPER_H_ +#define TENSORFLOW_COMPILER_JIT_MARK_FOR_COMPILATION_PASS_TEST_HELPER_H_ + +#include "tensorflow/compiler/jit/mark_for_compilation_pass.h" + +namespace tensorflow { +class MarkForCompilationPassTestHelper { + public: + // Runs the MarkForCompilation pass on `graph` after assigning all nodes in + // `graph` to the CPU device. To make testing easier, ignores device + // registration, _XlaCompile attributes, input deadness and global jit level. + static Status MarkForCompilation(std::unique_ptr* graph, + FunctionLibraryDefinition* flib_def, + SessionOptions* session_options); + + // Like `MarkForCompilation` but creates a default SessionOptions. + static Status MarkForCompilation(std::unique_ptr* graph, + FunctionLibraryDefinition* flib_def); + + // Like `MarkForCompilation` but creates `flib_def` from the op registry. + static Status MarkForCompilation(std::unique_ptr* graph); +}; +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_JIT_MARK_FOR_COMPILATION_PASS_TEST_HELPER_H_ diff --git a/linux/include/tensorflow/compiler/jit/partially_decluster_pass.h b/linux/include/tensorflow/compiler/jit/partially_decluster_pass.h new file mode 100644 index 000000000..cfc4ddb56 --- /dev/null +++ b/linux/include/tensorflow/compiler/jit/partially_decluster_pass.h @@ -0,0 +1,35 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_JIT_PARTIALLY_DECLUSTER_PASS_H_ +#define TENSORFLOW_COMPILER_JIT_PARTIALLY_DECLUSTER_PASS_H_ + +#include "tensorflow/core/common_runtime/optimization_registry.h" + +namespace tensorflow { + +// Clones or moves nodes from within a cluster to outside the cluster if +// profitable. There are two reasons why we do this: +// +// - Reducing device-to-host copies. +// - Reducing the number of XLA recompilations. +class PartiallyDeclusterPass : public GraphOptimizationPass { + public: + Status Run(const GraphOptimizationPassOptions& options) override; +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_JIT_PARTIALLY_DECLUSTER_PASS_H_ diff --git a/linux/include/tensorflow/compiler/jit/resource_operation_safety_analysis.h b/linux/include/tensorflow/compiler/jit/resource_operation_safety_analysis.h new file mode 100644 index 000000000..ae8cfeeca --- /dev/null +++ b/linux/include/tensorflow/compiler/jit/resource_operation_safety_analysis.h @@ -0,0 +1,73 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_JIT_RESOURCE_OPERATION_SAFETY_ANALYSIS_H_ +#define TENSORFLOW_COMPILER_JIT_RESOURCE_OPERATION_SAFETY_ANALYSIS_H_ + +#include "tensorflow/compiler/jit/graphcycles/graphcycles.h" +#include "tensorflow/core/framework/function.h" +#include "tensorflow/core/graph/graph.h" + +namespace tensorflow { +// An XLA cluster hoists all resource reads to be beginning of the cluster +// execution and all the resource writes to the end. This means it cannot +// enforce arbitrary ordering dependencies (via control or data edges) between +// resource operations. Since all resource reads happen before all resource +// writes, edges constraining resource reads to happen before resource writes +// are fine, but all other kinds of edges are problematic. This analysis +// returns the set of pairs of resource operations that cannot be put in the +// same cluster because XLA cannot respect the dependencies between them in the +// TensorFlow program. +// +// The restrictions are not transitive: it is fine to put A and C in the same +// cluster even if the returned set contains (A,B) and (B,C). +// +// In other words, if these pairs are seen as edges in an undirected graph of +// the nodes in `g` then auto-clustering is at least as constrained as the graph +// coloring problem on this graph. +// +// +// For instance if we auto-cluster all operations in this TensorFlow graph: +// +// ReadVariablepOp0 -> ReadVariableOp1 +// | +// v +// AssignVariableOp0 -> AssignVariableOp1 +// +// we will lose the ReadVariablepOp0 -> ReadVariableOp1 and the +// AssignVariableOp0 -> AssignVariableOp1 dependencies. I.e. it is possible for +// XlaLaunchOp to issue ReadVariableOp1 before ReadVariablepOp0 since it reads +// all the resource variables when the cluster starts executing without any +// particular ordering between them; same holds for the AssignVariableOp0 -> +// AssignVariableOp1 edge. The ReadVariableOp1 -> AssignVariableOp0 edge will +// be respected by XlaLaunchOp though because all reads happen before all +// writes. +// +// +// NB! The result computed by this analysis assumes that we don't auto-cluster +// back-edges (i.e. the edges from NextIteration to Merge). +// +// NB! The result computed by this analysis assumes that we don't auto-cluster +// functional control flow nodes containing resource operations. +// +// If `resource_ops_to_ignore` is set then nodes for which it returns true are +// ignored (we pretend these nodes are not resource operations). +Status ComputeIncompatibleResourceOperationPairs( + const Graph& g, const FunctionLibraryDefinition* flib_def, + const std::function& resource_ops_to_ignore, + std::vector>* result); +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_JIT_RESOURCE_OPERATION_SAFETY_ANALYSIS_H_ diff --git a/linux/include/tensorflow/compiler/jit/shape_inference_helpers.h b/linux/include/tensorflow/compiler/jit/shape_inference_helpers.h new file mode 100644 index 000000000..2f053c9a4 --- /dev/null +++ b/linux/include/tensorflow/compiler/jit/shape_inference_helpers.h @@ -0,0 +1,65 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_JIT_SHAPE_INFERENCE_HELPERS_H_ +#define TENSORFLOW_COMPILER_JIT_SHAPE_INFERENCE_HELPERS_H_ + +#include + +#include "tensorflow/core/graph/graph.h" + +namespace tensorflow { + +// Helper class to temporarily remove, then replace, the back edges in a +// graph. Simple algorithms for shape inference don't work with cycles, and this +// class can be used to remove cycles before running inference and replace them +// after. Correct usage requires exactly one call to Remove(), followed by any +// number of calls to RemovedEdges() and at most one call to Replace(). The call +// to Replace() is optional if the graph will be discarded without being +// executed, e.g., if it is being used purely for a shape inference pass. +class BackEdgeHelper { + public: + struct BackEdge { + const Edge* edge; + Node* src; + int src_output; + Node* dst; + int dst_input; + }; + + BackEdgeHelper() = default; + // Disallows copy and assign. + BackEdgeHelper(const BackEdgeHelper& other) = delete; + BackEdgeHelper& operator=(const BackEdgeHelper& other) = delete; + + // Temporarily removes all the back edges in graph. + Status Remove(Graph* graph); + + // Gets the list of removed edges. + const std::vector& RemovedEdges() const; + + // Replaces the back edges removed by a prior call to Remove. + Status Replace(); + + private: + Graph* graph_ = nullptr; // not owned + std::vector back_edges_; + // Set once Replace has been called. + bool replaced_ = false; +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_JIT_SHAPE_INFERENCE_HELPERS_H_ diff --git a/linux/include/tensorflow/compiler/jit/union_find.h b/linux/include/tensorflow/compiler/jit/union_find.h new file mode 100644 index 000000000..a1a7a6a4d --- /dev/null +++ b/linux/include/tensorflow/compiler/jit/union_find.h @@ -0,0 +1,81 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_JIT_UNION_FIND_H_ +#define TENSORFLOW_COMPILER_JIT_UNION_FIND_H_ + +namespace tensorflow { + +// Union-Find data structure. +// Each cluster has an associated value; when merging clusters we can control +// which value becomes the representative of the merged clusters. Values must be +// copyable. +template +class UnionFind { + public: + UnionFind() : rank_(0), size_(1), parent_(nullptr) {} + + // Returns the number of elements in a cluster. + int Size() { return FindRoot()->size_; } + + // Merges this cluster with 'other'. This cluster's value becomes + // the value of the merged cluster; the value of 'other' is ignored. + void Merge(UnionFind* other); + + // Each cluster has an associated value. Retrieves the value associated + // with this cluster. + T& Get() { return FindRoot()->value_; } + + private: + // Finds the root element of the cluster. Performs path compression. + UnionFind* FindRoot(); + + int rank_; + int size_; // Size of the cluster. + UnionFind* parent_; + T value_; +}; + +template +void UnionFind::Merge(UnionFind* other) { + UnionFind* a = FindRoot(); + UnionFind* b = other->FindRoot(); + if (a == b) return; + if (a->rank_ > b->rank_) { + b->parent_ = a; + a->size_ += b->size_; + return; + } + + a->parent_ = b; + if (a->rank_ == b->rank_) { + b->rank_++; + } + b->value_ = a->value_; + b->size_ += a->size_; +} + +template +UnionFind* UnionFind::FindRoot() { + if (!parent_) return this; + // Path compression: update intermediate nodes to point to the root of the + // equivalence class. + parent_ = parent_->FindRoot(); + return parent_; +} + +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_JIT_UNION_FIND_H_ diff --git a/linux/include/tensorflow/compiler/jit/xla_cluster_util.h b/linux/include/tensorflow/compiler/jit/xla_cluster_util.h new file mode 100644 index 000000000..ba218f331 --- /dev/null +++ b/linux/include/tensorflow/compiler/jit/xla_cluster_util.h @@ -0,0 +1,70 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// Contains utilities for clustering compilable graph nodes via XLA. + +#ifndef TENSORFLOW_COMPILER_JIT_XLA_CLUSTER_UTIL_H_ +#define TENSORFLOW_COMPILER_JIT_XLA_CLUSTER_UTIL_H_ + +#include "absl/types/optional.h" +#include "tensorflow/compiler/jit/graphcycles/graphcycles.h" +#include "tensorflow/core/graph/algorithm.h" + +namespace tensorflow { + +// The attribute that marks nodes to be grouped into functions by the +// encapsulate subgraphs pass. +extern const char* const kXlaClusterAttr; + +// The attribute that marks nodes in a cluster to be placed outside the xla +// compilation by the encapsulate subgraphs pass. +extern const char* const kXlaOutsideCompilationAttr; + +using OrderedNodeSet = std::set; + +// Returns the DeviceType corresponding to 'device'. +Status DeviceToDeviceType(const string& device, DeviceType* device_type); + +// Returns true if `node` has a ref tensor input that it forwards to its output. +bool HasForwardedRefInput(const Node& node); + +// Creates a graph representation to enable cycle detection when clustering. +// This representation handles loops in graph by disconnecting each loop from +// the enclosing graph. +Status CreateCycleDetectionGraph(const Graph* graph, GraphCycles* cycles); + +// Returns the XLA cluster in which `node` is placed if it is in an XLA cluster, +// otherwise returns nullopt. +absl::optional GetXlaClusterForNode(const Node& node); + +// Removes `node_def` its XLA cluster (by clearing its _XlaCluster attribute). +void RemoveFromXlaCluster(NodeDef* node_def); + +// Removes `node` its XLA cluster (by clearing its _XlaCluster attribute). +void RemoveFromXlaCluster(Node* node); + +// Returns true if `node` has a DT_RESOURCE typed input or output. +bool HasResourceInputOrOutput(const Node& node); + +// Adds edges to `cycles` to prevent clustering resource operations that cannot +// be legally clustered. +Status AdjustCycleDetectionGraphForResourceOps( + const Graph* graph, const FunctionLibraryDefinition* flib_def, + const std::function& resource_ops_to_ignore, + GraphCycles* cycles); + +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_JIT_XLA_CLUSTER_UTIL_H_ diff --git a/linux/include/tensorflow/compiler/jit/xla_compilation_cache.h b/linux/include/tensorflow/compiler/jit/xla_compilation_cache.h new file mode 100644 index 000000000..17c0321c1 --- /dev/null +++ b/linux/include/tensorflow/compiler/jit/xla_compilation_cache.h @@ -0,0 +1,176 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_JIT_XLA_COMPILATION_CACHE_H_ +#define TENSORFLOW_COMPILER_JIT_XLA_COMPILATION_CACHE_H_ + +#include "absl/container/flat_hash_map.h" +#include "tensorflow/compiler/tf2xla/xla_compiler.h" +#include "tensorflow/compiler/tf2xla/xla_context.h" +#include "tensorflow/compiler/xla/client/local_client.h" +#include "tensorflow/core/common_runtime/device.h" +#include "tensorflow/core/common_runtime/device_mgr.h" +#include "tensorflow/core/framework/graph.pb.h" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/lib/core/threadpool.h" +#include "tensorflow/core/platform/mutex.h" +#include "tensorflow/core/platform/thread_annotations.h" + +namespace tensorflow { + +// Struct that represents a possibly-absent Tensor. +struct OptionalTensor { + string name; // A descriptive name + bool present = false; // Is the tensor present? + Tensor value; // If present, what is the Tensor's value? +}; + +// The XlaCompilationCache class caches the results of the XlaCompiler class, +// which converts a Tensorflow graph into a compiled XLA compilation. +// +// Since XLA computations must have static shapes, the cache generates a new +// XLA computation for each new set of input shapes. +// +// Currently no cache eviction policy is implemented and the cache grows without +// bound. +class XlaCompilationCache : public ResourceBase { + public: + XlaCompilationCache(xla::LocalClient* client, DeviceType device_type); + ~XlaCompilationCache() override; + + // Compiles a function into a XlaCompiler::CompilationResult that can be used + // to execute an XLA Computation. Compilation results are cached. + // `function` is the name of a Tensorflow function to compile. + // `constant_args` is a map of tensorflow argument number to its constant + // value. + // `variable_args` is a snapshot of the current values of the + // resource variable arguments to `function`; uninitialized variables are + // represented by an absent OptionalTensor. + // The result of compilation is written to `*compilation_result`, which must + // be non-null. If `executable` is non-null, also builds an + // xla::LocalExecutable and sets `executable` to point to it. The resulting + // executable pointer may be null if the computation has no non-constant + // outputs. + Status Compile(const XlaCompiler::Options& options, + const NameAttrList& function, + const std::map& constant_args, + const std::map& variable_args, + OpKernelContext* ctx, + const XlaCompiler::CompilationResult** compilation_result, + xla::LocalExecutable** executable, + const XlaCompiler::CompileOptions& compile_options); + + // As above, but calls XlaCompiler::CompileSingleOp instead of + // XlaCompiler::CompileFunction. + Status CompileSingleOp( + const XlaCompiler::Options& options, + const std::map& constant_args, + const std::map& variable_args, OpKernelContext* ctx, + const XlaCompiler::CompilationResult** compilation_result, + xla::LocalExecutable** executable, + const XlaCompiler::CompileOptions& compile_options); + + xla::LocalClient* client() const { return client_; } + const DeviceType& device_type() const { return device_type_; } + + string DebugString() override; + + private: + // Common implementation of Compile and CompileSingleOp. + Status CompileImpl(const XlaCompiler::Options& options, + const NameAttrList& function, + const std::map& constant_args, + const std::map& variable_args, + OpKernelContext* ctx, + const XlaCompiler::CompilationResult** compilation_result, + xla::LocalExecutable** executable, + const XlaCompiler::CompileOptions& compile_options, + bool compile_single_op); + + // Takes `result` which has been compiled from a Tensorflow subgraph to a + // XLA computation already, and generates an XLA LocalExecutable `executable`. + Status BuildExecutable(const XlaCompiler::Options& options, + const XlaCompiler::CompilationResult& result, + std::unique_ptr* executable); + + xla::LocalClient* const client_; + const DeviceType device_type_; + + // Describes the types, shapes and any compile-time constant arguments + // to a kernel. Key that uniquely identifies a compilation output. + struct Signature { + string name; + + std::vector> arg_types; + + // List of Tensor values for compile-time constant arguments to the + // compilation, ordered by argument number. Tensors must be in host memory. + std::vector arg_values; + + bool operator==(const Signature& other) const; + + struct Hash { + uint64 operator()(const Signature& signature) const; + }; + }; + static string SignatureDebugString(const Signature& sig); + + // Builds the signature for a compilation. + Status BuildSignature(const NameAttrList& function, + const std::map& constant_args, + const std::map& variable_args, + OpKernelContext* ctx, Signature* signature); + + // The value associated with a cache entry. + struct Entry { + mutex mu; + + // Have we tried compiling this entry? + bool compiled = false; + + // Did compilation succeed? + Status compilation_status GUARDED_BY(mu); + + // Output of the XlaCompiler. + XlaCompiler::CompilationResult compilation_result GUARDED_BY(mu); + + // The XLA executable compiled from . May be null if no + // executable has been built. + std::unique_ptr executable GUARDED_BY(mu); + }; + + mutex compile_cache_mu_; + absl::flat_hash_map, Signature::Hash> cache_ + GUARDED_BY(compile_cache_mu_); + + struct CompileStats { + // Number of times the cluster has been (re-)compiled. + int64 compile_count = 0; + + // Cumulative time spent compiling the cluster. + int64 cumulative_compile_time_us = 0; + }; + mutex compile_stats_mu_; + + // Maps cluster names to compilation statistics for said cluster. + absl::flat_hash_map compile_stats_ + GUARDED_BY(compile_stats_mu_); + + TF_DISALLOW_COPY_AND_ASSIGN(XlaCompilationCache); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_JIT_XLA_COMPILATION_CACHE_H_ diff --git a/linux/include/tensorflow/compiler/jit/xla_compile_on_demand_op.h b/linux/include/tensorflow/compiler/jit/xla_compile_on_demand_op.h new file mode 100644 index 000000000..7cc3d0e00 --- /dev/null +++ b/linux/include/tensorflow/compiler/jit/xla_compile_on_demand_op.h @@ -0,0 +1,53 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// The XlaCompileOnDemandOp is an OpKernel that, when its Compute method is +// called, will generate an xla::Computation and run it asynchronously. + +#ifndef TENSORFLOW_COMPILER_JIT_XLA_COMPILE_ON_DEMAND_OP_H_ +#define TENSORFLOW_COMPILER_JIT_XLA_COMPILE_ON_DEMAND_OP_H_ + +#include "tensorflow/compiler/jit/xla_device.h" +#include "tensorflow/compiler/tf2xla/xla_compiler.h" +#include "tensorflow/compiler/xla/client/local_client.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/core/status.h" + +namespace tensorflow { + +// An OpKernel that compiles an op to an XLA computation and runs it. Unlike +// XlaLaunch this doesn't rely on any rewrites of the graphdef - it will run a +// vanilla TensorFlow op as long as the bridge supports it. +class XlaCompileOnDemandOp : public OpKernel { + public: + explicit XlaCompileOnDemandOp(OpKernelConstruction* ctx) : OpKernel(ctx) {} + void Compute(OpKernelContext* ctx) override; + + private: + XlaCompiler::Argument CreateCompilerArgument(OpKernelContext* ctx, int64 i); + bool ShouldArgumentBeConstant(const OpKernel* op_kernel, int64 argument_idx); + bool MustArgumentBeConstant(const OpKernel* op_kernel, int64 argument_idx); + Status Compile(OpKernelContext* ctx, const XlaDevice::Metadata& metadata, + const XlaCompiler::CompilationResult** result, + xla::LocalExecutable** executable); + Status Run(OpKernelContext* ctx, const XlaDevice::Metadata& metadata, + const XlaCompiler::CompilationResult* result, + xla::LocalExecutable* executable); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_JIT_XLA_COMPILE_ON_DEMAND_OP_H_ diff --git a/linux/include/tensorflow/compiler/jit/xla_device.h b/linux/include/tensorflow/compiler/jit/xla_device.h new file mode 100644 index 000000000..0f06b3fc8 --- /dev/null +++ b/linux/include/tensorflow/compiler/jit/xla_device.h @@ -0,0 +1,234 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// The XlaDevice executes a TensorFlow graph using the XLA linear algebra +// runtime. +// +// Operators assigned to an XlaDevice are compiled into XLA computations. +// Tensors on an XlaDevice are thin wrappers around XLA ScopedShapedBuffers. +// +// XlaDevice is instantiated separately for each XLA backend (e.g., CPU or GPU), +// under different names (e.g., XLA_CPU or XLA_GPU). + +#ifndef TENSORFLOW_COMPILER_JIT_XLA_DEVICE_H_ +#define TENSORFLOW_COMPILER_JIT_XLA_DEVICE_H_ + +#include "tensorflow/compiler/jit/xla_device_context.h" +#include "tensorflow/compiler/jit/xla_tensor.h" +#include "tensorflow/compiler/tf2xla/xla_compiler.h" +#include "tensorflow/compiler/tf2xla/xla_op_registry.h" +#include "tensorflow/compiler/xla/client/local_client.h" +#include "tensorflow/core/common_runtime/device_factory.h" +#include "tensorflow/core/common_runtime/local_device.h" +#include "tensorflow/core/framework/allocator.h" +#include "tensorflow/core/framework/device_base.h" +#include "tensorflow/core/framework/node_def_builder.h" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/resource_mgr.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/platform/mutex.h" +#include "tensorflow/core/platform/stream_executor_no_cuda.h" + +namespace tensorflow { + +class XlaDevice : public LocalDevice { + public: + // Given a tensor, sets `xla::Shape*` the shape of tensor's representation + // on device, fully padded. On error, the contents of `xla::Shape*` + // are undefined. + typedef std::function PaddedShapeFn; + + // Wrapper class to store metadata about the XlaDevice, where it can be + // retrieved e.g., when lazily creating the XlaCompilationCache device. + class Metadata { + public: + Metadata(int device_ordinal, se::Platform* platform, + const DeviceType& device_type, + XlaCompiler::ShapeRepresentationFn shape_representation_fn, + PaddedShapeFn padded_shape_fn, bool use_multiple_streams); + + // The index of the device on this host. + int device_ordinal() const; + + se::Platform* platform() const; + xla::LocalClient* client() const; + const DeviceType& jit_device_type() const; + const XlaCompiler::ShapeRepresentationFn& shape_representation_fn() const { + return shape_representation_fn_; + } + const PaddedShapeFn& padded_shape_fn() const { return padded_shape_fn_; } + + bool UseMultipleStreams() const { return use_multiple_streams_; } + + private: + const int device_ordinal_; + const DeviceType device_type_; + se::Platform* platform_; // Not owned. + XlaCompiler::ShapeRepresentationFn shape_representation_fn_; + PaddedShapeFn padded_shape_fn_; + const bool use_multiple_streams_; + + TF_DISALLOW_COPY_AND_ASSIGN(Metadata); + }; + + // Sets `*metadata` to the XlaDevice Metadata in the XLA device used by `ctx`. + static Status GetMetadata(OpKernelContext* ctx, const Metadata** metadata); + + // Sets `*metadata` to the XlaDevice Metadata in the XLA device used by `ctx`. + static Status GetMetadata(OpKernelConstruction* ctx, + const Metadata** metadata); + + // Factory function. 'platform_name' is the name of the XLA platform. + // 'device_name' is the name of the Tensorflow device to create. + // 'jit_device_name' is the name of the corresponding JIT device. + // 'transfer_as_literal' is true if device<->host transfers must be done using + // XLA's TransferLiteral{To,From}Device interface. If false, we can use + // ThenMemcpy instead. + // If 'use_multiple_streams' is true, we create separate streams for + // host-to-device and device-to-host communication. + // If padded_shape_fn is empty, a default implementation that returns + // the on-host shape is used. + static Status Create( + const string& platform_name, const string& device_name, + int device_ordinal, const string& jit_device_name, + const SessionOptions& options, const string& name_prefix, + const XlaOpRegistry::DeviceRegistration& registration, + bool transfer_as_literal, bool use_multiple_streams, + const XlaCompiler::ShapeRepresentationFn& shape_representation_fn, + const PaddedShapeFn& padded_shape_fn, std::unique_ptr* device); + + // Creates a new XLA Device. + // If padded_shape_fn is empty, a default implementation that returns + // the logical on-device shape without padding is used. + XlaDevice(const SessionOptions& options, const DeviceAttributes& attrs, + int device_ordinal, const DeviceType& jit_device_name, + se::Platform* platform, bool transfer_as_literal, + bool use_multiple_streams, + const XlaCompiler::ShapeRepresentationFn& shape_representation_fn, + const PaddedShapeFn& padded_shape_fn); + ~XlaDevice() override; + + Allocator* GetAllocator(AllocatorAttributes attr) override + LOCKS_EXCLUDED(mu_); + void Compute(OpKernel* op_kernel, OpKernelContext* context) override; + void ComputeAsync(AsyncOpKernel* op_kernel, OpKernelContext* context, + AsyncOpKernel::DoneCallback done) override; + Status Sync() override; + + Status FillContextMap(const Graph* graph, + DeviceContextMap* device_context_map) override + LOCKS_EXCLUDED(mu_); + + Status MakeTensorFromProto(const TensorProto& tensor_proto, + const AllocatorAttributes alloc_attrs, + Tensor* tensor) override LOCKS_EXCLUDED(mu_); + + const Metadata& metadata() { return xla_metadata_; } + + // Ensures the DeviceContext associated with this XlaDevice is created and + // valid (i.e. all streams are ok). If any state is not valid, a new + // DeviceContext will be created. + // + // TODO(b/111859745): The Eager context needs to call this method to recover + // from failures. + Status EnsureDeviceContextOk() LOCKS_EXCLUDED(mu_); + + // Instructs this XlaDevice to set a GpuDeviceInfo, which holds extra + // information for GPU and TPU devices. + Status UseGpuDeviceInfo() LOCKS_EXCLUDED(mu_); + + // Instructs this XlaDevice to return 'sync_on_completion' for + // RequiresSyncOnCompletion(). + void SetRequiresSyncOnCompletion(bool sync_on_completion) LOCKS_EXCLUDED(mu_); + + bool RequiresSyncOnCompletion() const override LOCKS_EXCLUDED(mu_); + + private: + xla::LocalClient* client() const; + Allocator* GetAllocatorLocked(AllocatorAttributes attr) + EXCLUSIVE_LOCKS_REQUIRED(mu_); + Status EnsureStreamOkLocked(xla::Backend* backend, const string& name, + std::shared_ptr* stream, + bool* stream_was_changed) + EXCLUSIVE_LOCKS_REQUIRED(mu_); + xla::StatusOr GetDeviceContextLocked() + EXCLUSIVE_LOCKS_REQUIRED(mu_); + + static Status GetMetadataFromDevice(DeviceBase* device, + const XlaDevice::Metadata** metadata); + + mutable mutex mu_; + // The metadata of this XlaDevice. + const Metadata xla_metadata_; + // Which hardware device in the client's platform this XlaDevice controls. + const int device_ordinal_; + // The name of the device that is used to compile Ops for this XlaDevice. + const DeviceType jit_device_name_; + // The platform for this device. + se::Platform* const platform_; // Not owned. + // Memory allocator associated with this device. + Allocator* xla_allocator_ GUARDED_BY(mu_) = nullptr; // Not owned. + // Stream associated with this device. Operations enqueued on this + // stream are executed on the device. Operations include data + // copying back and forth between CPU and the device, and + // computations enqueued by XLA. + std::shared_ptr stream_ GUARDED_BY(mu_); + // If false, only stream_ is valid and all computation and transfers use + // stream_. If true, computation is performed by stream_ and transfers are + // performed by host_to_device/device_to_host_stream. + const bool use_multiple_streams_; + // If use_multiple_streams_, host to device transfers are performed using this + // stream. + std::shared_ptr host_to_device_stream_ GUARDED_BY(mu_); + // If use_multiple_streams_, device to host transfers are performed using this + // stream. + std::shared_ptr device_to_host_stream_ GUARDED_BY(mu_); + // Must we use XLA's transfer manager for correct host<->device transfers? if + // false, we can use ThenMemcpy() instead. + const bool transfer_as_literal_; + const XlaCompiler::ShapeRepresentationFn shape_representation_fn_; + + // The device context accessed by all users of the XlaDevice, set by calls to + // EnsureDeviceContextOk. If gpu_device_info_ is non-null, this pointer is + // also filled in to that struct. XlaDeviceContext is a ref-counted object. + XlaDeviceContext* device_context_ GUARDED_BY(mu_) = nullptr; + + // Holds extra information for GPU and TPU devices, e.g. the device context. + bool use_gpu_device_info_ GUARDED_BY(mu_) = false; + std::unique_ptr gpu_device_info_ GUARDED_BY(mu_); + + // Thread pool used for running closures + std::unique_ptr thread_pool_; + + // True if the device requires XlaDevice::Sync to be called on completion + // regardless of status. + bool sync_on_completion_ GUARDED_BY(mu_) = false; +}; + +// Builds OpKernel registrations on 'device' for the JIT operators +// registered on 'jit_device'. Returns ownership of a XlaDeviceOpRegistrations +// object that encapsulates the kernel registrations. +struct XlaDeviceOpRegistrations { + std::vector> + op_kernel_registrars; +}; +XlaDeviceOpRegistrations* RegisterXlaDeviceKernels(const char* device, + const char* jit_device); + +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_JIT_XLA_DEVICE_H_ diff --git a/linux/include/tensorflow/compiler/jit/xla_device_context.h b/linux/include/tensorflow/compiler/jit/xla_device_context.h new file mode 100644 index 000000000..df8242129 --- /dev/null +++ b/linux/include/tensorflow/compiler/jit/xla_device_context.h @@ -0,0 +1,127 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_JIT_XLA_DEVICE_CONTEXT_H_ +#define TENSORFLOW_COMPILER_JIT_XLA_DEVICE_CONTEXT_H_ + +#include + +#include "tensorflow/compiler/jit/xla_tensor.h" +#include "tensorflow/compiler/tf2xla/xla_compiler.h" +#include "tensorflow/compiler/xla/client/global_data.h" +#include "tensorflow/compiler/xla/client/local_client.h" +#include "tensorflow/core/framework/allocator.h" +#include "tensorflow/core/framework/device_base.h" +#include "tensorflow/core/lib/core/status.h" + +namespace tensorflow { + +// The allocator used for Tensors assigned to the XLA device. The allocator +// ignores the alignment and size of the request and always returns a new, +// empty, XlaTensor. +class XlaDeviceAllocator : public Allocator { + public: + XlaDeviceAllocator(); + ~XlaDeviceAllocator() override; + + string Name() override; + + void* AllocateRaw(size_t alignment, size_t num_bytes) override; + void DeallocateRaw(void* ptr) override; + void GetStats(AllocatorStats* stats) override; +}; + +// Helper class for managing data transfers between host and XLA devices. +class XlaTransferManager { + public: + explicit XlaTransferManager( + std::shared_ptr compute_stream, + std::shared_ptr host_to_device_stream, + std::shared_ptr device_to_host_stream, + xla::LocalClient* client, bool transfer_as_literal, + XlaCompiler::ShapeRepresentationFn shape_representation_fn, + thread::ThreadPool* thread_pool); + + void CopyCPUTensorToDevice(const Tensor* cpu_tensor, Device* device, + Tensor* device_tensor, StatusCallback done) const; + void CopyDeviceTensorToCPU(const Tensor* device_tensor, + absl::string_view tensor_name, Device* device, + Tensor* cpu_tensor, StatusCallback done); + + void CopyDeviceTensorToDevice(const Tensor& src_tensor, Tensor* dst_tensor, + const StatusCallback& done); + + se::Stream* stream() const { return stream_.get(); } + + private: + Status TransferLiteralToDevice(const Tensor& host_tensor, + Tensor* device_tensor) const; + void TransferLiteralFromDevice(Tensor* host_tensor, + const Tensor& device_tensor, + const StatusCallback& done) const; + bool UseMultipleStreams() const { return stream_ != host_to_device_stream_; } + + // The main compute stream of the device, used to synchronize the transfer + // streams if they are set. + std::shared_ptr stream_; + // The stream to use for transferring data from host to device. Can be + // idential to stream_, but must not be nullptr. + std::shared_ptr host_to_device_stream_; + // The stream to use for transferring data from device to host. Can be + // idential to stream_, but must not be nullptr. + std::shared_ptr device_to_host_stream_; + // For the underlying memory allocator and XLA's TransferManager. + xla::LocalClient* client_; + // Transfer manager, for marshalling data to and from the device. + xla::TransferManager* transfer_manager_; + // True if we must use XLA's TransferManager for correct device transfers. + const bool transfer_as_literal_; + XlaCompiler::ShapeRepresentationFn shape_representation_fn_; + + // Thread pool used for running closures + thread::ThreadPool* thread_pool_; +}; + +// DeviceContext for operators assigned to XlaDevice devices. The +// implementation must inherit from DeviceContext but otherwise just +// wraps the methods in XlaTransferManager. +class XlaDeviceContext : public DeviceContext { + public: + explicit XlaDeviceContext( + std::shared_ptr compute_stream, + std::shared_ptr host_to_device_stream, + std::shared_ptr device_to_host_stream, + xla::LocalClient* client, bool transfer_as_literal, + XlaCompiler::ShapeRepresentationFn shape_representation_fn, + thread::ThreadPool* thread_pool); + + void CopyCPUTensorToDevice(const Tensor* cpu_tensor, Device* device, + Tensor* device_tensor, + StatusCallback done) const override; + void CopyDeviceTensorToCPU(const Tensor* device_tensor, + absl::string_view tensor_name, Device* device, + Tensor* cpu_tensor, StatusCallback done) override; + void CopyDeviceTensorToDevice(const Tensor& src_tensor, Tensor* dst_tensor, + const StatusCallback& done); + + se::Stream* stream() const override { return manager_.stream(); } + + private: + XlaTransferManager manager_; +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_JIT_XLA_DEVICE_CONTEXT_H_ diff --git a/linux/include/tensorflow/compiler/jit/xla_device_ops.h b/linux/include/tensorflow/compiler/jit/xla_device_ops.h new file mode 100644 index 000000000..6967ad1f0 --- /dev/null +++ b/linux/include/tensorflow/compiler/jit/xla_device_ops.h @@ -0,0 +1,264 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// Common kernel registrations for XLA devices. + +#ifndef TENSORFLOW_COMPILER_JIT_XLA_DEVICE_OPS_H_ +#define TENSORFLOW_COMPILER_JIT_XLA_DEVICE_OPS_H_ + +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/resource_mgr.h" +#include "tensorflow/core/kernels/cast_op.h" +#include "tensorflow/core/kernels/constant_op.h" +#include "tensorflow/core/kernels/control_flow_ops.h" +#include "tensorflow/core/kernels/data/generator_dataset_op.h" +#include "tensorflow/core/kernels/data/iterator_ops.h" +#include "tensorflow/core/kernels/data/prefetch_dataset_op.h" +#include "tensorflow/core/kernels/fifo_queue.h" +#include "tensorflow/core/kernels/function_ops.h" +#include "tensorflow/core/kernels/identity_n_op.h" +#include "tensorflow/core/kernels/identity_op.h" +#include "tensorflow/core/kernels/no_op.h" +#include "tensorflow/core/kernels/queue_op.h" +#include "tensorflow/core/kernels/resource_variable_ops.h" +#include "tensorflow/core/kernels/sendrecv_ops.h" +#include "tensorflow/core/kernels/shape_ops.h" +#include "tensorflow/core/kernels/variable_ops.h" + +namespace tensorflow { + +// Dummy OpKernel, used for kernels assigned to an XLA device that should be +// compiled. Should never be called at runtime since such ops should be +// rewritten to a XlaLaunch op. If it is called, it means the placer placed an +// operator on an XLA device but the compiler did not compile it. +class XlaDeviceDummyOp : public OpKernel { + public: + explicit XlaDeviceDummyOp(OpKernelConstruction* ctx); + void Compute(OpKernelContext* ctx) override; +}; + +class XlaAssignVariableOp : public AsyncOpKernel { + public: + explicit XlaAssignVariableOp(OpKernelConstruction* c); + void ComputeAsync(OpKernelContext* context, DoneCallback done) override; + + private: + DataType dtype_; +}; + +#define REGISTER_XLA_LAUNCH_KERNEL(DEVICE, KERNEL, TYPES) \ + REGISTER_KERNEL_BUILDER(Name("XlaLaunch") \ + .Device(DEVICE) \ + .HostMemory("constants") \ + .HostMemory("resources"), \ + KERNEL); + +#define REGISTER_XLA_COMPILE_KERNEL(DEVICE, KERNEL, TYPES) \ + REGISTER_KERNEL_BUILDER(Name("_XlaCompile") \ + .Device(DEVICE) \ + .HostMemory("constants") \ + .HostMemory("resources"), \ + KERNEL); + +#define REGISTER_XLA_RUN_KERNEL(DEVICE, KERNEL, TYPES) \ + REGISTER_KERNEL_BUILDER(Name("_XlaRun").Device(DEVICE), KERNEL); + +#define REGISTER_XLA_DEVICE_KERNELS(DEVICE, TYPES) \ + REGISTER_KERNEL_BUILDER(Name("_Send").Device(DEVICE), SendOp); \ + REGISTER_KERNEL_BUILDER(Name("_Recv").Device(DEVICE), RecvOp); \ + REGISTER_KERNEL_BUILDER( \ + Name("_HostSend").Device(DEVICE).HostMemory("tensor"), SendOp); \ + REGISTER_KERNEL_BUILDER( \ + Name("_HostRecv").Device(DEVICE).HostMemory("tensor"), RecvOp); \ + REGISTER_KERNEL_BUILDER( \ + Name("_HostCast").Device(DEVICE).HostMemory("x").HostMemory("y"), \ + CpuCastOp); \ + REGISTER_KERNEL_BUILDER(Name("NoOp").Device(DEVICE), NoOp); \ + REGISTER_KERNEL_BUILDER( \ + Name("Const").Device(DEVICE).TypeConstraint("dtype", TYPES), \ + ConstantOp); \ + REGISTER_KERNEL_BUILDER( \ + Name("Identity").Device(DEVICE).TypeConstraint("T", TYPES), IdentityOp); \ + REGISTER_KERNEL_BUILDER(Name("IdentityN").Device(DEVICE), IdentityNOp); \ + REGISTER_KERNEL_BUILDER(Name("Placeholder").Device(DEVICE), PlaceholderOp); \ + REGISTER_KERNEL_BUILDER(Name("PlaceholderV2").Device(DEVICE), \ + PlaceholderOp); \ + \ + REGISTER_KERNEL_BUILDER( \ + Name("VarHandleOp").Device(DEVICE).HostMemory("resource"), \ + ResourceHandleOp); \ + REGISTER_KERNEL_BUILDER( \ + Name("_VarHandlesOp").Device(DEVICE).HostMemory("resources"), \ + ResourceHandlesOp); \ + REGISTER_KERNEL_BUILDER( \ + Name("ReadVariableOp").Device(DEVICE).HostMemory("resource"), \ + ReadVariableOp); \ + REGISTER_KERNEL_BUILDER( \ + Name("_ReadVariablesOp").Device(DEVICE).HostMemory("resources"), \ + ReadVariablesOp); \ + REGISTER_KERNEL_BUILDER( \ + Name("DestroyResourceOp").Device(DEVICE).HostMemory("resource"), \ + DestroyResourceOp); \ + REGISTER_KERNEL_BUILDER(Name("Shape") \ + .Device(DEVICE) \ + .HostMemory("output") \ + .TypeConstraint("out_type") \ + .TypeConstraint("T", TYPES), \ + ShapeOp); \ + REGISTER_KERNEL_BUILDER(Name("Shape") \ + .Device(DEVICE) \ + .HostMemory("output") \ + .TypeConstraint("out_type") \ + .TypeConstraint("T", TYPES), \ + ShapeOp); \ + REGISTER_KERNEL_BUILDER(Name("ShapeN") \ + .Device(DEVICE) \ + .HostMemory("output") \ + .TypeConstraint("out_type") \ + .TypeConstraint("T", TYPES), \ + ShapeNOp); \ + REGISTER_KERNEL_BUILDER(Name("ShapeN") \ + .Device(DEVICE) \ + .HostMemory("output") \ + .TypeConstraint("out_type") \ + .TypeConstraint("T", TYPES), \ + ShapeNOp); \ + REGISTER_KERNEL_BUILDER(Name("Size") \ + .Device(DEVICE) \ + .HostMemory("output") \ + .TypeConstraint("out_type") \ + .TypeConstraint("T", TYPES), \ + SizeOp); \ + REGISTER_KERNEL_BUILDER(Name("Size") \ + .Device(DEVICE) \ + .HostMemory("output") \ + .TypeConstraint("out_type") \ + .TypeConstraint("T", TYPES), \ + SizeOp); \ + REGISTER_KERNEL_BUILDER( \ + Name("Rank").Device(DEVICE).HostMemory("output").TypeConstraint("T", \ + TYPES), \ + RankOp); \ + REGISTER_KERNEL_BUILDER( \ + Name("AssignVariableOp").Device(DEVICE).HostMemory("resource"), \ + XlaAssignVariableOp); \ + REGISTER_KERNEL_BUILDER(Name("ControlTrigger").Device(DEVICE), \ + ControlTriggerOp); \ + REGISTER_KERNEL_BUILDER(Name("Switch").Device(DEVICE).HostMemory("pred"), \ + SwitchOp); \ + REGISTER_KERNEL_BUILDER( \ + Name("Merge").Device(DEVICE).HostMemory("value_index"), MergeOp); \ + REGISTER_KERNEL_BUILDER(Name("Enter").Device(DEVICE), EnterOp); \ + REGISTER_KERNEL_BUILDER(Name("Exit").Device(DEVICE), ExitOp); \ + REGISTER_KERNEL_BUILDER(Name("NextIteration").Device(DEVICE), \ + NextIterationOp); \ + REGISTER_KERNEL_BUILDER(Name("LoopCond") \ + .Device(DEVICE) \ + .HostMemory("input") \ + .HostMemory("output"), \ + LoopCondOp); \ + \ + REGISTER_KERNEL_BUILDER( \ + Name("QueueEnqueueV2").Device(DEVICE).HostMemory("handle"), EnqueueOp); \ + REGISTER_KERNEL_BUILDER( \ + Name("QueueDequeueV2").Device(DEVICE).HostMemory("handle"), DequeueOp); \ + REGISTER_KERNEL_BUILDER( \ + Name("QueueCloseV2").Device(DEVICE).HostMemory("handle"), QueueCloseOp); \ + REGISTER_KERNEL_BUILDER(Name("QueueSizeV2") \ + .Device(DEVICE) \ + .HostMemory("size") \ + .HostMemory("handle"), \ + QueueSizeOp); \ + REGISTER_KERNEL_BUILDER( \ + Name("QueueIsClosedV2").Device(DEVICE).HostMemory("handle"), \ + QueueIsClosedOp); \ + \ + REGISTER_KERNEL_BUILDER( \ + Name("FIFOQueueV2").Device(DEVICE).HostMemory("handle"), FIFOQueueOp); \ + \ + REGISTER_KERNEL_BUILDER( \ + Name(kArgOp).Device(DEVICE).HostMemory("output").TypeConstraint("T", \ + TYPES), \ + ArgOp); \ + REGISTER_KERNEL_BUILDER(Name(kArgOp) \ + .Device(DEVICE) \ + .HostMemory("output") \ + .TypeConstraint("T"), \ + ArgOp); \ + \ + REGISTER_KERNEL_BUILDER(Name(kRetOp) \ + .Device(DEVICE) \ + .TypeConstraint("T", TYPES) \ + .HostMemory("input"), \ + RetvalOp); \ + REGISTER_KERNEL_BUILDER(Name(kRetOp) \ + .Device(DEVICE) \ + .TypeConstraint("T") \ + .HostMemory("input"), \ + RetvalOp); \ + \ + REGISTER_KERNEL_BUILDER( \ + Name("RemoteCall").Device(DEVICE).HostMemory("target"), RemoteCallOp); \ + \ + REGISTER_KERNEL_BUILDER( \ + Name("GeneratorDataset").Device(DEVICE).HostMemory("handle"), \ + data::GeneratorDatasetOp); \ + REGISTER_KERNEL_BUILDER(Name("PrefetchDataset") \ + .Device(DEVICE) \ + .HostMemory("buffer_size") \ + .HostMemory("input_dataset") \ + .HostMemory("handle"), \ + data::PrefetchDatasetOp); \ + \ + REGISTER_KERNEL_BUILDER(Name("IteratorV2").Device(DEVICE), \ + data::IteratorHandleOp); \ + REGISTER_KERNEL_BUILDER( \ + Name("MakeIterator").Device(DEVICE).HostMemory("dataset"), \ + data::MakeIteratorOp); \ + REGISTER_KERNEL_BUILDER(Name("AnonymousIterator").Device(DEVICE), \ + data::AnonymousIteratorHandleOp); \ + REGISTER_KERNEL_BUILDER(Name("IteratorGetNext").Device(DEVICE), \ + data::IteratorGetNextOp); \ + REGISTER_KERNEL_BUILDER(Name("IteratorGetNextSync").Device(DEVICE), \ + data::IteratorGetNextSyncOp); \ + REGISTER_KERNEL_BUILDER(Name("IteratorToStringHandle") \ + .Device(DEVICE) \ + .HostMemory("string_handle"), \ + data::IteratorToStringHandleOp); \ + REGISTER_KERNEL_BUILDER(Name("IteratorFromStringHandleV2") \ + .Device(DEVICE) \ + .HostMemory("string_handle"), \ + data::IteratorFromStringHandleOp); \ + REGISTER_KERNEL_BUILDER(Name(FunctionLibraryDefinition::kArgOp) \ + .Device(DEVICE) \ + .HostMemory("output") \ + .TypeConstraint("T"), \ + ArgOp); \ + REGISTER_KERNEL_BUILDER(Name(FunctionLibraryDefinition::kRetOp) \ + .Device(DEVICE) \ + .TypeConstraint("T") \ + .HostMemory("input"), \ + RetvalOp); + +// TODO(phawkins): currently we do not register the QueueEnqueueMany, +// QueueDequeueMany, or QueueDequeueUpTo kernels because they attempt to read +// and write the tensors they access in order to concatenate them into a batch. +// We would need either to call out to an XLA computation to perform the +// concatenation, or we would need to refactor those kernels so the splitting +// or merging is done in a separate operator that can be compiled. + +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_JIT_XLA_DEVICE_OPS_H_ diff --git a/linux/include/tensorflow/compiler/jit/xla_launch_util.h b/linux/include/tensorflow/compiler/jit/xla_launch_util.h new file mode 100644 index 000000000..326d70a02 --- /dev/null +++ b/linux/include/tensorflow/compiler/jit/xla_launch_util.h @@ -0,0 +1,181 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// Contains utilities for launching compiled XLA kernels for a KernelContext. + +#ifndef TENSORFLOW_COMPILER_JIT_XLA_LAUNCH_UTIL_H_ +#define TENSORFLOW_COMPILER_JIT_XLA_LAUNCH_UTIL_H_ + +#include "tensorflow/compiler/jit/xla_compilation_cache.h" +#include "tensorflow/compiler/jit/xla_tensor.h" +#include "tensorflow/compiler/tf2xla/xla_compiler.h" +#include "tensorflow/compiler/xla/client/local_client.h" +#include "tensorflow/compiler/xla/service/device_memory_allocator.h" +#include "tensorflow/compiler/xla/service/owning_device_memory.h" +#include "tensorflow/core/framework/allocation_description.pb.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/kernels/variable_ops.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/lib/gtl/array_slice.h" + +namespace tensorflow { +class XlaAllocator; + +// Takes a snapshot of the values of resource variable arguments, whose +// indices are specified in `variables` argument. We snapshot tensors that back +// resource variables since concurrent updates may modify the shape, and it is +// important that the shapes used for compilation match the true shapes of the +// buffers. +// +// Returns a map of TensorFlow argument index to resource variable. If a +// resource variable is not initialized, the corresponding OptionalTensor +// will have its `present` field set to false. +std::map SnapshotResourceVariables( + OpKernelContext* ctx, absl::Span variables); + +// Adapter class that wraps a Tensorflow allocator as an XLA allocator. +// Assumes that the Tensorflow allocator permits asynchronous deallocation: +// see comment on `AllowsAsynchronousDeallocation()`. +class XlaAllocator : public xla::DeviceMemoryAllocator { + public: + XlaAllocator(const se::Platform* platform, Allocator* wrapped); + ~XlaAllocator() override; + xla::StatusOr Allocate( + int device_ordinal, uint64 size, bool retry_on_failure) override; + Status Deallocate(int device_ordinal, se::DeviceMemoryBase mem) override; + + // The Tensorflow BFC allocator used on GPU allows host-side deallocation + // before GPU execution takes place. Tensorflow uses the ordering of the main + // compute stream to enforce a happens-before relationship between a memory + // allocation and code that reuses the same memory. If Tensorflow adds + // support for multiple GPU streams or allocators with different ordering + // requirements, this code may need to change. + // (This attribute has no effect on CPU.) + bool AllowsAsynchronousDeallocation() const override { return true; } + + private: + Allocator* wrapped_; +}; + +// Helper class to perform the marshalling of TensorFlow inputs and outputs to +// ShapedBuffers suitable for passing to an XLA computation. +class XlaComputationLaunchContext { + public: + // Create a new launch context. 'allocate_xla_tensors' is true if allocated + // output tensors and variables are always XlaTensors. If false they are + // assumed to be "normal" device pointers. + // If 'use_multiple_streams' is true, tensors may be defined and used on + // multiple streams and so se::Events must be defined and waited for. If + // 'use_multiple_streams' is true, 'allocate_xla_tensors' must also be true + // because we track inter-stream dependencies through events inside XlaTensor + // objects. + XlaComputationLaunchContext(xla::LocalClient* client, + xla::DeviceMemoryAllocator* xla_allocator, + bool allocate_xla_tensors, + bool use_multiple_streams); + + // Add all inputs within `ctx` as XLA arguments (returned by arguments()). + // `variables` is a map from TensorFlow argument number to resource variable. + // + // Assumes that the first `missing_ctx_input_prefix` inputs to the kernel are + // missing and adjusts input indices accordingly. All elements in kernel's + // input_mapping must be greater than or equal to `missing_ctx_input_prefix` + // (in other words, no inputs actually required by the kernel can be missing). + void PopulateInputs(OpKernelContext* ctx, + const XlaCompiler::CompilationResult* kernel, + const std::map& variables, + int missing_ctx_input_prefix); + + // Given the XLA output in `output`, populate all outputs of `ctx`. + // + // Assumes that the first `missing_ctx_input_prefix` inputs to the kernel are + // missing and adjusts input indices accordingly. + Status PopulateOutputs(OpKernelContext* ctx, + const XlaCompiler::CompilationResult* kernel, + xla::ScopedShapedBuffer output, + int missing_ctx_input_prefix); + + // Return the argument list. Only valid after PopulateInputs() has been + // called. + const std::vector& arguments() const { return arg_ptrs_; } + + private: + xla::LocalClient* client_; + xla::DeviceMemoryAllocator* xla_allocator_; + bool allocate_xla_tensors_; + bool use_multiple_streams_; + std::vector> arg_buffers_; + std::vector arg_ptrs_; +}; + +// A simple TensorBuffer implementation that allows us to create Tensors that +// take ownership of pre-allocated memory. +class XlaTensorBuffer : public TensorBuffer { + public: + XlaTensorBuffer(const void* ptr, size_t expected_size, size_t actual_size, + Allocator* allocator) + : expected_size_(expected_size), + actual_size_(actual_size), + allocator_(allocator) { + data_ = const_cast(ptr); + } + + ~XlaTensorBuffer() override { + if (data_) { + allocator_->DeallocateRaw(data_); + } + } + + void* data() const override { return data_; } + size_t size() const override { return expected_size_; } + + TensorBuffer* root_buffer() override { return this; } + + void FillAllocationDescription(AllocationDescription* proto) const override { + proto->set_allocated_bytes(actual_size_); + } + + static Tensor MakeTensor(DataType dtype, const TensorShape& shape, + se::DeviceMemoryBase buffer, Allocator* allocator) { + size_t expected_size = shape.num_elements() * DataTypeSize(dtype); + auto* tensor_buffer = new XlaTensorBuffer(buffer.opaque(), expected_size, + buffer.size(), allocator); + Tensor t(dtype, shape, tensor_buffer); + tensor_buffer->Unref(); + return t; + } + + private: + void* data_; + size_t expected_size_; + size_t actual_size_; + Allocator* allocator_; +}; + +// Exposed in this header file for microbenchmarking purposes, but this is an +// internal implementation detail. +namespace internal { +// Return the 'index''th subtree of the given ShapedBuffer as a +// ScopedShapedBuffer. The returned ScopedShapedBuffer takes ownership of the +// subtree, and sets the input's buffer pointers to nullptr for the subtree. +xla::ScopedShapedBuffer ExtractSubShapedBuffer( + xla::ShapedBuffer* shaped_buffer, int index, + xla::DeviceMemoryAllocator* allocator); +} // namespace internal + +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_JIT_XLA_LAUNCH_UTIL_H_ diff --git a/linux/include/tensorflow/compiler/jit/xla_tensor.h b/linux/include/tensorflow/compiler/jit/xla_tensor.h new file mode 100644 index 000000000..d95da6340 --- /dev/null +++ b/linux/include/tensorflow/compiler/jit/xla_tensor.h @@ -0,0 +1,131 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_JIT_XLA_TENSOR_H_ +#define TENSORFLOW_COMPILER_JIT_XLA_TENSOR_H_ + +#include + +#include "absl/memory/memory.h" +#include "tensorflow/compiler/xla/client/local_client.h" +#include "tensorflow/compiler/xla/service/shaped_buffer.h" +#include "tensorflow/core/framework/allocator.h" +#include "tensorflow/core/framework/device_base.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/platform/mutex.h" + +namespace tensorflow { + +// The implementation of a Tensor for an XlaDevice. All device tensors are +// actually one of these. +// +// To distinguish between "normal" device tensors and XlaTensors, the raw +// pointer data stored in the TensorBuffer is a tagged pointer. +class XlaTensor { + public: + // Downcast from a Tensor to an XlaTensor. Return nullptr if the downcast + // fails. + static XlaTensor* FromTensor(const Tensor* tensor); + + static bool RefCountIsOne(const Tensor& tensor); + + // Create a DeviceMemoryBase from a Tensor. The Tensor can be an XlaTensor, in + // which case the returned value is shaped_buffer()->root_buffer(), or a + // normal Tensor in which case the returned value is + // {tensor.tensor_data().data(), tensor.tensor_data().size}. + static se::DeviceMemoryBase DeviceMemoryFromTensor(const Tensor& tensor); + + // Assign the internal ShapedBuffer to new memory for the given dtype and + // shape. If a ShapedBuffer exists already (has_shaped_buffer() == true), it + // is replaced and the managed memory deallocated. + Status AllocateShapedBuffer(DataType dtype, const TensorShape& shape, + xla::LocalClient* client, int device_ordinal); + + // Some Tensors can have complex on-device shapes, including tuple shapes. To + // manage the memory for these tensors a ShapedBuffer may be required. + + // Return true if this XlaTensor contains a ShapedBuffer. + bool has_shaped_buffer() const { return shaped_buffer_ != nullptr; } + // Return the contained ShapedBuffer. + // REQUIRES: has_shaped_buffer() + const xla::ShapedBuffer& shaped_buffer() const { + CHECK(has_shaped_buffer()); + return *shaped_buffer_; + } + xla::ShapedBuffer& shaped_buffer() { + CHECK(has_shaped_buffer()); + return *shaped_buffer_; + } + // Mutates the XlaTensor to set the ShapedBuffer. + void set_shaped_buffer(xla::ScopedShapedBuffer shaped_buffer) { + shaped_buffer_ = + absl::make_unique(std::move(shaped_buffer)); + } + + // Some tensors on the device may have known values on the host. We use these + // in on-demand mode to avoid re-copying values from the device if we know the + // host value already. + + // Return true if this XlaTensor contains a host tensor. + bool has_host_tensor() const { return host_tensor_ != nullptr; } + // Return the contained host tensor. + // REQUIRES: has_host_tensor() + const Tensor& host_tensor() const { return *host_tensor_; } + // Sets the contained host tensor. + void set_host_tensor(const Tensor& tensor) { + host_tensor_.reset(new Tensor(tensor)); + } + + // If the tensor's content is not yet defined on 'stream', and there exists an + // se::Event declaring when the tensor's content is defined, return it. + // Otherwise, return nullptr. If this function returns nullptr then the + // tensor's content can be read on 'stream' without additional + // synchronization. + se::Event* GetDefinitionEvent(se::Stream* stream); + + // Assert that the tensor's content is defined on 'stream' by the time 'event' + // triggers. + void SetDefinedOn(se::Stream* stream, std::shared_ptr event); + + // Assert that the tensor's content is defined on 'stream'. This version does + // not provide an event, and must be called *after* SetDefinedOn(Stream, + // Event). This call can be read as an assertion that the definition event has + // been waited on by 'stream', so further calls to GetDefinitionEvent(stream) + // do not need to also wait on the event. + void SetDefinedOn(se::Stream* stream); + + // Convert from a raw pointer to an XlaTensor, removing the pointer tag. + static XlaTensor* FromOpaquePointer(void* ptr); + // Convert to a raw pointer from an XlaTensor, adding the pointer tag. + static void* ToOpaquePointer(XlaTensor* tensor); + + private: + // The optional contained ShapedBuffer. + std::unique_ptr shaped_buffer_; + // An optional host tensor value. + std::unique_ptr host_tensor_; + // An optional event that is triggered when the tensor's content has been + // defined. If this event is nullptr, it is assumed that the tensor's content + // is always defined. + std::shared_ptr definition_event_; + // A list of all streams for which the tensor's content is defined for any + // newly enqueued command. + absl::InlinedVector streams_defined_on_ GUARDED_BY(mu_); + mutex mu_; +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_JIT_XLA_TENSOR_H_ diff --git a/linux/include/tensorflow/compiler/tf2xla/cc/ops/xla_jit_ops.h b/linux/include/tensorflow/compiler/tf2xla/cc/ops/xla_jit_ops.h new file mode 100644 index 000000000..c483be888 --- /dev/null +++ b/linux/include/tensorflow/compiler/tf2xla/cc/ops/xla_jit_ops.h @@ -0,0 +1,108 @@ +// This file is MACHINE GENERATED! Do not edit. + +#ifndef TENSORFLOW_COMPILER_TF_XLA_CC_OPS_XLA_JIT_OPS_H_ +#define TENSORFLOW_COMPILER_TF_XLA_CC_OPS_XLA_JIT_OPS_H_ + +// This file is MACHINE GENERATED! Do not edit. + +#include "tensorflow/cc/framework/ops.h" +#include "tensorflow/cc/framework/scope.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/gtl/array_slice.h" + +namespace tensorflow { +namespace ops { + +/// @defgroup xla_jit_ops Xla Jit Ops +/// @{ + +/// Operator that connects the output of an XLA computation to other consumer graph nodes. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output`: The outputs tensor. +class XlaClusterOutput { + public: + XlaClusterOutput(const ::tensorflow::Scope& scope, ::tensorflow::Input input); + operator ::tensorflow::Output() const { return outputs; } + operator ::tensorflow::Input() const { return outputs; } + ::tensorflow::Node* node() const { return outputs.node(); } + + Operation operation; + ::tensorflow::Output outputs; +}; + +/// XLA Launch Op. For use by the XLA JIT only. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `OutputList`: The results tensor. +class XlaLaunch { + public: + XlaLaunch(const ::tensorflow::Scope& scope, ::tensorflow::InputList constants, + ::tensorflow::InputList args, ::tensorflow::InputList resources, + const DataTypeSlice& Tresults, const NameAttrList& function); + ::tensorflow::Output operator[](size_t index) const { return results[index]; } + + + Operation operation; + ::tensorflow::OutputList results; +}; + +/// XLA Compile Op. For use by the XLA JIT only. +/// +/// Compiles a TensorFlow function into an XLA LocalExecutable and returns a key +/// that _XlaRun can use to look up the LocalExecutable and execute it. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `Output` key: A key that can be used to look up the local executable compiled by the +/// node and associated metadata. +/// * `Output` compilation_successful: True iff the compilation was successful. Always true +/// for now. +class _XlaCompile { + public: + _XlaCompile(const ::tensorflow::Scope& scope, ::tensorflow::InputList + constants, ::tensorflow::InputList args, ::tensorflow::InputList + resources, const NameAttrList& function); + + Operation operation; + ::tensorflow::Output key; + ::tensorflow::Output compilation_successful; +}; + +/// XLA Run Op. For use by the XLA JIT only. +/// +/// Executes a TensorFlow function previously compiled into a LocalExecutable by an +/// _XlaCompile op. +/// +/// Arguments: +/// * scope: A Scope object +/// +/// Returns: +/// * `OutputList`: The results tensor. +class _XlaRun { + public: + _XlaRun(const ::tensorflow::Scope& scope, ::tensorflow::InputList args, + ::tensorflow::Input key, const DataTypeSlice& Tresults); + ::tensorflow::Output operator[](size_t index) const { return results[index]; } + + + Operation operation; + ::tensorflow::OutputList results; +}; + +/// @} + +} // namespace ops +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_TF_XLA_CC_OPS_XLA_JIT_OPS_H_ diff --git a/linux/include/tensorflow/compiler/tf2xla/const_analysis.h b/linux/include/tensorflow/compiler/tf2xla/const_analysis.h new file mode 100644 index 000000000..49b3c6d41 --- /dev/null +++ b/linux/include/tensorflow/compiler/tf2xla/const_analysis.h @@ -0,0 +1,45 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_TF2XLA_CONST_ANALYSIS_H_ +#define TENSORFLOW_COMPILER_TF2XLA_CONST_ANALYSIS_H_ + +#include + +#include "tensorflow/core/graph/graph.h" +#include "tensorflow/core/lib/core/status.h" + +namespace tensorflow { + +// Backwards dataflow analysis that finds nodes in a graph that must be +// compile-time constants for us to be able to lower the graph to XLA. +// +// The indices of the arguments to `graph` that must be constant are returned in +// `compile_time_const_arg_indices`, if `compile_time_const_arg_indices` is not +// null. +// +// The ids of the nodes in `graph` that must be constant are returned in +// `compile_time_const_nodes`, if `compile_time_const_nodes` is not null. +// +// Only propagate const-ness along edges for which `edge_filter` returns true. +Status BackwardsConstAnalysis(const Graph& g, + std::vector* compile_time_const_arg_indices, + std::vector* compile_time_const_nodes, + std::function edge_filter = + [](const Edge& e) { return true; }); + +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_TF2XLA_CONST_ANALYSIS_H_ diff --git a/linux/include/tensorflow/compiler/tf2xla/cpu_function_runtime.h b/linux/include/tensorflow/compiler/tf2xla/cpu_function_runtime.h new file mode 100644 index 000000000..dfc1e8b8a --- /dev/null +++ b/linux/include/tensorflow/compiler/tf2xla/cpu_function_runtime.h @@ -0,0 +1,165 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_TF2XLA_CPU_FUNCTION_RUNTIME_H_ +#define TENSORFLOW_COMPILER_TF2XLA_CPU_FUNCTION_RUNTIME_H_ + +#include "tensorflow/core/platform/types.h" + +#include + +namespace tensorflow { +namespace cpu_function_runtime { +// Stores information about one buffer used by an XLA:CPU compiled function. +// These buffers are used for holding inputs to the computation, outputs from +// the computation and as temporary scratch space. +class BufferInfo { + public: + // Creates a BufferInfo from a serialized encoding generated by `Encode`. + explicit BufferInfo(std::pair encoding) + : entry_param_number_(encoding.second) { + Kind kind; + uint64 size; + Unpack(encoding.first, &kind, &size); + kind_ = kind; + size_ = size; + } + + // Returns true if this buffer stores a constant. These never need to be + // allocated by the runtime. + bool is_constant() const { return kind() == Kind::kConstant; } + + // Returns true if this buffer stores an entry parameter. These may or may + // not need to be allocated by the runtime, depending on + // XlaCompiledCpuFunction::AllocMode. + bool is_entry_parameter() const { return kind() == Kind::kEntryParameter; } + + // Returns the entry parameter number of this buffer. + uint64 entry_parameter_number() const { + assert(is_entry_parameter()); + return entry_param_number_; + } + + // Returns true if this buffer is temporary scratch space required by the XLA + // computations. These are always allocated by the runtime. + bool is_temp_buffer() const { return kind() == Kind::kTempBuffer; } + + // Returns true if this buffer is allocated on the C stack or into registers. + // These buffers are never allocated by the runtime. + bool is_on_stack_buffer() const { return kind() == Kind::kOnStackBuffer; } + + // Returns the size for this buffer. + uint64 size() const { return size_; } + + // Encodes this BufferInfo into two 64 bit integers that can be used to + // reconstruct the BufferInfo later using the constructor. We need this + // because we use BufferInfo in places where using protocol buffers would + // negatively impact binary size. + std::pair Encode() const { + static_assert(sizeof(*this) == 16, ""); + uint64 upper = Pack(kind(), size_); + uint64 lower = entry_param_number_; + return {upper, lower}; + } + + bool operator==(const BufferInfo& buffer_info) const { + if (kind() != buffer_info.kind() || size() != buffer_info.size()) { + return false; + } + return !is_entry_parameter() || + entry_parameter_number() == buffer_info.entry_parameter_number(); + } + + // Factory methods: + + static BufferInfo MakeTempBuffer(uint64 size) { + return BufferInfo(Kind::kTempBuffer, /*size=*/size, + /*entry_param_number=*/-1); + } + static BufferInfo MakeConstant(uint64 size) { + return BufferInfo(Kind::kConstant, /*size=*/size, + /*entry_param_number=*/-1); + } + static BufferInfo MakeEntryParameter(uint64 size, uint64 param_number) { + return BufferInfo(Kind::kEntryParameter, /*size=*/size, + /*entry_param_number=*/param_number); + } + static BufferInfo MakeOnStackBuffer(uint64 size) { + return BufferInfo(Kind::kOnStackBuffer, /*size=*/size, + /*entry_param_number=*/-1); + } + + private: + BufferInfo() = default; + + enum class Kind : unsigned { + kConstant, + kTempBuffer, + kEntryParameter, + kOnStackBuffer + }; + + Kind kind() const { return static_cast(kind_); } + + explicit BufferInfo(Kind kind, uint64 size, uint64 entry_param_number) + : kind_(kind), size_(size), entry_param_number_(entry_param_number) {} + + static uint64 Pack(Kind kind, uint64 size) { + return (static_cast(size) << 2) | static_cast(kind); + } + + static void Unpack(uint64 packed, Kind* kind, uint64* size) { + *size = packed >> 2; + *kind = static_cast((packed << 62) >> 62); + } + + Kind kind_ : 2; + uint64 size_ : 62; + int64 entry_param_number_; +}; + +// Align to 64-bytes, to mimic tensorflow::Allocator::kAllocatorAlignment. +constexpr size_t kAlign = 64; + +// AlignedBufferBytes returns the sum of the size of each buffer in +// `buffer_infos`, skipping constants, on-stack buffers and, if +// allocate_entry_params is false, entry parameters. There are `n` entries in +// `buffer_infos`. Each buffer is aligned to kAlign byte boundaries. +size_t AlignedBufferBytes(const BufferInfo* buffer_infos, size_t n, + bool allocate_entry_params); + +// MallocContiguousBuffers allocates buffers for use by the entry point +// generated by tfcompile. There are `n` entries in `buffer_infos`. If +// `annotate_initialized` is set, the allocated memory will be annotated as +// having been initialized - this is useful when allocating temporary buffers. +// If allocate_entry_params is true then allocates temp buffers and entry +// parameters, otherwise allocated only temp buffers. Slots in `bufs` +// corresponding to unallocated buffers are set to nullptr. +// +// A single contiguous block of memory is allocated, and portions of it are +// parceled out into `bufs`, which must have space for `n` entries. Returns +// the head of the allocated contiguous block, which should be passed to +// FreeContiguous when the buffers are no longer in use. +void* MallocContiguousBuffers(const BufferInfo* buffer_infos, size_t n, + bool allocate_entry_params, void** bufs, + bool annotate_initialized); + +// FreeContiguous frees the contiguous block of memory allocated by +// MallocContiguousBuffers. +void FreeContiguous(void* contiguous); +} // namespace cpu_function_runtime +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_TF2XLA_CPU_FUNCTION_RUNTIME_H_ diff --git a/linux/include/tensorflow/compiler/tf2xla/dump_graph.h b/linux/include/tensorflow/compiler/tf2xla/dump_graph.h new file mode 100644 index 000000000..bbf01eb90 --- /dev/null +++ b/linux/include/tensorflow/compiler/tf2xla/dump_graph.h @@ -0,0 +1,50 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// Helper functions for dumping Graphs, GraphDefs, and FunctionDefs to files for +// debugging. + +#ifndef TENSORFLOW_COMPILER_TF2XLA_DUMP_GRAPH_H_ +#define TENSORFLOW_COMPILER_TF2XLA_DUMP_GRAPH_H_ + +#include "tensorflow/core/framework/function.h" +#include "tensorflow/core/framework/graph.pb.h" +#include "tensorflow/core/graph/graph.h" + +namespace tensorflow { +namespace dump_graph { + +// Dumps 'graph_def' to a file, as a GraphDef text proto. Returns the file name +// chosen. +// +// Automatically picks a file name. Prefixes 'name' with the value of the +// --tf_dump_graph_prefix flag and suffixes it with ".pbtxt" to form a name. +// If a graph has already been dumped by this process with the same name, +// suffixes with "_n.pbtxt", where 'n' is a sequence number. +string DumpGraphDefToFile(const string& name, GraphDef const& graph_def); + +// Similar to DumpGraphDefToFile, but builds the GraphDef to dump from a 'graph' +// and an optional function library 'flib_def'. Returns the file name chosen. +string DumpGraphToFile(const string& name, Graph const& graph, + const FunctionLibraryDefinition* flib_def = nullptr); + +// Similar to DumpGraphDefToFile, but dumps a function as a FunctionDef text +// proto. Returns the file name chosen. +string DumpFunctionDefToFile(const string& name, FunctionDef const& fdef); + +} // namespace dump_graph +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_TF2XLA_DUMP_GRAPH_H_ diff --git a/linux/include/tensorflow/compiler/tf2xla/dump_graph_flags.h b/linux/include/tensorflow/compiler/tf2xla/dump_graph_flags.h new file mode 100644 index 000000000..80a3307d9 --- /dev/null +++ b/linux/include/tensorflow/compiler/tf2xla/dump_graph_flags.h @@ -0,0 +1,48 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_TF2XLA_DUMP_GRAPH_FLAGS_H_ +#define TENSORFLOW_COMPILER_TF2XLA_DUMP_GRAPH_FLAGS_H_ + +// Legacy flags for the XLA bridge's dump_graph module. + +#include + +#include "tensorflow/core/platform/types.h" +#include "tensorflow/core/util/command_line_flags.h" + +namespace tensorflow { +namespace legacy_flags { + +// Append to *flag_list flag definitions associated with the XLA bridge's +// dump_graph module. +void AppendDumpGraphFlags(std::vector* flag_list); + +// The values of flags associated with the XLA bridge's +// dump_graph module. +typedef struct { + string tf_dump_graph_prefix; // Path prefix to which graphs dumped during + // debugging should be written. +} DumpGraphFlags; + +// Return a pointer to the DumpGraphFlags struct; +// repeated calls return the same pointer. +// This should be called only after Flags::Parse() has returned. +DumpGraphFlags* GetDumpGraphFlags(); + +} // namespace legacy_flags +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_TF2XLA_DUMP_GRAPH_FLAGS_H_ diff --git a/linux/include/tensorflow/compiler/tf2xla/functionalize_cond.h b/linux/include/tensorflow/compiler/tf2xla/functionalize_cond.h new file mode 100644 index 000000000..189980894 --- /dev/null +++ b/linux/include/tensorflow/compiler/tf2xla/functionalize_cond.h @@ -0,0 +1,243 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_TF2XLA_FUNCTIONALIZE_COND_H_ +#define TENSORFLOW_COMPILER_TF2XLA_FUNCTIONALIZE_COND_H_ + +#include +#include "tensorflow/compiler/xla/status_macros.h" +#include "tensorflow/core/framework/function.h" +#include "tensorflow/core/graph/graph.h" + +namespace tensorflow { + +// Functionalize all the switch-merge nodes of a loop-free graph into If +// nodes. That is, attempt to transform every remaining switch and merge nodes +// in the graph into If nodes. +// Precondition: All while loops have been removed from graph. +Status FunctionalizeCond(Graph* graph, FunctionLibraryDefinition* library); + +// Internal functions/classes exposed for testing purposes. +namespace functionalize_cond { + +// All nodes are assumed to be either in no branch, then branch, else branch, +// or both branches (such as merge nodes). +// The code below relies on Else and Then being 0 and 1 (corresponding to the +// switch outputs). Both and Neither are arbitrary. +enum class BranchType { + kElseBranch = 0, + kThenBranch = 1, + kBoth = 2, + kNeither = 3, +}; + +// StateMap is responsible for mapping from each graph Node to +// * a CondState, where each CondState is a map from predicate to branch (i,e., +// what predicates have to hold or not hold). +// * a AncestorState, where each AncestorState is a set of switch/merge nodes +// that are an ancestor of the node in the graph; +// For efficiency, this class interns the CondState (AncestorState), so that +// CondState (AncestorState) equality comparisons are simply pointer +// comparisons. +class StateMap { + public: + explicit StateMap(Graph* graph); + + // Compare two OutputTensors by (node id, index). + struct OutputTensorLess { + bool operator()(const OutputTensor& lhs, const OutputTensor& rhs) const; + }; + + // A node in the graph is executed when multiple conditions hold. Keep track + // of the predicates that must hold for a node to execute. + using CondState = std::map; + + // Every unique ID is mapped to a CondState. + using CondId = const CondState*; + + // Keep track of which switch/merge node's feed into a node's values. + using AncestorState = std::set; + + // Every unique ID is mapped to a AncestorState. + using AncestorId = const AncestorState*; + + // Returns the CondId for a given node. + CondId LookupCondId(const Node* node) const; + + // Returns the unique CondId for CondState. + CondId GetCondId(const CondState& state); + + // Resets the CondId for a given node. + void ResetCondId(const Node* node, CondId id); + + // Returns the AncestorId for a given node. + AncestorId LookupAncestorId(const Node* node) const; + + // Returns the unique AncestorId for CondState. + AncestorId GetAncestorId(const AncestorState& state); + + // Resets the AncestorId for a given node. + void ResetAncestorId(const Node* node, AncestorId id); + + // Marks `node` as dead. + void MarkDead(const Node* node); + + // Determine branch execution of CondState. + BranchType FindBranchOf(CondId id, OutputTensor predicate) const; + + // Returns textual representation of node's CondState. + string CondStateToString(const Node* node) const; + string CondStateToString(CondId id) const; + + // Returns textual representation of node's AncestorState. + string AncestorStateToString(const Node* node) const; + + // Returns whether the cond state is the dead state. + bool IsDead(CondId id) const; + + // Returns whether the cond state is the empty state. + bool IsEmpty(CondId id) const; + + private: + // Hash for CondState and AncestorState. + struct Hash { + size_t operator()(const CondState& map) const; + size_t operator()(const AncestorState& map) const; + }; + + // Set to keep track of unique CondStates. + // Pointers to the entries in the unordered set are used as identifiers: + // unordered_set guarantees that the pointers remain the same. + std::unordered_set condstate_set_; + + // Mapping from Node id to CondId. + std::vector node_to_condid_map_; + + // Track the CondId for newly inserted nodes. We use a vector to quickly map + // from Node id in the original graph to the CondId, but there will be nodes + // added to the original graph (such as If nodes) whose CondState needs to be + // tracked too. + std::unordered_map added_node_condid_mapping_; + + // AncestorId variants of the CondId members. + std::unordered_set ancestorstate_set_; + std::vector node_to_ancestorid_map_; + std::unordered_map added_node_ancestorid_mapping_; + + // Identifier of the dead flow state. The empty flow state is represented with + // a nullptr. + CondId dead_id_; +}; + +// FunctionalizeCond groups all the state used by functionalizing conditionals +// of the given graph together. +class FunctionalizeCond { + public: + // Functionalize all the switch-merge nodes of a loop-free graph into If + // nodes. That is, attempt to transform every remaining switch and merge nodes + // in the graph into If nodes. + // Precondition: All while loops have been removed from graph. + static Status Functionalize(Graph* graph, FunctionLibraryDefinition* library); + + // Build identity node with the same name as the merge that will be replaced + // in case the output is fetched/colocated. + Status AddIdentityNode(const Node* replacee, Node* if_node, int port); + + // Add a If node to the graph defined by def that will, amongst other, replace + // replacee in the graph. + xla::StatusOr AddIfNode(const NodeDef& def, const Node* replacee, + const OutputTensor& predicate); + + // Propagates the state of a newly inserted node. + Status PropagateUpdatedState(const Node* replacee); + + // Dump graph with the CondState annotated. + void DumpGraphWithCondState(const string& name); + + private: + FunctionalizeCond(Graph* graph, FunctionLibraryDefinition* library); + + // Performs the actual cond functionalization. Iterate over groups of merge + // nodes (linked by common predicates & ancestor IDs), from innermost to + // outermost, and extract into If nodes. + Status FunctionalizeInternal(); + + // Returns the forward flow state propagated along edge `e`. + // This may modify state_map_. + StateMap::CondId StateAlongEdge(const Edge* e); + + // Determines the CondState and AncestorState of all the nodes in the given + // vector where the input is expected in reverse topological order. + // This populates the state_map_. + Status DetermineStates(std::vector rev_topo_order); + + // Determine the CondState for a given node using the incomming edges + // to the node. Note: it is expected that this node's CondState is only + // determined once its input's CondState is. + Status DetermineCondState(Node* dst) { + if (IsMerge(dst)) return DetermineCondStateMerge(dst); + return DetermineCondStateNonMerge(dst); + } + + // Helper functions for DetermineCondState. + Status DetermineCondStateNonMerge(Node* dst); + Status DetermineCondStateMerge(Node* dst); + + // Determines the dst node's CondState by joining the src and dst's CondState + // where either the dst node is a merge or not. + // These may modify state_map_. + xla::StatusOr JoinCondStatesMerge(Node* merge, + StateMap::CondId src, + StateMap::CondId dst); + xla::StatusOr JoinCondStatesNonMerge(StateMap::CondId src, + StateMap::CondId dst); + + // Determines which switch/merge nodes are ancestors of this node. + Status DetermineAncestorState(Node* dst); + + // Checks if a merge node is redundant and if so removes it from the graph. + Status RemoveRedundantMerge(Node* node); + + // Checks if a switch node is redundant and if so removes it from the graph. + Status RemoveRedundantSwitch(Node* node); + + // Sorts merge nodes (in reverse topological order) in order of increasing + // nesting depth. + void SortMergeNodes(std::vector* merge_order); + + // Deletes all nodes in/consumers reachable from switch/merge nodes that were + // extracted. + void DeleteReachableAndDeadNodes(const std::vector& switch_ids, + const std::vector& merge_order); + + // Member used to unique the CondState to a unique CondId (AncestorState to a + // unique AncestorId) and keep track of CondState/CondId + // (AncestorState/AncestorId) per Node. + StateMap state_map_; + + // Mapping from merge nodes to predicate. + std::unordered_map merge_to_predicate_; + + FunctionLibraryDefinition* library_; + Graph* graph_; + + friend class FunctionalizeCondTest; +}; + +} // namespace functionalize_cond + +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_TF2XLA_FUNCTIONALIZE_COND_H_ diff --git a/linux/include/tensorflow/compiler/tf2xla/functionalize_control_flow.h b/linux/include/tensorflow/compiler/tf2xla/functionalize_control_flow.h new file mode 100644 index 000000000..ba9920564 --- /dev/null +++ b/linux/include/tensorflow/compiler/tf2xla/functionalize_control_flow.h @@ -0,0 +1,46 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_TF2XLA_FUNCTIONALIZE_CONTROL_FLOW_H_ +#define TENSORFLOW_COMPILER_TF2XLA_FUNCTIONALIZE_CONTROL_FLOW_H_ + +#include "tensorflow/compiler/xla/status_macros.h" +#include "tensorflow/core/common_runtime/optimization_registry.h" +#include "tensorflow/core/framework/function.h" +#include "tensorflow/core/graph/graph.h" + +namespace tensorflow { + +// Transformation that converts tf.while_loop() loops into functional While +// operators and tf.cond() conditionals into function If operators, suitable for +// XLA compilation. If lookup_library is provided, use it to make the library +// for control flow self-contained. +Status FunctionalizeControlFlow(Graph* graph, + FunctionLibraryDefinition* library); +Status FunctionalizeControlFlow(const FunctionLibraryDefinition* lookup_library, + Graph* graph, + FunctionLibraryDefinition* library); + +// This pass looks at the graph and all associated FunctionDefs, and turns +// traditional control flow structure (Switch/Merge/etc.) into functional +// control flow structure (If/While). +class FunctionalizeControlFlowPass : public GraphOptimizationPass { + public: + Status Run(const GraphOptimizationPassOptions& options) override; +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_TF2XLA_FUNCTIONALIZE_CONTROL_FLOW_H_ diff --git a/linux/include/tensorflow/compiler/tf2xla/functionalize_control_flow_util.h b/linux/include/tensorflow/compiler/tf2xla/functionalize_control_flow_util.h new file mode 100644 index 000000000..582b49d51 --- /dev/null +++ b/linux/include/tensorflow/compiler/tf2xla/functionalize_control_flow_util.h @@ -0,0 +1,56 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_TF2XLA_FUNCTIONALIZE_CONTROL_FLOW_UTIL_H_ +#define TENSORFLOW_COMPILER_TF2XLA_FUNCTIONALIZE_CONTROL_FLOW_UTIL_H_ + +#include "absl/strings/str_join.h" +#include "tensorflow/compiler/xla/status_macros.h" +#include "tensorflow/core/graph/graph.h" + +// Utility functions shared between functionalize cond and while. + +namespace tensorflow { + +// Check that the graph has no cycle containing the given node. +Status CheckNodeNotInCycle(const Node* node, const int num_nodes); + +// Comparison function used for sorting nodes consistently. +// a) resource variables are last, and +// b) sort lexicographically by name (for deterministic output). +struct NodeCmpByNameResourcesLast { + bool operator()(const Node* lhs, const Node* rhs) const; +}; + +// Returns the Node* created from the NodeDef in the Graph. +xla::StatusOr AddNodeDefToGraph(const NodeDef& node_def, Graph* graph); + +// Build a retval node of given type and index. +xla::StatusOr BuildRetvalNode(Graph* graph, DataType type, int index); + +// Returns a textual representation of the names of the nodes in the input. +template +string NodesToString(const T& nodes) { + return absl::StrCat("{", + absl::StrJoin(nodes, ",", + [](string* output, const Node* node) { + absl::StrAppend(output, node->name()); + }), + "}"); +} + +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_TF2XLA_FUNCTIONALIZE_CONTROL_FLOW_UTIL_H_ diff --git a/linux/include/tensorflow/compiler/tf2xla/functionalize_while.h b/linux/include/tensorflow/compiler/tf2xla/functionalize_while.h new file mode 100644 index 000000000..a708c6e4e --- /dev/null +++ b/linux/include/tensorflow/compiler/tf2xla/functionalize_while.h @@ -0,0 +1,32 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_TF2XLA_FUNCTIONALIZE_WHILE_H_ +#define TENSORFLOW_COMPILER_TF2XLA_FUNCTIONALIZE_WHILE_H_ + +#include "tensorflow/core/framework/function.h" +#include "tensorflow/core/graph/graph.h" + +namespace tensorflow { + +// Transformation that converts tf.while_loop() loops into functional While +// operators, suitable for XLA compilation. If lookup_library is provided, use +// it to make the library for control flow self-contained. +Status FunctionalizeWhileLoop(const FunctionLibraryDefinition* lookup_library, + Graph* graph, FunctionLibraryDefinition* library); + +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_TF2XLA_FUNCTIONALIZE_WHILE_H_ diff --git a/linux/include/tensorflow/compiler/tf2xla/graph_compiler.h b/linux/include/tensorflow/compiler/tf2xla/graph_compiler.h new file mode 100644 index 000000000..e9f02201c --- /dev/null +++ b/linux/include/tensorflow/compiler/tf2xla/graph_compiler.h @@ -0,0 +1,96 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_TF2XLA_GRAPH_COMPILER_H_ +#define TENSORFLOW_COMPILER_TF2XLA_GRAPH_COMPILER_H_ + +#include "tensorflow/compiler/tf2xla/xla_compilation_device.h" +#include "tensorflow/compiler/tf2xla/xla_context.h" +#include "tensorflow/compiler/xla/client/local_client.h" +#include "tensorflow/core/common_runtime/device.h" +#include "tensorflow/core/common_runtime/device_mgr.h" +#include "tensorflow/core/common_runtime/function.h" +#include "tensorflow/core/framework/function.h" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/platform/env.h" +#include "tensorflow/core/platform/mutex.h" +#include "tensorflow/core/platform/notification.h" +#include "tensorflow/core/platform/thread_annotations.h" +#include "tensorflow/core/public/version.h" + +namespace tensorflow { + +// GraphCompiler compiles the graph in topological order in the current +// thread. It also resolves the nondeterminism in the graph by enforcing a +// total order on all inputs to a node. This abstraction helps us create the +// same XLA computation given two structurally equivalent TensorFlow graphs. +// If a function call is visited during the graph traversal, it is then +// compiled through the xla_context into a computation and a `Call` operation +// is inserted to call into that computation. +// +// Note: GraphCompiler was created to remove our dependency to TF Executor in +// the history. There are still some todos so that we can completely decouple +// from Executor. +// +// TODO(yunxing): Remove usage of XlaCompilationDevice. +// +// TODO(yunxing): Remove the hack that wraps XlaExpression within a tensor now +// that we don't use TF Executor to pass around a tensor. +// +// TODO(yunxing): Make XlaOpkernel not a subclass of OpKernel so that it can +// handle a XlaExpression directly instead of a Tensor. This may require our own +// op registration infrastructure instead of FunctionLibraryRuntime. +class GraphCompiler { + public: + GraphCompiler(XlaCompilationDevice* device, Graph* graph, + FunctionLibraryRuntime* flib, + ScopedStepContainer* step_container) + : device_(device), + graph_(graph), + flib_(flib), + step_container_(step_container) {} + + // Compiles the graph. The results are written in xla_context stored in the + // resource_manager of the 'XlaCompilationDevice' that's passed into the + // constructor. + Status Compile(); + + private: + // Partially sets params. This partially set params can be reused + // across multiple nodes visit. + void PartiallySetupParams(OpKernelContext::Params* params); + + // Tests if a node is a functional node. A functional node represents a + // defined computation and should be compiled using `compiler_`. + bool IsFunctional(Node* n); + + // Compiles a functional node and writes result to OpkernelContext. A + // functional node represents a defined computation and should be compiled + // using `compiler_`. + Status CompileFunctionalNode(Node* n, OpKernelContext* op_context); + + XlaCompilationDevice* device_; + Graph* graph_; + FunctionLibraryRuntime* flib_; + ScopedStepContainer* step_container_; + // A buffer to hold tensor inputs to a node, this is reused across the graph + // traversal. + absl::InlinedVector tensor_inputs_; +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_TF2XLA_GRAPH_COMPILER_H_ diff --git a/linux/include/tensorflow/compiler/tf2xla/host_compute_metadata.pb.h b/linux/include/tensorflow/compiler/tf2xla/host_compute_metadata.pb.h new file mode 100644 index 000000000..0dbeb0ba9 --- /dev/null +++ b/linux/include/tensorflow/compiler/tf2xla/host_compute_metadata.pb.h @@ -0,0 +1,764 @@ +// Generated by the protocol buffer compiler. DO NOT EDIT! +// source: tensorflow/compiler/tf2xla/host_compute_metadata.proto + +#ifndef PROTOBUF_INCLUDED_tensorflow_2fcompiler_2ftf2xla_2fhost_5fcompute_5fmetadata_2eproto +#define PROTOBUF_INCLUDED_tensorflow_2fcompiler_2ftf2xla_2fhost_5fcompute_5fmetadata_2eproto + +#include + +#include + +#if GOOGLE_PROTOBUF_VERSION < 3006000 +#error This file was generated by a newer version of protoc which is +#error incompatible with your Protocol Buffer headers. Please update +#error your headers. +#endif +#if 3006000 < GOOGLE_PROTOBUF_MIN_PROTOC_VERSION +#error This file was generated by an older version of protoc which is +#error incompatible with your Protocol Buffer headers. Please +#error regenerate this file with a newer version of protoc. +#endif + +#include +#include +#include +#include +#include +#include +#include +#include +#include // IWYU pragma: export +#include // IWYU pragma: export +#include +#include "tensorflow/core/framework/tensor_shape.pb.h" +#include "tensorflow/core/framework/types.pb.h" +// @@protoc_insertion_point(includes) +#define PROTOBUF_INTERNAL_EXPORT_protobuf_tensorflow_2fcompiler_2ftf2xla_2fhost_5fcompute_5fmetadata_2eproto + +namespace protobuf_tensorflow_2fcompiler_2ftf2xla_2fhost_5fcompute_5fmetadata_2eproto { +// Internal implementation detail -- do not use these members. +struct TableStruct { + static const ::google::protobuf::internal::ParseTableField entries[]; + static const ::google::protobuf::internal::AuxillaryParseTableField aux[]; + static const ::google::protobuf::internal::ParseTable schema[3]; + static const ::google::protobuf::internal::FieldMetadata field_metadata[]; + static const ::google::protobuf::internal::SerializationTable serialization_table[]; + static const ::google::protobuf::uint32 offsets[]; +}; +void AddDescriptors(); +} // namespace protobuf_tensorflow_2fcompiler_2ftf2xla_2fhost_5fcompute_5fmetadata_2eproto +namespace tensorflow { +namespace tf2xla { +class HostComputeMetadata; +class HostComputeMetadataDefaultTypeInternal; +extern HostComputeMetadataDefaultTypeInternal _HostComputeMetadata_default_instance_; +class HostTransferMetadata; +class HostTransferMetadataDefaultTypeInternal; +extern HostTransferMetadataDefaultTypeInternal _HostTransferMetadata_default_instance_; +class TensorMetadata; +class TensorMetadataDefaultTypeInternal; +extern TensorMetadataDefaultTypeInternal _TensorMetadata_default_instance_; +} // namespace tf2xla +} // namespace tensorflow +namespace google { +namespace protobuf { +template<> ::tensorflow::tf2xla::HostComputeMetadata* Arena::CreateMaybeMessage<::tensorflow::tf2xla::HostComputeMetadata>(Arena*); +template<> ::tensorflow::tf2xla::HostTransferMetadata* Arena::CreateMaybeMessage<::tensorflow::tf2xla::HostTransferMetadata>(Arena*); +template<> ::tensorflow::tf2xla::TensorMetadata* Arena::CreateMaybeMessage<::tensorflow::tf2xla::TensorMetadata>(Arena*); +} // namespace protobuf +} // namespace google +namespace tensorflow { +namespace tf2xla { + +// =================================================================== + +class TensorMetadata : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.tf2xla.TensorMetadata) */ { + public: + TensorMetadata(); + virtual ~TensorMetadata(); + + TensorMetadata(const TensorMetadata& from); + + inline TensorMetadata& operator=(const TensorMetadata& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + TensorMetadata(TensorMetadata&& from) noexcept + : TensorMetadata() { + *this = ::std::move(from); + } + + inline TensorMetadata& operator=(TensorMetadata&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const TensorMetadata& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const TensorMetadata* internal_default_instance() { + return reinterpret_cast( + &_TensorMetadata_default_instance_); + } + static constexpr int kIndexInFileMessages = + 0; + + void UnsafeArenaSwap(TensorMetadata* other); + void Swap(TensorMetadata* other); + friend void swap(TensorMetadata& a, TensorMetadata& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline TensorMetadata* New() const final { + return CreateMaybeMessage(NULL); + } + + TensorMetadata* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const TensorMetadata& from); + void MergeFrom(const TensorMetadata& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(TensorMetadata* other); + protected: + explicit TensorMetadata(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // .tensorflow.TensorShapeProto shape = 2; + bool has_shape() const; + void clear_shape(); + static const int kShapeFieldNumber = 2; + private: + const ::tensorflow::TensorShapeProto& _internal_shape() const; + public: + const ::tensorflow::TensorShapeProto& shape() const; + ::tensorflow::TensorShapeProto* release_shape(); + ::tensorflow::TensorShapeProto* mutable_shape(); + void set_allocated_shape(::tensorflow::TensorShapeProto* shape); + void unsafe_arena_set_allocated_shape( + ::tensorflow::TensorShapeProto* shape); + ::tensorflow::TensorShapeProto* unsafe_arena_release_shape(); + + // .tensorflow.DataType type = 1; + void clear_type(); + static const int kTypeFieldNumber = 1; + ::tensorflow::DataType type() const; + void set_type(::tensorflow::DataType value); + + // @@protoc_insertion_point(class_scope:tensorflow.tf2xla.TensorMetadata) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::tensorflow::TensorShapeProto* shape_; + int type_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2ftf2xla_2fhost_5fcompute_5fmetadata_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class HostTransferMetadata : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.tf2xla.HostTransferMetadata) */ { + public: + HostTransferMetadata(); + virtual ~HostTransferMetadata(); + + HostTransferMetadata(const HostTransferMetadata& from); + + inline HostTransferMetadata& operator=(const HostTransferMetadata& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + HostTransferMetadata(HostTransferMetadata&& from) noexcept + : HostTransferMetadata() { + *this = ::std::move(from); + } + + inline HostTransferMetadata& operator=(HostTransferMetadata&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const HostTransferMetadata& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const HostTransferMetadata* internal_default_instance() { + return reinterpret_cast( + &_HostTransferMetadata_default_instance_); + } + static constexpr int kIndexInFileMessages = + 1; + + void UnsafeArenaSwap(HostTransferMetadata* other); + void Swap(HostTransferMetadata* other); + friend void swap(HostTransferMetadata& a, HostTransferMetadata& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline HostTransferMetadata* New() const final { + return CreateMaybeMessage(NULL); + } + + HostTransferMetadata* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const HostTransferMetadata& from); + void MergeFrom(const HostTransferMetadata& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(HostTransferMetadata* other); + protected: + explicit HostTransferMetadata(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated .tensorflow.tf2xla.TensorMetadata metadata = 2; + int metadata_size() const; + void clear_metadata(); + static const int kMetadataFieldNumber = 2; + ::tensorflow::tf2xla::TensorMetadata* mutable_metadata(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::tf2xla::TensorMetadata >* + mutable_metadata(); + const ::tensorflow::tf2xla::TensorMetadata& metadata(int index) const; + ::tensorflow::tf2xla::TensorMetadata* add_metadata(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::tf2xla::TensorMetadata >& + metadata() const; + + // string key = 1; + void clear_key(); + static const int kKeyFieldNumber = 1; + const ::std::string& key() const; + void set_key(const ::std::string& value); + #if LANG_CXX11 + void set_key(::std::string&& value); + #endif + void set_key(const char* value); + void set_key(const char* value, size_t size); + ::std::string* mutable_key(); + ::std::string* release_key(); + void set_allocated_key(::std::string* key); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_key(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_key( + ::std::string* key); + + // @@protoc_insertion_point(class_scope:tensorflow.tf2xla.HostTransferMetadata) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::tf2xla::TensorMetadata > metadata_; + ::google::protobuf::internal::ArenaStringPtr key_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2ftf2xla_2fhost_5fcompute_5fmetadata_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class HostComputeMetadata : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.tf2xla.HostComputeMetadata) */ { + public: + HostComputeMetadata(); + virtual ~HostComputeMetadata(); + + HostComputeMetadata(const HostComputeMetadata& from); + + inline HostComputeMetadata& operator=(const HostComputeMetadata& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + HostComputeMetadata(HostComputeMetadata&& from) noexcept + : HostComputeMetadata() { + *this = ::std::move(from); + } + + inline HostComputeMetadata& operator=(HostComputeMetadata&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const HostComputeMetadata& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const HostComputeMetadata* internal_default_instance() { + return reinterpret_cast( + &_HostComputeMetadata_default_instance_); + } + static constexpr int kIndexInFileMessages = + 2; + + void UnsafeArenaSwap(HostComputeMetadata* other); + void Swap(HostComputeMetadata* other); + friend void swap(HostComputeMetadata& a, HostComputeMetadata& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline HostComputeMetadata* New() const final { + return CreateMaybeMessage(NULL); + } + + HostComputeMetadata* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const HostComputeMetadata& from); + void MergeFrom(const HostComputeMetadata& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(HostComputeMetadata* other); + protected: + explicit HostComputeMetadata(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated .tensorflow.tf2xla.HostTransferMetadata device_to_host = 1; + int device_to_host_size() const; + void clear_device_to_host(); + static const int kDeviceToHostFieldNumber = 1; + ::tensorflow::tf2xla::HostTransferMetadata* mutable_device_to_host(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::tf2xla::HostTransferMetadata >* + mutable_device_to_host(); + const ::tensorflow::tf2xla::HostTransferMetadata& device_to_host(int index) const; + ::tensorflow::tf2xla::HostTransferMetadata* add_device_to_host(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::tf2xla::HostTransferMetadata >& + device_to_host() const; + + // repeated .tensorflow.tf2xla.HostTransferMetadata host_to_device = 2; + int host_to_device_size() const; + void clear_host_to_device(); + static const int kHostToDeviceFieldNumber = 2; + ::tensorflow::tf2xla::HostTransferMetadata* mutable_host_to_device(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::tf2xla::HostTransferMetadata >* + mutable_host_to_device(); + const ::tensorflow::tf2xla::HostTransferMetadata& host_to_device(int index) const; + ::tensorflow::tf2xla::HostTransferMetadata* add_host_to_device(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::tf2xla::HostTransferMetadata >& + host_to_device() const; + + // @@protoc_insertion_point(class_scope:tensorflow.tf2xla.HostComputeMetadata) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::tf2xla::HostTransferMetadata > device_to_host_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::tf2xla::HostTransferMetadata > host_to_device_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2ftf2xla_2fhost_5fcompute_5fmetadata_2eproto::TableStruct; +}; +// =================================================================== + + +// =================================================================== + +#ifdef __GNUC__ + #pragma GCC diagnostic push + #pragma GCC diagnostic ignored "-Wstrict-aliasing" +#endif // __GNUC__ +// TensorMetadata + +// .tensorflow.DataType type = 1; +inline void TensorMetadata::clear_type() { + type_ = 0; +} +inline ::tensorflow::DataType TensorMetadata::type() const { + // @@protoc_insertion_point(field_get:tensorflow.tf2xla.TensorMetadata.type) + return static_cast< ::tensorflow::DataType >(type_); +} +inline void TensorMetadata::set_type(::tensorflow::DataType value) { + + type_ = value; + // @@protoc_insertion_point(field_set:tensorflow.tf2xla.TensorMetadata.type) +} + +// .tensorflow.TensorShapeProto shape = 2; +inline bool TensorMetadata::has_shape() const { + return this != internal_default_instance() && shape_ != NULL; +} +inline const ::tensorflow::TensorShapeProto& TensorMetadata::_internal_shape() const { + return *shape_; +} +inline const ::tensorflow::TensorShapeProto& TensorMetadata::shape() const { + const ::tensorflow::TensorShapeProto* p = shape_; + // @@protoc_insertion_point(field_get:tensorflow.tf2xla.TensorMetadata.shape) + return p != NULL ? *p : *reinterpret_cast( + &::tensorflow::_TensorShapeProto_default_instance_); +} +inline ::tensorflow::TensorShapeProto* TensorMetadata::release_shape() { + // @@protoc_insertion_point(field_release:tensorflow.tf2xla.TensorMetadata.shape) + + ::tensorflow::TensorShapeProto* temp = shape_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + shape_ = NULL; + return temp; +} +inline ::tensorflow::TensorShapeProto* TensorMetadata::unsafe_arena_release_shape() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.tf2xla.TensorMetadata.shape) + + ::tensorflow::TensorShapeProto* temp = shape_; + shape_ = NULL; + return temp; +} +inline ::tensorflow::TensorShapeProto* TensorMetadata::mutable_shape() { + + if (shape_ == NULL) { + auto* p = CreateMaybeMessage<::tensorflow::TensorShapeProto>(GetArenaNoVirtual()); + shape_ = p; + } + // @@protoc_insertion_point(field_mutable:tensorflow.tf2xla.TensorMetadata.shape) + return shape_; +} +inline void TensorMetadata::set_allocated_shape(::tensorflow::TensorShapeProto* shape) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(shape_); + } + if (shape) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(shape)->GetArena(); + if (message_arena != submessage_arena) { + shape = ::google::protobuf::internal::GetOwnedMessage( + message_arena, shape, submessage_arena); + } + + } else { + + } + shape_ = shape; + // @@protoc_insertion_point(field_set_allocated:tensorflow.tf2xla.TensorMetadata.shape) +} + +// ------------------------------------------------------------------- + +// HostTransferMetadata + +// string key = 1; +inline void HostTransferMetadata::clear_key() { + key_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& HostTransferMetadata::key() const { + // @@protoc_insertion_point(field_get:tensorflow.tf2xla.HostTransferMetadata.key) + return key_.Get(); +} +inline void HostTransferMetadata::set_key(const ::std::string& value) { + + key_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.tf2xla.HostTransferMetadata.key) +} +#if LANG_CXX11 +inline void HostTransferMetadata::set_key(::std::string&& value) { + + key_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.tf2xla.HostTransferMetadata.key) +} +#endif +inline void HostTransferMetadata::set_key(const char* value) { + GOOGLE_DCHECK(value != NULL); + + key_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.tf2xla.HostTransferMetadata.key) +} +inline void HostTransferMetadata::set_key(const char* value, + size_t size) { + + key_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.tf2xla.HostTransferMetadata.key) +} +inline ::std::string* HostTransferMetadata::mutable_key() { + + // @@protoc_insertion_point(field_mutable:tensorflow.tf2xla.HostTransferMetadata.key) + return key_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* HostTransferMetadata::release_key() { + // @@protoc_insertion_point(field_release:tensorflow.tf2xla.HostTransferMetadata.key) + + return key_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void HostTransferMetadata::set_allocated_key(::std::string* key) { + if (key != NULL) { + + } else { + + } + key_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), key, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.tf2xla.HostTransferMetadata.key) +} +inline ::std::string* HostTransferMetadata::unsafe_arena_release_key() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.tf2xla.HostTransferMetadata.key) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return key_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void HostTransferMetadata::unsafe_arena_set_allocated_key( + ::std::string* key) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (key != NULL) { + + } else { + + } + key_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + key, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.tf2xla.HostTransferMetadata.key) +} + +// repeated .tensorflow.tf2xla.TensorMetadata metadata = 2; +inline int HostTransferMetadata::metadata_size() const { + return metadata_.size(); +} +inline void HostTransferMetadata::clear_metadata() { + metadata_.Clear(); +} +inline ::tensorflow::tf2xla::TensorMetadata* HostTransferMetadata::mutable_metadata(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.tf2xla.HostTransferMetadata.metadata) + return metadata_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::tf2xla::TensorMetadata >* +HostTransferMetadata::mutable_metadata() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.tf2xla.HostTransferMetadata.metadata) + return &metadata_; +} +inline const ::tensorflow::tf2xla::TensorMetadata& HostTransferMetadata::metadata(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.tf2xla.HostTransferMetadata.metadata) + return metadata_.Get(index); +} +inline ::tensorflow::tf2xla::TensorMetadata* HostTransferMetadata::add_metadata() { + // @@protoc_insertion_point(field_add:tensorflow.tf2xla.HostTransferMetadata.metadata) + return metadata_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::tf2xla::TensorMetadata >& +HostTransferMetadata::metadata() const { + // @@protoc_insertion_point(field_list:tensorflow.tf2xla.HostTransferMetadata.metadata) + return metadata_; +} + +// ------------------------------------------------------------------- + +// HostComputeMetadata + +// repeated .tensorflow.tf2xla.HostTransferMetadata device_to_host = 1; +inline int HostComputeMetadata::device_to_host_size() const { + return device_to_host_.size(); +} +inline void HostComputeMetadata::clear_device_to_host() { + device_to_host_.Clear(); +} +inline ::tensorflow::tf2xla::HostTransferMetadata* HostComputeMetadata::mutable_device_to_host(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.tf2xla.HostComputeMetadata.device_to_host) + return device_to_host_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::tf2xla::HostTransferMetadata >* +HostComputeMetadata::mutable_device_to_host() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.tf2xla.HostComputeMetadata.device_to_host) + return &device_to_host_; +} +inline const ::tensorflow::tf2xla::HostTransferMetadata& HostComputeMetadata::device_to_host(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.tf2xla.HostComputeMetadata.device_to_host) + return device_to_host_.Get(index); +} +inline ::tensorflow::tf2xla::HostTransferMetadata* HostComputeMetadata::add_device_to_host() { + // @@protoc_insertion_point(field_add:tensorflow.tf2xla.HostComputeMetadata.device_to_host) + return device_to_host_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::tf2xla::HostTransferMetadata >& +HostComputeMetadata::device_to_host() const { + // @@protoc_insertion_point(field_list:tensorflow.tf2xla.HostComputeMetadata.device_to_host) + return device_to_host_; +} + +// repeated .tensorflow.tf2xla.HostTransferMetadata host_to_device = 2; +inline int HostComputeMetadata::host_to_device_size() const { + return host_to_device_.size(); +} +inline void HostComputeMetadata::clear_host_to_device() { + host_to_device_.Clear(); +} +inline ::tensorflow::tf2xla::HostTransferMetadata* HostComputeMetadata::mutable_host_to_device(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.tf2xla.HostComputeMetadata.host_to_device) + return host_to_device_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::tf2xla::HostTransferMetadata >* +HostComputeMetadata::mutable_host_to_device() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.tf2xla.HostComputeMetadata.host_to_device) + return &host_to_device_; +} +inline const ::tensorflow::tf2xla::HostTransferMetadata& HostComputeMetadata::host_to_device(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.tf2xla.HostComputeMetadata.host_to_device) + return host_to_device_.Get(index); +} +inline ::tensorflow::tf2xla::HostTransferMetadata* HostComputeMetadata::add_host_to_device() { + // @@protoc_insertion_point(field_add:tensorflow.tf2xla.HostComputeMetadata.host_to_device) + return host_to_device_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::tf2xla::HostTransferMetadata >& +HostComputeMetadata::host_to_device() const { + // @@protoc_insertion_point(field_list:tensorflow.tf2xla.HostComputeMetadata.host_to_device) + return host_to_device_; +} + +#ifdef __GNUC__ + #pragma GCC diagnostic pop +#endif // __GNUC__ +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + + +// @@protoc_insertion_point(namespace_scope) + +} // namespace tf2xla +} // namespace tensorflow + +// @@protoc_insertion_point(global_scope) + +#endif // PROTOBUF_INCLUDED_tensorflow_2fcompiler_2ftf2xla_2fhost_5fcompute_5fmetadata_2eproto diff --git a/linux/include/tensorflow/compiler/tf2xla/kernels/conv_op_helpers.h b/linux/include/tensorflow/compiler/tf2xla/kernels/conv_op_helpers.h new file mode 100644 index 000000000..6e1b70a47 --- /dev/null +++ b/linux/include/tensorflow/compiler/tf2xla/kernels/conv_op_helpers.h @@ -0,0 +1,69 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_TF2XLA_KERNELS_CONV_OP_HELPERS_H_ +#define TENSORFLOW_COMPILER_TF2XLA_KERNELS_CONV_OP_HELPERS_H_ + +#include + +#include "tensorflow/compiler/xla/client/xla_builder.h" +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/util/padding.h" +#include "tensorflow/core/util/tensor_format.h" + +// This header exposes utilities for translating TensorFlow convolution ops into +// XLA ops. +// +// conv_ops.cc contains lowerings for many of these TF convolution ops (e.g. +// Conv2D, Conv3DBackpropFilterV2), but you might want to use the utilities in +// this header to implement a new and exciting convolution op, for example a +// fused TensorFlow op that contains a convolution and other things. + +namespace tensorflow { + +// ConvOpAttrs contains all of the metadata necessary to specify a TF or XLA +// convolution. +struct ConvOpAttrs { + // Constructs a ConvOpAttrs, reading most of the attributes from `ctx`. + static xla::StatusOr Create(int num_spatial_dims, bool depthwise, + OpKernelConstruction* ctx); + + bool depthwise; + int num_spatial_dims; + std::vector dilations; + std::vector strides; + Padding padding; + TensorFormat data_format; +}; + +// Creates a new XLA forward or backward convolution with the given inputs and +// attributes. +xla::StatusOr MakeXlaForwardConvOp(StringPiece type_string, + xla::XlaOp conv_input, + xla::XlaOp filter, + const ConvOpAttrs& attrs); +xla::StatusOr MakeXlaBackpropInputConvOp( + StringPiece type_string, const xla::Shape& input_shape, xla::XlaOp filter, + xla::XlaOp out_backprop, const ConvOpAttrs& attrs); +xla::StatusOr MakeXlaBackpropFilterConvOp( + StringPiece type_string, xla::XlaOp activations, + const xla::Shape& filter_shape, xla::XlaOp gradients, + const ConvOpAttrs& attrs); + +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_TF2XLA_KERNELS_CONV_OP_HELPERS_H_ diff --git a/linux/include/tensorflow/compiler/tf2xla/kernels/cwise_ops.h b/linux/include/tensorflow/compiler/tf2xla/kernels/cwise_ops.h new file mode 100644 index 000000000..6653944a9 --- /dev/null +++ b/linux/include/tensorflow/compiler/tf2xla/kernels/cwise_ops.h @@ -0,0 +1,76 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// XLA-specific base classes for Unary and Binary Ops. + +#ifndef TENSORFLOW_COMPILER_TF2XLA_KERNELS_CWISE_OPS_H_ +#define TENSORFLOW_COMPILER_TF2XLA_KERNELS_CWISE_OPS_H_ + +#include "tensorflow/compiler/tf2xla/xla_op_kernel.h" +#include "tensorflow/compiler/xla/client/client_library.h" +#include "tensorflow/compiler/xla/client/xla_builder.h" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/util/bcast.h" + +namespace tensorflow { + +// Coefficient-wise binary operations. Each binary Op expects two +// inputs that can be broadcast to the same shape. The base class +// contains pure virtual methods to override: description is a textual +// description of the operation; and Computation adds the +// implementation of the operation to a xla::XlaBuilder. For most +// arithmetic Ops XLA handles the broadcasting automatically given the input +// tensors. +class XlaBinaryOp : public XlaOpKernel { + public: + explicit XlaBinaryOp(OpKernelConstruction* ctx) : XlaOpKernel(ctx) { + const DataType lhs = BaseType(input_type(0)); + const DataType rhs = BaseType(input_type(1)); + OP_REQUIRES(ctx, lhs == rhs, + errors::InvalidArgument("Input types of binary op must match")); + } + ~XlaBinaryOp() override {} + + // Implement the (tensor,tensor)->tensor lambda that should be + // applied to the inputs. The desired computation should be added to + // 'tc->builder()' and '(lhs,rhs)' are the function's inputs and + // (lhs_shape,rhs_shape) are their respective + // shapes. 'broadcast_helper' contains metadata about the shapes of + // the inputs and the dimensions that need to be broadcast, which + // may be useful for Ops that can't use standard XLA automatic + // broadcasting. 'extend_dimension' is non-empty if lhs and rhs have + // different ranks, and indicates which dimensions of the + // higher-rank input should be matched when broadcasting the + // lower-rank input. See comment below and the documentation on broadcasting + // in the XLA documentation. + virtual xla::XlaOp Computation( + XlaOpKernelContext* ctx, const xla::XlaOp& lhs, + const absl::Span& lhs_shape, const xla::XlaOp& rhs, + const absl::Span& rhs_shape, const BCast& broadcast_helper, + const std::vector& extend_dimensions) = 0; + + void Compile(XlaOpKernelContext* ctx) override; + + // Helper function that performs the broadcasting described by + // 'broadcast_helper', yielding arguments 'lhs' and 'rhs' that have the same + // shape. + static std::pair Broadcast( + xla::XlaBuilder* builder, const xla::XlaOp& lhs, const xla::XlaOp& rhs, + const BCast& broadcast_helper); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_TF2XLA_KERNELS_CWISE_OPS_H_ diff --git a/linux/include/tensorflow/compiler/tf2xla/kernels/gather_op_helpers.h b/linux/include/tensorflow/compiler/tf2xla/kernels/gather_op_helpers.h new file mode 100644 index 000000000..92346283c --- /dev/null +++ b/linux/include/tensorflow/compiler/tf2xla/kernels/gather_op_helpers.h @@ -0,0 +1,44 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// Helper methods for XLA Gather Ops. + +#ifndef TENSORFLOW_COMPILER_TF2XLA_KERNELS_GATHER_OP_HELPERS_H_ +#define TENSORFLOW_COMPILER_TF2XLA_KERNELS_GATHER_OP_HELPERS_H_ + +#include "tensorflow/compiler/tf2xla/xla_op_kernel.h" +#include "tensorflow/compiler/xla/client/client_library.h" +#include "tensorflow/compiler/xla/client/xla_builder.h" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/util/bcast.h" + +namespace tensorflow { + +// Adds to builder an XLA computation that performs a gather on input (of +// shape input_shape) keyed on indices (of shape indices_shape). +// +// index_type must be must be DT_INT32 or DT_INT64. +// If `indices_are_nd` is true, the last dimension of `indices` are treated as +// a multidimensional index values. Otherwise, `indices` is treated as a tensor +// of scalar indices. +Status XlaGather(const xla::XlaOp& input, const TensorShape& input_shape, + const xla::XlaOp& indices, const TensorShape& indices_shape, + int64 axis, bool indices_are_nd, DataType dtype, + DataType index_type, xla::XlaBuilder* builder, + xla::XlaOp* gather_output); + +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_TF2XLA_KERNELS_GATHER_OP_HELPERS_H_ diff --git a/linux/include/tensorflow/compiler/tf2xla/kernels/if_op.h b/linux/include/tensorflow/compiler/tf2xla/kernels/if_op.h new file mode 100644 index 000000000..7783e13a8 --- /dev/null +++ b/linux/include/tensorflow/compiler/tf2xla/kernels/if_op.h @@ -0,0 +1,61 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_TF2XLA_KERNELS_IF_OP_H_ +#define TENSORFLOW_COMPILER_TF2XLA_KERNELS_IF_OP_H_ + +#include "tensorflow/compiler/tf2xla/xla_op_kernel.h" +#include "tensorflow/core/framework/attr_value.pb.h" + +namespace tensorflow { + +// This TensorFlow op provides a functional conditional primitive. +// +// The outputs of the then/else branches must agree on the number, types, and +// shapes of the Tensors carried around the two bodies. +// +// Computations in then/else bodies may read from and write to resource +// variables. +// Resource variables may be passed as arguments to the then/else function's +// bodies. The XlaCompiler converts resource variable arguments +// into parameters to the XLA computation and moves them to the end of the +// parameter list, and by using the `return_updated_values_for_all_variables` +// we ensure that all variables that appear in the input also appear at the +// end of the then/else bodies output. This ensures the then/else bodies output +// signatures match. +// +// It is the user's responsibility to ensure that each non-variable _Arg matches +// the corresponding _Retval. +class XlaIfOp : public XlaOpKernel { + public: + explicit XlaIfOp(OpKernelConstruction* ctx); + + void Compile(XlaOpKernelContext* ctx) override; + + private: + TF_DISALLOW_COPY_AND_ASSIGN(XlaIfOp); + + NameAttrList then_branch_; + NameAttrList else_branch_; + DataType cond_type_; + DataTypeVector input_types_; + DataTypeVector output_types_; + bool has_token_input_output_; + std::vector token_input_nodes_; +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_TF2XLA_KERNELS_IF_OP_H_ diff --git a/linux/include/tensorflow/compiler/tf2xla/kernels/index_ops.h b/linux/include/tensorflow/compiler/tf2xla/kernels/index_ops.h new file mode 100644 index 000000000..ef2b9e6b6 --- /dev/null +++ b/linux/include/tensorflow/compiler/tf2xla/kernels/index_ops.h @@ -0,0 +1,42 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// Declarations of the ArgMax/ArgMin ops using a pure XLA implementation. + +#ifndef TENSORFLOW_COMPILER_TF2XLA_KERNELS_INDEX_OPS_H_ +#define TENSORFLOW_COMPILER_TF2XLA_KERNELS_INDEX_OPS_H_ + +#include "tensorflow/compiler/tf2xla/xla_op_kernel.h" +#include "tensorflow/core/framework/op_kernel.h" + +namespace tensorflow { + +class XlaArgMinMaxOp : public XlaOpKernel { + public: + explicit XlaArgMinMaxOp(OpKernelConstruction* ctx, bool is_min); + void Compile(XlaOpKernelContext* ctx) override; + + private: + const bool is_min_; // Are we computing ArgMin (true) or ArgMax (false)? +}; + +class XlaArgMaxOp : public XlaArgMinMaxOp { + public: + explicit XlaArgMaxOp(OpKernelConstruction* ctx); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_TF2XLA_KERNELS_INDEX_OPS_H_ diff --git a/linux/include/tensorflow/compiler/tf2xla/kernels/reduction_ops.h b/linux/include/tensorflow/compiler/tf2xla/kernels/reduction_ops.h new file mode 100644 index 000000000..466e79828 --- /dev/null +++ b/linux/include/tensorflow/compiler/tf2xla/kernels/reduction_ops.h @@ -0,0 +1,72 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// XLA-specific base classes for Reduction Ops. + +#ifndef TENSORFLOW_COMPILER_TF2XLA_KERNELS_REDUCTION_OPS_H_ +#define TENSORFLOW_COMPILER_TF2XLA_KERNELS_REDUCTION_OPS_H_ + +#include "tensorflow/compiler/tf2xla/xla_op_kernel.h" +#include "tensorflow/compiler/xla/client/xla_builder.h" +#include "tensorflow/core/framework/op_kernel.h" + +namespace tensorflow { + +// Reduction operations. The base class contains pure virtual methods +// to override: description is a textual description of the mapped +// function; InitialValue constructs the base case for the reduction; +// BuildReducer adds the implementation of the reduction lambda to a +// xla::XlaBuilder and BuildFinalizer adds the +// implementation of the finalizer lambda (if there is one) to a +// xla::XlaBuilder. +class XlaReductionOp : public XlaOpKernel { + public: + XlaReductionOp(OpKernelConstruction* ctx, DataType reduction_type); + ~XlaReductionOp() override {} + + // Return the base case for the reduction. + virtual xla::XlaOp InitialValue(xla::XlaBuilder* builder) = 0; + + // Implement the (scalar,scalar)->scalar lambda that should be + // applied to each pair of elements to be reduced. The desired + // computation should be added to 'builder' and + // '(scalar_lhs,scalar_rhs)' are the function's inputs. + virtual void BuildReducer(xla::XlaBuilder* builder, + const xla::XlaOp& scalar_lhs, + const xla::XlaOp& scalar_rhs) = 0; + + // Applies a transformation to the output of the reduction. The desired + // computation should be added to 'builder'. Argument 'reduce_output' is the + // output of the reduction. 'num_elements_reduced' is the number of elements + // that contributed to the reduction. Returns the transformed reduction + // output, Defaults to returning 'reduce_output' unchanged. + virtual xla::XlaOp BuildFinalizer(xla::XlaBuilder* builder, + const xla::XlaOp& reduce_output, + int64 num_elements_reduced); + + void Compile(XlaOpKernelContext* ctx) override; + + private: + // True if the number of dimensions should be maintained. + bool keep_dims_; + + protected: + DataType reduction_type_; + xla::PrimitiveType xla_reduction_type_; +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_TF2XLA_KERNELS_REDUCTION_OPS_H_ diff --git a/linux/include/tensorflow/compiler/tf2xla/kernels/shape_util.h b/linux/include/tensorflow/compiler/tf2xla/kernels/shape_util.h new file mode 100644 index 000000000..ca57be3d4 --- /dev/null +++ b/linux/include/tensorflow/compiler/tf2xla/kernels/shape_util.h @@ -0,0 +1,34 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_TF2XLA_KERNELS_SHAPE_UTIL_H_ +#define TENSORFLOW_COMPILER_TF2XLA_KERNELS_SHAPE_UTIL_H_ + +#include + +#include "tensorflow/core/framework/tensor.h" + +namespace tensorflow { + +// Converts a TensorShape to a constant Tensor. +// +// The input TensorShape input_shape is used to populate the elements of +// shape_constant, which is modified in place. +Status TensorShapeToConstant(const TensorShape& input_shape, + Tensor* shape_constant); + +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_TF2XLA_KERNELS_SHAPE_UTIL_H_ diff --git a/linux/include/tensorflow/compiler/tf2xla/kernels/while_op.h b/linux/include/tensorflow/compiler/tf2xla/kernels/while_op.h new file mode 100644 index 000000000..aeeff40e6 --- /dev/null +++ b/linux/include/tensorflow/compiler/tf2xla/kernels/while_op.h @@ -0,0 +1,67 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_TF2XLA_KERNELS_WHILE_OP_H_ +#define TENSORFLOW_COMPILER_TF2XLA_KERNELS_WHILE_OP_H_ + +#include "tensorflow/compiler/tf2xla/xla_op_kernel.h" +#include "tensorflow/core/framework/attr_value.pb.h" + +namespace tensorflow { + +// This TensorFlow op provides a functional iteration primitive. +// +// The inputs and outputs of the loop body must agree on the number, types, and +// shapes of the Tensors carried around the loop body. +// +// Computations in while loops may read from and write to resource variables. +// Resource variables may be passed as arguments to a function's body and +// condition functions. The XlaCompiler converts resource variable arguments +// into parameters to the XLA computation and moves them to the end of the +// parameter list, and by using the `return_updated_values_for_all_variables` +// we ensure that all variables that appear in the input also appear at the +// end of the body's output. This ensures the loop body's input and output +// signatures match. +// +// It is the user's responsibility to ensure that each non-variable _Arg matches +// the corresponding _Retval. +// +// For example, suppose we have a loop body with arguments: +// DT_INT32, DT_RESOURCE (pointing to a DT_BOOL var), DT_FLOAT +// and return values +// DT_INT32, DT_FLOAT +// It is an error for the body to return DT_RESOURCE values. +// +// The body will be lowered into an XLA computation that takes and returns a +// tuple with XLA type (I32, F32, PRED). Note the resource variable appears at +// the end of both the loop body's input and output argument lists. +class XlaWhileOp : public XlaOpKernel { + public: + explicit XlaWhileOp(OpKernelConstruction* ctx); + + void Compile(XlaOpKernelContext* ctx) override; + + private: + NameAttrList cond_name_attr_; + NameAttrList body_name_attr_; + bool has_token_input_output_; + std::vector token_input_nodes_; + + TF_DISALLOW_COPY_AND_ASSIGN(XlaWhileOp); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_TF2XLA_KERNELS_WHILE_OP_H_ diff --git a/linux/include/tensorflow/compiler/tf2xla/lib/batch_dot.h b/linux/include/tensorflow/compiler/tf2xla/lib/batch_dot.h new file mode 100644 index 000000000..6edd63a4d --- /dev/null +++ b/linux/include/tensorflow/compiler/tf2xla/lib/batch_dot.h @@ -0,0 +1,54 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_TF2XLA_LIB_BATCH_DOT_H_ +#define TENSORFLOW_COMPILER_TF2XLA_LIB_BATCH_DOT_H_ + +#include "tensorflow/compiler/xla/client/xla_builder.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" + +namespace tensorflow { + +// Multiplies slices of two tensors in batches. + +// Multiplies all slices of `Tensor` `x` and `y` (each slice can be +// viewed as an element of a batch), and arranges the individual results +// in a single output tensor of the same batch size. Each of the +// individual slices can optionally be transposed before multiplication by +// setting the `transpose_x` or `transpose_y` flag to `true`. Similarly, each +// can be elementwise-complex-conjugated by setting the `conjugate_x` or +// `conjugate_y` flag to `true`. To apply a Hermitian adjoint to `x`, set both +// `transpose_x` and `conjugate_x` to `true`, and analogously for `y`. +// +// The input tensors `x` and `y` are 2-D or higher with shape `[..., r_x, c_x]` +// and `[..., r_y, c_y]`. +// +// The output tensor is 2-D or higher with shape `[..., r_o, c_o]`, where: +// +// r_o = c_x if transpose_x else r_x +// c_o = r_y if transpose_y else c_y +// +// It is computed as: +// +// output[..., :, :] = matrix(x[..., :, :]) * matrix(y[..., :, :]) +xla::XlaOp BatchDot( + xla::XlaOp x, xla::XlaOp y, bool transpose_x = false, + bool transpose_y = false, bool conjugate_x = false, + bool conjugate_y = false, + xla::PrecisionConfig::Precision precision = xla::PrecisionConfig::DEFAULT); + +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_TF2XLA_LIB_BATCH_DOT_H_ diff --git a/linux/include/tensorflow/compiler/tf2xla/lib/cholesky.h b/linux/include/tensorflow/compiler/tf2xla/lib/cholesky.h new file mode 100644 index 000000000..9a561c34b --- /dev/null +++ b/linux/include/tensorflow/compiler/tf2xla/lib/cholesky.h @@ -0,0 +1,39 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_TF2XLA_LIB_CHOLESKY_H_ +#define TENSORFLOW_COMPILER_TF2XLA_LIB_CHOLESKY_H_ + +#include "tensorflow/compiler/xla/client/xla_builder.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" + +namespace tensorflow { + +// Computes the Cholesky decompositions of a batch of symmetric positive +// definite matrices. +// `a` must be a (batched) square matrix; i.e., it must have rank >= 2 with the +// two minor dimensions equal. +// The algorithm implements a blocked Cholesky decomposition; `block_size` is +// the block size to use. +// TODO(phawkins): check for negative values on the diagonal and return an +// error, instead of silently yielding NaNs. +// TODO(znado): handle the complex Hermitian case +xla::XlaOp Cholesky( + xla::XlaOp a, int64 block_size = 256, + xla::PrecisionConfig::Precision precision = xla::PrecisionConfig::HIGHEST); + +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_TF2XLA_LIB_CHOLESKY_H_ diff --git a/linux/include/tensorflow/compiler/tf2xla/lib/qr.h b/linux/include/tensorflow/compiler/tf2xla/lib/qr.h new file mode 100644 index 000000000..24b537ac8 --- /dev/null +++ b/linux/include/tensorflow/compiler/tf2xla/lib/qr.h @@ -0,0 +1,42 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_TF2XLA_LIB_QR_H_ +#define TENSORFLOW_COMPILER_TF2XLA_LIB_QR_H_ + +#include "tensorflow/compiler/xla/client/xla_builder.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" + +namespace tensorflow { + +// Computes the QR decompositions of a batch of matrices. That is, +// given a (batched) matrix a, computes an orthonormal matrix Q and an +// upper-triangular matrix R such that a = QR. +// `a` must be a (batched) matrix of size [..., m, n]. +// The algorithm implements a blocked QR decomposition; `block_size` is +// the block size to use. +// TODO(phawkins): handle the complex case. +struct QRDecompositionResult { + xla::XlaOp q; + xla::XlaOp r; +}; + +xla::StatusOr QRDecomposition( + xla::XlaOp a, bool full_matrices, int64 block_size = 128, + xla::PrecisionConfig::Precision precision = xla::PrecisionConfig::HIGHEST); + +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_TF2XLA_LIB_QR_H_ diff --git a/linux/include/tensorflow/compiler/tf2xla/lib/random.h b/linux/include/tensorflow/compiler/tf2xla/lib/random.h new file mode 100644 index 000000000..59fc5d043 --- /dev/null +++ b/linux/include/tensorflow/compiler/tf2xla/lib/random.h @@ -0,0 +1,35 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_TF2XLA_LIB_RANDOM_H_ +#define TENSORFLOW_COMPILER_TF2XLA_LIB_RANDOM_H_ + +#include "tensorflow/compiler/xla/client/xla_builder.h" +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/core/framework/types.pb.h" + +namespace tensorflow { + +// Builds an array filled with values sampled from a truncated normal +// distribution such that no values are greater than two or less than negative +// two. +// +// The "uniform" parameter must be an array of random numbers distributed in +// (0,1). +xla::XlaOp TruncatedNormal(xla::XlaOp uniform); + +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_TF2XLA_LIB_RANDOM_H_ diff --git a/linux/include/tensorflow/compiler/tf2xla/lib/scatter.h b/linux/include/tensorflow/compiler/tf2xla/lib/scatter.h new file mode 100644 index 000000000..4cf478c4b --- /dev/null +++ b/linux/include/tensorflow/compiler/tf2xla/lib/scatter.h @@ -0,0 +1,55 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_TF2XLA_LIB_SCATTER_H_ +#define TENSORFLOW_COMPILER_TF2XLA_LIB_SCATTER_H_ + +#include + +#include "tensorflow/compiler/xla/client/xla_builder.h" +#include "tensorflow/compiler/xla/client/xla_computation.h" +#include "tensorflow/compiler/xla/statusor.h" + +namespace tensorflow { + +// Builds an XLA computation that performs a scatter operation on `buffer`, +// returning an updated buffer. +// For each i0, i1, ..., sets +// buffer[indices[i0, i1, ...], ...] := updates[i0, i1, ...] +// +// If `indices_are_vectors` is false, then each index in indices is a scalar, +// and the shape of `indices` must be a prefix of the shape of updates. +// Otherwise, `indices_are_vectors`, then indices are multidimensional and the +// minor dimension of `indices` represents a vector of indices. +// +// If `updates` is a scalar, then it will be broadcasted into the expected shape +// of updates. +// +// If any part of the update region is out-of-bounds, the corresponding update +// is discarded. +// +// If a `combiner` is provided, updates are combined with the existing values in +// the buffer using the combiner function. Otherwise, the updates replace the +// existing values. The order of updates is implementation-defined. +xla::StatusOr XlaScatter( + const xla::XlaOp& buffer, const xla::XlaOp& updates, + const xla::XlaOp& indices, bool indices_are_vectors, + const std::function& + combiner, + xla::XlaBuilder* builder); + +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_TF2XLA_LIB_SCATTER_H_ diff --git a/linux/include/tensorflow/compiler/tf2xla/lib/triangular_solve.h b/linux/include/tensorflow/compiler/tf2xla/lib/triangular_solve.h new file mode 100644 index 000000000..2303234f3 --- /dev/null +++ b/linux/include/tensorflow/compiler/tf2xla/lib/triangular_solve.h @@ -0,0 +1,67 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_TF2XLA_LIB_TRIANGULAR_SOLVE_H_ +#define TENSORFLOW_COMPILER_TF2XLA_LIB_TRIANGULAR_SOLVE_H_ + +#include "tensorflow/compiler/xla/client/xla_builder.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" + +namespace tensorflow { + +// Solves systems of linear equations with lower or upper triangular coefficient +// matrices by forward- or back-substitution. Broadcasting along leading +// dimensions, this routine solves one of the matrix systems +// `op(a) * x = b`, or `x * op(a) = b`, +// for the variable `x` given `a` and `b`, where `op(a)` is either +// `op(a) = a`, or `op(a) = transpose(a)`, or `op(a) = conj(transpose(a))`. +// That is, the innermost matrices in the output satisfy a scalar system +// depending on the value of the value of (left_side, transpose_a, conjugate_a) +// according to: +// (F, F, F) => `output[..., i, k] a[..., k, j] = b[..., i, j]`, +// (F, F, T) => `output[..., i, k] a*[..., k, j] = b[..., i, j]`, +// (F, T, F) => `output[..., i, k] a[..., j, k] = b[..., i, j]`, +// (F, T, T) => `output[..., i, k] a*[..., j, k] = b[..., i, j]`, +// (T, F, F) => ` a[..., i, k] output[..., k, j] = b[..., i, j]`, +// (T, F, T) => `a*[..., i, k] output[..., k, j] = b[..., i, j]`, +// (T, T, F) => ` a[..., i, k] output[..., j, k] = b[..., i, j]`, +// (T, T, T) => `a*[..., i, k] output[..., j, k] = b[..., i, j]`, +// where * denotes complex conjugation and where the index `k` is summed over. +// +// `a` is a tensor of shape `[..., M, M]` whose innermost 2 dimensions form +// square matrices. If lower is true (false), then the strictly upper (lower) +// triangular part of each innermost matrix in `a` is assumed to be zero and is +// not accessed. +// `b` is a tensor of shape `[..., M, K]` if left_side is true, otherwise a +// tensor of shape `[..., K, M]`. +// `left_side` is a boolean, indicating whether to solve a system of the form +// op(a) * x = b (true) or x * op(a) = b (false). +// `lower` is a boolean, indicating whether the argument `a` is lower-triangular +// (true) or upper-triangular (false). +// `transpose_a` is a boolean indicating whether the matrix `a` is transposed. +// `conjugate_a` is a boolean indicating whether the entries of `a` are complex +// conjugated (independently of whether they are transposed), so that when both +// transpose_a and conjugate_a are true the effect is a Hermitian adjoint. +// +// Uses a blocked algorithm if `block_size` is > 1; if block_size == 1 then no +// blocking is used. +xla::XlaOp TriangularSolve( + xla::XlaOp a, xla::XlaOp b, bool left_side, bool lower, bool transpose_a, + bool conjugate_a, int64 block_size = 128, + xla::PrecisionConfig::Precision precision = xla::PrecisionConfig::HIGHEST); + +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_TF2XLA_LIB_TRIANGULAR_SOLVE_H_ diff --git a/linux/include/tensorflow/compiler/tf2xla/lib/util.h b/linux/include/tensorflow/compiler/tf2xla/lib/util.h new file mode 100644 index 000000000..80e9e5b00 --- /dev/null +++ b/linux/include/tensorflow/compiler/tf2xla/lib/util.h @@ -0,0 +1,81 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_TF2XLA_LIB_UTIL_H_ +#define TENSORFLOW_COMPILER_TF2XLA_LIB_UTIL_H_ + +#include "absl/types/span.h" +#include "tensorflow/compiler/xla/client/xla_builder.h" +#include "tensorflow/compiler/xla/client/xla_computation.h" +#include "tensorflow/compiler/xla/statusor.h" + +namespace tensorflow { + +// Returns a floating point scalar constant of 'type' with 'value'. +// If 'type' is complex, returns a real value with zero imaginary component. +xla::XlaOp FloatLiteral(xla::XlaBuilder* builder, xla::PrimitiveType type, + double value); + +// Makes a 1D tensor [0, ..., x, y] from two tensors x and y with zeros +// prepended until the array is length n_dims. +xla::XlaOp PrependZerosInMajorDims(xla::XlaOp x, + absl::Span starts); + +// Returns a integer scalar constant of 'type' with 'value'. +// If 'type' is complex, returns a real value with zero imaginary component. +xla::XlaOp IntegerLiteral(xla::XlaBuilder* builder, xla::PrimitiveType type, + int64 value); + +// Builds a vector of zeros of length rank(x) with the last values being +// those in `starts`. +xla::XlaOp PrependZerosInMajorDims(xla::XlaOp x, + absl::Span starts); + +// Performs a slice in the minor dimensions of a Tensor. +xla::XlaOp SliceInMinorDims(xla::XlaOp x, absl::Span start, + absl::Span end); + +// Returns the concatenation of `xs` and `ys`. +std::vector ConcatVectors(absl::Span xs, + absl::Span ys); + +// Performs a dynamic slice in the minor dimensions of a Tensor. +xla::XlaOp DynamicSliceInMinorDims(xla::XlaOp x, + absl::Span starts, + absl::Span sizes); + +// Updates a slice of 'x', i.e., +// x[start[0], ..., start[n]] = update +xla::XlaOp UpdateSlice(xla::XlaOp x, xla::XlaOp update, + absl::Span start); + +// Updates a slice of 'x', where 'start' contains a list of minor dimensions: +// x[..., start[0], ..., start[n]] = update +xla::XlaOp UpdateSliceInMinorDims(xla::XlaOp x, xla::XlaOp update, + absl::Span start); + +xla::XlaOp DynamicUpdateSliceInMinorDims(xla::XlaOp x, xla::XlaOp update, + absl::Span starts); + +// Transposes a stack of matrices `x` by swapping the last two dimensions. +xla::XlaOp TransposeInMinorDims(xla::XlaOp x); + +// Applies a complex conjugation operation if `a` is complex and `conjugate_a` +// is true, otherwise returns its argument. +xla::XlaOp MaybeConjugate(xla::XlaOp x, bool conjugate); + +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_TF2XLA_LIB_UTIL_H_ diff --git a/linux/include/tensorflow/compiler/tf2xla/lib/while_loop.h b/linux/include/tensorflow/compiler/tf2xla/lib/while_loop.h new file mode 100644 index 000000000..f2134bb44 --- /dev/null +++ b/linux/include/tensorflow/compiler/tf2xla/lib/while_loop.h @@ -0,0 +1,73 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_TF2XLA_LIB_WHILE_LOOP_H_ +#define TENSORFLOW_COMPILER_TF2XLA_LIB_WHILE_LOOP_H_ + +#include +#include + +#include "absl/strings/string_view.h" +#include "absl/types/span.h" +#include "tensorflow/compiler/xla/client/xla_builder.h" +#include "tensorflow/compiler/xla/client/xla_computation.h" +#include "tensorflow/compiler/xla/statusor.h" + +namespace tensorflow { + +// Function that builds a loop condition. Takes as input a sequence of input +// values, and returns a boolean value representing if the condition succeeds. +typedef std::function(absl::Span, + xla::XlaBuilder*)> + LoopConditionFunction; + +// Function that builds a loop body. Takes as input a sequence of input values +// and returns a sequence of output values. +typedef std::function>( + absl::Span, xla::XlaBuilder*)> + LoopBodyFunction; + +// Helper function for building an XLA while loop, where the values carried by +// the loop are a tuple of values, e.g., (a, b, c): +// while( +// condition: (a, b, c) -> bool, +// body: (a, b, c) -> (a, b, c) +// init: (a, b, c) +// ) +// 'name' is a descriptive name for the loop. +xla::StatusOr> XlaWhileLoop( + const LoopConditionFunction& condition_function, + const LoopBodyFunction& body_function, + absl::Span initial_values, absl::string_view name, + xla::XlaBuilder* builder); + +// Builds an XLA loop that repeats a computation `num_iterations` times. +// +// The body function (ForEachIndexBodyFunction) takes as input a pair of +// (current iteration number, loop-carried values), and returns an updated +// vector of the loop-carried values. +typedef std::function>( + xla::XlaOp, absl::Span, xla::XlaBuilder*)> + ForEachIndexBodyFunction; + +xla::StatusOr> XlaForEachIndex( + int64 num_iterations, xla::PrimitiveType num_iterations_type, + const ForEachIndexBodyFunction& body_function, + absl::Span initial_values, absl::string_view name, + xla::XlaBuilder* builder); + +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_TF2XLA_LIB_WHILE_LOOP_H_ diff --git a/linux/include/tensorflow/compiler/tf2xla/literal_util.h b/linux/include/tensorflow/compiler/tf2xla/literal_util.h new file mode 100644 index 000000000..1db7470ee --- /dev/null +++ b/linux/include/tensorflow/compiler/tf2xla/literal_util.h @@ -0,0 +1,67 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// Utilities for working with XLA Literals. + +#ifndef TENSORFLOW_COMPILER_TF2XLA_LITERAL_UTIL_H_ +#define TENSORFLOW_COMPILER_TF2XLA_LITERAL_UTIL_H_ + +#include "absl/types/span.h" +#include "tensorflow/compiler/xla/literal.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/lib/core/status.h" + +namespace tensorflow { + +// Returns a BorrowingLiteral that utilizes the same underlying buffer owned by +// 'host_tensor'. +Status HostTensorToBorrowingLiteral(const Tensor& host_tensor, + xla::BorrowingLiteral* literal); +// Returns a MutableBorrowingLiteral that utilizes the same underlying buffer +// owned by 'host_tensor', but is mutable via the xla::Literal methods. +Status HostTensorToMutableBorrowingLiteral( + Tensor* host_tensor, xla::MutableBorrowingLiteral* literal); +// Similar as above, except the literal shape is explicitly provided and used +// instead of obtaining it from the 'host_tensor'. The provided literal shape +// 'xla_shape' must be compatible with the shape of 'host_tensor'. +Status HostTensorToMutableBorrowingLiteral( + const xla::Shape& xla_shape, Tensor* host_tensor, + xla::MutableBorrowingLiteral* literal); + +// Returns a BorrowingLiteral tuple that utilizes the same underlying buffers +// owned by 'host_tensors'. +Status HostTensorsToBorrowingLiteralTuple(absl::Span host_tensors, + xla::BorrowingLiteral* literal); + +// Copies 'literal' to freshly allocated 'host_tensor', which is allocated of +// type . +// Fails if the literal's primitive type != +// DataTypeToPrimitiveType(target_type). Note that is not +// derivable from the type of , because multiple tensorflow types map +// to the same XLA type (e.g. INT32 and QINT32 both map to INT32 in +// XLA). +Status LiteralToHostTensor(const xla::LiteralSlice& literal, + DataType target_type, Tensor* host_tensor); + +// Copies the contents of 'literal' to a previously allocated tensor +// 'host_tensor'. The tensor and the literal must have the same number of +// elements and the same type. +Status CopyLiteralToHostTensor(const xla::LiteralSlice& literal, + Tensor* host_tensor); + +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_TF2XLA_LITERAL_UTIL_H_ diff --git a/linux/include/tensorflow/compiler/tf2xla/resource_operation_table.h b/linux/include/tensorflow/compiler/tf2xla/resource_operation_table.h new file mode 100644 index 000000000..61c7a56ff --- /dev/null +++ b/linux/include/tensorflow/compiler/tf2xla/resource_operation_table.h @@ -0,0 +1,71 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_TF2XLA_RESOURCE_OPERATION_TABLE_H_ +#define TENSORFLOW_COMPILER_TF2XLA_RESOURCE_OPERATION_TABLE_H_ + +#include +#include + +#include "absl/strings/string_view.h" +#include "tensorflow/core/platform/logging.h" + +// Exposes information about the resource operations supported by tf2xla in a +// structured form. + +namespace tensorflow { +enum class XlaResourceOpKind { + kRead, // Only reads from resources. + kWrite, // Only writes to resources. + kReadWrite // Reads from and writes to resources. +}; + +enum class XlaResourceKind { + kVariable, // Operates on resource variables. + kStack, // Operates on stacks. + kTensorArray // Operates on tensor arrays. +}; + +class XlaResourceOpInfo { + public: + explicit XlaResourceOpInfo(XlaResourceOpKind op_kind, + XlaResourceKind resource_kind) + : op_kind_(op_kind), resource_kind_(resource_kind) {} + + XlaResourceOpKind kind() const { return op_kind_; } + XlaResourceKind resource_kind() const { return resource_kind_; } + + static absl::string_view XlaResourceOpKindToString(XlaResourceOpKind op_kind); + + private: + XlaResourceOpKind op_kind_; + XlaResourceKind resource_kind_; +}; + +// Returns a XlaResourceOpInfo describing `op` if it is a resource operation +// supported by tf2xla, otherwise returns null (i.e. if this returns null then +// `op` is either not a resource operation or is unsupported by XLA). +const XlaResourceOpInfo* GetResourceOpInfoForOp(absl::string_view op); + +namespace resource_op_table_internal { +// NB! Implementation detail exposed for unit testing, do not use. +// +// Returns the set of resource operations known by this module. +std::vector GetKnownResourceOps(); +} // namespace resource_op_table_internal + +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_TF2XLA_RESOURCE_OPERATION_TABLE_H_ diff --git a/linux/include/tensorflow/compiler/tf2xla/shape_util.h b/linux/include/tensorflow/compiler/tf2xla/shape_util.h new file mode 100644 index 000000000..f7e34a5b4 --- /dev/null +++ b/linux/include/tensorflow/compiler/tf2xla/shape_util.h @@ -0,0 +1,45 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// Utilities for working with XLA shapes. + +#ifndef TENSORFLOW_COMPILER_TF2XLA_SHAPE_UTIL_H_ +#define TENSORFLOW_COMPILER_TF2XLA_SHAPE_UTIL_H_ + +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.pb.h" + +namespace tensorflow { + +// Convert an XLA Shape into the equivalent TensorFlow shape. May fail since +// not all XLA shapes can be represented as TensorShapes. +Status XLAShapeToTensorShape(const xla::Shape& shape, + TensorShape* tensor_shape); + +// Convert a TensorShape into the equivalent XLA Shape proto. Unlike Tensorflow, +// XLA shapes include the type. Not all `dtype` values can be represented by +// XLA, so this conversion may fail. +Status TensorShapeToXLAShape(DataType dtype, const TensorShape& tensor_shape, + xla::Shape* shape); + +// Converts a TensorShape into the equivalent XLA Shape proto, taking an +// xla::PrimitiveType to specify the element type. This never fails. +xla::Shape TensorShapeToXLAShape(xla::PrimitiveType type, + const TensorShape& tensor_shape); + +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_TF2XLA_SHAPE_UTIL_H_ diff --git a/linux/include/tensorflow/compiler/tf2xla/sharding_util.h b/linux/include/tensorflow/compiler/tf2xla/sharding_util.h new file mode 100644 index 000000000..ab67d4f15 --- /dev/null +++ b/linux/include/tensorflow/compiler/tf2xla/sharding_util.h @@ -0,0 +1,50 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_COMPILER_TF2XLA_SHARDING_UTIL_H_ +#define TENSORFLOW_COMPILER_TF2XLA_SHARDING_UTIL_H_ + +#include + +#include "tensorflow/compiler/xla/client/sharding_builder.h" +#include "tensorflow/compiler/xla/status_macros.h" +#include "tensorflow/core/graph/graph.h" +#include "tensorflow/core/lib/core/status.h" + +namespace tensorflow { + +// Parses the op sharding from the 'replicated core' device_name . +// Returns an error: +// - if the device name is invalid. +// - the core is parsed and is out of the range [0, num_cores_per_replica). +// +// Otherwise, returns either: +// - explicit_sharding if explicit_sharding.has_value() +// - a non-value if there is no assigned core or +// - a sharding set as per xla::sharding_builder::AssignDevice. +xla::StatusOr> ParseShardingFromDevice( + const string& device_name, int num_cores_per_replica, + absl::optional explicit_sharding = absl::nullopt); + +xla::StatusOr> ParseShardingFromDevice( + const Node& node, int num_cores_per_replica); + +xla::StatusOr> ParseShardingFromDevice( + const NodeDef& node_def, int num_cores_per_replica); + +void SetShardingDeviceAssignmentFromNode(const Node& src, Node* dst); + +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_TF2XLA_SHARDING_UTIL_H_ diff --git a/linux/include/tensorflow/compiler/tf2xla/side_effect_util.h b/linux/include/tensorflow/compiler/tf2xla/side_effect_util.h new file mode 100644 index 000000000..ad0762472 --- /dev/null +++ b/linux/include/tensorflow/compiler/tf2xla/side_effect_util.h @@ -0,0 +1,47 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_TF2XLA_SIDE_EFFECT_UTIL_H_ +#define TENSORFLOW_COMPILER_TF2XLA_SIDE_EFFECT_UTIL_H_ + +#include + +#include "tensorflow/core/graph/graph.h" + +namespace tensorflow { + +// Side-effecting nodes will have this attribute set. Its value is the list of +// node names which this node has side-effect dependencies on. +// +// Nodes like HostCompute, SendToHost, RecvFromHost always have this attribute, +// because they always have side-effect. +// If and While nodes may or may not have this attribute, depending on whether +// their bodies have side-effecting nodes. +extern const char kXlaTokenInputNodesAttrName[]; + +// This node name is used in kXlaTokenInputNodesAttrName attr to signal that a +// node has side-effect dependency on current graph's token input. +extern const char kXlaTokenArgNodeName[]; + +// Calculates side-effect dependencies for the graph's token output. +// Returns a set of node names representing these dependencies. +std::set CalculateTokenInputsForOutputToken(const Graph& g); + +// Returns whether a graph contains side-effecting nodes. +bool HasSideEffectingNodes(const Graph& g); + +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_TF2XLA_SIDE_EFFECT_UTIL_H_ diff --git a/linux/include/tensorflow/compiler/tf2xla/tf2xla.pb.h b/linux/include/tensorflow/compiler/tf2xla/tf2xla.pb.h new file mode 100644 index 000000000..ee58a0e7e --- /dev/null +++ b/linux/include/tensorflow/compiler/tf2xla/tf2xla.pb.h @@ -0,0 +1,1223 @@ +// Generated by the protocol buffer compiler. DO NOT EDIT! +// source: tensorflow/compiler/tf2xla/tf2xla.proto + +#ifndef PROTOBUF_INCLUDED_tensorflow_2fcompiler_2ftf2xla_2ftf2xla_2eproto +#define PROTOBUF_INCLUDED_tensorflow_2fcompiler_2ftf2xla_2ftf2xla_2eproto + +#include + +#include + +#if GOOGLE_PROTOBUF_VERSION < 3006000 +#error This file was generated by a newer version of protoc which is +#error incompatible with your Protocol Buffer headers. Please update +#error your headers. +#endif +#if 3006000 < GOOGLE_PROTOBUF_MIN_PROTOC_VERSION +#error This file was generated by an older version of protoc which is +#error incompatible with your Protocol Buffer headers. Please +#error regenerate this file with a newer version of protoc. +#endif + +#include +#include +#include +#include +#include +#include +#include +#include +#include // IWYU pragma: export +#include // IWYU pragma: export +#include +#include "tensorflow/core/framework/tensor_shape.pb.h" +#include "tensorflow/core/framework/types.pb.h" +// @@protoc_insertion_point(includes) +#define PROTOBUF_INTERNAL_EXPORT_protobuf_tensorflow_2fcompiler_2ftf2xla_2ftf2xla_2eproto + +namespace protobuf_tensorflow_2fcompiler_2ftf2xla_2ftf2xla_2eproto { +// Internal implementation detail -- do not use these members. +struct TableStruct { + static const ::google::protobuf::internal::ParseTableField entries[]; + static const ::google::protobuf::internal::AuxillaryParseTableField aux[]; + static const ::google::protobuf::internal::ParseTable schema[4]; + static const ::google::protobuf::internal::FieldMetadata field_metadata[]; + static const ::google::protobuf::internal::SerializationTable serialization_table[]; + static const ::google::protobuf::uint32 offsets[]; +}; +void AddDescriptors(); +} // namespace protobuf_tensorflow_2fcompiler_2ftf2xla_2ftf2xla_2eproto +namespace tensorflow { +namespace tf2xla { +class Config; +class ConfigDefaultTypeInternal; +extern ConfigDefaultTypeInternal _Config_default_instance_; +class Feed; +class FeedDefaultTypeInternal; +extern FeedDefaultTypeInternal _Feed_default_instance_; +class Fetch; +class FetchDefaultTypeInternal; +extern FetchDefaultTypeInternal _Fetch_default_instance_; +class TensorId; +class TensorIdDefaultTypeInternal; +extern TensorIdDefaultTypeInternal _TensorId_default_instance_; +} // namespace tf2xla +} // namespace tensorflow +namespace google { +namespace protobuf { +template<> ::tensorflow::tf2xla::Config* Arena::CreateMaybeMessage<::tensorflow::tf2xla::Config>(Arena*); +template<> ::tensorflow::tf2xla::Feed* Arena::CreateMaybeMessage<::tensorflow::tf2xla::Feed>(Arena*); +template<> ::tensorflow::tf2xla::Fetch* Arena::CreateMaybeMessage<::tensorflow::tf2xla::Fetch>(Arena*); +template<> ::tensorflow::tf2xla::TensorId* Arena::CreateMaybeMessage<::tensorflow::tf2xla::TensorId>(Arena*); +} // namespace protobuf +} // namespace google +namespace tensorflow { +namespace tf2xla { + +// =================================================================== + +class TensorId : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.tf2xla.TensorId) */ { + public: + TensorId(); + virtual ~TensorId(); + + TensorId(const TensorId& from); + + inline TensorId& operator=(const TensorId& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + TensorId(TensorId&& from) noexcept + : TensorId() { + *this = ::std::move(from); + } + + inline TensorId& operator=(TensorId&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const TensorId& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const TensorId* internal_default_instance() { + return reinterpret_cast( + &_TensorId_default_instance_); + } + static constexpr int kIndexInFileMessages = + 0; + + void UnsafeArenaSwap(TensorId* other); + void Swap(TensorId* other); + friend void swap(TensorId& a, TensorId& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline TensorId* New() const final { + return CreateMaybeMessage(NULL); + } + + TensorId* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const TensorId& from); + void MergeFrom(const TensorId& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(TensorId* other); + protected: + explicit TensorId(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // string node_name = 1; + void clear_node_name(); + static const int kNodeNameFieldNumber = 1; + const ::std::string& node_name() const; + void set_node_name(const ::std::string& value); + #if LANG_CXX11 + void set_node_name(::std::string&& value); + #endif + void set_node_name(const char* value); + void set_node_name(const char* value, size_t size); + ::std::string* mutable_node_name(); + ::std::string* release_node_name(); + void set_allocated_node_name(::std::string* node_name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_node_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_node_name( + ::std::string* node_name); + + // int64 output_index = 2; + void clear_output_index(); + static const int kOutputIndexFieldNumber = 2; + ::google::protobuf::int64 output_index() const; + void set_output_index(::google::protobuf::int64 value); + + // @@protoc_insertion_point(class_scope:tensorflow.tf2xla.TensorId) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::internal::ArenaStringPtr node_name_; + ::google::protobuf::int64 output_index_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2ftf2xla_2ftf2xla_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class Feed : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.tf2xla.Feed) */ { + public: + Feed(); + virtual ~Feed(); + + Feed(const Feed& from); + + inline Feed& operator=(const Feed& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + Feed(Feed&& from) noexcept + : Feed() { + *this = ::std::move(from); + } + + inline Feed& operator=(Feed&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const Feed& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const Feed* internal_default_instance() { + return reinterpret_cast( + &_Feed_default_instance_); + } + static constexpr int kIndexInFileMessages = + 1; + + void UnsafeArenaSwap(Feed* other); + void Swap(Feed* other); + friend void swap(Feed& a, Feed& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline Feed* New() const final { + return CreateMaybeMessage(NULL); + } + + Feed* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const Feed& from); + void MergeFrom(const Feed& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(Feed* other); + protected: + explicit Feed(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // string name = 3; + void clear_name(); + static const int kNameFieldNumber = 3; + const ::std::string& name() const; + void set_name(const ::std::string& value); + #if LANG_CXX11 + void set_name(::std::string&& value); + #endif + void set_name(const char* value); + void set_name(const char* value, size_t size); + ::std::string* mutable_name(); + ::std::string* release_name(); + void set_allocated_name(::std::string* name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_name( + ::std::string* name); + + // .tensorflow.tf2xla.TensorId id = 1; + bool has_id() const; + void clear_id(); + static const int kIdFieldNumber = 1; + private: + const ::tensorflow::tf2xla::TensorId& _internal_id() const; + public: + const ::tensorflow::tf2xla::TensorId& id() const; + ::tensorflow::tf2xla::TensorId* release_id(); + ::tensorflow::tf2xla::TensorId* mutable_id(); + void set_allocated_id(::tensorflow::tf2xla::TensorId* id); + void unsafe_arena_set_allocated_id( + ::tensorflow::tf2xla::TensorId* id); + ::tensorflow::tf2xla::TensorId* unsafe_arena_release_id(); + + // .tensorflow.TensorShapeProto shape = 2; + bool has_shape() const; + void clear_shape(); + static const int kShapeFieldNumber = 2; + private: + const ::tensorflow::TensorShapeProto& _internal_shape() const; + public: + const ::tensorflow::TensorShapeProto& shape() const; + ::tensorflow::TensorShapeProto* release_shape(); + ::tensorflow::TensorShapeProto* mutable_shape(); + void set_allocated_shape(::tensorflow::TensorShapeProto* shape); + void unsafe_arena_set_allocated_shape( + ::tensorflow::TensorShapeProto* shape); + ::tensorflow::TensorShapeProto* unsafe_arena_release_shape(); + + // .tensorflow.DataType type = 4; + void clear_type(); + static const int kTypeFieldNumber = 4; + ::tensorflow::DataType type() const; + void set_type(::tensorflow::DataType value); + + // @@protoc_insertion_point(class_scope:tensorflow.tf2xla.Feed) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::internal::ArenaStringPtr name_; + ::tensorflow::tf2xla::TensorId* id_; + ::tensorflow::TensorShapeProto* shape_; + int type_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2ftf2xla_2ftf2xla_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class Fetch : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.tf2xla.Fetch) */ { + public: + Fetch(); + virtual ~Fetch(); + + Fetch(const Fetch& from); + + inline Fetch& operator=(const Fetch& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + Fetch(Fetch&& from) noexcept + : Fetch() { + *this = ::std::move(from); + } + + inline Fetch& operator=(Fetch&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const Fetch& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const Fetch* internal_default_instance() { + return reinterpret_cast( + &_Fetch_default_instance_); + } + static constexpr int kIndexInFileMessages = + 2; + + void UnsafeArenaSwap(Fetch* other); + void Swap(Fetch* other); + friend void swap(Fetch& a, Fetch& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline Fetch* New() const final { + return CreateMaybeMessage(NULL); + } + + Fetch* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const Fetch& from); + void MergeFrom(const Fetch& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(Fetch* other); + protected: + explicit Fetch(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // string name = 2; + void clear_name(); + static const int kNameFieldNumber = 2; + const ::std::string& name() const; + void set_name(const ::std::string& value); + #if LANG_CXX11 + void set_name(::std::string&& value); + #endif + void set_name(const char* value); + void set_name(const char* value, size_t size); + ::std::string* mutable_name(); + ::std::string* release_name(); + void set_allocated_name(::std::string* name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_name( + ::std::string* name); + + // .tensorflow.tf2xla.TensorId id = 1; + bool has_id() const; + void clear_id(); + static const int kIdFieldNumber = 1; + private: + const ::tensorflow::tf2xla::TensorId& _internal_id() const; + public: + const ::tensorflow::tf2xla::TensorId& id() const; + ::tensorflow::tf2xla::TensorId* release_id(); + ::tensorflow::tf2xla::TensorId* mutable_id(); + void set_allocated_id(::tensorflow::tf2xla::TensorId* id); + void unsafe_arena_set_allocated_id( + ::tensorflow::tf2xla::TensorId* id); + ::tensorflow::tf2xla::TensorId* unsafe_arena_release_id(); + + // @@protoc_insertion_point(class_scope:tensorflow.tf2xla.Fetch) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::internal::ArenaStringPtr name_; + ::tensorflow::tf2xla::TensorId* id_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2ftf2xla_2ftf2xla_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class Config : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.tf2xla.Config) */ { + public: + Config(); + virtual ~Config(); + + Config(const Config& from); + + inline Config& operator=(const Config& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + Config(Config&& from) noexcept + : Config() { + *this = ::std::move(from); + } + + inline Config& operator=(Config&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const Config& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const Config* internal_default_instance() { + return reinterpret_cast( + &_Config_default_instance_); + } + static constexpr int kIndexInFileMessages = + 3; + + void UnsafeArenaSwap(Config* other); + void Swap(Config* other); + friend void swap(Config& a, Config& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline Config* New() const final { + return CreateMaybeMessage(NULL); + } + + Config* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const Config& from); + void MergeFrom(const Config& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(Config* other); + protected: + explicit Config(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated .tensorflow.tf2xla.Feed feed = 1; + int feed_size() const; + void clear_feed(); + static const int kFeedFieldNumber = 1; + ::tensorflow::tf2xla::Feed* mutable_feed(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::tf2xla::Feed >* + mutable_feed(); + const ::tensorflow::tf2xla::Feed& feed(int index) const; + ::tensorflow::tf2xla::Feed* add_feed(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::tf2xla::Feed >& + feed() const; + + // repeated .tensorflow.tf2xla.Fetch fetch = 2; + int fetch_size() const; + void clear_fetch(); + static const int kFetchFieldNumber = 2; + ::tensorflow::tf2xla::Fetch* mutable_fetch(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::tf2xla::Fetch >* + mutable_fetch(); + const ::tensorflow::tf2xla::Fetch& fetch(int index) const; + ::tensorflow::tf2xla::Fetch* add_fetch(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::tf2xla::Fetch >& + fetch() const; + + // @@protoc_insertion_point(class_scope:tensorflow.tf2xla.Config) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::tf2xla::Feed > feed_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::tf2xla::Fetch > fetch_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2ftf2xla_2ftf2xla_2eproto::TableStruct; +}; +// =================================================================== + + +// =================================================================== + +#ifdef __GNUC__ + #pragma GCC diagnostic push + #pragma GCC diagnostic ignored "-Wstrict-aliasing" +#endif // __GNUC__ +// TensorId + +// string node_name = 1; +inline void TensorId::clear_node_name() { + node_name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& TensorId::node_name() const { + // @@protoc_insertion_point(field_get:tensorflow.tf2xla.TensorId.node_name) + return node_name_.Get(); +} +inline void TensorId::set_node_name(const ::std::string& value) { + + node_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.tf2xla.TensorId.node_name) +} +#if LANG_CXX11 +inline void TensorId::set_node_name(::std::string&& value) { + + node_name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.tf2xla.TensorId.node_name) +} +#endif +inline void TensorId::set_node_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + node_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.tf2xla.TensorId.node_name) +} +inline void TensorId::set_node_name(const char* value, + size_t size) { + + node_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.tf2xla.TensorId.node_name) +} +inline ::std::string* TensorId::mutable_node_name() { + + // @@protoc_insertion_point(field_mutable:tensorflow.tf2xla.TensorId.node_name) + return node_name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* TensorId::release_node_name() { + // @@protoc_insertion_point(field_release:tensorflow.tf2xla.TensorId.node_name) + + return node_name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void TensorId::set_allocated_node_name(::std::string* node_name) { + if (node_name != NULL) { + + } else { + + } + node_name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), node_name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.tf2xla.TensorId.node_name) +} +inline ::std::string* TensorId::unsafe_arena_release_node_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.tf2xla.TensorId.node_name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return node_name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void TensorId::unsafe_arena_set_allocated_node_name( + ::std::string* node_name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (node_name != NULL) { + + } else { + + } + node_name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + node_name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.tf2xla.TensorId.node_name) +} + +// int64 output_index = 2; +inline void TensorId::clear_output_index() { + output_index_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 TensorId::output_index() const { + // @@protoc_insertion_point(field_get:tensorflow.tf2xla.TensorId.output_index) + return output_index_; +} +inline void TensorId::set_output_index(::google::protobuf::int64 value) { + + output_index_ = value; + // @@protoc_insertion_point(field_set:tensorflow.tf2xla.TensorId.output_index) +} + +// ------------------------------------------------------------------- + +// Feed + +// .tensorflow.tf2xla.TensorId id = 1; +inline bool Feed::has_id() const { + return this != internal_default_instance() && id_ != NULL; +} +inline void Feed::clear_id() { + if (GetArenaNoVirtual() == NULL && id_ != NULL) { + delete id_; + } + id_ = NULL; +} +inline const ::tensorflow::tf2xla::TensorId& Feed::_internal_id() const { + return *id_; +} +inline const ::tensorflow::tf2xla::TensorId& Feed::id() const { + const ::tensorflow::tf2xla::TensorId* p = id_; + // @@protoc_insertion_point(field_get:tensorflow.tf2xla.Feed.id) + return p != NULL ? *p : *reinterpret_cast( + &::tensorflow::tf2xla::_TensorId_default_instance_); +} +inline ::tensorflow::tf2xla::TensorId* Feed::release_id() { + // @@protoc_insertion_point(field_release:tensorflow.tf2xla.Feed.id) + + ::tensorflow::tf2xla::TensorId* temp = id_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + id_ = NULL; + return temp; +} +inline ::tensorflow::tf2xla::TensorId* Feed::unsafe_arena_release_id() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.tf2xla.Feed.id) + + ::tensorflow::tf2xla::TensorId* temp = id_; + id_ = NULL; + return temp; +} +inline ::tensorflow::tf2xla::TensorId* Feed::mutable_id() { + + if (id_ == NULL) { + auto* p = CreateMaybeMessage<::tensorflow::tf2xla::TensorId>(GetArenaNoVirtual()); + id_ = p; + } + // @@protoc_insertion_point(field_mutable:tensorflow.tf2xla.Feed.id) + return id_; +} +inline void Feed::set_allocated_id(::tensorflow::tf2xla::TensorId* id) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete id_; + } + if (id) { + ::google::protobuf::Arena* submessage_arena = + ::google::protobuf::Arena::GetArena(id); + if (message_arena != submessage_arena) { + id = ::google::protobuf::internal::GetOwnedMessage( + message_arena, id, submessage_arena); + } + + } else { + + } + id_ = id; + // @@protoc_insertion_point(field_set_allocated:tensorflow.tf2xla.Feed.id) +} + +// .tensorflow.TensorShapeProto shape = 2; +inline bool Feed::has_shape() const { + return this != internal_default_instance() && shape_ != NULL; +} +inline const ::tensorflow::TensorShapeProto& Feed::_internal_shape() const { + return *shape_; +} +inline const ::tensorflow::TensorShapeProto& Feed::shape() const { + const ::tensorflow::TensorShapeProto* p = shape_; + // @@protoc_insertion_point(field_get:tensorflow.tf2xla.Feed.shape) + return p != NULL ? *p : *reinterpret_cast( + &::tensorflow::_TensorShapeProto_default_instance_); +} +inline ::tensorflow::TensorShapeProto* Feed::release_shape() { + // @@protoc_insertion_point(field_release:tensorflow.tf2xla.Feed.shape) + + ::tensorflow::TensorShapeProto* temp = shape_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + shape_ = NULL; + return temp; +} +inline ::tensorflow::TensorShapeProto* Feed::unsafe_arena_release_shape() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.tf2xla.Feed.shape) + + ::tensorflow::TensorShapeProto* temp = shape_; + shape_ = NULL; + return temp; +} +inline ::tensorflow::TensorShapeProto* Feed::mutable_shape() { + + if (shape_ == NULL) { + auto* p = CreateMaybeMessage<::tensorflow::TensorShapeProto>(GetArenaNoVirtual()); + shape_ = p; + } + // @@protoc_insertion_point(field_mutable:tensorflow.tf2xla.Feed.shape) + return shape_; +} +inline void Feed::set_allocated_shape(::tensorflow::TensorShapeProto* shape) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(shape_); + } + if (shape) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(shape)->GetArena(); + if (message_arena != submessage_arena) { + shape = ::google::protobuf::internal::GetOwnedMessage( + message_arena, shape, submessage_arena); + } + + } else { + + } + shape_ = shape; + // @@protoc_insertion_point(field_set_allocated:tensorflow.tf2xla.Feed.shape) +} + +// string name = 3; +inline void Feed::clear_name() { + name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& Feed::name() const { + // @@protoc_insertion_point(field_get:tensorflow.tf2xla.Feed.name) + return name_.Get(); +} +inline void Feed::set_name(const ::std::string& value) { + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.tf2xla.Feed.name) +} +#if LANG_CXX11 +inline void Feed::set_name(::std::string&& value) { + + name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.tf2xla.Feed.name) +} +#endif +inline void Feed::set_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.tf2xla.Feed.name) +} +inline void Feed::set_name(const char* value, + size_t size) { + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.tf2xla.Feed.name) +} +inline ::std::string* Feed::mutable_name() { + + // @@protoc_insertion_point(field_mutable:tensorflow.tf2xla.Feed.name) + return name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* Feed::release_name() { + // @@protoc_insertion_point(field_release:tensorflow.tf2xla.Feed.name) + + return name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void Feed::set_allocated_name(::std::string* name) { + if (name != NULL) { + + } else { + + } + name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.tf2xla.Feed.name) +} +inline ::std::string* Feed::unsafe_arena_release_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.tf2xla.Feed.name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void Feed::unsafe_arena_set_allocated_name( + ::std::string* name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (name != NULL) { + + } else { + + } + name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.tf2xla.Feed.name) +} + +// .tensorflow.DataType type = 4; +inline void Feed::clear_type() { + type_ = 0; +} +inline ::tensorflow::DataType Feed::type() const { + // @@protoc_insertion_point(field_get:tensorflow.tf2xla.Feed.type) + return static_cast< ::tensorflow::DataType >(type_); +} +inline void Feed::set_type(::tensorflow::DataType value) { + + type_ = value; + // @@protoc_insertion_point(field_set:tensorflow.tf2xla.Feed.type) +} + +// ------------------------------------------------------------------- + +// Fetch + +// .tensorflow.tf2xla.TensorId id = 1; +inline bool Fetch::has_id() const { + return this != internal_default_instance() && id_ != NULL; +} +inline void Fetch::clear_id() { + if (GetArenaNoVirtual() == NULL && id_ != NULL) { + delete id_; + } + id_ = NULL; +} +inline const ::tensorflow::tf2xla::TensorId& Fetch::_internal_id() const { + return *id_; +} +inline const ::tensorflow::tf2xla::TensorId& Fetch::id() const { + const ::tensorflow::tf2xla::TensorId* p = id_; + // @@protoc_insertion_point(field_get:tensorflow.tf2xla.Fetch.id) + return p != NULL ? *p : *reinterpret_cast( + &::tensorflow::tf2xla::_TensorId_default_instance_); +} +inline ::tensorflow::tf2xla::TensorId* Fetch::release_id() { + // @@protoc_insertion_point(field_release:tensorflow.tf2xla.Fetch.id) + + ::tensorflow::tf2xla::TensorId* temp = id_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + id_ = NULL; + return temp; +} +inline ::tensorflow::tf2xla::TensorId* Fetch::unsafe_arena_release_id() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.tf2xla.Fetch.id) + + ::tensorflow::tf2xla::TensorId* temp = id_; + id_ = NULL; + return temp; +} +inline ::tensorflow::tf2xla::TensorId* Fetch::mutable_id() { + + if (id_ == NULL) { + auto* p = CreateMaybeMessage<::tensorflow::tf2xla::TensorId>(GetArenaNoVirtual()); + id_ = p; + } + // @@protoc_insertion_point(field_mutable:tensorflow.tf2xla.Fetch.id) + return id_; +} +inline void Fetch::set_allocated_id(::tensorflow::tf2xla::TensorId* id) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete id_; + } + if (id) { + ::google::protobuf::Arena* submessage_arena = + ::google::protobuf::Arena::GetArena(id); + if (message_arena != submessage_arena) { + id = ::google::protobuf::internal::GetOwnedMessage( + message_arena, id, submessage_arena); + } + + } else { + + } + id_ = id; + // @@protoc_insertion_point(field_set_allocated:tensorflow.tf2xla.Fetch.id) +} + +// string name = 2; +inline void Fetch::clear_name() { + name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& Fetch::name() const { + // @@protoc_insertion_point(field_get:tensorflow.tf2xla.Fetch.name) + return name_.Get(); +} +inline void Fetch::set_name(const ::std::string& value) { + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.tf2xla.Fetch.name) +} +#if LANG_CXX11 +inline void Fetch::set_name(::std::string&& value) { + + name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.tf2xla.Fetch.name) +} +#endif +inline void Fetch::set_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.tf2xla.Fetch.name) +} +inline void Fetch::set_name(const char* value, + size_t size) { + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.tf2xla.Fetch.name) +} +inline ::std::string* Fetch::mutable_name() { + + // @@protoc_insertion_point(field_mutable:tensorflow.tf2xla.Fetch.name) + return name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* Fetch::release_name() { + // @@protoc_insertion_point(field_release:tensorflow.tf2xla.Fetch.name) + + return name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void Fetch::set_allocated_name(::std::string* name) { + if (name != NULL) { + + } else { + + } + name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.tf2xla.Fetch.name) +} +inline ::std::string* Fetch::unsafe_arena_release_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.tf2xla.Fetch.name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void Fetch::unsafe_arena_set_allocated_name( + ::std::string* name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (name != NULL) { + + } else { + + } + name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.tf2xla.Fetch.name) +} + +// ------------------------------------------------------------------- + +// Config + +// repeated .tensorflow.tf2xla.Feed feed = 1; +inline int Config::feed_size() const { + return feed_.size(); +} +inline void Config::clear_feed() { + feed_.Clear(); +} +inline ::tensorflow::tf2xla::Feed* Config::mutable_feed(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.tf2xla.Config.feed) + return feed_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::tf2xla::Feed >* +Config::mutable_feed() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.tf2xla.Config.feed) + return &feed_; +} +inline const ::tensorflow::tf2xla::Feed& Config::feed(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.tf2xla.Config.feed) + return feed_.Get(index); +} +inline ::tensorflow::tf2xla::Feed* Config::add_feed() { + // @@protoc_insertion_point(field_add:tensorflow.tf2xla.Config.feed) + return feed_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::tf2xla::Feed >& +Config::feed() const { + // @@protoc_insertion_point(field_list:tensorflow.tf2xla.Config.feed) + return feed_; +} + +// repeated .tensorflow.tf2xla.Fetch fetch = 2; +inline int Config::fetch_size() const { + return fetch_.size(); +} +inline void Config::clear_fetch() { + fetch_.Clear(); +} +inline ::tensorflow::tf2xla::Fetch* Config::mutable_fetch(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.tf2xla.Config.fetch) + return fetch_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::tf2xla::Fetch >* +Config::mutable_fetch() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.tf2xla.Config.fetch) + return &fetch_; +} +inline const ::tensorflow::tf2xla::Fetch& Config::fetch(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.tf2xla.Config.fetch) + return fetch_.Get(index); +} +inline ::tensorflow::tf2xla::Fetch* Config::add_fetch() { + // @@protoc_insertion_point(field_add:tensorflow.tf2xla.Config.fetch) + return fetch_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::tf2xla::Fetch >& +Config::fetch() const { + // @@protoc_insertion_point(field_list:tensorflow.tf2xla.Config.fetch) + return fetch_; +} + +#ifdef __GNUC__ + #pragma GCC diagnostic pop +#endif // __GNUC__ +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + + +// @@protoc_insertion_point(namespace_scope) + +} // namespace tf2xla +} // namespace tensorflow + +// @@protoc_insertion_point(global_scope) + +#endif // PROTOBUF_INCLUDED_tensorflow_2fcompiler_2ftf2xla_2ftf2xla_2eproto diff --git a/linux/include/tensorflow/compiler/tf2xla/tf2xla_util.h b/linux/include/tensorflow/compiler/tf2xla/tf2xla_util.h new file mode 100644 index 000000000..53eab8b63 --- /dev/null +++ b/linux/include/tensorflow/compiler/tf2xla/tf2xla_util.h @@ -0,0 +1,149 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_TF2XLA_TF2XLA_UTIL_H_ +#define TENSORFLOW_COMPILER_TF2XLA_TF2XLA_UTIL_H_ + +#include + +#include "tensorflow/compiler/tf2xla/tf2xla.pb.h" +#include "tensorflow/compiler/xla/status_macros.h" +#include "tensorflow/core/framework/graph.pb.h" +#include "tensorflow/core/framework/kernel_def.pb.h" +#include "tensorflow/core/framework/op.h" +#include "tensorflow/core/graph/graph.h" +#include "tensorflow/core/lib/core/status.h" + +namespace tensorflow { + +// ValidateConfig returns OK iff config is valid. +Status ValidateConfig(const tf2xla::Config& config); + +// Modifies to include placeholders for each fed tensor, and +// update references to the fed tensors to refer to the placeholders. +// The existing nodes referenced by the feeds are not removed or modified +// (except where their input edges are modified by the replacement of other +// feeds). +Status AddPlaceholdersForFeeds( + const tf2xla::Config& config, const OpRegistryInterface* op_registry, + std::unordered_map* feed_remapping, GraphDef* graph_def); + +// Returns in a copy of , pruned to only include fetches from +// . +Status PruneGraphDefInto(const tf2xla::Config& config, const GraphDef& in, + GraphDef* out); + +// Returns node:port for the given . +string TensorIdToString(const tf2xla::TensorId& id); + +// Updates the sharding of based on the sharding of its neighbors. +// If is true, outgoing edges from are considered; else incoming +// edges are considered. +Status SetNodeShardingFromNeighbors(Node* n, bool out_edges); + +// Add an allowed data type to the AttrConstraint with the given name. +void AddDtypeToKernalDefConstraint(absl::string_view name, DataType dtype, + KernelDef* kdef); + +// Returns the next random seed to use for seeding xla rng. +uint32 GetXLARandomSeed(); + +// Indicates how a FunctionDef is associated with a graph node (e.g. the node is +// a function call, or the node has function attrs). +class AssociatedFunctionInfo { + public: + enum AssociatedFunctionType { + kFunctionAttr = 0, + kFunctionCallNode = 1, + kSymbolicGradient = 2, + }; + + // The function is an attr of the node. + static AssociatedFunctionInfo FunctionAttr(const string& func_name, + const AttrValueMap& attrs, + const string& attr_name) { + return AssociatedFunctionInfo(kFunctionAttr, func_name, attrs, attr_name); + } + + // The node is a function call. + static AssociatedFunctionInfo FunctionCall(const string& func_name, + const AttrValueMap& attrs) { + // attr_name will not be used in this case. + return AssociatedFunctionInfo(kFunctionCallNode, func_name, attrs, + /*attr_name=*/""); + } + + // The node is a SymbolicGradient op. + static AssociatedFunctionInfo SymbolicGradient(const string& func_name, + const AttrValueMap& attrs) { + // attr_name will not be used in this case. + return AssociatedFunctionInfo(kSymbolicGradient, func_name, attrs, + /*attr_name=*/""); + } + + AssociatedFunctionType type() const { return type_; } + + const string& func_name() const { return func_name_; } + + const string& attr_name() const { return attr_name_; } + + const AttrValueMap& attrs() const { return attrs_; } + + private: + AssociatedFunctionInfo(AssociatedFunctionType type, const string& func_name, + const AttrValueMap& attrs, const string& attr_name) + : type_(type), + func_name_(func_name), + attrs_(attrs), + attr_name_(attr_name) {} + + // Available for all instances. + AssociatedFunctionType type_; + string func_name_; + AttrValueMap attrs_; + + // Only available if the function is defined in an attr. + string attr_name_; +}; + +// Returns if the NodeDef has associated function. +bool HasAssociatedFunction(const NodeDef& node_def, + FunctionLibraryRuntime* flr); + +// Gets functions associated with the node. Current cases: +// 1. For function call node, its function name; +// 2. For SymbolicGradient op, returned func_name will be "SymbolicGradient", +// and returned attrs will be this node's attributes; +// 3. For nodes like XlaWhile/XlaIf, all their function attributes. +std::vector GetAssociatedFunctions( + const Node& node, FunctionLibraryRuntime* flr); + +// Changes associated functions for the node. Current cases: +// 1. For function call node, creates a new node with the new function name and +// remove the old node; +// 2. For SymbolicGradient op, add or replace GradientDef in +// FunctionLibraryDefinition; +// 3. For nodes like XlaWhile/XlaIf, modify their function attributes. +Status RewriteAssociatedFunction( + Graph* graph, Node* node, FunctionLibraryDefinition* fld, + const AssociatedFunctionInfo& associated_function, + const string& rewritten_function_name); + +// Attribute to mark nodes to be executed on host. +extern const char kXlaOutsideCompilationAttrName[]; + +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_TF2XLA_TF2XLA_UTIL_H_ diff --git a/linux/include/tensorflow/compiler/tf2xla/type_util.h b/linux/include/tensorflow/compiler/tf2xla/type_util.h new file mode 100644 index 000000000..6354216ee --- /dev/null +++ b/linux/include/tensorflow/compiler/tf2xla/type_util.h @@ -0,0 +1,38 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_TF2XLA_TYPE_UTIL_H_ +#define TENSORFLOW_COMPILER_TF2XLA_TYPE_UTIL_H_ + +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/framework/types.pb.h" +#include "tensorflow/core/lib/core/status.h" + +namespace tensorflow { + +// Converts a Tensorflow DataType to an XLA PrimitiveType. +Status DataTypeToPrimitiveType(DataType data_type, xla::PrimitiveType* type); + +// N.B.: there is intentionally no function to convert an XLA PrimitiveType to +// a TensorFlow DataType. The mapping from TF types to XLA types is not +// one-to-one: for example, both DT_INT8 and DT_QINT8 map to xla::S8. So the +// inverse would not be a well-defined function. If you find that you want the +// inverse mapping, then most likely you should be preserving the original +// TensorFlow type, rather than trying to convert an XLA type into a TensorFlow +// type. + +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_TF2XLA_TYPE_UTIL_H_ diff --git a/linux/include/tensorflow/compiler/tf2xla/xla_compilation_device.h b/linux/include/tensorflow/compiler/tf2xla/xla_compilation_device.h new file mode 100644 index 000000000..a6e788253 --- /dev/null +++ b/linux/include/tensorflow/compiler/tf2xla/xla_compilation_device.h @@ -0,0 +1,106 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_TF2XLA_XLA_COMPILATION_DEVICE_H_ +#define TENSORFLOW_COMPILER_TF2XLA_XLA_COMPILATION_DEVICE_H_ + +#include + +#include "tensorflow/compiler/tf2xla/xla_resource.h" +#include "tensorflow/compiler/xla/client/xla_builder.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/common_runtime/local_device.h" +#include "tensorflow/core/framework/device_base.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/platform/mem.h" +#include "tensorflow/core/public/session_options.h" + +namespace tensorflow { + +// Class is defined in xla_compilation_device.cc, reference +// included here only so the XlaCompilationDevice allocator_ member can be +// declared. +class XlaCompilationAllocator; + +// This is a 'dummy' TensorFlow device that is only used to execute a +// subgraph of XLA compilation Ops to construct a compiled version +// of the subgraph's computation. It has a 'dummy' allocator that +// backs each Tensor with metadata indicating the computation the +// Tensor represents. +// +// We deliberately don't register a device factory because we *never* +// want placement to put Ops on a compilation device. The device is created +// manually, not using a factory. +// +// XLA compilation is not thread-safe. OpKernels registered on the +// XlaCompilationDevice must not use threads or concurrency. +class XlaCompilationDevice : public LocalDevice { + public: + XlaCompilationDevice(const SessionOptions& options, DeviceType type); + + ~XlaCompilationDevice() override; + + Allocator* GetAllocator(AllocatorAttributes attr) override; + + void Compute(OpKernel* op_kernel, OpKernelContext* context) override; + + Status Sync() override; + + Status MakeTensorFromProto(const TensorProto& tensor_proto, + const AllocatorAttributes alloc_attrs, + Tensor* tensor) override; + + private: + std::unique_ptr allocator_; +}; + +// A XlaExpression wraps an XLA computation. Each Tensor on an +// XlaCompilationDevice contains an XlaExpression, and the shape of the Tensor +// matches the shape of the subcomputation in the XlaOp. Each +// expression is either a constant, or a function of previously-compiled +// expressions. +class XlaExpression { + public: + XlaExpression(); + + // handle() stores the XLA handle of the computation that the + // expression represents. + void set_handle(const xla::XlaOp& h); + const xla::XlaOp& handle() const { return handle_; } + + void set_constant_value(Tensor value); + bool has_constant_value() const { return has_constant_value_; } + const Tensor& constant_value() const { return constant_value_; } + + void set_resource(XlaResource* resource) { resource_ = resource; } + XlaResource* resource() const { return resource_; } + + private: + // The XLA handle of the expression's computation. + xla::XlaOp handle_; + + // If this expression is a constant with a known value, 'constant_value' is a + // host-memory Tensor containing the value. Used to avoid invoking XLA for + // expressions that are trivially constant. + bool has_constant_value_ = false; + Tensor constant_value_; + + XlaResource* resource_ = nullptr; // Not owned. +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_TF2XLA_XLA_COMPILATION_DEVICE_H_ diff --git a/linux/include/tensorflow/compiler/tf2xla/xla_compiler.h b/linux/include/tensorflow/compiler/tf2xla/xla_compiler.h new file mode 100644 index 000000000..2cc603a58 --- /dev/null +++ b/linux/include/tensorflow/compiler/tf2xla/xla_compiler.h @@ -0,0 +1,479 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_TF2XLA_XLA_COMPILER_H_ +#define TENSORFLOW_COMPILER_TF2XLA_XLA_COMPILER_H_ + +#include + +#include "tensorflow/compiler/tf2xla/host_compute_metadata.pb.h" +#include "tensorflow/compiler/tf2xla/xla_compilation_device.h" +#include "tensorflow/compiler/tf2xla/xla_op_registry.h" +#include "tensorflow/compiler/xla/client/local_client.h" +#include "tensorflow/compiler/xla/client/xla_computation.h" +#include "tensorflow/compiler/xla/status_macros.h" +#include "tensorflow/core/common_runtime/device.h" +#include "tensorflow/core/common_runtime/device_mgr.h" +#include "tensorflow/core/common_runtime/function.h" +#include "tensorflow/core/framework/function.h" +#include "tensorflow/core/lib/core/errors.h" +#include "tensorflow/core/platform/env.h" +#include "tensorflow/core/platform/mutex.h" +#include "tensorflow/core/platform/notification.h" +#include "tensorflow/core/platform/thread_annotations.h" +#include "tensorflow/core/public/version.h" + +namespace tensorflow { + +class XlaContext; + +// The XlaCompiler class is responsible for compilation of a self-contained +// subgraph of a TensorFlow computation using the XLA linear algebra runtime. +// It does a symbolic execution of the graph starting from specific input +// shapes, using a JIT device to convert operators into XLA computations. +// +// XlaCompiler is typically invoked from an `XlaLaunch` operator once the +// shapes of all input parameters to the computation are known. This is +// because the symbolic execution requires known shapes for all operations. +// +// XlaCompiler compiles Tensorflow graphs that received inputs via _Arg nodes, +// and return outputs via _Retval nodes. +// +// The XlaCompiler requires one Argument struct for each _Arg index, that +// describes each argument. Arguments can be compile-time constants +// (kind kConstant), run-time parameters (kind kParameter), or resources +// (kind kResource). +// +// Only kParameter and initialized kResource arguments become runtime parameters +// to the generated XLA computation. +// +// The run-time outputs of the XLA computation are arranged in the following +// order: +// +------------------+-----------------------------------------+ +// | _Retval values | Updated values of kResource arguments | +// +------------------+-----------------------------------------+ +// _Retval values are ordered by _Retval index, whereas kResource values are +// ordered by the original _Arg position of the variable. +// +// If a shape representation function is provided as part of +// XlaCompiler::CompileOptions, kParameter arguments and return values to an +// entry computation will be reshaped in accordance to the shape function. +// Arguments and return values to a non-entry computation are not reshaped. +// Variable resource arguments are passed and returned in reshaped form, even +// for non-entry computations. This feature allows TensorFlow to keep on-device +// tensors with a different shape to their representation inside the XLA +// computation. +// +// In computation outputs, updated kResource values are placed the end. When +// emitting While loop bodies, we must ensure that the loop body has +// identical input and output signatures. By passing variable values +// at the end of the argument list and using the +// `return_updated_values_for_all_variables` option, we can ensure that the +// input and output values of resources appear at the same positions. +// +// Resources are passed as parameters or returned as resource updates in +// "packed" form. +// kStack resources are packed as (array, size of stack) XLA tuples. +// kTensorArray resources without gradients are packed as the array that +// backs the TensorArray. If gradients are present (`tensor_array_gradients`), +// the packed representation is a (array, gradient0, gradient1, ...) tuple, +// where gradient_k is the value of the k-th gradient in the +// `tensor_array_gradients` ordered set. +class XlaCompiler { + public: + // Describes how to derive the value of each _Arg node in the graph/function + // being compiled. There must be one Argument for each _Arg index. + struct Argument { + enum Kind { + // Default value; not a valid kind. + kInvalid, + + // Argument is a compile-time constant. No associated runtime parameter. + kConstant, + + // Argument is a Variable, TensorArray, or Stack resource. Has an + // associated runtime parameter iff `initialized` is true. + kResource, + + // Argument is a run-time parameter. + kParameter, + + // Argument is an XLA token. + kToken, + }; + + Kind kind = kInvalid; + + // The type of the argument. If the argument is a resource, this + // is the type of the variable's value, not DT_RESOURCE. + DataType type; + + // The shape of the argument. For: + // * a parameter: the shape of the parameter. + // * a constant: ignored; the shape given by constant_value is used + // instead. + // * an uninitialized resource: ignored. We don't yet know the shape of an + // uninitialized resource (otherwise we would have initialized it!) + // * an initialized variable: the shape of the variable's value. + // * an initialized TensorArray or Stack resource: the shape of an entry in + // the TensorArray/Stack. Note this is the size of a single entry, not the + // XLA data structure that represents the complete stack/array. + TensorShape shape; + + // The value of the argument, if it is a compile-time constant. Must be a + // host-memory tensor. + Tensor constant_value; + + // The name of this argument, used for debugging. + string name; + + // For a kResource, what kind of resource is it? + XlaResource::Kind resource_kind = XlaResource::kInvalid; + + // For a kResource, has this resource been initialized? + bool initialized = false; + + // For a TensorArray or Stack resource, what is the array's declared size? + // (Used for lazy initialization.) + int64 tensor_array_size = -1; + + // TensorArray resource parameters are passed as (array, gradient array 0, + // ..., gradient array k), where the gradient arrays are in the same order + // as `tensor_array_gradients`. + std::set tensor_array_gradients; + + bool operator==(const Argument& other) const; + }; + + // Options pertaining to an individual call to CompileGraph() or + // CompileFunction(). + struct CompileOptions { + // If `use_tuple_arg` is true, a single tuple parameter will be used for all + // arguments; if false, each argument gets its own parameter. + bool use_tuple_arg = false; + + // If 'return_updated_values_for_all_resources' is true, then updated + // values of all resource arguments will be included in the + // 'resource_updates' of the computation, even if the resource was not + // modified by the computation. Used when compiling loop bodies to ensure + // the input and output signatures match. + bool return_updated_values_for_all_resources = false; + + // If 'resolve_compile_time_constants' is true, then outputs of a + // computation that are known to be compile-time constants will be returned + // as Tensors at compile-time, rather than as run-time outputs of the + // computation. + bool resolve_compile_time_constants = true; + + // If 'always_return_tuple' is true, then the output of a computation will + // always be a tuple. Otherwise, a single-element output will not be wrapped + // in a tuple. + bool always_return_tuple = true; + + // True when compiling the entry computation, false for subcomputations + // (while, call, etc.) + bool is_entry_computation = true; + + // True when we should add XLA input & output to the graph/function. + bool add_token_input_output = false; + }; + + struct OutputDescription { + // Type and shape of the output. The shape is the unflattened shape. + // When `type` is DT_RESOURCE, `shape` is the shape of the resource + // variable's value. + DataType type; + TensorShape shape; + + // Constant output value, if known to be constant at JIT compilation time. + // 'Tensor' is in host memory. + bool is_constant = false; + Tensor constant_value; + + // When this output is a resource, i.e. `type == DT_RESOURCE`, this is + // the index of the input that contains the resource. + int input_index; + }; + + // Describes a variable write side effect of the computation. + struct ResourceUpdate { + // Index of the input that contains the variable resource to write to. + int input_index; + + // Type and shape of the tensor to be written back. + // The `shape` field has the same meaning as the Argument::shape field. + DataType type; + TensorShape shape; + + // Was the value of the variable modified by the computation? + // (Always true, unless `return_updated_values_for_all_resources` is true.) + bool modified; + + // If the resource is a TensorArray, the set of gradients read or written. + std::set tensor_array_gradients_accessed; + }; + + struct CompilationResult { + // Vector that maps from the parameters of the XLA computation to their + // original argument positions. To handle compile-time constant inputs, the + // parameters to the XLA computation may be a subset of the original + // arguments. The relative ordering of parameters are maintained. + std::vector input_mapping; + + // Input shapes of the computation. If we are flattening inputs, these are + // the flattened shapes. + std::vector xla_input_shapes; + + // Output shape in XLA format. The output shape is always a tuple. If we + // are flattening outputs, these are the flattened shapes. + xla::Shape xla_output_shape; + + // TensorFlow shapes of outputs, together with the values of any + // constant arguments. Vector indexed by Tensorflow _Retval number, + // containing both constant and non-constant results. + std::vector outputs; + + // TensorFlow shapes and types of sends/recvs from HostCompute Ops to their + // matching RecvAtHost/SendFromHost Ops in the outer graph. + tf2xla::HostComputeMetadata host_compute_metadata; + + // Resources whose values were updated by the computation, ordered + // by return value position (which is the same as the order the resources + // were passed as arguments). Resource updates follow the non-constant + // results in the outputs of XLA computation. + std::vector resource_updates; + + // The XLA computation built from the tensorflow subgraph. + std::shared_ptr computation; + }; + + typedef std::function(const TensorShape&, + DataType)> + ShapeRepresentationFn; + struct Options { + // Name of the compilation device to use. It must be set by the caller. + // The default empty value is invalid. + DeviceType device_type = DeviceType(""); + + // The device to use during compilation to execute instructions on, for + // example for auto-tuning. + // Valid values are defined by `xla::Backend::devices_ordinal_supported()`. + // -1 indicates the default device should be used. + int device_ordinal = -1; + + xla::Client* client = nullptr; + + // Function library in which to find function definitions. Must be non-null. + const FunctionLibraryDefinition* flib_def = nullptr; + + // The graph def version to be compiled. + int graph_def_version = TF_GRAPH_DEF_VERSION; + + // If 'allow_cpu_custom_calls' is true, kernels may make use of CustomCall() + // for CPU. + bool allow_cpu_custom_calls = false; + + // If set, the XLA representation of variables represented to XLA as the + // shape given by this shape function. Variables are reshaped to this shape + // on write, and reshaped to their original shape on read. + ShapeRepresentationFn shape_representation_fn; + + // If not nullptr, populate_resource_manager is called with the + // compilation device's resource manager when the compilation + // device is created, and can be used to create metadata objects + // that can be accessed by XLA op kernels. + std::function* populate_resource_manager = nullptr; + + // If not nullptr, this memory allocator can be used by the compiler for + // temporary allocations it might want to make during compilation. + // + // For example, the compiler may want to try out different algorithms and + // choose the fastest one, and it might run those algorithms over buffers + // created using this allocator. + // + // The compiler can function correctly without an explicit allocator given + // here, but on some devices (notably, GPUs), TensorFlow tends to eagerly + // allocate most or all available memory on the device, leaving none for the + // compiler to access, unless it can use TensorFlow's allocator. + xla::DeviceMemoryAllocator* device_allocator = nullptr; + }; + + explicit XlaCompiler(Options options); + + ~XlaCompiler(); + + Status CompileFunction(const CompileOptions& options, + const NameAttrList& fn_name_attrs, + std::vector args, CompilationResult* result); + + // Compiles a tensorflow::Graph into an xla::XlaComputation. + // Similar to CompileFunction, but takes a Graph as input rather than a + // function. + Status CompileGraph(const CompileOptions& options, string const& name, + std::unique_ptr graph, + const std::vector& args, + CompilationResult* result); + + // Compiles a single Op, given by an OpKernelContext, into an + // xla::XlaComputation. Similar to CompileFunction but takes a single Op as + // input. + Status CompileSingleOp(const CompileOptions& options, string const& name, + OpKernelContext* ctx, + const std::vector& args, + CompilationResult* result); + + // Returns the shape of the XLA parameter for an argument 'arg'. + // See the class comment for more details about the argument passing + // convention. + Status XLAShapeForArgument(const Argument& arg, bool is_entry_computation, + xla::Shape* xla_shape) const; + + // Retrieves the channel handle associated with `key`. Allocates + // a new channel handle if none exists. + // Channel handles can be used to communicate between different + // computations. Computations that communicate should be compiled with the + // same XlaCompiler. + Status GetChannelHandle(const string& key, xla::ChannelHandle* channel); + + // Retrieves the host-to-device channel handle associated with `key`. + // Allocates a new channel handle if none exists. + Status GetHostToDeviceChannelHandle(const string& key, + xla::ChannelHandle* channel); + + // Retrieves the device-to-host channel handle associated with `key`. + // Allocates a new channel handle if none exists. + Status GetDeviceToHostChannelHandle(const string& key, + xla::ChannelHandle* channel); + + // Sets the shapes and types for the device to host transfer associated with + // 'key'. + Status SetDeviceToHostMetadata(const string& key, + absl::Span types, + absl::Span shapes); + + // Gets the shapes the device to host transfer associated with 'key'. + Status GetDeviceToHostShapes(const string& key, + std::vector* shapes) const; + + // Sets the shapes and types for the host to device transfer associated with + // 'key'. + Status SetHostToDeviceMetadata(const string& key, + absl::Span types, + absl::Span shapes); + + // In order to avoid deadlocks from dependencies in host computations, it can + // be necessary to enforce a partial order on the execution of HostCompute + // Ops. In particular it may be necessary to constrain the SendToHost for one + // HostCompute to run before blocking on the RecvAtHost for another + // HostCompute. The compiler maintains a mapping from 'host_compute_name' to + // handle, where the handle is an 'output' of the HostCompute Op corresponding + // to 'host_compute_name'. Another HostCompute Op that needs to be sequenced + // later can add the handle as an 'input' to enforce the constraints. + // 'host_compute_name' can be any string the client wishes to use to identify + // a given HostCompute Op as long as the names are unique within the + // compilation. + Status GetHostComputeControlDependency(const string& host_compute_name, + xla::XlaOp* handle); + Status SetHostComputeControlDependency(const string& host_compute_name, + const xla::XlaOp& handle); + + const Options& options() const { return options_; } + xla::Client* client() const { return options_.client; } + FunctionLibraryRuntime* flib_runtime() const { return flib_runtime_; } + + void PushNodeTokenMapping(); + Status PopNodeTokenMapping(); + Status SetNodeToken(const string& node_name, const xla::XlaOp& op); + xla::StatusOr GetNodeToken(const string& node_name); + + private: + // Sets the function body `fbody` to the one registered as `function`. + Status FindFunctionBody(const NameAttrList& function, + const FunctionBody** fbody); + + // Returns the optimized graph object in this function body. + std::unique_ptr GetGraph(const FunctionBody* fbody); + + // Builds XLA computations for each of the arguments to the computation. + // `args` are the arguments to the computation. + Status BuildArguments(const Graph& graph, + const std::vector& args, + bool use_tuple_arg, xla::XlaBuilder* builder, + XlaContext* context, std::vector* arg_cores, + std::vector* arg_expressions, + std::vector* input_mapping, + std::vector* input_shapes, + bool is_entry_computation); + + // Graph compiler needs to know how to get an optimized graph from a function + // body. + friend class GraphCompiler; + friend class XlaCompilerTest; + + Options options_; + + // Status set to non-OK in the constructor if initialization fails. + Status initialization_status_; + + // Returns the next step sequence number. + int64 NextStepId(); + + // Internal sequence number for steps executed on the compilation device. + int64 next_step_id_; + + XlaCompilationDevice* device_; // Owned by device_mgr_ + DeviceMgr device_mgr_; + + // To avoid copying the client's function library, use a local function + // library and runtime for functions created as part of the functionalize + // control flow transformation. + std::unique_ptr local_flib_def_; + std::unique_ptr pflr_; + std::unique_ptr local_pflr_; + + FunctionLibraryRuntime* local_flib_runtime_; // owned by local_pflr_. + FunctionLibraryRuntime* flib_runtime_; // owned by pflr_. + + struct SignatureHash { + uint64 operator()( + const std::pair>& signature) const; + }; + + std::unordered_map>, + CompilationResult, SignatureHash> + cache_; + + std::unordered_map channels_; + + std::unordered_map host_compute_sends_; + std::unordered_map host_compute_recvs_; + + std::unordered_map host_compute_control_output_; + + // This is used to store mapping. Side-effecting + // ops call SetNodeToken() to record its token output, so later side-effecting + // ops can use GetNodeToken() to get it and use it as token input. + // + // It's a stack because we need a mapping like this for each level of nested + // CompileGraph() call. In CompileGraph(), we will push a new mapping to the + // stack, and pop the mapping before returning. + std::stack> node_token_mapping_stack_; + + TF_DISALLOW_COPY_AND_ASSIGN(XlaCompiler); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_TF2XLA_XLA_COMPILER_H_ diff --git a/linux/include/tensorflow/compiler/tf2xla/xla_context.h b/linux/include/tensorflow/compiler/tf2xla/xla_context.h new file mode 100644 index 000000000..d7dbdc957 --- /dev/null +++ b/linux/include/tensorflow/compiler/tf2xla/xla_context.h @@ -0,0 +1,203 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// This file defines the contexts used during XLA compilation. + +#ifndef TENSORFLOW_COMPILER_TF2XLA_XLA_CONTEXT_H_ +#define TENSORFLOW_COMPILER_TF2XLA_XLA_CONTEXT_H_ + +#include + +#include "tensorflow/compiler/tf2xla/xla_compilation_device.h" +#include "tensorflow/compiler/tf2xla/xla_compiler.h" +#include "tensorflow/compiler/xla/client/xla_builder.h" +#include "tensorflow/compiler/xla/client/xla_computation.h" +#include "tensorflow/compiler/xla/status_macros.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/resource_mgr.h" +#include "tensorflow/core/platform/macros.h" + +namespace tensorflow { + +class XlaOpKernelContext; + +// The XlaContext is the data structure that holds the state of an XLA +// compilation, that is accessible from OpKernelContexts when compiling a +// subgraph of Ops using XLA. +class XlaContext : public ResourceBase { + public: + // Retrieves the XlaContext of the current compilation. + static XlaContext& Get(const OpKernelContext* ctx); + static XlaContext& Get(const XlaOpKernelContext* ctx); + + // Creates a new XlaContext. See the documentation on the class data fields + // for descriptions of the arguments. + XlaContext(XlaCompiler* compiler, xla::XlaBuilder* builder, + bool allow_cpu_custom_calls, bool resolve_compile_time_constants, + bool is_entry_computation, + const std::function( + const TensorShape&, DataType)>* shape_representation_fn); + + // Virtual method defined by ResourceBase. + string DebugString() override; + + XlaCompiler* compiler() const { return compiler_; } + + // Returns the XlaBuilder that Ops use for compiling new expressions. + xla::XlaBuilder* builder(); + + bool allow_cpu_custom_calls() const { return allow_cpu_custom_calls_; } + + bool resolve_compile_time_constants() const { + return resolve_compile_time_constants_; + } + bool is_entry_computation() const { return is_entry_computation_; } + + const std::vector& args() const { return args_; } + void set_args(std::vector args); + + struct Retval { + DataType type; + TensorShape shape; + // An XlaExpression representing the Retval's value. + XlaExpression expression; + }; + const std::vector& retvals() { return retvals_; } + + // This is called by the Retval Op to associate a computed value + // with a specific return value of the subgraph. + void AddRetval(int retval_index, DataType type, const TensorShape& shape, + const xla::XlaOp& handle); + + // As for Retval, but for return values that are compile-time constants. + Status AddConstRetval(int retval_index, DataType dtype, + const xla::LiteralSlice& literal); + + // As for Retval, but for return values that are resource handles. + Status AddResourceRetval(int retval_index, XlaResource* resource); + + // As for Retval, but for return values that are XLA tokens. + Status AppendTokenRetval(const xla::XlaOp& token); + + // Creates a resource with resource `kind` and initial value `handle`. `name` + // is a descriptive name for use in error messages. See the `XlaResource` + // constructor for a description of the remaining arguments. + // Fails if the resource already exists. + Status CreateResource(XlaResource::Kind kind, int arg_num, string name, + DataType type, TensorShape shape, + const xla::XlaOp& handle, int64 tensor_array_size, + const std::set& tensor_array_gradients, + XlaResource** resource); + + const std::vector>& resources() { + return resources_; + } + + // Returns the XLA shape to be used to represent a variable of TF `shape` + // and `type`, or of an argument or return value of a top-level computation. + xla::StatusOr RepresentationShape(const TensorShape& shape, + DataType type) const; + + // Get an XLA lambda to compute Max. This is cached in the + // XlaContext since it may be used by multiple Ops. There is a + // separate specialization of the computation for each DataType. + const xla::XlaComputation* GetOrCreateMax(const DataType type); + + // Get an XLA lambda to compute Min. This is cached in the + // XlaContext since it may be used by multiple Ops. There is a + // separate specialization of the computation for each DataType. + const xla::XlaComputation* GetOrCreateMin(const DataType type); + + // Get an XLA lambda to compute Add. This is cached in the + // XlaContext since it may be used by multiple Ops. There is a + // separate specialization of the computation for each DataType. + const xla::XlaComputation* GetOrCreateAdd(const DataType type); + + // Get an XLA lambda to compute Mul. This is cached in the + // XlaContext since it may be used by multiple Ops. There is a + // separate specialization of the computation for each DataType. + const xla::XlaComputation* GetOrCreateMul(const DataType type); + + // The name of the XlaContext resource during symbolic graph execution. + static const char kXlaContextResourceName[]; + + private: + XlaCompiler* const compiler_; + + // The XlaBuilder used to construct the subgraph's compiled representation. + xla::XlaBuilder* builder_; + + // Allow ops to emit CustomCall operations for CPU. + const bool allow_cpu_custom_calls_; + + // If true, constant return values are returned as Tensors instead of + // run-time computation outputs. + const bool resolve_compile_time_constants_; + + // Arguments to the Tensorflow graph, indexed by _Arg index. + // Includes both compile-time constant arguments and runtime parameters. + std::vector args_; + + // Return values of the Tensorflow graph, indexed by _Retval index. + std::vector retvals_; + + // Holds ownership of resources. The resources are not ordered. + std::vector> resources_; + + // Is this a top-level computation, or an inner computation (e.g., a while + // body)? + const bool is_entry_computation_; + + // A function that describes how the shapes of + // a) argument and return value, for entry computations + // b) variables, for all computations, + // should be represented in XLA. Parameters/return values will be shaped + // according to this function, and reshaped back to/from their declared shapes + // for computations. Must be non-null. + const std::function(const TensorShape&, DataType)>* + shape_representation_fn_; + + // Cache of prebuilt computations indexed by their type. + using ComputationMap = std::map; + + // Finds the value for the given type in out map if it already + // exists or makes a new value with create function and keeps it the + // map. The returned value != nullptr and is owned by the map. + const xla::XlaComputation* LookupOrCreate( + DataType type, ComputationMap* out, + const std::function& create); + + // Cached computation to compute Max of two elements, specialized by type. + ComputationMap max_func_; + + // Cached computation to compute Min of two elements, specialized by type. + ComputationMap min_func_; + + // Cached computation to compute Sum of two elements, specialized by type. + ComputationMap add_func_; + + // Cached computation to compute Mul of two elements, specialized by type. + ComputationMap mul_func_; + + // Cached computation to compute Sigmoid of an element, specialized by type. + ComputationMap sigmoid_func_; + + TF_DISALLOW_COPY_AND_ASSIGN(XlaContext); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_TF2XLA_XLA_CONTEXT_H_ diff --git a/linux/include/tensorflow/compiler/tf2xla/xla_helpers.h b/linux/include/tensorflow/compiler/tf2xla/xla_helpers.h new file mode 100644 index 000000000..39578144c --- /dev/null +++ b/linux/include/tensorflow/compiler/tf2xla/xla_helpers.h @@ -0,0 +1,90 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// This file defines helper routines for the XLA device. + +#ifndef TENSORFLOW_COMPILER_TF2XLA_XLA_HELPERS_H_ +#define TENSORFLOW_COMPILER_TF2XLA_XLA_HELPERS_H_ + +#include "absl/types/span.h" +#include "tensorflow/compiler/tf2xla/xla_context.h" +#include "tensorflow/compiler/xla/client/xla_builder.h" +#include "tensorflow/core/framework/tensor.h" + +namespace tensorflow { + +// Helper methods for building XLA computations. +class XlaHelpers { + public: + // Returns a handle representing the zero value of a scalar + // element of data_type. + static xla::XlaOp Zero(xla::XlaBuilder* b, DataType data_type); + + // Returns a handle representing the one value of a scalar + // element of data_type. + static xla::XlaOp One(xla::XlaBuilder* b, DataType data_type); + + // Returns a handle representing the given value of an integer scalar + // element of data_type. + // Note that unlike One and Zero, does not work on boolean types. + static xla::XlaOp IntegerLiteral(xla::XlaBuilder* b, DataType data_type, + int64 value); + + // Returns a handle representing the given value of a floating-point scalar + // element of data_type. + static xla::XlaOp FloatLiteral(xla::XlaBuilder* b, DataType data_type, + double value); + + // Reshapes literal 'input' to have 'shape'. Both the original shape and + // 'shape' must contain the same number of elements. + static Status ReshapeLiteral(const xla::Literal& input, + absl::Span shape, + xla::Literal* output); + + // Returns the argmax of `input` along `axis`. `output_type` is the type to + // use for the output. + static xla::XlaOp ArgMax(xla::XlaOp input, xla::PrimitiveType output_type, + int axis); + + // Returns the argmin of `input` along `axis`. `output_type` is the type to + // use for the output. + static xla::XlaOp ArgMin(xla::XlaOp input, xla::PrimitiveType output_type, + int axis); + + // Converts `indices` into a one-hot representation. `depth` is the size + // of the new axis to add. `axis` is the position at which to add the new + // axis. `indices_shape` is the shape of `indices`. `on_value` and + // `off_value` represent the values to use for the on and off positions, + // respectively. + static Status OneHot(xla::XlaBuilder* builder, int64 depth, int axis, + DataType index_type, const TensorShape& indices_shape, + const xla::XlaOp& indices, const xla::XlaOp& on_value, + const xla::XlaOp& off_value, xla::XlaOp* one_hot); + + // Certain DataTypes should use increased precision DataTypes when performing + // reductions. This function remaps a given DataType to a higher precision + // DataType if needed. + static DataType SumAccumulationType(const DataType& dtype); + + // A helper for creating a ConvertElementType xla op given a DataType rather + // than the xla::PrimitiveType. + static xla::XlaOp ConvertElementType(xla::XlaBuilder* const builder, + const xla::XlaOp& operand, + const DataType new_element_type); +}; + +} // end namespace tensorflow + +#endif // TENSORFLOW_COMPILER_TF2XLA_XLA_HELPERS_H_ diff --git a/linux/include/tensorflow/compiler/tf2xla/xla_op_kernel.h b/linux/include/tensorflow/compiler/tf2xla/xla_op_kernel.h new file mode 100644 index 000000000..a3a0d10cc --- /dev/null +++ b/linux/include/tensorflow/compiler/tf2xla/xla_op_kernel.h @@ -0,0 +1,263 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_TF2XLA_XLA_OP_KERNEL_H_ +#define TENSORFLOW_COMPILER_TF2XLA_XLA_OP_KERNEL_H_ + +#include "tensorflow/compiler/tf2xla/xla_compiler.h" +#include "tensorflow/compiler/xla/client/xla_builder.h" +#include "tensorflow/compiler/xla/client/xla_computation.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/platform/macros.h" + +namespace tensorflow { + +class XlaOpKernelContext; + +// Implementations of operators that generate XLA code should usually subclass +// XlaOpKernel and implement the Compile() method. Unlike a regular OpKernel, +// an XlaOpKernel produces and consumes symbolic values during compilation. +// +// See the comments in xla_context.h for more details. +class XlaOpKernel : public OpKernel { + public: + explicit XlaOpKernel(OpKernelConstruction* construction); + + // Subclasses should implement Compile(), much as standard OpKernels implement + // Compute(). + virtual void Compile(XlaOpKernelContext* context) = 0; + + private: + void Compute(OpKernelContext* context) final; +}; + +// The context passed to the Compile() method of XlaOpKernel. An +// XlaOpKernelContext is a variant of the standard OpKernel class, tailored for +// implementing operators that perform symbolic execution as part of the XLA +// compiler. The key difference is that XlaOpKernelContext produces and consumes +// data as XLA computations, rather than as standard Tensors. +// +// Under the hood, symbolic execution communicates using special Tensors that +// wrap XlaExpression objects, however this is an implementation detail that +// this class hides. The *only* correct way to allocate a Tensor during +// compilation is using the XlaOpKernelContext methods, since they ensure there +// is a valid XlaExpression backing the tensor. No Op should ever call +// allocate_output or allocate_temp directly on the underlying OpKernelContext. +class XlaOpKernelContext { + public: + explicit XlaOpKernelContext(OpKernelContext* context); + + // Returns the XLA XlaBuilder containing the output of compilation. + xla::XlaBuilder* builder() const; + + // Inputs + + // Returns the number of inputs to the operator. + int num_inputs() const { return context_->num_inputs(); } + + // Returns the type of input `index`. + DataType input_type(int index) const; + + // Returns the type of input `name`. + DataType InputType(absl::string_view name); + + // Returns the type of input `index` as an xla::PrimitiveType. If the type + // is not representable as an XLA type, sets an error status and returns + // xla::PRIMITIVE_TYPE_INVALID. + xla::PrimitiveType input_xla_type(int index); + + // Returns the shape of input `index`. + TensorShape InputShape(int index); + + // Returns the shape of input with name `name`. + TensorShape InputShape(absl::string_view name); + + // Returns input `index` as a XlaOp. Unlike + // OpKernelContext::Input returns a symbolic value rather than a concrete + // Tensor. + const xla::XlaOp& Input(int index); + // Returns input `name` as a XlaOp. + const xla::XlaOp& Input(absl::string_view name); + + // Returns true if all inputs are the same shape, otherwise sets the + // status to a non-OK value and returns false. + // Usage: if (!context->ValidateInputsAreSameShape(this)) return; + bool ValidateInputsAreSameShape(OpKernel* op) TF_MUST_USE_RESULT; + + // Returns the named list-valued immutable input in "list", as + // defined in the OpDef. If the named output is not list-valued, + // returns a one-element list. + Status InputList(absl::string_view name, std::vector* handles, + std::vector* shapes); + + // Helper methods for constant inputs. + + // Evaluates input `index` and stores it in `*constant_literal`. If the + // expression cannot be evaluated, e.g., because it depends on unbound + // parameters, returns a non-OK status. + Status ConstantInput(int index, xla::Literal* constant_literal); + Status ConstantInput(absl::string_view name, xla::Literal* constant_literal); + + // Evaluates input `index`, reshapes it to `new_shape` if new_shape != + // InputShape(index), and stores it in `*constant_literal`. If the input + // cannot be evaluated, e.g., because it depends on unbound parameters, + // returns a non-Ok status. If InputShape(index).num_elements() != + // new_shape.num_elements(), returns an error status. + Status ConstantInputReshaped(int index, absl::Span new_dims, + xla::Literal* constant_literal); + + // Converts a constant scalar int32 or int64 tensor into an int64. + Status ConstantInputAsIntScalar(int index, int64* out); + Status ConstantInputAsIntScalar(absl::string_view name, int64* out); + + // Converts a constant scalar float32 or float64 tensor into a float64. + Status ConstantInputAsFloatScalar(int index, double* out); + + // Converts a constant 1D int32 or int64 tensor into a vector of int64s. + Status ConstantInputAsIntVector(int index, std::vector* out); + Status ConstantInputAsIntVector(absl::string_view name, + std::vector* out); + + // Reshapes and converts a constant int32 or int64 tensor into a vector of + // int64s. + Status ConstantInputReshapedToIntVector(int index, std::vector* out); + + // Converts a constant int32 or int64 Tensor into an xla int64 Literal. + Status ConstantInputAsInt64Literal(int index, xla::Literal* out); + Status ConstantInputAsInt64Literal(absl::string_view name, xla::Literal* out); + + // Converts a constant 1D int32 or int64 tensor into a TensorShape. + Status ConstantInputAsShape(int index, TensorShape* shape); + + // Returns the named list-valued immutable input in "list", as + // defined in the OpDef. If the named output is not list-valued, + // returns a one-element list. + Status ConstantInputList(absl::string_view name, + std::vector* literals); + + // Outputs + + int num_outputs() const { return context_->num_outputs(); } + DataType expected_output_dtype(int index) const { + return context_->expected_output_dtype(index); + } + + // Sets output `index` to the XlaOp `handle`. + // All outputs should be set using SetOutput and SetConstantOutput, not + // via the underlying OpKernelContext. + void SetOutput(int index, const xla::XlaOp& handle); + + // Sets output `index` to compile-time constant `host_tensor`, where + // `host_tensor` is a tensor in host memory. It is preferable to use + // SetConstantOutput where possible. + void SetConstantOutput(int index, const Tensor& host_tensor); + + // Sets output `index` to an invalid value. + // Any subsequent attempt to consume this output will cause an error. + void SetInvalidOutput(int index); + + // Status handling. + void SetStatus(const Status& status) { context_->SetStatus(status); } + Status status() { return context_->status(); } + + // Variables + + // Sets `*resource` to the resource associated with input `index`. + Status GetResourceInput(int index, XlaResource** resource); + + // Sets output `index` to be a reference to resource `resource`. + void SetResourceOutput(int index, XlaResource* resource); + + // Sets `*type` and `*shape` to the current type and shape of a variable's + // value. + Status GetVariableTypeAndShape(int index, DataType* type, + TensorShape* shape) const; + + // Reads the current value of the resouce variable referred to by input + // `index`. If `shape` is not nullptr, sets `*shape` to the shape of the + // variable. Returns an error if the variable has not been initialized, or if + // its type does not match `type`. + Status ReadVariableInput(int index, DataType type, TensorShape* shape, + xla::XlaOp* value); + // Reads the current value of the resouce variable referred to by input + // `name`. + Status ReadVariableInput(absl::string_view name, DataType type, + TensorShape* shape, xla::XlaOp* value); + + // Assigns the value `handle` to the variable referenced by input + // `input_index`. The variable must be of `type`. Returns an error if the + // variable has been initialized with a different type or with a + // different shape. + Status AssignVariable(int input_index, DataType type, xla::XlaOp handle); + // Assigns the value `handle` to the variable referenced by input `name`. + Status AssignVariable(absl::string_view name, DataType type, + xla::XlaOp handle); + + // Helper routines for the OP_REQUIRES macros + void CtxFailure(const Status& s); + void CtxFailureWithWarning(const Status& s); + void CtxFailure(const char* file, int line, const Status& s); + void CtxFailureWithWarning(const char* file, int line, const Status& s); + + // If this kernel invocation is within a function execution, + // call_frame() returns the call frame for the function call. + CallFrameInterface* call_frame() const { return context_->call_frame(); } + + FunctionLibraryRuntime* function_library() const { + return context_->function_library(); + } + + const OpKernel& op_kernel() const { return context_->op_kernel(); } + + // Returns the underlying OpKernelContext. Use rarely. + OpKernelContext* op_kernel_context() const { return context_; } + + // Returns the XlaCompiler that is performing the compilation. Used for, e.g., + // While to compile nested computations. + XlaCompiler* compiler() const; + + // TODO(phawkins): find a better home for these helpers. + + // Gets an XLA lambda to compute Max. This is cached in the + // XlaContext since it may be used by multiple Ops. There is a + // separate specialization of the computation for each DataType. + const xla::XlaComputation* GetOrCreateMax(const DataType type); + + // Gets an XLA lambda to compute Min. This is cached in the + // XlaContext since it may be used by multiple Ops. There is a + // separate specialization of the computation for each DataType. + const xla::XlaComputation* GetOrCreateMin(const DataType type); + + // Gets an XLA lambda to compute Add. This is cached in the + // XlaContext since it may be used by multiple Ops. There is a + // separate specialization of the computation for each DataType. + const xla::XlaComputation* GetOrCreateAdd(const DataType type); + + // Gets an XLA lambda to compute Mul. This is cached in the + // XlaContext since it may be used by multiple Ops. There is a + // separate specialization of the computation for each DataType. + const xla::XlaComputation* GetOrCreateMul(const DataType type); + + private: + // Returns the tensor of input `name`. + const Tensor& GetInputTensorByName(absl::string_view name); + + OpKernelContext* const context_; +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_TF2XLA_XLA_OP_KERNEL_H_ diff --git a/linux/include/tensorflow/compiler/tf2xla/xla_op_registry.h b/linux/include/tensorflow/compiler/tf2xla/xla_op_registry.h new file mode 100644 index 000000000..4b2c2bacd --- /dev/null +++ b/linux/include/tensorflow/compiler/tf2xla/xla_op_registry.h @@ -0,0 +1,317 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_TF2XLA_XLA_OP_REGISTRY_H_ +#define TENSORFLOW_COMPILER_TF2XLA_XLA_OP_REGISTRY_H_ + +#include +#include +#include +#include +#include + +#include "tensorflow/core/common_runtime/device_factory.h" +#include "tensorflow/core/common_runtime/local_device.h" +#include "tensorflow/core/framework/device_base.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/types.pb.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/platform/mem.h" +#include "tensorflow/core/platform/mutex.h" +#include "tensorflow/core/platform/thread_annotations.h" +#include "tensorflow/core/public/session_options.h" + +namespace tensorflow { + +// Names of the XLA compilation devices. These are not user-visible, and are +// used internally by the Tensorflow/XLA bridge to perform symbolic execution of +// a Tensorflow graph. + +extern const char* const DEVICE_CPU_XLA_JIT; // "CPU_XLA_JIT" +extern const char* const DEVICE_GPU_XLA_JIT; // "GPU_XLA_JIT" + +extern const char* const DEVICE_XLA_CPU; +extern const char* const DEVICE_XLA_GPU; + +constexpr std::array kFloatTypes = { + {DT_HALF, DT_FLOAT, DT_DOUBLE, DT_BFLOAT16}}; +constexpr std::array kNumericTypes = { + {DT_UINT8, DT_UINT32, DT_UINT64, DT_INT8, DT_INT32, DT_INT64, DT_HALF, + DT_FLOAT, DT_DOUBLE, DT_COMPLEX64, DT_BFLOAT16}}; + +constexpr std::array kCpuAllTypes = { + {DT_UINT8, DT_QUINT8, DT_UINT32, DT_UINT64, DT_INT8, DT_QINT8, DT_INT32, + DT_QINT32, DT_INT64, DT_HALF, DT_FLOAT, DT_DOUBLE, DT_COMPLEX64, DT_BOOL}}; + +constexpr std::array kGpuAllTypes = { + {DT_UINT8, DT_QUINT8, DT_UINT32, DT_UINT64, DT_INT8, DT_QINT8, DT_INT32, + DT_QINT32, DT_INT64, DT_HALF, DT_FLOAT, DT_DOUBLE, DT_COMPLEX64, DT_BOOL, + DT_BFLOAT16}}; + +// Class that manages registrations of operators and devices for the XLA JIT. +// Not thread-safe. +class XlaOpRegistry { + public: + typedef OpKernel* (*Factory)(OpKernelConstruction*); + + // Describes how to compile operators assigned to a device. + struct DeviceRegistration { + // The name of the an XLA compilation device to use to compile code. + string compilation_device_name; + + // Do operators assigned to this device require compilation? + bool requires_compilation; + + // If !requires_compilation, should we try to JIT operators on this device + // when XLA JIT compilation is enabled globally via the SessionOptions? + // (It is still possible to explicitly mark operators to JIT compile, even + // if enable_jit_by_default is false.) + bool enable_jit_by_default; + + // Enable compilation of operators that use DT_RESOURCE types? + bool compile_resource_ops = false; + }; + + // Registers an XLA backend. `compilation_device_name` is the name of the + // device used for symbolic execution during compilation. `supported_types` + // is the list of non-resource types supported by the device. Each operators + // will be registered for the intersection of the operator's supported types + // and the device's supported types. `backend_op_filter` is a function used + // to exclude or modify operator registrations on the device; it may be + // nullptr, in which case all ops are included. + // `backend_op_filter` should return true if the op should be registered on + // the device; it may optionally modify the KernelDef. + typedef bool (*BackendOpFilter)(KernelDef* kdef); + static void RegisterBackend(const string& compilation_device_name, + absl::Span supported_types, + BackendOpFilter op_filter); + + // Returns the names of the registered backends. + static std::vector BackendNames(); + + // Returns true iff a backend with the given name is registered. + static bool IsBackendRegistered(const string& name); + + // Registers `device_name` for XLA compilation, using information from + // `registration`. + static void RegisterCompilationDevice(const string& device_name, + const DeviceRegistration& registration); + + // Returns the JIT device name associated with 'device_name', setting + // 'jit_device_name', 'requires_jit', and 'enabled_jit_by_default', if they + // are not null. Returns false and leaves the outputs unchanged if no matching + // JIT device is registered. + // '*enable_jit_by_default' is set to true if we should try to JIT using this + // device when the JIT is enabled via the Session OptimizerOptions. + static bool GetCompilationDevice(const string& device_name, + const DeviceRegistration** registration); + + // Registers all JIT kernels on JIT devices, if not already registered. + // Does nothing otherwise. + static void RegisterCompilationKernels(); + + // Returns KernelDefs for compilation ops registered on + // 'compilation_device_name'. Does not include kernels registered as + // CompilationOnly, iff include_compilation_only_kernels=false. + static std::vector DeviceKernels( + const string& compilation_device_name, + bool include_compilation_only_kernels); + + // Returns all operations for which there are XLA kernels on any device. + static std::vector GetAllRegisteredOps(); + + // Returns the set of compile-time constant inputs to 'op'. Returns nullptr + // if the op is not registered. + static const std::unordered_set* CompileTimeConstantInputs( + const string& op); + + // Returns true if `op` is a "metadata" op, one that only looks at the shapes + // of its operands and not their values. + static bool IsMetadataOp(const string& op); + + private: + friend class XlaBackendRegistrar; + friend class XlaOpRegistrar; + friend class XlaOpRegistrationBuilder; + + static XlaOpRegistry& Instance(); + + XlaOpRegistry(); + ~XlaOpRegistry(); + + mutex mutex_; + + // Describes an XLA backend. + struct Backend { + // Which types are supported by this device? + std::set supported_types; + + // The per-backend operator filter function. See the comment on + // RegisterBackend() for details. + BackendOpFilter op_filter; + + // KernelDefs built by RegisterCompilationKernels() for each op supported + // by the device. + std::vector> kernel_defs; + }; + + // Map from compilation device names to a description of the backend. + std::unordered_map backends_ GUARDED_BY(mutex_); + + // Map from Tensorflow device names to the corresponding JIT device metadata. + std::unordered_map compilation_devices_ + GUARDED_BY(mutex_); + + // A description of a Tensorflow operator that can be compiled to XLA. + struct OpRegistration { + string name; + + // Should this operator be registered only on compilation devices, without a + // dummy kernel registered on the corresponding XLA device? + bool compilation_only = false; + + // Should we allow resource types for type attributes? Used by _Arg to + // allow DT_RESOURCE. + bool allow_resource_types = false; + + // Mapping from attribute name to a list of supported types. + std::unordered_map> type_constraints; + + // An optional whitelist of devices. If there is no whitelist, all devices + // are permitted. + bool has_device_whitelist = false; + std::unordered_set device_whitelist; + + // Names of arguments that must be compile-time constants. + std::unordered_set compile_time_constant_inputs; + + // True if this is a "metadata" op, one that only looks at the shapes of its + // operands and not their values. + bool is_metadata_op = false; + + // Factory used to build OpKernels that perform symbolic execution. + Factory factory; + }; + + // Returns true if registrations x and y can both be added to the registry. + // This is always the case if they refer to different ops. If they refer to + // the same op name, they must: have the same values for compilation_only and + // allow_resource_types; use a device_whitelist; and their + // whitelists must not intersect. + static bool IsCompatible(const OpRegistration& x, const OpRegistration& y); + + // Map from operator name to OpRegistrations, populated by REGISTER_XLA_OP. + // Registrations present under the same key must satisfy IsCompatible above, + // and this is checked during registration. + std::unordered_map>> ops_ + GUARDED_BY(mutex_); + + // Have we already registered the JIT kernels on the JIT devices? + bool jit_kernels_registered_ = false; + + // Holds ownership of OpKernelRegistrars that represent the Tensorflow kernel + // registrations created by RegisterCompilationKernels() and + // RegisterDeviceKernels(). + std::vector> + kernel_registrars_ GUARDED_BY(mutex_); +}; + +// REGISTER_XLA_OP() registers an XLA OpKernel by name, for example: +// REGISTER_XLA_OP(Name("Add"), AddOp); +// where 'AddOp' is the name of a JIT OpKernel class that implements "Add". +// +// We don't use a variadic macro here because we don't expect JIT operators to +// be templated. + +#define REGISTER_XLA_OP(NAME, OP) \ + REGISTER_XLA_OP_UNIQ_HELPER(__COUNTER__, NAME, OP) + +class XlaOpRegistrationBuilder { + public: + // Starts an operator registration chain. + static XlaOpRegistrationBuilder Name(absl::string_view name); + + // Specifies a whitelist of devices on which the operator may run. + XlaOpRegistrationBuilder& Device(absl::string_view devices); + XlaOpRegistrationBuilder& Device(absl::Span devices); + + // Specifies a type constraint for a type variable attribute. Each constraint + // specifies the set of types that the type variable may assume. + XlaOpRegistrationBuilder& TypeConstraint(absl::string_view attr_name, + DataType allowed); + + XlaOpRegistrationBuilder& TypeConstraint(absl::string_view attr_name, + absl::Span allowed); + + // Specifies that a dummy copy of this operator should not be registered on + // XLA_* devices, but may be used during compilation. + XlaOpRegistrationBuilder& CompilationOnly(); + + // Allow DT_RESOURCE types for type parameters. + XlaOpRegistrationBuilder& AllowResourceTypes(); + + // Mark 'input_name' as an argument whose value must be known at compile-time. + XlaOpRegistrationBuilder& CompileTimeConstInput(absl::string_view input_name); + + // Mark this op as a "metadata" op, one that only looks at the shapes of its + // operands and not their values. + XlaOpRegistrationBuilder& IsMetadataOp(); + + std::unique_ptr Build( + XlaOpRegistry::Factory factory); + + private: + XlaOpRegistrationBuilder(absl::string_view name); + + std::unique_ptr registration_; +}; + +// REGISTER_XLA_BACKEND() registers an XLA backend. Example usage: +// REGISTER_XLA_BACKEND(DEVICE_GPU_XLA_JIT, kGpuAllTypes, GpuOpFilter); +#define REGISTER_XLA_BACKEND(NAME, ...) \ + REGISTER_XLA_BACKEND_UNIQ_HELPER(__COUNTER__, NAME, __VA_ARGS__) + +// Implementation details. + +class XlaOpRegistrar { + public: + XlaOpRegistrar(std::unique_ptr registration); +}; + +#define REGISTER_XLA_OP_UNIQ_HELPER(COUNTER, BUILDER, OP) \ + REGISTER_XLA_OP_UNIQ(COUNTER, BUILDER, OP) + +#define REGISTER_XLA_OP_UNIQ(CTR, BUILDER, OP) \ + static ::tensorflow::XlaOpRegistrar xla_op_registrar__body__##CTR##__object( \ + ::tensorflow::XlaOpRegistrationBuilder::BUILDER.Build( \ + [](::tensorflow::OpKernelConstruction* context) \ + -> ::tensorflow::OpKernel* { return new OP(context); })); + +class XlaBackendRegistrar { + public: + XlaBackendRegistrar(absl::string_view name, absl::Span types, + XlaOpRegistry::BackendOpFilter op_filter = nullptr); +}; + +#define REGISTER_XLA_BACKEND_UNIQ_HELPER(COUNTER, NAME, ...) \ + REGISTER_XLA_BACKEND_UNIQ(COUNTER, NAME, __VA_ARGS__) + +#define REGISTER_XLA_BACKEND_UNIQ(CTR, NAME, ...) \ + static ::tensorflow::XlaBackendRegistrar \ + xla_backend_registrar__body__##CTR##__object(NAME, __VA_ARGS__); + +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_TF2XLA_XLA_OP_REGISTRY_H_ diff --git a/linux/include/tensorflow/compiler/tf2xla/xla_resource.h b/linux/include/tensorflow/compiler/tf2xla/xla_resource.h new file mode 100644 index 000000000..2438490be --- /dev/null +++ b/linux/include/tensorflow/compiler/tf2xla/xla_resource.h @@ -0,0 +1,152 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_TF2XLA_XLA_RESOURCE_H_ +#define TENSORFLOW_COMPILER_TF2XLA_XLA_RESOURCE_H_ + +#include + +#include "tensorflow/compiler/xla/client/xla_builder.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.pb.h" +#include "tensorflow/core/lib/core/status.h" + +namespace tensorflow { + +// Represents a resource, such as a Variable or TensorArray. +class XlaResource { + public: + enum Kind { + kInvalid, + kVariable, + kTensorArray, + kStack, + }; + + XlaResource(Kind kind, int arg_num, string name, DataType type, + TensorShape shape, const xla::XlaOp& initial_value, + int64 tensor_array_size, + const std::set& tensor_array_gradients); + + XlaResource(const XlaResource&) = delete; + XlaResource(XlaResource&&) = delete; + XlaResource& operator=(const XlaResource&) = delete; + XlaResource& operator=(XlaResource&&) = delete; + + Kind kind() const { return kind_; } + + // If this resource is visible externally to the computation, what was its + // argument number? + // < 0 means "not visible externally". + int arg_num() const { return arg_num_; } + + // A descriptive name for the resource, used in error messages. + const string& name() const { return name_; } + + // Current type and value of the resource. Uninitialized resources are + // represented by a default (zero) handle and type DT_INVALID. + // While the type of a resource is notionally fixed during execution, when + // a resource is first initialized we do not yet know its type, so we keep + // track of its type dynamically. + DataType type() const { return type_; } + + // Shape of the resource. For an uninitialized resource, this is ignored. + // For a Variable, this is the shape of the value. For a TensorArray or Stack + // this is the shape of each entry in the TensorArray/Stack. + const TensorShape& shape() const { return shape_; } + + const xla::XlaOp& value() const { return value_; } + + // Value of the resource at computation entry. Used to detect which + // variables have new values that need to be written back. + const xla::XlaOp& initial_value() const { return initial_value_; } + + // A variable is initialized if it has a value. + bool initialized() const { return value_.valid(); } + + // Sets the type and shape of the resource. The type and shape of a resource + // must not change once the variable has been initialized. + Status SetTypeAndShape(DataType type, const TensorShape& shape); + + // Sets the current value of the resource. Returns an error if the type is not + // set to a valid value. + Status SetValue(const xla::XlaOp& value); + + // Sets the current value of the resource to an all-zero value. + Status SetZeroValue(xla::XlaBuilder* builder); + + // Looks up the gradient for `source`, or creates it if it does not already + // exist. The call target must be an initialized TensorArray resource. A + // TensorArray can have multiple named gradients; see the operator + // documentation for TensorArrayGradV3 for details. + Status GetOrCreateTensorArrayGradient(const string& source, + xla::XlaBuilder* builder, + XlaResource** gradient_out); + + // Packs a resource into a single XLA value `pack`, suitable for use as + // an XlaCompiler::Argument. For non-TensorArrays or TensorArrays without + // gradients, sets `*pack` to `value`. + // For TensorArrays with gradients, packs the value and its gradient values in + // a tuple; the gradients values are packed in order by source name. + Status Pack(xla::XlaOp* pack, xla::XlaBuilder* builder) const; + + // Updates the resource with values from `pack`. If `gradient_sources` is + // non-empty, treats `pack` as a tuple that represents a TensorArray and + // its gradients, and unpacks and updates the gradient resources. + // If `reset_initial_values` is true, sets the initial_values as well as the + // values. + // Opposite of Pack(). + Status SetFromPack(const std::set& gradient_sources, + const xla::XlaOp& pack, xla::XlaBuilder* builder); + + // TensorArray and Stack specific fields + + // 'tensor_array_size' stores the expected size of the TensorArray or Stack. + // We need to store this since sometimes TensorArrays must be initialized + // lazily since we do not know the element shape at construction time. + // Used by both TensorArrays and Stacks. + int64 tensor_array_size() const { return tensor_array_size_; } + void set_tensor_array_size(int64 size) { tensor_array_size_ = size; } + + // 'tensor_array_gradient' is a map from TensorArrayGradV3 'source' attributes + // to an XlaResource containing the gradient TensorArrays. We store a pointer + // here since there should only be one gradient TensorArray per 'source' + // string, irrespective of the number of calls to TensorArrayGrad. The map + // is ordered since values are packed into tuples by Pack() sorted by name + // order. + const std::map>& tensor_array_gradients() + const { + return tensor_array_gradients_; + } + + private: + const Kind kind_; + const int arg_num_; + const string name_; + + DataType type_; + TensorShape shape_; + xla::XlaOp value_; + xla::XlaOp initial_value_; + + int64 tensor_array_size_ = -1; + + std::map> tensor_array_gradients_; +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_TF2XLA_XLA_RESOURCE_H_ diff --git a/linux/include/tensorflow/compiler/xla/array.h b/linux/include/tensorflow/compiler/xla/array.h new file mode 100644 index 000000000..58cc15758 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/array.h @@ -0,0 +1,571 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_ARRAY_H_ +#define TENSORFLOW_COMPILER_XLA_ARRAY_H_ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "absl/strings/str_cat.h" +#include "absl/strings/str_join.h" +#include "absl/types/span.h" +#include "tensorflow/compiler/xla/status.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/core/lib/core/bits.h" +#include "tensorflow/core/platform/logging.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/types.h" + +namespace xla { + +namespace array_impl { + +// conjunction +// +// Performs a compile-time logical AND operation on the passed types (which +// must have `::value` members convertible to `bool`. Short-circuits if it +// encounters any `false` members (and does not compare the `::value` members +// of any remaining arguments). +// +// This metafunction is designed to be a drop-in replacement for the C++17 +// `std::conjunction` metafunction. +template +struct conjunction; + +template +struct conjunction + : std::conditional, T>::type {}; + +template <> +struct conjunction<> : std::true_type {}; + +// A type trait that is valid when all elements in a parameter pack are of +// integral type. +template +using pack_is_integral = conjunction...>; + +// Compares three same-sized vectors elementwise. For each item in `values`, +// returns false if any of values[i] is outside the half-open range [starts[i], +// ends[i]). +template +bool all_inside_range(const C1& values, const C2& range_starts, + const C3& range_ends) { + for (size_t i = 0, e = values.size(); i < e; ++i) { + if (values[i] < range_starts[i] || values[i] >= range_ends[i]) { + return false; + } + } + return true; +} + +} // namespace array_impl + +// General N dimensional array class with arbitrary value type. +template +class Array { + public: + // Type inference can have a hard time parsing very deep initializer list + // nests, especially if one or more dimensions is one as the compiler just + // sees a single-element integer initializer. These typedefs allow casting + // explicitly with less typing. + using InitializerList1D = std::initializer_list; + using InitializerList2D = std::initializer_list; + using InitializerList3D = std::initializer_list; + using InitializerList4D = std::initializer_list; + + using value_type = T; + + // Creates a new array with the specified dimensions. + explicit Array(absl::Span sizes) : Array(sizes, T()) {} + + // Creates a new array with the specified dimensions and specified value for + // every cell. + Array(absl::Span sizes, T value) + : sizes_(sizes.begin(), sizes.end()), values_(new T[num_elements()]) { + Fill(value); + } + + // Creates a 2D array from the given nested initializer list. The outer + // initializer list is the first dimension, the inner is the second dimension. + // For example, {{1, 2, 3}, {4, 5, 6}} results in an array with n1=2 and n2=3. + Array(InitializerList2D values) + : Array(ToInt64Vector({values.size(), values.begin()->size()})) { + int64 idx = 0; + for (const auto& it1 : values) { + for (const auto& it2 : it1) { + values_[idx] = it2; + ++idx; + } + } + CHECK(idx == num_elements()); + } + + // Creates a 1D array of a floating-point type (half, bfloat16, float, + // or double) from an initializer list of float values. + template ::value || + std::is_same::value || + std::is_same::value || + std::is_same::value) && + std::is_same::value>::type> + Array(std::initializer_list values) + : Array(ToInt64Vector({values.size()})) { + int64 idx = 0; + for (const auto& it1 : values) { + values_[idx] = static_cast(it1); + ++idx; + } + CHECK(idx == num_elements()); + } + + // Creates a 2D array of a floating-point type (half, bfloat16, float, + // or double) from an initializer list of float values. + template ::value || + std::is_same::value || + std::is_same::value || + std::is_same::value) && + std::is_same::value>::type> + Array(std::initializer_list> values) + : Array(ToInt64Vector({values.size(), values.begin()->size()})) { + int64 idx = 0; + for (const auto& it1 : values) { + for (const auto& it2 : it1) { + values_[idx] = static_cast(it2); + ++idx; + } + } + CHECK(idx == num_elements()); + } + + // Creates a 3D array from the given nested initializer list. The outer + // initializer list is the first dimension, and so on. + Array(InitializerList3D values) + : Array(ToInt64Vector({values.size(), values.begin()->size(), + values.begin()->begin()->size()})) { + int64 idx = 0; + for (const auto& it1 : values) { + for (const auto& it2 : it1) { + for (const auto& it3 : it2) { + values_[idx] = it3; + ++idx; + } + } + } + CHECK(idx == num_elements()); + } + + // Creates a 3D array of a floating-point type (half, bfloat16, float, + // or double) from an initializer list of float values. + template ::value || + std::is_same::value || + std::is_same::value || + std::is_same::value) && + std::is_same::value>::type> + Array(std::initializer_list>> + values) + : Array(ToInt64Vector({values.size(), values.begin()->size(), + values.begin()->begin()->size()})) { + int64 idx = 0; + for (const auto& it1 : values) { + for (const auto& it2 : it1) { + for (const auto& it3 : it2) { + values_[idx] = static_cast(it3); + ++idx; + } + } + } + CHECK(idx == num_elements()); + } + + // Creates a 4D array from the given nested initializer list. The outer + // initializer list is the first dimension, and so on. + Array(InitializerList4D values) + : Array(ToInt64Vector({values.size(), values.begin()->size(), + values.begin()->begin()->size(), + values.begin()->begin()->begin()->size()})) { + int64 idx = 0; + for (const auto& it1 : values) { + for (const auto& it2 : it1) { + for (const auto& it3 : it2) { + for (const auto& it4 : it3) { + values_[idx] = it4; + ++idx; + } + } + } + } + CHECK(idx == num_elements()); + } + + // Creates a 4D array of a floating-point type (half, bfloat16, float, + // or double) from an initializer list of float values. + template ::value || + std::is_same::value || + std::is_same::value || + std::is_same::value) && + std::is_same::value>::type> + Array(std::initializer_list< + std::initializer_list>>> + values) + : Array(ToInt64Vector({values.size(), values.begin()->size(), + values.begin()->begin()->size(), + values.begin()->begin()->begin()->size()})) { + int64 idx = 0; + for (const auto& it1 : values) { + for (const auto& it2 : it1) { + for (const auto& it3 : it2) { + for (const auto& it4 : it3) { + values_[idx] = static_cast(it4); + ++idx; + } + } + } + } + CHECK(idx == num_elements()); + } + + Array(const Array& other) + : sizes_(other.sizes_), values_(new T[num_elements()]) { + std::copy(&other.values_[0], &other.values_[0] + num_elements(), + &values_[0]); + } + + Array& operator=(const Array& other) { + sizes_ = other.sizes_; + values_.reset(new T[num_elements()]); + std::copy(&other.values_[0], &other.values_[0] + num_elements(), + &values_[0]); + return *this; + } + + // Fills the array with the specified value. + void Fill(const T& value) { + std::fill(&values_[0], &values_[0] + num_elements(), value); + } + + // Fills the array with sequentially increasing values. + void FillIota(const T& value) { + std::iota(&values_[0], &values_[0] + num_elements(), value); + } + + // Fills the array with the sequence i*multiplier for i=0,1,... + void FillWithMultiples(const T& multiplier) { + for (int64 i = 0; i < num_elements(); ++i) { + values_[i] = static_cast(i) * multiplier; + } + } + + // Fills the array with random normal variables with the specified mean. + void FillRandom(const T& value, const double mean = 0.0, + const int seed = 12345) { + std::mt19937 g(seed); + std::normal_distribution distribution(mean, + static_cast(value)); + for (int64 i = 0; i < num_elements(); ++i) { + values_[i] = static_cast(distribution(g)); + } + } + + // Sets all the values in the array to values specified in the container. + template > + void SetValues(const Container& container) { + CHECK_EQ(std::distance(std::begin(container), std::end(container)), + num_elements()); + std::copy(std::begin(container), std::end(container), &values_[0]); + } + + // Invokes a callback with the (indices, value_ptr) for each cell in the + // array. + void Each(std::function, T*)> f) { + std::vector index(sizes_.size()); + for (int64 i = 0; i < num_elements(); ++i, next_index(&index)) { + f(index, &values_[i]); + } + } + + // Invokes a callback with the (indices, value) for each cell in the array. + void Each(std::function, T)> f) const { + std::vector index(sizes_.size()); + for (int64 i = 0; i < num_elements(); ++i, next_index(&index)) { + f(index, values_[i]); + } + } + + // Invokes a callback with the (indices, value_ptr) for each cell in the + // array. If a callback returns a non-OK status, returns that else returns + // Status::OK(). + Status EachStatus(std::function, T*)> f) { + std::vector index(sizes_.size()); + for (int64 i = 0; i < num_elements(); ++i, next_index(&index)) { + Status s = f(index, &values_[i]); + if (!s.ok()) { + return s; + } + } + return Status::OK(); + } + + // Invokes a callback with the (indices, value) for each cell in the array. + // If a callback returns a non-OK status, returns that else returns + // Status::OK(). + Status EachStatus(std::function, T)> f) const { + std::vector index(sizes_.size()); + for (int64 i = 0; i < num_elements(); ++i, next_index(&index)) { + Status s = f(index, values_[i]); + if (!s.ok()) { + return s; + } + } + return Status::OK(); + } + + // Returns the value at the cell specified by the indexes. The number of + // arguments have to match with the number of dimensions for the array. + // + // The type trait is required to avoid this overload participating too + // eagerly; a parameter pack can take zero or more elements, so we must + // restrict this to only parameter packs that are all of integral type. + template + typename std::enable_if::value, + const T&>::type + operator()(Dims... dims) const { + // We are using a std::array to avoid having to allocate memory in this + // function for performance reasons. + std::array indexes{{static_cast(dims)...}}; + return values_[calculate_index(indexes)]; + } + + // Returns the value at the cell specified by the indexes. The number of + // arguments have to match with the number of dimensions for the array. + template + typename std::enable_if::value, + T&>::type + operator()(Dims... dims) { + // We are using a std::array to avoid having to allocate memory in this + // function for performance reasons. + std::array indexes{{static_cast(dims)...}}; + return values_[calculate_index(indexes)]; + } + + // Returns the value at the cell specified by the indexes. The number of + // arguments have to match with the number of dimensions for the array. + const T& operator()(absl::Span indexes) const { + return values_[calculate_index(indexes)]; + } + + // Returns the value at the cell specified by the indexes. The number of + // arguments have to match with the number of dimensions for the array. + T& operator()(absl::Span indexes) { + return values_[calculate_index(indexes)]; + } + + // Low-level accessor for stuff like memcmp, handle with care. Returns pointer + // to the underlying storage of the array (similarly to std::vector::data()). + T* data() const { + // TODO(tberghammer): Get rid of the const_cast. Currently it is needed + // because the Eigen backend needs a non-const pointers even for reading + // from the array. + return const_cast(this)->values_.get(); + } + + // Returns the size of the dimension at the given index. + int64 dim(int64 n) const { + CHECK(n < sizes_.size()); + return sizes_[n]; + } + + // Returns a vector containing the dimensions of the array. + const std::vector& dimensions() const { return sizes_; } + + int64 num_dimensions() const { return sizes_.size(); } + + // Returns the total number of elements in the array. + int64 num_elements() const { + return std::accumulate(sizes_.begin(), sizes_.end(), 1LL, + std::multiplies()); + } + + const T* begin() const { return &values_[0]; } + T* begin() { return &values_[0]; } + const T* end() const { return &values_[num_elements()]; } + T* end() { return &values_[num_elements()]; } + + bool operator==(const Array& other) const { + if (sizes_.size() != other.sizes_.size()) { + return false; + } + for (int64 i = 0; i < sizes_.size(); ++i) { + if (sizes_[i] != other.sizes_[i]) { + return false; + } + } + for (int64 i = 0; i < num_elements(); ++i) { + if (values_[i] != other.values_[i]) { + return false; + } + } + return true; + } + + bool operator!=(const Array& other) const { return !(*this == other); } + + // Performs the equivalent of a slice operation on this array. + Array Slice(absl::Span starts, + absl::Span limits) const { + CHECK_EQ(starts.size(), num_dimensions()); + CHECK_EQ(limits.size(), num_dimensions()); + + std::vector sizes; + std::transform(starts.begin(), starts.end(), limits.begin(), + std::back_inserter(sizes), + [](int64 start, int64 limit) { return limit - start; }); + Array result(sizes); + + std::vector index(sizes_.size()); + int64 slice_i = 0; + for (int64 i = 0; i < num_elements(); ++i, next_index(&index)) { + if (array_impl::all_inside_range(index, starts, limits)) { + // Even though the bounds of result are different to our bounds, we're + // iterating in the same order. So we can simply write successive linear + // indices instead of recalculating a multi-dimensional index. + result.values_[slice_i++] = values_[i]; + } + } + return result; + } + + // Performs the equivalent of a DynamicUpdateSlice in-place on this array. + void UpdateSlice(const Array& from, + absl::Span start_indices) { + CHECK_EQ(from.num_dimensions(), num_dimensions()); + std::vector limit_indices; + std::transform(start_indices.begin(), start_indices.end(), + from.dimensions().begin(), std::back_inserter(limit_indices), + std::plus{}); + std::vector index(sizes_.size()); + int64 from_i = 0; + for (int64 i = 0; i < num_elements(); ++i, next_index(&index)) { + if (array_impl::all_inside_range(index, start_indices, limit_indices)) { + // Even though the bounds of from are different to our bounds, we're + // iterating in the same order. So we can simply write successive linear + // indices instead of recalculating a multi-dimensional index. + values_[i] = from.values_[from_i++]; + } + } + } + + // Performs an in-place reshape, modifying the dimensions but not the + // underlying data. + void Reshape(absl::Span new_dimensions) { + int64 old_num_elements = num_elements(); + sizes_ = std::vector(new_dimensions.begin(), new_dimensions.end()); + CHECK_EQ(num_elements(), old_num_elements); + } + + // Returns a string representation of the array suitable for debugging. + string ToString() const { + std::vector pieces; + std::vector index(sizes_.size()); + do { + // Emit leading spaces and opening square brackets + if (index.back() == 0) { + for (int64 i = sizes_.size() - 1; i >= 0; --i) { + if (i == 0 || index[i - 1] != 0) { + for (int64 j = 0; j < sizes_.size(); ++j) { + pieces.push_back(j < i ? " " : "["); + } + break; + } + } + } + + pieces.push_back(absl::StrCat(values_[calculate_index(index)])); + + // Emit comma if it isn't the last element + if (index.back() != sizes_.back() - 1) { + pieces.push_back(", "); + } + + // Emit closing square brackets + for (int64 i = sizes_.size() - 1; i >= 0; --i) { + if (index[i] != sizes_[i] - 1) { + break; + } + pieces.push_back("]"); + if (i != 0 && index[i - 1] != sizes_[i - 1] - 1) { + pieces.push_back(",\n"); + } + } + } while (next_index(&index)); + return absl::StrJoin(pieces, ""); + } + + private: + // Converts an initializer_list of type U to a vector of type int64. Used by + // the initializer list based constructors to convert the size type into int64 + // to be passed to the size based constructor. + template + static std::vector ToInt64Vector( + const std::initializer_list& data) { + return std::vector(data.begin(), data.end()); + } + + // Returns the linear index from the list of per-dimension indexes. Function + // is templated so can be used with an std::array from operator() to avoid + // memory allocation. + template + int64 calculate_index(const U& indexes) const { + CHECK_EQ(sizes_.size(), indexes.size()); + int64 index = 0; + for (int64 i = 0; i < sizes_.size(); ++i) { + index *= sizes_[i]; + index += indexes[i]; + } + return index; + } + + // Advances the specified set of indexes and returns true if we haven't + // wrapped around (i.e. result isn't {0, 0, ...}). + bool next_index(std::vector* index) const { + CHECK_EQ(index->size(), sizes_.size()); + for (int64 i = sizes_.size() - 1; i >= 0; --i) { + (*index)[i]++; + if ((*index)[i] < sizes_[i]) { + return true; + } + (*index)[i] = 0; + } + return false; + } + + std::vector sizes_; + std::unique_ptr values_; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_ARRAY_H_ diff --git a/linux/include/tensorflow/compiler/xla/array2d.h b/linux/include/tensorflow/compiler/xla/array2d.h new file mode 100644 index 000000000..782c966b4 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/array2d.h @@ -0,0 +1,119 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_ARRAY2D_H_ +#define TENSORFLOW_COMPILER_XLA_ARRAY2D_H_ + +#include +#include +#include +#include +#include +#include +#include + +#include "absl/memory/memory.h" +#include "absl/strings/str_cat.h" +#include "tensorflow/compiler/xla/array.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/core/lib/core/bits.h" +#include "tensorflow/core/platform/logging.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/types.h" + +namespace xla { + +template +class Array2D : public Array { + public: + Array2D() : Array(std::vector{0, 0}) {} + + Array2D(const int64 n1, const int64 n2) + : Array(std::vector{n1, n2}) {} + + Array2D(const int64 n1, const int64 n2, const T value) + : Array({n1, n2}, value) {} + + // Creates an array from the given nested initializer list. The outer + // initializer list is the first dimension; the inner is the second dimension. + // For example, {{1, 2, 3}, {4, 5, 6}} results in an array with n1=2 and n2=3. + Array2D(std::initializer_list> values) + : Array(values) {} + + // Creates an array of a floating-point type (half, bfloat16, float, + // or double) from the given nested initializer list of float values. + template ::value || + std::is_same::value || + std::is_same::value || + std::is_same::value) && + std::is_same::value>::type> + Array2D(std::initializer_list> values) + : Array(values) {} + + Array2D(const Array2D& other) : Array(other) {} + + int64 n1() const { return this->dim(0); } + int64 n2() const { return this->dim(1); } + + int64 height() const { return this->dim(0); } + int64 width() const { return this->dim(1); } + + // Fills the array with a pattern of values of the form: + // + // (rowno << log2ceil(width) | colno) + start_value + // + // This makes it easy to see distinct row/column values in the array. + void FillUnique(T start_value = 0) { + for (int64 i0 = 0; i0 < n1(); ++i0) { + for (int64 i1 = 0; i1 < n2(); ++i1) { + (*this)(i0, i1) = + ((i0 << tensorflow::Log2Ceiling64(n2())) | i1) + start_value; + } + } + } + + // Applies f to all cells in this array, in row-major order. + void Each(std::function f) { + for (int64 i0 = 0; i0 < n1(); ++i0) { + for (int64 i1 = 0; i1 < n2(); ++i1) { + f(i0, i1, &(*this)(i0, i1)); + } + } + } +}; + +// Returns a linspace-populated Array2D in the range [from, to] (inclusive) +// with dimensions n1 x n2. +template +std::unique_ptr> MakeLinspaceArray2D(double from, double to, + int64 n1, int64 n2) { + auto array = absl::make_unique>(n1, n2); + int64 count = n1 * n2; + NativeT step = + static_cast((count > 1) ? (to - from) / (count - 1) : 0); + auto set = [&array, n1, n2](int64 index, NativeT value) { + (*array)(index / n2, index % n2) = value; + }; + for (int64 i = 0; i < count - 1; ++i) { + set(i, (static_cast(from) + + static_cast(i) * static_cast(step))); + } + set(count - 1, static_cast(to)); + return array; +} +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_ARRAY2D_H_ diff --git a/linux/include/tensorflow/compiler/xla/array3d.h b/linux/include/tensorflow/compiler/xla/array3d.h new file mode 100644 index 000000000..0e9a0722a --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/array3d.h @@ -0,0 +1,80 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_ARRAY3D_H_ +#define TENSORFLOW_COMPILER_XLA_ARRAY3D_H_ + +#include +#include +#include +#include +#include +#include +#include + +#include "tensorflow/compiler/xla/array.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/core/platform/logging.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/types.h" + +namespace xla { + +// Simple 3D array structure. +template +class Array3D : public Array { + public: + Array3D() : Array(std::vector{0, 0, 0}) {} + + // Creates an array of dimensions n1 x n2 x n3, uninitialized values. + Array3D(const int64 n1, const int64 n2, const int64 n3) + : Array(std::vector{n1, n2, n3}) {} + + // Creates an array of dimensions n1 x n2 x n3, initialized to value. + Array3D(const int64 n1, const int64 n2, const int64 n3, const T value) + : Array(std::vector{n1, n2, n3}, value) {} + + // Creates an array from the given nested initializer list. The outer + // initializer list is the first dimension, and so on. + // + // For example {{{1, 2}, {3, 4}, {5, 6}, {7, 8}}, + // {{9, 10}, {11, 12}, {13, 14}, {15, 16}}, + // {{17, 18}, {19, 20}, {21, 22}, {23, 24}}} + // results in an array with n1=3, n2=4, n3=2. + Array3D(std::initializer_list>> + values) + : Array(values) {} + + // Creates an array of a floating-point type (half, bfloat16, float, + // or double) from the given nested initializer list of float values. + template ::value || + std::is_same::value || + std::is_same::value || + std::is_same::value) && + std::is_same::value>::type> + Array3D( + std::initializer_list>> + values) + : Array(values) {} + + int64 n1() const { return this->dim(0); } + int64 n2() const { return this->dim(1); } + int64 n3() const { return this->dim(2); } +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_ARRAY3D_H_ diff --git a/linux/include/tensorflow/compiler/xla/array4d.h b/linux/include/tensorflow/compiler/xla/array4d.h new file mode 100644 index 000000000..e23d317ba --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/array4d.h @@ -0,0 +1,160 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_ARRAY4D_H_ +#define TENSORFLOW_COMPILER_XLA_ARRAY4D_H_ + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "absl/strings/str_cat.h" +#include "absl/types/span.h" +#include "tensorflow/compiler/xla/array.h" +#include "tensorflow/compiler/xla/array2d.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/core/platform/logging.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/types.h" + +namespace xla { + +// Simple 4D array structure, similar in form to Array2D, for use primarily in +// testing and describing to XLA APIs values in the 4D array structures used +// in convolutions. +// +// The data layout is, in order from major to minor: +// +// First dimension: plane, batch, n1 +// Second dimension: depth, feature, z, n2 +// Third dimension: height, y, n3 +// Fourth dimension: width, x, n4 +// +// These dimensions are referred to by various names, so that is why +// more than one name is given above. See operator() for the exact +// calculation of 1d indices from 4d indices. +template +class Array4D : public Array { + public: + // Creates a 4D array, uninitialized values. + Array4D(int64 planes, int64 depth, int64 height, int64 width) + : Array(std::vector{planes, depth, height, width}) {} + + // Creates a 4D array, initialized to value. + Array4D(int64 planes, int64 depth, int64 height, int64 width, T value) + : Array(std::vector{planes, depth, height, width}, value) {} + + // Creates a 4D array, filled with values. + // + // We need to set a default type for Container so that code like + // Array4D(1, 1, 1, 1, {1}) will work. The template cannot infer the + // initializer_list type in that case without this default. + template > + Array4D(int64 planes, int64 depth, int64 height, int64 width, + const Container& values) + : Array4D(planes, depth, height, width) { + this->SetValues(values); + } + + // Construct an Array4D with the given nested initializer list. + Array4D(std::initializer_list>>> + values) + : Array(values) {} + + // Creates an array of a floating-point type (half, bfloat16, float, + // or double) from the given nested initializer list of float values. + template ::value || + std::is_same::value || + std::is_same::value || + std::is_same::value) && + std::is_same::value>::type> + Array4D(std::initializer_list>>> + values) + : Array(values) {} + + // Numerically-named aliases for the various dimensions. This matches the + // dimension names used in array3d. + int64 n4() const { return this->dim(3); } + int64 n3() const { return this->dim(2); } + int64 n2() const { return this->dim(1); } + int64 n1() const { return this->dim(0); } + + int64 width() const { return this->dim(3); } + int64 height() const { return this->dim(2); } + int64 depth() const { return this->dim(1); } + int64 planes() const { return this->dim(0); } + + // Fills all of the {p,z} with the array provided, which specifies {y,x}. + void FillWithYX(const Array2D& value) { + CHECK_EQ(value.height(), height()); + CHECK_EQ(value.width(), width()); + for (int64 plane = 0; plane < planes(); ++plane) { + for (int64 depth = 0; depth < this->depth(); ++depth) { + for (int64 height = 0; height < this->height(); ++height) { + for (int64 width = 0; width < this->width(); ++width) { + (*this)(plane, depth, height, width) = value(height, width); + } + } + } + } + } + + // Fills all of the {x,y} with the array provided, which specifies {p,z}. + void FillWithPZ(const Array2D& value) { + CHECK_EQ(value.height(), planes()); + CHECK_EQ(value.width(), depth()); + for (int64 height = 0; height < this->height(); ++height) { + for (int64 width = 0; width < this->width(); ++width) { + for (int64 plane = 0; plane < planes(); ++plane) { + for (int64 depth = 0; depth < this->depth(); ++depth) { + (*this)(plane, depth, height, width) = value(plane, depth); + } + } + } + } + } + + // Fills each of the minor-dim matrices with a number designating which minor + // dim matrix is enclosed by the shape. + void FillWithMinorDimNum() { + LOG(INFO) << "width: " << this->width(); + LOG(INFO) << "height: " << this->height(); + LOG(INFO) << "depth: " << this->depth(); + LOG(INFO) << "planes: " << this->planes(); + for (int64 height = 0; height < this->height(); ++height) { + for (int64 width = 0; width < this->width(); ++width) { + for (int64 plane = 0; plane < planes(); ++plane) { + for (int64 depth = 0; depth < this->depth(); ++depth) { + float this_val = plane * this->depth() + depth; + (*this)(plane, depth, height, width) = this_val; + } + } + } + } + } +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_ARRAY4D_H_ diff --git a/linux/include/tensorflow/compiler/xla/client/client.h b/linux/include/tensorflow/compiler/xla/client/client.h new file mode 100644 index 000000000..6f4d33c46 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/client/client.h @@ -0,0 +1,210 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_CLIENT_CLIENT_H_ +#define TENSORFLOW_COMPILER_XLA_CLIENT_CLIENT_H_ + +#include +#include + +#include "absl/types/span.h" +#include "tensorflow/compiler/xla/client/global_data.h" +#include "tensorflow/compiler/xla/client/xla_computation.h" +#include "tensorflow/compiler/xla/literal.h" +#include "tensorflow/compiler/xla/service/hlo.pb.h" +#include "tensorflow/compiler/xla/service_interface.h" +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/compiler/xla/xla.pb.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/platform/macros.h" + +namespace xla { + +// XLA service's client object -- wraps the service with convenience and +// lifetime-oriented methods. +class Client { + public: + explicit Client(ServiceInterface* stub); + virtual ~Client(); + + // Executes the computation with the given arguments and returns the global + // data that was produced from the execution. + // * If execution_options is not nullptr, these options are passed to the + // service to affect how it compiles our computation. (The pointer does not + // need to live beyond this call.) + // * If execution_options.device_handles is not empty, the computation is + // executed on the devices associated with the handles by partitioning the + // computation based on the attached sharding attributes. Otherwise, a + // device is chosen by the service. + // * If execution_profile is not nullptr then the pointed-to ExecutionProfile + // will be filled with profile data from the execution. + StatusOr> Execute( + const XlaComputation& computation, + absl::Span arguments, + const ExecutionOptions* execution_options = nullptr, + ExecutionProfile* execution_profile = nullptr); + + // A struct to represent a computation instance to be executed. + // * If execution_options.device_handles is not empty, the computation is + // executed on the devices associated with the handles by partitioning the + // computation based on the attached sharding attributes. Otherwise, a + // device is chosen by the service. + struct XlaComputationInstance { + const XlaComputation& computation; + std::vector arguments; + ExecutionOptions execution_options; + ExecutionProfile* execution_profile; + + XlaComputationInstance(const XlaComputation& computation, + std::vector arguments, + ExecutionOptions execution_options, + ExecutionProfile* execution_profile) + : computation(computation), + arguments(std::move(arguments)), + execution_options(execution_options), + execution_profile(execution_profile) {} + }; + + // Executes a list XlaComputationInstances and returns global data produced + // from each computation. + // + StatusOr>> ExecuteParallel( + absl::Span computations); + + // Requests device_count device handles available on the target. The returned + // device handles are used to specify the devices to execute the computations + // (see ExecuteParallel) or to transfer data (see TransferToServer or + // TransferToInfeed). + StatusOr> GetDeviceHandles(int64 device_count); + + // Transfer the global data provided to this client process, which is + // returned in the provided literal. Use sparingly to avoid transfer + // overheads. + // + // If shape_with_layout is not nullptr, it points to a shape whose layout will + // be the layout of the returned literal. + StatusOr Transfer(const GlobalData& data, + const Shape* shape_with_layout = nullptr); + + // Transfer the given literal to the server. This allocates memory on the + // device and copies the literal's contents over. Returns a global data handle + // that can be used to refer to this value from the client. + // + // If device_handle is not nullptr, data is transferred to the associated + // device (and its replicas if replication is enabled). Otherwise, data is + // transferred to the default device (and its replicas). + StatusOr> TransferToServer( + const LiteralSlice& literal, const DeviceHandle* device_handle = nullptr); + + // Transfer the given literal to the Infeed interface of the device. + // + // device_handle and replica_id together specify a particular device; a device + // assigned for the given replica_id among the replicas that the given device + // handle belongs to. + Status TransferToInfeed(const LiteralSlice& literal, int64 replica_id = 0, + const DeviceHandle* device_handle = nullptr); + + // Transfers from the Outfeed of the device. + // + // device_handle and replica_id together specify a particular device; a device + // assigned for the given replica_id among the replicas that the given device + // handle belongs to. + StatusOr TransferFromOutfeed( + const Shape* shape_with_layout, int64 replica_id = 0, + const DeviceHandle* device_handle = nullptr); + + // Resets the device, clearing all existing state on the device. + Status ResetDevice(); + + // Executes the computation with the given arguments and transfers the result + // to the client as a literal. Parameters are defined the same as for + // Execute() and Transfer(). + StatusOr ExecuteAndTransfer( + const XlaComputation& computation, + absl::Span arguments, + const ExecutionOptions* execution_options = nullptr, + ExecutionProfile* execution_profile = nullptr); + + // Computes the value of the given computation using a non-optimized + // interpreter on the host. + // + // The computation must not depend on any parameters, or on stateful operators + // such as `RngNormal` or `Infeed`. + // + // This functionality can be useful when translating a computation into XLA + // where something that looked dynamic is required by XLA to be specified as a + // constant. E.g. the source computation (outside of XLA) may include a + // dynamic computation of the shape of something and ComputeConstant lets you + // determine what the value of that computation is in the case where the value + // can be determined at compile time. + // + // If output_layout is non-null, then the output of the computation will be + // stored using that layout. + StatusOr ComputeConstant( + const XlaComputation& computation, + const Layout* output_layout = nullptr) const; + + // Unregister the memory for the given GlobalData on the device. + Status Unregister(const GlobalData& data); + + // Returns a vector of global data handles that point to the tuple elements. + StatusOr>> DeconstructTuple( + const GlobalData& data); + + // Retrieves the statistics of the given computation. + StatusOr GetComputationStats( + const XlaComputation& computation, + const DebugOptions& debug_options) const; + + // Returns the Shape of the given array specified by 'data'. The shape + // includes the Layout of the array as it is stored on the service. + StatusOr GetShape(const GlobalData& data); + + // As above, but returns the shape of the provided computation (parameter + // types/names and return type). + StatusOr> GetComputationShape( + const XlaComputation& computation); + + // Creates a channel handle that can be used to transfer data between two + // computations on different devices via a pair of Send and Recv instructions. + StatusOr CreateChannelHandle(); + + // Create a channel for communicating with the host via a SendtoHost or + // RecvFromHost operation. + StatusOr CreateHostToDeviceChannelHandle(); + StatusOr CreateDeviceToHostChannelHandle(); + + StatusOr LoadSnapshot(const HloSnapshot& module); + + ServiceInterface* stub() { return stub_; } + + private: + // Returns the execution statistics (e.g., gflop/s) as a string from the + // ExecutionProfile returned from an execution of the computation. + StatusOr ExecutionStatsAsString(const XlaComputation& computation, + const ExecutionProfile& profile); + + StatusOr CreateChannelHandleByType( + ChannelHandle::ChannelType type); + + ServiceInterface* stub_; // Stub that this client is connected on. + + TF_DISALLOW_COPY_AND_ASSIGN(Client); +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_CLIENT_CLIENT_H_ diff --git a/linux/include/tensorflow/compiler/xla/client/client_library.h b/linux/include/tensorflow/compiler/xla/client/client_library.h new file mode 100644 index 000000000..3ad558fa5 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/client/client_library.h @@ -0,0 +1,134 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// The "client library" instantiates a local (in-process) XLA service for +// use by this process, and connects to it with a singleton XLA local +// client. ClientLibrary::GetOrCreateLocalClient will spawn a local service, +// and return a client that's connected to it and ready to run XLA +// computations. +#ifndef TENSORFLOW_COMPILER_XLA_CLIENT_CLIENT_LIBRARY_H_ +#define TENSORFLOW_COMPILER_XLA_CLIENT_CLIENT_LIBRARY_H_ + +#include +#include +#include +#include + +#include "tensorflow/compiler/xla/client/compile_only_client.h" +#include "tensorflow/compiler/xla/client/local_client.h" +#include "tensorflow/compiler/xla/service/compile_only_service.h" +#include "tensorflow/compiler/xla/service/device_memory_allocator.h" +#include "tensorflow/compiler/xla/service/local_service.h" +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/mutex.h" +#include "tensorflow/core/platform/stream_executor_no_cuda.h" +#include "tensorflow/core/platform/thread_annotations.h" + +namespace xla { + +// Options to configure the local client when it is created. +class LocalClientOptions { + public: + LocalClientOptions(se::Platform* platform = nullptr, + int number_of_replicas = 1, + int intra_op_parallelism_threads = -1); + + // Set the platform backing the service, or nullptr for the default platform. + LocalClientOptions& set_platform(se::Platform* platform); + se::Platform* platform() const; + + // Set the number of replicas to use when compiling replicated + // programs. + LocalClientOptions& set_number_of_replicas(int number_of_replicas); + int number_of_replicas() const; + + // Sets the thread pool size for parallel execution of an individual operator. + LocalClientOptions& set_intra_op_parallelism_threads(int num_threads); + int intra_op_parallelism_threads() const; + + private: + se::Platform* platform_; + int number_of_replicas_; + int intra_op_parallelism_threads_; +}; + +class ClientLibrary { + public: + // Singleton constructor-or-accessor -- returns a client for the application + // to issue XLA commands on. Arguments: + // + // platform : The platform the underlying XLA service should target. If + // null then default platform is used. + static StatusOr GetOrCreateLocalClient( + se::Platform* platform = nullptr); + static StatusOr GetOrCreateLocalClient( + const LocalClientOptions& options); + + // Convenience "or-die" wrapper around the above which returns the existing + // client library or creates one with default platform and allocator. + static LocalClient* LocalClientOrDie(); + + // Returns the service from the service thread. Only used in unit tests to + // access user computations from client. + static LocalService* GetXlaService(se::Platform* platform); + + // Singleton constructor-or-accessor for compile-only clients. Arguments: + // + // platform : The platform the underlying XLA service should target. If + // null then default platform is used. + static StatusOr GetOrCreateCompileOnlyClient( + se::Platform* platform = nullptr); + + // Clears the local instance and compile only instance caches. The client + // pointers returned by the previous GetOrCreateLocalClient() or + // GetOrCreateCompileOnlyClient() invocations are not valid anymore. + static void DestroyLocalInstances(); + + private: + // Returns the singleton instance of ClientLibrary. + static ClientLibrary& Singleton(); + + ClientLibrary(); + ~ClientLibrary(); + + struct LocalInstance { + // Service that is wrapped by the singleton client object. + std::unique_ptr service; + // Singleton client object. + std::unique_ptr client; + }; + + struct CompileOnlyInstance { + // Service that is wrapped by the singleton client object. + std::unique_ptr service; + // Singleton client object. + std::unique_ptr client; + }; + + tensorflow::mutex service_mutex_; // Guards the singleton creation state. + std::unordered_map> + local_instances_ GUARDED_BY(service_mutex_); + + std::unordered_map> + compile_only_instances_ GUARDED_BY(service_mutex_); + + TF_DISALLOW_COPY_AND_ASSIGN(ClientLibrary); +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_CLIENT_CLIENT_LIBRARY_H_ diff --git a/linux/include/tensorflow/compiler/xla/client/compile_only_client.h b/linux/include/tensorflow/compiler/xla/client/compile_only_client.h new file mode 100644 index 000000000..9e3ed2373 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/client/compile_only_client.h @@ -0,0 +1,68 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_CLIENT_COMPILE_ONLY_CLIENT_H_ +#define TENSORFLOW_COMPILER_XLA_CLIENT_COMPILE_ONLY_CLIENT_H_ + +#include "tensorflow/compiler/xla/client/client.h" +#include "tensorflow/compiler/xla/client/xla_computation.h" +#include "tensorflow/compiler/xla/service/compile_only_service.h" +#include "tensorflow/compiler/xla/service/compiler.h" +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/platform/stream_executor_no_cuda.h" + +namespace xla { + +// An XLA Client specialization for doing ahead-of-time compilation. This does +// not require (or attempt to instantiate) an execution-capable backend for the +// relevant platform. +class CompileOnlyClient : public Client { + public: + explicit CompileOnlyClient(CompileOnlyService* service) + : Client(service), compiler_service_(service) {} + + CompileOnlyClient(const CompileOnlyClient&) = delete; + void operator=(const CompileOnlyClient&) = delete; + + // A description of an xla computation to compile using CompileAheadOfTime. + struct AotXlaComputationInstance { + const XlaComputation* computation; + // Inform the compiler of the expected layout for arguments. + std::vector argument_layouts; + // Specifies the expected result layout. + const Shape* result_layout; + }; + + // Compiles a list of xla computations for ahead-of-time execution. + // This is intended for use in static compilation. The |options| + // parameter describes the target for which the compiler should emit + // code. |metadata|, if provided, is populated during compilation. + StatusOr>> + CompileAheadOfTime( + const absl::Span computations, + const AotCompilationOptions& options, + std::unique_ptr* metadata = nullptr); + + // Returns the size of a pointer in bytes for a given triple. + static int64 PointerSizeForTriple(absl::string_view triple); + + private: + CompileOnlyService* compiler_service_; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_CLIENT_COMPILE_ONLY_CLIENT_H_ diff --git a/linux/include/tensorflow/compiler/xla/client/executable_build_options.h b/linux/include/tensorflow/compiler/xla/client/executable_build_options.h new file mode 100644 index 000000000..93334db88 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/client/executable_build_options.h @@ -0,0 +1,112 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_CLIENT_EXECUTABLE_BUILD_OPTIONS_H_ +#define TENSORFLOW_COMPILER_XLA_CLIENT_EXECUTABLE_BUILD_OPTIONS_H_ + +#include "absl/strings/string_view.h" +#include "absl/types/optional.h" +#include "tensorflow/compiler/xla/service/device_memory_allocator.h" +#include "tensorflow/compiler/xla/util.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" + +namespace xla { + +// Class containing options for building an LocalExecutable with +// LocalClient::Compile. +class ExecutableBuildOptions { + public: + // If set, this is the device to build the computation for. Valid + // device_ordinal values are: 0 to # of devices - 1. These values are + // identical to the device ordinal values used by StreamExecutor. The built + // executable will be executable on any device equivalent to the specified + // device as determined by Backend::devices_equivalent(). A value of -1 + // indicates this option has not been set. + ExecutableBuildOptions& set_device_ordinal(int device_ordinal); + int device_ordinal() const; + + // If set, this specifies the layout of the result of the computation. If not + // set, the service will chose the layout of the result. A Shape is used to + // store the layout to accommodate tuple result shapes. A value of nullptr + // indicates the option has not been set. + ExecutableBuildOptions& set_result_layout(const Shape& shape_with_layout); + const Shape* result_layout() const; + + // If set, this specifies an allocator that can be used to allocate temporary + // space on the device during compilation. For example, the compiler might + // want to run various algorithms on the device and pick the fastest one -- it + // might allocate buffers for use by these algorithms using this allocator. + // + // This does not need to be the same as the DeviceMemoryAllocator passed when + // running the executable. + ExecutableBuildOptions& set_device_allocator( + DeviceMemoryAllocator* allocator); + DeviceMemoryAllocator* device_allocator() const; + + // If set, specifies a regexp of HLO graphs to dump (as in DebugOptions). + ExecutableBuildOptions& set_generate_hlo_graph(string regex); + const absl::optional& generate_hlo_graph() const; + + // If set, specifies a dirpath to dump the end-of-optimization-pipeline HLO + // protobuf to (as in DebugOptions). + ExecutableBuildOptions& set_dump_optimized_hlo_proto_to( + absl::string_view dirpath); + const absl::optional& dump_optimized_hlo_proto_to() const; + + // If set, specifies a dirpath to dump the start-of-optimization-pipeline HLO + // protobuf to (as in DebugOptions). + ExecutableBuildOptions& set_dump_unoptimized_hlo_proto_to( + absl::string_view dirpath); + const absl::optional& dump_unoptimized_hlo_proto_to() const; + + // If set, specifies a dirpath to dump the per-pass-in-pipeline HLO protobufs + // to (as in DebugOptions). + ExecutableBuildOptions& set_dump_per_pass_hlo_proto_to( + absl::string_view dirpath); + const absl::optional& dump_per_pass_hlo_proto_to() const; + + // If true, specifies that we should record an HLO profile during execution + // and log it after execution (as in DebugOptions). If nullopt the default is + // used. + ExecutableBuildOptions& set_hlo_profile(bool enabled); + absl::optional hlo_profile() const; + + void add_disabled_hlo_pass(absl::string_view pass_name) { + disabled_hlo_passes_.push_back(std::string(pass_name)); + } + const absl::Span disabled_hlo_passes() const { + return disabled_hlo_passes_; + } + + // Returns a string representation of the build options, suitable for + // debugging. + string ToString() const; + + private: + absl::optional hlo_profile_; + int device_ordinal_ = -1; + Shape result_layout_; + bool result_layout_set_ = false; + absl::optional generate_hlo_graph_; + absl::optional dump_optimized_hlo_proto_to_; + absl::optional dump_unoptimized_hlo_proto_to_; + absl::optional dump_per_pass_hlo_proto_to_; + DeviceMemoryAllocator* device_allocator_ = nullptr; + std::vector disabled_hlo_passes_; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_CLIENT_EXECUTABLE_BUILD_OPTIONS_H_ diff --git a/linux/include/tensorflow/compiler/xla/client/global_data.h b/linux/include/tensorflow/compiler/xla/client/global_data.h new file mode 100644 index 000000000..b7929357d --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/client/global_data.h @@ -0,0 +1,48 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_CLIENT_GLOBAL_DATA_H_ +#define TENSORFLOW_COMPILER_XLA_CLIENT_GLOBAL_DATA_H_ + +#include "tensorflow/compiler/xla/service_interface.h" +#include "tensorflow/compiler/xla/xla.pb.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/platform/macros.h" + +namespace xla { + +// A GlobalData object represents a globally-accessible allocation of +// data in the associated XLA service. +class GlobalData { + public: + // Gives ownership of the global data handle to this object. + GlobalData(ServiceInterface* parent, GlobalDataHandle handle); + + // Unregisters the wrapped handle, which causes the service to + // deallocate the associated data. + ~GlobalData(); + + const GlobalDataHandle& handle() const { return handle_; } + + private: + GlobalDataHandle handle_; // Handle being wrapped. + ServiceInterface* parent_; // Service used to unregister handle_. + + TF_DISALLOW_COPY_AND_ASSIGN(GlobalData); +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_CLIENT_GLOBAL_DATA_H_ diff --git a/linux/include/tensorflow/compiler/xla/client/lib/arithmetic.h b/linux/include/tensorflow/compiler/xla/client/lib/arithmetic.h new file mode 100644 index 000000000..632e8cc8b --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/client/lib/arithmetic.h @@ -0,0 +1,62 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_CLIENT_LIB_ARITHMETIC_H_ +#define TENSORFLOW_COMPILER_XLA_CLIENT_LIB_ARITHMETIC_H_ + +#include + +#include "tensorflow/compiler/xla/client/xla_builder.h" +#include "tensorflow/compiler/xla/client/xla_computation.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" + +namespace xla { + +// Creates a scalar add computation and returns it. +XlaComputation CreateScalarAddComputation(PrimitiveType type, + XlaBuilder* builder); + +// Creates a scalar multiply computation and returns it. +XlaComputation CreateScalarMultiplyComputation(PrimitiveType type, + XlaBuilder* builder); + +// Creates a scalar ge computation and returns it. +XlaComputation CreateScalarGeComputation(PrimitiveType type, + XlaBuilder* builder); + +// Creates a scalar max computation and returns it. +XlaComputation CreateScalarMaxComputation(PrimitiveType type, + XlaBuilder* builder); + +// Creates a scalar min computation and returns it. +XlaComputation CreateScalarMinComputation(PrimitiveType type, + XlaBuilder* builder); + +// Creates a scalar logical AND computation and returns it. +XlaComputation CreateScalarAndComputation(PrimitiveType type, + XlaBuilder* builder); + +// Creates a scalar logical OR computation and returns it. +XlaComputation CreateScalarOrComputation(PrimitiveType type, + XlaBuilder* builder); + +// Returns whether any predicate in "predicates" is set. +// +// Note: if predicates is zero-sized, Any() vacuously returns false. +XlaOp Any(XlaOp predicates); + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_CLIENT_LIB_ARITHMETIC_H_ diff --git a/linux/include/tensorflow/compiler/xla/client/lib/constants.h b/linux/include/tensorflow/compiler/xla/client/lib/constants.h new file mode 100644 index 000000000..81624614c --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/client/lib/constants.h @@ -0,0 +1,124 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_CLIENT_LIB_CONSTANTS_H_ +#define TENSORFLOW_COMPILER_XLA_CLIENT_LIB_CONSTANTS_H_ + +#include + +#include "tensorflow/compiler/xla/client/xla_builder.h" +#include "tensorflow/compiler/xla/primitive_util.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" + +namespace xla { + +// Returns scalar 'value' as a scalar of 'type'. Unlike ConstantR0, 'type' is +// determined at C++ run-time, rather than C++ compile-time. +// If 'value' is floating point but 'type' is not, or if 'value' is complex but +// 'type' is not, an error will be returned. This is to catch accidental +// truncation; in such cases, use an explicit cast. +template +XlaOp ConstantR0WithType(XlaBuilder* builder, PrimitiveType type, T value) { + if (std::is_floating_point::value && + !(primitive_util::IsFloatingPointType(type) || + primitive_util::IsComplexType(type))) { + return builder->ReportError(InvalidArgument( + "Invalid cast from floating point type to %s in ConstantR0WithType.", + PrimitiveType_Name(type))); + } + if (std::is_same::value && + !primitive_util::IsComplexType(type)) { + return builder->ReportError(InvalidArgument( + "Invalid cast from complex type to %s in ConstantR0WithType.", + PrimitiveType_Name(type))); + } + switch (type) { + case F16: + return ConstantR0(builder, static_cast(value)); + case BF16: + return ConstantR0(builder, static_cast(value)); + case F32: + return ConstantR0(builder, static_cast(value)); + case F64: + return ConstantR0(builder, static_cast(value)); + case C64: + return ConstantR0(builder, static_cast(value)); + case U8: + return ConstantR0(builder, static_cast(value)); + case U32: + return ConstantR0(builder, static_cast(value)); + case U64: + return ConstantR0(builder, static_cast(value)); + case S8: + return ConstantR0(builder, static_cast(value)); + case S32: + return ConstantR0(builder, static_cast(value)); + case S64: + return ConstantR0(builder, static_cast(value)); + default: + return builder->ReportError( + InvalidArgument("Invalid type for ConstantR0WithType (%s).", + PrimitiveType_Name(type))); + } +} + +// Returns a scalar containing 'value' cast to the same run-time type as +// 'prototype'. +// If 'value' is floating point but 'prototype' is not, or if 'value' is complex +// 'prototype' is not, an error will be returned. +template +XlaOp ScalarLike(XlaOp prototype, T value) { + XlaBuilder* builder = prototype.builder(); + return builder->ReportErrorOrReturn([&]() -> StatusOr { + TF_ASSIGN_OR_RETURN(Shape shape, builder->GetShape(prototype)); + return ConstantR0WithType(builder, shape.element_type(), value); + }); +} + +// Returns a scalar with value '0' of 'type'. +XlaOp Zero(XlaBuilder* builder, PrimitiveType type); + +// Returns a zero-filled tensor with shape `shape`. +XlaOp Zeros(XlaBuilder* builder, const Shape& shape); + +// Returns a zero-filled tensor with the same shape as `prototype`. +XlaOp ZerosLike(XlaOp prototype); + +// Returns a scalar with value '1' of 'type'. +XlaOp One(XlaBuilder* builder, PrimitiveType type); + +// Returns the machine epsilon for floating-point type `type`, i.e., +// the difference between 1.0 and the next representable value. +XlaOp Epsilon(XlaBuilder* builder, PrimitiveType type); + +// Returns the minimum representable finite or infinite value for 'type'. +// Returns '-inf' for floating-point types. +XlaOp MinValue(XlaBuilder* builder, PrimitiveType type); + +// Returns the minimum representable finite value for 'type'. For a floating +// point type, this is equal to -MaxFiniteValue(). +XlaOp MinFiniteValue(XlaBuilder* builder, PrimitiveType type); + +// Returns the maximum representable finite or infinite value for 'type'. +// Returns 'inf' for floating-point types. +XlaOp MaxValue(XlaBuilder* builder, PrimitiveType type); + +// Returns the maximum representable finite value for 'type'. +XlaOp MaxFiniteValue(XlaBuilder* builder, PrimitiveType type); + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_CLIENT_LIB_CONSTANTS_H_ diff --git a/linux/include/tensorflow/compiler/xla/client/lib/conv_grad_size_util.h b/linux/include/tensorflow/compiler/xla/client/lib/conv_grad_size_util.h new file mode 100644 index 000000000..0ad01728e --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/client/lib/conv_grad_size_util.h @@ -0,0 +1,44 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_CLIENT_LIB_CONV_GRAD_SIZE_UTIL_H_ +#define TENSORFLOW_COMPILER_XLA_CLIENT_LIB_CONV_GRAD_SIZE_UTIL_H_ + +#include "tensorflow/compiler/xla/client/padding.h" +#include "tensorflow/core/platform/types.h" + +namespace xla { + +// Information about a single spatial dimension for a convolution gradients and +// windowed operations. +struct SpatialDimensionOutputSizeAndPadding { + // Effective size of the operation output (potentially expanded). + int64 output_size; + // Number of padding elements to be added before/after this dimension of + // the input when computing the input gradient. + int64 pad_before; + int64 pad_after; +}; + +// Verifies that the dimensions all match, and computes the size and padding of +// a spatial dimension for convolution gradient operations. +StatusOr +ConvGradExtractAndVerifyDimension(int64 input_size, int64 filter_size, + int64 output_size, int64 dilation, + int64 stride, Padding padding); + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_CLIENT_LIB_CONV_GRAD_SIZE_UTIL_H_ diff --git a/linux/include/tensorflow/compiler/xla/client/lib/math.h b/linux/include/tensorflow/compiler/xla/client/lib/math.h new file mode 100644 index 000000000..a6cafd420 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/client/lib/math.h @@ -0,0 +1,87 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_CLIENT_LIB_MATH_H_ +#define TENSORFLOW_COMPILER_XLA_CLIENT_LIB_MATH_H_ + +#include "tensorflow/compiler/xla/client/xla_builder.h" + +namespace xla { + +// Computes the square root of 'operand'. +XlaOp Sqrt(XlaOp operand); + +// Computes the reciprocal of the square root of 'operand'. +XlaOp Rsqrt(XlaOp operand); + +// Computes the square of 'operand'. +XlaOp Square(XlaOp operand); + +// Computes the reciprocal of 'operand'. +XlaOp Reciprocal(XlaOp operand); + +// Evaluates a polynomial given coefficients and `x`. +// N.B. Coefficients should be supplied in decreasing order. +XlaOp EvaluatePolynomial(XlaOp x, absl::Span coefficients); + +// Computes an approximation of the error function complement (1 - erf(x)). +XlaOp Erfc(XlaOp x); + +// Computes an approximation of the error function. +XlaOp Erf(XlaOp x); + +// Computes an approximation of the inverse of the error function. +XlaOp ErfInv(XlaOp x); + +// Computes an approximation of the lgamma function. +XlaOp Lgamma(XlaOp input); + +// Computes an approximation of the digamma function. +XlaOp Digamma(XlaOp input); + +// Trigonometric functions + +// Computes the arc cosine of 'x'. +XlaOp Acos(XlaOp x); + +// Computes the arc sine of 'x'. +XlaOp Asin(XlaOp x); + +// Computes the arc tangent of 'x'. +XlaOp Atan(XlaOp x); + +// Computes the tangent of 'x'. +XlaOp Tan(XlaOp x); + +// Hyperbolic trigonometric functions + +// Computes the inverse hyperbolic cosine of 'x'. +XlaOp Acosh(XlaOp x); + +// Computes the inverse hyperbolic sine of 'x'. +XlaOp Asinh(XlaOp x); + +// Computes the inverse hyperbolic tangent of 'x'. +XlaOp Atanh(XlaOp x); + +// Computes the hyperbolic cosine of 'x'. +XlaOp Cosh(XlaOp x); + +// Computes the hyperbolic sine of 'x'. +XlaOp Sinh(XlaOp x); + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_CLIENT_LIB_MATH_H_ diff --git a/linux/include/tensorflow/compiler/xla/client/lib/numeric.h b/linux/include/tensorflow/compiler/xla/client/lib/numeric.h new file mode 100644 index 000000000..efd8cdc25 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/client/lib/numeric.h @@ -0,0 +1,48 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_CLIENT_LIB_NUMERIC_H_ +#define TENSORFLOW_COMPILER_XLA_CLIENT_LIB_NUMERIC_H_ + +#include "tensorflow/compiler/xla/client/xla_builder.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" + +namespace xla { + +// Returns a rank 1 tensor of `type` containing values [0, 1, 2, ...]. +XlaOp Iota(XlaBuilder* builder, PrimitiveType type, int64 size); + +// Returns an m x n matrix with 1s on the diagonal elements, zeros everywhere +// else. +XlaOp IdentityMatrix(XlaBuilder* builder, PrimitiveType type, int64 m, int64 n); + +// Get the diagonals of the last two dimensions. If 'x' has shape +// [..., M, N], then the output has shape [..., min(M, N)], containing the +// diagonal elements (i.e., with indices [..., i, i]). +XlaOp GetMatrixDiagonal(XlaOp x); + +// Get the upper or lower triangle part of the last two dimensions +XlaOp Triangle(XlaOp x, bool lower); + +// Get the upper triangle part of the last two dimensions +XlaOp UpperTriangle(XlaOp x); + +// Get the lower triangle part of the last two dimensions +XlaOp LowerTriangle(XlaOp x); + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_CLIENT_LIB_NUMERIC_H_ diff --git a/linux/include/tensorflow/compiler/xla/client/lib/pooling.h b/linux/include/tensorflow/compiler/xla/client/lib/pooling.h new file mode 100644 index 000000000..5c0054857 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/client/lib/pooling.h @@ -0,0 +1,80 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_CLIENT_LIB_POOLING_H_ +#define TENSORFLOW_COMPILER_XLA_CLIENT_LIB_POOLING_H_ + +#include "absl/container/inlined_vector.h" +#include "tensorflow/compiler/xla/client/xla_builder.h" + +namespace xla { + +// Tensor format for reduce window operations. +class TensorFormat { + public: + TensorFormat(int batch_dimension, int feature_dimension, + absl::Span spatial_dimensions) + : batch_dimension_(batch_dimension), + feature_dimension_(feature_dimension), + spatial_dimensions_(spatial_dimensions.begin(), + spatial_dimensions.end()) {} + + int batch_dimension() const { return batch_dimension_; } + + int feature_dimension() const { return feature_dimension_; } + + int spatial_dimension(int dim) const { return spatial_dimensions_[dim]; } + + int num_spatial_dims() const { return spatial_dimensions_.size(); } + + private: + // The number of the dimension that represents the batch. + int batch_dimension_; + // The number of the dimension that represents the features. + int feature_dimension_; + // The dimension numbers for the spatial dimensions. + absl::InlinedVector spatial_dimensions_; +}; + +// Computes the max pool of 'operand'. +XlaOp MaxPool(XlaOp operand, absl::Span kernel_size, + absl::Span stride, Padding padding, + const TensorFormat& data_format); + +// Computes the average pool of 'operand'. +XlaOp AvgPool(XlaOp operand, absl::Span kernel_size, + absl::Span stride, + absl::Span> padding, + const TensorFormat& data_format, + const bool counts_include_padding); + +// Returns the list of low and high padding elements in each spatial dimension +// for the given 'padding' specification. +std::vector> MakeSpatialPadding( + absl::Span input_size, absl::Span kernel_size, + absl::Span stride, Padding padding, + const TensorFormat& data_format); + +// Computes the average pool gradient. +XlaOp AvgPoolGrad(XlaOp out_backprop, absl::Span gradients_size, + absl::Span kernel_size, + absl::Span stride, + absl::Span> spatial_padding, + const TensorFormat& data_format, + const bool counts_include_padding); + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_CLIENT_LIB_POOLING_H_ diff --git a/linux/include/tensorflow/compiler/xla/client/lib/prng.h b/linux/include/tensorflow/compiler/xla/client/lib/prng.h new file mode 100644 index 000000000..ad000b1fa --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/client/lib/prng.h @@ -0,0 +1,34 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_CLIENT_LIB_PRNG_H_ +#define TENSORFLOW_COMPILER_XLA_CLIENT_LIB_PRNG_H_ + +#include + +#include "tensorflow/compiler/xla/client/xla_builder.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" + +namespace xla { + +// Returns a tensor containing 'shape' random values uniformly distributed in +// the range [minval, maxval). Requires 2 32-bit integer seeds. +// Currently only 'shape's of type F32 are implemented. +XlaOp StatelessRngUniform(std::array seeds, const Shape& shape, + XlaOp minval, XlaOp maxval); + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_CLIENT_LIB_PRNG_H_ diff --git a/linux/include/tensorflow/compiler/xla/client/lib/sorting.h b/linux/include/tensorflow/compiler/xla/client/lib/sorting.h new file mode 100644 index 000000000..b9dfafdd6 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/client/lib/sorting.h @@ -0,0 +1,31 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_CLIENT_LIB_SORTING_H_ +#define TENSORFLOW_COMPILER_XLA_CLIENT_LIB_SORTING_H_ + +#include "tensorflow/compiler/xla/client/xla_builder.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" + +namespace xla { + +// Returns a tuple composed of the top `k` values and corresponding indices in +// `input`. Output values are in descending order, from largest to smallest. +XlaOp TopK(XlaOp input, int64 k); + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_CLIENT_LIB_SORTING_H_ diff --git a/linux/include/tensorflow/compiler/xla/client/local_client.h b/linux/include/tensorflow/compiler/xla/client/local_client.h new file mode 100644 index 000000000..feb2f8ec9 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/client/local_client.h @@ -0,0 +1,190 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_CLIENT_LOCAL_CLIENT_H_ +#define TENSORFLOW_COMPILER_XLA_CLIENT_LOCAL_CLIENT_H_ + +#include + +#include "absl/types/span.h" +#include "tensorflow/compiler/xla/client/client.h" +#include "tensorflow/compiler/xla/client/executable_build_options.h" +#include "tensorflow/compiler/xla/client/xla_computation.h" +#include "tensorflow/compiler/xla/executable_run_options.h" +#include "tensorflow/compiler/xla/service/compiler.h" +#include "tensorflow/compiler/xla/service/device_memory_allocator.h" +#include "tensorflow/compiler/xla/service/executable.h" +#include "tensorflow/compiler/xla/service/hlo.pb.h" +#include "tensorflow/compiler/xla/service/local_service.h" +#include "tensorflow/compiler/xla/service/shaped_buffer.h" +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/platform/stream_executor_no_cuda.h" + +namespace xla { + +class LocalExecutable { + public: + // Run the compiled computation with the given arguments and options and + // return the result. + StatusOr Run( + const absl::Span arguments, + ExecutableRunOptions run_options); + + // Return the options used to build the executable. + const ExecutableBuildOptions& build_options() const { return build_options_; } + + // Return the built executable. + Executable* executable() const { return executable_.get(); } + + private: + // Only a local client can construct these objects. + friend class LocalClient; + + // Constructor invoked by LocalClient. + LocalExecutable(std::unique_ptr executable, Backend* backend, + ExecutableBuildOptions build_options); + + // Validates that the given arguments and options satisfy various constraints + // of the computation. + // + // The given ExecutableRunOptions override any values from legacy_flags + // (TF_XLA_FLAGS environment variable). + Status ValidateExecutionOptions( + const absl::Span arguments, + const ExecutableRunOptions& run_options, const Backend& backend); + + // Records the computation in a SessionModule proto with the arguments used to + // invoke it, and the result. Enabled by flag: --tla_dump_executions_to. + // + // The given ServiceExecutableRunOptions override any values from legacy_flags + // (TF_XLA_FLAGS environment variable). + StatusOr ExecuteAndDump( + const ServiceExecutableRunOptions* run_options, + const absl::Span arguments); + + // Records the arguments used to invoke the computation in a SessionModule + // proto. + Status RecordArguments(const absl::Span arguments, + HloSnapshot* hlo_snapshot); + + // Records the result of the computation in a SessionModule proto. + Status RecordResult(const ShapedBuffer* result, HloSnapshot* hlo_snapshot); + + // Returns a literal containing the contents of the given ShapedBuffer. + StatusOr LiteralFromShapedBuffer(const ShapedBuffer& shaped_buffer); + + // The ordinal of the device which this executable was compiled for. The + // executable can run on all equivalent devices (as determined by + // Backend::devices_equivalent). + int build_device_ordinal() const { return build_options_.device_ordinal(); } + + // Compiled computation. + std::unique_ptr executable_; + + // Execution backend. + Backend* backend_ = nullptr; + + // Options used to build the executable. + const ExecutableBuildOptions build_options_; +}; + +// An XLA Client specialization for use when the client and service run in +// the same process. +class LocalClient : public Client { + public: + explicit LocalClient(LocalService* service) + : Client(service), local_service_(service) {} + + LocalClient(const LocalClient&) = delete; + void operator=(const LocalClient&) = delete; + + // Build and return a LocalExecutable object. The executable is compiled using + // the given XlaComputation, argument layouts and options. + // + // The given ExecutableBuildOptions override any values from legacy_flags + // (TF_XLA_FLAGS environment variable). + StatusOr> Compile( + const XlaComputation& computation, + const absl::Span argument_layouts, + const ExecutableBuildOptions& options); + + // Copy the literal data to the device with the given ordinal and return as a + // ScopedShapedBuffer. If non-null the given memory allocator is used for + // device memory allocation. If null, the default memory allocator for the + // device is used. + StatusOr LiteralToShapedBuffer( + const Literal& literal, int device_ordinal, + DeviceMemoryAllocator* allocator = nullptr); + + // Copy the data from the device contained in the given ShapedBuffer and + // return as a Literal. + StatusOr ShapedBufferToLiteral(const ShapedBuffer& shaped_buffer); + + // Converts a GlobalDataHandle into a pointer to a ShapedBuffer that's valid + // as long as the handle is valid. + StatusOr GlobalDataToShapedBuffer( + const GlobalDataHandle& data, int replica_number); + + // Transfer the given literal to the infeed queue of the given device. + // TODO(b/69670845): Remove the 'Local' from the name when LocalClient does + // not inherit from Client and there is no possibility of confusion with + // Client::TransferToInfeed. + Status TransferToInfeedLocal(const Literal& literal, int device_ordinal); + + // Transfer and return a value of the given shape from the outfeed of the + // given device. + // TODO(b/69670845): Remove the 'Local' from the name when LocalClient does + // not inherit from Client and there is no possibility of confusion with + // Client::TransferFromOutfeed. + StatusOr TransferFromOutfeedLocal(const Shape& shape, + int device_ordinal); + + // Returns the device ordinal that corresponds to the given replica number. + // + // This returns an error if there is not a one-to-one correspondence of + // replicas to device ordinals, but is useful as a short term mechanism for + // the "easy" case where a single replica is a single device. + StatusOr ReplicaNumberToDeviceOrdinal(int replica_number); + + // Returns the platform that the underlying service targets. + se::Platform* platform() const; + + // Returns the number of devices on the system of the service platform + // type. Not all devices may be supported by the service (see + // device_ordinal_supported method). + int device_count() const; + + // Returns the default device ordinal that the service will run computations + // on if no device ordinal is specified in execute options. + int default_device_ordinal() const; + + // Returns whether the device with the given ordinal can be used by the + // service to execute computations. Not all devices of a particular platform + // may be usable by the service (eg, a GPU with insufficient CUDA compute + // capability). + bool device_ordinal_supported(int device_ordinal) const; + + // Returns the backend used to execute computations. + const Backend& backend() const; + Backend* mutable_backend(); + + private: + LocalService* local_service_; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_CLIENT_LOCAL_CLIENT_H_ diff --git a/linux/include/tensorflow/compiler/xla/client/padding.h b/linux/include/tensorflow/compiler/xla/client/padding.h new file mode 100644 index 000000000..5c009bd49 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/client/padding.h @@ -0,0 +1,66 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_CLIENT_PADDING_H_ +#define TENSORFLOW_COMPILER_XLA_CLIENT_PADDING_H_ + +#include +#include + +#include "absl/types/span.h" +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/compiler/xla/types.h" + +namespace xla { + +// Describes the padding applied for a windowed operation like +// convolution, where a window is placed inside a base area. +enum class Padding { + // Make the output have the same dimensions as the base area. For + // example, for a 3x3 base area and a 2x2 window, the output will be + // 3x3, so that requires padding the 3x3 base area to 4x4. + kSame, + + // Use no padding. For example, for a 4x4 base area and a 2x2 + // window, the output will be 3x3. + kValid, +}; + +// Validates that the slices are acceptable for determining padding -- this can +// be used to check the preconditions of MakePadding below to produce an error +// message that can be returned to the user. +Status ValidatePaddingValues(absl::Span input_dimensions, + absl::Span window_dimensions, + absl::Span window_strides); + +// Returns the padding needed for the base area, given the base area dimensions, +// window dimensions, strides, and the type of padding. +// +// If v is the returned vector, then for each dimension number i, +// v[i].first is the padding to the left (i.e. in the direction of +// lower indices) and v[i].second is the padding to the right (i.e. in +// the direction of higher indices). +// +// Precondition: The number of dimensions (i.e., rank) in input_dimensions, +// window_dimensions, and strides must match, which is equal to the number +// of elements in the result vector. +std::vector> MakePadding( + absl::Span input_dimensions, + absl::Span window_dimensions, + absl::Span window_strides, Padding padding); + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_CLIENT_PADDING_H_ diff --git a/linux/include/tensorflow/compiler/xla/client/sharding_builder.h b/linux/include/tensorflow/compiler/xla/client/sharding_builder.h new file mode 100644 index 000000000..59df3a876 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/client/sharding_builder.h @@ -0,0 +1,59 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_CLIENT_SHARDING_BUILDER_H_ +#define TENSORFLOW_COMPILER_XLA_CLIENT_SHARDING_BUILDER_H_ + +#include + +#include "tensorflow/compiler/xla/array.h" +#include "tensorflow/compiler/xla/shape_tree.h" +#include "tensorflow/compiler/xla/shape_util.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/compiler/xla/util.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" + +namespace xla { +namespace sharding_builder { +// A shaped array used to describe the assignment of tiles to devices. +using TileAssignment = Array; + +// Creates a replicated sharding - replicate a tensor on every device. +OpSharding Replicate(); + +// Creates a sharding that assigns a tensor to just one device. +OpSharding AssignDevice(int device); + +// Creates a tiled sharding with the given tile shape and assignment of tiles +// to devices. +// +// If tile_shape is not evenly divisible by the number of devices in +// tile_assignment, operations behave as if implicit padding had been inserted. +// The value of this padding is undefined. +OpSharding Tile(const Shape& tile_shape, const TileAssignment& tile_assignment); + +// Creates a sharding in one dimension, with the given tile shape which must +// be rank 1 and using devices [0..num_tiles). +// +// This is simply a convenience wrapper for Tile(). +OpSharding Tile1D(const Shape& tile_shape, int64 num_tiles); + +// Creates a tuple sharding from the given ShapeTree of element shardings. +OpSharding Tuple(const ShapeTree& shardings); + +} // namespace sharding_builder +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_CLIENT_SHARDING_BUILDER_H_ diff --git a/linux/include/tensorflow/compiler/xla/client/xla_builder.h b/linux/include/tensorflow/compiler/xla/client/xla_builder.h new file mode 100644 index 000000000..cd0d5ca5d --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/client/xla_builder.h @@ -0,0 +1,2289 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_CLIENT_XLA_BUILDER_H_ +#define TENSORFLOW_COMPILER_XLA_CLIENT_XLA_BUILDER_H_ + +#include +#include +#include +#include + +#include "absl/container/flat_hash_map.h" +#include "absl/container/flat_hash_set.h" +#include "absl/strings/string_view.h" +#include "absl/types/span.h" +#include "tensorflow/compiler/xla/client/padding.h" +#include "tensorflow/compiler/xla/client/xla_computation.h" +#include "tensorflow/compiler/xla/literal.h" +#include "tensorflow/compiler/xla/literal_util.h" +#include "tensorflow/compiler/xla/service/hlo.pb.h" +#include "tensorflow/compiler/xla/service/hlo_opcode.h" +#include "tensorflow/compiler/xla/shape_util.h" +#include "tensorflow/compiler/xla/status_macros.h" +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/stacktrace.h" +#include "tensorflow/core/platform/types.h" + +namespace xla { + +class XlaBuilder; + +// This represents an instruction that has been enqueued using the XlaBuilder. +// This is used to pass to subsequent computations that depends upon the +// instruction as an operand. +class XlaOp { + public: + XlaOp() : handle_(-1), builder_(nullptr) { + static_assert(std::is_trivially_destructible::value, + "XlaOp should be trivially destructible"); + } + ~XlaOp() = default; + + // Precondition: !IsUninitialized(). + // + // It's very common to do foo.builder()->bar(). Without this precondition, if + // foo.builder() is null, the call to bar will segfault at some point possibly + // deep in the callstack when we finally dereference `this`. The precondition + // lets us avoid this tricky-to-debug problem. + XlaBuilder* builder() const { + CHECK(builder_ != nullptr); + return builder_; + } + + // Returns true if the XlaOp represents valid, non-erroneous value. + bool valid() const { return handle_ >= 0; } + + // Returns true if the XlaOp was created by the XlaOp() constructor and + // not returned by a builder. + bool IsUninitialized() const { return builder_ == nullptr; } + + bool IsIdenticalTo(const XlaOp& rhs) const { + return handle_ == rhs.handle_ && builder_ == rhs.builder_; + } + + friend std::ostream& operator<<(std::ostream& out, const XlaOp& op) { + out << op.handle(); + return out; + } + + private: + explicit XlaOp(XlaBuilder* builder) : handle_(-1), builder_(builder) {} + XlaOp(int64 handle, XlaBuilder* builder) + : handle_(handle), builder_(builder) {} + + int64 handle() const { return handle_; } + + friend class XlaBuilder; + + // < 0 means "invalid handle". + int64 handle_; + + // Not owned. Non-null for any handle returned by XlaBuilder, even if the + // handle is invalid. + XlaBuilder* builder_; +}; + +// Arithmetic operator overloads for the XlaOp type. +XlaOp operator-(const XlaOp& x); +XlaOp operator+(const XlaOp& x, const XlaOp& y); +XlaOp operator-(const XlaOp& x, const XlaOp& y); +XlaOp operator*(const XlaOp& x, const XlaOp& y); +XlaOp operator/(const XlaOp& x, const XlaOp& y); +XlaOp operator%(const XlaOp& x, const XlaOp& y); + +// Bitwise operator overloads for the XlaOp type. +XlaOp operator~(const XlaOp& x); +XlaOp operator&(const XlaOp& x, const XlaOp& y); +XlaOp operator|(const XlaOp& x, const XlaOp& y); +XlaOp operator^(const XlaOp& x, const XlaOp& y); +XlaOp operator<<(const XlaOp& x, const XlaOp& y); +// Performs a right arithmetic shift if 'x' is a signed type, otherwise performs +// a right logical shift. +XlaOp operator>>(const XlaOp& x, const XlaOp& y); + +// We don't overload the relational operators (==, !=, <, <=, >, >=) because the +// semantics might be surprising since their result types are usually 'bool'. +// Further programmers may expect == to be a structural equality. +// We also choose not to overload any of the mutating operators (e.g., +=, -=) +// because the semantics might be misleading — XLA computations are immutable. + +// A convenient interface for building up computations. +// +// Thread-compatible. +class XlaBuilder { + public: + // computation_name: name to use for the built computation. + XlaBuilder(const string& computation_name); + + XlaBuilder(const XlaBuilder&) = delete; + XlaBuilder& operator=(const XlaBuilder&) = delete; + + ~XlaBuilder(); + + // Returns the computation name. + const string& name() const { return name_; } + + // Sets OpMetadata that will be added to all instructions until cleared. + // + // OpMetadata is often applied to a series of XLA HLO instructions. As a + // result, OpMetadata is set on the Computation Builder. All subsequent + // instructions generated via this Computation Builder will have the same + // OpMetadata attached until a call to ClearOpMetadata. + void SetOpMetadata(const OpMetadata& metadata) { metadata_ = metadata; } + + // Clears the HloMetadata state. + void ClearOpMetadata() { metadata_.Clear(); } + + // Sets an OpSharding that will be attached to all instructions until cleared. + void SetSharding(const OpSharding& sharding) { sharding_ = sharding; } + + // Clears the sharding. Ops will be sharded according to the default placement + // policy. + void ClearSharding() { sharding_ = absl::nullopt; } + + // Returns the OpSharding that will be attached to all instructions. + const absl::optional& sharding() const { return sharding_; } + + // Sets the builder to a mode where it will die immediately when an error is + // encountered, rather than producing it in a deferred fashion when Build() is + // called (which is the default). + void set_die_immediately_on_error(bool enabled) { + die_immediately_on_error_ = enabled; + } + + // Default dimension numbers used for a 2D convolution. + static constexpr int64 kConvBatchDimension = 0; + static constexpr int64 kConvFeatureDimension = 1; + static constexpr int64 kConvFirstSpatialDimension = 2; + static constexpr int64 kConvSecondSpatialDimension = 3; + static constexpr int64 kConvKernelOutputDimension = 0; + static constexpr int64 kConvKernelInputDimension = 1; + static constexpr int64 kConvKernelFirstSpatialDimension = 2; + static constexpr int64 kConvKernelSecondSpatialDimension = 3; + + // Creates a default ConvolutionDimensionNumbers. For a 2D convolution, for + // the input operand {batch, feature, height, width} = {0, 1, 2, 3} and for + // the kernel operand + // {output_feature, input_feature, height, width} = {0, 1, 2, 3}. + static ConvolutionDimensionNumbers CreateDefaultConvDimensionNumbers( + int num_spatial_dims = 2); + + // Returns an error if the convolution dimension numbers have conflicts. + static Status Validate(const ConvolutionDimensionNumbers& dnum); + + // Returns a new XlaBuilder whose resultant Computation is used only by this + // XlaBuilder. The sub-XlaBuilder has the same die_immediately_on_error + // behavior as the parent. + std::unique_ptr CreateSubBuilder(const string& computation_name); + + // Builds the computation with the requested operations, or returns a non-ok + // status. Note that all ops that have been enqueued will be moved to the + // computation being returned. The root of the computation will be the last + // added operation. + StatusOr Build(); + + // Overload of Build which specifies a particular root instruction for the + // computation. + StatusOr Build(XlaOp root); + + // Builds the computation with the requested operations, or notes an error in + // the parent XlaBuilder and returns an empty computation if building failed. + // This function is intended to be used where the returned XlaComputation is + // only used by the parent XlaBuilder and hence further operation on the + // returned XlaComputation will simply be error'ed out if an error occurred + // while building this computation. If the built computation is to be used by + // a XlaBuilder other than the parent XlaBuilder then Build() should be used + // instead. + XlaComputation BuildAndNoteError(); + + // Returns a subgraph that roots on the given root. If the root is not a + // compile-time constant (see `IsConstant`), returns an error. + // + // This will copy the needed ops/computations to the subgraph. + StatusOr BuildConstantSubGraph(const XlaOp& root_op) const; + + // Returns the first error that was encountered while building the + // computation. When an error is encountered, by default we return a vacuous + // XlaOp and inform the user of the error that occurred while + // building the computation when they make a final call to Build(). + // + // See also set_die_immediately_on_error(). + Status first_error() const { return first_error_; } + + // Returns the shape of the given op. + StatusOr GetShape(const XlaOp& op) const; + + // Returns the (inferred) result for the current computation's shape. This + // assumes the root instruction is the last added instruction. + StatusOr GetProgramShape() const; + + // Returns the (inferred) result for the current computation's shape using the + // given operation as the root. + StatusOr GetProgramShape(XlaOp root) const; + + // Reports an error to the builder, by + // * storing it internally and capturing a backtrace if it's the first error + // (this deferred value will be produced on the call to + // Build()/GetShape()/...) + // * dying if die_immediately_on_error_ is true. + // Returns an XlaOp with an invalid handle but a valid builder. This value can + // be returned in place of a value in APIs that return an XlaOp. + XlaOp ReportError(const Status& error); + + // A helper function that converts a StatusOr into an XlaOp. + // If the Status was an error, reports the error to builder and returns an + // invalid XlaOp handle. + XlaOp ReportErrorOrReturn(const StatusOr& op); + + // A helper function that runs a function that returns a StatusOr and + // returns an XlaOp. + XlaOp ReportErrorOrReturn(const std::function()>& op_creator); + + // Returns true if 'operand' is a compile-time constant. A compile-time + // constant does not depend on any parameters, or on stateful operators such + // as `RngNormal` or `Infeed`. + // + // This tests whether a computation is a compile-time constant without + // evaluating the computation. + StatusOr IsConstant(const XlaOp& operand) const; + + private: + // Build helper which takes the id of the root operation.. + StatusOr Build(int64 root_id); + + // Enqueues a "retrieve parameter value" instruction for a parameter that was + // passed to the computation. + XlaOp Parameter(int64 parameter_number, const Shape& shape, + const string& name); + + // Enqueues a constant with the value of the given literal onto the + // computation. + XlaOp ConstantLiteral(const LiteralSlice& literal); + + // Enqueues a constant onto the computation. Methods are templated on the + // native host type (NativeT) which corresponds to a specific XLA + // PrimitiveType as given in the following table: + // + // Native Type PrimitiveType + // ----------------------------- + // bool PRED + // int32 S32 + // int64 S64 + // uint32 U32 + // uint64 U64 + // float F32 + // double F64 + // + // Note: not all primitive types defined in xla_data.proto have a + // corresponding native type yet. + template + XlaOp ConstantR0(NativeT value); + template + XlaOp ConstantR1(absl::Span values); + XlaOp ConstantR1(const tensorflow::core::Bitmap& values); + template + XlaOp ConstantR2( + std::initializer_list> values); + template + XlaOp ConstantFromArrayWithLayout(const Array& values, + const Layout& layout); + template + XlaOp ConstantFromArray(const Array& values); + template + XlaOp ConstantR2FromArray2DWithLayout(const Array2D& values, + const Layout& layout); + template + XlaOp ConstantR2FromArray2D(const Array2D& values); + template + XlaOp ConstantR3FromArray3DWithLayout(const Array3D& values, + const Layout& layout); + template + XlaOp ConstantR3FromArray3D(const Array3D& values); + template + XlaOp ConstantR4FromArray4DWithLayout(const Array4D& values, + const Layout& layout); + template + XlaOp ConstantR4FromArray4D(const Array4D& values); + + // Enqueues a rank one constant (vector) onto the computation. The vector has + // size 'length' and every element has the value 'value'. + template + XlaOp ConstantR1(int64 length, NativeT value); + + // Adds dimensions to an array by duplicating the data in the array. + // + // The new dimensions are inserted on the left, i.e. if + // broadcast_sizes has values {a0, ..., aN} and the operand shape + // has dimensions {b0, ..., bM} then the shape of the output has + // dimensions {a0, ..., aN, b0, ..., bM}. + // + // The new dimensions index into copies of the operand, i.e. + // + // output[i0, ..., iN, j0, ..., jM] = operand[j0, ..., jM] + XlaOp Broadcast(const XlaOp& operand, + absl::Span broadcast_sizes); + + // Performs in-dimension-style broadcast. + // + // Operand specifies the input to be broadcast. "shape" is expected output + // shape. "broadcast_dimensions" are the dimensions to be broadcasting into. + // Dimension numbers in broadcast_dimensions map to individual dimensions + // of the operand, and specify what dimension of the output shape they + // should be broadcast. + // e.g. + // Say operand = [1, 2], i.e., a 1D tensor with 2 elements. + // and dimension of shape is [2,2]. + // Specifying {1} as brodcast_dimension will generate output + // [1 , 2] + // [1 , 2] + // On the other hand, specifying {0} as broadcast_dimension + // will generate output + // [1 , 1] + // [2 , 2] + XlaOp BroadcastInDim(const XlaOp& operand, const Shape& shape, + const absl::Span broadcast_dimensions); + + // Enqueues a pad operation onto the computation that pads the given value on + // the edges as well as between the elements of the input. padding_config + // specifies the padding amount for each dimension. + XlaOp Pad(const XlaOp& operand, const XlaOp& padding_value, + const PaddingConfig& padding_config); + + // Enqueues an operation onto the computation that flattens the operand based + // on the dimension order (major/slowest-varying to minor/fastest-varying) + // given, followed by reshaping it into the shape with the given dimension + // sizes (also major to minor). Conceptually, this is a limited form of + // "shape casting". + XlaOp Reshape(const XlaOp& operand, absl::Span dimensions, + absl::Span new_sizes); + + // Enqueues an operation onto the computation that collapses the operand, from + // first to last dimension (C order), then reshapes it to the given dimension + // sizes. Conceptually, this is a limited form of "shape casting". + XlaOp Reshape(const XlaOp& operand, absl::Span new_sizes); + + // Wrapper for Reshape. + // Enqueues an operation to collapse the provided dimensions; e.g. an + // operand with dimensions {x=256, y=2, z=2, p=32} can be collapsed to + // {x=1024, y=32} by collapsing dims {0, 1, 2}. Collapsing dimensions must + // be a consecutive, in-order subsequence of the operand dimensions. + // + // Note that collapsing a single dimension does nothing: + // + // {256} collapsing {0} => {256} + // {1} collapsing {0} => {1} + // + // Collapsing multiple dimensions produces a single result dimension: + // + // {256, 2} collapsing {0,1} => {512} + // {256, 2, 3} collapsing {0,1} => {512, 3} + // + // This could potentially cause data to be moved -- it provides a more + // structured form of reshaping than an arbitrary Reshape operation. + XlaOp Collapse(const XlaOp& operand, absl::Span dimensions); + + // Enqueues a slice operation onto the computation that slices the operand + // from the start indices to the limit indices; e.g. + // + // x + // [ 0 1 2 3 ] + // y [ 4 5 6 7 ] => slice(start={1, 1}, limit={2, 3}) => [ 5 6 ] + // [ 8 9 a b ] + // + // Note that "limit" means up-to-but-not-including; i.e. [start, limit) in 1D + // range notation. + // The strides parameter determines the stride over the slice + XlaOp Slice(const XlaOp& operand, absl::Span start_indices, + absl::Span limit_indices, + absl::Span strides); + + // Enqueues a slice operation in a given dimension, taking all other + // dimensions as they are; e.g. if dimno is 1 from start_index 2 to + // limit_index 4 by 1, and the shape is f32[7,8,9], this call is short-hand + // for: + // + // array[:, 2:4:1, :] + XlaOp SliceInDim(const XlaOp& operand, int64 start_index, int64 limit_index, + int64 stride, int64 dimno); + + // Enqueues a slice operation onto the computation that slices the 'operand' + // from dynamic start indices which are passed in 'start_indices'. + // The size of the slice in each dimension is passed in 'slice_sizes', + // which specify the end point of exclusive slice intervals in each + // dimension [start, start + size). + // The shape of 'start_indices' must be rank == 1, with dimension size + // equal to the rank of the 'operand'. + // Slice index calculations are computed modulo input dimension sizes to + // prevent dynamic start indices from generating out-of-bound array accesses. + XlaOp DynamicSlice(const XlaOp& operand, const XlaOp& start_indices, + absl::Span slice_sizes); + + // Enqueues a dynamic update slice operation onto the computation, which + // updates a slice of 'operand' with 'update' at dynamic 'start_indices'. + // The shape of 'update' determines the shape of the slice of 'operand' + // which is updated. + // The indices specified in 'start_indices' specify the offset of the slice + // of 'operand' which is updated. + // + // update = {10, 11} // calculated at runtime. + // [1 2 3] start = {1, 1} // calculated at runtime. [1 2 3 ] + // [4 5 6] => DynamicUpdateslice(data, update, start) => [4 10 11] + // [7 8 9] [7 8 9 ] + // + // The shape of 'start_indices' must be rank == 1, with dimension size + // equal to the rank of the 'operand'. + // Slice index calculations are computed modulo update dimension sizes to + // prevent dynamic start indices from generating out-of-bound array accesses. + XlaOp DynamicUpdateSlice(const XlaOp& operand, const XlaOp& update, + const XlaOp& start_indices); + + // Enqueues a concatenate instruction onto the computation. 'operands' must + // have >= 1 entry. + XlaOp ConcatInDim(absl::Span operands, int64 dimension); + + // Enqueue a tracing operation onto the computation; the computation will emit + // a logging message with the operand. + void Trace(const string& tag, const XlaOp& operand); + + // Enqueues a conditional-move-like select operation onto the computation; + // predicated on pred, selects between on_true and on_false. + XlaOp Select(const XlaOp& pred, const XlaOp& on_true, const XlaOp& on_false); + + // Enqueues a tuple-creation instruction onto the computation. + XlaOp Tuple(absl::Span elements); + + // Enqueues a tuple-element-get instruction onto the computation. + XlaOp GetTupleElement(const XlaOp& tuple_data, int64 index); + + // Enqueues an equal-to comparison instruction onto the computation. + XlaOp Eq(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions = {}); + + // Enqueues a not-equal comparison instruction onto the computation. + XlaOp Ne(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions = {}); + + // Enqueues a greater-or-equal comparison instruction onto the computation. + XlaOp Ge(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions = {}); + + // Enqueues a greater-than comparison instruction onto the computation. + XlaOp Gt(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions = {}); + + // Enqueues a less-than comparison instruction onto the computation. + XlaOp Lt(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions = {}); + + // Enqueues a less-or-equal comparison instruction onto the computation. + XlaOp Le(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions = {}); + + // Enqueues a dot instruction onto the computation. + XlaOp Dot(const XlaOp& lhs, const XlaOp& rhs, + const PrecisionConfig* precision_config = nullptr); + + // Enqueues a general dot instruction onto the computation. + XlaOp DotGeneral(const XlaOp& lhs, const XlaOp& rhs, + const DotDimensionNumbers& dimension_numbers, + const PrecisionConfig* precision_config = nullptr); + + // Enqueues a convolution instruction onto the computation, which uses the + // default convolution dimension numbers. + XlaOp Conv(const XlaOp& lhs, const XlaOp& rhs, + absl::Span window_strides, Padding padding, + int64 feature_group_count = 1, + const PrecisionConfig* precision_config = nullptr); + + // Enqueues a convolution instruction onto the computation, with the caller + // provided padding configuration in the format returned by MakePadding(). + XlaOp ConvWithGeneralPadding( + const XlaOp& lhs, const XlaOp& rhs, + absl::Span window_strides, + absl::Span> padding, + int64 feature_group_count = 1, + const PrecisionConfig* precision_config = nullptr); + + // Enqueues a convolution instruction onto the computation, with the caller + // provided dimension numbers configuration. + XlaOp ConvWithGeneralDimensions( + const XlaOp& lhs, const XlaOp& rhs, + absl::Span window_strides, Padding padding, + const ConvolutionDimensionNumbers& dimension_numbers, + int64 feature_group_count = 1, + const PrecisionConfig* precision_config = nullptr); + + // Enqueues a convolution instruction onto the computation, with the caller + // provided padding configuration as well as the dimension numbers. + XlaOp ConvGeneral(const XlaOp& lhs, const XlaOp& rhs, + absl::Span window_strides, + absl::Span> padding, + const ConvolutionDimensionNumbers& dimension_numbers, + int64 feature_group_count = 1, + const PrecisionConfig* precision_config = nullptr); + + // Enqueues a convolution instruction onto the computation, with the caller + // provided padding configuration, dilation factors and dimension numbers. + XlaOp ConvGeneralDilated(const XlaOp& lhs, const XlaOp& rhs, + absl::Span window_strides, + absl::Span> padding, + absl::Span lhs_dilation, + absl::Span rhs_dilation, + const ConvolutionDimensionNumbers& dimension_numbers, + int64 feature_group_count = 1, + const PrecisionConfig* precision_config = nullptr); + + // Enqueues an FFT instruction onto the computation, of the given type and + // with the given FFT length. + XlaOp Fft(const XlaOp& operand, FftType fft_type, + absl::Span fft_length); + + // Enqueues an infeed instruction onto the computation, which writes data of + // the given shape to the infeed buffer of the device. + XlaOp Infeed(const Shape& shape, const string& config = ""); + XlaOp InfeedWithToken(const XlaOp& token, const Shape& shape, + const string& config = ""); + + // Enqueues an outfeed instruction onto the computation. This instruction + // generates outgoing data transfers for the given data. + // + // shape_with_layout communicates the laid out shape that we want to outfeed + // -- if !ShapeUtil::Compatible(GetShape(operand), shape_with_layout) an error + // will occur. + void Outfeed(const XlaOp& operand, const Shape& shape_with_layout, + const string& outfeed_config); + XlaOp OutfeedWithToken(const XlaOp& operand, const XlaOp& token, + const Shape& shape_with_layout, + const string& outfeed_config); + + // Enqueues a call instruction onto the computation. + XlaOp Call(const XlaComputation& computation, + absl::Span operands); + + // Enqueues a custom call instruction onto the computation. + XlaOp CustomCall(const string& call_target_name, + absl::Span operands, const Shape& shape, + const string& opaque); + + // The following methods enqueue element-wise binary arithmetic operations + // onto the computation. The shapes of the operands have to match unless one + // of the operands is a scalar, or an explicit broadcast dimension is given + // (see g3doc for more details). + + // Enqueues a complex compose instruction onto the computation. + XlaOp Complex(const XlaOp& real, const XlaOp& imag, + absl::Span broadcast_dimensions = {}); + + // Enqueues a complex conjugate instruction onto the computation. + XlaOp Conj(const XlaOp& operand); + + // Enqueues an add instruction onto the computation. + XlaOp Add(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions = {}); + + // Enqueues a subtract instruction onto the computation. + XlaOp Sub(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions = {}); + + // Enqueues a multiply instruction onto the computation. + XlaOp Mul(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions = {}); + + // Enqueues a divide instruction onto the computation. + XlaOp Div(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions = {}); + + // Enqueues a remainder instruction onto the computation. + XlaOp Rem(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions = {}); + + // Enqueues a max instruction onto the computation. + XlaOp Max(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions = {}); + + // Enqueues a min instruction onto the computation. + XlaOp Min(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions = {}); + + // Element-wise logical operators + XlaOp And(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions = {}); + + XlaOp Or(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions = {}); + + XlaOp Xor(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions = {}); + + XlaOp Not(const XlaOp& operand); + + XlaOp ShiftLeft(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions = {}); + XlaOp ShiftRightArithmetic(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions = {}); + XlaOp ShiftRightLogical(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions = {}); + + // Reduces an array among the provided dimensions, given "computation" as a + // reduction operator. + XlaOp Reduce(const XlaOp& operand, const XlaOp& init_value, + const XlaComputation& computation, + absl::Span dimensions_to_reduce); + + // Reduces several arrays simultaneously among the provided dimensions, given + // "computation" as a reduction operator. + XlaOp Reduce(absl::Span operands, + absl::Span init_values, + const XlaComputation& computation, + absl::Span dimensions_to_reduce); + + // Convenience wrapper around the above that reduces all the dimensions in the + // operand shape. + XlaOp ReduceAll(const XlaOp& operand, const XlaOp& init_value, + const XlaComputation& computation); + + // Enqueues a windowed reduce instruction onto the computation. + XlaOp ReduceWindow(const XlaOp& operand, const XlaOp& init_value, + const XlaComputation& computation, + absl::Span window_dimensions, + absl::Span window_strides, Padding padding); + + // As ReduceWindow(), but the padding is given in the format + // returned by MakePadding(). + XlaOp ReduceWindowWithGeneralPadding( + const XlaOp& operand, const XlaOp& init_value, + const XlaComputation& computation, + absl::Span window_dimensions, + absl::Span window_strides, + absl::Span> padding); + + // Returns the sum of the operand value within each subgroup of replicas. All + // replicas supply one input to the sum and all replicas receive the resulting + // sum for each subgroup. + XlaOp CrossReplicaSum(const XlaOp& operand, + absl::Span replica_groups = {}); + + // Enqueues an operation that do an AllReduce of the operand cross cores. Here + // AllReduce means doing a reduction on the input operand cross cores and then + // broadcasting the reduction result to those cores. The reduction function is + // defined by `computation`, which should be a commutative computation on + // scalars, e.g., add, min, or max. The way that AllReduce is applied is + // configured by: + // + // - `replica_groups`: each ReplicaGroup contains a list of replica id. If + // empty, all replicas belong to one group. Allreduce will be applied within + // subgroups. For example, we have 4 replicas, then + // replica_groups={{0,2},{1,3}} means, replica 0 and 2 are in subgroup 0, + // replica 1 and 3 are in subgroup 1. + // + // - `channel_id`: for Allreduce nodes from different modules, if they have + // the same channel_id, they will be 'Allreduce'd. If empty, Allreduce will + // not be applied cross modules. + // + // TODO(b/79737069): Rename this to AllReduce when it's ready to use. + XlaOp CrossReplicaSum( + const XlaOp& operand, const XlaComputation& computation, + absl::Span replica_groups = {}, + const absl::optional& channel_id = absl::nullopt); + + // Enqueues an operation that do an Alltoall of the operand cross cores. + XlaOp AllToAll(const XlaOp& operand, int64 split_dimension, + int64 concat_dimension, int64 split_count, + const std::vector& replica_groups); + + // Enqueues an operation that do an CollectivePermute of the operand cross + // cores. + XlaOp CollectivePermute( + const XlaOp& operand, + const std::vector>& source_target_pairs); + + // Enqueues an operation that scatters the `source` array to the selected + // indices of each window. + XlaOp SelectAndScatter(const XlaOp& operand, const XlaComputation& select, + absl::Span window_dimensions, + absl::Span window_strides, + Padding padding, const XlaOp& source, + const XlaOp& init_value, + const XlaComputation& scatter); + + // As SelectAndScatter(), but the padding is given in the format + // returned by MakePadding(). + XlaOp SelectAndScatterWithGeneralPadding( + const XlaOp& operand, const XlaComputation& select, + absl::Span window_dimensions, + absl::Span window_strides, + absl::Span> padding, const XlaOp& source, + const XlaOp& init_value, const XlaComputation& scatter); + + // Enqueues an abs instruction onto the computation. + XlaOp Abs(const XlaOp& operand); + + // Enqueues a atan2 instruction onto the computation. + XlaOp Atan2(const XlaOp& y, const XlaOp& x, + absl::Span broadcast_dimensions = {}); + + // Enqueues an exp instruction onto the computation. + XlaOp Exp(const XlaOp& operand); + + // Enqueues an expm1 instruction onto the computation. + XlaOp Expm1(const XlaOp& operand); + + // Enqueues a floor instruction onto the computation. + XlaOp Floor(const XlaOp& operand); + + // Enqueues a ceil instruction onto the computation. + XlaOp Ceil(const XlaOp& operand); + + // Enqueues a round instruction onto the computation, rounding to nearest even + // with half-way cases rounding away from zero. + XlaOp Round(const XlaOp& operand); + + // Enqueues an log instruction (natural logarithm) onto the computation. + XlaOp Log(const XlaOp& operand); + + // Enqueues an log1p instruction (log(x+1)) onto the computation. + XlaOp Log1p(const XlaOp& operand); + + // Enqueues a sign instruction onto the computation. + XlaOp Sign(const XlaOp& operand); + + // Enqueues a count leading zeros instruction onto the computation. + XlaOp Clz(const XlaOp& operand); + + // Enqueues a cosine instruction onto the computation. + XlaOp Cos(const XlaOp& operand); + + // Enqueues a sine instruction onto the computation. + XlaOp Sin(const XlaOp& operand); + + // Enqueues a tanh instruction onto the computation. + XlaOp Tanh(const XlaOp& operand); + + // Enqueues a real-part instruction onto the computation. + XlaOp Real(const XlaOp& operand); + + // Enqueues an imaginary-part instruction onto the computation. + XlaOp Imag(const XlaOp& operand); + + // Enqueues a lhs^rhs computation onto the computation. + XlaOp Pow(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions = {}); + + // Enqueues an operator that tests if the operand's values are finite, i.e., + // not Inf or NaN. Defined only for floating-point types. Returns an array of + // booleans with the same shape where entries are true iff the corresponding + // entry was NaN. + XlaOp IsFinite(const XlaOp& operand); + + // Enqueues an iota operation onto the computation. + XlaOp Iota(const Shape& shape, int64 iota_dimension); + + // Enqueues a rank-1 iota operation onto the computation. + XlaOp Iota(PrimitiveType type, int64 size); + + // Enqueues a convert instruction onto the computation that changes the + // element type of the operand array to primitive_type. + XlaOp ConvertElementType(const XlaOp& operand, + PrimitiveType new_element_type); + + // Enqueues a no-op instruction onto the computation that changes + // the element type of the operand array to primitive_type. The + // bit-widths of the source and destination element types must be + // identical. + XlaOp BitcastConvertType(const XlaOp& operand, + PrimitiveType new_element_type); + + // Enqueues a negate instruction onto the computation. + XlaOp Neg(const XlaOp& operand); + + // Enqueues a transpose instruction onto the computation. + XlaOp Transpose(const XlaOp& operand, absl::Span permutation); + + // Enqueues a reverse instruction onto the computation. The order of the + // elements in the given dimensions is reversed (i.e., the element at index i + // is moved to index dimension_size - 1 - i). + XlaOp Rev(const XlaOp& operand, absl::Span dimensions); + + // Enqueues a sort (as increasing order) instruction onto the computation. + // If only keys are provided: + // * If the keys are an rank-1 tensor (an array), the result is a sorted array + // of keys, in ascending order. + // * If the keys have higher rank, the keys are sorted along the provided + // dimension. For example, for a rank-2 tensor (a matrix) of keys, a dimension + // value of 0 will indepenently sort every column, and a dimension value of 1 + // will independently sort each row. If no dimension number is provided, then + // the last dimension is chosen by default. + // + // If both keys and values are provided: + // * The keys and the values must tensors with the same dimensions. The + // element types of the tensors may be different. + // * The result is a tuple that consists of a sorted tensor of keys (along the + // provided dimension, as above) as the first element, and a tensor with their + // corresponding values as the second element. + XlaOp Sort(XlaOp keys, absl::optional values = absl::nullopt, + int64 dimension = -1); + + // Enqueues a clamp instruction onto the computation. + XlaOp Clamp(const XlaOp& min, const XlaOp& operand, const XlaOp& max); + + // Enqueues a map instruction onto the computation. + XlaOp Map(absl::Span operands, const XlaComputation& computation, + absl::Span dimensions, + absl::Span static_operands = {}); + + // Enqueues a N(mu, sigma) random number generation instruction onto the + // computation. + XlaOp RngNormal(const XlaOp& mu, const XlaOp& sigma, const Shape& shape); + + // Enqueues a U(a, b) random number generation instruction onto the + // computation. Returns values in the semi-open interval [a, b). + XlaOp RngUniform(const XlaOp& a, const XlaOp& b, const Shape& shape); + + // Enqueues a while node onto the computation. + XlaOp While(const XlaComputation& condition, const XlaComputation& body, + const XlaOp& init); + + // Enqueues a conditional node onto the computation. + XlaOp Conditional(const XlaOp& predicate, const XlaOp& true_operand, + const XlaComputation& true_computation, + const XlaOp& false_operand, + const XlaComputation& false_computation); + + // Enqueues a ReducePrecision node onto the computation. + XlaOp ReducePrecision(const XlaOp& operand, const int exponent_bits, + const int mantissa_bits); + + // Enqueues a Gather node onto the computation. + XlaOp Gather(const XlaOp& input, const XlaOp& start_indices, + const GatherDimensionNumbers& dimension_numbers, + absl::Span slice_sizes); + + // Enqueues a Scatter node onto the computation. + XlaOp Scatter(const XlaOp& input, const XlaOp& scatter_indices, + const XlaOp& updates, const XlaComputation& update_computation, + const ScatterDimensionNumbers& dimension_numbers); + + // Enqueues a Send node onto the computation for device-to-device + // communication, to send the given operand to a Recv instruction that shares + // the same channel handle. + void Send(const XlaOp& operand, const ChannelHandle& handle); + XlaOp SendWithToken(const XlaOp& operand, const XlaOp& token, + const ChannelHandle& handle); + + // Enqueues a Send node which sends data to the host. + XlaOp SendToHost(const XlaOp& operand, const XlaOp& token, + const Shape& shape_with_layout, const ChannelHandle& handle); + + // Enqueues a Recv node which receives data from the host. + XlaOp RecvFromHost(const XlaOp& token, const Shape& shape, + const ChannelHandle& handle); + + // Enqueues an AfterAll operation with no operands producing a token-shaped + // value. + XlaOp CreateToken(); + + // Enqueues an AfterAll operation with no operands producing a token-shaped + // value. + XlaOp AfterAll(absl::Span tokens); + + // Enqueues a Recv node onto the computation. The data comes from a Send + // instruction that shares the same channel handle and its shape must + // be the same as the given shape. + XlaOp Recv(const Shape& shape, const ChannelHandle& handle); + XlaOp RecvWithToken(const XlaOp& token, const Shape& shape, + const ChannelHandle& handle); + + // Normalizes operand across spatial and batch dimensions for each feature. + // + // Returns a tuple (normalized, batch_mean, batch_var) where `normalized` + // is the normalized result and batch_mean and batch_var are the mean and + // variance, respectively, across batch for the operand. + XlaOp BatchNormTraining(const XlaOp& operand, const XlaOp& scale, + const XlaOp& offset, float epsilon, + int64 feature_index); + + // Normalizes operand across spatial and batch dimensions for each feature. + // + // `BatchNormInference` is equivalent to calling `BatchNormTraining` without + // computing `mean` and `variance` for each batch inside the operation. It + // uses the input `mean` and `variance` instead as estimated values. The + // purpose of this op is to reduce latency in inference, hence the name + // `BatchNormInference`. + // + // The output has the same shape as `operand`, and contains the normalized + // values for each batch. + XlaOp BatchNormInference(const XlaOp& operand, const XlaOp& scale, + const XlaOp& offset, const XlaOp& mean, + const XlaOp& variance, float epsilon, + int64 feature_index); + + // Calculates the gradients of a batch norm op. + // + // The inputs `batch_mean` and `batch_var` represent the mean and variance + // across the batch. + // + // Returns a tuple of three elements: + // - grad_operand: Gradient with respect to input `operand` + // - grad_offset: Gradient with respect to input `offset` + // - grad_scale: Gradient with respect to input `scale` + XlaOp BatchNormGrad(const XlaOp& operand, const XlaOp& scale, + const XlaOp& batch_mean, const XlaOp& batch_var, + const XlaOp& grad_output, float epsilon, + int64 feature_index); + + StatusOr AddInstruction(HloInstructionProto&& instr, HloOpcode opcode, + absl::Span operands = {}); + + void AddCalledComputation(const XlaComputation& computation, + HloInstructionProto* instr); + + StatusOr LookUpInstruction(const XlaOp& op) const; + StatusOr LookUpInstructionByHandle( + int64 handle) const; + + // Internal helper method that does the building for an arbitrary unary op. + XlaOp UnaryOp(HloOpcode unop, const XlaOp& operand); + + // Internal helper method that does the building for an arbitrary binary op. + // broadcast_dimensions specifies which dimensions to use for broadcasting + // when the operation is between tensors of different ranks. + XlaOp BinaryOp(HloOpcode binop, const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions); + + // Internal helper method that does the building for an arbitrary ternary op. + XlaOp TernaryOp(HloOpcode triop, const XlaOp& lhs, const XlaOp& rhs, + const XlaOp& ehs); + + XlaOp RngOp(RandomDistribution distribution, + absl::Span parameters, const Shape& shape); + + StatusOr InDimBroadcast(const Shape& shape, const XlaOp& operand, + absl::Span broadcast_dimensions); + + // Internal helper method that creates a sequence of instructions that + // performs an explicit broadcast of the operand to the target shape. + StatusOr AddBroadcastSequence(const Shape& output_shape, + const XlaOp& operand); + + // Internal helper method for creating a Reshape op with the already inferred + // shape. + StatusOr Reshape(const Shape& shape, const XlaOp& operand); + + // Returns the (inferred) result for the program shape using the given root. + StatusOr GetProgramShape(int64 root_id) const; + + // Returns shapes for the operands. + StatusOr> GetOperandShapes( + absl::Span operands) const; + + // A visitor which checks whether an operation is a compile-time constant, + // meaning that it doesn't depend on any parameters, or on any stateful + // operation such as `RngNormal` or `Infeed`. The visitor walks the + // computation starting at a given operation and sets is_constant to false iff + // a parameter or stateful operation is encountered. + void IsConstantVisitor(const int64 op_handle, std::set* visited, + bool* is_constant) const; + + // Checks bounds for convolution parameters. + Status VerifyConvolution( + const Shape& lhs_shape, const Shape& rhs_shape, + const ConvolutionDimensionNumbers& dimension_numbers) const; + + // Helper function for creating a Window proto from user-supplied data. + // Returns error if the user-supplied data was invalid. + StatusOr MakeWindow(absl::Span window_dimensions, + absl::Span window_strides, + absl::Span> padding, + absl::Span lhs_dilation, + absl::Span rhs_dilation) const; + + string name_; // Name to use for the built computation. + + // The first error encountered while building the computation. + // This is OK until the first error is encountered. + Status first_error_; + + // The saved stack trace from the point at which the first error occurred. + tensorflow::SavedStackTrace first_error_backtrace_; + + // The instructions of this computation. + std::vector instructions_; + + // A map from XlaOp::Handle to the index in the instructions_ vector where the + // instruction is held. + absl::flat_hash_map handle_to_index_; + + // The embedded computations used by this computation. Each computation was + // the entry computation of some XlaComputation, the key is the unique id of + // that XlaComputation. + std::map embedded_; + + // The unique parameter numbers. + absl::flat_hash_set parameter_numbers_; + + // The metadata to attach to each op. This is structured as a "modal"-like + // operation, in order to simplify client code (and not sprinkle this metadata + // throughout the TensorFlow op kernel implementations). + OpMetadata metadata_; + + // Sharding for this operator. This is structured as a "model"-like operation, + // in order to simplify client code, similar to metadata_. + absl::optional sharding_; + + // Mode bit that indicates whether to die when a first error is encountered. + bool die_immediately_on_error_ = false; + + XlaBuilder* parent_builder_{nullptr}; + + friend XlaOp Parameter(XlaBuilder* builder, int64 parameter_number, + const Shape& shape, const string& name); + friend XlaOp ConstantLiteral(XlaBuilder* builder, + const LiteralSlice& literal); + template + friend XlaOp ConstantR0(XlaBuilder* builder, NativeT value); + template + friend XlaOp ConstantR1(XlaBuilder* builder, + absl::Span values); + friend XlaOp ConstantR1(XlaBuilder* builder, + const tensorflow::core::Bitmap& values); + template + friend XlaOp ConstantR2( + XlaBuilder* builder, + std::initializer_list> values); + template + friend XlaOp ConstantFromArrayWithLayout(XlaBuilder* builder, + const Array& values, + const Layout& layout); + template + friend XlaOp ConstantFromArray(XlaBuilder* builder, + const Array& values); + template + friend XlaOp ConstantR2FromArray2DWithLayout(XlaBuilder* builder, + const Array2D& values, + const Layout& layout); + template + friend XlaOp ConstantR2FromArray2D(XlaBuilder* builder, + const Array2D& values); + template + friend XlaOp ConstantR3FromArray3DWithLayout(XlaBuilder* builder, + const Array3D& values, + const Layout& layout); + template + friend XlaOp ConstantR3FromArray3D(XlaBuilder* builder, + const Array3D& values); + template + friend XlaOp ConstantR4FromArray4DWithLayout(XlaBuilder* builder, + const Array4D& values, + const Layout& layout); + template + friend XlaOp ConstantR4FromArray4D(XlaBuilder* builder, + const Array4D& values); + + template + friend XlaOp ConstantR1(XlaBuilder* builder, int64 length, NativeT value); + + friend XlaOp Broadcast(const XlaOp& operand, + absl::Span broadcast_sizes); + + friend XlaOp BroadcastInDim( + const XlaOp& operand, const Shape& shape, + const absl::Span broadcast_dimensions); + + friend XlaOp Pad(const XlaOp& operand, const XlaOp& padding_value, + const PaddingConfig& padding_config); + + friend XlaOp Reshape(const XlaOp& operand, absl::Span dimensions, + absl::Span new_sizes); + + friend XlaOp Reshape(const XlaOp& operand, absl::Span new_sizes); + + friend XlaOp Collapse(const XlaOp& operand, + absl::Span dimensions); + + friend XlaOp Slice(const XlaOp& operand, + absl::Span start_indices, + absl::Span limit_indices, + absl::Span strides); + + friend XlaOp SliceInDim(const XlaOp& operand, int64 start_index, + int64 limit_index, int64 stride, int64 dimno); + + friend XlaOp DynamicSlice(const XlaOp& operand, const XlaOp& start_indices, + absl::Span slice_sizes); + + friend XlaOp DynamicUpdateSlice(const XlaOp& operand, const XlaOp& update, + const XlaOp& start_indices); + + friend XlaOp ConcatInDim(XlaBuilder* builder, + absl::Span operands, int64 dimension); + + friend void Trace(const string& tag, const XlaOp& operand); + + friend XlaOp Select(const XlaOp& pred, const XlaOp& on_true, + const XlaOp& on_false); + friend XlaOp Tuple(XlaBuilder* builder, absl::Span elements); + friend XlaOp GetTupleElement(const XlaOp& tuple_data, int64 index); + friend XlaOp Eq(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions); + friend XlaOp Ne(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions); + friend XlaOp Ge(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions); + friend XlaOp Gt(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions); + friend XlaOp Lt(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions); + friend XlaOp Le(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions); + friend XlaOp Dot(const XlaOp& lhs, const XlaOp& rhs, + const PrecisionConfig* precision_config); + friend XlaOp DotGeneral(const XlaOp& lhs, const XlaOp& rhs, + const DotDimensionNumbers& dimension_number, + const PrecisionConfig* precision_config); + friend XlaOp Conv(const XlaOp& lhs, const XlaOp& rhs, + absl::Span window_strides, Padding padding, + int64 feature_group_count, + const PrecisionConfig* precision_config); + friend XlaOp ConvWithGeneralPadding( + const XlaOp& lhs, const XlaOp& rhs, + absl::Span window_strides, + absl::Span> padding, + int64 feature_group_count, const PrecisionConfig* precision_config); + friend XlaOp ConvWithGeneralDimensions( + const XlaOp& lhs, const XlaOp& rhs, + absl::Span window_strides, Padding padding, + const ConvolutionDimensionNumbers& dimension_numbers, + int64 feature_group_count, const PrecisionConfig* precision_config); + friend XlaOp ConvGeneral(const XlaOp& lhs, const XlaOp& rhs, + absl::Span window_strides, + absl::Span> padding, + const ConvolutionDimensionNumbers& dimension_numbers, + int64 feature_group_count, + const PrecisionConfig* precision_config); + friend XlaOp ConvGeneralDilated( + const XlaOp& lhs, const XlaOp& rhs, + absl::Span window_strides, + absl::Span> padding, + absl::Span lhs_dilation, + absl::Span rhs_dilation, + const ConvolutionDimensionNumbers& dimension_numbers, + int64 feature_group_count, const PrecisionConfig* precision_config); + friend XlaOp Fft(const XlaOp& operand, FftType fft_type, + absl::Span fft_length); + friend XlaOp Infeed(XlaBuilder* builder, const Shape& shape, + const string& config); + friend void Outfeed(const XlaOp& operand, const Shape& shape_with_layout, + const string& outfeed_config); + friend XlaOp Call(XlaBuilder* builder, const XlaComputation& computation, + absl::Span operands); + friend XlaOp CustomCall(XlaBuilder* builder, const string& call_target_name, + absl::Span operands, const Shape& shape, + const string& opaque); + friend XlaOp Complex(const XlaOp& real, const XlaOp& imag, + absl::Span broadcast_dimensions); + friend XlaOp Conj(const XlaOp& operand); + friend XlaOp Add(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions); + friend XlaOp Sub(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions); + friend XlaOp Mul(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions); + friend XlaOp Div(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions); + friend XlaOp Rem(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions); + friend XlaOp Max(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions); + friend XlaOp Min(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions); + friend XlaOp And(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions); + friend XlaOp Or(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions); + friend XlaOp Xor(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions); + friend XlaOp Not(const XlaOp& operand); + friend XlaOp ShiftLeft(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions); + friend XlaOp ShiftRightArithmetic( + const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions); + friend XlaOp ShiftRightLogical(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions); + friend XlaOp Reduce(const XlaOp& operand, const XlaOp& init_value, + const XlaComputation& computation, + absl::Span dimensions_to_reduce); + friend XlaOp Reduce(XlaBuilder* builder, absl::Span operands, + absl::Span init_values, + const XlaComputation& computation, + absl::Span dimensions_to_reduce); + friend XlaOp ReduceAll(const XlaOp& operand, const XlaOp& init_value, + const XlaComputation& computation); + friend XlaOp ReduceWindow(const XlaOp& operand, const XlaOp& init_value, + const XlaComputation& computation, + absl::Span window_dimensions, + absl::Span window_strides, + Padding padding); + friend XlaOp ReduceWindowWithGeneralPadding( + const XlaOp& operand, const XlaOp& init_value, + const XlaComputation& computation, + absl::Span window_dimensions, + absl::Span window_strides, + absl::Span> padding); + friend XlaOp CrossReplicaSum(const XlaOp& operand, + absl::Span replica_groups); + friend XlaOp CrossReplicaSum(const XlaOp& operand, + const XlaComputation& computation, + absl::Span replica_groups, + const absl::optional& channel_id); + friend XlaOp AllToAll(const XlaOp& operand, int64 split_dimension, + int64 concat_dimension, int64 split_count, + const std::vector& replica_groups); + friend XlaOp CollectivePermute( + const XlaOp& operand, + const std::vector>& source_target_pairs); + friend XlaOp SelectAndScatter(const XlaOp& operand, + const XlaComputation& select, + absl::Span window_dimensions, + absl::Span window_strides, + Padding padding, const XlaOp& source, + const XlaOp& init_value, + const XlaComputation& scatter); + friend XlaOp SelectAndScatterWithGeneralPadding( + const XlaOp& operand, const XlaComputation& select, + absl::Span window_dimensions, + absl::Span window_strides, + absl::Span> padding, const XlaOp& source, + const XlaOp& init_value, const XlaComputation& scatter); + friend XlaOp Abs(const XlaOp& operand); + friend XlaOp Atan2(const XlaOp& y, const XlaOp& x, + absl::Span broadcast_dimensions); + friend XlaOp Exp(const XlaOp& operand); + friend XlaOp Expm1(const XlaOp& operand); + friend XlaOp Floor(const XlaOp& operand); + friend XlaOp Ceil(const XlaOp& operand); + friend XlaOp Round(const XlaOp& operand); + friend XlaOp Log(const XlaOp& operand); + friend XlaOp Log1p(const XlaOp& operand); + friend XlaOp Sign(const XlaOp& operand); + friend XlaOp Clz(const XlaOp& operand); + friend XlaOp Cos(const XlaOp& operand); + friend XlaOp Sin(const XlaOp& operand); + friend XlaOp Tanh(const XlaOp& operand); + friend XlaOp Real(const XlaOp& operand); + friend XlaOp Imag(const XlaOp& operand); + friend XlaOp Pow(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions); + friend XlaOp IsFinite(const XlaOp& operand); + friend XlaOp Iota(XlaBuilder* builder, const Shape& shape, + int64 iota_dimension); + friend XlaOp Iota(XlaBuilder* builder, PrimitiveType type, int64 size); + friend XlaOp ConvertElementType(const XlaOp& operand, + PrimitiveType new_element_type); + friend XlaOp BitcastConvertType(const XlaOp& operand, + PrimitiveType new_element_type); + friend XlaOp Neg(const XlaOp& operand); + friend XlaOp Transpose(const XlaOp& operand, + absl::Span permutation); + friend XlaOp Rev(const XlaOp& operand, absl::Span dimensions); + friend XlaOp Sort(XlaOp keys, absl::optional values, int64 dimension); + friend XlaOp Clamp(const XlaOp& min, const XlaOp& operand, const XlaOp& max); + friend XlaOp Map(XlaBuilder* builder, absl::Span operands, + const XlaComputation& computation, + absl::Span dimensions, + absl::Span static_operands); + friend XlaOp RngNormal(const XlaOp& mu, const XlaOp& sigma, + const Shape& shape); + friend XlaOp RngUniform(const XlaOp& a, const XlaOp& b, const Shape& shape); + friend XlaOp While(const XlaComputation& condition, + const XlaComputation& body, const XlaOp& init); + friend XlaOp Conditional(const XlaOp& predicate, const XlaOp& true_operand, + const XlaComputation& true_computation, + const XlaOp& false_operand, + const XlaComputation& false_computation); + friend XlaOp ReducePrecision(const XlaOp& operand, const int exponent_bits, + const int mantissa_bits); + friend XlaOp Gather(const XlaOp& input, const XlaOp& start_indices, + const GatherDimensionNumbers& dimension_numbers, + absl::Span slice_sizes); + friend XlaOp Scatter(const XlaOp& input, const XlaOp& scatter_indices, + const XlaOp& updates, + const XlaComputation& update_computation, + const ScatterDimensionNumbers& dimension_numbers); + friend void Send(const XlaOp& operand, const ChannelHandle& handle); + friend XlaOp Recv(XlaBuilder* builder, const Shape& shape, + const ChannelHandle& handle); + friend XlaOp BatchNormTraining(const XlaOp& operand, const XlaOp& scale, + const XlaOp& offset, float epsilon, + int64 feature_index); + friend XlaOp BatchNormInference(const XlaOp& operand, const XlaOp& scale, + const XlaOp& offset, const XlaOp& mean, + const XlaOp& variance, float epsilon, + int64 feature_index); + friend XlaOp BatchNormGrad(const XlaOp& operand, const XlaOp& scale, + const XlaOp& batch_mean, const XlaOp& batch_var, + const XlaOp& grad_output, float epsilon, + int64 feature_index); + friend XlaOp SendWithToken(const XlaOp& operand, const XlaOp& token, + const ChannelHandle& handle); + friend XlaOp RecvWithToken(const XlaOp& token, const Shape& shape, + const ChannelHandle& handle); + friend XlaOp SendToHost(const XlaOp& operand, const XlaOp& token, + const Shape& shape_with_layout, + const ChannelHandle& handle); + friend XlaOp RecvFromHost(const XlaOp& token, const Shape& shape, + const ChannelHandle& handle); + friend XlaOp InfeedWithToken(const XlaOp& token, const Shape& shape, + const string& config); + friend XlaOp OutfeedWithToken(const XlaOp& operand, const XlaOp& token, + const Shape& shape_with_layout, + const string& outfeed_config); + friend XlaOp CreateToken(XlaBuilder* builder); + friend XlaOp AfterAll(XlaBuilder* builder, absl::Span tokens); +}; + +// RAII-style object: sets the current sharding assignment in builder on +// construction, and sets back to the previous assignment on destruction. +class XlaScopedShardingAssignment { + public: + XlaScopedShardingAssignment(xla::XlaBuilder* builder, + absl::optional sharding) + : builder_(builder), prev_sharding_(builder->sharding()) { + SetSharding(sharding); + } + + XlaScopedShardingAssignment(const XlaScopedShardingAssignment&) = delete; + XlaScopedShardingAssignment& operator=(const XlaScopedShardingAssignment&) = + delete; + + ~XlaScopedShardingAssignment() { SetSharding(prev_sharding_); } + + private: + void SetSharding(const absl::optional& sharding) { + if (sharding.has_value()) { + builder_->SetSharding(sharding.value()); + } else { + builder_->ClearSharding(); + } + } + + xla::XlaBuilder* const builder_; + absl::optional prev_sharding_; +}; + +// Free functions for building XlaOps. The intention is that these will +// become the public API for building XlaOps rather than calling methods on +// XlaBuilder directly. + +// Enqueues a "retrieve parameter value" instruction for a parameter that was +// passed to the computation. +XlaOp Parameter(XlaBuilder* builder, int64 parameter_number, const Shape& shape, + const string& name); + +// Enqueues a constant with the value of the given literal onto the +// computation. +XlaOp ConstantLiteral(XlaBuilder* builder, const LiteralSlice& literal); + +// Enqueues a constant onto the computation. Methods are templated on the +// native host type (NativeT) which corresponds to a specific XLA +// PrimitiveType as given in the following table: +// +// Native Type PrimitiveType +// ----------------------------- +// bool PRED +// int32 S32 +// int64 S64 +// uint32 U32 +// uint64 U64 +// float F32 +// double F64 +// +// Note: not all primitive types defined in xla_data.proto have a +// corresponding native type yet. +template +XlaOp ConstantR0(XlaBuilder* builder, NativeT value); +template +XlaOp ConstantR1(XlaBuilder* builder, absl::Span values); +XlaOp ConstantR1(XlaBuilder* builder, const tensorflow::core::Bitmap& values); +template +XlaOp ConstantR2(XlaBuilder* builder, + std::initializer_list> values); +template +XlaOp ConstantFromArrayWithLayout(XlaBuilder* builder, + const Array& values, + const Layout& layout); +template +XlaOp ConstantFromArray(XlaBuilder* builder, const Array& values); +template +XlaOp ConstantR2FromArray2DWithLayout(XlaBuilder* builder, + const Array2D& values, + const Layout& layout); +template +XlaOp ConstantR2FromArray2D(XlaBuilder* builder, + const Array2D& values); +template +XlaOp ConstantR3FromArray3DWithLayout(XlaBuilder* builder, + const Array3D& values, + const Layout& layout); +template +XlaOp ConstantR3FromArray3D(XlaBuilder* builder, + const Array3D& values); +template +XlaOp ConstantR4FromArray4DWithLayout(XlaBuilder* builder, + const Array4D& values, + const Layout& layout); +template +XlaOp ConstantR4FromArray4D(XlaBuilder* builder, + const Array4D& values); + +// Enqueues a rank one constant (XlaBuilder* builder, vector) onto the +// computation. The vector has size 'length' and every element has the value +// 'value'. +template +XlaOp ConstantR1(XlaBuilder* builder, int64 length, NativeT value); + +// Adds dimensions to an array by duplicating the data in the array. +// +// The new dimensions are inserted on the left, i.e. if +// broadcast_sizes has values {a0, ..., aN} and the operand shape +// has dimensions {b0, ..., bM} then the shape of the output has +// dimensions {a0, ..., aN, b0, ..., bM}. +// +// The new dimensions index into copies of the operand, i.e. +// +// output[i0, ..., iN, j0, ..., jM] = operand[j0, ..., jM] +XlaOp Broadcast(const XlaOp& operand, absl::Span broadcast_sizes); + +// Performs in-dimension-style broadcast. +// +// Operand specifies the input to be broadcast. "shape" is expected output +// shape. "broadcast_dimensions" are the dimensions to be broadcasting into. +// Dimension numbers in broadcast_dimensions map to individual dimensions +// of the operand, and specify what dimension of the output shape they +// should be broadcast. +// e.g. +// Say operand = [1, 2], i.e., a 1D tensor with 2 elements. +// and dimension of shape is [2,2]. +// Specifying {1} as brodcast_dimension will generate output +// [1 , 2] +// [1 , 2] +// On the other hand, specifying {0} as broadcast_dimension +// will generate output +// [1 , 1] +// [2 , 2] +XlaOp BroadcastInDim(const XlaOp& operand, const Shape& shape, + const absl::Span broadcast_dimensions); + +// Enqueues a pad operation onto the computation that pads the given value on +// the edges as well as between the elements of the input. padding_config +// specifies the padding amount for each dimension. +XlaOp Pad(const XlaOp& operand, const XlaOp& padding_value, + const PaddingConfig& padding_config); + +// Enqueues an operation onto the computation that flattens the operand based +// on the dimension order (major/slowest-varying to minor/fastest-varying) +// given, followed by reshaping it into the shape with the given dimension +// sizes (also major to minor). Conceptually, this is a limited form of +// "shape casting". +XlaOp Reshape(const XlaOp& operand, absl::Span dimensions, + absl::Span new_sizes); + +// Enqueues an operation onto the computation that collapses the operand, from +// first to last dimension (C order), then reshapes it to the given dimension +// sizes. Conceptually, this is a limited form of "shape casting". +XlaOp Reshape(const XlaOp& operand, absl::Span new_sizes); + +// Wrapper for Reshape. +// Enqueues an operation to collapse the provided dimensions; e.g. an +// operand with dimensions {x=256, y=2, z=2, p=32} can be collapsed to +// {x=1024, y=32} by collapsing dims {0, 1, 2}. Collapsing dimensions must +// be a consecutive, in-order subsequence of the operand dimensions. +// +// Note that collapsing a single dimension does nothing: +// +// {256} collapsing {0} => {256} +// {1} collapsing {0} => {1} +// +// Collapsing multiple dimensions produces a single result dimension: +// +// {256, 2} collapsing {0,1} => {512} +// {256, 2, 3} collapsing {0,1} => {512, 3} +// +// This could potentially cause data to be moved -- it provides a more +// structured form of reshaping than an arbitrary Reshape operation. +XlaOp Collapse(const XlaOp& operand, absl::Span dimensions); + +// Enqueues a slice operation onto the computation that slices the operand +// from the start indices to the limit indices; e.g. +// +// x +// [ 0 1 2 3 ] +// y [ 4 5 6 7 ] => slice(start={1, 1}, limit={2, 3}) => [ 5 6 ] +// [ 8 9 a b ] +// +// Note that "limit" means up-to-but-not-including; i.e. [start, limit) in 1D +// range notation. +// The strides parameter determines the stride over the slice +XlaOp Slice(const XlaOp& operand, absl::Span start_indices, + absl::Span limit_indices, + absl::Span strides); + +// Enqueues a slice operation in a given dimension, taking all other +// dimensions as they are; e.g. if dimno is 1 from start_index 2 to +// limit_index 4 by 1, and the shape is f32[7,8,9], this call is short-hand +// for: +// +// array[:, 2:4:1, :] +XlaOp SliceInDim(const XlaOp& operand, int64 start_index, int64 limit_index, + int64 stride, int64 dimno); + +// Enqueues a slice operation onto the computation that slices the 'operand' +// from dynamic start indices which are passed in 'start_indices'. +// The size of the slice in each dimension is passed in 'slice_sizes', +// which specify the end point of exclusive slice intervals in each +// dimension [start, start + size). +// The shape of 'start_indices' must be rank == 1, with dimension size +// equal to the rank of the 'operand'. +// Slice index calculations are computed modulo input dimension sizes to +// prevent dynamic start indices from generating out-of-bound array accesses. +XlaOp DynamicSlice(const XlaOp& operand, const XlaOp& start_indices, + absl::Span slice_sizes); + +// Enqueues a dynamic update slice operation onto the computation, which +// updates a slice of 'operand' with 'update' at dynamic 'start_indices'. +// The shape of 'update' determines the shape of the slice of 'operand' +// which is updated. +// The indices specified in 'start_indices' specify the offset of the slice +// of 'operand' which is updated. +// +// update = {10, 11} // calculated at runtime. +// [1 2 3] start = {1, 1} // calculated at runtime. [1 2 3 ] +// [4 5 6] => DynamicUpdateslice(data, update, start) => [4 10 11] +// [7 8 9] [7 8 9 ] +// +// The shape of 'start_indices' must be rank == 1, with dimension size +// equal to the rank of the 'operand'. +// Slice index calculations are computed modulo update dimension sizes to +// prevent dynamic start indices from generating out-of-bound array accesses. +XlaOp DynamicUpdateSlice(const XlaOp& operand, const XlaOp& update, + const XlaOp& start_indices); + +// Enqueues a concatenate instruction onto the computation. 'operands' must +// have >= 1 entry. +XlaOp ConcatInDim(XlaBuilder* builder, absl::Span operands, + int64 dimension); + +// Enqueue a tracing operation onto the computation; the computation will emit +// a logging message with the operand. +void Trace(const string& tag, const XlaOp& operand); + +// Enqueues a conditional-move-like select operation onto the computation; +// predicated on pred, selects between on_true and on_false. +XlaOp Select(const XlaOp& pred, const XlaOp& on_true, const XlaOp& on_false); + +// Enqueues a tuple-creation instruction onto the computation. +XlaOp Tuple(XlaBuilder* builder, absl::Span elements); + +// Enqueues a tuple-element-get instruction onto the computation. +XlaOp GetTupleElement(const XlaOp& tuple_data, int64 index); + +// Enqueues an equal-to comparison instruction onto the computation. +XlaOp Eq(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions = {}); + +// Enqueues a not-equal comparison instruction onto the computation. +XlaOp Ne(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions = {}); + +// Enqueues a greater-or-equal comparison instruction onto the computation. +XlaOp Ge(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions = {}); + +// Enqueues a greater-than comparison instruction onto the computation. +XlaOp Gt(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions = {}); + +// Enqueues a less-than comparison instruction onto the computation. +XlaOp Lt(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions = {}); + +// Enqueues a less-or-equal comparison instruction onto the computation. +XlaOp Le(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions = {}); + +// Enqueues a dot instruction onto the computation. +XlaOp Dot(const XlaOp& lhs, const XlaOp& rhs, + const PrecisionConfig* precision_config = nullptr); + +// Enqueues a general dot instruction onto the computation. +XlaOp DotGeneral(const XlaOp& lhs, const XlaOp& rhs, + const DotDimensionNumbers& dimension_numbers, + const PrecisionConfig* precision_config = nullptr); + +// Enqueues a convolution instruction onto the computation, which uses the +// default convolution dimension numbers. +XlaOp Conv(const XlaOp& lhs, const XlaOp& rhs, + absl::Span window_strides, Padding padding, + int64 feature_group_count = 1, + const PrecisionConfig* precision_config = nullptr); + +// Enqueues a convolution instruction onto the computation, with the caller +// provided padding configuration in the format returned by MakePadding(). +XlaOp ConvWithGeneralPadding(const XlaOp& lhs, const XlaOp& rhs, + absl::Span window_strides, + absl::Span> padding, + int64 feature_group_count = 1, + const PrecisionConfig* precision_config = nullptr); + +// Enqueues a convolution instruction onto the computation, with the caller +// provided dimension numbers configuration. +XlaOp ConvWithGeneralDimensions( + const XlaOp& lhs, const XlaOp& rhs, absl::Span window_strides, + Padding padding, const ConvolutionDimensionNumbers& dimension_numbers, + int64 feature_group_count = 1, + const PrecisionConfig* precision_config = nullptr); + +// Enqueues a convolution instruction onto the computation, with the caller +// provided padding configuration as well as the dimension numbers. +XlaOp ConvGeneral(const XlaOp& lhs, const XlaOp& rhs, + absl::Span window_strides, + absl::Span> padding, + const ConvolutionDimensionNumbers& dimension_numbers, + int64 feature_group_count = 1, + const PrecisionConfig* precision_config = nullptr); + +// Enqueues a convolution instruction onto the computation, with the caller +// provided padding configuration, dilation factors and dimension numbers. +XlaOp ConvGeneralDilated(const XlaOp& lhs, const XlaOp& rhs, + absl::Span window_strides, + absl::Span> padding, + absl::Span lhs_dilation, + absl::Span rhs_dilation, + const ConvolutionDimensionNumbers& dimension_numbers, + int64 feature_group_count = 1, + const PrecisionConfig* precision_config = nullptr); + +// Enqueues an FFT instruction onto the computation, of the given type and +// with the given FFT length. +XlaOp Fft(const XlaOp& operand, FftType fft_type, + absl::Span fft_length); + +// Enqueues an infeed instruction onto the computation, which writes data of +// the given shape to the infeed buffer of the device. +XlaOp Infeed(XlaBuilder* builder, const Shape& shape, + const string& config = ""); + +// Variant of Infeed which takes a token-shaped operand and produces a +// two-element tuple containing the data value and a token-shaped value. +// Tokens are used for ordering side-effecting operations. +// TODO(b/110532604): Replace all uses of the non-token form with this variant. +XlaOp InfeedWithToken(const XlaOp& token, const Shape& shape, + const string& config = ""); + +// Enqueues an outfeed instruction onto the computation. This instruction +// generates outgoing data transfers for the given data. +// +// shape_with_layout communicates the laid out shape that we want to outfeed +// -- if !ShapeUtil::Compatible(GetShape(operand), shape_with_layout) an error +// will occur. +void Outfeed(const XlaOp& operand, const Shape& shape_with_layout, + const string& outfeed_config); + +// Variant of Outfeed which takes a token-shaped operand and produces a +// token-shaped value. Tokens are used for ordering side-effecting operations. +// TODO(b/110532604): Replace all uses of the non-token form with this variant. +XlaOp OutfeedWithToken(const XlaOp& operand, const XlaOp& token, + const Shape& shape_with_layout, + const string& outfeed_config); + +// Enqueues a call instruction onto the computation. +XlaOp Call(XlaBuilder* builder, const XlaComputation& computation, + absl::Span operands); + +// Enqueues a custom call instruction onto the computation. A custom call +// invokes code external to XLA. The |operands| are passed to the external code, +// and the external code is expected to produce a result of the given +// |shape|. The exact mechanism is backend-specific. For example, in the CPU +// backend, a call instruction is emitted which targets a symbol with the name +// |call_target_name|. |call_target_name| and |opaque| can arbitrary strings, +// but |call_target_name| should be short as it may be used in labels. |opaque| +// can encode arbitrarily large amounts of information. +XlaOp CustomCall(XlaBuilder* builder, const string& call_target_name, + absl::Span operands, const Shape& shape, + const string& opaque = ""); + +// The following methods enqueue element-wise binary arithmetic operations +// onto the computation. The shapes of the operands have to match unless one +// of the operands is a scalar, or an explicit broadcast dimension is given +// (see g3doc for more details). + +// Enqueues a complex compose instruction onto the computation. +XlaOp Complex(const XlaOp& real, const XlaOp& imag, + absl::Span broadcast_dimensions = {}); + +// Enqueues a complex conjugate instruction onto the computation. +XlaOp Conj(const XlaOp& operand); + +// Enqueues an add instruction onto the computation. +XlaOp Add(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions = {}); + +// Enqueues a subtract instruction onto the computation. +XlaOp Sub(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions = {}); + +// Enqueues a multiply instruction onto the computation. +XlaOp Mul(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions = {}); + +// Enqueues a divide instruction onto the computation. +XlaOp Div(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions = {}); + +// Enqueues a remainder instruction onto the computation. +XlaOp Rem(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions = {}); + +// Enqueues a max instruction onto the computation. +XlaOp Max(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions = {}); + +// Enqueues a min instruction onto the computation. +XlaOp Min(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions = {}); + +// Element-wise logical operators +XlaOp And(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions = {}); + +XlaOp Or(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions = {}); + +XlaOp Xor(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions = {}); + +XlaOp Not(const XlaOp& operand); + +XlaOp ShiftLeft(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions = {}); +XlaOp ShiftRightArithmetic(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions = {}); +XlaOp ShiftRightLogical(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions = {}); + +// Reduces an array among the provided dimensions, given "computation" as a +// reduction operator. +XlaOp Reduce(const XlaOp& operand, const XlaOp& init_value, + const XlaComputation& computation, + absl::Span dimensions_to_reduce); + +// Reduces several arrays simultaneously among the provided dimensions, given +// "computation" as a reduction operator. +XlaOp Reduce(XlaBuilder* builder, absl::Span operands, + absl::Span init_values, + const XlaComputation& computation, + absl::Span dimensions_to_reduce); + +// Convenience wrapper around the above that reduces all the dimensions in the +// operand shape. +XlaOp ReduceAll(const XlaOp& operand, const XlaOp& init_value, + const XlaComputation& computation); + +// Enqueues a windowed reduce instruction onto the computation. +XlaOp ReduceWindow(const XlaOp& operand, const XlaOp& init_value, + const XlaComputation& computation, + absl::Span window_dimensions, + absl::Span window_strides, Padding padding); + +// As ReduceWindow(), but the padding is given in the format +// returned by MakePadding(). +XlaOp ReduceWindowWithGeneralPadding( + const XlaOp& operand, const XlaOp& init_value, + const XlaComputation& computation, + absl::Span window_dimensions, + absl::Span window_strides, + absl::Span> padding); + +// Returns the sum of the operand value within each subgroup of replicas. All +// replicas supply one input to the sum and all replicas receive the resulting +// sum for each subgroup. +XlaOp CrossReplicaSum(const XlaOp& operand, + absl::Span replica_groups = {}); + +// Enqueues an operation that do an AllReduce of the operand cross cores. Here +// AllReduce means doing a reduction on the input operand cross cores and then +// broadcasting the reduction result to those cores. The reduction function is +// defined by `computation`, which should be a commutative computation on +// scalars, e.g., add, min, or max. The way that AllReduce is applied is +// configured by: +// +// - `replica_groups`: each ReplicaGroup contains a list of replica id. If +// empty, all replicas belong to one group. Allreduce will be applied within +// subgroups. For example, we have 4 replicas, then replica_groups={{0,2},{1,3}} +// means, replica 0 and 2 are in subgroup 0, replica 1 and 3 are in subgroup 1. +// +// - `channel_id`: for Allreduce nodes from different modules, if they have the +// same channel_id, they will be 'Allreduce'd. If empty, Allreduce will not be +// applied cross modules. +// +// TODO(b/79737069): Rename this to AllReduce when it's ready to use. +XlaOp CrossReplicaSum( + const XlaOp& operand, const XlaComputation& computation, + absl::Span replica_groups = {}, + const absl::optional& channel_id = absl::nullopt); + +// Enqueues an operation that do an Alltoall of the operand cross cores. +XlaOp AllToAll(const XlaOp& operand, int64 split_dimension, + int64 concat_dimension, int64 split_count, + const std::vector& replica_groups = {}); + +// Enqueues an collective operation that sends and receives data cross replicas. +// +// - `source_target_pair`: a list of (source_replica_id, target_replica_id) +// pairs. For each pair, the operand is sent from source replica to target +// replica. Note that, 1) any two pairs should not have the same target replica +// id, and they should not have the same source replica id; 2) if a replica id +// is not a target in any pair, then the output on that replica is a tensor +// consists of 0(s) with the same shape as the input. +XlaOp CollectivePermute( + const XlaOp& operand, + const std::vector>& source_target_pairs); + +// Enqueues an operation that scatters the `source` array to the selected +// indices of each window. +XlaOp SelectAndScatter(const XlaOp& operand, const XlaComputation& select, + absl::Span window_dimensions, + absl::Span window_strides, Padding padding, + const XlaOp& source, const XlaOp& init_value, + const XlaComputation& scatter); + +// As SelectAndScatter(), but the padding is given in the format +// returned by MakePadding(). +XlaOp SelectAndScatterWithGeneralPadding( + const XlaOp& operand, const XlaComputation& select, + absl::Span window_dimensions, + absl::Span window_strides, + absl::Span> padding, const XlaOp& source, + const XlaOp& init_value, const XlaComputation& scatter); + +// Enqueues an abs instruction onto the computation. +XlaOp Abs(const XlaOp& operand); + +// Enqueues a atan2 instruction onto the computation. +XlaOp Atan2(const XlaOp& y, const XlaOp& x, + absl::Span broadcast_dimensions = {}); + +// Enqueues an exp instruction onto the computation. +XlaOp Exp(const XlaOp& operand); + +// Enqueues an expm1 instruction onto the computation. +XlaOp Expm1(const XlaOp& operand); + +// Enqueues a floor instruction onto the computation. +XlaOp Floor(const XlaOp& operand); + +// Enqueues a ceil instruction onto the computation. +XlaOp Ceil(const XlaOp& operand); + +// Enqueues a round instruction onto the computation, rounding to nearest even +// with half-way cases rounding away from zero. +XlaOp Round(const XlaOp& operand); + +// Enqueues an log instruction (natural logarithm) onto the computation. +XlaOp Log(const XlaOp& operand); + +// Enqueues an log1p instruction (log(x+1)) onto the computation. +XlaOp Log1p(const XlaOp& operand); + +// Enqueues a sign instruction onto the computation. +XlaOp Sign(const XlaOp& operand); + +// Enqueues a count leading zeros instruction onto the computation. +XlaOp Clz(const XlaOp& operand); + +// Enqueues a cosine instruction onto the computation. +XlaOp Cos(const XlaOp& operand); + +// Enqueues a sine instruction onto the computation. +XlaOp Sin(const XlaOp& operand); + +// Enqueues a tanh instruction onto the computation. +XlaOp Tanh(const XlaOp& operand); + +// Enqueues a real-part instruction onto the computation. +XlaOp Real(const XlaOp& operand); + +// Enqueues an imaginary-part instruction onto the computation. +XlaOp Imag(const XlaOp& operand); + +// Enqueues a lhs^rhs computation onto the computation. +XlaOp Pow(const XlaOp& lhs, const XlaOp& rhs, + absl::Span broadcast_dimensions = {}); + +// Enqueues an operator that tests if the operand's values are finite, i.e., +// not Inf or NaN. Defined only for floating-point types. Returns an array of +// booleans with the same shape where entries are true iff the corresponding +// entry was NaN. +XlaOp IsFinite(const XlaOp& operand); + +// Enqueues an iota operation onto the computation. +XlaOp Iota(XlaBuilder* builder, const Shape& shape, int64 iota_dimension); + +// Enqueues a rank-1 iota operation onto the computation. +XlaOp Iota(XlaBuilder* builder, PrimitiveType type, int64 size); + +// Enqueues a convert instruction onto the computation that changes the +// element type of the operand array to primitive_type. +XlaOp ConvertElementType(const XlaOp& operand, PrimitiveType new_element_type); + +// Enqueues a no-op instruction onto the computation that changes +// the element type of the operand array to primitive_type. The +// bit-widths of the source and destination element types must be +// identical. +XlaOp BitcastConvertType(const XlaOp& operand, PrimitiveType new_element_type); + +// Enqueues a negate instruction onto the computation. +XlaOp Neg(const XlaOp& operand); + +// Enqueues a transpose instruction onto the computation. +XlaOp Transpose(const XlaOp& operand, absl::Span permutation); + +// Enqueues a reverse instruction onto the computation. The order of the +// elements in the given dimensions is reversed (i.e., the element at index i +// is moved to index dimension_size - 1 - i). +XlaOp Rev(const XlaOp& operand, absl::Span dimensions); + +// Enqueues a sort (as increasing order) instruction onto the computation. +// If only keys are provided: +// * If the keys are an rank-1 tensor (an array), the result is a sorted array +// of keys, in ascending order. +// * If the keys have higher rank, the keys are sorted along the provided +// dimension. For example, for a rank-2 tensor (a matrix) of keys, a dimension +// value of 0 will indepenently sort every column, and a dimension value of 1 +// will independently sort each row. If no dimension number is provided, then +// the last dimension is chosen by default. +// +// If both keys and values are provided: +// * The keys and the values must tensors with the same dimensions. The +// element types of the tensors may be different. +// * The result is a tuple that consists of a sorted tensor of keys (along the +// provided dimension, as above) as the first element, and a tensor with their +// corresponding values as the second element. +XlaOp Sort(XlaOp keys, absl::optional values = absl::nullopt, + int64 dimension = -1); + +// Enqueues a clamp instruction onto the computation. +XlaOp Clamp(const XlaOp& min, const XlaOp& operand, const XlaOp& max); + +// Enqueues a map instruction onto the computation. +XlaOp Map(XlaBuilder* builder, absl::Span operands, + const XlaComputation& computation, absl::Span dimensions, + absl::Span static_operands = {}); + +// Enqueues a N(mu, sigma) random number generation instruction onto the +// computation. +XlaOp RngNormal(const XlaOp& mu, const XlaOp& sigma, const Shape& shape); + +// Enqueues a U(a, b) random number generation instruction onto the +// computation. Returns values in the semi-open interval [a, b). +XlaOp RngUniform(const XlaOp& a, const XlaOp& b, const Shape& shape); + +// Enqueues a while node onto the computation. +XlaOp While(const XlaComputation& condition, const XlaComputation& body, + const XlaOp& init); + +// Enqueues a conditional node onto the computation. +XlaOp Conditional(const XlaOp& predicate, const XlaOp& true_operand, + const XlaComputation& true_computation, + const XlaOp& false_operand, + const XlaComputation& false_computation); + +// Enqueues a ReducePrecision node onto the computation. +XlaOp ReducePrecision(const XlaOp& operand, const int exponent_bits, + const int mantissa_bits); + +// Enqueues a Gather node onto the computation. +XlaOp Gather(const XlaOp& input, const XlaOp& start_indices, + const GatherDimensionNumbers& dimension_numbers, + absl::Span slice_sizes); + +// Enqueues a Scatter node onto the computation. +XlaOp Scatter(const XlaOp& input, const XlaOp& scatter_indices, + const XlaOp& updates, const XlaComputation& update_computation, + const ScatterDimensionNumbers& dimension_numbers); + +// Enqueues a Send node onto the computation for device-to-device +// communication. This operation sends the given operand to +// a Recv instruction in a different computation that shares the same channel +// handle. +void Send(const XlaOp& operand, const ChannelHandle& handle); + +// Variant of Send which takes a token-shaped operand and produces a +// token-shaped value. Tokens are used for ordering side-effecting operations. +// TODO(b/110532604): Replace all uses of the non-token form with this variant. +XlaOp SendWithToken(const XlaOp& operand, const XlaOp& token, + const ChannelHandle& handle); + +// Enqueues a Recv node onto the computation for device-to-device +// communication. The data comes from a Send instruction in a different +// computation that shares the same channel handle and its shape must be the +// same as the given shape. +XlaOp Recv(XlaBuilder* builder, const Shape& shape, + const ChannelHandle& handle); + +// Variant of Recv which takes a token-shaped operand and produces a two-element +// tuple containing the data value and a token-shaped value. Tokens are used +// for ordering side-effecting operations. +// TODO(b/110532604): Replace all uses of the non-token form with this variant. +XlaOp RecvWithToken(const XlaOp& token, const Shape& shape, + const ChannelHandle& handle); + +// Enqueues a Send node which transfers data from the device to the host. The +// 'shape_with_layout' argument defines the layout of the data transferred; its +// shape must be compatible with the shape of the operand. The operand must be +// array-shaped. +// TODO(b/111544877): Support tuple shapes. +XlaOp SendToHost(const XlaOp& operand, const XlaOp& token, + const Shape& shape_with_layout, const ChannelHandle& handle); + +// Enqueues a Recv node which transfers data from the host to the device. The +// given shape must contain a layout and must be an array. +// TODO(b/111544877): Support tuple shapes. +XlaOp RecvFromHost(const XlaOp& token, const Shape& shape, + const ChannelHandle& handle); + +// Enqueues an operation (AfterAll) with no operands that produces a +// token-shaped value. Tokens are used for ordering side-effecting operations. +// This is a separate method from AfterAll to facility the removal of +// operand-less AfterAll instructions. +// TODO(b/110532604): Remove this function when all tokens are derived from a +// single token generated or passed into the entry computation. +XlaOp CreateToken(XlaBuilder* builder); + +// Enqueues an AfterAll instruction which produces a token-shaped value and +// takes a variadic number of token-shaped operands. The number of operands must +// be greater than zero. Used for joining tokens. +XlaOp AfterAll(XlaBuilder* builder, absl::Span tokens); + +// Normalizes operand across spatial and batch dimensions for each feature. +// +// Returns a tuple (normalized, batch_mean, batch_var) where `normalized` +// is the normalized result and batch_mean and batch_var are the mean and +// variance, respectively, across batch for the operand. +XlaOp BatchNormTraining(const XlaOp& operand, const XlaOp& scale, + const XlaOp& offset, float epsilon, + int64 feature_index); + +// Normalizes operand across spatial and batch dimensions for each feature. +// +// `BatchNormInference` is equivalent to calling `BatchNormTraining` without +// computing `mean` and `variance` for each batch inside the operation. It +// uses the input `mean` and `variance` instead as estimated values. The +// purpose of this op is to reduce latency in inference, hence the name +// `BatchNormInference`. +// +// The output has the same shape as `operand`, and contains the normalized +// values for each batch. +XlaOp BatchNormInference(const XlaOp& operand, const XlaOp& scale, + const XlaOp& offset, const XlaOp& mean, + const XlaOp& variance, float epsilon, + int64 feature_index); + +// Calculates the gradients of a batch norm op. +// +// The inputs `batch_mean` and `batch_var` represent the mean and variance +// across the batch. +// +// Returns a tuple of three elements: +// - grad_operand: Gradient with respect to input `operand` +// - grad_offset: Gradient with respect to input `offset` +// - grad_scale: Gradient with respect to input `scale` +XlaOp BatchNormGrad(const XlaOp& operand, const XlaOp& scale, + const XlaOp& batch_mean, const XlaOp& batch_var, + const XlaOp& grad_output, float epsilon, + int64 feature_index); + +// Implementation details below this point. + +template +XlaOp XlaBuilder::ConstantR0(NativeT value) { + return ConstantLiteral(LiteralUtil::CreateR0(value)); +} + +template +XlaOp XlaBuilder::ConstantR1(absl::Span values) { + return ConstantLiteral(LiteralUtil::CreateR1(values)); +} + +template +XlaOp XlaBuilder::ConstantR1(int64 length, NativeT value) { + Literal literal(ShapeUtil::MakeShape( + primitive_util::NativeToPrimitiveType(), {length})); + literal.PopulateWithValue(value); + return ConstantLiteral(literal); +} + +inline XlaOp XlaBuilder::ConstantR1(const tensorflow::core::Bitmap& values) { + return ConstantLiteral(LiteralUtil::CreateR1(values)); +} + +template +XlaOp XlaBuilder::ConstantR2( + std::initializer_list> values) { + return ConstantLiteral(LiteralUtil::CreateR2(values)); +} + +template +XlaOp XlaBuilder::ConstantFromArrayWithLayout(const Array& values, + const Layout& layout) { + return ConstantLiteral( + LiteralUtil::CreateFromArrayWithLayout(values, layout)); +} + +template +XlaOp XlaBuilder::ConstantFromArray(const Array& values) { + return ConstantLiteral(LiteralUtil::CreateFromArray(values)); +} + +template +XlaOp XlaBuilder::ConstantR2FromArray2DWithLayout( + const Array2D& values, const Layout& layout) { + return ConstantLiteral( + LiteralUtil::CreateFromArrayWithLayout(values, layout)); +} + +template +XlaOp XlaBuilder::ConstantR2FromArray2D(const Array2D& values) { + return ConstantLiteral(LiteralUtil::CreateR2FromArray2D(values)); +} + +template +XlaOp XlaBuilder::ConstantR3FromArray3DWithLayout( + const Array3D& values, const Layout& layout) { + return ConstantLiteral( + LiteralUtil::CreateR3FromArray3DWithLayout(values, layout)); +} + +template +XlaOp XlaBuilder::ConstantR3FromArray3D(const Array3D& values) { + return ConstantFromArray(values); +} + +template +XlaOp XlaBuilder::ConstantR4FromArray4DWithLayout( + const Array4D& values, const Layout& layout) { + return ConstantFromArrayWithLayout(values, layout); +} + +template +XlaOp XlaBuilder::ConstantR4FromArray4D(const Array4D& values) { + return ConstantFromArray(values); +} + +// Free function template implementations. + +template +XlaOp ConstantR0(XlaBuilder* builder, NativeT value) { + return ConstantLiteral(builder, LiteralUtil::CreateR0(value)); +} + +template +XlaOp ConstantR1(XlaBuilder* builder, absl::Span values) { + return ConstantLiteral(builder, LiteralUtil::CreateR1(values)); +} + +template +XlaOp ConstantR1(XlaBuilder* builder, int64 length, NativeT value) { + Literal literal(ShapeUtil::MakeShape( + primitive_util::NativeToPrimitiveType(), {length})); + literal.PopulateWithValue(value); + return ConstantLiteral(builder, literal); +} + +inline XlaOp ConstantR1(XlaBuilder* builder, + const tensorflow::core::Bitmap& values) { + return ConstantLiteral(builder, LiteralUtil::CreateR1(values)); +} + +template +XlaOp ConstantR2(XlaBuilder* builder, + std::initializer_list> values) { + return ConstantLiteral(builder, LiteralUtil::CreateR2(values)); +} + +template +XlaOp ConstantFromArrayWithLayout(XlaBuilder* builder, + const Array& values, + const Layout& layout) { + return ConstantLiteral( + builder, LiteralUtil::CreateFromArrayWithLayout(values, layout)); +} + +template +XlaOp ConstantFromArray(XlaBuilder* builder, const Array& values) { + return ConstantLiteral(builder, + LiteralUtil::CreateFromArray(values)); +} + +template +XlaOp ConstantR2FromArray2DWithLayout(XlaBuilder* builder, + const Array2D& values, + const Layout& layout) { + return ConstantLiteral( + builder, LiteralUtil::CreateFromArrayWithLayout(values, layout)); +} + +template +XlaOp ConstantR2FromArray2D(XlaBuilder* builder, + const Array2D& values) { + return ConstantLiteral(builder, + LiteralUtil::CreateR2FromArray2D(values)); +} + +template +XlaOp ConstantR3FromArray3DWithLayout(XlaBuilder* builder, + const Array3D& values, + const Layout& layout) { + return ConstantLiteral( + builder, + LiteralUtil::CreateR3FromArray3DWithLayout(values, layout)); +} + +template +XlaOp ConstantR3FromArray3D(XlaBuilder* builder, + const Array3D& values) { + return ConstantFromArray(builder, values); +} + +template +XlaOp ConstantR4FromArray4DWithLayout(XlaBuilder* builder, + const Array4D& values, + const Layout& layout) { + return ConstantFromArrayWithLayout(builder, values, layout); +} + +template +XlaOp ConstantR4FromArray4D(XlaBuilder* builder, + const Array4D& values) { + return ConstantFromArray(builder, values); +} + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_CLIENT_XLA_BUILDER_H_ diff --git a/linux/include/tensorflow/compiler/xla/client/xla_computation.h b/linux/include/tensorflow/compiler/xla/client/xla_computation.h new file mode 100644 index 000000000..71598ef8b --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/client/xla_computation.h @@ -0,0 +1,67 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_CLIENT_XLA_COMPUTATION_H_ +#define TENSORFLOW_COMPILER_XLA_CLIENT_XLA_COMPUTATION_H_ + +#include + +#include "tensorflow/compiler/xla/service/hlo.pb.h" +#include "tensorflow/compiler/xla/status_macros.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" + +namespace xla { + +// The computation graph that the user builds up with the XlaBuilder. +class XlaComputation { + public: + XlaComputation() : unique_id_(-1) {} + XlaComputation(const HloModuleProto& proto) + : unique_id_(proto.id()), proto_(proto) {} + + ~XlaComputation() {} + + XlaComputation(const XlaComputation&) = delete; + XlaComputation& operator=(const XlaComputation&) = delete; + + XlaComputation(XlaComputation&& from) = default; + + XlaComputation& operator=(XlaComputation&& from) = default; + + // Returns the "program shape" (parameter and return shapes) for this + // computation. + StatusOr GetProgramShape() const; + + const HloModuleProto& proto() const { return proto_; } + + // Requests that we snapshot the computation into a serializable protocol + // buffer form. + StatusOr> Snapshot() const; + + // Returns true if this object is a null Computation. + bool IsNull() const { return unique_id_ == -1; } + + private: + XlaComputation(const int64 unique_id) : unique_id_(unique_id) {} + HloModuleProto* mutable_proto() { return &proto_; } + friend class XlaBuilder; + + int64 unique_id_; + HloModuleProto proto_; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_CLIENT_XLA_COMPUTATION_H_ diff --git a/linux/include/tensorflow/compiler/xla/executable_run_options.h b/linux/include/tensorflow/compiler/xla/executable_run_options.h new file mode 100644 index 000000000..ba3217f31 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/executable_run_options.h @@ -0,0 +1,105 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_EXECUTABLE_RUN_OPTIONS_H_ +#define TENSORFLOW_COMPILER_XLA_EXECUTABLE_RUN_OPTIONS_H_ + +// Pulls in the ::stream_executor -> ::xla::se namespace alias. +#include "tensorflow/compiler/xla/types.h" + +// These classes are forward declared so that ExecutableRunOptions can be linked +// into an XLA-compiled binary without having to link all of the pointed-to +// objects (e.g., for an ahead-of-time compiled CPU binary, the gpu tools don't +// need to be linked). +namespace stream_executor { +class Stream; +class Platform; +} // namespace stream_executor + +namespace tensorflow { +namespace thread { +class ThreadPool; +} // namespace thread +} // namespace tensorflow + +namespace Eigen { +struct ThreadPoolDevice; +} // namespace Eigen + +namespace xla { + +class DeviceMemoryAllocator; +class DeviceAssignment; +class ExecutionProfile; + +// Class containing options for running a LocalExecutable. +class ExecutableRunOptions { + public: + // Specifies the allocator to use during execution. + ExecutableRunOptions& set_allocator(DeviceMemoryAllocator* allocator); + DeviceMemoryAllocator* allocator() const; + + // If set, this is the device to run the computation on. Valid device_ordinal + // values are: 0 to # of devices - 1. These values are identical to the device + // ordinal values used by StreamExecutor. The device must be of the same type + // as the executable was compiled for. A value of -1 indicates this option has + // not been set. + ExecutableRunOptions& set_device_ordinal(int device_ordinal); + int device_ordinal() const; + + // If set, this is the stream to run the computation on. The platform of the + // stream must match the platform the executable was built for. A value of + // nullptr indicates the option has not been set. + ExecutableRunOptions& set_stream(stream_executor::Stream* stream); + stream_executor::Stream* stream() const; + + // If set, this is the stream to perform any pre-computation transfers on. + // The platform of the stream must match the platform the executable was + // built for. A value of nullptr indicates the option has not been set. + ExecutableRunOptions& set_host_to_device_stream( + stream_executor::Stream* stream); + stream_executor::Stream* host_to_device_stream() const; + + // Sets the thread pool device on which to run Eigen subcomputations. + // Does not take ownership. + ExecutableRunOptions& set_intra_op_thread_pool( + const Eigen::ThreadPoolDevice* intra_op_thread_pool); + const Eigen::ThreadPoolDevice* intra_op_thread_pool() const; + + // If set, profiling information is written to 'profile'. + ExecutionProfile* execution_profile() const; + ExecutableRunOptions& set_execution_profile(ExecutionProfile* profile); + + ExecutableRunOptions& set_device_assignment( + DeviceAssignment* device_assignment); + const DeviceAssignment* device_assignment() const; + + ExecutableRunOptions& set_rng_seed(int rng_seed); + int rng_seed() const; + + private: + DeviceMemoryAllocator* allocator_ = nullptr; + int device_ordinal_ = -1; + DeviceAssignment* device_assignment_ = nullptr; + stream_executor::Stream* stream_ = nullptr; + const Eigen::ThreadPoolDevice* intra_op_thread_pool_ = nullptr; + ExecutionProfile* execution_profile_ = nullptr; + int rng_seed_ = 0; + stream_executor::Stream* host_to_device_stream_ = nullptr; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_EXECUTABLE_RUN_OPTIONS_H_ diff --git a/linux/include/tensorflow/compiler/xla/execution_options_util.h b/linux/include/tensorflow/compiler/xla/execution_options_util.h new file mode 100644 index 000000000..a8ca27ec8 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/execution_options_util.h @@ -0,0 +1,29 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_EXECUTION_OPTIONS_UTIL_H_ +#define TENSORFLOW_COMPILER_XLA_EXECUTION_OPTIONS_UTIL_H_ + +#include "tensorflow/compiler/xla/xla.pb.h" + +namespace xla { + +// Create a default ExecutionOptions proto; this proto has its debug options +// popupated to the default values taken from flags. +ExecutionOptions CreateDefaultExecutionOptions(); + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_EXECUTION_OPTIONS_UTIL_H_ diff --git a/linux/include/tensorflow/compiler/xla/index_util.h b/linux/include/tensorflow/compiler/xla/index_util.h new file mode 100644 index 000000000..2979cf87d --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/index_util.h @@ -0,0 +1,88 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// Utility functions related to layouts of Shapes. + +#ifndef TENSORFLOW_COMPILER_XLA_INDEX_UTIL_H_ +#define TENSORFLOW_COMPILER_XLA_INDEX_UTIL_H_ + +#include + +#include "absl/types/span.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/platform/macros.h" + +namespace xla { + +// Namespaced collection of (static) utilities related to indexing into +// multidimensional arrays. +class IndexUtil { + public: + // Converts a multidimensional index (eg {x, y, z}) into a linear index based + // on the shape and its layout. The first index in the multi_index is + // dimension 0. + static int64 MultidimensionalIndexToLinearIndex( + const Shape& shape, absl::Span multi_index); + + // Converts a linear index into multidimensional index (eg {x, y, z}) based on + // the shape and its layout. The first index in the returned multidimensional + // index is dimension 0. + static std::vector LinearIndexToMultidimensionalIndex( + const Shape& shape, int64 linear_index); + + // Bumps a sequence of indices; e.g. {0,0,0,0} up by one index value; e.g. to + // {0,0,0,1}. This is akin to std::next_permutation. If the index hits a limit + // for the provided shape, the next most significant index is bumped, in a + // counting-up process. + // + // E.g. for shape f32[2,3] + // {0,0}=>{0,1} + // {0,1}=>{0,2} + // {0,2}=>{1,0} + // etc. + // + // This is useful for traversing the indices in a literal. + // + // Returns true iff the indices were successfully bumped; false if we've hit + // the limit where it can no longer be bumped in-bounds. + static bool BumpIndices(const Shape& shape, absl::Span indices); + + // Calculates the stride size (in number of elements, not byte size) of a + // given logical shape dimension (from 0 to rank-1). If available, padded + // dimensions are used. + // Example: + // GetDimensionStride(F32[5,8,10,4]{3,2,1,0}, 1) == + // sizeof(dimension(3)) * sizeof(dimension(2)) == 4 * 10 + static int64 GetDimensionStride(const Shape& shape, int64 dimension); + + // Returns true iff the given multi-index is contained in the bounds for the + // shape. + static bool IndexInBounds(const Shape& shape, absl::Span index); + + // Compares the given indices in lexicographic order. lhs[0] and rhs[0] are + // compared first, and lhs[rank-1] and rhs[rank-1] last. If lhs is larger, + // then -1 is returned. If rhs is larger, then 1 is returned. Otherwise, 0 is + // returned. + static int CompareIndices(absl::Span lhs, + absl::Span rhs); + + private: + TF_DISALLOW_COPY_AND_ASSIGN(IndexUtil); +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_INDEX_UTIL_H_ diff --git a/linux/include/tensorflow/compiler/xla/iterator_util.h b/linux/include/tensorflow/compiler/xla/iterator_util.h new file mode 100644 index 000000000..3a3ee21e7 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/iterator_util.h @@ -0,0 +1,98 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_ITERATOR_UTIL_H_ +#define TENSORFLOW_COMPILER_XLA_ITERATOR_UTIL_H_ + +#include +#include + +namespace xla { + +// UnwrappingIterator is a transforming iterator that calls get() on the +// elements it returns. +// +// Together with tensorflow::gtl::iterator_range, this lets classes which +// contain a collection of smart pointers expose a view of raw pointers to +// consumers. For example: +// +// class MyContainer { +// public: +// tensorflow::gtl::iterator_range< +// UnwrappingIterator>::iterator>> +// things() { +// return {MakeUnwrappingIterator(things_.begin()), +// MakeUnwrappingIterator(things_.end())}; +// } +// +// tensorflow::gtl::iterator_range>::const_iterator>> +// things() const { +// return {MakeUnwrappingIterator(things_.begin()), +// MakeUnwrappingIterator(things_.end())}; +// } +// +// private: +// std::vector> things_; +// }; +// +// MyContainer container = ...; +// for (Thing* t : container.things()) { +// ... +// } +// +// For simplicity, UnwrappingIterator is currently unconditionally an +// input_iterator -- it doesn't inherit any superpowers NestedIterator may have. +template +class UnwrappingIterator + : public std::iterator()->get())> { + private: + NestedIter iter_; + + public: + explicit UnwrappingIterator(NestedIter iter) : iter_(std::move(iter)) {} + + auto operator*() -> decltype(iter_->get()) { return iter_->get(); } + auto operator-> () -> decltype(iter_->get()) { return iter_->get(); } + UnwrappingIterator& operator++() { + ++iter_; + return *this; + } + UnwrappingIterator operator++(int) { + UnwrappingIterator temp(iter_); + operator++(); + return temp; + } + + friend bool operator==(const UnwrappingIterator& a, + const UnwrappingIterator& b) { + return a.iter_ == b.iter_; + } + + friend bool operator!=(const UnwrappingIterator& a, + const UnwrappingIterator& b) { + return !(a == b); + } +}; + +template +UnwrappingIterator MakeUnwrappingIterator(NestedIter iter) { + return UnwrappingIterator(std::move(iter)); +} + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_ITERATOR_UTIL_H_ diff --git a/linux/include/tensorflow/compiler/xla/layout_util.h b/linux/include/tensorflow/compiler/xla/layout_util.h new file mode 100644 index 000000000..b78883c2d --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/layout_util.h @@ -0,0 +1,211 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// Utility functions related to layouts of Shapes. + +#ifndef TENSORFLOW_COMPILER_XLA_LAYOUT_UTIL_H_ +#define TENSORFLOW_COMPILER_XLA_LAYOUT_UTIL_H_ + +#include + +#include "absl/types/span.h" +#include "tensorflow/compiler/xla/status.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/types.h" + +namespace xla { + +// Namespaced collection of (static) Layout utilities. +class LayoutUtil { + public: + // Creates a layout with the given minor-to-major dimension order. (This is a + // convenience function for protobuf construction.) + static Layout MakeLayout(absl::Span minor_to_major); + + // Similar to MakeLayout, but take indices in reverse order. + static Layout MakeLayoutFromMajorToMinor( + absl::Span major_to_minor); + + // Creates a sparse layout with the given maximum number of elements. (This is + // a convenience function for protobuf construction.) + static Layout MakeSparseLayout(int64 max_sparse_elements); + + // Returns default layout for the given shape. + static Layout GetDefaultLayoutForShape(const Shape& shape); + + // Helper functions that create default layouts for various ranks. + static Layout GetDefaultLayoutForRank(int64 rank); + static Layout GetDefaultLayoutForR2(); + static Layout GetDefaultLayoutForR3(); + static Layout GetDefaultLayoutForR4(); + + // Sets the default layout on the Shape. + static void SetToDefaultLayout(Shape* shape); + + // Returns a shape with the same dimensions as `shape` but with the default + // layout. + static Shape GetWithDefaultLayout(const Shape& shape); + + // Sets the layouts of all Shapes within the given ProgramShape to the + // default. + static void SetToDefaultLayout(ProgramShape* program_shape); + + // Validates that the layout within the given shape is correct. + static Status ValidateLayoutInShape(const Shape& shape); + + // Validates that the provided layout satisfies invariants for the given + // shape. + static Status ValidateLayoutForShape(const Layout& layout, + const Shape& shape); + + // Clears the layout in the given Shape. After this function is called, + // HasLayout will return false for the shape. + static void ClearLayout(Shape* shape); + + // Clears the layout on all Shapes within the given ProgramShape. + static void ClearLayout(ProgramShape* program_shape); + + // Returns whether the given Shape is an array and has a dense format layout. + static bool IsDenseArray(const Shape& shape); + + // Returns whether the given Layout has a dense format. + static bool IsDense(const Layout& layout); + + // Returns whether the layout is monotonic and dim 0 is minor in the layout. + // * R0 and R1: this is always trivially true. + // * R2+: equivalent to column-major. Dimension 0 is the minor, dimension 1 is + // more major, and so on until dimension N-1 which is the major. + static bool IsMonotonicWithDim0Minor(const Layout& layout); + + // Returns whether the layout is monotonic and dim 0 is major in the layout. + // * R0 and R1: this is always trivially true. + // * R2+: equivalent to row-major. Dimension 0 is the major, dimension 1 is + // more minor, and so on until dimension N-1 which is the minor. + static bool IsMonotonicWithDim0Major(const Layout& layout); + + // Returns whether the layout of the given shape has padding (a + // padded_dimension value in Layout is greater than the corresponding + // dimension size). + static bool IsPadded(const Shape& shape); + + // Returns the padded_dimensions array for the given Shape. Requires that the + // shape is an array and has a dense layout. + static absl::Span PaddedDimensions(const Shape& shape); + + // Returns the given index of the padded_dimensions array for the given Shape. + // Requires that the shape is an array and has a dense layout. + static int64 PaddedDimension(const Shape& shape, int64 index); + + // Returns the padding_value for the given Shape. Requires that the shape is + // an array and has a dense layout. + static PaddingValue GetPaddingValue(const Shape& shape); + + // Returns whether the given Shape is an array (i.e. not a tuple) and has a + // sparse format layout. + static bool IsSparseArray(const Shape& shape); + + // Returns whether the given Layout has a sparse format. + static bool IsSparse(const Layout& layout); + + // Returns the maximum number of elements that can be stored in a sparse + // layout. + static int64 MaxSparseElements(const Layout& layout); + + // Returns whether the given shape has a layout. For tuple shapes, true is + // returned only if all elements have layouts. + static bool HasLayout(const Shape& shape); + + // Returns whether all Shapes within the given ProgramShape have layouts. + static bool HasLayout(const ProgramShape& program_shape); + + // Returns whether lhs and rhs are identical. + static bool Equal(const Layout& lhs, const Layout& rhs); + + // Returns the minor_to_major array for the given Shape. Requires that the + // shape is an array and has a dense layout. + static absl::Span MinorToMajor(const Shape& shape); + static absl::Span MinorToMajor(const Layout& layout); + + // Major(0) is the most major logical dimension number, Major(1) is the + // second-most-major logical dimension number and so on. + // + // This can be used to translate physical dimension numbers to logical + // dimension numbers. Assume that we are numbering the physical dimensions so + // that the most major physical dimension has physical dimension number 0 and + // so on. Then a physical dimension number p corresponds to the logical + // dimension number Major(p). So this function could also be called + // PhysicalToLogical(). + // + // As an example, consider physical dimension number 0, which by definition is + // the most major. Then Major(0) is the most major logical dimension, so Major + // maps the physical dimension number 0 to the most major logical dimension + // number Major(0). + static int64 Major(const Layout& layout, int64 physical_dimension_number); + + // Minor(0) is the most minor logical dimension number, minor(1) is the + // second-most-minor logical dimension number and so on. + static int64 Minor(const Layout& layout, int64 physical_dimension_number); + + // Returns the inverse mapping of the Major() function. More precisely, return + // a vector v such that if l == Major(p), then v[l] == p. + // + // This can be used to translate logical dimension numbers into physical + // dimension numbers. Assume that we are numbering the physical dimensions so + // that the most major physical dimension has physical dimension number 0 and + // so on. Then a logical dimension number l corresponds to the physical + // dimension number MakeLogicalToPhysical(layout)[l]. + // + // As an example, consider physical dimension number 0, which by definition is + // the most major. Then l := Major(0) is the most major logical dimension. If + // v is the vector returned from this function, then v[l] == 0. So v maps the + // most major logical dimension l to the physical dimension number 0. + static std::vector MakeLogicalToPhysical(const Layout& layout); + + // Returns a human-readable string that represents the given layout. + static string HumanString(const Layout& layout); + + // Copies the layout from 'src' to 'dst'. Recursively copies layouts of + // tuples. 'src' and 'dst' need not be compatible but the two shapes must + // have the same tuple structure (if any) and arrays must have the same + // rank. within the shapes must have the same number of dimensions. + static Status CopyLayoutBetweenShapes(const Shape& src, Shape* dst); + + // Returns true if the layouts of lhs and rhs are equal, false + // otherwise. Recursively compares layouts of tuples. + // + // lhs and rhs need not be compatible to have the same layout but the two + // shapes must have the same tuple structure (if any) and arrays must have the + // same rank. Element type is ignored. + static bool LayoutsInShapesEqual(const Shape& lhs, const Shape& rhs); + + // Returns whether the given dimensions are consecutive in the given layout, + // not necessarily in the order given. + static bool AreDimensionsConsecutive(const Layout& layout, + absl::Span dims); + + // Compute a hash for `layout`. + static size_t Hash(const Layout& layout); + + private: + TF_DISALLOW_COPY_AND_ASSIGN(LayoutUtil); +}; + +std::ostream& operator<<(std::ostream& out, const Layout& layout); + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_LAYOUT_UTIL_H_ diff --git a/linux/include/tensorflow/compiler/xla/legacy_flags/debug_options_flags.h b/linux/include/tensorflow/compiler/xla/legacy_flags/debug_options_flags.h new file mode 100644 index 000000000..b53157f59 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/legacy_flags/debug_options_flags.h @@ -0,0 +1,38 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_LEGACY_FLAGS_DEBUG_OPTIONS_FLAGS_H_ +#define TENSORFLOW_COMPILER_XLA_LEGACY_FLAGS_DEBUG_OPTIONS_FLAGS_H_ + +#include + +#include "tensorflow/compiler/xla/xla.pb.h" +#include "tensorflow/core/util/command_line_flags.h" + +namespace xla { +namespace legacy_flags { + +// Appends flag definitions for debug options to flag_list. +void AppendDebugOptionsFlags(std::vector* flag_list); + +// Fetches a DebugOptions proto message from flags provided to the program. +// Flags must be registered with the flags parser using AppendDebugOptionsFlags +// first. +xla::DebugOptions GetDebugOptionsFromFlags(); + +} // namespace legacy_flags +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_LEGACY_FLAGS_DEBUG_OPTIONS_FLAGS_H_ diff --git a/linux/include/tensorflow/compiler/xla/legacy_flags/debug_options_parsers.h b/linux/include/tensorflow/compiler/xla/legacy_flags/debug_options_parsers.h new file mode 100644 index 000000000..ee7eb019c --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/legacy_flags/debug_options_parsers.h @@ -0,0 +1,147 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_LEGACY_FLAGS_DEBUG_OPTIONS_PARSERS_H_ +#define TENSORFLOW_COMPILER_XLA_LEGACY_FLAGS_DEBUG_OPTIONS_PARSERS_H_ + +#include +#include "absl/strings/numbers.h" +#include "absl/strings/str_split.h" +#include "tensorflow/compiler/xla/service/hlo_opcode.h" +#include "tensorflow/compiler/xla/xla.pb.h" + +namespace xla { +namespace legacy_flags { +namespace impl { + +template +void parse_xla_backend_extra_options(T* extra_options_map, + string comma_separated_values) { + std::vector extra_options_parts = + absl::StrSplit(comma_separated_values, ','); + + // The flag contains a comma-separated list of options; some options + // have arguments following "=", some don't. + for (const auto& part : extra_options_parts) { + size_t eq_pos = part.find_first_of('='); + if (eq_pos == string::npos) { + (*extra_options_map)[part] = ""; + } else { + string value = ""; + if (eq_pos + 1 < part.size()) { + value = part.substr(eq_pos + 1); + } + (*extra_options_map)[part.substr(0, eq_pos)] = value; + } + } +} + +// The --xla_reduce_precision option has the format "LOCATION=E,M:OPS;NAME", +// where LOCATION is an HloReducePrecisionOptions::location, E and M are +// integers for the exponent and matissa bit counts respectively, and OPS and +// NAMES are comma-separated of the operation types and names to which to +// attach the reduce-precision operations. The OPS values are matches to the +// strings produced by HloOpcodeString, while the NAME values are arbitrary +// strings subject to the requirements that they not contain any of "=,:;". +// The NAME string (with its preceding semicolon) is optional. +inline bool parse_xla_reduce_precision_option( + HloReducePrecisionOptions* options, string option_string) { + // Split off "LOCATION" from remainder of string. + std::vector eq_split = absl::StrSplit(option_string, '='); + if (eq_split.size() != 2) { + return false; + } + string& location = eq_split[0]; + if (location == "OP_INPUTS") { + options->set_location(HloReducePrecisionOptions::OP_INPUTS); + } else if (location == "OP_OUTPUTS") { + options->set_location(HloReducePrecisionOptions::OP_OUTPUTS); + } else if (location == "UNFUSED_OP_OUTPUTS") { + options->set_location(HloReducePrecisionOptions::UNFUSED_OP_OUTPUTS); + } else if (location == "FUSION_INPUTS_BY_CONTENT") { + options->set_location(HloReducePrecisionOptions::FUSION_INPUTS_BY_CONTENT); + } else if (location == "FUSION_OUTPUTS_BY_CONTENT") { + options->set_location(HloReducePrecisionOptions::FUSION_OUTPUTS_BY_CONTENT); + } else { + return false; + } + + // Split off "E,M" from remainder of string. + std::vector colon_split = absl::StrSplit(eq_split[1], ':'); + if (colon_split.size() != 2) { + return false; + } + + // Split E and M, and parse. + std::vector bitsizes; + for (const auto& s : absl::StrSplit(colon_split[0], ',')) { + bitsizes.emplace_back(); + if (!absl::SimpleAtoi(s, &bitsizes.back())) { + return false; + } + } + options->set_exponent_bits(bitsizes[0]); + options->set_mantissa_bits(bitsizes[1]); + + // Split off OPS comma-separated list from remainder of string, if the + // remainder exists. + std::vector semicolon_split = absl::StrSplit(colon_split[1], ';'); + if (semicolon_split.size() > 2) { + return false; + } + // The opcode values are either 'all' (meaning all opcodes), or matches to + // the strings returned by HloOpcodeString. An empty string is also + // interpreted as 'all', for convenience. Note that 'all' may not be part + // of a comma-separated list; it must stand alone. + string& opcode_string = semicolon_split[0]; + if (opcode_string == "" || opcode_string == "all") { + for (int i = 0; i < HloOpcodeCount(); i++) { + options->add_opcodes_to_suffix(i); + } + } else { + std::vector opcodes = absl::StrSplit(opcode_string, ','); + for (const string& opcode : opcodes) { + bool found = false; + for (int i = 0; i < HloOpcodeCount(); i++) { + if (opcode == HloOpcodeString(static_cast(i))) { + options->add_opcodes_to_suffix(i); + found = true; + break; + } + } + if (!found) { + return false; + } + } + } + + // Process the NAMES string, if it exists. + if (semicolon_split.size() == 2) { + std::vector opnames = absl::StrSplit(semicolon_split[1], ','); + for (const string& opname : opnames) { + if (opname.length() > 0) { + options->add_opname_substrings_to_suffix(opname); + } + } + } + + return true; +} + +} // namespace impl +} // namespace legacy_flags +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_LEGACY_FLAGS_DEBUG_OPTIONS_PARSERS_H_ diff --git a/linux/include/tensorflow/compiler/xla/legacy_flags/parse_flags_from_env.h b/linux/include/tensorflow/compiler/xla/legacy_flags/parse_flags_from_env.h new file mode 100644 index 000000000..b54482ad2 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/legacy_flags/parse_flags_from_env.h @@ -0,0 +1,66 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_LEGACY_FLAGS_PARSE_FLAGS_FROM_ENV_H_ +#define TENSORFLOW_COMPILER_XLA_LEGACY_FLAGS_PARSE_FLAGS_FROM_ENV_H_ + +// This module exports ParseFlagsFromEnv(), which allows other modules to parse +// flags from the environtment variable TF_XLA_FLAGS, or (if the first +// non-whitespace in the variable value is not '-'), a file named by that +// environment variable. The accepted syntax is that flags arguments are of +// the form --flag=value or (for boolean flags) --flag, and are whitespace +// separated. The may be one of: +// - +// in which case the effective value is the string itself +// - in which case the effective value is the +// string with the single-quotes removed +// - in which case the effective value if the +// string with the double-quotes removed, and escaped sequences of +// replaced by . +// +// Flags values inconsistent with the type of the flag will be rejected by the +// flag parser. +// +// Examples: +// TF_XLA_FLAGS="--foo=bar --wombat='value with a space'" +// +// TF_XLA_FLAGS=/tmp/flagfile +// where /tmp/flagfile might contain +// --some_flag="This is a string containing a \" and a '." +// --another_flag=wombats + +#include + +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/core/platform/types.h" +#include "tensorflow/core/util/command_line_flags.h" + +namespace xla { +namespace legacy_flags { + +// Call tensorflow::Flags::Parse(argc, argv, flag_list) against any as yet +// unrecognized flags passed in from the environment, and return its +// return value. +bool ParseFlagsFromEnv(const std::vector& flag_list); + +// Used only for testing. Not to be used by clients. +void ResetFlagsFromEnvForTesting(int** pargc, std::vector** pargv); + +} // namespace legacy_flags +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_LEGACY_FLAGS_PARSE_FLAGS_FROM_ENV_H_ diff --git a/linux/include/tensorflow/compiler/xla/literal.h b/linux/include/tensorflow/compiler/xla/literal.h new file mode 100644 index 000000000..3cd3541fe --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/literal.h @@ -0,0 +1,1167 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_LITERAL_H_ +#define TENSORFLOW_COMPILER_XLA_LITERAL_H_ + +#include +#include +#include +#include +#include +#include +#include +#include + +#include "absl/memory/memory.h" +#include "absl/strings/string_view.h" +#include "absl/types/span.h" +#include "tensorflow/compiler/xla/array2d.h" +#include "tensorflow/compiler/xla/array3d.h" +#include "tensorflow/compiler/xla/array4d.h" +#include "tensorflow/compiler/xla/index_util.h" +#include "tensorflow/compiler/xla/layout_util.h" +#include "tensorflow/compiler/xla/primitive_util.h" +#include "tensorflow/compiler/xla/shape_util.h" +#include "tensorflow/compiler/xla/sparse_index_array.h" +#include "tensorflow/compiler/xla/status_macros.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/compiler/xla/util.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/lib/core/bitmap.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/platform/logging.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/protobuf.h" +#include "tensorflow/core/platform/types.h" + +namespace xla { + +// Forward declare Literal and LiteralSlice class to be used by the creation +// methods in the base class. +class Literal; +class LiteralSlice; + +// Abstract base class for literals. +class LiteralBase { + public: + virtual ~LiteralBase() = 0; + + // Literals are equal if they have compatible shapes and the same data + // values. Layout is not compared. + bool operator==(const LiteralBase& other) const; + bool operator!=(const LiteralBase& other) const { return !(*this == other); } + + // Returns the shape of the literal. + const Shape& shape() const { return root_piece().subshape(); } + + // Serialize to proto. + LiteralProto ToProto() const; + + // Returns a Span of the array for this literal for the given NativeT + // (e.g., float). CHECKs if the subshape of the literal at the given + // ShapeIndex is not array. See primitive_util.h for the mapping from XLA type + // to native type. + template + absl::Span data(const ShapeIndex& shape_index = {}) const; + + // Returns a const pointer to the sparse index array. Returns nullptr if the + // literal is not a sparse array. + const SparseIndexArray* sparse_indices( + const ShapeIndex& shape_index = {}) const; + + // Returns a const pointer to (or size of) the underlying buffer holding the + // array at the given shape index. CHECKs if the subshape of the literal at + // the given ShapeIndex is not array. + const void* untyped_data(const ShapeIndex& shape_index = {}) const; + int64 size_bytes(const ShapeIndex& shape_index = {}) const; + + // Returns this literal's data as a string. This literal must be a rank-1 U8 + // array. + string GetR1U8AsString() const; + + // Returns a string representation of the literal value. + // Warning: this function can take minutes for multi-million element Literals. + string ToString(bool print_layout = false) const; + + // Gets an element in the literal at the given index. The multi_index is + // CHECKed against the dimension sizes. + template + NativeT Get(absl::Span multi_index, + const ShapeIndex& shape_index) const; + // Overloads of Get for array literals. CHECKs if the literal is not + // array-shaped and dense. + template + NativeT Get(absl::Span multi_index) const; + + // Returns the element value at index (0, ..., 0), however many zeroes are + // required for that index. + template + NativeT GetFirstElement() const; + + // As Get(), but determines the correct type and converts the value + // into text. + string GetAsString(absl::Span multi_index, + const ShapeIndex& shape_index = {}) const; + // As GetSparseElement(), but determines the correct type and converts the + // value into text. + string GetSparseElementAsString(int64 sparse_element_number, + const ShapeIndex& shape_index = {}) const; + // As Get(), but determines the correct type and converts the value into + // int64. This literal must be an array. + StatusOr GetIntegralAsS64(absl::Span multi_index) const; + + // Returns the multi-index of the element in a sparse literal at the given + // sparse element number. The sparse element number is the position with in + // the sparse array's list of (index, value) pairs, and is checked against the + // total number of (index, value) pairs in the sparse array. + absl::Span GetSparseIndex( + int64 sparse_element_number, const ShapeIndex& shape_index = {}) const; + + // Returns the value of the element in a sparse literal at the given sparse + // element number. The sparse element number is the position with in the + // sparse array's list of (index, value) pairs, and is checked against the + // total number of (index, value) pairs in the sparse array. + template + NativeT GetSparseElement(int64 sparse_element_number, + const ShapeIndex& shape_index = {}) const; + + // Invokes the "per cell" callback for each element in the provided + // literal with the element's indices and a string representation of + // the element's value. + // + // This function is useful if you want a polymorphic representation + // of the tensor's elements (turning it to a string for something + // like representation in a protobuf). + // + // This literal must have a dense layout. + void EachCellAsString( + const std::function indices, + const string& value)>& per_cell) const; + template + void EachCell( + std::function indices, NativeT value)> + per_cell) const; + + // Returns whether every element in this literal is equal to value. + // + // value is an int8 because we expect this to be called with small + // compile-time constants (0, -1, etc.) and so that whatever value you pass + // can be represented exactly by floating-point types as small as 16 bits. + // + // If value doesn't fit in this literal's type, returns false. Values of 1/0 + // are considered equal to true/false; other values are not considered equal + // to true. Also if this literal is not array-shaped false is returned. + bool IsAll(int8 value) const; + + // Like IsAll(const Literal&, int8), except we check whether the literal is + // equal to a particular floating-point number. + // + // If the literal is not a floating-point value, this always returns false. + // + // This casts value to the type of literal, then compares using ==. The usual + // admonishments about floating-point equality checks apply. We expect you to + // use this to check for values that can be expressed precisely as a float, + // e.g. -0.5. Also if this literal is not array-shaped false is returned. + bool IsAllFloat(float value) const; + + // Like IsAll(const Literal&, int8), except we check whether the literal is + // equal to a particular complex number. + // + // If the literal is not a complex value, this always returns false. + // + // This casts value to the type of literal, then compares using ==. The usual + // admonishments about floating-point equality checks apply. We expect you to + // use this to check for complex values that can be expressed precisely as + // float pairs e.g. (-0.5, 1.0). + // + // This literal must have a dense layout. + bool IsAllComplex(complex64 value) const; + + // Literal consists entirely of the first element of the literal. + bool IsAllFirst() const; + + // Literal consists entirely of an iota. + bool IsR1Iota() const; + + // Returns whether this literal is zero at the specified index. This literal + // must be an array with a dense layout. + bool IsZero(absl::Span indices) const; + + // Returns the count of the elements in the array at the given shape index in + // this literal. + int64 element_count(const ShapeIndex& index = {}) const { + if (index.empty()) { + // Common case, avoid GetSubshape(). + return ShapeUtil::ElementsIn(shape()); + } + return ShapeUtil::ElementsIn(ShapeUtil::GetSubshape(shape(), index)); + } + + // Returns the count of the elements in the sparse array at the given shape + // index in this literal, which will be no larger than + // LayoutUtil::MaxSparseElements(SetSubshape(shape(), index).layout()). + int64 sparse_element_count() const; + + // Compute a hash for this literal. This literal must not be a sparse tensor + // or a tuple containing a sparse tensor. + size_t Hash() const; + + // Converts this literal to the given shape. Returns an error is the + // conversion is not possible. + StatusOr ConvertToShape(const Shape& dest_shape) const; + + // Converts this literal to another primitive type using a bitcast + // conversion. The to and from primitive types must have the same bit + // width. Returns an error if the conversion is not possible. This literal + // must be array-shaped. + StatusOr BitcastConvert(PrimitiveType primitive_dest_type) const; + + // Converts this literal to another primitive type. Returns an error if the + // conversion is not possible. This literal must be array-shaped. + StatusOr Convert(PrimitiveType primitive_dest_type) const; + + // Clones the underlying buffers into a new Literal. + Literal Clone() const; + + // TODO(b/67651157): The methods below which perform computation on Literals + // (Reshape, Slice, etc) should be moved elsewhere, and perhaps combined with + // evaluator code which operates on Literals. + // + // Creates a new value that has the equivalent value as this + // literal, but conforms to new_layout; e.g. a literal matrix that was in {0, + // 1} minor-to-major dimension layout can be re-layed-out as {1, 0} + // minor-to-major dimension layout and the value in the cell at any given + // logical index (i0, i1) will be the same. + // + // For tuple shaped literals, shape_index should be used to select the inner + // array that the new layout applies to. + // + // Note: this is useful when the client wants to ensure that a value placed in + // the XLA allocation tracker has a particular layout; for efficiency + // purposes or avoiding unimplemented operation/layout combinations. + Literal Relayout(const Layout& new_layout, + const ShapeIndex& shape_index = {}) const; + + // An overload of Relayout which changes the layout of the entire shape rather + // than being limited to a single array within the shape. + Literal Relayout(const Shape& shape_with_layout) const; + + // Creates a new literal by reshaping this literal to have the given + // dimensions. The total number of elements must not change; The + // implementation currently only supports monotonic dim0-major layouts. + // This literal must be an array. + StatusOr Reshape(absl::Span dimensions) const; + + // Creates a new literal by broadcasting this literal with `dimensions` to + // yield a literal of shape `result_shape`. + StatusOr Broadcast(const Shape& result_shape, + absl::Span dimensions) const; + + // Creates a new literal by reordering the dimensions of this literal. + // The given `permutation` must be a permutation of the dimension numbers + // in the original literal, and it specifies the order of the new dimensions + // in the result literal (i.e., new_order[i] = old_order[permutation[i]]). + // For example, a transpose call on a literal of shape [3 x 8 x 4] and + // `permutation` = {2, 0, 1} returns a new literal of shape [4 x 3 x 8]. + // This literal must be an array. + Literal Transpose(absl::Span permutation) const; + + // Creates a sub-array from this literal by extracting the indices + // [start_index, limit_index) of each dimension. The result literal has the + // same rank and layout as for the given literal. The number of indices in + // start_indices and limit_indices must be the rank of the literal, and the + // indices follow the order of the dimensions. + // This literal must be an array. + Literal Slice(absl::Span start_indices, + absl::Span limit_indices) const; + + // Creates a literal with a prepended dimension with bound "times"; e.g. a + // f32[3x2] with times=4 will produce a f32[4x3x2] with the 3x2 from this + // literal replicated four times. + // This literal must be an array. + template + Literal Replicate(int64 times) const; + + // Creates a new Literal object with the shape specified as parameter. + // The content of the literal values is the default value of the primitive + // type of literal itself (0 for numeric types, and false for predicates). + // + // Note: It's an antipattern to use this method then immediately call + // MutableLiteralBase::Populate on the result (since that results in zero + // initialization, then reinitialization. Conside if a call to + // absl::make_unique(shape), followed by the call to + // MutableLiteralBase::Populate can be used instead. + static Literal CreateFromShape(const Shape& shape); + + protected: + // A data structure representing a subshape at a particular ShapeIndex within + // the literal. For array-shaped ShapeIndexes, this data structure holds the + // pointer to the memory allocated for the array data. + class Piece { + public: + // Returns the buffer holding the array data for this piece as an array + // slice. This piece must be array-shaped. + template + absl::Span data() const; + template + absl::Span data(); + + // Returns the buffer holding the array data for this piece as a void*. This + // piece must be array-shaped. + void* untyped_data(); + const void* untyped_data() const; + + // Gets or sets an element in the array at the given index. The multi_index + // is CHECKed against the dimension sizes of the array. This piece must be + // array-shaped. + template + NativeT Get(absl::Span index) const; + template + void Set(absl::Span index, NativeT value); + + // Gets/sets the buffer holding the array data. + char* buffer() const { return buffer_; } + void set_buffer(char* buffer) { buffer_ = buffer; } + + // The array of multi-indices that provide the locations of non-zero + // elements in a sparse array. Only used if + // LayoutUtil::IsSparseArray(shape()) is true. + SparseIndexArray* sparse_indices() const { return sparse_indices_; } + void set_sparse_indices(SparseIndexArray* sparse_indices) { + sparse_indices_ = sparse_indices; + } + + // Gets or sets the subshape of this piece. This reference points to a + // subshape within the shape in the containing Literal (Literal::shape_). + const Shape& subshape() const { return *subshape_; } + void set_subshape(const Shape* subshape) { subshape_ = subshape; } + + // Returns the size in bytes of the buffer holding the array data. + int64 size_bytes() const { return ShapeUtil::ByteSizeOf(subshape()); } + + // Returns the number of elements in this piece's array. + int64 element_count() const { + // If this is a sparse array, use the number of elements represented by + // the indices in the associated SparseIndexArray. + return LayoutUtil::IsSparseArray(subshape()) + ? sparse_indices()->index_count() + : ShapeUtil::ElementsIn(subshape()); + } + + // Returns the child piece at 'index' of this piece. + Piece& child(int64 index) { return children_[index]; } + + // Adds a child piece to this piece's children. + void emplace_back(Piece child_piece) { + children_.emplace_back(std::move(child_piece)); + } + + // Returns the size of children pieces of this piece. + int64 children_size() { return children_.size(); } + + // Visitor functions that recursively traverses the piece and calls the + // given function at each child piece. The function has the type: + // void (const ShapeIndex& index, const Piece& piece) + template + void ForEachSubpiece(const Fn& func) const { + ShapeIndex index; + return ForEachHelper( + [&func](const ShapeIndex& index, const Piece& piece) { + func(index, piece); + return Status::OK(); + }, + *this, &index) + .IgnoreError(); + } + // Same as above, but the function has the type: + // Status (const ShapeIndex& index, const Piece& piece) + // The first non-OK return value is returned by the function. + template + Status ForEachSubpieceWithStatus(const Fn& func) const { + ShapeIndex index; + return ForEachHelper(func, *this, &index); + } + // Same as above, but the function has the type: + // Bool (const ShapeIndex& index, const Piece& piece) + // The first non-true return value is returned by the function. + template + bool ForEachSubpieceWithBool(const Fn& func) const { + ShapeIndex index; + return ForEachHelperBool(func, *this, &index); + } + // Same as above, but the function has the type: + // Void (const ShapeIndex& index, Piece& piece) + template + void ForEachMutableSubpiece(const Fn& func) { + ShapeIndex index; + return ForEachMutableHelper( + [&func](const ShapeIndex& index, Piece* piece) { + func(index, piece); + return Status::OK(); + }, + const_cast(this), &index) + .IgnoreError(); + } + // Same as above, but the function has the type: + // Status (const ShapeIndex& index, Piece& piece) + // The first non-OK return value is returned by the function. + template + Status ForEachMutableSubpieceWithStatus(const Fn& func) { + ShapeIndex index; + return ForEachMutableHelper( + func, const_cast(this), &index); + } + + // Returns true if this piece and 'other' contain the same data. This piece + // and 'other' must be array-shaped and compatible. + bool EqualElements(const Piece& other) const; + + // Writes the shape and data (if array-shaped) into the given proto. + void WriteToProto(LiteralProto* proto) const; + + // Copy the data from 'src' into this piece's buffer. Shapes of this piece + // and src must be compatible. + Status CopyFrom(const Piece& src); + + // Copies the data from the given proto into this piece. The shape of this + // piece must be equal (not just compatible) to the shape of the proto. + Status CopyFromProto(const LiteralProto& proto); + + // Sorts the elements in a sparse array. + void SortSparseElements(); + + private: + // Helpers for traversing the piece via ForEachSubpiece rooted at 'index'. + // The first non-OK (or non-true) value is returned by the function. + // The callable 'func' has the same signature as described above in + // ForEachSubpiece*. + template + Status ForEachHelper(const Fn& func, const Piece& piece, + ShapeIndex* index) const { + TF_RETURN_IF_ERROR(func(*index, piece)); + for (int64 i = 0; i < piece.children_.size(); ++i) { + index->push_back(i); + TF_RETURN_IF_ERROR(ForEachHelper(func, piece.children_[i], index)); + index->pop_back(); + } + return Status::OK(); + } + template + bool ForEachHelperBool(const Fn& func, const Piece& piece, + ShapeIndex* index) const { + if (!func(*index, piece)) { + return false; + } + for (int64 i = 0; i < piece.children_.size(); ++i) { + index->push_back(i); + if (!ForEachHelperBool(func, piece.children_[i], index)) { + return false; + } + index->pop_back(); + } + return true; + } + template + Status ForEachMutableHelper(const Fn& func, Piece* piece, + ShapeIndex* index) { + TF_RETURN_IF_ERROR(func(*index, piece)); + for (int64 i = 0; i < piece->children_.size(); ++i) { + index->push_back(i); + TF_RETURN_IF_ERROR( + ForEachMutableHelper(func, &piece->children_[i], index)); + index->pop_back(); + } + return Status::OK(); + } + + // Recursive helper for EqualElements. + template + bool EqualElementsInternal(const Piece& other, + std::vector* multi_index) const; + + // Helper for SortSparseElements that has the element type as a template + // parameter. + template + void SortSparseElementsInternal(); + + // For array-shaped pieces, this is the buffer holding the literal data. + char* buffer_ = nullptr; + + // For sparse arrays, this is the array of indices. + SparseIndexArray* sparse_indices_ = nullptr; + + // The shape of piece. This points into the shape of the containing Literal + // (Literal::shape_). + const Shape* subshape_ = nullptr; + + // Children pieces for tuple shaped pieces. + std::vector children_ = {}; + }; // class Piece + + const Piece& piece(const ShapeIndex& shape_index) const { + Piece* piece = &const_cast(root_piece()); + for (const auto i : shape_index) { + DCHECK_GE(i, 0); + DCHECK_LT(i, piece->children_size()); + piece = &piece->child(i); + } + return *piece; + } + + // Returns the piece at the root of the shape. + virtual const Piece& root_piece() const = 0; + + // LiteralSlice and Literal must access Pieces of other Literals. + friend class MutableLiteralBase; + friend class LiteralSlice; + friend class BorrowingLiteral; + + private: + template + Literal SliceInternal(const Shape& result_shape, + absl::Span start_indices) const; +}; + +// Abstract base class representing a mutable literal in XLA. +class MutableLiteralBase : public LiteralBase { + public: + virtual ~MutableLiteralBase() = 0; + + // Returns a Span view of the array for this literal for the + // given NativeT (e.g., float). CHECKs if the subshape of the literal at the + // given ShapeIndex is not array. See primitive_util.h for the mapping from + // XLA type to native type. + template + absl::Span data(const ShapeIndex& shape_index = {}); + // Unhide const method from parent class. + using LiteralBase::data; + + // Returns a pointer to the sparse index array. Returns nullptr if the literal + // is not a sparse array. + SparseIndexArray* sparse_indices(const ShapeIndex& shape_index = {}); + + // TODO(b/67651157): Remove this accessor. Literal users should not be able to + // mutate the shape as this can produce malformed Literals. + Shape* mutable_shape_do_not_use() { return shape_.get(); } + + // Returns a pointer to the underlying buffer holding the array at the given + // shape index. CHECKs if the subshape of the literal at the given ShapeIndex + // is not array. + void* untyped_data(const ShapeIndex& shape_index = {}); + // Unhide const method from parent class. + using LiteralBase::untyped_data; + + // Populates a literal with a sparse layout with the given indices and values. + // Each index in the indices array is CHECKed against the dimensions in the + // literal's shape. If sort is true, then the indices and values will be + // sorted. If sort is false, then the indices and values are assumed to + // already be in sorted order. See CreateSparse for an example of how data + // are populated. + template + void PopulateSparse(SparseIndexArray indices, + absl::Span values, bool sort = true); + + // Copy values from 'src_literal' rooted at 'src_shape_index' into this + // literal rooted at 'dest_shape_index'. The subshape of this literal rooted + // at 'dest_shape_index' must be compatible with the subshape of 'src_literal' + // rooted at 'src_shape_index', but need not be arrays. + Status CopyFrom(const LiteralSlice& src_literal, + const ShapeIndex& dest_shape_index = {}, + const ShapeIndex& src_shape_index = {}); + + // Copies the values from src_literal, starting at src_base shape indexes, + // to this literal, starting at dest_base, where the copy size in each + // dimension is specified by copy_size. + // The src_literal and this literal must have the same primitive type, + // src_base+copy_size must fit the source literal dimensions, as well as + // dest_base+copy_size must fit the destination literal dimensions. + // Note: if either src_literal or this literal contains dimensions with zero + // element, then copy_size must be 0 in these dimensions while the + // corresponding base indices being 0. + // This literal and 'src_literal' must be arrays. + Status CopySliceFrom(const LiteralSlice& src_literal, + absl::Span src_base, + absl::Span dest_base, + absl::Span copy_size); + + // Copies one element from src_literal[src_index] to (*this)[dest_index]. + Status CopyElementFrom(const LiteralSlice& src_literal, + absl::Span src_index, + absl::Span dest_index); + + // Sets an element in the literal at the given index. The multi_index is + // CHECKed against the dimension sizes. + template + void Set(absl::Span multi_index, const ShapeIndex& shape_index, + NativeT value); + // Overloads of Set for array literals. CHECKs if the literal is not + // array-shaped and dense. + template + void Set(absl::Span multi_index, NativeT value); + + // Appends the given element to the literal. If the elements are not appended + // in sorted order, then SortSparseElements should be called before calling + // other methods. This literal must have a sparse layout. + template + void AppendSparseElement(absl::Span multi_index, NativeT value, + const ShapeIndex& shape_index = {}); + + // Sorts the elements in a sparse array. + void SortSparseElements(const ShapeIndex& shape_index = {}); + + // As Set(), but truncates `value` to the literal element type before storing. + // This literal must be an array. + Status SetIntegralAsS64(absl::Span multi_index, int64 value); + + // Populate this literal with the given values. Examples: + // + // // Populate with floats. + // Array2D float_values = ... + // literal.PopulateR2FromArray2D(values); + // + // // Populate with int32s. + // literal.PopulateR2({{1, 2}, {3, 4}}); + // + // The shape and element type of this literal must match given values. For + // example, in the call above to literal.PopulateR2(), 'literal' must be a 2x2 + // array of S32. + template + void PopulateR1(absl::Span values); + void PopulateR1(const tensorflow::core::Bitmap& values); + template + void PopulateR2(std::initializer_list> values); + template + void PopulateFromArray(const Array& values); + template + void PopulateR2FromArray2D(const Array2D& values); + template + void PopulateR3FromArray3D(const Array3D& values); + template + void PopulateR4FromArray4D(const Array4D& values); + + // Populates literal values by calling the generator function for every cell + // in this literal object. + // + // generator must be a callable of the type + // NativeT(absl::Span indexes) or compatible. + // + // This literal must have a dense layout. + template + Status Populate(const FnType& generator); + + // A parallel version of Populate(). This can be used if the generator is + // thread-safe and the values for the shape's different elements are + // independent. + template + Status PopulateParallel(const FnType& generator); + + // Fills this literal with the given value. + template + void PopulateWithValue(NativeT value); + + // This operation is the inverse of DecomposeTuple. The given elements are + // moved into the tuple elements of a new tuple-shaped Literal which is + // returned. Upon return, each of the Literals in 'elements' is set to a nil + // shape (empty tuple). + static Literal MoveIntoTuple(absl::Span elements); + + // Serialize from a proto. + static StatusOr CreateFromProto(const LiteralProto& proto); + + protected: + // Returns the piece at the given ShapeIndex. + Piece& piece(const ShapeIndex& shape_index) { + return const_cast(LiteralBase::piece(shape_index)); + } + + Piece& root_piece() const override { return *root_piece_; }; + + // Internal template helper for the Literal::CopySliceFrom(), matching its + // arguments one by one. + template + Status CopySliceFromInternal(const LiteralBase& src_literal, + absl::Span src_base, + absl::Span dest_base, + absl::Span copy_size); + + // Utility structure which is used to create the optimal configuration for + // a ShapeUtil::ForEachIndex() scan across two literals. + struct StrideConfig { + StrideConfig(const Shape& source_shape, const Shape& dest_shape, + absl::Span dimensions); + + // The dimensions of the stride operation. Essentially every dimension + // will be iterated from base[i] to base[i]+dimensions[i], in step[i] + // steps. + absl::Span dimensions; + DimensionVector base; + DimensionVector step; + int64 minor_dimension = 0; + // The size of the strides for source and destination. One of the two + // (the one looping through its most minor dimension) will be 1, while + // the other will be the stride size at the dimension matching the other + // shape most minor dimension being scanned. + int64 dest_stride = 1; + int64 source_stride = 1; + // The size of the inner loop on the most minor dimension. + int64 minor_loop_size = 1; + }; + + // Literal class always owns the shape. The parent class borrows this shape. + std::unique_ptr shape_; + + Piece* root_piece_ = nullptr; + + // Implementation details shared between Populate() and PopulateParallel() + template + Status PopulateInternal(const FnType& generator, bool parallel); + + friend class LiteralBase; + friend class MutableBorrowingLiteral; +}; +std::ostream& operator<<(std::ostream& out, const Literal& literal); + +// The underlying buffer and shape is always owned by this class. +class Literal : public MutableLiteralBase { + public: + Literal() : Literal(ShapeUtil::MakeNil()) {} + + // Create a literal of the given shape. The literal is allocated sufficient + // memory to hold the shape. Memory is uninitialized. + explicit Literal(const Shape& shape); + virtual ~Literal(); + + // Literals are moveable, but not copyable. To copy a literal use + // Literal::Clone or Literal::CloneToUnique. This prevents inadvertent copies + // of literals which can be expensive. + Literal(const Literal& other) = delete; + Literal& operator=(const Literal& other) = delete; + Literal(Literal&& other); + // 'allocate_arrays' indicates whether to allocate memory for the arrays in + // the shape. If false, buffer pointers inside of the Literal::Pieces are set + // to nullptr. + Literal(const Shape& shape, bool allocate_arrays); + Literal& operator=(Literal&& other); + + // Similar to CopyFrom, but with move semantincs. The subshape of this literal + // rooted at 'dest_shape_index' must be *equal* to the shape 'src_literal' + // (layouts and shapes must match), but need not be arrays. The memory + // allocated in this literal for the subshape at dest_shape_index is + // deallocated, and the respective buffers are replaced with those in + // src_literal. Upon return, src_literal is set to a nil shape (empty tuple). + virtual Status MoveFrom(Literal&& src_literal, + const ShapeIndex& dest_shape_index = {}); + + // Returns a vector containing the tuple elements of this Literal as separate + // Literals. This Literal must be tuple-shaped and can be a nested tuple. The + // elements are moved into the new Literals; no data is copied. Upon return + // this Literal is set to a nil shape (empty tuple) + std::vector DecomposeTuple(); + + private: + // Deallocate the buffers held by this literal. + void DeallocateBuffers(); + + // Recursively sets the subshapes and buffers of all subpieces rooted at + // 'piece'. If 'allocate_array' is true, memory is allocated for the arrays in + // the shape. + void SetPiece(const Shape& shape, Piece* piece, bool allocate_arrays); +}; + +// The underlying buffer is not owned by this class and is always owned by +// others. The shape is not owned by this class and not mutable. +class MutableBorrowingLiteral : public MutableLiteralBase { + public: + virtual ~MutableBorrowingLiteral(); + + MutableBorrowingLiteral() : MutableLiteralBase() {} + + MutableBorrowingLiteral(const MutableBorrowingLiteral& literal); + MutableBorrowingLiteral& operator=(const MutableBorrowingLiteral& literal); + + // Implicit conversion constructors. + MutableBorrowingLiteral(const MutableLiteralBase& literal); + MutableBorrowingLiteral(MutableLiteralBase* literal); + MutableBorrowingLiteral(MutableBorrowingLiteral literal, + const ShapeIndex& view_root); + MutableBorrowingLiteral(const char* src_buf_ptr, const Shape& shape); + + private: + // Recursively copies the subtree from the `src_piece` at the given child + // index to the `dest_piece`. For buffers only the pointers are copied, but + // not the content. + void CopyPieceSubtree(const Shape& shape, Piece* src_piece, + Piece* dest_piece); +}; + +// A read-only view of a Literal. A LiteralSlice contains pointers to shape and +// literal buffers always owned by others. +class LiteralSlice : public LiteralBase { + public: + LiteralSlice() : LiteralBase() {} + + // Implicit conversion constructors. + LiteralSlice(const LiteralBase& literal); + LiteralSlice(const LiteralBase& literal, const ShapeIndex& view_root); + + private: + const Piece& root_piece() const override { return *root_piece_; }; + + const Piece* root_piece_; // Not owned. +}; + +// A read-only Literal where the underlying buffers are never owned by this +// class. +class BorrowingLiteral : public LiteralBase { + public: + BorrowingLiteral() : LiteralBase() {} + + // 'src_buf_ptr' is not owned by this class and must outlive the + // lifetime of this class. It points to an appropirately sized buffer with + // data interpretered as indicated by 'shape'. + // This constructor is only used for array shapes. + BorrowingLiteral(const char* src_buf_ptr, const Shape& shape); + // Similar as above, except to be used for constructing non-nested tuples. + BorrowingLiteral(absl::Span src_buf_ptrs, + const Shape& shape); + // TODO(b/79707221): adding constructors for nested tuples as well. + + private: + // Recursively builds the subtree for the given piece and sets the subshapes + // of the given piece with the given shape. + void BuildPieceSubtree(const Shape& shape, Piece* piece); + + // Accessor for the root piece of this literal. + const Piece& root_piece() const override { return root_piece_; }; + Piece root_piece_; + + // Shape of this literal. Stored as unique_ptr such that the (default) move + // construction of this class would be trivially correct: the pointer to Shape + // root_piece_ stores will still point to the correct address. + std::unique_ptr shape_; +}; + +template +absl::Span LiteralBase::Piece::data() const { + DCHECK(ShapeUtil::IsArray(subshape())) << ShapeUtil::HumanString(subshape()); + DCHECK_EQ(subshape().element_type(), + primitive_util::NativeToPrimitiveType()) + << "Attempting to access " + << PrimitiveType_Name(primitive_util::NativeToPrimitiveType()) + << " type, but literal element type is " + << PrimitiveType_Name(subshape().element_type()); + return absl::Span(reinterpret_cast(buffer()), + element_count()); +} + +template +absl::Span LiteralBase::Piece::data() { + DCHECK(ShapeUtil::IsArray(subshape())) << ShapeUtil::HumanString(subshape()); + DCHECK_EQ(subshape().element_type(), + primitive_util::NativeToPrimitiveType()) + << "Attempting to access " + << PrimitiveType_Name(primitive_util::NativeToPrimitiveType()) + << " type, but literal element type is " + << PrimitiveType_Name(subshape().element_type()); + return absl::Span(reinterpret_cast(buffer()), + element_count()); +} + +template +NativeT LiteralBase::Piece::Get(absl::Span multi_index) const { + CHECK(LayoutUtil::IsDenseArray(subshape())); + return data()[IndexUtil::MultidimensionalIndexToLinearIndex( + subshape(), multi_index)]; +} + +template +void LiteralBase::Piece::Set(absl::Span multi_index, + NativeT value) { + CHECK(LayoutUtil::IsDenseArray(subshape())); + data()[IndexUtil::MultidimensionalIndexToLinearIndex( + subshape(), multi_index)] = value; +} + +template +absl::Span LiteralBase::data( + const ShapeIndex& shape_index) const { + return piece(shape_index).data(); +} + +template +absl::Span MutableLiteralBase::data(const ShapeIndex& shape_index) { + return piece(shape_index).data(); +} + +template +inline NativeT LiteralBase::Get(absl::Span multi_index, + const ShapeIndex& shape_index) const { + return piece(shape_index).Get(multi_index); +} + +template +inline NativeT LiteralBase::Get(absl::Span multi_index) const { + return root_piece().Get(multi_index); +} + +template +inline void MutableLiteralBase::Set(absl::Span multi_index, + const ShapeIndex& shape_index, + NativeT value) { + return piece(shape_index).Set(multi_index, value); +} + +template +inline void MutableLiteralBase::Set(absl::Span multi_index, + NativeT value) { + return root_piece().Set(multi_index, value); +} + +template +NativeT LiteralBase::GetFirstElement() const { + return data().at(0); +} + +template +NativeT LiteralBase::GetSparseElement(int64 sparse_element_number, + const ShapeIndex& shape_index) const { + CHECK( + LayoutUtil::IsSparseArray(ShapeUtil::GetSubshape(shape(), shape_index))); + return data(shape_index)[sparse_element_number]; +} + +template +void MutableLiteralBase::AppendSparseElement( + absl::Span multi_index, NativeT value, + const ShapeIndex& shape_index) { + Piece& p = piece(shape_index); + const Shape& subshape = p.subshape(); + CHECK(LayoutUtil::IsSparseArray(subshape)); + int64 rank = ShapeUtil::Rank(subshape); + CHECK_EQ(multi_index.size(), rank); + int64 last_element = p.sparse_indices()->index_count(); + CHECK_LT(last_element, LayoutUtil::MaxSparseElements(subshape.layout())); + p.sparse_indices()->Append(multi_index); + CHECK_LT(last_element, p.data().size()); + p.data()[last_element] = value; +} + +template +void LiteralBase::EachCell( + std::function indices, NativeT value)> + per_cell) const { + if (ShapeUtil::IsZeroElementArray(shape())) { + return; + } + std::vector indices(ShapeUtil::Rank(shape()), 0); + do { + per_cell(indices, Get(indices)); + } while (IndexUtil::BumpIndices(shape(), absl::MakeSpan(indices))); +} + +template +inline void MutableLiteralBase::PopulateR1(absl::Span values) { + CHECK(ShapeUtil::IsArray(shape())); + CHECK_EQ(ShapeUtil::Rank(shape()), 1); + CHECK_EQ(ShapeUtil::ElementsIn(shape()), values.size()); + CHECK_EQ(shape().element_type(), + primitive_util::NativeToPrimitiveType()); + for (int64 i = 0; i < values.size(); ++i) { + Set({i}, values[i]); + } +} + +template +void MutableLiteralBase::PopulateR2( + std::initializer_list> values) { + CHECK(ShapeUtil::IsArray(shape())); + CHECK_EQ(ShapeUtil::Rank(shape()), 2); + CHECK_EQ(shape().element_type(), + primitive_util::NativeToPrimitiveType()); + + const int64 dim0_size = values.size(); + const int64 dim1_size = values.begin()->size(); + CHECK_EQ(dim0_size, shape().dimensions(0)); + CHECK_EQ(dim1_size, shape().dimensions(1)); + + int64 dim0 = 0; + for (auto inner_list : values) { + int64 dim1 = 0; + for (auto value : inner_list) { + Set({dim0, dim1}, value); + ++dim1; + } + CHECK_EQ(dim1_size, dim1); + ++dim0; + } +} + +template +void MutableLiteralBase::PopulateFromArray(const Array& values) { + CHECK(ShapeUtil::IsArray(shape())); + CHECK_EQ(shape().element_type(), + primitive_util::NativeToPrimitiveType()); + CHECK_EQ(ShapeUtil::Rank(shape()), values.num_dimensions()); + for (int dim = 0; dim < values.num_dimensions(); ++dim) { + CHECK_EQ(values.dim(dim), shape().dimensions(dim)); + } + values.Each([this](absl::Span indices, NativeT value) { + this->Set(indices, value); + }); +} + +template +void MutableLiteralBase::PopulateR2FromArray2D(const Array2D& values) { + PopulateFromArray(values); +} + +template +void MutableLiteralBase::PopulateR3FromArray3D(const Array3D& values) { + PopulateFromArray(values); +} + +template +void MutableLiteralBase::PopulateR4FromArray4D(const Array4D& values) { + PopulateFromArray(values); +} + +template +void MutableLiteralBase::PopulateSparse(SparseIndexArray indices, + absl::Span values, + bool sort) { + CHECK(LayoutUtil::IsSparseArray(shape())); + int rank = ShapeUtil::Rank(shape()); + CHECK_EQ(indices.rank(), rank); + int64 max_elements = LayoutUtil::MaxSparseElements(shape().layout()); + CHECK_LE(indices.max_indices(), max_elements); + int64 num_elements = values.size(); + CHECK_LE(num_elements, max_elements); + CHECK_EQ(num_elements, indices.index_count()); + auto root_data = root_piece().data(); + // Piece::data() returns a Span of size equal to the number of indices + // in the SparseIndexArray. So there is no need to adjust the size of the data + // here. It is enough to just copy the incoming values into the data buffer. + std::copy(values.begin(), values.end(), root_data.begin()); + *this->root_piece().sparse_indices() = std::move(indices); + if (sort) { + auto root_data = this->root_piece().data(); + this->root_piece().sparse_indices()->SortWithValues(root_data); + } + DCHECK(this->root_piece().sparse_indices()->Validate(shape())); +} + +template +Status MutableLiteralBase::PopulateInternal(const FnType& generator, + bool parallel) { + const Shape& this_shape = shape(); + const int64 rank = ShapeUtil::Rank(this_shape); + TF_RET_CHECK(LayoutUtil::IsDenseArray(this_shape)); + TF_RET_CHECK(this_shape.element_type() == + primitive_util::NativeToPrimitiveType()); + absl::Span literal_data = data(); + if (rank > 0) { + StrideConfig stride_config(this_shape, this_shape, + AsInt64Slice(this_shape.dimensions())); + int64 minor_dimension_size = + ShapeUtil::GetDimension(this_shape, stride_config.minor_dimension); + + auto init_function = [&](absl::Span indexes) { + DimensionVector minor_scan_indexes(rank, 0); + const int64 index = + IndexUtil::MultidimensionalIndexToLinearIndex(shape(), indexes); + std::copy(indexes.begin(), indexes.end(), minor_scan_indexes.begin()); + for (int64 i = 0; i < minor_dimension_size; ++i) { + minor_scan_indexes[stride_config.minor_dimension] = i; + literal_data.at(index + i) = generator(minor_scan_indexes); + } + }; + if (parallel) { + ShapeUtil::ForEachIndexParallel(this_shape, stride_config.base, + stride_config.dimensions, + stride_config.step, init_function); + } else { + ShapeUtil::ForEachIndex( + this_shape, stride_config.base, stride_config.dimensions, + stride_config.step, + [&init_function](absl::Span indexes) { + init_function(indexes); + return true; + }); + } + } else { + // For scalars. + literal_data.at(0) = generator({}); + } + return Status::OK(); +} +template +Status MutableLiteralBase::Populate(const FnType& generator) { + return PopulateInternal(generator, /*parallel=*/false); +} + +template +Status MutableLiteralBase::PopulateParallel(const FnType& generator) { + return PopulateInternal(generator, /*parallel=*/true); +} + +template +void MutableLiteralBase::PopulateWithValue(NativeT value) { + CHECK(ShapeUtil::IsArray(shape())); + CHECK_EQ(shape().element_type(), + primitive_util::NativeToPrimitiveType()); + for (NativeT& element : data()) { + element = value; + } +} + +template +Literal LiteralBase::Replicate(int64 times) const { + DimensionVector bounds = {times}; + bounds.reserve(shape().dimensions_size() + 1); + for (int64 bound : shape().dimensions()) { + bounds.push_back(bound); + } + Literal literal(ShapeUtil::MakeShape(shape().element_type(), bounds)); + int64 elements = ShapeUtil::ElementsIn(literal.shape()); + if (elements == 0) { + return literal; + } + + DimensionVector output_indices(bounds.size(), 0); + absl::Span input_indices = output_indices; + input_indices.remove_prefix(1); + + bool done = false; + while (!done) { + const auto element = Get(input_indices); + literal.Set(output_indices, element); + + done = true; + for (int n = 0; n < output_indices.size(); ++n) { + ++output_indices[n]; + if (output_indices[n] < bounds[n]) { + done = false; + break; + } + output_indices[n] = 0; + } + } + return literal; +} + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_LITERAL_H_ diff --git a/linux/include/tensorflow/compiler/xla/literal_util.h b/linux/include/tensorflow/compiler/xla/literal_util.h new file mode 100644 index 000000000..2b181621e --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/literal_util.h @@ -0,0 +1,591 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// Utilities for dealing with Literal protobufs. + +#ifndef TENSORFLOW_COMPILER_XLA_LITERAL_UTIL_H_ +#define TENSORFLOW_COMPILER_XLA_LITERAL_UTIL_H_ + +#include +#include +#include +#include +#include +#include +#include +#include + +#include "absl/memory/memory.h" +#include "absl/strings/string_view.h" +#include "absl/types/span.h" +#include "tensorflow/compiler/xla/array2d.h" +#include "tensorflow/compiler/xla/array3d.h" +#include "tensorflow/compiler/xla/array4d.h" +#include "tensorflow/compiler/xla/index_util.h" +#include "tensorflow/compiler/xla/layout_util.h" +#include "tensorflow/compiler/xla/literal.h" +#include "tensorflow/compiler/xla/primitive_util.h" +#include "tensorflow/compiler/xla/shape_util.h" +#include "tensorflow/compiler/xla/sparse_index_array.h" +#include "tensorflow/compiler/xla/status_macros.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/compiler/xla/util.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/lib/core/bitmap.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/platform/logging.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/protobuf.h" +#include "tensorflow/core/platform/types.h" + +namespace xla { + +class LiteralUtil { + public: + LiteralUtil() = delete; + + // Returns a literal scalar representing the first element. + static Literal GetFirstScalarLiteral(const LiteralSlice& literal); + + // Creates a new literal of a given rank. To minimize ambiguity (for users + // and the compiler) these CreateR[0-2] methods should explicitly specify the + // native type. For example: + // + // CreateR1({1.0, 42.0}); + // CreateR2({{1, 2}, {3, 4}}); + // + // The variants not ending with WithLayout use the default XLA layout for the + // literal's linear representation in memory. + template + static Literal CreateR0(NativeT value); + template + static Literal CreateR1(absl::Span values); + static Literal CreateR1(const tensorflow::core::Bitmap& values); + template + static Literal CreateR2( + std::initializer_list> values); + template + static Literal CreateR2WithLayout( + std::initializer_list> values, + const Layout& layout); + template + static Literal CreateR3(std::initializer_list< + std::initializer_list>> + values); + template + static Literal CreateR3WithLayout( + std::initializer_list< + std::initializer_list>> + values, + const Layout& layout); + template + static Literal CreateR4( + std::initializer_list>>> + values); + template + static Literal CreateR4WithLayout( + std::initializer_list>>> + values, + const Layout& layout); + + // Creates a literal with a sparse layout and the given indices and values. + // The shape is initialized from the given dimensions. The minor dimension of + // the indices array must equal the rank of the shape (i.e. size of the + // dimensions array). The major dimension of the indices array must equal the + // number of elements in the values array. The maximum number of elements in + // the array is taken from the max_indices() value of the index array. + // + // XLA assumes that sparse literals are in sorted order for all operations. If + // the `sort` argument is true, then the indices and values will be sorted + // while copying them into the literal. If you have ensured that the indices + // and values are already sorted, then you may set the `sort` argument to + // false to skip the sorting step. + // + // For example: + // + // CreateSparse( + // {12, 12, 12}, + // SparseIndexArray(10, 3, + // Array2D{ + // {0, 1, 2}, + // {3, 4, 5}, + // {6, 7, 8}, + // {9, 10, 11}, + // }), + // {1.0, 2.0 3.0, 4.0}) + // + // This creates an array with shape F64[12,12,12]sparse{10}, that has the + // following non-zero values: + // + // [0, 1, 2]: 1.0 + // [3, 4, 5]: 2.0 + // [6, 7, 8]: 3.0 + // [9, 10, 11]: 4.0 + // + template + static Literal CreateSparse(absl::Span dimensions, + SparseIndexArray indices, + absl::Span values, + bool sort = true); + + // Creates a scalar literal value zero of the given primitive type. + static Literal Zero(PrimitiveType primitive_type); + // Creates a scalar literal value one of the given primitive type. + static Literal One(PrimitiveType primitive_type); + // Creates a scalar literal value containing the minimum value of the given + // primitive type. For floating-point types, returns -inf. + static Literal MinValue(PrimitiveType primitive_type); + // Creates a scalar literal value containing the maximum value of the given + // primitive type. For floating-point types, returns inf. + static Literal MaxValue(PrimitiveType primitive_type); + // Creates a literal of the given shape where each element is `value`. + template + static Literal CreateFullWithDescendingLayout( + absl::Span dimensions, NativeT value); + + // Creates a new literal from an Array type. The variants not ending with + // WithLayout use the default XLA layout for the literal's linear + // representation in memory. + template + static Literal CreateFromArray(const Array& values); + template + static Literal CreateFromArrayWithLayout(const Array& values, + const Layout& layout); + template + static Literal CreateR2FromArray2D(const Array2D& values); + template + static Literal CreateR2FromArray2DWithLayout(const Array2D& values, + const Layout& layout); + template + static Literal CreateR3FromArray3D(const Array3D& values); + template + static Literal CreateR3FromArray3DWithLayout(const Array3D& values, + const Layout& layout); + template + static Literal CreateR4FromArray4D(const Array4D& values); + template + static Literal CreateR4FromArray4DWithLayout(const Array4D& values, + const Layout& layout); + + // Creates a new vector of U8s literal value from a string. + static Literal CreateR1U8(absl::string_view value); + + // Creates a linspace-populated literal with the given number of rows and + // columns. + static Literal CreateR2F32Linspace(float from, float to, int64 rows, + int64 cols); + + // Creates a literal that projects the (x, y) dimensions given in values into + // the z dimension given by "projection". + template + static Literal CreateR3Projected( + std::initializer_list> values, + int64 projection); + + // Creates a literal that projects the (x, y) dimensions given in values into + // the z and p dimensions given. + template + static Literal CreateR4Projected( + std::initializer_list> values, + int64 projection_p, int64 projection_z); + + // Returns an identity matrix (rank 2) with the given row and column count. + template + static Literal MakeIdentityR2(int64 size); + + // Returns a tuple literal composed of given literals. Data is copied from the + // given elements into the returned literal. + static Literal MakeTuple(absl::Span elements); + + static Literal MakeTupleFromSlices(absl::Span elements); + + // As above, but intended to be invoked with move semantics; i.e. + // + // std::vector elements = ...; + // auto result = LiteralUtil::MakeTupleOwned(std::move(elements)); + // + // This would have been declared as an overload, but there is ambiguity + // in invocation between the above signature and this one. + static Literal MakeTupleOwned(std::vector elements); + + // This overload lets you pass a braced list of Literals to + // MakeTupleOwned: + // + // LiteralUtil::MakeTupleOwned(LiteralUtil::CreateR1(...), ...). + // + // Simply relying on the MakeTupleOwned(std::vector) + // overload doesn't work because std::initializer_list's elements are always + // const. + // + // The arguments to this function must all be Literal. + template + static Literal MakeTupleOwned(Ts... elements) { + std::array arr{std::move(elements)...}; + std::vector v; + v.insert(v.begin(), std::make_move_iterator(arr.begin()), + std::make_move_iterator(arr.end())); + return MakeTupleOwned(std::move(v)); + } + + // Create a constant token literal. Token types have no value. + static Literal CreateToken(); + + // Creates a new Literal object with its values havings the primitive_type + // type, and with dimensions defined by the dimensions parameter. + // The content of the literal values is the default value of the primitive + // type of literal itself (0 for numeric types, and false for predicates). + static Literal CreateFromDimensions(PrimitiveType primitive_type, + absl::Span dimensions); + + // If the given literal's data type is bfloat16, converts it to a float + // literal; otherwise, returns a copy of it. If the literal is a tuple, + // recursively converts its elements. + static Literal ConvertBF16ToF32(const LiteralSlice& bf16_literal); + + // If the given literal's data type is float, converts it to a bfloat16 + // literal; otherwise, returns a copy of it. If the literal is a tuple, + // recursively converts its elements. + static Literal ConvertF32ToBF16(const LiteralSlice& f32_literal); + + // Creates a literal with a new shape with the given new dimensions using the + // data in the given input literal. For reshaping purposes the (flat) data + // buffer of the input literal is assumed to have the given minor_to_major + // layout order. + static Literal ReshapeSlice(absl::Span new_dimensions, + absl::Span minor_to_major, + const LiteralSlice& literal); + + // Creates a literal with the supplied shape, and uses the provided value + // generator to populate the literal's values. + // Returns the new literal object, or an error Status if failed. + template < + PrimitiveType type, + typename T = typename primitive_util::PrimitiveTypeToNative::type> + static StatusOr CreateRandomLiteral( + const Shape& shape, + const std::function)>& generator); + + // Creates a literal with the supplied shape, and initializes the literal + // values using a normal distribution with given mean and stddev standard + // deviation, and using the engine as entropy generator. + // Returns the new literal object, or an error Status if failed. + template < + PrimitiveType type, typename E, + typename T = typename primitive_util::PrimitiveTypeToNative::type> + static StatusOr CreateRandomLiteral(const Shape& shape, E* engine, + T mean, T stddev); + + // Creates a literal with the supplied shape, and initializes the literal + // values using a normal distribution with given mean and stddev standard + // deviation. + // Returns the new literal object, or an error Status if failed. + template < + PrimitiveType type, + typename T = typename primitive_util::PrimitiveTypeToNative::type> + static StatusOr CreateRandomLiteral(const Shape& shape, T mean, + T stddev); + + // + // End of factory methods. + + // Returns a multi-dimensional index as a string. For example: '{7, 8}' will + // be returned for a 2-dimensional index with dimension 0 index equal to 7, + // dimension 1 equal to 8. + static string MultiIndexAsString(absl::Span multi_index); +}; + +std::ostream& operator<<(std::ostream& out, const Literal& literal); + +template +/* static */ Literal LiteralUtil::CreateR0(NativeT value) { + Literal literal(ShapeUtil::MakeShape( + primitive_util::NativeToPrimitiveType(), {})); + literal.Set({}, value); + return literal; +} + +template +/* static */ Literal LiteralUtil::CreateR1(absl::Span values) { + Literal literal( + ShapeUtil::MakeShape(primitive_util::NativeToPrimitiveType(), + {static_cast(values.size())})); + literal.PopulateR1(values); + return literal; +} + +template +/* static */ Literal LiteralUtil::CreateR2WithLayout( + std::initializer_list> values, + const Layout& layout) { + Literal literal(ShapeUtil::MakeShapeWithLayout( + primitive_util::NativeToPrimitiveType(), + {static_cast(values.size()), + static_cast(values.begin()->size())}, + AsInt64Slice(layout.minor_to_major()))); + literal.PopulateR2(values); + return literal; +} + +template +/* static */ Literal LiteralUtil::CreateR2( + std::initializer_list> values) { + return CreateR2WithLayout(values, LayoutUtil::GetDefaultLayoutForR2()); +} + +template +/* static */ Literal LiteralUtil::CreateR3WithLayout( + std::initializer_list>> + values, + const Layout& layout) { + const int64 d0 = values.size(); + const int64 d1 = values.begin()->size(); + const int64 d2 = values.begin()->begin()->size(); + Array3D tmp(d0, d1, d2); + int64 i0 = 0; + for (auto d1_values : values) { + int64 i1 = 0; + for (auto d2_values : d1_values) { + int64 i2 = 0; + for (auto value : d2_values) { + tmp(i0, i1, i2) = value; + ++i2; + } + ++i1; + } + ++i0; + } + return CreateR3FromArray3DWithLayout(tmp, layout); +} + +template +/* static */ Literal LiteralUtil::CreateR3( + std::initializer_list>> + values) { + return CreateR3WithLayout(values, LayoutUtil::GetDefaultLayoutForR3()); +} + +template +/* static */ Literal LiteralUtil::CreateR4WithLayout( + std::initializer_list>>> + values, + const Layout& layout) { + const int64 d0 = values.size(); + const int64 d1 = values.begin()->size(); + const int64 d2 = values.begin()->begin()->size(); + const int64 d3 = values.begin()->begin()->begin()->size(); + Array4D tmp(d0, d1, d2, d3); + int64 i0 = 0; + for (auto d1_values : values) { + int64 i1 = 0; + for (auto d2_values : d1_values) { + int64 i2 = 0; + for (auto d3_values : d2_values) { + int64 i3 = 0; + for (auto value : d3_values) { + tmp(i0, i1, i2, i3) = value; + ++i3; + } + ++i2; + } + ++i1; + } + ++i0; + } + return CreateR4FromArray4DWithLayout(tmp, layout); +} + +template +/* static */ Literal LiteralUtil::CreateSparse( + absl::Span dimensions, SparseIndexArray indices, + absl::Span values, bool sort) { + int64 num_elements = values.size(); + int64 rank = dimensions.size(); + CHECK_EQ(num_elements, indices.index_count()); + CHECK_EQ(rank, indices.rank()); + Literal literal(ShapeUtil::MakeShapeWithSparseLayout( + primitive_util::NativeToPrimitiveType(), dimensions, + indices.max_indices())); + literal.PopulateSparse(indices, values, sort); + return literal; +} + +template +/* static */ Literal LiteralUtil::CreateR4( + std::initializer_list>>> + values) { + return CreateR4WithLayout(values, LayoutUtil::GetDefaultLayoutForR4()); +} + +template +/* static */ Literal LiteralUtil::CreateFromArrayWithLayout( + const Array& values, const Layout& layout) { + Literal literal(ShapeUtil::MakeShapeWithLayout( + primitive_util::NativeToPrimitiveType(), values.dimensions(), + AsInt64Slice(layout.minor_to_major()))); + literal.PopulateFromArray(values); + return literal; +} + +template +/* static */ Literal LiteralUtil::CreateFromArray( + const Array& values) { + return CreateFromArrayWithLayout( + values, LayoutUtil::GetDefaultLayoutForRank(values.num_dimensions())); +} + +template +/* static */ Literal LiteralUtil::CreateR2FromArray2DWithLayout( + const Array2D& values, const Layout& layout) { + return CreateFromArrayWithLayout(values, layout); +} + +template +/* static */ Literal LiteralUtil::CreateR2FromArray2D( + const Array2D& values) { + return CreateFromArray(values); +} + +template +/* static */ Literal LiteralUtil::CreateR3FromArray3DWithLayout( + const Array3D& values, const Layout& layout) { + return CreateFromArrayWithLayout(values, layout); +} + +template +/* static */ Literal LiteralUtil::CreateR3FromArray3D( + const Array3D& values) { + return CreateFromArray(values); +} + +template +/* static */ Literal LiteralUtil::CreateR3Projected( + std::initializer_list> values, + int64 projection) { + int64 dim0_size = projection; + int64 dim1_size = values.size(); + int64 dim2_size = values.begin()->size(); + + Array3D array(dim0_size, dim1_size, dim2_size); + for (int64 dim0 = 0; dim0 < dim0_size; ++dim0) { + int64 dim1 = 0; + for (auto inner_list : values) { + int64 dim2 = 0; + for (auto value : inner_list) { + array(dim0, dim1, dim2) = value; + ++dim2; + } + CHECK_EQ(dim2_size, dim2); + ++dim1; + } + CHECK_EQ(dim1_size, dim1); + } + return CreateR3FromArray3D(array); +} + +template +/* static */ Literal LiteralUtil::CreateR4Projected( + std::initializer_list> values, + int64 projection_p, int64 projection_z) { + int64 dim0_size = projection_p; + int64 dim1_size = projection_z; + int64 dim2_size = values.size(); + int64 dim3_size = values.begin()->size(); + + Array4D array(dim0_size, dim1_size, dim2_size, dim3_size); + for (int64 dim0 = 0; dim0 < dim0_size; ++dim0) { + for (int64 dim1 = 0; dim1 < dim1_size; ++dim1) { + int64 dim2 = 0; + for (auto inner_list : values) { + int64 dim3 = 0; + for (auto value : inner_list) { + array(dim0, dim1, dim2, dim3) = value; + ++dim3; + } + CHECK_EQ(dim3_size, dim3); + ++dim2; + } + CHECK_EQ(dim2_size, dim2); + } + } + return CreateR4FromArray4D(array); +} + +template +/* static */ Literal LiteralUtil::CreateR4FromArray4D( + const Array4D& values) { + return CreateFromArray(values); +} + +template +/* static */ Literal LiteralUtil::CreateR4FromArray4DWithLayout( + const Array4D& values, const Layout& layout) { + return CreateFromArrayWithLayout(values, layout); +} + +// Returns an identity matrix (rank 2) with the given row and column count. +template +/* static */ Literal LiteralUtil::MakeIdentityR2(int64 size) { + Array2D array(size, size, 0); + for (int64 i = 0; i < size; ++i) { + array(i, i) = 1; + } + return CreateR2FromArray2D(array); +} + +template +/* static */ Literal LiteralUtil::CreateFullWithDescendingLayout( + absl::Span dimensions, NativeT value) { + Literal literal(ShapeUtil::MakeShapeWithDescendingLayout( + primitive_util::NativeToPrimitiveType(), dimensions)); + literal.PopulateWithValue(value); + return literal; +} + +template +/* static */ StatusOr LiteralUtil::CreateRandomLiteral( + const Shape& shape, + const std::function)>& generator) { + using NativeT = typename primitive_util::PrimitiveTypeToNative::type; + TF_RET_CHECK(shape.element_type() == type); + Literal literal(shape); + TF_RETURN_IF_ERROR(literal.Populate( + [&](absl::Span indexes) { return generator(indexes); })); + return std::move(literal); +} + +template +/* static */ StatusOr LiteralUtil::CreateRandomLiteral( + const Shape& shape, E* engine, T mean, T stddev) { + using NativeT = typename primitive_util::PrimitiveTypeToNative::type; + std::normal_distribution generator(mean, stddev); + return CreateRandomLiteral( + shape, + [&](absl::Span /*indexes*/) { return generator(*engine); }); +} + +template +/* static */ StatusOr LiteralUtil::CreateRandomLiteral( + const Shape& shape, T mean, T stddev) { + std::minstd_rand0 engine; + return CreateRandomLiteral(shape, &engine, mean, stddev); +} + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_LITERAL_UTIL_H_ diff --git a/linux/include/tensorflow/compiler/xla/map_util.h b/linux/include/tensorflow/compiler/xla/map_util.h new file mode 100644 index 000000000..fcff48b6b --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/map_util.h @@ -0,0 +1,111 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_MAP_UTIL_H_ +#define TENSORFLOW_COMPILER_XLA_MAP_UTIL_H_ + +#include +#include + +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/compiler/xla/util.h" +#include "tensorflow/core/platform/logging.h" + +namespace xla { + +// FindOrDie returns a const reference to the value associated with +// the given key if it exists. Crashes otherwise. +// +// This is intended as a replacement for operator[] as an rvalue (for reading) +// when the key is guaranteed to exist. +template +const typename Collection::value_type::second_type& FindOrDie( + const Collection& collection, + const typename Collection::value_type::first_type& key) { + typename Collection::const_iterator it = collection.find(key); + CHECK(it != collection.end()) << "Map key not found: " << key; + return it->second; +} + +// Same as above, but returns a non-const reference. +template +typename Collection::value_type::second_type& FindOrDie( + Collection& collection, // NOLINT + const typename Collection::value_type::first_type& key) { + typename Collection::iterator it = collection.find(key); + CHECK(it != collection.end()) << "Map key not found: " << key; + return it->second; +} + +// Like FindOrDie but returns an error instead of dying if `key` is not in +// `container`. +template +StatusOr< + std::reference_wrapper> +MaybeFind(const Collection& collection, + const typename Collection::value_type::first_type& key) { + typename Collection::const_iterator it = collection.find(key); + if (it == collection.end()) { + std::ostringstream os; + os << key; + return NotFound("key not found: %s", os.str()); + } + return {it->second}; +} + +// Returns a const reference to the value associated with the given key if it +// exists, otherwise returns a const reference to the provided default value. +// +// WARNING: If a temporary object is passed as the default "value," +// this function will return a reference to that temporary object, +// which will be destroyed at the end of the statement. A common +// example: if you have a map with string values, and you pass a char* +// as the default "value," either use the returned value immediately +// or store it in a string (not string&). +template +const typename Collection::value_type::second_type& FindOrDefault( + const Collection& collection, + const typename Collection::value_type::first_type& key, + const typename Collection::value_type::second_type& value) { + auto it = collection.find(key); + if (it != collection.end()) return it->second; + return value; +} + +// Inserts the key-value pair into the collection. Dies if key was already +// present. +template +void InsertOrDie(Collection* const collection, Key&& key, Value&& value) { + auto p = collection->insert( + std::make_pair(std::forward(key), std::forward(value))); + CHECK(p.second) << "duplicate key: " << key; +} + +// Returns true if and only if the given collection contains the given key. +template +bool ContainsKey(const Collection& collection, const Key& key) { + return collection.find(key) != collection.end(); +} + +// Inserts `value` into `set`. Dies if it was already present. +template +void InsertOrDie(Set* const set, Value&& value) { + CHECK(set->insert(std::forward(value)).second) + << "duplicate value: " << value; +} + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_MAP_UTIL_H_ diff --git a/linux/include/tensorflow/compiler/xla/metric_table_report.h b/linux/include/tensorflow/compiler/xla/metric_table_report.h new file mode 100644 index 000000000..062d8ed99 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/metric_table_report.h @@ -0,0 +1,174 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_METRIC_TABLE_REPORT_H_ +#define TENSORFLOW_COMPILER_XLA_METRIC_TABLE_REPORT_H_ + +#include + +#include "absl/strings/str_cat.h" +#include "tensorflow/compiler/xla/util.h" + +namespace xla { + +// Class for creating a text format table showing entries with a metric +// (e.g. cycles) and a text (e.g. name of function taking that many +// cycles). Entries are grouped by a category and sorted in decreasing order of +// the metric. +// +// Example of a categories table generated using this class: +// +// ********** microseconds report ********** +// There are 3,912,517 microseconds in total. +// There are 123 microseconds ( 0.00%) not accounted for by the data. +// There are 3002 ops. +// +// ********** categories table ********** +// The left hand side numbers are microseconds. +// 1,749,414 (44.71% Σ44.72%) convolution (206 ops) +// * 10.51% %convolution.202 +// * 10.51% %convolution.204 +// * 10.51% %convolution.203 +// * ... (203 more ops) +// 884,939 (22.62% Σ67.33%) convolution window-dilated (7 ops) +// * 7.50% %convolution-window-dilated.7 +// [...] +// +// The entry table is similar, it just has the entries directly as the entries +// instead of grouping by categories first. +class MetricTableReport { + public: + // Represents an entry in the table. + struct Entry { + // Text to show in the entry table for this entry. + string text; + + // Text to show in the category table for this entry. + string short_text; + + // Text that represents the category of this entry - entries with the same + // category are grouped together in the category table. + string category_text; + + // The value of the metric for this entry. + double metric = 0.0; + }; + + void AddEntry(Entry entry); + + // The default name for the metric is "units", this function allows setting a + // more meaningful name. + void SetMetricName(string metric_name); + + // The default name for referring to entries is "entries", this functions + // allows setting a more meaningful name. + void SetEntryName(string entry_name); + + // By default the size of the table is limited. Calling this function forces + // all entries to be shown no matter how many there are. + void SetShowAllEntries(); + + // Set option to show a table with data on the categories of entries. + void SetShowCategoryTable(); + + // Set option to show a table with data on the entries. + void SetShowEntryTable(); + + // Returns the report as a string. expected_metric_sum is the expected sum of + // the metric across the entries. It is not an error for the actual sum to be + // different from the expectation - the report will include the + // discrepancy. All metric percentages are for the ratio with respect to the + // expected sum, not the actual sum. + string MakeReport(double expected_metric_sum); + + // As MakeReport(), but writes the report to the INFO log in a way that avoids + // cutting the report off if it is longer than the maximum line length for a + // logged line. Individual lines in the report may still be cut off, but they + // would have to be very long for that to happen. + void WriteReportToInfoLog(double expected_metric_sum); + + private: + static constexpr double kDefaultMaxMetricProportionToShow = 0.99; + static constexpr int64 kDefaultMaxEntriesToShow = 100; + static constexpr int64 kDefaultMaxEntriesPerCategoryToShow = 5; + + // Append all parameters to the report. + template + void AppendLine(Args... args) { + absl::StrAppend(&report_, std::forward(args)..., "\n"); + } + + // Represents a set of entries with the same category_text. + struct Category { + string category_text; + double metric_sum = 0.0; // Sum of metric across entries. + std::vector entries; + }; + + // Returns a vector of categories of entries with the same category_text. The + // vector is sorted in order of decreasing metric sum. + // + // The returned categories contain pointers into the entries parameter. The + // style guide requires parameters to which references/pointers are retained + // to be taken by pointer, even for const parameters, so that is why entries + // is taken by pointer. + static std::vector MakeCategories( + const std::vector* entries); + + // Append a header to the report. + void AppendHeader(); + + // Append a table of categories to the report. + void AppendCategoryTable(); + + // Append a table of entries to the report. + void AppendEntryTable(); + + // Appends a row of a table to the report. + void AppendTableRow(const string& text, const double metric, + const double running_metric_sum); + + // Returns the discrepancy between the expected sum of the metric of the + // entries and the actual sum. + double UnaccountedMetric(); + + // Formats the metric value as a string. + string MetricString(double metric); + + // Returns a string representing the metric value as a proportion of the + // expected metric sum. + string MetricPercent(double metric); + + // The entries to make a report about. + std::vector entries_; + + double expected_metric_sum_ = 0.0; + string metric_name_ = "units"; + string entry_name_ = "entries"; + bool show_category_table_ = false; + bool show_entry_table_ = false; + + // These members control how many categories and entries to show in tables. + int64 max_entries_to_show_ = kDefaultMaxEntriesToShow; + int64 max_entries_per_category_to_show_ = kDefaultMaxEntriesPerCategoryToShow; + double max_metric_proportion_to_show_ = kDefaultMaxMetricProportionToShow; + + // The report that is being created. + string report_; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_METRIC_TABLE_REPORT_H_ diff --git a/linux/include/tensorflow/compiler/xla/overflow_util.h b/linux/include/tensorflow/compiler/xla/overflow_util.h new file mode 100644 index 000000000..8657d3a4b --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/overflow_util.h @@ -0,0 +1,50 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_OVERFLOW_UTIL_H_ +#define TENSORFLOW_COMPILER_XLA_OVERFLOW_UTIL_H_ + +#include "tensorflow/core/platform/logging.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/types.h" + +namespace xla { + +// Multiply two nonnegative int64's, returning negative for overflow +inline int64 MultiplyWithoutOverflow(const int64 x, const int64 y) { + // Multiply in uint64 rather than int64 since signed overflow is undefined. + // Negative values will wrap around to large unsigned values in the casts + // (see section 4.7 [conv.integral] of the C++14 standard). + const uint64 ux = x; + const uint64 uy = y; + const uint64 uxy = ux * uy; + + // Check if we overflow uint64, using a cheap check if both inputs are small + if (TF_PREDICT_FALSE((ux | uy) >> 32 != 0)) { + // Ensure nonnegativity. Note that negative numbers will appear "large" + // to the unsigned comparisons above. + CHECK(x >= 0 && y >= 0); + + // Otherwise, detect overflow using a division + if (ux != 0 && uxy / ux != uy) return -1; + } + + // Cast back to signed. Any negative value will signal an error. + return static_cast(uxy); +} + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_OVERFLOW_UTIL_H_ diff --git a/linux/include/tensorflow/compiler/xla/primitive_util.h b/linux/include/tensorflow/compiler/xla/primitive_util.h new file mode 100644 index 000000000..889e9a1ce --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/primitive_util.h @@ -0,0 +1,227 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// Utilities for dealing with XLA primitive types. + +#ifndef TENSORFLOW_COMPILER_XLA_PRIMITIVE_UTIL_H_ +#define TENSORFLOW_COMPILER_XLA_PRIMITIVE_UTIL_H_ + +#include + +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" + +namespace xla { +namespace primitive_util { + +// The number of exponent bits in a BF16 value. +const int kBFloat16ExponentBits = 8; + +// The number of mantissa bits in a BF16 value. There is an implicit leading +// 1, so there is an implicit additional bit of precision. +const int kBFloat16MantissaBits = 7; + +// Returns the XLA primitive type (eg, F32) corresponding to the given +// template parameter native type (eg, float). +template +PrimitiveType NativeToPrimitiveType() { + // Make the expression depend on the template parameter NativeT so + // that this compile-time error only apperas if this function is + // instantiated with some concrete type that is not specialized + // below. + static_assert(!std::is_same::value, + "Cannot map native type to primitive type."); + return PRIMITIVE_TYPE_INVALID; +} + +// Declarations of specializations for each native type which correspond to a +// XLA primitive type. As an optimization, these are declared inline in the +// header. +template <> +inline PrimitiveType NativeToPrimitiveType() { + return PRED; +} + +// Unsigned integer +template <> +inline PrimitiveType NativeToPrimitiveType() { + return U8; +} + +template <> +inline PrimitiveType NativeToPrimitiveType() { + return U16; +} + +template <> +inline PrimitiveType NativeToPrimitiveType() { + return U32; +} + +template <> +inline PrimitiveType NativeToPrimitiveType() { + return U64; +} + +// Signed integer +template <> +inline PrimitiveType NativeToPrimitiveType() { + return S8; +} + +template <> +inline PrimitiveType NativeToPrimitiveType() { + return S16; +} + +template <> +inline PrimitiveType NativeToPrimitiveType() { + return S32; +} + +template <> +inline PrimitiveType NativeToPrimitiveType() { + return S64; +} + +// Floating point +template <> +inline PrimitiveType NativeToPrimitiveType() { + return F32; +} + +template <> +inline PrimitiveType NativeToPrimitiveType() { + return F64; +} + +template <> +inline PrimitiveType NativeToPrimitiveType() { + return F16; +} + +template <> +inline PrimitiveType NativeToPrimitiveType() { + return BF16; +} + +// Complex +template <> +inline PrimitiveType NativeToPrimitiveType() { + return C64; +} + +bool IsFloatingPointType(PrimitiveType type); + +bool IsComplexType(PrimitiveType type); + +bool IsSignedIntegralType(PrimitiveType type); + +bool IsUnsignedIntegralType(PrimitiveType type); + +bool IsIntegralType(PrimitiveType type); + +// Returns true if values of the given primitive type are held in array shapes. +bool IsArrayType(PrimitiveType primitive_type); + +// Returns the number of bits in the representation for a given type. +int BitWidth(PrimitiveType type); + +// Returns the real, imag component type underlying the given complex type. +// LOG(FATAL)'s if complex_type is not complex. +PrimitiveType ComplexComponentType(PrimitiveType complex_type); + +// Returns the native type (eg, float) corresponding to the given template +// parameter XLA primitive type (eg, F32). +template +struct PrimitiveTypeToNative; + +// Declarations of specializations for each native type which correspond to a +// XLA primitive type. +template <> +struct PrimitiveTypeToNative { + using type = bool; +}; + +// Unsigned integer +template <> +struct PrimitiveTypeToNative { + using type = uint8; +}; + +template <> +struct PrimitiveTypeToNative { + using type = uint16; +}; + +template <> +struct PrimitiveTypeToNative { + using type = uint32; +}; + +template <> +struct PrimitiveTypeToNative { + using type = uint64; +}; + +// Signed integer +template <> +struct PrimitiveTypeToNative { + using type = int8; +}; + +template <> +struct PrimitiveTypeToNative { + using type = int16; +}; + +template <> +struct PrimitiveTypeToNative { + using type = int32; +}; + +template <> +struct PrimitiveTypeToNative { + using type = int64; +}; + +// Floating point +template <> +struct PrimitiveTypeToNative { + using type = float; +}; +template <> +struct PrimitiveTypeToNative { + using type = double; +}; +template <> +struct PrimitiveTypeToNative { + using type = half; +}; + +template <> +struct PrimitiveTypeToNative { + using type = bfloat16; +}; + +// Complex +template <> +struct PrimitiveTypeToNative { + using type = complex64; +}; +} // namespace primitive_util +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_PRIMITIVE_UTIL_H_ diff --git a/linux/include/tensorflow/compiler/xla/protobuf_util.h b/linux/include/tensorflow/compiler/xla/protobuf_util.h new file mode 100644 index 000000000..f22fc8b84 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/protobuf_util.h @@ -0,0 +1,49 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_PROTOBUF_UTIL_H_ +#define TENSORFLOW_COMPILER_XLA_PROTOBUF_UTIL_H_ + +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/core/platform/protobuf.h" + +namespace xla { +namespace protobuf_util { + +// Returns true if m1 is equal to m2. +// +// WARNING: We use protocol buffer serialization and then check for +// equality of the serialized representation, which may miss some +// cases of equality. However, for the purposes of the XLA code +// base, this form of equality checking is sufficient. +extern bool ProtobufEquals(const tensorflow::protobuf::Message& m1, + const tensorflow::protobuf::Message& m2); + +// Writes the given message in binary proto to the path formed by joining +// 'directory/file_name.pb'. The 'directory' is recursively created if it +// doesn't already exist, and the 'file_name' is sanitized by replacing +// illegal characters with underscore '_'. +Status DumpProtoToDirectory(const tensorflow::protobuf::Message& message, + const string& directory, const string& file_name); + +// Registers a function that may either expand a dirpath or forward the original +// dirpath along as-is. +void RegisterDirectoryExpander(const std::function& expander); + +} // namespace protobuf_util +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_PROTOBUF_UTIL_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/algebraic_simplifier.h b/linux/include/tensorflow/compiler/xla/service/algebraic_simplifier.h new file mode 100644 index 000000000..9f8d0ee88 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/algebraic_simplifier.h @@ -0,0 +1,69 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_ALGEBRAIC_SIMPLIFIER_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_ALGEBRAIC_SIMPLIFIER_H_ + +#include + +#include "tensorflow/compiler/xla/service/hlo_module.h" +#include "tensorflow/compiler/xla/service/hlo_pass_interface.h" + +namespace xla { + +// A pass which performs algebraic simplifications. +class AlgebraicSimplifier : public HloModulePass { + public: + // Given shapes 'from_shape' and 'to_shape', determines if it is valid to + // bitcast from 'from_shape' to 'to_shape' after considering platform + // dependent effects on layout like alignment restrictions. Precondition: the + // two shapes have layouts, the same number of elements and + // ShapeUtil::ReshapeIsBitcast returns true. + using ValidBitcastCallback = + std::function; + + // If is_layout_sensitive is true, then the simplifier preserves layout during + // transformation. Otherwise, layout is ignored. If valid_bitcast_callback + // returns true, then the pass will replace reshapes and transposes with + // bitcasts. + AlgebraicSimplifier(bool is_layout_sensitive, + ValidBitcastCallback valid_bitcast_callback, + bool enable_dot_strength_reduction = true, + bool enable_conv_simplification = true) + : is_layout_sensitive_(is_layout_sensitive), + valid_bitcast_callback_(std::move(valid_bitcast_callback)), + enable_dot_strength_reduction_(enable_dot_strength_reduction), + enable_conv_simplification_(enable_conv_simplification) {} + ~AlgebraicSimplifier() override = default; + absl::string_view name() const override { return "algsimp"; } + + // Run algebraic simplification on the given computation. Returns whether the + // computation was changed. + StatusOr Run(HloModule* module) override; + + private: + bool is_layout_sensitive_; + ValidBitcastCallback valid_bitcast_callback_; + + // Enable dot simplification on platforms where it is profitable. + bool enable_dot_strength_reduction_; + + // Enable convolution simplification on platforms where it is profitable. + bool enable_conv_simplification_; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_ALGEBRAIC_SIMPLIFIER_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/allocation_tracker.h b/linux/include/tensorflow/compiler/xla/service/allocation_tracker.h new file mode 100644 index 000000000..af227fe4d --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/allocation_tracker.h @@ -0,0 +1,158 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_ALLOCATION_TRACKER_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_ALLOCATION_TRACKER_H_ + +#include +#include +#include +#include +#include + +#include "absl/container/flat_hash_map.h" +#include "tensorflow/compiler/xla/service/backend.h" +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/mutex.h" +#include "tensorflow/core/platform/thread_annotations.h" +#include "tensorflow/core/platform/types.h" + +namespace xla { + +// Tracks allocations for the XLA service; allocations can be registered +// with shape/device/tag and resolved from a handle for later use. +class AllocationTracker { + public: + // The allocator is used for deallocating memory when allocations are + // deregistered. All registered allocations must have the same platform as the + // allocator. + AllocationTracker(Backend* backend) : backend_(backend), next_handle_(1) {} + + // Registers a shaped buffer of device memory, and returns a corresponding + // handle that can be used for talking to XLA clients. The given shaped buffer + // will be treated as the buffer corresponding to the only replica. + StatusOr Register(ScopedShapedBuffer shaped_buffer, + const string& tag); + + // Registers a vector of shaped buffers of device memory, one per replica, and + // returns a corresponding handle that can be used for talking to XLA clients. + StatusOr RegisterReplicatedBuffers( + std::vector replicated_buffers, const string& tag); + + // Unregister the allocation for the given data handle. + Status Unregister(const GlobalDataHandle& data); + + // Returns a vector of global data handles that point to the tuple elements. + StatusOr> DeconstructTuple( + const GlobalDataHandle& Data); + + // Resolve a handle from an XLA client to a vector of shaped buffers, one per + // replica, or provide an error status to say whether any of those buffers + // were not found (or found, but found deallocated). + StatusOr> Resolve( + const GlobalDataHandle& data); + + // Resolves a handle from an XLA client and replica id to a shaped buffer, or + // provide an error status to say whether it was not found (or found, but + // found deallocated). + StatusOr ResolveForReplica(const GlobalDataHandle& data, + int replica_id); + + private: + // Data structure encapsulating single memory allocation on the device. + struct Allocation { + // The pointer to this allocation. + OwningDeviceMemory device_memory; + + // This is the number of times this memory allocation is referred to by + // registered data handles. + int ref_count; + }; + + // Internal helper which resolves the given GlobalDataHandle to a + // list of ScopedShapedBuffers. + StatusOr> ResolveInternal( + const GlobalDataHandle& data) EXCLUSIVE_LOCKS_REQUIRED(mutex_); + + // Internal helper which registers a vector of shaped buffers, one per + // replica. ShapedBufferTy is either ScopedShapedBuffer or ShapedBuffer. If + // it's ShapedBuffer, all of the given buffers must already be tracked by this + // object -- presumably this is a call from DeconstructTuple. + template + StatusOr RegisterInternal( + std::vector replicated_buffers, const string& tag) + EXCLUSIVE_LOCKS_REQUIRED(mutex_); + + // Adds the given device address to the allocation tracker, or if it already + // exists, then increment its reference count. + void AddAllocationOrIncrementRefCount(se::DeviceMemoryBase device_memory, + int device_ordinal) + EXCLUSIVE_LOCKS_REQUIRED(mutex_); + + // Decrements the reference count of the given device memory. Then, if it is + // zero, deallocate the memory. + Status DecrementRefCount(se::DeviceMemoryBase device_memory, + int device_ordinal) EXCLUSIVE_LOCKS_REQUIRED(mutex_); + + // A map from device memory opaque value to allocation. One such map is + // maintained per device ordinal. + using AllocationMap = absl::flat_hash_map; + + tensorflow::mutex mutex_; + + // Backend to use with this tracker. The backend supplies the memory allocator + // to use when deallocating memory. + Backend* backend_; + + // The next handle to assign to an allocation, guarded by the same mutex as + // the mapping as they'll be mutated at the same time. + int64 next_handle_ GUARDED_BY(mutex_); + + // A map from device ordinal to AllocationMap. + // + // This is not a TF FlatMap because (currently) FlatMap (and therefore + // AllocationMap) is not movable. + std::unordered_map opaque_to_allocation_map_ + GUARDED_BY(mutex_); + + // A map from data handle to a vector of shaped buffers that represent the + // buffers for different replicas. + // + // The ShapedBuffers in this map's vectors need to be unique_ptrs, because our + // public API returns pointers to them. We expect the concrete class to be + // ShapedBuffer and never ScopedShapedBuffer; deallocation of buffers is + // handled by opaque_to_allocation_map_. + // + // The elements of the vectors need to be unique_ptrs because we return + // pointers to them. (In theory we could use std::list or something instead, + // but we also want to be able to null out these elements.) + // + // The reason that the elements can't be unique_ptrs is + // the existence of DeconstructTuple(). This function allows us to create a + // non-owning "view" into a tuple's sub-buffers. The sub-buffers are then + // free'd when both the view *and* the original tuple are Unregistered. This + // refcounting is managed in opaque_to_allocation_map_. + absl::flat_hash_map>> + handle_to_shaped_buffers_ GUARDED_BY(mutex_); + + TF_DISALLOW_COPY_AND_ASSIGN(AllocationTracker); +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_ALLOCATION_TRACKER_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/backend.h b/linux/include/tensorflow/compiler/xla/service/backend.h new file mode 100644 index 000000000..a2dafbe80 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/backend.h @@ -0,0 +1,181 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_BACKEND_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_BACKEND_H_ + +#include +#include +#include +#include + +#include "absl/strings/str_cat.h" +#include "absl/types/span.h" +#include "tensorflow/compiler/xla/service/compiler.h" +#include "tensorflow/compiler/xla/service/computation_placer.h" +#include "tensorflow/compiler/xla/service/device_memory_allocator.h" +#include "tensorflow/compiler/xla/service/stream_pool.h" +#include "tensorflow/compiler/xla/service/transfer_manager.h" +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/core/platform/mutex.h" +#include "tensorflow/core/platform/stream_executor_no_cuda.h" +#include "tensorflow/core/platform/thread_annotations.h" + +namespace Eigen { +struct ThreadPoolDevice; +} + +namespace xla { + +// Options to configure the backend when it is created. +class BackendOptions { + public: + // Set the platform backing the backend, or nullptr for the default platform. + BackendOptions& set_platform(se::Platform* platform); + se::Platform* platform() const; + + // Sets the thread pool size for parallel execution of an individual operator. + // The default value of -1 will result in initializing the thread pool with + // the number of threads equal to the number of cores in the system. + BackendOptions& set_intra_op_parallelism_threads(int num_threads); + int intra_op_parallelism_threads() const; + + private: + se::Platform* platform_ = nullptr; + int intra_op_parallelism_threads_ = -1; +}; + +// Class which encapsulates an XLA backend. It includes everything necessary +// to compile and execute computations on a particular platform. +// +// It also offers a pooling API for creation/use of initialized streams: +// +// StreamPool::Ptr stream = backend->BorrowStream().ConsumeValueOrDie(); +class Backend { + public: + // Creates a new backend. + static StatusOr> CreateBackend( + const BackendOptions& options); + + // Creates a backend for the default platform. The default platform is defined + // in PlatformUtil. + static StatusOr> CreateDefaultBackend(); + + ~Backend(); + + // Accessors for the various objects. + se::Platform* platform() const { return platform_; } + Compiler* compiler() const { return compiler_; } + DeviceMemoryAllocator* memory_allocator() const { + return memory_allocator_.get(); + } + TransferManager* transfer_manager() const { return transfer_manager_; } + ComputationPlacer* computation_placer() const { return computation_placer_; } + + // Returns the number of devices of the platform type which are visible. Not + // all of these devices may be usable by XLA. + int device_count() const { return stream_executors_.size(); } + + // Returns the device ordinal number of the default device. + int default_device_ordinal() const; + + // Returns stream executors of all supported devices for this backend. The + // executors are ordered by the device ordinal. + const std::vector& stream_executors() const { + return stream_executors_; + } + + // Returns the stream executor for the given device ordinal. + StatusOr stream_executor(int device_ordinal) const; + + // Returns the stream executor for the default device ordinal. This stream + // executor can only be used when the number of computations is 1 (replication + // can be > 1). + se::StreamExecutor* default_stream_executor() const { + CHECK(!stream_executors_.empty()); + return stream_executors_[0]; + } + + // Borrows a stream for use by the caller, either by grabbing it from an + // internal pool, or by constructing/initializating it, and returns the result + // to the caller. + StatusOr BorrowStream(int device_ordinal); + StatusOr BorrowStream(se::StreamExecutor* executor); + + // Returns a function to borrow a stream, as `BorrowStream` above does. + // Purely for convenience, the caller could rather make this anonymous + // function itself. + std::function(int)> StreamBorrower() { + return [this](int device_ordinal) { return BorrowStream(device_ordinal); }; + } + + // Returns whether the given device ordinal of the backend is supported. + bool device_ordinal_supported(int device_ordinal) const { + return (device_ordinal >= 0 && device_ordinal < device_count() && + stream_executors_[device_ordinal] != nullptr); + } + + // Return a string identifier for the given device, eg: "GPU:3". + string device_name(int device_ordinal) const { + return absl::StrCat(platform_->Name(), ":", device_ordinal); + } + + // Returns true if the devices with the given ordinals are equivalent from + // XLA's perspective. That is, an executable compiled for one device would + // be equivalent to an executable compiled for the other. + StatusOr devices_equivalent(int device_ordinal_a, int device_ordinal_b); + + // For the host platform, returns the configured eigen threadpool device to be + // used for scheduling work. For other platforms, returns NULL. + const Eigen::ThreadPoolDevice* eigen_intra_op_thread_pool_device() const; + tensorflow::thread::ThreadPool* eigen_intra_op_thread_pool() const; + + // Resets the devices associated with this backend. + Status ResetDevices(); + + private: + struct EigenThreadPoolWrapper; + Backend(se::Platform* platform, Compiler* compiler, + absl::Span stream_executors, + TransferManager* transfer_manager, + ComputationPlacer* computation_placer, + int intra_op_parallelism_threads); + Backend(const Backend&) = delete; + Backend& operator=(const Backend&) = delete; + + se::Platform* platform_; + Compiler* compiler_; + TransferManager* transfer_manager_; + ComputationPlacer* computation_placer_; + + // Vector of stream executors. stream_executors_[0] is the default executor. + std::vector stream_executors_; + + tensorflow::mutex mu_; + + // Mapping from stream executor to stream pools, used by `BorrowStream` above. + std::map stream_pools_ GUARDED_BY(mu_); + + // The default memory allocator to use. + std::unique_ptr memory_allocator_; + + // For the CPU backend, an Eigen threadpool device for use by Eigen code. + std::unique_ptr intra_op_thread_pool_wrapper_; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_BACKEND_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/batch_dot_simplification.h b/linux/include/tensorflow/compiler/xla/service/batch_dot_simplification.h new file mode 100644 index 000000000..5b625bf3b --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/batch_dot_simplification.h @@ -0,0 +1,39 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_BATCH_DOT_SIMPLIFICATION_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_BATCH_DOT_SIMPLIFICATION_H_ + +#include "tensorflow/compiler/xla/service/hlo_module.h" +#include "tensorflow/compiler/xla/service/hlo_pass_interface.h" + +namespace xla { +// Simplifies batch dot operations. +// +// Normally these would live in the algebraic simplifier, but we want to run +// this to fixpoint (this pass reaches fixed point in one execution) before we +// run the DotDecomposer. +class BatchDotSimplification : public HloModulePass { + public: + StatusOr Run(HloModule* module) override; + absl::string_view name() const override; + + private: + StatusOr ElideDegenerateBatchDimensionFromBatchDot( + HloInstruction* batch_dot); +}; +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_BATCH_DOT_SIMPLIFICATION_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/batchnorm_expander.h b/linux/include/tensorflow/compiler/xla/service/batchnorm_expander.h new file mode 100644 index 000000000..147f3ae7b --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/batchnorm_expander.h @@ -0,0 +1,53 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_BATCHNORM_EXPANDER_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_BATCHNORM_EXPANDER_H_ + +#include + +#include "tensorflow/compiler/xla/service/hlo_module.h" +#include "tensorflow/compiler/xla/service/hlo_pass_interface.h" + +namespace xla { + +// A pass which rewrites batch norm operations into more operations. Breaking a +// big operation into smaller operations helps leverage our generic fusion +// logic. +class BatchNormExpander : public HloModulePass { + public: + // When use_fusion is set, a multi-output fusion node is created. + BatchNormExpander(bool rewrite_training_op = false, + bool rewrite_inference_op = false, + bool rewrite_grad_op = false) + : rewrite_training_op_(rewrite_training_op), + rewrite_inference_op_(rewrite_inference_op), + rewrite_grad_op_(rewrite_grad_op) {} + ~BatchNormExpander() = default; + absl::string_view name() const override { return "batchnorm_expander"; } + + // Run operation expander on the given computation. Returns whether the + // computation was changed. + StatusOr Run(HloModule* module) override; + + private: + bool rewrite_training_op_; + bool rewrite_inference_op_; + bool rewrite_grad_op_; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_BATCHNORM_EXPANDER_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/buffer_assignment.h b/linux/include/tensorflow/compiler/xla/service/buffer_assignment.h new file mode 100644 index 000000000..899cd36e1 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/buffer_assignment.h @@ -0,0 +1,645 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_BUFFER_ASSIGNMENT_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_BUFFER_ASSIGNMENT_H_ + +#include +#include +#include +#include +#include + +#include "absl/container/flat_hash_map.h" +#include "absl/container/flat_hash_set.h" +#include "absl/types/span.h" +#include "tensorflow/compiler/xla/service/buffer_liveness.h" +#include "tensorflow/compiler/xla/service/heap_simulator.h" +#include "tensorflow/compiler/xla/service/hlo.pb.h" +#include "tensorflow/compiler/xla/service/hlo_computation.h" +#include "tensorflow/compiler/xla/service/hlo_instruction.h" +#include "tensorflow/compiler/xla/service/hlo_module.h" +#include "tensorflow/compiler/xla/service/logical_buffer.h" +#include "tensorflow/compiler/xla/service/tuple_points_to_analysis.h" +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/core/platform/logging.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/types.h" + +namespace xla { + +// Walk the call graph of the HLO module and place each computation into either +// thread_local_computations or global_computations depending upon whether the +// computation requires thread-local allocations or global allocations. The +// elements in thread_local_computations and global_computations are in post +// order (if computation A has an instruction which calls computation B, then A +// will appear after B in the vector). +Status GatherComputationsByAllocationType( + const HloModule* module, + std::vector* thread_local_computations, + std::vector* global_computations); + +// This class abstracts an allocation of contiguous memory which can hold the +// values described by LogicalBuffers. Each LogicalBuffer occupies a sub-range +// of the allocation, represented by a Slice. A single BufferAllocation may hold +// LogicalBuffers with disjoint liveness, which may have overlapping Slices. A +// single BufferAllocation may also hold LogicalBuffers with overlapping +// liveness, which must have disjoint Slices. +// +// The abstraction includes information required by the backends for allocation, +// use, and deallocation of the buffer. This includes the LogicalBuffers which +// are held in this allocation through the execution of the computation. +class BufferAllocation { + public: + // Holds a unique identifier for each allocation. Values are assigned + // contiguously and can be used as array indexes. + using Index = int64; + + BufferAllocation(Index index, int64 size, LogicalBuffer::Color color) + : index_(index), size_(size), color_(color) {} + ~BufferAllocation() {} + + // Returns the index of this allocation. + Index index() const { return index_; } + + // Whether this allocation is used in a parallel calling context such as + // inside of a map or reduce computation. Such allocations need to be thread + // local. + bool is_thread_local() const { return is_thread_local_; } + void set_is_thread_local(bool is_thread_local) { + is_thread_local_ = is_thread_local; + } + + // Whether this allocation can be used by more than one logical buffer. + bool is_reusable() const { + // We do not reuse thread-local buffers for now, because they are + // dynamically allocated and their lifetimes are hard to compute. + // + // TODO(b/34669761): Don't reuse tuple buffers because the GPU backend + // assumes longer buffer liveness than indicated by the analysis. + return !is_thread_local() && !is_tuple(); + } + + // Whether this allocation is readonly i.e. backed by memory we cannot write + // to. + bool is_readonly() const { + return is_entry_computation_parameter() || is_constant(); + } + + bool is_tuple() const { return is_tuple_; } + void set_is_tuple(bool is_tuple) { is_tuple_ = is_tuple; } + + // Whether this allocation holds a LogicalBuffer from a parameter of the entry + // computation. These buffers have lifetimes which may be longer than the + // XLA computation. + bool is_entry_computation_parameter() const { + return is_entry_computation_parameter_; + } + + // Whether this allocation holds a constant. On the CPU and GPU backends + // constant allocations are not allocated dynamically, instead we resolve + // references to these buffer allocations to a global in the readonly section + // of the binary. + bool is_constant() const { return is_constant_; } + + // If this allocation holds a Buffer from a parameter of the entry + // computation, this methods returns the parameter number. CHECKs otherwise. + int64 parameter_number() const { + CHECK(is_entry_computation_parameter_); + return parameter_number_; + } + + // If this allocation is for a parameter of the entry computation, this + // function returns which subshape of the parameter the allocation is for. + const ShapeIndex& param_shape_index() const { + CHECK(is_entry_computation_parameter_); + return param_shape_index_; + } + + // Returns whether this allocation is assigned a LogicalBuffer which may + // be live out of the entry computation. + bool maybe_live_out() const { return maybe_live_out_; } + + // Returns the size of the allocation. Necessarily this must be at least as + // large as any LogicalBuffer assigned to this allocation. + int64 size() const { return size_; } + + // Returns the color of the allocation. Only logical buffers with a matching + // color can reside in this allocation. + LogicalBuffer::Color color() const { return color_; } + + struct OffsetSize { + int64 offset = 0; + int64 size = 0; + }; + + // Access to the logical buffers assigned to this allocation, and their + // associated logical offsets and sizes. + const absl::flat_hash_map& + assigned_buffers() const { + return assigned_buffers_; + } + + // A Slice represents a contiguous portion of a memory allocation. It is used + // to identify the memory range that a LogicalBuffer corresponds to. + class Slice { + public: + Slice() {} + Slice(const BufferAllocation* allocation, int64 offset, int64 size) + : allocation_(allocation), offset_(offset), size_(size) {} + + const BufferAllocation* allocation() const { return allocation_; } + Index index() const { return allocation_->index(); } + int64 offset() const { return offset_; } + int64 size() const { return size_; } + + bool operator==(const Slice& other) const { + return index() == other.index() && offset_ == other.offset_ && + size_ == other.size_; + } + bool operator!=(const Slice& other) const { return !(*this == other); } + bool operator<(const Slice& other) const { + if (index() != other.index()) return index() < other.index(); + if (offset_ != other.offset_) return offset_ < other.offset_; + return size_ < other.size_; + } + + // Returns true iff this slice's memory range has a non-empty intersection + // with the other slice's memory range. + bool OverlapsWith(const Slice& other) const { + const int64 end = offset_ + size_; + const int64 other_end = other.offset_ + other.size_; + return index() == other.index() && offset_ < other_end && + end > other.offset_; + } + + struct Hasher { + size_t operator()(Slice s) const; + }; + + string ToString() const; + + private: + const BufferAllocation* allocation_ = nullptr; + int64 offset_ = 0; + int64 size_ = 0; + }; + + // GetSlice returns the Slice of contiguous memory that holds the value + // described by the given 'buffer'. + // REQUIRES: 'buffer' must be assigned to this allocation. + Slice GetSlice(const LogicalBuffer& buffer) const; + + string ToString() const; + BufferAllocationProto ToProto() const; + + // Whether the buffer is a parameter to or live out of the entry computation. + bool IsInputOrOutput() const { + return is_entry_computation_parameter() || maybe_live_out(); + } + + // Whether the buffer is a temporary buffer allocated before + // Executable::ExecuteOnStream. + bool IsPreallocatedTempBuffer() const { + // Parameters do not need temporary buffers. + return !is_entry_computation_parameter() && + // LogicalBuffers that maybe pointed to by the output should live out + // of the computation. + !maybe_live_out() && + // Thread-local buffers are allocated using `alloca`s. + !is_thread_local() && + // Constant buffers are allocated as global values. + !is_constant(); + } + + // Add a heap trace which was used to assign slices to logical buffers in this + // allocation. A single BufferAllocation may include multiple heap traces + // in the case of the temporary block where there is a heap trace per + // computation. + void AddHeapTrace(const HeapSimulatorTrace& heap_trace) { + heap_traces_.push_back(heap_trace); + } + + // Return the set of heap traces used to assign slices to logical buffers in + // this allocation. + const std::vector HeapTraces() const { + return heap_traces_; + } + + // Returns the LogicalBuffers which are live at the point of peak memory usage + // for this allocation. The point of peak memory usage is the point at which + // the total size of all live logical buffers is maximal. If peak memory is + // reached at multiple points, the set of logical buffers live at the earliest + // maximal point is returned. The vector is stabily sorted by + // LogicalBuffer::Index. + const std::vector& PeakMemoryLogicalBuffers() const { + return peak_buffers_; + } + + // Get the number of bytes lost to fragmentation. This is equal to the + // difference between the size of the allocation and the size of the maximal + // live set. + int64 fragmentation_bytes() const { return fragmentation_bytes_; } + + bool operator==(const BufferAllocation& other) const { + return index_ == other.index_; + } + bool operator!=(const BufferAllocation& other) const { + return !(*this == other); + } + bool operator<(const BufferAllocation& other) const { + return index() < other.index(); + } + + private: + // Only BufferAssigner and BufferAssignment can modify BufferAllocation. + friend class BufferAssigner; + friend class BufferAssignment; + + // Adds a LogicalBuffer to the set assigned to this buffer. + void AddAssignment(const LogicalBuffer& buffer, int64 offset, int64 size); + + void set_entry_computation_parameter(int64 parameter_number, + ShapeIndex param_shape_index) { + is_entry_computation_parameter_ = true; + parameter_number_ = parameter_number; + param_shape_index_ = std::move(param_shape_index); + } + + void set_constant(bool is_constant) { is_constant_ = is_constant; } + void set_maybe_live_out(bool value) { maybe_live_out_ = value; } + void set_index(Index index) { index_ = index; } + void set_size(int64 size) { size_ = size; } + + // The index of the allocation in the BufferAssignment. + Index index_; + + // Size of the allocation in bytes. + int64 size_; + + // Whether this buffer needs to be thread-local. + bool is_thread_local_ = false; + + // Whether this buffer holds a tuple. + bool is_tuple_ = false; + + // Color of the allocation. + LogicalBuffer::Color color_; + + // Whether this allocation holds an entry computation parameter. Entry + // computation parameters are special be cause they have lifetimes which may + // outlast the computation. + bool is_entry_computation_parameter_ = false; + + // If this allocation holds an entry computation parameter, this field + // indicates the index (starting from 0) of the parameter. + int64 parameter_number_ = 0; + + // If this buffer is for an entry computation parameter, which subshape of the + // parameter is it for? + ShapeIndex param_shape_index_; + + // Whether the allocation contains a LogicalBuffer which may be live-out of + // the entry computation. Note that this flag is conservatively computed by + // TuplePointsToAnalysis. That is, an allocation marked `maybe_live_out_` + // might not actually escape. + bool maybe_live_out_ = false; + + // See comment on the is_constant() accessor. + bool is_constant_ = false; + + // Mapping from the set of buffers assigned to this allocation to their + // logical offsets and sizes. + absl::flat_hash_map assigned_buffers_; + + int64 fragmentation_bytes_ = 0; + std::vector heap_traces_; + + // Set of buffers live at the point of peak memory usage for this allocation. + std::vector peak_buffers_; +}; + +// Add stream operators for nicer output of CHECK/RET_CHECK failures. +std::ostream& operator<<(std::ostream& out, const BufferAllocation& s); +std::ostream& operator<<(std::ostream& out, const BufferAllocation::Slice& s); + +// This class encapsulates an assignment of the LogicalBuffers in an XLA +// module to a set of BufferAllocations. +class BufferAssignment { + public: + // Returns the vector containing all buffer allocations in this assignment. + const std::vector& Allocations() const { + return allocations_; + } + + // Returns the total size allocation holding all temporary buffers. + int64 temp_allocation_total_size() const { + return temp_allocation_total_size_; + } + + // Returns whether the given buffer has been assigned an allocation. + bool HasAllocation(const LogicalBuffer& buffer) const; + + // Returns the allocation that a particular LogicalBuffer has been assigned + // to. CHECKs if buffer has not been assigned an allocation. + const BufferAllocation& GetAssignedAllocation( + const LogicalBuffer& buffer) const; + + // Returns the allocation with the given index. CHECKs if no allocation exists + // with the given index. + const BufferAllocation& GetAllocation(BufferAllocation::Index index) const; + + // Builds and returns a vector containing the slices which might contain the + // subvalue at the given index of given instruction. + std::set GetAllSlices( + const HloInstruction* instruction, const ShapeIndex& index) const; + + // Convenience function which returns whether the buffer of the + // instruction at the given index is assigned an allocation. + bool HasAllocationAt(const HloInstruction* instruction, + const ShapeIndex& index) const; + + // Convenience function which returns whether the top-level buffer of the + // instruction (index == {}) is assigned an allocation. + bool HasTopLevelAllocation(const HloInstruction* instruction) const; + + // Convenience function which returns the unique slice containing the buffer + // at the given index of the given instruction. If a slice is not assigned or + // the slice cannot be determined at compile time then an error is returned. + StatusOr GetUniqueSlice( + const HloInstruction* instruction, const ShapeIndex& index) const; + // Like GetUniqueSlice but fixes the index to the top-level of the shape + // (index = {}). + StatusOr GetUniqueTopLevelSlice( + const HloInstruction* instruction) const; + // Like GetUniqueTopLevelSlice but returns the slice for the output of the + // entry computation of the HLO module (ie, the result of the XLA + // computation). + StatusOr GetUniqueTopLevelOutputSlice() const; + + // Returns the set LogicalBuffers which may be the source of the value at the + // given index and instruction. + const PointsToSet::BufferList& GetSourceBuffers( + const HloInstruction* instruction, const ShapeIndex& index) const { + return GetPointsToSet(instruction).element(index); + } + + // Returns true if 'hlo_a{shape_index_a}' and 'hlo_b{shape_index_b}' + // share the same BufferAllocation::Slice. + // Returns false otherwise. + // REQUIRES: BufferAssignment assigned allocations to both instructions. + bool SharesSliceAtIndex(const HloInstruction* hlo_a, + const ShapeIndex& shape_index_a, + const HloInstruction* hlo_b, + const ShapeIndex& shape_index_b) const; + + // Returns true if the top-level buffers of hlo_a and hlo_b are the same. + // REQUIRES: HasTopLevelAllocation(hlo_a) && HasTopLevelAllocation(hlo_b). + bool SharesTopLevelSlice(const HloInstruction* hlo_a, + const HloInstruction* hlo_b) const { + return SharesSliceAtIndex(hlo_a, {}, hlo_b, {}); + } + + // Returns true if hlo_a and hlo_b both have at least one buffer assigned for + // their top-level and each of their nested shape indices, and if hlo_a's + // buffers are all different from hlo_b's buffers. + bool HaveDisjointSlices(const HloInstruction* hlo_a, + const HloInstruction* hlo_b) const; + + // Returns the underlying points-to analysis used for this assignment. + const TuplePointsToAnalysis& points_to_analysis() const { + return liveness_->points_to_analysis(); + } + + // Returns the BufferLiveness object used to construct this assignment. + const BufferLiveness& liveness() const { return *liveness_; } + + string ToString() const; + BufferAssignmentProto ToProto() const; + + // Statistics for the assignment. Values initialized to -1 are not always + // collected; fragmentation is only collected for instructions that have a + // sequential total ordering. + struct Stats { + int64 parameter_allocation_count = 0; + int64 parameter_allocation_bytes = 0; + int64 constant_allocation_count = 0; + int64 constant_allocation_bytes = 0; + int64 maybe_live_out_allocation_count = 0; + int64 maybe_live_out_allocation_bytes = 0; + int64 preallocated_temp_allocation_count = 0; + int64 preallocated_temp_allocation_bytes = 0; + int64 preallocated_temp_fragmentation_bytes = -1; + int64 total_allocation_count = 0; + int64 total_allocation_bytes = 0; + int64 total_fragmentation_bytes = -1; + + string ToString() const; + }; + const Stats& GetStats() const { return stats_; } + + private: + // Only BufferAssigner can build or modify BufferAssignments. + friend class BufferAssigner; + + BufferAssignment(const HloModule* module, + std::unique_ptr liveness, + LogicalBuffer::SizeFunction buffer_size, + LogicalBuffer::AlignmentFunction color_alignment) + : module_(module), + liveness_(std::move(liveness)), + buffer_size_(std::move(buffer_size)), + color_alignment_(std::move(color_alignment)) {} + + // Creates and returns a new BufferAllocation, with no assigned + // LogicalBuffers. Ownership is maintained internally. + BufferAllocation* NewEmptyAllocation(int64 size, LogicalBuffer::Color color); + + // Helper that calls NewEmptyAllocation and AddAssignment in one call, + // creating an allocation containing a single LogicalBuffer. + BufferAllocation* NewAllocation(const LogicalBuffer& buffer, int64 size); + + // Adds a LogicalBuffer to the set assigned to the given allocation. + void AddAssignment(BufferAllocation* allocation, const LogicalBuffer& buffer, + int64 offset, int64 size); + + // Returns the HloModule used to construct this assignment. + const HloModule& module() const { return *module_; } + + // Convenience function which returns the PointsToSet for the given + // instruction. Extracted from the liveness object. + const PointsToSet& GetPointsToSet(const HloInstruction* instruction) const; + + // Mutable accessors for allocations. + BufferAllocation* GetMutableAssignedAllocation(const LogicalBuffer& buffer); + BufferAllocation* GetMutableAllocation(BufferAllocation::Index index); + + // Combines allocations of temporary buffers into one big BufferAllocation. + void CombineTempAllocations(); + + // Computes stats for the assignment, to be retrieved by GetStats. + Status ComputeSummaryStats(); + + // The vector of buffer allocations. Indexed by BufferAllocation::Index. + std::vector allocations_; + + // The total size of all temporary buffers. + int64 temp_allocation_total_size_ = 0; + + // Maps Buffers to the index of the BufferAllocation which holds the buffer. + absl::flat_hash_map + allocation_index_for_buffer_; + + const HloModule* module_; + const std::unique_ptr liveness_; + + // Function which returns the buffer size for a given logical buffer (shape). + LogicalBuffer::SizeFunction buffer_size_; + + // Function which returns the alignment for a given logical buffer color. + LogicalBuffer::AlignmentFunction color_alignment_; + + Stats stats_; + + TF_DISALLOW_COPY_AND_ASSIGN(BufferAssignment); +}; + +// A class which constructs a buffer assignment. +class BufferAssigner { + public: + // Build and return a BufferAssignment for the given module. The given + // HloOrdering is used to determine buffer liveness. buffer_size and + // color_alignment are functions which returns the size and alignment of a + // LogicalBuffer. allow_input_output_aliasing specifies whether input buffer + // are allowed to be reused as outbut buffers by the client code. + static StatusOr> Run( + const HloModule* module, std::unique_ptr hlo_ordering, + LogicalBuffer::SizeFunction buffer_size, + LogicalBuffer::AlignmentFunction color_alignment, + bool allow_input_output_aliasing = false, + bool allocate_buffers_for_constants = false, + BufferLiveness::Colorer colorer = BufferLiveness::DefaultColorer()); + + private: + BufferAssigner(bool allow_input_output_aliasing, + bool allocate_buffers_for_constants, + BufferLiveness::Colorer colorer) + : allow_input_output_aliasing_(allow_input_output_aliasing), + allocate_buffers_for_constants_(allocate_buffers_for_constants), + colorer_(colorer) {} + virtual ~BufferAssigner() = default; + + // Create a buffer assignment. + StatusOr> CreateAssignment( + const HloModule* module, std::unique_ptr hlo_ordering, + LogicalBuffer::SizeFunction buffer_size, + LogicalBuffer::AlignmentFunction color_alignment); + + // Assigns buffers to the instructions in the given computation. "assignment" + // is modified to reflect the new buffer assignments. If is_thread_local is + // true, then all assigned buffers have the is_thread_local flag set to + // true. + Status AssignBuffersForComputation( + const HloComputation* computation, bool is_thread_local, + const absl::flat_hash_set& colocated_buffers, + const absl::flat_hash_set& colocated_allocations, + absl::flat_hash_map>* + buffers_to_assign_sequentially, + BufferAssignment* assignment); + + // Assigns 'buffers_to_assign_sequentially' using heap simulation, assuming + // the HLO instructions will be executed in the sequential order given by + // assignment->liveness().hlo_ordering().SequentialOrder. If + // 'run_whole_module_heap_simulation' is true, the heap simulation will be run + // assuming all global computations are sequentially ordered. + Status AssignBuffersWithSequentialOrdering( + const absl::flat_hash_map>& + buffers_to_assign_sequentially, + bool run_whole_module_heap_simulation, BufferAssignment* assignment); + + // Uses the results of the heap simulator to create a single allocation, with + // LogicalBuffers packed to specific offsets. + void AssignBuffersFromHeapSimulator(const HeapSimulator::Result& result, + BufferAssignment* assignment, + LogicalBuffer::Color color); + + // Tries to assign the given instruction to the given buffer. Returns if the + // assignment was successful. + bool MaybeAssignBuffer(BufferAllocation* allocation, + const LogicalBuffer& buffer, + BufferAssignment* assignment); + + // Colocated buffers are logical buffers from different computations which + // alias. Explicitly handling these colocated buffers is necessary because + // points-to analysis is computation level scope and does not recognize + // aliasing across computations (b/32491382). + using ColocatedBufferSet = absl::flat_hash_set; + + // Returns a vector of ColocatedBufferSet objects, where each + // ColocatedBufferSet aggregates a set of related LogicalBuffers from 'module' + // which should be colocated in the same buffer allocation. + void BuildColocatedBufferSets( + const HloModule* module, const BufferLiveness& buffer_liveness, + const LogicalBuffer::SizeFunction& buffer_size, + std::vector* colocated_buffer_sets); + + // For each buffer set in 'colocated_buffer_sets', assigns all buffers in the + // same set to the same buffer allocation in 'assignment'. + void AssignColocatedBufferSets( + const std::vector& colocated_buffer_sets, + BufferAssignment* assignment, + absl::flat_hash_set* colocated_buffers, + absl::flat_hash_set* colocated_allocations); + + // Adds the 'colocated_set' of buffers to 'colocated_buffer_sets', maintaining + // the invariant that all sets in 'colocated_buffer_sets' are disjoint. + void AddSetToColocatedBufferSets( + const std::vector& colocated_set, + std::vector* colocated_buffer_sets); + + // Given a list of colocated buffer sets (each colocated buffer set represents + // the logical buffers that would be assigned to the same physical buffer), + // try to merge the sets if the buffers can be shared. Returns the merged set. + std::vector MergeColocatedBufferSets( + const std::vector& colocated_buffer_sets, + const BufferLiveness& buffer_liveness, + const LogicalBuffer::SizeFunction& buffer_size); + + // Split a set of buffers into several sets, each of which contains buffers + // colored with the same color. + absl::flat_hash_map, + LogicalBuffer::Color::Hasher> + SplitBuffersByColor(const absl::flat_hash_set& buffers); + + // If true, buffer assignments assumes that input parameter buffers and output + // buffers can be shared if their sizes match. + bool allow_input_output_aliasing_; + + // If true, allocate buffers for constant instructions. + bool allocate_buffers_for_constants_; + + // Functor used to assign colors to newly allocated logical buffers. + BufferLiveness::Colorer colorer_; + + TF_DISALLOW_COPY_AND_ASSIGN(BufferAssigner); +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_BUFFER_ASSIGNMENT_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/buffer_liveness.h b/linux/include/tensorflow/compiler/xla/service/buffer_liveness.h new file mode 100644 index 000000000..f939a426e --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/buffer_liveness.h @@ -0,0 +1,114 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_BUFFER_LIVENESS_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_BUFFER_LIVENESS_H_ + +#include +#include +#include + +#include "absl/container/flat_hash_set.h" +#include "tensorflow/compiler/xla/service/hlo_instruction.h" +#include "tensorflow/compiler/xla/service/hlo_module.h" +#include "tensorflow/compiler/xla/service/hlo_ordering.h" +#include "tensorflow/compiler/xla/service/tuple_points_to_analysis.h" +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/core/lib/core/status.h" + +namespace xla { + +// Class which computes liveness of the output buffers of HLOs and their +// interference. +class BufferLiveness { + public: + using Colorer = std::function; + + // Constructs a buffer liveness object for the given module assuming the given + // HLO instruction ordering. + static StatusOr> Run( + const HloModule* module, std::unique_ptr hlo_ordering); + + // Returns true if the live range of the buffer containing the output of 'a' + // may overlap with the live range of the buffer of 'b'. If instruction 'a' + // interferes with instruction 'b' then they cannot share the same buffer. + bool MayInterfere(const LogicalBuffer& a, const LogicalBuffer& b) const; + + // Returns true if the buffer for the given instruction may be live out of the + // module. That is, the instruction's buffer may be included in the output of + // the entry computation. + bool MaybeLiveOut(const LogicalBuffer& buffer) const; + + // Returns the complete set of buffers that may be live out of the module. + const PointsToSet::BufferSet& maybe_live_out_buffers() const { + return maybe_live_out_buffers_; + } + + // Returns the underlying points-to analysis used for this liveness analysis. + const TuplePointsToAnalysis& points_to_analysis() const { + return *points_to_analysis_; + } + + // Returns the underlying hlo ordering used for this liveness analysis. + const HloOrdering& hlo_ordering() const { return *hlo_ordering_; } + + const HloModule& module() const { return *module_; } + + string ToString() const; + + static Colorer DefaultColorer() { + return [](const BufferLiveness& buffer_liveness) { + for (LogicalBuffer::Id id = 0; + id < buffer_liveness.points_to_analysis().num_logical_buffers(); + id++) { + auto& buffer = buffer_liveness.points_to_analysis().logical_buffer(id); + buffer.set_color(LogicalBuffer::Color(0)); + } + return Status::OK(); + }; + } + + private: + explicit BufferLiveness(const HloModule* module, + std::unique_ptr hlo_ordering) + : module_(module), hlo_ordering_(std::move(hlo_ordering)) {} + + // Perform buffer liveness analysis. This method must be called prior to + // MayInterfere or MaybeLiveOut. + Status Analyze(); + + // Returns true if the live range of the buffer of 'a' is strictly before the + // live range of the buffer of 'b' (they do not overlap). + bool live_range_strictly_before(const LogicalBuffer& a, + const LogicalBuffer& b) const; + + const HloModule* module_; + std::unique_ptr hlo_ordering_; + + // Set of LogicalBuffers which are aliased in the output of other + // instructions. For example, a LogicalBuffer which is inserted into a tuple + // is considered to be aliased and will be in this set. + absl::flat_hash_set aliased_buffers_; + + // LogicalBuffers that may be live out of the entry computation. + PointsToSet::BufferSet maybe_live_out_buffers_; + + std::unique_ptr points_to_analysis_; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_BUFFER_LIVENESS_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/buffer_value.h b/linux/include/tensorflow/compiler/xla/service/buffer_value.h new file mode 100644 index 000000000..69b364635 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/buffer_value.h @@ -0,0 +1,177 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_BUFFER_VALUE_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_BUFFER_VALUE_H_ + +#include +#include + +#include "absl/types/span.h" +#include "tensorflow/compiler/xla/service/hlo.pb.h" +#include "tensorflow/compiler/xla/service/hlo_instruction.h" +#include "tensorflow/compiler/xla/shape_util.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/lib/gtl/int_type.h" +#include "tensorflow/core/platform/logging.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/types.h" + +namespace xla { + +// Abstract class describing a value used by one of the dataflow analyses - +// TuplePointsToAnalysis or HloDataflowAnalysis. +// TODO(b/78906445) Delete this class when TuplePointsToAnalysis is unused. +// +// XLA arrays are trivially a single BufferValue. Tuples are made up of more +// than one BufferValue: an BufferValue for the pointer vector, and an +// BufferValue for each child element. +// +// Every BufferValue is defined by a particular instruction and most +// instructions define only a single BufferValue. Instructions which define a +// single BufferValue include array-shaped instructions such as Add but also +// includes Tuple-shaped instructions such as Tuple. The Tuple instruction +// defines a single BufferValue which is a vector of pointers to the values +// containing the Tuple instruction's operands. Though the result of the Tuple +// instruction includes multiple values only the top-level BufferValue (the +// vector of pointers) is defined by the Tuple instruction. The values +// containing the tuple elements are defined by earlier instructions, usually +// the operands of the Tuple instruction. +// +// Instructions which construct both the tuple *and* the tuple elements define +// more than one BufferValue. This includes (at least) tuple-shaped Constant, +// Parameter, Infeed and While instructions. These tuple-shaped instructions do +// not assemble a tuple from existing BufferValues like the Tuple instruction +// does, but rather define all the BufferValues in the tuple. +// +// Some instructions, such as Bitcast, define no buffers. These instructions +// simply forward buffers from their operands. +// +// The BufferValue object describes which HLO instruction defines a buffer and +// where within that instruction's output shape the buffer is defined. The +// location within the output shape is indicated by BufferValue::index() which +// is defined identically to the index used in ShapeUtil::GetSubshape(). +// Examples: +// +// %add = Add(%foo, %bar) +// %tuple_constant = Constant({1, {42, 43}}) +// +// %add defines a single array-shaped buffer BufferValue(%add, {}) which holds +// the array result of the add operation. The nested-tuple-shaped +// %tuple_constant defines 5 buffers described by the following BufferValue +// objects: +// +// BufferValue(%tuple_constant, {}) // "Top-level" buffer: vector of +// // pointers to BufferValues at +// // indices {0} and {1} +// BufferValue(%tuple_constant, {0}) // Holds value "1" +// BufferValue(%tuple_constant, {1}) // Holds nested tuple: vector of +// // pointers to BufferValues at +// // indices {1, 0} and {1, 1} +// BufferValue(%tuple_constant, {1, 0}) // Holds value "42" +// BufferValue(%tuple_constant, {1, 1}) // Holds value "43" + +class BufferValue { + public: + TF_LIB_GTL_DEFINE_INT_TYPE(Color, int64); + + // Id is a unique identifier for the BufferValue to facilitate efficient + // collections of BufferValues with stable iteration order. + using Id = int64; + + // Functions which return the size and alignment of a logical buffer in bytes. + using SizeFunction = std::function; + using AlignmentFunction = std::function; + + virtual ~BufferValue(); + + Id id() const { return id_; } + + // Return the instruction that defines the buffer. + virtual HloInstruction* instruction() const = 0; + + // Return the index within the output of the instruction where the buffer is + // defined. Index used defined as in ShapeUtil::GetSubshape() + virtual const ShapeIndex& index() const = 0; + + // Return the color of the BufferValue. Differently colored buffers can not be + // parts of the same allocation. + Color color() const { + CHECK_NE(color_, kInvalidColor) + << "Should not query the color of a buffer that was never colored"; + return color_; + } + + void set_color(Color color) { + CHECK_NE(color, kInvalidColor) + << "Should not set the color of a buffer to the invalid color"; + color_ = color; + } + + bool has_color() const { return color_ != kInvalidColor; } + + // Return the shape of the buffer. This reference points into the shape field + // of the instruction defining the buffer. Therefore, the returned shape will + // contain the layout of instruction, if any. + virtual const Shape& shape() const = 0; + + // Returns true if this buffer is the top-level output buffer of the defining + // HLO instruction. This is equivalent to index == {}. + bool IsTopLevel() const { return index().empty(); } + + // Whether this buffer contains a tuple. + bool IsTuple() const { return is_tuple_; } + + // Whether this buffer contains an array. + bool IsArray() const { return is_array_; } + + // operator< is required for std::set. + bool operator<(const BufferValue& other) const { return id_ < other.id_; } + + virtual string ToString() const = 0; + + // TODO(lauj) rename LogicalBufferProto to BufferValueProto. + LogicalBufferProto ToProto(const SizeFunction& size_fn) const; + + // Returns the LogicalBufferProto::Location that serializes the given + // instruction and index. + static LogicalBufferProto::Location ToLocationProto( + const HloInstruction& instruction, const ShapeIndex& index); + + const Color kInvalidColor = Color(-1); + + protected: + BufferValue(HloInstruction* instruction, const ShapeIndex& index, Id id); + + private: + // The definining instruction and index are not stored here; they can be found + // in the LogicalBuffer and HloValue subclasses. This class exists only to + // support migrations from TuplePointsToAnalysis to HloDataflowAnalysis, by + // allowing abstract use of LogicalBuffer or HloValue. After those migrations + // are complete, this class should be deleted (b/78906445). Because we plan to + // delete LogicalBuffer and this class, we don't refactor all the shared + // features from LogicalBuffer and HloValue into this class. + Id id_ : 62; + bool is_array_ : 1; + bool is_tuple_ : 1; + Color color_ = kInvalidColor; +}; + +std::ostream& operator<<(std::ostream& out, const BufferValue& buffer); + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_BUFFER_VALUE_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/buffer_value_containers.h b/linux/include/tensorflow/compiler/xla/service/buffer_value_containers.h new file mode 100644 index 000000000..cc46af5ee --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/buffer_value_containers.h @@ -0,0 +1,55 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_BUFFER_VALUE_CONTAINERS_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_BUFFER_VALUE_CONTAINERS_H_ + +#include "absl/container/flat_hash_set.h" +#include "tensorflow/compiler/xla/service/buffer_value.h" +#include "tensorflow/compiler/xla/service/logical_buffer.h" +#include "tensorflow/core/lib/gtl/compactptrset.h" + +namespace xla { + +// Define various containers of BufferValues, and utilities to convert from +// containers of LogicalBuffers to containers of BufferValues. + +using BufferValueCompactPointerSet = + tensorflow::gtl::CompactPointerSet; +template +BufferValueCompactPointerSet ToBufferValueCompactPointerSet( + const LogicalBufferContainerT& logical_buffer_container) { + BufferValueCompactPointerSet output; + for (const LogicalBuffer* buffer : logical_buffer_container) { + output.insert(buffer); + } + return output; +} + +using BufferValueFlatSet = absl::flat_hash_set; +template +BufferValueFlatSet ToBufferValueFlatSet( + const LogicalBufferContainerT& logical_buffer_container) { + BufferValueFlatSet output; + output.reserve(logical_buffer_container.size()); + for (const LogicalBuffer* buffer : logical_buffer_container) { + output.insert(buffer); + } + return output; +} + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_BUFFER_VALUE_CONTAINERS_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/call_graph.h b/linux/include/tensorflow/compiler/xla/service/call_graph.h new file mode 100644 index 000000000..cb56f4789 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/call_graph.h @@ -0,0 +1,275 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// Call graph for an HLO module. + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_CALL_GRAPH_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_CALL_GRAPH_H_ + +#include + +#include "absl/container/flat_hash_map.h" +#include "absl/container/flat_hash_set.h" +#include "tensorflow/compiler/xla/service/hlo_computation.h" +#include "tensorflow/compiler/xla/service/hlo_instruction.h" +#include "tensorflow/compiler/xla/service/hlo_module.h" + +namespace xla { + +// The context in which a computation is called by another computation. +enum class CallContext { + // In a parallel contex the computation is applied to each element of the + // array argument(s). kMap and kReduce instructions call computations in + // parallel context. + kParallel, + + // In a sequential context the computation is applied to the entire argument + // shape(s). kCall and kWhile (body and condition) call computations in + // sequential context. + kSequential, + + // A computation is called from both a parallel and sequential context. + kBoth, + + // During call graph construction kNone is used to indicate that the context + // has not been determined. This is the top value for the context + // lattice. After construction, no call sites or call graph nodes should have + // this value. + kNone +}; + +string CallContextToString(CallContext context); +std::ostream& operator<<(std::ostream& out, const CallContext& context); + +CallContext GetInstructionCallContext(HloOpcode opcode); + +// Represents an HLO instruction which calls one or more computations. +class CallSite { + public: + CallSite(HloInstruction* instruction, + const std::vector& called_computations, + CallContext context) + : instruction_(CHECK_NOTNULL(instruction)), + called_computations_(called_computations), + context_(context) {} + + // Returns the instruction associated with this call site. + HloInstruction* instruction() const { return instruction_; } + + // Returns the computations called at this call site. + const std::vector& called_computations() const { + return called_computations_; + } + + // Returns the context in which computations are called at this call site. + CallContext context() const { return context_; } + + string ToString() const; + + private: + // The calling instruction. + HloInstruction* instruction_; + + // The computations called by this callsite. + const std::vector called_computations_; + + // The context in which the computations are called. + const CallContext context_; +}; + +// A node in the call graph representing an HLO computation. +class CallGraphNode { + public: + CallGraphNode(HloComputation* computation); + + // Returns the computation represented by this call graph node. + HloComputation* computation() const { return computation_; } + + // Returns the call sites in this computation. These are the instructions in + // this computation which call other computations. + const std::vector& callsites() const { return callsites_; } + + // Returns the callsite associated with the given instruction. If this + // instruction calls no computations nullptr is returned. + // Prerequisite: instruction is in the computation associated with this call + // graph node. + const CallSite* GetCallSite(const HloInstruction* instruction) const; + + // Returns the computations called by this computation. + const std::vector& callees() const { return callees_; } + + // Returns the call sites in other computations which call this computation. + const std::vector& caller_callsites() const { + return caller_callsites_; + } + + // Returns the computations which call this computation. + const std::vector& callers() const { return callers_; } + + // Returns the context in which this computation is called. + CallContext context() const { return context_; } + + string ToString() const; + + private: + // Only CallGraph can modify CallGraphNode. + friend class CallGraph; + + // Sets the context in which this computation is called. + void set_context(CallContext value) { context_ = value; } + + // Adds a callsite which calls this computation. Updates callers to include + // the calling computation. + void AddCallerCallSite(const CallSite& caller_callsite); + + // If instruction calls any computations adds a call site for this instruction + // to the call graph node. If the instruction calls no computations then no + // call site is added. + void AddCallSiteForInstruction(HloInstruction* instruction); + + // Computation represented by this call graph node. + HloComputation* computation_; + + // The computations called by this computation. The vector is used for a + // stable ordering and the set enables fast membership testing. + std::vector callees_; + absl::flat_hash_set callee_set_; + + // The computations which call this computation. The vector is used for a + // stable ordering and the set enables fast membership testing. + std::vector callers_; + absl::flat_hash_set caller_set_; + + // The call sites in this computation + std::vector callsites_; + + // The map from instruction to index in callsites_ for looking up the callsite + // (if any) associated with a particular instruction in this computation. + absl::flat_hash_map callsite_instructions_; + + // The call sites in other computations which call this computation. + std::vector caller_callsites_; + + // The context in which this computation is called. + CallContext context_ = CallContext::kNone; +}; + +// The call graph for an HLO module. The graph includes a node for each +// computation in the module. +class CallGraph { + public: + using VisitorFunction = std::function; + + // Builds and returns a call graph for the given HLO module. + static std::unique_ptr Build(const HloModule* module); + + // Returns the node associated with the given computation. + const CallGraphNode& GetNode(const HloComputation* computation) const; + CallGraphNode& GetNode(const HloComputation* computation); + + // Returns the vector of all nodes in the call graph. + const std::vector& nodes() const { return nodes_; } + + // Calls the given function on each node in the call graph. Nodes are visited + // in post order (callees before callers). If visit_unreachable_nodes is true + // then all nodes in the call graph are visited. Otherwise only those nodes + // reachable from the entry computation are visited. + Status VisitNodes(const VisitorFunction& visitor_func, + bool visit_unreachable_nodes = true) const; + + // Returns true if 'a' dominates 'b' in the call graph. Computation 'a' + // dominates computation 'b' iff all callgraph paths in the caller-to-callee + // direction from a root computation to 'b' pass through computation + // 'a'. Trivially, a computation dominates itself. + bool Dominates(const HloComputation* a, const HloComputation* b) const; + + // Returns whether 'instruction' is contained in 'computation' either directly + // ('instruction->parent' is 'computation') or indirectly ('computation' + // dominates 'instruction->parent' in the call graph). + bool InstructionIsNestedIn(const HloInstruction* instruction, + const HloComputation* computation) const { + return Dominates(computation, instruction->parent()); + } + + // Returns the nearest call graph ancestors of instructions 'a' and 'b' for + // which the ancestors are in the same computation. An instruction is an call + // graph ancestor of 'a' if the instruction calls the computation containing + // 'a' either directly or transitively. Degeneratively an instruction is an + // ancestor of itself. nullptr is returned if there is no common ancestor or + // if the caller chain of 'a' or 'b' diverges (has multiple callers) before + // the nearest common ancestor. + // + // Example: + // + // Entry computation: + // %x = Call(A, {Constant(42.0)}) + // %y = Call(B, {%x}) + // + // Computation A: + // %a = Negate(Param()) + // + // Computation B: + // %b = Exp(Param()); + // + // If called with %a and %b, this function would return (%x, %y). %x is an + // ancestor of %a, and %y is an ancestor of %b, and %x and %y are in the same + // computation. + std::pair NearestAncestorsInSameComputation( + HloInstruction* a, HloInstruction* b) const; + + // Returns whether the call graph is flattened. A call graph is flattened if + // every computation called in a sequential context (eg, kWhile or kCall) has + // zero or one callsite, and no computation is called from both a parallel and + // sequential context. The call graph of a module can be flattened with + // FlattenCallGraph. + bool IsFlattened() const; + + string ToString() const; + + private: + CallGraph(const HloModule* module); + + // Sets the call contexts for every node in the graph. + void SetCallContexts(); + + // Helper method for VisitNodes(). Traverses the call graph from 'node' in DFS + // post order (callee before caller) calling visitor_func on each node. Adds + // nodes to 'visited' as each node is visited. Skips nodes already in + // 'visited'. + Status VisitNodesInternal( + const VisitorFunction& visitor_func, const CallGraphNode& node, + absl::flat_hash_set* visited) const; + + // Recursive helper for computing whether 'a' dominates 'b' in the call + // graph. 'b_ancestor' is the currently visited node (which starts at 'b'), + // and 'visited' is the set of computations which have been visited. + bool DominatesHelper( + const HloComputation* a, const HloComputation* b, + absl::flat_hash_set* visited) const; + + // The HLO module represented by this call graph. + const HloModule* module_ = nullptr; + + // Vector of all nodes in the call graph. + std::vector nodes_; + + // Map from HLO computation to the index of the corresponding call graph node + // in nodes_. + absl::flat_hash_map node_indices_; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_CALL_GRAPH_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/call_inliner.h b/linux/include/tensorflow/compiler/xla/service/call_inliner.h new file mode 100644 index 000000000..08c4aff4f --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/call_inliner.h @@ -0,0 +1,45 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_CALL_INLINER_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_CALL_INLINER_H_ + +#include + +#include "tensorflow/compiler/xla/service/hlo_pass_interface.h" +#include "tensorflow/compiler/xla/statusor.h" + +namespace xla { + +// For every kCall operation in the main computation, we inline the body of the +// called function, and proceed recursively. +class CallInliner : public HloModulePass { + public: + using InlinedInstructionMap = + std::unordered_map; + + // Inlines one call instruction. Returns a mapping from the original + // instructions to their inlined versions. + static StatusOr Inline(HloInstruction* call); + + ~CallInliner() override = default; + absl::string_view name() const override { return "CallInliner"; } + + StatusOr Run(HloModule* module) override; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_CALL_INLINER_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/channel_tracker.h b/linux/include/tensorflow/compiler/xla/service/channel_tracker.h new file mode 100644 index 000000000..52037bf9b --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/channel_tracker.h @@ -0,0 +1,93 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_CHANNEL_TRACKER_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_CHANNEL_TRACKER_H_ + +#include + +#include "absl/types/span.h" +#include "tensorflow/compiler/xla/service/hlo_module.h" +#include "tensorflow/compiler/xla/status.h" +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/mutex.h" +#include "tensorflow/core/platform/thread_annotations.h" +#include "tensorflow/core/platform/types.h" + +namespace xla { + +// Tracks channels between computations in the XLA service. Channels +// are associated with a unique handle and can be resolved from the handle for +// later use. +// +// TODO(b/34027823): Destruct channels when all the associated computations that +// communicate via each channel are destructed. +class ChannelTracker { + public: + ChannelTracker(); + + // A struct that keeps the current status of each channel. has_sender and + // receiver_count fields are initialized with false and 0 respectively when + // the struct is created and are updated by RegisterSend() and RegisterRecev() + // as Send or Recv instructions using the channel are requested. + struct Channel { + bool has_sender; + int64 receiver_count; + ChannelHandle::ChannelType type; + }; + + // Creates a new Channel object and returns the corresponding + // ChannelHandle for it. + StatusOr NewChannel(ChannelHandle::ChannelType type); + + // Informs that the given channel handle is used for a Send operation. + // Returns an error status if the handle is already used by another Send. + Status RegisterSend(const ChannelHandle& handle); + + // Informs that the given channel handle is used for a Recv operation. + // Returns an error status if the handle is already used by another Recv. + Status RegisterRecv(const ChannelHandle& handle); + + private: + // Bumps the next_channel_ number and returns the allocated number + // wrapped in a ChannelHandle. + ChannelHandle AllocateHandle(ChannelHandle::ChannelType type) + EXCLUSIVE_LOCKS_REQUIRED(channel_mutex_); + + Status RegisterSendInternal(const ChannelHandle& handle) + EXCLUSIVE_LOCKS_REQUIRED(channel_mutex_); + + Status RegisterRecvInternal(const ChannelHandle& handle) + EXCLUSIVE_LOCKS_REQUIRED(channel_mutex_); + + // Guards the channel mapping. + tensorflow::mutex channel_mutex_; + + // The next sequence number to assign to a channel. + int64 next_channel_ GUARDED_BY(channel_mutex_); + + // Mapping from ChannelHandle value to the corresponding registered + // Channel object. + std::map opaque_to_channel_ GUARDED_BY(channel_mutex_); + + TF_DISALLOW_COPY_AND_ASSIGN(ChannelTracker); +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_CHANNEL_TRACKER_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/compile_only_service.h b/linux/include/tensorflow/compiler/xla/service/compile_only_service.h new file mode 100644 index 000000000..61136a3e1 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/compile_only_service.h @@ -0,0 +1,104 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_COMPILE_ONLY_SERVICE_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_COMPILE_ONLY_SERVICE_H_ + +#include "tensorflow/compiler/xla/service/backend.h" +#include "tensorflow/compiler/xla/service/compiler.h" +#include "tensorflow/compiler/xla/service/service.h" +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/platform/stream_executor_no_cuda.h" + +namespace xla { + +// An XLA Service specialization for ahead-of-time compilation. This only +// instantiates a Compiler object for the relevant platform; it does not +// instantiate or require an execution backend. +class CompileOnlyService : public Service { + public: + // Factory for creating a CompileOnlyService. The parameter platform is the + // platform that the service should target. If platform is null then the + // default platform is used. + static StatusOr> NewService( + se::Platform* platform); + static StatusOr> NewService( + const ServiceOptions& options); + + // A description of a xla computation to compile using CompileAheadOfTime. + struct AotXlaComputationInstance { + HloModuleProto computation; + std::vector argument_layouts; + const Shape* result_layout = nullptr; + }; + + // Compiles a list of xla computations for ahead-of-time execution. This is + // intended for use in static compilation. See + // |CompileOnlyClient::CompileAheadOfTime| for additional details. + StatusOr>> + CompileAheadOfTime( + const absl::Span computations, + const AotCompilationOptions& options); + + StatusOr>> + CompileAheadOfTime( + const absl::Span computations, + const AotCompilationOptions& options, + std::unique_ptr* metadata); + + Status GetDeviceHandles(const GetDeviceHandlesRequest* arg, + GetDeviceHandlesResponse* result) override { + return Unimplemented("CompileOnlyService does not support devices."); + } + Status WaitForExecution(const WaitForExecutionRequest* arg, + WaitForExecutionResponse* result) override { + return Unimplemented("CompileOnlyService does not support execution."); + } + Status TransferToServer(const TransferToServerRequest* arg, + TransferToServerResponse* result) override { + return Unimplemented( + "CompileOnlyService does not support device data transfers."); + } + Status TransferToInfeed(const TransferToInfeedRequest* arg, + TransferToInfeedResponse* result) override { + return Unimplemented( + "CompileOnlyService does not support device data transfers."); + } + Status TransferFromOutfeed(const TransferFromOutfeedRequest* arg, + TransferFromOutfeedResponse* result) override { + return Unimplemented( + "CompileOnlyService does not support device data transfers."); + } + Status ResetDevice(const ResetDeviceRequest* arg, + ResetDeviceResponse* result) override { + return Unimplemented("CompileOnlyService does not support devices."); + } + + private: + explicit CompileOnlyService(const ServiceOptions& options, + Compiler* compiler); + CompileOnlyService(const CompileOnlyService&) = delete; + void operator=(const CompileOnlyService&) = delete; + + // The compiler for the target platform. This is included in place of + // the Service::execute_backend_'s compiler, since execute_backend_ is a + // nullptr in CompileOnlyService. + Compiler* compiler_; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_COMPILE_ONLY_SERVICE_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/compiler.h b/linux/include/tensorflow/compiler/xla/service/compiler.h new file mode 100644 index 000000000..1fdda31c3 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/compiler.h @@ -0,0 +1,246 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// The compiler API is used by the XLA service to generate executables that +// run on a given platform. This is a registry and abstract interface, for +// pluggability by the various platforms. + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_COMPILER_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_COMPILER_H_ + +#include +#include +#include +#include +#include + +#include "absl/types/span.h" +#include "tensorflow/compiler/xla/service/buffer_value.h" +#include "tensorflow/compiler/xla/service/executable.h" +#include "tensorflow/compiler/xla/service/hlo_instruction.h" +#include "tensorflow/compiler/xla/service/hlo_module.h" +#include "tensorflow/compiler/xla/service/hlo_module_config.h" +#include "tensorflow/compiler/xla/service/logical_buffer.h" +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/core/platform/mutex.h" +#include "tensorflow/core/platform/protobuf.h" +#include "tensorflow/core/platform/stream_executor_no_cuda.h" +#include "tensorflow/core/platform/thread_annotations.h" + +namespace xla { + +// The following types are used for ahead of time compilation. + +// Contains the object file data created as a result of ahead-of-time +// compuation. +using ObjectFileData = std::vector; + +// Abstract superclass describing the result of an ahead-of-time compilation. +class AotCompilationResult { + public: + AotCompilationResult(const AotCompilationResult&) = delete; + AotCompilationResult& operator=(AotCompilationResult const&) = delete; + + virtual ~AotCompilationResult() = default; + + protected: + AotCompilationResult() = default; +}; + +// Abstract superclass describing options to an ahead-of-time compilation. +class AotCompilationOptions { + public: + AotCompilationOptions(const AotCompilationOptions&) = delete; + AotCompilationOptions& operator=(AotCompilationOptions const&) = delete; + + virtual ~AotCompilationOptions() = default; + + // Returns the ID of the platform to which these options apply. + virtual se::Platform::Id PlatformId() const = 0; + + // Optional allocator that may be used for allocating temp space on the device + // during compilation. + DeviceMemoryAllocator* device_allocator() const { return device_allocator_; } + void set_device_allocator(DeviceMemoryAllocator* device_allocator) { + device_allocator_ = device_allocator; + } + + const DebugOptions& debug_options() const { return debug_options_; } + DebugOptions* mutable_debug_options() { return &debug_options_; } + + protected: + AotCompilationOptions(); + + private: + DeviceMemoryAllocator* device_allocator_ = nullptr; + DebugOptions debug_options_; +}; + +// Abstract superclass describing metadata produced during ahead-of-time +// compilation. +class AotCompilationMetadata { + public: + AotCompilationMetadata(const AotCompilationMetadata&) = delete; + AotCompilationMetadata& operator=(AotCompilationMetadata const&) = delete; + + virtual ~AotCompilationMetadata() = default; + + protected: + AotCompilationMetadata() = default; +}; + +// Abstract compiler interface that is subclassed for compilation on a +// particular platform. +// +// The compiler ties together high level optimization (HLO) and low level +// optimization (LLO) / codegen (CG) to generate efficient executables for the +// target platform. +// +// The platform-based compiler singletons are registered via module initializers +// in their corresponding XLA compiler libraries, and are registered via the +// RegisterCompilerFactory API below. +// +// Thread-safety: subclasses of Compiler must be thread-safe, as multiple +// XLA clients may be requesting compilation concurrently for a given +// platform. +class Compiler { + public: + virtual ~Compiler() {} + + // Returns the ID of the platform that this compiler targets. + virtual se::Platform::Id PlatformId() const = 0; + + // Runs Hlo passes to optimize the given Hlo module, returns the optimized + // module. + // + // If device_allocator is not null, the compiler may use it to allocate temp + // space on the device for use during compilation. For example, the compiler + // may allocate buffers on the device and then run variants of a given + // algorithm over those buffers, to see which variant is fastest. Any space + // allocated should be deallocated before this function returns. + virtual StatusOr> RunHloPasses( + std::unique_ptr module, se::StreamExecutor* executor, + DeviceMemoryAllocator* device_allocator) = 0; + + // Compiles the HLO module for execution on a device given by the executor, + // and returns an executable object or an error status. No HLO passes are + // applied to module. Generally a module should be passed through RunHloPasses + // prior to calling this method because some HLO passes are required for + // correctness. Takes ownership of the HLO module and is free to transform it. + // + // The compiler may optionally specialize to the individual device + // (not just type of device) indicated by the executor. + // + // device_allocator is optional; see RunHloPasses. + // + // Use the overload below to compile computations that run in parallel. + virtual StatusOr> RunBackend( + std::unique_ptr module, se::StreamExecutor* executor, + DeviceMemoryAllocator* device_allocator) = 0; + + // Compiles a set of HLO modules that can run in parallel, potentially + // communicating data between the modules, and returns a corresponding + // sequence of executable objects. + // + // device_allocator is optional; see RunHloPasses. + // + // TODO(b/68666782): Remove this method after adding support for multiple + // modules to RunHloPasses and RunBackends. + virtual StatusOr>> Compile( + std::vector> modules, + std::vector> stream_exec, + DeviceMemoryAllocator* device_allocator) = 0; + + // Returns the backend configurations that the backend will consider for the + // given HLO. Returns no configurations if the backend does not support + // configurations for the given HLO. + // + // The stream executor is passed in to provide information about the hardware + // that the backend configurations would be targeting. + virtual std::vector> + ComputeBackendConfigs(const HloInstruction& hlo, + se::StreamExecutor* executor) const; + + // Returns the backend configuration that the backend chooses by default for + // the given HLO. Returns no configuration if the backend does not support + // configurations for the given HLO. + // + // The stream executor is passed in to provide information about the hardware + // that the backend configurations would be targeting. + virtual std::unique_ptr + ComputeDefaultBackendConfig(const HloInstruction& hlo, + se::StreamExecutor* executor) const; + + // Compiles the HLO module for ahead-of-time execution. This is intended for + // use in static compilation. + virtual StatusOr>> + CompileAheadOfTime(std::vector> modules, + const AotCompilationOptions& options) = 0; + + // Similar to CompileAheadOfTime above but AotCompilationMetadata + // has an argument that can be populated during compilation. + virtual StatusOr>> + CompileAheadOfTime(std::vector> modules, + const AotCompilationOptions& options, + std::unique_ptr* metadata); + + ///// + // The Compiler class also serves as a point to register compiler objects + // for the various platforms. + + using CompilerFactory = std::function()>; + + // Registers the compiler singleton for the platform. This is assumed to + // be a singleton, so no ownership is transferred. + // + // Precondition: a platform kind must not be registered more than once. + static void RegisterCompilerFactory(se::Platform::Id platform_id, + CompilerFactory compiler_factory); + + // Returns the compiler singleton pointer if it is available for the given + // platform, or an error status if it is not. + static StatusOr GetForPlatform(const se::Platform* platform); + + // Returns a function that computes the size in bytes of the logical + // buffer that contains a shape. + virtual HloCostAnalysis::ShapeSizeFunction ShapeSizeBytesFunction() const = 0; + + // Returns a function that computes the size in bytes of a given + // logical buffer. + std::function BufferSizeBytesFunction() { + HloCostAnalysis::ShapeSizeFunction shape_size = ShapeSizeBytesFunction(); + return [shape_size](const BufferValue& buffer) { + return shape_size(buffer.shape()); + }; + } + + private: + // Mutex that guards the platform-compiler map. + static tensorflow::mutex platform_compiler_mutex_; + + // Map from platform kind to compiler factory. + static std::map* + GetPlatformCompilerFactories(); + + // Map from platform kind to compiler instance, if we made one already (based + // on the factories above). + static std::map>* + GetPlatformCompilers(); +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_COMPILER_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/computation_layout.h b/linux/include/tensorflow/compiler/xla/service/computation_layout.h new file mode 100644 index 000000000..6975f387b --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/computation_layout.h @@ -0,0 +1,93 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_COMPUTATION_LAYOUT_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_COMPUTATION_LAYOUT_H_ + +#include +#include +#include + +#include "tensorflow/compiler/xla/shape_layout.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/platform/types.h" + +namespace xla { + +// Class which contains the layouts of the parameters and results of a +// computation. The layouts are stored as ShapeLayouts with immutable shapes and +// mutable layouts. +class ComputationLayout { + public: + // Creates a new ComputationLayout with the given result layout. + explicit ComputationLayout(ShapeLayout result_layout) + : result_layout_(std::move(result_layout)) {} + + // Constructs a ComputationLayout from a ProgramShape. The layouts of the + // parameters and results are set to the default layout. Layouts in the + // ProgramShape are ignored if ignore_layouts is true. + explicit ComputationLayout(const ProgramShape& program_shape, + bool ignore_layouts = true); + + // Adds a new parameter layout to the computation layout. + void add_parameter_layout(ShapeLayout shape_layout) { + parameter_layouts_.push_back(std::move(shape_layout)); + } + + // Returns the layout of a particular parameter. + const ShapeLayout& parameter_layout(int64 param_no) const { + return parameter_layouts_[param_no]; + } + ShapeLayout* mutable_parameter_layout(int64 param_no) { + return ¶meter_layouts_[param_no]; + } + + // Returns the number of parameters in the computation. + int parameter_count() const { return parameter_layouts_.size(); } + + // Returns the ShapeLayouts of the parameters of the computation. + const std::vector& parameter_layouts() const { + return parameter_layouts_; + } + + // Returns the ShapeLayout of a result of the computation. + const ShapeLayout& result_layout() const { return result_layout_; } + ShapeLayout* mutable_result_layout() { return &result_layout_; } + + // Returns the shape of the particular parameter or result of the computation + // with layout. + const Shape& parameter_shape(int64 param_no) const { + return parameter_layouts_[param_no].shape(); + } + const Shape& result_shape() const { return result_layout_.shape(); } + + // Sets layouts of all parameters and the result to the default layout. + void SetToDefaultLayout(); + + // Returns true if all layouts (parameters and result) have been set. + bool LayoutIsSet() const; + + // Returns a string representation of this object. + string ToString() const; + + private: + std::vector parameter_layouts_; + ShapeLayout result_layout_; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_COMPUTATION_LAYOUT_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/computation_placer.h b/linux/include/tensorflow/compiler/xla/service/computation_placer.h new file mode 100644 index 000000000..c899ffb9d --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/computation_placer.h @@ -0,0 +1,115 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_COMPUTATION_PLACER_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_COMPUTATION_PLACER_H_ + +#include +#include +#include + +#include "tensorflow/compiler/xla/array2d.h" +#include "tensorflow/compiler/xla/status.h" +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/stream_executor_no_cuda.h" +#include "tensorflow/core/platform/types.h" + +namespace xla { + +// Class that represents the device assignment for a set of XLA replicated +// computations. For R replicas and C computations, R * C devices are required +// execute the computation in parallel. The assigned device ids can be accessed +// by assignment(replica, computation). +class DeviceAssignment : public Array2D { + public: + DeviceAssignment() {} + DeviceAssignment(int replica_count, int computation_count) + : Array2D(replica_count, computation_count, -1) { + CHECK_GT(replica_count, 0); + CHECK_GT(computation_count, 0); + } + + int replica_count() const { return height(); } + int computation_count() const { return width(); } + + // Protocol buffer serialization and deserialization. + Status Serialize(DeviceAssignmentProto* proto) const; + + // Return a std::unique_ptr instead of a DeviceAssignment + // directly because one of the supported TF platforms (mac) does not compile + // due to a StatusOr of an incomplete type (DeviceAssignment). + static StatusOr> Deserialize( + const DeviceAssignmentProto& proto); + + string ToString() const; +}; + +// A generic implementation of the XLA computation placer, which assigns device +// ids to a set of replicated computations. +class ComputationPlacer { + public: + ComputationPlacer() {} + virtual ~ComputationPlacer() {} + + // Returns the device id assigned to the given replica and computation + // instance for [replica_count x computation_count] setup. The returned device + // id must match the assignement from PlaceReplicatedComputation(). + virtual StatusOr DeviceId(int replica, int computation, + int replica_count, int computation_count); + + // Returns the device ids assigned to a set of replicated computations, given + // the number of replicas and the number of computations. + virtual StatusOr AssignDevices(int replica_count, + int computation_count); + + using ComputationPlacerCreationFunction = + std::unique_ptr (*)(); + + // Registers a computation placer creation function for a particular platform. + static void RegisterComputationPlacer( + se::Platform::Id platform_id, + ComputationPlacerCreationFunction creation_function); + + // Returns the computation placer singleton pointer if it is available for the + // given platform, or an error status if it is not. + static StatusOr GetForPlatform( + const se::Platform* platform); + + private: + // The mutex that guards the platform-to-computation placer map. + static tensorflow::mutex platform_computation_placer_mutex_; + + // State kept for each kind of ComputationPlacer. Registration functions set + // up creation_function, and then we use that to lazily create "placer" the + // first time GetForPlatform is invoked for a particular id. + struct State { + std::unique_ptr placer; + ComputationPlacerCreationFunction creation_function = nullptr; + }; + + // Map from platform kind to computation placer singleton. + static std::map* GetPlatformComputationPlacers(); + + se::Platform::Id platform_id_; + + TF_DISALLOW_COPY_AND_ASSIGN(ComputationPlacer); +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_COMPUTATION_PLACER_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/conditional_simplifier.h b/linux/include/tensorflow/compiler/xla/service/conditional_simplifier.h new file mode 100644 index 000000000..2223ad675 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/conditional_simplifier.h @@ -0,0 +1,36 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_CONDITIONAL_SIMPLIFIER_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_CONDITIONAL_SIMPLIFIER_H_ + +#include "absl/strings/string_view.h" +#include "tensorflow/compiler/xla/service/hlo_module.h" +#include "tensorflow/compiler/xla/service/hlo_pass_interface.h" +#include "tensorflow/compiler/xla/statusor.h" + +namespace xla { + +// HLO pass that removes kConditional with a constant predicate, replacing them +// with their true or false computation as appropriate. +class ConditionalSimplifier : public HloModulePass { + public: + absl::string_view name() const override { return "simplify-conditional"; } + StatusOr Run(HloModule* module) override; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_CONDITIONAL_SIMPLIFIER_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/convolution_feature_group_converter.h b/linux/include/tensorflow/compiler/xla/service/convolution_feature_group_converter.h new file mode 100644 index 000000000..ce0138e56 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/convolution_feature_group_converter.h @@ -0,0 +1,43 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_CONVOLUTION_FEATURE_GROUP_CONVERTER_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_CONVOLUTION_FEATURE_GROUP_CONVERTER_H_ + +#include "absl/strings/string_view.h" +#include "tensorflow/compiler/xla/service/hlo_module.h" +#include "tensorflow/compiler/xla/service/hlo_pass_interface.h" +#include "tensorflow/compiler/xla/status_macros.h" + +namespace xla { + +// A pass which rewrites convolutions with feature_group_count > 1 into +// convolutions with feature_group_count = 1. +class ConvolutionFeatureGroupConverter : public HloModulePass { + public: + ConvolutionFeatureGroupConverter() {} + + absl::string_view name() const override { + return "convolution-feature-group-converter"; + } + + // Run convolution rewriting on the given computation. Returns whether the + // computation was changed. + StatusOr Run(HloModule* module) override; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_CONVOLUTION_FEATURE_GROUP_CONVERTER_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/copy_insertion.h b/linux/include/tensorflow/compiler/xla/service/copy_insertion.h new file mode 100644 index 000000000..c097089e3 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/copy_insertion.h @@ -0,0 +1,110 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_COPY_INSERTION_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_COPY_INSERTION_H_ + +#include "tensorflow/compiler/xla/service/buffer_liveness.h" +#include "tensorflow/compiler/xla/service/hlo_computation.h" +#include "tensorflow/compiler/xla/service/hlo_instruction.h" +#include "tensorflow/compiler/xla/service/hlo_module.h" +#include "tensorflow/compiler/xla/service/hlo_pass_interface.h" + +namespace xla { + +// Copy insertion is a legalization HLO pass which inserts copies (kCopy +// instructions) to eliminate several kinds of problems in the HLO module. +// +// (1) Entry parameter or a constant live out of the entry computation. Entry +// computation arguments and constants have different lifetimes than the +// computation result and cannot share the same allocation. Parameters and +// constants live out of non-entry computations do not need copies. +// +// (2) Different values which are simultaneously live and which must be held +// in the same buffer. This can occur in while bodies. Specifically, the +// while loop state (the arguments to the while instruction) is updated +// in-place and the update may clobber the value from the previous +// iteration before the previous value is dead. Computations called from +// kCall instructions do not need such copies because kCall has no update +// in-place semantics. +// +// (3) The buffer set of the root instruction of the entry computation must be +// unambiguous and distinct. That is, InstructionAliasSet::IsAmbiguous and +// InstructionAliasSet::IsDistinct return true. +class CopyInsertion : public HloModulePass { + public: + absl::string_view name() const override { return "copy-insertion"; } + + // fusion_can_share_buffer: backend specific function that decides whether a + // fusion can share buffer with its operand. + // + // TODO(b/80315712): Find a better way to tell whether a fusion can share + // buffer. + CopyInsertion(const HloDataflowAnalysis::FusionCanShareBufferFunction& + fusion_can_share_buffer = nullptr) + : fusion_can_share_buffer_(fusion_can_share_buffer) {} + + // Run the pass on the given module. Returns whether the module was changed + // (copies were inserted). + StatusOr Run(HloModule* module) override; + + // The CPU and GPU backend need additional copies added due to deficiencies in + // buffer assignment. Specifically, copies are needed for constants live-out + // of computations, and for values which are live-in and live-out of the same + // computation. These copies are needed because buffer-assignment uses a + // computation-scoped analyis (TuplePointsToAnalysis) and has limited + // visibility across computation boundaries. This method adds these necessary + // copies. Returns whether the module was modified. + // + // TODO(b/62548313): Remove this when buffer assignment is module-scoped. + static StatusOr AddCopiesForBufferAssignment(HloModule* module); + + // Try to remove as many copies from the module as possible without + // introducing live range interference. Only copy instructions that are + // eligible for copy elision are considered for removal. + Status RemoveUnnecessaryCopies(const HloOrdering& ordering, + HloModule* module); + + // Add copies to address special constraints on the roots of computations not + // related to live range interference: + // + // (1) Entry computation root must be unambiguous and distinct. + // + // (2) Any computation called by a kCall instruction must have an + // unambiguous root. + // + // (3) Constants and parameters cannot be live out of the entry computation + // + Status AddSpecialCaseCopies(HloModule* module); + + // Verifies that no HLO values have interfering live ranges using the given + // ordering. + Status VerifyNoLiveRangeInterference(const HloOrdering& ordering, + HloModule* module); + + private: + // Override which requires the caller to pass in a call graph. + Status AddSpecialCaseCopies(const CallGraph& call_graph, HloModule* module); + + Status AddCopiesToResolveInterference(HloModule* module); + + // Backend specific function that decides whether a fusion can share buffer + // with its operand. + HloDataflowAnalysis::FusionCanShareBufferFunction fusion_can_share_buffer_; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_COPY_INSERTION_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/cpu/buffer_info_util.h b/linux/include/tensorflow/compiler/xla/service/cpu/buffer_info_util.h new file mode 100644 index 000000000..e9ee928ab --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/cpu/buffer_info_util.h @@ -0,0 +1,42 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_CPU_BUFFER_INFO_UTIL_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_CPU_BUFFER_INFO_UTIL_H_ + +#include "absl/types/span.h" +#include "tensorflow/compiler/tf2xla/cpu_function_runtime.h" +#include "tensorflow/compiler/xla/service/buffer_assignment.h" + +namespace xla { +namespace cpu { +// Creates and returns a list of BufferInfo instances containing relevant +// information from `buffer_assignment`. +std::vector<::tensorflow::cpu_function_runtime::BufferInfo> +CreateBufferInfosFromBufferAssignment( + const BufferAssignment& buffer_assignment); + +// Creates and returns a table containing the mapping from entry computation +// parameters to buffer allocation indices. +// +// If this function returns V then entry parameter i has buffer allocation index +// V[i]. +std::vector CreateArgIndexTableFromBufferInfos( + absl::Span + buffer_infos); +} // namespace cpu +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_CPU_BUFFER_INFO_UTIL_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/cpu/compiler_functor.h b/linux/include/tensorflow/compiler/xla/service/cpu/compiler_functor.h new file mode 100644 index 000000000..c38b896c5 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/cpu/compiler_functor.h @@ -0,0 +1,76 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_CPU_COMPILER_FUNCTOR_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_CPU_COMPILER_FUNCTOR_H_ + +#include "llvm/IR/LegacyPassManager.h" +#include "llvm/IR/Module.h" +#include "llvm/Object/ObjectFile.h" +#include "llvm/Target/TargetMachine.h" +#include "tensorflow/compiler/xla/service/cpu/disassembler.h" +#include "tensorflow/compiler/xla/service/llvm_compiler.h" +#include "tensorflow/core/platform/logging.h" + +namespace xla { +namespace cpu { + +// Functor class for compiling an LLVM module down to an object file. For use by +// Orc JIT compile layer. +class CompilerFunctor { + public: + explicit CompilerFunctor( + llvm::TargetMachine* target_machine, const Disassembler* disassembler, + int opt_level, bool optimize_for_size, bool enable_fast_math, + bool disable_expensive_passes, + LLVMCompiler::ModuleHook pre_optimization_hook = nullptr, + LLVMCompiler::ModuleHook post_optimization_hook = nullptr) + : target_machine_(target_machine), + disassembler_(CHECK_NOTNULL(disassembler)), + opt_level_(opt_level), + optimize_for_size_(optimize_for_size), + enable_fast_math_(enable_fast_math), + disable_expensive_passes_(disable_expensive_passes), + pre_optimization_hook_(pre_optimization_hook), + post_optimization_hook_(post_optimization_hook) {} + + // Compile a Module to an ObjectFile. + std::unique_ptr operator()( + llvm::Module& module) const; // NOLINT + + private: + // Populates the given pass manager with TargetLibraryInfo and + // TargetTransformInfo passes. + void AddTargetInfoPasses(llvm::legacy::PassManagerBase* passes) const; + + // Populates the given pass managers based on the optimization level. + void AddOptimizationPasses(llvm::legacy::PassManagerBase* module_passes, + llvm::legacy::FunctionPassManager* function_passes, + unsigned opt_level, unsigned size_level) const; + + llvm::TargetMachine* target_machine_; + const Disassembler* disassembler_; + const unsigned opt_level_; + const bool optimize_for_size_; + const bool enable_fast_math_; + const bool disable_expensive_passes_; + LLVMCompiler::ModuleHook pre_optimization_hook_; + LLVMCompiler::ModuleHook post_optimization_hook_; +}; + +} // namespace cpu +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_CPU_COMPILER_FUNCTOR_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/cpu/conv_canonicalization.h b/linux/include/tensorflow/compiler/xla/service/cpu/conv_canonicalization.h new file mode 100644 index 000000000..becee3f81 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/cpu/conv_canonicalization.h @@ -0,0 +1,54 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_CPU_CONV_CANONICALIZATION_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_CPU_CONV_CANONICALIZATION_H_ + +#include "tensorflow/compiler/xla/service/cpu/target_machine_features.h" +#include "tensorflow/compiler/xla/service/hlo_module.h" +#include "tensorflow/compiler/xla/service/hlo_pass_interface.h" + +namespace xla { +namespace cpu { + +// An HLO pass that canonicalizes the dimension numbers of all top-level +// convolutions in the given module. +// +// In order to hit the fast path of using Eigen's convolution implementation, a +// convolution's dimension numbers need to satisfy certain constraints (so +// called canonical convolutions). This pass expands non-canonical convolutions +// into reshapes and canonical convolutions, so that these non-canonical +// convolutions can run faster. +class ConvCanonicalization : public HloModulePass { + public: + explicit ConvCanonicalization( + const TargetMachineFeatures* target_machine_features) + : target_machine_features_(*target_machine_features) {} + + ~ConvCanonicalization() override {} + absl::string_view name() const override { + return "convolution-canonicalization"; + } + + StatusOr Run(HloModule* module) override; + + private: + const TargetMachineFeatures& target_machine_features_; +}; + +} // namespace cpu +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_CPU_CONV_CANONICALIZATION_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/cpu/cpu_compiler.h b/linux/include/tensorflow/compiler/xla/service/cpu/cpu_compiler.h new file mode 100644 index 000000000..f2af92378 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/cpu/cpu_compiler.h @@ -0,0 +1,177 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_CPU_CPU_COMPILER_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_CPU_CPU_COMPILER_H_ + +#include + +#include "absl/types/span.h" +#include "llvm/Target/TargetMachine.h" +#include "tensorflow/compiler/tf2xla/cpu_function_runtime.h" +#include "tensorflow/compiler/xla/service/cpu/target_machine_features.h" +#include "tensorflow/compiler/xla/service/executable.h" +#include "tensorflow/compiler/xla/service/hlo_module.h" +#include "tensorflow/compiler/xla/service/llvm_compiler.h" +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/stream_executor_no_cuda.h" + +namespace xla { +namespace cpu { + +// This class wraps the configurability options that LLVM exposes including: the +// target triple, the target cpu and the target features. It also includes the +// desired linkage name for the computation entry point. +class CpuAotCompilationOptions : public AotCompilationOptions { + public: + // Relocation models available for compilation. + enum class RelocationModel { + // Corresponds to the -fno-pic compiler option. + Static, + // Corresponds to the -fpic compiler option. + SmallPic, + // Corresponds to the -fPIC compiler option. + BigPic, + // Corresponds to the -fpie compiler option. + SmallPie, + // Corresponds to the -fPIE compiler option. + BigPie + }; + + CpuAotCompilationOptions(string triple, string cpu_name, string features, + string entry_point_name, + RelocationModel relocation_model); + ~CpuAotCompilationOptions() override; + + se::Platform::Id PlatformId() const override; + + // The triple used for compilation, similar to clang's -target flag. + const string& triple() const { return triple_; } + // The CPU name used for compilation, similar to clang's -mcpu flag. + const string& cpu_name() const { return cpu_name_; } + // The target features used for compilation ("+avx2", "+neon", etc). + const string& features() const { return features_; } + // The name to be used for the compiled code's entry point. + const string& entry_point_name() const { return entry_point_name_; } + // The relocation model used for compilation. + RelocationModel relocation_model() const { return relocation_model_; } + + private: + const string triple_; + const string cpu_name_; + const string features_; + const string entry_point_name_; + const RelocationModel relocation_model_; +}; + +class CpuAotCompilationResult : public AotCompilationResult { + public: + CpuAotCompilationResult( + ObjectFileData object_file_data, + std::vector<::tensorflow::cpu_function_runtime::BufferInfo> buffer_infos, + int64 result_buffer_index, + std::unique_ptr hlo_profile_printer_data); + ~CpuAotCompilationResult(); + + HloProfilePrinterData* hlo_profile_printer_data() const { + return hlo_profile_printer_data_.get(); + } + + const ObjectFileData& object_file_data() const { return object_file_data_; } + const std::vector<::tensorflow::cpu_function_runtime::BufferInfo>& + buffer_infos() const { + return buffer_infos_; + } + int64 result_buffer_index() const { return result_buffer_index_; } + + private: + // Contains the compiled computation: an object file. + const ObjectFileData object_file_data_; + + // A list of BufferInfo objects describing the buffers used by the XLA + // computation. + const std::vector<::tensorflow::cpu_function_runtime::BufferInfo> + buffer_infos_; + + // Contains which buffer index into |buffer_sizes| was designated to the + // result of the computation. This buffer should be passed into the output + // parameter when calling the compiled computation. + const int64 result_buffer_index_; + + // Contains an instance of HloProfilePrinterData if HLO profiling is enabled, + // otherwise is nullptr. + std::unique_ptr hlo_profile_printer_data_; +}; + +// CPU-targeting implementation of the XLA Compiler interface. +// +// The compiler translates XLA HLO code into LLVM IR and uses LLVM's JIT +// infrastructure to create an executable "blob" that can then be returned +// wrapped in CpuExecutable and actually invoked. +class CpuCompiler : public LLVMCompiler { + public: + CpuCompiler(); + ~CpuCompiler() override {} + + // Bring in + // StatusOr>> Compile( + // std::vector> modules, + // std::vector> + // stream_execs) + using LLVMCompiler::Compile; + + StatusOr> RunHloPasses( + std::unique_ptr module, se::StreamExecutor* stream_exec, + DeviceMemoryAllocator* device_allocator) override; + + StatusOr> RunBackend( + std::unique_ptr module, se::StreamExecutor* stream_exec, + DeviceMemoryAllocator* device_allocator) override; + + StatusOr>> + CompileAheadOfTime(std::vector> modules, + const AotCompilationOptions& options) override; + + se::Platform::Id PlatformId() const override; + + HloCostAnalysis::ShapeSizeFunction ShapeSizeBytesFunction() const override; + + private: + // Initialize the LLVM target. + static void InitializeLLVMTarget(); + + // Runs the HLO passes which are necessary for both optimizations and + // correctness. + Status RunHloPasses(HloModule* module, bool is_aot_compile, + llvm::TargetMachine* target_machine); + + // Runs HLO passes up to and including layout assignment. + Status RunHloPassesThroughLayoutAssn( + HloModule* module, bool /*is_aot_compile*/, + LLVMTargetMachineFeatures* target_machine_features); + + // Runs HLO passes after layout assignment. + Status RunHloPassesAfterLayoutAssn( + HloModule* module, bool is_aot_compile, + LLVMTargetMachineFeatures* target_machine_features); + + TF_DISALLOW_COPY_AND_ASSIGN(CpuCompiler); +}; + +} // namespace cpu +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_CPU_CPU_COMPILER_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/cpu/cpu_copy_insertion.h b/linux/include/tensorflow/compiler/xla/service/cpu/cpu_copy_insertion.h new file mode 100644 index 000000000..076235f88 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/cpu/cpu_copy_insertion.h @@ -0,0 +1,42 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_CPU_CPU_COPY_INSERTION_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_CPU_CPU_COPY_INSERTION_H_ + +#include "tensorflow/compiler/xla/service/hlo_module.h" +#include "tensorflow/compiler/xla/service/hlo_pass_interface.h" + +namespace xla { + +// Besides the modifications made by the generic xla::CopyInsertion, this +// CPU-specific copy insertion pass also adds copies to values live out of +// computations satisfying certain conditions (defined by constant or parameter, +// etc). This is necessary because of deficiencies of buffer +// assignment. Specifically, buffer assignment is computation-scoped and does +// not recognized aliasing between arguments and outputs of computations. +// +// TODO(b/62548313): Remove this when buffer assignment is smarter +// (module-scoped). +class CpuCopyInsertion : public HloModulePass { + public: + absl::string_view name() const override { return "copy-insertion"; } + + StatusOr Run(HloModule* module) override; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_CPU_CPU_COPY_INSERTION_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/cpu/cpu_executable.h b/linux/include/tensorflow/compiler/xla/service/cpu/cpu_executable.h new file mode 100644 index 000000000..3c3c047bf --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/cpu/cpu_executable.h @@ -0,0 +1,158 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_CPU_CPU_EXECUTABLE_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_CPU_CPU_EXECUTABLE_H_ + +#include +#include +#include +#include +#include + +#include "absl/types/span.h" +#include "tensorflow/compiler/xla/service/buffer_assignment.h" +#include "tensorflow/compiler/xla/service/cpu/simple_orc_jit.h" +#include "tensorflow/compiler/xla/service/device_memory_allocator.h" +#include "tensorflow/compiler/xla/service/executable.h" +#include "tensorflow/compiler/xla/service/hlo_execution_profile.h" +#include "tensorflow/compiler/xla/service/hlo_instruction.h" +#include "tensorflow/compiler/xla/service/hlo_module.h" +#include "tensorflow/compiler/xla/service/shaped_buffer.h" +#include "tensorflow/compiler/xla/service/tuple_points_to_analysis.h" +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/stream_executor_no_cuda.h" +#include "tensorflow/core/platform/types.h" + +namespace xla { +namespace cpu { + +// CPU-targeting implementation of the XLA Executable interface. +// +// Wraps a JIT-ed object that can be executed "on device". We JIT for the host +// architecture, so JIT-ed code and host code share the same ABI. +class CpuExecutable : public Executable { + public: + CpuExecutable(std::unique_ptr jit, + std::unique_ptr assignment, + std::unique_ptr hlo_module, + const string& entry_function_name, + std::unique_ptr hlo_profile_printer_data, + std::unique_ptr hlo_profile_index_map); + ~CpuExecutable() override {} + + StatusOr ExecuteOnStream( + const ServiceExecutableRunOptions* run_options, + absl::Span arguments, + HloExecutionProfile* hlo_execution_profile) override; + + StatusOr ExecuteAsyncOnStream( + const ServiceExecutableRunOptions* run_options, + absl::Span arguments) override; + + // This should be called after set_ir_module_string. + const string& ir_module_string() const { return ir_module_string_; } + + void set_ir_module_string(const string& ir_module_string) { + ir_module_string_ = ir_module_string; + } + + static int64 ShapeSizeBytes(const Shape& shape); + + // Type of the computation function we expect in the JIT. + using ComputeFunctionType = + void (*)(void* /*result*/, const ExecutableRunOptions* /*run_options*/, + const void** /*args*/, void** /*buffer_table*/, + int64* /*profile_counters*/); + + const ComputeFunctionType& compute_function() const { + return compute_function_; + } + + const BufferAssignment& buffer_assignment() const { return *assignment_; } + + private: + // This is for sharing the code between ExecuteOnStream and + // ExecuteAsyncOnStream. + // + // Notice that it's tricky to use correctly, as the profile object (when it + // exists) must out-live the task. + StatusOr ExecuteAsyncOnStreamImpl( + const ServiceExecutableRunOptions* run_options, + absl::Span arguments, + HloExecutionProfile* hlo_execution_profile); + + // Creates an array suitable for passing as the "buffer_table" argument to the + // JIT compiled function pointer. + // + // Returns (unowning_buffers, owning_buffers) where: + // + // - unowning_buffers.data() can be passed as the buffer_table argument as-is + // and includes pointers to the scratch storage required by the + // computation, the live-out buffer into which the result will be written + // and entry computation parameters. + // + // - owning_buffers contains owning pointers to the buffers that were + // allocated by this routine. This routine allocates buffers for temporary + // storage and the live-out buffer into which the computation writes it + // result. + StatusOr, + std::vector>> + CreateBufferTable(DeviceMemoryAllocator* memory_allocator, int device_ordinal, + absl::Span arguments); + + // Calls the generated function performing the computation with the given + // arguments using the supplied buffers. + Status ExecuteComputeFunction(const ExecutableRunOptions* run_options, + absl::Span buffers, + HloExecutionProfile* hlo_execution_profile); + + // Creates a ScopedShapedBuffer for holding the result of the computation, + // moving buffers out of allocated_buffers and into the result as appropriate. + // The addresses are set according to buffer assignment. + StatusOr CreateResultShapedBuffer( + const ServiceExecutableRunOptions* run_options, + absl::Span buffers); + + // Returns the points-to set of the root instruction of the entry + // computation. Uses points-to analysis from buffer assignment. + const PointsToSet& GetRootPointsToSet() const; + + // The JIT containing compiled modules. + const std::unique_ptr jit_; + + // Buffer assignment for the buffers we need to allocate. + const std::unique_ptr assignment_; + + // The LLVM IR, in string format, of the unoptimized module generated for this + // CpuExecutable. We save a string instead of an llvm::Module* because leaving + // llvm::Module* in a singleton can cause the heap checker to emit false + // positives. + string ir_module_string_; + + ComputeFunctionType compute_function_; + + // Entry function name for the computation. + const string entry_function_name_; + + TF_DISALLOW_COPY_AND_ASSIGN(CpuExecutable); +}; + +} // namespace cpu +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_CPU_CPU_EXECUTABLE_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/cpu/cpu_hlo_support_checker.h b/linux/include/tensorflow/compiler/xla/service/cpu/cpu_hlo_support_checker.h new file mode 100644 index 000000000..a39a9d472 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/cpu/cpu_hlo_support_checker.h @@ -0,0 +1,40 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_CPU_CPU_HLO_SUPPORT_CHECKER_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_CPU_CPU_HLO_SUPPORT_CHECKER_H_ + +#include "tensorflow/compiler/xla/service/hlo_pass_interface.h" + +namespace xla { + +// This pass should run early in the HLO pipeline and checks for HLO constructs +// which are not supported by the CPU backend and cannot be removed via HLO +// transformations (eg, sparse layouts). +class CpuHloSupportChecker : public HloModulePass { + public: + CpuHloSupportChecker() = default; + ~CpuHloSupportChecker() override = default; + + absl::string_view name() const override { return "cpu_hlo_support_checker"; } + + // Note: always returns false (no instructions are ever modified by this + // pass). + StatusOr Run(HloModule* module) override; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_CPU_CPU_HLO_SUPPORT_CHECKER_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/cpu/cpu_instruction_fusion.h b/linux/include/tensorflow/compiler/xla/service/cpu/cpu_instruction_fusion.h new file mode 100644 index 000000000..07aff3497 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/cpu/cpu_instruction_fusion.h @@ -0,0 +1,40 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_CPU_CPU_INSTRUCTION_FUSION_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_CPU_CPU_INSTRUCTION_FUSION_H_ + +#include "tensorflow/compiler/xla/service/hlo_instruction.h" +#include "tensorflow/compiler/xla/service/instruction_fusion.h" + +namespace xla { +namespace cpu { + +class CpuInstructionFusion : public InstructionFusion { + public: + CpuInstructionFusion() + : InstructionFusion(CpuInstructionFusion::IsExpensive) {} + ~CpuInstructionFusion() override = default; + + protected: + bool ShouldFuse(HloInstruction* consumer, int64 operand_index) override; + HloInstruction::FusionKind ChooseKind( + const HloInstruction* producer, const HloInstruction* consumer) override; +}; + +} // namespace cpu +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_CPU_CPU_INSTRUCTION_FUSION_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/cpu/cpu_layout_assignment.h b/linux/include/tensorflow/compiler/xla/service/cpu/cpu_layout_assignment.h new file mode 100644 index 000000000..f4da35dd3 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/cpu/cpu_layout_assignment.h @@ -0,0 +1,50 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_CPU_CPU_LAYOUT_ASSIGNMENT_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_CPU_CPU_LAYOUT_ASSIGNMENT_H_ + +#include "tensorflow/compiler/xla/service/computation_layout.h" +#include "tensorflow/compiler/xla/service/cpu/target_machine_features.h" +#include "tensorflow/compiler/xla/service/layout_assignment.h" +#include "tensorflow/core/lib/core/status.h" + +namespace xla { +namespace cpu { + +// CPU-specific layout assignment pass which preassigns layouts to satisfy +// layout constraints for operands and results of library calls. +class CpuLayoutAssignment : public LayoutAssignment { + public: + explicit CpuLayoutAssignment( + ComputationLayout* entry_computation_layout, + std::function + instruction_can_change_layout_func, + const TargetMachineFeatures* target_machine_features) + : LayoutAssignment(entry_computation_layout, + std::move(instruction_can_change_layout_func)), + target_machine_features_(*target_machine_features) {} + ~CpuLayoutAssignment() override {} + + protected: + Status AddBackendConstraints(LayoutConstraints* constraints) override; + + const TargetMachineFeatures& target_machine_features_; +}; + +} // namespace cpu +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_CPU_CPU_LAYOUT_ASSIGNMENT_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/cpu/cpu_options.h b/linux/include/tensorflow/compiler/xla/service/cpu/cpu_options.h new file mode 100644 index 000000000..47c7eb13b --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/cpu/cpu_options.h @@ -0,0 +1,38 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_CPU_CPU_OPTIONS_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_CPU_CPU_OPTIONS_H_ + +#include "tensorflow/compiler/xla/service/hlo_module_config.h" + +// Helper functions for querying options that are specific to the CPU backend. + +namespace xla { +namespace cpu { +namespace options { + +bool OptimizeForSizeRequested(const HloModuleConfig& config); +bool VectorizedReduceDisabled(const HloModuleConfig& config); +bool EnableExperimentalLlvmIrGemm(const HloModuleConfig& config); +absl::optional LlvmIrGemvTilingFactor(const HloModuleConfig& config); +absl::optional> LlvmIrGemmTileSize( + const HloModuleConfig& config); + +} // namespace options +} // namespace cpu +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_CPU_CPU_OPTIONS_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/cpu/cpu_runtime.h b/linux/include/tensorflow/compiler/xla/service/cpu/cpu_runtime.h new file mode 100644 index 000000000..b2e760a22 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/cpu/cpu_runtime.h @@ -0,0 +1,158 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// This header declares functions which may be called by the generated code on +// the CPU. Calls to these functions must be resolved explicitly in the JIT in +// xla::cpu::SimpleResolver. It also defines a per-CpuExecutable context +// which is used to cache expensive state and resources utilized by the +// aforementioned functions. +// +// Other functions are declared in individual libraries as well, such as +// runtime_conv2d and runtime_matmul. As individual libraries, callers for +// ahead-of-time compilation can link only the required subset. + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_CPU_CPU_RUNTIME_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_CPU_CPU_RUNTIME_H_ + +#include "tensorflow/compiler/xla/executable_run_options.h" +#include "tensorflow/compiler/xla/service/cpu/xfeed_manager.h" +#include "tensorflow/compiler/xla/types.h" + +namespace xla { +namespace cpu { +namespace runtime { + +// Names of runtime functions. These get resolved from the generated code to the +// right symbol at link time in one of two ways: +// 1. When using the JIT, the symbol resolver (SimpleResolver in +// third_party/tensorflow/compiler/xla/service/cpu/simple_orc_jit.cc) maps +// this symbol name to +// the actual symbol. +// 2. When using ahead-of-time compilation, the linker can resolve the name +// because it is a symbol in the cpu_runtime library. +extern const char* const kEigenMatMulF16SymbolName; +extern const char* const kEigenMatMulF32SymbolName; +extern const char* const kEigenMatMulF64SymbolName; +extern const char* const kMKLConvF32SymbolName; +extern const char* const kMKLMatMulF32SymbolName; +extern const char* const kMKLMatMulF64SymbolName; +extern const char* const kMKLSingleThreadedMatMulF32SymbolName; +extern const char* const kMKLSingleThreadedMatMulF64SymbolName; +extern const char* const kEigenConvF16SymbolName; +extern const char* const kEigenConvF32SymbolName; +extern const char* const kEigenFftSymbolName; +extern const char* const kEigenSingleThreadedFftSymbolName; +extern const char* const kEigenSingleThreadedMatMulF16SymbolName; +extern const char* const kEigenSingleThreadedMatMulF32SymbolName; +extern const char* const kEigenSingleThreadedMatMulF64SymbolName; +extern const char* const kEigenSingleThreadedConvF16SymbolName; +extern const char* const kEigenSingleThreadedConvF32SymbolName; +extern const char* const kAcquireInfeedBufferForDequeueSymbolName; +extern const char* const kReleaseInfeedBufferAfterDequeueSymbolName; +extern const char* const kAcquireOutfeedBufferForPopulationSymbolName; +extern const char* const kReleaseOutfeedBufferAfterPopulationSymbolName; +extern const char* const kParallelForkJoinSymbolName; +extern const char* const kKeyValueSortPREDSymbolName; +extern const char* const kKeyValueSortS8SymbolName; +extern const char* const kKeyValueSortU8SymbolName; +extern const char* const kKeyValueSortS16SymbolName; +extern const char* const kKeyValueSortU16SymbolName; +extern const char* const kKeyValueSortF16SymbolName; +extern const char* const kKeyValueSortS32SymbolName; +extern const char* const kKeyValueSortU32SymbolName; +extern const char* const kKeyValueSortF32SymbolName; +extern const char* const kKeyValueSortS64SymbolName; +extern const char* const kKeyValueSortU64SymbolName; +extern const char* const kKeyValueSortF64SymbolName; + +// All symbol names for XLA CPU runtime functions need to start with this +// prefix. +extern const char* const kXlaCpuRuntimeSymbolNamePrefix; + +// Returns the infeed manager used by the CPU runtime for the CPU device +// `device_ordinal`. Note the device ordinal does not name a CPU +XfeedManager* GetXfeedManager(int device_ordinal); + +} // namespace runtime +} // namespace cpu +} // namespace xla + +extern "C" { + +// Some things common to all of the runtime entry points below: +// +// * The shape pointer and shape_length reflect values that can be deserialized +// via llvm_ir::DecodeSelfDescribingShapeConstant. This is the way we pass +// reified type information from the generated program to the runtime, which +// helps check the type safety and contract for the emitted-code/runtime +// communication. +// +// * run_options is used to look up the device ordinal for the stream executor +// we're executing under. If it is null the device ordinal is assumed to be +// 0 (this behavior helps in writing tests). + +// Note: in the runtime entry points below, the shape pointer and shape_length +// reflect values that can be deserialized via +// llvm_ir::DecodeSelfDescribingShapeConstant. This is the way we pass reified +// type information from the generated program to the runtime, which helps check +// the type safety and contract for the emitted-code/runtime communication. + +// Blocks until the next infeed buffer is ready to be dequeued, then +// returns it. Fails catastrophically if the next enqueued buffer is +// not of the correct length in bytes. Checking the shape rather than +// the length would be more exact, but the length check is chosen as a +// tradeoff between error checking and speed/simplicity. +extern void* __xla_cpu_runtime_AcquireInfeedBufferForDequeue( + const xla::ExecutableRunOptions* run_options, xla::int32 buffer_length, + const void* shape, xla::int32 shape_length); + +// Relinquishes the next infeed buffer that was returned by +// __xla_cpu_runtime_AcquireInfeedBufferForDequeue. Once this call +// completes the data at buffer_ptr may no longer be +// accessed. buffer_length must match the length passed to the call to +// __xla_cpu_runtime_AcquireInfeedBufferForDequeue that returned +// buffer_ptr. This function must be called before the next buffer is +// acquired, i.e., there may only be one outstanding infeed buffer in +// use by the runtime. TODO(b/31340454) investigate whether or not it +// is worth supporting zero-copy infeed where the buffer is retained +// by the compiled code until it has been used. If zero-copy infeed is +// implemented we will add support for multiple outstanding buffers +// that can be returned out of order. +extern void __xla_cpu_runtime_ReleaseInfeedBufferAfterDequeue( + const xla::ExecutableRunOptions* run_options, xla::int32 buffer_length, + void* buffer_ptr, const void* shape_ptr, xla::int32 shape_length); + +// Blocks until the next outfeed buffer is available to be populated, then +// returns it. +extern void* __xla_cpu_runtime_AcquireOutfeedBufferForPopulation( + const xla::ExecutableRunOptions* run_options, xla::int32 buffer_length, + const void* shape_ptr, xla::int32 shape_length); + +// Relinquishes the outfeed buffer after it has been populated. +// buffer_ptr must have been previously returned by +// __xla_cpu_runtime_AcquireOutfeedBufferForPopulation. +// Once this call completes, buffer_ptr may no longer be accessed. +// buffer_length must match the length passed to the call to +// __xla_cpu_runtime_AcquireInfeedBufferForDequeue that returned +// buffer_ptr. This function must be called before the next buffer is +// acquired, i.e., there may only be one outstanding outfeed buffer in +// use by the runtime. +extern void __xla_cpu_runtime_ReleaseOutfeedBufferAfterPopulation( + const xla::ExecutableRunOptions* run_options, xla::int32 buffer_length, + void* buffer_ptr, const void* shape_ptr, xla::int32 shape_length); + +} // extern "C" + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_CPU_CPU_RUNTIME_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/cpu/cpu_transfer_manager.h b/linux/include/tensorflow/compiler/xla/service/cpu/cpu_transfer_manager.h new file mode 100644 index 000000000..361d4b9c8 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/cpu/cpu_transfer_manager.h @@ -0,0 +1,78 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_CPU_CPU_TRANSFER_MANAGER_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_CPU_CPU_TRANSFER_MANAGER_H_ + +#include + +#include "absl/types/span.h" +#include "tensorflow/compiler/xla/literal.h" +#include "tensorflow/compiler/xla/service/cpu/xfeed_manager.h" +#include "tensorflow/compiler/xla/service/generic_transfer_manager.h" +#include "tensorflow/compiler/xla/service/transfer_manager.h" +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/stream_executor_no_cuda.h" +#include "tensorflow/core/platform/types.h" + +namespace xla { + +// An implementation of the XLA GenericTransferManager that +// handles CPU-specific infeed. +class CpuTransferManager : public GenericTransferManager { + public: + CpuTransferManager(); + ~CpuTransferManager() override {} + + Status TransferLiteralToInfeed(se::StreamExecutor* executor, + const LiteralSlice& literal) override; + Status TransferLiteralFromOutfeed(se::StreamExecutor* executor, + const Shape& literal_shape, + MutableBorrowingLiteral literal) override; + + private: + Status TransferBufferToInfeed(se::StreamExecutor* executor, int64 size, + const void* source); + + // Transfers infeed data to device. InfeedBuffer->Done() must be + // called to clean up the memory allocated for InfeedBuffer. + StatusOr TransferBufferToInfeedInternal( + se::StreamExecutor* executor, int64 size, const void* source); + + // Helper that transfers a tuple of element buffers from the device's outfeed. + StatusOr TransferTupleBuffersFromOutfeed( + se::StreamExecutor* executor, + absl::Span> buffer_data); + + // Helper that transfers an array buffer from the device's outfeed. + StatusOr TransferArrayBufferFromOutfeed(se::StreamExecutor* executor, + void* destination, + int64 size_bytes); + + // On success, returns the shape that was transferred from the outfeed -- if + // is_tuple is true, the returned shape will be a tuple of the returned shapes + // for the given buffers. + StatusOr TransferBuffersFromOutfeedInternal( + se::StreamExecutor* executor, + absl::Span> buffer_data, bool is_tuple); + + TF_DISALLOW_COPY_AND_ASSIGN(CpuTransferManager); +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_CPU_CPU_TRANSFER_MANAGER_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/cpu/custom_call_target_registry.h b/linux/include/tensorflow/compiler/xla/service/cpu/custom_call_target_registry.h new file mode 100644 index 000000000..664125ecc --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/cpu/custom_call_target_registry.h @@ -0,0 +1,74 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_CPU_CUSTOM_CALL_TARGET_REGISTRY_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_CPU_CUSTOM_CALL_TARGET_REGISTRY_H_ + +// This file is depended on by kernels that have to build for mobile devices. +// For this reason, we avoid relying on TensorFlow and instead only use the +// standard C++ library. + +#include // NOLINT +#include +#include + +namespace xla { +namespace cpu { + +// The CPU JIT compiler uses this registry to resolve symbolic CustomCall +// targets; so when using the CPU JIT, CustomCall targets need to be registered +// here with the symbol name used in the CustomCall. +// +// The XLA AOT compiler links using a standard offline linker; so when compiling +// in AOT mode, you *also* need to make sure the name of the callee (presumably +// implemented in C++) matches up with the symbolic name used in the CustomCall. +// +// We maintain the registry in both the JIT and the AOT cases for simplicity, +// but we only use it when running in JIT mode. +class CustomCallTargetRegistry { + public: + static CustomCallTargetRegistry* Global(); + + void Register(const std::string& symbol, void* address); + void* Lookup(const std::string& symbol) const; + + private: + std::unordered_map registered_symbols_; + mutable std::mutex mu_; +}; + +class RegisterCustomCallTarget { + public: + explicit RegisterCustomCallTarget(const std::string& name, void* address) { + CustomCallTargetRegistry::Global()->Register(name, address); + } +}; + +#define REGISTER_CUSTOM_CALL_CONCAT(a, b) a##b + +#define REGISTER_CUSTOM_CALL_TARGET_WITH_SYM_HELPER(symbol, address, counter) \ + static ::xla::cpu::RegisterCustomCallTarget REGISTER_CUSTOM_CALL_CONCAT( \ + custom_call_target_register, counter)(symbol, \ + reinterpret_cast(address)) + +#define REGISTER_CUSTOM_CALL_TARGET_WITH_SYM(symbol, address) \ + REGISTER_CUSTOM_CALL_TARGET_WITH_SYM_HELPER(symbol, address, __COUNTER__) + +#define REGISTER_CUSTOM_CALL_TARGET(function) \ + REGISTER_CUSTOM_CALL_TARGET_WITH_SYM(#function, function) + +} // namespace cpu +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_CPU_CUSTOM_CALL_TARGET_REGISTRY_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/cpu/disassembler.h b/linux/include/tensorflow/compiler/xla/service/cpu/disassembler.h new file mode 100644 index 000000000..5e302f889 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/cpu/disassembler.h @@ -0,0 +1,73 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_CPU_DISASSEMBLER_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_CPU_DISASSEMBLER_H_ + +#include +#include + +#include "llvm/MC/MCContext.h" +#include "llvm/MC/MCDisassembler/MCDisassembler.h" +#include "llvm/MC/MCInstPrinter.h" +#include "llvm/MC/MCInstrAnalysis.h" +#include "llvm/MC/MCObjectFileInfo.h" +#include "llvm/MC/MCSubtargetInfo.h" +#include "llvm/Object/ObjectFile.h" +#include "llvm/Target/TargetMachine.h" +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/compiler/xla/types.h" + +namespace xla { +namespace cpu { + +struct DisassemblerResult { + DisassemblerResult(const string& text, size_t code_size_bytes) + : text(text), code_size_bytes(code_size_bytes) {} + + // The disassembled text sections of the object file. + string text; + // The total number of bytes of executable code in the object file. + uint64_t code_size_bytes; +}; + +// Class for disassembling object files (and potentially other constructs) into +// x86/ARM assembly. Builds all the LLVM disassembly and instruction printing +// constructs from a given TargetMachine. +class Disassembler { + public: + explicit Disassembler(const llvm::TargetMachine& target_machine); + + // Returns a DisassemblerResult for the given object file, containing the + // disassembled code. + // + // If we couldn't retrieve a disassembler for this platform, an error status + // is returned. + StatusOr DisassembleObjectFile( + const llvm::object::ObjectFile& object_file) const; + + private: + const llvm::MCSubtargetInfo& subtarget_info_; + std::unique_ptr objfile_info_; + std::unique_ptr mc_context_; + std::unique_ptr disassembler_; + std::unique_ptr inst_printer_; + std::unique_ptr inst_analysis_; +}; + +} // namespace cpu +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_CPU_DISASSEMBLER_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/cpu/dot_op_emitter.h b/linux/include/tensorflow/compiler/xla/service/cpu/dot_op_emitter.h new file mode 100644 index 000000000..4c2041b55 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/cpu/dot_op_emitter.h @@ -0,0 +1,170 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_CPU_DOT_OP_EMITTER_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_CPU_DOT_OP_EMITTER_H_ + +#include "absl/strings/string_view.h" +#include "llvm/IR/IRBuilder.h" +#include "tensorflow/compiler/xla/service/cpu/cpu_options.h" +#include "tensorflow/compiler/xla/service/cpu/target_machine_features.h" +#include "tensorflow/compiler/xla/service/hlo_instruction.h" +#include "tensorflow/compiler/xla/service/hlo_module_config.h" +#include "tensorflow/compiler/xla/service/llvm_ir/ir_array.h" +#include "tensorflow/compiler/xla/service/llvm_ir/llvm_loop.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/platform/types.h" + +namespace xla { +namespace cpu { + +bool PotentiallyImplementedAsEigenDot( + const HloInstruction& hlo, + const TargetMachineFeatures& target_machine_features); + +// Returns the index for an operand to `hlo` that should ideally be column +// major. Returns nullopt if there is no such operand or if `hlo` is not a dot +// or a fusion containing a dot. +absl::optional ProfitableToMakeDotOperandColumnMajor( + const HloInstruction& hlo); + +// Returns true to indicate that we can generate a tiled LLVM IR implementation +// for |dot|. +bool ProfitableToImplementDotInTiledLlvmIr(const HloInstruction& dot); + +// Helper class for emitting LLVM IR to perform the dot operation. +class DotOpEmitter { + public: + // Emit LLVM IR to perform the dot operation on lhs_array and rhs_array and + // place the result in target_array. IR is emitted at current insert point of + // the builder. Upon completion of the method, the insert point is set to the + // end of all instructions emitted for this operation. + // + // If `addend_array` is not nullptr then it must be an array of the same + // dimensions as the result, and the result is computed as `addend_array` + + // dot(`lhs_array`, `rhs_array`). A non-null `addend_array` is only supported + // for Matrix-vector products. + static Status EmitDotOperation( + const HloInstruction& dot, const llvm_ir::IrArray& target_array, + const llvm_ir::IrArray& lhs_array, const llvm_ir::IrArray& rhs_array, + const llvm_ir::IrArray* addend_array, + llvm::Value* executable_run_options_value, llvm::IRBuilder<>* b, + const HloModuleConfig& hlo_module_config, + const TargetMachineFeatures& target_machine_features); + + private: + DotOpEmitter(const HloInstruction& dot, const llvm_ir::IrArray& target_array, + const llvm_ir::IrArray& lhs_array, + const llvm_ir::IrArray& rhs_array, + const llvm_ir::IrArray* addend_array, + llvm::Value* executable_run_options_value, llvm::IRBuilder<>* b, + const HloModuleConfig& hlo_module_config, + const TargetMachineFeatures& target_machine_features); + + // Emits the IR to perform the dot operation. + Status Emit(); + + // Emits instructions to perform a scalar dot product (a multiply of the + // LHS and RHS) and store the results in the target. + Status EmitScalarDot(); + + // Emit an LLVM IR implementation of the dot operation if we can. Returns + // true if an LLVM IR implementation was emitted. + bool EmitLlvmIrDotIfProfitable(); + + // Emits a call to the CPU runtime to perform the matrix multiply. + Status EmitCallToRuntime(); + + // Represents the dimensions of a matrix-matrix multiply operation. + struct MatMultDims { + // The number of rows in the LHS. + int64 m; + + // The number of columns in the LHS, which is also must be equal to the + // number of rows in the RHS. + int64 k; + + // The number of columns on the RHS. + int64 n; + + // True if the LHS matrix is column major. + bool lhs_column_major; + + // True if the LHS contraction dimension is not 1. + bool lhs_non_canonical; + + // True if the RHS matrix is column major. + bool rhs_column_major; + + // True if the RHS contraction dimension is not 0. + bool rhs_non_canonical; + + // True if the result matrix is column major. + bool target_column_major; + }; + + // Get the MatMultDims instance for the dot product this DotOpEmitter + // represents. Precondition: the dot is of rank 2 (and thus its operands are + // of rank 2 as well). + MatMultDims GetMatMultDims() const; + + bool EmitSmallGemmIfProfitable(const MatMultDims& mat_mult_dims); + + // When doing a tiled GEMV in LLVM IR, a "tile" consists of this many vector + // registers. + int64 GetGemvTilingFactor() const { + const int64 kDefaultTilingFactor = 8; + return options::LlvmIrGemvTilingFactor(hlo_module_config_) + .value_or(kDefaultTilingFactor); + } + + std::tuple GetGemmTileSize() const { + // Tuned for broadwell - Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz + // + // TODO(b/80093688): Tune for other architectures and centralize this + // information in one place. + const std::tuple kDefaultTileSize = + std::tuple(11, 9, 1); + return options::LlvmIrGemmTileSize(hlo_module_config_) + .value_or(kDefaultTileSize); + } + + // Returns true if we should use an experimental implementation of GEMM + // (general matrix matrix multiplication) if possible. + bool EnableExperimentalLlvmIrGemm() const { + return options::EnableExperimentalLlvmIrGemm(hlo_module_config_); + } + + // Returns true if we should call into multi-threaded Eigen routines. + bool ShouldUseMultiThreadedEigen() { + return hlo_module_config_.debug_options().xla_cpu_multi_thread_eigen(); + } + + const HloInstruction& dot_; + const llvm_ir::IrArray& target_array_; + const llvm_ir::IrArray& lhs_array_; + const llvm_ir::IrArray& rhs_array_; + const llvm_ir::IrArray* addend_array_; + llvm::Value* executable_run_options_value_; + llvm::IRBuilder<>* b_; + const HloModuleConfig& hlo_module_config_; + const TargetMachineFeatures& target_machine_features_; +}; + +} // namespace cpu +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_CPU_DOT_OP_EMITTER_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/cpu/elemental_ir_emitter.h b/linux/include/tensorflow/compiler/xla/service/cpu/elemental_ir_emitter.h new file mode 100644 index 000000000..e3fba9306 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/cpu/elemental_ir_emitter.h @@ -0,0 +1,53 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_CPU_ELEMENTAL_IR_EMITTER_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_CPU_ELEMENTAL_IR_EMITTER_H_ + +#include "llvm/IR/IRBuilder.h" +#include "llvm/IR/Module.h" +#include "llvm/IR/Value.h" +#include "tensorflow/compiler/xla/service/cpu/ir_emitter.h" +#include "tensorflow/compiler/xla/service/elemental_ir_emitter.h" +#include "tensorflow/compiler/xla/service/hlo_instruction.h" +#include "tensorflow/compiler/xla/statusor.h" + +namespace xla { +namespace cpu { + +class CpuElementalIrEmitter : public ElementalIrEmitter { + public: + CpuElementalIrEmitter(const HloModuleConfig& module_config, + IrEmitter* ir_emitter, llvm::Module* module) + : ElementalIrEmitter(module_config, module, ir_emitter->b()), + ir_emitter_(ir_emitter) {} + + llvm_ir::ElementGenerator MakeElementGenerator( + const HloInstruction* hlo, + const HloToElementGeneratorMap& operand_to_generator) override; + + protected: + StatusOr EmitAtan2(PrimitiveType prim_type, llvm::Value* lhs, + llvm::Value* rhs) override; + StatusOr EmitTanh(PrimitiveType prim_type, + llvm::Value* value) override; + + IrEmitter* ir_emitter_; +}; + +} // namespace cpu +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_CPU_ELEMENTAL_IR_EMITTER_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/cpu/ir_emission_utils.h b/linux/include/tensorflow/compiler/xla/service/cpu/ir_emission_utils.h new file mode 100644 index 000000000..68fbc7caa --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/cpu/ir_emission_utils.h @@ -0,0 +1,50 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_CPU_IR_EMISSION_UTILS_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_CPU_IR_EMISSION_UTILS_H_ + +#include "llvm/IR/Value.h" +#include "tensorflow/compiler/xla/service/cpu/target_machine_features.h" +#include "tensorflow/compiler/xla/service/hlo_instruction.h" + +namespace xla { +namespace cpu { + +bool PotentiallyImplementedAsEigenConvolution( + const HloInstruction& convolution, + const TargetMachineFeatures& target_machine_features); + +// Computes the minimum alignment guaranteed for a tensor of shape `shape` on +// the target machine. +int64 GetMinimumAlignmentForArray( + const Shape& shape, const TargetMachineFeatures& target_machine_features); + +// Dynamic loop bounds are specified as an array of dimension index +// [start, limit) pairs of ir values (one for each partitioned outer dimension). +// +// EX: Let 'shape' = [8, 16, 32], with the loop bounds of the two-most major +// dimensions dynamic. Then 'dynamic_loop_bounds' will contain the +// following ir values for the two most-major dimensions: +// [dim0_index_start_ir_value, dim0_index_limit_ir_value] +// [dim1_index_start_ir_value, dim1_index_limit_ir_value] +// +// See IrFunction and ParallelLoopEmitter for details. +using DynamicLoopBounds = std::vector>; + +} // namespace cpu +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_CPU_IR_EMISSION_UTILS_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/cpu/ir_emitter.h b/linux/include/tensorflow/compiler/xla/service/cpu/ir_emitter.h new file mode 100644 index 000000000..586f27b10 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/cpu/ir_emitter.h @@ -0,0 +1,586 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_CPU_IR_EMITTER_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_CPU_IR_EMITTER_H_ + +#include +#include +#include +#include +#include +#include + +#include "absl/container/flat_hash_map.h" +#include "absl/strings/string_view.h" +#include "absl/types/span.h" +#include "llvm/ADT/Triple.h" +#include "llvm/IR/Function.h" +#include "llvm/IR/IRBuilder.h" +#include "llvm/IR/Module.h" +#include "llvm/IR/Value.h" +#include "llvm/Target/TargetMachine.h" +#include "tensorflow/compiler/xla/service/buffer_assignment.h" +#include "tensorflow/compiler/xla/service/cpu/ir_function.h" +#include "tensorflow/compiler/xla/service/cpu/target_machine_features.h" +#include "tensorflow/compiler/xla/service/dfs_hlo_visitor_with_default.h" +#include "tensorflow/compiler/xla/service/hlo_computation.h" +#include "tensorflow/compiler/xla/service/hlo_instruction.h" +#include "tensorflow/compiler/xla/service/hlo_instructions.h" +#include "tensorflow/compiler/xla/service/hlo_module_config.h" +#include "tensorflow/compiler/xla/service/llvm_ir/alias_analysis.h" +#include "tensorflow/compiler/xla/service/llvm_ir/ir_array.h" +#include "tensorflow/compiler/xla/service/llvm_ir/ir_builder_mixin.h" +#include "tensorflow/compiler/xla/service/llvm_ir/loop_emitter.h" +#include "tensorflow/compiler/xla/service/name_uniquer.h" +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/types.h" + +namespace xla { +namespace cpu { +// This class is the top-level API for the XLA HLO --> LLVM IR compiler. It +// implements the DfsHloVisitor interface and emits HLO computations as LLVM IR +// functions. +class IrEmitter : public DfsHloVisitorWithDefault, + public IrBuilderMixin { + public: + // Create a new LLVM IR emitter. + // + // hlo_module: the HLO module we are emitting IR for. + // assignment: a BufferAssignment from which we know which buffers are used by + // the HLO nodes. + // llvm_module: the LLVM module to emit IR into. + // instruction_to_profile_idx: the mapping from HLO instructions to their + // index in the profiling array. + // computation_to_profile_idx: the mapping from HLO computations to their + // index in the profiling array. + IrEmitter(const HloModule& hlo_module, const BufferAssignment& assignment, + llvm::Module* llvm_module, + std::unordered_map + instruction_to_profile_idx, + std::unordered_map + computation_to_profile_idx, + const TargetMachineFeatures* target_machine); + ~IrEmitter() override; + + // Emit and return the given HLO computation as an LLVM IR + // function. + // + // function_name_prefix is the desired name of the function. If the name is + // not unique among already emitted functions then a suffix is appended to + // make the name unique. + // + // 'is_top_level_computation' has the following meanings for each CPU backend: + // *) sequential: indicates that this is the entry computation of the HLO + // module. + // *) parallel: indices that this is the callee of a kCall HLO in the entry + // computation of the HLO module. + // + // If 'instruction_order' is not NULL, then the HLO instructions are emitted + // in the given order. In this case, 'instruction_order' must be a + // topological sort of the set of nodes accessible from the root of the + // computation. + StatusOr EmitComputation( + HloComputation* computation, const string& function_name_prefix, + bool is_top_level_computation, + const std::vector* instruction_order); + + llvm::IRBuilder<>* b() { return &b_; } + + // builder() is for IrBuilderMixin. + llvm::IRBuilder<>* builder() { return &b_; } + + // Emit an LLVM global variable for every constant buffer allocation. + Status EmitConstantGlobals(); + + // Emit code to map one element according to `map_instr`. + llvm::Value* EmitElementalMap( + const HloMapInstruction& map_instr, + absl::Span elemental_operands, + absl::string_view name); + + protected: + // + // The following methods implement the DfsHloVisitor interface. + // + // Default action which emits code for most operations. Operations which are + // special in some way are handled explicitly in HandleFoo methods. + Status DefaultAction(HloInstruction* hlo) override; + + Status HandleBitcast(HloInstruction* bitcast) override; + Status HandleConstant(HloInstruction* constant) override; + Status HandleCopy(HloInstruction* copy) override; + Status HandleGetTupleElement(HloInstruction* get_tuple_element) override; + Status HandleSelect(HloInstruction* select) override; + Status HandleTupleSelect(HloInstruction* tuple_select) override; + Status HandleDot(HloInstruction* dot) override; + Status HandleConvolution(HloInstruction* convolution) override; + Status HandleFft(HloInstruction* fft) override; + Status HandleCrossReplicaSum(HloInstruction* crs) override; + Status HandleInfeed(HloInstruction* infeed) override; + Status HandleOutfeed(HloInstruction* outfeed) override; + Status HandleSort(HloInstruction* sort) override; + Status HandleParameter(HloInstruction* parameter) override; + Status HandleReduce(HloInstruction* reduce) override; + Status HandleReduceWindow(HloInstruction* reduce_window) override; + Status HandleSelectAndScatter(HloInstruction* select_and_scatter) override; + Status HandleSend(HloInstruction* send) override; + Status HandleSendDone(HloInstruction* send_done) override; + Status HandleSlice(HloInstruction* slice) override; + Status HandleDynamicSlice(HloInstruction* dynamic_slice) override; + Status HandleDynamicUpdateSlice( + HloInstruction* dynamic_update_slice) override; + Status HandleRecv(HloInstruction* recv) override; + Status HandleRecvDone(HloInstruction* recv_done) override; + Status HandlePad(HloInstruction* pad) override; + Status HandleTuple(HloInstruction* tuple) override; + Status HandleFusion(HloInstruction* fusion) override; + Status HandleCall(HloInstruction* call) override; + Status HandleCustomCall(HloInstruction* custom_call) override; + Status HandleWhile(HloInstruction* xla_while) override; + Status HandleConcatenate(HloInstruction* concatenate) override; + Status HandleConditional(HloInstruction* conditional) override; + Status HandleScatter(HloInstruction* scatter) override; + Status HandleAfterAll(HloInstruction* gen_token) override; + Status HandleRng(HloInstruction* rng) override; + Status FinishVisit(HloInstruction* root) override; + + Status Preprocess(HloInstruction* hlo) override; + Status Postprocess(HloInstruction* hlo) override; + + // A convenient helper for calling BufferAssignment::GetUniqueSlice. + BufferAllocation::Slice GetAllocationSlice( + const HloInstruction& hlo, const ShapeIndex& index = {}) const { + return assignment_.GetUniqueSlice(&hlo, index).ConsumeValueOrDie(); + } + + private: + // Private helper to initialize an IR function for the computation. + void InitializeIrFunction(const string& function_name); + + template + llvm::Value* GetProfileCounterCommon( + const T& hlo, + const std::unordered_map& profile_index_map); + + // Convenience functions to generate a GEP into the profile counter parameter + // which would correspond to the index for a given HLO instruction or + // computation. + llvm::Value* GetProfileCounterFor(const HloInstruction& instruction) { + return GetProfileCounterCommon(instruction, + instruction_to_profile_idx_); + } + + llvm::Value* GetProfileCounterFor(const HloComputation& computation) { + return GetProfileCounterCommon(computation, + computation_to_profile_idx_); + } + + // Gets the IR Value emitted previously for the given hlo. + // + // Prefer calling GetIrArrayFor if the value you're reading is a buffer, + // because GetIrArrayFor annotates buffer's loads/stores with noalias + // metadata. + // + // Make sure to call this only when you're certain a value *was* emitted - if + // not found, this will log a fatal error. + llvm::Value* GetEmittedValueFor(const HloInstruction* hlo); + + // Gets an IrArray representing the given hlo. + llvm_ir::IrArray GetIrArrayFor(const HloInstruction* hlo); + + // Gets a list of IrArrays, one for each of hlo's operands. + std::vector GetIrArraysForOperandsOf( + const HloInstruction* hlo); + + // Augments IrArray with aliasing information. + void AddAliasingInformationToIrArray(const HloInstruction& hlo, + llvm_ir::IrArray* array) { + alias_analysis_.AddAliasingInformationToIrArray(hlo, array); + } + + // Convenience function to get the IR type matching the given shape. + llvm::Type* IrShapeType(const Shape& shape); + + // Get the llvm::Value* that represents the "prof_counters" argument of the + // computation function being emitted by this emitter. + llvm::Value* GetProfileCountersArgument(); + + // Get the xla::ExecutableRunOptions that represents the "run_options" + // argument of the computation function being emitted by this emitter. + llvm::Value* GetExecutableRunOptionsArgument(); + + // Get the llvm::Value* that represents the "buffer_table" argument of the + // computation function being emitted by this emitter. + llvm::Value* GetBufferTableArgument(); + + // Helper for EmitBufferPointer. + llvm::Value* EmitGlobalBufferPointer(const BufferAllocation::Slice& slice, + const Shape& target_shape); + + // Helper for EmitBufferPointer. + llvm::Value* EmitThreadLocalBufferPointer( + const BufferAllocation::Slice& slice, const Shape& target_shape); + + // Emits code that computes the address of the given buffer allocation slice. + llvm::Value* EmitBufferPointer(const BufferAllocation::Slice& slice, + const Shape& target_shape); + + // Emits a function into the current module. This can be used for + // computations embedded inside other computations, such as the + // function that a map operation applies. + StatusOr EmitFunction( + HloComputation* function, // The function to emit. + absl::string_view + function_name_suffix); // Used for LLVM IR register names. + + // Emits a call to a thread local function (e.g. to the computation nested + // within a reduce or a map). Thread local callees (by definition) only write + // to and read from thread local allocations. + // + // `parameters` holds the *scalar values* that need to be passed to the + // callee. The return value is the scalar returned by the callee. + llvm::Value* EmitThreadLocalCall(const HloComputation& callee, + absl::Span parameters, + absl::string_view name); + + // Emits a call to a "global" function (e.g. to the computation nested within + // a kWhile or a kCall). Buffer assignment unabiguously assignes buffers to + // the parameters and return values for these computations so there is no need + // to explicitly pass parameters or return results. + void EmitGlobalCall(const HloComputation& callee, absl::string_view name); + + // Returns the buffer to which a global call to `callee` would have written + // its result. + llvm::Value* GetBufferForGlobalCallReturnValue(const HloComputation& callee); + + // Verifies that the element types of all of the given operand instructions + // match and are of one of the given supported types. + Status ElementTypesSameAndSupported( + const HloInstruction& instruction, + absl::Span operands, + absl::Span supported_types); + + // Emit IR to perform a computation for every element in the given target op. + // This produces a series of nested loops (one for each dimension of the op's + // shape). The body of the inner-most loop is provided by the body_emitter + // function. + // + // desc is an optional human-readable string that's added to the loop name in + // IR. Regardless of whether desc is provided, target_op->name() is included + // in the loop name. + // + // TODO(jingyue): target_op should be a `const HloInstruction*`. + Status EmitTargetElementLoop( + HloInstruction* target_op, + const llvm_ir::ElementGenerator& element_generator); + Status EmitTargetElementLoop( + HloInstruction* target_op, absl::string_view desc, + const llvm_ir::ElementGenerator& element_generator); + + // Emits a memcpy from the source instruction's result value to the + // destination's. Both source and destination must have an entry in the + // emitted_value_ table. + Status EmitMemcpy(const HloInstruction& source, + const HloInstruction& destination); + + // Emits IR to compute the target address of the buffer for the given op. + // After calling this function, you can get a pointer to this buffer by + // calling GetIrArrayForOp or GetEmittedValueFor. + Status EmitTargetAddressForOp(const HloInstruction* op); + + // Structurizes "array_elements" into an MD array that represents "shape". + // This is a recursive function, and "dimension_index" indicates the index of + // the current dimension that the function is considering (0 means the + // most-minor dimension). + llvm::Constant* CreateInitializerForConstantArray( + const std::vector& array_elements, const Shape& shape, + int64 dimension_index); + + // Tries to codegen a reduction operation using vectorized instructions. + // Returns true if successful, and false on failure. On failure, sets + // "failure_reason" to a string describing why it could not vectorize the + // reduction. + // + // TODO(sanjoy): Some of the things we do here can be abstracted out into + // concepts that generalize over other vectorizable operations. We should + // consider pulling out these abstractions into a VectorizingIrEmitter or + // something similar. + StatusOr EmitVectorizedReduce(HloInstruction* reduce, + HloInstruction* arg, + HloInstruction* init_value, + absl::Span dimensions, + HloComputation* function, + string* failure_reason); + + // We'd like to keep one or two one cache-line's worth of data in registers + // without generating IR with illegal (e.g. excessively large or + // non-power-of-two) vector types. We do this by introducing a layer of + // abstraction: we introduce a high level vector-like concept called a + // "sharded vector" that models data paralleism, and is mapped to a sequence + // scalar and vector llvm::Value s. + // + // For example, we can represent 29 f32 elements by a sharded vector mapped to + // a sequence of LLVM values of types [<16 x f32>, <8 x f32>, <4 x f32>, f32]. + // Note that the last element is scalar. + // + // There is no requirement on the ordering or the uniqueness of the elements + // mapped to sharded vectors -- we allow repeated elements, and we allow + // elements to appear in any order. + using ShardedVector = std::vector; + + // A sharded vector type is the element-wise llvm::Type's of some + // ShardedVector. + using ShardedVectorType = std::vector; + + // Create a sharded vector type corresponding to a "element_count" long + // sequence of "element_type" values. + ShardedVectorType CreateShardedVectorType(PrimitiveType element_type, + unsigned element_count); + + // Emit LLVM IR to store the sharded vector "value_to_store" to + // "store_address". + void EmitShardedVectorStore(llvm::Value* store_address, + const ShardedVector& value_to_store, + const int alignment, + const llvm_ir::IrArray& containing_array); + + using ReductionGenerator = std ::function*, llvm::Value*, llvm::Value*)>; + + // Tries to match the reduction function "function" to a known reduction + // pattern. Returns a non-null ReductionGenerator on a successful match, + // which can be used to generate the LLVM IR corresponding to said reduction. + // On failure, this stores a reason string into "failure_reason". + ReductionGenerator MatchReductionGenerator(HloComputation* function, + string* failure_reason) const; + + // Emits the inner loop nest that runs the reduction. Helper function for + // EmitVectorizedReduce. + StatusOr EmitInnerLoopForVectorizedReduction( + const ReductionGenerator& reduction_generator, + const llvm_ir::IrArray::Index& output_index, + const ShardedVectorType& accumulator_type, HloInstruction* init_value, + HloInstruction* arg, absl::Span dimensions, + unsigned element_alignment); + + // Tries to emit a fast concatenate operation using memcpy. Returns true if + // successful, and false on failure. On failure, sets "failure_reason" to a + // string describing why it could not emit a fast concatenate. + StatusOr EmitFastConcatenate(HloInstruction* concatenate, + absl::Span operands, + string* failure_reason); + + // Emits LLVM IR to transfer "element_count" elements of type "primitive_type" + // from the address "source" to the address "target". + void EmitTransferElements(llvm::Value* target, llvm::Value* source, + int64 element_count, PrimitiveType primitive_type, + const llvm_ir::IrArray& target_array, + const llvm_ir::IrArray& source_array); + + // Assignment of the buffers needed by the computation and their shape + // information. + const BufferAssignment& assignment_; + + // The LLVM module into which IR will be emitted. + llvm::Module* module_; + + // The target architecture. + llvm::Triple::ArchType arch_type_; + + // Used to produce unique names for generated functions. + NameUniquer name_uniquer_; + + // Map containing all previously emitted computations. + std::map emitted_functions_; + + // Map containing all previously emitted thread-local temporary buffers. + std::map, llvm::Value*> + thread_local_buffers_; + + // The following fields track the IR emission state. According to LLVM memory + // management rules, their memory is owned by the module (Note that IrFunction + // creates the encapsulated llvm::Function s.t. it is added to the llvm + // module's function list). + std::unique_ptr compute_function_; + llvm::IRBuilder<> b_; + + // The buffer allocation slice for the root of the computation being compiled. + // Only relevant for thread local computations. + BufferAllocation::Slice computation_root_allocation_; + + // Maps the buffer allocation slices for the parameters to the computation + // being compiled to their parameter numbers. Only relevant for thread local + // computations. + absl::flat_hash_map + computation_parameter_allocations_; + + // Maps HLO instructions to their index into the profile counter array. + const std::unordered_map + instruction_to_profile_idx_; + + // Maps HLO computations to their index into the profile counter array. + const std::unordered_map + computation_to_profile_idx_; + + // Maps HLOs to Values emitted for them. + std::unordered_map emitted_value_; + + llvm_ir::AliasAnalysis alias_analysis_; + + // The number of root instruction outer dimensions used in parallel loop + // emission (ParallelLoopEmitter). + int64 num_dynamic_loop_bounds_ = 0; + + // Returns whether the given instruction should be emitted as a parallel loop. + bool ShouldEmitParallelLoopFor(const HloInstruction& op) const { + // Emit parallel loop for root instruction if dynamic outer-dimension loop + // bounds were specified. + return num_dynamic_loop_bounds_ > 0 && + op.parent()->root_instruction() == &op; + } + + // This struct contains all the state needed to emit instructions for + // profiling a computation. + class ProfilingState { + public: + ProfilingState() : use_rdtscp_(false), prof_counters_(nullptr) {} + ProfilingState(bool use_rdtscp, llvm::Value* prof_counters) + : use_rdtscp_(use_rdtscp), prof_counters_(prof_counters) {} + + // Record the cycle counter before an HLO executes. + void RecordCycleStart(llvm::IRBuilder<>* b, HloInstruction* hlo); + // Record the number of cycles it took for an HLO to execute. + void RecordCycleDelta(llvm::IRBuilder<>* b, HloInstruction* hlo, + llvm::Value* prof_counter); + // Record the number of cycles it took for the entire computation to + // execute. + void RecordCompleteComputation(llvm::IRBuilder<>* b, + llvm::Value* prof_counter); + + // Convenience function to generate a call to an intrinsic which reads the + // CPU cycle counter. + llvm::Value* ReadCycleCounter(llvm::IRBuilder<>* b); + + // Store the cycle counter delta to the per-HLO profile counter. + void UpdateProfileCounter(llvm::IRBuilder<>* b, llvm::Value* prof_counter, + llvm::Value* cycle_end, llvm::Value* cycle_start); + + private: + // Should we use the x86-specific rdtscp or the generic readcyclecounter + // intrinsic? + bool use_rdtscp_; + + // The argument which corresponds to the profile counter buffer. + llvm::Value* prof_counters_; + + // The first read cycle counter in the program. + llvm::Value* first_read_cycle_start_ = nullptr; + + // The last read cycle counter in the program. + llvm::Value* last_read_cycle_end_ = nullptr; + + // An alloca used to hold the output of the aux value returned by the rdtscp + // intrinsic. + llvm::Value* aux_i8ptr_ = nullptr; + + // Maps HLOs to the value the cycle counter contained right before the HLO + // began to execute. + std::unordered_map cycle_starts_; + }; + + ProfilingState profiling_state_; + + // Given a load instruction and a shape or buffer size, annotate the load's + // result with the alignment required by the shape or size. + void AttachAlignmentMetadataForLoad(llvm::LoadInst* load, const Shape& shape); + void AttachAlignmentMetadataForLoad(llvm::LoadInst* load, int64 buffer_size); + + // Given a load instruction and a shape or buffer size, annotate the load's + // result with the dereferenceable bytes required by the shape / buffer size. + void AttachDereferenceableMetadataForLoad(llvm::LoadInst* load, + const Shape& shape); + void AttachDereferenceableMetadataForLoad(llvm::LoadInst* load, + int64 buffer_size); + + // Calculate the alignment of a buffer allocated for a given shape. + int MinimumAlignmentForShape(const Shape& shape); + + // Calculate the alignment of a buffer allocated for a given primitive type. + int MinimumAlignmentForPrimitiveType(PrimitiveType primitive_type); + + // Returns the number of bytes within the shape. + int64 ByteSizeOf(const Shape& shape) const; + + StatusOr EmitTargetElementLoopBodyForMap( + HloMapInstruction* map, const llvm_ir::IrArray::Index& index); + StatusOr EmitTargetElementLoopBodyForReduceWindow( + HloReduceWindowInstruction* reduce_window, + const llvm_ir::IrArray::Index& index); + StatusOr EmitTargetElementLoopBodyForConvolution( + HloConvolutionInstruction* convolution, + const llvm_ir::IrArray::Index& index); + StatusOr EmitTargetElementLoopBodyForReduce( + HloReduceInstruction* reduce, const llvm_ir::IrArray::Index& index); + + enum class XfeedKind { + kInfeed, + kOutfeed, + }; + + // Emit IR to transfer between a {infeed,outfeed} buffer and an in-program + // address. + Status EmitXfeedTransfer(XfeedKind kind, const Shape& shape, + llvm::Value* program_buffer_address); + + // Returns a ConstExpr bitcast. + llvm::Constant* EmitGlobalForLiteral(const Literal& literal); + + const HloModuleConfig& hlo_module_config_; + + bool is_top_level_computation_; + + const TargetMachineFeatures& target_machine_features_; + + struct LiteralPtrHashFunctor { + size_t operator()(const Literal* literal) const { return literal->Hash(); } + }; + + struct LiteralPtrEqualityFunctor { + bool operator()(const Literal* lhs, const Literal* rhs) const { + return *lhs == *rhs; + } + }; + + absl::flat_hash_map + emitted_literals_; + + absl::flat_hash_map + constant_buffer_to_global_; + + std::vector thread_local_computations_; + std::vector global_computations_; + + TF_DISALLOW_COPY_AND_ASSIGN(IrEmitter); +}; + +} // namespace cpu +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_CPU_IR_EMITTER_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/cpu/ir_function.h b/linux/include/tensorflow/compiler/xla/service/cpu/ir_function.h new file mode 100644 index 000000000..623a5f185 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/cpu/ir_function.h @@ -0,0 +1,134 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_CPU_IR_FUNCTION_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_CPU_IR_FUNCTION_H_ + +#include "absl/types/span.h" +#include "llvm/IR/Function.h" +#include "llvm/IR/IRBuilder.h" +#include "llvm/IR/Module.h" +#include "llvm/IR/Value.h" +#include "tensorflow/compiler/xla/service/cpu/ir_emission_utils.h" +#include "tensorflow/compiler/xla/shape_util.h" +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/compiler/xla/types.h" + +namespace xla { +namespace cpu { + +// IrFunction creates and encapsulates an llvm::Function, exposing methods to +// emitters for function and function argument access. +// The llvm::Function is created with the standard function signature +// used in the XLA CPU backend (see ir_function.cc for argument details). +// In addition IrFunction saves the callers IR insert point during construction, +// and restores it after destruction. +// +// Example usage: +// +// // Create and initialize new IrFunction. +// std::unique_ptr compute_function(new IrFunction(...)); +// // Emit IR for function body using IrFunction helper methods. +// ... +// // Store reference to llvm::Function for future invocation. +// ir_functions.push_back(compute_function.function()); +// // Delete IrFunction (finalizes IR function and restores caller insertion +// // point). +// compute_function.reset(); +// + +class IrFunction { + public: + IrFunction(const string& function_name, llvm::Function::LinkageTypes linkage, + const bool optimize_for_size_requested, + const bool enable_fast_math, llvm::Module* llvm_module, + llvm::IRBuilder<>* b, int64 num_dynamic_loop_bounds); + ~IrFunction(); + + // Emit ir to read and return the set of ir values representing the dynamic + // loop bounds argument of this function. + // Each element in returned vector is a pair of ir values representing + // the loop bounds for a specific dimension, where the first element of the + // pair is the dimension start index, and the second element of the pair + // is the dimension limit. + // EX: [dimension_i_index_start_ir_value, dimension_i_index_limit_ir_value] + // + DynamicLoopBounds GetDynamicLoopBounds(); + + // Returns the encapculated llvm::Function. + llvm::Function* function() { return function_; } + + // Get the llvm::Value* that represents this functions "retval" argument. + llvm::Argument* result_arg() { return result_arg_; } + + // Get the xla::ExecutableRunOptions that represents this functions + // "run_options" argument. + llvm::Value* exec_run_options_arg() { return exec_run_options_arg_; } + + // Get the llvm::Value* that represents this functions parameters argument. + llvm::Value* parameters_arg() { return parameters_arg_; } + + // Get the llvm::Value* that represents this functions "buffer_table" + // argument. + llvm::Value* buffer_table_arg() { return buffer_table_arg_; } + + // Get the llvm::Value* that represents this functions "prof_counters" + // argument. + llvm::Value* profile_counters_arg() { return profile_counters_arg_; } + + private: + // Initialize an llvm::Function with standard signature based on arguments. + void Initialize(const string& function_name, + llvm::Function::LinkageTypes linkage, + bool optimize_for_size_requested, bool enable_fast_math); + + // Emit ir to read and return the ir value for the dynamic loop bound at + // 'offset' from the "dynamic_loop_bounds" argument of this function. + llvm::Value* GetDynamicLoopBound(int64 offset); + + llvm::IRBuilder<>* b_; + llvm::Module* llvm_module_; + llvm::IRBuilder<>::InsertPointGuard caller_insert_point_guard_; + + int64 num_dynamic_loop_bounds_ = 0; + // Encapsulated llvm::Function. + llvm::Function* function_; + // Function argument IR values. + llvm::Argument* result_arg_; + llvm::Value* exec_run_options_arg_; + llvm::Value* parameters_arg_; + llvm::Value* buffer_table_arg_; + llvm::Value* dynamic_loop_bounds_arg_ = nullptr; + llvm::Value* profile_counters_arg_; +}; + +// Returns an array of compute function call argument ir values. +std::vector GetArrayFunctionCallArguments( + absl::Span parameter_addresses, llvm::IRBuilder<>* b, + absl::string_view name, llvm::Value* return_value_buffer, + llvm::Value* exec_run_options_arg, llvm::Value* buffer_table_arg, + llvm::Value* profile_counters_arg); + +// Emits a call to a runtime fork/join function which dispatches parallel +// calls to 'parallel_function' (and joins threads before returning). +Status EmitCallToParallelForkJoin( + const std::vector& arguments, const Shape& shape, + const std::vector& dimension_partition_counts, llvm::IRBuilder<>* b, + llvm::Function* parallel_function, const string& name); + +} // namespace cpu +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_CPU_IR_FUNCTION_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/cpu/llvm_ir_runtime.h b/linux/include/tensorflow/compiler/xla/service/cpu/llvm_ir_runtime.h new file mode 100644 index 000000000..555397267 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/cpu/llvm_ir_runtime.h @@ -0,0 +1,47 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_CPU_LLVM_IR_RUNTIME_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_CPU_LLVM_IR_RUNTIME_H_ + +#include "llvm/IR/Module.h" +#include "tensorflow/compiler/xla/service/llvm_ir/llvm_util.h" + +namespace xla { +namespace cpu { +namespace runtime { + +extern const char* const kTanhV4F32SymbolName; +extern const char* const kTanhV8F32SymbolName; +extern const char* const kExpV4F32SymbolName; +extern const char* const kExpV8F32SymbolName; +extern const char* const kLogV4F32SymbolName; +extern const char* const kLogV8F32SymbolName; + +// The following CPU runtime functions have LLVM-IR only implementations: +// +// - __xla_cpu_runtime_TanhV4F32 +// - __xla_cpu_runtime_TanhV8F32 +// +// |LinkIRRuntimeFunctions| rewrites calls to these functions into generic LLVM +// IR. + +void RewriteIRRuntimeFunctions(llvm::Module* module, bool enable_fast_math); + +} // namespace runtime +} // namespace cpu +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_CPU_LLVM_IR_RUNTIME_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/cpu/orc_jit_memory_mapper.h b/linux/include/tensorflow/compiler/xla/service/cpu/orc_jit_memory_mapper.h new file mode 100644 index 000000000..f89638411 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/cpu/orc_jit_memory_mapper.h @@ -0,0 +1,56 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_CPU_ORC_JIT_MEMORY_MAPPER_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_CPU_ORC_JIT_MEMORY_MAPPER_H_ + +#include + +#include "llvm/ExecutionEngine/SectionMemoryManager.h" + +namespace xla { +namespace cpu { + +namespace orc_jit_memory_mapper { +// Returns the registered memory mapper if there is one. Returns nullptr if no +// memory mapper is registered. +llvm::SectionMemoryManager::MemoryMapper* GetInstance(); + +class Registrar { + public: + // Registers the `mapper` as a memory mapper. This is a no-op if `mapper` is + // null. Precondition: no other memory mapper has been registered yet. + explicit Registrar( + std::unique_ptr mapper); +}; +} // namespace orc_jit_memory_mapper + +#define XLA_INTERNAL_REGISTER_ORC_JIT_MEMORY_MAPPER(mapper_instance, ctr) \ + static ::xla::cpu::orc_jit_memory_mapper::Registrar \ + XLA_INTERNAL_REGISTER_ORC_JIT_MEMORY_MAPPER_NAME(ctr)(mapper_instance) + +// __COUNTER__ must go through another macro to be properly expanded +#define XLA_INTERNAL_REGISTER_ORC_JIT_MEMORY_MAPPER_NAME(ctr) \ + __orc_jit_memory_mapper_registrar_##ctr + +// Registers the std::unique_ptr +// returned by the `factory` expression. `factory` is allowed to evaluate to +// a null unique_ptr in which case this macro does nothing. +#define XLA_REGISTER_ORC_JIT_MEMORY_MAPPER(factory) \ + XLA_INTERNAL_REGISTER_ORC_JIT_MEMORY_MAPPER(factory, __COUNTER__) +} // namespace cpu +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_CPU_ORC_JIT_MEMORY_MAPPER_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/cpu/parallel_loop_emitter.h b/linux/include/tensorflow/compiler/xla/service/cpu/parallel_loop_emitter.h new file mode 100644 index 000000000..a604e1db2 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/cpu/parallel_loop_emitter.h @@ -0,0 +1,73 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_CPU_PARALLEL_LOOP_EMITTER_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_CPU_PARALLEL_LOOP_EMITTER_H_ + +#include "llvm/IR/IRBuilder.h" +#include "llvm/IR/Value.h" +#include "tensorflow/compiler/xla/service/cpu/ir_emission_utils.h" +#include "tensorflow/compiler/xla/service/llvm_ir/ir_array.h" +#include "tensorflow/compiler/xla/service/llvm_ir/loop_emitter.h" + +namespace xla { +namespace cpu { + +// ParallelLoopEmitter emits a loop nest for the target array shape. +// The outer loop bounds of the loop nest are passed as ir values at runtime +// (specified in 'dynamic_loop_bounds'), and the inner loop bounds are static. +// Dynamic loop bounds are specified as an array of dimension index +// [start, limit) pairs of ir values (one for each partitioned outer dimension). +// +// EX: Let 'shape' = [8, 16, 32], with the loop bounds of the two-most major +// dimensions dynamic. Then 'dynamic_loop_bounds' will contain the +// following ir values for the two most-major dimensions: +// [dim0_index_start_ir_value, dim0_index_limit_ir_value] +// [dim1_index_start_ir_value, dim1_index_limit_ir_value] +// +// Code emitted by ParallelLoopEmitter will be called in a multi-threaded +// context where each thread will be assigned a different set of outer dimension +// partitions, and where all threads will collectively iterate over the +// entire target array shape. +// +// Outer dimension partitions can be generated using the ShapePartitionAssigner +// and ShapePartitionIterator utility classes from shape_partition.cc. +// +class ParallelLoopEmitter : public llvm_ir::LoopEmitter { + public: + // Constructs a ParallelLoopEmitter which uses 'target_element_generator' to + // generate elements, 'dynamic_loop_bounds' to set the loop bounds of the + // most-major dimensions, and 'target_array.' shape to set the static loop + // bounds for the most-minor dimensions. + ParallelLoopEmitter(const llvm_ir::ElementGenerator& target_element_generator, + const llvm_ir::IrArray& target_array, + const DynamicLoopBounds* dynamic_loop_bounds, + llvm::IRBuilder<>* b); + + ParallelLoopEmitter(const ParallelLoopEmitter&) = delete; + ParallelLoopEmitter& operator=(const ParallelLoopEmitter&) = delete; + ~ParallelLoopEmitter() override = default; + + std::vector EmitIndexAndSetExitBasicBlock( + absl::string_view loop_name, llvm::Type* index_type) override; + + private: + const DynamicLoopBounds* dynamic_loop_bounds_; +}; + +} // namespace cpu +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_CPU_PARALLEL_LOOP_EMITTER_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/cpu/parallel_task_assignment.h b/linux/include/tensorflow/compiler/xla/service/cpu/parallel_task_assignment.h new file mode 100644 index 000000000..3822d5300 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/cpu/parallel_task_assignment.h @@ -0,0 +1,109 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_CPU_PARALLEL_TASK_ASSIGNMENT_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_CPU_PARALLEL_TASK_ASSIGNMENT_H_ + +#include "tensorflow/compiler/xla/service/cpu/target_machine_features.h" +#include "tensorflow/compiler/xla/service/hlo_cost_analysis.h" +#include "tensorflow/compiler/xla/service/hlo_module.h" +#include "tensorflow/compiler/xla/service/hlo_pass_interface.h" + +namespace xla { +namespace cpu { + +// Simple interface for different parallel cost model implementations. +class ParallelCostModel { + public: + virtual ~ParallelCostModel() = default; + virtual int64 GetParallelTaskCount(HloInstruction* instruction) = 0; +}; + +// ParallelTaskAssignment computes parallel task counts for HLOs in 'module'. +class ParallelTaskAssignment { + public: + // 'max_parallelism': the maximum parallel task count per instruction. + // 'shape_size': shape size function used by HloCostAnalysis during parallel + // task assignment. + // 'module': the containing HloModule. + ParallelTaskAssignment(const int64 max_parallelism, + const HloCostAnalysis::ShapeSizeFunction& shape_size, + HloModule* module, + const TargetMachineFeatures* target_machine_features); + ~ParallelTaskAssignment() {} + + // Computes and returns the target parallel task count for 'instruction'. + int64 GetTargetParallelTaskCount(HloInstruction* instruction); + + private: + std::unique_ptr cost_model_; + const TargetMachineFeatures& target_machine_features_; +}; + +// ParallelTaskAssigner computes target parallel task counts for all HLOs +// in the module, then assigns parallel task counts to HLOs in the entry +// computation, or to HLOs in embedded computations invoked by (potentially +// nested) kWhile or kCall instructions. +// Each HLO which is assigned parallel task counts is outlined into its +// own embedded computation, which is compiled as a parallel compute function, +// and which is invoked from a kCall instruction that is lowered in codegen to +// a runtime parallel fork/join call. +class ParallelTaskAssigner : public HloModulePass { + public: + // 'max_parallelism': the maximum parallel task count per instruction. + // 'shape_size': shape size function used by HloCostAnalysis during parallel + // task assignment. + ParallelTaskAssigner(const int64 max_parallelism, + const HloCostAnalysis::ShapeSizeFunction& shape_size, + const TargetMachineFeatures* target_machine_features) + : max_parallelism_(max_parallelism), + shape_size_function_(shape_size), + target_machine_features_(*target_machine_features) {} + ~ParallelTaskAssigner() override {} + + absl::string_view name() const override { + return "cpu-parallel-task-assigner"; + } + + // Run parallel task assigner on 'module'. + // Returns true if the computation was changed, false otherwise. + StatusOr Run(HloModule* module) override; + + private: + using HloToParallelTasks = std::unordered_map; + + // Assigns target parallel tasks from 'hlo_to_parallel_tasks' to HLOs in + // 'module'. + // Returns true if the computation was changed, false otherwise. + bool AssignParallelTasks(HloModule* module, + const HloToParallelTasks& hlo_to_parallel_tasks); + bool AssignParallelTasksHelper( + HloModule* module, HloComputation* computation, + const HloToParallelTasks& hlo_to_parallel_tasks); + + // Computes target parallel task counts (returned in 'parallel_task_counts') + // for parallelizable instructions in 'module'. + void ComputeTargetParallelTasks(HloModule* module, + HloToParallelTasks* hlo_to_parallel_tasks); + + int64 max_parallelism_; + HloCostAnalysis::ShapeSizeFunction shape_size_function_; + const TargetMachineFeatures& target_machine_features_; +}; + +} // namespace cpu +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_CPU_PARALLEL_TASK_ASSIGNMENT_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/cpu/runtime_conv2d.h b/linux/include/tensorflow/compiler/xla/service/cpu/runtime_conv2d.h new file mode 100644 index 000000000..7337c907f --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/cpu/runtime_conv2d.h @@ -0,0 +1,54 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_CPU_RUNTIME_CONV2D_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_CPU_RUNTIME_CONV2D_H_ + +#include "third_party/eigen3/Eigen/Core" +#include "tensorflow/core/platform/types.h" + +extern "C" { + +extern void __xla_cpu_runtime_EigenConvF32( + const void* /* xla::ExecutableRunOptions* */ run_options_ptr, float* out, + float* lhs, float* rhs, tensorflow::int64 input_batch, + tensorflow::int64 input_rows, tensorflow::int64 input_cols, + tensorflow::int64 input_channels, tensorflow::int64 kernel_rows, + tensorflow::int64 kernel_cols, tensorflow::int64 kernel_channels, + tensorflow::int64 kernel_filters, tensorflow::int64 output_rows, + tensorflow::int64 output_cols, tensorflow::int64 row_stride, + tensorflow::int64 col_stride, tensorflow::int64 padding_top, + tensorflow::int64 padding_bottom, tensorflow::int64 padding_left, + tensorflow::int64 padding_right, tensorflow::int64 lhs_row_dilation, + tensorflow::int64 lhs_col_dilation, tensorflow::int64 rhs_row_dilation, + tensorflow::int64 rhs_col_dilation); + +extern void __xla_cpu_runtime_EigenConvF16( + const void* /* xla::ExecutableRunOptions* */ run_options_ptr, + Eigen::half* out, Eigen::half* lhs, Eigen::half* rhs, + tensorflow::int64 input_batch, tensorflow::int64 input_rows, + tensorflow::int64 input_cols, tensorflow::int64 input_channels, + tensorflow::int64 kernel_rows, tensorflow::int64 kernel_cols, + tensorflow::int64 kernel_channels, tensorflow::int64 kernel_filters, + tensorflow::int64 output_rows, tensorflow::int64 output_cols, + tensorflow::int64 row_stride, tensorflow::int64 col_stride, + tensorflow::int64 padding_top, tensorflow::int64 padding_bottom, + tensorflow::int64 padding_left, tensorflow::int64 padding_right, + tensorflow::int64 lhs_row_dilation, tensorflow::int64 lhs_col_dilation, + tensorflow::int64 rhs_row_dilation, tensorflow::int64 rhs_col_dilation); + +} // extern "C" + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_CPU_RUNTIME_CONV2D_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/cpu/runtime_conv2d_impl.h b/linux/include/tensorflow/compiler/xla/service/cpu/runtime_conv2d_impl.h new file mode 100644 index 000000000..85af63bb0 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/cpu/runtime_conv2d_impl.h @@ -0,0 +1,88 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_CPU_RUNTIME_CONV2D_IMPL_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_CPU_RUNTIME_CONV2D_IMPL_H_ + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/kernels/eigen_spatial_convolutions.h" +#include "tensorflow/core/platform/types.h" + +// 'tensorflow' namespace is used so that int64 and other types don't require +// qualification. +namespace tensorflow { +namespace xla { + +template +void EigenConvImpl(const EigenDevice& device, ScalarType* out, ScalarType* lhs, + ScalarType* rhs, int64 input_batch, int64 input_rows, + int64 input_cols, int64 input_channels, int64 kernel_rows, + int64 kernel_cols, int64 kernel_channels, + int64 kernel_filters, int64 output_rows, int64 output_cols, + int64 row_stride, int64 col_stride, int64 padding_top, + int64 padding_bottom, int64 padding_left, + int64 padding_right, int64 lhs_row_dilation, + int64 lhs_col_dilation, int64 rhs_row_dilation, + int64 rhs_col_dilation) { + const Eigen::TensorMap, + Eigen::Aligned> + input(lhs, input_batch, input_rows, input_cols, input_channels); + + const Eigen::TensorMap, + Eigen::Aligned> + kernel(rhs, kernel_rows, kernel_cols, kernel_channels, kernel_filters); + + Eigen::TensorMap, + Eigen::Aligned> + output(out, input_batch, output_rows, output_cols, kernel_filters); + + Eigen::array, 1> contract_dims; + contract_dims[0] = Eigen::IndexPair(1, 0); + + // Molds the output of the patch extraction code into a 2d tensor: + // - the first dimension (dims[0]): the patch values to be multiplied with the + // kernels + // - the second dimension (dims[1]): everything else + Eigen::DSizes pre_contract_dims; + pre_contract_dims[0] = output_cols * output_rows * input_batch; + pre_contract_dims[1] = kernel_channels * kernel_cols * kernel_rows; + + // Molds the output of the contraction into the shape expected by the user: + Eigen::DSizes post_contract_dims; + post_contract_dims[0] = input_batch; + post_contract_dims[1] = output_rows; + post_contract_dims[2] = output_cols; + post_contract_dims[3] = kernel_filters; + + Eigen::DSizes kernel_dims; + kernel_dims[0] = kernel_channels * kernel_cols * kernel_rows; + kernel_dims[1] = kernel_filters; + + // The row and column dimensions must be flipped when passed to Eigen. + output.device(device) = + input + .extract_image_patches(kernel_cols, kernel_rows, col_stride, + row_stride, rhs_col_dilation, rhs_row_dilation, + lhs_col_dilation, lhs_row_dilation, + padding_left, padding_right, padding_top, + padding_bottom, static_cast(0.0f)) + .reshape(pre_contract_dims) + .contract(kernel.reshape(kernel_dims), contract_dims) + .reshape(post_contract_dims); +} + +} // namespace xla +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_CPU_RUNTIME_CONV2D_IMPL_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/cpu/runtime_conv2d_mkl.h b/linux/include/tensorflow/compiler/xla/service/cpu/runtime_conv2d_mkl.h new file mode 100644 index 000000000..b239e71d2 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/cpu/runtime_conv2d_mkl.h @@ -0,0 +1,39 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_CPU_RUNTIME_CONV2D_MKL_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_CPU_RUNTIME_CONV2D_MKL_H_ + +#include +#include "tensorflow/core/platform/types.h" + +extern "C" { + +extern void __xla_cpu_runtime_MKLConvF32( + const void* /* xla::ExecutableRunOptions* */ run_options_ptr, float* out, + float* lhs, float* rhs, tensorflow::int64 input_batch, + tensorflow::int64 input_rows, tensorflow::int64 input_cols, + tensorflow::int64 input_channels, tensorflow::int64 kernel_rows, + tensorflow::int64 kernel_cols, tensorflow::int64 kernel_channels, + tensorflow::int64 kernel_filters, tensorflow::int64 output_rows, + tensorflow::int64 output_cols, tensorflow::int64 row_stride, + tensorflow::int64 col_stride, tensorflow::int64 padding_top, + tensorflow::int64 padding_bottom, tensorflow::int64 padding_left, + tensorflow::int64 padding_right, tensorflow::int64 lhs_row_dilation, + tensorflow::int64 lhs_col_dilation, tensorflow::int64 rhs_row_dilation, + tensorflow::int64 rhs_col_dilation); +} + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_CPU_RUNTIME_CONV2D_MKL_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/cpu/runtime_fft.h b/linux/include/tensorflow/compiler/xla/service/cpu/runtime_fft.h new file mode 100644 index 000000000..f20c5aa0a --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/cpu/runtime_fft.h @@ -0,0 +1,31 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_CPU_RUNTIME_FFT_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_CPU_RUNTIME_FFT_H_ + +#include "tensorflow/core/platform/types.h" + +extern "C" { + +extern void __xla_cpu_runtime_EigenFft( + const void* /* xla::ExecutableRunOptions* */ run_options_ptr, void* out, + void* operand, tensorflow::int32 fft_type, tensorflow::int32 fft_rank, + tensorflow::int64 input_batch, tensorflow::int64 fft_length0, + tensorflow::int64 fft_length1, tensorflow::int64 fft_length2); + +} // extern "C" + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_CPU_RUNTIME_FFT_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/cpu/runtime_fft_impl.h b/linux/include/tensorflow/compiler/xla/service/cpu/runtime_fft_impl.h new file mode 100644 index 000000000..0bf693edd --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/cpu/runtime_fft_impl.h @@ -0,0 +1,234 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_CPU_RUNTIME_FFT_IMPL_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_CPU_RUNTIME_FFT_IMPL_H_ + +#include + +#include "third_party/eigen3/Eigen/Core" +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/framework/numeric_types.h" +#include "tensorflow/core/platform/types.h" + +// 'tensorflow' namespace is used so that int64 and other types don't require +// qualification. +namespace tensorflow { +namespace xla { + +namespace internal { + +// Computes either a forward or reverse complex-to-complex FFT. +template +void EigenFftC2C(const EigenDevice& device, complex64* out, complex64* operand, + int64 input_batch, int64 fft_length0, int64 fft_length1, + int64 fft_length2) { + // Create the axes (which are always trailing). + const auto axes = Eigen::ArrayXi::LinSpaced(FFTRank, 1, FFTRank); + constexpr auto direction = Forward ? Eigen::FFT_FORWARD : Eigen::FFT_REVERSE; + + const std::array fft_shape = { + {fft_length0, fft_length1, fft_length2}}; + + Eigen::DSizes dims; + dims[0] = input_batch; + for (int i = 0; i < FFTRank; i++) { + dims[i + 1] = fft_shape[i]; + } + const Eigen::TensorMap, + Eigen::Aligned> + input(operand, dims); + Eigen::TensorMap, + Eigen::Aligned> + output(out, dims); + output.device(device) = input.template fft(axes); +} + +// Computes a forward real->complex FFT, slicing out redundant negative +// frequencies from the innermost dimension. +template +void EigenFftR2C(const EigenDevice& device, complex64* out, float* operand, + int64 input_batch, int64 fft_length0, int64 fft_length1, + int64 fft_length2) { + const std::array fft_shape = { + {fft_length0, fft_length1, fft_length2}}; + + Eigen::DSizes in_dims; + in_dims[0] = input_batch; + Eigen::DSizes out_dims; + out_dims[0] = input_batch; + for (int i = 0; i < FFTRank; i++) { + in_dims[i + 1] = fft_shape[i]; + out_dims[i + 1] = i == FFTRank - 1 ? fft_shape[i] / 2 + 1 : fft_shape[i]; + } + const Eigen::TensorMap, + Eigen::Aligned> + input(operand, in_dims); + Eigen::TensorMap, + Eigen::Aligned> + output(out, out_dims); + + // Create the axes (which are always trailing). + const auto axes = Eigen::ArrayXi::LinSpaced(FFTRank, 1, FFTRank); + + // Compute the full FFT using a temporary tensor. + Eigen::Tensor full_fft(in_dims); + + const Eigen::DSizes zero_start_indices; + full_fft.device(device) = + input.template fft(axes); + + // Slice away the negative frequency components. + output.device(device) = full_fft.slice(zero_start_indices, out_dims); +} + +// Computes a reverse complex->real FFT, reconstructing redundant negative +// frequencies using reverse conjugate on innermost dimension after doing IFFT +// on outer dimensions. +template +void EigenFftC2R(const EigenDevice& device, float* out, complex64* operand, + int64 input_batch, int64 fft_length0, int64 fft_length1, + int64 fft_length2) { + const std::array fft_shape = { + {fft_length0, fft_length1, fft_length2}}; + + Eigen::DSizes in_dims; + in_dims[0] = input_batch; + Eigen::DSizes out_dims; + out_dims[0] = input_batch; + for (int i = 0; i < FFTRank; i++) { + in_dims[i + 1] = i == FFTRank - 1 ? fft_shape[i] / 2 + 1 : fft_shape[i]; + out_dims[i + 1] = fft_shape[i]; + } + const Eigen::TensorMap, + Eigen::Aligned> + input(operand, in_dims); + Eigen::TensorMap, + Eigen::Aligned> + output(out, out_dims); + + // Calculate the shape of the temporary tensor for the full FFT and the + // region we will slice from input given fft_shape. We slice input to + // fft_shape on its inner-most dimensions, except the last (which we + // slice to fft_shape[-1] / 2 + 1). + Eigen::Tensor full_fft(out_dims); + + // Calculate the starting point and range of the source of + // negative frequency part. + auto neg_sizes = in_dims; + neg_sizes[FFTRank] = fft_shape[FFTRank - 1] - in_dims[FFTRank]; + Eigen::DSizes neg_target_indices; + neg_target_indices[FFTRank] = in_dims[FFTRank]; + + const Eigen::DSizes zero_start_indices; + Eigen::DSizes neg_start_indices; + neg_start_indices[FFTRank] = 1; + + full_fft.slice(zero_start_indices, in_dims).device(device) = input; + + // First, conduct IFFTs on outer dimensions. We save computation (and + // avoid touching uninitialized memory) by slicing full_fft to the + // subregion we wrote input to. + if (FFTRank > 1) { + const auto outer_axes = + Eigen::ArrayXi::LinSpaced(FFTRank - 1, 1, FFTRank - 1); + full_fft.slice(zero_start_indices, in_dims).device(device) = + full_fft.slice(zero_start_indices, in_dims) + .template fft(outer_axes); + } + + // Reconstruct the full FFT by appending reversed and conjugated + // spectrum as the negative frequency part. + Eigen::array reverse_last_axis; + for (auto i = 0; i <= FFTRank; i++) { + reverse_last_axis[i] = i == FFTRank; + } + + if (neg_sizes[FFTRank] != 0) { + full_fft.slice(neg_target_indices, neg_sizes).device(device) = + full_fft.slice(neg_start_indices, neg_sizes) + .reverse(reverse_last_axis) + .conjugate(); + } + + auto inner_axis = Eigen::array{FFTRank}; + output.device(device) = + full_fft.template fft(inner_axis); +} + +template +void EigenFftWithRank(const EigenDevice& device, void* out, void* operand, + int32 fft_type, int64 input_batch, int64 fft_length0, + int64 fft_length1, int64 fft_length2) { + switch (fft_type) { + case ::xla::FftType::FFT: + EigenFftC2C( + device, static_cast(out), + static_cast(operand), input_batch, fft_length0, + fft_length1, fft_length2); + break; + case ::xla::FftType::IFFT: + EigenFftC2C( + device, static_cast(out), + static_cast(operand), input_batch, fft_length0, + fft_length1, fft_length2); + break; + case ::xla::FftType::RFFT: + EigenFftR2C( + device, static_cast(out), static_cast(operand), + input_batch, fft_length0, fft_length1, fft_length2); + break; + case ::xla::FftType::IRFFT: + EigenFftC2R( + device, static_cast(out), static_cast(operand), + input_batch, fft_length0, fft_length1, fft_length2); + break; + default: + // Unsupported FFT type + abort(); + } +} + +} // namespace internal + +template +void EigenFftImpl(const EigenDevice& device, void* out, void* operand, + int32 fft_type, int32 fft_rank, int64 input_batch, + int64 fft_length0, int64 fft_length1, int64 fft_length2) { + switch (fft_rank) { + case 1: + internal::EigenFftWithRank<1, EigenDevice>( + device, out, operand, fft_type, input_batch, fft_length0, 0, 0); + break; + case 2: + internal::EigenFftWithRank<2, EigenDevice>(device, out, operand, fft_type, + input_batch, fft_length0, + fft_length1, 0); + break; + case 3: + internal::EigenFftWithRank<3, EigenDevice>(device, out, operand, fft_type, + input_batch, fft_length0, + fft_length1, fft_length2); + break; + default: + // Unsupported FFT rank + abort(); + } +} + +} // namespace xla +} // namespace tensorflow + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_CPU_RUNTIME_FFT_IMPL_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/cpu/runtime_fork_join.h b/linux/include/tensorflow/compiler/xla/service/cpu/runtime_fork_join.h new file mode 100644 index 000000000..a279c7d2d --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/cpu/runtime_fork_join.h @@ -0,0 +1,33 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_CPU_RUNTIME_FORK_JOIN_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_CPU_RUNTIME_FORK_JOIN_H_ + +#include "tensorflow/core/platform/types.h" + +extern "C" { + +// Dispatches 'num_partitions' parallel calls to 'function_ptr' and joins +// threads before returning. See comments in runtime_fork_join.cc for details. +extern void __xla_cpu_runtime_ParallelForkJoin( + void* result_ptr, const void* run_options_ptr, const void** params, + void** buffer_table, tensorflow::uint64* prof_counters, + tensorflow::int32 num_partitions, tensorflow::int64* partitions, + tensorflow::int32 num_partitioned_dims, void* function_ptr); + +} // extern "C" + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_CPU_RUNTIME_FORK_JOIN_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/cpu/runtime_fp16.h b/linux/include/tensorflow/compiler/xla/service/cpu/runtime_fp16.h new file mode 100644 index 000000000..01d92d031 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/cpu/runtime_fp16.h @@ -0,0 +1,27 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_CPU_RUNTIME_FP16_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_CPU_RUNTIME_FP16_H_ + +#include "tensorflow/core/platform/types.h" + +// Converts an F32 value to a F16. +extern "C" tensorflow::uint16 __gnu_f2h_ieee(float); + +// Converts an F16 value to a F32. +extern "C" float __gnu_h2f_ieee(tensorflow::uint16); + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_CPU_RUNTIME_FP16_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/cpu/runtime_key_value_sort.h b/linux/include/tensorflow/compiler/xla/service/cpu/runtime_key_value_sort.h new file mode 100644 index 000000000..28e35e82c --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/cpu/runtime_key_value_sort.h @@ -0,0 +1,88 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_CPU_RUNTIME_KEY_VALUE_SORT_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_CPU_RUNTIME_KEY_VALUE_SORT_H_ + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/platform/types.h" + +extern "C" { + +// 'keys' represents a 3-dimensional shape with dimensions [a, b, c]. The 'b' +// dimension of 'keys' is sorted into ascending order. 'values' can be nullptr. +// If 'values' is not nullptr, the elements in 'values' are reordered in such a +// way that if the element at index 'i' in 'keys' was moved to index 'j', the +// element at index 'i' in 'values' is also moved to index 'j' (which means that +// the same elements correspond to each other as before). +extern void __xla_cpu_runtime_KeyValueSortPRED( + bool* keys, tensorflow::int64 a, tensorflow::int64 b, tensorflow::int64 c, + char* values, tensorflow::int32 values_primitive_type_size_in_bytes); + +extern void __xla_cpu_runtime_KeyValueSortS8( + tensorflow::int8* keys, tensorflow::int64 a, tensorflow::int64 b, + tensorflow::int64 c, char* values, + tensorflow::int32 values_primitive_type_size_in_bytes); + +extern void __xla_cpu_runtime_KeyValueSortU8( + tensorflow::uint8* keys, tensorflow::int64 a, tensorflow::int64 b, + tensorflow::int64 c, char* values, + tensorflow::int32 values_primitive_type_size_in_bytes); + +extern void __xla_cpu_runtime_KeyValueSortS16( + tensorflow::int16* keys, tensorflow::int64 a, tensorflow::int64 b, + tensorflow::int64 c, char* values, + tensorflow::int32 values_primitive_type_size_in_bytes); + +extern void __xla_cpu_runtime_KeyValueSortU16( + tensorflow::uint16* keys, tensorflow::int64 a, tensorflow::int64 b, + tensorflow::int64 c, char* values, + tensorflow::int32 values_primitive_type_size_in_bytes); + +extern void __xla_cpu_runtime_KeyValueSortF16( + Eigen::half* keys, tensorflow::int64 a, tensorflow::int64 b, + tensorflow::int64 c, char* values, + tensorflow::int32 values_primitive_type_size_in_bytes); + +extern void __xla_cpu_runtime_KeyValueSortS32( + tensorflow::int32* keys, tensorflow::int64 a, tensorflow::int64 b, + tensorflow::int64 c, char* values, + tensorflow::int32 values_primitive_type_size_in_bytes); + +extern void __xla_cpu_runtime_KeyValueSortU32( + tensorflow::uint32* keys, tensorflow::int64 a, tensorflow::int64 b, + tensorflow::int64 c, char* values, + tensorflow::int32 values_primitive_type_size_in_bytes); + +extern void __xla_cpu_runtime_KeyValueSortF32( + float* keys, tensorflow::int64 a, tensorflow::int64 b, tensorflow::int64 c, + char* values, tensorflow::int32 values_primitive_type_size_in_bytes); + +extern void __xla_cpu_runtime_KeyValueSortS64( + tensorflow::int64* keys, tensorflow::int64 a, tensorflow::int64 b, + tensorflow::int64 c, char* values, + tensorflow::int32 values_primitive_type_size_in_bytes); + +extern void __xla_cpu_runtime_KeyValueSortU64( + tensorflow::uint64* keys, tensorflow::int64 a, tensorflow::int64 b, + tensorflow::int64 c, char* values, + tensorflow::int32 values_primitive_type_size_in_bytes); + +extern void __xla_cpu_runtime_KeyValueSortF64( + double* keys, tensorflow::int64 a, tensorflow::int64 b, tensorflow::int64 c, + char* values, tensorflow::int32 values_primitive_type_size_in_bytes); +} + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_CPU_RUNTIME_KEY_VALUE_SORT_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/cpu/runtime_matmul.h b/linux/include/tensorflow/compiler/xla/service/cpu/runtime_matmul.h new file mode 100644 index 000000000..d96fe3d58 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/cpu/runtime_matmul.h @@ -0,0 +1,49 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_CPU_RUNTIME_MATMUL_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_CPU_RUNTIME_MATMUL_H_ + +#include "third_party/eigen3/Eigen/Core" +#include "tensorflow/core/platform/types.h" + +extern "C" { + +// Performs a multi-threaded matrix multiplication using Eigen. 'lhs' and 'rhs' +// are pointers to buffers containing input matrices in column-major +// order. 'out' is a pointer to a buffer sufficiently large to hold the result +// of the operation. Following standard nomenclature: lhs is m x k, +// rhs is k x n, and out is m x n. +extern void __xla_cpu_runtime_EigenMatMulF16( + const void* /* xla::ExecutableRunOptions* */ run_options_ptr, + Eigen::half* out, Eigen::half* lhs, Eigen::half* rhs, tensorflow::int64 m, + tensorflow::int64 n, tensorflow::int64 k, tensorflow::int32 transpose_lhs, + tensorflow::int32 transpose_rhs); + +extern void __xla_cpu_runtime_EigenMatMulF32( + const void* /* xla::ExecutableRunOptions* */ run_options_ptr, float* out, + float* lhs, float* rhs, tensorflow::int64 m, tensorflow::int64 n, + tensorflow::int64 k, tensorflow::int32 transpose_lhs, + tensorflow::int32 transpose_rhs); + +extern void __xla_cpu_runtime_EigenMatMulF64( + const void* /* xla::ExecutableRunOptions* */ run_options_ptr, double* out, + double* lhs, double* rhs, tensorflow::int64 m, tensorflow::int64 n, + tensorflow::int64 k, tensorflow::int32 transpose_lhs, + tensorflow::int32 transpose_rhs); + +} // extern "C" + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_CPU_RUNTIME_MATMUL_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/cpu/runtime_matmul_mkl.h b/linux/include/tensorflow/compiler/xla/service/cpu/runtime_matmul_mkl.h new file mode 100644 index 000000000..831b796ef --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/cpu/runtime_matmul_mkl.h @@ -0,0 +1,84 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_CPU_RUNTIME_MATMUL_MKL_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_CPU_RUNTIME_MATMUL_MKL_H_ + +#include +#include "tensorflow/core/platform/types.h" +#ifdef INTEL_MKL +#include "third_party/intel_mkl_ml/include/mkl_cblas.h" + +extern void __xla_cpu_runtime_MKLMatMulF32( + const void* /* xla::ExecutableRunOptions* */ run_options_ptr, float* out, + float* lhs, float* rhs, tensorflow::int64 m, tensorflow::int64 n, + tensorflow::int64 k, tensorflow::int32 transpose_lhs, + tensorflow::int32 transpose_rhs); +extern void __xla_cpu_runtime_MKLMatMulF64( + const void* /* xla::ExecutableRunOptions* */ run_options_ptr, double* out, + double* lhs, double* rhs, tensorflow::int64 m, tensorflow::int64 n, + tensorflow::int64 k, tensorflow::int32 transpose_lhs, + tensorflow::int32 transpose_rhs); +extern void __xla_cpu_runtime_MKLSingleThreadedMatMulF32( + const void* /* xla::ExecutableRunOptions* */ run_options_ptr, float* out, + float* lhs, float* rhs, tensorflow::int64 m, tensorflow::int64 n, + tensorflow::int64 k, tensorflow::int32 transpose_lhs, + tensorflow::int32 transpose_rhs); +extern void __xla_cpu_runtime_MKLSingleThreadedMatMulF64( + const void* /* xla::ExecutableRunOptions* */ run_options_ptr, double* out, + double* lhs, double* rhs, tensorflow::int64 m, tensorflow::int64 n, + tensorflow::int64 k, tensorflow::int32 transpose_lhs, + tensorflow::int32 transpose_rhs); + +#else +extern void __xla_cpu_runtime_MKLMatMulF32( + const void* /* xla::ExecutableRunOptions* */ run_options_ptr, float* out, + float* lhs, float* rhs, tensorflow::int64 m, tensorflow::int64 n, + tensorflow::int64 k, tensorflow::int32 transpose_lhs, + tensorflow::int32 transpose_rhs) { + std::cerr << "Attempt to call MKL MatMul runtime library without defining " + "INTEL_MKL. Add --config=mkl to build with MKL."; + exit(1); +} +extern void __xla_cpu_runtime_MKLMatMulF64( + const void* /* xla::ExecutableRunOptions* */ run_options_ptr, double* out, + double* lhs, double* rhs, tensorflow::int64 m, tensorflow::int64 n, + tensorflow::int64 k, tensorflow::int32 transpose_lhs, + tensorflow::int32 transpose_rhs) { + std::cerr << "Attempt to call MKL MatMul runtime library without defining " + "INTEL_MKL. Add --config=mkl to build with MKL."; + exit(1); +} +extern void __xla_cpu_runtime_MKLSingleThreadedMatMulF32( + const void* /* xla::ExecutableRunOptions* */ run_options_ptr, float* out, + float* lhs, float* rhs, tensorflow::int64 m, tensorflow::int64 n, + tensorflow::int64 k, tensorflow::int32 transpose_lhs, + tensorflow::int32 transpose_rhs) { + std::cerr << "Attempt to call MKL MatMul runtime library without defining " + "INTEL_MKL. Add --config=mkl to build with MKL."; + exit(1); +} +extern void __xla_cpu_runtime_MKLSingleThreadedMatMulF64( + const void* /* xla::ExecutableRunOptions* */ run_options_ptr, double* out, + double* lhs, double* rhs, tensorflow::int64 m, tensorflow::int64 n, + tensorflow::int64 k, tensorflow::int32 transpose_lhs, + tensorflow::int32 transpose_rhs) { + std::cerr << "Attempt to call MKL MatMul runtime library without defining " + "INTEL_MKL. Add --config=mkl to build with MKL."; + exit(1); +} + +#endif // INTEL_MKL +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_CPU_RUNTIME_MATMUL_MKL_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/cpu/runtime_matvec.h b/linux/include/tensorflow/compiler/xla/service/cpu/runtime_matvec.h new file mode 100644 index 000000000..70eb98c54 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/cpu/runtime_matvec.h @@ -0,0 +1,121 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_CPU_RUNTIME_MATVEC_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_CPU_RUNTIME_MATVEC_H_ + +#include "third_party/eigen3/Eigen/Core" + +#include "tensorflow/core/platform/types.h" + +namespace xla { + +namespace detail { + +using tensorflow::int32; +using tensorflow::int64; + +// Does mat * x or mat^T * x. +template +void MatVec(T* out_buf, T* mat_buf, T* x_buf, int64 rows, int64 cols, + int32 transpose) { + // Use an Eigen Matrix instead of a Tensor, as the GEMV from Matrix seems to + // be faster (b/30223679). See also: the matmul op kernel in TensorFlow, + // which implements the same optimization. + using Matrix = Eigen::Matrix; + using MatrixMap = Eigen::Map; + + using Vector = Eigen::Matrix; + using VectorMap = Eigen::Map; + + auto x = VectorMap(x_buf, cols); + auto out = VectorMap(out_buf, rows); + + int64 mat_rows = rows; + int64 mat_cols = cols; + + if (transpose) { + std::swap(mat_rows, mat_cols); + } + + auto mat = MatrixMap(mat_buf, mat_rows, mat_cols); + + if (transpose) { + out = mat.transpose() * x; + } else { + out = mat * x; + } +} + +// Converts matmul-style args to matvec. +template +void DispatchMatVec(T* out, T* lhs, T* rhs, int64 m, int64 n, int64 k, + int32 transpose_lhs, int32 transpose_rhs) { + // If the input is in the form x * A, where x is the vector, then bring A back + // over to the left hand side. We make use of the identity + // + // (x * A)^T = A^T * x^T + // + // We do not need to take the transpose of x or of the result since taking + // the transpose of a vector does not change the memory layout. + const int64 cols = k; + + T* mat; + T* vec; + int64 rows; + bool transpose_mat; + + bool is_mat_vec = (n == 1); + + if (is_mat_vec) { + mat = lhs; + vec = rhs; + rows = m; + transpose_mat = transpose_lhs; + } else { + mat = rhs; + vec = lhs; + rows = n; + transpose_mat = !transpose_rhs; + } + + MatVec(out, mat, vec, rows, cols, transpose_mat); +} + +} // namespace detail + +// Performs a matrix-vector multiplication using Eigen. 'lhs' and 'rhs' are +// pointers to buffers containing input matrices in column-major order. 'out' is +// a pointer to a buffer sufficiently large to hold the result of the +// operation. Following standard nomenclature: lhs is m x k, rhs is k x n, and +// out is m x n. +// +// This requires that m = 1 or n = 1. +// +// TODO(b/64684907): Compare runtime performance of these functions with dot +// simplification. +template +void EigenMatVec(T* out, T* lhs, T* rhs, tensorflow::int64 m, + tensorflow::int64 n, tensorflow::int64 k, + tensorflow::int32 transpose_lhs, + tensorflow::int32 transpose_rhs) { + assert((m == 1 || n == 1) && "not a matrix-vector multiply"); + detail::DispatchMatVec(out, lhs, rhs, m, n, k, transpose_lhs, + transpose_rhs); +} + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_CPU_RUNTIME_MATVEC_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/cpu/runtime_single_threaded_conv2d.h b/linux/include/tensorflow/compiler/xla/service/cpu/runtime_single_threaded_conv2d.h new file mode 100644 index 000000000..44b201725 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/cpu/runtime_single_threaded_conv2d.h @@ -0,0 +1,54 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_CPU_RUNTIME_SINGLE_THREADED_CONV2D_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_CPU_RUNTIME_SINGLE_THREADED_CONV2D_H_ + +#include "third_party/eigen3/Eigen/Core" +#include "tensorflow/core/platform/types.h" + +extern "C" { + +extern void __xla_cpu_runtime_EigenSingleThreadedConvF16( + const void* /* xla::ExecutableRunOptions* */ run_options_ptr, + Eigen::half* out, Eigen::half* lhs, Eigen::half* rhs, + tensorflow::int64 input_batch, tensorflow::int64 input_rows, + tensorflow::int64 input_cols, tensorflow::int64 input_channels, + tensorflow::int64 kernel_rows, tensorflow::int64 kernel_cols, + tensorflow::int64 kernel_channels, tensorflow::int64 kernel_filters, + tensorflow::int64 output_rows, tensorflow::int64 output_cols, + tensorflow::int64 row_stride, tensorflow::int64 col_stride, + tensorflow::int64 padding_top, tensorflow::int64 padding_bottom, + tensorflow::int64 padding_left, tensorflow::int64 padding_right, + tensorflow::int64 lhs_row_dilation, tensorflow::int64 lhs_col_dilation, + tensorflow::int64 rhs_row_dilation, tensorflow::int64 rhs_col_dilation); + +extern void __xla_cpu_runtime_EigenSingleThreadedConvF32( + const void* /* xla::ExecutableRunOptions* */ run_options_ptr, float* out, + float* lhs, float* rhs, tensorflow::int64 input_batch, + tensorflow::int64 input_rows, tensorflow::int64 input_cols, + tensorflow::int64 input_channels, tensorflow::int64 kernel_rows, + tensorflow::int64 kernel_cols, tensorflow::int64 kernel_channels, + tensorflow::int64 kernel_filters, tensorflow::int64 output_rows, + tensorflow::int64 output_cols, tensorflow::int64 row_stride, + tensorflow::int64 col_stride, tensorflow::int64 padding_top, + tensorflow::int64 padding_bottom, tensorflow::int64 padding_left, + tensorflow::int64 padding_right, tensorflow::int64 lhs_row_dilation, + tensorflow::int64 lhs_col_dilation, tensorflow::int64 rhs_row_dilation, + tensorflow::int64 rhs_col_dilation); + +} // extern "C" + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_CPU_RUNTIME_SINGLE_THREADED_CONV2D_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/cpu/runtime_single_threaded_fft.h b/linux/include/tensorflow/compiler/xla/service/cpu/runtime_single_threaded_fft.h new file mode 100644 index 000000000..dcd133d01 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/cpu/runtime_single_threaded_fft.h @@ -0,0 +1,31 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_CPU_RUNTIME_SINGLE_THREADED_FFT_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_CPU_RUNTIME_SINGLE_THREADED_FFT_H_ + +#include "tensorflow/core/platform/types.h" + +extern "C" { + +extern void __xla_cpu_runtime_EigenSingleThreadedFft( + const void* /* xla::ExecutableRunOptions* */ run_options_ptr, void* out, + void* operand, tensorflow::int32 fft_type, tensorflow::int32 fft_rank, + tensorflow::int64 input_batch, tensorflow::int64 fft_length0, + tensorflow::int64 fft_length1, tensorflow::int64 fft_length2); + +} // extern "C" + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_CPU_RUNTIME_SINGLE_THREADED_FFT_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/cpu/runtime_single_threaded_matmul.h b/linux/include/tensorflow/compiler/xla/service/cpu/runtime_single_threaded_matmul.h new file mode 100644 index 000000000..82a1fcce5 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/cpu/runtime_single_threaded_matmul.h @@ -0,0 +1,49 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_CPU_RUNTIME_SINGLE_THREADED_MATMUL_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_CPU_RUNTIME_SINGLE_THREADED_MATMUL_H_ + +#include "third_party/eigen3/Eigen/Core" +#include "tensorflow/core/platform/types.h" + +extern "C" { + +// Performs a single-threaded matrix multiplication using Eigen. 'lhs' and 'rhs' +// are pointers to buffers containing input matrices in column-major order. +// 'out' is a pointer to a buffer sufficiently large to hold the result of the +// operation. Following standard nomenclature: lhs is m x k, rhs is k x n, and +// out is m x n. +extern void __xla_cpu_runtime_EigenSingleThreadedMatMulF16( + const void* /* xla::ExecutableRunOptions* */ run_options_ptr, + Eigen::half* out, Eigen::half* lhs, Eigen::half* rhs, tensorflow::int64 m, + tensorflow::int64 n, tensorflow::int64 k, tensorflow::int32 transpose_lhs, + tensorflow::int32 transpose_rhs); + +extern void __xla_cpu_runtime_EigenSingleThreadedMatMulF32( + const void* /* xla::ExecutableRunOptions* */ run_options_ptr, float* out, + float* lhs, float* rhs, tensorflow::int64 m, tensorflow::int64 n, + tensorflow::int64 k, tensorflow::int32 transpose_lhs, + tensorflow::int32 transpose_rhs); + +extern void __xla_cpu_runtime_EigenSingleThreadedMatMulF64( + const void* /* xla::ExecutableRunOptions* */ run_options_ptr, double* out, + double* lhs, double* rhs, tensorflow::int64 m, tensorflow::int64 n, + tensorflow::int64 k, tensorflow::int32 transpose_lhs, + tensorflow::int32 transpose_rhs); + +} // extern "C" + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_CPU_RUNTIME_SINGLE_THREADED_MATMUL_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/cpu/shape_partition.h b/linux/include/tensorflow/compiler/xla/service/cpu/shape_partition.h new file mode 100644 index 000000000..db2cda293 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/cpu/shape_partition.h @@ -0,0 +1,105 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_CPU_SHAPE_PARTITION_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_CPU_SHAPE_PARTITION_H_ + +#include + +#include "tensorflow/compiler/xla/shape_util.h" + +namespace xla { +namespace cpu { + +// ShapePartitionAssigner partitions the most-major dimensions of 'shape' such +// that the total partition count <= 'target_partition_count'. +// +// Example 1: +// +// Let 'shape' = [8, 16, 32] and 'target_partition_count' = 6. +// +// Because the most-major dimension size is <= 'target_partition_count', we +// can generate our target number of partitions by partition the most-major +// dimensions. +// +// This will result in the following partitions of the most-major dimension: +// +// [0, 1), [1, 2), [2, 3), [3, 4), [4, 5) [5, 8) +// +// Note that the last partition has residual because the dimension size is +// not a multiple of the partition count. +// +// +// Example 2: +// +// Let 'shape' = [8, 16, 32] and 'target_partition_count' = 16. +// +// Because the most-major dimension only has size 8, we must also partition +// the next most-major dimension to generate the target of 16 partitions. +// We factor 'target_partition_count' by the number of most-major dimensions +// we need to partition, to get a per-dimension target partition count: +// +// target_dimension_partition_count = 16 ^ (1 / 2) == 4 +// +// This will result in the following partitions of the most-major dimension: +// +// [0, 2), [2, 4), [4, 6), [6, 8) +// +// This will result in the following partitions of the second most-major +// dimension: +// +// [0, 4), [4, 8), [8, 12), [12, 16) +// +class ShapePartitionAssigner { + public: + ShapePartitionAssigner(const Shape& shape) : shape_(shape) {} + + // Returns dimension partition counts (starting at outer-most dimension). + std::vector Run(int64 target_partition_count); + + // Returns the total partition count based on 'dimension_partition_counts'. + static int64 GetTotalPartitionCount( + const std::vector& dimension_partition_counts); + + private: + const Shape& shape_; +}; + +// ShapePartitionIterator iterates through outer-dimension partitions of +// 'shape' as specified by 'dimension_partition_counts'. +class ShapePartitionIterator { + public: + ShapePartitionIterator(const Shape& shape, + const std::vector& dimension_partition_counts); + + // Returns a partition [start, size] for each dimension. + // Partitions are listed starting from outer-most dimension first. + std::vector> GetPartition(int64 index) const; + + int64 GetTotalPartitionCount() const; + + private: + const Shape& shape_; + const std::vector dimension_partition_counts_; + + std::vector dimensions_; + std::vector dimension_partition_sizes_; + std::vector dimension_partition_strides_; +}; + +} // namespace cpu +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_CPU_SHAPE_PARTITION_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/cpu/simple_orc_jit.h b/linux/include/tensorflow/compiler/xla/service/cpu/simple_orc_jit.h new file mode 100644 index 000000000..d74b63fcf --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/cpu/simple_orc_jit.h @@ -0,0 +1,115 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_CPU_SIMPLE_ORC_JIT_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_CPU_SIMPLE_ORC_JIT_H_ + +#include +#include +#include + +#include "llvm/ADT/Triple.h" +#include "llvm/ExecutionEngine/Orc/Core.h" +#include "llvm/ExecutionEngine/Orc/IRCompileLayer.h" +#include "llvm/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.h" +#include "llvm/ExecutionEngine/Orc/SymbolStringPool.h" +#include "llvm/IR/Module.h" +#include "llvm/Target/TargetMachine.h" +#include "tensorflow/compiler/xla/service/cpu/compiler_functor.h" +#include "tensorflow/compiler/xla/service/cpu/disassembler.h" +#include "tensorflow/compiler/xla/types.h" + +namespace xla { +namespace cpu { + +// Simplified LLVM JIT based on the new Orc API. +// +// This class wraps Orc's functionality into a single interface that only +// exposes what we need for XLA. +// +// Supports JIT-ing multiple modules but without cross-module linking. +// Implements eager compilation - the module is lowered to binary as soon as +// it's added to the JIT. +class SimpleOrcJIT { + public: + using ObjLayerT = llvm::orc::RTDyldObjectLinkingLayer; + using CompileFtor = std::function; + using CompileLayerT = llvm::orc::IRCompileLayer; + using VModuleKeyT = llvm::orc::VModuleKey; + + // Create a new JIT, targeting the host architecture. + // The |target_options| parameter allows customization of certain code + // generation properties of the TargetMachine (whether or not float point math + // can be reassociated, etc.). + // The |opt_level| parameter controls the optimization level of the code + // generator. + // The |optimize_for_size| parameter specifies that the code generator should + // optimize to reduce code size, potentially at the cost of performance. + // The |disable_expensive_passes| parameter will disable certain optimization + // passes + // The |pre_optimization_hook| is invoked on the module before any IR + // level optimizations are applied. + // The |post_optimization_hook| is invoked on the module after all IR + // level optimizations are applied. + SimpleOrcJIT(const llvm::TargetOptions& target_options, + llvm::CodeGenOpt::Level opt_level, bool optimize_for_size, + bool enable_fast_math, bool disable_expensive_passes, + LLVMCompiler::ModuleHook pre_optimization_hook, + LLVMCompiler::ModuleHook post_optimization_hook); + + // Data layout this JIT was created with. + const llvm::DataLayout& data_layout() const { return data_layout_; } + + // Target triple (host) this JIT was created with. + const llvm::Triple& target_triple() const { + return target_machine_->getTargetTriple(); + } + + // Add a module to the JIT. Returns an opaque key that can be used to later + // remove this module. + VModuleKeyT AddModule(std::unique_ptr module); + + // Remove a module from the JIT and free the memory associated with it. + void RemoveModule(VModuleKeyT key); + + // Get the runtime address of the compiled symbol whose name is given. Returns + // nullptr if the symbol cannot be found. + llvm::JITSymbol FindCompiledSymbol(const std::string& name); + + llvm::TargetMachine* target_machine() const { return target_machine_.get(); } + + // Creates an llvm::TargetMachine suitable for JITting code that will run on + // the current machine. + static std::unique_ptr InferTargetMachineForJIT( + const llvm::TargetOptions& target_options, + llvm::CodeGenOpt::Level opt_level); + + private: + llvm::JITSymbol ResolveRuntimeSymbol(const std::string& name); + + std::vector module_keys_; + std::unique_ptr target_machine_; + const Disassembler disassembler_; + const llvm::DataLayout data_layout_; + llvm::orc::ExecutionSession execution_session_; + std::shared_ptr symbol_resolver_; + ObjLayerT object_layer_; + CompileLayerT compile_layer_; +}; + +} // namespace cpu +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_CPU_SIMPLE_ORC_JIT_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/cpu/target_machine_features.h b/linux/include/tensorflow/compiler/xla/service/cpu/target_machine_features.h new file mode 100644 index 000000000..a383b4a4a --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/cpu/target_machine_features.h @@ -0,0 +1,108 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_CPU_TARGET_MACHINE_FEATURES_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_CPU_TARGET_MACHINE_FEATURES_H_ + +#include "absl/container/flat_hash_map.h" +#include "llvm/Analysis/TargetTransformInfo.h" +#include "llvm/Target/TargetMachine.h" +#include "tensorflow/compiler/xla/primitive_util.h" + +namespace xla { +namespace cpu { + +// Abstract interface for classes providing information about the target we're +// compiling for. +class TargetMachineFeatures { + public: + static constexpr int kX86AvxVectorByteSize = 32; + + // Input and output tensor buffers must be aligned to this many bytes if we + // want to call an Eigen backed GEMM or Convolution. + static constexpr int kEigenExpectedTensorAlignment = 16; + + // Return the vectorization factor, which is the number of bytes of data + // explicitly vectorized routines will try to process at once. + virtual int vectorization_factor_in_bytes() const = 0; + + // Return the size of the largest vector size in bytes. We need to pass in + // "function" since llvm functions can contain annotations for specializing + // them to specific micro-architectures (though currently XLA does not use + // this functionality). + virtual int vector_register_byte_size( + const llvm::Function& function) const = 0; + + // Return the number of elements of type `type` that can fit into the largest + // vector register available. We need to pass in "function" since llvm + // functions can contain annotations for specializing them to specific + // micro-architectures (though currently XLA does not use this functionality). + virtual int vector_register_num_elements(const llvm::Function& function, + PrimitiveType type) const = 0; + + // Returns the minimum alignment for a buffer of size size_bytes. + virtual int64 minimum_alignment_for_allocation(int64 size_bytes) const = 0; + + virtual ~TargetMachineFeatures() = default; +}; + +// Implements the TargetMachineFeatures interface using an llvm::TargetMachine. +class LLVMTargetMachineFeatures : public TargetMachineFeatures { + public: + static constexpr int kX86AvxVectorByteSize = 32; + + LLVMTargetMachineFeatures(llvm::TargetMachine* target_machine) + : target_machine_(target_machine) {} + + int vectorization_factor_in_bytes() const override { + // Ideally this should be a function of the cache line size (which we can + // get from llvm::TargetTransformInfo::getCacheLineSize) of the target + // machine. Guess a value of 128 bytes for now. + return 128; + } + + int vector_register_byte_size(const llvm::Function& function) const override { + llvm::TargetTransformInfo* tti = GetTargetTransformInfoFor(function); + return tti->getRegisterBitWidth(/*Vector=*/true) / 8; + } + + int vector_register_num_elements(const llvm::Function& function, + PrimitiveType type) const override { + return vector_register_byte_size(function) / + (primitive_util::BitWidth(type) / 8); + } + + int64 minimum_alignment_for_allocation(int64 size_bytes) const override; + + private: + llvm::TargetTransformInfo* GetTargetTransformInfoFor( + const llvm::Function& function) const; + + // This cache saves us from having to create a llvm::TargetTransformInfo for + // every call to GetTargetTransformInfoFor (creating a TargetTransformInfo + // costs one heap allocation on X86). + // + // This is mutated from within `GetTargetTransformInfoFor` which is + // semantically a getter (and thus `const`); and is therefore declared + // mutable. Making this mutable is okay because it has cache semantics. + mutable absl::flat_hash_map + target_transform_info_cache_; + llvm::TargetMachine* target_machine_; +}; + +} // namespace cpu +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_CPU_TARGET_MACHINE_FEATURES_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/cpu/vector_support_library.h b/linux/include/tensorflow/compiler/xla/service/cpu/vector_support_library.h new file mode 100644 index 000000000..5690d2be2 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/cpu/vector_support_library.h @@ -0,0 +1,335 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_CPU_VECTOR_SUPPORT_LIBRARY_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_CPU_VECTOR_SUPPORT_LIBRARY_H_ + +#include + +#include "absl/types/span.h" +#include "llvm/IR/IRBuilder.h" +#include "llvm/IR/Value.h" +#include "tensorflow/compiler/xla/primitive_util.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" + +namespace xla { +namespace cpu { + +// Simple wrappers around llvm::APFloat::APFloat to make the calling code more +// obvious. + +inline llvm::APFloat GetIeeeF32(float f) { return llvm::APFloat(f); } +inline llvm::APFloat GetIeeeF32FromBitwiseRep(int32 bitwise_value) { + return llvm::APFloat(llvm::APFloat::IEEEsingle(), + llvm::APInt(/*numBits=*/32, /*val=*/bitwise_value)); +} + +// A thin wrapper around llvm_util.h to make code generating vector math flow +// more readable. +class VectorSupportLibrary { + public: + // This VectorSupportLibrary instance remembers `primitive_type` and + // `vector_size`, and these are implicitly used by the methods on this + // instance (i.e. LoadVector will load a vector of type <`vector_size` x + // `primitive_type`>). + VectorSupportLibrary(PrimitiveType primitive_type, int64 vector_size, + llvm::IRBuilder<>* b, std::string name); + + llvm::Value* Mul(llvm::Value* lhs, llvm::Value* rhs); + llvm::Value* Mul(int64 lhs, llvm::Value* rhs) { + return Mul(b()->getInt64(lhs), rhs); + } + llvm::Value* Mul(const llvm::APFloat& lhs, llvm::Value* rhs) { + return Mul(GetConstantFloat(rhs->getType(), lhs), rhs); + } + + // If your call resolved to these then you probably wanted the versions taking + // APFloat. + llvm::Value* Mul(double lhs, llvm::Value* rhs) = delete; + llvm::Value* Mul(float lhs, llvm::Value* rhs) = delete; + + llvm::Value* Add(llvm::Value* lhs, llvm::Value* rhs); + llvm::Value* Add(int64 lhs, llvm::Value* rhs) { + return Add(b()->getInt64(lhs), rhs); + } + llvm::Value* Add(const llvm::APFloat& lhs, llvm::Value* rhs) { + return Add(GetConstantFloat(rhs->getType(), lhs), rhs); + } + + // If your call resolved to these then you probably wanted the versions taking + // APFloat. + llvm::Value* Add(double lhs, llvm::Value* rhs) = delete; + llvm::Value* Add(float lhs, llvm::Value* rhs) = delete; + + llvm::Value* Sub(llvm::Value* lhs, llvm::Value* rhs); + llvm::Value* Sub(llvm::Value* lhs, const llvm::APFloat& rhs) { + return Sub(lhs, GetConstantFloat(lhs->getType(), rhs)); + } + llvm::Value* Max(llvm::Value* lhs, llvm::Value* rhs); + llvm::Value* Max(const llvm::APFloat& lhs, llvm::Value* rhs) { + return Max(GetConstantFloat(rhs->getType(), lhs), rhs); + } + llvm::Value* Div(llvm::Value* lhs, llvm::Value* rhs); + + llvm::Value* MulAdd(llvm::Value* a, llvm::Value* b, llvm::Value* c) { + return Add(c, Mul(a, b)); + } + + llvm::Value* MulAdd(llvm::Value* a, llvm::Value* b, const llvm::APFloat& c) { + return Add(GetConstantFloat(vector_type(), c), Mul(a, b)); + } + + llvm::Value* MulAdd(llvm::Value* a, const llvm::APFloat& b, + const llvm::APFloat& c) { + return Add(GetConstantFloat(a->getType(), c), + Mul(a, GetConstantFloat(a->getType(), b))); + } + + llvm::Value* Floor(llvm::Value* a); + + llvm::Value* Clamp(llvm::Value* a, const llvm::APFloat& low, + const llvm::APFloat& high); + llvm::Value* SplatFloat(const llvm::APFloat& d) { + return GetConstantFloat(vector_type(), d); + } + + // These compare instructions return a floating point typed mask instead of an + // i1. For instance, on a vector typed input, lanes where the predicate is + // true get a float with all ones and other lanes get a float with all zeros. + // This is slightly odd from the perspective of LLVM's type system, but it + // makes kernel IR generation code written using VectorSupportLibrary (its + // raison d'etre) less cluttered. + + llvm::Value* FCmpEQMask(llvm::Value* lhs, llvm::Value* rhs); + llvm::Value* FCmpULEMask(llvm::Value* lhs, llvm::Value* rhs); + llvm::Value* FCmpOLTMask(llvm::Value* lhs, llvm::Value* rhs); + llvm::Value* FCmpOLTMask(llvm::Value* lhs, const llvm::APFloat& rhs) { + return FCmpOLTMask(lhs, GetConstantFloat(lhs->getType(), rhs)); + } + + // These boolean operations operate on the bitwise values of the floating + // point inputs. They return a (vector of) float(s) but like in the mask + // generating predicates above this type system oddity makes the kernel IR + // generation code less cluttered. + llvm::Value* FloatAnd(llvm::Value* lhs, llvm::Value* rhs); + llvm::Value* FloatAnd(llvm::Value* lhs, const llvm::APFloat& rhs) { + return FloatAnd(lhs, GetConstantFloat(lhs->getType(), rhs)); + } + llvm::Value* FloatOr(llvm::Value* lhs, llvm::Value* rhs); + llvm::Value* FloatOr(llvm::Value* lhs, const llvm::APFloat& rhs) { + return FloatOr(lhs, GetConstantFloat(lhs->getType(), rhs)); + } + llvm::Value* FloatNot(llvm::Value* lhs); + llvm::Value* FloatAndNot(llvm::Value* lhs, llvm::Value* rhs) { + return FloatAnd(FloatNot(lhs), rhs); + } + + llvm::Value* BroadcastScalar(llvm::Value* x); + llvm::Value* BroadcastScalar(const llvm::APFloat& d) { + return BroadcastScalar(GetConstantFloat(scalar_type(), d)); + } + + llvm::Value* ComputeOffsetPointer(llvm::Value* base_pointer, + llvm::Value* offset_elements); + llvm::Value* ComputeOffsetPointer(llvm::Value* base_pointer, + llvm::Value* offset_elements, int64 scale) { + return ComputeOffsetPointer( + base_pointer, b_->CreateMul(b_->getInt64(scale), offset_elements)); + } + llvm::Value* ComputeOffsetPointer(llvm::Value* base_pointer, + int64 offset_elements) { + return ComputeOffsetPointer(base_pointer, b()->getInt64(offset_elements)); + } + + llvm::Value* LoadVector(llvm::Value* pointer); + + llvm::Value* LoadVector(llvm::Value* base_pointer, + llvm::Value* offset_elements) { + return LoadVector(ComputeOffsetPointer(base_pointer, offset_elements)); + } + + llvm::Value* LoadVector(llvm::Value* base_pointer, int64 offset_elements) { + return LoadVector(base_pointer, b()->getInt64(offset_elements)); + } + + llvm::Value* LoadScalar(llvm::Value* pointer); + + llvm::Value* LoadScalar(llvm::Value* base_pointer, + llvm::Value* offset_elements) { + return LoadScalar(ComputeOffsetPointer(base_pointer, offset_elements)); + } + + llvm::Value* LoadScalar(llvm::Value* base_pointer, int64 offset_elements) { + return LoadScalar(base_pointer, b()->getInt64(offset_elements)); + } + + void StoreVector(llvm::Value* value, llvm::Value* pointer); + + void StoreVector(llvm::Value* value, llvm::Value* base_pointer, + llvm::Value* offset_elements) { + StoreVector(value, ComputeOffsetPointer(base_pointer, offset_elements)); + } + + void StoreVector(llvm::Value* value, llvm::Value* base_pointer, + int64 offset_elements) { + StoreVector(value, base_pointer, b()->getInt64(offset_elements)); + } + + void StoreScalar(llvm::Value* value, llvm::Value* pointer); + void StoreScalar(llvm::Value* value, llvm::Value* base_pointer, + llvm::Value* offset_elements) { + StoreScalar(value, ComputeOffsetPointer(base_pointer, offset_elements)); + } + + void StoreScalar(llvm::Value* value, llvm::Value* base_pointer, + int64 offset_elements) { + StoreScalar(base_pointer, b()->getInt64(offset_elements)); + } + + llvm::Value* LoadBroadcast(llvm::Value* pointer); + llvm::Value* LoadBroadcast(llvm::Value* base_pointer, + llvm::Value* offset_elements) { + return LoadBroadcast(ComputeOffsetPointer(base_pointer, offset_elements)); + } + llvm::Value* LoadBroadcast(llvm::Value* base_pointer, int64 offset_elements) { + return LoadBroadcast(base_pointer, b()->getInt64(offset_elements)); + } + + // Compute the horizontal sum of each vector in `vectors`. The i'th element + // in the result vector is the (scalar) horizontal sum of the i'th vector in + // `vectors`. If `init_values` is not nullptr then the value in the i'th lane + // in `init_values` is added to the i'th horizontal sum. + std::vector ComputeHorizontalSums( + std::vector vectors, llvm::Value* init_values = nullptr); + + llvm::Value* GetZeroVector(); + llvm::Value* GetZeroScalar(); + + llvm::IRBuilder<>* b() const { return b_; } + int64 vector_size() const { return vector_size_; } + llvm::Type* vector_type() const { return vector_type_; } + llvm::Type* vector_pointer_type() const { return vector_pointer_type_; } + llvm::Type* scalar_type() const { return scalar_type_; } + llvm::Type* scalar_pointer_type() const { return scalar_pointer_type_; } + int64 scalar_byte_size() const { + return primitive_util::BitWidth(primitive_type_) / 8; + } + + const std::string& name() const { return name_; } + + private: + llvm::Value* ExtractLowHalf(llvm::Value*); + llvm::Value* ExtractHighHalf(llvm::Value*); + + llvm::Value* MulInternal(llvm::Value* lhs, llvm::Value* rhs); + llvm::Value* AddInternal(llvm::Value* lhs, llvm::Value* rhs); + + llvm::Value* AddReduce(llvm::Value* vector); + + // Checks that each value in `values` is either of type scalar_type() or + // vector_type(). This LOG(FATAL)'s so it should only be called in cases + // where a mismatching type is a programmer bug. + void AssertCorrectTypes(std::initializer_list values); + + // Perform an X86 AVX style horizontal add between `lhs` and `rhs`. The + // resulting IR for an 8-float wide vector is expected to lower to a single + // vhaddps instruction on a CPU that supports vhaddps, and not be too bad in + // other cases. + // + // For a vector width of 8, the result vector is computed as: + // Result[0] = Lhs[0] + Lhs[1] + // Result[1] = Lhs[2] + Lhs[3] + // Result[2] = Rhs[0] + Rhs[1] + // Result[3] = Rhs[2] + Rhs[3] + // Result[4] = Lhs[4] + Lhs[5] + // Result[5] = Lhs[6] + Lhs[7] + // Result[6] = Rhs[4] + Rhs[5] + // Result[7] = Rhs[6] + Rhs[7] + llvm::Value* AvxStyleHorizontalAdd(llvm::Value* lhs, llvm::Value* rhs); + + std::vector ComputeAvxOptimizedHorizontalSums( + std::vector vectors, llvm::Value* init_values); + + llvm::Type* IntegerTypeForFloatSize(bool vector); + llvm::Value* I1ToFloat(llvm::Value* i1); + llvm::Value* GetConstantFloat(llvm::Type* type, const llvm::APFloat& f) { + llvm::Constant* scalar_value = llvm::ConstantFP::get(type->getContext(), f); + if (llvm::isa(type)) { + return llvm::ConstantVector::getSplat(vector_size(), scalar_value); + } + return scalar_value; + } + + int64 vector_size_; + PrimitiveType primitive_type_; + llvm::IRBuilder<>* b_; + llvm::Type* vector_type_; + llvm::Type* vector_pointer_type_; + llvm::Type* scalar_type_; + llvm::Type* scalar_pointer_type_; + std::string name_; +}; + +// This wraps an alloca-backed stack variable which LLVM's SSA construction pass +// can later convert to a SSA value. +class LlvmVariable { + public: + LlvmVariable(llvm::Type*, llvm::IRBuilder<>* b); + + llvm::Value* Get() const; + void Set(llvm::Value* new_value); + + private: + llvm::AllocaInst* alloca_; + llvm::IRBuilder<>* b_; +}; + +class VectorVariable : public LlvmVariable { + public: + VectorVariable(VectorSupportLibrary* vector_support, + llvm::Value* initial_value) + : LlvmVariable(vector_support->vector_type(), vector_support->b()) { + Set(initial_value); + } +}; + +class ScalarVariable : public LlvmVariable { + public: + ScalarVariable(VectorSupportLibrary* vector_support, + llvm::Value* initial_value) + : LlvmVariable(vector_support->scalar_type(), vector_support->b()) { + Set(initial_value); + } +}; + +// This wraps a set of alloca-backed stack variables that can, as a whole, store +// a tile. A "tile" is a sequence of vectors that is typically used as a 2D +// grid of scalar values (e.g. for tiled GEMMs). +class TileVariable { + public: + TileVariable(VectorSupportLibrary* vector_support, + std::vector initial_value); + + std::vector Get() const; + void Set(absl::Span value); + + private: + std::vector storage_; +}; +} // namespace cpu +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_CPU_VECTOR_SUPPORT_LIBRARY_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/cpu/windows_compatibility.h b/linux/include/tensorflow/compiler/xla/service/cpu/windows_compatibility.h new file mode 100644 index 000000000..262f379d8 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/cpu/windows_compatibility.h @@ -0,0 +1,31 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_CPU_WINDOWS_COMPATIBILITY_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_CPU_WINDOWS_COMPATIBILITY_H_ + +#ifdef _MSC_VER + +extern "C" { + +// MSVC does not have sincos[f]. +void sincos(double x, double *sinv, double *cosv); +void sincosf(float x, float *sinv, float *cosv); + +} + +#endif // _MSC_VER + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_CPU_WINDOWS_COMPATIBILITY_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/cpu/xfeed_manager.h b/linux/include/tensorflow/compiler/xla/service/cpu/xfeed_manager.h new file mode 100644 index 000000000..990ff94ba --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/cpu/xfeed_manager.h @@ -0,0 +1,124 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// This header declares the abstract class for the infeed manager that +// is used by the CPU runtime to transfer buffers into an executing +// CPU computation, e.g., to feed data into a while loop. + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_CPU_XFEED_MANAGER_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_CPU_XFEED_MANAGER_H_ + +#include + +#include "absl/types/span.h" +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/platform/mutex.h" + +namespace xla { +namespace cpu { +namespace runtime { + +// Abstract class defining an infeed buffer that is passed to the +// runtime by a client. The client manages the storage of the buffer. +class XfeedBuffer { + public: + virtual ~XfeedBuffer() = default; + + virtual int32 length() = 0; + virtual void* data() = 0; + + // The 'shape' parameter reflects what shape the embedded program was + // expecting / producing with respect to this XfeedBuffer. E.g. this will + // contain information about the layout of an outfed buffer. + virtual void Done(StatusOr shape) = 0; +}; + +// Reusable component for managing the infeed and outfeed queue state. +class XfeedQueueManager { + public: + XfeedQueueManager(string queue_name) : queue_name_(queue_name) {} + + // Calls the completion callback for any enqueued buffers that have + // not been dequeued by the runtime, and empties the + // queue. Reset may not be called while a runtime computation is + // processing a dequeued buffer. The only safe way to ensure this + // condition is to call Reset when no computation is taking place. + void Reset(); + + // Adds a sequence of buffers to the queue atomically. buffer->Done will be + // called when the buffer will no longer be accessed by the XfeedManager, + // either as a result of a call to Reset or because the runtime has dequeued + // and used the buffer. + void EnqueueBuffersAtomically(absl::Span buffers); + + // Blocks until the queue is non-empty, then returns the buffer at the head of + // the queue. Sets the current buffer to be the returned buffer. It is an + // error to call BlockingDequeueBuffer if there is an unreleased current + // buffer, i.e., ReleaseCurrentBuffer must be called between calls to + // BlockingDequeueBuffer. + XfeedBuffer* BlockingDequeueBuffer(); + + // Releases the current buffer, which is the last buffer returned by + // BlockingDequeuBuffer and not yet released. length and data must + // match the buffer->length() and buffer->data() for the current + // buffer. + // + // 'shape' communicates the shape of the buffer being released. If the program + // passed a value that could not be decoded as a shape, 'shape' will be an + // error status. In the case of outfeed, this indicates the layout of the + // shape that has been outfed. In the case of infeed, this can be used for + // sanity checking purposes. + void ReleaseCurrentBuffer(int32 length, void* data, StatusOr shape); + + private: + const string queue_name_; + + tensorflow::mutex mu_; + + // Condition variable that is signaled every time a buffer is + // enqueued to an empty queue. + tensorflow::condition_variable cv_; + + // XfeedBuffer* queue contents are not owned, but buffer->Done must + // be called when the buffer is no longer needed by the runtime. + std::deque enqueued_buffers_; + + // If non-NULL, the buffer that is currently being processed by the + // runtime. Not owned. + XfeedBuffer* current_buffer_ = nullptr; +}; + +// Client-side class used to enqueue infeed buffers. +class XfeedManager { + public: + XfeedManager() = default; + + void Reset(); + + XfeedQueueManager* infeed() { return &infeed_; } + XfeedQueueManager* outfeed() { return &outfeed_; } + + private: + XfeedQueueManager infeed_ = {"infeed"}; + XfeedQueueManager outfeed_ = {"outfeed"}; +}; + +} // namespace runtime +} // namespace cpu +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_CPU_XFEED_MANAGER_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/device_memory_allocator.h b/linux/include/tensorflow/compiler/xla/service/device_memory_allocator.h new file mode 100644 index 000000000..a2308ee7a --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/device_memory_allocator.h @@ -0,0 +1,106 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_DEVICE_MEMORY_ALLOCATOR_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_DEVICE_MEMORY_ALLOCATOR_H_ + +#include + +#include "absl/types/span.h" +#include "tensorflow/compiler/xla/service/owning_device_memory.h" +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/core/platform/stream_executor_no_cuda.h" +#include "tensorflow/core/platform/types.h" + +namespace xla { + +// Interface for device memory allocators used within the XLA service. An +// allocator is responsible for allocating memory on all devices of a particular +// platform. +class DeviceMemoryAllocator { + public: + // Parameter platform indicates which platform the allocator allocates memory + // on. Must be non-null. + explicit DeviceMemoryAllocator(const se::Platform* platform) + : platform_(platform) {} + virtual ~DeviceMemoryAllocator() {} + + // Allocates memory on the device. + // + // If size > 0 and the returned StatusOr is OK, the wrapped OwningDeviceMemory + // must not be null. If size == 0, must return a null OwningDeviceMemory. + // + // 'retry_on_failure': If false, and the first attempt to allocate the memory + // fails, the allocation should return immediately without retrying. An + // example use case is optional scratch spaces where a failure has only + // performance impact. + virtual StatusOr Allocate(int device_ordinal, uint64 size, + bool retry_on_failure) = 0; + + // Two-arg version of Allocate(), which sets retry-on-failure to true. + // + // (We don't simply use a default argument on the virtual Allocate function + // because default args on virtual functions are disallowed by the Google + // style guide.) + StatusOr Allocate(int device_ordinal, uint64 size) { + return Allocate(device_ordinal, size, /*retry_on_failure=*/true); + } + + // Must be a nop for null pointers. + virtual Status Deallocate(int device_ordinal, se::DeviceMemoryBase mem) = 0; + + // Return the platform that the allocator allocates memory on. + const se::Platform* platform() const { return platform_; } + + // Can we call Deallocate() as soon as a computation has been scheduled on + // a stream, or do we have to wait for the computation to complete first? + virtual bool AllowsAsynchronousDeallocation() const = 0; + + protected: + friend class OwningDeviceMemory; + const se::Platform* platform_; +}; + +// Default memory allocator for a platform which uses +// StreamExecutor::Allocate/Deallocate. +class StreamExecutorMemoryAllocator : public DeviceMemoryAllocator { + public: + StreamExecutorMemoryAllocator( + const se::Platform* platform, + absl::Span stream_executors); + + StatusOr Allocate(int device_ordinal, uint64 size, + bool retry_on_failure) override; + + // Pull in two-arg overload that sets retry_on_failure to true. + using DeviceMemoryAllocator::Allocate; + + Status Deallocate(int device_ordinal, se::DeviceMemoryBase mem) override; + + bool AllowsAsynchronousDeallocation() const override; + + private: + StatusOr GetStreamExecutor(int device_ordinal); + + // A vector indexed by device ordinal of StreamExecutors for each device of + // the allocator's platform type. If an element is nullptr, then the device + // with the respective device ordinal is not supported by XLA. + std::vector stream_executors_; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_DEVICE_MEMORY_ALLOCATOR_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/dfs_hlo_visitor.h b/linux/include/tensorflow/compiler/xla/service/dfs_hlo_visitor.h new file mode 100644 index 000000000..68d01d75a --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/dfs_hlo_visitor.h @@ -0,0 +1,378 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_DFS_HLO_VISITOR_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_DFS_HLO_VISITOR_H_ + +#include +#include + +#include "absl/strings/string_view.h" +#include "absl/types/span.h" +#include "tensorflow/compiler/xla/literal.h" +#include "tensorflow/compiler/xla/service/hlo_opcode.h" +#include "tensorflow/compiler/xla/status.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/types.h" + +namespace xla { + +class HloComputation; +class HloInstruction; + +// A postorder depth-first HloInstruction visitor. When Handle* is called on an +// instruction, all its operands were already visited. User code can subclass +// this to iterate over an HloInstruction DAG. The Handle* routines have +// operands / data unpacked for ease of use in the visitor subclass. +// +// No instruction will ever be visited twice; however, the root instruction will +// be reported again when the traversal is done via a call to FinishVisit. +// +// A subclass must override at least +// (either HandleElementwiseUnary or all the Handle methods for unary ops) and +// (either HandleElementwiseBinary or all the Handle methods for binary ops)). +// The default Handle methods for (unary, binary) ops call +// (HandleElementwiseUnary, HandleElementwiseBinary). +// The default (HandleElementwiseUnary, HandleElementwiseBinary) return an +// "unimplemented" error status. +// +// Note: this may change to an iterator in the future for flexibility purposes. +// +// Users should not use this class directly, but use the type-aliases +// DfsHloVisitor/ConstDfsHloVisitor instead. +template +class DfsHloVisitorBase { + static_assert( + std::is_same::value || + std::is_same::value, + "Template argument expected to be HloInstruction* or const " + "HloInstruction*"); + + public: + DfsHloVisitorBase() {} + virtual ~DfsHloVisitorBase() {} + + // These routines are self-descriptive, see class comment for usage + // information. + + virtual Status HandleElementwiseUnary(HloInstructionPtr hlo); + virtual Status HandleElementwiseBinary(HloInstructionPtr hlo); + + virtual Status HandleClamp(HloInstructionPtr hlo) = 0; + virtual Status HandleSelect(HloInstructionPtr hlo) = 0; + virtual Status HandleTupleSelect(HloInstructionPtr hlo) = 0; + virtual Status HandleMaximum(HloInstructionPtr hlo) { + return HandleElementwiseBinary(hlo); + } + virtual Status HandleMinimum(HloInstructionPtr hlo) { + return HandleElementwiseBinary(hlo); + } + virtual Status HandleConcatenate(HloInstructionPtr hlo) = 0; + virtual Status HandleConvert(HloInstructionPtr hlo) { + return HandleElementwiseUnary(hlo); + } + virtual Status HandleBitcastConvert(HloInstructionPtr hlo) { + return HandleElementwiseUnary(hlo); + } + virtual Status HandleCopy(HloInstructionPtr hlo) { + return HandleElementwiseUnary(hlo); + } + virtual Status HandleComplex(HloInstructionPtr hlo) { + return HandleElementwiseBinary(hlo); + } + virtual Status HandleMultiply(HloInstructionPtr hlo) { + return HandleElementwiseBinary(hlo); + } + virtual Status HandleDot(HloInstructionPtr hlo) = 0; + virtual Status HandlePower(HloInstructionPtr hlo) { + return HandleElementwiseBinary(hlo); + } + virtual Status HandleConvolution(HloInstructionPtr hlo) = 0; + virtual Status HandleFft(HloInstructionPtr fft) = 0; + virtual Status HandleCrossReplicaSum(HloInstructionPtr hlo) = 0; + virtual Status HandleAllToAll(HloInstructionPtr hlo) = 0; + virtual Status HandleCollectivePermute(HloInstructionPtr hlo) = 0; + virtual Status HandleCompare(HloInstructionPtr hlo) { + return HandleElementwiseBinary(hlo); + } + virtual Status HandleAdd(HloInstructionPtr hlo) { + return HandleElementwiseBinary(hlo); + } + virtual Status HandleDivide(HloInstructionPtr hlo) { + return HandleElementwiseBinary(hlo); + } + virtual Status HandleRemainder(HloInstructionPtr hlo) { + return HandleElementwiseBinary(hlo); + } + virtual Status HandleSubtract(HloInstructionPtr hlo) { + return HandleElementwiseBinary(hlo); + } + virtual Status HandleAbs(HloInstructionPtr hlo) { + return HandleElementwiseUnary(hlo); + } + virtual Status HandleAtan2(HloInstructionPtr hlo) { + return HandleElementwiseBinary(hlo); + } + virtual Status HandleRound(HloInstructionPtr hlo) { + return HandleElementwiseUnary(hlo); + } + virtual Status HandleSign(HloInstructionPtr hlo) { + return HandleElementwiseUnary(hlo); + } + virtual Status HandleNegate(HloInstructionPtr hlo) { + return HandleElementwiseUnary(hlo); + } + virtual Status HandleExp(HloInstructionPtr hlo) { + return HandleElementwiseUnary(hlo); + } + virtual Status HandleExpm1(HloInstructionPtr hlo) { + return HandleElementwiseUnary(hlo); + } + virtual Status HandleFloor(HloInstructionPtr hlo) { + return HandleElementwiseUnary(hlo); + } + virtual Status HandleCeil(HloInstructionPtr hlo) { + return HandleElementwiseUnary(hlo); + } + virtual Status HandleLog(HloInstructionPtr hlo) { + return HandleElementwiseUnary(hlo); + } + virtual Status HandleClz(HloInstructionPtr hlo) { + return HandleElementwiseUnary(hlo); + } + virtual Status HandleLog1p(HloInstructionPtr hlo) { + return HandleElementwiseUnary(hlo); + } + virtual Status HandleCos(HloInstructionPtr hlo) { + return HandleElementwiseUnary(hlo); + } + virtual Status HandleSin(HloInstructionPtr hlo) { + return HandleElementwiseUnary(hlo); + } + virtual Status HandleTanh(HloInstructionPtr hlo) { + return HandleElementwiseUnary(hlo); + } + virtual Status HandleReal(HloInstructionPtr hlo) { + return HandleElementwiseUnary(hlo); + } + virtual Status HandleImag(HloInstructionPtr hlo) { + return HandleElementwiseUnary(hlo); + } + virtual Status HandleIsFinite(HloInstructionPtr hlo) { + return HandleElementwiseUnary(hlo); + } + virtual Status HandleAnd(HloInstructionPtr hlo) { + return HandleElementwiseBinary(hlo); + } + virtual Status HandleNot(HloInstructionPtr hlo) { + return HandleElementwiseUnary(hlo); + } + virtual Status HandleOr(HloInstructionPtr hlo) { + return HandleElementwiseBinary(hlo); + } + virtual Status HandleXor(HloInstructionPtr hlo) { + return HandleElementwiseBinary(hlo); + } + virtual Status HandleShiftLeft(HloInstructionPtr hlo) { + return HandleElementwiseBinary(hlo); + } + virtual Status HandleShiftRightArithmetic(HloInstructionPtr hlo) { + return HandleElementwiseBinary(hlo); + } + virtual Status HandleShiftRightLogical(HloInstructionPtr hlo) { + return HandleElementwiseBinary(hlo); + } + + virtual Status HandleReducePrecision(HloInstructionPtr hlo) { + return HandleElementwiseUnary(hlo); + } + + virtual Status HandleDomain(HloInstructionPtr hlo) { + return HandleElementwiseUnary(hlo); + } + + virtual Status HandleInfeed(HloInstructionPtr hlo) = 0; + virtual Status HandleOutfeed(HloInstructionPtr hlo) = 0; + virtual Status HandleRng(HloInstructionPtr hlo) = 0; + virtual Status HandleReverse(HloInstructionPtr hlo) = 0; + virtual Status HandleSort(HloInstructionPtr hlo) = 0; + virtual Status HandleConstant(HloInstructionPtr hlo) = 0; + virtual Status HandleIota(HloInstructionPtr hlo) = 0; + virtual Status HandleGetTupleElement(HloInstructionPtr hlo) = 0; + virtual Status HandleReduce(HloInstructionPtr hlo) = 0; + virtual Status HandleBitcast(HloInstructionPtr hlo) = 0; + virtual Status HandleBroadcast(HloInstructionPtr hlo) = 0; + virtual Status HandleReshape(HloInstructionPtr hlo) = 0; + virtual Status HandleTranspose(HloInstructionPtr hlo) = 0; + virtual Status HandleParameter(HloInstructionPtr hlo) = 0; + virtual Status HandleFusion(HloInstructionPtr hlo) = 0; + virtual Status HandleCall(HloInstructionPtr hlo) = 0; + virtual Status HandleCustomCall(HloInstructionPtr hlo) = 0; + virtual Status HandleSlice(HloInstructionPtr hlo) = 0; + virtual Status HandleDynamicSlice(HloInstructionPtr hlo) = 0; + virtual Status HandleDynamicUpdateSlice(HloInstructionPtr hlo) = 0; + virtual Status HandleTuple(HloInstructionPtr hlo) = 0; + virtual Status HandleMap(HloInstructionPtr hlo) = 0; + virtual Status HandleReduceWindow(HloInstructionPtr hlo) = 0; + virtual Status HandleSelectAndScatter(HloInstructionPtr hlo) = 0; + virtual Status HandleWhile(HloInstructionPtr hlo) = 0; + virtual Status HandleConditional(HloInstructionPtr hlo) = 0; + virtual Status HandleGather(HloInstructionPtr hlo) = 0; + virtual Status HandleScatter(HloInstructionPtr hlo) = 0; + + virtual Status HandlePad(HloInstructionPtr hlo) = 0; + + virtual Status HandleSend(HloInstructionPtr send) = 0; + virtual Status HandleSendDone(HloInstructionPtr send_done) = 0; + + virtual Status HandleRecv(HloInstructionPtr recv) = 0; + virtual Status HandleRecvDone(HloInstructionPtr recv_done) = 0; + + virtual Status HandleBatchNormTraining(HloInstructionPtr hlo) = 0; + + virtual Status HandleBatchNormInference(HloInstructionPtr hlo) = 0; + + virtual Status HandleBatchNormGrad(HloInstructionPtr hlo) = 0; + + virtual Status HandleAfterAll(HloInstructionPtr token) = 0; + + // Invoked to inform the visitor that the traversal has completed, and that + // the root was "root". + virtual Status FinishVisit(HloInstructionPtr root) = 0; + + // 3 possible visitation states of HLO instructions. Each instruction's + // state only flows one way: kNotVisited -> kVisiting -> kVisited. + enum VisitState { + kNotVisited = 0, + kVisiting = 1, + kVisited = 2, + }; + + VisitState GetVisitState(int id) { return visit_state_.GetState(id); } + VisitState GetVisitState(const HloInstruction& instruction); + + // Resize internal state if necessary to hold state for ids <= num. + // This call is purely a performance hint and can be omitted without + // affecting correctness. + void ReserveVisitStates(int num) { visit_state_.Reserve(num); } + + // Useful when we want to visit the same computation more than once with the + // same visitor. + void ResetVisitStates() { visit_state_.Reset(); } + + void SetVisitState(int id, VisitState state) { + visit_state_.SetState(id, state); + } + + // Sets the visitation state of the given instruction as kVisiting. + // + // Precondition: current state must be kNotVisited. + void SetVisiting(const HloInstruction& instruction); + + // Sets the visitation state of the given instruction as kVisited. + // + // Precondition: current state must be either kNotVisited or kVisiting. + void SetVisited(const HloInstruction& instruction); + + // Returns whether the state of the given instruction is kVisiting. + bool IsVisiting(const HloInstruction& instruction) { + return GetVisitState(instruction) == kVisiting; + } + + // Returns whether the state of the given instruction is kVisited. + bool DidVisit(const HloInstruction& instruction) { + return GetVisitState(instruction) == kVisited; + } + + // Returns whether the state of the given instruction is kNotVisited. + bool NotVisited(const HloInstruction& instruction) { + return GetVisitState(instruction) == kNotVisited; + } + + // This method should be overridden by subclasses that wish to run some + // operation on an op before its Handle* visitor method is called. + // + // For any HLO op, the order of calls is: + // + // Preprocess(op); + // Handle/OpType/(op); + // Postprocess(op); + // + // Overriding methods should call DfsHloVisitor::Preprocess before doing their + // own preprocessing. + virtual Status Preprocess(HloInstructionPtr hlo); + + // This method should be overridden by subclasses that wish to run some + // operation on an op after its Handle* visitor method is called. See + // Preprocess for more details. + // + // Overriding methods should call DfsHloVisitor::Postprocess after doing their + // own postprocessing. + virtual Status Postprocess(HloInstructionPtr hlo); + + private: + class DFSVisitStates { + public: + DFSVisitStates() {} + void Reserve(uint64 num) { + states_.reserve((num + kStatesPerWord - 1) / kStatesPerWord); + } + VisitState GetState(uint64 id) { + uint64 word_index = id / kStatesPerWord; + if (word_index >= states_.size()) { + return VisitState::kNotVisited; + } + static_assert(static_cast(VisitState::kVisited) < 3, + "VisitState must fit in two bits"); + uint64 w = states_[word_index]; + uint32 shift = 2 * (id % kStatesPerWord); // 2 bits per state + return static_cast((w >> shift) & 0x3); + } + void SetState(uint64 id, VisitState state) { + uint64 word_index = id / kStatesPerWord; + if (word_index >= states_.size()) { + states_.resize(word_index + 1, 0); + } + uint64* w = &states_[word_index]; + uint32 shift = 2 * (id % kStatesPerWord); // 2 bits per state + uint64 mask = 0x3ull << shift; + *w = (*w & ~mask) | (static_cast(state) << shift); + DCHECK_EQ(GetState(id), state); + } + void Reset() { states_.clear(); } + + private: + static const uint32 kStatesPerWord = sizeof(uint64) / 2 /*bits per entry*/; + // Map from id to two-bit states. We store 32 such states per 64-bit + // value + std::vector states_; + }; + + DFSVisitStates visit_state_; + + TF_DISALLOW_COPY_AND_ASSIGN(DfsHloVisitorBase); +}; + +// Users should use one of these two type aliases, which are the only two valid +// instantiations of DfsHloVisitorBase. +using DfsHloVisitor = DfsHloVisitorBase; +using ConstDfsHloVisitor = DfsHloVisitorBase; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_DFS_HLO_VISITOR_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/dfs_hlo_visitor_with_default.h b/linux/include/tensorflow/compiler/xla/service/dfs_hlo_visitor_with_default.h new file mode 100644 index 000000000..4cd10ab06 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/dfs_hlo_visitor_with_default.h @@ -0,0 +1,251 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_DFS_HLO_VISITOR_WITH_DEFAULT_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_DFS_HLO_VISITOR_WITH_DEFAULT_H_ + +#include "absl/strings/string_view.h" +#include "absl/types/span.h" +#include "tensorflow/compiler/xla/literal.h" +#include "tensorflow/compiler/xla/service/dfs_hlo_visitor.h" +#include "tensorflow/compiler/xla/service/hlo_opcode.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/types.h" + +namespace xla { + +class HloComputation; +class HloInstruction; + +// DfsHloVisitor with default action based on the HloInstruction being visited. +// Users should not use this class directly, but use the type aliases +// DfsHloVisitorWithDefault/ConstDfsHloVisitorWithDefault instead. +// +// Do *not* add an override to this class if the opcode is covered by +// HandleElementwiseUnary/Binary. These opcode handlers dispatch to +// HandleElementwiseUnary/Binary in DfsHloVisitorBase. Adding such a handler +// here will break passes which rely on the HandleElementwiseUnary/Binary +// handling these opcodes. +template +class DfsHloVisitorWithDefaultBase + : public DfsHloVisitorBase { + public: + DfsHloVisitorWithDefaultBase() {} + ~DfsHloVisitorWithDefaultBase() override {} + + // Default action performed on HloInstruction. + virtual Status DefaultAction(HloInstructionPtr hlo_instruction) = 0; + + Status HandleElementwiseUnary(HloInstructionPtr hlo) override { + return DefaultAction(hlo); + } + Status HandleElementwiseBinary(HloInstructionPtr hlo) override { + return DefaultAction(hlo); + } + + Status HandleBatchNormTraining(HloInstructionPtr hlo) override { + return DefaultAction(hlo); + } + + Status HandleBatchNormInference(HloInstructionPtr hlo) override { + return DefaultAction(hlo); + } + + Status HandleBatchNormGrad(HloInstructionPtr hlo) override { + return DefaultAction(hlo); + } + + Status HandleClamp(HloInstructionPtr clamp) override { + return DefaultAction(clamp); + } + Status HandleConcatenate(HloInstructionPtr concatenate) override { + return DefaultAction(concatenate); + } + Status HandleSelect(HloInstructionPtr select) override { + return DefaultAction(select); + } + Status HandleTupleSelect(HloInstructionPtr tuple_select) override { + return DefaultAction(tuple_select); + } + Status HandleDot(HloInstructionPtr dot) override { + return DefaultAction(dot); + } + Status HandleConvolution(HloInstructionPtr convolution) override { + return DefaultAction(convolution); + } + Status HandleFft(HloInstructionPtr fft) override { + return DefaultAction(fft); + } + Status HandleCrossReplicaSum(HloInstructionPtr crs) override { + return DefaultAction(crs); + } + Status HandleAllToAll(HloInstructionPtr hlo) override { + return DefaultAction(hlo); + } + Status HandleCollectivePermute(HloInstructionPtr hlo) override { + return DefaultAction(hlo); + } + Status HandleRng(HloInstructionPtr random) override { + return DefaultAction(random); + } + Status HandleInfeed(HloInstructionPtr infeed) override { + return DefaultAction(infeed); + } + Status HandleOutfeed(HloInstructionPtr outfeed) override { + return DefaultAction(outfeed); + } + Status HandleReverse(HloInstructionPtr reverse) override { + return DefaultAction(reverse); + } + Status HandleSort(HloInstructionPtr sort) override { + return DefaultAction(sort); + } + Status HandleConstant(HloInstructionPtr constant) override { + return DefaultAction(constant); + } + Status HandleIota(HloInstructionPtr iota) override { + return DefaultAction(iota); + } + Status HandleGetTupleElement(HloInstructionPtr get_tuple_element) override { + return DefaultAction(get_tuple_element); + } + Status HandleParameter(HloInstructionPtr parameter) override { + return DefaultAction(parameter); + } + Status HandleFusion(HloInstructionPtr fusion) override { + return DefaultAction(fusion); + } + Status HandleCall(HloInstructionPtr call) override { + return DefaultAction(call); + } + Status HandleCustomCall(HloInstructionPtr custom_call) override { + return DefaultAction(custom_call); + } + Status HandleSlice(HloInstructionPtr slice) override { + return DefaultAction(slice); + } + Status HandleDynamicSlice(HloInstructionPtr dynamic_slice) override { + return DefaultAction(dynamic_slice); + } + Status HandleDynamicUpdateSlice( + HloInstructionPtr dynamic_update_slice) override { + return DefaultAction(dynamic_update_slice); + } + Status HandleTuple(HloInstructionPtr tuple) override { + return DefaultAction(tuple); + } + Status HandleMap(HloInstructionPtr map) override { + return DefaultAction(map); + } + Status HandleReduce(HloInstructionPtr reduce) override { + return DefaultAction(reduce); + } + Status HandleReduceWindow(HloInstructionPtr reduce_window) override { + return DefaultAction(reduce_window); + } + Status HandleSelectAndScatter(HloInstructionPtr select_and_scatter) override { + return DefaultAction(select_and_scatter); + } + Status HandleBitcast(HloInstructionPtr bitcast) override { + return DefaultAction(bitcast); + } + Status HandleBroadcast(HloInstructionPtr broadcast) override { + return DefaultAction(broadcast); + } + Status HandlePad(HloInstructionPtr pad) override { + return DefaultAction(pad); + } + Status HandleReshape(HloInstructionPtr reshape) override { + return DefaultAction(reshape); + } + Status HandleTranspose(HloInstructionPtr transpose) override { + return DefaultAction(transpose); + } + Status HandleWhile(HloInstructionPtr xla_while) override { + return DefaultAction(xla_while); + } + Status HandleConditional(HloInstructionPtr conditional) override { + return DefaultAction(conditional); + } + Status HandleRecv(HloInstructionPtr recv) override { + return DefaultAction(recv); + } + Status HandleRecvDone(HloInstructionPtr recv_done) override { + return DefaultAction(recv_done); + } + Status HandleSend(HloInstructionPtr send) override { + return DefaultAction(send); + } + Status HandleSendDone(HloInstructionPtr send_done) override { + return DefaultAction(send_done); + } + Status HandleGather(HloInstructionPtr gather) override { + return DefaultAction(gather); + } + Status HandleScatter(HloInstructionPtr scatter) override { + return DefaultAction(scatter); + } + Status HandleAfterAll(HloInstructionPtr token) override { + return DefaultAction(token); + } + + // Invoked to inform the visitor that the traversal has completed, and that + // the root was "root". + Status FinishVisit(HloInstructionPtr /*root*/) override { + return Status::OK(); + } + + private: + TF_DISALLOW_COPY_AND_ASSIGN(DfsHloVisitorWithDefaultBase); +}; + +// Users should use these type aliases which are only two valid instantiations. +using DfsHloVisitorWithDefault = DfsHloVisitorWithDefaultBase; +using ConstDfsHloVisitorWithDefault = + DfsHloVisitorWithDefaultBase; + +// (Const)FunctionVisitor lets you transform an +// std::function into a (Const)DfsHloVisitor. +// +// This is useful if you have code that needs to handle visitors in the form of +// both std::function and DfsHloVisitor. You can wrap the function in a +// FunctionVisitor and then treat it like any other DfsHloVisitor. +template +class FunctionVisitorBase + : public DfsHloVisitorWithDefaultBase { + public: + explicit FunctionVisitorBase( + std::function visitor_func) + : visitor_func_(std::move(visitor_func)) {} + + Status DefaultAction(HloInstructionPtr hlo_instruction) override { + return visitor_func_(hlo_instruction); + } + + private: + TF_DISALLOW_COPY_AND_ASSIGN(FunctionVisitorBase); + + std::function visitor_func_; +}; + +using FunctionVisitor = FunctionVisitorBase; +using ConstFunctionVisitor = FunctionVisitorBase; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_DFS_HLO_VISITOR_WITH_DEFAULT_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/dot_decomposer.h b/linux/include/tensorflow/compiler/xla/service/dot_decomposer.h new file mode 100644 index 000000000..40e7a3b4c --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/dot_decomposer.h @@ -0,0 +1,44 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_DOT_DECOMPOSER_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_DOT_DECOMPOSER_H_ + +#include "tensorflow/compiler/xla/service/hlo_module.h" +#include "tensorflow/compiler/xla/service/hlo_pass_interface.h" + +namespace xla { + +// DotDecomposer is a pass which decomposes batch Dot operations into a +// sequence of smaller (R2) Dot operations. +class DotDecomposer : public HloModulePass { + public: + // Decomposes batch Dot operations when 'decompose_batch_dot' is true. + DotDecomposer(bool decompose_batch_dot = true) + : decompose_batch_dot_(decompose_batch_dot) {} + ~DotDecomposer() = default; + absl::string_view name() const override { return "dot_decomposer"; } + + // Run DotDecomposer pass on computations in 'module'. + // Returns whether the 'module' was changed. + StatusOr Run(HloModule* module) override; + + private: + bool decompose_batch_dot_; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_DOT_DECOMPOSER_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/elemental_ir_emitter.h b/linux/include/tensorflow/compiler/xla/service/elemental_ir_emitter.h new file mode 100644 index 000000000..d3e2acaab --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/elemental_ir_emitter.h @@ -0,0 +1,229 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_ELEMENTAL_IR_EMITTER_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_ELEMENTAL_IR_EMITTER_H_ + +#include + +#include "llvm/IR/IRBuilder.h" +#include "llvm/IR/Module.h" +#include "llvm/IR/Value.h" +#include "tensorflow/compiler/xla/service/hlo_instruction.h" +#include "tensorflow/compiler/xla/service/hlo_module_config.h" +#include "tensorflow/compiler/xla/service/llvm_ir/ir_builder_mixin.h" +#include "tensorflow/compiler/xla/service/llvm_ir/loop_emitter.h" +#include "tensorflow/compiler/xla/statusor.h" + +namespace xla { + +class ElementalIrEmitter : public IrBuilderMixin { + public: + using HloToElementGeneratorMap = + std::unordered_map; + + ElementalIrEmitter(const HloModuleConfig& hlo_module_config, + llvm::Module* module, llvm::IRBuilder<>* b) + : b_(b), module_(module), hlo_module_config_(hlo_module_config) {} + + virtual ~ElementalIrEmitter() = default; + + virtual StatusOr EmitUnaryOp(const HloInstruction* op, + llvm::Value* operand_value); + + virtual StatusOr EmitBinaryOp(const HloInstruction* op, + llvm::Value* lhs_value, + llvm::Value* rhs_value); + + // Returns a function to generate an element of the output of `hlo`, given a + // map of functions to generate elements of its operands. + virtual llvm_ir::ElementGenerator MakeElementGenerator( + const HloInstruction* hlo, + const HloToElementGeneratorMap& operand_to_generator); + + llvm::IRBuilder<>* b() { return b_; } + + // builder() is for IrBuilderMixin. + llvm::IRBuilder<>* builder() { return b_; } + + llvm::Module* module() { return module_; } + + protected: + virtual StatusOr EmitIntegerUnaryOp(const HloInstruction* op, + llvm::Value* operand_value); + + virtual StatusOr EmitFloatUnaryOp(const HloInstruction* op, + llvm::Value* operand_value); + + virtual StatusOr EmitComplexUnaryOp(const HloInstruction* op, + llvm::Value* operand_value); + + llvm::Value* IsZero(llvm::Value* v); + llvm::Value* IsIntMinDivisionOverflow(llvm::Value* lhs, llvm::Value* rhs); + llvm::Value* GetZero(llvm::Type* type); + llvm::Value* GetOne(llvm::Type* type); + llvm::Value* GetIntSMin(llvm::Type* type); + llvm::Value* GetMinusOne(llvm::Type* type); + + llvm::Value* EmitIntegerDivide(llvm::Value* lhs, llvm::Value* rhs, + bool is_signed); + llvm::Value* EmitIntegerRemainder(llvm::Value* lhs, llvm::Value* rhs, + bool is_signed); + + virtual StatusOr EmitIntegerBinaryOp(const HloInstruction* op, + llvm::Value* lhs_value, + llvm::Value* rhs_value, + bool is_signed); + + virtual StatusOr EmitFloatBinaryOp(const HloInstruction* op, + llvm::Value* lhs_value, + llvm::Value* rhs_value); + + virtual StatusOr EmitComplexBinaryOp(const HloInstruction* op, + llvm::Value* lhs_value, + llvm::Value* rhs_value); + + virtual llvm::Value* EmitFloatMax(llvm::Value* lhs_value, + llvm::Value* rhs_value); + + virtual llvm::Value* EmitFloatMin(llvm::Value* lhs_value, + llvm::Value* rhs_value); + + llvm::Value* EmitIntegralMax(llvm::Value* lhs_value, llvm::Value* rhs_value, + bool is_signed); + + llvm::Value* EmitIntegralMin(llvm::Value* lhs_value, llvm::Value* rhs_value, + bool is_signed); + + virtual StatusOr EmitErfInv(PrimitiveType prim_type, + llvm::Value* value); + + virtual StatusOr EmitErfcInv(PrimitiveType prim_type, + llvm::Value* value); + + virtual StatusOr EmitAtan2(PrimitiveType prim_type, + llvm::Value* lhs, llvm::Value* rhs); + + virtual StatusOr EmitLog(PrimitiveType prim_type, + llvm::Value* value); + + virtual StatusOr EmitLog1p(PrimitiveType prim_type, + llvm::Value* value); + + virtual StatusOr EmitSin(PrimitiveType prim_type, + llvm::Value* value); + + virtual StatusOr EmitCos(PrimitiveType prim_type, + llvm::Value* value); + + virtual StatusOr EmitExp(PrimitiveType prim_type, + llvm::Value* value); + + virtual StatusOr EmitExpm1(PrimitiveType prim_type, + llvm::Value* value); + + virtual StatusOr EmitPow(PrimitiveType prim_type, + llvm::Value* lhs, llvm::Value* rhs); + + virtual StatusOr EmitTanh(PrimitiveType prim_type, + llvm::Value* value); + + virtual StatusOr EmitReducePrecision(const HloInstruction* hlo, + llvm::Value* x); + + virtual llvm::Value* EmitExtractReal(llvm::Value* value); + virtual llvm::Value* EmitExtractImag(llvm::Value* value); + + // Composes a complex struct. imag may be nullptr for simple cast operations. + llvm::Value* EmitComposeComplex(const HloInstruction* op, llvm::Value* real, + llvm::Value* imag); + + // A helper method for MakeElementGenerator. Given an elementwise op `hlo` and + // the target array index, computes the source array index of its + // `operand_no`-th operand. + // + // Precondition: `hlo` is an elementwise op. + llvm_ir::IrArray::Index ElementwiseSourceIndex( + const llvm_ir::IrArray::Index& target_index, const HloInstruction& hlo, + int64 operand_no); + + // Identifier of the thread unique among all threads on the device + virtual llvm::Value* EmitThreadId() { return b_->getIntN(128, 0); } + + StatusOr EmitElementalSelect( + const HloInstruction* hlo, + const HloToElementGeneratorMap& operand_to_generator, + const llvm_ir::IrArray::Index& index); + + StatusOr EmitElementalClamp( + const HloInstruction* hlo, + const HloToElementGeneratorMap& operand_to_generator, + const llvm_ir::IrArray::Index& index); + + StatusOr EmitElementalConcatenate( + const HloInstruction* hlo, + const HloToElementGeneratorMap& operand_to_generator, + const llvm_ir::IrArray::Index& target_index); + + StatusOr EmitElementalDynamicSlice( + const HloInstruction* hlo, + const HloToElementGeneratorMap& operand_to_generator, + const llvm_ir::IrArray::Index& index); + + StatusOr EmitElementalGather( + const HloInstruction* hlo, + const HloToElementGeneratorMap& operand_to_generator, + const llvm_ir::IrArray::Index& index); + + StatusOr EmitElementalDynamicUpdateSlice( + const HloInstruction* hlo, + const HloToElementGeneratorMap& operand_to_generator, + const llvm_ir::IrArray::Index& index); + + StatusOr EmitElementalPad( + const HloInstruction* hlo, + const HloToElementGeneratorMap& operand_to_generator, + const llvm_ir::IrArray::Index& padded_index); + + StatusOr EmitElementalDot( + const HloInstruction* hlo, + const HloToElementGeneratorMap& operand_to_generator, + const llvm_ir::IrArray::Index& dot_result_index); + + llvm::IRBuilder<>* const b_; + + llvm::Module* module_; + + // The HloModuleConfig which gathers all settings and values which affect the + // compiled executable outside of the HLO code itself. + const HloModuleConfig& hlo_module_config_; + + private: + // Returns a ElementGenerator for an RNG HloInstruction using the Philox + // random number generation algorithm. + llvm_ir::ElementGenerator MakePhiloxRngElementGenerator( + const HloInstruction* hlo, + const HloToElementGeneratorMap& operand_to_generator); + // Converts the raw value generated by a random number generation algorithm + // to the distribution requested by the RNG HloInstruction. + StatusOr ConvertValueForDistribution( + const HloInstruction* hlo, + const ElementalIrEmitter::HloToElementGeneratorMap& operand_to_generator, + const llvm_ir::IrArray::Index& index, llvm::Value* raw_value); +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_ELEMENTAL_IR_EMITTER_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/executable.h b/linux/include/tensorflow/compiler/xla/service/executable.h new file mode 100644 index 000000000..3a6780f2a --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/executable.h @@ -0,0 +1,217 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_EXECUTABLE_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_EXECUTABLE_H_ + +#include +#include +#include + +#include "absl/types/span.h" +#include "absl/types/variant.h" +#include "tensorflow/compiler/xla/legacy_flags/debug_options_flags.h" +#include "tensorflow/compiler/xla/service/computation_layout.h" +#include "tensorflow/compiler/xla/service/device_memory_allocator.h" +#include "tensorflow/compiler/xla/service/hlo.pb.h" +#include "tensorflow/compiler/xla/service/hlo_execution_profile.h" +#include "tensorflow/compiler/xla/service/hlo_graph_dumper.h" +#include "tensorflow/compiler/xla/service/hlo_module.h" +#include "tensorflow/compiler/xla/service/maybe_owning_device_memory.h" +#include "tensorflow/compiler/xla/service/owning_device_memory.h" +#include "tensorflow/compiler/xla/service/service_executable_run_options.h" +#include "tensorflow/compiler/xla/service/shaped_buffer.h" +#include "tensorflow/compiler/xla/shape_tree.h" +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/compiler/xla/util.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/platform/mutex.h" +#include "tensorflow/core/platform/stream_executor_no_cuda.h" +#include "tensorflow/core/platform/thread_annotations.h" + +namespace xla { + +// ExecutionOutput encapsulates the output buffers of a execution and the +// leftover buffers to be released by the caller. +struct ExecutionOutput { + ExecutionOutput(ScopedShapedBuffer result, + std::vector to_be_released) + : result(std::move(result)), to_be_released(std::move(to_be_released)) {} + ScopedShapedBuffer result; + + // Leftover buffers for the caller to release. Elements in this list are + // donated input memory buffers that are not reused by XLA as outputs. + std::vector to_be_released; +}; + +// A given platform's compiler will produce an Executable -- this is a uniform +// interface that is used for launching compiled programs across platforms. +class Executable { + public: + explicit Executable( + std::unique_ptr hlo_module, + std::unique_ptr hlo_profile_printer_data, + std::unique_ptr hlo_profile_index_map) + : hlo_module_(std::move(hlo_module)), + hlo_profile_printer_data_(std::move(hlo_profile_printer_data)), + hlo_profile_index_map_(std::move(hlo_profile_index_map)) { + CHECK_EQ(hlo_profile_printer_data_.get() == nullptr, + hlo_profile_index_map_.get() == nullptr); + } + virtual ~Executable() {} + + // Enqueues the compilation result on the provided stream, passing the given + // arguments. This call is blocking and returns after the execution is done. + // + // If the hlo_execution_profile is provided as non-nullptr, profiling will be + // enabled. + // + // Returns a shaped buffer containing the result of the computation. + virtual StatusOr ExecuteOnStream( + const ServiceExecutableRunOptions* run_options, + absl::Span arguments, + HloExecutionProfile* hlo_execution_profile) = 0; + + // Same as ExecuteOnStream(), but this call is non-blocking and returns as + // soon as all of the operations are enqueued for launch on the stream. + virtual StatusOr ExecuteAsyncOnStream( + const ServiceExecutableRunOptions* run_options, + absl::Span arguments) = 0; + + // Starts the given program executing on the given stream/executor. + // + // `arguments` are ShapeTree containing the input parameters. For each element + // in the shape tree, if the element holds the ownership of the memory, it is + // considered donated and XLA will potentially reuse it as output buffers. For + // all donated inputs, XLA is also responsible for freeing them. + // + // If an input is donated to XLA but is not reused as output, it is returned + // as an leftover buffer for the caller to release. + virtual StatusOr ExecuteOnStream( + const ServiceExecutableRunOptions* run_options, + std::vector> arguments, + HloExecutionProfile* hlo_execution_profile) { + return Unimplemented( + "MaybeOwningDeviceMemory version of overload is not implemented "); + } + + virtual StatusOr ExecuteAsyncOnStream( + const ServiceExecutableRunOptions* run_options, + std::vector> arguments) { + return Unimplemented( + "MaybeOwningDeviceMemory version of overload is not implemented "); + } + + // Same as ExecuteOnStream(), but runs this executable on multiple + // streams. arguments[i] contains the arguments to the execution on + // run_options[i]->stream() and the returned value is at index i of the + // returned vector. + virtual StatusOr> ExecuteOnStreams( + absl::Span run_options, + absl::Span> arguments); + + // Populates `hlo_execution_profile` from `executor`. This is implicit in any + // Execute* API call that takes a hlo_execution_profile argument, but must be + // called explicitly for other (async, for example) variants after the stream + // has completed. + virtual Status PopulateExecutionProfile( + HloExecutionProfile* hlo_execution_profile, se::Stream* stream) { + return Status::OK(); + } + + // Convenience wrapper for calling Executable::ExecuteOnStream. Sets up a + // timer for the execution, sets up HLO profiling if enabled, and fills in the + // given ExecutionProfile if non-null. + StatusOr ExecuteOnStreamWrapper( + const ServiceExecutableRunOptions* run_options, ExecutionProfile* profile, + absl::Span arguments); + + // Returns the ExecutionProfile from executing on the device. This includes + // the number of cycles taken for the computation or the compilation time. + ExecutionProfile execution_profile() const { + tensorflow::mutex_lock lock(mutex_); + return execution_profile_; + } + + const HloProfilePrinterData& hlo_profile_printer_data() const { + CHECK(hlo_profiling_enabled()); + return *hlo_profile_printer_data_; + } + + const HloProfileIndexMap& hlo_profile_index_map() const { + CHECK(hlo_profiling_enabled()); + return *hlo_profile_index_map_; + } + + // Returns whether this executable was compiled with HLO profilings support + // enabled. If not, the caller should not expect an hlo_execution_profile + // passed to ExecuteOnStream above to be populated during execution. + bool hlo_profiling_enabled() const { + return hlo_profile_printer_data_ != nullptr; + } + + const HloModule& module() const { return *hlo_module_; } + + const bool has_module() const { return hlo_module_ != nullptr; } + + const HloModuleConfig& module_config() const { return hlo_module_->config(); } + + // The shape (including layout) that results from this execution. This is the + // shape of the DeviceMemoryBase result value in ExecuteOnStream above. + const Shape& result_shape() const { + return hlo_module_->config().entry_computation_layout().result_shape(); + } + + // Returns the size of the executable in bytes. Returns -1 by default if the + // method is not overridden to support this kind of query. + virtual int64 SizeInBytes(); + + // Dumping helpers. + void set_hlo_snapshot(std::unique_ptr hlo_snapshot) { + hlo_snapshot_ = std::move(hlo_snapshot); + } + bool dumping_snapshot() const { return hlo_snapshot_ != nullptr; } + HloSnapshot* hlo_snapshot() const { return hlo_snapshot_.get(); } + Status DumpHloSnapshot(); + + // Dump hlo snapshot to directory_path/filename. + static Status DumpToDirectory(const string& directory_path, string filename, + const HloSnapshot& hlo_session); + + protected: + mutable tensorflow::mutex mutex_; + + // Execution profile data on the device. + ExecutionProfile execution_profile_ GUARDED_BY(mutex_); + + // HloModule this was compiled from. BufferAssignment keeps pointers to + // HloInstructions owned by the HloModule so we need to keep the HloModule + // around. + const std::unique_ptr hlo_module_; + + // HloSnapshot this was compiled from. Null if not dumping executions. + std::unique_ptr hlo_snapshot_; + + // Execution count, used to generate a unique filename for each dumped + // execution. + int64 execution_count_ = 0; + + std::unique_ptr hlo_profile_printer_data_; + std::unique_ptr hlo_profile_index_map_; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_EXECUTABLE_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/execution_tracker.h b/linux/include/tensorflow/compiler/xla/service/execution_tracker.h new file mode 100644 index 000000000..4e9b9f883 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/execution_tracker.h @@ -0,0 +1,103 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_EXECUTION_TRACKER_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_EXECUTION_TRACKER_H_ + +#include +#include +#include + +#include "tensorflow/compiler/xla/executable_run_options.h" +#include "tensorflow/compiler/xla/service/backend.h" +#include "tensorflow/compiler/xla/service/stream_pool.h" +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/compiler/xla/util.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/platform/logging.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/mutex.h" +#include "tensorflow/core/platform/stream_executor_no_cuda.h" +#include "tensorflow/core/platform/thread_annotations.h" +#include "tensorflow/core/platform/types.h" + +namespace xla { + +// Represents an asynchronously launched execution. Owns the stream (from the +// passed run_options->stream()) on which the execution is launched and releases +// the stream when destructed. +class AsyncExecution { + public: + AsyncExecution(Backend* backend, std::vector streams, + const ExecutionProfile& profile, GlobalDataHandle result); + + Status BlockUntilDone() const; + + const GlobalDataHandle& result() const { return result_; } + + const ExecutionProfile& profile() const { return profile_; } + + private: + // Backend to execute the computation on. + Backend* backend_; + + // Stream on which the execution is launched. + std::vector streams_; + + // Profile object of the execution to be returned to the user. + ExecutionProfile profile_; + + // Data handle to the result of the execution. Data represented by this handle + // is valid only after BlockUntilDone() is called. + GlobalDataHandle result_; +}; + +// Tracks asynchronously launched executions for the XLA service. +class ExecutionTracker { + public: + ExecutionTracker(); + + // Registers an execution with its backend, streams, and data handle to the + // execution result. Returns a handle for the registered execution. + ExecutionHandle Register(Backend* backend, + std::vector stream, + const ExecutionProfile& profile, + GlobalDataHandle data); + + // Unregisters the execution for the given handle. + Status Unregister(const ExecutionHandle& handle); + + // Resolves the given ExecutionHandle to an AsyncExecution. Returns an + // error status if the given handle is not found, which means that the + // execution is not yet registered or already unregistered. + StatusOr Resolve(const ExecutionHandle& handle); + + private: + // The next handle to assign to an execution. + int64 next_handle_ GUARDED_BY(execution_mutex_); + + // Mapping from ExecutionHandle handle to the corresponding registered + // AsyncExecution object. + std::map> handle_to_execution_ + GUARDED_BY(execution_mutex_); + + tensorflow::mutex execution_mutex_; // Guards the execution mapping. + + TF_DISALLOW_COPY_AND_ASSIGN(ExecutionTracker); +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_EXECUTION_TRACKER_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/flatten_call_graph.h b/linux/include/tensorflow/compiler/xla/service/flatten_call_graph.h new file mode 100644 index 000000000..986970f88 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/flatten_call_graph.h @@ -0,0 +1,40 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// Flatten the call graph for an HLO module into a tree. + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_FLATTEN_CALL_GRAPH_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_FLATTEN_CALL_GRAPH_H_ + +#include "tensorflow/compiler/xla/service/hlo_pass_interface.h" +#include "tensorflow/compiler/xla/statusor.h" + +namespace xla { + +// Flattening associates each call site with a unique computation (for +// sequential calling contexts) This simplifies buffer assignment and +// points-to analysis (see b/36865746 for details). +class FlattenCallGraph : public HloModulePass { + public: + absl::string_view name() const override { return "flatten-call-graph"; } + + // Duplicates computations called from multiple call- or while-nodes to + // flatten the call graph. + StatusOr Run(HloModule* module) override; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_FLATTEN_CALL_GRAPH_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/generic_transfer_manager.h b/linux/include/tensorflow/compiler/xla/service/generic_transfer_manager.h new file mode 100644 index 000000000..86c8b1c14 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/generic_transfer_manager.h @@ -0,0 +1,83 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_GENERIC_TRANSFER_MANAGER_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_GENERIC_TRANSFER_MANAGER_H_ + +#include + +#include "tensorflow/compiler/xla/service/transfer_manager.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/stream_executor_no_cuda.h" +#include "tensorflow/core/platform/types.h" + +namespace xla { + +// A generic implementation of the XLA TransferManager interface +// that is the base class for both CPU and GPU. For GPU, it transfers +// data between host and device (GPU). For CPU, since the "device" +// here is the host itself, there's not much for this transfer manager +// to do except memcpy the result. There is a CpuTransferManager that +// inherits from GenericTransferManager and handles CPU-specific +// infeed. +class GenericTransferManager : public TransferManager { + public: + GenericTransferManager(se::Platform::Id platform_id, size_t pointer_size); + ~GenericTransferManager() override {} + + se::Platform::Id PlatformId() const override; + + void TransferLiteralFromDevice(se::Stream* stream, + const ShapedBuffer& device_buffer, + MutableBorrowingLiteral literal, + std::function done) override; + + Status TransferLiteralToDeviceAsync( + se::Stream* stream, const LiteralSlice& literal, + const ShapedBuffer& device_buffer) override; + + Status TransferLiteralToInfeed(se::StreamExecutor* executor, + const LiteralSlice& literal) override; + Status TransferLiteralFromOutfeed(se::StreamExecutor* executor, + const Shape& literal_shape, + MutableBorrowingLiteral literal) override; + + Status ResetDevices(absl::Span executors) override; + + int64 GetByteSizeRequirement(const Shape& shape) const override; + + protected: + Status WriteSingleTupleIndexTable( + se::Stream* stream, absl::Span elements, + const Shape& shape, se::DeviceMemoryBase* region) override; + + private: + Status TransferLiteralFromDeviceInternal(se::StreamExecutor* executor, + const ShapedBuffer& device_buffer, + MutableBorrowingLiteral literal); + + // The platform this transfer manager targets. + const se::Platform::Id platform_id_; + + // The size in bytes of pointers on this platform. + const size_t pointer_size_; + + TF_DISALLOW_COPY_AND_ASSIGN(GenericTransferManager); +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_GENERIC_TRANSFER_MANAGER_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/gpu/parallel_loop_emitter.h b/linux/include/tensorflow/compiler/xla/service/gpu/parallel_loop_emitter.h new file mode 100644 index 000000000..f32ea1ce4 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/gpu/parallel_loop_emitter.h @@ -0,0 +1,71 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_GPU_PARALLEL_LOOP_EMITTER_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_GPU_PARALLEL_LOOP_EMITTER_H_ + +#include "llvm/IR/IRBuilder.h" +#include "tensorflow/compiler/xla/service/gpu/partition_assignment.h" +#include "tensorflow/compiler/xla/service/llvm_ir/ir_array.h" +#include "tensorflow/compiler/xla/service/llvm_ir/loop_emitter.h" + +namespace xla { +namespace gpu { + +// Emits a parallel loop for every element in the given array shape. This loop +// emitted will be executed by multiple threads in parallel. Therefore, each +// thread instance of the loop iterates over part of the array, and they +// collectively iterates over the entire array. +class ParallelLoopEmitter : public llvm_ir::LoopEmitter { + public: + // `thread_count` is the number of threads to parallelize the loop on. + // The meanings of other parameters are the same as LoopEmitter. + ParallelLoopEmitter(BodyEmitter body_emitter, const Shape& shape, + const LaunchDimensions& launch_dimensions, + llvm::IRBuilder<>* b, int unroll_factor = 1); + // Constructs a ParallelLoopEmitter from an element generator that generates + // each element of the given target array. + ParallelLoopEmitter(const llvm_ir::ElementGenerator& target_element_generator, + const llvm_ir::IrArray& target_array, + const LaunchDimensions& launch_dimensions, + llvm::IRBuilder<>* b, int unroll_factor = 1); + + // Constructs a loop emitter for a loop that generates on element of each of N + // arrays on each iteration. + // + // This is used in multi-output fusion. target_element_generator should + // produce a struct with N elements, one for each of target_arrays. + ParallelLoopEmitter(const llvm_ir::ElementGenerator& target_element_generator, + absl::Span target_arrays, + const LaunchDimensions& launch_dimensions, + llvm::IRBuilder<>* b, int unroll_factor = 1); + + ParallelLoopEmitter(const ParallelLoopEmitter&) = delete; + ParallelLoopEmitter& operator=(const ParallelLoopEmitter&) = delete; + ~ParallelLoopEmitter() override = default; + + std::vector EmitIndexAndSetExitBasicBlock( + absl::string_view loop_name, llvm::Type* index_type) override; + + private: + // The thread and block dimension to parallelize the loop on. + const LaunchDimensions launch_dimensions_; + const int unroll_factor_; +}; + +} // namespace gpu +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_GPU_PARALLEL_LOOP_EMITTER_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/gpu/partition_assignment.h b/linux/include/tensorflow/compiler/xla/service/gpu/partition_assignment.h new file mode 100644 index 000000000..02471129e --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/gpu/partition_assignment.h @@ -0,0 +1,67 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// Algorithms and data structures for partition assignment. + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_GPU_PARTITION_ASSIGNMENT_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_GPU_PARTITION_ASSIGNMENT_H_ + +#include +#include +#include + +#include "tensorflow/compiler/xla/service/hlo_instruction.h" +#include "tensorflow/compiler/xla/service/hlo_module.h" +#include "tensorflow/core/platform/stream_executor_no_cuda.h" +#include "tensorflow/core/platform/types.h" + +namespace xla { +namespace gpu { + +// Encapsulates the launch dimensions of a kernel, e.g., the block count and the +// number of threads per block. +class LaunchDimensions { + public: + // The default constructor creates a launch dimension that indicate + // single-threaded execution. + LaunchDimensions() : block_count_(1), threads_per_block_(1) {} + + LaunchDimensions(int64 block_count, int64 threads_per_block) + : block_count_(block_count), threads_per_block_(threads_per_block) {} + + bool IsSinglethreaded() const { + return block_count_ == 1 && threads_per_block_ == 1; + } + + int64 block_count() const { return block_count_; } + int64 threads_per_block() const { return threads_per_block_; } + int64 launch_bound() const { return block_count() * threads_per_block(); } + + private: + int64 block_count_; + int64 threads_per_block_; +}; + +std::ostream& operator<<(std::ostream& out, + const LaunchDimensions& launch_dims); + +LaunchDimensions CalculateLaunchDimensions( + const Shape& shape, const se::DeviceDescription& device_desc, + int unroll_factor = 1); + +} // namespace gpu +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_GPU_PARTITION_ASSIGNMENT_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/heap_simulator.h b/linux/include/tensorflow/compiler/xla/service/heap_simulator.h new file mode 100644 index 000000000..b0295a616 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/heap_simulator.h @@ -0,0 +1,417 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_HEAP_SIMULATOR_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_HEAP_SIMULATOR_H_ + +#include +#include +#include +#include + +#include "absl/container/flat_hash_map.h" +#include "absl/container/flat_hash_set.h" +#include "tensorflow/compiler/xla/service/buffer_value.h" +#include "tensorflow/compiler/xla/service/buffer_value_containers.h" +#include "tensorflow/compiler/xla/service/hlo.pb.h" +#include "tensorflow/compiler/xla/service/hlo_computation.h" +#include "tensorflow/compiler/xla/service/hlo_instruction.h" +#include "tensorflow/compiler/xla/service/hlo_ordering.h" +#include "tensorflow/compiler/xla/service/hlo_schedule.h" +#include "tensorflow/compiler/xla/service/tuple_points_to_analysis.h" +#include "tensorflow/compiler/xla/statusor.h" + +namespace xla { + +// Forward declare classes defined below. +class HeapAlgorithm; +class NoFragmentationStatsHeap; + +// HeapSimulator assigns buffer offsets by running a simulation of a regular +// memory heap with Alloc and Free calls. It only works for completely +// sequential instruction sequences. Unlike regular heaps, we have the +// advantage that the sequence of Alloc and Free calls is known up-front; we +// don't need to return the assignment of buffer offsets until the very end. +class HeapSimulator { + public: + // Chunk represents a contiguous piece of memory. Each BufferValue will be + // associated with a chunk in the assignment result. + struct Chunk { + int64 offset; + int64 size; + + int64 chunk_end() const { return offset + size; } + }; + + // Result represents the result of the heap simulation. + struct Result { + // The assignment of buffers to chunks. + absl::flat_hash_map chunk_map; + + // The total size in bytes of the heap, containing all assigned chunks. + int64 heap_size = 0; + + // The total size in bytes of heap fragmentation. + int64 fragmentation_size = 0; + + // A trace of heap simulation events. + HeapSimulatorTrace debug_trace; + }; + + // The different options to be passed to the Run() APIs. + struct Options { + Options() + : may_reuse_operand_buffers(true), + alloc_constants(false), + buffers_to_assign(nullptr) {} + + // Whether a buffer about to be Free()-ed, can be recycled for a new born + // one, hence collapsing Free()+Alloc() calls (default true). + bool may_reuse_operand_buffers; + // Whether to issue Alloc() and Free() calls for constants (default false). + bool alloc_constants; + // If 'buffers_to_assign' is provided, only those buffers are assigned + // offsets, otherwise all buffers defined by the instructions are assigned. + const BufferValueFlatSet* buffers_to_assign; + }; + + // Returns the minimum memory required to compute an HLO module where all + // computations have been scheduled (represented by the given + // schedule), assuming no fragmentation. + static StatusOr MinimumMemoryForModule( + const HloSchedule& schedule, + const LogicalBuffer::SizeFunction& size_function); + + // Returns the minimum memory required to compute the given computation, + // assuming no fragmentation. + static StatusOr MinimumMemoryForComputation( + const HloComputation& computation, const HloInstructionSequence& sequence, + const TuplePointsToAnalysis& points_to_analysis, + const LogicalBuffer::SizeFunction& size_function, + const absl::flat_hash_map* + memory_by_computation = nullptr); + + // Run the heap simulation with the given algorithm, assuming the given + // schedule, which must contain a topologically-consistent total + // ordering of all instructions within each computation. The result is invalid + // if instructions are not run in exactly this sequence. + // + // Running heap simulation on the whole module tends to save memory, compared + // to running on a per-computation basis, since we can re-use buffer space for + // called sub-computations. + // + static StatusOr Run(std::unique_ptr algorithm, + const HloModule& module, + const HloSchedule& schedule, + const TuplePointsToAnalysis& points_to_analysis, + const BufferValue::SizeFunction& size_fn, + const Options& options = Options()); + + // Same as above, but runs on a single computation. The 'instruction_sequence' + // must contain a topologically-consistent total ordering of all instructions + // in the computation. The result is invalid if instructions are not run in + // exactly this sequence. + static StatusOr Run( + std::unique_ptr algorithm, + const HloComputation& computation, + const HloInstructionSequence& instruction_sequence, + const TuplePointsToAnalysis& points_to_analysis, + const BufferValue::SizeFunction& size_fn, + const Options& options = Options(), + const absl::flat_hash_map* + memory_by_computation = nullptr); + + private: + // If 'schedule' is non-null, it is used to find kCall and kWhile + // sub-computations, and the heap simulation for those sub-computations will + // be run recursively. I.e. the simulation is run over the whole module. + HeapSimulator(std::unique_ptr algorithm, + const BufferValue::SizeFunction& size_fn, + const Options& options, const HloSchedule* schedule = nullptr, + const absl::flat_hash_map* + memory_by_computation = nullptr); + ~HeapSimulator(); + + Status RunComputation(const HloComputation& computation, + const HloInstructionSequence& instruction_sequence, + const TuplePointsToAnalysis& points_to_analysis); + + bool IgnoreBuffer(const BufferValue* buffer) const; + void Alloc(const BufferValue* buffer, const HloInstruction* instruction); + void Free(const BufferValue* buffer, const HloInstruction* instruction); + void ShareBuffer(const BufferValue* buffer, const BufferValue* shared, + const HloInstruction* instruction); + Result Finish(); + + void FillDebugTrace(HeapSimulatorTrace::Event::Kind kind, + const BufferValue* buffer, + const HloInstruction* instruction, + const BufferValue* shared_with_canonical); + + // Counterintuitive: the algorithm_ itself can be a NoFragmentationStatsHeap, + // in which case we are calculating the same allocs/frees twice in the + // simulation. + const std::unique_ptr no_fragmentation_stats_; + const std::unique_ptr algorithm_; + const BufferValue::SizeFunction size_fn_; + const Options options_; + // schedule_ is set by buffer assignment, and memory_by_computation_ is + // set by hlo scheduling. Then, in RunComputation, we check both in order to + // handle subcomputations. It would be good to unify the handling of + // subcomputations, but it's not clear how. + const HloSchedule* schedule_; + const absl::flat_hash_map* + memory_by_computation_; + + // In addition to Alloc and Free, the heap simulator exposes a concept of + // buffer sharing. When ShareBuffer is called, instead of allocating new + // space for the buffer, it associates the buffer with a previously allocated + // (or shared) buffer. Each group of mutually-shared buffers points to a + // single SharedGroup instance, which is a shared control block. + // + // This forced buffer sharing is hidden from the underlying heap algorithm, + // which only sees a regular Alloc call on the canonical buffer. The + // corresponding Free call is delayed until the liveness of all shared buffers + // in the group has expired, which is tracked via the refcount. The results + // are post-processed in Finish to add chunks for shared buffers. + // + // The shared_buffers_ map associates each shared buffer (including the + // canonical) to its SharedGroup control block. + struct SharedGroup { + const BufferValue* canonical = nullptr; + int64 refcount = 0; + }; + absl::flat_hash_map> + shared_buffers_; + + // Hold some sets for error-checking the sequence of Alloc and Free calls. + absl::flat_hash_set allocated_buffers_; + absl::flat_hash_set freed_buffers_; + + // Debugging information filled in while the heap simulator runs. + HeapSimulatorTrace debug_trace_; +}; + +// Abstract base class describing a heap simulation algorithm that assigns +// offsets to buffers. A sequence of Alloc / Free calls will be made, with the +// same semantics as a regular memory heap. Finish will be called at the end to +// collect the simulation results. +class HeapAlgorithm { + public: + using Chunk = HeapSimulator::Chunk; + using Result = HeapSimulator::Result; + + virtual ~HeapAlgorithm() = default; + + // Alloc allocates a buffer of 'size' bytes. + virtual void Alloc(const BufferValue* buffer, int64 size) = 0; + + // NoFragmentationStatsHeap overrides this method. + virtual void Alloc(const BufferValue* buffer, int64 size, + const HloInstruction* instruction) { + Alloc(buffer, size); + } + + // Takes memory usage of subcomputations into account when calculating the + // memory usage of a computation. Currently, we don't handle buffer aliasing + // between computations entirely correctly. We are careful to not double count + // for the output buffers of whiles/conds/calls. But we don't take into + // account other aliases, such as for the while init. A more thorough solution + // would require something like BufferAssignment::BuildColocatedBufferSets. + // TODO(b/65835246): + // Since TuplePointsToAnalysis is being replaced with a module-aware alias + // analysis, it's not worth making major changes to HeapSimulator now. + virtual void AccountForSubcomputationMemory( + const HloInstruction* instruction, + const absl::flat_hash_map& + memory_by_computation) {} + + // Free de-allocates a previously allocated buffer. + virtual void Free(const BufferValue* buffer, int64 size) = 0; + + // Finish collects the buffer offset assignment results. Free may only be + // called once, after the Alloc and Free calls. + virtual Result Finish() = 0; +}; + +// NoFragmentationStatsHeap computes the heap size assuming no fragmentation; +// this is the absolute minimum size for a given instruction sequence. The +// result.chunk_map returned in Finish is always empty, since we only collect +// stats, and don't actually compute chunk assignments. +class NoFragmentationStatsHeap : public HeapAlgorithm { + public: + NoFragmentationStatsHeap() = default; + ~NoFragmentationStatsHeap() override = default; + + void Alloc(const BufferValue* buffer, int64 size) override; + + void Alloc(const BufferValue* buffer, int64 size, + const HloInstruction* instruction) override; + + void AccountForSubcomputationMemory( + const HloInstruction* instruction, + const absl::flat_hash_map& + memory_by_computation) override; + + void Free(const BufferValue* buffer, int64 size) override; + + Result Finish() override; + + private: + int64 current_heap_size_ = 0; + int64 max_heap_size_ = 0; +}; + +// DecreasingSizeRunsHeap collects runs of Alloc and Free calls, sorts them by +// decreasing size, and delegates the actual calls to another heap algorithm. +// This greedy heuristic tends to reduce fragmentation for all algorithms. +class DecreasingSizeRunsHeap : public HeapAlgorithm { + public: + DecreasingSizeRunsHeap(std::unique_ptr algorithm) + : algorithm_(std::move(algorithm)) {} + ~DecreasingSizeRunsHeap() override {} + + void Alloc(const BufferValue* buffer, int64 size) override; + void Free(const BufferValue* buffer, int64 size) override; + Result Finish() override; + + private: + // A single Alloc or Free operation that we've buffered in run_. + struct Op { + const BufferValue* buffer; + int64 size; + }; + + // Current collection mode; kInit means no ops have been collected yet. + enum Mode { kInit, kAlloc, kFree }; + + void SetMode(Mode mode); + void CallAndDrainRun(); + + const std::unique_ptr algorithm_; + std::vector run_; + Mode mode_ = kInit; +}; + +// LazyBestFitHeap is a variant of the traditional best-fit heap. This is a +// greedy heuristic, based on the idea that delaying offset assignment helps +// reduce fragmentation. Here's an example of a "bad" offset assignment, where +// a tiny buffer A prevents adjacent free chunks from being coalesced: +// BAD: | free |A| free | +// If we could have delayed the assignment of A, we might have ended up with: +// GOOD: | free |A| +// +// In general it's actually hard to say whether GOOD is better than BAD; the +// heuristic we use is we try to leave large contiguous chunks free, and we try +// to avoid growing the overall heap size unless necessary. +// +// Just like regular best-fit, in Alloc we look for the smallest free chunk that +// fits the requested size. Unlike regular best-fit, we postpone offset +// assignment for buffers that cannot re-use existing free chunks (and force us +// to grow the heap); these buffers are "lazily" assigned offsets in Free. +class LazyBestFitHeap : public HeapAlgorithm { + public: + LazyBestFitHeap(int64 alignment) : alignment_(alignment) {} + ~LazyBestFitHeap() override {} + + void Alloc(const BufferValue* buffer, int64 size) override; + void Free(const BufferValue* buffer, int64 size) override; + Result Finish() override; + + private: + // Sentry value used to indicate a chunk that wasn't assigned an offset in + // Alloc, and will instead be assigned an offset in Free. + enum { kLazyAllocOffset = -1 }; + + struct OrderChunkByIncreasingSize { + bool operator()(const Chunk& a, const Chunk& b) const { + if (a.size != b.size) return a.size < b.size; + return a.offset < b.offset; + } + }; + + void AddFreeChunk(int64 offset, int64 size); + + const int64 alignment_; + Result result_; + + // Maintain the set of free chunks, ordered by increasing size. + std::set free_; +}; + +// GlobalDecreasingSizeBestFitHeap collects the live intervals of all buffers, +// then allocates them in decreasing sizes regardless of the alloc/free time. It +// internally tracks the allocated buffers and their live intervals; when +// allocating a buffer, it finds the best-fit free chunk during its live +// interval. +class GlobalDecreasingSizeBestFitHeap : public HeapAlgorithm { + public: + GlobalDecreasingSizeBestFitHeap(int64 alignment) : alignment_(alignment) {} + ~GlobalDecreasingSizeBestFitHeap() override {} + + void Alloc(const BufferValue* buffer, int64 size) override; + void Free(const BufferValue* buffer, int64 size) override; + Result Finish() override; + + private: + int64 alignment_; + Result result_; + + // The current time represented as an integer. It increments by 1 at each + // Alloc or Free call. + int64 current_time_ = 0; + + // BufferInterval stores a buffer's size and time interval. + struct BufferInterval { + const BufferValue* buffer; + int64 size; + // Alloc time of the buffer. + int64 start; + // Free time of the buffer. + int64 end; + }; + absl::flat_hash_map buffer_intervals_; +}; + +// A heap algorithm that chooses the best results from other algorithms added to +// it. +class ChooseBestHeapAlgorithm : public HeapAlgorithm { + public: + ChooseBestHeapAlgorithm( + std::unique_ptr>> algorithms) + : algorithms_(std::move(*algorithms)) {} + ~ChooseBestHeapAlgorithm() override {} + + void Alloc(const BufferValue* buffer, int64 size) override { + for (auto& algorithm : algorithms_) { + algorithm->Alloc(buffer, size); + } + } + + void Free(const BufferValue* buffer, int64 size) override { + for (auto& algorithm : algorithms_) { + algorithm->Free(buffer, size); + } + } + + Result Finish() override; + + private: + std::vector> algorithms_; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_HEAP_SIMULATOR_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/hlo.pb.h b/linux/include/tensorflow/compiler/xla/service/hlo.pb.h new file mode 100644 index 000000000..69e971d95 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/hlo.pb.h @@ -0,0 +1,7612 @@ +// Generated by the protocol buffer compiler. DO NOT EDIT! +// source: tensorflow/compiler/xla/service/hlo.proto + +#ifndef PROTOBUF_INCLUDED_tensorflow_2fcompiler_2fxla_2fservice_2fhlo_2eproto +#define PROTOBUF_INCLUDED_tensorflow_2fcompiler_2fxla_2fservice_2fhlo_2eproto + +#include + +#include + +#if GOOGLE_PROTOBUF_VERSION < 3006000 +#error This file was generated by a newer version of protoc which is +#error incompatible with your Protocol Buffer headers. Please update +#error your headers. +#endif +#if 3006000 < GOOGLE_PROTOBUF_MIN_PROTOC_VERSION +#error This file was generated by an older version of protoc which is +#error incompatible with your Protocol Buffer headers. Please +#error regenerate this file with a newer version of protoc. +#endif + +#include +#include +#include +#include +#include +#include +#include +#include +#include // IWYU pragma: export +#include // IWYU pragma: export +#include // IWYU pragma: export +#include +#include +#include +#include +#include "tensorflow/compiler/xla/xla_data.pb.h" +// @@protoc_insertion_point(includes) +#define PROTOBUF_INTERNAL_EXPORT_protobuf_tensorflow_2fcompiler_2fxla_2fservice_2fhlo_2eproto + +namespace protobuf_tensorflow_2fcompiler_2fxla_2fservice_2fhlo_2eproto { +// Internal implementation detail -- do not use these members. +struct TableStruct { + static const ::google::protobuf::internal::ParseTableField entries[]; + static const ::google::protobuf::internal::AuxillaryParseTableField aux[]; + static const ::google::protobuf::internal::ParseTable schema[18]; + static const ::google::protobuf::internal::FieldMetadata field_metadata[]; + static const ::google::protobuf::internal::SerializationTable serialization_table[]; + static const ::google::protobuf::uint32 offsets[]; +}; +void AddDescriptors(); +} // namespace protobuf_tensorflow_2fcompiler_2fxla_2fservice_2fhlo_2eproto +namespace xla { +class BufferAllocationProto; +class BufferAllocationProtoDefaultTypeInternal; +extern BufferAllocationProtoDefaultTypeInternal _BufferAllocationProto_default_instance_; +class BufferAllocationProto_Assigned; +class BufferAllocationProto_AssignedDefaultTypeInternal; +extern BufferAllocationProto_AssignedDefaultTypeInternal _BufferAllocationProto_Assigned_default_instance_; +class BufferAssignmentProto; +class BufferAssignmentProtoDefaultTypeInternal; +extern BufferAssignmentProtoDefaultTypeInternal _BufferAssignmentProto_default_instance_; +class BufferAssignmentProto_BufferAlias; +class BufferAssignmentProto_BufferAliasDefaultTypeInternal; +extern BufferAssignmentProto_BufferAliasDefaultTypeInternal _BufferAssignmentProto_BufferAlias_default_instance_; +class HeapSimulatorTrace; +class HeapSimulatorTraceDefaultTypeInternal; +extern HeapSimulatorTraceDefaultTypeInternal _HeapSimulatorTrace_default_instance_; +class HeapSimulatorTrace_Event; +class HeapSimulatorTrace_EventDefaultTypeInternal; +extern HeapSimulatorTrace_EventDefaultTypeInternal _HeapSimulatorTrace_Event_default_instance_; +class HloComputationProto; +class HloComputationProtoDefaultTypeInternal; +extern HloComputationProtoDefaultTypeInternal _HloComputationProto_default_instance_; +class HloInstructionProto; +class HloInstructionProtoDefaultTypeInternal; +extern HloInstructionProtoDefaultTypeInternal _HloInstructionProto_default_instance_; +class HloInstructionProto_SliceDimensions; +class HloInstructionProto_SliceDimensionsDefaultTypeInternal; +extern HloInstructionProto_SliceDimensionsDefaultTypeInternal _HloInstructionProto_SliceDimensions_default_instance_; +class HloModuleGroupProto; +class HloModuleGroupProtoDefaultTypeInternal; +extern HloModuleGroupProtoDefaultTypeInternal _HloModuleGroupProto_default_instance_; +class HloModuleProto; +class HloModuleProtoDefaultTypeInternal; +extern HloModuleProtoDefaultTypeInternal _HloModuleProto_default_instance_; +class HloProto; +class HloProtoDefaultTypeInternal; +extern HloProtoDefaultTypeInternal _HloProto_default_instance_; +class HloScheduleProto; +class HloScheduleProtoDefaultTypeInternal; +extern HloScheduleProtoDefaultTypeInternal _HloScheduleProto_default_instance_; +class HloScheduleProto_InstructionSequence; +class HloScheduleProto_InstructionSequenceDefaultTypeInternal; +extern HloScheduleProto_InstructionSequenceDefaultTypeInternal _HloScheduleProto_InstructionSequence_default_instance_; +class HloScheduleProto_SequencesEntry_DoNotUse; +class HloScheduleProto_SequencesEntry_DoNotUseDefaultTypeInternal; +extern HloScheduleProto_SequencesEntry_DoNotUseDefaultTypeInternal _HloScheduleProto_SequencesEntry_DoNotUse_default_instance_; +class HloSnapshot; +class HloSnapshotDefaultTypeInternal; +extern HloSnapshotDefaultTypeInternal _HloSnapshot_default_instance_; +class LogicalBufferProto; +class LogicalBufferProtoDefaultTypeInternal; +extern LogicalBufferProtoDefaultTypeInternal _LogicalBufferProto_default_instance_; +class LogicalBufferProto_Location; +class LogicalBufferProto_LocationDefaultTypeInternal; +extern LogicalBufferProto_LocationDefaultTypeInternal _LogicalBufferProto_Location_default_instance_; +} // namespace xla +namespace google { +namespace protobuf { +template<> ::xla::BufferAllocationProto* Arena::CreateMaybeMessage<::xla::BufferAllocationProto>(Arena*); +template<> ::xla::BufferAllocationProto_Assigned* Arena::CreateMaybeMessage<::xla::BufferAllocationProto_Assigned>(Arena*); +template<> ::xla::BufferAssignmentProto* Arena::CreateMaybeMessage<::xla::BufferAssignmentProto>(Arena*); +template<> ::xla::BufferAssignmentProto_BufferAlias* Arena::CreateMaybeMessage<::xla::BufferAssignmentProto_BufferAlias>(Arena*); +template<> ::xla::HeapSimulatorTrace* Arena::CreateMaybeMessage<::xla::HeapSimulatorTrace>(Arena*); +template<> ::xla::HeapSimulatorTrace_Event* Arena::CreateMaybeMessage<::xla::HeapSimulatorTrace_Event>(Arena*); +template<> ::xla::HloComputationProto* Arena::CreateMaybeMessage<::xla::HloComputationProto>(Arena*); +template<> ::xla::HloInstructionProto* Arena::CreateMaybeMessage<::xla::HloInstructionProto>(Arena*); +template<> ::xla::HloInstructionProto_SliceDimensions* Arena::CreateMaybeMessage<::xla::HloInstructionProto_SliceDimensions>(Arena*); +template<> ::xla::HloModuleGroupProto* Arena::CreateMaybeMessage<::xla::HloModuleGroupProto>(Arena*); +template<> ::xla::HloModuleProto* Arena::CreateMaybeMessage<::xla::HloModuleProto>(Arena*); +template<> ::xla::HloProto* Arena::CreateMaybeMessage<::xla::HloProto>(Arena*); +template<> ::xla::HloScheduleProto* Arena::CreateMaybeMessage<::xla::HloScheduleProto>(Arena*); +template<> ::xla::HloScheduleProto_InstructionSequence* Arena::CreateMaybeMessage<::xla::HloScheduleProto_InstructionSequence>(Arena*); +template<> ::xla::HloScheduleProto_SequencesEntry_DoNotUse* Arena::CreateMaybeMessage<::xla::HloScheduleProto_SequencesEntry_DoNotUse>(Arena*); +template<> ::xla::HloSnapshot* Arena::CreateMaybeMessage<::xla::HloSnapshot>(Arena*); +template<> ::xla::LogicalBufferProto* Arena::CreateMaybeMessage<::xla::LogicalBufferProto>(Arena*); +template<> ::xla::LogicalBufferProto_Location* Arena::CreateMaybeMessage<::xla::LogicalBufferProto_Location>(Arena*); +} // namespace protobuf +} // namespace google +namespace xla { + +enum HeapSimulatorTrace_Event_Kind { + HeapSimulatorTrace_Event_Kind_ALLOC = 0, + HeapSimulatorTrace_Event_Kind_FREE = 1, + HeapSimulatorTrace_Event_Kind_SHARE_WITH = 2, + HeapSimulatorTrace_Event_Kind_HeapSimulatorTrace_Event_Kind_INT_MIN_SENTINEL_DO_NOT_USE_ = ::google::protobuf::kint32min, + HeapSimulatorTrace_Event_Kind_HeapSimulatorTrace_Event_Kind_INT_MAX_SENTINEL_DO_NOT_USE_ = ::google::protobuf::kint32max +}; +bool HeapSimulatorTrace_Event_Kind_IsValid(int value); +const HeapSimulatorTrace_Event_Kind HeapSimulatorTrace_Event_Kind_Kind_MIN = HeapSimulatorTrace_Event_Kind_ALLOC; +const HeapSimulatorTrace_Event_Kind HeapSimulatorTrace_Event_Kind_Kind_MAX = HeapSimulatorTrace_Event_Kind_SHARE_WITH; +const int HeapSimulatorTrace_Event_Kind_Kind_ARRAYSIZE = HeapSimulatorTrace_Event_Kind_Kind_MAX + 1; + +const ::google::protobuf::EnumDescriptor* HeapSimulatorTrace_Event_Kind_descriptor(); +inline const ::std::string& HeapSimulatorTrace_Event_Kind_Name(HeapSimulatorTrace_Event_Kind value) { + return ::google::protobuf::internal::NameOfEnum( + HeapSimulatorTrace_Event_Kind_descriptor(), value); +} +inline bool HeapSimulatorTrace_Event_Kind_Parse( + const ::std::string& name, HeapSimulatorTrace_Event_Kind* value) { + return ::google::protobuf::internal::ParseNamedEnum( + HeapSimulatorTrace_Event_Kind_descriptor(), name, value); +} +// =================================================================== + +class HloInstructionProto_SliceDimensions : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.HloInstructionProto.SliceDimensions) */ { + public: + HloInstructionProto_SliceDimensions(); + virtual ~HloInstructionProto_SliceDimensions(); + + HloInstructionProto_SliceDimensions(const HloInstructionProto_SliceDimensions& from); + + inline HloInstructionProto_SliceDimensions& operator=(const HloInstructionProto_SliceDimensions& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + HloInstructionProto_SliceDimensions(HloInstructionProto_SliceDimensions&& from) noexcept + : HloInstructionProto_SliceDimensions() { + *this = ::std::move(from); + } + + inline HloInstructionProto_SliceDimensions& operator=(HloInstructionProto_SliceDimensions&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const HloInstructionProto_SliceDimensions& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const HloInstructionProto_SliceDimensions* internal_default_instance() { + return reinterpret_cast( + &_HloInstructionProto_SliceDimensions_default_instance_); + } + static constexpr int kIndexInFileMessages = + 0; + + void UnsafeArenaSwap(HloInstructionProto_SliceDimensions* other); + void Swap(HloInstructionProto_SliceDimensions* other); + friend void swap(HloInstructionProto_SliceDimensions& a, HloInstructionProto_SliceDimensions& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline HloInstructionProto_SliceDimensions* New() const final { + return CreateMaybeMessage(NULL); + } + + HloInstructionProto_SliceDimensions* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const HloInstructionProto_SliceDimensions& from); + void MergeFrom(const HloInstructionProto_SliceDimensions& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(HloInstructionProto_SliceDimensions* other); + protected: + explicit HloInstructionProto_SliceDimensions(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // int64 start = 1; + void clear_start(); + static const int kStartFieldNumber = 1; + ::google::protobuf::int64 start() const; + void set_start(::google::protobuf::int64 value); + + // int64 limit = 2; + void clear_limit(); + static const int kLimitFieldNumber = 2; + ::google::protobuf::int64 limit() const; + void set_limit(::google::protobuf::int64 value); + + // int64 stride = 3; + void clear_stride(); + static const int kStrideFieldNumber = 3; + ::google::protobuf::int64 stride() const; + void set_stride(::google::protobuf::int64 value); + + // @@protoc_insertion_point(class_scope:xla.HloInstructionProto.SliceDimensions) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::int64 start_; + ::google::protobuf::int64 limit_; + ::google::protobuf::int64 stride_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fservice_2fhlo_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class HloInstructionProto : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.HloInstructionProto) */ { + public: + HloInstructionProto(); + virtual ~HloInstructionProto(); + + HloInstructionProto(const HloInstructionProto& from); + + inline HloInstructionProto& operator=(const HloInstructionProto& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + HloInstructionProto(HloInstructionProto&& from) noexcept + : HloInstructionProto() { + *this = ::std::move(from); + } + + inline HloInstructionProto& operator=(HloInstructionProto&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const HloInstructionProto& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const HloInstructionProto* internal_default_instance() { + return reinterpret_cast( + &_HloInstructionProto_default_instance_); + } + static constexpr int kIndexInFileMessages = + 1; + + void UnsafeArenaSwap(HloInstructionProto* other); + void Swap(HloInstructionProto* other); + friend void swap(HloInstructionProto& a, HloInstructionProto& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline HloInstructionProto* New() const final { + return CreateMaybeMessage(NULL); + } + + HloInstructionProto* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const HloInstructionProto& from); + void MergeFrom(const HloInstructionProto& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(HloInstructionProto* other); + protected: + explicit HloInstructionProto(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + typedef HloInstructionProto_SliceDimensions SliceDimensions; + + // accessors ------------------------------------------------------- + + // repeated int64 dimensions = 14; + int dimensions_size() const; + void clear_dimensions(); + static const int kDimensionsFieldNumber = 14; + ::google::protobuf::int64 dimensions(int index) const; + void set_dimensions(int index, ::google::protobuf::int64 value); + void add_dimensions(::google::protobuf::int64 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& + dimensions() const; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* + mutable_dimensions(); + + // repeated .xla.HloInstructionProto.SliceDimensions slice_dimensions = 17; + int slice_dimensions_size() const; + void clear_slice_dimensions(); + static const int kSliceDimensionsFieldNumber = 17; + ::xla::HloInstructionProto_SliceDimensions* mutable_slice_dimensions(int index); + ::google::protobuf::RepeatedPtrField< ::xla::HloInstructionProto_SliceDimensions >* + mutable_slice_dimensions(); + const ::xla::HloInstructionProto_SliceDimensions& slice_dimensions(int index) const; + ::xla::HloInstructionProto_SliceDimensions* add_slice_dimensions(); + const ::google::protobuf::RepeatedPtrField< ::xla::HloInstructionProto_SliceDimensions >& + slice_dimensions() const; + + // repeated int64 dynamic_slice_sizes = 20; + int dynamic_slice_sizes_size() const; + void clear_dynamic_slice_sizes(); + static const int kDynamicSliceSizesFieldNumber = 20; + ::google::protobuf::int64 dynamic_slice_sizes(int index) const; + void set_dynamic_slice_sizes(int index, ::google::protobuf::int64 value); + void add_dynamic_slice_sizes(::google::protobuf::int64 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& + dynamic_slice_sizes() const; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* + mutable_dynamic_slice_sizes(); + + // repeated int64 fft_length = 32; + int fft_length_size() const; + void clear_fft_length(); + static const int kFftLengthFieldNumber = 32; + ::google::protobuf::int64 fft_length(int index) const; + void set_fft_length(int index, ::google::protobuf::int64 value); + void add_fft_length(::google::protobuf::int64 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& + fft_length() const; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* + mutable_fft_length(); + + // repeated int64 gather_slice_sizes = 34; + int gather_slice_sizes_size() const; + void clear_gather_slice_sizes(); + static const int kGatherSliceSizesFieldNumber = 34; + ::google::protobuf::int64 gather_slice_sizes(int index) const; + void set_gather_slice_sizes(int index, ::google::protobuf::int64 value); + void add_gather_slice_sizes(::google::protobuf::int64 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& + gather_slice_sizes() const; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* + mutable_gather_slice_sizes(); + + // repeated int64 operand_ids = 36; + int operand_ids_size() const; + void clear_operand_ids(); + static const int kOperandIdsFieldNumber = 36; + ::google::protobuf::int64 operand_ids(int index) const; + void set_operand_ids(int index, ::google::protobuf::int64 value); + void add_operand_ids(::google::protobuf::int64 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& + operand_ids() const; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* + mutable_operand_ids(); + + // repeated int64 control_predecessor_ids = 37; + int control_predecessor_ids_size() const; + void clear_control_predecessor_ids(); + static const int kControlPredecessorIdsFieldNumber = 37; + ::google::protobuf::int64 control_predecessor_ids(int index) const; + void set_control_predecessor_ids(int index, ::google::protobuf::int64 value); + void add_control_predecessor_ids(::google::protobuf::int64 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& + control_predecessor_ids() const; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* + mutable_control_predecessor_ids(); + + // repeated int64 called_computation_ids = 38; + int called_computation_ids_size() const; + void clear_called_computation_ids(); + static const int kCalledComputationIdsFieldNumber = 38; + ::google::protobuf::int64 called_computation_ids(int index) const; + void set_called_computation_ids(int index, ::google::protobuf::int64 value); + void add_called_computation_ids(::google::protobuf::int64 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& + called_computation_ids() const; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* + mutable_called_computation_ids(); + + // repeated .xla.ReplicaGroup replica_groups = 49; + int replica_groups_size() const; + void clear_replica_groups(); + static const int kReplicaGroupsFieldNumber = 49; + ::xla::ReplicaGroup* mutable_replica_groups(int index); + ::google::protobuf::RepeatedPtrField< ::xla::ReplicaGroup >* + mutable_replica_groups(); + const ::xla::ReplicaGroup& replica_groups(int index) const; + ::xla::ReplicaGroup* add_replica_groups(); + const ::google::protobuf::RepeatedPtrField< ::xla::ReplicaGroup >& + replica_groups() const; + + // repeated .xla.SourceTarget source_target_pairs = 52; + int source_target_pairs_size() const; + void clear_source_target_pairs(); + static const int kSourceTargetPairsFieldNumber = 52; + ::xla::SourceTarget* mutable_source_target_pairs(int index); + ::google::protobuf::RepeatedPtrField< ::xla::SourceTarget >* + mutable_source_target_pairs(); + const ::xla::SourceTarget& source_target_pairs(int index) const; + ::xla::SourceTarget* add_source_target_pairs(); + const ::google::protobuf::RepeatedPtrField< ::xla::SourceTarget >& + source_target_pairs() const; + + // string name = 1; + void clear_name(); + static const int kNameFieldNumber = 1; + const ::std::string& name() const; + void set_name(const ::std::string& value); + #if LANG_CXX11 + void set_name(::std::string&& value); + #endif + void set_name(const char* value); + void set_name(const char* value, size_t size); + ::std::string* mutable_name(); + ::std::string* release_name(); + void set_allocated_name(::std::string* name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_name( + ::std::string* name); + + // string opcode = 2; + void clear_opcode(); + static const int kOpcodeFieldNumber = 2; + const ::std::string& opcode() const; + void set_opcode(const ::std::string& value); + #if LANG_CXX11 + void set_opcode(::std::string&& value); + #endif + void set_opcode(const char* value); + void set_opcode(const char* value, size_t size); + ::std::string* mutable_opcode(); + ::std::string* release_opcode(); + void set_allocated_opcode(::std::string* opcode); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_opcode(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_opcode( + ::std::string* opcode); + + // string fusion_kind = 11; + void clear_fusion_kind(); + static const int kFusionKindFieldNumber = 11; + const ::std::string& fusion_kind() const; + void set_fusion_kind(const ::std::string& value); + #if LANG_CXX11 + void set_fusion_kind(::std::string&& value); + #endif + void set_fusion_kind(const char* value); + void set_fusion_kind(const char* value, size_t size); + ::std::string* mutable_fusion_kind(); + ::std::string* release_fusion_kind(); + void set_allocated_fusion_kind(::std::string* fusion_kind); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_fusion_kind(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_fusion_kind( + ::std::string* fusion_kind); + + // bytes outfeed_config = 22; + void clear_outfeed_config(); + static const int kOutfeedConfigFieldNumber = 22; + const ::std::string& outfeed_config() const; + void set_outfeed_config(const ::std::string& value); + #if LANG_CXX11 + void set_outfeed_config(::std::string&& value); + #endif + void set_outfeed_config(const char* value); + void set_outfeed_config(const void* value, size_t size); + ::std::string* mutable_outfeed_config(); + ::std::string* release_outfeed_config(); + void set_allocated_outfeed_config(::std::string* outfeed_config); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_outfeed_config(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_outfeed_config( + ::std::string* outfeed_config); + + // bytes infeed_config = 27; + void clear_infeed_config(); + static const int kInfeedConfigFieldNumber = 27; + const ::std::string& infeed_config() const; + void set_infeed_config(const ::std::string& value); + #if LANG_CXX11 + void set_infeed_config(::std::string&& value); + #endif + void set_infeed_config(const char* value); + void set_infeed_config(const void* value, size_t size); + ::std::string* mutable_infeed_config(); + ::std::string* release_infeed_config(); + void set_allocated_infeed_config(::std::string* infeed_config); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_infeed_config(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_infeed_config( + ::std::string* infeed_config); + + // string custom_call_target = 28; + void clear_custom_call_target(); + static const int kCustomCallTargetFieldNumber = 28; + const ::std::string& custom_call_target() const; + void set_custom_call_target(const ::std::string& value); + #if LANG_CXX11 + void set_custom_call_target(::std::string&& value); + #endif + void set_custom_call_target(const char* value); + void set_custom_call_target(const char* value, size_t size); + ::std::string* mutable_custom_call_target(); + ::std::string* release_custom_call_target(); + void set_allocated_custom_call_target(::std::string* custom_call_target); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_custom_call_target(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_custom_call_target( + ::std::string* custom_call_target); + + // string channel_name = 41; + void clear_channel_name(); + static const int kChannelNameFieldNumber = 41; + const ::std::string& channel_name() const; + void set_channel_name(const ::std::string& value); + #if LANG_CXX11 + void set_channel_name(::std::string&& value); + #endif + void set_channel_name(const char* value); + void set_channel_name(const char* value, size_t size); + ::std::string* mutable_channel_name(); + ::std::string* release_channel_name(); + void set_allocated_channel_name(::std::string* channel_name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_channel_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_channel_name( + ::std::string* channel_name); + + // string backend_config = 43; + void clear_backend_config(); + static const int kBackendConfigFieldNumber = 43; + const ::std::string& backend_config() const; + void set_backend_config(const ::std::string& value); + #if LANG_CXX11 + void set_backend_config(::std::string&& value); + #endif + void set_backend_config(const char* value); + void set_backend_config(const char* value, size_t size); + ::std::string* mutable_backend_config(); + ::std::string* release_backend_config(); + void set_allocated_backend_config(::std::string* backend_config); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_backend_config(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_backend_config( + ::std::string* backend_config); + + // string cross_replica_sum_barrier = 46; + void clear_cross_replica_sum_barrier(); + static const int kCrossReplicaSumBarrierFieldNumber = 46; + const ::std::string& cross_replica_sum_barrier() const; + void set_cross_replica_sum_barrier(const ::std::string& value); + #if LANG_CXX11 + void set_cross_replica_sum_barrier(::std::string&& value); + #endif + void set_cross_replica_sum_barrier(const char* value); + void set_cross_replica_sum_barrier(const char* value, size_t size); + ::std::string* mutable_cross_replica_sum_barrier(); + ::std::string* release_cross_replica_sum_barrier(); + void set_allocated_cross_replica_sum_barrier(::std::string* cross_replica_sum_barrier); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_cross_replica_sum_barrier(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_cross_replica_sum_barrier( + ::std::string* cross_replica_sum_barrier); + + // string custom_call_opaque = 53; + void clear_custom_call_opaque(); + static const int kCustomCallOpaqueFieldNumber = 53; + const ::std::string& custom_call_opaque() const; + void set_custom_call_opaque(const ::std::string& value); + #if LANG_CXX11 + void set_custom_call_opaque(::std::string&& value); + #endif + void set_custom_call_opaque(const char* value); + void set_custom_call_opaque(const char* value, size_t size); + ::std::string* mutable_custom_call_opaque(); + ::std::string* release_custom_call_opaque(); + void set_allocated_custom_call_opaque(::std::string* custom_call_opaque); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_custom_call_opaque(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_custom_call_opaque( + ::std::string* custom_call_opaque); + + // .xla.Shape shape = 3; + bool has_shape() const; + void clear_shape(); + static const int kShapeFieldNumber = 3; + private: + const ::xla::Shape& _internal_shape() const; + public: + const ::xla::Shape& shape() const; + ::xla::Shape* release_shape(); + ::xla::Shape* mutable_shape(); + void set_allocated_shape(::xla::Shape* shape); + void unsafe_arena_set_allocated_shape( + ::xla::Shape* shape); + ::xla::Shape* unsafe_arena_release_shape(); + + // .xla.OpMetadata metadata = 7; + bool has_metadata() const; + void clear_metadata(); + static const int kMetadataFieldNumber = 7; + private: + const ::xla::OpMetadata& _internal_metadata() const; + public: + const ::xla::OpMetadata& metadata() const; + ::xla::OpMetadata* release_metadata(); + ::xla::OpMetadata* mutable_metadata(); + void set_allocated_metadata(::xla::OpMetadata* metadata); + void unsafe_arena_set_allocated_metadata( + ::xla::OpMetadata* metadata); + ::xla::OpMetadata* unsafe_arena_release_metadata(); + + // .xla.LiteralProto literal = 8; + bool has_literal() const; + void clear_literal(); + static const int kLiteralFieldNumber = 8; + private: + const ::xla::LiteralProto& _internal_literal() const; + public: + const ::xla::LiteralProto& literal() const; + ::xla::LiteralProto* release_literal(); + ::xla::LiteralProto* mutable_literal(); + void set_allocated_literal(::xla::LiteralProto* literal); + void unsafe_arena_set_allocated_literal( + ::xla::LiteralProto* literal); + ::xla::LiteralProto* unsafe_arena_release_literal(); + + // .xla.Window window = 15; + bool has_window() const; + void clear_window(); + static const int kWindowFieldNumber = 15; + private: + const ::xla::Window& _internal_window() const; + public: + const ::xla::Window& window() const; + ::xla::Window* release_window(); + ::xla::Window* mutable_window(); + void set_allocated_window(::xla::Window* window); + void unsafe_arena_set_allocated_window( + ::xla::Window* window); + ::xla::Window* unsafe_arena_release_window(); + + // .xla.ConvolutionDimensionNumbers convolution_dimension_numbers = 16; + bool has_convolution_dimension_numbers() const; + void clear_convolution_dimension_numbers(); + static const int kConvolutionDimensionNumbersFieldNumber = 16; + private: + const ::xla::ConvolutionDimensionNumbers& _internal_convolution_dimension_numbers() const; + public: + const ::xla::ConvolutionDimensionNumbers& convolution_dimension_numbers() const; + ::xla::ConvolutionDimensionNumbers* release_convolution_dimension_numbers(); + ::xla::ConvolutionDimensionNumbers* mutable_convolution_dimension_numbers(); + void set_allocated_convolution_dimension_numbers(::xla::ConvolutionDimensionNumbers* convolution_dimension_numbers); + void unsafe_arena_set_allocated_convolution_dimension_numbers( + ::xla::ConvolutionDimensionNumbers* convolution_dimension_numbers); + ::xla::ConvolutionDimensionNumbers* unsafe_arena_release_convolution_dimension_numbers(); + + // .xla.PaddingConfig padding_config = 21; + bool has_padding_config() const; + void clear_padding_config(); + static const int kPaddingConfigFieldNumber = 21; + private: + const ::xla::PaddingConfig& _internal_padding_config() const; + public: + const ::xla::PaddingConfig& padding_config() const; + ::xla::PaddingConfig* release_padding_config(); + ::xla::PaddingConfig* mutable_padding_config(); + void set_allocated_padding_config(::xla::PaddingConfig* padding_config); + void unsafe_arena_set_allocated_padding_config( + ::xla::PaddingConfig* padding_config); + ::xla::PaddingConfig* unsafe_arena_release_padding_config(); + + // .xla.Shape outfeed_shape = 29; + bool has_outfeed_shape() const; + void clear_outfeed_shape(); + static const int kOutfeedShapeFieldNumber = 29; + private: + const ::xla::Shape& _internal_outfeed_shape() const; + public: + const ::xla::Shape& outfeed_shape() const; + ::xla::Shape* release_outfeed_shape(); + ::xla::Shape* mutable_outfeed_shape(); + void set_allocated_outfeed_shape(::xla::Shape* outfeed_shape); + void unsafe_arena_set_allocated_outfeed_shape( + ::xla::Shape* outfeed_shape); + ::xla::Shape* unsafe_arena_release_outfeed_shape(); + + // .xla.DotDimensionNumbers dot_dimension_numbers = 30; + bool has_dot_dimension_numbers() const; + void clear_dot_dimension_numbers(); + static const int kDotDimensionNumbersFieldNumber = 30; + private: + const ::xla::DotDimensionNumbers& _internal_dot_dimension_numbers() const; + public: + const ::xla::DotDimensionNumbers& dot_dimension_numbers() const; + ::xla::DotDimensionNumbers* release_dot_dimension_numbers(); + ::xla::DotDimensionNumbers* mutable_dot_dimension_numbers(); + void set_allocated_dot_dimension_numbers(::xla::DotDimensionNumbers* dot_dimension_numbers); + void unsafe_arena_set_allocated_dot_dimension_numbers( + ::xla::DotDimensionNumbers* dot_dimension_numbers); + ::xla::DotDimensionNumbers* unsafe_arena_release_dot_dimension_numbers(); + + // .xla.GatherDimensionNumbers gather_dimension_numbers = 33; + bool has_gather_dimension_numbers() const; + void clear_gather_dimension_numbers(); + static const int kGatherDimensionNumbersFieldNumber = 33; + private: + const ::xla::GatherDimensionNumbers& _internal_gather_dimension_numbers() const; + public: + const ::xla::GatherDimensionNumbers& gather_dimension_numbers() const; + ::xla::GatherDimensionNumbers* release_gather_dimension_numbers(); + ::xla::GatherDimensionNumbers* mutable_gather_dimension_numbers(); + void set_allocated_gather_dimension_numbers(::xla::GatherDimensionNumbers* gather_dimension_numbers); + void unsafe_arena_set_allocated_gather_dimension_numbers( + ::xla::GatherDimensionNumbers* gather_dimension_numbers); + ::xla::GatherDimensionNumbers* unsafe_arena_release_gather_dimension_numbers(); + + // .xla.OpSharding sharding = 40; + bool has_sharding() const; + void clear_sharding(); + static const int kShardingFieldNumber = 40; + private: + const ::xla::OpSharding& _internal_sharding() const; + public: + const ::xla::OpSharding& sharding() const; + ::xla::OpSharding* release_sharding(); + ::xla::OpSharding* mutable_sharding(); + void set_allocated_sharding(::xla::OpSharding* sharding); + void unsafe_arena_set_allocated_sharding( + ::xla::OpSharding* sharding); + ::xla::OpSharding* unsafe_arena_release_sharding(); + + // .xla.ScatterDimensionNumbers scatter_dimension_numbers = 48; + bool has_scatter_dimension_numbers() const; + void clear_scatter_dimension_numbers(); + static const int kScatterDimensionNumbersFieldNumber = 48; + private: + const ::xla::ScatterDimensionNumbers& _internal_scatter_dimension_numbers() const; + public: + const ::xla::ScatterDimensionNumbers& scatter_dimension_numbers() const; + ::xla::ScatterDimensionNumbers* release_scatter_dimension_numbers(); + ::xla::ScatterDimensionNumbers* mutable_scatter_dimension_numbers(); + void set_allocated_scatter_dimension_numbers(::xla::ScatterDimensionNumbers* scatter_dimension_numbers); + void unsafe_arena_set_allocated_scatter_dimension_numbers( + ::xla::ScatterDimensionNumbers* scatter_dimension_numbers); + ::xla::ScatterDimensionNumbers* unsafe_arena_release_scatter_dimension_numbers(); + + // .xla.PrecisionConfig precision_config = 51; + bool has_precision_config() const; + void clear_precision_config(); + static const int kPrecisionConfigFieldNumber = 51; + private: + const ::xla::PrecisionConfig& _internal_precision_config() const; + public: + const ::xla::PrecisionConfig& precision_config() const; + ::xla::PrecisionConfig* release_precision_config(); + ::xla::PrecisionConfig* mutable_precision_config(); + void set_allocated_precision_config(::xla::PrecisionConfig* precision_config); + void unsafe_arena_set_allocated_precision_config( + ::xla::PrecisionConfig* precision_config); + ::xla::PrecisionConfig* unsafe_arena_release_precision_config(); + + // int64 parameter_number = 9; + void clear_parameter_number(); + static const int kParameterNumberFieldNumber = 9; + ::google::protobuf::int64 parameter_number() const; + void set_parameter_number(::google::protobuf::int64 value); + + // int64 tuple_index = 13; + void clear_tuple_index(); + static const int kTupleIndexFieldNumber = 13; + ::google::protobuf::int64 tuple_index() const; + void set_tuple_index(::google::protobuf::int64 value); + + // int32 exponent_bits = 18; + void clear_exponent_bits(); + static const int kExponentBitsFieldNumber = 18; + ::google::protobuf::int32 exponent_bits() const; + void set_exponent_bits(::google::protobuf::int32 value); + + // int32 mantissa_bits = 19; + void clear_mantissa_bits(); + static const int kMantissaBitsFieldNumber = 19; + ::google::protobuf::int32 mantissa_bits() const; + void set_mantissa_bits(::google::protobuf::int32 value); + + // .xla.RandomDistribution distribution = 23; + void clear_distribution(); + static const int kDistributionFieldNumber = 23; + ::xla::RandomDistribution distribution() const; + void set_distribution(::xla::RandomDistribution value); + + // float epsilon = 24; + void clear_epsilon(); + static const int kEpsilonFieldNumber = 24; + float epsilon() const; + void set_epsilon(float value); + + // int64 feature_index = 25; + void clear_feature_index(); + static const int kFeatureIndexFieldNumber = 25; + ::google::protobuf::int64 feature_index() const; + void set_feature_index(::google::protobuf::int64 value); + + // int64 channel_id = 26; + void clear_channel_id(); + static const int kChannelIdFieldNumber = 26; + ::google::protobuf::int64 channel_id() const; + void set_channel_id(::google::protobuf::int64 value); + + // int64 id = 35; + void clear_id(); + static const int kIdFieldNumber = 35; + ::google::protobuf::int64 id() const; + void set_id(::google::protobuf::int64 value); + + // .xla.FftType fft_type = 31; + void clear_fft_type(); + static const int kFftTypeFieldNumber = 31; + ::xla::FftType fft_type() const; + void set_fft_type(::xla::FftType value); + + // bool is_host_transfer = 47; + void clear_is_host_transfer(); + static const int kIsHostTransferFieldNumber = 47; + bool is_host_transfer() const; + void set_is_host_transfer(bool value); + + // int64 cost_estimate_ns = 42; + void clear_cost_estimate_ns(); + static const int kCostEstimateNsFieldNumber = 42; + ::google::protobuf::int64 cost_estimate_ns() const; + void set_cost_estimate_ns(::google::protobuf::int64 value); + + // int64 all_reduce_id = 45; + void clear_all_reduce_id(); + static const int kAllReduceIdFieldNumber = 45; + ::google::protobuf::int64 all_reduce_id() const; + void set_all_reduce_id(::google::protobuf::int64 value); + + // int64 feature_group_count = 50; + void clear_feature_group_count(); + static const int kFeatureGroupCountFieldNumber = 50; + ::google::protobuf::int64 feature_group_count() const; + void set_feature_group_count(::google::protobuf::int64 value); + + // @@protoc_insertion_point(class_scope:xla.HloInstructionProto) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 > dimensions_; + mutable int _dimensions_cached_byte_size_; + ::google::protobuf::RepeatedPtrField< ::xla::HloInstructionProto_SliceDimensions > slice_dimensions_; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 > dynamic_slice_sizes_; + mutable int _dynamic_slice_sizes_cached_byte_size_; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 > fft_length_; + mutable int _fft_length_cached_byte_size_; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 > gather_slice_sizes_; + mutable int _gather_slice_sizes_cached_byte_size_; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 > operand_ids_; + mutable int _operand_ids_cached_byte_size_; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 > control_predecessor_ids_; + mutable int _control_predecessor_ids_cached_byte_size_; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 > called_computation_ids_; + mutable int _called_computation_ids_cached_byte_size_; + ::google::protobuf::RepeatedPtrField< ::xla::ReplicaGroup > replica_groups_; + ::google::protobuf::RepeatedPtrField< ::xla::SourceTarget > source_target_pairs_; + ::google::protobuf::internal::ArenaStringPtr name_; + ::google::protobuf::internal::ArenaStringPtr opcode_; + ::google::protobuf::internal::ArenaStringPtr fusion_kind_; + ::google::protobuf::internal::ArenaStringPtr outfeed_config_; + ::google::protobuf::internal::ArenaStringPtr infeed_config_; + ::google::protobuf::internal::ArenaStringPtr custom_call_target_; + ::google::protobuf::internal::ArenaStringPtr channel_name_; + ::google::protobuf::internal::ArenaStringPtr backend_config_; + ::google::protobuf::internal::ArenaStringPtr cross_replica_sum_barrier_; + ::google::protobuf::internal::ArenaStringPtr custom_call_opaque_; + ::xla::Shape* shape_; + ::xla::OpMetadata* metadata_; + ::xla::LiteralProto* literal_; + ::xla::Window* window_; + ::xla::ConvolutionDimensionNumbers* convolution_dimension_numbers_; + ::xla::PaddingConfig* padding_config_; + ::xla::Shape* outfeed_shape_; + ::xla::DotDimensionNumbers* dot_dimension_numbers_; + ::xla::GatherDimensionNumbers* gather_dimension_numbers_; + ::xla::OpSharding* sharding_; + ::xla::ScatterDimensionNumbers* scatter_dimension_numbers_; + ::xla::PrecisionConfig* precision_config_; + ::google::protobuf::int64 parameter_number_; + ::google::protobuf::int64 tuple_index_; + ::google::protobuf::int32 exponent_bits_; + ::google::protobuf::int32 mantissa_bits_; + int distribution_; + float epsilon_; + ::google::protobuf::int64 feature_index_; + ::google::protobuf::int64 channel_id_; + ::google::protobuf::int64 id_; + int fft_type_; + bool is_host_transfer_; + ::google::protobuf::int64 cost_estimate_ns_; + ::google::protobuf::int64 all_reduce_id_; + ::google::protobuf::int64 feature_group_count_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fservice_2fhlo_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class HloComputationProto : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.HloComputationProto) */ { + public: + HloComputationProto(); + virtual ~HloComputationProto(); + + HloComputationProto(const HloComputationProto& from); + + inline HloComputationProto& operator=(const HloComputationProto& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + HloComputationProto(HloComputationProto&& from) noexcept + : HloComputationProto() { + *this = ::std::move(from); + } + + inline HloComputationProto& operator=(HloComputationProto&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const HloComputationProto& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const HloComputationProto* internal_default_instance() { + return reinterpret_cast( + &_HloComputationProto_default_instance_); + } + static constexpr int kIndexInFileMessages = + 2; + + void UnsafeArenaSwap(HloComputationProto* other); + void Swap(HloComputationProto* other); + friend void swap(HloComputationProto& a, HloComputationProto& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline HloComputationProto* New() const final { + return CreateMaybeMessage(NULL); + } + + HloComputationProto* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const HloComputationProto& from); + void MergeFrom(const HloComputationProto& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(HloComputationProto* other); + protected: + explicit HloComputationProto(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated .xla.HloInstructionProto instructions = 2; + int instructions_size() const; + void clear_instructions(); + static const int kInstructionsFieldNumber = 2; + ::xla::HloInstructionProto* mutable_instructions(int index); + ::google::protobuf::RepeatedPtrField< ::xla::HloInstructionProto >* + mutable_instructions(); + const ::xla::HloInstructionProto& instructions(int index) const; + ::xla::HloInstructionProto* add_instructions(); + const ::google::protobuf::RepeatedPtrField< ::xla::HloInstructionProto >& + instructions() const; + + // string name = 1; + void clear_name(); + static const int kNameFieldNumber = 1; + const ::std::string& name() const; + void set_name(const ::std::string& value); + #if LANG_CXX11 + void set_name(::std::string&& value); + #endif + void set_name(const char* value); + void set_name(const char* value, size_t size); + ::std::string* mutable_name(); + ::std::string* release_name(); + void set_allocated_name(::std::string* name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_name( + ::std::string* name); + + // .xla.ProgramShape program_shape = 4; + bool has_program_shape() const; + void clear_program_shape(); + static const int kProgramShapeFieldNumber = 4; + private: + const ::xla::ProgramShape& _internal_program_shape() const; + public: + const ::xla::ProgramShape& program_shape() const; + ::xla::ProgramShape* release_program_shape(); + ::xla::ProgramShape* mutable_program_shape(); + void set_allocated_program_shape(::xla::ProgramShape* program_shape); + void unsafe_arena_set_allocated_program_shape( + ::xla::ProgramShape* program_shape); + ::xla::ProgramShape* unsafe_arena_release_program_shape(); + + // int64 id = 5; + void clear_id(); + static const int kIdFieldNumber = 5; + ::google::protobuf::int64 id() const; + void set_id(::google::protobuf::int64 value); + + // int64 root_id = 6; + void clear_root_id(); + static const int kRootIdFieldNumber = 6; + ::google::protobuf::int64 root_id() const; + void set_root_id(::google::protobuf::int64 value); + + // @@protoc_insertion_point(class_scope:xla.HloComputationProto) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedPtrField< ::xla::HloInstructionProto > instructions_; + ::google::protobuf::internal::ArenaStringPtr name_; + ::xla::ProgramShape* program_shape_; + ::google::protobuf::int64 id_; + ::google::protobuf::int64 root_id_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fservice_2fhlo_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class HloScheduleProto_InstructionSequence : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.HloScheduleProto.InstructionSequence) */ { + public: + HloScheduleProto_InstructionSequence(); + virtual ~HloScheduleProto_InstructionSequence(); + + HloScheduleProto_InstructionSequence(const HloScheduleProto_InstructionSequence& from); + + inline HloScheduleProto_InstructionSequence& operator=(const HloScheduleProto_InstructionSequence& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + HloScheduleProto_InstructionSequence(HloScheduleProto_InstructionSequence&& from) noexcept + : HloScheduleProto_InstructionSequence() { + *this = ::std::move(from); + } + + inline HloScheduleProto_InstructionSequence& operator=(HloScheduleProto_InstructionSequence&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const HloScheduleProto_InstructionSequence& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const HloScheduleProto_InstructionSequence* internal_default_instance() { + return reinterpret_cast( + &_HloScheduleProto_InstructionSequence_default_instance_); + } + static constexpr int kIndexInFileMessages = + 3; + + void UnsafeArenaSwap(HloScheduleProto_InstructionSequence* other); + void Swap(HloScheduleProto_InstructionSequence* other); + friend void swap(HloScheduleProto_InstructionSequence& a, HloScheduleProto_InstructionSequence& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline HloScheduleProto_InstructionSequence* New() const final { + return CreateMaybeMessage(NULL); + } + + HloScheduleProto_InstructionSequence* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const HloScheduleProto_InstructionSequence& from); + void MergeFrom(const HloScheduleProto_InstructionSequence& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(HloScheduleProto_InstructionSequence* other); + protected: + explicit HloScheduleProto_InstructionSequence(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated int64 instruction_ids = 1; + int instruction_ids_size() const; + void clear_instruction_ids(); + static const int kInstructionIdsFieldNumber = 1; + ::google::protobuf::int64 instruction_ids(int index) const; + void set_instruction_ids(int index, ::google::protobuf::int64 value); + void add_instruction_ids(::google::protobuf::int64 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& + instruction_ids() const; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* + mutable_instruction_ids(); + + // @@protoc_insertion_point(class_scope:xla.HloScheduleProto.InstructionSequence) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 > instruction_ids_; + mutable int _instruction_ids_cached_byte_size_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fservice_2fhlo_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class HloScheduleProto_SequencesEntry_DoNotUse : public ::google::protobuf::internal::MapEntry { +public: + typedef ::google::protobuf::internal::MapEntry SuperType; + HloScheduleProto_SequencesEntry_DoNotUse(); + HloScheduleProto_SequencesEntry_DoNotUse(::google::protobuf::Arena* arena); + void MergeFrom(const HloScheduleProto_SequencesEntry_DoNotUse& other); + static const HloScheduleProto_SequencesEntry_DoNotUse* internal_default_instance() { return reinterpret_cast(&_HloScheduleProto_SequencesEntry_DoNotUse_default_instance_); } + void MergeFrom(const ::google::protobuf::Message& other) final; + ::google::protobuf::Metadata GetMetadata() const; +}; + +// ------------------------------------------------------------------- + +class HloScheduleProto : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.HloScheduleProto) */ { + public: + HloScheduleProto(); + virtual ~HloScheduleProto(); + + HloScheduleProto(const HloScheduleProto& from); + + inline HloScheduleProto& operator=(const HloScheduleProto& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + HloScheduleProto(HloScheduleProto&& from) noexcept + : HloScheduleProto() { + *this = ::std::move(from); + } + + inline HloScheduleProto& operator=(HloScheduleProto&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const HloScheduleProto& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const HloScheduleProto* internal_default_instance() { + return reinterpret_cast( + &_HloScheduleProto_default_instance_); + } + static constexpr int kIndexInFileMessages = + 5; + + void UnsafeArenaSwap(HloScheduleProto* other); + void Swap(HloScheduleProto* other); + friend void swap(HloScheduleProto& a, HloScheduleProto& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline HloScheduleProto* New() const final { + return CreateMaybeMessage(NULL); + } + + HloScheduleProto* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const HloScheduleProto& from); + void MergeFrom(const HloScheduleProto& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(HloScheduleProto* other); + protected: + explicit HloScheduleProto(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + typedef HloScheduleProto_InstructionSequence InstructionSequence; + + // accessors ------------------------------------------------------- + + // map sequences = 1; + int sequences_size() const; + void clear_sequences(); + static const int kSequencesFieldNumber = 1; + const ::google::protobuf::Map< ::google::protobuf::int64, ::xla::HloScheduleProto_InstructionSequence >& + sequences() const; + ::google::protobuf::Map< ::google::protobuf::int64, ::xla::HloScheduleProto_InstructionSequence >* + mutable_sequences(); + + // @@protoc_insertion_point(class_scope:xla.HloScheduleProto) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::internal::MapField< + HloScheduleProto_SequencesEntry_DoNotUse, + ::google::protobuf::int64, ::xla::HloScheduleProto_InstructionSequence, + ::google::protobuf::internal::WireFormatLite::TYPE_INT64, + ::google::protobuf::internal::WireFormatLite::TYPE_MESSAGE, + 0 > sequences_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fservice_2fhlo_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class HloModuleProto : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.HloModuleProto) */ { + public: + HloModuleProto(); + virtual ~HloModuleProto(); + + HloModuleProto(const HloModuleProto& from); + + inline HloModuleProto& operator=(const HloModuleProto& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + HloModuleProto(HloModuleProto&& from) noexcept + : HloModuleProto() { + *this = ::std::move(from); + } + + inline HloModuleProto& operator=(HloModuleProto&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const HloModuleProto& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const HloModuleProto* internal_default_instance() { + return reinterpret_cast( + &_HloModuleProto_default_instance_); + } + static constexpr int kIndexInFileMessages = + 6; + + void UnsafeArenaSwap(HloModuleProto* other); + void Swap(HloModuleProto* other); + friend void swap(HloModuleProto& a, HloModuleProto& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline HloModuleProto* New() const final { + return CreateMaybeMessage(NULL); + } + + HloModuleProto* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const HloModuleProto& from); + void MergeFrom(const HloModuleProto& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(HloModuleProto* other); + protected: + explicit HloModuleProto(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated .xla.HloComputationProto computations = 3; + int computations_size() const; + void clear_computations(); + static const int kComputationsFieldNumber = 3; + ::xla::HloComputationProto* mutable_computations(int index); + ::google::protobuf::RepeatedPtrField< ::xla::HloComputationProto >* + mutable_computations(); + const ::xla::HloComputationProto& computations(int index) const; + ::xla::HloComputationProto* add_computations(); + const ::google::protobuf::RepeatedPtrField< ::xla::HloComputationProto >& + computations() const; + + // string name = 1; + void clear_name(); + static const int kNameFieldNumber = 1; + const ::std::string& name() const; + void set_name(const ::std::string& value); + #if LANG_CXX11 + void set_name(::std::string&& value); + #endif + void set_name(const char* value); + void set_name(const char* value, size_t size); + ::std::string* mutable_name(); + ::std::string* release_name(); + void set_allocated_name(::std::string* name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_name( + ::std::string* name); + + // string entry_computation_name = 2; + void clear_entry_computation_name(); + static const int kEntryComputationNameFieldNumber = 2; + const ::std::string& entry_computation_name() const; + void set_entry_computation_name(const ::std::string& value); + #if LANG_CXX11 + void set_entry_computation_name(::std::string&& value); + #endif + void set_entry_computation_name(const char* value); + void set_entry_computation_name(const char* value, size_t size); + ::std::string* mutable_entry_computation_name(); + ::std::string* release_entry_computation_name(); + void set_allocated_entry_computation_name(::std::string* entry_computation_name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_entry_computation_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_entry_computation_name( + ::std::string* entry_computation_name); + + // .xla.ProgramShape program_shape = 4; + bool has_program_shape() const; + void clear_program_shape(); + static const int kProgramShapeFieldNumber = 4; + private: + const ::xla::ProgramShape& _internal_program_shape() const; + public: + const ::xla::ProgramShape& program_shape() const; + ::xla::ProgramShape* release_program_shape(); + ::xla::ProgramShape* mutable_program_shape(); + void set_allocated_program_shape(::xla::ProgramShape* program_shape); + void unsafe_arena_set_allocated_program_shape( + ::xla::ProgramShape* program_shape); + ::xla::ProgramShape* unsafe_arena_release_program_shape(); + + // .xla.HloScheduleProto schedule = 7; + bool has_schedule() const; + void clear_schedule(); + static const int kScheduleFieldNumber = 7; + private: + const ::xla::HloScheduleProto& _internal_schedule() const; + public: + const ::xla::HloScheduleProto& schedule() const; + ::xla::HloScheduleProto* release_schedule(); + ::xla::HloScheduleProto* mutable_schedule(); + void set_allocated_schedule(::xla::HloScheduleProto* schedule); + void unsafe_arena_set_allocated_schedule( + ::xla::HloScheduleProto* schedule); + ::xla::HloScheduleProto* unsafe_arena_release_schedule(); + + // int64 id = 5; + void clear_id(); + static const int kIdFieldNumber = 5; + ::google::protobuf::int64 id() const; + void set_id(::google::protobuf::int64 value); + + // int64 entry_computation_id = 6; + void clear_entry_computation_id(); + static const int kEntryComputationIdFieldNumber = 6; + ::google::protobuf::int64 entry_computation_id() const; + void set_entry_computation_id(::google::protobuf::int64 value); + + // @@protoc_insertion_point(class_scope:xla.HloModuleProto) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedPtrField< ::xla::HloComputationProto > computations_; + ::google::protobuf::internal::ArenaStringPtr name_; + ::google::protobuf::internal::ArenaStringPtr entry_computation_name_; + ::xla::ProgramShape* program_shape_; + ::xla::HloScheduleProto* schedule_; + ::google::protobuf::int64 id_; + ::google::protobuf::int64 entry_computation_id_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fservice_2fhlo_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class LogicalBufferProto_Location : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.LogicalBufferProto.Location) */ { + public: + LogicalBufferProto_Location(); + virtual ~LogicalBufferProto_Location(); + + LogicalBufferProto_Location(const LogicalBufferProto_Location& from); + + inline LogicalBufferProto_Location& operator=(const LogicalBufferProto_Location& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + LogicalBufferProto_Location(LogicalBufferProto_Location&& from) noexcept + : LogicalBufferProto_Location() { + *this = ::std::move(from); + } + + inline LogicalBufferProto_Location& operator=(LogicalBufferProto_Location&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const LogicalBufferProto_Location& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const LogicalBufferProto_Location* internal_default_instance() { + return reinterpret_cast( + &_LogicalBufferProto_Location_default_instance_); + } + static constexpr int kIndexInFileMessages = + 7; + + void UnsafeArenaSwap(LogicalBufferProto_Location* other); + void Swap(LogicalBufferProto_Location* other); + friend void swap(LogicalBufferProto_Location& a, LogicalBufferProto_Location& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline LogicalBufferProto_Location* New() const final { + return CreateMaybeMessage(NULL); + } + + LogicalBufferProto_Location* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const LogicalBufferProto_Location& from); + void MergeFrom(const LogicalBufferProto_Location& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(LogicalBufferProto_Location* other); + protected: + explicit LogicalBufferProto_Location(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated int64 shape_index = 3; + int shape_index_size() const; + void clear_shape_index(); + static const int kShapeIndexFieldNumber = 3; + ::google::protobuf::int64 shape_index(int index) const; + void set_shape_index(int index, ::google::protobuf::int64 value); + void add_shape_index(::google::protobuf::int64 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& + shape_index() const; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* + mutable_shape_index(); + + // string computation_name = 1; + void clear_computation_name(); + static const int kComputationNameFieldNumber = 1; + const ::std::string& computation_name() const; + void set_computation_name(const ::std::string& value); + #if LANG_CXX11 + void set_computation_name(::std::string&& value); + #endif + void set_computation_name(const char* value); + void set_computation_name(const char* value, size_t size); + ::std::string* mutable_computation_name(); + ::std::string* release_computation_name(); + void set_allocated_computation_name(::std::string* computation_name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_computation_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_computation_name( + ::std::string* computation_name); + + // string instruction_name = 2; + void clear_instruction_name(); + static const int kInstructionNameFieldNumber = 2; + const ::std::string& instruction_name() const; + void set_instruction_name(const ::std::string& value); + #if LANG_CXX11 + void set_instruction_name(::std::string&& value); + #endif + void set_instruction_name(const char* value); + void set_instruction_name(const char* value, size_t size); + ::std::string* mutable_instruction_name(); + ::std::string* release_instruction_name(); + void set_allocated_instruction_name(::std::string* instruction_name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_instruction_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_instruction_name( + ::std::string* instruction_name); + + // @@protoc_insertion_point(class_scope:xla.LogicalBufferProto.Location) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 > shape_index_; + mutable int _shape_index_cached_byte_size_; + ::google::protobuf::internal::ArenaStringPtr computation_name_; + ::google::protobuf::internal::ArenaStringPtr instruction_name_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fservice_2fhlo_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class LogicalBufferProto : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.LogicalBufferProto) */ { + public: + LogicalBufferProto(); + virtual ~LogicalBufferProto(); + + LogicalBufferProto(const LogicalBufferProto& from); + + inline LogicalBufferProto& operator=(const LogicalBufferProto& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + LogicalBufferProto(LogicalBufferProto&& from) noexcept + : LogicalBufferProto() { + *this = ::std::move(from); + } + + inline LogicalBufferProto& operator=(LogicalBufferProto&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const LogicalBufferProto& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const LogicalBufferProto* internal_default_instance() { + return reinterpret_cast( + &_LogicalBufferProto_default_instance_); + } + static constexpr int kIndexInFileMessages = + 8; + + void UnsafeArenaSwap(LogicalBufferProto* other); + void Swap(LogicalBufferProto* other); + friend void swap(LogicalBufferProto& a, LogicalBufferProto& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline LogicalBufferProto* New() const final { + return CreateMaybeMessage(NULL); + } + + LogicalBufferProto* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const LogicalBufferProto& from); + void MergeFrom(const LogicalBufferProto& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(LogicalBufferProto* other); + protected: + explicit LogicalBufferProto(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + typedef LogicalBufferProto_Location Location; + + // accessors ------------------------------------------------------- + + // .xla.LogicalBufferProto.Location defined_at = 3; + bool has_defined_at() const; + void clear_defined_at(); + static const int kDefinedAtFieldNumber = 3; + private: + const ::xla::LogicalBufferProto_Location& _internal_defined_at() const; + public: + const ::xla::LogicalBufferProto_Location& defined_at() const; + ::xla::LogicalBufferProto_Location* release_defined_at(); + ::xla::LogicalBufferProto_Location* mutable_defined_at(); + void set_allocated_defined_at(::xla::LogicalBufferProto_Location* defined_at); + void unsafe_arena_set_allocated_defined_at( + ::xla::LogicalBufferProto_Location* defined_at); + ::xla::LogicalBufferProto_Location* unsafe_arena_release_defined_at(); + + // int64 id = 1; + void clear_id(); + static const int kIdFieldNumber = 1; + ::google::protobuf::int64 id() const; + void set_id(::google::protobuf::int64 value); + + // int64 size = 2; + void clear_size(); + static const int kSizeFieldNumber = 2; + ::google::protobuf::int64 size() const; + void set_size(::google::protobuf::int64 value); + + // int64 color = 4; + void clear_color(); + static const int kColorFieldNumber = 4; + ::google::protobuf::int64 color() const; + void set_color(::google::protobuf::int64 value); + + // @@protoc_insertion_point(class_scope:xla.LogicalBufferProto) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::xla::LogicalBufferProto_Location* defined_at_; + ::google::protobuf::int64 id_; + ::google::protobuf::int64 size_; + ::google::protobuf::int64 color_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fservice_2fhlo_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class BufferAllocationProto_Assigned : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.BufferAllocationProto.Assigned) */ { + public: + BufferAllocationProto_Assigned(); + virtual ~BufferAllocationProto_Assigned(); + + BufferAllocationProto_Assigned(const BufferAllocationProto_Assigned& from); + + inline BufferAllocationProto_Assigned& operator=(const BufferAllocationProto_Assigned& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + BufferAllocationProto_Assigned(BufferAllocationProto_Assigned&& from) noexcept + : BufferAllocationProto_Assigned() { + *this = ::std::move(from); + } + + inline BufferAllocationProto_Assigned& operator=(BufferAllocationProto_Assigned&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const BufferAllocationProto_Assigned& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const BufferAllocationProto_Assigned* internal_default_instance() { + return reinterpret_cast( + &_BufferAllocationProto_Assigned_default_instance_); + } + static constexpr int kIndexInFileMessages = + 9; + + void UnsafeArenaSwap(BufferAllocationProto_Assigned* other); + void Swap(BufferAllocationProto_Assigned* other); + friend void swap(BufferAllocationProto_Assigned& a, BufferAllocationProto_Assigned& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline BufferAllocationProto_Assigned* New() const final { + return CreateMaybeMessage(NULL); + } + + BufferAllocationProto_Assigned* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const BufferAllocationProto_Assigned& from); + void MergeFrom(const BufferAllocationProto_Assigned& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(BufferAllocationProto_Assigned* other); + protected: + explicit BufferAllocationProto_Assigned(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // int64 logical_buffer_id = 1; + void clear_logical_buffer_id(); + static const int kLogicalBufferIdFieldNumber = 1; + ::google::protobuf::int64 logical_buffer_id() const; + void set_logical_buffer_id(::google::protobuf::int64 value); + + // int64 offset = 2; + void clear_offset(); + static const int kOffsetFieldNumber = 2; + ::google::protobuf::int64 offset() const; + void set_offset(::google::protobuf::int64 value); + + // int64 size = 3; + void clear_size(); + static const int kSizeFieldNumber = 3; + ::google::protobuf::int64 size() const; + void set_size(::google::protobuf::int64 value); + + // @@protoc_insertion_point(class_scope:xla.BufferAllocationProto.Assigned) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::int64 logical_buffer_id_; + ::google::protobuf::int64 offset_; + ::google::protobuf::int64 size_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fservice_2fhlo_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class BufferAllocationProto : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.BufferAllocationProto) */ { + public: + BufferAllocationProto(); + virtual ~BufferAllocationProto(); + + BufferAllocationProto(const BufferAllocationProto& from); + + inline BufferAllocationProto& operator=(const BufferAllocationProto& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + BufferAllocationProto(BufferAllocationProto&& from) noexcept + : BufferAllocationProto() { + *this = ::std::move(from); + } + + inline BufferAllocationProto& operator=(BufferAllocationProto&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const BufferAllocationProto& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const BufferAllocationProto* internal_default_instance() { + return reinterpret_cast( + &_BufferAllocationProto_default_instance_); + } + static constexpr int kIndexInFileMessages = + 10; + + void UnsafeArenaSwap(BufferAllocationProto* other); + void Swap(BufferAllocationProto* other); + friend void swap(BufferAllocationProto& a, BufferAllocationProto& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline BufferAllocationProto* New() const final { + return CreateMaybeMessage(NULL); + } + + BufferAllocationProto* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const BufferAllocationProto& from); + void MergeFrom(const BufferAllocationProto& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(BufferAllocationProto* other); + protected: + explicit BufferAllocationProto(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + typedef BufferAllocationProto_Assigned Assigned; + + // accessors ------------------------------------------------------- + + // repeated .xla.BufferAllocationProto.Assigned assigned = 9; + int assigned_size() const; + void clear_assigned(); + static const int kAssignedFieldNumber = 9; + ::xla::BufferAllocationProto_Assigned* mutable_assigned(int index); + ::google::protobuf::RepeatedPtrField< ::xla::BufferAllocationProto_Assigned >* + mutable_assigned(); + const ::xla::BufferAllocationProto_Assigned& assigned(int index) const; + ::xla::BufferAllocationProto_Assigned* add_assigned(); + const ::google::protobuf::RepeatedPtrField< ::xla::BufferAllocationProto_Assigned >& + assigned() const; + + // repeated int64 parameter_shape_index = 10; + int parameter_shape_index_size() const; + void clear_parameter_shape_index(); + static const int kParameterShapeIndexFieldNumber = 10; + ::google::protobuf::int64 parameter_shape_index(int index) const; + void set_parameter_shape_index(int index, ::google::protobuf::int64 value); + void add_parameter_shape_index(::google::protobuf::int64 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& + parameter_shape_index() const; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* + mutable_parameter_shape_index(); + + // int64 index = 1; + void clear_index(); + static const int kIndexFieldNumber = 1; + ::google::protobuf::int64 index() const; + void set_index(::google::protobuf::int64 value); + + // int64 size = 2; + void clear_size(); + static const int kSizeFieldNumber = 2; + ::google::protobuf::int64 size() const; + void set_size(::google::protobuf::int64 value); + + // int64 parameter_number = 6; + void clear_parameter_number(); + static const int kParameterNumberFieldNumber = 6; + ::google::protobuf::int64 parameter_number() const; + void set_parameter_number(::google::protobuf::int64 value); + + // bool maybe_live_out = 7; + void clear_maybe_live_out(); + static const int kMaybeLiveOutFieldNumber = 7; + bool maybe_live_out() const; + void set_maybe_live_out(bool value); + + // bool is_thread_local = 3; + void clear_is_thread_local(); + static const int kIsThreadLocalFieldNumber = 3; + bool is_thread_local() const; + void set_is_thread_local(bool value); + + // bool is_tuple = 11; + void clear_is_tuple(); + static const int kIsTupleFieldNumber = 11; + bool is_tuple() const; + void set_is_tuple(bool value); + + // bool is_entry_computation_parameter = 5; + void clear_is_entry_computation_parameter(); + static const int kIsEntryComputationParameterFieldNumber = 5; + bool is_entry_computation_parameter() const; + void set_is_entry_computation_parameter(bool value); + + // bool is_constant = 12; + void clear_is_constant(); + static const int kIsConstantFieldNumber = 12; + bool is_constant() const; + void set_is_constant(bool value); + + // int64 color = 8; + void clear_color(); + static const int kColorFieldNumber = 8; + ::google::protobuf::int64 color() const; + void set_color(::google::protobuf::int64 value); + + // @@protoc_insertion_point(class_scope:xla.BufferAllocationProto) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedPtrField< ::xla::BufferAllocationProto_Assigned > assigned_; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 > parameter_shape_index_; + mutable int _parameter_shape_index_cached_byte_size_; + ::google::protobuf::int64 index_; + ::google::protobuf::int64 size_; + ::google::protobuf::int64 parameter_number_; + bool maybe_live_out_; + bool is_thread_local_; + bool is_tuple_; + bool is_entry_computation_parameter_; + bool is_constant_; + ::google::protobuf::int64 color_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fservice_2fhlo_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class HeapSimulatorTrace_Event : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.HeapSimulatorTrace.Event) */ { + public: + HeapSimulatorTrace_Event(); + virtual ~HeapSimulatorTrace_Event(); + + HeapSimulatorTrace_Event(const HeapSimulatorTrace_Event& from); + + inline HeapSimulatorTrace_Event& operator=(const HeapSimulatorTrace_Event& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + HeapSimulatorTrace_Event(HeapSimulatorTrace_Event&& from) noexcept + : HeapSimulatorTrace_Event() { + *this = ::std::move(from); + } + + inline HeapSimulatorTrace_Event& operator=(HeapSimulatorTrace_Event&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const HeapSimulatorTrace_Event& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const HeapSimulatorTrace_Event* internal_default_instance() { + return reinterpret_cast( + &_HeapSimulatorTrace_Event_default_instance_); + } + static constexpr int kIndexInFileMessages = + 11; + + void UnsafeArenaSwap(HeapSimulatorTrace_Event* other); + void Swap(HeapSimulatorTrace_Event* other); + friend void swap(HeapSimulatorTrace_Event& a, HeapSimulatorTrace_Event& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline HeapSimulatorTrace_Event* New() const final { + return CreateMaybeMessage(NULL); + } + + HeapSimulatorTrace_Event* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const HeapSimulatorTrace_Event& from); + void MergeFrom(const HeapSimulatorTrace_Event& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(HeapSimulatorTrace_Event* other); + protected: + explicit HeapSimulatorTrace_Event(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + typedef HeapSimulatorTrace_Event_Kind Kind; + static const Kind ALLOC = + HeapSimulatorTrace_Event_Kind_ALLOC; + static const Kind FREE = + HeapSimulatorTrace_Event_Kind_FREE; + static const Kind SHARE_WITH = + HeapSimulatorTrace_Event_Kind_SHARE_WITH; + static inline bool Kind_IsValid(int value) { + return HeapSimulatorTrace_Event_Kind_IsValid(value); + } + static const Kind Kind_MIN = + HeapSimulatorTrace_Event_Kind_Kind_MIN; + static const Kind Kind_MAX = + HeapSimulatorTrace_Event_Kind_Kind_MAX; + static const int Kind_ARRAYSIZE = + HeapSimulatorTrace_Event_Kind_Kind_ARRAYSIZE; + static inline const ::google::protobuf::EnumDescriptor* + Kind_descriptor() { + return HeapSimulatorTrace_Event_Kind_descriptor(); + } + static inline const ::std::string& Kind_Name(Kind value) { + return HeapSimulatorTrace_Event_Kind_Name(value); + } + static inline bool Kind_Parse(const ::std::string& name, + Kind* value) { + return HeapSimulatorTrace_Event_Kind_Parse(name, value); + } + + // accessors ------------------------------------------------------- + + // string computation_name = 3; + void clear_computation_name(); + static const int kComputationNameFieldNumber = 3; + const ::std::string& computation_name() const; + void set_computation_name(const ::std::string& value); + #if LANG_CXX11 + void set_computation_name(::std::string&& value); + #endif + void set_computation_name(const char* value); + void set_computation_name(const char* value, size_t size); + ::std::string* mutable_computation_name(); + ::std::string* release_computation_name(); + void set_allocated_computation_name(::std::string* computation_name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_computation_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_computation_name( + ::std::string* computation_name); + + // string instruction_name = 4; + void clear_instruction_name(); + static const int kInstructionNameFieldNumber = 4; + const ::std::string& instruction_name() const; + void set_instruction_name(const ::std::string& value); + #if LANG_CXX11 + void set_instruction_name(::std::string&& value); + #endif + void set_instruction_name(const char* value); + void set_instruction_name(const char* value, size_t size); + ::std::string* mutable_instruction_name(); + ::std::string* release_instruction_name(); + void set_allocated_instruction_name(::std::string* instruction_name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_instruction_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_instruction_name( + ::std::string* instruction_name); + + // int64 buffer_id = 2; + void clear_buffer_id(); + static const int kBufferIdFieldNumber = 2; + ::google::protobuf::int64 buffer_id() const; + void set_buffer_id(::google::protobuf::int64 value); + + // int64 share_with_canonical_id = 5; + void clear_share_with_canonical_id(); + static const int kShareWithCanonicalIdFieldNumber = 5; + ::google::protobuf::int64 share_with_canonical_id() const; + void set_share_with_canonical_id(::google::protobuf::int64 value); + + // .xla.HeapSimulatorTrace.Event.Kind kind = 1; + void clear_kind(); + static const int kKindFieldNumber = 1; + ::xla::HeapSimulatorTrace_Event_Kind kind() const; + void set_kind(::xla::HeapSimulatorTrace_Event_Kind value); + + // @@protoc_insertion_point(class_scope:xla.HeapSimulatorTrace.Event) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::internal::ArenaStringPtr computation_name_; + ::google::protobuf::internal::ArenaStringPtr instruction_name_; + ::google::protobuf::int64 buffer_id_; + ::google::protobuf::int64 share_with_canonical_id_; + int kind_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fservice_2fhlo_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class HeapSimulatorTrace : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.HeapSimulatorTrace) */ { + public: + HeapSimulatorTrace(); + virtual ~HeapSimulatorTrace(); + + HeapSimulatorTrace(const HeapSimulatorTrace& from); + + inline HeapSimulatorTrace& operator=(const HeapSimulatorTrace& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + HeapSimulatorTrace(HeapSimulatorTrace&& from) noexcept + : HeapSimulatorTrace() { + *this = ::std::move(from); + } + + inline HeapSimulatorTrace& operator=(HeapSimulatorTrace&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const HeapSimulatorTrace& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const HeapSimulatorTrace* internal_default_instance() { + return reinterpret_cast( + &_HeapSimulatorTrace_default_instance_); + } + static constexpr int kIndexInFileMessages = + 12; + + void UnsafeArenaSwap(HeapSimulatorTrace* other); + void Swap(HeapSimulatorTrace* other); + friend void swap(HeapSimulatorTrace& a, HeapSimulatorTrace& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline HeapSimulatorTrace* New() const final { + return CreateMaybeMessage(NULL); + } + + HeapSimulatorTrace* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const HeapSimulatorTrace& from); + void MergeFrom(const HeapSimulatorTrace& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(HeapSimulatorTrace* other); + protected: + explicit HeapSimulatorTrace(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + typedef HeapSimulatorTrace_Event Event; + + // accessors ------------------------------------------------------- + + // repeated .xla.HeapSimulatorTrace.Event events = 1; + int events_size() const; + void clear_events(); + static const int kEventsFieldNumber = 1; + ::xla::HeapSimulatorTrace_Event* mutable_events(int index); + ::google::protobuf::RepeatedPtrField< ::xla::HeapSimulatorTrace_Event >* + mutable_events(); + const ::xla::HeapSimulatorTrace_Event& events(int index) const; + ::xla::HeapSimulatorTrace_Event* add_events(); + const ::google::protobuf::RepeatedPtrField< ::xla::HeapSimulatorTrace_Event >& + events() const; + + // bool whole_module_simulation = 2; + void clear_whole_module_simulation(); + static const int kWholeModuleSimulationFieldNumber = 2; + bool whole_module_simulation() const; + void set_whole_module_simulation(bool value); + + // @@protoc_insertion_point(class_scope:xla.HeapSimulatorTrace) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedPtrField< ::xla::HeapSimulatorTrace_Event > events_; + bool whole_module_simulation_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fservice_2fhlo_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class HloModuleGroupProto : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.HloModuleGroupProto) */ { + public: + HloModuleGroupProto(); + virtual ~HloModuleGroupProto(); + + HloModuleGroupProto(const HloModuleGroupProto& from); + + inline HloModuleGroupProto& operator=(const HloModuleGroupProto& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + HloModuleGroupProto(HloModuleGroupProto&& from) noexcept + : HloModuleGroupProto() { + *this = ::std::move(from); + } + + inline HloModuleGroupProto& operator=(HloModuleGroupProto&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const HloModuleGroupProto& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const HloModuleGroupProto* internal_default_instance() { + return reinterpret_cast( + &_HloModuleGroupProto_default_instance_); + } + static constexpr int kIndexInFileMessages = + 13; + + void UnsafeArenaSwap(HloModuleGroupProto* other); + void Swap(HloModuleGroupProto* other); + friend void swap(HloModuleGroupProto& a, HloModuleGroupProto& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline HloModuleGroupProto* New() const final { + return CreateMaybeMessage(NULL); + } + + HloModuleGroupProto* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const HloModuleGroupProto& from); + void MergeFrom(const HloModuleGroupProto& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(HloModuleGroupProto* other); + protected: + explicit HloModuleGroupProto(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated .xla.HloModuleProto hlo_modules = 2; + int hlo_modules_size() const; + void clear_hlo_modules(); + static const int kHloModulesFieldNumber = 2; + ::xla::HloModuleProto* mutable_hlo_modules(int index); + ::google::protobuf::RepeatedPtrField< ::xla::HloModuleProto >* + mutable_hlo_modules(); + const ::xla::HloModuleProto& hlo_modules(int index) const; + ::xla::HloModuleProto* add_hlo_modules(); + const ::google::protobuf::RepeatedPtrField< ::xla::HloModuleProto >& + hlo_modules() const; + + // string name = 1; + void clear_name(); + static const int kNameFieldNumber = 1; + const ::std::string& name() const; + void set_name(const ::std::string& value); + #if LANG_CXX11 + void set_name(::std::string&& value); + #endif + void set_name(const char* value); + void set_name(const char* value, size_t size); + ::std::string* mutable_name(); + ::std::string* release_name(); + void set_allocated_name(::std::string* name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_name( + ::std::string* name); + + // @@protoc_insertion_point(class_scope:xla.HloModuleGroupProto) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedPtrField< ::xla::HloModuleProto > hlo_modules_; + ::google::protobuf::internal::ArenaStringPtr name_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fservice_2fhlo_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class BufferAssignmentProto_BufferAlias : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.BufferAssignmentProto.BufferAlias) */ { + public: + BufferAssignmentProto_BufferAlias(); + virtual ~BufferAssignmentProto_BufferAlias(); + + BufferAssignmentProto_BufferAlias(const BufferAssignmentProto_BufferAlias& from); + + inline BufferAssignmentProto_BufferAlias& operator=(const BufferAssignmentProto_BufferAlias& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + BufferAssignmentProto_BufferAlias(BufferAssignmentProto_BufferAlias&& from) noexcept + : BufferAssignmentProto_BufferAlias() { + *this = ::std::move(from); + } + + inline BufferAssignmentProto_BufferAlias& operator=(BufferAssignmentProto_BufferAlias&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const BufferAssignmentProto_BufferAlias& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const BufferAssignmentProto_BufferAlias* internal_default_instance() { + return reinterpret_cast( + &_BufferAssignmentProto_BufferAlias_default_instance_); + } + static constexpr int kIndexInFileMessages = + 14; + + void UnsafeArenaSwap(BufferAssignmentProto_BufferAlias* other); + void Swap(BufferAssignmentProto_BufferAlias* other); + friend void swap(BufferAssignmentProto_BufferAlias& a, BufferAssignmentProto_BufferAlias& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline BufferAssignmentProto_BufferAlias* New() const final { + return CreateMaybeMessage(NULL); + } + + BufferAssignmentProto_BufferAlias* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const BufferAssignmentProto_BufferAlias& from); + void MergeFrom(const BufferAssignmentProto_BufferAlias& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(BufferAssignmentProto_BufferAlias* other); + protected: + explicit BufferAssignmentProto_BufferAlias(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // .xla.LogicalBufferProto.Location location = 2; + bool has_location() const; + void clear_location(); + static const int kLocationFieldNumber = 2; + private: + const ::xla::LogicalBufferProto_Location& _internal_location() const; + public: + const ::xla::LogicalBufferProto_Location& location() const; + ::xla::LogicalBufferProto_Location* release_location(); + ::xla::LogicalBufferProto_Location* mutable_location(); + void set_allocated_location(::xla::LogicalBufferProto_Location* location); + void unsafe_arena_set_allocated_location( + ::xla::LogicalBufferProto_Location* location); + ::xla::LogicalBufferProto_Location* unsafe_arena_release_location(); + + // int64 source_buffer_id = 1; + void clear_source_buffer_id(); + static const int kSourceBufferIdFieldNumber = 1; + ::google::protobuf::int64 source_buffer_id() const; + void set_source_buffer_id(::google::protobuf::int64 value); + + // @@protoc_insertion_point(class_scope:xla.BufferAssignmentProto.BufferAlias) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::xla::LogicalBufferProto_Location* location_; + ::google::protobuf::int64 source_buffer_id_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fservice_2fhlo_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class BufferAssignmentProto : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.BufferAssignmentProto) */ { + public: + BufferAssignmentProto(); + virtual ~BufferAssignmentProto(); + + BufferAssignmentProto(const BufferAssignmentProto& from); + + inline BufferAssignmentProto& operator=(const BufferAssignmentProto& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + BufferAssignmentProto(BufferAssignmentProto&& from) noexcept + : BufferAssignmentProto() { + *this = ::std::move(from); + } + + inline BufferAssignmentProto& operator=(BufferAssignmentProto&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const BufferAssignmentProto& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const BufferAssignmentProto* internal_default_instance() { + return reinterpret_cast( + &_BufferAssignmentProto_default_instance_); + } + static constexpr int kIndexInFileMessages = + 15; + + void UnsafeArenaSwap(BufferAssignmentProto* other); + void Swap(BufferAssignmentProto* other); + friend void swap(BufferAssignmentProto& a, BufferAssignmentProto& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline BufferAssignmentProto* New() const final { + return CreateMaybeMessage(NULL); + } + + BufferAssignmentProto* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const BufferAssignmentProto& from); + void MergeFrom(const BufferAssignmentProto& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(BufferAssignmentProto* other); + protected: + explicit BufferAssignmentProto(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + typedef BufferAssignmentProto_BufferAlias BufferAlias; + + // accessors ------------------------------------------------------- + + // repeated .xla.LogicalBufferProto logical_buffers = 1; + int logical_buffers_size() const; + void clear_logical_buffers(); + static const int kLogicalBuffersFieldNumber = 1; + ::xla::LogicalBufferProto* mutable_logical_buffers(int index); + ::google::protobuf::RepeatedPtrField< ::xla::LogicalBufferProto >* + mutable_logical_buffers(); + const ::xla::LogicalBufferProto& logical_buffers(int index) const; + ::xla::LogicalBufferProto* add_logical_buffers(); + const ::google::protobuf::RepeatedPtrField< ::xla::LogicalBufferProto >& + logical_buffers() const; + + // repeated .xla.BufferAssignmentProto.BufferAlias buffer_aliases = 2; + int buffer_aliases_size() const; + void clear_buffer_aliases(); + static const int kBufferAliasesFieldNumber = 2; + ::xla::BufferAssignmentProto_BufferAlias* mutable_buffer_aliases(int index); + ::google::protobuf::RepeatedPtrField< ::xla::BufferAssignmentProto_BufferAlias >* + mutable_buffer_aliases(); + const ::xla::BufferAssignmentProto_BufferAlias& buffer_aliases(int index) const; + ::xla::BufferAssignmentProto_BufferAlias* add_buffer_aliases(); + const ::google::protobuf::RepeatedPtrField< ::xla::BufferAssignmentProto_BufferAlias >& + buffer_aliases() const; + + // repeated .xla.BufferAllocationProto buffer_allocations = 3; + int buffer_allocations_size() const; + void clear_buffer_allocations(); + static const int kBufferAllocationsFieldNumber = 3; + ::xla::BufferAllocationProto* mutable_buffer_allocations(int index); + ::google::protobuf::RepeatedPtrField< ::xla::BufferAllocationProto >* + mutable_buffer_allocations(); + const ::xla::BufferAllocationProto& buffer_allocations(int index) const; + ::xla::BufferAllocationProto* add_buffer_allocations(); + const ::google::protobuf::RepeatedPtrField< ::xla::BufferAllocationProto >& + buffer_allocations() const; + + // repeated .xla.HeapSimulatorTrace heap_simulator_traces = 4; + int heap_simulator_traces_size() const; + void clear_heap_simulator_traces(); + static const int kHeapSimulatorTracesFieldNumber = 4; + ::xla::HeapSimulatorTrace* mutable_heap_simulator_traces(int index); + ::google::protobuf::RepeatedPtrField< ::xla::HeapSimulatorTrace >* + mutable_heap_simulator_traces(); + const ::xla::HeapSimulatorTrace& heap_simulator_traces(int index) const; + ::xla::HeapSimulatorTrace* add_heap_simulator_traces(); + const ::google::protobuf::RepeatedPtrField< ::xla::HeapSimulatorTrace >& + heap_simulator_traces() const; + + // @@protoc_insertion_point(class_scope:xla.BufferAssignmentProto) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedPtrField< ::xla::LogicalBufferProto > logical_buffers_; + ::google::protobuf::RepeatedPtrField< ::xla::BufferAssignmentProto_BufferAlias > buffer_aliases_; + ::google::protobuf::RepeatedPtrField< ::xla::BufferAllocationProto > buffer_allocations_; + ::google::protobuf::RepeatedPtrField< ::xla::HeapSimulatorTrace > heap_simulator_traces_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fservice_2fhlo_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class HloProto : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.HloProto) */ { + public: + HloProto(); + virtual ~HloProto(); + + HloProto(const HloProto& from); + + inline HloProto& operator=(const HloProto& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + HloProto(HloProto&& from) noexcept + : HloProto() { + *this = ::std::move(from); + } + + inline HloProto& operator=(HloProto&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const HloProto& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const HloProto* internal_default_instance() { + return reinterpret_cast( + &_HloProto_default_instance_); + } + static constexpr int kIndexInFileMessages = + 16; + + void UnsafeArenaSwap(HloProto* other); + void Swap(HloProto* other); + friend void swap(HloProto& a, HloProto& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline HloProto* New() const final { + return CreateMaybeMessage(NULL); + } + + HloProto* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const HloProto& from); + void MergeFrom(const HloProto& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(HloProto* other); + protected: + explicit HloProto(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // .xla.HloModuleProto hlo_module = 1; + bool has_hlo_module() const; + void clear_hlo_module(); + static const int kHloModuleFieldNumber = 1; + private: + const ::xla::HloModuleProto& _internal_hlo_module() const; + public: + const ::xla::HloModuleProto& hlo_module() const; + ::xla::HloModuleProto* release_hlo_module(); + ::xla::HloModuleProto* mutable_hlo_module(); + void set_allocated_hlo_module(::xla::HloModuleProto* hlo_module); + void unsafe_arena_set_allocated_hlo_module( + ::xla::HloModuleProto* hlo_module); + ::xla::HloModuleProto* unsafe_arena_release_hlo_module(); + + // .xla.BufferAssignmentProto buffer_assignment = 3; + bool has_buffer_assignment() const; + void clear_buffer_assignment(); + static const int kBufferAssignmentFieldNumber = 3; + private: + const ::xla::BufferAssignmentProto& _internal_buffer_assignment() const; + public: + const ::xla::BufferAssignmentProto& buffer_assignment() const; + ::xla::BufferAssignmentProto* release_buffer_assignment(); + ::xla::BufferAssignmentProto* mutable_buffer_assignment(); + void set_allocated_buffer_assignment(::xla::BufferAssignmentProto* buffer_assignment); + void unsafe_arena_set_allocated_buffer_assignment( + ::xla::BufferAssignmentProto* buffer_assignment); + ::xla::BufferAssignmentProto* unsafe_arena_release_buffer_assignment(); + + // @@protoc_insertion_point(class_scope:xla.HloProto) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::xla::HloModuleProto* hlo_module_; + ::xla::BufferAssignmentProto* buffer_assignment_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fservice_2fhlo_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class HloSnapshot : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.HloSnapshot) */ { + public: + HloSnapshot(); + virtual ~HloSnapshot(); + + HloSnapshot(const HloSnapshot& from); + + inline HloSnapshot& operator=(const HloSnapshot& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + HloSnapshot(HloSnapshot&& from) noexcept + : HloSnapshot() { + *this = ::std::move(from); + } + + inline HloSnapshot& operator=(HloSnapshot&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const HloSnapshot& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const HloSnapshot* internal_default_instance() { + return reinterpret_cast( + &_HloSnapshot_default_instance_); + } + static constexpr int kIndexInFileMessages = + 17; + + void UnsafeArenaSwap(HloSnapshot* other); + void Swap(HloSnapshot* other); + friend void swap(HloSnapshot& a, HloSnapshot& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline HloSnapshot* New() const final { + return CreateMaybeMessage(NULL); + } + + HloSnapshot* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const HloSnapshot& from); + void MergeFrom(const HloSnapshot& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(HloSnapshot* other); + protected: + explicit HloSnapshot(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated .xla.LiteralProto arguments = 2; + int arguments_size() const; + void clear_arguments(); + static const int kArgumentsFieldNumber = 2; + ::xla::LiteralProto* mutable_arguments(int index); + ::google::protobuf::RepeatedPtrField< ::xla::LiteralProto >* + mutable_arguments(); + const ::xla::LiteralProto& arguments(int index) const; + ::xla::LiteralProto* add_arguments(); + const ::google::protobuf::RepeatedPtrField< ::xla::LiteralProto >& + arguments() const; + + // string execution_platform = 4; + void clear_execution_platform(); + static const int kExecutionPlatformFieldNumber = 4; + const ::std::string& execution_platform() const; + void set_execution_platform(const ::std::string& value); + #if LANG_CXX11 + void set_execution_platform(::std::string&& value); + #endif + void set_execution_platform(const char* value); + void set_execution_platform(const char* value, size_t size); + ::std::string* mutable_execution_platform(); + ::std::string* release_execution_platform(); + void set_allocated_execution_platform(::std::string* execution_platform); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_execution_platform(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_execution_platform( + ::std::string* execution_platform); + + // .xla.HloProto hlo = 1; + bool has_hlo() const; + void clear_hlo(); + static const int kHloFieldNumber = 1; + private: + const ::xla::HloProto& _internal_hlo() const; + public: + const ::xla::HloProto& hlo() const; + ::xla::HloProto* release_hlo(); + ::xla::HloProto* mutable_hlo(); + void set_allocated_hlo(::xla::HloProto* hlo); + void unsafe_arena_set_allocated_hlo( + ::xla::HloProto* hlo); + ::xla::HloProto* unsafe_arena_release_hlo(); + + // .xla.LiteralProto result = 3; + bool has_result() const; + void clear_result(); + static const int kResultFieldNumber = 3; + private: + const ::xla::LiteralProto& _internal_result() const; + public: + const ::xla::LiteralProto& result() const; + ::xla::LiteralProto* release_result(); + ::xla::LiteralProto* mutable_result(); + void set_allocated_result(::xla::LiteralProto* result); + void unsafe_arena_set_allocated_result( + ::xla::LiteralProto* result); + ::xla::LiteralProto* unsafe_arena_release_result(); + + // @@protoc_insertion_point(class_scope:xla.HloSnapshot) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedPtrField< ::xla::LiteralProto > arguments_; + ::google::protobuf::internal::ArenaStringPtr execution_platform_; + ::xla::HloProto* hlo_; + ::xla::LiteralProto* result_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fservice_2fhlo_2eproto::TableStruct; +}; +// =================================================================== + + +// =================================================================== + +#ifdef __GNUC__ + #pragma GCC diagnostic push + #pragma GCC diagnostic ignored "-Wstrict-aliasing" +#endif // __GNUC__ +// HloInstructionProto_SliceDimensions + +// int64 start = 1; +inline void HloInstructionProto_SliceDimensions::clear_start() { + start_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 HloInstructionProto_SliceDimensions::start() const { + // @@protoc_insertion_point(field_get:xla.HloInstructionProto.SliceDimensions.start) + return start_; +} +inline void HloInstructionProto_SliceDimensions::set_start(::google::protobuf::int64 value) { + + start_ = value; + // @@protoc_insertion_point(field_set:xla.HloInstructionProto.SliceDimensions.start) +} + +// int64 limit = 2; +inline void HloInstructionProto_SliceDimensions::clear_limit() { + limit_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 HloInstructionProto_SliceDimensions::limit() const { + // @@protoc_insertion_point(field_get:xla.HloInstructionProto.SliceDimensions.limit) + return limit_; +} +inline void HloInstructionProto_SliceDimensions::set_limit(::google::protobuf::int64 value) { + + limit_ = value; + // @@protoc_insertion_point(field_set:xla.HloInstructionProto.SliceDimensions.limit) +} + +// int64 stride = 3; +inline void HloInstructionProto_SliceDimensions::clear_stride() { + stride_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 HloInstructionProto_SliceDimensions::stride() const { + // @@protoc_insertion_point(field_get:xla.HloInstructionProto.SliceDimensions.stride) + return stride_; +} +inline void HloInstructionProto_SliceDimensions::set_stride(::google::protobuf::int64 value) { + + stride_ = value; + // @@protoc_insertion_point(field_set:xla.HloInstructionProto.SliceDimensions.stride) +} + +// ------------------------------------------------------------------- + +// HloInstructionProto + +// string name = 1; +inline void HloInstructionProto::clear_name() { + name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& HloInstructionProto::name() const { + // @@protoc_insertion_point(field_get:xla.HloInstructionProto.name) + return name_.Get(); +} +inline void HloInstructionProto::set_name(const ::std::string& value) { + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:xla.HloInstructionProto.name) +} +#if LANG_CXX11 +inline void HloInstructionProto::set_name(::std::string&& value) { + + name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:xla.HloInstructionProto.name) +} +#endif +inline void HloInstructionProto::set_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:xla.HloInstructionProto.name) +} +inline void HloInstructionProto::set_name(const char* value, + size_t size) { + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:xla.HloInstructionProto.name) +} +inline ::std::string* HloInstructionProto::mutable_name() { + + // @@protoc_insertion_point(field_mutable:xla.HloInstructionProto.name) + return name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* HloInstructionProto::release_name() { + // @@protoc_insertion_point(field_release:xla.HloInstructionProto.name) + + return name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void HloInstructionProto::set_allocated_name(::std::string* name) { + if (name != NULL) { + + } else { + + } + name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:xla.HloInstructionProto.name) +} +inline ::std::string* HloInstructionProto::unsafe_arena_release_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.HloInstructionProto.name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void HloInstructionProto::unsafe_arena_set_allocated_name( + ::std::string* name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (name != NULL) { + + } else { + + } + name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:xla.HloInstructionProto.name) +} + +// string opcode = 2; +inline void HloInstructionProto::clear_opcode() { + opcode_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& HloInstructionProto::opcode() const { + // @@protoc_insertion_point(field_get:xla.HloInstructionProto.opcode) + return opcode_.Get(); +} +inline void HloInstructionProto::set_opcode(const ::std::string& value) { + + opcode_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:xla.HloInstructionProto.opcode) +} +#if LANG_CXX11 +inline void HloInstructionProto::set_opcode(::std::string&& value) { + + opcode_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:xla.HloInstructionProto.opcode) +} +#endif +inline void HloInstructionProto::set_opcode(const char* value) { + GOOGLE_DCHECK(value != NULL); + + opcode_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:xla.HloInstructionProto.opcode) +} +inline void HloInstructionProto::set_opcode(const char* value, + size_t size) { + + opcode_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:xla.HloInstructionProto.opcode) +} +inline ::std::string* HloInstructionProto::mutable_opcode() { + + // @@protoc_insertion_point(field_mutable:xla.HloInstructionProto.opcode) + return opcode_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* HloInstructionProto::release_opcode() { + // @@protoc_insertion_point(field_release:xla.HloInstructionProto.opcode) + + return opcode_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void HloInstructionProto::set_allocated_opcode(::std::string* opcode) { + if (opcode != NULL) { + + } else { + + } + opcode_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), opcode, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:xla.HloInstructionProto.opcode) +} +inline ::std::string* HloInstructionProto::unsafe_arena_release_opcode() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.HloInstructionProto.opcode) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return opcode_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void HloInstructionProto::unsafe_arena_set_allocated_opcode( + ::std::string* opcode) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (opcode != NULL) { + + } else { + + } + opcode_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + opcode, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:xla.HloInstructionProto.opcode) +} + +// .xla.Shape shape = 3; +inline bool HloInstructionProto::has_shape() const { + return this != internal_default_instance() && shape_ != NULL; +} +inline const ::xla::Shape& HloInstructionProto::_internal_shape() const { + return *shape_; +} +inline const ::xla::Shape& HloInstructionProto::shape() const { + const ::xla::Shape* p = shape_; + // @@protoc_insertion_point(field_get:xla.HloInstructionProto.shape) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_Shape_default_instance_); +} +inline ::xla::Shape* HloInstructionProto::release_shape() { + // @@protoc_insertion_point(field_release:xla.HloInstructionProto.shape) + + ::xla::Shape* temp = shape_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + shape_ = NULL; + return temp; +} +inline ::xla::Shape* HloInstructionProto::unsafe_arena_release_shape() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.HloInstructionProto.shape) + + ::xla::Shape* temp = shape_; + shape_ = NULL; + return temp; +} +inline ::xla::Shape* HloInstructionProto::mutable_shape() { + + if (shape_ == NULL) { + auto* p = CreateMaybeMessage<::xla::Shape>(GetArenaNoVirtual()); + shape_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.HloInstructionProto.shape) + return shape_; +} +inline void HloInstructionProto::set_allocated_shape(::xla::Shape* shape) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(shape_); + } + if (shape) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(shape)->GetArena(); + if (message_arena != submessage_arena) { + shape = ::google::protobuf::internal::GetOwnedMessage( + message_arena, shape, submessage_arena); + } + + } else { + + } + shape_ = shape; + // @@protoc_insertion_point(field_set_allocated:xla.HloInstructionProto.shape) +} + +// .xla.OpMetadata metadata = 7; +inline bool HloInstructionProto::has_metadata() const { + return this != internal_default_instance() && metadata_ != NULL; +} +inline const ::xla::OpMetadata& HloInstructionProto::_internal_metadata() const { + return *metadata_; +} +inline const ::xla::OpMetadata& HloInstructionProto::metadata() const { + const ::xla::OpMetadata* p = metadata_; + // @@protoc_insertion_point(field_get:xla.HloInstructionProto.metadata) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_OpMetadata_default_instance_); +} +inline ::xla::OpMetadata* HloInstructionProto::release_metadata() { + // @@protoc_insertion_point(field_release:xla.HloInstructionProto.metadata) + + ::xla::OpMetadata* temp = metadata_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + metadata_ = NULL; + return temp; +} +inline ::xla::OpMetadata* HloInstructionProto::unsafe_arena_release_metadata() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.HloInstructionProto.metadata) + + ::xla::OpMetadata* temp = metadata_; + metadata_ = NULL; + return temp; +} +inline ::xla::OpMetadata* HloInstructionProto::mutable_metadata() { + + if (metadata_ == NULL) { + auto* p = CreateMaybeMessage<::xla::OpMetadata>(GetArenaNoVirtual()); + metadata_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.HloInstructionProto.metadata) + return metadata_; +} +inline void HloInstructionProto::set_allocated_metadata(::xla::OpMetadata* metadata) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(metadata_); + } + if (metadata) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(metadata)->GetArena(); + if (message_arena != submessage_arena) { + metadata = ::google::protobuf::internal::GetOwnedMessage( + message_arena, metadata, submessage_arena); + } + + } else { + + } + metadata_ = metadata; + // @@protoc_insertion_point(field_set_allocated:xla.HloInstructionProto.metadata) +} + +// .xla.LiteralProto literal = 8; +inline bool HloInstructionProto::has_literal() const { + return this != internal_default_instance() && literal_ != NULL; +} +inline const ::xla::LiteralProto& HloInstructionProto::_internal_literal() const { + return *literal_; +} +inline const ::xla::LiteralProto& HloInstructionProto::literal() const { + const ::xla::LiteralProto* p = literal_; + // @@protoc_insertion_point(field_get:xla.HloInstructionProto.literal) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_LiteralProto_default_instance_); +} +inline ::xla::LiteralProto* HloInstructionProto::release_literal() { + // @@protoc_insertion_point(field_release:xla.HloInstructionProto.literal) + + ::xla::LiteralProto* temp = literal_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + literal_ = NULL; + return temp; +} +inline ::xla::LiteralProto* HloInstructionProto::unsafe_arena_release_literal() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.HloInstructionProto.literal) + + ::xla::LiteralProto* temp = literal_; + literal_ = NULL; + return temp; +} +inline ::xla::LiteralProto* HloInstructionProto::mutable_literal() { + + if (literal_ == NULL) { + auto* p = CreateMaybeMessage<::xla::LiteralProto>(GetArenaNoVirtual()); + literal_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.HloInstructionProto.literal) + return literal_; +} +inline void HloInstructionProto::set_allocated_literal(::xla::LiteralProto* literal) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(literal_); + } + if (literal) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(literal)->GetArena(); + if (message_arena != submessage_arena) { + literal = ::google::protobuf::internal::GetOwnedMessage( + message_arena, literal, submessage_arena); + } + + } else { + + } + literal_ = literal; + // @@protoc_insertion_point(field_set_allocated:xla.HloInstructionProto.literal) +} + +// int64 parameter_number = 9; +inline void HloInstructionProto::clear_parameter_number() { + parameter_number_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 HloInstructionProto::parameter_number() const { + // @@protoc_insertion_point(field_get:xla.HloInstructionProto.parameter_number) + return parameter_number_; +} +inline void HloInstructionProto::set_parameter_number(::google::protobuf::int64 value) { + + parameter_number_ = value; + // @@protoc_insertion_point(field_set:xla.HloInstructionProto.parameter_number) +} + +// string fusion_kind = 11; +inline void HloInstructionProto::clear_fusion_kind() { + fusion_kind_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& HloInstructionProto::fusion_kind() const { + // @@protoc_insertion_point(field_get:xla.HloInstructionProto.fusion_kind) + return fusion_kind_.Get(); +} +inline void HloInstructionProto::set_fusion_kind(const ::std::string& value) { + + fusion_kind_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:xla.HloInstructionProto.fusion_kind) +} +#if LANG_CXX11 +inline void HloInstructionProto::set_fusion_kind(::std::string&& value) { + + fusion_kind_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:xla.HloInstructionProto.fusion_kind) +} +#endif +inline void HloInstructionProto::set_fusion_kind(const char* value) { + GOOGLE_DCHECK(value != NULL); + + fusion_kind_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:xla.HloInstructionProto.fusion_kind) +} +inline void HloInstructionProto::set_fusion_kind(const char* value, + size_t size) { + + fusion_kind_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:xla.HloInstructionProto.fusion_kind) +} +inline ::std::string* HloInstructionProto::mutable_fusion_kind() { + + // @@protoc_insertion_point(field_mutable:xla.HloInstructionProto.fusion_kind) + return fusion_kind_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* HloInstructionProto::release_fusion_kind() { + // @@protoc_insertion_point(field_release:xla.HloInstructionProto.fusion_kind) + + return fusion_kind_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void HloInstructionProto::set_allocated_fusion_kind(::std::string* fusion_kind) { + if (fusion_kind != NULL) { + + } else { + + } + fusion_kind_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), fusion_kind, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:xla.HloInstructionProto.fusion_kind) +} +inline ::std::string* HloInstructionProto::unsafe_arena_release_fusion_kind() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.HloInstructionProto.fusion_kind) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return fusion_kind_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void HloInstructionProto::unsafe_arena_set_allocated_fusion_kind( + ::std::string* fusion_kind) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (fusion_kind != NULL) { + + } else { + + } + fusion_kind_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + fusion_kind, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:xla.HloInstructionProto.fusion_kind) +} + +// int64 tuple_index = 13; +inline void HloInstructionProto::clear_tuple_index() { + tuple_index_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 HloInstructionProto::tuple_index() const { + // @@protoc_insertion_point(field_get:xla.HloInstructionProto.tuple_index) + return tuple_index_; +} +inline void HloInstructionProto::set_tuple_index(::google::protobuf::int64 value) { + + tuple_index_ = value; + // @@protoc_insertion_point(field_set:xla.HloInstructionProto.tuple_index) +} + +// repeated int64 dimensions = 14; +inline int HloInstructionProto::dimensions_size() const { + return dimensions_.size(); +} +inline void HloInstructionProto::clear_dimensions() { + dimensions_.Clear(); +} +inline ::google::protobuf::int64 HloInstructionProto::dimensions(int index) const { + // @@protoc_insertion_point(field_get:xla.HloInstructionProto.dimensions) + return dimensions_.Get(index); +} +inline void HloInstructionProto::set_dimensions(int index, ::google::protobuf::int64 value) { + dimensions_.Set(index, value); + // @@protoc_insertion_point(field_set:xla.HloInstructionProto.dimensions) +} +inline void HloInstructionProto::add_dimensions(::google::protobuf::int64 value) { + dimensions_.Add(value); + // @@protoc_insertion_point(field_add:xla.HloInstructionProto.dimensions) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& +HloInstructionProto::dimensions() const { + // @@protoc_insertion_point(field_list:xla.HloInstructionProto.dimensions) + return dimensions_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* +HloInstructionProto::mutable_dimensions() { + // @@protoc_insertion_point(field_mutable_list:xla.HloInstructionProto.dimensions) + return &dimensions_; +} + +// .xla.Window window = 15; +inline bool HloInstructionProto::has_window() const { + return this != internal_default_instance() && window_ != NULL; +} +inline const ::xla::Window& HloInstructionProto::_internal_window() const { + return *window_; +} +inline const ::xla::Window& HloInstructionProto::window() const { + const ::xla::Window* p = window_; + // @@protoc_insertion_point(field_get:xla.HloInstructionProto.window) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_Window_default_instance_); +} +inline ::xla::Window* HloInstructionProto::release_window() { + // @@protoc_insertion_point(field_release:xla.HloInstructionProto.window) + + ::xla::Window* temp = window_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + window_ = NULL; + return temp; +} +inline ::xla::Window* HloInstructionProto::unsafe_arena_release_window() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.HloInstructionProto.window) + + ::xla::Window* temp = window_; + window_ = NULL; + return temp; +} +inline ::xla::Window* HloInstructionProto::mutable_window() { + + if (window_ == NULL) { + auto* p = CreateMaybeMessage<::xla::Window>(GetArenaNoVirtual()); + window_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.HloInstructionProto.window) + return window_; +} +inline void HloInstructionProto::set_allocated_window(::xla::Window* window) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(window_); + } + if (window) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(window)->GetArena(); + if (message_arena != submessage_arena) { + window = ::google::protobuf::internal::GetOwnedMessage( + message_arena, window, submessage_arena); + } + + } else { + + } + window_ = window; + // @@protoc_insertion_point(field_set_allocated:xla.HloInstructionProto.window) +} + +// .xla.ConvolutionDimensionNumbers convolution_dimension_numbers = 16; +inline bool HloInstructionProto::has_convolution_dimension_numbers() const { + return this != internal_default_instance() && convolution_dimension_numbers_ != NULL; +} +inline const ::xla::ConvolutionDimensionNumbers& HloInstructionProto::_internal_convolution_dimension_numbers() const { + return *convolution_dimension_numbers_; +} +inline const ::xla::ConvolutionDimensionNumbers& HloInstructionProto::convolution_dimension_numbers() const { + const ::xla::ConvolutionDimensionNumbers* p = convolution_dimension_numbers_; + // @@protoc_insertion_point(field_get:xla.HloInstructionProto.convolution_dimension_numbers) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_ConvolutionDimensionNumbers_default_instance_); +} +inline ::xla::ConvolutionDimensionNumbers* HloInstructionProto::release_convolution_dimension_numbers() { + // @@protoc_insertion_point(field_release:xla.HloInstructionProto.convolution_dimension_numbers) + + ::xla::ConvolutionDimensionNumbers* temp = convolution_dimension_numbers_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + convolution_dimension_numbers_ = NULL; + return temp; +} +inline ::xla::ConvolutionDimensionNumbers* HloInstructionProto::unsafe_arena_release_convolution_dimension_numbers() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.HloInstructionProto.convolution_dimension_numbers) + + ::xla::ConvolutionDimensionNumbers* temp = convolution_dimension_numbers_; + convolution_dimension_numbers_ = NULL; + return temp; +} +inline ::xla::ConvolutionDimensionNumbers* HloInstructionProto::mutable_convolution_dimension_numbers() { + + if (convolution_dimension_numbers_ == NULL) { + auto* p = CreateMaybeMessage<::xla::ConvolutionDimensionNumbers>(GetArenaNoVirtual()); + convolution_dimension_numbers_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.HloInstructionProto.convolution_dimension_numbers) + return convolution_dimension_numbers_; +} +inline void HloInstructionProto::set_allocated_convolution_dimension_numbers(::xla::ConvolutionDimensionNumbers* convolution_dimension_numbers) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(convolution_dimension_numbers_); + } + if (convolution_dimension_numbers) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(convolution_dimension_numbers)->GetArena(); + if (message_arena != submessage_arena) { + convolution_dimension_numbers = ::google::protobuf::internal::GetOwnedMessage( + message_arena, convolution_dimension_numbers, submessage_arena); + } + + } else { + + } + convolution_dimension_numbers_ = convolution_dimension_numbers; + // @@protoc_insertion_point(field_set_allocated:xla.HloInstructionProto.convolution_dimension_numbers) +} + +// int64 feature_group_count = 50; +inline void HloInstructionProto::clear_feature_group_count() { + feature_group_count_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 HloInstructionProto::feature_group_count() const { + // @@protoc_insertion_point(field_get:xla.HloInstructionProto.feature_group_count) + return feature_group_count_; +} +inline void HloInstructionProto::set_feature_group_count(::google::protobuf::int64 value) { + + feature_group_count_ = value; + // @@protoc_insertion_point(field_set:xla.HloInstructionProto.feature_group_count) +} + +// repeated .xla.HloInstructionProto.SliceDimensions slice_dimensions = 17; +inline int HloInstructionProto::slice_dimensions_size() const { + return slice_dimensions_.size(); +} +inline void HloInstructionProto::clear_slice_dimensions() { + slice_dimensions_.Clear(); +} +inline ::xla::HloInstructionProto_SliceDimensions* HloInstructionProto::mutable_slice_dimensions(int index) { + // @@protoc_insertion_point(field_mutable:xla.HloInstructionProto.slice_dimensions) + return slice_dimensions_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::xla::HloInstructionProto_SliceDimensions >* +HloInstructionProto::mutable_slice_dimensions() { + // @@protoc_insertion_point(field_mutable_list:xla.HloInstructionProto.slice_dimensions) + return &slice_dimensions_; +} +inline const ::xla::HloInstructionProto_SliceDimensions& HloInstructionProto::slice_dimensions(int index) const { + // @@protoc_insertion_point(field_get:xla.HloInstructionProto.slice_dimensions) + return slice_dimensions_.Get(index); +} +inline ::xla::HloInstructionProto_SliceDimensions* HloInstructionProto::add_slice_dimensions() { + // @@protoc_insertion_point(field_add:xla.HloInstructionProto.slice_dimensions) + return slice_dimensions_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::xla::HloInstructionProto_SliceDimensions >& +HloInstructionProto::slice_dimensions() const { + // @@protoc_insertion_point(field_list:xla.HloInstructionProto.slice_dimensions) + return slice_dimensions_; +} + +// int32 exponent_bits = 18; +inline void HloInstructionProto::clear_exponent_bits() { + exponent_bits_ = 0; +} +inline ::google::protobuf::int32 HloInstructionProto::exponent_bits() const { + // @@protoc_insertion_point(field_get:xla.HloInstructionProto.exponent_bits) + return exponent_bits_; +} +inline void HloInstructionProto::set_exponent_bits(::google::protobuf::int32 value) { + + exponent_bits_ = value; + // @@protoc_insertion_point(field_set:xla.HloInstructionProto.exponent_bits) +} + +// int32 mantissa_bits = 19; +inline void HloInstructionProto::clear_mantissa_bits() { + mantissa_bits_ = 0; +} +inline ::google::protobuf::int32 HloInstructionProto::mantissa_bits() const { + // @@protoc_insertion_point(field_get:xla.HloInstructionProto.mantissa_bits) + return mantissa_bits_; +} +inline void HloInstructionProto::set_mantissa_bits(::google::protobuf::int32 value) { + + mantissa_bits_ = value; + // @@protoc_insertion_point(field_set:xla.HloInstructionProto.mantissa_bits) +} + +// repeated int64 dynamic_slice_sizes = 20; +inline int HloInstructionProto::dynamic_slice_sizes_size() const { + return dynamic_slice_sizes_.size(); +} +inline void HloInstructionProto::clear_dynamic_slice_sizes() { + dynamic_slice_sizes_.Clear(); +} +inline ::google::protobuf::int64 HloInstructionProto::dynamic_slice_sizes(int index) const { + // @@protoc_insertion_point(field_get:xla.HloInstructionProto.dynamic_slice_sizes) + return dynamic_slice_sizes_.Get(index); +} +inline void HloInstructionProto::set_dynamic_slice_sizes(int index, ::google::protobuf::int64 value) { + dynamic_slice_sizes_.Set(index, value); + // @@protoc_insertion_point(field_set:xla.HloInstructionProto.dynamic_slice_sizes) +} +inline void HloInstructionProto::add_dynamic_slice_sizes(::google::protobuf::int64 value) { + dynamic_slice_sizes_.Add(value); + // @@protoc_insertion_point(field_add:xla.HloInstructionProto.dynamic_slice_sizes) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& +HloInstructionProto::dynamic_slice_sizes() const { + // @@protoc_insertion_point(field_list:xla.HloInstructionProto.dynamic_slice_sizes) + return dynamic_slice_sizes_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* +HloInstructionProto::mutable_dynamic_slice_sizes() { + // @@protoc_insertion_point(field_mutable_list:xla.HloInstructionProto.dynamic_slice_sizes) + return &dynamic_slice_sizes_; +} + +// .xla.PaddingConfig padding_config = 21; +inline bool HloInstructionProto::has_padding_config() const { + return this != internal_default_instance() && padding_config_ != NULL; +} +inline const ::xla::PaddingConfig& HloInstructionProto::_internal_padding_config() const { + return *padding_config_; +} +inline const ::xla::PaddingConfig& HloInstructionProto::padding_config() const { + const ::xla::PaddingConfig* p = padding_config_; + // @@protoc_insertion_point(field_get:xla.HloInstructionProto.padding_config) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_PaddingConfig_default_instance_); +} +inline ::xla::PaddingConfig* HloInstructionProto::release_padding_config() { + // @@protoc_insertion_point(field_release:xla.HloInstructionProto.padding_config) + + ::xla::PaddingConfig* temp = padding_config_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + padding_config_ = NULL; + return temp; +} +inline ::xla::PaddingConfig* HloInstructionProto::unsafe_arena_release_padding_config() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.HloInstructionProto.padding_config) + + ::xla::PaddingConfig* temp = padding_config_; + padding_config_ = NULL; + return temp; +} +inline ::xla::PaddingConfig* HloInstructionProto::mutable_padding_config() { + + if (padding_config_ == NULL) { + auto* p = CreateMaybeMessage<::xla::PaddingConfig>(GetArenaNoVirtual()); + padding_config_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.HloInstructionProto.padding_config) + return padding_config_; +} +inline void HloInstructionProto::set_allocated_padding_config(::xla::PaddingConfig* padding_config) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(padding_config_); + } + if (padding_config) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(padding_config)->GetArena(); + if (message_arena != submessage_arena) { + padding_config = ::google::protobuf::internal::GetOwnedMessage( + message_arena, padding_config, submessage_arena); + } + + } else { + + } + padding_config_ = padding_config; + // @@protoc_insertion_point(field_set_allocated:xla.HloInstructionProto.padding_config) +} + +// bytes outfeed_config = 22; +inline void HloInstructionProto::clear_outfeed_config() { + outfeed_config_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& HloInstructionProto::outfeed_config() const { + // @@protoc_insertion_point(field_get:xla.HloInstructionProto.outfeed_config) + return outfeed_config_.Get(); +} +inline void HloInstructionProto::set_outfeed_config(const ::std::string& value) { + + outfeed_config_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:xla.HloInstructionProto.outfeed_config) +} +#if LANG_CXX11 +inline void HloInstructionProto::set_outfeed_config(::std::string&& value) { + + outfeed_config_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:xla.HloInstructionProto.outfeed_config) +} +#endif +inline void HloInstructionProto::set_outfeed_config(const char* value) { + GOOGLE_DCHECK(value != NULL); + + outfeed_config_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:xla.HloInstructionProto.outfeed_config) +} +inline void HloInstructionProto::set_outfeed_config(const void* value, + size_t size) { + + outfeed_config_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:xla.HloInstructionProto.outfeed_config) +} +inline ::std::string* HloInstructionProto::mutable_outfeed_config() { + + // @@protoc_insertion_point(field_mutable:xla.HloInstructionProto.outfeed_config) + return outfeed_config_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* HloInstructionProto::release_outfeed_config() { + // @@protoc_insertion_point(field_release:xla.HloInstructionProto.outfeed_config) + + return outfeed_config_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void HloInstructionProto::set_allocated_outfeed_config(::std::string* outfeed_config) { + if (outfeed_config != NULL) { + + } else { + + } + outfeed_config_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), outfeed_config, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:xla.HloInstructionProto.outfeed_config) +} +inline ::std::string* HloInstructionProto::unsafe_arena_release_outfeed_config() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.HloInstructionProto.outfeed_config) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return outfeed_config_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void HloInstructionProto::unsafe_arena_set_allocated_outfeed_config( + ::std::string* outfeed_config) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (outfeed_config != NULL) { + + } else { + + } + outfeed_config_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + outfeed_config, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:xla.HloInstructionProto.outfeed_config) +} + +// .xla.RandomDistribution distribution = 23; +inline void HloInstructionProto::clear_distribution() { + distribution_ = 0; +} +inline ::xla::RandomDistribution HloInstructionProto::distribution() const { + // @@protoc_insertion_point(field_get:xla.HloInstructionProto.distribution) + return static_cast< ::xla::RandomDistribution >(distribution_); +} +inline void HloInstructionProto::set_distribution(::xla::RandomDistribution value) { + + distribution_ = value; + // @@protoc_insertion_point(field_set:xla.HloInstructionProto.distribution) +} + +// float epsilon = 24; +inline void HloInstructionProto::clear_epsilon() { + epsilon_ = 0; +} +inline float HloInstructionProto::epsilon() const { + // @@protoc_insertion_point(field_get:xla.HloInstructionProto.epsilon) + return epsilon_; +} +inline void HloInstructionProto::set_epsilon(float value) { + + epsilon_ = value; + // @@protoc_insertion_point(field_set:xla.HloInstructionProto.epsilon) +} + +// int64 feature_index = 25; +inline void HloInstructionProto::clear_feature_index() { + feature_index_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 HloInstructionProto::feature_index() const { + // @@protoc_insertion_point(field_get:xla.HloInstructionProto.feature_index) + return feature_index_; +} +inline void HloInstructionProto::set_feature_index(::google::protobuf::int64 value) { + + feature_index_ = value; + // @@protoc_insertion_point(field_set:xla.HloInstructionProto.feature_index) +} + +// int64 channel_id = 26; +inline void HloInstructionProto::clear_channel_id() { + channel_id_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 HloInstructionProto::channel_id() const { + // @@protoc_insertion_point(field_get:xla.HloInstructionProto.channel_id) + return channel_id_; +} +inline void HloInstructionProto::set_channel_id(::google::protobuf::int64 value) { + + channel_id_ = value; + // @@protoc_insertion_point(field_set:xla.HloInstructionProto.channel_id) +} + +// bytes infeed_config = 27; +inline void HloInstructionProto::clear_infeed_config() { + infeed_config_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& HloInstructionProto::infeed_config() const { + // @@protoc_insertion_point(field_get:xla.HloInstructionProto.infeed_config) + return infeed_config_.Get(); +} +inline void HloInstructionProto::set_infeed_config(const ::std::string& value) { + + infeed_config_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:xla.HloInstructionProto.infeed_config) +} +#if LANG_CXX11 +inline void HloInstructionProto::set_infeed_config(::std::string&& value) { + + infeed_config_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:xla.HloInstructionProto.infeed_config) +} +#endif +inline void HloInstructionProto::set_infeed_config(const char* value) { + GOOGLE_DCHECK(value != NULL); + + infeed_config_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:xla.HloInstructionProto.infeed_config) +} +inline void HloInstructionProto::set_infeed_config(const void* value, + size_t size) { + + infeed_config_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:xla.HloInstructionProto.infeed_config) +} +inline ::std::string* HloInstructionProto::mutable_infeed_config() { + + // @@protoc_insertion_point(field_mutable:xla.HloInstructionProto.infeed_config) + return infeed_config_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* HloInstructionProto::release_infeed_config() { + // @@protoc_insertion_point(field_release:xla.HloInstructionProto.infeed_config) + + return infeed_config_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void HloInstructionProto::set_allocated_infeed_config(::std::string* infeed_config) { + if (infeed_config != NULL) { + + } else { + + } + infeed_config_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), infeed_config, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:xla.HloInstructionProto.infeed_config) +} +inline ::std::string* HloInstructionProto::unsafe_arena_release_infeed_config() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.HloInstructionProto.infeed_config) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return infeed_config_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void HloInstructionProto::unsafe_arena_set_allocated_infeed_config( + ::std::string* infeed_config) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (infeed_config != NULL) { + + } else { + + } + infeed_config_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + infeed_config, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:xla.HloInstructionProto.infeed_config) +} + +// string custom_call_target = 28; +inline void HloInstructionProto::clear_custom_call_target() { + custom_call_target_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& HloInstructionProto::custom_call_target() const { + // @@protoc_insertion_point(field_get:xla.HloInstructionProto.custom_call_target) + return custom_call_target_.Get(); +} +inline void HloInstructionProto::set_custom_call_target(const ::std::string& value) { + + custom_call_target_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:xla.HloInstructionProto.custom_call_target) +} +#if LANG_CXX11 +inline void HloInstructionProto::set_custom_call_target(::std::string&& value) { + + custom_call_target_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:xla.HloInstructionProto.custom_call_target) +} +#endif +inline void HloInstructionProto::set_custom_call_target(const char* value) { + GOOGLE_DCHECK(value != NULL); + + custom_call_target_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:xla.HloInstructionProto.custom_call_target) +} +inline void HloInstructionProto::set_custom_call_target(const char* value, + size_t size) { + + custom_call_target_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:xla.HloInstructionProto.custom_call_target) +} +inline ::std::string* HloInstructionProto::mutable_custom_call_target() { + + // @@protoc_insertion_point(field_mutable:xla.HloInstructionProto.custom_call_target) + return custom_call_target_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* HloInstructionProto::release_custom_call_target() { + // @@protoc_insertion_point(field_release:xla.HloInstructionProto.custom_call_target) + + return custom_call_target_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void HloInstructionProto::set_allocated_custom_call_target(::std::string* custom_call_target) { + if (custom_call_target != NULL) { + + } else { + + } + custom_call_target_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), custom_call_target, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:xla.HloInstructionProto.custom_call_target) +} +inline ::std::string* HloInstructionProto::unsafe_arena_release_custom_call_target() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.HloInstructionProto.custom_call_target) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return custom_call_target_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void HloInstructionProto::unsafe_arena_set_allocated_custom_call_target( + ::std::string* custom_call_target) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (custom_call_target != NULL) { + + } else { + + } + custom_call_target_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + custom_call_target, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:xla.HloInstructionProto.custom_call_target) +} + +// string custom_call_opaque = 53; +inline void HloInstructionProto::clear_custom_call_opaque() { + custom_call_opaque_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& HloInstructionProto::custom_call_opaque() const { + // @@protoc_insertion_point(field_get:xla.HloInstructionProto.custom_call_opaque) + return custom_call_opaque_.Get(); +} +inline void HloInstructionProto::set_custom_call_opaque(const ::std::string& value) { + + custom_call_opaque_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:xla.HloInstructionProto.custom_call_opaque) +} +#if LANG_CXX11 +inline void HloInstructionProto::set_custom_call_opaque(::std::string&& value) { + + custom_call_opaque_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:xla.HloInstructionProto.custom_call_opaque) +} +#endif +inline void HloInstructionProto::set_custom_call_opaque(const char* value) { + GOOGLE_DCHECK(value != NULL); + + custom_call_opaque_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:xla.HloInstructionProto.custom_call_opaque) +} +inline void HloInstructionProto::set_custom_call_opaque(const char* value, + size_t size) { + + custom_call_opaque_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:xla.HloInstructionProto.custom_call_opaque) +} +inline ::std::string* HloInstructionProto::mutable_custom_call_opaque() { + + // @@protoc_insertion_point(field_mutable:xla.HloInstructionProto.custom_call_opaque) + return custom_call_opaque_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* HloInstructionProto::release_custom_call_opaque() { + // @@protoc_insertion_point(field_release:xla.HloInstructionProto.custom_call_opaque) + + return custom_call_opaque_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void HloInstructionProto::set_allocated_custom_call_opaque(::std::string* custom_call_opaque) { + if (custom_call_opaque != NULL) { + + } else { + + } + custom_call_opaque_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), custom_call_opaque, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:xla.HloInstructionProto.custom_call_opaque) +} +inline ::std::string* HloInstructionProto::unsafe_arena_release_custom_call_opaque() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.HloInstructionProto.custom_call_opaque) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return custom_call_opaque_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void HloInstructionProto::unsafe_arena_set_allocated_custom_call_opaque( + ::std::string* custom_call_opaque) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (custom_call_opaque != NULL) { + + } else { + + } + custom_call_opaque_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + custom_call_opaque, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:xla.HloInstructionProto.custom_call_opaque) +} + +// .xla.Shape outfeed_shape = 29; +inline bool HloInstructionProto::has_outfeed_shape() const { + return this != internal_default_instance() && outfeed_shape_ != NULL; +} +inline const ::xla::Shape& HloInstructionProto::_internal_outfeed_shape() const { + return *outfeed_shape_; +} +inline const ::xla::Shape& HloInstructionProto::outfeed_shape() const { + const ::xla::Shape* p = outfeed_shape_; + // @@protoc_insertion_point(field_get:xla.HloInstructionProto.outfeed_shape) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_Shape_default_instance_); +} +inline ::xla::Shape* HloInstructionProto::release_outfeed_shape() { + // @@protoc_insertion_point(field_release:xla.HloInstructionProto.outfeed_shape) + + ::xla::Shape* temp = outfeed_shape_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + outfeed_shape_ = NULL; + return temp; +} +inline ::xla::Shape* HloInstructionProto::unsafe_arena_release_outfeed_shape() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.HloInstructionProto.outfeed_shape) + + ::xla::Shape* temp = outfeed_shape_; + outfeed_shape_ = NULL; + return temp; +} +inline ::xla::Shape* HloInstructionProto::mutable_outfeed_shape() { + + if (outfeed_shape_ == NULL) { + auto* p = CreateMaybeMessage<::xla::Shape>(GetArenaNoVirtual()); + outfeed_shape_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.HloInstructionProto.outfeed_shape) + return outfeed_shape_; +} +inline void HloInstructionProto::set_allocated_outfeed_shape(::xla::Shape* outfeed_shape) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(outfeed_shape_); + } + if (outfeed_shape) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(outfeed_shape)->GetArena(); + if (message_arena != submessage_arena) { + outfeed_shape = ::google::protobuf::internal::GetOwnedMessage( + message_arena, outfeed_shape, submessage_arena); + } + + } else { + + } + outfeed_shape_ = outfeed_shape; + // @@protoc_insertion_point(field_set_allocated:xla.HloInstructionProto.outfeed_shape) +} + +// .xla.DotDimensionNumbers dot_dimension_numbers = 30; +inline bool HloInstructionProto::has_dot_dimension_numbers() const { + return this != internal_default_instance() && dot_dimension_numbers_ != NULL; +} +inline const ::xla::DotDimensionNumbers& HloInstructionProto::_internal_dot_dimension_numbers() const { + return *dot_dimension_numbers_; +} +inline const ::xla::DotDimensionNumbers& HloInstructionProto::dot_dimension_numbers() const { + const ::xla::DotDimensionNumbers* p = dot_dimension_numbers_; + // @@protoc_insertion_point(field_get:xla.HloInstructionProto.dot_dimension_numbers) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_DotDimensionNumbers_default_instance_); +} +inline ::xla::DotDimensionNumbers* HloInstructionProto::release_dot_dimension_numbers() { + // @@protoc_insertion_point(field_release:xla.HloInstructionProto.dot_dimension_numbers) + + ::xla::DotDimensionNumbers* temp = dot_dimension_numbers_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + dot_dimension_numbers_ = NULL; + return temp; +} +inline ::xla::DotDimensionNumbers* HloInstructionProto::unsafe_arena_release_dot_dimension_numbers() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.HloInstructionProto.dot_dimension_numbers) + + ::xla::DotDimensionNumbers* temp = dot_dimension_numbers_; + dot_dimension_numbers_ = NULL; + return temp; +} +inline ::xla::DotDimensionNumbers* HloInstructionProto::mutable_dot_dimension_numbers() { + + if (dot_dimension_numbers_ == NULL) { + auto* p = CreateMaybeMessage<::xla::DotDimensionNumbers>(GetArenaNoVirtual()); + dot_dimension_numbers_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.HloInstructionProto.dot_dimension_numbers) + return dot_dimension_numbers_; +} +inline void HloInstructionProto::set_allocated_dot_dimension_numbers(::xla::DotDimensionNumbers* dot_dimension_numbers) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(dot_dimension_numbers_); + } + if (dot_dimension_numbers) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(dot_dimension_numbers)->GetArena(); + if (message_arena != submessage_arena) { + dot_dimension_numbers = ::google::protobuf::internal::GetOwnedMessage( + message_arena, dot_dimension_numbers, submessage_arena); + } + + } else { + + } + dot_dimension_numbers_ = dot_dimension_numbers; + // @@protoc_insertion_point(field_set_allocated:xla.HloInstructionProto.dot_dimension_numbers) +} + +// .xla.FftType fft_type = 31; +inline void HloInstructionProto::clear_fft_type() { + fft_type_ = 0; +} +inline ::xla::FftType HloInstructionProto::fft_type() const { + // @@protoc_insertion_point(field_get:xla.HloInstructionProto.fft_type) + return static_cast< ::xla::FftType >(fft_type_); +} +inline void HloInstructionProto::set_fft_type(::xla::FftType value) { + + fft_type_ = value; + // @@protoc_insertion_point(field_set:xla.HloInstructionProto.fft_type) +} + +// repeated int64 fft_length = 32; +inline int HloInstructionProto::fft_length_size() const { + return fft_length_.size(); +} +inline void HloInstructionProto::clear_fft_length() { + fft_length_.Clear(); +} +inline ::google::protobuf::int64 HloInstructionProto::fft_length(int index) const { + // @@protoc_insertion_point(field_get:xla.HloInstructionProto.fft_length) + return fft_length_.Get(index); +} +inline void HloInstructionProto::set_fft_length(int index, ::google::protobuf::int64 value) { + fft_length_.Set(index, value); + // @@protoc_insertion_point(field_set:xla.HloInstructionProto.fft_length) +} +inline void HloInstructionProto::add_fft_length(::google::protobuf::int64 value) { + fft_length_.Add(value); + // @@protoc_insertion_point(field_add:xla.HloInstructionProto.fft_length) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& +HloInstructionProto::fft_length() const { + // @@protoc_insertion_point(field_list:xla.HloInstructionProto.fft_length) + return fft_length_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* +HloInstructionProto::mutable_fft_length() { + // @@protoc_insertion_point(field_mutable_list:xla.HloInstructionProto.fft_length) + return &fft_length_; +} + +// .xla.GatherDimensionNumbers gather_dimension_numbers = 33; +inline bool HloInstructionProto::has_gather_dimension_numbers() const { + return this != internal_default_instance() && gather_dimension_numbers_ != NULL; +} +inline const ::xla::GatherDimensionNumbers& HloInstructionProto::_internal_gather_dimension_numbers() const { + return *gather_dimension_numbers_; +} +inline const ::xla::GatherDimensionNumbers& HloInstructionProto::gather_dimension_numbers() const { + const ::xla::GatherDimensionNumbers* p = gather_dimension_numbers_; + // @@protoc_insertion_point(field_get:xla.HloInstructionProto.gather_dimension_numbers) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_GatherDimensionNumbers_default_instance_); +} +inline ::xla::GatherDimensionNumbers* HloInstructionProto::release_gather_dimension_numbers() { + // @@protoc_insertion_point(field_release:xla.HloInstructionProto.gather_dimension_numbers) + + ::xla::GatherDimensionNumbers* temp = gather_dimension_numbers_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + gather_dimension_numbers_ = NULL; + return temp; +} +inline ::xla::GatherDimensionNumbers* HloInstructionProto::unsafe_arena_release_gather_dimension_numbers() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.HloInstructionProto.gather_dimension_numbers) + + ::xla::GatherDimensionNumbers* temp = gather_dimension_numbers_; + gather_dimension_numbers_ = NULL; + return temp; +} +inline ::xla::GatherDimensionNumbers* HloInstructionProto::mutable_gather_dimension_numbers() { + + if (gather_dimension_numbers_ == NULL) { + auto* p = CreateMaybeMessage<::xla::GatherDimensionNumbers>(GetArenaNoVirtual()); + gather_dimension_numbers_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.HloInstructionProto.gather_dimension_numbers) + return gather_dimension_numbers_; +} +inline void HloInstructionProto::set_allocated_gather_dimension_numbers(::xla::GatherDimensionNumbers* gather_dimension_numbers) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(gather_dimension_numbers_); + } + if (gather_dimension_numbers) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(gather_dimension_numbers)->GetArena(); + if (message_arena != submessage_arena) { + gather_dimension_numbers = ::google::protobuf::internal::GetOwnedMessage( + message_arena, gather_dimension_numbers, submessage_arena); + } + + } else { + + } + gather_dimension_numbers_ = gather_dimension_numbers; + // @@protoc_insertion_point(field_set_allocated:xla.HloInstructionProto.gather_dimension_numbers) +} + +// repeated int64 gather_slice_sizes = 34; +inline int HloInstructionProto::gather_slice_sizes_size() const { + return gather_slice_sizes_.size(); +} +inline void HloInstructionProto::clear_gather_slice_sizes() { + gather_slice_sizes_.Clear(); +} +inline ::google::protobuf::int64 HloInstructionProto::gather_slice_sizes(int index) const { + // @@protoc_insertion_point(field_get:xla.HloInstructionProto.gather_slice_sizes) + return gather_slice_sizes_.Get(index); +} +inline void HloInstructionProto::set_gather_slice_sizes(int index, ::google::protobuf::int64 value) { + gather_slice_sizes_.Set(index, value); + // @@protoc_insertion_point(field_set:xla.HloInstructionProto.gather_slice_sizes) +} +inline void HloInstructionProto::add_gather_slice_sizes(::google::protobuf::int64 value) { + gather_slice_sizes_.Add(value); + // @@protoc_insertion_point(field_add:xla.HloInstructionProto.gather_slice_sizes) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& +HloInstructionProto::gather_slice_sizes() const { + // @@protoc_insertion_point(field_list:xla.HloInstructionProto.gather_slice_sizes) + return gather_slice_sizes_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* +HloInstructionProto::mutable_gather_slice_sizes() { + // @@protoc_insertion_point(field_mutable_list:xla.HloInstructionProto.gather_slice_sizes) + return &gather_slice_sizes_; +} + +// string channel_name = 41; +inline void HloInstructionProto::clear_channel_name() { + channel_name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& HloInstructionProto::channel_name() const { + // @@protoc_insertion_point(field_get:xla.HloInstructionProto.channel_name) + return channel_name_.Get(); +} +inline void HloInstructionProto::set_channel_name(const ::std::string& value) { + + channel_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:xla.HloInstructionProto.channel_name) +} +#if LANG_CXX11 +inline void HloInstructionProto::set_channel_name(::std::string&& value) { + + channel_name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:xla.HloInstructionProto.channel_name) +} +#endif +inline void HloInstructionProto::set_channel_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + channel_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:xla.HloInstructionProto.channel_name) +} +inline void HloInstructionProto::set_channel_name(const char* value, + size_t size) { + + channel_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:xla.HloInstructionProto.channel_name) +} +inline ::std::string* HloInstructionProto::mutable_channel_name() { + + // @@protoc_insertion_point(field_mutable:xla.HloInstructionProto.channel_name) + return channel_name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* HloInstructionProto::release_channel_name() { + // @@protoc_insertion_point(field_release:xla.HloInstructionProto.channel_name) + + return channel_name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void HloInstructionProto::set_allocated_channel_name(::std::string* channel_name) { + if (channel_name != NULL) { + + } else { + + } + channel_name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), channel_name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:xla.HloInstructionProto.channel_name) +} +inline ::std::string* HloInstructionProto::unsafe_arena_release_channel_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.HloInstructionProto.channel_name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return channel_name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void HloInstructionProto::unsafe_arena_set_allocated_channel_name( + ::std::string* channel_name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (channel_name != NULL) { + + } else { + + } + channel_name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + channel_name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:xla.HloInstructionProto.channel_name) +} + +// int64 cost_estimate_ns = 42; +inline void HloInstructionProto::clear_cost_estimate_ns() { + cost_estimate_ns_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 HloInstructionProto::cost_estimate_ns() const { + // @@protoc_insertion_point(field_get:xla.HloInstructionProto.cost_estimate_ns) + return cost_estimate_ns_; +} +inline void HloInstructionProto::set_cost_estimate_ns(::google::protobuf::int64 value) { + + cost_estimate_ns_ = value; + // @@protoc_insertion_point(field_set:xla.HloInstructionProto.cost_estimate_ns) +} + +// int64 id = 35; +inline void HloInstructionProto::clear_id() { + id_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 HloInstructionProto::id() const { + // @@protoc_insertion_point(field_get:xla.HloInstructionProto.id) + return id_; +} +inline void HloInstructionProto::set_id(::google::protobuf::int64 value) { + + id_ = value; + // @@protoc_insertion_point(field_set:xla.HloInstructionProto.id) +} + +// repeated int64 operand_ids = 36; +inline int HloInstructionProto::operand_ids_size() const { + return operand_ids_.size(); +} +inline void HloInstructionProto::clear_operand_ids() { + operand_ids_.Clear(); +} +inline ::google::protobuf::int64 HloInstructionProto::operand_ids(int index) const { + // @@protoc_insertion_point(field_get:xla.HloInstructionProto.operand_ids) + return operand_ids_.Get(index); +} +inline void HloInstructionProto::set_operand_ids(int index, ::google::protobuf::int64 value) { + operand_ids_.Set(index, value); + // @@protoc_insertion_point(field_set:xla.HloInstructionProto.operand_ids) +} +inline void HloInstructionProto::add_operand_ids(::google::protobuf::int64 value) { + operand_ids_.Add(value); + // @@protoc_insertion_point(field_add:xla.HloInstructionProto.operand_ids) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& +HloInstructionProto::operand_ids() const { + // @@protoc_insertion_point(field_list:xla.HloInstructionProto.operand_ids) + return operand_ids_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* +HloInstructionProto::mutable_operand_ids() { + // @@protoc_insertion_point(field_mutable_list:xla.HloInstructionProto.operand_ids) + return &operand_ids_; +} + +// repeated int64 control_predecessor_ids = 37; +inline int HloInstructionProto::control_predecessor_ids_size() const { + return control_predecessor_ids_.size(); +} +inline void HloInstructionProto::clear_control_predecessor_ids() { + control_predecessor_ids_.Clear(); +} +inline ::google::protobuf::int64 HloInstructionProto::control_predecessor_ids(int index) const { + // @@protoc_insertion_point(field_get:xla.HloInstructionProto.control_predecessor_ids) + return control_predecessor_ids_.Get(index); +} +inline void HloInstructionProto::set_control_predecessor_ids(int index, ::google::protobuf::int64 value) { + control_predecessor_ids_.Set(index, value); + // @@protoc_insertion_point(field_set:xla.HloInstructionProto.control_predecessor_ids) +} +inline void HloInstructionProto::add_control_predecessor_ids(::google::protobuf::int64 value) { + control_predecessor_ids_.Add(value); + // @@protoc_insertion_point(field_add:xla.HloInstructionProto.control_predecessor_ids) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& +HloInstructionProto::control_predecessor_ids() const { + // @@protoc_insertion_point(field_list:xla.HloInstructionProto.control_predecessor_ids) + return control_predecessor_ids_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* +HloInstructionProto::mutable_control_predecessor_ids() { + // @@protoc_insertion_point(field_mutable_list:xla.HloInstructionProto.control_predecessor_ids) + return &control_predecessor_ids_; +} + +// repeated int64 called_computation_ids = 38; +inline int HloInstructionProto::called_computation_ids_size() const { + return called_computation_ids_.size(); +} +inline void HloInstructionProto::clear_called_computation_ids() { + called_computation_ids_.Clear(); +} +inline ::google::protobuf::int64 HloInstructionProto::called_computation_ids(int index) const { + // @@protoc_insertion_point(field_get:xla.HloInstructionProto.called_computation_ids) + return called_computation_ids_.Get(index); +} +inline void HloInstructionProto::set_called_computation_ids(int index, ::google::protobuf::int64 value) { + called_computation_ids_.Set(index, value); + // @@protoc_insertion_point(field_set:xla.HloInstructionProto.called_computation_ids) +} +inline void HloInstructionProto::add_called_computation_ids(::google::protobuf::int64 value) { + called_computation_ids_.Add(value); + // @@protoc_insertion_point(field_add:xla.HloInstructionProto.called_computation_ids) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& +HloInstructionProto::called_computation_ids() const { + // @@protoc_insertion_point(field_list:xla.HloInstructionProto.called_computation_ids) + return called_computation_ids_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* +HloInstructionProto::mutable_called_computation_ids() { + // @@protoc_insertion_point(field_mutable_list:xla.HloInstructionProto.called_computation_ids) + return &called_computation_ids_; +} + +// .xla.OpSharding sharding = 40; +inline bool HloInstructionProto::has_sharding() const { + return this != internal_default_instance() && sharding_ != NULL; +} +inline const ::xla::OpSharding& HloInstructionProto::_internal_sharding() const { + return *sharding_; +} +inline const ::xla::OpSharding& HloInstructionProto::sharding() const { + const ::xla::OpSharding* p = sharding_; + // @@protoc_insertion_point(field_get:xla.HloInstructionProto.sharding) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_OpSharding_default_instance_); +} +inline ::xla::OpSharding* HloInstructionProto::release_sharding() { + // @@protoc_insertion_point(field_release:xla.HloInstructionProto.sharding) + + ::xla::OpSharding* temp = sharding_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + sharding_ = NULL; + return temp; +} +inline ::xla::OpSharding* HloInstructionProto::unsafe_arena_release_sharding() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.HloInstructionProto.sharding) + + ::xla::OpSharding* temp = sharding_; + sharding_ = NULL; + return temp; +} +inline ::xla::OpSharding* HloInstructionProto::mutable_sharding() { + + if (sharding_ == NULL) { + auto* p = CreateMaybeMessage<::xla::OpSharding>(GetArenaNoVirtual()); + sharding_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.HloInstructionProto.sharding) + return sharding_; +} +inline void HloInstructionProto::set_allocated_sharding(::xla::OpSharding* sharding) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(sharding_); + } + if (sharding) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(sharding)->GetArena(); + if (message_arena != submessage_arena) { + sharding = ::google::protobuf::internal::GetOwnedMessage( + message_arena, sharding, submessage_arena); + } + + } else { + + } + sharding_ = sharding; + // @@protoc_insertion_point(field_set_allocated:xla.HloInstructionProto.sharding) +} + +// string backend_config = 43; +inline void HloInstructionProto::clear_backend_config() { + backend_config_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& HloInstructionProto::backend_config() const { + // @@protoc_insertion_point(field_get:xla.HloInstructionProto.backend_config) + return backend_config_.Get(); +} +inline void HloInstructionProto::set_backend_config(const ::std::string& value) { + + backend_config_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:xla.HloInstructionProto.backend_config) +} +#if LANG_CXX11 +inline void HloInstructionProto::set_backend_config(::std::string&& value) { + + backend_config_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:xla.HloInstructionProto.backend_config) +} +#endif +inline void HloInstructionProto::set_backend_config(const char* value) { + GOOGLE_DCHECK(value != NULL); + + backend_config_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:xla.HloInstructionProto.backend_config) +} +inline void HloInstructionProto::set_backend_config(const char* value, + size_t size) { + + backend_config_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:xla.HloInstructionProto.backend_config) +} +inline ::std::string* HloInstructionProto::mutable_backend_config() { + + // @@protoc_insertion_point(field_mutable:xla.HloInstructionProto.backend_config) + return backend_config_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* HloInstructionProto::release_backend_config() { + // @@protoc_insertion_point(field_release:xla.HloInstructionProto.backend_config) + + return backend_config_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void HloInstructionProto::set_allocated_backend_config(::std::string* backend_config) { + if (backend_config != NULL) { + + } else { + + } + backend_config_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), backend_config, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:xla.HloInstructionProto.backend_config) +} +inline ::std::string* HloInstructionProto::unsafe_arena_release_backend_config() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.HloInstructionProto.backend_config) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return backend_config_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void HloInstructionProto::unsafe_arena_set_allocated_backend_config( + ::std::string* backend_config) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (backend_config != NULL) { + + } else { + + } + backend_config_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + backend_config, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:xla.HloInstructionProto.backend_config) +} + +// repeated .xla.ReplicaGroup replica_groups = 49; +inline int HloInstructionProto::replica_groups_size() const { + return replica_groups_.size(); +} +inline ::xla::ReplicaGroup* HloInstructionProto::mutable_replica_groups(int index) { + // @@protoc_insertion_point(field_mutable:xla.HloInstructionProto.replica_groups) + return replica_groups_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::xla::ReplicaGroup >* +HloInstructionProto::mutable_replica_groups() { + // @@protoc_insertion_point(field_mutable_list:xla.HloInstructionProto.replica_groups) + return &replica_groups_; +} +inline const ::xla::ReplicaGroup& HloInstructionProto::replica_groups(int index) const { + // @@protoc_insertion_point(field_get:xla.HloInstructionProto.replica_groups) + return replica_groups_.Get(index); +} +inline ::xla::ReplicaGroup* HloInstructionProto::add_replica_groups() { + // @@protoc_insertion_point(field_add:xla.HloInstructionProto.replica_groups) + return replica_groups_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::xla::ReplicaGroup >& +HloInstructionProto::replica_groups() const { + // @@protoc_insertion_point(field_list:xla.HloInstructionProto.replica_groups) + return replica_groups_; +} + +// int64 all_reduce_id = 45; +inline void HloInstructionProto::clear_all_reduce_id() { + all_reduce_id_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 HloInstructionProto::all_reduce_id() const { + // @@protoc_insertion_point(field_get:xla.HloInstructionProto.all_reduce_id) + return all_reduce_id_; +} +inline void HloInstructionProto::set_all_reduce_id(::google::protobuf::int64 value) { + + all_reduce_id_ = value; + // @@protoc_insertion_point(field_set:xla.HloInstructionProto.all_reduce_id) +} + +// string cross_replica_sum_barrier = 46; +inline void HloInstructionProto::clear_cross_replica_sum_barrier() { + cross_replica_sum_barrier_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& HloInstructionProto::cross_replica_sum_barrier() const { + // @@protoc_insertion_point(field_get:xla.HloInstructionProto.cross_replica_sum_barrier) + return cross_replica_sum_barrier_.Get(); +} +inline void HloInstructionProto::set_cross_replica_sum_barrier(const ::std::string& value) { + + cross_replica_sum_barrier_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:xla.HloInstructionProto.cross_replica_sum_barrier) +} +#if LANG_CXX11 +inline void HloInstructionProto::set_cross_replica_sum_barrier(::std::string&& value) { + + cross_replica_sum_barrier_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:xla.HloInstructionProto.cross_replica_sum_barrier) +} +#endif +inline void HloInstructionProto::set_cross_replica_sum_barrier(const char* value) { + GOOGLE_DCHECK(value != NULL); + + cross_replica_sum_barrier_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:xla.HloInstructionProto.cross_replica_sum_barrier) +} +inline void HloInstructionProto::set_cross_replica_sum_barrier(const char* value, + size_t size) { + + cross_replica_sum_barrier_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:xla.HloInstructionProto.cross_replica_sum_barrier) +} +inline ::std::string* HloInstructionProto::mutable_cross_replica_sum_barrier() { + + // @@protoc_insertion_point(field_mutable:xla.HloInstructionProto.cross_replica_sum_barrier) + return cross_replica_sum_barrier_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* HloInstructionProto::release_cross_replica_sum_barrier() { + // @@protoc_insertion_point(field_release:xla.HloInstructionProto.cross_replica_sum_barrier) + + return cross_replica_sum_barrier_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void HloInstructionProto::set_allocated_cross_replica_sum_barrier(::std::string* cross_replica_sum_barrier) { + if (cross_replica_sum_barrier != NULL) { + + } else { + + } + cross_replica_sum_barrier_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), cross_replica_sum_barrier, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:xla.HloInstructionProto.cross_replica_sum_barrier) +} +inline ::std::string* HloInstructionProto::unsafe_arena_release_cross_replica_sum_barrier() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.HloInstructionProto.cross_replica_sum_barrier) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return cross_replica_sum_barrier_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void HloInstructionProto::unsafe_arena_set_allocated_cross_replica_sum_barrier( + ::std::string* cross_replica_sum_barrier) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (cross_replica_sum_barrier != NULL) { + + } else { + + } + cross_replica_sum_barrier_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + cross_replica_sum_barrier, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:xla.HloInstructionProto.cross_replica_sum_barrier) +} + +// bool is_host_transfer = 47; +inline void HloInstructionProto::clear_is_host_transfer() { + is_host_transfer_ = false; +} +inline bool HloInstructionProto::is_host_transfer() const { + // @@protoc_insertion_point(field_get:xla.HloInstructionProto.is_host_transfer) + return is_host_transfer_; +} +inline void HloInstructionProto::set_is_host_transfer(bool value) { + + is_host_transfer_ = value; + // @@protoc_insertion_point(field_set:xla.HloInstructionProto.is_host_transfer) +} + +// .xla.ScatterDimensionNumbers scatter_dimension_numbers = 48; +inline bool HloInstructionProto::has_scatter_dimension_numbers() const { + return this != internal_default_instance() && scatter_dimension_numbers_ != NULL; +} +inline const ::xla::ScatterDimensionNumbers& HloInstructionProto::_internal_scatter_dimension_numbers() const { + return *scatter_dimension_numbers_; +} +inline const ::xla::ScatterDimensionNumbers& HloInstructionProto::scatter_dimension_numbers() const { + const ::xla::ScatterDimensionNumbers* p = scatter_dimension_numbers_; + // @@protoc_insertion_point(field_get:xla.HloInstructionProto.scatter_dimension_numbers) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_ScatterDimensionNumbers_default_instance_); +} +inline ::xla::ScatterDimensionNumbers* HloInstructionProto::release_scatter_dimension_numbers() { + // @@protoc_insertion_point(field_release:xla.HloInstructionProto.scatter_dimension_numbers) + + ::xla::ScatterDimensionNumbers* temp = scatter_dimension_numbers_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + scatter_dimension_numbers_ = NULL; + return temp; +} +inline ::xla::ScatterDimensionNumbers* HloInstructionProto::unsafe_arena_release_scatter_dimension_numbers() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.HloInstructionProto.scatter_dimension_numbers) + + ::xla::ScatterDimensionNumbers* temp = scatter_dimension_numbers_; + scatter_dimension_numbers_ = NULL; + return temp; +} +inline ::xla::ScatterDimensionNumbers* HloInstructionProto::mutable_scatter_dimension_numbers() { + + if (scatter_dimension_numbers_ == NULL) { + auto* p = CreateMaybeMessage<::xla::ScatterDimensionNumbers>(GetArenaNoVirtual()); + scatter_dimension_numbers_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.HloInstructionProto.scatter_dimension_numbers) + return scatter_dimension_numbers_; +} +inline void HloInstructionProto::set_allocated_scatter_dimension_numbers(::xla::ScatterDimensionNumbers* scatter_dimension_numbers) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(scatter_dimension_numbers_); + } + if (scatter_dimension_numbers) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(scatter_dimension_numbers)->GetArena(); + if (message_arena != submessage_arena) { + scatter_dimension_numbers = ::google::protobuf::internal::GetOwnedMessage( + message_arena, scatter_dimension_numbers, submessage_arena); + } + + } else { + + } + scatter_dimension_numbers_ = scatter_dimension_numbers; + // @@protoc_insertion_point(field_set_allocated:xla.HloInstructionProto.scatter_dimension_numbers) +} + +// .xla.PrecisionConfig precision_config = 51; +inline bool HloInstructionProto::has_precision_config() const { + return this != internal_default_instance() && precision_config_ != NULL; +} +inline const ::xla::PrecisionConfig& HloInstructionProto::_internal_precision_config() const { + return *precision_config_; +} +inline const ::xla::PrecisionConfig& HloInstructionProto::precision_config() const { + const ::xla::PrecisionConfig* p = precision_config_; + // @@protoc_insertion_point(field_get:xla.HloInstructionProto.precision_config) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_PrecisionConfig_default_instance_); +} +inline ::xla::PrecisionConfig* HloInstructionProto::release_precision_config() { + // @@protoc_insertion_point(field_release:xla.HloInstructionProto.precision_config) + + ::xla::PrecisionConfig* temp = precision_config_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + precision_config_ = NULL; + return temp; +} +inline ::xla::PrecisionConfig* HloInstructionProto::unsafe_arena_release_precision_config() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.HloInstructionProto.precision_config) + + ::xla::PrecisionConfig* temp = precision_config_; + precision_config_ = NULL; + return temp; +} +inline ::xla::PrecisionConfig* HloInstructionProto::mutable_precision_config() { + + if (precision_config_ == NULL) { + auto* p = CreateMaybeMessage<::xla::PrecisionConfig>(GetArenaNoVirtual()); + precision_config_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.HloInstructionProto.precision_config) + return precision_config_; +} +inline void HloInstructionProto::set_allocated_precision_config(::xla::PrecisionConfig* precision_config) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(precision_config_); + } + if (precision_config) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(precision_config)->GetArena(); + if (message_arena != submessage_arena) { + precision_config = ::google::protobuf::internal::GetOwnedMessage( + message_arena, precision_config, submessage_arena); + } + + } else { + + } + precision_config_ = precision_config; + // @@protoc_insertion_point(field_set_allocated:xla.HloInstructionProto.precision_config) +} + +// repeated .xla.SourceTarget source_target_pairs = 52; +inline int HloInstructionProto::source_target_pairs_size() const { + return source_target_pairs_.size(); +} +inline ::xla::SourceTarget* HloInstructionProto::mutable_source_target_pairs(int index) { + // @@protoc_insertion_point(field_mutable:xla.HloInstructionProto.source_target_pairs) + return source_target_pairs_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::xla::SourceTarget >* +HloInstructionProto::mutable_source_target_pairs() { + // @@protoc_insertion_point(field_mutable_list:xla.HloInstructionProto.source_target_pairs) + return &source_target_pairs_; +} +inline const ::xla::SourceTarget& HloInstructionProto::source_target_pairs(int index) const { + // @@protoc_insertion_point(field_get:xla.HloInstructionProto.source_target_pairs) + return source_target_pairs_.Get(index); +} +inline ::xla::SourceTarget* HloInstructionProto::add_source_target_pairs() { + // @@protoc_insertion_point(field_add:xla.HloInstructionProto.source_target_pairs) + return source_target_pairs_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::xla::SourceTarget >& +HloInstructionProto::source_target_pairs() const { + // @@protoc_insertion_point(field_list:xla.HloInstructionProto.source_target_pairs) + return source_target_pairs_; +} + +// ------------------------------------------------------------------- + +// HloComputationProto + +// string name = 1; +inline void HloComputationProto::clear_name() { + name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& HloComputationProto::name() const { + // @@protoc_insertion_point(field_get:xla.HloComputationProto.name) + return name_.Get(); +} +inline void HloComputationProto::set_name(const ::std::string& value) { + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:xla.HloComputationProto.name) +} +#if LANG_CXX11 +inline void HloComputationProto::set_name(::std::string&& value) { + + name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:xla.HloComputationProto.name) +} +#endif +inline void HloComputationProto::set_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:xla.HloComputationProto.name) +} +inline void HloComputationProto::set_name(const char* value, + size_t size) { + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:xla.HloComputationProto.name) +} +inline ::std::string* HloComputationProto::mutable_name() { + + // @@protoc_insertion_point(field_mutable:xla.HloComputationProto.name) + return name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* HloComputationProto::release_name() { + // @@protoc_insertion_point(field_release:xla.HloComputationProto.name) + + return name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void HloComputationProto::set_allocated_name(::std::string* name) { + if (name != NULL) { + + } else { + + } + name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:xla.HloComputationProto.name) +} +inline ::std::string* HloComputationProto::unsafe_arena_release_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.HloComputationProto.name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void HloComputationProto::unsafe_arena_set_allocated_name( + ::std::string* name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (name != NULL) { + + } else { + + } + name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:xla.HloComputationProto.name) +} + +// repeated .xla.HloInstructionProto instructions = 2; +inline int HloComputationProto::instructions_size() const { + return instructions_.size(); +} +inline void HloComputationProto::clear_instructions() { + instructions_.Clear(); +} +inline ::xla::HloInstructionProto* HloComputationProto::mutable_instructions(int index) { + // @@protoc_insertion_point(field_mutable:xla.HloComputationProto.instructions) + return instructions_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::xla::HloInstructionProto >* +HloComputationProto::mutable_instructions() { + // @@protoc_insertion_point(field_mutable_list:xla.HloComputationProto.instructions) + return &instructions_; +} +inline const ::xla::HloInstructionProto& HloComputationProto::instructions(int index) const { + // @@protoc_insertion_point(field_get:xla.HloComputationProto.instructions) + return instructions_.Get(index); +} +inline ::xla::HloInstructionProto* HloComputationProto::add_instructions() { + // @@protoc_insertion_point(field_add:xla.HloComputationProto.instructions) + return instructions_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::xla::HloInstructionProto >& +HloComputationProto::instructions() const { + // @@protoc_insertion_point(field_list:xla.HloComputationProto.instructions) + return instructions_; +} + +// .xla.ProgramShape program_shape = 4; +inline bool HloComputationProto::has_program_shape() const { + return this != internal_default_instance() && program_shape_ != NULL; +} +inline const ::xla::ProgramShape& HloComputationProto::_internal_program_shape() const { + return *program_shape_; +} +inline const ::xla::ProgramShape& HloComputationProto::program_shape() const { + const ::xla::ProgramShape* p = program_shape_; + // @@protoc_insertion_point(field_get:xla.HloComputationProto.program_shape) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_ProgramShape_default_instance_); +} +inline ::xla::ProgramShape* HloComputationProto::release_program_shape() { + // @@protoc_insertion_point(field_release:xla.HloComputationProto.program_shape) + + ::xla::ProgramShape* temp = program_shape_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + program_shape_ = NULL; + return temp; +} +inline ::xla::ProgramShape* HloComputationProto::unsafe_arena_release_program_shape() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.HloComputationProto.program_shape) + + ::xla::ProgramShape* temp = program_shape_; + program_shape_ = NULL; + return temp; +} +inline ::xla::ProgramShape* HloComputationProto::mutable_program_shape() { + + if (program_shape_ == NULL) { + auto* p = CreateMaybeMessage<::xla::ProgramShape>(GetArenaNoVirtual()); + program_shape_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.HloComputationProto.program_shape) + return program_shape_; +} +inline void HloComputationProto::set_allocated_program_shape(::xla::ProgramShape* program_shape) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(program_shape_); + } + if (program_shape) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(program_shape)->GetArena(); + if (message_arena != submessage_arena) { + program_shape = ::google::protobuf::internal::GetOwnedMessage( + message_arena, program_shape, submessage_arena); + } + + } else { + + } + program_shape_ = program_shape; + // @@protoc_insertion_point(field_set_allocated:xla.HloComputationProto.program_shape) +} + +// int64 id = 5; +inline void HloComputationProto::clear_id() { + id_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 HloComputationProto::id() const { + // @@protoc_insertion_point(field_get:xla.HloComputationProto.id) + return id_; +} +inline void HloComputationProto::set_id(::google::protobuf::int64 value) { + + id_ = value; + // @@protoc_insertion_point(field_set:xla.HloComputationProto.id) +} + +// int64 root_id = 6; +inline void HloComputationProto::clear_root_id() { + root_id_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 HloComputationProto::root_id() const { + // @@protoc_insertion_point(field_get:xla.HloComputationProto.root_id) + return root_id_; +} +inline void HloComputationProto::set_root_id(::google::protobuf::int64 value) { + + root_id_ = value; + // @@protoc_insertion_point(field_set:xla.HloComputationProto.root_id) +} + +// ------------------------------------------------------------------- + +// HloScheduleProto_InstructionSequence + +// repeated int64 instruction_ids = 1; +inline int HloScheduleProto_InstructionSequence::instruction_ids_size() const { + return instruction_ids_.size(); +} +inline void HloScheduleProto_InstructionSequence::clear_instruction_ids() { + instruction_ids_.Clear(); +} +inline ::google::protobuf::int64 HloScheduleProto_InstructionSequence::instruction_ids(int index) const { + // @@protoc_insertion_point(field_get:xla.HloScheduleProto.InstructionSequence.instruction_ids) + return instruction_ids_.Get(index); +} +inline void HloScheduleProto_InstructionSequence::set_instruction_ids(int index, ::google::protobuf::int64 value) { + instruction_ids_.Set(index, value); + // @@protoc_insertion_point(field_set:xla.HloScheduleProto.InstructionSequence.instruction_ids) +} +inline void HloScheduleProto_InstructionSequence::add_instruction_ids(::google::protobuf::int64 value) { + instruction_ids_.Add(value); + // @@protoc_insertion_point(field_add:xla.HloScheduleProto.InstructionSequence.instruction_ids) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& +HloScheduleProto_InstructionSequence::instruction_ids() const { + // @@protoc_insertion_point(field_list:xla.HloScheduleProto.InstructionSequence.instruction_ids) + return instruction_ids_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* +HloScheduleProto_InstructionSequence::mutable_instruction_ids() { + // @@protoc_insertion_point(field_mutable_list:xla.HloScheduleProto.InstructionSequence.instruction_ids) + return &instruction_ids_; +} + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// HloScheduleProto + +// map sequences = 1; +inline int HloScheduleProto::sequences_size() const { + return sequences_.size(); +} +inline void HloScheduleProto::clear_sequences() { + sequences_.Clear(); +} +inline const ::google::protobuf::Map< ::google::protobuf::int64, ::xla::HloScheduleProto_InstructionSequence >& +HloScheduleProto::sequences() const { + // @@protoc_insertion_point(field_map:xla.HloScheduleProto.sequences) + return sequences_.GetMap(); +} +inline ::google::protobuf::Map< ::google::protobuf::int64, ::xla::HloScheduleProto_InstructionSequence >* +HloScheduleProto::mutable_sequences() { + // @@protoc_insertion_point(field_mutable_map:xla.HloScheduleProto.sequences) + return sequences_.MutableMap(); +} + +// ------------------------------------------------------------------- + +// HloModuleProto + +// string name = 1; +inline void HloModuleProto::clear_name() { + name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& HloModuleProto::name() const { + // @@protoc_insertion_point(field_get:xla.HloModuleProto.name) + return name_.Get(); +} +inline void HloModuleProto::set_name(const ::std::string& value) { + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:xla.HloModuleProto.name) +} +#if LANG_CXX11 +inline void HloModuleProto::set_name(::std::string&& value) { + + name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:xla.HloModuleProto.name) +} +#endif +inline void HloModuleProto::set_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:xla.HloModuleProto.name) +} +inline void HloModuleProto::set_name(const char* value, + size_t size) { + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:xla.HloModuleProto.name) +} +inline ::std::string* HloModuleProto::mutable_name() { + + // @@protoc_insertion_point(field_mutable:xla.HloModuleProto.name) + return name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* HloModuleProto::release_name() { + // @@protoc_insertion_point(field_release:xla.HloModuleProto.name) + + return name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void HloModuleProto::set_allocated_name(::std::string* name) { + if (name != NULL) { + + } else { + + } + name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:xla.HloModuleProto.name) +} +inline ::std::string* HloModuleProto::unsafe_arena_release_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.HloModuleProto.name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void HloModuleProto::unsafe_arena_set_allocated_name( + ::std::string* name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (name != NULL) { + + } else { + + } + name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:xla.HloModuleProto.name) +} + +// string entry_computation_name = 2; +inline void HloModuleProto::clear_entry_computation_name() { + entry_computation_name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& HloModuleProto::entry_computation_name() const { + // @@protoc_insertion_point(field_get:xla.HloModuleProto.entry_computation_name) + return entry_computation_name_.Get(); +} +inline void HloModuleProto::set_entry_computation_name(const ::std::string& value) { + + entry_computation_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:xla.HloModuleProto.entry_computation_name) +} +#if LANG_CXX11 +inline void HloModuleProto::set_entry_computation_name(::std::string&& value) { + + entry_computation_name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:xla.HloModuleProto.entry_computation_name) +} +#endif +inline void HloModuleProto::set_entry_computation_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + entry_computation_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:xla.HloModuleProto.entry_computation_name) +} +inline void HloModuleProto::set_entry_computation_name(const char* value, + size_t size) { + + entry_computation_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:xla.HloModuleProto.entry_computation_name) +} +inline ::std::string* HloModuleProto::mutable_entry_computation_name() { + + // @@protoc_insertion_point(field_mutable:xla.HloModuleProto.entry_computation_name) + return entry_computation_name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* HloModuleProto::release_entry_computation_name() { + // @@protoc_insertion_point(field_release:xla.HloModuleProto.entry_computation_name) + + return entry_computation_name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void HloModuleProto::set_allocated_entry_computation_name(::std::string* entry_computation_name) { + if (entry_computation_name != NULL) { + + } else { + + } + entry_computation_name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), entry_computation_name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:xla.HloModuleProto.entry_computation_name) +} +inline ::std::string* HloModuleProto::unsafe_arena_release_entry_computation_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.HloModuleProto.entry_computation_name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return entry_computation_name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void HloModuleProto::unsafe_arena_set_allocated_entry_computation_name( + ::std::string* entry_computation_name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (entry_computation_name != NULL) { + + } else { + + } + entry_computation_name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + entry_computation_name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:xla.HloModuleProto.entry_computation_name) +} + +// int64 entry_computation_id = 6; +inline void HloModuleProto::clear_entry_computation_id() { + entry_computation_id_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 HloModuleProto::entry_computation_id() const { + // @@protoc_insertion_point(field_get:xla.HloModuleProto.entry_computation_id) + return entry_computation_id_; +} +inline void HloModuleProto::set_entry_computation_id(::google::protobuf::int64 value) { + + entry_computation_id_ = value; + // @@protoc_insertion_point(field_set:xla.HloModuleProto.entry_computation_id) +} + +// repeated .xla.HloComputationProto computations = 3; +inline int HloModuleProto::computations_size() const { + return computations_.size(); +} +inline void HloModuleProto::clear_computations() { + computations_.Clear(); +} +inline ::xla::HloComputationProto* HloModuleProto::mutable_computations(int index) { + // @@protoc_insertion_point(field_mutable:xla.HloModuleProto.computations) + return computations_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::xla::HloComputationProto >* +HloModuleProto::mutable_computations() { + // @@protoc_insertion_point(field_mutable_list:xla.HloModuleProto.computations) + return &computations_; +} +inline const ::xla::HloComputationProto& HloModuleProto::computations(int index) const { + // @@protoc_insertion_point(field_get:xla.HloModuleProto.computations) + return computations_.Get(index); +} +inline ::xla::HloComputationProto* HloModuleProto::add_computations() { + // @@protoc_insertion_point(field_add:xla.HloModuleProto.computations) + return computations_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::xla::HloComputationProto >& +HloModuleProto::computations() const { + // @@protoc_insertion_point(field_list:xla.HloModuleProto.computations) + return computations_; +} + +// .xla.ProgramShape program_shape = 4; +inline bool HloModuleProto::has_program_shape() const { + return this != internal_default_instance() && program_shape_ != NULL; +} +inline const ::xla::ProgramShape& HloModuleProto::_internal_program_shape() const { + return *program_shape_; +} +inline const ::xla::ProgramShape& HloModuleProto::program_shape() const { + const ::xla::ProgramShape* p = program_shape_; + // @@protoc_insertion_point(field_get:xla.HloModuleProto.program_shape) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_ProgramShape_default_instance_); +} +inline ::xla::ProgramShape* HloModuleProto::release_program_shape() { + // @@protoc_insertion_point(field_release:xla.HloModuleProto.program_shape) + + ::xla::ProgramShape* temp = program_shape_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + program_shape_ = NULL; + return temp; +} +inline ::xla::ProgramShape* HloModuleProto::unsafe_arena_release_program_shape() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.HloModuleProto.program_shape) + + ::xla::ProgramShape* temp = program_shape_; + program_shape_ = NULL; + return temp; +} +inline ::xla::ProgramShape* HloModuleProto::mutable_program_shape() { + + if (program_shape_ == NULL) { + auto* p = CreateMaybeMessage<::xla::ProgramShape>(GetArenaNoVirtual()); + program_shape_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.HloModuleProto.program_shape) + return program_shape_; +} +inline void HloModuleProto::set_allocated_program_shape(::xla::ProgramShape* program_shape) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(program_shape_); + } + if (program_shape) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(program_shape)->GetArena(); + if (message_arena != submessage_arena) { + program_shape = ::google::protobuf::internal::GetOwnedMessage( + message_arena, program_shape, submessage_arena); + } + + } else { + + } + program_shape_ = program_shape; + // @@protoc_insertion_point(field_set_allocated:xla.HloModuleProto.program_shape) +} + +// int64 id = 5; +inline void HloModuleProto::clear_id() { + id_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 HloModuleProto::id() const { + // @@protoc_insertion_point(field_get:xla.HloModuleProto.id) + return id_; +} +inline void HloModuleProto::set_id(::google::protobuf::int64 value) { + + id_ = value; + // @@protoc_insertion_point(field_set:xla.HloModuleProto.id) +} + +// .xla.HloScheduleProto schedule = 7; +inline bool HloModuleProto::has_schedule() const { + return this != internal_default_instance() && schedule_ != NULL; +} +inline void HloModuleProto::clear_schedule() { + if (GetArenaNoVirtual() == NULL && schedule_ != NULL) { + delete schedule_; + } + schedule_ = NULL; +} +inline const ::xla::HloScheduleProto& HloModuleProto::_internal_schedule() const { + return *schedule_; +} +inline const ::xla::HloScheduleProto& HloModuleProto::schedule() const { + const ::xla::HloScheduleProto* p = schedule_; + // @@protoc_insertion_point(field_get:xla.HloModuleProto.schedule) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_HloScheduleProto_default_instance_); +} +inline ::xla::HloScheduleProto* HloModuleProto::release_schedule() { + // @@protoc_insertion_point(field_release:xla.HloModuleProto.schedule) + + ::xla::HloScheduleProto* temp = schedule_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + schedule_ = NULL; + return temp; +} +inline ::xla::HloScheduleProto* HloModuleProto::unsafe_arena_release_schedule() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.HloModuleProto.schedule) + + ::xla::HloScheduleProto* temp = schedule_; + schedule_ = NULL; + return temp; +} +inline ::xla::HloScheduleProto* HloModuleProto::mutable_schedule() { + + if (schedule_ == NULL) { + auto* p = CreateMaybeMessage<::xla::HloScheduleProto>(GetArenaNoVirtual()); + schedule_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.HloModuleProto.schedule) + return schedule_; +} +inline void HloModuleProto::set_allocated_schedule(::xla::HloScheduleProto* schedule) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete schedule_; + } + if (schedule) { + ::google::protobuf::Arena* submessage_arena = + ::google::protobuf::Arena::GetArena(schedule); + if (message_arena != submessage_arena) { + schedule = ::google::protobuf::internal::GetOwnedMessage( + message_arena, schedule, submessage_arena); + } + + } else { + + } + schedule_ = schedule; + // @@protoc_insertion_point(field_set_allocated:xla.HloModuleProto.schedule) +} + +// ------------------------------------------------------------------- + +// LogicalBufferProto_Location + +// string computation_name = 1; +inline void LogicalBufferProto_Location::clear_computation_name() { + computation_name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& LogicalBufferProto_Location::computation_name() const { + // @@protoc_insertion_point(field_get:xla.LogicalBufferProto.Location.computation_name) + return computation_name_.Get(); +} +inline void LogicalBufferProto_Location::set_computation_name(const ::std::string& value) { + + computation_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:xla.LogicalBufferProto.Location.computation_name) +} +#if LANG_CXX11 +inline void LogicalBufferProto_Location::set_computation_name(::std::string&& value) { + + computation_name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:xla.LogicalBufferProto.Location.computation_name) +} +#endif +inline void LogicalBufferProto_Location::set_computation_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + computation_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:xla.LogicalBufferProto.Location.computation_name) +} +inline void LogicalBufferProto_Location::set_computation_name(const char* value, + size_t size) { + + computation_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:xla.LogicalBufferProto.Location.computation_name) +} +inline ::std::string* LogicalBufferProto_Location::mutable_computation_name() { + + // @@protoc_insertion_point(field_mutable:xla.LogicalBufferProto.Location.computation_name) + return computation_name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* LogicalBufferProto_Location::release_computation_name() { + // @@protoc_insertion_point(field_release:xla.LogicalBufferProto.Location.computation_name) + + return computation_name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void LogicalBufferProto_Location::set_allocated_computation_name(::std::string* computation_name) { + if (computation_name != NULL) { + + } else { + + } + computation_name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), computation_name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:xla.LogicalBufferProto.Location.computation_name) +} +inline ::std::string* LogicalBufferProto_Location::unsafe_arena_release_computation_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.LogicalBufferProto.Location.computation_name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return computation_name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void LogicalBufferProto_Location::unsafe_arena_set_allocated_computation_name( + ::std::string* computation_name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (computation_name != NULL) { + + } else { + + } + computation_name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + computation_name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:xla.LogicalBufferProto.Location.computation_name) +} + +// string instruction_name = 2; +inline void LogicalBufferProto_Location::clear_instruction_name() { + instruction_name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& LogicalBufferProto_Location::instruction_name() const { + // @@protoc_insertion_point(field_get:xla.LogicalBufferProto.Location.instruction_name) + return instruction_name_.Get(); +} +inline void LogicalBufferProto_Location::set_instruction_name(const ::std::string& value) { + + instruction_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:xla.LogicalBufferProto.Location.instruction_name) +} +#if LANG_CXX11 +inline void LogicalBufferProto_Location::set_instruction_name(::std::string&& value) { + + instruction_name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:xla.LogicalBufferProto.Location.instruction_name) +} +#endif +inline void LogicalBufferProto_Location::set_instruction_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + instruction_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:xla.LogicalBufferProto.Location.instruction_name) +} +inline void LogicalBufferProto_Location::set_instruction_name(const char* value, + size_t size) { + + instruction_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:xla.LogicalBufferProto.Location.instruction_name) +} +inline ::std::string* LogicalBufferProto_Location::mutable_instruction_name() { + + // @@protoc_insertion_point(field_mutable:xla.LogicalBufferProto.Location.instruction_name) + return instruction_name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* LogicalBufferProto_Location::release_instruction_name() { + // @@protoc_insertion_point(field_release:xla.LogicalBufferProto.Location.instruction_name) + + return instruction_name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void LogicalBufferProto_Location::set_allocated_instruction_name(::std::string* instruction_name) { + if (instruction_name != NULL) { + + } else { + + } + instruction_name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), instruction_name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:xla.LogicalBufferProto.Location.instruction_name) +} +inline ::std::string* LogicalBufferProto_Location::unsafe_arena_release_instruction_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.LogicalBufferProto.Location.instruction_name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return instruction_name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void LogicalBufferProto_Location::unsafe_arena_set_allocated_instruction_name( + ::std::string* instruction_name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (instruction_name != NULL) { + + } else { + + } + instruction_name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + instruction_name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:xla.LogicalBufferProto.Location.instruction_name) +} + +// repeated int64 shape_index = 3; +inline int LogicalBufferProto_Location::shape_index_size() const { + return shape_index_.size(); +} +inline void LogicalBufferProto_Location::clear_shape_index() { + shape_index_.Clear(); +} +inline ::google::protobuf::int64 LogicalBufferProto_Location::shape_index(int index) const { + // @@protoc_insertion_point(field_get:xla.LogicalBufferProto.Location.shape_index) + return shape_index_.Get(index); +} +inline void LogicalBufferProto_Location::set_shape_index(int index, ::google::protobuf::int64 value) { + shape_index_.Set(index, value); + // @@protoc_insertion_point(field_set:xla.LogicalBufferProto.Location.shape_index) +} +inline void LogicalBufferProto_Location::add_shape_index(::google::protobuf::int64 value) { + shape_index_.Add(value); + // @@protoc_insertion_point(field_add:xla.LogicalBufferProto.Location.shape_index) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& +LogicalBufferProto_Location::shape_index() const { + // @@protoc_insertion_point(field_list:xla.LogicalBufferProto.Location.shape_index) + return shape_index_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* +LogicalBufferProto_Location::mutable_shape_index() { + // @@protoc_insertion_point(field_mutable_list:xla.LogicalBufferProto.Location.shape_index) + return &shape_index_; +} + +// ------------------------------------------------------------------- + +// LogicalBufferProto + +// int64 id = 1; +inline void LogicalBufferProto::clear_id() { + id_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 LogicalBufferProto::id() const { + // @@protoc_insertion_point(field_get:xla.LogicalBufferProto.id) + return id_; +} +inline void LogicalBufferProto::set_id(::google::protobuf::int64 value) { + + id_ = value; + // @@protoc_insertion_point(field_set:xla.LogicalBufferProto.id) +} + +// int64 size = 2; +inline void LogicalBufferProto::clear_size() { + size_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 LogicalBufferProto::size() const { + // @@protoc_insertion_point(field_get:xla.LogicalBufferProto.size) + return size_; +} +inline void LogicalBufferProto::set_size(::google::protobuf::int64 value) { + + size_ = value; + // @@protoc_insertion_point(field_set:xla.LogicalBufferProto.size) +} + +// .xla.LogicalBufferProto.Location defined_at = 3; +inline bool LogicalBufferProto::has_defined_at() const { + return this != internal_default_instance() && defined_at_ != NULL; +} +inline void LogicalBufferProto::clear_defined_at() { + if (GetArenaNoVirtual() == NULL && defined_at_ != NULL) { + delete defined_at_; + } + defined_at_ = NULL; +} +inline const ::xla::LogicalBufferProto_Location& LogicalBufferProto::_internal_defined_at() const { + return *defined_at_; +} +inline const ::xla::LogicalBufferProto_Location& LogicalBufferProto::defined_at() const { + const ::xla::LogicalBufferProto_Location* p = defined_at_; + // @@protoc_insertion_point(field_get:xla.LogicalBufferProto.defined_at) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_LogicalBufferProto_Location_default_instance_); +} +inline ::xla::LogicalBufferProto_Location* LogicalBufferProto::release_defined_at() { + // @@protoc_insertion_point(field_release:xla.LogicalBufferProto.defined_at) + + ::xla::LogicalBufferProto_Location* temp = defined_at_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + defined_at_ = NULL; + return temp; +} +inline ::xla::LogicalBufferProto_Location* LogicalBufferProto::unsafe_arena_release_defined_at() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.LogicalBufferProto.defined_at) + + ::xla::LogicalBufferProto_Location* temp = defined_at_; + defined_at_ = NULL; + return temp; +} +inline ::xla::LogicalBufferProto_Location* LogicalBufferProto::mutable_defined_at() { + + if (defined_at_ == NULL) { + auto* p = CreateMaybeMessage<::xla::LogicalBufferProto_Location>(GetArenaNoVirtual()); + defined_at_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.LogicalBufferProto.defined_at) + return defined_at_; +} +inline void LogicalBufferProto::set_allocated_defined_at(::xla::LogicalBufferProto_Location* defined_at) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete defined_at_; + } + if (defined_at) { + ::google::protobuf::Arena* submessage_arena = + ::google::protobuf::Arena::GetArena(defined_at); + if (message_arena != submessage_arena) { + defined_at = ::google::protobuf::internal::GetOwnedMessage( + message_arena, defined_at, submessage_arena); + } + + } else { + + } + defined_at_ = defined_at; + // @@protoc_insertion_point(field_set_allocated:xla.LogicalBufferProto.defined_at) +} + +// int64 color = 4; +inline void LogicalBufferProto::clear_color() { + color_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 LogicalBufferProto::color() const { + // @@protoc_insertion_point(field_get:xla.LogicalBufferProto.color) + return color_; +} +inline void LogicalBufferProto::set_color(::google::protobuf::int64 value) { + + color_ = value; + // @@protoc_insertion_point(field_set:xla.LogicalBufferProto.color) +} + +// ------------------------------------------------------------------- + +// BufferAllocationProto_Assigned + +// int64 logical_buffer_id = 1; +inline void BufferAllocationProto_Assigned::clear_logical_buffer_id() { + logical_buffer_id_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 BufferAllocationProto_Assigned::logical_buffer_id() const { + // @@protoc_insertion_point(field_get:xla.BufferAllocationProto.Assigned.logical_buffer_id) + return logical_buffer_id_; +} +inline void BufferAllocationProto_Assigned::set_logical_buffer_id(::google::protobuf::int64 value) { + + logical_buffer_id_ = value; + // @@protoc_insertion_point(field_set:xla.BufferAllocationProto.Assigned.logical_buffer_id) +} + +// int64 offset = 2; +inline void BufferAllocationProto_Assigned::clear_offset() { + offset_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 BufferAllocationProto_Assigned::offset() const { + // @@protoc_insertion_point(field_get:xla.BufferAllocationProto.Assigned.offset) + return offset_; +} +inline void BufferAllocationProto_Assigned::set_offset(::google::protobuf::int64 value) { + + offset_ = value; + // @@protoc_insertion_point(field_set:xla.BufferAllocationProto.Assigned.offset) +} + +// int64 size = 3; +inline void BufferAllocationProto_Assigned::clear_size() { + size_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 BufferAllocationProto_Assigned::size() const { + // @@protoc_insertion_point(field_get:xla.BufferAllocationProto.Assigned.size) + return size_; +} +inline void BufferAllocationProto_Assigned::set_size(::google::protobuf::int64 value) { + + size_ = value; + // @@protoc_insertion_point(field_set:xla.BufferAllocationProto.Assigned.size) +} + +// ------------------------------------------------------------------- + +// BufferAllocationProto + +// int64 index = 1; +inline void BufferAllocationProto::clear_index() { + index_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 BufferAllocationProto::index() const { + // @@protoc_insertion_point(field_get:xla.BufferAllocationProto.index) + return index_; +} +inline void BufferAllocationProto::set_index(::google::protobuf::int64 value) { + + index_ = value; + // @@protoc_insertion_point(field_set:xla.BufferAllocationProto.index) +} + +// int64 size = 2; +inline void BufferAllocationProto::clear_size() { + size_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 BufferAllocationProto::size() const { + // @@protoc_insertion_point(field_get:xla.BufferAllocationProto.size) + return size_; +} +inline void BufferAllocationProto::set_size(::google::protobuf::int64 value) { + + size_ = value; + // @@protoc_insertion_point(field_set:xla.BufferAllocationProto.size) +} + +// bool is_thread_local = 3; +inline void BufferAllocationProto::clear_is_thread_local() { + is_thread_local_ = false; +} +inline bool BufferAllocationProto::is_thread_local() const { + // @@protoc_insertion_point(field_get:xla.BufferAllocationProto.is_thread_local) + return is_thread_local_; +} +inline void BufferAllocationProto::set_is_thread_local(bool value) { + + is_thread_local_ = value; + // @@protoc_insertion_point(field_set:xla.BufferAllocationProto.is_thread_local) +} + +// bool is_tuple = 11; +inline void BufferAllocationProto::clear_is_tuple() { + is_tuple_ = false; +} +inline bool BufferAllocationProto::is_tuple() const { + // @@protoc_insertion_point(field_get:xla.BufferAllocationProto.is_tuple) + return is_tuple_; +} +inline void BufferAllocationProto::set_is_tuple(bool value) { + + is_tuple_ = value; + // @@protoc_insertion_point(field_set:xla.BufferAllocationProto.is_tuple) +} + +// bool is_entry_computation_parameter = 5; +inline void BufferAllocationProto::clear_is_entry_computation_parameter() { + is_entry_computation_parameter_ = false; +} +inline bool BufferAllocationProto::is_entry_computation_parameter() const { + // @@protoc_insertion_point(field_get:xla.BufferAllocationProto.is_entry_computation_parameter) + return is_entry_computation_parameter_; +} +inline void BufferAllocationProto::set_is_entry_computation_parameter(bool value) { + + is_entry_computation_parameter_ = value; + // @@protoc_insertion_point(field_set:xla.BufferAllocationProto.is_entry_computation_parameter) +} + +// bool is_constant = 12; +inline void BufferAllocationProto::clear_is_constant() { + is_constant_ = false; +} +inline bool BufferAllocationProto::is_constant() const { + // @@protoc_insertion_point(field_get:xla.BufferAllocationProto.is_constant) + return is_constant_; +} +inline void BufferAllocationProto::set_is_constant(bool value) { + + is_constant_ = value; + // @@protoc_insertion_point(field_set:xla.BufferAllocationProto.is_constant) +} + +// int64 parameter_number = 6; +inline void BufferAllocationProto::clear_parameter_number() { + parameter_number_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 BufferAllocationProto::parameter_number() const { + // @@protoc_insertion_point(field_get:xla.BufferAllocationProto.parameter_number) + return parameter_number_; +} +inline void BufferAllocationProto::set_parameter_number(::google::protobuf::int64 value) { + + parameter_number_ = value; + // @@protoc_insertion_point(field_set:xla.BufferAllocationProto.parameter_number) +} + +// repeated int64 parameter_shape_index = 10; +inline int BufferAllocationProto::parameter_shape_index_size() const { + return parameter_shape_index_.size(); +} +inline void BufferAllocationProto::clear_parameter_shape_index() { + parameter_shape_index_.Clear(); +} +inline ::google::protobuf::int64 BufferAllocationProto::parameter_shape_index(int index) const { + // @@protoc_insertion_point(field_get:xla.BufferAllocationProto.parameter_shape_index) + return parameter_shape_index_.Get(index); +} +inline void BufferAllocationProto::set_parameter_shape_index(int index, ::google::protobuf::int64 value) { + parameter_shape_index_.Set(index, value); + // @@protoc_insertion_point(field_set:xla.BufferAllocationProto.parameter_shape_index) +} +inline void BufferAllocationProto::add_parameter_shape_index(::google::protobuf::int64 value) { + parameter_shape_index_.Add(value); + // @@protoc_insertion_point(field_add:xla.BufferAllocationProto.parameter_shape_index) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& +BufferAllocationProto::parameter_shape_index() const { + // @@protoc_insertion_point(field_list:xla.BufferAllocationProto.parameter_shape_index) + return parameter_shape_index_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* +BufferAllocationProto::mutable_parameter_shape_index() { + // @@protoc_insertion_point(field_mutable_list:xla.BufferAllocationProto.parameter_shape_index) + return ¶meter_shape_index_; +} + +// bool maybe_live_out = 7; +inline void BufferAllocationProto::clear_maybe_live_out() { + maybe_live_out_ = false; +} +inline bool BufferAllocationProto::maybe_live_out() const { + // @@protoc_insertion_point(field_get:xla.BufferAllocationProto.maybe_live_out) + return maybe_live_out_; +} +inline void BufferAllocationProto::set_maybe_live_out(bool value) { + + maybe_live_out_ = value; + // @@protoc_insertion_point(field_set:xla.BufferAllocationProto.maybe_live_out) +} + +// int64 color = 8; +inline void BufferAllocationProto::clear_color() { + color_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 BufferAllocationProto::color() const { + // @@protoc_insertion_point(field_get:xla.BufferAllocationProto.color) + return color_; +} +inline void BufferAllocationProto::set_color(::google::protobuf::int64 value) { + + color_ = value; + // @@protoc_insertion_point(field_set:xla.BufferAllocationProto.color) +} + +// repeated .xla.BufferAllocationProto.Assigned assigned = 9; +inline int BufferAllocationProto::assigned_size() const { + return assigned_.size(); +} +inline void BufferAllocationProto::clear_assigned() { + assigned_.Clear(); +} +inline ::xla::BufferAllocationProto_Assigned* BufferAllocationProto::mutable_assigned(int index) { + // @@protoc_insertion_point(field_mutable:xla.BufferAllocationProto.assigned) + return assigned_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::xla::BufferAllocationProto_Assigned >* +BufferAllocationProto::mutable_assigned() { + // @@protoc_insertion_point(field_mutable_list:xla.BufferAllocationProto.assigned) + return &assigned_; +} +inline const ::xla::BufferAllocationProto_Assigned& BufferAllocationProto::assigned(int index) const { + // @@protoc_insertion_point(field_get:xla.BufferAllocationProto.assigned) + return assigned_.Get(index); +} +inline ::xla::BufferAllocationProto_Assigned* BufferAllocationProto::add_assigned() { + // @@protoc_insertion_point(field_add:xla.BufferAllocationProto.assigned) + return assigned_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::xla::BufferAllocationProto_Assigned >& +BufferAllocationProto::assigned() const { + // @@protoc_insertion_point(field_list:xla.BufferAllocationProto.assigned) + return assigned_; +} + +// ------------------------------------------------------------------- + +// HeapSimulatorTrace_Event + +// .xla.HeapSimulatorTrace.Event.Kind kind = 1; +inline void HeapSimulatorTrace_Event::clear_kind() { + kind_ = 0; +} +inline ::xla::HeapSimulatorTrace_Event_Kind HeapSimulatorTrace_Event::kind() const { + // @@protoc_insertion_point(field_get:xla.HeapSimulatorTrace.Event.kind) + return static_cast< ::xla::HeapSimulatorTrace_Event_Kind >(kind_); +} +inline void HeapSimulatorTrace_Event::set_kind(::xla::HeapSimulatorTrace_Event_Kind value) { + + kind_ = value; + // @@protoc_insertion_point(field_set:xla.HeapSimulatorTrace.Event.kind) +} + +// int64 buffer_id = 2; +inline void HeapSimulatorTrace_Event::clear_buffer_id() { + buffer_id_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 HeapSimulatorTrace_Event::buffer_id() const { + // @@protoc_insertion_point(field_get:xla.HeapSimulatorTrace.Event.buffer_id) + return buffer_id_; +} +inline void HeapSimulatorTrace_Event::set_buffer_id(::google::protobuf::int64 value) { + + buffer_id_ = value; + // @@protoc_insertion_point(field_set:xla.HeapSimulatorTrace.Event.buffer_id) +} + +// string computation_name = 3; +inline void HeapSimulatorTrace_Event::clear_computation_name() { + computation_name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& HeapSimulatorTrace_Event::computation_name() const { + // @@protoc_insertion_point(field_get:xla.HeapSimulatorTrace.Event.computation_name) + return computation_name_.Get(); +} +inline void HeapSimulatorTrace_Event::set_computation_name(const ::std::string& value) { + + computation_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:xla.HeapSimulatorTrace.Event.computation_name) +} +#if LANG_CXX11 +inline void HeapSimulatorTrace_Event::set_computation_name(::std::string&& value) { + + computation_name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:xla.HeapSimulatorTrace.Event.computation_name) +} +#endif +inline void HeapSimulatorTrace_Event::set_computation_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + computation_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:xla.HeapSimulatorTrace.Event.computation_name) +} +inline void HeapSimulatorTrace_Event::set_computation_name(const char* value, + size_t size) { + + computation_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:xla.HeapSimulatorTrace.Event.computation_name) +} +inline ::std::string* HeapSimulatorTrace_Event::mutable_computation_name() { + + // @@protoc_insertion_point(field_mutable:xla.HeapSimulatorTrace.Event.computation_name) + return computation_name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* HeapSimulatorTrace_Event::release_computation_name() { + // @@protoc_insertion_point(field_release:xla.HeapSimulatorTrace.Event.computation_name) + + return computation_name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void HeapSimulatorTrace_Event::set_allocated_computation_name(::std::string* computation_name) { + if (computation_name != NULL) { + + } else { + + } + computation_name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), computation_name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:xla.HeapSimulatorTrace.Event.computation_name) +} +inline ::std::string* HeapSimulatorTrace_Event::unsafe_arena_release_computation_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.HeapSimulatorTrace.Event.computation_name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return computation_name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void HeapSimulatorTrace_Event::unsafe_arena_set_allocated_computation_name( + ::std::string* computation_name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (computation_name != NULL) { + + } else { + + } + computation_name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + computation_name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:xla.HeapSimulatorTrace.Event.computation_name) +} + +// string instruction_name = 4; +inline void HeapSimulatorTrace_Event::clear_instruction_name() { + instruction_name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& HeapSimulatorTrace_Event::instruction_name() const { + // @@protoc_insertion_point(field_get:xla.HeapSimulatorTrace.Event.instruction_name) + return instruction_name_.Get(); +} +inline void HeapSimulatorTrace_Event::set_instruction_name(const ::std::string& value) { + + instruction_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:xla.HeapSimulatorTrace.Event.instruction_name) +} +#if LANG_CXX11 +inline void HeapSimulatorTrace_Event::set_instruction_name(::std::string&& value) { + + instruction_name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:xla.HeapSimulatorTrace.Event.instruction_name) +} +#endif +inline void HeapSimulatorTrace_Event::set_instruction_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + instruction_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:xla.HeapSimulatorTrace.Event.instruction_name) +} +inline void HeapSimulatorTrace_Event::set_instruction_name(const char* value, + size_t size) { + + instruction_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:xla.HeapSimulatorTrace.Event.instruction_name) +} +inline ::std::string* HeapSimulatorTrace_Event::mutable_instruction_name() { + + // @@protoc_insertion_point(field_mutable:xla.HeapSimulatorTrace.Event.instruction_name) + return instruction_name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* HeapSimulatorTrace_Event::release_instruction_name() { + // @@protoc_insertion_point(field_release:xla.HeapSimulatorTrace.Event.instruction_name) + + return instruction_name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void HeapSimulatorTrace_Event::set_allocated_instruction_name(::std::string* instruction_name) { + if (instruction_name != NULL) { + + } else { + + } + instruction_name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), instruction_name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:xla.HeapSimulatorTrace.Event.instruction_name) +} +inline ::std::string* HeapSimulatorTrace_Event::unsafe_arena_release_instruction_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.HeapSimulatorTrace.Event.instruction_name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return instruction_name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void HeapSimulatorTrace_Event::unsafe_arena_set_allocated_instruction_name( + ::std::string* instruction_name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (instruction_name != NULL) { + + } else { + + } + instruction_name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + instruction_name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:xla.HeapSimulatorTrace.Event.instruction_name) +} + +// int64 share_with_canonical_id = 5; +inline void HeapSimulatorTrace_Event::clear_share_with_canonical_id() { + share_with_canonical_id_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 HeapSimulatorTrace_Event::share_with_canonical_id() const { + // @@protoc_insertion_point(field_get:xla.HeapSimulatorTrace.Event.share_with_canonical_id) + return share_with_canonical_id_; +} +inline void HeapSimulatorTrace_Event::set_share_with_canonical_id(::google::protobuf::int64 value) { + + share_with_canonical_id_ = value; + // @@protoc_insertion_point(field_set:xla.HeapSimulatorTrace.Event.share_with_canonical_id) +} + +// ------------------------------------------------------------------- + +// HeapSimulatorTrace + +// repeated .xla.HeapSimulatorTrace.Event events = 1; +inline int HeapSimulatorTrace::events_size() const { + return events_.size(); +} +inline void HeapSimulatorTrace::clear_events() { + events_.Clear(); +} +inline ::xla::HeapSimulatorTrace_Event* HeapSimulatorTrace::mutable_events(int index) { + // @@protoc_insertion_point(field_mutable:xla.HeapSimulatorTrace.events) + return events_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::xla::HeapSimulatorTrace_Event >* +HeapSimulatorTrace::mutable_events() { + // @@protoc_insertion_point(field_mutable_list:xla.HeapSimulatorTrace.events) + return &events_; +} +inline const ::xla::HeapSimulatorTrace_Event& HeapSimulatorTrace::events(int index) const { + // @@protoc_insertion_point(field_get:xla.HeapSimulatorTrace.events) + return events_.Get(index); +} +inline ::xla::HeapSimulatorTrace_Event* HeapSimulatorTrace::add_events() { + // @@protoc_insertion_point(field_add:xla.HeapSimulatorTrace.events) + return events_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::xla::HeapSimulatorTrace_Event >& +HeapSimulatorTrace::events() const { + // @@protoc_insertion_point(field_list:xla.HeapSimulatorTrace.events) + return events_; +} + +// bool whole_module_simulation = 2; +inline void HeapSimulatorTrace::clear_whole_module_simulation() { + whole_module_simulation_ = false; +} +inline bool HeapSimulatorTrace::whole_module_simulation() const { + // @@protoc_insertion_point(field_get:xla.HeapSimulatorTrace.whole_module_simulation) + return whole_module_simulation_; +} +inline void HeapSimulatorTrace::set_whole_module_simulation(bool value) { + + whole_module_simulation_ = value; + // @@protoc_insertion_point(field_set:xla.HeapSimulatorTrace.whole_module_simulation) +} + +// ------------------------------------------------------------------- + +// HloModuleGroupProto + +// string name = 1; +inline void HloModuleGroupProto::clear_name() { + name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& HloModuleGroupProto::name() const { + // @@protoc_insertion_point(field_get:xla.HloModuleGroupProto.name) + return name_.Get(); +} +inline void HloModuleGroupProto::set_name(const ::std::string& value) { + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:xla.HloModuleGroupProto.name) +} +#if LANG_CXX11 +inline void HloModuleGroupProto::set_name(::std::string&& value) { + + name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:xla.HloModuleGroupProto.name) +} +#endif +inline void HloModuleGroupProto::set_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:xla.HloModuleGroupProto.name) +} +inline void HloModuleGroupProto::set_name(const char* value, + size_t size) { + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:xla.HloModuleGroupProto.name) +} +inline ::std::string* HloModuleGroupProto::mutable_name() { + + // @@protoc_insertion_point(field_mutable:xla.HloModuleGroupProto.name) + return name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* HloModuleGroupProto::release_name() { + // @@protoc_insertion_point(field_release:xla.HloModuleGroupProto.name) + + return name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void HloModuleGroupProto::set_allocated_name(::std::string* name) { + if (name != NULL) { + + } else { + + } + name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:xla.HloModuleGroupProto.name) +} +inline ::std::string* HloModuleGroupProto::unsafe_arena_release_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.HloModuleGroupProto.name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void HloModuleGroupProto::unsafe_arena_set_allocated_name( + ::std::string* name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (name != NULL) { + + } else { + + } + name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:xla.HloModuleGroupProto.name) +} + +// repeated .xla.HloModuleProto hlo_modules = 2; +inline int HloModuleGroupProto::hlo_modules_size() const { + return hlo_modules_.size(); +} +inline void HloModuleGroupProto::clear_hlo_modules() { + hlo_modules_.Clear(); +} +inline ::xla::HloModuleProto* HloModuleGroupProto::mutable_hlo_modules(int index) { + // @@protoc_insertion_point(field_mutable:xla.HloModuleGroupProto.hlo_modules) + return hlo_modules_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::xla::HloModuleProto >* +HloModuleGroupProto::mutable_hlo_modules() { + // @@protoc_insertion_point(field_mutable_list:xla.HloModuleGroupProto.hlo_modules) + return &hlo_modules_; +} +inline const ::xla::HloModuleProto& HloModuleGroupProto::hlo_modules(int index) const { + // @@protoc_insertion_point(field_get:xla.HloModuleGroupProto.hlo_modules) + return hlo_modules_.Get(index); +} +inline ::xla::HloModuleProto* HloModuleGroupProto::add_hlo_modules() { + // @@protoc_insertion_point(field_add:xla.HloModuleGroupProto.hlo_modules) + return hlo_modules_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::xla::HloModuleProto >& +HloModuleGroupProto::hlo_modules() const { + // @@protoc_insertion_point(field_list:xla.HloModuleGroupProto.hlo_modules) + return hlo_modules_; +} + +// ------------------------------------------------------------------- + +// BufferAssignmentProto_BufferAlias + +// int64 source_buffer_id = 1; +inline void BufferAssignmentProto_BufferAlias::clear_source_buffer_id() { + source_buffer_id_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 BufferAssignmentProto_BufferAlias::source_buffer_id() const { + // @@protoc_insertion_point(field_get:xla.BufferAssignmentProto.BufferAlias.source_buffer_id) + return source_buffer_id_; +} +inline void BufferAssignmentProto_BufferAlias::set_source_buffer_id(::google::protobuf::int64 value) { + + source_buffer_id_ = value; + // @@protoc_insertion_point(field_set:xla.BufferAssignmentProto.BufferAlias.source_buffer_id) +} + +// .xla.LogicalBufferProto.Location location = 2; +inline bool BufferAssignmentProto_BufferAlias::has_location() const { + return this != internal_default_instance() && location_ != NULL; +} +inline void BufferAssignmentProto_BufferAlias::clear_location() { + if (GetArenaNoVirtual() == NULL && location_ != NULL) { + delete location_; + } + location_ = NULL; +} +inline const ::xla::LogicalBufferProto_Location& BufferAssignmentProto_BufferAlias::_internal_location() const { + return *location_; +} +inline const ::xla::LogicalBufferProto_Location& BufferAssignmentProto_BufferAlias::location() const { + const ::xla::LogicalBufferProto_Location* p = location_; + // @@protoc_insertion_point(field_get:xla.BufferAssignmentProto.BufferAlias.location) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_LogicalBufferProto_Location_default_instance_); +} +inline ::xla::LogicalBufferProto_Location* BufferAssignmentProto_BufferAlias::release_location() { + // @@protoc_insertion_point(field_release:xla.BufferAssignmentProto.BufferAlias.location) + + ::xla::LogicalBufferProto_Location* temp = location_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + location_ = NULL; + return temp; +} +inline ::xla::LogicalBufferProto_Location* BufferAssignmentProto_BufferAlias::unsafe_arena_release_location() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.BufferAssignmentProto.BufferAlias.location) + + ::xla::LogicalBufferProto_Location* temp = location_; + location_ = NULL; + return temp; +} +inline ::xla::LogicalBufferProto_Location* BufferAssignmentProto_BufferAlias::mutable_location() { + + if (location_ == NULL) { + auto* p = CreateMaybeMessage<::xla::LogicalBufferProto_Location>(GetArenaNoVirtual()); + location_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.BufferAssignmentProto.BufferAlias.location) + return location_; +} +inline void BufferAssignmentProto_BufferAlias::set_allocated_location(::xla::LogicalBufferProto_Location* location) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete location_; + } + if (location) { + ::google::protobuf::Arena* submessage_arena = + ::google::protobuf::Arena::GetArena(location); + if (message_arena != submessage_arena) { + location = ::google::protobuf::internal::GetOwnedMessage( + message_arena, location, submessage_arena); + } + + } else { + + } + location_ = location; + // @@protoc_insertion_point(field_set_allocated:xla.BufferAssignmentProto.BufferAlias.location) +} + +// ------------------------------------------------------------------- + +// BufferAssignmentProto + +// repeated .xla.LogicalBufferProto logical_buffers = 1; +inline int BufferAssignmentProto::logical_buffers_size() const { + return logical_buffers_.size(); +} +inline void BufferAssignmentProto::clear_logical_buffers() { + logical_buffers_.Clear(); +} +inline ::xla::LogicalBufferProto* BufferAssignmentProto::mutable_logical_buffers(int index) { + // @@protoc_insertion_point(field_mutable:xla.BufferAssignmentProto.logical_buffers) + return logical_buffers_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::xla::LogicalBufferProto >* +BufferAssignmentProto::mutable_logical_buffers() { + // @@protoc_insertion_point(field_mutable_list:xla.BufferAssignmentProto.logical_buffers) + return &logical_buffers_; +} +inline const ::xla::LogicalBufferProto& BufferAssignmentProto::logical_buffers(int index) const { + // @@protoc_insertion_point(field_get:xla.BufferAssignmentProto.logical_buffers) + return logical_buffers_.Get(index); +} +inline ::xla::LogicalBufferProto* BufferAssignmentProto::add_logical_buffers() { + // @@protoc_insertion_point(field_add:xla.BufferAssignmentProto.logical_buffers) + return logical_buffers_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::xla::LogicalBufferProto >& +BufferAssignmentProto::logical_buffers() const { + // @@protoc_insertion_point(field_list:xla.BufferAssignmentProto.logical_buffers) + return logical_buffers_; +} + +// repeated .xla.BufferAssignmentProto.BufferAlias buffer_aliases = 2; +inline int BufferAssignmentProto::buffer_aliases_size() const { + return buffer_aliases_.size(); +} +inline void BufferAssignmentProto::clear_buffer_aliases() { + buffer_aliases_.Clear(); +} +inline ::xla::BufferAssignmentProto_BufferAlias* BufferAssignmentProto::mutable_buffer_aliases(int index) { + // @@protoc_insertion_point(field_mutable:xla.BufferAssignmentProto.buffer_aliases) + return buffer_aliases_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::xla::BufferAssignmentProto_BufferAlias >* +BufferAssignmentProto::mutable_buffer_aliases() { + // @@protoc_insertion_point(field_mutable_list:xla.BufferAssignmentProto.buffer_aliases) + return &buffer_aliases_; +} +inline const ::xla::BufferAssignmentProto_BufferAlias& BufferAssignmentProto::buffer_aliases(int index) const { + // @@protoc_insertion_point(field_get:xla.BufferAssignmentProto.buffer_aliases) + return buffer_aliases_.Get(index); +} +inline ::xla::BufferAssignmentProto_BufferAlias* BufferAssignmentProto::add_buffer_aliases() { + // @@protoc_insertion_point(field_add:xla.BufferAssignmentProto.buffer_aliases) + return buffer_aliases_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::xla::BufferAssignmentProto_BufferAlias >& +BufferAssignmentProto::buffer_aliases() const { + // @@protoc_insertion_point(field_list:xla.BufferAssignmentProto.buffer_aliases) + return buffer_aliases_; +} + +// repeated .xla.BufferAllocationProto buffer_allocations = 3; +inline int BufferAssignmentProto::buffer_allocations_size() const { + return buffer_allocations_.size(); +} +inline void BufferAssignmentProto::clear_buffer_allocations() { + buffer_allocations_.Clear(); +} +inline ::xla::BufferAllocationProto* BufferAssignmentProto::mutable_buffer_allocations(int index) { + // @@protoc_insertion_point(field_mutable:xla.BufferAssignmentProto.buffer_allocations) + return buffer_allocations_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::xla::BufferAllocationProto >* +BufferAssignmentProto::mutable_buffer_allocations() { + // @@protoc_insertion_point(field_mutable_list:xla.BufferAssignmentProto.buffer_allocations) + return &buffer_allocations_; +} +inline const ::xla::BufferAllocationProto& BufferAssignmentProto::buffer_allocations(int index) const { + // @@protoc_insertion_point(field_get:xla.BufferAssignmentProto.buffer_allocations) + return buffer_allocations_.Get(index); +} +inline ::xla::BufferAllocationProto* BufferAssignmentProto::add_buffer_allocations() { + // @@protoc_insertion_point(field_add:xla.BufferAssignmentProto.buffer_allocations) + return buffer_allocations_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::xla::BufferAllocationProto >& +BufferAssignmentProto::buffer_allocations() const { + // @@protoc_insertion_point(field_list:xla.BufferAssignmentProto.buffer_allocations) + return buffer_allocations_; +} + +// repeated .xla.HeapSimulatorTrace heap_simulator_traces = 4; +inline int BufferAssignmentProto::heap_simulator_traces_size() const { + return heap_simulator_traces_.size(); +} +inline void BufferAssignmentProto::clear_heap_simulator_traces() { + heap_simulator_traces_.Clear(); +} +inline ::xla::HeapSimulatorTrace* BufferAssignmentProto::mutable_heap_simulator_traces(int index) { + // @@protoc_insertion_point(field_mutable:xla.BufferAssignmentProto.heap_simulator_traces) + return heap_simulator_traces_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::xla::HeapSimulatorTrace >* +BufferAssignmentProto::mutable_heap_simulator_traces() { + // @@protoc_insertion_point(field_mutable_list:xla.BufferAssignmentProto.heap_simulator_traces) + return &heap_simulator_traces_; +} +inline const ::xla::HeapSimulatorTrace& BufferAssignmentProto::heap_simulator_traces(int index) const { + // @@protoc_insertion_point(field_get:xla.BufferAssignmentProto.heap_simulator_traces) + return heap_simulator_traces_.Get(index); +} +inline ::xla::HeapSimulatorTrace* BufferAssignmentProto::add_heap_simulator_traces() { + // @@protoc_insertion_point(field_add:xla.BufferAssignmentProto.heap_simulator_traces) + return heap_simulator_traces_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::xla::HeapSimulatorTrace >& +BufferAssignmentProto::heap_simulator_traces() const { + // @@protoc_insertion_point(field_list:xla.BufferAssignmentProto.heap_simulator_traces) + return heap_simulator_traces_; +} + +// ------------------------------------------------------------------- + +// HloProto + +// .xla.HloModuleProto hlo_module = 1; +inline bool HloProto::has_hlo_module() const { + return this != internal_default_instance() && hlo_module_ != NULL; +} +inline void HloProto::clear_hlo_module() { + if (GetArenaNoVirtual() == NULL && hlo_module_ != NULL) { + delete hlo_module_; + } + hlo_module_ = NULL; +} +inline const ::xla::HloModuleProto& HloProto::_internal_hlo_module() const { + return *hlo_module_; +} +inline const ::xla::HloModuleProto& HloProto::hlo_module() const { + const ::xla::HloModuleProto* p = hlo_module_; + // @@protoc_insertion_point(field_get:xla.HloProto.hlo_module) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_HloModuleProto_default_instance_); +} +inline ::xla::HloModuleProto* HloProto::release_hlo_module() { + // @@protoc_insertion_point(field_release:xla.HloProto.hlo_module) + + ::xla::HloModuleProto* temp = hlo_module_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + hlo_module_ = NULL; + return temp; +} +inline ::xla::HloModuleProto* HloProto::unsafe_arena_release_hlo_module() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.HloProto.hlo_module) + + ::xla::HloModuleProto* temp = hlo_module_; + hlo_module_ = NULL; + return temp; +} +inline ::xla::HloModuleProto* HloProto::mutable_hlo_module() { + + if (hlo_module_ == NULL) { + auto* p = CreateMaybeMessage<::xla::HloModuleProto>(GetArenaNoVirtual()); + hlo_module_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.HloProto.hlo_module) + return hlo_module_; +} +inline void HloProto::set_allocated_hlo_module(::xla::HloModuleProto* hlo_module) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete hlo_module_; + } + if (hlo_module) { + ::google::protobuf::Arena* submessage_arena = + ::google::protobuf::Arena::GetArena(hlo_module); + if (message_arena != submessage_arena) { + hlo_module = ::google::protobuf::internal::GetOwnedMessage( + message_arena, hlo_module, submessage_arena); + } + + } else { + + } + hlo_module_ = hlo_module; + // @@protoc_insertion_point(field_set_allocated:xla.HloProto.hlo_module) +} + +// .xla.BufferAssignmentProto buffer_assignment = 3; +inline bool HloProto::has_buffer_assignment() const { + return this != internal_default_instance() && buffer_assignment_ != NULL; +} +inline void HloProto::clear_buffer_assignment() { + if (GetArenaNoVirtual() == NULL && buffer_assignment_ != NULL) { + delete buffer_assignment_; + } + buffer_assignment_ = NULL; +} +inline const ::xla::BufferAssignmentProto& HloProto::_internal_buffer_assignment() const { + return *buffer_assignment_; +} +inline const ::xla::BufferAssignmentProto& HloProto::buffer_assignment() const { + const ::xla::BufferAssignmentProto* p = buffer_assignment_; + // @@protoc_insertion_point(field_get:xla.HloProto.buffer_assignment) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_BufferAssignmentProto_default_instance_); +} +inline ::xla::BufferAssignmentProto* HloProto::release_buffer_assignment() { + // @@protoc_insertion_point(field_release:xla.HloProto.buffer_assignment) + + ::xla::BufferAssignmentProto* temp = buffer_assignment_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + buffer_assignment_ = NULL; + return temp; +} +inline ::xla::BufferAssignmentProto* HloProto::unsafe_arena_release_buffer_assignment() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.HloProto.buffer_assignment) + + ::xla::BufferAssignmentProto* temp = buffer_assignment_; + buffer_assignment_ = NULL; + return temp; +} +inline ::xla::BufferAssignmentProto* HloProto::mutable_buffer_assignment() { + + if (buffer_assignment_ == NULL) { + auto* p = CreateMaybeMessage<::xla::BufferAssignmentProto>(GetArenaNoVirtual()); + buffer_assignment_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.HloProto.buffer_assignment) + return buffer_assignment_; +} +inline void HloProto::set_allocated_buffer_assignment(::xla::BufferAssignmentProto* buffer_assignment) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete buffer_assignment_; + } + if (buffer_assignment) { + ::google::protobuf::Arena* submessage_arena = + ::google::protobuf::Arena::GetArena(buffer_assignment); + if (message_arena != submessage_arena) { + buffer_assignment = ::google::protobuf::internal::GetOwnedMessage( + message_arena, buffer_assignment, submessage_arena); + } + + } else { + + } + buffer_assignment_ = buffer_assignment; + // @@protoc_insertion_point(field_set_allocated:xla.HloProto.buffer_assignment) +} + +// ------------------------------------------------------------------- + +// HloSnapshot + +// .xla.HloProto hlo = 1; +inline bool HloSnapshot::has_hlo() const { + return this != internal_default_instance() && hlo_ != NULL; +} +inline void HloSnapshot::clear_hlo() { + if (GetArenaNoVirtual() == NULL && hlo_ != NULL) { + delete hlo_; + } + hlo_ = NULL; +} +inline const ::xla::HloProto& HloSnapshot::_internal_hlo() const { + return *hlo_; +} +inline const ::xla::HloProto& HloSnapshot::hlo() const { + const ::xla::HloProto* p = hlo_; + // @@protoc_insertion_point(field_get:xla.HloSnapshot.hlo) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_HloProto_default_instance_); +} +inline ::xla::HloProto* HloSnapshot::release_hlo() { + // @@protoc_insertion_point(field_release:xla.HloSnapshot.hlo) + + ::xla::HloProto* temp = hlo_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + hlo_ = NULL; + return temp; +} +inline ::xla::HloProto* HloSnapshot::unsafe_arena_release_hlo() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.HloSnapshot.hlo) + + ::xla::HloProto* temp = hlo_; + hlo_ = NULL; + return temp; +} +inline ::xla::HloProto* HloSnapshot::mutable_hlo() { + + if (hlo_ == NULL) { + auto* p = CreateMaybeMessage<::xla::HloProto>(GetArenaNoVirtual()); + hlo_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.HloSnapshot.hlo) + return hlo_; +} +inline void HloSnapshot::set_allocated_hlo(::xla::HloProto* hlo) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete hlo_; + } + if (hlo) { + ::google::protobuf::Arena* submessage_arena = + ::google::protobuf::Arena::GetArena(hlo); + if (message_arena != submessage_arena) { + hlo = ::google::protobuf::internal::GetOwnedMessage( + message_arena, hlo, submessage_arena); + } + + } else { + + } + hlo_ = hlo; + // @@protoc_insertion_point(field_set_allocated:xla.HloSnapshot.hlo) +} + +// repeated .xla.LiteralProto arguments = 2; +inline int HloSnapshot::arguments_size() const { + return arguments_.size(); +} +inline ::xla::LiteralProto* HloSnapshot::mutable_arguments(int index) { + // @@protoc_insertion_point(field_mutable:xla.HloSnapshot.arguments) + return arguments_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::xla::LiteralProto >* +HloSnapshot::mutable_arguments() { + // @@protoc_insertion_point(field_mutable_list:xla.HloSnapshot.arguments) + return &arguments_; +} +inline const ::xla::LiteralProto& HloSnapshot::arguments(int index) const { + // @@protoc_insertion_point(field_get:xla.HloSnapshot.arguments) + return arguments_.Get(index); +} +inline ::xla::LiteralProto* HloSnapshot::add_arguments() { + // @@protoc_insertion_point(field_add:xla.HloSnapshot.arguments) + return arguments_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::xla::LiteralProto >& +HloSnapshot::arguments() const { + // @@protoc_insertion_point(field_list:xla.HloSnapshot.arguments) + return arguments_; +} + +// .xla.LiteralProto result = 3; +inline bool HloSnapshot::has_result() const { + return this != internal_default_instance() && result_ != NULL; +} +inline const ::xla::LiteralProto& HloSnapshot::_internal_result() const { + return *result_; +} +inline const ::xla::LiteralProto& HloSnapshot::result() const { + const ::xla::LiteralProto* p = result_; + // @@protoc_insertion_point(field_get:xla.HloSnapshot.result) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_LiteralProto_default_instance_); +} +inline ::xla::LiteralProto* HloSnapshot::release_result() { + // @@protoc_insertion_point(field_release:xla.HloSnapshot.result) + + ::xla::LiteralProto* temp = result_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + result_ = NULL; + return temp; +} +inline ::xla::LiteralProto* HloSnapshot::unsafe_arena_release_result() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.HloSnapshot.result) + + ::xla::LiteralProto* temp = result_; + result_ = NULL; + return temp; +} +inline ::xla::LiteralProto* HloSnapshot::mutable_result() { + + if (result_ == NULL) { + auto* p = CreateMaybeMessage<::xla::LiteralProto>(GetArenaNoVirtual()); + result_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.HloSnapshot.result) + return result_; +} +inline void HloSnapshot::set_allocated_result(::xla::LiteralProto* result) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(result_); + } + if (result) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(result)->GetArena(); + if (message_arena != submessage_arena) { + result = ::google::protobuf::internal::GetOwnedMessage( + message_arena, result, submessage_arena); + } + + } else { + + } + result_ = result; + // @@protoc_insertion_point(field_set_allocated:xla.HloSnapshot.result) +} + +// string execution_platform = 4; +inline void HloSnapshot::clear_execution_platform() { + execution_platform_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& HloSnapshot::execution_platform() const { + // @@protoc_insertion_point(field_get:xla.HloSnapshot.execution_platform) + return execution_platform_.Get(); +} +inline void HloSnapshot::set_execution_platform(const ::std::string& value) { + + execution_platform_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:xla.HloSnapshot.execution_platform) +} +#if LANG_CXX11 +inline void HloSnapshot::set_execution_platform(::std::string&& value) { + + execution_platform_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:xla.HloSnapshot.execution_platform) +} +#endif +inline void HloSnapshot::set_execution_platform(const char* value) { + GOOGLE_DCHECK(value != NULL); + + execution_platform_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:xla.HloSnapshot.execution_platform) +} +inline void HloSnapshot::set_execution_platform(const char* value, + size_t size) { + + execution_platform_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:xla.HloSnapshot.execution_platform) +} +inline ::std::string* HloSnapshot::mutable_execution_platform() { + + // @@protoc_insertion_point(field_mutable:xla.HloSnapshot.execution_platform) + return execution_platform_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* HloSnapshot::release_execution_platform() { + // @@protoc_insertion_point(field_release:xla.HloSnapshot.execution_platform) + + return execution_platform_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void HloSnapshot::set_allocated_execution_platform(::std::string* execution_platform) { + if (execution_platform != NULL) { + + } else { + + } + execution_platform_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), execution_platform, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:xla.HloSnapshot.execution_platform) +} +inline ::std::string* HloSnapshot::unsafe_arena_release_execution_platform() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.HloSnapshot.execution_platform) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return execution_platform_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void HloSnapshot::unsafe_arena_set_allocated_execution_platform( + ::std::string* execution_platform) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (execution_platform != NULL) { + + } else { + + } + execution_platform_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + execution_platform, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:xla.HloSnapshot.execution_platform) +} + +#ifdef __GNUC__ + #pragma GCC diagnostic pop +#endif // __GNUC__ +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + + +// @@protoc_insertion_point(namespace_scope) + +} // namespace xla + +namespace google { +namespace protobuf { + +template <> struct is_proto_enum< ::xla::HeapSimulatorTrace_Event_Kind> : ::std::true_type {}; +template <> +inline const EnumDescriptor* GetEnumDescriptor< ::xla::HeapSimulatorTrace_Event_Kind>() { + return ::xla::HeapSimulatorTrace_Event_Kind_descriptor(); +} + +} // namespace protobuf +} // namespace google + +// @@protoc_insertion_point(global_scope) + +#endif // PROTOBUF_INCLUDED_tensorflow_2fcompiler_2fxla_2fservice_2fhlo_2eproto diff --git a/linux/include/tensorflow/compiler/xla/service/hlo_alias_analysis.h b/linux/include/tensorflow/compiler/xla/service/hlo_alias_analysis.h new file mode 100644 index 000000000..372f99ff0 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/hlo_alias_analysis.h @@ -0,0 +1,123 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_HLO_ALIAS_ANALYSIS_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_HLO_ALIAS_ANALYSIS_H_ + +#include +#include +#include + +#include "absl/container/flat_hash_map.h" +#include "absl/types/span.h" +#include "tensorflow/compiler/xla/service/hlo_buffer.h" +#include "tensorflow/compiler/xla/service/hlo_dataflow_analysis.h" +#include "tensorflow/compiler/xla/service/hlo_instruction.h" +#include "tensorflow/compiler/xla/service/hlo_module.h" +#include "tensorflow/compiler/xla/service/hlo_ordering.h" +#include "tensorflow/compiler/xla/status.h" +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/platform/macros.h" + +namespace xla { + +// Analysis which allocates HloBuffers to HloValues. +class HloAliasAnalysis { + public: + // The callgraph of the given HloModule must be flattened + // (xla::FlattenCallGraph) prior to running the analysis. + static StatusOr> Run( + HloModule* module, + const HloDataflowAnalysis::FusionCanShareBufferFunction& + fusion_can_share_buffer); + + string ToString() const; + + // Return the buffer containing the given value. + const HloBuffer& GetBufferContainingValue(const HloValue& value) const { + return *value_to_buffer_.at(&value); + } + HloBuffer& GetBufferContainingValue(const HloValue& value) { + return *value_to_buffer_.at(&value); + } + + // Return the HloBuffer with the given ID. + const HloBuffer& GetBuffer(HloBuffer::Id buffer_id) const { + return buffers_.at(buffer_id); + } + HloBuffer& GetBuffer(HloBuffer::Id buffer_id) { + return buffers_.at(buffer_id); + } + + // Returns the unique buffer at the given position. CHECK fails if the buffer + // set at that position does not contain exactly one buffer. + const HloBuffer& GetUniqueBufferAt(const HloInstruction* instruction, + const ShapeIndex& index = {}) const; + HloBuffer& GetUniqueBufferAt(const HloInstruction* instruction, + const ShapeIndex& index = {}); + + // Compute the set of buffers at the given instruction and index and return as + // a vector. This set is exactly the union of the buffers containing the + // HloValues at this position. + std::vector ComputeBuffersAt( + const HloInstruction* instruction, const ShapeIndex& index = {}) const; + + // Return a vector of all HloBuffers stabily sorted by HloBuffer::Id. This + // vector is lazily computed. Mutating operations on HloAliasAnalysis may + // invalidate the underlying vector requiring recomputation. + const std::vector& buffers() const { return buffers_; } + + // Returns the underlying dataflow analysis used by this alias analysis. + const HloDataflowAnalysis& dataflow_analysis() const { + return *dataflow_analysis_; + } + + // Returns true if any index in the output of the given instruction has more + // than one buffer. That is, ComputeBuffersAt returns a vector with more than + // one element. + bool InstructionBuffersAreAmbiguous(const HloInstruction* instruction) const; + + // Returns true if no HloBuffer appears in more than one shape index in the + // output of the given instruction. + bool InstructionBuffersAreDistinct(const HloInstruction* instruction) const; + + // Returns true if any HLO values in the module have interfering live ranges + // assuming the given ordering. + bool HasLiveRangeInterference(const HloOrdering& ordering) const; + + protected: + explicit HloAliasAnalysis(HloModule* module); + + // Verify various invariants of the alias analysis. + Status Verify() const; + + HloModule* module_; + + // The underlying dataflow analysis used by this alias analysis. + std::unique_ptr dataflow_analysis_; + + // A map indicating which buffer a value is contained in. + absl::flat_hash_map value_to_buffer_; + + // A lazily constructed vector containing all HloBuffers sorted by + // HloBuffer::Id. + std::vector buffers_; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_HLO_ALIAS_ANALYSIS_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/hlo_buffer.h b/linux/include/tensorflow/compiler/xla/service/hlo_buffer.h new file mode 100644 index 000000000..a88c87e46 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/hlo_buffer.h @@ -0,0 +1,123 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_HLO_BUFFER_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_HLO_BUFFER_H_ + +#include +#include +#include + +#include "tensorflow/compiler/xla/service/hlo_value.h" +#include "tensorflow/compiler/xla/shape_tree.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/platform/macros.h" + +namespace xla { + +// A container which can hold one or more HloValues. An HLO buffer abstractly +// represents the allocation which HLO instructions write into and read +// from. Generally there is a one-to-one correspondence between HloBuffers and +// HloValue where each HloValue in the module is held in a unique HloBuffer. An +// exception is the while instruction which updates the loop state in-place. In +// this case, we have a single HloBuffer for each HloPosition in the loop state, +// but multiple HloValues. For example: +// +// %init = ... +// %while = While(%init, body, condition) +// +// body: +// %body_param = Param(0) +// ... +// %body_root = ... +// +// condition: +// %cond_param = Param(0) +// ... +// +// For simplicity, assume that %while is array-shaped. In this case, we have a +// single HloBuffer which holds the following HloValues: HloValue{%init}, +// HloValue{%while}, HloValue{%body_param}, HloValue{%body_root}, and +// HloValue{%cond_param}. +// +// HloBuffers may appear at different HloPositions in the module mirroring the +// same propery of HloValues. For example: +// +// %sub = Sub(...) +// %add = Add(...) +// %tuple = Tuple(%add, %sub) +// %gte = GetTupleElement(%tuple, 0) +// +// In this case, the HloBuffer containing %add appears at the following +// positions: HloPosition{%add, {}}, HloPosition{%tuple, {0}}, and +// HloPosition{%gte, {}}. +// +// Different HloPositions which share the same HloBuffer indicate mandatory +// aliasing in the HLO module. These positions must share the same memory +// allocation for correctness (the backends rely on this property). This differs +// from incidental aliasing introduced by memory reuse in BufferAssignment where +// different instructions may happen to get the same allocation. +class HloBuffer { + public: + using Id = int64; + + // Predicate comparing HloBuffers by increasing id, useful for std::sort. + static bool IdLessThan(const HloBuffer* a, const HloBuffer* b) { + return a->id() < b->id(); + } + + // Predicate comparing HloBuffers by equal id, useful for std::unique. + static bool IdEqual(const HloBuffer* a, const HloBuffer* b) { + return a->id() == b->id(); + } + + HloBuffer(Id id, absl::Span values) + : id_(id), values_(values.begin(), values.end()) {} + + // Return the unique identifier for this HloBuffer. + Id id() const { return id_; } + + // Return all values contained in this buffer. + const std::vector& values() const { return values_; } + + // Return the unique HLO value in the buffer. CHECK fails if the buffer does + // not contain exactly one value. + const HloValue& GetUniqueValue() const { + CHECK_EQ(values_.size(), 1); + return *values_[0]; + } + + std::vector ComputePositions() const; + + string ToString() const; + + bool operator==(const HloBuffer& other) const; + bool operator!=(const HloBuffer& other) const { return !(*this == other); } + + private: + // Unique identifier for this HloBuffer. + const Id id_; + + // The set of values contained in this buffer. Vector contains no duplicates + // and is sorted stably by HloValue::Id. + const std::vector values_; +}; + +std::ostream& operator<<(std::ostream& out, const HloBuffer& buffer); + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_HLO_BUFFER_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/hlo_casting_utils.h b/linux/include/tensorflow/compiler/xla/service/hlo_casting_utils.h new file mode 100644 index 000000000..7f73bba03 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/hlo_casting_utils.h @@ -0,0 +1,104 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// Casting utilitiy functions for HLO instructions. + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_HLO_CASTING_UTILS_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_HLO_CASTING_UTILS_H_ + +#include +#include "tensorflow/core/platform/logging.h" + +namespace xla { + +class HloInstruction; + +template +using EnableIfDerivedFromHlo = + typename std::enable_if::value>::type; + +// TODO(b/93238915): Switch implementation from C++'s dynamic_cast to LLVM-like +// RTTI if it turns out to be a performance issue. +// Casts an HloInstruction pointer to one of its subclasses, dies if argument is +// nullptr or runtime information does not match. +// +// Similar to LLVM's cast. +template * = nullptr> +const T* Cast(const HloInstruction* instruction) { + CHECK(instruction != nullptr); + const T* casted = dynamic_cast(instruction); + CHECK(casted != nullptr); + return casted; +} + +// Non-const overload of Cast. +template * = nullptr> +T* Cast(HloInstruction* instruction) { + return const_cast( + Cast(const_cast(instruction))); +} + +// Works just like the Cast, except that it allows for a null pointer as an +// argument which it then propagates. +// +// Similar to LLVM's cast_or_null. +template * = nullptr> +const T* CastOrNull(const HloInstruction* instruction) { + return instruction != nullptr ? Cast(instruction) : nullptr; +} + +// Non-const overload of CastOrNull. +template * = nullptr> +T* CastOrNull(HloInstruction* instruction) { + return const_cast( + CastOrNull(const_cast(instruction))); +} + +// Casts an HloInstruction pointer to one of its subclasses, dies if argument is +// nullptr, returns nullptr if runtime information does not match. +// +// Similar to LLVM's dyn_cast. +template * = nullptr> +const T* DynCast(const HloInstruction* instruction) { + CHECK(instruction != nullptr); + return dynamic_cast(instruction); +} + +// Non-const overload of DynCast. +template * = nullptr> +T* DynCast(HloInstruction* instruction) { + return const_cast( + DynCast(const_cast(instruction))); +} + +// Works just like the DynCast, except that it allows for a null pointer as an +// argument which it then propagates. +// +// Similar to LLVM's dyn_cast_or_null. +template * = nullptr> +const T* DynCastOrNull(const HloInstruction* instruction) { + return instruction != nullptr ? DynCast(instruction) : nullptr; +} + +// Non-const overload of DynCastOrNull. +template * = nullptr> +T* DynCastOrNull(HloInstruction* instruction) { + return const_cast( + DynCastOrNull(const_cast(instruction))); +} + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_HLO_CASTING_UTILS_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/hlo_clone_context.h b/linux/include/tensorflow/compiler/xla/service/hlo_clone_context.h new file mode 100644 index 000000000..24910ca07 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/hlo_clone_context.h @@ -0,0 +1,95 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_HLO_CLONE_CONTEXT_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_HLO_CLONE_CONTEXT_H_ + +#include + +#include "absl/container/flat_hash_map.h" +#include "tensorflow/compiler/xla/map_util.h" + +namespace xla { + +class HloInstruction; +class HloComputation; +class HloModule; + +// Data structure used to track the cloning of HloInstruction and HloComputation +// objects. +class HloCloneContext { + public: + // Creates a new HloCloneContext object to clone HloInstruction and + // HloComputation objects to be added to the module specified as argument. + // The suffix string will be appended to computation names. + explicit HloCloneContext(HloModule* module, const string& suffix = "") + : module_(module), suffix_(suffix) {} + + HloModule* module() const { return module_; } + + const string& suffix() const { return suffix_; } + + void MapInstruction(const HloInstruction* old_instruction, + HloInstruction* new_instruction) { + instructions_[old_instruction] = new_instruction; + } + + void MapComputation(const HloComputation* old_computation, + HloComputation* new_computation) { + computations_[old_computation] = new_computation; + } + + // Finds the new instruction mapped to its old copy, or return nullptr in case + // it is not found. + HloInstruction* FindInstruction(const HloInstruction* old_instruction) const { + return FindOrDefault(instructions_, old_instruction, nullptr); + } + + // Finds the new computation mapped to its old copy, or return nullptr in case + // it is not found. + HloComputation* FindComputation(const HloComputation* old_computation) const { + return FindOrDefault(computations_, old_computation, nullptr); + } + + // Retrieves the new instruction mapped to its old copy, or fail if not found. + HloInstruction* GetInstruction(const HloInstruction* old_instruction) const { + return FindOrDie(instructions_, old_instruction); + } + + // Retrieves the new computation mapped to its old copy, or fail if not found. + HloComputation* GetComputation(const HloComputation* old_computation) const { + return FindOrDie(computations_, old_computation); + } + + const absl::flat_hash_map& + cloned_instructions() const { + return instructions_; + } + + const absl::flat_hash_map& + cloned_computations() const { + return computations_; + } + + private: + HloModule* module_; + string suffix_; + absl::flat_hash_map instructions_; + absl::flat_hash_map computations_; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_HLO_CLONE_CONTEXT_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/hlo_computation.h b/linux/include/tensorflow/compiler/xla/service/hlo_computation.h new file mode 100644 index 000000000..d87ab4bda --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/hlo_computation.h @@ -0,0 +1,452 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_HLO_COMPUTATION_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_HLO_COMPUTATION_H_ + +#include +#include +#include +#include +#include +#include +#include +#include + +#include "absl/container/flat_hash_map.h" +#include "absl/container/flat_hash_set.h" +#include "absl/types/span.h" +#include "tensorflow/compiler/xla/iterator_util.h" +#include "tensorflow/compiler/xla/map_util.h" +#include "tensorflow/compiler/xla/service/dfs_hlo_visitor.h" +#include "tensorflow/compiler/xla/service/dfs_hlo_visitor_with_default.h" +#include "tensorflow/compiler/xla/service/hlo.pb.h" +#include "tensorflow/compiler/xla/service/hlo_clone_context.h" +#include "tensorflow/compiler/xla/service/hlo_instruction.h" +#include "tensorflow/compiler/xla/service/hlo_reachability.h" +#include "tensorflow/compiler/xla/service/name_uniquer.h" +#include "tensorflow/compiler/xla/shape_tree.h" +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/types.h" + +namespace xla { + +class HloModule; + +// Describes a computation at the HLO level. +// +// You can think of an HloComputation like a function. It has some inputs +// (parameters) and returns exactly one value (the value of its root node). If +// you want to return multiple values, you can return a tuple. +// +// The instructions inside of a computation do not have an explicit total order. +// Instead, they have a partial order determined by their data and control +// dependencies. +// +// An HloModule contains one "entry computation" -- this is like main() in a C +// program. Every other computation inside of a module is attached to one or +// more HloInstructions, as a "nested computation". For example, the kMap +// instruction has a nested computation and "applies" it to every element of its +// input, elementwise. (That is, the input [x, y, z] is transformed to [f(x), +// f(y), f(z)].) +class HloComputation { + public: + // Builder class for HloComputation. + class Builder { + public: + explicit Builder(const string& name, + HloInstruction* fusion_instruction = nullptr) + : name_(name), + last_added_instruction_(nullptr), + fusion_instruction_(fusion_instruction) {} + + // Build and return an HloComputation. The parameter root_instruction + // specifies the already-added instruction to use as the root. If + // root_instruction is nullptr then use the last added instruction as the + // root. + std::unique_ptr Build( + HloInstruction* root_instruction = nullptr); + + HloInstruction* AddInstruction( + std::unique_ptr instruction) { + instructions_.push_back(std::move(instruction)); + last_added_instruction_ = instructions_.back().get(); + return last_added_instruction_; + } + + Status ForEachInstruction( + const std::function& func) const { + for (const auto& instruction : instructions_) { + TF_RETURN_IF_ERROR(func(instruction.get())); + } + return Status::OK(); + } + + private: + const string name_; + HloInstruction* last_added_instruction_; + HloInstruction* fusion_instruction_; + std::vector> instructions_; + }; + + // Add an instruction to the computation. The computation takes ownership of + // the instruction. + HloInstruction* AddInstruction(std::unique_ptr instruction); + + // Remove the param_no'th parameter from the computation. + // Note this is only applicatable to the computation for the fusion + // instruction. + Status RemoveParameter(int64 param_no); + + // Remove unused parameters from the computation. + // Note this is only applicatable to the computation for the fusion + // instruction. + Status RemoveUnusedParameters(); + + // Add new parameter instruction to the computation. + // This should be a new parameter. Instruction will be appended to parameters + // and inserted to the instruction list. + HloInstruction* AddParameter(std::unique_ptr instruction); + + // Remove an instruction from the computation. The instruction must have no + // users. Instruction is deallocated with this call. + Status RemoveInstruction(HloInstruction* instruction); + + // Remove an instruction from the computation and also transitively any + // operand that has no users post removing an instruction. The instruction + // must have no users. Instruction is deallocated with this call. + Status RemoveInstructionAndUnusedOperands(HloInstruction* instruction); + + // Set the root of the computation to the given instruction. The instruction + // must have already been added to the computation. In addition it must have + // the same shape as the result of the computation for non fusion + // computations, except if accept_different_shape is set to true. + void set_root_instruction(HloInstruction* new_root_instruction, + bool accept_different_shape = false); + + // Return the root instruction of the computation. The root instruction is the + // instruction which produces the output of the computation. + HloInstruction* root_instruction() const { return root_instruction_; } + + // Returns the number of parameters for this computation. + int64 num_parameters() const { return param_instructions_.size(); } + + // Returns the parameter instruction for the given parameter number. + HloInstruction* parameter_instruction(int64 param_no) const { + CHECK_GE(param_no, 0); + CHECK_LT(param_no, static_cast(param_instructions_.size())) + << "Computation " << name() << " has no parameter number " << param_no; + return param_instructions_[param_no]; + } + + const std::vector& parameter_instructions() const { + return param_instructions_; + } + + const string& name() const { return name_; } + + // Use the given NameUniquer to select a unique name for the computation based + // on the computation's existing name. + void UniquifyName(NameUniquer* name_uniquer); + + // Return a string representation of the computation. + // + // (We express the default options using an overload rather than a default + // param because gdb ignores default params, but does resolve overloads.) + string ToString() const { return ToString(HloPrintOptions()); } + string ToString(const HloPrintOptions& options) const; + + // Overload which accepts an order to emit the instructions in. + string ToString( + const HloPrintOptions& options, + absl::Span instruction_order) const; + + // Returns a serialized representation of this computation. + HloComputationProto ToProto() const; + + // Creates a computation from the given proto. Arguments: + // + // proto: the proto to convert from. + // computation_map: a map from computation id to HloComputation*. This map + // must contain all computations which the newly constructed computation + // calls. + static StatusOr> CreateFromProto( + const HloComputationProto& proto, + const absl::flat_hash_map& computation_map); + + // Gets the instructions in this computation. + // + // The returned type is a range of HloInstruction*s, so you can iterate over + // it using a range-based for loop in the natural way: + // + // for (HloInstruction* instr : computation->instructions()) { ... } + // + tensorflow::gtl::iterator_range>::const_iterator>> + instructions() const { + return {MakeUnwrappingIterator(instructions_.begin()), + MakeUnwrappingIterator(instructions_.end())}; + } + tensorflow::gtl::iterator_range< + UnwrappingIterator>::iterator>> + instructions() { + return {MakeUnwrappingIterator(instructions_.begin()), + MakeUnwrappingIterator(instructions_.end())}; + } + + // Compute and return a post-order of the instructions in the computation. In + // this order, definitions of values always appear before their uses. + std::vector MakeInstructionPostOrder() const; + + // Computes and returns the reachability between HLO instructions in the + // computation. The returned HloReachabilityMap is constructed such that + // HloReachabilityMap::IsReachable(a, b) returns true iff there exists a + // directed path (from producer to consumer) from 'a' to 'b'. Both data + // dependencies (operands) and control dependencies are considered for + // reachability. Trivially an instruction is reachable from itself. + std::unique_ptr ComputeReachability() const; + + // Updates the given reachability map after the immediate predecessor set + // (operands and control predecessors) of 'instruction' has changed. + void UpdateReachabilityThroughInstruction( + const HloInstruction* instruction, HloReachabilityMap* reachability_map); + + int64 instruction_count() const { return instruction_iterators_.size(); } + + // Creates and returns a list of the embedded computations called by this + // computation. This includes all embedded computations called directly or + // transitively. The embedded computations are sorted such that if computation + // A calls computation B (eg, via a map instruction) then A will appear after + // B in the list. + std::vector MakeEmbeddedComputationsList() const; + + // Creates a fusion instruction containing the given instructions. + // `fusion_kind` indicates the type of the fusion, e.g., loop fusion or fusion + // into a library call. Instructions must be in reverse topological order + // (root of the fused expression first). Replaces all uses of the original + // root instruction with the fusion instruction. The original instructions are + // removed if they have no uses after fusion (this is necessarily true for at + // least the root). + HloInstruction* CreateFusionInstruction( + absl::Span instructions_to_fuse, + HloInstruction::FusionKind fusion_kind); + + // Create a deep copy of the given instruction and return the instruction + // producing the copied result. All instructions performing the copy are added + // to the computation. For array-shaped values, this method trivially returns + // a kCopy instruction. For tuple-shaped instructions, the copy is performed + // with a series of kGetTupleElement and kTuple instructions. If + // indices_to_copy is non-null then this ShapeTree indicates which elements + // (arrays) of the shape to copy. Non-copied elements are passed through + // transparently. If copies_added is non-null, then the added kCopy + // instructions will be inserted in the respective index in the given + // ShapeTree. + StatusOr DeepCopyInstruction( + HloInstruction* instruction, + const ShapeTree* indices_to_copy = nullptr, + ShapeTree* copies_added = nullptr); + + // As above, but uses a custom function to copy the leaf nodes, which could + // create alternative HLOs other than kCopy, or even pass-throughs. + StatusOr DeepCopyInstructionWithCustomCopier( + HloInstruction* instruction, + const std::function< + HloInstruction*(HloInstruction* leaf, const ShapeIndex& leaf_index, + HloComputation* computation)>& copy_leaf); + + // Computes and returns the ProgramShape of this computation (shape of + // parameters and result with layout). + ProgramShape ComputeProgramShape() const; + + // Return whether `*this` and `other` are functionally equivalent. + bool operator==(const HloComputation& other) const; + + // Replaces old instruction with newly created instruction. Removes old + // instruction from computation. Updates uses and root instruction. + Status ReplaceWithNewInstruction( + HloInstruction* old_instruction, + std::unique_ptr new_instruction); + + // Replace old instruction with new instruction. Updates uses and root + // instruction. Removes old instruction from computation. Precondition: + // old_instruction and new_instruction must have the compatible shapes. + Status ReplaceInstruction(HloInstruction* old_instruction, + HloInstruction* new_instruction); + + // Set/get the module containing this computation. + void set_parent(HloModule* module) { parent_ = module; } + const HloModule* parent() const { return parent_; } + HloModule* parent() { return parent_; } + + // Visit every node in the computation in DFS post-order with the given + // visitor. This is similar to calling HloInstruction::Accept on the root of + // the computation except this method also visits instructions not reachable + // via the root. The root instruction of the computation is visited last, and + // the visitor's FinishVisit method is called once upon completion (with the + // root instruction as the argument). + template + Status Accept(DfsHloVisitorBase* visitor) const; + + // Same as Accept() above, but the order of operand and control predecessor + // visitation is determined by the given operand order; if compare(A, B) == + // true, A is visited before B. + Status AcceptWithOperandOrder( + DfsHloVisitor* visitor, + const HloInstruction::CompareFunction& operand_order) const; + + // Visit every node in the computation in the given order. 'order' must + // be a topological sort of all instructions in the computation. + template + Status AcceptOrdered(DfsHloVisitorBase* visitor, + const std::vector& order) const; + + // Same as Accept() above, but the visitor is given as a function. + Status Accept(const std::function& visitor_func); + Status Accept( + const std::function& visitor_func) const; + + // Returns a deep copy of this computation including all instructions. + // If the clone context is specified, it will be populated with the cloned + // object mappings, and its module() will be used to add new computations + // into. + std::unique_ptr Clone(const string& suffix = "clone", + HloCloneContext* context = nullptr); + + // Like Clone(), but if an instruction is present in replacement_map, we use + // the map's value to replace that instruction in the cloned computation. + // + // If replacements maps a key to nullptr, we remove that instruction from the + // new computation. + // If additional instructions are used by instructions in replacement map, + // they must be passed in post-order in the extras span. + std::unique_ptr CloneWithReplacements( + std::unordered_map> + replacements, + absl::Span extras, HloCloneContext* context = nullptr, + const string& suffix = "clone"); + + // Returns true if the given instruction can be removed from the computation. + // Parameter instructions cannot be removed without violating invariants of + // the HLO computation with the exception of fusion computation. A parameter + // instruction is removable for a fusion computation. + // + // Note that IsRemovable() is a necessariy condition to remove an instruction + // rather than a sufficient condition. For example, instructions with + // side-effect (e.g., Send, Infeed) may be removed from a computation, but the + // transformation must guarantee the invariants relevant to the instructions + // still hold (e.g., Send and Recv must be removed together to make each + // channel complete). + bool IsRemovable(const HloInstruction* instruction); + + // Returns true if this computation has a side effect. A computation has a + // side effect if it contains one or more instructions with a side effect. + bool HasSideEffect() const; + + // Returns if this computation is a fusion computation. + bool IsFusionComputation() const { return fusion_instruction_ != nullptr; } + + // Returns the owning fusion instruction, or nullptr if this is not a fusion + // computation. + HloInstruction* FusionInstruction() const { return fusion_instruction_; } + void SetFusionInstruction(HloInstruction* fusion_instruction) { + fusion_instruction_ = fusion_instruction; + } + + // The id of this computation should be unique within the module. + void SetUniqueId(int64 id) { + CHECK_EQ(unique_id_, -1); + CHECK_GE(id, 0); + unique_id_ = id; + } + + // Returns the instruction in this computation that has name `name`. Returns + // null if there is no such computation. + HloInstruction* GetInstructionWithName(absl::string_view name); + + int64 unique_id() const { return unique_id_; } + + private: + explicit HloComputation( + const string& name, int parameter_count, + std::vector>* instructions, + HloInstruction* root_instruction, HloInstruction* fusion_instruction); + + // Internal helper for adding instructions. + HloInstruction* AddInstructionInternal( + std::unique_ptr instruction); + + // Fuses HLOs in instructions_to_fuse into fusion_instruction. + // + // Pre-condition: fusion_instruction's opcode is kFusion. + void FuseInstructionsInto( + absl::Span instructions_to_fuse, + HloInstruction* fusion_instruction); + + // Internal helper for recursive copying of an instruction. Creates and + // returns a deep copy of the given instruction. + StatusOr DeepCopyHelper( + HloInstruction* instruction, ShapeIndex* index, + const std::function< + HloInstruction*(HloInstruction* leaf, const ShapeIndex& leaf_index, + HloComputation* computation)>& copy_leaf); + + // Internal helper to collect unreachable roots. + std::vector CollectUnreachableRoots() const; + + // Returns a map from channel-id to directed dependencies of the channel + // instructions. For send&recv pairs it means the send instruction and for + // cross-replica-sum the union of the dependencies for all participating + // instructions. + using ChannelDependencyMap = + absl::flat_hash_map>; + ChannelDependencyMap ComputeChannelDependencies() const; + + enum VisitState { kVisiting, kVisited }; + void ComputeInstructionPostOrder( + const HloComputation::ChannelDependencyMap& channel_dependency_map, + std::vector* post_order, HloInstruction* root, + absl::flat_hash_map* visited) const; + + string name_; + int64 unique_id_; + HloInstruction* root_instruction_; + + // If this computation is a fusion computation, this field points to the + // corresponding fusion instruction. Otherwise, this is null. + HloInstruction* fusion_instruction_; + + // Module containing this computation. + HloModule* parent_ = nullptr; + + // Store instructions in std::list as they can be added and removed + // arbitrarily and we want a stable iteration order. Keep a map from + // instruction pointer to location in the list for fast lookup. + using InstructionList = std::list>; + InstructionList instructions_; + absl::flat_hash_map + instruction_iterators_; + + std::vector param_instructions_; + + TF_DISALLOW_COPY_AND_ASSIGN(HloComputation); +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_HLO_COMPUTATION_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/hlo_constant_folding.h b/linux/include/tensorflow/compiler/xla/service/hlo_constant_folding.h new file mode 100644 index 000000000..4a624cc7b --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/hlo_constant_folding.h @@ -0,0 +1,37 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_HLO_CONSTANT_FOLDING_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_HLO_CONSTANT_FOLDING_H_ + +#include "tensorflow/compiler/xla/service/hlo_module.h" +#include "tensorflow/compiler/xla/service/hlo_pass_interface.h" + +namespace xla { + +// A pass which performs constant folding in order to avoid unnecessary +// computation on constants. +class HloConstantFolding : public HloModulePass { + public: + absl::string_view name() const override { return "constant_folding"; } + + // Run constant folding operations on the given module. Returns whether the + // module was changed (constant expressions folded). + StatusOr Run(HloModule* module) override; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_HLO_CONSTANT_FOLDING_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/hlo_cost_analysis.h b/linux/include/tensorflow/compiler/xla/service/hlo_cost_analysis.h new file mode 100644 index 000000000..46b4bbeef --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/hlo_cost_analysis.h @@ -0,0 +1,206 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_HLO_COST_ANALYSIS_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_HLO_COST_ANALYSIS_H_ + +#include "absl/types/span.h" +#include "tensorflow/compiler/xla/service/dfs_hlo_visitor.h" +#include "tensorflow/compiler/xla/service/hlo_computation.h" +#include "tensorflow/compiler/xla/service/hlo_instruction.h" +#include "tensorflow/compiler/xla/service/hlo_opcode.h" +#include "tensorflow/compiler/xla/shape_util.h" +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/types.h" + +namespace xla { + +// HloCostAnalysis traverses an HLO graph and calculates the amount of +// computations required for the graph. Each HLO instruction handler provides +// the computation cost of the instruction, and the values are accumulated +// during the traversal for the entire graph. We treat normal floating point +// operations separately from transcendental operations. +class HloCostAnalysis : public ConstDfsHloVisitor { + public: + // Each HLO is associated to a vector of properties with the indices given + // below. Sub-classes can add further properties. + typedef std::map Properties; + static constexpr char kFlopsKey[] = "flops"; + static constexpr char kTranscendentalsKey[] = "transcendentals"; + static constexpr char kBytesAccessedKey[] = "bytes accessed"; + static constexpr char kOptimalSecondsKey[] = "optimal_seconds"; + + // shape_size is a function which returns the size in bytes of the top-level + // buffer of a shape. + using ShapeSizeFunction = std::function; + explicit HloCostAnalysis(const ShapeSizeFunction& shape_size); + + Status HandleElementwiseUnary(const HloInstruction* hlo) override; + Status HandleElementwiseBinary(const HloInstruction* hlo) override; + Status HandleConstant(const HloInstruction* constant) override; + Status HandleIota(const HloInstruction* iota) override; + Status HandleGetTupleElement( + const HloInstruction* get_tuple_element) override; + Status HandleSelect(const HloInstruction* hlo) override; + Status HandleTupleSelect(const HloInstruction* hlo) override; + Status HandleCompare(const HloInstruction* compare) override; + Status HandleClamp(const HloInstruction* clamp) override; + Status HandleReducePrecision(const HloInstruction* hlo) override; + Status HandleConcatenate(const HloInstruction* concatenate) override; + Status HandleSend(const HloInstruction* send) override; + Status HandleSendDone(const HloInstruction* send_done) override; + Status HandleRecv(const HloInstruction* recv) override; + Status HandleRecvDone(const HloInstruction* recv_done) override; + Status HandleConvert(const HloInstruction* convert) override; + Status HandleCopy(const HloInstruction* copy) override; + Status HandleDomain(const HloInstruction* domain) override; + Status HandleDot(const HloInstruction* dot) override; + Status HandleConvolution(const HloInstruction* convolution) override; + Status HandleFft(const HloInstruction* fft) override; + Status HandleCrossReplicaSum(const HloInstruction* crs) override; + Status HandleAllToAll(const HloInstruction* hlo) override; + Status HandleCollectivePermute(const HloInstruction* hlo) override; + Status HandleInfeed(const HloInstruction* infeed) override; + Status HandleOutfeed(const HloInstruction* outfeed) override; + Status HandleRng(const HloInstruction* random) override; + Status HandleReverse(const HloInstruction* reverse) override; + Status HandleSort(const HloInstruction* sort) override; + Status HandleParameter(const HloInstruction* parameter) override; + Status HandleReduce(const HloInstruction* reduce) override; + Status HandleBatchNormTraining( + const HloInstruction* batch_norm_training) override; + Status HandleBatchNormInference( + const HloInstruction* batch_norm_inference) override; + Status HandleBatchNormGrad(const HloInstruction* batch_norm_grad) override; + Status HandleFusion(const HloInstruction* fusion) override; + Status HandleCall(const HloInstruction* call) override; + Status HandleCustomCall(const HloInstruction* custom_call) override; + Status HandleSlice(const HloInstruction* slice) override; + Status HandleDynamicSlice(const HloInstruction* dynamic_slice) override; + Status HandleDynamicUpdateSlice( + const HloInstruction* dynamic_update_slice) override; + Status HandleTuple(const HloInstruction* tuple) override; + Status HandleMap(const HloInstruction* map) override; + Status HandleReduceWindow(const HloInstruction* reduce_window) override; + Status HandleSelectAndScatter(const HloInstruction* instruction) override; + Status HandleBitcast(const HloInstruction* bitcast) override; + Status HandleBroadcast(const HloInstruction* broadcast) override; + Status HandlePad(const HloInstruction* pad) override; + Status HandleReshape(const HloInstruction* reshape) override; + Status HandleAfterAll(const HloInstruction* token) override; + Status HandleTranspose(const HloInstruction* transpose) override; + Status HandleWhile(const HloInstruction* xla_while) override; + Status HandleConditional(const HloInstruction* conditional) override; + Status HandleGather(const HloInstruction* gather) override; + Status HandleScatter(const HloInstruction* scatter) override; + Status FinishVisit(const HloInstruction* root) override; + + Status Preprocess(const HloInstruction* hlo) override; + Status Postprocess(const HloInstruction* hlo) override; + + // Set the rates used to calculate the time taken by the computation. These + // need to be set before visiting starts. + void set_flops_per_second(float value) { + per_second_rates_[kFlopsKey] = value; + } + void set_transcendentals_per_second(float value) { + per_second_rates_[kTranscendentalsKey] = value; + } + void set_bytes_per_second(float value) { + per_second_rates_[kBytesAccessedKey] = value; + } + + // Returns properties for the computation. + float flop_count() const; + float transcendental_count() const; + float bytes_accessed() const; + float optimal_seconds() const; + + // Returns the respective cost computed for a particular HLO instruction, or 0 + // if the HLO was not found to have a cost in the analysis. + int64 flop_count(const HloInstruction& hlo) const; + int64 transcendental_count(const HloInstruction& hlo) const; + int64 bytes_accessed(const HloInstruction& hlo) const; + float optimal_seconds(const HloInstruction& hlo) const; + + const Properties& properties() const { return properties_sum_; } + const float property(const string& key) const { + return GetProperty(key, properties()); + } + + protected: + typedef std::unordered_map HloToProperties; + + // An FMA counts as two floating point operations in these analyzes. + static constexpr int64 kFmaFlops = 2; + + HloCostAnalysis(const ShapeSizeFunction& shape_size, + const Properties& per_second_rates); + + // Returns the properties computed from visiting the computation rooted at the + // given hlo. + StatusOr ProcessSubcomputation(HloComputation* computation); + + // Utility function to handle all element-wise operations. + Status HandleElementwiseOp(const HloInstruction* hlo_instruction); + + // Returns the default value if the key is not present in the + // properties. Otherwise, returns the value that the key maps to from the + // properties parameter. + static float GetProperty(const string& key, const Properties& properties, + float default_value = 0.0f); + + // Returns 0.0f if the hlo is not present in hlo_to_properties or if the key + // is not present in hlo_to_properties[hlo]. Otherwise, returns the value that + // the key maps to in the properties of the given hlo. + static float GetPropertyForHlo(const HloInstruction& hlo, const string& key, + const HloToProperties& hlo_to_properties); + + // Decorates shape_size_ by returning 0 immediately if the shape does not have + // a layout. + int64 GetShapeSize(const Shape& shape) const; + + // Function which computes the size of the top-level of a given shape (not + // including nested elements, if any). If null then bytes_accessed methods + // return an error. + const ShapeSizeFunction shape_size_; + + HloToProperties hlo_properties_; + + // If true, the time taken will be computed from the rates for each property + // and the total time will be the maximum time, which is the time of the + // bottleneck. + bool current_should_compute_bottleneck_time_; + + // The properties of the currently visited instruction. A HandleFoo method can + // modify these to change the default values computed in Preprocess. + Properties current_properties_; + + // The sum of the properties of all HLOs in the computation. + Properties properties_sum_; + + // How much of each property can be processed per second. E.g. if the property + // is bytes accessed, this is the number of bytes that can be processed per + // second. Is empty if no rates have been set. + Properties per_second_rates_; + + TF_DISALLOW_COPY_AND_ASSIGN(HloCostAnalysis); +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_HLO_COST_ANALYSIS_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/hlo_creation_utils.h b/linux/include/tensorflow/compiler/xla/service/hlo_creation_utils.h new file mode 100644 index 000000000..8e5ddbbd5 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/hlo_creation_utils.h @@ -0,0 +1,213 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_HLO_CREATION_UTILS_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_HLO_CREATION_UTILS_H_ + +#include "tensorflow/compiler/xla/literal_util.h" +#include "tensorflow/compiler/xla/service/hlo_computation.h" +#include "tensorflow/compiler/xla/service/hlo_instruction.h" +#include "tensorflow/compiler/xla/statusor.h" + +namespace xla { + +// Some lightweight utilities intended to make HLO instruction creation more +// ergonomic. We don't have a complete set of helpers yet -- I expect we'll +// expand this interface as needed on an ad-hoc basis. + +// Creates a binary HLO instruction and adds it to the computation containing +// `lhs` and `rhs` (`lhs` and `rhs` must be in the same computation). +StatusOr MakeBinaryHlo(HloOpcode opcode, HloInstruction* lhs, + HloInstruction* rhs); + +// Creates a pad HLO instruction and adds it to the computation containing +// `operand` and `padding_value` (`operand` and `padding_value` must be in the +// same computation). +StatusOr MakePadHlo(HloInstruction* operand, + HloInstruction* padding_value, + const PaddingConfig& padding_config); + +// Creates a slice HLO instruction and adds it to the computation containing +// `operand`. +StatusOr MakeSliceHlo(HloInstruction* operand, + absl::Span start_indices, + absl::Span limit_indices, + absl::Span strides); + +// Creates a convolution HLO instruction and adds it to the computation +// containing `lhs` and `rhs` (`lhs` and `rhs` must be in the same computation). +StatusOr MakeConvolveHlo( + HloInstruction* lhs, HloInstruction* rhs, int64 feature_group_count, + const Window& window, const ConvolutionDimensionNumbers& dimension_numbers, + const PrecisionConfig& precision_config); + +// Creates a transpose HLO instruction and adds it to the computation containing +// `operand`. +StatusOr MakeTransposeHlo(HloInstruction* operand, + absl::Span dimensions); + +// Creates a reshape HLO instruction and adds it to the computation containing +// `operand`. +StatusOr MakeReshapeHlo(const Shape& result_shape, + HloInstruction* operand); + +StatusOr MakeReshapeHlo( + absl::Span result_shape_dim_bounds, HloInstruction* operand); + +// Creates a dynamic-slice HLO instruction and adds it to the computation +// containing `operand` and `start_indices` (`operand` and `start_indices` must +// be in the same computation). +StatusOr MakeDynamicSliceHlo( + HloInstruction* operand, HloInstruction* start_indices, + absl::Span slice_sizes); + +// Creates a dynamic-update-slice HLO instruction and adds it to the computation +// containing `operand`, `update` and `start_indices` (`operand`, `update` and +// `start_indices` must be in the same computation). +StatusOr MakeDynamicUpdateSliceHlo( + HloInstruction* operand, HloInstruction* update, + HloInstruction* start_indices); + +// Creates a broadcast HLO instruction and adds it to the computation containing +// `operand`. +StatusOr MakeBroadcastHlo( + HloInstruction* operand, absl::Span broadcast_dimensions, + absl::Span result_shape_bounds); + +// Creates a GetTupleElement HLO instruction and adds it to the computation +// containing `operand`. +StatusOr MakeGetTupleElementHlo(HloInstruction* operand, + int64 index); + +// Creates a Concatenate HLO instruction and adds it to the computation +// containing `operands` (`operands` must be non-empty and every element must be +// contained in the same computation). +StatusOr MakeConcatHlo( + absl::Span operands, int64 dimension); + +// Creates a Dot HLO instruction and adds it to the computation containing `lhs` +// and `rhs` (both must be in the same computation). +StatusOr MakeDotHlo(HloInstruction* lhs, HloInstruction* rhs, + const DotDimensionNumbers& dim_numbers, + const PrecisionConfig& precision_config); + +// Creates a Map HLO instruction and adds it to the computation containing the +// operands. All operands must be in the same computation. +StatusOr MakeMapHlo(absl::Span operands, + HloComputation* map_computation); + +// Creates a Reduce HLO instruction and adds it to the computation containing +// the operand. This will create the sub-computation needed for the reduction in +// the given module. binary_opcode should represent a binary operation. +StatusOr MakeReduceHlo(HloInstruction* operand, + HloInstruction* init_value, + HloOpcode binary_opcode, + HloModule* module); + +// Creates a Select HLO instruction and adds it to the computation containing +// the predicate. The on_true and on_false instructions must also be contained +// in the same computation. +StatusOr MakeSelectHlo(HloInstruction* pred, + HloInstruction* on_true, + HloInstruction* on_false); + +// Creates an R1 Constant HLO instruction of the given PrimitiveType with the +// given values and adds it to the given computation. +template +StatusOr MakeR1ConstantHlo(HloComputation* computation, + PrimitiveType type, + absl::Span values) { + Literal literal = LiteralUtil::CreateR1(values); + if (literal.shape().element_type() != type) { + TF_ASSIGN_OR_RETURN(literal, literal.Convert(type)); + } + return computation->AddInstruction( + HloInstruction::CreateConstant(std::move(literal))); +} + +// ----------------------------------------------------------------------------- +// Some other miscellaneous helpers to generate common HLO patterns. All of +// these add all the instructions they generate into the computation containing +// their operand(s). + +// Collapses (via reshape) the first N (logical) dimensions of `operand` into a +// single leading dimension. `operand` must have rank > `n` and `n` must not be +// 0. +// +// For instance if `operand` has shape f32[7,8,9] and n is 2 then the output is +// the `operand` reshaped to [56,9]. +StatusOr CollapseFirstNDims(HloInstruction* operand, int64 n); + +// Prepends `n` degenerate dimensions (dimensions with bound = 1) to `operand` +// using a reshape. +// +// For instance if operand has shape f32[3,4,5] then this returns the operand +// reshaped to f32[1,3,4,5]. If the operand is a f32 scalar (i.e. has shape +// f32[]) then this returns the operand reshaped to f32[1]. +StatusOr PrependDegenerateDims(HloInstruction* operand, + int64 n); + +// Expands (via reshape) the first (logical) dimension of `operand` into a +// sequence of `expanded_dims` dimensions. `operand` must at least be of rank 1 +// and the number of elements in its first dimension must be equal to the +// product of `expanded_dims`. +// +// For instance if `operand` has shape f32[200,9,7] and expanded_dims is +// {2,5,20} the result is `operand` reshaped to [2,5,20,9,7]. +StatusOr ExpandFirstDimIntoNDims( + HloInstruction* operand, absl::Span expanded_dims); + +// Elides (via reshape) a set of degenerate dimensions (dimensions containing +// exactly one element), `dims_to_elide` from `operand`. Every dimension in +// `dims_to_elide` must be a degenerate dimension. `dims_to_elide` must be +// sorted and not contain duplicates. +// +// For example if `operand` is of shape f32[19,1,20,1,7,1,9] and dims_to_elide +// is {1,5} then the result is `operand` reshaped to [19,20,1,7,9]. +StatusOr ElideDegenerateDims( + HloInstruction* operand, absl::Span dims_to_elide); + +// Inserts (via reshape) a set of degenerate dimensions (dimensions containing +// exactly one element), `dims_to_insert` into `operand`. The dimensions in +// `dims_to_insert` refer to the dimensions in the result, and hence should be +// less than the rank of the result. Also, `dims_to_insert` must be sorted. +// +// For example, if `operand` is of shape f32[12,21,8,34] and dims_to_insert is +// {0, 2}, then the result is `operand` reshaped to [1,12,1,21,8,34]. +StatusOr InsertDegenerateDims( + HloInstruction* operand, absl::Span dims_to_insert); + +// Pads `operand` (which must have rank 1) with `zeros_to_prepend` zeros in the +// front and `zeros_to_append` zeros in the back. +StatusOr PadVectorWithZeros(HloInstruction* operand, + int64 zeros_to_prepend, + int64 zeros_to_append); + +// Broadcasts a zero value of type `element_type` into a tensor with element +// type `element_type` and dimension bounds `broadcast_dimensions`. The +// broadcast instruction is emitted into `computation`. +StatusOr BroadcastZeros( + HloComputation* computation, PrimitiveType element_type, + absl::Span broadcast_dimensions); + +// Creates a HLO computation that takes arguments of type `domain` and produces +// a value of type `range`. +StatusOr> CreateComputationWithSignature( + absl::Span domain, const Shape& range, + absl::string_view name); + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_HLO_CREATION_UTILS_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/hlo_cse.h b/linux/include/tensorflow/compiler/xla/service/hlo_cse.h new file mode 100644 index 000000000..e4857fd3f --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/hlo_cse.h @@ -0,0 +1,50 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_HLO_CSE_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_HLO_CSE_H_ + +#include "tensorflow/compiler/xla/service/hlo_module.h" +#include "tensorflow/compiler/xla/service/hlo_pass_interface.h" + +namespace xla { + +// A pass which performs common-subexpression elimination. Identical constants +// and identical instructions with the same operands are commoned. The pass +// iterates over the instructions in topological order which enables the pass to +// find arbitrarily large common expressions. +class HloCSE : public HloModulePass { + public: + // If is_layout_sensitive is true, then the simplifier preserves layout during + // transformation. Otherwise, layout is ignored. + explicit HloCSE(bool is_layout_sensitive, + bool only_fusion_computations = false) + : is_layout_sensitive_(is_layout_sensitive), + only_fusion_computations_(only_fusion_computations) {} + ~HloCSE() override = default; + absl::string_view name() const override { return "cse"; } + + // Run CSE on the given module. Returns whether the module was changed (common + // subexpressions were found and eliminated). + StatusOr Run(HloModule* module) override; + + private: + const bool is_layout_sensitive_; + const bool only_fusion_computations_; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_HLO_CSE_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/hlo_dataflow_analysis.h b/linux/include/tensorflow/compiler/xla/service/hlo_dataflow_analysis.h new file mode 100644 index 000000000..e62c1c2ac --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/hlo_dataflow_analysis.h @@ -0,0 +1,253 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// Analysis for determining the possible set of values for all positions +// (instructions and ShapeIndexes) in the HLO module. Analysis is module-scoped +// tracking values across computation boundaries. + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_HLO_DATAFLOW_ANALYSIS_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_HLO_DATAFLOW_ANALYSIS_H_ + +#include +#include +#include +#include + +#include "absl/types/span.h" +#include "tensorflow/compiler/xla/service/call_graph.h" +#include "tensorflow/compiler/xla/service/hlo_instruction.h" +#include "tensorflow/compiler/xla/service/hlo_module.h" +#include "tensorflow/compiler/xla/service/hlo_value.h" +#include "tensorflow/compiler/xla/shape_util.h" +#include "tensorflow/compiler/xla/status.h" +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/platform/macros.h" + +namespace xla { + +// Analysis which identifies all HLO values and their uses in an HLO module. +class HloDataflowAnalysis { + public: + // Different backends can have very different ways to do fusion, so we give + // backends the flexibility to decide whether an fusion instruction can share + // buffer with it's operands. If this is not specified, a default strategy + // will be used; if this is specified, it will be applied *in addition* to the + // default strategy. + // + // The first parameter of the function should be the fusion instruction, the + // second parameter should be an operand of the fusion instruction. + // + // TODO(b/80315712): Find a better way to tell whether a fusion can share + // buffer. + using FusionCanShareBufferFunction = std::function; + + // Run dataflow analysis on the given module. Parameters: + // + // ssa_form : If true then new values are defined at the merge points of + // kWhile instructions. Abusing nomenclature somewhat, we call these "phi + // values". The merge is formed by the init value and loop backedge. The + // SSA form is minimal in that a new phi value is defined only if the + // merge point is reachable by multiple different values. The SSA form is + // also in loop-closed form in that no values defined inside of a loop + // (while body) is used outside of the loop. + // + // If ssa_form is false, then merge points do not define new + // values. Rather, the HloValueSet for the merge point contains the union + // of the merged HloValues. + // + // bitcast_defines_value : If true then the Bitcast HLO instruction defines + // a new HLO value in the analysis. If false then Bitcast forwards the + // value of its operand. + static StatusOr> Run( + const HloModule& module, bool ssa_form = false, + bool bitcast_defines_value = false, + const FusionCanShareBufferFunction& fusion_can_share_buffer = nullptr); + + static bool AreTransitiveUsesElementwiseOrTuple(const HloInstruction* inst); + + // Returns true if 'instruction' defines an HLO value at the given shape index + // of its output. + bool ValueIsDefinedAt(const HloInstruction* instruction, + const ShapeIndex& index = {}) const; + + // Return the HloValue defined by 'instruction' at the given shape index of + // its output. + // + // Precondition: ValueIsDefinedAt is true for this instruction and index. + const HloValue& GetValueDefinedAt(const HloInstruction* instruction, + const ShapeIndex& index = {}) const; + HloValue& GetValueDefinedAt(const HloInstruction* instruction, + const ShapeIndex& index = {}); + + // Return the InstructionValueSet for the given instruction. + const InstructionValueSet& GetInstructionValueSet( + const HloInstruction* instruction) const; + InstructionValueSet& GetInstructionValueSet( + const HloInstruction* instruction); + + // Return the HloValueSet for the given instruction at the given index or the + // given position. + const HloValueSet& GetValueSet(const HloInstruction* instruction, + const ShapeIndex& index = {}) const; + const HloValueSet& GetValueSet(const HloPosition& position) const; + HloValueSet& GetValueSet(const HloPosition& position); + HloValueSet& GetValueSet(const HloInstruction* instruction, + const ShapeIndex& index = {}); + + // Return the unique value in the HloValueSet at the given instruction and + // shape index. CHECKs if the value set does not contain a exactly one value. + const HloValue& GetUniqueValueAt(const HloInstruction* instruction, + const ShapeIndex& index = {}) const { + return GetValueSet(instruction, index).GetUniqueValue(); + } + HloValue& GetUniqueValueAt(const HloInstruction* instruction, + const ShapeIndex& index = {}) { + return GetValue(GetValueSet(instruction, index).GetUniqueValue().id()); + } + + // Return the HloValue with the given Id. + const HloValue& GetValue(HloValue::Id value_id) const; + HloValue& GetValue(HloValue::Id value_id); + + // Return the total number of HloValues. + int64 value_count() const { return values_.size(); } + + // Return a vector of all HloValues stabily sorted by HloValue::Id. + const std::vector& values() const { return values_vector_; } + + // Return the call graph used for computing the dataflow. + const CallGraph& call_graph() const { return *call_graph_; } + + string ToString() const; + + // Returns true if 'user' cannot possibly use the buffer at 'index' in + // 'operand'. Returns false otherwise. + // + // 'operand' does not have to be an operand of 'user'. This can be the case + // with indirect uses. + bool DoesNotUseOperandBuffer(const HloInstruction* operand, + const ShapeIndex& index, + const HloInstruction* user) const; + + // Returns true if 'user' (at 'user_index') can share a buffer with its + // operand 'operand' (at 'operand_index'). Returns false otherwise. + // + // REQUIRES: 'operand' is an operand of 'user'. + bool CanShareOperandBufferWithUser(HloInstruction* operand, + const ShapeIndex& operand_index, + HloInstruction* user, + const ShapeIndex& user_index) const; + + protected: + HloDataflowAnalysis( + const HloModule& module, bool ssa_form, + bool bitcast_defines_value = false, + const FusionCanShareBufferFunction& fusion_can_share_buffer = nullptr); + + // Returns a new HloValue defined at the given instruction and shape index. + HloValue* NewHloValue(HloInstruction* instruction, const ShapeIndex& index, + bool is_phi = false); + + // Mark the HloValue with the given ID for deletion. + void MarkValueForDeletion(HloValue::Id value_id); + + // Delete all HloValues marked for deletion. Should be called after + // propagation is complete. + void DeleteMarkedValues(); + + // Constructs and initializes the InstructionValueSets of all instructions to + // contain exactly the HloValues defined by each instruction. These values can + // then propagated throughout the HLO graph by calling Propagate. + Status InitializeInstructionValueSets(); + + // Updates the value set of the given instruction based on the values flowing + // into the instruction (operands and cross-computation dataflow). + bool UpdateInstructionValueSet(HloInstruction* instruction); + + // Updates the value set for a particular instruction type. Returns whether + // the instruction value set changed. + bool UpdateBitcastValueSet(HloInstruction* bitcast); + bool UpdateSliceValueSet(HloInstruction* slice); + bool UpdateCallValueSet(HloInstruction* call); + bool UpdateConditionalValueSet(HloInstruction* conditional); + bool UpdateCopyValueSet(HloInstruction* copy); + bool UpdateDomainValueSet(HloInstruction* domain); + bool UpdateGetTupleElementValueSet(HloInstruction* gte); + bool UpdateParameterValueSet(HloInstruction* parameter); + bool UpdateRecvDoneValueSet(HloInstruction* recv_done); + bool UpdateTupleSelectValueSet(HloInstruction* select); + bool UpdateSendValueSet(HloInstruction* send); + bool UpdateTupleValueSet(HloInstruction* tuple); + bool UpdateWhileValueSet(HloInstruction* xla_while); + + // Propagate the dataflow through the module. + void Propagate(); + + // Return the result of the SSA Phi function applied to the given inputs at + // the given instruction. If skip_top_level is true, then the top level of the + // value set of 'instruction' is not modified. + bool Phi(HloInstruction* instruction, + absl::Span inputs); + + // Updates the positions of the HloValues in the output of the given + // instruction. This should be called after the instruction value set of + // 'instruction' has been changed. 'prev_value_set' must point to the previous + // state of the value set prior to the change. 'prev_value_set' may be null if + // this is the first time positions are being computed. The previous state is + // necessary to efficiently remove positions which have been eliminated due to + // changes in the instructions' InstructionValueSet. + void UpdatePositionsOfValuesAt( + HloInstruction* instruction, const InstructionValueSet& new_value_set, + const InstructionValueSet* prev_value_set = nullptr); + + // Verify various invariants of the dataflow analysis. + Status Verify() const; + + const HloModule& module_; + const bool ssa_form_; + const bool bitcast_defines_value_; + + std::unique_ptr call_graph_; + + // The map of all HloValues in the module. We pass around pointers to the + // mapped HloValues, so the underlying container must keep them valid despite + // mutations touching other map entries. + std::unordered_map values_; + + // A map from instruction to InstructionValueSet. + std::unordered_map value_sets_; + + // Values marked for deletion during construction. We don't delete them + // immediately because references to them may remain in ValueSets temporarily + // during propagation. After construction, these values are deleted. + std::vector value_ids_to_delete_; + + // A vector containing all HloValues sorted by HloValue::Id. + std::vector values_vector_; + + // The Id to use for the next HloValue. + HloValue::Id next_value_id_ = 0; + + // Backend specific function that decides whether a fusion can share buffer + // with its operand. + FusionCanShareBufferFunction fusion_can_share_buffer_ = nullptr; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_HLO_DATAFLOW_ANALYSIS_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/hlo_dce.h b/linux/include/tensorflow/compiler/xla/service/hlo_dce.h new file mode 100644 index 000000000..401204267 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/hlo_dce.h @@ -0,0 +1,48 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_HLO_DCE_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_HLO_DCE_H_ + +#include "tensorflow/compiler/xla/service/hlo_computation.h" +#include "tensorflow/compiler/xla/service/hlo_instruction.h" +#include "tensorflow/compiler/xla/service/hlo_module.h" +#include "tensorflow/compiler/xla/service/hlo_pass_interface.h" +#include "tensorflow/compiler/xla/statusor.h" + +namespace xla { + +// HLO pass which removes dead instructions from each computation in the module +// and removes dead computations from the module. +// +// An instruction is dead if it is not reachable from the root. A computation is +// dead if it is not the entry computation of the module and it is not reachable +// from the entry computation. +// +// This pass does not remove dead parameter instructions, as parameter +// instructions cannot be deleted. +class HloDCE : public HloModulePass { + public: + ~HloDCE() override {} + absl::string_view name() const override { return "dce"; } + + // Run the pass on the given module. Returns whether the module was changed + // (instructions were removed). + StatusOr Run(HloModule* module) override; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_HLO_DCE_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/hlo_domain_map.h b/linux/include/tensorflow/compiler/xla/service/hlo_domain_map.h new file mode 100644 index 000000000..c8d581b74 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/hlo_domain_map.h @@ -0,0 +1,128 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_HLO_DOMAIN_MAP_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_HLO_DOMAIN_MAP_H_ + +#include +#include + +#include "absl/container/flat_hash_map.h" +#include "absl/container/flat_hash_set.h" +#include "tensorflow/compiler/xla/service/hlo_computation.h" +#include "tensorflow/compiler/xla/service/hlo_domain_metadata.h" +#include "tensorflow/compiler/xla/service/hlo_instruction.h" +#include "tensorflow/compiler/xla/service/hlo_module.h" +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/core/lib/core/status.h" + +namespace xla { + +// The HloDomainMap splits a set of instructions within a module or computation, +// into different domains, separated by kDomain instructions. +// A domain is composed by a set of instructions which can reach each other via +// operand/user edges, without crossing a kDomain insutrction of a given kind. +// A domain never crosses computation boundaries. +class HloDomainMap { + public: + // Creates a new HloDomainMap, creating all the domains within the input + // computation, of the given kind. If domain_kind is not empty, only the + // kDomain instructions of domain_kind will be considered as separators. + // Otherwise every kDomain instruction will be splitting domains. + static StatusOr> Create( + HloComputation* computation, string domain_kind); + + // Creates a new HloDomainMap, creating all the domains within the input + // module, of the given kind. If domain_kind is not empty, only the + // kDomain instructions of domain_kind will be considered as separators. + // Otherwise every kDomain instruction will be splitting domains. + static StatusOr> Create(HloModule* module, + string domain_kind); + + // Retrieves all the domains the input module or computation are composed by. + const std::vector>& GetDomains() + const { + return instruction_domains_; + } + + // Checks whether two instructions are within the same domain. + bool InSameDomain(HloInstruction* instruction1, + HloInstruction* instruction2) const; + + // Checks whether instruction is a kDomain instruction of the kind we are + // currently processing. + bool IsDomainInstruction(HloInstruction* instruction) const; + + // Retrieves the domain identifier of the instruction, or -1 in case + // instruction is not found within any domain. + int64 GetDomainId(HloInstruction* instruction) const; + + // Returns the unique id of the domain metadata for the domain the given + // instruction belongs to. The given instruction must not be a kDomain + // instruction since each domain instruction is associated with 2 domains. + int64 GetDomainMetadataId(HloInstruction* instruction) const; + + private: + // Map used for representing instruction ordering, i.e. + // order_map[a] < order_map[b] means a must be ordered before b. + using InstructionOrderMap = absl::flat_hash_map; + + HloDomainMap(string domain_kind) : domain_kind_(std::move(domain_kind)) {} + + // Check if the kDomain instruction is facing (via its operand link) another + // kDomain instruction of the same kind, hence defining an empty domain. + // If that is the case, create the empty domain and call the proper + // normalizer. + Status TryProcessEmptyDomain(HloInstruction* instruction); + + Status Populate(HloComputation* computation); + + // Inserts the provided domain into the ones tracked by this object, + // creating a new domain ID. + Status InsertDomain(std::unique_ptr domain); + + // From the given instruction, epxands operand and user wise, the set of + // instructions which can be reached without crossing a kDomain instruction + // of the kind specified by domain_kind_. + // The domain data structure will be populated with all the reached + // instructions, and the boundaries of the domain, with the kDomain + // instructions encountered while expanding the reach. + Status ExpandDomain(HloInstruction* instruction, + DomainMetadata::Domain* domain) const; + + // Creates a domain data structure using the ExpandDomain() API. + StatusOr> CreateDomain( + HloInstruction* instruction, + const InstructionOrderMap& instructions_order) const; + + // Out of an instruction set, returns a vector of all the ones which are not + // a kDomain kind. + static std::vector MakeNonDomainInstructions( + const absl::flat_hash_set& instruction_set, + const InstructionOrderMap& instructions_order); + + // Populates domain_metadata_id_ that maps each HloInstruction to the unique + // ID of its associated domain metatadata. + Status PopulateDomainMetadataMap(); + + string domain_kind_; + std::vector> instruction_domains_; + absl::flat_hash_map instruction_to_domain_; + absl::flat_hash_map domain_metadata_id_; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_HLO_DOMAIN_MAP_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/hlo_domain_metadata.h b/linux/include/tensorflow/compiler/xla/service/hlo_domain_metadata.h new file mode 100644 index 000000000..d3c83c15a --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/hlo_domain_metadata.h @@ -0,0 +1,84 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_HLO_DOMAIN_METADATA_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_HLO_DOMAIN_METADATA_H_ + +#include +#include +#include + +#include "absl/container/flat_hash_set.h" +#include "absl/strings/string_view.h" +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/core/lib/core/status.h" + +namespace xla { + +// Cannot include hlo_instruction.h as this file is included from there. +class HloInstruction; + +// The DomainMetadata represents the base class for metadata which can be +// attached to kDomain HLO instructions. +class DomainMetadata { + public: + // A Domain data structure captures all the information about a kDomain + // bounded instruction set. + struct Domain { + // The set of instructions which are reachable from each other via + // operand/user pathways, without crossing a kDomain instruction of a given + // kind. The reach_set can contain kDomain instructions of other kinds, if + // two domains of different kind intersect each other. + absl::flat_hash_set reach_set; + + // The same instructions in reach_set, but purged from kDomain instructions + // and ordered according to their computation graph post-order, i.e. + // if instructions[pos_a] depends on instructions[pos_b], then pos_a > + // pos_b. + std::vector instructions; + + // If we consider a graph edge as an arrow oriented from the operand to the + // user, the enter_domains will contain the set of kDomain instructions + // whose dataflow enters the reach set (domain), while the exit_domains + // contains the set of kDomain instructions whose dataflow exit the reach + // set. + absl::flat_hash_set enter_domains; + absl::flat_hash_set exit_domains; + }; + + virtual ~DomainMetadata() = default; + + // Clones the metadata object. + virtual std::unique_ptr Clone() const = 0; + + // Returns the metadata type. A unique identifier which describes the real + // metadata type. + virtual absl::string_view Kind() const = 0; + + // Compares the metadata object with another one and returns true if the + // two matches. + virtual bool Matches(const DomainMetadata& other) const = 0; + + // Returns the hash value of the metadata. + virtual size_t Hash() const = 0; + + // Returns a string representation of the metadata. + virtual string ToString() const = 0; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_HLO_DOMAIN_METADATA_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/hlo_element_type_converter.h b/linux/include/tensorflow/compiler/xla/service/hlo_element_type_converter.h new file mode 100644 index 000000000..4d2a94292 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/hlo_element_type_converter.h @@ -0,0 +1,47 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_HLO_ELEMENT_TYPE_CONVERTER_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_HLO_ELEMENT_TYPE_CONVERTER_H_ + +#include "tensorflow/compiler/xla/service/hlo_module.h" +#include "tensorflow/compiler/xla/service/hlo_pass_interface.h" + +namespace xla { + +// A pass that eliminates certain element types as the input or output of ops by +// inserting Convert ops. This allows a backend to support an element type while +// only actually implementing the Convert op for that element type. This is +// generally not the fastest approach, but it works. +class HloElementTypeConverter : public HloModulePass { + public: + // eliminate_type is the type to eliminate as the input or output of ops, + // using Convert ops to replace it with replace_with_type. + HloElementTypeConverter(PrimitiveType eliminate_type, + PrimitiveType replace_with_type); + + absl::string_view name() const override { return "element_type_converter"; } + + // Returns the pass on the module and returns whether the module was modified. + StatusOr Run(HloModule* module) override; + + private: + PrimitiveType eliminate_type_; + PrimitiveType replace_with_type_; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_HLO_ELEMENT_TYPE_CONVERTER_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/hlo_evaluator.h b/linux/include/tensorflow/compiler/xla/service/hlo_evaluator.h new file mode 100644 index 000000000..6c2662eba --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/hlo_evaluator.h @@ -0,0 +1,265 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_HLO_EVALUATOR_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_HLO_EVALUATOR_H_ + +#include + +#include "absl/memory/memory.h" +#include "absl/types/span.h" +#include "tensorflow/compiler/xla/service/dfs_hlo_visitor_with_default.h" +#include "tensorflow/compiler/xla/service/hlo_computation.h" +#include "tensorflow/compiler/xla/service/hlo_instruction.h" +#include "tensorflow/compiler/xla/service/hlo_module.h" +#include "tensorflow/compiler/xla/service/shape_inference.h" +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/compiler/xla/util.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/lib/gtl/flatmap.h" +#include "tensorflow/core/platform/macros.h" + +namespace xla { + +// Responsible for evaluating HLO and obtain literal as the evaluation results. +// +// This class is not thread-safe. +class HloEvaluator : public DfsHloVisitorWithDefault { + public: + // Only evaluate up to max_loop_iterations per while-loop execution if + // specified. + explicit HloEvaluator(int64 max_loop_iterations = -1); + + // Evaluates an HLO module and an array of pointers to literals. + // Returns the evaluated result as a literal if successful. + // Precondition: The indices of arg_literals correspond to the parameter + // numbers of the HLO parameters in the computation. See comment below for an + // example. + // `LiteralPtr` accepts either Literal or const Literal* + // type. + template + StatusOr Evaluate(const HloModule& module, + absl::Span arg_literals); + + // Evaluates an HLO computation and an array of pointers to literals. + // Returns the evaluated result as a literal if successful. + // Precondition: The indices of arg_literals correspond to the parameter + // numbers of the HLO parameters in the computation. For e.g., consider the + // following graph: + // + // * + // / \ + // + Parameter1 + // / \ + // / \ + // Parameter0 Constant + // + // where Parameter0 has parameter_number 0 and Parameter1 has parameter_number + // 1 in this computation. The input literals array will then have its first + // literal map to Parameter0 and the second map to Parameter1. + // `LiteralPtr` accepts either Literal or const Literal* + // type. + template + StatusOr Evaluate(const HloComputation& computation, + absl::Span arg_literals); + + // Evaluates a single HLO instruction and an array of pointers to literals. + // Return the evaluated result as literal if successful. + // Precondition: + // 1. argument literals correspond to the input instruction's parameters in + // their post-ordering. + // 2. the instruction's operands must be of either Parameter or Constant type. + // `LiteralPtr` accepts either Literal or const Literal* + // type. + template + StatusOr Evaluate(HloInstruction* instruction, + absl::Span arg_literals); + + // Evaluates a single HLO instruction with constant operands. + // Returns the evaluated result as literal if successful. + // Precondition: + // 1. all operands of the input instruction are constants. + // 2. the instruction is not a Parameter operation. + StatusOr Evaluate(HloInstruction* instruction); + + // Same as Evaluate, except returning false on error and accepts an output + // pointer. + bool TryEvaluate(HloInstruction* instruction, Literal* result); + + // Evaluates a single HLO instruction, substituting the given literals for + // some of the instruction's operands. + // + // For example, given instruction = op(A, B, C) and the map + // {A = x, C = y}, this evaluates op(x, B, y). + StatusOr EvaluateWithSubstitutions( + const HloInstruction* instruction, + const std::unordered_map& + substitutions); + + StatusOr EvaluateElementwiseBinaryOp(HloOpcode opcode, + const Literal& lhs, + const Literal& rhs); + + StatusOr EvaluateElementwiseUnaryOp(HloOpcode opcode, + const Literal& operand); + + StatusOr EvaluateDotOp(const DotDimensionNumbers& dim_numbers, + const PrecisionConfig& precision_config, + const Literal& lhs, const Literal& rhs); + + protected: + // Make HloEvaluatorTypedVisitor a friend because it is logically part of this + // class. + // + // A straightforward implementation would be to make it a nested class + // declared and defined in hlo_evaluator.cc. Instead HloEvaluatorTypedVisitor + // lives as a separate class with its own header because its template gets + // instantiated many times and we want to use extern templates to shard out + // the compilation of those instantiations across multiple cc files. + template + friend class HloEvaluatorTypedVisitor; + + // Wraps around instruction handling to infer types before dispatching to + // the corresponding typed Visitor. + Status DefaultAction(HloInstruction* hlo) override { + return hlo->Visit(typed_visitors_.at(hlo->shape().element_type()).get()); + } + + Status Preprocess(HloInstruction* hlo) override; + + Status Postprocess(HloInstruction* hlo) override; + + // Operations that are type-agnostic or always return a specific type, such as + // HandleIsFinite where boolean is always returned. + // + Status HandleParameter(HloInstruction* parameter) override; + + Status HandleConstant(HloInstruction* constant) override; + + Status HandleConcatenate(HloInstruction* concatenate) override; + + Status HandleReshape(HloInstruction* reshape) override; + + Status HandleTranspose(HloInstruction* transpose) override; + + Status HandleIsFinite(HloInstruction* is_finite) override; + + Status HandleCompare(HloInstruction* compare) override; + + Status HandleTuple(HloInstruction* tuple) override; + + Status HandleGather(HloInstruction* gather) override; + + Status HandleGetTupleElement(HloInstruction* get_tuple_element) override; + + Status HandleCopy(HloInstruction* copy) override; + + Status HandleConditional(HloInstruction* conditional) override; + + Status HandleCall(HloInstruction* call) override; + + Status HandleFusion(HloInstruction* fusion) override; + + Status HandleWhile(HloInstruction* while_hlo) override; + + Status HandleSelect(HloInstruction* select) override; + + Status HandleTupleSelect(HloInstruction* tuple_select) override; + + Status HandleBroadcast(HloInstruction* broadcast) override; + + Status HandleAfterAll(HloInstruction* token) override; + + Status HandleSort(HloInstruction* sort) override; + + Status HandleReal(HloInstruction* real) override; + + Status HandleImag(HloInstruction* imag) override; + + Status HandleReduce(HloInstruction* reduce) override; + + // Returns the already-evaluated literal result for the instruction. + // A Constant instruction is considered evaluated and its literal will be + // returned directly without looking up the cache. + // Crash with log if the given instruction has not been evaluated previously. + const Literal& GetEvaluatedLiteralFor(const HloInstruction* hlo) { + if (hlo->IsConstant()) { + return hlo->literal(); + } + auto it = evaluated_.find(hlo); + CHECK(it != evaluated_.end()) + << "could not find evaluated value for: " << hlo->ToString(); + return it->second; + } + + // Tracks the HLO instruction and its evaluated literal result. + // TODO(b/35950897): have better memory management here to free instructions + // that are no longer a parent for any other subsequent instruction in + // post-orderring. + // Must be cleared for each evaluation. + // Storing Literal in place require the container to have pointer stability so + // we cannot use FlatMap any more. + std::unordered_map evaluated_; + + private: + template + static StatusOr ElementWiseUnaryOpImpl( + HloInstruction* instruction, + const std::function& unary_op, + const Literal& operand_literal) { + const auto shape = instruction->shape(); + const auto* operand = instruction->operand(0); + + // TODO(b/35950897, b/27796129): add DCHECK back once implicit broadcast is + // removed. + if (!ShapeUtil::SameDimensions(shape, operand->shape())) { + return Unimplemented( + "Implicit broadcasting is currently unsupported in HLO evaluator " + "Shape Mismatch: %s vs %s", + ShapeUtil::HumanString(shape), + ShapeUtil::HumanString(operand->shape())); + } + + Literal result(shape); + TF_RETURN_IF_ERROR( + result.Populate([&](absl::Span multi_index) { + return unary_op(operand_literal.Get(multi_index)); + })); + return std::move(result); + } + + // Map from a primitive type to its associated (templated) DfsHloVisitor. + // Note: the hash function here is only needed because current gcc std::hash + // does not specialize for enum types. This should however be fixed in the + // future: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=60970#c5 + tensorflow::gtl::FlatMap, + std::hash> + typed_visitors_; + + // Caches pointers to input literals, assuming they are in post-order. + // Literals are not owned by this class, and they must outlive the lifetime of + // each invocation to the Evaluate* method. + // Must be cleared for each evaluation. + std::vector arg_literals_; + + // Max loop iterations to execute with no maximum if negative. + int64 max_loop_iterations_; + + TF_DISALLOW_COPY_AND_ASSIGN(HloEvaluator); +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_HLO_EVALUATOR_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/hlo_evaluator_typed_visitor.h b/linux/include/tensorflow/compiler/xla/service/hlo_evaluator_typed_visitor.h new file mode 100644 index 000000000..b2d12c94b --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/hlo_evaluator_typed_visitor.h @@ -0,0 +1,2809 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_HLO_EVALUATOR_TYPED_VISITOR_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_HLO_EVALUATOR_TYPED_VISITOR_H_ + +#include + +#include "absl/algorithm/container.h" +#include "absl/container/inlined_vector.h" +#include "absl/memory/memory.h" +#include "absl/types/optional.h" +#include "tensorflow/compiler/xla/literal_util.h" +#include "tensorflow/compiler/xla/service/hlo_casting_utils.h" +#include "tensorflow/compiler/xla/service/hlo_evaluator.h" +#include "tensorflow/compiler/xla/service/hlo_instructions.h" +#include "tensorflow/compiler/xla/service/shape_inference.h" +#include "tensorflow/core/lib/core/casts.h" + +namespace xla { + +// TODO(b/79274244): We'd like these type traits to live inside of +// HloEvaluatorTypedVisitor so they don't pollute namespace xla, but that +// crashes clang in the frontend. +// +// Anyway this is relatively safe as-is because hlo_evaluator_typed_visitor.h is +// a "private" header that's not exposed outside of hlo_evaluator.cc. +template +using is_complex_t = std::is_same; +template +using is_complex64_t = std::is_same; + +// It's UB to use std::sort with std::less, because of NaNs. Define +// "safe" less functions which are actually strict weak orders. -NaN and NaN +// should appear at the beginning and end of the ordering, and -0.0 should +// appear before 0.0. +template < + typename NativeT, + typename std::enable_if::value>::type* = nullptr> +bool SafeLess(const NativeT& a, const NativeT& b) { + return a < b; +} + +template ::value>::type* = nullptr> +bool SafeLess(const NativeT& a, const NativeT& b) { + bool lhs_is_negative = std::signbit(a); + bool rhs_is_negative = std::signbit(b); + // If the signs are different, we can just compare the signs. + if (lhs_is_negative != rhs_is_negative) { + return lhs_is_negative && !rhs_is_negative; + } + bool lhs_nan = std::isnan(a); + bool rhs_nan = std::isnan(b); + // Exactly one number is nan? + if (lhs_nan != rhs_nan) { + if (lhs_nan) { + return lhs_is_negative; + } + return !rhs_is_negative; + } + return a < b; +} + +template ::value || + std::is_same::value>::type* = nullptr> +bool SafeLess(const NativeT& a, const NativeT& b) { + return SafeLess(static_cast(a), static_cast(b)); +} + +// Templated DfsHloVisitor for use by HloEvaluator. +// +// Typically ReturnT here indicates the resulting literal type of each evaluated +// Handle* method of a TypedVisitor. There are however a few notable exceptions +// to this rule, notably: +// - HandleCompare and HandleIsFinite: where the resulting literal type is +// always boolean. +// - HandleImag and HandleReal: where the resulting literal type is always float +// and the operand is always complex, or real in the case of HandleReal. +// These operations are handled outside of the parent HloEvaluator handlers +// instead of from within TypedVisitor. +// +// Type params: +// - ReturnT: The type of input and output of each operation. +// - ElementwiseT: The type in which internal computation are done. +// +// This a logically a private part of HloEvaluator. It lives in this header +// file rather than in hlo_evaluator.cc because we use extern templates and a +// bunch of independent cc files to speed up compiling the many instantiations +// of this class. +template +class HloEvaluatorTypedVisitor : public DfsHloVisitorWithDefault { + private: + // Get the value in the given literal static_cast as a double. + template < + typename NativeT, + typename std::enable_if::value>::type* = nullptr> + double GetAsDouble(const Literal& literal, + absl::Span input_index) { + return static_cast(literal.Get(input_index)); + } + + // Specialization for complex types. In this case it is not possible to + // static_cast value to a double so just CHECK fail. This method is not used + // at run-time, but must be available at compile-time to keep the compiler + // happy. + template < + typename NativeT, + typename std::enable_if::value>::type* = nullptr> + double GetAsDouble(const Literal& literal, + absl::Span input_index) { + LOG(FATAL) << "Trying to get complex literal as double: " + << literal.ToString(); + } + + public: + explicit HloEvaluatorTypedVisitor(HloEvaluator* p) : parent_(p) {} + + // The following higher-order functions convert a function with ElementwiseT + // to a function with ReturnT. + std::function ConvertUnaryFunction( + const std::function& unary_op) { + return [&unary_op](ReturnT arg) { + return static_cast(unary_op(static_cast(arg))); + }; + } + std::function ConvertBinaryFunction( + const std::function& + binary_op) { + return [&binary_op](ReturnT arg1, ReturnT arg2) { + return static_cast(binary_op(static_cast(arg1), + static_cast(arg2))); + }; + } + std::function ConvertTernaryFunction( + const std::function& ternary_op) { + return [&ternary_op](ReturnT arg1, ReturnT arg2, ReturnT arg3) { + return static_cast(ternary_op(static_cast(arg1), + static_cast(arg2), + static_cast(arg3))); + }; + } + + Status DefaultAction(HloInstruction* hlo_instruction) override { + return Unimplemented("unhandled HLO ops for HloEvaluator: %s.", + HloOpcodeString(hlo_instruction->opcode())); + } + + // TODO(b/35950897): many of the stl functions used in the handlers are not + // overloaded for every XLA primitive type. + + template ::value>::type* = + nullptr> + Status HandleAbs(HloInstruction* abs) { + TF_ASSIGN_OR_RETURN(parent_->evaluated_[abs], + ElementWiseUnaryOp(abs, [](NativeT elem_operand) { + return elem_operand; + })); + return Status::OK(); + } + + template < + typename NativeT, + typename std::enable_if::value>::type* = nullptr> + Status HandleAbs(HloInstruction* abs) { + TF_ASSIGN_OR_RETURN(parent_->evaluated_[abs], + ElementWiseUnaryOp(abs, [](NativeT elem_operand) { + return std::abs(elem_operand); + })); + return Status::OK(); + } + + template < + typename NativeT, + typename std::enable_if::value>::type* = nullptr> + Status HandleAbs(HloInstruction* abs) { + const Literal& operand_literal = + parent_->GetEvaluatedLiteralFor(abs->operand(0)); + TF_ASSIGN_OR_RETURN( + parent_->evaluated_[abs], + (HloEvaluator::ElementWiseUnaryOpImpl( + abs, [](NativeT elem_operand) { return std::abs(elem_operand); }, + operand_literal))); + + return Status::OK(); + } + + Status HandleAbs(HloInstruction* abs) override { + // If the operand is of C64 type, the return type of abs will be F32. + // However, ElementwiseT would still be the return type, F32, and thus + // specifying the ElementwiseT explicitly as C64 is needed below. + if (abs->operand(0)->shape().element_type() == C64) { + return HandleAbs(abs); + } + return HandleAbs(abs); + } + + template < + typename NativeT, + typename std::enable_if::value>::type* = nullptr> + Status HandleRound(HloInstruction* round) { + TF_ASSIGN_OR_RETURN( + parent_->evaluated_[round], + ElementWiseUnaryOp(round, [](ElementwiseT elem_operand) { + return std::round(elem_operand); + })); + return Status::OK(); + } + + template < + typename NativeT, + typename std::enable_if::value>::type* = nullptr> + Status HandleRound(HloInstruction* round) { + return InvalidArgument("Unsupported type for Round"); + } + + Status HandleRound(HloInstruction* round) override { + return HandleRound(round); + } + + template < + typename NativeT, + typename std::enable_if::value>::type* = nullptr> + Status HandleCeil(HloInstruction* ceil) { + TF_ASSIGN_OR_RETURN(parent_->evaluated_[ceil], + ElementWiseUnaryOp(ceil, [](ElementwiseT elem_operand) { + return std::ceil(elem_operand); + })); + return Status::OK(); + } + + template < + typename NativeT, + typename std::enable_if::value>::type* = nullptr> + Status HandleCeil(HloInstruction* ceil) { + return InvalidArgument("Unsupported type for Ceil"); + } + + Status HandleCeil(HloInstruction* ceil) override { + return HandleCeil(ceil); + } + + Status HandleConvert(HloInstruction* convert) override { + const HloInstruction* operand = convert->operand(0); + TF_RET_CHECK(ShapeUtil::SameDimensions(operand->shape(), convert->shape())); + TF_ASSIGN_OR_RETURN(Literal result, + parent_->GetEvaluatedLiteralFor(operand).Convert( + convert->shape().element_type())); + parent_->evaluated_[convert] = std::move(result); + return Status::OK(); + } + + Status HandleBitcastConvert(HloInstruction* convert) override { + const HloInstruction* operand = convert->operand(0); + TF_RET_CHECK(ShapeUtil::SameDimensions(operand->shape(), convert->shape())); + TF_ASSIGN_OR_RETURN(Literal result, + parent_->GetEvaluatedLiteralFor(operand).BitcastConvert( + convert->shape().element_type())); + + parent_->evaluated_[convert] = std::move(result); + return Status::OK(); + } + + Status HandleExp(HloInstruction* exp) override { + TF_ASSIGN_OR_RETURN(parent_->evaluated_[exp], + ElementWiseUnaryOp(exp, [](ElementwiseT elem_operand) { + return std::exp(elem_operand); + })); + return Status::OK(); + } + + template < + typename NativeT, + typename std::enable_if::value>::type* = nullptr> + Status HandleExpm1(HloInstruction* expm1) { + TF_ASSIGN_OR_RETURN( + parent_->evaluated_[expm1], + ElementWiseUnaryOp(expm1, [](ElementwiseT elem_operand) { + return std::expm1(elem_operand); + })); + return Status::OK(); + } + + template < + typename NativeT, + typename std::enable_if::value>::type* = nullptr> + Status HandleExpm1(HloInstruction* floor) { + return InvalidArgument("Unsupported type for Expm1"); + } + + Status HandleExpm1(HloInstruction* floor) override { + return HandleExpm1(floor); + } + + template < + typename NativeT, + typename std::enable_if::value>::type* = nullptr> + Status HandleFloor(HloInstruction* floor) { + TF_ASSIGN_OR_RETURN( + parent_->evaluated_[floor], + ElementWiseUnaryOp(floor, [](ElementwiseT elem_operand) { + return std::floor(elem_operand); + })); + return Status::OK(); + } + + template < + typename NativeT, + typename std::enable_if::value>::type* = nullptr> + Status HandleFloor(HloInstruction* floor) { + return InvalidArgument("Unsupported type for Floor"); + } + + Status HandleFloor(HloInstruction* floor) override { + return HandleFloor(floor); + } + + Status HandleLog(HloInstruction* log) override { + TF_ASSIGN_OR_RETURN(parent_->evaluated_[log], + ElementWiseUnaryOp(log, [](ElementwiseT elem_operand) { + return std::log(elem_operand); + })); + return Status::OK(); + } + + template < + typename NativeT, + typename std::enable_if::value>::type* = nullptr> + Status HandleLog1p(HloInstruction* expm1) { + TF_ASSIGN_OR_RETURN( + parent_->evaluated_[expm1], + ElementWiseUnaryOp(expm1, [](ElementwiseT elem_operand) { + return std::log1p(elem_operand); + })); + return Status::OK(); + } + + template < + typename NativeT, + typename std::enable_if::value>::type* = nullptr> + Status HandleLog1p(HloInstruction* floor) { + return InvalidArgument("Unsupported type for Log1p"); + } + + Status HandleLog1p(HloInstruction* floor) override { + return HandleLog1p(floor); + } + + template ::value && + !std::is_same::value>::type* = nullptr> + Status HandleNot(HloInstruction* not_) { + TF_ASSIGN_OR_RETURN(parent_->evaluated_[not_], + ElementWiseUnaryOp(not_, [](ElementwiseT elem_operand) { + return ~elem_operand; + })); + return Status::OK(); + } + + template ::value>::type* = nullptr> + Status HandleNot(HloInstruction* not_) { + TF_ASSIGN_OR_RETURN(parent_->evaluated_[not_], + ElementWiseUnaryOp(not_, [](ElementwiseT elem_operand) { + return !elem_operand; + })); + return Status::OK(); + } + + template ::value>::type* = + nullptr> + Status HandleNot(HloInstruction* not_) { + TF_ASSIGN_OR_RETURN(parent_->evaluated_[not_], + ElementWiseUnaryOp(not_, [](ElementwiseT elem_operand) { + return !elem_operand; + })); + return Status::OK(); + } + + template < + typename NativeT, + typename std::enable_if::value>::type* = nullptr> + Status HandleNot(HloInstruction* not_) { + return InvalidArgument("Unsupported type for Not"); + } + + Status HandleNot(HloInstruction* not_) override { + return HandleNot(not_); + } + + template ::value && + !std::is_floating_point::value>::type* = nullptr> + Status HandleNegate(HloInstruction* negate) { + using type = typename std::make_unsigned::type; + TF_ASSIGN_OR_RETURN( + parent_->evaluated_[negate], + ElementWiseUnaryOp(negate, [](ElementwiseT elem_operand) { + return NativeT(-type(elem_operand)); + })); + return Status::OK(); + } + + template ::value || + std::is_floating_point::value>::type* = nullptr> + Status HandleNegate(HloInstruction* negate) { + TF_ASSIGN_OR_RETURN( + parent_->evaluated_[negate], + ElementWiseUnaryOp( + negate, [](ElementwiseT elem_operand) { return -elem_operand; })); + return Status::OK(); + } + + Status HandleNegate(HloInstruction* negate) override { + return HandleNegate(negate); + } + + template < + typename NativeT, + typename std::enable_if::value>::type* = nullptr> + Status HandleSign(HloInstruction* sign) { + TF_ASSIGN_OR_RETURN(parent_->evaluated_[sign], + ElementWiseUnaryOp(sign, [](ElementwiseT elem_operand) { + return (ElementwiseT(0) < elem_operand) - + (elem_operand < ElementwiseT(0)); + })); + return Status::OK(); + } + + template < + typename NativeT, + typename std::enable_if::value>::type* = nullptr> + Status HandleSign(HloInstruction* sign) { + TF_ASSIGN_OR_RETURN(parent_->evaluated_[sign], + ElementWiseUnaryOp(sign, [](ElementwiseT elem_operand) { + auto abs_val = std::abs(elem_operand); + return 0 == abs_val ? ElementwiseT(0) + : elem_operand / abs_val; + })); + return Status::OK(); + } + + Status HandleSign(HloInstruction* sign) override { + return HandleSign(sign); + } + + template ::value>::type* = nullptr> + Status HandleAtan2(HloInstruction* atan2) { + TF_ASSIGN_OR_RETURN(parent_->evaluated_[atan2], + ElementWiseBinaryOp(atan2, [](ElementwiseT lhs_elem, + ElementwiseT rhs_elem) { + return std::atan2(lhs_elem, rhs_elem); + })); + return Status::OK(); + } + + template ::value>::type* = nullptr> + Status HandleAtan2(HloInstruction* atan2) { + return InvalidArgument("Unsupported type for Atan2"); + } + + Status HandleAtan2(HloInstruction* atan2) override { + return HandleAtan2(atan2); + } + + Status HandleTanh(HloInstruction* tanh) override { + TF_ASSIGN_OR_RETURN(parent_->evaluated_[tanh], + ElementWiseUnaryOp(tanh, [](ElementwiseT elem_operand) { + return std::tanh(elem_operand); + })); + return Status::OK(); + } + + template ::value && + !std::is_floating_point::value>::type* = nullptr> + Status HandleMultiply(HloInstruction* multiply) { + using type = typename std::make_unsigned::type; + TF_ASSIGN_OR_RETURN( + parent_->evaluated_[multiply], + ElementWiseBinaryOp(multiply, + [](ElementwiseT lhs_elem, ElementwiseT rhs_elem) { + return NativeT(type(lhs_elem) * type(rhs_elem)); + })); + return Status::OK(); + } + + template < + typename NativeT, + typename std::enable_if::value || + std::is_floating_point::value || + is_complex_t::value>::type* = nullptr> + Status HandleMultiply(HloInstruction* multiply) { + TF_ASSIGN_OR_RETURN( + parent_->evaluated_[multiply], + ElementWiseBinaryOp(multiply, + [](ElementwiseT lhs_elem, ElementwiseT rhs_elem) { + return lhs_elem * rhs_elem; + })); + return Status::OK(); + } + + Status HandleMultiply(HloInstruction* multiply) override { + return HandleMultiply(multiply); + } + + Status HandleSubtract(HloInstruction* subtract) override { + TF_ASSIGN_OR_RETURN( + parent_->evaluated_[subtract], + ElementWiseBinaryOp(subtract, + [](ElementwiseT lhs_elem, ElementwiseT rhs_elem) { + return lhs_elem - rhs_elem; + })); + return Status::OK(); + } + + Status HandleAdd(HloInstruction* add) override { + TF_ASSIGN_OR_RETURN(parent_->evaluated_[add], + ElementWiseBinaryOp(add, [](ElementwiseT lhs_elem, + ElementwiseT rhs_elem) { + return lhs_elem + rhs_elem; + })); + return Status::OK(); + } + + template < + typename NativeT, + typename std::enable_if::value || + is_complex_t::value>::type* = nullptr> + Status HandleDivide(HloInstruction* divide) { + TF_ASSIGN_OR_RETURN(parent_->evaluated_[divide], + ElementWiseBinaryOp(divide, [](ElementwiseT lhs_elem, + ElementwiseT rhs_elem) { + return lhs_elem / rhs_elem; + })); + return Status::OK(); + } + + template ::value && + std::is_integral::value>::type* = + nullptr> + Status HandleDivide(HloInstruction* divide) { + TF_ASSIGN_OR_RETURN( + parent_->evaluated_[divide], + ElementWiseBinaryOp( + divide, + [](ElementwiseT lhs_elem, ElementwiseT rhs_elem) -> ElementwiseT { + if (rhs_elem == 0) { + return static_cast(-1); + } + if (rhs_elem == -1 && + lhs_elem == std::numeric_limits::min()) { + return lhs_elem; + } + return lhs_elem / rhs_elem; + })); + return Status::OK(); + } + + template ::value>::type* = + nullptr> + Status HandleDivide(HloInstruction* divide) { + TF_ASSIGN_OR_RETURN(parent_->evaluated_[divide], + ElementWiseBinaryOp(divide, [](ElementwiseT lhs_elem, + ElementwiseT rhs_elem) { + return rhs_elem == 0 + ? std::numeric_limits::max() + : (lhs_elem / rhs_elem); + })); + return Status::OK(); + } + + Status HandleDivide(HloInstruction* divide) { + return HandleDivide(divide); + } + + template ::value>::type* = + nullptr> + Status HandleMaximum(HloInstruction* maximum) { + TF_ASSIGN_OR_RETURN( + parent_->evaluated_[maximum], + ElementWiseBinaryOp(maximum, [](ElementwiseT lhs, ElementwiseT rhs) { + return std::max(lhs, rhs); + })); + return Status::OK(); + } + + template ::value>::type* = nullptr> + Status HandleMaximum(HloInstruction* maximum) { + TF_ASSIGN_OR_RETURN( + parent_->evaluated_[maximum], + ElementWiseBinaryOp(maximum, [](ElementwiseT lhs, ElementwiseT rhs) { + return ((lhs >= rhs) || std::isnan(lhs)) ? lhs : rhs; + })); + return Status::OK(); + } + + template < + typename NativeT, + typename std::enable_if::value>::type* = nullptr> + Status HandleMaximum(HloInstruction* maximum) { + return InvalidArgument("Unsupported type for Maximum"); + } + + Status HandleMaximum(HloInstruction* maximum) override { + return HandleMaximum(maximum); + } + + template ::value>::type* = + nullptr> + Status HandleMinimum(HloInstruction* minimum) { + TF_ASSIGN_OR_RETURN(parent_->evaluated_[minimum], + ElementWiseBinaryOp(minimum, [](ElementwiseT lhs_el, + ElementwiseT rhs_el) { + return std::min(lhs_el, rhs_el); + })); + return Status::OK(); + } + + template ::value>::type* = nullptr> + Status HandleMinimum(HloInstruction* minimum) { + TF_ASSIGN_OR_RETURN( + parent_->evaluated_[minimum], + ElementWiseBinaryOp(minimum, [](ElementwiseT lhs_el, + ElementwiseT rhs_el) { + return ((lhs_el <= rhs_el) || std::isnan(lhs_el)) ? lhs_el : rhs_el; + })); + return Status::OK(); + } + + template < + typename NativeT, + typename std::enable_if::value>::type* = nullptr> + Status HandleMinimum(HloInstruction* minimum) { + return InvalidArgument("Unsupported type for Minimum"); + } + + Status HandleMinimum(HloInstruction* minimum) override { + return HandleMinimum(minimum); + } + + Status HandlePower(HloInstruction* power) override { + TF_ASSIGN_OR_RETURN(parent_->evaluated_[power], + ElementWiseBinaryOp(power, [](ElementwiseT lhs_el, + ElementwiseT rhs_el) { + return std::pow(lhs_el, rhs_el); + })); + return Status::OK(); + } + + template ::value>::type* = nullptr> + Status HandleRemainder(HloInstruction* remainder) { + TF_ASSIGN_OR_RETURN(parent_->evaluated_[remainder], + ElementWiseBinaryOp(remainder, [](ElementwiseT lhs_el, + ElementwiseT rhs_el) { + return std::fmod(lhs_el, rhs_el); + })); + return Status::OK(); + } + + template ::value>::type* = + nullptr> + Status HandleRemainder(HloInstruction* remainder) { + TF_ASSIGN_OR_RETURN(parent_->evaluated_[remainder], + ElementWiseBinaryOp(remainder, [](ElementwiseT lhs_el, + ElementwiseT rhs_el) { + return rhs_el == 0 ? lhs_el : (lhs_el % rhs_el); + })); + return Status::OK(); + } + + template ::value && + std::is_integral::value>::type* = + nullptr> + Status HandleRemainder(HloInstruction* remainder) { + TF_ASSIGN_OR_RETURN( + parent_->evaluated_[remainder], + ElementWiseBinaryOp( + remainder, + [](ElementwiseT lhs_el, ElementwiseT rhs_el) -> ElementwiseT { + if (rhs_el == 0) { + return lhs_el; + } + if (rhs_el == -1 && + lhs_el == std::numeric_limits::min()) { + return 0; + } + return lhs_el % rhs_el; + })); + return Status::OK(); + } + + template < + typename NativeT, + typename std::enable_if::value>::type* = nullptr> + Status HandleRemainder(HloInstruction* remainder) { + return InvalidArgument("Unsupported type for Remainder"); + } + + Status HandleRemainder(HloInstruction* remainder) override { + return HandleRemainder(remainder); + } + + template ::value>::type* = + nullptr> + Status HandleAnd(HloInstruction* and_) { + TF_ASSIGN_OR_RETURN( + parent_->evaluated_[and_], + ElementWiseBinaryOp(and_, [](ElementwiseT lhs_el, ElementwiseT rhs_el) { + return lhs_el & rhs_el; + })); + return Status::OK(); + } + + template ::value>::type* = nullptr> + Status HandleAnd(HloInstruction* and_) { + return InvalidArgument("Unsupported type for And"); + } + + template < + typename NativeT, + typename std::enable_if::value>::type* = nullptr> + Status HandleAnd(HloInstruction* and_) { + return InvalidArgument("Unsupported type for And"); + } + + Status HandleAnd(HloInstruction* and_) override { + return HandleAnd(and_); + } + + template ::value>::type* = + nullptr> + Status HandleOr(HloInstruction* or_) { + TF_ASSIGN_OR_RETURN( + parent_->evaluated_[or_], + ElementWiseBinaryOp(or_, [](ElementwiseT lhs_el, ElementwiseT rhs_el) { + return lhs_el | rhs_el; + })); + return Status::OK(); + } + + template ::value>::type* = nullptr> + Status HandleOr(HloInstruction* or_) { + return InvalidArgument("Unsupported type for Or"); + } + + template < + typename NativeT, + typename std::enable_if::value>::type* = nullptr> + Status HandleOr(HloInstruction* or_) { + return InvalidArgument("Unsupported type for Or"); + } + + Status HandleOr(HloInstruction* or_) override { + return HandleOr(or_); + } + + template ::value>::type* = + nullptr> + Status HandleXor(HloInstruction* xor_) { + TF_ASSIGN_OR_RETURN( + parent_->evaluated_[xor_], + ElementWiseBinaryOp(xor_, [](ElementwiseT lhs_el, ElementwiseT rhs_el) { + return lhs_el ^ rhs_el; + })); + return Status::OK(); + } + + template ::value>::type* = nullptr> + Status HandleXor(HloInstruction* xor_) { + return InvalidArgument("Unsupported type for Xor"); + } + + template < + typename NativeT, + typename std::enable_if::value>::type* = nullptr> + Status HandleXor(HloInstruction* xor_) { + return InvalidArgument("Unsupported type for Xor"); + } + + Status HandleXor(HloInstruction* xor_) override { + return HandleXor(xor_); + } + + template ::value && + !std::is_same::value>::type* = nullptr> + Status HandleShiftLeft(HloInstruction* shl) { + TF_ASSIGN_OR_RETURN( + parent_->evaluated_[shl], + ElementWiseBinaryOp(shl, [](NativeT lhs_elem, NativeT rhs_elem) { + return IsShiftOutOfBounds(rhs_elem) ? 0 + : (lhs_elem << rhs_elem); + })); + return Status::OK(); + } + + template ::value || + std::is_same::value>::type* = + nullptr> + Status HandleShiftLeft(HloInstruction*) { + return InvalidArgument("Unsupported type for ShiftLeft"); + } + + Status HandleShiftLeft(HloInstruction* shl) override { + return HandleShiftLeft(shl); + } + template ::value && + !std::is_same::value>::type* = nullptr> + Status HandleShiftRightArithmetic(HloInstruction* shr) { + typedef typename std::make_signed::type SignedT; + TF_ASSIGN_OR_RETURN( + parent_->evaluated_[shr], + ElementWiseBinaryOp(shr, [](NativeT lhs_elem, NativeT rhs_elem) { + SignedT lhs_signed = static_cast(lhs_elem); + if (IsShiftOutOfBounds(rhs_elem)) { + return lhs_signed < 0 ? static_cast(-1) : 0; + } else { + return lhs_signed >> rhs_elem; + } + })); + return Status::OK(); + } + + template ::value || + std::is_same::value>::type* = + nullptr> + Status HandleShiftRightArithmetic(HloInstruction*) { + return InvalidArgument("Unsupported type for ShiftRightArithmetic"); + } + + Status HandleShiftRightArithmetic(HloInstruction* shra) override { + return HandleShiftRightArithmetic(shra); + } + + template ::value && + !std::is_same::value>::type* = nullptr> + Status HandleShiftRightLogical(HloInstruction* shr) { + typedef typename std::make_unsigned::type UnsignedT; + TF_ASSIGN_OR_RETURN( + parent_->evaluated_[shr], + ElementWiseBinaryOp(shr, [](NativeT lhs_elem, NativeT rhs_elem) { + // If shift amount is greater than the number of bits, then return 0. + if (IsShiftOutOfBounds(rhs_elem)) { + return static_cast(0); + } + return static_cast(static_cast(lhs_elem) >> + rhs_elem); + })); + return Status::OK(); + } + + template ::value || + std::is_same::value>::type* = + nullptr> + Status HandleShiftRightLogical(HloInstruction*) { + return InvalidArgument("Unsupported type for ShiftRightLogical"); + } + + Status HandleShiftRightLogical(HloInstruction* shrl) override { + return HandleShiftRightLogical(shrl); + } + + template < + typename NativeT, + typename std::enable_if::value>::type* = nullptr> + Status HandleClamp(HloInstruction* clamp) { + std::function + clamp_op = [](ElementwiseT low, ElementwiseT value, ElementwiseT high) { + return std::fmin(high, std::fmax(value, low)); + }; + TF_ASSIGN_OR_RETURN( + parent_->evaluated_[clamp], + ElementwiseTernaryOp(clamp, + std::move(ConvertTernaryFunction(clamp_op)))); + return Status::OK(); + } + + template < + typename NativeT, + typename std::enable_if::value>::type* = nullptr> + Status HandleClamp(HloInstruction*) { + return InvalidArgument("Unsupported type for Clamp"); + } + + Status HandleClamp(HloInstruction* clamp) override { + return HandleClamp(clamp); + } + + Status HandleSelect(HloInstruction* select) override { + CHECK(!ShapeUtil::IsScalar(select->operand(0)->shape())); + CHECK(ShapeUtil::IsArray(select->shape())); + std::function select_op = + [](bool pred, ReturnT on_true, ReturnT on_false) { + if (pred) { + return on_true; + } + return on_false; + }; + TF_ASSIGN_OR_RETURN(parent_->evaluated_[select], + ElementwiseTernaryOp(select, std::move(select_op))); + return Status::OK(); + } + + Status HandleReverse(HloInstruction* reverse) override { + const auto result_shape = reverse->shape(); + const auto reverse_dimensions = reverse->dimensions(); + + auto operand = reverse->operand(0); + TF_ASSIGN_OR_RETURN(auto inferred_return_shape, + ShapeInference::InferReverseShape(operand->shape(), + reverse_dimensions)); + + TF_RET_CHECK(ShapeUtil::Compatible(result_shape, inferred_return_shape)) + << "return shape set to: " << ShapeUtil::HumanString(result_shape) + << " but is inferred to be: " + << ShapeUtil::HumanString(inferred_return_shape); + + const Literal& operand_literal = parent_->GetEvaluatedLiteralFor(operand); + Literal result(result_shape); + + TF_RETURN_IF_ERROR( + result.Populate([&](absl::Span out_index) { + std::vector from_index(out_index.begin(), out_index.end()); + for (const int64 dim : reverse_dimensions) { + from_index[dim] = result_shape.dimensions(dim) - 1 - out_index[dim]; + } + return operand_literal.Get(from_index); + })); + + parent_->evaluated_[reverse] = std::move(result); + return Status::OK(); + } + + Status HandleConvolution(HloInstruction* conv) override { + auto lhs = conv->operand(0); + auto rhs = conv->operand(1); + const auto& window = conv->window(); + const Shape& result_shape = conv->shape(); + const Shape& lhs_shape = lhs->shape(); + const Shape& rhs_shape = rhs->shape(); + + TF_CHECK_OK(ShapeUtil::ValidateShape(lhs_shape)); + TF_CHECK_OK(ShapeUtil::ValidateShape(rhs_shape)); + CHECK(ShapeUtil::IsArray(lhs_shape)); + CHECK(ShapeUtil::IsArray(rhs_shape)); + CHECK(ShapeUtil::SameElementType(lhs_shape, rhs_shape)); + CHECK(ShapeUtil::SameElementType(lhs_shape, result_shape)); + + const auto& dnums = conv->convolution_dimension_numbers(); + const int64 num_spatial_dims = dnums.output_spatial_dimensions_size(); + CHECK_EQ(num_spatial_dims, dnums.input_spatial_dimensions_size()); + CHECK_EQ(num_spatial_dims, dnums.kernel_spatial_dimensions_size()); + CHECK_GE(num_spatial_dims, 0); + CHECK_EQ(window.dimensions_size(), num_spatial_dims); + + const auto lhs_rank = ShapeUtil::Rank(lhs_shape); + const auto rhs_rank = ShapeUtil::Rank(rhs_shape); + + CHECK_EQ(num_spatial_dims + 2, lhs_rank); + CHECK_EQ(num_spatial_dims + 2, rhs_rank); + + TF_ASSIGN_OR_RETURN( + auto inferred_return_shape, + ShapeInference::InferConvolveShape( + lhs_shape, rhs_shape, conv->feature_group_count(), window, dnums)); + CHECK(ShapeUtil::Compatible(result_shape, inferred_return_shape)) + << "return shape set to: " << ShapeUtil::HumanString(result_shape) + << " but is inferred to be: " + << ShapeUtil::HumanString(inferred_return_shape); + + const Literal& lhs_literal = parent_->GetEvaluatedLiteralFor(lhs); + const Literal& rhs_literal = parent_->GetEvaluatedLiteralFor(rhs); + + std::vector window_dimension_sizes; + for (auto i : dnums.kernel_spatial_dimensions()) { + window_dimension_sizes.push_back(ShapeUtil::GetDimension(rhs_shape, i)); + } + + const Shape& window_shape = + ShapeUtil::MakeShape(rhs_shape.element_type(), window_dimension_sizes); + + DimensionVector lhs_dim_multipliers = MakeDimMultipliers(lhs_shape); + DimensionVector rhs_dim_multipliers = MakeDimMultipliers(rhs_shape); + + auto lhs_literal_data = lhs_literal.data(); + auto rhs_literal_data = rhs_literal.data(); + + int64 feature_group_count = conv->feature_group_count(); + + auto func = [&window_shape, &dnums, &lhs_shape, &rhs_shape, &window, + &lhs_dim_multipliers, &rhs_dim_multipliers, lhs_literal_data, + rhs_literal_data, + feature_group_count](const absl::Span out_index) { + // Dimension number applicable for input (lhs). + const int64 input_batch_dim = dnums.input_batch_dimension(); + const int64 input_z_dim = dnums.input_feature_dimension(); + // Dimension number applicable for kernel (rhs). + const int64 kernel_input_z_dim = dnums.kernel_input_feature_dimension(); + const int64 kernel_output_z_dim = dnums.kernel_output_feature_dimension(); + // Dimension number applicable for output. + const int64 output_batch_dim = dnums.output_batch_dimension(); + const int64 output_z_dim = dnums.output_feature_dimension(); + + const int64 input_z_size = + ShapeUtil::GetDimension(lhs_shape, input_z_dim); + // The size of an input feature group. + const int64 input_feature_group_size = input_z_size / feature_group_count; + + const int64 output_z_size = + ShapeUtil::GetDimension(rhs_shape, kernel_output_z_dim); + // The output feature dimension is a concatenation of convolution results + // from the different groups. + const int64 output_feature_group_size = + output_z_size / feature_group_count; + + // Calculate the group index to which the current output index + // belongs. + const int64 feature_group_index = + out_index[output_z_dim] / output_feature_group_size; + + ElementwiseT result_val = static_cast(0); + DimensionVector rhs_spatial_index(dnums.kernel_spatial_dimensions_size(), + 0); + + // Convolve input feature with kernel. + do { + for (int64 rhs_iz = 0; rhs_iz < input_feature_group_size; ++rhs_iz) { + const int64 iz = + feature_group_index * input_feature_group_size + rhs_iz; + + int64 lhs_linear_index = 0; + lhs_linear_index += out_index[output_batch_dim] * + lhs_dim_multipliers[input_batch_dim]; + lhs_linear_index += iz * lhs_dim_multipliers[input_z_dim]; + + int64 rhs_linear_index = 0; + rhs_linear_index += out_index[output_z_dim] * + rhs_dim_multipliers[kernel_output_z_dim]; + rhs_linear_index += rhs_iz * rhs_dim_multipliers[kernel_input_z_dim]; + + // Find corresponding spatial dimension index for input (lhs). + for (int64 ki = 0; ki < rhs_spatial_index.size(); ++ki) { + // Spatial dimension number for input (lhs) and output. + const int64 input_spatial_dim = dnums.input_spatial_dimensions(ki); + const int64 output_spatial_dim = + dnums.output_spatial_dimensions(ki); + + // Calculate lhs (input) index without taking base dilation into + // account. + const auto& window_dim = window.dimensions(ki); + const int64 undilated_index = + out_index[output_spatial_dim] * window_dim.stride() - + window_dim.padding_low() + + rhs_spatial_index[ki] * window_dim.window_dilation(); + // Skip if the lhs (input) index is to be dilated. As an + // optimization, skip this mod if there's no dilation. + if (window_dim.base_dilation() > 1 && + undilated_index % window_dim.base_dilation() != 0) { + goto cnt; + } + + // Calculate the actual lhs (input) index after dilation. As an + // optimization, skip this integer divide if there's no dilation. + int64 lhs_spatial_index; + if (window_dim.base_dilation() > 1) { + lhs_spatial_index = undilated_index / window_dim.base_dilation(); + } else { + lhs_spatial_index = undilated_index; + } + lhs_linear_index += + lhs_spatial_index * lhs_dim_multipliers[input_spatial_dim]; + + // Skip if input index is not in bounds. + if (!(lhs_spatial_index >= 0 && + lhs_spatial_index < + lhs_shape.dimensions(input_spatial_dim))) { + goto cnt; + } + + rhs_linear_index += + (window_dim.window_reversal() + ? ((window_dim.size() - 1) - rhs_spatial_index[ki]) + : rhs_spatial_index[ki]) * + rhs_dim_multipliers[dnums.kernel_spatial_dimensions(ki)]; + } + + result_val += + static_cast(lhs_literal_data[lhs_linear_index]) * + static_cast(rhs_literal_data[rhs_linear_index]); + } + cnt : {} + } while (IndexUtil::BumpIndices(window_shape, + absl::MakeSpan(rhs_spatial_index))); + + return static_cast(result_val); + }; + + Literal result(result_shape); + TF_RETURN_IF_ERROR(result.PopulateParallel(func)); + + parent_->evaluated_[conv] = std::move(result); + return Status::OK(); + } + + Status HandleDot(HloInstruction* dot) override { + auto lhs = dot->operand(0); + auto rhs = dot->operand(1); + CHECK(ShapeUtil::IsArray(dot->shape())); + CHECK(ShapeUtil::IsArray(lhs->shape())); + CHECK(ShapeUtil::IsArray(rhs->shape())); + + const auto& dnums = dot->dot_dimension_numbers(); + + const auto lhs_rank = ShapeUtil::Rank(lhs->shape()); + const auto rhs_rank = ShapeUtil::Rank(rhs->shape()); + + CHECK(ShapeUtil::SameElementType(lhs->shape(), rhs->shape())); + CHECK(ShapeUtil::SameElementType(lhs->shape(), dot->shape())); + + // There must be 1 and only 1 Contracting dimension for lhs and rhs. + CHECK_EQ(dnums.lhs_contracting_dimensions_size(), 1); + CHECK_EQ(dnums.rhs_contracting_dimensions_size(), 1); + const int64 lhs_contracting_dimension = dnums.lhs_contracting_dimensions(0); + const int64 rhs_contracting_dimension = dnums.rhs_contracting_dimensions(0); + // Contracted dimension sizes must be the same. + CHECK_EQ(lhs->shape().dimensions(lhs_contracting_dimension), + rhs->shape().dimensions(rhs_contracting_dimension)) + << "lhs contracted dimension: " + << lhs->shape().dimensions(lhs_contracting_dimension) + << " rhs contracted dimension: " + << rhs->shape().dimensions(rhs_contracting_dimension); + const int64 contracted_dimension_size = + lhs->shape().dimensions(lhs_contracting_dimension); + + const Literal& lhs_literal = parent_->GetEvaluatedLiteralFor(lhs); + const Literal& rhs_literal = parent_->GetEvaluatedLiteralFor(rhs); + + CHECK_EQ(dnums.lhs_batch_dimensions_size(), + dnums.rhs_batch_dimensions_size()); + + DimensionVector lhs_index(lhs_rank); + DimensionVector rhs_index(rhs_rank); + + // result_index_locations[i] contains one or two pointers to the locations + // in lhs_index or rhs_index where the i'th result index should go. + absl::InlinedVector, kInlineRank> + result_index_locations; + result_index_locations.reserve(lhs_rank + rhs_rank - 2); + + // The first components in the output shape are the LHS and RHS batch + // dimensions: + for (int64 i = 0; i < dnums.lhs_batch_dimensions_size(); i++) { + result_index_locations.push_back( + {&lhs_index[dnums.lhs_batch_dimensions(i)], + &rhs_index[dnums.rhs_batch_dimensions(i)]}); + } + + // Then we have the LHS and RHS non-contracting dimensions, if any: + for (int64 i = 0; i < lhs_rank; i++) { + if (i != lhs_contracting_dimension && + !absl::c_linear_search(dnums.lhs_batch_dimensions(), i)) { + result_index_locations.push_back({&lhs_index[i], nullptr}); + } + } + for (int64 i = 0; i < rhs_rank; i++) { + if (i != rhs_contracting_dimension && + !absl::c_linear_search(dnums.rhs_batch_dimensions(), i)) { + result_index_locations.push_back({&rhs_index[i], nullptr}); + } + } + + Literal result(dot->shape()); + TF_RETURN_IF_ERROR( + result.Populate([&](absl::Span result_index) { + ElementwiseT result_val = static_cast(0); + + for (int64 i = 0; i < result_index.size(); i++) { + *result_index_locations[i].first = result_index[i]; + if (result_index_locations[i].second) { + *result_index_locations[i].second = result_index[i]; + } + } + + // Accumulates resulting product along the contracted dimension. + for (int64 i = 0; i < contracted_dimension_size; ++i) { + lhs_index[lhs_contracting_dimension] = i; + rhs_index[rhs_contracting_dimension] = i; + + result_val += + static_cast(lhs_literal.Get(lhs_index)) * + static_cast(rhs_literal.Get(rhs_index)); + } + + return static_cast(result_val); + })); + + parent_->evaluated_[dot] = std::move(result); + return Status::OK(); + } + + Status HandlePad(HloInstruction* pad) override { + CHECK(ShapeUtil::IsArray(pad->operand(0)->shape())); + // Padding value must be scalar. + CHECK(ShapeUtil::IsScalar(pad->operand(1)->shape())); + CHECK_EQ(ShapeUtil::Rank(pad->operand(0)->shape()), + pad->padding_config().dimensions_size()); + + TF_ASSIGN_OR_RETURN(auto inferred_return_shape, + ShapeInference::InferPadShape( + /*operand_shape=*/pad->operand(0)->shape(), + /*padding_value_shape=*/pad->operand(1)->shape(), + /*padding_config=*/pad->padding_config())); + CHECK(ShapeUtil::Compatible(pad->shape(), inferred_return_shape)) + << "return shape is set to: " << ShapeUtil::HumanString(pad->shape()) + << " but is inferred to be: " + << ShapeUtil::HumanString(inferred_return_shape); + + // Create new HLO of padded shape with padding value. + ReturnT scalar = + parent_->GetEvaluatedLiteralFor(pad->operand(1)).Get({}); + Literal result(pad->shape()); + TF_RETURN_IF_ERROR(result.Populate( + [&scalar](absl::Span multi_index) { return scalar; })); + + const Literal& evaluated_operand = + parent_->GetEvaluatedLiteralFor(pad->operand(0)); + + std::vector input_index(ShapeUtil::Rank(evaluated_operand.shape()), + 0); + std::vector target_index(ShapeUtil::Rank(result.shape()), 0); + + // Loop through each element of the operand, assign them to the + // corresponding index of the resulting padded literal. + const PaddingConfig& pad_config = pad->padding_config(); + + auto func = [&](absl::Span input_index) { + for (auto i = 0; i < input_index.size(); ++i) { + // Interior padding occurs logically before edge padding, so in the case + // of negative edge padding elements are removed from the + // interior-padded operand. + target_index[i] = + pad_config.dimensions(i).edge_padding_low() + + input_index[i] * (pad_config.dimensions(i).interior_padding() + 1); + + // Account for negative low and high padding: skip assignment if the + // any target index is out of range. + if (!(target_index[i] >= 0 && + target_index[i] < pad->shape().dimensions(i))) { + return true; + } + } + result.Set(target_index, + evaluated_operand.Get(input_index)); + return true; + }; + + std::vector zero_base(evaluated_operand.shape().dimensions_size(), + 0); + std::vector step(evaluated_operand.shape().dimensions_size(), 1); + + ShapeUtil::ForEachIndex( + evaluated_operand.shape(), zero_base, + AsInt64Slice(evaluated_operand.shape().dimensions()), step, func); + + parent_->evaluated_[pad] = std::move(result); + return Status::OK(); + } + + Status HandleDynamicSlice(HloInstruction* dynamic_slice) override { + auto operand = dynamic_slice->operand(0); + auto start_indices = dynamic_slice->operand(1); + auto result_shape = dynamic_slice->shape(); + TF_ASSIGN_OR_RETURN(auto inferred_return_shape, + ShapeInference::InferDynamicSliceShape( + operand->shape(), start_indices->shape(), + dynamic_slice->dynamic_slice_sizes())); + TF_RET_CHECK(ShapeUtil::Compatible(result_shape, inferred_return_shape)) + << "return shape is set to: " << ShapeUtil::HumanString(result_shape) + << " but is inferred to be: " + << ShapeUtil::HumanString(inferred_return_shape); + TF_RET_CHECK( + primitive_util::IsIntegralType(start_indices->shape().element_type())); + + const Literal& operand_literal = parent_->GetEvaluatedLiteralFor(operand); + const Literal& start_indices_literal = + parent_->GetEvaluatedLiteralFor(start_indices); + + switch (start_indices->shape().element_type()) { + case S32: { + TF_ASSIGN_OR_RETURN( + parent_->evaluated_[dynamic_slice], + DynamicSlice(operand_literal, start_indices_literal, + result_shape)); + } break; + case S64: { + TF_ASSIGN_OR_RETURN( + parent_->evaluated_[dynamic_slice], + DynamicSlice(operand_literal, start_indices_literal, + result_shape)); + } break; + case U32: { + TF_ASSIGN_OR_RETURN( + parent_->evaluated_[dynamic_slice], + DynamicSlice(operand_literal, start_indices_literal, + result_shape)); + } break; + case U64: { + TF_ASSIGN_OR_RETURN( + parent_->evaluated_[dynamic_slice], + DynamicSlice(operand_literal, start_indices_literal, + result_shape)); + } break; + default: + LOG(FATAL) << "HandleDynamicSlice: unhandled primitive type for " + "start_indices: " + << PrimitiveType_Name(start_indices->shape().element_type()); + } + + return Status::OK(); + } + + Status HandleDynamicUpdateSlice( + HloInstruction* dynamic_update_slice) override { + auto operand = dynamic_update_slice->operand(0); + auto update = dynamic_update_slice->operand(1); + auto start_indices = dynamic_update_slice->operand(2); + auto result_shape = dynamic_update_slice->shape(); + TF_ASSIGN_OR_RETURN( + auto inferred_return_shape, + ShapeInference::InferDynamicUpdateSliceShape( + operand->shape(), update->shape(), start_indices->shape())); + TF_RET_CHECK(ShapeUtil::Compatible(result_shape, inferred_return_shape)) + << "return shape is set to: " << ShapeUtil::HumanString(result_shape) + << " but is inferred to be: " + << ShapeUtil::HumanString(inferred_return_shape); + TF_RET_CHECK( + primitive_util::IsIntegralType(start_indices->shape().element_type())); + TF_RET_CHECK(ShapeUtil::Compatible(result_shape, operand->shape())); + + const Literal& operand_literal = parent_->GetEvaluatedLiteralFor(operand); + const Literal& update_literal = parent_->GetEvaluatedLiteralFor(update); + const Literal& start_indices_literal = + parent_->GetEvaluatedLiteralFor(start_indices); + + switch (start_indices->shape().element_type()) { + case S32: { + TF_ASSIGN_OR_RETURN( + parent_->evaluated_[dynamic_update_slice], + DynamicUpdateSlice(operand_literal, update_literal, + start_indices_literal)); + } break; + case S64: { + TF_ASSIGN_OR_RETURN( + parent_->evaluated_[dynamic_update_slice], + DynamicUpdateSlice(operand_literal, update_literal, + start_indices_literal)); + } break; + case U32: { + TF_ASSIGN_OR_RETURN( + parent_->evaluated_[dynamic_update_slice], + DynamicUpdateSlice(operand_literal, update_literal, + start_indices_literal)); + } break; + case U64: { + TF_ASSIGN_OR_RETURN( + parent_->evaluated_[dynamic_update_slice], + DynamicUpdateSlice(operand_literal, update_literal, + start_indices_literal)); + } break; + default: + LOG(FATAL) << "HandleDynamicUpdateSlice: unhandled primitive type for " + "start_indices: " + << PrimitiveType_Name(start_indices->shape().element_type()); + } + + return Status::OK(); + } + + template + StatusOr MapImpl(HloInstruction* map) { + auto operands = map->operands(); + HloComputation* computation = map->to_apply(); + + Literal result(map->shape()); + + HloEvaluator embedded_evaluator(parent_->max_loop_iterations_); + TF_RETURN_IF_ERROR( + result.Populate([&](absl::Span multi_index) { + std::vector arg_literals; + arg_literals.reserve(operands.size()); + + // Construct scalar literal parameters to be passed to the map + // computation. + for (auto operand : operands) { + const Literal& arg_literal = + parent_->GetEvaluatedLiteralFor(operand); + + auto curr_val = arg_literal.Get(multi_index); + auto curr_val_literal = LiteralUtil::CreateR0(curr_val); + + arg_literals.push_back(std::move(curr_val_literal)); + } + + Literal computed_result = + embedded_evaluator.Evaluate(*computation, arg_literals) + .ConsumeValueOrDie(); + // Clear visit states so that the we can use the evaluate again on + // the same computation. + embedded_evaluator.ResetVisitStates(); + + return computed_result.Get({}); + })); + return std::move(result); + } + + Status HandleMap(HloInstruction* map) override { + switch (map->operand(0)->shape().element_type()) { + case PRED: { + TF_ASSIGN_OR_RETURN(parent_->evaluated_[map], MapImpl(map)); + break; + } + case U8: { + TF_ASSIGN_OR_RETURN(parent_->evaluated_[map], MapImpl(map)); + break; + } + case U32: { + TF_ASSIGN_OR_RETURN(parent_->evaluated_[map], MapImpl(map)); + break; + } + case U64: { + TF_ASSIGN_OR_RETURN(parent_->evaluated_[map], MapImpl(map)); + break; + } + case S8: { + TF_ASSIGN_OR_RETURN(parent_->evaluated_[map], MapImpl(map)); + break; + } + case S32: { + TF_ASSIGN_OR_RETURN(parent_->evaluated_[map], MapImpl(map)); + break; + } + case S64: { + TF_ASSIGN_OR_RETURN(parent_->evaluated_[map], MapImpl(map)); + break; + } + case F16: { + TF_ASSIGN_OR_RETURN(parent_->evaluated_[map], + MapImpl(map)); + break; + } + case F32: { + TF_ASSIGN_OR_RETURN(parent_->evaluated_[map], MapImpl(map)); + break; + } + case F64: { + TF_ASSIGN_OR_RETURN(parent_->evaluated_[map], MapImpl(map)); + break; + } + case C64: { + TF_ASSIGN_OR_RETURN(parent_->evaluated_[map], MapImpl(map)); + break; + } + default: + LOG(FATAL) << "HandleMap: unhandled primitive type for " + "input operand: " + << PrimitiveType_Name( + map->operand(0)->shape().element_type()); + } + + return Status::OK(); + } + + template ::value && + !std::is_same::value>::type* = nullptr> + Status HandleSort(HloInstruction* sort) { + auto keys = sort->operand(0); + TF_RET_CHECK(sort->operand_count() == 1) + << "Typed visitor does not support key-value sort"; + + const Literal& keys_literal = parent_->GetEvaluatedLiteralFor(keys); + int64 sort_dim = sort->dimensions(0); + int64 sort_dim_elements = keys->shape().dimensions(sort_dim); + int64 rank = ShapeUtil::Rank(keys->shape()); + if (rank == 0) { + // Nothing to sort. + parent_->evaluated_[sort] = keys_literal.Clone(); + return Status::OK(); + } + Literal result_literal(keys_literal.shape()); + std::vector zero_base(rank, 0); + std::vector increment(rank, 1); + increment[sort_dim] = sort_dim_elements; + // Iterate through each dimension except 'sort_dim'. + TF_RETURN_IF_ERROR(ShapeUtil::ForEachIndexWithStatus( + keys->shape(), zero_base, AsInt64Slice(keys->shape().dimensions()), + increment, [&](absl::Span indices) -> StatusOr { + // Extract a slice from the literal that corresponds to exactly the + // row in dimension 'sort_dim'. + std::vector limit_indices(indices.begin(), indices.end()); + std::for_each(limit_indices.begin(), limit_indices.end(), + [](int64& index) { ++index; }); + limit_indices[sort_dim] = sort_dim_elements; + TF_ASSIGN_OR_RETURN(auto row_to_sort, + keys_literal.Slice(indices, limit_indices) + .Reshape({sort_dim_elements})); + const auto& row_data = row_to_sort.data(); + + std::vector result_data(row_data.begin(), row_data.end()); + std::sort(result_data.begin(), result_data.end(), + [](const NativeT& a, const NativeT& b) { + return SafeLess(a, b); + }); + Literal sorted_row(ShapeUtil::MakeShape(keys->shape().element_type(), + {sort_dim_elements})); + sorted_row.PopulateR1(absl::Span(result_data)); + std::vector slice_dimensions(rank, 1); + slice_dimensions[sort_dim] = sort_dim_elements; + TF_ASSIGN_OR_RETURN(auto sorted_row_reshaped, + sorted_row.Reshape(slice_dimensions)); + std::vector start_indices(rank, 0); + TF_RETURN_IF_ERROR(result_literal.CopySliceFrom( + sorted_row_reshaped, start_indices, indices, slice_dimensions)); + return true; + })); + parent_->evaluated_[sort] = std::move(result_literal); + return Status::OK(); + } + + template ::value || + std::is_same::value>::type* = + nullptr> + Status HandleSort(HloInstruction* sort) { + return InvalidArgument("Unsupported type for Sort"); + } + + Status HandleSort(HloInstruction* sort) override { + return HandleSort(sort); + } + + Status HandleReduce(HloInstruction* hlo) override { + HloReduceInstruction* reduce = Cast(hlo); + int64 num_args = reduce->inputs().size(); + bool has_tuple_output = ShapeUtil::IsTuple(reduce->shape()); + absl::Span dimensions(reduce->dimensions()); + HloComputation* function = reduce->to_apply(); + + absl::InlinedVector operand_shapes; + for (const HloInstruction* operand : reduce->operands()) { + operand_shapes.push_back(&operand->shape()); + } + TF_ASSIGN_OR_RETURN(auto inferred_return_shape, + ShapeInference::InferReduceShape( + operand_shapes, + /*dimensions_to_reduce=*/dimensions, + /*to_apply=*/function->ComputeProgramShape())); + TF_RET_CHECK(ShapeUtil::Compatible(reduce->shape(), inferred_return_shape)) + << "return shape is set to: " << ShapeUtil::HumanString(reduce->shape()) + << " but is inferred to be: " + << ShapeUtil::HumanString(inferred_return_shape); + + absl::InlinedVector arg_literals(num_args); + absl::InlinedVector init_literals(num_args); + for (int64 i = 0; i < num_args; ++i) { + arg_literals[i] = &parent_->GetEvaluatedLiteralFor(reduce->inputs()[i]); + VLOG(3) << "HandleReduce arg_literal: " << arg_literals[i]->ToString(); + init_literals[i] = + &parent_->GetEvaluatedLiteralFor(reduce->init_values()[i]); + VLOG(3) << "HandleReduce init_literal: " << init_literals[i]->ToString(); + TF_RET_CHECK(ShapeUtil::IsScalar(init_literals[i]->shape())); + } + + // All args and results have the same dimensions, so pick an arbitrary one. + const Shape& arg_shape = arg_literals[0]->shape(); + const Shape& result_shape = ShapeUtil::IsTuple(reduce->shape()) + ? reduce->shape().tuple_shapes(0) + : reduce->shape(); + const auto arg_dimensions = AsInt64Slice(arg_shape.dimensions()); + std::vector arg_dim_steps(arg_dimensions.size()); + std::vector arg_dim_counts(arg_dimensions.size()); + for (const int64 dim : dimensions) { + arg_dim_steps[dim] = 1; + arg_dim_counts[dim] = arg_dimensions[dim]; + } + + // Map each dimension in the result to a dimension in arg that isn't + // being reduced. + std::vector result_to_arg_index; + for (int64 i = 0; i < arg_dimensions.size(); ++i) { + if (arg_dim_steps[i] == 0) { + result_to_arg_index.push_back(i); + } + } + + HloEvaluator embedded_evaluator(parent_->max_loop_iterations_); + absl::InlinedVector results(num_args); + for (int64 i = 0; i < num_args; ++i) { + results[i] = Literal(result_shape); + } + + Status eval_status; + // For each resulting dimension, calculate and assign computed values. + // This is really wasteful when num_args > 1, since we re-run the + // reduction num_args time. The alternative is to teach Populate() about + // tuples, which we should probably do. + absl::InlinedVector init_scalars(num_args); + for (int i = 0; i < num_args; ++i) { + init_scalars[i] = init_literals[i]->Get({}); + } + + for (int64 input = 0; input < num_args; ++input) { + TF_RETURN_IF_ERROR(results[input].Populate( + [&](absl::Span multi_index) { + if (!eval_status.ok()) { + return init_scalars[input]; + } + absl::InlinedVector result_values(init_scalars.begin(), + init_scalars.end()); + std::vector base(arg_dimensions.size()); + for (int64 i = 0; i < multi_index.size(); ++i) { + base[result_to_arg_index[i]] = multi_index[i]; + } + + // When the reduction is addition of floats, accumulate in a double + // for better precision. Also, avoid creating Literals for the + // intermediate results; it's much faster. + if (ShapeUtil::ElementIsFloating(init_literals[0]->shape()) && + IsScalarAdd(function)) { + CHECK_EQ(num_args, 1); + double computed_result = 0; + auto func = [&](absl::Span input_index) { + computed_result += + GetAsDouble(*arg_literals[0], input_index); + return true; + }; + ShapeUtil::ForEachIndex(arg_literals[0]->shape(), base, + arg_dim_counts, arg_dim_steps, func); + return static_cast(computed_result); + } + auto func = + [&](absl::Span input_index) -> StatusOr { + absl::InlinedVector arg_values(num_args); + for (int64 i = 0; i < num_args; ++i) { + arg_values[i] = arg_literals[i]->Get(input_index); + } + + // Evaluate computation with specified literal operands. + absl::InlinedVector embedded_operands; + for (ReturnT value : result_values) { + embedded_operands.push_back( + LiteralUtil::CreateR0(value)); + } + for (ReturnT value : arg_values) { + embedded_operands.push_back( + LiteralUtil::CreateR0(value)); + } + absl::InlinedVector embedded_operands_ptrs( + embedded_operands.size()); + std::transform(embedded_operands.begin(), embedded_operands.end(), + embedded_operands_ptrs.begin(), + [](Literal& literal) { return &literal; }); + + TF_ASSIGN_OR_RETURN(Literal computed_result, + embedded_evaluator.Evaluate( + *function, embedded_operands_ptrs)); + // Clear visit states so that we can use the evaluator again on + // the same computation. + embedded_evaluator.ResetVisitStates(); + // Assign computed result to result_val. + if (!has_tuple_output) { + result_values[0] = computed_result.Get({}); + } else { + for (int64 i = 0; i < num_args; ++i) { + result_values[i] = computed_result.Get( + /*multi_index=*/{}, /*shape_index=*/{i}); + } + } + return true; + }; + // Computes one element of the result, reducing all dimensions that + // contribute to that element. + eval_status = ShapeUtil::ForEachIndexWithStatus( + arg_shape, base, arg_dim_counts, arg_dim_steps, func); + return result_values[input]; + })); + } + if (!has_tuple_output) { + parent_->evaluated_[reduce] = std::move(results[0]); + } else { + Literal tuple_result(reduce->shape()); + for (int64 i = 0; i < num_args; ++i) { + TF_CHECK_OK(tuple_result.MoveFrom(std::move(results[i]), {i})); + } + parent_->evaluated_[reduce] = std::move(tuple_result); + } + return eval_status; + } + + bool IsScalarAdd(HloComputation* computation) { + HloInstruction* instruction = computation->root_instruction(); + if (instruction->opcode() == HloOpcode::kAdd && + computation->num_parameters() == 2) { + const HloInstruction* lhs = instruction->operand(0); + const HloInstruction* rhs = instruction->operand(1); + return lhs->opcode() == HloOpcode::kParameter && + ShapeUtil::IsScalar(lhs->shape()) && + rhs->opcode() == HloOpcode::kParameter && + ShapeUtil::IsScalar(rhs->shape()) && lhs != rhs; + } + return false; + } + + Status HandleSelectAndScatter(HloInstruction* select_and_scatter) override { + auto operand = select_and_scatter->operand(0); + auto source = select_and_scatter->operand(1); + const Window& window = select_and_scatter->window(); + + const Literal& init_literal = + parent_->GetEvaluatedLiteralFor(select_and_scatter->operand(2)); + TF_RET_CHECK(ShapeUtil::IsScalar(init_literal.shape())); + auto init_scalar = init_literal.Get({}); + + Literal result(select_and_scatter->shape()); + + // Initialize result array with the init value. + TF_RETURN_IF_ERROR(result.Populate( + [&](absl::Span output_index) { return init_scalar; })); + + std::vector window_dimension_sizes; + for (const auto& window_dimension : window.dimensions()) { + window_dimension_sizes.push_back(window_dimension.size()); + } + const Shape window_shape = ShapeUtil::MakeShape( + operand->shape().element_type(), window_dimension_sizes); + + HloComputation* select = select_and_scatter->select(); + HloComputation* scatter = select_and_scatter->scatter(); + + const Literal& operand_literal = parent_->GetEvaluatedLiteralFor(operand); + const Literal& source_literal = parent_->GetEvaluatedLiteralFor(source); + + int64 rank = ShapeUtil::Rank(operand_literal.shape()); + + HloEvaluator embedded_evaluator(parent_->max_loop_iterations_); + DimensionVector source_index(rank, 0); + + // Used in the dual IterateThroughWindow lambdas below. Hoisted to avoid + // dynamic memory allocations. + auto curr_val_literal = LiteralUtil::CreateR0(ReturnT()); + auto selected_val_literal = LiteralUtil::CreateR0(ReturnT()); + auto source_literal_scatter = LiteralUtil::CreateR0(ReturnT()); + auto scattered_literal = LiteralUtil::CreateR0(ReturnT()); + do { + // For each element in `source`, we place a window in `operand`. For each + // window placement, we iterate inside the window twice: + // + // 1. Find the selected index by applying `select` function to all + // elements. E.g., If the `select` function is GreaterEqual, the first + // iteration through the window finds the biggest value and returns its + // index. + // + // 2. Using the selected index, scatter value from `source` to result. We + // do this by iterating through the window, and compare each index with + // the selected index. + absl::optional selected_val; + absl::optional> selected_index; + + IterateThroughWindow( + window_shape, window, operand_literal.shape(), source_index, + [&](const std::vector& operand_index) { + auto curr_val = operand_literal.Get(operand_index); + if (!selected_val) { + selected_val = curr_val; + selected_index = operand_index; + } + curr_val_literal.Set({}, curr_val); + selected_val_literal.Set({}, *selected_val); + Literal computed_result = + embedded_evaluator + .Evaluate( + *select, {&selected_val_literal, &curr_val_literal}) + .ConsumeValueOrDie(); + bool selected = !computed_result.Get({}); + if (selected) { + selected_val = curr_val; + selected_index = operand_index; + } + embedded_evaluator.ResetVisitStates(); + }); + + IterateThroughWindow( + window_shape, window, operand_literal.shape(), source_index, + [&](const std::vector& operand_index) { + if (std::equal(operand_index.begin(), operand_index.end(), + selected_index->begin())) { + auto source = source_literal.Get(source_index); + auto scattered = result.Get(operand_index); + source_literal_scatter.Set({}, source); + scattered_literal.Set({}, scattered); + Literal computed_result = + embedded_evaluator + .Evaluate( + *scatter, + {&source_literal_scatter, &scattered_literal}) + .ConsumeValueOrDie(); + result.Set(operand_index, computed_result.Get({})); + // Clear visit states so that the we can use the evaluator again + // on the same computation. + embedded_evaluator.ResetVisitStates(); + } + }); + } while ( + IndexUtil::BumpIndices(source->shape(), absl::MakeSpan(source_index))); + + parent_->evaluated_[select_and_scatter] = std::move(result); + return Status::OK(); + } + + Status HandleReduceWindow(HloInstruction* reduce_window) override { + auto operand = reduce_window->operand(0); + const Window& window = reduce_window->window(); + HloComputation* function = reduce_window->to_apply(); + TF_ASSIGN_OR_RETURN( + auto inferred_return_shape, + ShapeInference::InferReduceWindowShape( + /*operand_shape=*/reduce_window->operand(0)->shape(), + /*init_value=*/reduce_window->operand(1)->shape(), window, + /*to_apply_shape=*/function->ComputeProgramShape())); + TF_RET_CHECK( + ShapeUtil::Compatible(reduce_window->shape(), inferred_return_shape)) + << "return shape is set to: " + << ShapeUtil::HumanStringWithLayout(reduce_window->shape()) + << " but is inferred to be: " + << ShapeUtil::HumanStringWithLayout(inferred_return_shape); + + const Literal& operand_literal = + parent_->GetEvaluatedLiteralFor(reduce_window->operand(0)); + VLOG(3) << "HandleReduceWindow arg_literal: " << operand_literal.ToString(); + const Literal& init_literal = + parent_->GetEvaluatedLiteralFor(reduce_window->operand(1)); + VLOG(3) << "HandleReduceWindow init_literal: " << init_literal.ToString(); + TF_RET_CHECK(ShapeUtil::IsScalar(init_literal.shape())); + auto init_scalar = init_literal.Get({}); + + // Creates a Shape object from window, for iteration below. + std::vector window_dimension_sizes; + for (const auto& window_dimension : window.dimensions()) { + window_dimension_sizes.push_back(window_dimension.size()); + } + const Shape window_shape = ShapeUtil::MakeShape( + operand->shape().element_type(), window_dimension_sizes); + + DimensionVector window_index(window.dimensions_size()); + DimensionVector operand_index(ShapeUtil::Rank(operand_literal.shape())); + + HloEvaluator embedded_evaluator(parent_->max_loop_iterations_); + Literal result(reduce_window->shape()); + // For each resulting dimension, calculate and assign computed value. + TF_RETURN_IF_ERROR( + result.Populate([&](absl::Span output_index) { + ReturnT result_val = init_scalar; + + std::fill(window_index.begin(), window_index.end(), 0); + std::fill(operand_index.begin(), operand_index.end(), 0); + + IterateThroughWindow( + window_shape, window, operand_literal.shape(), output_index, + [&](const std::vector& operand_index) { + auto curr_val = operand_literal.Get(operand_index); + + // Evaluate computation with specified literal operands. + const auto curr_val_literal = + LiteralUtil::CreateR0(curr_val); + const auto result_val_literal = + LiteralUtil::CreateR0(result_val); + Literal computed_result = + embedded_evaluator + .Evaluate( + *function, {&result_val_literal, &curr_val_literal}) + .ConsumeValueOrDie(); + + // Clear visit states so that the we can use the evaluate again + // on the same computation. + embedded_evaluator.ResetVisitStates(); + + result_val = computed_result.Get({}); + }); + + return result_val; + })); + + parent_->evaluated_[reduce_window] = std::move(result); + return Status::OK(); + } + + // Reshapes the scatter indices input to have a trailing degenerate `1` + // dimension if necessary. Hands over the ownership of the newly created + // literal (if there is one) to `reshaped_indices`. + StatusOr> ReshapedScatterIndices( + int64 index_vector_dim, const Literal& indices, + Literal* reshaped_indices) { + if (indices.shape().dimensions_size() != index_vector_dim) { + return std::cref(indices); + } + + std::vector new_shape(indices.shape().dimensions().begin(), + indices.shape().dimensions().end()); + new_shape.push_back(1); + TF_ASSIGN_OR_RETURN(*reshaped_indices, indices.Reshape(new_shape)); + return std::cref(*reshaped_indices); + } + + // Returns an ShapeUtil::IndexIterationSpace that iterates over the update + // scatter dimensions while keeping the rest of the update dimensions clamped + // to 0. + ShapeUtil::IndexIterationSpace IterationSpaceForUpdateScatterIndices( + const Shape& updates_shape, const ScatterDimensionNumbers& dim_numbers) { + int64 updates_rank = updates_shape.dimensions_size(); + std::vector index_base(updates_rank, 0); + std::vector index_count(updates_rank, 1); + for (int64 i = 0; i < updates_rank; i++) { + bool is_update_scatter_dim = + !absl::c_binary_search(dim_numbers.update_window_dims(), i); + if (is_update_scatter_dim) { + index_count[i] = updates_shape.dimensions(i); + } + } + return {std::move(index_base), std::move(index_count), + std::vector(updates_rank, 1)}; + } + + // Return an ShapeUtil::IndexIterationSpace that iterates over the update + // window dimensions while keeping the rest of the update dimensions clamped + // to 0. + ShapeUtil::IndexIterationSpace IterationSpaceForUpdateWindowIndices( + const Shape& updates_shape, const ScatterDimensionNumbers& dim_numbers) { + int64 updates_rank = updates_shape.dimensions_size(); + std::vector index_base(updates_rank, 0); + std::vector index_count(updates_rank, 1); + for (int64 i = 0; i < updates_rank; i++) { + bool is_update_window_dim = + absl::c_binary_search(dim_numbers.update_window_dims(), i); + if (is_update_window_dim) { + index_count[i] = updates_shape.dimensions(i); + } + } + return {std::move(index_base), std::move(index_count), + std::vector(updates_rank, 1)}; + } + + // This functor computes the contribution of scatter_indices to an input index + // corresponding to an update index. That is, given an update index I, it + // picks out the scatter indices in I and uses them to look up a scatter + // index, S, from the scatter indices tensor, and expands S into the input + // space according to scatter_dims_to_operand_dims. + // + // This is similar to the class HloEvaluator::OutputGatherIndexToInputIndex + // that does the corresponding function for Gather. + class UpdateScatterIndexToInputIndex { + public: + // The constructor does some setup work that is amortized across all + // iterations. + explicit UpdateScatterIndexToInputIndex( + const ScatterDimensionNumbers* dim_numbers, const Shape& input_shape, + const Shape& updates_shape, const Literal* scatter_indices) + : dim_numbers_(*dim_numbers), scatter_indices_(*scatter_indices) { + for (int64 i = 0; i < updates_shape.dimensions_size(); i++) { + update_dim_is_scatter_dims_.push_back( + !absl::c_binary_search(dim_numbers_.update_window_dims(), i)); + } + + for (int64 i = 0; i < input_shape.dimensions_size(); i++) { + int64 index_of_input_dim_in_index_vector = + FindIndex(dim_numbers_.scatter_dims_to_operand_dims(), i); + if (index_of_input_dim_in_index_vector == + dim_numbers_.scatter_dims_to_operand_dims_size()) { + input_dim_value_to_index_vector_.push_back(-1); + } else { + input_dim_value_to_index_vector_.push_back( + index_of_input_dim_in_index_vector); + } + } + + index_vector_index_.resize(scatter_indices_.shape().dimensions_size()); + input_index_.resize(input_shape.dimensions_size()); + int64 index_vector_size = + scatter_indices_.shape().dimensions(dim_numbers_.index_vector_dim()); + index_vector_.resize(index_vector_size); + } + + // Returns the contribution of scatter_indices to the input index + // corresponding to update_index. See scatter_inner_loop_body. + // + // This is conceptually a stateless transformation from update_index to the + // scatter input index, but: + // + // - Instead of allocating memory to represent the scatter input index on + // every invocation we reuse the same storage for the result + // (input_index_), mutating it in place. + // - Instead of allocating buffers for temporary values like + // index_vector_index_ and index_vector on every invocation, we reuse the + // same storage for all invocations. + // + // This returns a Span into memory owned by the class. + StatusOr> operator()( + absl::Span update_index) { + PropagateUpdateIndexScatterDimsToIndexVectorIndex(update_index); + TF_RETURN_IF_ERROR(FetchIndexVector()); + PropagateIndexVectorToInputIndex(); + return absl::Span(input_index_); + } + + private: + // Propagates the scatter index dimensions from the update index into + // index_vector_index_ by mutating index_vector_index_ in place. Does not + // update the dim_numbers.index_vector_dim() dimension -- that's the + // dimension we iterate over in FetchIndexVector. + void PropagateUpdateIndexScatterDimsToIndexVectorIndex( + absl::Span update_index) { + int64 index_vector_index_i = 0; + for (int64 i = 0, e = update_index.size(); i < e; i++) { + if (!update_dim_is_scatter_dims_[i]) { + continue; + } + + if (index_vector_index_i == dim_numbers_.index_vector_dim()) { + index_vector_index_i++; + } + + index_vector_index_[index_vector_index_i++] = update_index[i]; + } + } + + // Populates index_vector_ by iterating over scatter_indices_ according to + // index_vector_index_. + Status FetchIndexVector() { + int64 index_vector_dim = dim_numbers_.index_vector_dim(); + for (int64 i = 0, e = index_vector_.size(); i < e; i++) { + index_vector_index_[index_vector_dim] = i; + TF_ASSIGN_OR_RETURN(index_vector_[i], scatter_indices_.GetIntegralAsS64( + index_vector_index_)); + } + return Status::OK(); + } + + // Populates input_index_. + void PropagateIndexVectorToInputIndex() { + for (int64 i = 0, e = input_index_.size(); i < e; i++) { + if (input_dim_value_to_index_vector_[i] != -1) { + input_index_[i] = index_vector_[input_dim_value_to_index_vector_[i]]; + } + + // If input_dim_value_to_index_vector_[i] == -1 then input_index_[i] + // remains 0, as set by the constructor. + } + } + + // input_dim_value_to_index_vector_[i] tells us how to compute dimension i + // of the input index from the index vector. See + // PropagateIndexVectorToInputIndex. + std::vector input_dim_value_to_index_vector_; + + // update_dim_is_scatter_dims_[i] is true iff the update index i is a + // scatter dimension. + std::vector update_dim_is_scatter_dims_; + + // The buffer into which we construct an index into scatter_indices_ to + // fetch the index vector. + std::vector index_vector_index_; + + // The index vector fetched from scatter_indices_. + std::vector index_vector_; + + // The result computed by this functor. operator() returns a Span + // into this vector. + std::vector input_index_; + + const ScatterDimensionNumbers& dim_numbers_; + const Literal& scatter_indices_; + }; + + // This functor computes the contribution of the window indices in an update + // index to an input index. That is, given an update index I it picks out the + // update window indices in I and expands it into a window index into the + // input shape. + // + // This is similar to the class HloEvaluator::OutputWindowIndexToInputIndex + // that does the corresponding function for Gather. + class UpdateWindowIndexToInputIndex { + public: + // The constructor does some setup work that is amortized across all + // iterations. + explicit UpdateWindowIndexToInputIndex( + const ScatterDimensionNumbers& dim_numbers, const Shape& input_shape, + const Shape& updates_shape) { + std::vector window_index_to_update_index; + int64 update_index_count = 0; + for (int64 i = 0; i < updates_shape.dimensions_size(); i++) { + if (absl::c_binary_search(dim_numbers.update_window_dims(), i)) { + window_index_to_update_index.push_back(update_index_count++); + } else { + update_index_count++; + } + } + + int64 window_dim_count = 0; + for (int64 i = 0; i < input_shape.dimensions_size(); i++) { + if (absl::c_binary_search(dim_numbers.inserted_window_dims(), i)) { + input_dim_value_to_update_index_.push_back(-1); + } else { + input_dim_value_to_update_index_.push_back( + window_index_to_update_index[window_dim_count++]); + } + } + + input_index_.resize(input_shape.dimensions_size()); + } + + // Returns the contribution of the window indices to the input index + // corresponding to update_index. See scatter_inner_loop_body. + // + // This is conceptually a stateless transformation from update_index to the + // window input index, but instead of allocating memory to represent the + // scatter input index on every invocation we reuse the same storage for the + // result (input_index_), mutating it in place. + // + // This returns a Span into memory owned by the class. + StatusOr> operator()( + absl::Span update_index) { + PropagateUpdateIndexWindowDimsToInputIndex(update_index); + return absl::Span(input_index_); + } + + // Returns for a given 'input_dim' the corresponding update dimension index, + // or -1 if 'input_dim' is an elided window dimension. + int64 input_dim_value_to_update_index(int64 input_dim) { + return input_dim_value_to_update_index_[input_dim]; + } + + private: + // Propagates window dimensions from the update index to input_index_ by + // mutating input_index_ in place. + void PropagateUpdateIndexWindowDimsToInputIndex( + absl::Span update_index) { + for (int64 i = 0, e = input_index_.size(); i < e; i++) { + if (input_dim_value_to_update_index_[i] != -1) { + input_index_[i] = update_index[input_dim_value_to_update_index_[i]]; + } + + // If input_dim_value_to_index_vector_[i] == -1 then input_index_[i] + // remains 0, as set by the constructor. + } + } + + // input_dim_value_to_index_vector_[i] tells us how to compute dimension i + // of the input index from the update index. See + // PropagateUpdateIndexWindowDimsToInputIndex. + std::vector input_dim_value_to_update_index_; + + // The result computed by this functor. operator() returns a Span + // into this vector. + std::vector input_index_; + }; + + Status HandleScatter(HloInstruction* scatter) override { + const ScatterDimensionNumbers& dim_numbers = + scatter->scatter_dimension_numbers(); + const Literal& operand = + parent_->GetEvaluatedLiteralFor(scatter->operand(0)); + Literal reshaped_scatter_indices; + TF_ASSIGN_OR_RETURN(const Literal& scatter_indices, + ReshapedScatterIndices(dim_numbers.index_vector_dim(), + parent_->GetEvaluatedLiteralFor( + scatter->operand(1)), + &reshaped_scatter_indices)); + const Literal& updates = + parent_->GetEvaluatedLiteralFor(scatter->operand(2)); + const Shape& updates_shape = updates.shape(); + const Shape& operand_shape = operand.shape(); + + ShapeUtil::IndexIterationSpace scatter_indices_iteration_space = + IterationSpaceForUpdateScatterIndices(updates_shape, dim_numbers); + ShapeUtil::IndexIterationSpace window_indices_iteration_space = + IterationSpaceForUpdateWindowIndices(updates_shape, dim_numbers); + + std::vector input_index(operand_shape.dimensions_size()); + std::vector update_index(updates_shape.dimensions_size()); + std::vector input_scatter_index_clamped( + operand_shape.dimensions_size()); + + UpdateScatterIndexToInputIndex update_scatter_index_to_input_index( + &scatter->scatter_dimension_numbers(), /*input_shape=*/operand_shape, + updates_shape, &scatter_indices); + UpdateWindowIndexToInputIndex update_window_index_to_input_index( + scatter->scatter_dimension_numbers(), /*input_shape=*/operand_shape, + updates_shape); + + // Initialize the result with the operand. This makes it easier to handle + // the updates even when the indices are repeated. + Literal result = operand.Clone(); + HloEvaluator embedded_evaluator; + auto scatter_inner_loop_body = + [&](absl::Span update_window_index, + absl::Span input_scatter_index, + absl::Span update_scatter_index) -> StatusOr { + TF_ASSIGN_OR_RETURN( + absl::Span input_window_index, + update_window_index_to_input_index(update_window_index)); + for (int i = 0, e = update_index.size(); i < e; i++) { + update_index[i] = update_scatter_index[i] + update_window_index[i]; + DCHECK_LT(update_index[i], updates_shape.dimensions(i)); + } + for (int i = 0, e = input_scatter_index.size(); i < e; i++) { + int64 update_dim = + update_window_index_to_input_index.input_dim_value_to_update_index( + i); + // If 'update_dim' is -1, it means 'i' is an elided window dim. This + // means we set the iteration index to 0, so for the purpose of the + // following calculations we can consider the update dimension size to + // be 1. + int64 update_dim_size = + update_dim == -1 ? 1 : updates_shape.dimensions(update_dim); + // If any part of the update region is out-of-bounds, then do not + // perform any update on the input. + if ((input_scatter_index[i] < 0) || + (input_scatter_index[i] > + operand_shape.dimensions(i) - update_dim_size)) { + return true; + } + } + for (int i = 0, e = input_index.size(); i < e; i++) { + input_index[i] = input_scatter_index[i] + input_window_index[i]; + } + + auto result_value_literal = + LiteralUtil::CreateR0(result.Get(input_index)); + auto update_value_literal = + LiteralUtil::CreateR0(updates.Get(update_index)); + Literal updated_result = + embedded_evaluator + .Evaluate( + *scatter->to_apply(), + {&result_value_literal, &update_value_literal}) + .ConsumeValueOrDie(); + // Clear visit states so that the we can use the evaluate again on the + // same computation. + embedded_evaluator.ResetVisitStates(); + result.Set(input_index, updated_result.Get({})); + return true; + }; + + auto scatter_outer_loop_body = + [&](absl::Span update_scatter_index) -> StatusOr { + TF_ASSIGN_OR_RETURN( + absl::Span input_scatter_index, + update_scatter_index_to_input_index(update_scatter_index)); + TF_RETURN_IF_ERROR(ShapeUtil::ForEachIndexWithStatus( + updates_shape, window_indices_iteration_space, + [&](absl::Span update_window_index) { + return scatter_inner_loop_body( + update_window_index, input_scatter_index, update_scatter_index); + })); + return true; + }; + + TF_RETURN_IF_ERROR(ShapeUtil::ForEachIndexWithStatus( + updates_shape, scatter_indices_iteration_space, + scatter_outer_loop_body)); + parent_->evaluated_[scatter] = std::move(result); + return Status::OK(); + } + + Status HandleSlice(HloInstruction* slice) override { + auto operand = slice->operand(0); + const Shape& shape = slice->shape(); + TF_ASSIGN_OR_RETURN(auto inferred_return_shape, + ShapeInference::InferSliceShape( + operand->shape(), slice->slice_starts(), + slice->slice_limits(), slice->slice_strides())); + TF_RET_CHECK(ShapeUtil::Compatible(shape, inferred_return_shape)) + << "return shape set to: " << ShapeUtil::HumanString(shape) + << " but is inferred to be: " + << ShapeUtil::HumanString(inferred_return_shape); + + const int64 rank = ShapeUtil::Rank(operand->shape()); + const Literal& operand_literal = parent_->GetEvaluatedLiteralFor(operand); + auto func = [&](absl::Span out_index) { + DimensionVector operand_index(rank); + for (int64 i = 0; i < rank; ++i) { + operand_index[i] = + slice->slice_starts(i) + out_index[i] * slice->slice_strides(i); + } + return operand_literal.Get(operand_index); + }; + + Literal result(shape); + TF_RETURN_IF_ERROR(result.Populate(func)); + parent_->evaluated_[slice] = std::move(result); + return Status::OK(); + } + + // Enable CLZ only for int32, uint32, int64 and uint64. + template < + typename NativeT, + typename std::enable_if< + (std::is_floating_point::value || + std::is_integral::value || is_complex_t::value) && + !(std::is_same::value || + std::is_same::value || + std::is_same::value || + std::is_same::value)>::type* = nullptr> + Status HandleClz(HloInstruction* clz) { + return InvalidArgument("Unsupported type for Clz"); + } + + template ::value || + std::is_same::value>::type* = nullptr> + Status HandleClz(HloInstruction* clz) { + TF_ASSIGN_OR_RETURN(parent_->evaluated_[clz], + ElementWiseUnaryOp(clz, [](ElementwiseT elem_operand) { + return 31 - tensorflow::Log2Floor(elem_operand); + })); + return Status::OK(); + } + + template ::value || + std::is_same::value>::type* = nullptr> + Status HandleClz(HloInstruction* clz) { + TF_ASSIGN_OR_RETURN(parent_->evaluated_[clz], + ElementWiseUnaryOp(clz, [](ElementwiseT elem_operand) { + return 63 - tensorflow::Log2Floor64(elem_operand); + })); + return Status::OK(); + } + + Status HandleClz(HloInstruction* clz) override { + return HandleClz(clz); + } + + template ::value>::type* = nullptr> + Status HandleSin(HloInstruction* sin) { + TF_ASSIGN_OR_RETURN(parent_->evaluated_[sin], + ElementWiseUnaryOp(sin, [](ElementwiseT elem_operand) { + return std::sin(elem_operand); + })); + return Status::OK(); + } + + template < + typename NativeT, + typename std::enable_if::value || + is_complex_t::value>::type* = nullptr> + Status HandleSin(HloInstruction* sin) { + return InvalidArgument("Unsupported type for Sin"); + } + + Status HandleSin(HloInstruction* sin) override { + return HandleSin(sin); + } + + template ::value>::type* = nullptr> + Status HandleCos(HloInstruction* cos) { + TF_ASSIGN_OR_RETURN(parent_->evaluated_[cos], + ElementWiseUnaryOp(cos, [](ElementwiseT elem_operand) { + return std::cos(elem_operand); + })); + return Status::OK(); + } + + template < + typename NativeT, + typename std::enable_if::value || + is_complex_t::value>::type* = nullptr> + Status HandleCos(HloInstruction* cos) { + return InvalidArgument("Unsupported type for Cos"); + } + + Status HandleCos(HloInstruction* cos) override { + return HandleCos(cos); + } + + template ::value>::type* = nullptr> + Status HandleReducePrecision(HloInstruction* reduce_precision) { + TF_ASSIGN_OR_RETURN( + parent_->evaluated_[reduce_precision], + ElementWiseUnaryOp(reduce_precision, [reduce_precision]( + ElementwiseT elem) { + uint32_t value_as_int = tensorflow::bit_cast(elem); + const uint32_t mantissa_bits = reduce_precision->mantissa_bits(); + const uint32_t exponent_bits = reduce_precision->exponent_bits(); + + // Code is based on the CPU/GPU implementation in LLVM-emitting code. + // + // Bits in float type: + // mantissa : bits [0:22] + // exponent : bits [23:30] + // sign : bits [31] + if (mantissa_bits < 23) { + const uint32_t last_mantissa_bit_mask = 1u << (23 - mantissa_bits); + + // Compute rounding bias for round-to-nearest with ties to even. + // This is equal to a base value of 0111... plus one bit if the last + // remaining mantissa bit is 1. + const uint32_t base_rounding_bias = + (last_mantissa_bit_mask >> 1) - 1; + const uint32_t x_last_mantissa_bit = + (value_as_int & last_mantissa_bit_mask) >> (23 - mantissa_bits); + const uint32_t x_rounding_bias = + x_last_mantissa_bit + base_rounding_bias; + + // Add rounding bias, and mask out truncated bits. Note that the + // case where adding the rounding bias overflows into the exponent + // bits is correct; the non-masked mantissa bits will all be zero, + // and the exponent will be incremented by one. + const uint32_t truncation_mask = ~(last_mantissa_bit_mask - 1); + value_as_int = value_as_int + x_rounding_bias; + value_as_int = value_as_int & truncation_mask; + } + if (exponent_bits < 8) { + // Masks for f32 values. + const uint32_t f32_sign_bit_mask = 1u << 31; + const uint32_t f32_exp_bits_mask = 0xffu << 23; + + // An exponent of 2^(n-1)-1 -- that is, 0111... with the zero in the + // most- significant bit -- is equal to 1.0f for all exponent sizes. + // Adding 2^(n-1)-1 to this gives us the highest non-infinite + // exponent for a bit- size of n, and subtracting 2^(n-1)-1 from + // this gives us the lowest' exponent (corresponding to 0.0f). + // + // Thus, the f32 exponent corresponding to the highest non-infinite + // exponent for a bit size of n is (2^7-1) + 2^(n-1)-1, and the f32 + // exponent corresponding to the lowest exponent for a bit size of n + // is (2^7-1) - 2^(n-1)-1. + // + // Note that we have already checked that exponents_bits >= 1. + const uint32_t f32_exponent_bias = (1 << 7) - 1; + const uint32_t reduced_exponent_bias = + (1 << (exponent_bits - 1)) - 1; + const uint32_t reduced_max_exponent = + f32_exponent_bias + reduced_exponent_bias; + const uint32_t reduced_min_exponent = + f32_exponent_bias - reduced_exponent_bias; + + // Do we overflow or underflow? + const uint32_t x_exponent = value_as_int & f32_exp_bits_mask; + const bool x_overflows = x_exponent > (reduced_max_exponent << 23); + const bool x_underflows = + x_exponent <= (reduced_min_exponent << 23); + + // Compute appropriately-signed values of zero and infinity. + const uint32_t x_signed_zero = value_as_int & f32_sign_bit_mask; + const uint32_t x_signed_inf = x_signed_zero | f32_exp_bits_mask; + + // Force to zero or infinity if overflow or underflow. (Note that + // this truncates all denormal values to zero, rather than rounding + // them.) + value_as_int = x_overflows ? x_signed_inf : value_as_int; + value_as_int = x_underflows ? x_signed_zero : value_as_int; + } + + float reduced_result = tensorflow::bit_cast(value_as_int); + if (std::isnan(elem)) { + reduced_result = mantissa_bits > 0 + ? elem + : std::numeric_limits::infinity(); + } + return reduced_result; + })); + return Status::OK(); + } + + template ::value>::type* = nullptr> + Status HandleReducePrecision(HloInstruction* reduce_precision) { + return InvalidArgument("Double not supported for reduce precision"); + } + + template < + typename NativeT, + typename std::enable_if::value || + is_complex_t::value>::type* = nullptr> + Status HandleReducePrecision(HloInstruction* reduce_precision) { + return InvalidArgument("Unsupported type for reduce precision"); + } + + Status HandleReducePrecision(HloInstruction* reduce_precision) override { + return HandleReducePrecision(reduce_precision); + } + + template ::value || + std::is_same::value || + std::is_same::value>::type* = nullptr> + Status HandleIota(HloInstruction* instruction) { + auto* iota = Cast(instruction); + std::vector data(iota->shape().dimensions(iota->iota_dimension())); + std::iota(data.begin(), data.end(), 0); + auto result = LiteralUtil::CreateR1(data); + + if (ShapeUtil::Rank(iota->shape()) > 1) { + TF_ASSIGN_OR_RETURN( + parent_->evaluated_[iota], + result.Broadcast(iota->shape(), {iota->iota_dimension()})); + } else { + TF_RET_CHECK(ShapeUtil::Rank(iota->shape()) == 1); + parent_->evaluated_[iota] = std::move(result); + } + + return Status::OK(); + } + template ::value || + std::is_same::value || + std::is_same::value)>::type* = nullptr> + Status HandleIota(HloInstruction* iota) { + return InvalidArgument("Unsupported type for iota"); + } + Status HandleIota(HloInstruction* iota) override { + return HandleIota(iota); + } + + private: + // Creates a vector of multipliers which can be used to create a linear index + // into shape. + // + // Given the multidimensional index {i1, ..., iN} and + // M = MakeDimMultipliers(shape), the corresponding linear index LI is simply + // + // LI = i1 * M[1] + i2 * M[2] + ... + iN * M[N]. + // + // This lets you calculate LI given the multidimensional indices in any order. + static DimensionVector MakeDimMultipliers(const Shape& shape) { + DimensionVector v(ShapeUtil::Rank(shape)); + int64 scale = 1; + for (auto dim : LayoutUtil::MinorToMajor(shape)) { + v[dim] = scale; + scale *= shape.dimensions(dim); + } + return v; + } + + // For one particular placement of a window in a base shape (the placement is + // represented as `window_count_index`), iterates inside the window. + // Translates the window index into base index. If the base index is within + // bound, call `f` with the base index. + static void IterateThroughWindow( + const Shape& window_shape, const Window& window, const Shape& base_shape, + const absl::Span& window_count_index, + const std::function&)>& f) { + const int64 rank = ShapeUtil::Rank(base_shape); + DimensionVector window_index(rank); + std::fill(window_index.begin(), window_index.end(), 0); + do { + std::vector base_index(rank); + bool out_of_bound = false; + for (int64 i = 0; i < rank; ++i) { + base_index[i] = window_count_index[i] * window.dimensions(i).stride() + + window_index[i] - window.dimensions(i).padding_low(); + if (base_index[i] < 0 || base_index[i] >= base_shape.dimensions(i)) { + out_of_bound = true; + break; + } + } + if (!out_of_bound) { + f(base_index); + } + } while ( + IndexUtil::BumpIndices(window_shape, absl::MakeSpan(window_index))); + } + + template + StatusOr DynamicSlice(const Literal& operand_literal, + const Literal& start_indices_literal, + const Shape& result_shape) { + auto start_indices_typed = start_indices_literal.data(); + std::vector start(start_indices_typed.begin(), + start_indices_typed.end()); + + // Clamp the start indices so the slice is in-bounds w.r.t the operand. + for (int64 i = 0; i < start.size(); ++i) { + start[i] = std::min( + std::max(int64{0}, start[i]), + operand_literal.shape().dimensions(i) - result_shape.dimensions(i)); + } + + std::vector operand_indices(start.size()); + Literal result(result_shape); + TF_RETURN_IF_ERROR( + result.Populate([&](absl::Span multi_index) { + for (int64 i = 0; i < operand_indices.size(); ++i) { + CHECK_GE(multi_index[i] + start[i], 0); + operand_indices[i] = multi_index[i] + start[i]; + } + + auto result = operand_literal.Get(operand_indices); + return result; + })); + + return std::move(result); + } + + template + StatusOr DynamicUpdateSlice(const Literal& operand_literal, + const Literal& update_literal, + const Literal& start_indices_literal) { + auto result = operand_literal.Clone(); + auto start_indices_typed = start_indices_literal.data(); + const auto rank = ShapeUtil::Rank(result.shape()); + std::vector start(start_indices_typed.begin(), + start_indices_typed.end()); + // Clamp the update start indices so the slice is in-bounds w.r.t the + // operand. + for (int64 i = 0; i < rank; ++i) { + start[i] = std::min( + std::max(0, start[i]), + result.shape().dimensions(i) - update_literal.shape().dimensions(i)); + } + std::vector result_index(rank, 0); + + auto func = [&](absl::Span update_index) { + std::transform(update_index.begin(), update_index.end(), start.begin(), + result_index.begin(), std::plus()); + result.Set(result_index, + update_literal.Get(update_index)); + return true; + }; + + std::vector base(update_literal.shape().dimensions_size(), 0); + std::vector step(update_literal.shape().dimensions_size(), 1); + ShapeUtil::ForEachIndex(update_literal.shape(), base, + AsInt64Slice(update_literal.shape().dimensions()), + step, func); + + return std::move(result); + } + + StatusOr ElementWiseUnaryOp( + HloInstruction* instruction, + const std::function& unary_op) { + const Literal& operand_literal = + parent_->GetEvaluatedLiteralFor(instruction->operand(0)); + TF_ASSIGN_OR_RETURN( + auto result_literal, + (HloEvaluator::ElementWiseUnaryOpImpl( + instruction, ConvertUnaryFunction(unary_op), operand_literal))); + + return std::move(result_literal); + } + + StatusOr ElementWiseBinaryOp( + HloInstruction* instruction, + const std::function& + binary_op) { + const auto shape = instruction->shape(); + const auto* lhs = instruction->operand(0); + const auto* rhs = instruction->operand(1); + + // TODO(b/35950897, b/27796129): add DCHECK back once implicit broadcast + // is removed. + if (!(ShapeUtil::SameDimensions(shape, rhs->shape()) && + ShapeUtil::SameDimensions(lhs->shape(), rhs->shape()))) { + return Unimplemented( + "Implicit broadcasting is currently unsupported in HLO evaluator " + "Shape Mismatch: %s vs %s vs %s: ", + ShapeUtil::HumanString(shape), ShapeUtil::HumanString(lhs->shape()), + ShapeUtil::HumanString(rhs->shape())); + } + + const Literal& lhs_literal = parent_->GetEvaluatedLiteralFor(lhs); + const Literal& rhs_literal = parent_->GetEvaluatedLiteralFor(rhs); + + Literal result(shape); + + TF_RETURN_IF_ERROR( + result.Populate([&](absl::Span multi_index) { + return ConvertBinaryFunction(binary_op)( + lhs_literal.Get(multi_index), + rhs_literal.Get(multi_index)); + })); + return std::move(result); + } + + template + StatusOr ElementwiseTernaryOp( + HloInstruction* instruction, + const std::function& ternary_op) { + const auto shape = instruction->shape(); + const auto* lhs = instruction->operand(0); + const auto* rhs = instruction->operand(1); + const auto* ehs = instruction->operand(2); + + // TODO(b/35950897, b/27796129): add DCHECK back once implicit + // broadcast is removed. + if (!(ShapeUtil::SameDimensions(shape, lhs->shape()) && + ShapeUtil::SameDimensions(lhs->shape(), rhs->shape()) && + ShapeUtil::SameDimensions(rhs->shape(), ehs->shape()))) { + return Unimplemented( + "Implicit broadcasting is currently unsupported in HLO evaluator " + "Shape Mismatch: %s vs %s vs %s vs %s: ", + ShapeUtil::HumanString(shape), ShapeUtil::HumanString(lhs->shape()), + ShapeUtil::HumanString(rhs->shape()), + ShapeUtil::HumanString(ehs->shape())); + } + + const Literal& lhs_literal = parent_->GetEvaluatedLiteralFor(lhs); + const Literal& rhs_literal = parent_->GetEvaluatedLiteralFor(rhs); + const Literal& ehs_literal = parent_->GetEvaluatedLiteralFor(ehs); + + Literal result(shape); + + TF_RETURN_IF_ERROR( + result.Populate([&](absl::Span multi_index) { + return ternary_op(lhs_literal.Get(multi_index), + rhs_literal.Get(multi_index), + ehs_literal.Get(multi_index)); + })); + + return std::move(result); + } + + template + static bool IsShiftOutOfBounds(NativeT rhs) { + typedef typename std::make_unsigned::type UnsignedT; + UnsignedT lhs_size_unsigned = sizeof(NativeT) * CHAR_BIT; + UnsignedT rhs_unsigned = static_cast(rhs); + return rhs_unsigned >= lhs_size_unsigned; + } + + HloEvaluator* parent_; +}; + +// These extern templates prevent users of this class from implicitly +// instantiating it. We explicitly instantiate this class in the various +// hlo_evaluator_typed_visitor*.cc files. +extern template class HloEvaluatorTypedVisitor; +extern template class HloEvaluatorTypedVisitor; +extern template class HloEvaluatorTypedVisitor; +extern template class HloEvaluatorTypedVisitor; +extern template class HloEvaluatorTypedVisitor; +extern template class HloEvaluatorTypedVisitor; +extern template class HloEvaluatorTypedVisitor; +extern template class HloEvaluatorTypedVisitor; +extern template class HloEvaluatorTypedVisitor; +extern template class HloEvaluatorTypedVisitor; +extern template class HloEvaluatorTypedVisitor; +extern template class HloEvaluatorTypedVisitor; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_HLO_EVALUATOR_TYPED_VISITOR_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/hlo_execution_profile.h b/linux/include/tensorflow/compiler/xla/service/hlo_execution_profile.h new file mode 100644 index 000000000..be989846e --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/hlo_execution_profile.h @@ -0,0 +1,142 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_HLO_EXECUTION_PROFILE_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_HLO_EXECUTION_PROFILE_H_ + +#include + +#include "tensorflow/compiler/xla/map_util.h" +#include "tensorflow/compiler/xla/service/hlo_cost_analysis.h" +#include "tensorflow/compiler/xla/service/hlo_profile_printer.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/core/platform/stream_executor_no_cuda.h" +#include "tensorflow/core/platform/types.h" + +namespace xla { + +class HloInstruction; + +// Maps all HloInstructions and HloComputations in an HloModule to integers. +// These integers form the contiguous range [0, total_count()). +class HloProfileIndexMap { + public: + // Scans `module` to populate this instance of HloProfileIndexMap. + explicit HloProfileIndexMap(const HloModule& module); + + HloProfileIndexMap(const HloProfileIndexMap&) = default; + HloProfileIndexMap(HloProfileIndexMap&&) = default; + + HloProfileIndexMap& operator=(const HloProfileIndexMap&) = default; + HloProfileIndexMap& operator=(HloProfileIndexMap&&) = default; + + size_t GetProfileIndexFor(const HloInstruction& instruction) const { + return FindOrDie(instruction_to_profile_idx(), &instruction); + } + + size_t GetProfileIndexFor(const HloComputation& computation) const { + return FindOrDie(computation_to_profile_idx(), &computation); + } + + size_t instruction_count() const { + return instruction_to_profile_idx().size(); + } + + size_t computation_count() const { + return computation_to_profile_idx().size(); + } + + size_t total_count() const { + return instruction_count() + computation_count(); + } + + const std::unordered_map& + instruction_to_profile_idx() const { + return instruction_to_profile_idx_; + } + + const std::unordered_map& + computation_to_profile_idx() const { + return computation_to_profile_idx_; + } + + private: + std::unordered_map instruction_to_profile_idx_; + std::unordered_map computation_to_profile_idx_; +}; + +// Create an instance of `HloProfilePrinterData`. +std::unique_ptr CreateHloProfilePrinterData( + const HloProfileIndexMap& hlo_profile_index_map, + const HloCostAnalysis& cost_analysis); + +// Describes how much time each HLO operation took. +// +// Each HloComputation takes a certain number of cycles. This class helps break +// down how much time each HLO took. +class HloExecutionProfile { + public: + using DeviceDescription = se::DeviceDescription; + + HloExecutionProfile(const HloProfilePrinterData* hlo_profile_printer_data, + const HloProfileIndexMap* hlo_profile_index_map); + + // Record how many cycles this HLO took to execute. + void SetCyclesTakenBy(const HloInstruction* hlo, uint64 cycles_taken); + + // Returns how many cycles this HLO took to execute. Profiling information + // may not be available for some instructions in which case zero is returned. + uint64 GetCyclesTakenBy(const HloInstruction& hlo) const; + + // Return the number of cycles this computation took to execute. + uint64 total_cycles_executed(const HloComputation& computation) const { + return profile_counters_[hlo_profile_index_map_.GetProfileIndexFor( + computation)]; + } + + // Record how many cycles a computation took to execute. + void set_total_cycles_executed(const HloComputation& computation, + uint64 total_cycles_executed) { + profile_counters_[hlo_profile_index_map_.GetProfileIndexFor(computation)] = + total_cycles_executed; + } + + // Returns a version of the execution profile suitable for performance + // debugging; e.g. emits cycle counts, execution time at the nominal device + // frequency, and the effective throughput given the provided cost_analysis + // for the operations in a given computation. Returns an empty string if it + // wasn't possible to generate a printable version. + string ToString(const DeviceDescription& device_description) const { + return PrintHloProfile(hlo_profile_printer_data_, profile_counters_.data(), + device_description.clock_rate_ghz()); + } + + std::vector* mutable_profile_counters() { return &profile_counters_; } + const std::vector& profile_counters() const { + return profile_counters_; + } + + private: + const HloProfilePrinterData& hlo_profile_printer_data_; + const HloProfileIndexMap& hlo_profile_index_map_; + + // Stores per-Hlo profile counters. This is the only thing that changes when + // we execute an XLA computation. + std::vector profile_counters_; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_HLO_EXECUTION_PROFILE_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/hlo_graph_dumper.h b/linux/include/tensorflow/compiler/xla/service/hlo_graph_dumper.h new file mode 100644 index 000000000..0b11f34ab --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/hlo_graph_dumper.h @@ -0,0 +1,104 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_HLO_GRAPH_DUMPER_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_HLO_GRAPH_DUMPER_H_ + +#include + +#include "tensorflow/compiler/xla/service/hlo_computation.h" +#include "tensorflow/compiler/xla/service/hlo_execution_profile.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/compiler/xla/xla.pb.h" + +namespace xla { +namespace hlo_graph_dumper { + +// Abstract interface for classes that render HLO graphs (e.g. DOT graph, +// tensorflow GraphDef). +class GraphRendererInterface { + public: + enum GraphKind { + DOT_GRAPH, + TF_GRAPHDEF, + }; + + virtual ~GraphRendererInterface() = default; + + // Renders a DOT graph, returning a description of the rendered output + // (e.g., a URL) + virtual string RenderGraph(const string& graph, GraphKind graph_kind, + const DebugOptions& debug_options) = 0; +}; + +// Dump the given HLO module if a dump is requested in its debug options. Based +// on the debug options, either a graph dump, a text dump or both may be +// generated. If a graph dump is generated, the description (e.g. an URL) is +// returned; otherwise an empty string is returned. +string MaybeDumpHloModule(const HloModule& module, const string& label, + const HloExecutionProfile* profile = nullptr); + +// Dumps a graph of the computation and returns a description of the rendered +// graph (e.g., a URL) based on the renderer. The "best" renderer in the +// registry is used. +string DumpGraph(const HloComputation& computation, const string& label, + const DebugOptions& debug_options, + const HloExecutionProfile* hlo_execution_profile = nullptr, + bool show_backend_config = false); + +// Like DumpGraph, but renders only nodes "near" the given node in the graph. +// +// The number of nodes dumped is controlled by the radius parameter, which +// (roughly) corresponds to the max distance a node may be from the primary node +// before it's omitted from the graph. +string DumpNeighborhoodAround(const HloInstruction& node, int radius, + bool show_backend_config = false); + +// Dumps the HloModule::ToString() as a file into the provided directory path +// suffixed with the provided label. +// +// If do_prefix is true, a timestamp will be prepended onto the label to +// construct a filename in the directory path; otherwise, the label is used +// as the filename directly. +void DumpText(const HloModule& module, const string& label, + const string& directory_path, bool do_prefix = true); + +// Graph renderers may be added using a registration mechanism, e.g.: +// XLA_REGISTER_GRAPH_RENDERER(AGraphRendererClass, 100) +// The renderer with the highest numeric priority value is used. + +#define XLA_REGISTER_GRAPH_RENDERER(factory, ...) \ + XLA_INTERNAL_REGISTER_GRAPH_RENDERER(factory, __COUNTER__, ##__VA_ARGS__) + +// Internal implementation details below this point. + +// Class that registers a graph renderer. +class Registrar { + public: + Registrar(GraphRendererInterface* dumper); +}; + +#define XLA_INTERNAL_REGISTER_GRAPH_RENDERER(factory, ctr, ...) \ + static ::xla::hlo_graph_dumper::Registrar \ + XLA_INTERNAL_REGISTER_GRAPH_RENDERER_NAME(ctr)(new factory, \ + ##__VA_ARGS__) + +// __COUNTER__ must go through another macro to be properly expanded +#define XLA_INTERNAL_REGISTER_GRAPH_RENDERER_NAME(ctr) ___##ctr##__object_ + +} // namespace hlo_graph_dumper +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_HLO_GRAPH_DUMPER_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/hlo_instruction.h b/linux/include/tensorflow/compiler/xla/service/hlo_instruction.h new file mode 100644 index 000000000..1bfdc88ab --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/hlo_instruction.h @@ -0,0 +1,1715 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// HLO instructions are in DAG form and represent the computations that the user +// has built up via the XLA service interface. They are ultimately lowered +// in a platform-aware way by traversing the HLO DAG and emitting a lowered +// form; e.g. see DfsHloVisitor. + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_HLO_INSTRUCTION_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_HLO_INSTRUCTION_H_ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "absl/container/flat_hash_map.h" +#include "absl/container/inlined_vector.h" +#include "absl/memory/memory.h" +#include "absl/strings/str_cat.h" +#include "absl/strings/string_view.h" +#include "absl/types/span.h" +#include "tensorflow/compiler/xla/iterator_util.h" +#include "tensorflow/compiler/xla/literal.h" +#include "tensorflow/compiler/xla/map_util.h" +#include "tensorflow/compiler/xla/service/dfs_hlo_visitor.h" +#include "tensorflow/compiler/xla/service/hlo.pb.h" +#include "tensorflow/compiler/xla/service/hlo_clone_context.h" +#include "tensorflow/compiler/xla/service/hlo_domain_metadata.h" +#include "tensorflow/compiler/xla/service/hlo_opcode.h" +#include "tensorflow/compiler/xla/service/hlo_sharding.h" +#include "tensorflow/compiler/xla/service/name_uniquer.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/lib/gtl/iterator_range.h" +#include "tensorflow/core/platform/logging.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/protobuf.h" +#include "tensorflow/core/platform/types.h" + +namespace xla { + +class HloComputation; +class HloModule; + +// A bunch of switches that control how the hlo text should be printed. +class HloPrintOptions { + public: + enum class PrintSubcomputationMode { + kOff, // Do not print anything about subcomputations. + kNameOnly, // Only print the name of subcomputations. + kFullBodies, // Print the full bodies of subcomputations. + }; + + // Constructs the default print options: don't print large constants, don't + // compact operands, no indentation. + HloPrintOptions() + : print_large_constants_(false), + print_subcomputation_mode_(PrintSubcomputationMode::kNameOnly), + print_metadata_(true), + print_backend_config_(true), + compact_operands_(false), + print_operand_shape_(true), + print_program_shape_(true), + print_percent_(true), + print_control_dependencies_(true), + canonicalize_instruction_names_(false), + indent_amount_(0), + is_in_nested_computation_(false) {} + + static HloPrintOptions ShortParsable() { + return HloPrintOptions() + .set_print_large_constants(true) + .set_print_subcomputation_mode(PrintSubcomputationMode::kNameOnly) + .set_print_metadata(false) + .set_print_backend_config(false) + .set_print_operand_shape(false) + .set_print_program_shape(false) + .set_print_percent(false) + .set_print_control_dependencies(false); + } + + // Options to produce the canonical string representing an isomorphic + // computation graph. + static HloPrintOptions Canonical() { + return HloPrintOptions() + .set_print_subcomputation_mode(PrintSubcomputationMode::kFullBodies) + .set_print_metadata(false) + .set_print_backend_config(false) + .set_compact_operands(true) + .set_print_operand_shape(true) + .set_print_program_shape(false) + .set_print_percent(false) + .set_print_control_dependencies(false) + .set_canonicalize_instruction_names(true); + } + + // If true, large constants will be printed out. + HloPrintOptions& set_print_large_constants(bool value) { + print_large_constants_ = value; + return *this; + } + + HloPrintOptions& set_print_subcomputation_mode( + PrintSubcomputationMode value) { + print_subcomputation_mode_ = value; + return *this; + } + + // If true, metadata will be printed. + HloPrintOptions& set_print_metadata(bool value) { + print_metadata_ = value; + return *this; + } + + // If true, backend_config will be printed. + HloPrintOptions& set_print_backend_config(bool value) { + print_backend_config_ = value; + return *this; + } + + // If true, operands' shapes will be printed. + HloPrintOptions& set_print_operand_shape(bool value) { + print_operand_shape_ = value; + return *this; + } + + // If true, program shape of hlo computations will be printed. + HloPrintOptions& set_print_program_shape(bool value) { + print_program_shape_ = value; + return *this; + } + + // If true, names will be printed with prefix '%'. + HloPrintOptions& set_print_percent(bool value) { + print_percent_ = value; + return *this; + } + + // If true, control dependencies will be printed. + HloPrintOptions& set_print_control_dependencies(bool value) { + print_control_dependencies_ = value; + return *this; + } + + // If true, only a part of operands will be printed out, and their names will + // be omitted (note that in this case the text will not be parsable). + HloPrintOptions& set_compact_operands(bool value) { + compact_operands_ = value; + return *this; + } + + // If true, canonicalizes instructions' name. Instead of using "%foo.1" as + // the name of an instruction, we use "%tmp_1", "%tmp_2" etc. + HloPrintOptions& set_canonicalize_instruction_names(bool value) { + canonicalize_instruction_names_ = value; + return *this; + } + + // The indent of the hlo text block. + HloPrintOptions& set_indent_amount(int value) { + indent_amount_ = value; + return *this; + } + + // If true, indicates the instruction being printed is inside a nested + // computation. + HloPrintOptions& set_is_in_nested_computation(bool value) { + is_in_nested_computation_ = value; + return *this; + } + + bool print_large_constants() const { return print_large_constants_; } + PrintSubcomputationMode print_subcomputation_mode() const { + return print_subcomputation_mode_; + } + bool print_metadata() const { return print_metadata_; } + bool print_backend_config() const { return print_backend_config_; } + bool compact_operands() const { return compact_operands_; } + bool print_operand_shape() const { return print_operand_shape_; } + bool print_program_shape() const { return print_program_shape_; } + bool print_percent() const { return print_percent_; } + bool print_control_dependencies() const { + return print_control_dependencies_; + } + bool canonicalize_instruction_names() const { + return canonicalize_instruction_names_; + } + int indent_amount() const { return indent_amount_; } + int is_in_nested_computation() const { return is_in_nested_computation_; } + + private: + bool print_large_constants_; + PrintSubcomputationMode print_subcomputation_mode_; + bool print_metadata_; + bool print_backend_config_; + bool compact_operands_; + bool print_operand_shape_; + bool print_program_shape_; + bool print_percent_; + bool print_control_dependencies_; + bool canonicalize_instruction_names_; + int indent_amount_; + bool is_in_nested_computation_; +}; + +// For canonical string output, we need to have a canonical way to rename +// each instruction and its operands. Each operand is renamed as "tmp_", +// where is an index starting from 0. +class CanonicalNameMap { + public: + CanonicalNameMap() : index(0) {} + + string LookupOrInsert(const string& old_name) { + auto iter = canonical_name_map.find(old_name); + if (iter != canonical_name_map.end()) { + return iter->second; + } + + string new_name = absl::StrCat("tmp_", index++); + canonical_name_map[old_name] = new_name; + return new_name; + } + void Clear() { + canonical_name_map.clear(); + index = 0; + } + + private: + int64 index; + absl::flat_hash_map canonical_name_map; +}; + +// HLO instructions are the atomic unit of the high-level compiler's IR. +// +// HloInstructions live inside of an HloComputation, which is analogous to a +// function in other programming languages. Nodes have no total order within +// their computation. Instead, they have a partial ordering determined by their +// data and control dependencies. +// +// HLO does not have basic blocks or explicit "branch" instructions. Instead, +// certain HloInstructions -- namely, kWhile, kConditional, and kCall -- encode +// control flow. For example, the kConditional HLO executes one of two possible +// computations, depending on the runtime value of a predicate. +// +// HLO is pure (mostly). It has no concept of mutable state. Instead, data +// values are produced by one HLO and flow into consumers across dependency +// edges. +class HloInstruction { + public: + // A fusion node computes the same value a call to its fusion computation + // would compute. However, the choice of fusion kind dictates codegen + // strategy for the backend. + // + // To generate code for a kFusion HloInstruction, most backends do something + // like the following: + // + // 1) Identify the "primary" HloInstruction of the fused computation. + // 2) Emit code that does the work of the primary node, creating its inputs + // and transforming its outputs as specified by the fused computation. + // + // In step (2), the code emitted is usually similar to the code that would be + // emitted for an *unfused* version of the primary node, except that + // + // - when the primary node reads an element of one of its operands, instead + // of loading the value from memory, it *computes* the value based on the + // contents of the fused computation. + // - when the primary node outputs a value, instead of storing it to memory, + // it forwards the value to its users, which then perform additional + // computations before the value is finally stored to memory at the root of + // the fusion node. + // + // An HloInstruction's FusionKind helps us find the kFusion instruction's + // primary node, and can also affect how we generate code in step (2). + // + // - kInput: The primary node is the root of the fused instruction. + // + // - kOutput: The primary node is not the root of the fused instruction. + // This fusion kind requires that one operand buffer of the fusion + // instruction be able to alias the output buffer. This constraint is + // usually enough to let backends find the primary node unambiguously. + // + // - kLoop: The primary node is the root of the fused computation, but, + // unlike in input fusion, we prescribe a specific implementation for + // codegen. Rather than generating code that looks like the code we'd emit + // for an unfused version of the primary/root node, we emit code that + // generates one element of the root at a time. + // + // - kCustom: Custom category for backend-specific fusions that don't fit + // into the above patterns. + // + // Not all backends support all fusion kinds, and given a particular fused + // computation, it's not in general safe to change its fusion kind. Creation + // of fusion nodes is always backend-specific. + // + // For elementwise ops (e.g. kAdd), most backends would emit a + // one-element-at-a-time implementation for the unfused version, so loop + // fusion and input fusion are probably equivalent if the root node is + // elementwise. They're not necessarily equivalent e.g. for kReduce, where an + // implementation might emit something more sophisticated for an unfused or + // input-fusion reduce, but will emit the naive code that reduces one element + // at a time for loop fusion with a reduce as the root. + // + // Another way to think of loop fusion is that it's equivalent to input + // fusion, but where the root node is an implicit identity node, whose + // unfused implementation is "read one element, write one element". + // + // TODO(b/79869434): This categorization scheme is not great. For one thing, + // input and loop fusion are basically the same thing: There is no reason for + // the HLO to encode backend-specific decisions about how e.g. a reduce that's + // the root of a fusion should be lowered. In addition, this scheme as + // written doesn't work for multi-output fusion, where the primary node is + // never actually the root (which is a kTuple instruction that gathers the + // multiple outputs of the fusion). + enum class FusionKind { + kLoop, + kInput, + kOutput, + kCustom, + }; + + virtual ~HloInstruction(); + + // Creates an instruction from the given proto. Arguments: + // + // proto: the proto to convert from. + // instruction_map: a map from instruction id to HloInstruction*. This map + // must contain all operands of the newly constructed instruction. + // computation_map: a map from computation id to HloComputation*. This map + // must contain all computations which the newly constructed instruction + // calls. + static StatusOr> CreateFromProto( + const HloInstructionProto& proto, + const absl::flat_hash_map& instruction_map, + const absl::flat_hash_map& computation_map); + + // Creates a parameter-retrieving instruction. + static std::unique_ptr CreateParameter(int64 parameter_number, + const Shape& shape, + const string& name); + + // Creates a literal constant instruction. + static std::unique_ptr CreateConstant(Literal literal); + + // Creates an Iota instruction. + static std::unique_ptr CreateIota(const Shape& shape, + int64 iota_dimension); + + // Creates a get tuple element instruction. + static std::unique_ptr CreateGetTupleElement( + const Shape& shape, HloInstruction* operand, int64 index); + + // Creates a trace instruction that logs the input operand in the computation. + static std::unique_ptr CreateTrace(const string& tag, + HloInstruction* operand); + + // Creates a random number generation instruction that fills a shape with + // random numbers from a given distribution. + static std::unique_ptr CreateRng( + const Shape& shape, RandomDistribution distribution, + absl::Span parameters); + + // Creates a unary instruction (one operand). + // Precondition: opcode must be a legitimate unary operation. + static std::unique_ptr CreateUnary(const Shape& shape, + HloOpcode opcode, + HloInstruction* operand); + + // Creates a binary instruction (two operands). + // Precondition: opcode must be a legitimate binary operation. + static std::unique_ptr CreateBinary(const Shape& shape, + HloOpcode opcode, + HloInstruction* lhs, + HloInstruction* rhs); + + // Creates a ternary instruction (three operands). + // Precondition: opcode must be a legitimate ternary operation. + static std::unique_ptr CreateTernary(const Shape& shape, + HloOpcode opcode, + HloInstruction* lhs, + HloInstruction* rhs, + HloInstruction* ehs); + + // Creates a variadic instruction (variable number of operands). + // Precondition: opcode must be a legitimate variadic operation. + static std::unique_ptr CreateVariadic( + const Shape& shape, HloOpcode opcode, + absl::Span operands); + + // Creates a map instruction, where the computation (given by the handle) is + // applied element-wise to every element in operands (across the operands, + // at a given index) + static std::unique_ptr CreateMap( + const Shape& shape, absl::Span operands, + HloComputation* map_computation); + + // Creates a convolution op, where rhs is the convolutional filter + // and window describes how the filter is applied to lhs. + static std::unique_ptr CreateConvolve( + const Shape& shape, HloInstruction* lhs, HloInstruction* rhs, + int64 feature_group_count, const Window& window, + const ConvolutionDimensionNumbers& dimension_numbers, + const PrecisionConfig& precision_config); + + // Creates an FFT op, of the type indicated by fft_type. + static std::unique_ptr CreateFft( + const Shape& shape, HloInstruction* operand, FftType fft_type, + absl::Span fft_length); + + // Creates a dot op with operands 'lhs' and 'rhs' with contracting and batch + // dimensions specified in 'dimension_numbers'. + static std::unique_ptr CreateDot( + const Shape& shape, HloInstruction* lhs, HloInstruction* rhs, + const DotDimensionNumbers& dimension_numbers, + const PrecisionConfig& precision_config); + + // Creates a reduce-precision op, where operand is the data to reduce in + // precision, and exponent_bits and mantissa_bits describe the precision to + // reduce it to. + static std::unique_ptr CreateReducePrecision( + const Shape& shape, HloInstruction* operand, const int exponent_bits, + const int mantissa_bits); + + // Creates a cross replica reduction op. + // + // `reduction_computation`: the reduction function. + // + // `replica_groups`: each ReplicaGroup contains a list of replica id. If + // empty, all replicas belong to one group in the order of 0 - (n-1). + // Allreduce will be applied within subgroups. + // For example, we have 4 replicas, then replica_groups={{0,2},{1,3}} means, + // replica 0 and 2 are in subgroup 0, replica 1 and 3 are in subgroup 1. + // + // `all_reduce_id`: for Allreduce nodes from different modules, if they have + // the same all_reduce_id, they will be 'Allreduce'd. If empty, Allreduce will + // not be applied cross modules. + // + // TODO(b/79737069): Rename this to AllReduce. + static std::unique_ptr CreateCrossReplicaSum( + const Shape& shape, absl::Span operands, + HloComputation* reduce_computation, + const std::vector& replica_groups, + absl::string_view barrier, const absl::optional& all_reduce_id); + + // This op handles the communication of an Alltoall operation. On each core, + // the operands are N ops in the same shape, where N is the number of cores + // participating the Alltoall. Then the N operands are scattered to N cores, + // e.g., the ith operand is sent to the ith core. Then each core gathers the + // received data into a tuple. + // + // - `replica_groups`: each ReplicaGroup contains a list of replica id. If + // empty, all replicas belong to one group in the order of 0 - (n-1). Alltoall + // will be applied within subgroups in the specified order. For example, + // replica groups = {{1,2,3},{4,5,0}} means, an Alltoall will be applied + // within replica 1, 2, 3, and in the gather phase, the received blocks will + // be concatenated in the order of 1, 2, 3; another Alltoall will be applied + // within replica 4, 5, 0, and the concatenation order is 4, 5, 0. + static std::unique_ptr CreateAllToAll( + const Shape& shape, absl::Span operands, + const std::vector& replica_groups); + + // Creates a communitation instructions that permutes data cross replicas. + // Data is sent/received according to the (source_replica_id, + // target_replica_id) pairs in `source_target_pairs`. If a replica id is not a + // target_replica_id in any pair, the output on that replica is a tensor + // conssits of 0(s) in `shape`. + static std::unique_ptr CreateCollectivePermute( + const Shape& shape, HloInstruction* operand, + const std::vector>& source_target_pairs); + + // Creates a conversion instruction, where operand is the data to convert and + // shape is the target shape for the conversion. + static std::unique_ptr CreateConvert(const Shape& shape, + HloInstruction* operand); + + // Creates a bitcast conversion instruction, where operand is the data to + // convert and shape is the target shape for the conversion. + static std::unique_ptr CreateBitcastConvert( + const Shape& shape, HloInstruction* operand); + + // Creates an infeed instruction, which reads data of the given shape from the + // Infeed interface of the device. infeed_shape is the shape of the data + // received from the infeed *not* the shape of the infeed instruction which + // is a tuple containing the infeed_shape and the TOKEN. + static std::unique_ptr CreateInfeed( + const Shape& infeed_shape, HloInstruction* token_operand, + const string& config); + + // Creates an outfeed instruction, which outputs data. outfeed_shape is the + // shape of the data being outfed *not* the shape of the outfeed instruction + // which is a TOKEN. + static std::unique_ptr CreateOutfeed( + const Shape& outfeed_shape, HloInstruction* operand, + HloInstruction* token_operand, absl::string_view outfeed_config); + + // Creates an asynchronous send instruction with the given channel id, which + // initiates sending the operand data to a unique receive instruction in + // another computation that has the same channel id. If is_host_transfer is + // true, then this Send operation transfers data to the host. + static std::unique_ptr CreateSend( + HloInstruction* operand, HloInstruction* token, int64 channel_id, + bool is_host_transfer = false); + + // Blocks until data transfer for the Send instruction (operand) is complete. + // The operand must be kSend. + static std::unique_ptr CreateSendDone( + HloInstruction* operand, bool is_host_transfer = false); + + // Creates an asynchronous receive instruction with the given channel id, + // which allocates resources to receive data of the given shape from a unique + // send instruction in another computation that has the same channel id. If + // is_host_transfer is true, then this Send operation transfers data from the + // host. + static std::unique_ptr CreateRecv( + const Shape& shape, HloInstruction* token, int64 channel_id, + bool is_host_transfer = false); + + // Blocks until data transfer for the Recv instruction (operand) is complete + // and returns the receive buffer. The operand must be kRecv. + static std::unique_ptr CreateRecvDone( + HloInstruction* operand, bool is_host_transfer = false); + + // Creates a slice instruction, where the operand is sliced by the given + // start/limit indices. + static std::unique_ptr CreateSlice( + const Shape& shape, HloInstruction* operand, + absl::Span start_indices, + absl::Span limit_indices, absl::Span strides); + + // Creates a slice instruction, where the first operand is sliced by + // start indices specified in the second operand, and by size specified in + // 'slice_sizes'. + static std::unique_ptr CreateDynamicSlice( + const Shape& shape, HloInstruction* operand, + HloInstruction* start_indices, absl::Span slice_sizes); + + // Creates a dynamic update slice instruction, which updates a slice + // of 'operand' with 'update' and 'start_indices'. + static std::unique_ptr CreateDynamicUpdateSlice( + const Shape& shape, HloInstruction* operand, HloInstruction* update, + HloInstruction* start_indices); + + // Creates a concatenate instruction, where the operands are concatenated on + // the provided dimension. + static std::unique_ptr CreateConcatenate( + const Shape& shape, absl::Span operands, + int64 dimension); + + // Creates a reduce instruction, where the computation (given by the handle) + // is applied successively to every element in operand. For example, let f be + // the function to apply, which takes 2 arguments, an accumulator and the + // current value. Let init be an initial value (which is normally chosen to be + // the identity element for f, e.g. 0 if f is addition). + // Then the reduce HLO will compute: + // f(f(init, value0), value1), ...) + static std::unique_ptr CreateReduce( + const Shape& shape, HloInstruction* operand, HloInstruction* init_value, + absl::Span dimensions_to_reduce, + HloComputation* reduce_computation); + + // A more general, multiple-argument version of the above. + // The function to apply, f, now takes N arguments: + // [accumulator0, accumulator1, ..., accumulatorN, value0, value1, ..., + // init_valueN], and returns an N-tuple. The performed computation is (for + // commutative and associative f operators) equivalent to: + // + // f_1 = f(init0, ... initN, input0.value0, ..., inputN.value0) + // f_2 = f(f_1.tuple_element(0), ..., f_1.tuple_element(N), input0.value1, + // ..., inputN.value1) + // ... + // TODO(b/112040122): Add support to this in HLO passes and in backends. + static std::unique_ptr CreateReduce( + const Shape& shape, absl::Span operands, + absl::Span init_values, + absl::Span dimensions_to_reduce, + HloComputation* reduce_computation); + + // Creates a reduce-window instruction, where the computation (given + // by the handle) is applied window-wise at each valid window + // position in the operand. + static std::unique_ptr CreateReduceWindow( + const Shape& shape, HloInstruction* operand, HloInstruction* init_value, + const Window& window, HloComputation* reduce_computation); + + // Creates a batch-norm-training instruction. + static std::unique_ptr CreateBatchNormTraining( + const Shape& shape, HloInstruction* operand, HloInstruction* scale, + HloInstruction* offset, float epsilon, int64 feature_index); + + // Creates a batch-norm-inference instruction. + static std::unique_ptr CreateBatchNormInference( + const Shape& shape, HloInstruction* operand, HloInstruction* scale, + HloInstruction* offset, HloInstruction* mean, HloInstruction* variance, + float epsilon, int64 feature_index); + + // Creates a batch-norm-grad instruction. + static std::unique_ptr CreateBatchNormGrad( + const Shape& shape, HloInstruction* operand, HloInstruction* scale, + HloInstruction* mean, HloInstruction* variance, + HloInstruction* grad_output, float epsilon, int64 feature_index); + + // Creates a scatter computation that scatters the `source` array to the + // selected indices of each window. + static std::unique_ptr CreateSelectAndScatter( + const Shape& shape, HloInstruction* operand, HloComputation* select, + const Window& window, HloInstruction* source, HloInstruction* init_value, + HloComputation* scatter); + + // Creates a broadcast instruction. + static std::unique_ptr CreateBroadcast( + const Shape& shape, HloInstruction* operand, + absl::Span broadcast_dimensions); + + // Creates a sequence of instructions that performs an explicit broadcast of + // the operand to the target shape. + // + // Interior HLOs are passed to "adder", but the "root" HLO of the sequence is + // returned as a unique_ptr for API consistency with other factory methods in + // this interface. + // + // TODO(b/72173833) Ideally HloComputations would always be present, and so + // the adder being passed by the caller would not be necessary. + static std::unique_ptr CreateBroadcastSequence( + const Shape& output_shape, HloInstruction* operand, + const std::function)>& + adder); + + // Creates a pad instruction, where the operand is padded on the edges and + // between the elements with the given padding value. + static std::unique_ptr CreatePad( + const Shape& shape, HloInstruction* operand, + HloInstruction* padding_value, const PaddingConfig& padding_config); + + // Creates a reshape instruction, where the operand is flattened row-major + // order and then reshaped to the given result shape. + static std::unique_ptr CreateReshape(const Shape& shape, + HloInstruction* operand); + + // Creates a transpose instruction which permutes the operand dimensions. + static std::unique_ptr CreateTranspose( + const Shape& shape, HloInstruction* operand, + absl::Span dimensions); + + // Creates a sort op, with a keys operand, and an optional values operand. + static std::unique_ptr CreateSort( + const Shape& shape, int64 dimension, HloInstruction* keys, + HloInstruction* values = nullptr); + + // Creates a while instruction, given a condition computation, a body + // computation, and the initial value for the input of the computations. For + // example, shape: S32, condition: i -> i < 1000, body: i -> i * 2, init: 1 + // corresponds to the C code below. + // int32 i = 1; int32 result = while(i < 1000) { i = i * 2 } + static std::unique_ptr CreateWhile(const Shape& shape, + HloComputation* condition, + HloComputation* body, + HloInstruction* init); + + static std::unique_ptr CreateConditional( + const Shape& shape, HloInstruction* pred, + HloInstruction* true_computation_arg, HloComputation* true_computation, + HloInstruction* false_computation_arg, HloComputation* false_computation); + + static std::unique_ptr CreateGather( + const Shape& shape, HloInstruction* operand, + HloInstruction* start_indices, + const GatherDimensionNumbers& gather_dim_numbers, + absl::Span slice_sizes); + + static std::unique_ptr CreateScatter( + const Shape& shape, HloInstruction* operand, + HloInstruction* scatter_indices, HloInstruction* updates, + HloComputation* update_computation, + const ScatterDimensionNumbers& scatter_dim_numbers); + + // Creates a kDomain instruction which delimits an HLO domain which have + // the provided user and operand side metadata. + static std::unique_ptr CreateDomain( + const Shape& shape, HloInstruction* operand, + std::unique_ptr operand_side_metadata, + std::unique_ptr user_side_metadata); + + // Creates a fusion instruction. A fusion instruction contains one or more + // fused instructions forming an expression with a single root + // "fused_root". Additional instructions can be added to the fusion + // instruction with the method FuseInstruction. + static std::unique_ptr CreateFusion( + const Shape& shape, FusionKind fusion_kind, HloInstruction* fused_root); + + static std::unique_ptr CreateFusion( + const Shape& shape, FusionKind fusion_kind, + absl::Span operands, + HloComputation* fusion_computation); + + // Creates a call instruction that applies the given computation on the given + // operands. "shape" is the resultant shape. + static std::unique_ptr CreateCall( + const Shape& shape, absl::Span operands, + HloComputation* computation); + + // Creates a custom call instruction that applies the given custom call target + // to the given operands. "opaque" can be an arbitrary string with a + // backend-specific interpretation. "shape" is the resultant shape. + static std::unique_ptr CreateCustomCall( + const Shape& shape, absl::Span operands, + absl::string_view custom_call_target, absl::string_view opaque = ""); + + // Creates a tuple instruction with the given elements. This is a convenience + // wrapper around CreateVariadic. + static std::unique_ptr CreateTuple( + absl::Span elements); + + // Creates a reverse instruction, which reverses the order of the elements + // in the specified dimensions. + static std::unique_ptr CreateReverse( + const Shape& shape, HloInstruction* operand, + absl::Span dimensions); + + // Creates a Afterall instruction used for joining or creating new values of + // token type which thread through side-effecting operations. Operands must + // all be tokens, and there must be at least one operand. + static std::unique_ptr CreateAfterAll( + absl::Span operands); + + // Creates an AfterAll instruction which creates a token type out of thin air + // (no operands). This is a separate method from CreateAfterAll to facility + // the removal of operand-less AfterAll instructions. + // TODO(b/110532604): Remove this capability of creating a token from nothing + // when we plumb a primordial token from the entry computation. + static std::unique_ptr CreateToken(); + + // Returns the opcode for this instruction. + HloOpcode opcode() const { return opcode_; } + + // Returns true if this instruction has a side effect, irrespective of whether + // any called computations may contain an instruction with side effects. + bool HasSideEffectNoRecurse() const; + + // Returns true if this instruction has a side effect. An instruction has a + // side effect if it uses certain opcodes or calls a computation with a side + // effect. + bool HasSideEffect() const; + + // Returns the result shape of this instruction. + const Shape& shape() const; + + // Returns the (mutable) result shape of this instruction. + Shape* mutable_shape() { return &shape_; } + + // Returns the ith operand to this instruction. + const HloInstruction* operand(int64 i) const; + + // Returns the ith operand to this instruction. + HloInstruction* mutable_operand(int64 i); + + // Returns the number of operands to this instruction. + int64 operand_count() const { return operands_.size(); } + + // Returns the vector of operands of this instruction. + using InstructionVector = absl::InlinedVector; + const InstructionVector& operands() const { return operands_; } + + // Returns the vector of unique operands, in the same order they are found + // within the operand vector. + InstructionVector unique_operands() const; + + // Returns the index of 'target' in the operands sequence. + // Precondition: target must be an operand (or a fatal error will occur). + int64 operand_index(const HloInstruction* target) const; + + // Returns the number of users of this instruction. + int64 user_count() const { return users_.size(); } + + // Returns the users of this instruction. + const std::vector& users() const { return users_; } + + // Returns true if this instruction is a user of 'instruction'. + bool IsUserOf(const HloInstruction* instruction) const { + return ContainsKey(instruction->user_set_, this); + } + + // Adds a control dependency from this instruction to the given + // instruction. This instruction becomes a control predecessor of + // 'instruction', and 'instruction' becomes a control successor of this + // instruction. Returns an error status if either of the given instructions + // does not belong to the same computation. + // + // This is used to enforce an additional ordering requirement that is not + // captured by normal data dependencies, such as ordering among Send or Recv + // operations to avoid deadlock. + Status AddControlDependencyTo(HloInstruction* instruction); + + // Removes a previously added control dependency from this instruction to + // 'instruction'. + Status RemoveControlDependencyTo(HloInstruction* instruction); + + // Drops all control predecessors and successors from this HLO instruction. + Status DropAllControlDeps(); + + // Copies the control predecessors and successors on this HLO instruction to + // `inst`. Does not do a deep copy so this makes sense only if `inst` and + // this HLO are in the same module. + // + // Depending on the use cases we see in practice, in the future we may + // consider folding the logic here into Clone, CloneWithNewOperands and + // ReplaceAllUsesWith by treating control dependencies like data dependencies. + Status CopyAllControlDepsFrom(const HloInstruction* inst); + + // Returns the set of control predecessors (successors) of this + // instruction. Control predecessors (successors) must execute before (after) + // the current instruction. + const std::vector& control_predecessors() const { + return control_predecessors_; + } + const std::vector& control_successors() const { + return control_successors_; + } + + // Returns true if "other" performs the same computation as this instruction. + bool Identical( + const HloInstruction& other, + const std::function& + eq_operands = std::equal_to(), + const std::function& + eq_computations = std::equal_to(), + bool layout_sensitive = true) const { + // An instruction is always identical to itself. + if (this == &other) { + return true; + } + + // Identical instruction must have the same opcode, shape, and identical + // operands. + if (opcode() != other.opcode()) { + return false; + } + if (!(layout_sensitive ? ShapeUtil::Equal(shape(), other.shape()) + : ShapeUtil::Compatible(shape(), other.shape()))) { + return false; + } + if (operands().size() != other.operands().size()) { + return false; + } + + // Use an explicit loop rather than ContainerEquals, because copying around + // std::functions may be too expensive in some cases. + for (size_t i = 0; i < operands().size(); ++i) { + if (!eq_operands(operand(i), other.operand(i))) { + return false; + } + } + + if (backend_config_ != other.backend_config_) { + return false; + } + + return IdenticalSlowPath(other, eq_computations); + } + + // Returns whether the instruction has a constant operand. + bool HasConstantOperand() const; + + // Replaces the use of this instruction in "user" with "new_producer". Note + // that there might be multiple uses of this instruction in "user"; all will + // be replaced. + // + // If user is a fusion instruction, this function will remove any duplicated + // operands of it which could be created due to this replacement. + Status ReplaceUseWith(HloInstruction* user, HloInstruction* new_producer); + + // Replaces the specified operand with new_operand. + // + // This function does NOT remove duplicated operands even if this instruction + // is a fusion, so that the existing operand numbers do not change. + Status ReplaceOperandWith(int64 operand_no, HloInstruction* new_operand); + + // Replaces all uses of this instruction with the new producer. If + // new_producer is a user of this instruction then new_producer remains a use + // of this instruction to avoid introducing cycles into the graph. + // + // If this instruction is the root of its computation, sets the computation's + // root to new_producer. + // + // If a user is a fusion instruction, this function will remove any duplicated + // operands of it which could be created due to this replacement. + Status ReplaceAllUsesWith(HloInstruction* new_producer); + + // Performs a postorder DFS visit using this node as the root. If + // call_finish_visit is true, then DfsHloVisitor::FinishVisit is called when + // complete. If ignore_control_predecessors is true, instructions only + // reachable via control dependencies will not be visited, and the postorder + // will not take control dependencies into account. It is as if the control + // dependencies didn't exist in the graph at all. + template + Status Accept(DfsHloVisitorBase* visitor, + bool call_finish_visit = true, + bool ignore_control_predecessors = false); + Status Accept(ConstDfsHloVisitor* visitor, bool call_finish_visit = true, + bool ignore_control_predecessors = false) const { + return const_cast(this)->Accept( + visitor, call_finish_visit, ignore_control_predecessors); + } + + // Same as Accept() above, but the order of operand and control predecessor + // visitation is determined by the given operand order; if compare(A, B) == + // true, A is visited before B. + using CompareFunction = + std::function; + Status AcceptWithOperandOrder(DfsHloVisitor* visitor, + const CompareFunction& operand_order, + bool call_finish_visit = true); + + // Performs a postorder DFS visit using this node as the root. Calls the given + // visitor function at each instruction. + Status Accept(const std::function& visitor_func); + Status Accept( + const std::function& visitor_func) const; + + // Visits all instructions rooted at this instruction using the given visitor + // in the given order. 'order' must contain at least the set of instructions + // rooted at this node (ie, those accessible from a DFS traversal from this + // instruction). Instructions contained in 'order' which are not in the set of + // instructions rooted at this node are ignored. 'order' must also be a valid + // topological sort of these instructions (defs appear before uses) though + // need not be a DFS post-order. + Status AcceptOrdered(DfsHloVisitor* visitor, + const std::vector& order); + + // Visit this instruction and only this instruction with the given visitor. + template + Status Visit(DfsHloVisitorBase* visitor); + + // Returns the first non-GetTupleElement ancestor instruction of 'hlo'. + // If the first non-GTE ancestor is tuple-shaped, populates 'index' with the + // (possibly nested) tuple indices used on the path from ancestor to 'hlo'. + std::pair LatestNonGteAncestorAndIndex() + const; + + std::pair LatestNonGteAncestorAndIndex() { + auto rv = + const_cast(this)->LatestNonGteAncestorAndIndex(); + return {const_cast(rv.first), rv.second}; + } + + // Same as LatestNonGteAncestorAndIndex, but just returns the HloInstruction. + const HloInstruction* LatestNonGteAncestor() const; + + HloInstruction* LatestNonGteAncestor() { + return const_cast( + const_cast(this)->LatestNonGteAncestor()); + } + + // Gets/sets the to_apply HloComputation for Call, Map, Reduce, etc. + // The setter should only be called by HloModule or HloComputation methods. + // + // Precondition: The instruction has a valid to_apply_ field. + HloComputation* to_apply() const; + void set_to_apply(HloComputation* to_apply); + + // Gets/sets the while_condition or while_body HloComputation for While. The + // setters should only be called by HloModule or HloComputation methods. + // + // Precondition: The instruction is a While instruction. + HloComputation* while_condition() const; + HloComputation* while_body() const; + void set_while_condition(HloComputation* while_condition); + void set_while_body(HloComputation* while_body); + + // Gets/sets the true and false HloComputation for Conditional. The setters + // should only be called by HloModule or HloComputation methods. + // + // Precondition: The instruction is a Conditional instruction. + HloComputation* true_computation() const; + HloComputation* false_computation() const; + void set_true_computation(HloComputation* true_computation); + void set_false_computation(HloComputation* false_computation); + + // Returns a string for the signature of this instruction if considered as a + // function, e.g. the signature of an F32 add is (F32, F32) -> F32. + string SignatureString() const; + + // Returns a debugging string that represents this instruction. + // + // (We express the default options using an overload rather than a default + // param because gdb ignores default params, but does resolve overloads.) + // + // TODO(b/73348663): Make ToString() adaptive to the size of the string by + // default, backing off on providing full information for very large strings, + // or provide a different name for a ToString-like function that does that. + string ToString() const { return ToString(HloPrintOptions()); } + string ToString(const HloPrintOptions& options) const; + + // Components of the ToString() representation: + + // Returns a string representation of the operand list. + string OperandsToString(const HloPrintOptions& options) const; + + // Returns string representation of op-specific attributes. + std::vector ExtraAttributesToString( + const HloPrintOptions& options) const; + + // As ToString, but returns a shorter string. + string ToShortString() const; + + // Returns a serialized representation of this instruction. + virtual HloInstructionProto ToProto() const; + + // Returns a category for the HLO. This could be something like "convolution" + // or "elementwise". + virtual string ToCategory() const; + + // Returns a logging instruction, if the output of this instruction is logged. + // + // Postcondition: retval == nullptr || retval->opcode() == HloOpcode::kTrace + HloInstruction* tracing() const; + void set_tracing(HloInstruction* trace_instruction); + + // Returns true if this instruction is fused, ie contained within a fusion + // instruction. + bool IsFused() const; + + // Returns true if this instruction can be legally fused into a fusion + // instruction. + bool IsFusible() const; + + // Returns the sharding applied to this operator. + // REQUIRES: has_sharding() is true. + const HloSharding& sharding() const { + CHECK(has_sharding()); + return *sharding_; + } + std::shared_ptr sharding_ptr() const { return sharding_; } + + // Returns the sharding applied to this operator, or default_ if none exists. + const HloSharding& sharding_or_default(const HloSharding& default_) const { + return sharding_ ? *sharding_ : default_; + } + // Returns the sharding unique device, if any. + absl::optional sharding_unique_device() const { + if (sharding_ == nullptr) { + return absl::optional(); + } + return sharding_->UniqueDevice(); + } + // Sets the sharding of this operator. Should only be called by HloModule or + // HloComputation methods. + void set_sharding(const HloSharding& sharding) { + sharding_ = std::make_shared(sharding); + } + void set_sharding(std::shared_ptr sharding) { + sharding_ = std::move(sharding); + } + void set_single_sharding(const HloSharding& sharding); + // Sets a sharding that assigns the current instruction to device. + void set_device_sharding(int64 device) { + set_single_sharding(HloSharding::AssignDevice(device)); + } + // Remove any sharding from this operator. + void clear_sharding() { sharding_ = nullptr; } + // Return true if this operator has a sharding assigned. + bool has_sharding() const { return sharding_ != nullptr; } + // Checks whether the instruction has compatible sharding with the other + // instruction. + bool has_compatible_sharding(const HloInstruction* other) const { + if (!has_sharding()) { + return !other->has_sharding(); + } + return other->has_sharding() ? sharding() == other->sharding() : false; + } + + // When creating a new instruction which either replaces, or shifts up (kCopy + // insertion case), another instruction, we need to make sure the certain + // properties of the new instruction are copied into the derived one. As of + // today, the metadata and sharding will be propagated to the derived + // instruction. + void SetupDerivedInstruction(HloInstruction* derived_instruction) const; + + // Clones the HLO instruction. The clone will have the same opcode, shape, and + // operands. After creation the clone has no uses. "this" (the instruction + // cloned from) is not changed. Suffix is the string to append to the name of + // the instruction to form the name of the cloned instruction. + // Ignores the control predecessors and successors of this HLO instruction. + std::unique_ptr Clone( + const string& suffix = "clone", HloCloneContext* context = nullptr) const; + + // Clones the HLO instruction as above but with new shape and operands. + std::unique_ptr CloneWithNewOperands( + const Shape& shape, absl::Span new_operands, + HloCloneContext* context = nullptr) const; + + // Returns the computations this instruction directly calls (if any). + const std::vector& called_computations() const { + return called_computations_; + } + + // Replaces all called computations based on a map function. This is needed + // when we clone hlo_computations and want to let the instructions to point + // to the newly cloned nodes. + void ReplaceCalledComputations( + std::function map_function) { + for (int64 i = 0; i < called_computations_.size(); ++i) { + called_computations_[i] = map_function(called_computations_[i]); + } + } + + // Clears out the called computations. + // + // This is, in particular, necessary when inlining function bodies into their + // caller. If there were side-effecting operations in the called computations, + // the call itself is considered side-effecting and thus cannot be removed. By + // clearing out the computations, we reflect the fact that all side-effecting + // properties have been reflected in the caller, and make the call HLO + // removable. + void ClearCalledComputations() { called_computations_.clear(); } + + // Returns true if this instruction performs an elementwise operation on + // `operand_idx`-th operand. An instruction is elementwise on an operand iff, + // after performing necessary implicit broadcast + // (cs/IrArray::EmitArrayElementAddress), to compute the output at index + // {i_0,i_1,...,i_n}, the only element required from the operand (if any) is + // the element at {i_0,i_1,...,i_n}. + // + // Note on performance: when this instruction is kFusion, this method, in the + // worst case, scans all fused instructions. We could speed this up by + // caching. + bool IsElementwiseOnOperand(int64 operand_idx) const; + + // Returns true if this instruction is elementwise on all its operands. + bool IsElementwise() const; + + // Returns true if this is an cross module all-reduce instrucion. + bool IsCrossModuleAllReduce() const; + + // Returns true if this elementwise instruction implicitly broadcasts operand + // `operand_idx`. + // + // Precondition: this instruction should be an elementwise operation. + bool ImplicitlyBroadcastsOperand(int64 operand_idx) const; + + // Returns true if this instruction is binary and elementwise. + bool IsElementwiseBinary() const; + + // Returns whether this instruction may reuse elements of its `i`th operand. + bool ReusesOperandElements(int64 i) const { + return OperandElementUse(i) == UseKind::kReuse; + } + + // Returns the indices that the given operand appear in the operand list of + // this instruction. Note that an instruction can use the same operand + // multiple times. + std::vector OperandIndices(const HloInstruction* operand) const; + + // Convenience helper for ShapeUtil::InsertedOrDeleted1SizedDimensions. If + // this reshape merely inserts or deletes 1-sized dimensions, return the input + // indices of the deleted dimensions and the output indices of the inserted + // dimensions. + // + // Precondition: this op must be a reshape. + std::tuple, std::vector> + ReshapeMerelyInsertsOrDeletes1SizedDimensions() const; + + // Gets the string identifier for this instruction. + const string& name() const { return name_; } + + // Sets the string identifier for this instruction. Name will be sanitized to + // match the regexp "[a-zA-Z_][a-zA-Z0-9_.-]*". + void SetAndSanitizeName(const string& name) { + name_ = NameUniquer::GetSanitizedName(name); + } + + // Use the given NameUniquer to select a unique name for the instruction based + // on the instruction's existing name. + void UniquifyName(NameUniquer* name_uniquer); + + // Set the unique id for this instruction to "id" + void SetUniqueId(int id) { + CHECK_EQ(unique_id_, -1); // Should not be assigned already + CHECK_GE(id, 0); + unique_id_ = id; + } + + // Return the unique ID assigned to this node via SetUniqueId (or -1 + // if no id has been assigned yet). + int unique_id() const { return unique_id_; } + + // Returns the backend-specific configuration for how a backend should compile + // this HLO. The meaning of the field is backend specific. Not for use before + // or during general HLO optimization, since HLO optimizations do not preserve + // this field and they cannot interpret it due to its meaning being backend + // specific. + // + // ConfigProto should be a protobuf Message type. + template + StatusOr backend_config() const { + ConfigProto proto; + TF_RETURN_IF_ERROR(GetBackendConfigInternal(&proto)); + return std::move(proto); + } + Status set_backend_config(const tensorflow::protobuf::Message& proto); + + // Getter/setter for raw JSON-encoded backend config. Prefer the + // functions above that deal in proto Messages where possible. + const string& raw_backend_config_string() const { return backend_config_; } + void set_raw_backend_config_string(string config_str) { + backend_config_ = std::move(config_str); + } + + // Returns a string representation of a proto in the format used by + // raw_backend_config_string. + // + // This is morally equivalent to: + // + // HloInstruction instr; + // TF_RETURN_IF_ERROR(instr.set_backend_config(proto)); + // return instr.raw_backend_config_string(); + // + static StatusOr BackendConfigToRawString( + const tensorflow::protobuf::Message& proto); + + // Returns the information used to tell the implementation information about + // what sort of precision is requested. The meaning of the field is backend + // specific. At the moment, it is only supported for kConvolution and kDot. + // Transformations on one kDot or kConvolution to another will preserve this + // information. Transformations to other HLOs will not preserve this + // information but it is presumed that the alternate lowering is strictly + // superior. + // Precondition: opcode must be kConvolution or kDot. + const PrecisionConfig& precision_config() const; + + // Sets the debug metadata for this instruction. + void set_metadata(const OpMetadata& metadata) { metadata_ = metadata; } + const OpMetadata& metadata() const { return metadata_; } + + // Set/get the computation containing this instruction. set_parent should only + // be called by HloComputation methods which add/remove instructions to + // computations. + void set_parent(HloComputation* computation) { parent_ = computation; } + const HloComputation* parent() const { return parent_; } + HloComputation* parent() { return parent_; } + + // Returns the module for this instruction. + HloModule* GetModule() const; + + // Returns whether we could assign input and output layouts to this + // instruction to make it a bitcast. + bool CouldBeBitcast() const; + + // Get/Set the number of partitions per outer dimension (in order, starting + // with outer-most dimension first). Currently used by the parallel cpu + // backend to partition HLOs into parallel tasks. + // + // TODO(b/62783254) Replace these methods with a more general way to + // annotate HLOs with backend-specific information. + const std::vector& outer_dimension_partitions() const { + return outer_dimension_partitions_; + } + void set_outer_dimension_partitions( + const std::vector& outer_dimension_partitions); + + // Old methods kept for smooth subclassing transition BEGIN. + // TODO(b/80131774): Remove this code. + + // Delegates to HloBatchNormInstruction::feature_index. + int64 feature_index() const; + + // Delegates to HloBatchNormInstruction::epsilon. + float epsilon() const; + + // Delegates to HloFftInstruction::fft_type. + FftType fft_type() const; + + // Delegates to HloFftInstruction::fft_length. + const std::vector& fft_length() const; + + // Delegates to HloSendRecvInstruction::channel_id. + int64 channel_id() const; + + // Returns the dimension sizes or numbers associated with this instruction. + virtual const std::vector& dimensions() const { + LOG(FATAL) << "Unimplemented method."; + } + virtual int64 dimensions(int64 index) const { + LOG(FATAL) << "Unimplemented method."; + } + + // Delegates to HloConcatenateInstruction::concatenate_dimension. + int64 concatenate_dimension() const; + + // Returns whether this instruction does a rank-2 transposition. + bool IsRank2Transpose() const; + + // Delegates to HloSliceInstruction::slice_start. + int64 slice_starts(int64 dimension) const; + const std::vector& slice_starts() const; + + // Delegates to HloSliceInstruction::slice_limits. + int64 slice_limits(int64 dimension) const; + const std::vector& slice_limits() const; + + // Delegates to HloSliceInstruction::slice_strides. + int64 slice_strides(int64 dimension) const; + const std::vector& slice_strides() const; + + // Delegates to HloSliceInstruction::IsInPlaceSlice. + bool IsInPlaceSlice() const; + + // Returns the literal associated with this instruction. + const Literal& literal() const; + + // Returns whether the instruction is a constant. + bool IsConstant() const; + + // Delegate to HloConstantInstruction::RelayoutConstant. + void RelayoutConstant(const Layout& new_layout, + const ShapeIndex& shape_index = {}); + + // Delegates to HloTraceInstruction::TracingTag. + string TracingTag() const; + + // Delegates to HloFusionInstruction::AddFusionOperand. + HloInstruction* AddFusionOperand(HloInstruction* new_operand); + + // Delegates to HloFusionInstruction::MergeFusionInstruction. + void MergeFusionInstruction(HloInstruction* instruction_to_merge); + + // Delegates to HloFusionInstruction::MergeFusionInstructionIntoMultiOutput. + void MergeFusionInstructionIntoMultiOutput( + HloInstruction* instruction_to_merge); + + // Delegates to HloFusionInstruction::FuseInstruction. + HloInstruction* FuseInstruction(HloInstruction* instruction_to_fuse); + + // Delegates to HloFusionInstruction::FuseInstructionIntoMultiOutput. + HloInstruction* FuseInstructionIntoMultiOutput( + HloInstruction* instruction_to_fuse); + + // Delegates to HloFusionInstruction::fused_instruction. + HloComputation* fused_instructions_computation() const; + + // Delegates to HloFusionInstruction::fused_expression_root. + HloInstruction* fused_expression_root() const; + + // Delegates to HloFusionInstruction::fused_instructions. + const tensorflow::gtl::iterator_range>::const_iterator>> + fused_instructions() const; + + const tensorflow::gtl::iterator_range< + UnwrappingIterator>::iterator>> + fused_instructions(); + + // Delegates to HloFusionInstruction::fused_instruction_count. + int64 fused_instruction_count() const; + + // Delegates to HloFusionInstruction::fused_parameter. + HloInstruction* fused_parameter(int64 parameter_number) const; + + // Delegates to HloFusionInstruction::fused_parameters. + const std::vector& fused_parameters() const; + + // Returns true if this instruction is a fusion instruction that generates + // multiple outputs. + const bool IsMultiOutputFusion() const; + + // Delegates to HloFusionInstruction::fusion_kind. + FusionKind fusion_kind() const; + + // Delegates to HloFusionInstruction::set_fusion_kind. + void set_fusion_kind(FusionKind kind); + + // Delegates to HloRngInstruction::random_distribution. + RandomDistribution random_distribution() const; + + // Delegates to HloParameterInstruction::parameter_number. + int64 parameter_number() const; + + // Delegates to HloGetTupleElementInstruction::tuple_index. + int64 tuple_index() const; + + // Delegates to HloReducePrecisionInstruction::exponent_bits. + int32 exponent_bits() const; + + // Delegates to HloReducePrecisionInstruction::mantissa_bits. + int32 mantissa_bits() const; + + // Delegates to HloInfeedInstruction::infeed_config. + string infeed_config() const; + + // Delegates to HloInfeedInstruction::set_infeed_config. + void set_infeed_config(const string& config); + + // Returns the config for the Outfeed instruction. + const string& outfeed_config() const; + + // Returns the shape for the Outfeed instruction. + const Shape& outfeed_shape() const; + + // Delegates to HloCollectiveInstruction::replica_groups. + const std::vector& replica_groups() const; + + // Delegates to HloCollectivePermuteInstruction::source_target_pairs. + const std::vector>& source_target_pairs() const; + + // Delegates to HloAllReduceInstruction::cross_replica_sum_barrier. + string cross_replica_sum_barrier() const; + void set_cross_replica_sum_barrier(const string& barrier); + + // Delegates to HloAllReduceInstruction::all_reduce_id. + absl::optional all_reduce_id() const; + + // Returns data on the window in a windowed operation such as + // convolution. + virtual const Window& window() const { + LOG(FATAL) << "Unimplemented method."; + } + + // Sets the window data in a windowed operation such as convolution. + virtual void set_window(const Window& window) { + LOG(FATAL) << "Unimplemented method."; + } + + // Returns data on the dimension numbers used for a convolution operation, + // which may be a kConvolution instruction or a kCustomCall that implements a + // convolution. + const ConvolutionDimensionNumbers& convolution_dimension_numbers() const; + + // Sets the convolution dimension numbers on this instruction. In general you + // shouldn't need to call this; instead, specify the convolution dimension + // numbers when you create the instruction. + void set_convolution_dimension_numbers( + const ConvolutionDimensionNumbers& dnums); + + // The number of feature groups. Must be a divisor of the input feature + // dimension and output feature dimension. + int64 feature_group_count() const; + + void set_feature_group_count(int64 feature_group_count); + + // Delegates to HloSelectAndScatterInstruction::select. + HloComputation* select() const; + + // Delegates to HloSelectAndScatterInstruction::scatter. + HloComputation* scatter() const; + + // Delegates to HloSelectAndScatterInstruction::set_select. + void set_select(HloComputation* computation); + + // Delegates to HloSelectAndScatterInstruction::set_scatter. + void set_scatter(HloComputation* computation); + + // Delegates to HloCustomCallInstruction::custom_call_target. + const string& custom_call_target() const; + + // Delegates to HloPadInstruction::padding_config. + const PaddingConfig& padding_config() const; + + // Delegates to HloDynamicSliceInstruction::slice_sizes. + int64 slice_sizes(int64 dimension) const; + + // Delegates to HloDynamicSliceInstruction::dynamic_slice_sizes. + const std::vector& dynamic_slice_sizes() const; + + // Delegates to HloGatherInstruction::gather_dimension_numbers. + const GatherDimensionNumbers& gather_dimension_numbers() const; + // Delegates to HloGatherInstruction::gather_slice_sizes. + absl::Span gather_slice_sizes() const; + + // Delegates to HloScatterInstruction::scatter_dimension_numbers(). + const ScatterDimensionNumbers& scatter_dimension_numbers() const; + + // Delegates to HloDotInstruction::dot_dimension_numbers(). + const DotDimensionNumbers& dot_dimension_numbers() const; + + // Delegates to HloDomainInstruction::operand_side_metadata(). + const DomainMetadata& operand_side_metadata() const; + + // Delegates to HloDomainInstruction::user_side_metadata(). + const DomainMetadata& user_side_metadata() const; + + // Old methods kept for smooth subclassing transition END. + + protected: + enum class UseKind { kNoUse, kReuse, kUsePermutingElements, kUse }; + // Helper class for computing OperandElementUse for kFusion. + class FusionReusesParamElements; + + // Internal constructor for a given opcode/shape, other fields must be filled + // by factory methods. + HloInstruction(HloOpcode opcode, const Shape& shape); + + // Appends operand to the list of operands and adds this instruction as a user + // of the operand. + void AppendOperand(HloInstruction* operand); + + void RemoveOperandAt(int index) { + operands_.erase(operands_.begin() + index); + } + + // Removes a list of operands with the given indices in ascending order. + void RemoveOperandsAtAscendingIndices( + absl::Span ascending_indices); + + void AppendComputation(HloComputation* computation) { + called_computations_.push_back(computation); + } + + void DetachFrom(HloInstruction* usee) { usee->RemoveUser(this); } + + void set_called_computation(int index, HloComputation* computation) { + called_computations_[index] = computation; + } + // Indices of computations in called_computations_ for instructions which call + // multiple computations. + enum { + // kWhile computations. + kBodyComputationIndex = 0, + kConditionComputationIndex = 1, + + // kSelectAndScatter computations. + kSelectComputationIndex = 0, + kScatterComputationIndex = 1, + + // kConditional computations. + kTrueComputationIndex = 0, + kFalseComputationIndex = 1, + }; + + private: + // Implementation for non-common logic of CloneWithNewOperands. + virtual std::unique_ptr CloneWithNewOperandsImpl( + const Shape& shape, absl::Span new_operands, + HloCloneContext* context) const { + // TODO(b/80131774): This should be pure virtual. + LOG(FATAL) << "Unimplemented method."; + } + + // Implementation for non-common logic of ExtraAttributesToString. + virtual std::vector ExtraAttributesToStringImpl( + const HloPrintOptions& options) const { + return {}; + } + + // Implementation for IsElementwise if operand_idx is nullopt and for + // IsElementwiseOnOperand if otherwise. + // + // NOTE: For all instructions other than kFusion, being elementwise on one of + // the operands is equivalent to being elementwise on all the operands. + virtual bool IsElementwiseImpl( + const absl::optional& operand_idx) const; + // Prints an instruction to a string. + // + // The canonical string representation needs to name operands and instruction + // names in a consistent way. This is implemented through the + // canonical_name_map. + string ToStringWithCanonicalNameMap( + const HloPrintOptions& options, + CanonicalNameMap* canonical_name_map) const; + + // Prints an operand to a string. + virtual string OperandsToStringWithCanonicalNameMap( + const HloPrintOptions& options, + CanonicalNameMap* canonical_name_map) const; + + // Allow HloInstruction to access the ToStringWithCanonicalNameMap() and + // OperandsToStringWithCanonicalNameMap() functions. + friend class HloComputation; + + // See comments on Identical(). + virtual bool IdenticalSlowPath( + const HloInstruction& other, + const std::function& + eq_computations) const; + + // Creates an n-ary elementwise operation. + static std::unique_ptr CreateNary( + const Shape& shape, HloOpcode opcode, + absl::Span operands); + + // Adds a user for this instruction. + void AddUser(HloInstruction* user); + + // Removes a user for this instruction. + void RemoveUser(HloInstruction* user); + + // Returns how this instruction uses elements of its `i`th operand. + UseKind OperandElementUse(int64 i) const; + + // Helper for implementing backend_config(). Parses backend_config_ into the + // given proto. + Status GetBackendConfigInternal(tensorflow::protobuf::Message* proto) const; + + int unique_id_; // Unique to this HloInstruction within a HloModule + + // Opcode for this instruction. + HloOpcode opcode_; + + // Instruction operands. + InstructionVector operands_; + + // The set of control predecessors of this instruction. + // Note that the order of the instructions in the vector influences the order + // computed in HloComputation::ComputeInstructionPostOrder, which may + // influence the result of the compilation by changing the scheduling. We are + // not sure if it matters. + std::vector control_predecessors_; + + // The users of this instruction. Users are HLOs where this instruction is an + // operand. The vector users_ and the set user_set_ contain identical + // members. The set enables fast membership testing and the vector enables + // fast, stable iteration. + std::vector users_; + std::unordered_set user_set_; + + // The set of control successors of this instruction. + std::vector control_successors_; + + // The computation in which this instruction is contained. + HloComputation* parent_ = nullptr; + + // Result shape of this instruction. + Shape shape_; + + // The sharding, if one exists. + // Uses std::shared_ptr to allow reuse of the same sharding object between + // HloInstructions and other components as HloSharding can be very large for + // many element tuples. + std::shared_ptr sharding_; + + // Computations called by this instruction. + std::vector called_computations_; + + // A trace instruction that consumes this instruction. + // + // Invariant: if trace_instruction_ != nullptr, trace_instruction has this as + // an operand. + HloInstruction* trace_instruction_ = nullptr; + + // The backend-specific configuration for how a backend should compile this + // HLO. See the documentation on backend_config(). + string backend_config_; + + // String identifier for instruction. + string name_; + + // Metadata for debugging. + OpMetadata metadata_; + + // The number of partitions per outer dimension (listed in order from + // outer-most dimension first). + std::vector outer_dimension_partitions_; + + TF_DISALLOW_COPY_AND_ASSIGN(HloInstruction); +}; + +string ToString(HloInstruction::FusionKind kind); +StatusOr StringToFusionKind( + const string& kind_name); + +// Custom (de)stringification functions for protos that live inside +// HloInstruction. +string PaddingConfigToString(const PaddingConfig& padding); +string OpMetadataToString(const OpMetadata& metadata); +string RandomDistributionToString(const RandomDistribution& distribution); +string PrecisionToString(const PrecisionConfig::Precision& precision); +string ConvolutionDimensionNumbersToString( + const ConvolutionDimensionNumbers& dnums); + +StatusOr StringToRandomDistribution(const string& name); +StatusOr StringToPrecision(const string& name); + +std::ostream& operator<<(std::ostream& os, HloInstruction::FusionKind kind); + +// Map classes that guarantee a deterministic iteration order when the key is +// an HloInstruction* or a const HloInstruction*. +// To make the iteration order over the map deterministic, the comparator +// should not be using the pointer values, but rather an intrinsic property of +// the hlo. Exception: null pointer values compare less than non-null. +struct HloPtrComparator { + bool operator()(const HloInstruction* const& lhs, + const HloInstruction* const& rhs) const; +}; + +template +using HloInstructionMap = std::map; + +template +using ConstHloInstructionMap = + std::map; + +using HloInstructionSet = std::set; +using ConstHloInstructionSet = + std::set; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_HLO_INSTRUCTION_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/hlo_instructions.h b/linux/include/tensorflow/compiler/xla/service/hlo_instructions.h new file mode 100644 index 000000000..e2aa50c8d --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/hlo_instructions.h @@ -0,0 +1,1369 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// All HloInstruction subclasses are put in this file. + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_HLO_INSTRUCTIONS_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_HLO_INSTRUCTIONS_H_ + +#include "absl/memory/memory.h" +#include "tensorflow/compiler/xla/service/hlo_instruction.h" + +namespace xla { + +class HloBatchNormInstruction : public HloInstruction { + public: + // Returns feature_index field associated with the instruction. The index + // represents the index of the feature dimension. + int64 feature_index() const { return feature_index_; } + + // Returns a epsilon value associated with the instruction. The is a small + // number added to the variance to avoid divide-by-zero error. + float epsilon() const { return epsilon_; } + + // Returns a serialized representation of this instruction. + HloInstructionProto ToProto() const override; + + protected: + explicit HloBatchNormInstruction(HloOpcode opcode, const Shape& shape, + HloInstruction* operand, + HloInstruction* scale, float epsilon, + int64 feature_index); + + private: + std::vector ExtraAttributesToStringImpl( + const HloPrintOptions& options) const override; + bool IdenticalSlowPath( + const HloInstruction& other, + const std::function& + eq_computations) const override; + // A small float number added to the variance to avoid divide-by-zero error. + float epsilon_ = 0.0f; + + // An integer value representing the index of the feature dimension. + int64 feature_index_ = -1; +}; + +class HloBatchNormTrainingInstruction : public HloBatchNormInstruction { + public: + explicit HloBatchNormTrainingInstruction(const Shape& shape, + HloInstruction* operand, + HloInstruction* scale, + HloInstruction* offset, + float epsilon, int64 feature_index); + + private: + // Implementation for non-common logic of CloneWithNewOperands. + std::unique_ptr CloneWithNewOperandsImpl( + const Shape& shape, absl::Span new_operands, + HloCloneContext* context) const override; +}; + +class HloBatchNormInferenceInstruction : public HloBatchNormInstruction { + public: + explicit HloBatchNormInferenceInstruction( + const Shape& shape, HloInstruction* operand, HloInstruction* scale, + HloInstruction* offset, HloInstruction* mean, HloInstruction* variance, + float epsilon, int64 feature_index); + + private: + // Implementation for non-common logic of CloneWithNewOperands. + std::unique_ptr CloneWithNewOperandsImpl( + const Shape& shape, absl::Span new_operands, + HloCloneContext* context) const override; +}; + +class HloBatchNormGradInstruction : public HloBatchNormInstruction { + public: + explicit HloBatchNormGradInstruction( + const Shape& shape, HloInstruction* operand, HloInstruction* scale, + HloInstruction* mean, HloInstruction* variance, + HloInstruction* grad_output, float epsilon, int64 feature_index); + + private: + // Implementation for non-common logic of CloneWithNewOperands. + std::unique_ptr CloneWithNewOperandsImpl( + const Shape& shape, absl::Span new_operands, + HloCloneContext* context) const override; +}; + +class HloFftInstruction : public HloInstruction { + public: + explicit HloFftInstruction(const Shape& shape, HloInstruction* operand, + FftType fft_type, + absl::Span fft_length); + FftType fft_type() const { return fft_type_; } + + const std::vector& fft_length() const { return fft_length_; } + + // Returns a serialized representation of this instruction. + HloInstructionProto ToProto() const override; + + private: + std::vector ExtraAttributesToStringImpl( + const HloPrintOptions& options) const override; + bool IdenticalSlowPath( + const HloInstruction& other, + const std::function& + eq_computations) const override; + + // Implementation for non-common logic of CloneWithNewOperands. + std::unique_ptr CloneWithNewOperandsImpl( + const Shape& shape, absl::Span new_operands, + HloCloneContext* context) const override; + + // Describes FFT type for an FFT instruction. + FftType fft_type_ = FftType::FFT; + + // Indicates the FFT length for an FFT instruction. + std::vector fft_length_; +}; + +class HloSendRecvInstruction : public HloInstruction { + public: + // Returns the channel id associated with the instruction. The id is + // shared between each Send/Recv pair and is globally unique to identify each + // channel. + int64 channel_id() const { return channel_id_; } + + // Returns whether this send/recv instruction sends data to/from the host. + bool is_host_transfer() const { return is_host_transfer_; } + + // Returns a serialized representation of this instruction. + HloInstructionProto ToProto() const override; + + protected: + explicit HloSendRecvInstruction(HloOpcode opcode, const Shape& shape, + int64 channel_id, bool is_host_transfer); + + private: + std::vector ExtraAttributesToStringImpl( + const HloPrintOptions& options) const override; + bool IdenticalSlowPath( + const HloInstruction& other, + const std::function& + eq_computations) const override; + // Represents a unique identifier for each Send/Recv instruction pair. + int64 channel_id_; + + // Whether this send/recv instruction sends data to/from the host. + bool is_host_transfer_; +}; + +class HloSendInstruction : public HloSendRecvInstruction { + public: + explicit HloSendInstruction(HloInstruction* operand, HloInstruction* token, + int64 channel_id, bool is_host_transfer); + + private: + // Implementation for non-common logic of CloneWithNewOperands. + std::unique_ptr CloneWithNewOperandsImpl( + const Shape& shape, absl::Span new_operands, + HloCloneContext* context) const override; +}; + +class HloSendDoneInstruction : public HloSendRecvInstruction { + public: + explicit HloSendDoneInstruction(HloSendInstruction* operand, + bool is_host_transfer); + + private: + // Implementation for non-common logic of CloneWithNewOperands. + std::unique_ptr CloneWithNewOperandsImpl( + const Shape& shape, absl::Span new_operands, + HloCloneContext* context) const override; +}; + +class HloRecvInstruction : public HloSendRecvInstruction { + public: + explicit HloRecvInstruction(const Shape& shape, HloInstruction* token, + int64 channel_id, bool is_host_transfer); + + private: + // Implementation for non-common logic of CloneWithNewOperands. + std::unique_ptr CloneWithNewOperandsImpl( + const Shape& shape, absl::Span new_operands, + HloCloneContext* context) const override; +}; + +class HloRecvDoneInstruction : public HloSendRecvInstruction { + public: + explicit HloRecvDoneInstruction(HloRecvInstruction* operand, + bool is_host_transfer); + + private: + // Implementation for non-common logic of CloneWithNewOperands. + std::unique_ptr CloneWithNewOperandsImpl( + const Shape& shape, absl::Span new_operands, + HloCloneContext* context) const override; +}; + +class HloCollectiveInstruction : public HloInstruction { + public: + const std::vector& replica_groups() const { + return replica_groups_; + } + + protected: + explicit HloCollectiveInstruction( + HloOpcode opcode, const Shape& shape, + absl::Span operands, + const std::vector& replica_groups); + + HloInstructionProto ToProto() const override; + + std::vector ExtraAttributesToStringImpl( + const HloPrintOptions& options) const override; + bool IdenticalSlowPath( + const HloInstruction& other, + const std::function& + eq_computations) const override; + + std::vector replica_groups_; +}; + +class HloAllReduceInstruction : public HloCollectiveInstruction { + public: + explicit HloAllReduceInstruction( + const Shape& shape, absl::Span operands, + HloComputation* reduce_computation, + const std::vector& replica_groups, + absl::string_view barrier, const absl::optional& all_reduce_id); + + // Returns the barrier config used for the CrossReplicaSum implementation of + // each backend. + string cross_replica_sum_barrier() const { + return cross_replica_sum_barrier_; + } + void set_cross_replica_sum_barrier(string barrier) { + cross_replica_sum_barrier_ = barrier; + } + + absl::optional all_reduce_id() const { return all_reduce_id_; } + + // Returns a serialized representation of this instruction. + HloInstructionProto ToProto() const override; + + private: + std::vector ExtraAttributesToStringImpl( + const HloPrintOptions& options) const override; + bool IdenticalSlowPath( + const HloInstruction& other, + const std::function& + eq_computations) const override; + + // Implementation for non-common logic of CloneWithNewOperands. + std::unique_ptr CloneWithNewOperandsImpl( + const Shape& shape, absl::Span new_operands, + HloCloneContext* context) const override; + + // The string representation of the barrier config used for CrossReplicaSum. + string cross_replica_sum_barrier_; + + // For Allreduce nodes from different modules, if they have the same + // all_reduce_id, they will be 'Allreduce'd. If empty, Allreduce will not be + // applied cross modules. + absl::optional all_reduce_id_; +}; + +class HloAllToAllInstruction : public HloCollectiveInstruction { + public: + explicit HloAllToAllInstruction( + const Shape& shape, absl::Span operands, + const std::vector& replica_groups); + + private: + // Implementation for non-common logic of CloneWithNewOperands. + std::unique_ptr CloneWithNewOperandsImpl( + const Shape& shape, absl::Span new_operands, + HloCloneContext* context) const override; +}; + +class HloCollectivePermuteInstruction : public HloInstruction { + public: + explicit HloCollectivePermuteInstruction( + const Shape& shape, HloInstruction* operand, + const std::vector>& source_target_pairs); + + const std::vector>& source_target_pairs() const { + return source_target_pairs_; + } + + // Returns a serialized representation of this instruction. + HloInstructionProto ToProto() const override; + + private: + std::vector ExtraAttributesToStringImpl( + const HloPrintOptions& options) const override; + bool IdenticalSlowPath( + const HloInstruction& other, + const std::function& + eq_computations) const override; + + // Implementation for non-common logic of CloneWithNewOperands. + std::unique_ptr CloneWithNewOperandsImpl( + const Shape& shape, absl::Span new_operands, + HloCloneContext* context) const override; + + const std::vector> source_target_pairs_; +}; + +class HloReverseInstruction : public HloInstruction { + public: + explicit HloReverseInstruction(const Shape& shape, HloInstruction* operand, + absl::Span dimensions); + // Returns the dimension sizes or numbers associated with this instruction. + const std::vector& dimensions() const override { return dimensions_; } + int64 dimensions(int64 index) const override { return dimensions()[index]; } + // Returns a serialized representation of this instruction. + HloInstructionProto ToProto() const override; + + private: + std::vector ExtraAttributesToStringImpl( + const HloPrintOptions& options) const override; + bool IdenticalSlowPath( + const HloInstruction& other, + const std::function& + eq_computations) const override; + // Implementation for non-common logic of CloneWithNewOperands. + std::unique_ptr CloneWithNewOperandsImpl( + const Shape& shape, absl::Span new_operands, + HloCloneContext* context) const override; + + std::vector dimensions_; +}; + +class HloConcatenateInstruction : public HloInstruction { + public: + explicit HloConcatenateInstruction(const Shape& shape, + absl::Span operands, + int64 dimension); + // Returns the dimension sizes or numbers associated with this instruction. + const std::vector& dimensions() const override { return dimensions_; } + int64 dimensions(int64 index) const override { return dimensions()[index]; } + // Accessor for the dimension in which a concatenate HLO should occur. + int64 concatenate_dimension() const { return dimensions(0); } + // Returns a serialized representation of this instruction. + HloInstructionProto ToProto() const override; + + private: + std::vector ExtraAttributesToStringImpl( + const HloPrintOptions& options) const override; + bool IdenticalSlowPath( + const HloInstruction& other, + const std::function& + eq_computations) const override; + // Implementation for non-common logic of CloneWithNewOperands. + std::unique_ptr CloneWithNewOperandsImpl( + const Shape& shape, absl::Span new_operands, + HloCloneContext* context) const override; + + std::vector dimensions_; +}; + +class HloReduceInstruction : public HloInstruction { + public: + explicit HloReduceInstruction(const Shape& shape, + absl::Span args, + absl::Span dimensions_to_reduce, + HloComputation* reduce_computation); + // Returns the dimension sizes or numbers associated with this instruction. + const std::vector& dimensions() const override { return dimensions_; } + int64 dimensions(int64 index) const override { return dimensions()[index]; } + // Returns a serialized representation of this instruction. + HloInstructionProto ToProto() const override; + + // Returns the number of input arrays (and, consequentially, the number of + // init values) this reduce has. + int64 input_count() const { return operand_count() / 2; } + + // Returns the input tensors to be reduced. + absl::Span inputs() const { + return absl::MakeSpan(operands()).subspan(0, input_count()); + } + + // Returns the init values of the reduction. + absl::Span init_values() const { + return absl::MakeSpan(operands()).subspan(input_count(), operand_count()); + } + + private: + std::vector ExtraAttributesToStringImpl( + const HloPrintOptions& options) const override; + bool IdenticalSlowPath( + const HloInstruction& other, + const std::function& + eq_computations) const override; + // Implementation for non-common logic of CloneWithNewOperands. + std::unique_ptr CloneWithNewOperandsImpl( + const Shape& shape, absl::Span new_operands, + HloCloneContext* context) const override; + + std::vector dimensions_; +}; + +class HloSortInstruction : public HloInstruction { + public: + explicit HloSortInstruction(const Shape& shape, int64 dimension, + HloInstruction* keys, + HloInstruction* values = nullptr); + // Returns the dimension sizes or numbers associated with this instruction. + const std::vector& dimensions() const override { return dimensions_; } + int64 dimensions(int64 index) const override { return dimensions()[index]; } + // Returns the sort dimension for this instruction + int64 sort_dimension() { return dimensions(0); } + // Returns a serialized representation of this instruction. + HloInstructionProto ToProto() const override; + + private: + std::vector ExtraAttributesToStringImpl( + const HloPrintOptions& options) const override; + bool IdenticalSlowPath( + const HloInstruction& other, + const std::function& + eq_computations) const override; + // Implementation for non-common logic of CloneWithNewOperands. + std::unique_ptr CloneWithNewOperandsImpl( + const Shape& shape, absl::Span new_operands, + HloCloneContext* context) const override; + + std::vector dimensions_; +}; + +class HloTransposeInstruction : public HloInstruction { + public: + explicit HloTransposeInstruction(const Shape& shape, HloInstruction* operand, + absl::Span dimensions); + // Returns the dimension sizes or numbers associated with this instruction. + const std::vector& dimensions() const override { return dimensions_; } + int64 dimensions(int64 index) const override { return dimensions()[index]; } + // Returns whether this instruction does a rank-2 transposition. + bool IsRank2Transpose() const; + // Returns a serialized representation of this instruction. + HloInstructionProto ToProto() const override; + + private: + std::vector ExtraAttributesToStringImpl( + const HloPrintOptions& options) const override; + bool IdenticalSlowPath( + const HloInstruction& other, + const std::function& + eq_computations) const override; + // Implementation for non-common logic of CloneWithNewOperands. + std::unique_ptr CloneWithNewOperandsImpl( + const Shape& shape, absl::Span new_operands, + HloCloneContext* context) const override; + + std::vector dimensions_; +}; + +class HloBroadcastInstruction : public HloInstruction { + public: + explicit HloBroadcastInstruction(const Shape& shape, HloInstruction* operand, + absl::Span broadcast_dimension); + // Returns the dimension sizes or numbers associated with this instruction. + const std::vector& dimensions() const override { return dimensions_; } + int64 dimensions(int64 index) const override { return dimensions()[index]; } + // Returns a serialized representation of this instruction. + HloInstructionProto ToProto() const override; + + private: + std::vector ExtraAttributesToStringImpl( + const HloPrintOptions& options) const override; + bool IdenticalSlowPath( + const HloInstruction& other, + const std::function& + eq_computations) const override; + // Implementation for non-common logic of CloneWithNewOperands. + std::unique_ptr CloneWithNewOperandsImpl( + const Shape& shape, absl::Span new_operands, + HloCloneContext* context) const override; + + std::vector dimensions_; +}; + +class HloMapInstruction : public HloInstruction { + public: + explicit HloMapInstruction(const Shape& shape, + absl::Span operands, + HloComputation* map_computation); + // Returns the dimension sizes or numbers associated with this instruction. + const std::vector& dimensions() const override { return dimensions_; } + int64 dimensions(int64 index) const override { return dimensions()[index]; } + // Returns a serialized representation of this instruction. + HloInstructionProto ToProto() const override; + + private: + bool IsElementwiseImpl( + const absl::optional& operand_idx) const override; + std::vector ExtraAttributesToStringImpl( + const HloPrintOptions& options) const override; + bool IdenticalSlowPath( + const HloInstruction& other, + const std::function& + eq_computations) const override; + // Implementation for non-common logic of CloneWithNewOperands. + std::unique_ptr CloneWithNewOperandsImpl( + const Shape& shape, absl::Span new_operands, + HloCloneContext* context) const override; + + std::vector dimensions_; +}; + +class HloSliceInstruction : public HloInstruction { + public: + explicit HloSliceInstruction(const Shape& shape, HloInstruction* operand, + absl::Span start_indices, + absl::Span limit_indices, + absl::Span strides); + + HloInstructionProto ToProto() const override; + + // Returns the start index in the given dimension for a slice node. + int64 slice_starts(int64 dimension) const { return slice_starts_[dimension]; } + const std::vector& slice_starts() const { return slice_starts_; } + + // Returns the (exclusive) limit index in the given dimension for a slice + // node. + int64 slice_limits(int64 dimension) const { return slice_limits_[dimension]; } + const std::vector& slice_limits() const { return slice_limits_; } + + // Returns the stride in the given dimension for a slice node. + int64 slice_strides(int64 dimension) const { + return slice_strides_[dimension]; + } + const std::vector& slice_strides() const { return slice_strides_; } + + // Returns the flag that describes whether a slice must be lowered into an + // offset into the original operand. + bool IsInPlaceSlice() const { return is_in_place_slice_; } + + // Sets and returns the flag that describes whether a slice must be lowered + // into an offset into the original operand. + bool SetIsInPlaceSlice(bool value) { + is_in_place_slice_ = value; + return value; + } + + private: + std::vector ExtraAttributesToStringImpl( + const HloPrintOptions& options) const override; + bool IdenticalSlowPath( + const HloInstruction& other, + const std::function& + eq_computations) const override; + // Implementation for non-common logic of CloneWithNewOperands. + std::unique_ptr CloneWithNewOperandsImpl( + const Shape& shape, absl::Span new_operands, + HloCloneContext* context) const override; + + // Describes the [begin, end) index range for a slice. + std::vector slice_starts_; + std::vector slice_limits_; + std::vector slice_strides_; + + // Describes whether the slice can be lowered to an offset into the operand. + bool is_in_place_slice_ = false; +}; + +class HloConstantInstruction : public HloInstruction { + public: + explicit HloConstantInstruction(Literal literal); + // Used when the literal is too large and dropped. + explicit HloConstantInstruction(const Shape& shape); + // Returns the literal associated with this instruction. + const Literal& literal() const { return *literal_; } + // Returns whether there is literal associated with this instruction. + bool HasLiteral() const { return literal_.has_value(); } + // Returns a serialized representation of this instruction. + HloInstructionProto ToProto() const override; + + // Change the layout for an Constant Hlo instruction to match new_layout. For + // tuple shaped constants shape_index is the path to the internal array + // subshape whose layout needs to be changed. + void RelayoutConstant(const Layout& new_layout, + const ShapeIndex& shape_index = {}); + + private: + bool IsElementwiseImpl( + const absl::optional& operand_idx) const override; + bool IdenticalSlowPath( + const HloInstruction& other, + const std::function& + eq_computations) const override; + string OperandsToStringWithCanonicalNameMap( + const HloPrintOptions& options, + CanonicalNameMap* canonical_name_map) const override; + // Implementation for non-common logic of CloneWithNewOperands. + std::unique_ptr CloneWithNewOperandsImpl( + const Shape& shape, absl::Span new_operands, + HloCloneContext* context) const override; + absl::optional literal_; +}; + +class HloTraceInstruction : public HloInstruction { + public: + explicit HloTraceInstruction(const string& tag, HloInstruction* operand); + // Returns a tag to be used in tracing. + string TracingTag() const { return literal_.GetR1U8AsString(); } + // Returns a serialized representation of this instruction. + HloInstructionProto ToProto() const override; + + private: + bool IdenticalSlowPath( + const HloInstruction& other, + const std::function& + eq_computations) const override; + // Implementation for non-common logic of CloneWithNewOperands. + std::unique_ptr CloneWithNewOperandsImpl( + const Shape& shape, absl::Span new_operands, + HloCloneContext* context) const override; + Literal literal_; +}; + +class HloFusionInstruction : public HloInstruction { + public: + explicit HloFusionInstruction(const Shape& shape, FusionKind fusion_kind, + HloInstruction* fused_root); + + explicit HloFusionInstruction(const Shape& shape, FusionKind fusion_kind, + absl::Span operands, + HloComputation* fusion_computation); + + string ToCategory() const override; + // Returns a serialized representation of this instruction. + HloInstructionProto ToProto() const override; + + // Adds a new operand the fusion instruction. + HloInstruction* AddFusionOperand(HloInstruction* new_operand); + + // Merges the fused instructions from 'instruction_to_merge' into the + // fused instruction set of 'this', updating operands as necessary. + // + // Predondition: 'instruction_to_merge' must be an operand of 'this'. + void MergeFusionInstruction(HloFusionInstruction* instruction_to_merge); + + // Merges the fused instructions from instruction_to_merge into the fused + // instruction set of 'this' and generates multioutput fusion instructions. + // All the users of instruction_to_merge will be redirected to 'this' + // instruction. instruction_to_merge will be removed from its parent + // computation. + void MergeFusionInstructionIntoMultiOutput( + HloFusionInstruction* instruction_to_merge); + + // Fuses the given instruction in this fusion instruction. instruction_to_fuse + // is cloned and the clone is placed in the fusion + // instruction. instruction_to_fuse is unchanged. Instruction is cloned rather + // than moved to cleanly handle the case where the instruction has a use + // outside the fusion instruction. Moving such an instruction into a fusion + // instruction would violate the single-result invariant of HLO instructions + // and significantly complicate code generation. + HloInstruction* FuseInstruction(HloInstruction* instruction_to_fuse) { + return FuseInstructionInternal(instruction_to_fuse); + } + + // Fuses the given instruction in this fusion instruction and generate + // multioutput fusion instruction. A clone of the instruction_to_fuse will + // be part of the output of fusion instructions. The users of + // instruction_to_fuse will be redirected to this fusion instructions. + // instruction_to_fuse will be removed from its parent computation. + HloInstruction* FuseInstructionIntoMultiOutput( + HloInstruction* instruction_to_fuse) { + return FuseInstructionInternal(instruction_to_fuse, /* add_output */ true); + } + + // Returns the computation for this fused instruction. + HloComputation* fused_instructions_computation() const; + + // Returns the root instruction of the fused expression contained within this + // fusion instruction. + HloInstruction* fused_expression_root() const; + + // Returns the list of fused instructions inside this fusion instruction. The + // returned type is a range of HloInstruction*s. + const tensorflow::gtl::iterator_range>::const_iterator>> + fused_instructions() const; + + const tensorflow::gtl::iterator_range< + UnwrappingIterator>::iterator>> + fused_instructions(); + + // Gets the number of instructions inside this fusion instruction. + int64 fused_instruction_count() const; + + // Returns the fused parameter instruction in this fusion instruction + // corresponding to the given parameter number. + HloInstruction* fused_parameter(int64 parameter_number) const; + + // Returns the vector of fused parameters inside this fusion instruction. + const std::vector& fused_parameters() const; + + // Returns true if this instruction is a fusion instruction that generates + // multiple outputs. + const bool IsMultiOutputFusion() const { + return fused_expression_root()->opcode() == HloOpcode::kTuple; + } + + FusionKind fusion_kind() const { return fusion_kind_; } + + void set_fusion_kind(FusionKind kind) { fusion_kind_ = kind; } + + // If multiple operands are the same instruction, keeps only one of them. + Status DeduplicateFusionOperands(); + + private: + // Fuses the given instruction into this fusion instruction. When add_output + // is false (which is the default), instruction_to_fuse is cloned and the + // clone is placed in the fusion instruction. instruction_to_fuse is + // unchanged. + // + // When add_output is true, a clone of the instruction_to_fuse will be part + // of the output of fusion instructions. The users of instruction_to_fuse + // will be redirected to this fusion instructions. instruction_to_fuse will + // be removed from its parent computation. + HloInstruction* FuseInstructionInternal(HloInstruction* instruction_to_fuse, + bool add_output = false); + // Clones the given instruction_to_fuse and insert the clone into this fusion + // instruction. If add_output is true, a clone of instruction_to_fuse will + // be in the output of the this fusion instruction (part of the tuple of the + // fusion root). + HloInstruction* CloneAndFuseInternal(HloInstruction* instruction_to_fuse, + bool add_output = false); + + bool IsElementwiseImpl( + const absl::optional& operand_idx) const override; + std::vector ExtraAttributesToStringImpl( + const HloPrintOptions& options) const override; + bool IdenticalSlowPath( + const HloInstruction& other, + const std::function& + eq_computations) const override; + // Implementation for non-common logic of CloneWithNewOperands. + std::unique_ptr CloneWithNewOperandsImpl( + const Shape& shape, absl::Span new_operands, + HloCloneContext* context) const override; + + // The type of the fusion. Used by kFusion only. + FusionKind fusion_kind_; +}; + +class HloRngInstruction : public HloInstruction { + public: + explicit HloRngInstruction(const Shape& shape, + RandomDistribution distribution, + absl::Span parameters); + // Returns the random distribution for this rng node. + RandomDistribution random_distribution() const { return distribution_; } + // Returns a serialized representation of this instruction. + HloInstructionProto ToProto() const override; + + private: + bool IsElementwiseImpl( + const absl::optional& operand_idx) const override; + std::vector ExtraAttributesToStringImpl( + const HloPrintOptions& options) const override; + bool IdenticalSlowPath( + const HloInstruction& other, + const std::function& + eq_computations) const override; + // Implementation for non-common logic of CloneWithNewOperands. + std::unique_ptr CloneWithNewOperandsImpl( + const Shape& shape, absl::Span new_operands, + HloCloneContext* context) const override; + + // The distribution requested for random number generation. + RandomDistribution distribution_; +}; + +class HloParameterInstruction : public HloInstruction { + public: + explicit HloParameterInstruction(int64 parameter_number, const Shape& shape, + const string& name); + int64 parameter_number() const { return parameter_number_; } + // Returns a serialized representation of this instruction. + HloInstructionProto ToProto() const override; + + private: + bool IdenticalSlowPath( + const HloInstruction& other, + const std::function& + eq_computations) const override; + string OperandsToStringWithCanonicalNameMap( + const HloPrintOptions& options, + CanonicalNameMap* canonical_name_map) const override; + // Implementation for non-common logic of CloneWithNewOperands. + std::unique_ptr CloneWithNewOperandsImpl( + const Shape& shape, absl::Span new_operands, + HloCloneContext* context) const override; + + int64 parameter_number_ = 0; +}; + +class HloGetTupleElementInstruction : public HloInstruction { + public: + explicit HloGetTupleElementInstruction(const Shape& shape, + HloInstruction* operand, int64 index); + // Returns the tuple index associated with this instruction. + int64 tuple_index() const { return tuple_index_; } + // Returns a serialized representation of this instruction. + HloInstructionProto ToProto() const override; + + private: + std::vector ExtraAttributesToStringImpl( + const HloPrintOptions& options) const override; + bool IdenticalSlowPath( + const HloInstruction& other, + const std::function& + eq_computations) const override; + // Implementation for non-common logic of CloneWithNewOperands. + std::unique_ptr CloneWithNewOperandsImpl( + const Shape& shape, absl::Span new_operands, + HloCloneContext* context) const override; + + int64 tuple_index_ = -1; +}; + +class HloReducePrecisionInstruction : public HloInstruction { + public: + explicit HloReducePrecisionInstruction(const Shape& shape, + HloInstruction* operand, + const int exponent_bits, + const int mantissa_bits); + // Returns the number of exponent bits for a reduce-precision node. + int32 exponent_bits() const { return exponent_bits_; } + // Returns the number of mantissa bits for a reduce-precision node. + int32 mantissa_bits() const { return mantissa_bits_; } + // Returns a serialized representation of this instruction. + HloInstructionProto ToProto() const override; + + private: + std::vector ExtraAttributesToStringImpl( + const HloPrintOptions& options) const override; + bool IdenticalSlowPath( + const HloInstruction& other, + const std::function& + eq_computations) const override; + // Implementation for non-common logic of CloneWithNewOperands. + std::unique_ptr CloneWithNewOperandsImpl( + const Shape& shape, absl::Span new_operands, + HloCloneContext* context) const override; + + // The bit sizes for a reduce-precision operation. + int32 exponent_bits_ = 0; + int32 mantissa_bits_ = 0; +}; + +class HloInfeedInstruction : public HloInstruction { + public: + explicit HloInfeedInstruction(const Shape& infeed_shape, + HloInstruction* token_operand, + const string& config); + // Returns the infeed configuration string. The infeed configuration includes + // any metadata needed for the backend compiler (e.g., infeed buffer address) + // and is target-dependent. + string infeed_config() const { return infeed_config_; } + void set_infeed_config(const string& config) { infeed_config_ = config; } + // Returns the shape of the data received by the infeed. This is not the same + // as the shape of the infeed instruction which produces a tuple containing + // the infeed data shape and a TOKEN. + const Shape& infeed_shape() const { + TF_DCHECK_OK(ShapeUtil::ValidateShapeWithOptionalLayout(shape())); + return ShapeUtil::GetSubshape(shape(), {0}); + } + // Returns a serialized representation of this instruction. + HloInstructionProto ToProto() const override; + + private: + std::vector ExtraAttributesToStringImpl( + const HloPrintOptions& options) const override; + bool IdenticalSlowPath( + const HloInstruction& other, + const std::function& + eq_computations) const override; + // Implementation for non-common logic of CloneWithNewOperands. + std::unique_ptr CloneWithNewOperandsImpl( + const Shape& shape, absl::Span new_operands, + HloCloneContext* context) const override; + + // The string representation of the infeed configuration. + string infeed_config_; +}; + +class HloOutfeedInstruction : public HloInstruction { + public: + explicit HloOutfeedInstruction(const Shape& outfeed_shape, + HloInstruction* operand, + HloInstruction* token_operand, + absl::string_view outfeed_config); + // Returns the shape for the Outfeed instruction. + const Shape& outfeed_shape() const { + return outfeed_shape_; + } + // Returns the config for the Outfeed instruction. + const string& outfeed_config() const { return outfeed_config_; } + // Returns a serialized representation of this instruction. + HloInstructionProto ToProto() const override; + + private: + std::vector ExtraAttributesToStringImpl( + const HloPrintOptions& options) const override; + bool IdenticalSlowPath( + const HloInstruction& other, + const std::function& + eq_computations) const override; + // Implementation for non-common logic of CloneWithNewOperands. + std::unique_ptr CloneWithNewOperandsImpl( + const Shape& shape, absl::Span new_operands, + HloCloneContext* context) const override; + + // Shape of outfeed request. + Shape outfeed_shape_; + // Outfeed configuration information, only present for kOutfeed. + string outfeed_config_; +}; + +class HloConvolutionInstruction : public HloInstruction { + public: + explicit HloConvolutionInstruction( + const Shape& shape, HloInstruction* lhs, HloInstruction* rhs, + int64 feature_group_count, const Window& window, + const ConvolutionDimensionNumbers& dimension_numbers, + const PrecisionConfig& precision_config); + const Window& window() const override { return window_; } + void set_window(const Window& window) override { window_ = window; } + const ConvolutionDimensionNumbers& convolution_dimension_numbers() const { + return convolution_dimension_numbers_; + } + void set_convolution_dimension_numbers( + const ConvolutionDimensionNumbers& dnums) { + convolution_dimension_numbers_ = dnums; + } + // The number of feature groups. Must be a divisor of the input feature + // dimension and output feature dimension. + int64 feature_group_count() const { return feature_group_count_; } + + // Returns the information used to tell the implementation information about + // what sort of precision is requested. The meaning of the field is backend + // specific. At the moment, it is only supported for kConvolution and kDot. + // Transformations on one kDot or kConvolution to another will preserve this + // information. Transformations to other HLOs will not preserve this + // information but it is presumed that the alternate lowering is strictly + // superior. + const PrecisionConfig& precision_config() const { return precision_config_; } + + string ToCategory() const override; + // Returns a serialized representation of this instruction. + HloInstructionProto ToProto() const override; + + private: + std::vector ExtraAttributesToStringImpl( + const HloPrintOptions& options) const override; + bool IdenticalSlowPath( + const HloInstruction& other, + const std::function& + eq_computations) const override; + // Implementation for non-common logic of CloneWithNewOperands. + std::unique_ptr CloneWithNewOperandsImpl( + const Shape& shape, absl::Span new_operands, + HloCloneContext* context) const override; + // The number of feature groups. Must be a divisor of the input feature + // dimension and output feature dimension. + int64 feature_group_count_; + // Describes the window used for a convolution. + Window window_; + // Describes the dimension numbers used for a convolution. + ConvolutionDimensionNumbers convolution_dimension_numbers_; + // Information used to communicate to the implementation about the algorithm + // used to produce results. See the documentation on precision_config(). + PrecisionConfig precision_config_; +}; + +class HloReduceWindowInstruction : public HloInstruction { + public: + explicit HloReduceWindowInstruction(const Shape& shape, + HloInstruction* operand, + HloInstruction* init_value, + const Window& window, + HloComputation* reduce_computation); + const Window& window() const override { return window_; } + void set_window(const Window& window) override { window_ = window; } + // Returns a serialized representation of this instruction. + HloInstructionProto ToProto() const override; + + private: + std::vector ExtraAttributesToStringImpl( + const HloPrintOptions& options) const override; + bool IdenticalSlowPath( + const HloInstruction& other, + const std::function& + eq_computations) const override; + // Implementation for non-common logic of CloneWithNewOperands. + std::unique_ptr CloneWithNewOperandsImpl( + const Shape& shape, absl::Span new_operands, + HloCloneContext* context) const override; + Window window_; +}; + +class HloSelectAndScatterInstruction : public HloInstruction { + public: + explicit HloSelectAndScatterInstruction( + const Shape& shape, HloInstruction* operand, HloComputation* select, + const Window& window, HloInstruction* source, HloInstruction* init_value, + HloComputation* scatter); + const Window& window() const override { return window_; } + void set_window(const Window& window) override { window_ = window; } + // Gets/sets the select or scatter HloComputation for SelectAndScatter. The + // setters should only be called by HloModule or HloComputation methods. + HloComputation* select() const { + return called_computations()[kSelectComputationIndex]; + } + + HloComputation* scatter() const { + return called_computations()[kScatterComputationIndex]; + } + + void set_select(HloComputation* computation) { + // Don't allow changing the computation for fused instructions so we don't + // have to recompute called_instructions for the entire fusion instruction. + CHECK(!IsFused()); + set_called_computation(kSelectComputationIndex, computation); + } + + void set_scatter(HloComputation* computation) { + // Don't allow changing the computation for fused instructions so we don't + // have to recompute called_instructions for the entire fusion instruction. + CHECK(!IsFused()); + set_called_computation(kScatterComputationIndex, computation); + } + // Returns a serialized representation of this instruction. + HloInstructionProto ToProto() const override; + + private: + std::vector ExtraAttributesToStringImpl( + const HloPrintOptions& options) const override; + bool IdenticalSlowPath( + const HloInstruction& other, + const std::function& + eq_computations) const override; + // Implementation for non-common logic of CloneWithNewOperands. + std::unique_ptr CloneWithNewOperandsImpl( + const Shape& shape, absl::Span new_operands, + HloCloneContext* context) const override; + Window window_; +}; + +class HloCustomCallInstruction : public HloInstruction { + public: + explicit HloCustomCallInstruction(const Shape& shape, + absl::Span operands, + absl::string_view custom_call_target, + absl::string_view opaque); + const Window& window() const override { + CHECK(window_ != nullptr); + return *window_; + } + + void set_window(const Window& window) override { + window_ = absl::make_unique(window); + } + + const ConvolutionDimensionNumbers& convolution_dimension_numbers() const { + CHECK(convolution_dimension_numbers_ != nullptr); + return *convolution_dimension_numbers_; + } + + void set_convolution_dimension_numbers( + const ConvolutionDimensionNumbers& dnums) { + convolution_dimension_numbers_ = + absl::make_unique(dnums); + } + const string& opaque() const { return opaque_; } + const string& custom_call_target() const { return custom_call_target_; } + void set_feature_group_count(int64 feature_group_count) { + feature_group_count_ = feature_group_count; + } + int64 feature_group_count() const { return feature_group_count_; } + // Returns a serialized representation of this instruction. + HloInstructionProto ToProto() const override; + + private: + std::vector ExtraAttributesToStringImpl( + const HloPrintOptions& options) const override; + bool IdenticalSlowPath( + const HloInstruction& other, + const std::function& + eq_computations) const override; + // Implementation for non-common logic of CloneWithNewOperands. + std::unique_ptr CloneWithNewOperandsImpl( + const Shape& shape, absl::Span new_operands, + HloCloneContext* context) const override; + // Name of a global symbol to call. + string custom_call_target_; + // Opaque string interpreted by the backend. + string opaque_; + // Describes the window in a windowed operation such as convolution. + std::unique_ptr window_; + // Describes the dimension numbers used for a convolution. + std::unique_ptr convolution_dimension_numbers_; + // The number of feature groups. This is used for grouped convolutions. + int64 feature_group_count_; +}; + +class HloPadInstruction : public HloInstruction { + public: + explicit HloPadInstruction(const Shape& shape, HloInstruction* operand, + HloInstruction* padding_value, + const PaddingConfig& padding_config); + // Returns the padding configuration for a pad node. + const PaddingConfig& padding_config() const { return padding_config_; } + // Returns a serialized representation of this instruction. + HloInstructionProto ToProto() const override; + + private: + std::vector ExtraAttributesToStringImpl( + const HloPrintOptions& options) const override; + bool IdenticalSlowPath( + const HloInstruction& other, + const std::function& + eq_computations) const override; + // Implementation for non-common logic of CloneWithNewOperands. + std::unique_ptr CloneWithNewOperandsImpl( + const Shape& shape, absl::Span new_operands, + HloCloneContext* context) const override; + + // The padding configuration that describes the edge padding and interior + // padding of this pad instruction. + PaddingConfig padding_config_; +}; + +class HloDynamicSliceInstruction : public HloInstruction { + public: + explicit HloDynamicSliceInstruction(const Shape& shape, + HloInstruction* operand, + HloInstruction* start_indices, + absl::Span slice_sizes); + // Old methods kept for smooth subclassing transition END. + // Returns the size of the slice in the given dimension for a dynamic + // slice node. + int64 slice_sizes(int64 dimension) const { + return dynamic_slice_sizes_[dimension]; + } + const std::vector& dynamic_slice_sizes() const { + return dynamic_slice_sizes_; + } + // Returns a serialized representation of this instruction. + HloInstructionProto ToProto() const override; + + private: + std::vector ExtraAttributesToStringImpl( + const HloPrintOptions& options) const override; + bool IdenticalSlowPath( + const HloInstruction& other, + const std::function& + eq_computations) const override; + // Implementation for non-common logic of CloneWithNewOperands. + std::unique_ptr CloneWithNewOperandsImpl( + const Shape& shape, absl::Span new_operands, + HloCloneContext* context) const override; + + // Describes the [start, start + size) range size for a dynamic slice + // ('start' is specified dynamically in the second operand of the operation). + std::vector dynamic_slice_sizes_; +}; + +class HloGatherInstruction : public HloInstruction { + public: + explicit HloGatherInstruction( + const Shape& shape, HloInstruction* operand, + HloInstruction* start_indices, + const GatherDimensionNumbers& gather_dim_numbers, + absl::Span slice_sizes); + const GatherDimensionNumbers& gather_dimension_numbers() const { + CHECK(gather_dimension_numbers_ != nullptr); + return *gather_dimension_numbers_; + } + absl::Span gather_slice_sizes() const { + return gather_slice_sizes_; + } + // Returns the dump string of the gather dimension numbers. + string GatherDimensionNumbersToString() const; + // Returns a serialized representation of this instruction. + HloInstructionProto ToProto() const override; + + // Creates an instance of GatherDimensionNumbers. + static GatherDimensionNumbers MakeGatherDimNumbers( + absl::Span offset_dims, + absl::Span collapsed_slice_dims, + absl::Span start_index_map, int64 index_vector_dim); + + private: + std::vector ExtraAttributesToStringImpl( + const HloPrintOptions& options) const override; + bool IdenticalSlowPath( + const HloInstruction& other, + const std::function& + eq_computations) const override; + std::unique_ptr CloneWithNewOperandsImpl( + const Shape& shape, absl::Span new_operands, + HloCloneContext* context) const override; + + std::unique_ptr gather_dimension_numbers_; + std::vector gather_slice_sizes_; +}; + +class HloScatterInstruction : public HloInstruction { + public: + explicit HloScatterInstruction( + const Shape& shape, HloInstruction* operand, + HloInstruction* scatter_indices, HloInstruction* updates, + HloComputation* update_computation, + const ScatterDimensionNumbers& scatter_dim_numbers); + const ScatterDimensionNumbers& scatter_dimension_numbers() const { + CHECK(scatter_dimension_numbers_ != nullptr); + return *scatter_dimension_numbers_; + } + // Returns the dump string of the scatter dimension numbers. + string ScatterDimensionNumbersToString() const; + // Returns a serialized representation of this instruction. + HloInstructionProto ToProto() const override; + + // Creates an instance of ScatterDimensionNumbers. + static ScatterDimensionNumbers MakeScatterDimNumbers( + absl::Span update_window_dims, + absl::Span inserted_window_dims, + absl::Span scatter_dims_to_operand_dims, + int64 index_vector_dim); + + private: + std::vector ExtraAttributesToStringImpl( + const HloPrintOptions& options) const override; + bool IdenticalSlowPath( + const HloInstruction& other, + const std::function& + eq_computations) const override; + // Implementation for non-common logic of CloneWithNewOperands. + std::unique_ptr CloneWithNewOperandsImpl( + const Shape& shape, absl::Span new_operands, + HloCloneContext* context) const override; + + std::unique_ptr scatter_dimension_numbers_; +}; + +class HloIotaInstruction : public HloInstruction { + public: + explicit HloIotaInstruction(const Shape& shape, int64 iota_dimension); + // Returns the dimension sizes or numbers associated with this instruction. + int64 iota_dimension() const { return iota_dimension_; } + // Returns a serialized representation of this instruction. + HloInstructionProto ToProto() const override; + + private: + std::vector ExtraAttributesToStringImpl( + const HloPrintOptions& options) const override; + bool IdenticalSlowPath( + const HloInstruction& other, + const std::function& + eq_computations) const override; + // Implementation for non-common logic of CloneWithNewOperands. + std::unique_ptr CloneWithNewOperandsImpl( + const Shape& shape, absl::Span new_operands, + HloCloneContext* context) const override; + + const int64 iota_dimension_; +}; + +class HloDotInstruction : public HloInstruction { + public: + // Creates a dot op with operands 'lhs' and 'rhs' with contracting and batch + // dimensions specified in 'dimension_numbers'. + explicit HloDotInstruction(const Shape& shape, HloInstruction* lhs, + HloInstruction* rhs, + const DotDimensionNumbers& dimension_numbers, + const PrecisionConfig& precision_config); + + // Returns data on the dimension numbers used for a dot operation. + const DotDimensionNumbers& dot_dimension_numbers() const { + return dot_dimension_numbers_; + } + + // Returns the information used to tell the implementation information about + // what sort of precision is requested. The meaning of the field is backend + // specific. At the moment, it is only supported for kConvolution and kDot. + // Transformations on one kDot or kConvolution to another will preserve this + // information. Transformations to other HLOs will not preserve this + // information but it is presumed that the alternate lowering is strictly + // superior. + const PrecisionConfig& precision_config() const { return precision_config_; } + + // Returns a serialized representation of this instruction. + HloInstructionProto ToProto() const override; + + private: + std::vector ExtraAttributesToStringImpl( + const HloPrintOptions& options) const override; + bool IdenticalSlowPath( + const HloInstruction& other, + const std::function& + eq_computations) const override; + // Implementation for non-common logic of CloneWithNewOperands. + std::unique_ptr CloneWithNewOperandsImpl( + const Shape& shape, absl::Span new_operands, + HloCloneContext* context) const override; + // Returns the dump string of the dot dimension numbers. + string DotDimensionNumbersToString() const; + + // Describes the dimension numbers used for a dot. + DotDimensionNumbers dot_dimension_numbers_; + + // Information used to communicate to the implementation about the algorithm + // used to produce results. See the documentation on precision_config(). + PrecisionConfig precision_config_; +}; + +class HloDomainInstruction : public HloInstruction { + public: + explicit HloDomainInstruction( + const Shape& shape, HloInstruction* operand, + std::unique_ptr operand_side_metadata, + std::unique_ptr user_side_metadata); + + // Retrieves the operand side metadata of a kDomain instruction. + const DomainMetadata& operand_side_metadata() const { + return *operand_side_metadata_; + } + // Retrieves the user side metadata of a kDomain instruction. + const DomainMetadata& user_side_metadata() const { + return *user_side_metadata_; + } + + private: + std::vector ExtraAttributesToStringImpl( + const HloPrintOptions& options) const override; + bool IdenticalSlowPath( + const HloInstruction& other, + const std::function& + eq_computations) const override; + // Implementation for non-common logic of CloneWithNewOperands. + std::unique_ptr CloneWithNewOperandsImpl( + const Shape& shape, absl::Span new_operands, + HloCloneContext* context) const override; + + std::unique_ptr operand_side_metadata_; + std::unique_ptr user_side_metadata_; +}; +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_HLO_INSTRUCTIONS_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/hlo_memory_scheduler.h b/linux/include/tensorflow/compiler/xla/service/hlo_memory_scheduler.h new file mode 100644 index 000000000..a4c1d3db8 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/hlo_memory_scheduler.h @@ -0,0 +1,124 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_HLO_MEMORY_SCHEDULER_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_HLO_MEMORY_SCHEDULER_H_ + +#include + +#include "absl/container/flat_hash_map.h" +#include "tensorflow/compiler/xla/service/hlo_instruction.h" +#include "tensorflow/compiler/xla/service/hlo_module.h" +#include "tensorflow/compiler/xla/service/hlo_ordering.h" +#include "tensorflow/compiler/xla/service/hlo_pass_interface.h" +#include "tensorflow/compiler/xla/service/hlo_schedule.h" +#include "tensorflow/compiler/xla/service/logical_buffer.h" +#include "tensorflow/compiler/xla/service/tuple_points_to_analysis.h" +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/compiler/xla/types.h" + +namespace xla { + +// A memory scheduler computes an execution sequence for the HLO instructions in +// 'computation' that minimizes peak memory, given a points-to analysis result +// that describes buffer aliasing, together with a target-specific size function +// that maps a tensor's logical size to its padded size. +typedef std::function( + const HloComputation&, const TuplePointsToAnalysis&, + const LogicalBuffer::SizeFunction&, + const absl::flat_hash_map&)> + MemorySchedulerAlgorithm; + +// List scheduler +StatusOr ListMemoryScheduler( + const HloComputation& computation, + const TuplePointsToAnalysis& points_to_analysis, + const LogicalBuffer::SizeFunction& size_function, + const absl::flat_hash_map& + memory_by_computation); + +// DFS-order scheduler +StatusOr DFSMemoryScheduler( + const HloComputation& computation, + const TuplePointsToAnalysis& points_to_analysis, + const LogicalBuffer::SizeFunction& size_function, + const absl::flat_hash_map& + memory_by_computation); + +// Naive Post Order scheduler +StatusOr PostOrderMemoryScheduler( + const HloComputation& computation, + const TuplePointsToAnalysis& points_to_analysis, + const LogicalBuffer::SizeFunction& size_function, + const absl::flat_hash_map& + memory_by_computation); + +// The default scheduling algorithm. Runs both the list scheduler +// and the DFS scheduler, and chooses whichever returns a lower min-memory, +// not accounting for fragmentation. +StatusOr DefaultMemoryScheduler( + const HloComputation& computation, + const TuplePointsToAnalysis& points_to_analysis, + const LogicalBuffer::SizeFunction& size_function, + const absl::flat_hash_map& + memory_by_computation); + +// Returns an HloSchedule which seeks to minimize the memory required for +// the computation. size_function is the function returning the number of bytes +// required for a LogicalBuffer. +StatusOr ScheduleModule( + const HloModule& module, const LogicalBuffer::SizeFunction& size_function, + const MemorySchedulerAlgorithm& algorithm = {}); + +// Computes the schedule for a single computation. +// Currently only used by the GPU backend. +StatusOr ScheduleComputation( + const HloComputation& computation, + const LogicalBuffer::SizeFunction& size_function); + +// A pass which schedules the HLO instructions in a module. The HloModule's +// schedule field is set to the resulting HloSchedule using +// HloModule::set_schedule. +class HloMemoryScheduler : public HloModulePass { + public: + // size_function is the function returning the number of bytes required for a + // LogicalBuffer. algorithm is the memory scheduling algorithm to use. If not + // specified, then DefaultMemoryScheduler is used. + HloMemoryScheduler(const LogicalBuffer::SizeFunction& size_function, + const MemorySchedulerAlgorithm& algorithm = {}); + ~HloMemoryScheduler() override = default; + absl::string_view name() const override { return "hlo-memory-scheduler"; } + + StatusOr Run(HloModule* module) override; + + private: + LogicalBuffer::SizeFunction size_function_; + MemorySchedulerAlgorithm algorithm_; +}; + +// A trivial pass which clears the schedule currently set on the +// HloModule. After this pass runs HloModudle::has_schedule will return false. +class HloDescheduler : public HloModulePass { + public: + HloDescheduler() = default; + ~HloDescheduler() override = default; + absl::string_view name() const override { return "hlo-descheduler"; } + + StatusOr Run(HloModule* module) override; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_HLO_MEMORY_SCHEDULER_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/hlo_module.h b/linux/include/tensorflow/compiler/xla/service/hlo_module.h new file mode 100644 index 000000000..735804e82 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/hlo_module.h @@ -0,0 +1,291 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_HLO_MODULE_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_HLO_MODULE_H_ + +#include +#include +#include +#include +#include +#include +#include + +#include "absl/strings/string_view.h" +#include "absl/types/optional.h" +#include "absl/types/span.h" +#include "tensorflow/compiler/xla/iterator_util.h" +#include "tensorflow/compiler/xla/service/hlo.pb.h" +#include "tensorflow/compiler/xla/service/hlo_clone_context.h" +#include "tensorflow/compiler/xla/service/hlo_computation.h" +#include "tensorflow/compiler/xla/service/hlo_instruction.h" +#include "tensorflow/compiler/xla/service/hlo_module_config.h" +#include "tensorflow/compiler/xla/service/hlo_schedule.h" +#include "tensorflow/compiler/xla/service/name_uniquer.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/core/lib/gtl/iterator_range.h" +#include "tensorflow/core/platform/logging.h" +#include "tensorflow/core/platform/mutex.h" + +namespace xla { + +// Describes a compilation unit at the HLO level. +// +// HloModule is the top-level unit in the HLO IR. It corresponds to a whole +// "program". Running a module, from beginning to end, is the only way to run +// an XLA program. +// +// A module contains one "entry computation"; this HloComputation is like main() +// in a C program. The result of running the module is the result of running +// this computation. +// +// A module also contains some number of "nested computations". Each nested +// computation is attached to an HloInstruction within some other computation. +// The meaning of the nested computation depends on the instruction it's +// attached to. +class HloModule { + public: + // Constructor without a versioned computation handle. This constructor should + // only be used for HloModules used outside of the XLA service (eg + // tests). The versioned handle is used by the service in the compilation + // cache. A default configuration is created for this module. + explicit HloModule(const string& name, const HloModuleConfig& config); + virtual ~HloModule() {} + + // Adds an entry computation to the module. A module can only have one entry + // computation. Returns a pointer to the newly added computation. + HloComputation* AddEntryComputation( + std::unique_ptr computation); + + // Adds an embedded computation to the module. + HloComputation* AddEmbeddedComputation( + std::unique_ptr computation); + + // Removes an embedded computation. + Status RemoveEmbeddedComputation(HloComputation* to_remove); + + // Replaces all uses of computations that are keys of 'replacements' with + // the corresponding values in 'replacements'. Replaces the entry computation, + // if applicable. + // + // This function iterates over all instructions in the module to find + // computations to replace. We could speed it up by keeping track of users of + // computations. + void ReplaceComputations( + const std::unordered_map& replacements); + + const string& name() const { return name_; } + void set_name(string name) { name_ = std::move(name); } + + // Returns a deep copy of this module including all computations. + std::unique_ptr Clone(const string& suffix = "clone") const; + + // Performs a deep clone of the computation, by recursively cloning all + // the called computations as well. If the clone context is specified, it + // will be populated with the cloned object mappings. + HloComputation* DeepCloneComputation(HloComputation* computation, + HloCloneContext* context = nullptr); + + // Return a pointer to the entry computation of the module.. + const HloComputation* entry_computation() const { + CHECK_NE(nullptr, entry_computation_); + return entry_computation_; + } + HloComputation* entry_computation() { + CHECK_NE(nullptr, entry_computation_); + return entry_computation_; + } + + // Creates the ComputationLayout which describes the current status of the HLO + // module entry computation. + ComputationLayout compute_computation_layout() const { + return ComputationLayout(entry_computation()->ComputeProgramShape(), + /*ignore_layouts=*/false); + } + + ComputationLayout* mutable_entry_computation_layout() { + return config_.mutable_entry_computation_layout(); + } + + const ComputationLayout& entry_computation_layout() const { + return config_.entry_computation_layout(); + } + + // Gets the computations in this module. + // + // Returns a view of HloComputation*s, so you can iterate over this in the + // natural way: + // + // for (HloComputation* c : module->computations()) { ... } + // + tensorflow::gtl::iterator_range>::const_iterator>> + computations() const { + return {MakeUnwrappingIterator(computations_.begin()), + MakeUnwrappingIterator(computations_.end())}; + } + tensorflow::gtl::iterator_range>::iterator>> + computations() { + return {MakeUnwrappingIterator(computations_.begin()), + MakeUnwrappingIterator(computations_.end())}; + } + + // Returns the computation in this module that has the name `name`. Returns + // null if there is no such computation. + HloComputation* GetComputationWithName(absl::string_view name); + + // Gets the number of computations in this module. + int64 computation_count() const { return computations_.size(); } + + // Gets the number of instructions in this module. + int64 instruction_count() const; + + // Compute and return a post order of all computations in the module. The sort + // is defined like so: if computation A has an instruction which calls + // computation B, then A will appear after B in the sort. + std::vector MakeComputationPostOrder() const; + + // Gets the computations in this module which aren't for fusion nodes. + // + // Postcondition: All computations in the returned list have + // !IsFusionComputation(). + // + // Note: Callers can and do rely on the return value here being a *snapshot* + // of the module's non-fusion computations -- that is, it's OK to add or + // remove computations from a module while iterating over + // MakeNonfusionComputations(). + std::vector MakeNonfusionComputations() const; + + const HloModuleConfig& config() const { return config_; } + + // Return a string representation of the module. + // + // (We express the default options using an overload rather than a default + // param because gdb ignores default params, but does resolve overloads.) + string ToString() const { return ToString(HloPrintOptions()); } + string ToString(const HloPrintOptions& options) const; + + // Convert an HloModule to or from a proto. + HloModuleProto ToProto() const; + static StatusOr> CreateFromProto( + const HloModuleProto& proto, const HloModuleConfig& module_config); + + // Creates and returns an HloModuleConfig with an appropriate program shape + // for the HLO module in the given proto. + static StatusOr CreateModuleConfigFromProto( + const HloModuleProto& module, const DebugOptions& debug_options); + + // Outlines the given expression from the given computation. + // instructions_to_outline contains the instructions that form the expression. + // + // Precondition: instructions in instructions_to_outline are in topological + // order (root of outlined instructions last). TODO(jingyue): takes a set of + // instructions and topologically sorts them. + HloInstruction* OutlineExpressionFromComputation( + absl::Span instructions_to_outline, + const string& outlined_computation_name, HloComputation* computation); + + // Returns a randomly generated uint64. + uint64 RandomNew64() const; + + // Returns the NameUniquer for uniquing instruction names in this module. + NameUniquer& instruction_name_uniquer() { return instruction_name_uniquer_; } + + // Assign a new unique dense id for an instruction + int NewUniqueInstructionId() { + int result = next_unique_id_; + next_unique_id_++; + return result; + } + + // Returns the number of unique intruction ids given out. All ids up to + // this point are guaranteed to be in the range [0..NumUniqueInstructionIds()) + int NumUniqueInstructionIds() const { return next_unique_id_; } + + // Returns an id that is unique to this module across all modules created over + // the lifetime of this process. + int unique_id() const { return unique_id_; } + + // Returns a non-const version of the passed-in const HloInstruction*. This is + // safe on the argument that if you have a non-const module, then you can + // access all instructions in the module as non-const. + // + // Returns an error if the passed-in instruction is not from this module, + // except that it is allowed to pass in a null pointer. + // + // TODO(b/78350259): Eliminate const laundering. The argument above is not + // reliable since at any time someone could add or discover a way for a + // non-const module to transitively contain a const HloInstruction. The + // reliable way to do this would be to create a const laundering map from a + // module, mapping each encountered HloInstruction to its non-const version + // and then look up each instruction in need of laundering in that map, but + // this is much more expensive and complicated. This returns a Status instead + // of doing a CHECK-failure in part to make it strongly apparent that this is + // something that can fail. + StatusOr LaunderConstInstructionFromModule( + const HloInstruction* hlo); + + // Sets the schedule of the module to the given schedule. + Status set_schedule(HloSchedule schedule); + + // Clears the schedule of the module. + void clear_schedule() { schedule_.reset(); } + + // Returns true if the module has a schedule set. + bool has_schedule() const { return schedule_.has_value(); } + + // Returns the schedue of the module. CHECK fails if no schedule is set. + const HloSchedule& schedule() const { return *schedule_; } + HloSchedule& schedule() { return *schedule_; } + + private: + HloComputation* AddComputationInternal( + std::unique_ptr computation, bool is_entry, + bool uniquify_identifiers); + + string name_; + HloModuleConfig config_; + HloComputation* entry_computation_ = nullptr; + std::vector> computations_; + + // Random number generator engine to use when generating random numbers per + // HloModule compilation. + // TODO(b/25995601): Replace with better seed setting or dev/random for + // where we don't need deterministic execution. + mutable std::mt19937_64 rng_{42}; + mutable tensorflow::mutex rng_mutex_; + + // Unique name generator for computation and instruction names, which are + // unique per module. + NameUniquer computation_name_uniquer_{/*separator=*/"."}; + NameUniquer instruction_name_uniquer_{/*separator=*/"."}; + int next_unique_id_ = 0; + + // Used to keep track of the next unique module id that should be assigned. + static std::atomic next_unique_module_id_; + // A unique id to label modules with. + int unique_id_; + + // The HloSchedule of the module. The schedule if it exists contains a + // sequential order of instructions for each non-fusion computation in the + // module. + absl::optional schedule_; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_HLO_MODULE_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/hlo_module_config.h b/linux/include/tensorflow/compiler/xla/service/hlo_module_config.h new file mode 100644 index 000000000..68c18836e --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/hlo_module_config.h @@ -0,0 +1,124 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_HLO_MODULE_CONFIG_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_HLO_MODULE_CONFIG_H_ + +#include + +#include "absl/types/optional.h" +#include "tensorflow/compiler/xla/service/computation_layout.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/compiler/xla/xla.pb.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" + +namespace xla { + +// This class gathers all settings and values which affect the compiled +// executable outside of the HLO code itself. This include layouts of inputs and +// outputs to the module and settings such as HLO profiling. Together the +// HloModule and HloModuleConfig unambiguously determine a particular +// executable. +class HloModuleConfig { + public: + // A configuration can be created either with, or without an entry + // ComputationLayout. The default ctor creates it without -- in this case + // accessing entry_computation_layout will CHECK-fail. The ctor accepting a + // ProgramShape creates a computation layout using this shape. + // The layouts in the ProgramShape will be reset to default unless + // ignore_layouts is set to false. + HloModuleConfig() = default; + + explicit HloModuleConfig(const ProgramShape& program_shape, + bool ignore_layouts = true); + + // Checks if this config has an entry computation layout already. + bool has_entry_computation_layout() const { + return entry_computation_layout_.has_value(); + } + + // Sets the entry computation layout for this config. If the entry computation + // layout already exists, it is silently replaced. + void SetDefaultComputationLayout(const ProgramShape& program_shape); + + // Returns a constant reference to the layout of the entry computation. + // Assumes the layout was set. + const ComputationLayout& entry_computation_layout() const { + CHECK(entry_computation_layout_.has_value()); + return *entry_computation_layout_; + } + + // Returns a mutable pointer to the layout of the entry computation. + // Assumes the layout was set. + ComputationLayout* mutable_entry_computation_layout() { + CHECK(entry_computation_layout_.has_value()); + return &(*entry_computation_layout_); + } + + // Returns whether to enable HLO-level profiling. + bool hlo_profiling_enabled() const { + return debug_options_.xla_hlo_profile(); + } + + // Sets/returns the module seed set during execution. + void set_seed(uint64 seed) { seed_ = seed; } + uint64 seed() const { return seed_; } + + void set_replica_count(int64 replica_count) { + replica_count_ = replica_count; + } + int64 replica_count() const { return replica_count_; } + + // Return a string which unambiguously represents all the fields of this data + // structure. Used for generating a cache key for storing the compiled + // executable. + string compilation_cache_key() const; + + const DebugOptions& debug_options() const { return debug_options_; } + + void set_debug_options(const DebugOptions& debug_options) { + debug_options_ = debug_options; + } + + // Sets/returns the number of intra op threads for this module. + void set_intra_op_parallelism_threads( + const int intra_op_parallelism_threads) { + intra_op_parallelism_threads_ = intra_op_parallelism_threads; + } + int64 intra_op_parallelism_threads() const { + return intra_op_parallelism_threads_; + } + + private: + // If you add new members, be sure to update compilation_cache_key. + + absl::optional entry_computation_layout_; + + // Module/graph-level seed handle. + uint64 seed_ = 0; + + // The number of replicas to compile this binary for. + int64 replica_count_ = 1; + + // The target maximum parallelism at which to partition HLOs for parallel + // execution on the CPU backend. + int64 intra_op_parallelism_threads_ = -1; + + DebugOptions debug_options_; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_HLO_MODULE_CONFIG_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/hlo_module_group.h b/linux/include/tensorflow/compiler/xla/service/hlo_module_group.h new file mode 100644 index 000000000..7338be8b9 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/hlo_module_group.h @@ -0,0 +1,81 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_HLO_MODULE_GROUP_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_HLO_MODULE_GROUP_H_ + +#include +#include +#include + +#include "absl/strings/string_view.h" +#include "absl/types/span.h" +#include "tensorflow/compiler/xla/service/hlo.pb.h" +#include "tensorflow/compiler/xla/service/hlo_module.h" + +namespace xla { + +// An abstraction representing a ordered set of HLO module built to run +// concurrently across different devices. +class HloModuleGroup { + public: + // Construct an empty module group. + explicit HloModuleGroup(absl::string_view name) : name_(name) {} + + // Construct a module group containing a single module. + HloModuleGroup(absl::string_view name, std::unique_ptr module); + + // Construct a module group containing any number of modules. + HloModuleGroup(absl::string_view name, + absl::Span> modules); + + // Returns the modules contained in the group. + const std::vector& modules() const { return module_ptrs_; } + + // Returns a module at a particular index. + HloModule& module(int index) const { return *module_ptrs_.at(index); } + + // Add a module to the back of vector of modules in the group. + void push_back(std::unique_ptr module); + + // Moves all modules from the group into the returned vector. After this + // method runs, the module group will be empty. + std::vector> ConsumeModules(); + + string name() const { return name_; } + string ToString() const; + + // Serialize the module group to/from a proto. + HloModuleGroupProto ToProto() const; + static StatusOr CreateFromProto( + const HloModuleGroupProto& proto, + absl::Span module_configs); + + private: + string name_; + + // Vector of modules as std::unique_ptrs. + std::vector> modules_; + + // Vector of modules as normal pointers. This vector is kept in sync with + // modules_ as modules are added to the group with push_back. + std::vector module_ptrs_; +}; + +std::ostream& operator<<(std::ostream& out, const HloModuleGroup& group); + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_HLO_MODULE_GROUP_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/hlo_opcode.h b/linux/include/tensorflow/compiler/xla/service/hlo_opcode.h new file mode 100644 index 000000000..e6bfb8025 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/hlo_opcode.h @@ -0,0 +1,181 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_HLO_OPCODE_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_HLO_OPCODE_H_ + +#include +#include +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/compiler/xla/types.h" + +namespace xla { + +// High-level optimizer instruction opcodes -- these are linear-algebra level +// opcodes. They are a flattened form of the UnaryOp, BinaryOp, ... opcodes +// present in the XLA service protobuf. +// +// See the XLA documentation for the semantics of each opcode. +// +// Each entry has the format: +// (enum_name, opcode_name) +// or +// (enum_name, opcode_name, p1 | p2 | ...) +// +// with p1, p2, ... are members of HloOpcodeProperty. They are combined +// using bitwise-or. +// +// Note: Do not use ':' in opcode names. It is used as a special character +// in these places: +// - In extended opcode strings (HloInstruction::ExtendedOpcodeString()), to +// separate the opcode from the fusion kind +// - In fully qualified names (HloInstruction::FullyQualifiedName()), to +// separate the qualifiers (name of the computation and potentially the +// fusion instruction) from the name +#define HLO_OPCODE_LIST(V) \ + V(kAbs, "abs") \ + V(kAdd, "add") \ + V(kAllToAll, "all-to-all") \ + V(kAtan2, "atan2") \ + V(kBatchNormGrad, "batch-norm-grad") \ + V(kBatchNormInference, "batch-norm-inference") \ + V(kBatchNormTraining, "batch-norm-training") \ + V(kBitcast, "bitcast") \ + V(kBitcastConvert, "bitcast-convert") \ + V(kBroadcast, "broadcast") \ + V(kCall, "call", kHloOpcodeIsVariadic) \ + V(kCeil, "ceil") \ + V(kClamp, "clamp") \ + V(kCollectivePermute, "collective-permute") \ + V(kClz, "count-leading-zeros") \ + V(kComplex, "complex") \ + V(kConcatenate, "concatenate", kHloOpcodeIsVariadic) \ + V(kConditional, "conditional") \ + V(kConstant, "constant") \ + V(kConvert, "convert") \ + V(kConvolution, "convolution") \ + V(kCopy, "copy") \ + V(kCos, "cosine") \ + V(kCrossReplicaSum, "cross-replica-sum") \ + V(kCustomCall, "custom-call") \ + V(kDivide, "divide") \ + V(kDomain, "domain") \ + V(kDot, "dot") \ + V(kDynamicSlice, "dynamic-slice") \ + V(kDynamicUpdateSlice, "dynamic-update-slice") \ + V(kEq, "equal-to", kHloOpcodeIsComparison) \ + V(kExp, "exponential") \ + V(kExpm1, "exponential-minus-one") \ + V(kFft, "fft") \ + V(kFloor, "floor") \ + V(kFusion, "fusion", kHloOpcodeIsVariadic) \ + V(kGather, "gather") \ + V(kGe, "greater-than-or-equal-to", kHloOpcodeIsComparison) \ + V(kAfterAll, "after-all", kHloOpcodeIsVariadic) \ + V(kGetTupleElement, "get-tuple-element") \ + V(kGt, "greater-than", kHloOpcodeIsComparison) \ + V(kImag, "imag") \ + V(kInfeed, "infeed") \ + V(kIota, "iota") \ + V(kIsFinite, "is-finite") \ + V(kLe, "less-than-or-equal-to", kHloOpcodeIsComparison) \ + V(kLog, "log") \ + V(kLog1p, "log-plus-one") \ + V(kAnd, "and") \ + V(kNot, "not") \ + V(kOr, "or") \ + V(kXor, "xor") \ + V(kLt, "less-than", kHloOpcodeIsComparison) \ + V(kMap, "map", kHloOpcodeIsVariadic) \ + V(kMaximum, "maximum") \ + V(kMinimum, "minimum") \ + V(kMultiply, "multiply") \ + V(kNe, "not-equal-to", kHloOpcodeIsComparison) \ + V(kNegate, "negate") \ + V(kOutfeed, "outfeed") \ + V(kPad, "pad") \ + V(kParameter, "parameter") \ + V(kPower, "power") \ + V(kReal, "real") \ + V(kRecv, "recv") \ + V(kRecvDone, "recv-done") \ + V(kReduce, "reduce") \ + V(kReducePrecision, "reduce-precision") \ + V(kReduceWindow, "reduce-window") \ + V(kRemainder, "remainder") \ + V(kReshape, "reshape") \ + V(kReverse, "reverse") \ + V(kRng, "rng") \ + V(kRoundNearestAfz, "round-nearest-afz") \ + V(kScatter, "scatter") \ + V(kSelect, "select") \ + V(kSelectAndScatter, "select-and-scatter") \ + V(kSend, "send") \ + V(kSendDone, "send-done") \ + V(kShiftLeft, "shift-left") \ + V(kShiftRightArithmetic, "shift-right-arithmetic") \ + V(kShiftRightLogical, "shift-right-logical") \ + V(kSign, "sign") \ + V(kSin, "sine") \ + V(kSlice, "slice") \ + V(kSort, "sort") \ + V(kSubtract, "subtract") \ + V(kTanh, "tanh") \ + V(kTrace, "trace") \ + V(kTranspose, "transpose") \ + V(kTuple, "tuple", kHloOpcodeIsVariadic) \ + V(kTupleSelect, "tuple-select") \ + V(kWhile, "while") + +enum class HloOpcode { +#define DECLARE_ENUM(enum_name, opcode_name, ...) enum_name, + HLO_OPCODE_LIST(DECLARE_ENUM) +#undef DECLARE_ENUM +}; + +// List of properties associated with opcodes. +// Properties are defined as increasing powers of two, so that we can use +// bitwise-or to combine properties, and bitwise-and to test for them. +enum HloOpcodeProperty { + kHloOpcodeIsComparison = 1 << 0, + kHloOpcodeIsVariadic = 1 << 1, +}; + +// Returns a string representation of the opcode. +string HloOpcodeString(HloOpcode opcode); + +// Retrieves the opcode enum by name if the opcode exists. +StatusOr StringToHloOpcode(const string& opcode_name); + +inline std::ostream& operator<<(std::ostream& os, HloOpcode opcode) { + return os << HloOpcodeString(opcode); +} + +// Returns true iff the given opcode is a comparison operation. +bool HloOpcodeIsComparison(HloOpcode opcode); + +// Returns true iff the given opcode has variadic operands. +bool HloOpcodeIsVariadic(HloOpcode opcode); + +// Returns the number of HloOpcode values. +inline const uint32_t HloOpcodeCount() { +#define HLO_COUNT_ONE(...) +1 +#define HLO_XLIST_LENGTH(list) list(HLO_COUNT_ONE) + return HLO_XLIST_LENGTH(HLO_OPCODE_LIST); +} + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_HLO_OPCODE_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/hlo_ordering.h b/linux/include/tensorflow/compiler/xla/service/hlo_ordering.h new file mode 100644 index 000000000..66313492e --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/hlo_ordering.h @@ -0,0 +1,212 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_HLO_ORDERING_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_HLO_ORDERING_H_ + +#include +#include +#include + +#include "absl/container/flat_hash_map.h" +#include "tensorflow/compiler/xla/service/call_graph.h" +#include "tensorflow/compiler/xla/service/hlo.pb.h" +#include "tensorflow/compiler/xla/service/hlo_dataflow_analysis.h" +#include "tensorflow/compiler/xla/service/hlo_instruction.h" +#include "tensorflow/compiler/xla/service/hlo_module.h" +#include "tensorflow/compiler/xla/service/hlo_schedule.h" +#include "tensorflow/compiler/xla/service/hlo_value.h" +#include "tensorflow/compiler/xla/types.h" + +namespace xla { + +// Base class for describing a partial ordering of HLO instructions. Used to +// determine live range overlap of HLO instruction output buffers. +class HloOrdering { + public: + HloOrdering(const HloModule* module) + : module_(module), call_graph_(CallGraph::Build(module)) {} + virtual ~HloOrdering() = default; + + // Returns true if instruction 'a' executes before instruction 'b'. This is + // not reflexive, that is, an instruction does not execute before itself. + bool ExecutesBefore(const HloInstruction* a, const HloInstruction* b) const; + + // Returns whether the value 'a' is defined before the value 'b' under the + // given ordering. + bool IsDefinedBefore(const HloValue& a, const HloValue& b) const; + + // Returns whether the given use is before the given value definition under + // the given ordering. + bool UseIsBeforeValueDefinition(const HloUse& use, const HloValue& value, + const HloDataflowAnalysis& dataflow) const; + // Returns whether the given values interfere. Two values interfere if they + // may both be simultaneously live. + bool MayInterfere(const HloValue& a, const HloValue& b, + const HloDataflowAnalysis& dataflow) const; + + // Returns true if the live range of the given value 'a' is strictly before + // the live range of value 'b' using the given HLO ordering. + bool LiveRangeStrictlyBefore(const HloValue& a, const HloValue& b, + const HloDataflowAnalysis& dataflow) const; + + // Returns the sequential instruction order for the given computation, or + // nullptr if the computation does not have a sequential ordering. + virtual const std::vector* SequentialOrder( + const HloComputation& computation) const = 0; + + // Return the call graph of the module used to compute ordering. + const CallGraph& call_graph() const { return *call_graph_; } + + virtual string ToString() const = 0; + + protected: + // Returns true if instruction 'a' executes before instruction 'b'. + // Precondition: 'a' and 'b' are in the same computation. + // + // Derived classes should implement this method for determining order of + // instructions in the same computation. ExecutesBefore() analyzes the + // callgraph and uses this method to determine ordering of instructions in + // different computations. + virtual bool ExecutesBeforeInSameComputation( + const HloInstruction* a, const HloInstruction* b) const = 0; + + const HloModule* module_; + + std::unique_ptr call_graph_; +}; + +// Base class for partial orderings implemented by a map of predecessors for +// each instruction. Subclasses should fill in predecessors_. +class PredecessorHloOrdering : public HloOrdering { + public: + ~PredecessorHloOrdering() override = default; + + // Returns nullptr indicating the computation does not have a sequential + // ordering. + const std::vector* SequentialOrder( + const HloComputation& computation) const override { + return nullptr; + } + + HloReachabilityMap& reachability_map(const HloComputation* computation) { + return *predecessors_.at(computation); + } + const HloReachabilityMap& reachability_map( + const HloComputation* computation) const { + return *predecessors_.at(computation); + } + + protected: + explicit PredecessorHloOrdering(const HloModule* module); + string ToStringHelper(const string& name) const; + + bool ExecutesBeforeInSameComputation(const HloInstruction* a, + const HloInstruction* b) const override; + + // For each computation in the module, this is the set of the instruction's + // predecessors. An instruction is an element of its own predecessor set. + // + // Subclasses should fill this in to define the desired ordering. + absl::flat_hash_map> + predecessors_; +}; + +// An HLO ordering based on data dependencies in the HLO graph. In this partial +// order, instruction A executes before instruction B only if there is a path +// from A to B in the HLO graph. For example, given the following graph: +/* + param + / \ + negate exp + \ / + add +*/ +// DependencyHloOrdering gives the following executes-before relations: +// param executes before negate, exp, and add +// negate executes before add +// exp executes before add +// add executes before nothing +// negate and exp are not ordered because the dependencies allow either to +// execute before the other (or in parallel). DependencyHloOrdering ordering +// allows maximum parallelism and enables any execution order which satisfies +// data dependencies. This requires pessimistic assumptions about buffer live +// ranges and can result in more memory used than more constrained orderings. +class DependencyHloOrdering : public PredecessorHloOrdering { + public: + explicit DependencyHloOrdering(const HloModule* module); + ~DependencyHloOrdering() override = default; + + string ToString() const override; +}; + +// An HLO ordering based on a total order of instructions in each computation. +// The computation total order is a sequencing of all of its instructions in +// the computation (eg, {inst0, inst1, inst2,...}) as in single-threaded +// execution. For example, given the following HLO graph: +/* + param + / \ + negate exp + \ / + add +*/ +// and the following sequence: +// +// {param, negate, exp, add} +// +// SequentialHloOrdering gives the following executes-before relations: +// param executes before negate, exp, and add +// negate executes before exp and add +// exp executes before add +// add executes before nothing +// This is more constrained than DependencyHloOrdering in this example because +// negate and exp are ordered (negate before exp). This enables param to share +// the same buffer as exp (param buffer is dead after exp). Generally, this +// ordering enables more buffer sharing (reduced memory usage) because buffer +// interference is reduced relative to DependencyHloOrdering. +class SequentialHloOrdering : public HloOrdering { + public: + SequentialHloOrdering(const HloSchedule& schedule); + SequentialHloOrdering(HloSchedule&& schedule); + ~SequentialHloOrdering() override = default; + + // Returns the sequential instruction order for the given computation. + const std::vector* SequentialOrder( + const HloComputation& computation) const override; + + string ToString() const override; + + protected: + void Initialize(); + + bool ExecutesBeforeInSameComputation(const HloInstruction* a, + const HloInstruction* b) const override; + + const HloSchedule schedule_; + + // The position of every instruction in the HLO module in its respective + // computation sequence (a value of zero indicates the instruction is first in + // the sequence, etc). Instructions from all computations are contained in + // this map so more than one instruction may have the same position + // value. This is not a problem because ExecutesBefore also verifies + // instructions are in the same computation. + absl::flat_hash_map order_position_; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_HLO_ORDERING_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/hlo_pass_fix.h b/linux/include/tensorflow/compiler/xla/service/hlo_pass_fix.h new file mode 100644 index 000000000..791b1a97b --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/hlo_pass_fix.h @@ -0,0 +1,59 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_HLO_PASS_FIX_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_HLO_PASS_FIX_H_ + +#include + +#include "tensorflow/compiler/xla/service/hlo_module.h" +#include "tensorflow/compiler/xla/status_macros.h" +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/core/platform/macros.h" + +namespace xla { + +// Do an HLO pass to a fix point. +template +class HloPassFix : public Pass { + public: + template + explicit HloPassFix(Args&&... args) : Pass(args...) {} + + StatusOr Run(HloModule* module) override { + bool changed = false; + bool changed_this_iteration = true; + int64 iteration_count = 0; + int64 limit = + std::max(static_cast(1000), module->instruction_count()); + while (changed_this_iteration) { + TF_ASSIGN_OR_RETURN(changed_this_iteration, Pass::Run(module)); + changed |= changed_this_iteration; + ++iteration_count; + if (iteration_count == limit) { + LOG(ERROR) + << "Unexpectedly high number of iterations in HLO passes (" + << iteration_count + << ")\nIf compilation hangs here, please file a bug with XLA."; + } + } + return changed; + } +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_HLO_PASS_FIX_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/hlo_pass_interface.h b/linux/include/tensorflow/compiler/xla/service/hlo_pass_interface.h new file mode 100644 index 000000000..fdaac3438 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/hlo_pass_interface.h @@ -0,0 +1,72 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_HLO_PASS_INTERFACE_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_HLO_PASS_INTERFACE_H_ + +#include "tensorflow/compiler/xla/service/hlo_module.h" +#include "tensorflow/compiler/xla/service/hlo_module_group.h" +#include "tensorflow/compiler/xla/status_macros.h" +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/core/platform/macros.h" + +namespace xla { + +// Base class for HLO passes. These are used with the HloPassPipeline to +// organize a sequence of passes. An HLO pass should not extend this class +// directly; it should extend HloModulePass or HloModuleGroupPass. +class HloPassInterface { + public: + virtual ~HloPassInterface() = default; + virtual absl::string_view name() const = 0; + + // Run the pass on the given HLO module. Returns whether it modified the + // module. + virtual StatusOr Run(HloModule* module) = 0; + + // Run the pass on the given HLO module group. Returns whether it modified the + // module group. Ideally, the module group variant would be named "Run" as + // well, but C++ does not handle overloaded virtual methods well. + virtual StatusOr RunOnModuleGroup(HloModuleGroup* module_group) = 0; +}; + +// Base class for passes which are module-scoped. +class HloModulePass : public HloPassInterface { + public: + // Runs the pass on a module group by iterating through each module in the + // group. + StatusOr RunOnModuleGroup(HloModuleGroup* module_group) override { + bool changed = false; + for (HloModule* module : module_group->modules()) { + TF_ASSIGN_OR_RETURN(bool module_changed, Run(module)); + changed |= module_changed; + } + return changed; + }; +}; + +// Base class for passes which are module-group scoped. These passes cannot run +// on an HLO module. +class HloModuleGroupPass : public HloPassInterface { + public: + StatusOr Run(HloModule* module) override { + return InternalError("Module group pass cannot be run on a module"); + } +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_HLO_PASS_INTERFACE_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/hlo_pass_pipeline.h b/linux/include/tensorflow/compiler/xla/service/hlo_pass_pipeline.h new file mode 100644 index 000000000..09e7033ea --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/hlo_pass_pipeline.h @@ -0,0 +1,114 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_HLO_PASS_PIPELINE_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_HLO_PASS_PIPELINE_H_ + +#include +#include +#include +#include + +#include "absl/memory/memory.h" +#include "absl/strings/str_cat.h" +#include "tensorflow/compiler/xla/service/hlo_module.h" +#include "tensorflow/compiler/xla/service/hlo_pass_interface.h" +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/core/platform/macros.h" + +namespace xla { + +// Pipeline of HLO passes. +class HloPassPipeline : public HloPassInterface { + public: + explicit HloPassPipeline(const string& name) : name_(name) {} + absl::string_view name() const override { return name_; } + + // Add a pass to the pipeline. It should be called with the arguments for the + // pass constructor: + // + // pipeline.AddPass(constructor_arg1, constructor_arg2); + // + // Returns a reference to the added pass. + template + T& AddPass(Args&&... args) { + CHECK(!run_called_) << "AddPass cannot be called after Run"; + auto pass = new T(std::forward(args)...); + passes_.push_back(std::unique_ptr(pass)); + return *pass; + } + + // Add an invariant-checking pass to the pipeline. It will be run before and + // after each HLO pass. The invariant checking pass must not mutate the graph + // (it is required to always return "false" from its Run() method). + template + T& AddInvariantChecker(Args&&... args) { + CHECK(!run_called_) << "AddInvariantChecker cannot be called after Run"; + auto pass = new T(std::forward(args)...); + invariant_checkers_.push_back(std::unique_ptr(pass)); + return *pass; + } + + StatusOr Run(HloModule* module) override; + StatusOr RunOnModuleGroup(HloModuleGroup* module_group) override; + + private: + // Returns the set of passes which are enabled. DebugOptions can selectively + // disable passes via --xla_disable_hlo_passes flag. + std::vector GetEnabledPasses( + const DebugOptions& debug_options); + + // Maybe dumps the given module or module group depending on flag values + // contained in DebugOptions of module config. + void MaybeDumpHlo(const HloModuleGroup& module_group, + absl::string_view after_pass_name, + absl::string_view before_pass_name); + void MaybeDumpHlo(const HloModule& module, absl::string_view after_pass_name, + absl::string_view before_pass_name); + + // Runs the invariant checker on the given HLO. HloT can be either HloModule + // or HloModuleGroup. + template + Status RunInvariantCheckers(HloT* hlo, absl::string_view after_pass_name); + + // Helper which runs the given pass on the given HLO. HloT can be either + // HloModule or HloModuleGroup. + template + StatusOr RunPassesInternal(HloT* hlo, + absl::Span passes); + + // Helpers which run the given passes on the given HLO construct. These + // helpers enable templating of the core of the pipeline logic by providing + // HloModule and HloModuleGroup specific methods with the same name. + static StatusOr RunHelper(HloPassInterface* pass, HloModule* module) { + return pass->Run(module); + } + static StatusOr RunHelper(HloPassInterface* pass, + HloModuleGroup* module_group) { + return pass->RunOnModuleGroup(module_group); + } + + const string name_; + std::vector> passes_; + std::vector> invariant_checkers_; + bool run_called_ = false; + + TF_DISALLOW_COPY_AND_ASSIGN(HloPassPipeline); +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_HLO_PASS_PIPELINE_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/hlo_profile_printer.h b/linux/include/tensorflow/compiler/xla/service/hlo_profile_printer.h new file mode 100644 index 000000000..b72325c75 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/hlo_profile_printer.h @@ -0,0 +1,32 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_HLO_PROFILE_PRINTER_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_HLO_PROFILE_PRINTER_H_ + +#include +#include +#include + +#include "tensorflow/compiler/xla/service/hlo_profile_printer_data.pb.h" +#include "tensorflow/compiler/xla/types.h" + +namespace xla { +// Pretty-print an array of profile counters using hlo_profile_printer_data. +string PrintHloProfile(const HloProfilePrinterData& hlo_profile_printer_data, + const int64* counters, double clock_rate_ghz); +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_HLO_PROFILE_PRINTER_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/hlo_profile_printer_data.pb.h b/linux/include/tensorflow/compiler/xla/service/hlo_profile_printer_data.pb.h new file mode 100644 index 000000000..d7e2bb533 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/hlo_profile_printer_data.pb.h @@ -0,0 +1,1066 @@ +// Generated by the protocol buffer compiler. DO NOT EDIT! +// source: tensorflow/compiler/xla/service/hlo_profile_printer_data.proto + +#ifndef PROTOBUF_INCLUDED_tensorflow_2fcompiler_2fxla_2fservice_2fhlo_5fprofile_5fprinter_5fdata_2eproto +#define PROTOBUF_INCLUDED_tensorflow_2fcompiler_2fxla_2fservice_2fhlo_5fprofile_5fprinter_5fdata_2eproto + +#include + +#include + +#if GOOGLE_PROTOBUF_VERSION < 3006000 +#error This file was generated by a newer version of protoc which is +#error incompatible with your Protocol Buffer headers. Please update +#error your headers. +#endif +#if 3006000 < GOOGLE_PROTOBUF_MIN_PROTOC_VERSION +#error This file was generated by an older version of protoc which is +#error incompatible with your Protocol Buffer headers. Please +#error regenerate this file with a newer version of protoc. +#endif + +#include +#include +#include +#include +#include +#include +#include +#include +#include // IWYU pragma: export +#include // IWYU pragma: export +#include +// @@protoc_insertion_point(includes) +#define PROTOBUF_INTERNAL_EXPORT_protobuf_tensorflow_2fcompiler_2fxla_2fservice_2fhlo_5fprofile_5fprinter_5fdata_2eproto + +namespace protobuf_tensorflow_2fcompiler_2fxla_2fservice_2fhlo_5fprofile_5fprinter_5fdata_2eproto { +// Internal implementation detail -- do not use these members. +struct TableStruct { + static const ::google::protobuf::internal::ParseTableField entries[]; + static const ::google::protobuf::internal::AuxillaryParseTableField aux[]; + static const ::google::protobuf::internal::ParseTable schema[3]; + static const ::google::protobuf::internal::FieldMetadata field_metadata[]; + static const ::google::protobuf::internal::SerializationTable serialization_table[]; + static const ::google::protobuf::uint32 offsets[]; +}; +void AddDescriptors(); +} // namespace protobuf_tensorflow_2fcompiler_2fxla_2fservice_2fhlo_5fprofile_5fprinter_5fdata_2eproto +namespace xla { +class HloProfilePrinterData; +class HloProfilePrinterDataDefaultTypeInternal; +extern HloProfilePrinterDataDefaultTypeInternal _HloProfilePrinterData_default_instance_; +class HloProfilePrinterData_HloComputationInfo; +class HloProfilePrinterData_HloComputationInfoDefaultTypeInternal; +extern HloProfilePrinterData_HloComputationInfoDefaultTypeInternal _HloProfilePrinterData_HloComputationInfo_default_instance_; +class HloProfilePrinterData_HloInstructionInfo; +class HloProfilePrinterData_HloInstructionInfoDefaultTypeInternal; +extern HloProfilePrinterData_HloInstructionInfoDefaultTypeInternal _HloProfilePrinterData_HloInstructionInfo_default_instance_; +} // namespace xla +namespace google { +namespace protobuf { +template<> ::xla::HloProfilePrinterData* Arena::CreateMaybeMessage<::xla::HloProfilePrinterData>(Arena*); +template<> ::xla::HloProfilePrinterData_HloComputationInfo* Arena::CreateMaybeMessage<::xla::HloProfilePrinterData_HloComputationInfo>(Arena*); +template<> ::xla::HloProfilePrinterData_HloInstructionInfo* Arena::CreateMaybeMessage<::xla::HloProfilePrinterData_HloInstructionInfo>(Arena*); +} // namespace protobuf +} // namespace google +namespace xla { + +// =================================================================== + +class HloProfilePrinterData_HloInstructionInfo : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.HloProfilePrinterData.HloInstructionInfo) */ { + public: + HloProfilePrinterData_HloInstructionInfo(); + virtual ~HloProfilePrinterData_HloInstructionInfo(); + + HloProfilePrinterData_HloInstructionInfo(const HloProfilePrinterData_HloInstructionInfo& from); + + inline HloProfilePrinterData_HloInstructionInfo& operator=(const HloProfilePrinterData_HloInstructionInfo& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + HloProfilePrinterData_HloInstructionInfo(HloProfilePrinterData_HloInstructionInfo&& from) noexcept + : HloProfilePrinterData_HloInstructionInfo() { + *this = ::std::move(from); + } + + inline HloProfilePrinterData_HloInstructionInfo& operator=(HloProfilePrinterData_HloInstructionInfo&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const HloProfilePrinterData_HloInstructionInfo& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const HloProfilePrinterData_HloInstructionInfo* internal_default_instance() { + return reinterpret_cast( + &_HloProfilePrinterData_HloInstructionInfo_default_instance_); + } + static constexpr int kIndexInFileMessages = + 0; + + void UnsafeArenaSwap(HloProfilePrinterData_HloInstructionInfo* other); + void Swap(HloProfilePrinterData_HloInstructionInfo* other); + friend void swap(HloProfilePrinterData_HloInstructionInfo& a, HloProfilePrinterData_HloInstructionInfo& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline HloProfilePrinterData_HloInstructionInfo* New() const final { + return CreateMaybeMessage(NULL); + } + + HloProfilePrinterData_HloInstructionInfo* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const HloProfilePrinterData_HloInstructionInfo& from); + void MergeFrom(const HloProfilePrinterData_HloInstructionInfo& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(HloProfilePrinterData_HloInstructionInfo* other); + protected: + explicit HloProfilePrinterData_HloInstructionInfo(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // string long_name = 1; + void clear_long_name(); + static const int kLongNameFieldNumber = 1; + const ::std::string& long_name() const; + void set_long_name(const ::std::string& value); + #if LANG_CXX11 + void set_long_name(::std::string&& value); + #endif + void set_long_name(const char* value); + void set_long_name(const char* value, size_t size); + ::std::string* mutable_long_name(); + ::std::string* release_long_name(); + void set_allocated_long_name(::std::string* long_name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_long_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_long_name( + ::std::string* long_name); + + // string short_name = 2; + void clear_short_name(); + static const int kShortNameFieldNumber = 2; + const ::std::string& short_name() const; + void set_short_name(const ::std::string& value); + #if LANG_CXX11 + void set_short_name(::std::string&& value); + #endif + void set_short_name(const char* value); + void set_short_name(const char* value, size_t size); + ::std::string* mutable_short_name(); + ::std::string* release_short_name(); + void set_allocated_short_name(::std::string* short_name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_short_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_short_name( + ::std::string* short_name); + + // string category = 3; + void clear_category(); + static const int kCategoryFieldNumber = 3; + const ::std::string& category() const; + void set_category(const ::std::string& value); + #if LANG_CXX11 + void set_category(::std::string&& value); + #endif + void set_category(const char* value); + void set_category(const char* value, size_t size); + ::std::string* mutable_category(); + ::std::string* release_category(); + void set_allocated_category(::std::string* category); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_category(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_category( + ::std::string* category); + + // float flop_count = 4; + void clear_flop_count(); + static const int kFlopCountFieldNumber = 4; + float flop_count() const; + void set_flop_count(float value); + + // float transcendental_count = 5; + void clear_transcendental_count(); + static const int kTranscendentalCountFieldNumber = 5; + float transcendental_count() const; + void set_transcendental_count(float value); + + // float bytes_accessed = 6; + void clear_bytes_accessed(); + static const int kBytesAccessedFieldNumber = 6; + float bytes_accessed() const; + void set_bytes_accessed(float value); + + // float optimal_seconds = 7; + void clear_optimal_seconds(); + static const int kOptimalSecondsFieldNumber = 7; + float optimal_seconds() const; + void set_optimal_seconds(float value); + + // int64 profile_index = 8; + void clear_profile_index(); + static const int kProfileIndexFieldNumber = 8; + ::google::protobuf::int64 profile_index() const; + void set_profile_index(::google::protobuf::int64 value); + + // @@protoc_insertion_point(class_scope:xla.HloProfilePrinterData.HloInstructionInfo) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::internal::ArenaStringPtr long_name_; + ::google::protobuf::internal::ArenaStringPtr short_name_; + ::google::protobuf::internal::ArenaStringPtr category_; + float flop_count_; + float transcendental_count_; + float bytes_accessed_; + float optimal_seconds_; + ::google::protobuf::int64 profile_index_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fservice_2fhlo_5fprofile_5fprinter_5fdata_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class HloProfilePrinterData_HloComputationInfo : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.HloProfilePrinterData.HloComputationInfo) */ { + public: + HloProfilePrinterData_HloComputationInfo(); + virtual ~HloProfilePrinterData_HloComputationInfo(); + + HloProfilePrinterData_HloComputationInfo(const HloProfilePrinterData_HloComputationInfo& from); + + inline HloProfilePrinterData_HloComputationInfo& operator=(const HloProfilePrinterData_HloComputationInfo& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + HloProfilePrinterData_HloComputationInfo(HloProfilePrinterData_HloComputationInfo&& from) noexcept + : HloProfilePrinterData_HloComputationInfo() { + *this = ::std::move(from); + } + + inline HloProfilePrinterData_HloComputationInfo& operator=(HloProfilePrinterData_HloComputationInfo&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const HloProfilePrinterData_HloComputationInfo& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const HloProfilePrinterData_HloComputationInfo* internal_default_instance() { + return reinterpret_cast( + &_HloProfilePrinterData_HloComputationInfo_default_instance_); + } + static constexpr int kIndexInFileMessages = + 1; + + void UnsafeArenaSwap(HloProfilePrinterData_HloComputationInfo* other); + void Swap(HloProfilePrinterData_HloComputationInfo* other); + friend void swap(HloProfilePrinterData_HloComputationInfo& a, HloProfilePrinterData_HloComputationInfo& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline HloProfilePrinterData_HloComputationInfo* New() const final { + return CreateMaybeMessage(NULL); + } + + HloProfilePrinterData_HloComputationInfo* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const HloProfilePrinterData_HloComputationInfo& from); + void MergeFrom(const HloProfilePrinterData_HloComputationInfo& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(HloProfilePrinterData_HloComputationInfo* other); + protected: + explicit HloProfilePrinterData_HloComputationInfo(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated .xla.HloProfilePrinterData.HloInstructionInfo instruction_infos = 3; + int instruction_infos_size() const; + void clear_instruction_infos(); + static const int kInstructionInfosFieldNumber = 3; + ::xla::HloProfilePrinterData_HloInstructionInfo* mutable_instruction_infos(int index); + ::google::protobuf::RepeatedPtrField< ::xla::HloProfilePrinterData_HloInstructionInfo >* + mutable_instruction_infos(); + const ::xla::HloProfilePrinterData_HloInstructionInfo& instruction_infos(int index) const; + ::xla::HloProfilePrinterData_HloInstructionInfo* add_instruction_infos(); + const ::google::protobuf::RepeatedPtrField< ::xla::HloProfilePrinterData_HloInstructionInfo >& + instruction_infos() const; + + // string name = 1; + void clear_name(); + static const int kNameFieldNumber = 1; + const ::std::string& name() const; + void set_name(const ::std::string& value); + #if LANG_CXX11 + void set_name(::std::string&& value); + #endif + void set_name(const char* value); + void set_name(const char* value, size_t size); + ::std::string* mutable_name(); + ::std::string* release_name(); + void set_allocated_name(::std::string* name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_name( + ::std::string* name); + + // int64 profile_index = 2; + void clear_profile_index(); + static const int kProfileIndexFieldNumber = 2; + ::google::protobuf::int64 profile_index() const; + void set_profile_index(::google::protobuf::int64 value); + + // @@protoc_insertion_point(class_scope:xla.HloProfilePrinterData.HloComputationInfo) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedPtrField< ::xla::HloProfilePrinterData_HloInstructionInfo > instruction_infos_; + ::google::protobuf::internal::ArenaStringPtr name_; + ::google::protobuf::int64 profile_index_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fservice_2fhlo_5fprofile_5fprinter_5fdata_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class HloProfilePrinterData : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.HloProfilePrinterData) */ { + public: + HloProfilePrinterData(); + virtual ~HloProfilePrinterData(); + + HloProfilePrinterData(const HloProfilePrinterData& from); + + inline HloProfilePrinterData& operator=(const HloProfilePrinterData& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + HloProfilePrinterData(HloProfilePrinterData&& from) noexcept + : HloProfilePrinterData() { + *this = ::std::move(from); + } + + inline HloProfilePrinterData& operator=(HloProfilePrinterData&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const HloProfilePrinterData& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const HloProfilePrinterData* internal_default_instance() { + return reinterpret_cast( + &_HloProfilePrinterData_default_instance_); + } + static constexpr int kIndexInFileMessages = + 2; + + void UnsafeArenaSwap(HloProfilePrinterData* other); + void Swap(HloProfilePrinterData* other); + friend void swap(HloProfilePrinterData& a, HloProfilePrinterData& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline HloProfilePrinterData* New() const final { + return CreateMaybeMessage(NULL); + } + + HloProfilePrinterData* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const HloProfilePrinterData& from); + void MergeFrom(const HloProfilePrinterData& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(HloProfilePrinterData* other); + protected: + explicit HloProfilePrinterData(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + typedef HloProfilePrinterData_HloInstructionInfo HloInstructionInfo; + typedef HloProfilePrinterData_HloComputationInfo HloComputationInfo; + + // accessors ------------------------------------------------------- + + // repeated .xla.HloProfilePrinterData.HloComputationInfo computation_infos = 1; + int computation_infos_size() const; + void clear_computation_infos(); + static const int kComputationInfosFieldNumber = 1; + ::xla::HloProfilePrinterData_HloComputationInfo* mutable_computation_infos(int index); + ::google::protobuf::RepeatedPtrField< ::xla::HloProfilePrinterData_HloComputationInfo >* + mutable_computation_infos(); + const ::xla::HloProfilePrinterData_HloComputationInfo& computation_infos(int index) const; + ::xla::HloProfilePrinterData_HloComputationInfo* add_computation_infos(); + const ::google::protobuf::RepeatedPtrField< ::xla::HloProfilePrinterData_HloComputationInfo >& + computation_infos() const; + + // int64 profile_counters_size = 2; + void clear_profile_counters_size(); + static const int kProfileCountersSizeFieldNumber = 2; + ::google::protobuf::int64 profile_counters_size() const; + void set_profile_counters_size(::google::protobuf::int64 value); + + // @@protoc_insertion_point(class_scope:xla.HloProfilePrinterData) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedPtrField< ::xla::HloProfilePrinterData_HloComputationInfo > computation_infos_; + ::google::protobuf::int64 profile_counters_size_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fservice_2fhlo_5fprofile_5fprinter_5fdata_2eproto::TableStruct; +}; +// =================================================================== + + +// =================================================================== + +#ifdef __GNUC__ + #pragma GCC diagnostic push + #pragma GCC diagnostic ignored "-Wstrict-aliasing" +#endif // __GNUC__ +// HloProfilePrinterData_HloInstructionInfo + +// string long_name = 1; +inline void HloProfilePrinterData_HloInstructionInfo::clear_long_name() { + long_name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& HloProfilePrinterData_HloInstructionInfo::long_name() const { + // @@protoc_insertion_point(field_get:xla.HloProfilePrinterData.HloInstructionInfo.long_name) + return long_name_.Get(); +} +inline void HloProfilePrinterData_HloInstructionInfo::set_long_name(const ::std::string& value) { + + long_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:xla.HloProfilePrinterData.HloInstructionInfo.long_name) +} +#if LANG_CXX11 +inline void HloProfilePrinterData_HloInstructionInfo::set_long_name(::std::string&& value) { + + long_name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:xla.HloProfilePrinterData.HloInstructionInfo.long_name) +} +#endif +inline void HloProfilePrinterData_HloInstructionInfo::set_long_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + long_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:xla.HloProfilePrinterData.HloInstructionInfo.long_name) +} +inline void HloProfilePrinterData_HloInstructionInfo::set_long_name(const char* value, + size_t size) { + + long_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:xla.HloProfilePrinterData.HloInstructionInfo.long_name) +} +inline ::std::string* HloProfilePrinterData_HloInstructionInfo::mutable_long_name() { + + // @@protoc_insertion_point(field_mutable:xla.HloProfilePrinterData.HloInstructionInfo.long_name) + return long_name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* HloProfilePrinterData_HloInstructionInfo::release_long_name() { + // @@protoc_insertion_point(field_release:xla.HloProfilePrinterData.HloInstructionInfo.long_name) + + return long_name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void HloProfilePrinterData_HloInstructionInfo::set_allocated_long_name(::std::string* long_name) { + if (long_name != NULL) { + + } else { + + } + long_name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), long_name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:xla.HloProfilePrinterData.HloInstructionInfo.long_name) +} +inline ::std::string* HloProfilePrinterData_HloInstructionInfo::unsafe_arena_release_long_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.HloProfilePrinterData.HloInstructionInfo.long_name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return long_name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void HloProfilePrinterData_HloInstructionInfo::unsafe_arena_set_allocated_long_name( + ::std::string* long_name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (long_name != NULL) { + + } else { + + } + long_name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + long_name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:xla.HloProfilePrinterData.HloInstructionInfo.long_name) +} + +// string short_name = 2; +inline void HloProfilePrinterData_HloInstructionInfo::clear_short_name() { + short_name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& HloProfilePrinterData_HloInstructionInfo::short_name() const { + // @@protoc_insertion_point(field_get:xla.HloProfilePrinterData.HloInstructionInfo.short_name) + return short_name_.Get(); +} +inline void HloProfilePrinterData_HloInstructionInfo::set_short_name(const ::std::string& value) { + + short_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:xla.HloProfilePrinterData.HloInstructionInfo.short_name) +} +#if LANG_CXX11 +inline void HloProfilePrinterData_HloInstructionInfo::set_short_name(::std::string&& value) { + + short_name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:xla.HloProfilePrinterData.HloInstructionInfo.short_name) +} +#endif +inline void HloProfilePrinterData_HloInstructionInfo::set_short_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + short_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:xla.HloProfilePrinterData.HloInstructionInfo.short_name) +} +inline void HloProfilePrinterData_HloInstructionInfo::set_short_name(const char* value, + size_t size) { + + short_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:xla.HloProfilePrinterData.HloInstructionInfo.short_name) +} +inline ::std::string* HloProfilePrinterData_HloInstructionInfo::mutable_short_name() { + + // @@protoc_insertion_point(field_mutable:xla.HloProfilePrinterData.HloInstructionInfo.short_name) + return short_name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* HloProfilePrinterData_HloInstructionInfo::release_short_name() { + // @@protoc_insertion_point(field_release:xla.HloProfilePrinterData.HloInstructionInfo.short_name) + + return short_name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void HloProfilePrinterData_HloInstructionInfo::set_allocated_short_name(::std::string* short_name) { + if (short_name != NULL) { + + } else { + + } + short_name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), short_name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:xla.HloProfilePrinterData.HloInstructionInfo.short_name) +} +inline ::std::string* HloProfilePrinterData_HloInstructionInfo::unsafe_arena_release_short_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.HloProfilePrinterData.HloInstructionInfo.short_name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return short_name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void HloProfilePrinterData_HloInstructionInfo::unsafe_arena_set_allocated_short_name( + ::std::string* short_name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (short_name != NULL) { + + } else { + + } + short_name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + short_name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:xla.HloProfilePrinterData.HloInstructionInfo.short_name) +} + +// string category = 3; +inline void HloProfilePrinterData_HloInstructionInfo::clear_category() { + category_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& HloProfilePrinterData_HloInstructionInfo::category() const { + // @@protoc_insertion_point(field_get:xla.HloProfilePrinterData.HloInstructionInfo.category) + return category_.Get(); +} +inline void HloProfilePrinterData_HloInstructionInfo::set_category(const ::std::string& value) { + + category_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:xla.HloProfilePrinterData.HloInstructionInfo.category) +} +#if LANG_CXX11 +inline void HloProfilePrinterData_HloInstructionInfo::set_category(::std::string&& value) { + + category_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:xla.HloProfilePrinterData.HloInstructionInfo.category) +} +#endif +inline void HloProfilePrinterData_HloInstructionInfo::set_category(const char* value) { + GOOGLE_DCHECK(value != NULL); + + category_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:xla.HloProfilePrinterData.HloInstructionInfo.category) +} +inline void HloProfilePrinterData_HloInstructionInfo::set_category(const char* value, + size_t size) { + + category_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:xla.HloProfilePrinterData.HloInstructionInfo.category) +} +inline ::std::string* HloProfilePrinterData_HloInstructionInfo::mutable_category() { + + // @@protoc_insertion_point(field_mutable:xla.HloProfilePrinterData.HloInstructionInfo.category) + return category_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* HloProfilePrinterData_HloInstructionInfo::release_category() { + // @@protoc_insertion_point(field_release:xla.HloProfilePrinterData.HloInstructionInfo.category) + + return category_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void HloProfilePrinterData_HloInstructionInfo::set_allocated_category(::std::string* category) { + if (category != NULL) { + + } else { + + } + category_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), category, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:xla.HloProfilePrinterData.HloInstructionInfo.category) +} +inline ::std::string* HloProfilePrinterData_HloInstructionInfo::unsafe_arena_release_category() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.HloProfilePrinterData.HloInstructionInfo.category) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return category_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void HloProfilePrinterData_HloInstructionInfo::unsafe_arena_set_allocated_category( + ::std::string* category) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (category != NULL) { + + } else { + + } + category_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + category, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:xla.HloProfilePrinterData.HloInstructionInfo.category) +} + +// float flop_count = 4; +inline void HloProfilePrinterData_HloInstructionInfo::clear_flop_count() { + flop_count_ = 0; +} +inline float HloProfilePrinterData_HloInstructionInfo::flop_count() const { + // @@protoc_insertion_point(field_get:xla.HloProfilePrinterData.HloInstructionInfo.flop_count) + return flop_count_; +} +inline void HloProfilePrinterData_HloInstructionInfo::set_flop_count(float value) { + + flop_count_ = value; + // @@protoc_insertion_point(field_set:xla.HloProfilePrinterData.HloInstructionInfo.flop_count) +} + +// float transcendental_count = 5; +inline void HloProfilePrinterData_HloInstructionInfo::clear_transcendental_count() { + transcendental_count_ = 0; +} +inline float HloProfilePrinterData_HloInstructionInfo::transcendental_count() const { + // @@protoc_insertion_point(field_get:xla.HloProfilePrinterData.HloInstructionInfo.transcendental_count) + return transcendental_count_; +} +inline void HloProfilePrinterData_HloInstructionInfo::set_transcendental_count(float value) { + + transcendental_count_ = value; + // @@protoc_insertion_point(field_set:xla.HloProfilePrinterData.HloInstructionInfo.transcendental_count) +} + +// float bytes_accessed = 6; +inline void HloProfilePrinterData_HloInstructionInfo::clear_bytes_accessed() { + bytes_accessed_ = 0; +} +inline float HloProfilePrinterData_HloInstructionInfo::bytes_accessed() const { + // @@protoc_insertion_point(field_get:xla.HloProfilePrinterData.HloInstructionInfo.bytes_accessed) + return bytes_accessed_; +} +inline void HloProfilePrinterData_HloInstructionInfo::set_bytes_accessed(float value) { + + bytes_accessed_ = value; + // @@protoc_insertion_point(field_set:xla.HloProfilePrinterData.HloInstructionInfo.bytes_accessed) +} + +// float optimal_seconds = 7; +inline void HloProfilePrinterData_HloInstructionInfo::clear_optimal_seconds() { + optimal_seconds_ = 0; +} +inline float HloProfilePrinterData_HloInstructionInfo::optimal_seconds() const { + // @@protoc_insertion_point(field_get:xla.HloProfilePrinterData.HloInstructionInfo.optimal_seconds) + return optimal_seconds_; +} +inline void HloProfilePrinterData_HloInstructionInfo::set_optimal_seconds(float value) { + + optimal_seconds_ = value; + // @@protoc_insertion_point(field_set:xla.HloProfilePrinterData.HloInstructionInfo.optimal_seconds) +} + +// int64 profile_index = 8; +inline void HloProfilePrinterData_HloInstructionInfo::clear_profile_index() { + profile_index_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 HloProfilePrinterData_HloInstructionInfo::profile_index() const { + // @@protoc_insertion_point(field_get:xla.HloProfilePrinterData.HloInstructionInfo.profile_index) + return profile_index_; +} +inline void HloProfilePrinterData_HloInstructionInfo::set_profile_index(::google::protobuf::int64 value) { + + profile_index_ = value; + // @@protoc_insertion_point(field_set:xla.HloProfilePrinterData.HloInstructionInfo.profile_index) +} + +// ------------------------------------------------------------------- + +// HloProfilePrinterData_HloComputationInfo + +// string name = 1; +inline void HloProfilePrinterData_HloComputationInfo::clear_name() { + name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& HloProfilePrinterData_HloComputationInfo::name() const { + // @@protoc_insertion_point(field_get:xla.HloProfilePrinterData.HloComputationInfo.name) + return name_.Get(); +} +inline void HloProfilePrinterData_HloComputationInfo::set_name(const ::std::string& value) { + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:xla.HloProfilePrinterData.HloComputationInfo.name) +} +#if LANG_CXX11 +inline void HloProfilePrinterData_HloComputationInfo::set_name(::std::string&& value) { + + name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:xla.HloProfilePrinterData.HloComputationInfo.name) +} +#endif +inline void HloProfilePrinterData_HloComputationInfo::set_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:xla.HloProfilePrinterData.HloComputationInfo.name) +} +inline void HloProfilePrinterData_HloComputationInfo::set_name(const char* value, + size_t size) { + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:xla.HloProfilePrinterData.HloComputationInfo.name) +} +inline ::std::string* HloProfilePrinterData_HloComputationInfo::mutable_name() { + + // @@protoc_insertion_point(field_mutable:xla.HloProfilePrinterData.HloComputationInfo.name) + return name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* HloProfilePrinterData_HloComputationInfo::release_name() { + // @@protoc_insertion_point(field_release:xla.HloProfilePrinterData.HloComputationInfo.name) + + return name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void HloProfilePrinterData_HloComputationInfo::set_allocated_name(::std::string* name) { + if (name != NULL) { + + } else { + + } + name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:xla.HloProfilePrinterData.HloComputationInfo.name) +} +inline ::std::string* HloProfilePrinterData_HloComputationInfo::unsafe_arena_release_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.HloProfilePrinterData.HloComputationInfo.name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void HloProfilePrinterData_HloComputationInfo::unsafe_arena_set_allocated_name( + ::std::string* name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (name != NULL) { + + } else { + + } + name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:xla.HloProfilePrinterData.HloComputationInfo.name) +} + +// int64 profile_index = 2; +inline void HloProfilePrinterData_HloComputationInfo::clear_profile_index() { + profile_index_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 HloProfilePrinterData_HloComputationInfo::profile_index() const { + // @@protoc_insertion_point(field_get:xla.HloProfilePrinterData.HloComputationInfo.profile_index) + return profile_index_; +} +inline void HloProfilePrinterData_HloComputationInfo::set_profile_index(::google::protobuf::int64 value) { + + profile_index_ = value; + // @@protoc_insertion_point(field_set:xla.HloProfilePrinterData.HloComputationInfo.profile_index) +} + +// repeated .xla.HloProfilePrinterData.HloInstructionInfo instruction_infos = 3; +inline int HloProfilePrinterData_HloComputationInfo::instruction_infos_size() const { + return instruction_infos_.size(); +} +inline void HloProfilePrinterData_HloComputationInfo::clear_instruction_infos() { + instruction_infos_.Clear(); +} +inline ::xla::HloProfilePrinterData_HloInstructionInfo* HloProfilePrinterData_HloComputationInfo::mutable_instruction_infos(int index) { + // @@protoc_insertion_point(field_mutable:xla.HloProfilePrinterData.HloComputationInfo.instruction_infos) + return instruction_infos_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::xla::HloProfilePrinterData_HloInstructionInfo >* +HloProfilePrinterData_HloComputationInfo::mutable_instruction_infos() { + // @@protoc_insertion_point(field_mutable_list:xla.HloProfilePrinterData.HloComputationInfo.instruction_infos) + return &instruction_infos_; +} +inline const ::xla::HloProfilePrinterData_HloInstructionInfo& HloProfilePrinterData_HloComputationInfo::instruction_infos(int index) const { + // @@protoc_insertion_point(field_get:xla.HloProfilePrinterData.HloComputationInfo.instruction_infos) + return instruction_infos_.Get(index); +} +inline ::xla::HloProfilePrinterData_HloInstructionInfo* HloProfilePrinterData_HloComputationInfo::add_instruction_infos() { + // @@protoc_insertion_point(field_add:xla.HloProfilePrinterData.HloComputationInfo.instruction_infos) + return instruction_infos_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::xla::HloProfilePrinterData_HloInstructionInfo >& +HloProfilePrinterData_HloComputationInfo::instruction_infos() const { + // @@protoc_insertion_point(field_list:xla.HloProfilePrinterData.HloComputationInfo.instruction_infos) + return instruction_infos_; +} + +// ------------------------------------------------------------------- + +// HloProfilePrinterData + +// repeated .xla.HloProfilePrinterData.HloComputationInfo computation_infos = 1; +inline int HloProfilePrinterData::computation_infos_size() const { + return computation_infos_.size(); +} +inline void HloProfilePrinterData::clear_computation_infos() { + computation_infos_.Clear(); +} +inline ::xla::HloProfilePrinterData_HloComputationInfo* HloProfilePrinterData::mutable_computation_infos(int index) { + // @@protoc_insertion_point(field_mutable:xla.HloProfilePrinterData.computation_infos) + return computation_infos_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::xla::HloProfilePrinterData_HloComputationInfo >* +HloProfilePrinterData::mutable_computation_infos() { + // @@protoc_insertion_point(field_mutable_list:xla.HloProfilePrinterData.computation_infos) + return &computation_infos_; +} +inline const ::xla::HloProfilePrinterData_HloComputationInfo& HloProfilePrinterData::computation_infos(int index) const { + // @@protoc_insertion_point(field_get:xla.HloProfilePrinterData.computation_infos) + return computation_infos_.Get(index); +} +inline ::xla::HloProfilePrinterData_HloComputationInfo* HloProfilePrinterData::add_computation_infos() { + // @@protoc_insertion_point(field_add:xla.HloProfilePrinterData.computation_infos) + return computation_infos_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::xla::HloProfilePrinterData_HloComputationInfo >& +HloProfilePrinterData::computation_infos() const { + // @@protoc_insertion_point(field_list:xla.HloProfilePrinterData.computation_infos) + return computation_infos_; +} + +// int64 profile_counters_size = 2; +inline void HloProfilePrinterData::clear_profile_counters_size() { + profile_counters_size_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 HloProfilePrinterData::profile_counters_size() const { + // @@protoc_insertion_point(field_get:xla.HloProfilePrinterData.profile_counters_size) + return profile_counters_size_; +} +inline void HloProfilePrinterData::set_profile_counters_size(::google::protobuf::int64 value) { + + profile_counters_size_ = value; + // @@protoc_insertion_point(field_set:xla.HloProfilePrinterData.profile_counters_size) +} + +#ifdef __GNUC__ + #pragma GCC diagnostic pop +#endif // __GNUC__ +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + + +// @@protoc_insertion_point(namespace_scope) + +} // namespace xla + +// @@protoc_insertion_point(global_scope) + +#endif // PROTOBUF_INCLUDED_tensorflow_2fcompiler_2fxla_2fservice_2fhlo_5fprofile_5fprinter_5fdata_2eproto diff --git a/linux/include/tensorflow/compiler/xla/service/hlo_proto_util.h b/linux/include/tensorflow/compiler/xla/service/hlo_proto_util.h new file mode 100644 index 000000000..3d9c375cd --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/hlo_proto_util.h @@ -0,0 +1,49 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// Utilities to manipulate data in hlo.proto. + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_HLO_PROTO_UTIL_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_HLO_PROTO_UTIL_H_ + +#include + +#include "tensorflow/compiler/xla/service/buffer_assignment.h" +#include "tensorflow/compiler/xla/service/hlo.pb.h" +#include "tensorflow/compiler/xla/service/hlo_module.h" +#include "tensorflow/compiler/xla/status.h" + +namespace xla { + +// Returns a serialized representation of the HLO state. +HloProto MakeHloProto(const HloModule& module, + const BufferAssignment& assignment); + +// Returns a serialized representation of the HLO state, but buffer assignment +// will not be included in the output. +HloProto MakeHloProto(const HloModule& module); + +// Returns the shapes of the parameters of the entry computation. Shape pointers +// refer to shapes inside of the given HloProto. +StatusOr> EntryComputationParameterShapes( + const HloProto& hlo_proto); + +// Returns the shape of the output of the entry computation. The shape pointer +// refers to the output shape inside of the given HloProto. +StatusOr EntryComputationOutputShape(const HloProto& hlo_proto); + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_HLO_PROTO_UTIL_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/hlo_query.h b/linux/include/tensorflow/compiler/xla/service/hlo_query.h new file mode 100644 index 000000000..c0826a6ae --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/hlo_query.h @@ -0,0 +1,70 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_HLO_QUERY_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_HLO_QUERY_H_ + +#include "tensorflow/compiler/xla/service/hlo_instruction.h" + +namespace xla { + +// Helper interface for making queries about the HLO IR. +namespace hlo_query { + +// Returns whether the instruction provided is a constant rank-0 float32, and +// if so, places the constant value into out. +// Precondition: out != nullptr +bool IsConstantR0F32(HloInstruction* instruction, float* out); + +// Returns whether all of an instruction's operands are of the types constants +// and parameters. +bool AllOperandsAreParametersOrConstants(const HloInstruction& instruction); + +// Returns whether all of an instruction's operands are parameters. +bool AllOperandsAreParameters(const HloInstruction& instruction); + +// Returns whether all of an instruction's operands are constants. +bool AllOperandsAreConstants(const HloInstruction& instruction); + +// Returns whether the instruction is a scalar constant. +bool IsScalarConstant(const HloInstruction* instruction); + +// Returns an operand of an instruction with the given opcode. If there are +// multiple matching operands, then the first matching operand is returned. If +// there are no matching operands then nullptr is returned. +HloInstruction* GetMatchingOperand( + const std::function& matcher, + HloInstruction* instruction); + +// Returns whether a binary instruction has a matching operand. Sets +// matching_operand to the matching operand and the other operand to +// other_operand. Note: in the case where both operands match, the first operand +// of the instruction is returned. +bool MatchBinaryInstructionOperand( + const std::function& matcher, + HloInstruction* instruction, HloInstruction** matching_operand, + HloInstruction** other_operand); + +// Returns whether a binary instruction has a operand with a given opcode. +// This is a special case of MatchingBinaryInstructionOperand. +bool MatchBinaryInstructionOperandOpcode(HloOpcode opcode, + HloInstruction* instruction, + HloInstruction** matching_operand, + HloInstruction** other_operand); + +} // namespace hlo_query +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_HLO_QUERY_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/hlo_reachability.h b/linux/include/tensorflow/compiler/xla/service/hlo_reachability.h new file mode 100644 index 000000000..5a5f01f8f --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/hlo_reachability.h @@ -0,0 +1,170 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_HLO_REACHABILITY_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_HLO_REACHABILITY_H_ + +#include +#include + +#include "absl/container/flat_hash_map.h" +#include "absl/types/span.h" +#include "tensorflow/compiler/xla/map_util.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/platform/types.h" + +namespace xla { + +class HloInstruction; + +// A class for representing reachability between HloInstructions. +// +// !!! THIS CLASS DOES NOT COMPUTE REACHABILITY !!! It has an adjacency matrix +// and it is up to the user of the class to set the adjacency matrix such that +// it represents reachability, i.e. such that it is transitive. That the graph +// be transitive is thus not an invariant of this class, but it is required for +// the name of the class and its methods to make sense. +class HloReachabilityMap { + public: + // Sets up a graph with no edges and where the nodes correspond to the given + // instructions. + explicit HloReachabilityMap( + absl::Span instructions); + + // Set the reachability set of 'instruction' to the union of the reachability + // sets of 'inputs'. Upon return, IsReachable(x, instruction) where + // 'x' is not 'instruction' will return true iff IsReachable(x, input) is true + // for some 'input' in 'inputs'. Also sets 'instruction' to be reachable from + // itself. Returns whether the reachability set of 'instruction' changed. + // + // !!! THIS FUNCTION DOES NOT COMPUTE REACHABILITY !!! It sets the adjacency + // vector in the internal graph of this HloReachabilityMap for the given + // instruction and does not transitively update any other part of the + // adjacency matrix. + bool SetReachabilityToUnion(absl::Span inputs, + const HloInstruction* instruction); + + // As above, but faster because it does not check if the reachability changed. + void FastSetReachabilityToUnion( + absl::Span inputs, + const HloInstruction* instruction); + + // Sets entry so that IsReachable(a, b) will return true + // + // !!! THIS FUNCTION DOES NOT COMPUTE REACHABILITY !!! It sets the adjacency + // matrix in the internal graph of this HloReachabilityMap to have an edge + // from a to b and does not transitively update any other part of the + // adjacency matrix. + void SetReachable(const HloInstruction* a, const HloInstruction* b); + + // Returns true if "b" is reachable from "a" + // + // Note that this function only correctly answers queries about reachability + // if the set of edges that have been provided to this class are transitive. + bool IsReachable(const HloInstruction* a, const HloInstruction* b) const; + + // Returns true if "b" is reachable from "a" or "a" is reachable from "b" + // + // Note that this function only correctly answers queries about reachability + // if the set of edges that have been provided to this class are transitive. + bool IsConnected(const HloInstruction* a, const HloInstruction* b) const; + + private: + // A bit-vector implementation specialized for this use case which provides a + // fast bitwise OR operation not available in tensorflow::gtl::BitMap. + class BitVector { + public: + BitVector() = default; + BitVector(size_t size) + : size_(size), vector_((size + kBits - 1) / kBits, 0) {} + + // Return the bit at the given index. + bool Get(size_t index) const { + DCHECK(index >= 0 && index < size_); + return vector_[index / kBits] & (1ull << (index % kBits)); + } + + // Set the bit at the given index. + void Set(size_t index) { + DCHECK(index >= 0 && index < size_); + vector_[index / kBits] |= 1ull << (index % kBits); + } + + // Set this bitvector to the Logical OR of this bitvector and 'other'. + void OrWith(const BitVector& other) { + for (size_t i = 0; i < vector_.size(); ++i) { + vector_[i] |= other.vector_[i]; + } + } + + // Set the bitvector to all zeros. + void SetToZero() { std::fill(vector_.begin(), vector_.end(), 0); } + + bool operator==(const BitVector& other) const { + return vector_ == other.vector_; + } + bool operator!=(const BitVector& other) const { + return vector_ != other.vector_; + } + + private: + using Word = uint64; + static const size_t kBits = 64; + + // Number of bits in the bitvector. + size_t size_; + + std::vector vector_; + }; + + // Return the bitvector storing the reachability-to of the given instruction. + const BitVector& GetBitVector(const HloInstruction* instruction) const { + return bit_vectors_[GetIndex(instruction)]; + } + BitVector& GetBitVector(const HloInstruction* instruction) { + return bit_vectors_[GetIndex(instruction)]; + } + + // Helper for SetReachabilityToUnion/FastSetReachabilityToUnion. + void SetReachabilityToUnionHelper( + absl::Span inputs, + const HloInstruction* instruction, BitVector* bit_vector); + + // Return the index of the given instruction. The value is used to index into + // the vector of BitVectors and the BitVectors themselves. + int GetIndex(const HloInstruction* instruction) const { + return FindOrDie(indices_, instruction); + } + + // The number of instructions in the reachability map. + const size_t size_; + + // Dense assignment from HloInstruction* to number. These numbers index + // into the bit_vectors_ vector and into the bits within a BitVector. + absl::flat_hash_map indices_; + + // Bitvectors holding the reachability to each instruction. The bit vector for + // instruction X includes ones for each instruction which X is reachable from. + std::vector bit_vectors_; + + // A temporary used by SetReachabilityToUnion to avoid an allocation with each + // call to the method. + BitVector tmp_bit_vector_; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_HLO_REACHABILITY_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/hlo_schedule.h b/linux/include/tensorflow/compiler/xla/service/hlo_schedule.h new file mode 100644 index 000000000..0a714101e --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/hlo_schedule.h @@ -0,0 +1,158 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_HLO_SCHEDULE_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_HLO_SCHEDULE_H_ + +#include + +#include "absl/container/flat_hash_map.h" +#include "absl/types/span.h" +#include "tensorflow/compiler/xla/service/hlo_computation.h" +#include "tensorflow/compiler/xla/service/hlo_instruction.h" +#include "tensorflow/compiler/xla/service/hlo_schedule.h" +#include "tensorflow/compiler/xla/status.h" + +namespace xla { + +class HloModule; + +// Class representing a sequence of HLO instructions such as the sequential +// execution order of an HLO computation. +class HloInstructionSequence { + public: + HloInstructionSequence() = default; + explicit HloInstructionSequence( + absl::Span instructions) { + for (const HloInstruction* instruction : instructions) { + push_back(instruction); + } + } + + // Adds the instruction to the end of the sequence. + void push_back(const HloInstruction* instruction) { + instruction_sequence_.push_back(instruction); + id_sequence_.push_back(instruction->unique_id()); + } + + // Clears the sequence of all instructions. + void clear() { + instruction_sequence_.clear(); + id_sequence_.clear(); + } + + int64 size() const { return instruction_sequence_.size(); } + + // Returns the sequence of HLO instructions. + const std::vector& instructions() const { + return instruction_sequence_; + } + + // Returns the unique IDs of the instructions in the sequence (in order). + const std::vector& ids() const { return id_sequence_; } + + private: + // The sequence as HloInstructions. + std::vector instruction_sequence_; + + // The sequence of HLO instructions, represented by their unique IDs. The + // sequence is stored as both HloInstructions and unique IDs because the + // sequence may be referenced after transformations to the HLO graph and HLO + // pointers can be invalidated or recycled in this process (see + // HloSchedule::Update). + std::vector id_sequence_; +}; + +// A class representing a sequential schedule of instructions for an HLO +// module. A complete HLO schedule contains an instruction sequence for every +// non-fusion computation in the HLO module. +class HloSchedule { + public: + explicit HloSchedule(const HloModule* module) : module_(module) {} + + // (De)Serialize an HloSchedule to/from a HloScheduleProto. + static StatusOr CreateFromProto(const HloModule* module, + const HloScheduleProto& proto); + StatusOr ToProto() const; + + // Returns a reference to the sequence for the given computation. + const HloInstructionSequence& sequence( + const HloComputation* computation) const; + + // Returns the sequence for the given computation. An empty sequence is + // created if none exists for the computation. + HloInstructionSequence& GetOrCreateSequence( + const HloComputation* computation); + + // Sets the sequence for the given computation to the given sequence. + void set_sequence(const HloComputation* computation, + absl::Span sequence); + void set_sequence(const HloComputation* computation, + HloInstructionSequence sequence); + + // Returns a map from HloComputation unique ID to instruction sequence. The + // map contains all sequences in the schedule. + const absl::flat_hash_map& sequences() const { + return sequences_; + } + + // Returns true if the schedule has a sequence for the given computation. + bool is_computation_scheduled(const HloComputation* computation) const { + return sequences_.count(computation->unique_id()) == 1; + } + + // Updates the schedule such that it is (again) a valid schedule for the + // module. This is used to update a schedule after the HLO module has been + // transformed in some way. In general, the only transformations to the module + // for which a schedule can be updated is the addition or removal of + // instructions and removal of computations. Updating the schedule after new + // dependencies between existing instructions in the module is not supported + // and may result in an error status returned. + // + // Instructions in the module which also exist in the given schedule will + // remain in the same order in the updated schedule. Instructions which exist + // in the module but not in the given schedule will be placed as early as + // possible in the updated schedule. + Status Update(); + + // Verifies that the given schedule is valid for the given module. + // Specifically, the schedule contains exactly the instructions in the + // non-fusion computations in the module and every dependency in the module is + // satisfied in the schedule. + Status Verify() const; + + string ToString() const; + + bool empty() const { return sequences_.empty(); } + + const HloModule* module() const { return module_; } + + private: + // Updates the instruction sequence for the given computation. + Status UpdateComputationSchedule(const HloComputation* computation); + + const HloModule* module_; + + // A map from computation unique ID to instruction sequence. Unique IDs are + // used rather than HloComputation pointers because HLO pointers are not + // unique across HLO transformations because pointers may be recycled. + absl::flat_hash_map sequences_; +}; + +std::ostream& operator<<(std::ostream& out, const HloSchedule& schedule); + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_HLO_SCHEDULE_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/hlo_sharding.h b/linux/include/tensorflow/compiler/xla/service/hlo_sharding.h new file mode 100644 index 000000000..9775505f8 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/hlo_sharding.h @@ -0,0 +1,273 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// HLO shardings describe how an HLO instruction is split across multiple +// computations. + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_HLO_SHARDING_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_HLO_SHARDING_H_ + +#include +#include +#include + +#include "absl/types/span.h" +#include "tensorflow/compiler/xla/array.h" +#include "tensorflow/compiler/xla/literal.h" +#include "tensorflow/compiler/xla/protobuf_util.h" +#include "tensorflow/compiler/xla/shape_tree.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/lib/hash/hash.h" +#include "tensorflow/core/platform/logging.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/types.h" + +namespace xla { + +// HLO shardings describe how an HLO instruction is split across multiple +// computations. +class HloSharding { + public: + // Creates a trivial sharding that replicates a maximal tile across all + // devices. + static HloSharding Replicate() { return HloSharding(); } + + // Creates a sharding that emulates device placement; a tile shape equal to + // the input shape (one tile) assigned to a single device. + static HloSharding AssignDevice(int64 device_id); + + // Creates a new sharding which splits a shape into tiles amongst the devices + // specified by `tile_assignment`. + static HloSharding Tile(const Array& tile_assignment) { + return HloSharding(tile_assignment); + } + + // Creates a new sharding which splits a one-dimensional input shape into + // `num_tiles` tiles. + static HloSharding Tile1D(const Shape& input_shape, int64 num_tiles); + + // Creates a new sharding for a tuple type. The given ShapeTree must have + // elements for every leaf shape contained in the tuple. + static HloSharding Tuple(const ShapeTree& sub_shardings); + + // Creates a new sharding for a tuple type. The number of elements in + // shardings must match the number of leaf nodes in tuple_shape. For + // empty tuples, the shardings array must have one element. + static HloSharding Tuple(const Shape& tuple_shape, + absl::Span shardings); + + // Creates a new sharding for a tuple type, with a single input sharding + // repeated on each leaf. + static HloSharding SingleTuple(const Shape& tuple_shape, + const HloSharding& sharding); + + // If shape is an array, returns sharding, otherwise returns the tuple shaped + // sharding with all the leaf nodes having the same input sharding. + static HloSharding Single(const Shape& shape, const HloSharding& sharding); + + // Create a new sharding from a protobuf OpSharding. + static StatusOr FromProto(const OpSharding& proto); + + // Checks whether device is a reserved device number. A reserved device number + // has usually a special meaning, with dedicated handling logic. + static bool IsReservedDevice(int64 device) { return device < 0; } + + OpSharding ToProto() const; + + // Note that this string canonically has outer curly braces, e.g. + // "{replicated}". + string ToString() const; + + // Validate that this sharding can be applied to a tensor with shape `shape`. + Status Validate(const Shape& shape, int64 num_devices) const; + + // Returns true if the sharding has tuple type. + bool IsTuple() const { return tuple_; } + + // Returns true if the sharding is trivial: replicate on all devices. + bool IsReplicated() const { + if (!IsTuple()) { + return replicated_; + } + return std::all_of(tuple_elements_.begin(), tuple_elements_.end(), + [](const HloSharding& s) { return s.IsReplicated(); }); + } + + // Returns true if the tile size is the same as the input size. + bool IsTileMaximal() const { + if (!IsTuple()) { + return maximal_; + } + return std::all_of(tuple_elements_.begin(), tuple_elements_.end(), + [](const HloSharding& s) { return s.IsTileMaximal(); }); + } + + // Returns true if the sharding defines an operation on the given device. + bool UsesDevice(int64 device) const; + + // Retrieves an histogram of the devices used by the sharding. The returned + // map has the device number as key, and the occurrence count as value. + // If a sharding does not have a device, it will not be incuded in the + // histogram. The count argument, if not nullptr, will receive the total + // number of elements this sharding is made of (one for array, N leaves for + // tuples). + std::map UsedDevices(int64* count) const; + + // Returns the tile that should be executed on the given device. + // REQUIRES: !IsTuple() + std::vector TileIndexForDevice(int64 device) const; + + // Returns the device that should execute the given tile. + // It is an error to call this if is_replicated() is true. + // REQUIRES: !IsTuple() + int64 DeviceForTileIndex(absl::Span index) const; + + // Given a device ID, returns the offset within the specified shape of the + // tile that should be executed on the given core. This returns the lower + // extent of the tile in the input space. + // REQUIRES: !IsTuple() + std::vector TileOffsetForDevice(const Shape& shape, + int64 device) const; + + // Given a device ID, returns the limit within the specified shape of the + // tile that should be executed on the given core. This returns the upper + // extent of the tile in the input space. + // REQUIRES: !IsTuple() + std::vector TileLimitForDevice(const Shape& shape, int64 device) const; + + // Returns the single device this op operates on. If the sharding does not + // span a single device, the return value will be empty. + // In order for a sharding to span a single device, every leaf sharding must + // be maximal and not replicated, and the used device must match. + absl::optional UniqueDevice() const; + + // Retrieves the unique device or fails with a CHECK. + int64 GetUniqueDevice() const; + + // Returns true if this op only uses a single device. + bool HasUniqueDevice() const { return UniqueDevice().has_value(); } + + // Returns the ShapeTree containing the shardings for each element of this + // tuple, if IsTuple, or a ShapeTree with a single element containing this + // sharding. Only the leaf elements are populated. This creates a new + // ShapeTree object so is not cheap. + StatusOr> AsShapeTree(const Shape& shape) const; + ShapeTree GetAsShapeTree(const Shape& shape) const { + return AsShapeTree(shape).ValueOrDie(); + } + + // Retrieves the sub sharding at a given index, out of a tuple sharding. + // REQUIRES: IsTuple() + HloSharding GetSubSharding(const Shape& shape, const ShapeIndex& index) const; + + // If the current sharding is a tuple sharding, return itself as result. + // Otherwise returns a tuple sharding for the input shape, with all the leaves + // having this object sharding. + StatusOr GetTupleSharding(const Shape& shape) const; + + // Extracts the sharding that is common within the current sharding. + // If the current sharding is not a tuple sharding, the current sharding will + // be returned. If it is a tuple, and all the tuple elements are common, the + // common element will be returned. Otherwise the optional will contain no + // value. + absl::optional ExtractSingleSharding() const; + + bool operator==(const HloSharding& other) const { + return replicated_ == other.replicated_ && maximal_ == other.maximal_ && + tile_assignment_ == other.tile_assignment_ && + tuple_elements_ == other.tuple_elements_; + } + bool operator!=(const HloSharding& other) const { return !(*this == other); } + + size_t Hash() const; + + struct Hasher { + size_t operator()(const HloSharding& sharding) const { + return sharding.Hash(); + } + }; + + // Gets the tile assignment tensor. + // REQUIRES: !IsReplicated() && !IsTuple() + const Array& tile_assignment() const { return tile_assignment_; } + + // Returns the flattened list of all the leaf shardings in a tuple shape, by + // pre-order walk (ShapeTree iterator order). + // REQUIRES: IsTuple(). + const std::vector& tuple_elements() const { + return tuple_elements_; + } + + // Gets the tile shape. + // REQUIRES: !IsTuple() + Shape TileShape(const Shape& shape) const; + + private: + HloSharding() + : replicated_(true), + maximal_(true), + tuple_(false), + tile_assignment_({0}) {} + // device_id values: + // -2: magic number to mean unassigned device, used by spatial partitioning + // -1: the id of the host + // 0 or positive: the id of a device + // NOTE(dimvar): -1 is needed for outside compilation. It can be removed once + // we have fully switched to the side-effect tokens. + explicit HloSharding(int64 device_id) + : replicated_(false), + maximal_(true), + tuple_(false), + tile_assignment_({1}, device_id) {} + explicit HloSharding(const Array& tile_assignment) + : replicated_(false), + maximal_(false), + tuple_(false), + tile_assignment_(tile_assignment) {} + explicit HloSharding(const std::vector& tuple_shardings) + : replicated_(false), + maximal_(false), + tuple_(true), + tile_assignment_({0}), + tuple_elements_(tuple_shardings) {} + + // Checks that the number of elements in tuple_elements_ is consistent with + // the tuple shape passes as argument. + Status CheckLeafCount(const Shape& shape) const; + + // Internal helper to validate a tuple sharding. + Status ValidateTuple(const Shape& shape, int64 num_devices) const; + + // Internal helper to validate a non-tuple (leaf) sharding. + Status ValidateNonTuple(const Shape& shape, int64 num_devices) const; + + // Returns the number of tuple_elements_ entries to fit the shape. + static int64 RequiredLeaves(const Shape& shape); + + bool replicated_; + bool maximal_; + bool tuple_; + Array tile_assignment_; + // Only non-empty when tuple_ is true. If a tuple is empty then one entry is + // present for the root. This is a flattened list of all the leaf shardings in + // a tuple shape, by pre-order walk (ShapeTree iterator order). + std::vector tuple_elements_; +}; + +std::ostream& operator<<(std::ostream& out, const HloSharding& sharding); + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_HLO_SHARDING_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/hlo_subcomputation_unification.h b/linux/include/tensorflow/compiler/xla/service/hlo_subcomputation_unification.h new file mode 100644 index 000000000..fa34bddde --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/hlo_subcomputation_unification.h @@ -0,0 +1,36 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_HLO_SUBCOMPUTATION_UNIFICATION_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_HLO_SUBCOMPUTATION_UNIFICATION_H_ + +#include "tensorflow/compiler/xla/service/hlo_pass_interface.h" + +namespace xla { + +// Unify subcomputations of a `HloModule`: if any computations are equal, choose +// one arbitrarily to use and delete the others. +class HloSubcomputationUnification : public HloModulePass { + public: + absl::string_view name() const override { + return "subcomputation-unification"; + } + + StatusOr Run(HloModule* module) override; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_HLO_SUBCOMPUTATION_UNIFICATION_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/hlo_tfgraph_builder.h b/linux/include/tensorflow/compiler/xla/service/hlo_tfgraph_builder.h new file mode 100644 index 000000000..c4876b852 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/hlo_tfgraph_builder.h @@ -0,0 +1,59 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_HLO_TFGRAPH_BUILDER_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_HLO_TFGRAPH_BUILDER_H_ + +#include "tensorflow/compiler/xla/service/hlo_computation.h" +#include "tensorflow/compiler/xla/xla.pb.h" +#include "tensorflow/core/framework/graph.pb.h" +#include "tensorflow/core/framework/node_def.pb.h" + +namespace xla { +namespace hlo_graph_dumper { + +// This constructs a tensorflow graph for HLO computations. +class HloTfGraphBuilder { + public: + HloTfGraphBuilder(const DebugOptions& debug_options = DebugOptions()); + + // Adds a computation to the graph. + Status AddComputation(const HloComputation& computation); + + const tensorflow::GraphDef& GetGraphDef() const; + + private: + // Gets the node name of an instruction. The node name is hierarchical. For + // example, if an instruction is fused, it will be put in a subgraph of the + // fusion instruction. + const string& GetNodeNameForInstruction(const HloInstruction* instruction); + + void SetNodeAttrs(const HloInstruction* instruction, + tensorflow::NodeDef* node_def) const; + + Status AddInstruction(const HloInstruction* instruction); + + DebugOptions debug_options_; + tensorflow::GraphDef graph_def_; + // This records instructions that have been visited. + std::unordered_set visited_instructions_; + // A cache that maps instruction to the node name. + std::unordered_map instruction_to_node_name_; +}; + +} // namespace hlo_graph_dumper +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_HLO_TFGRAPH_BUILDER_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/hlo_value.h b/linux/include/tensorflow/compiler/xla/service/hlo_value.h new file mode 100644 index 000000000..b6670d409 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/hlo_value.h @@ -0,0 +1,259 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_HLO_VALUE_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_HLO_VALUE_H_ + +#include +#include +#include + +#include "absl/types/span.h" +#include "tensorflow/compiler/xla/service/buffer_value.h" +#include "tensorflow/compiler/xla/service/hlo_instruction.h" +#include "tensorflow/compiler/xla/shape_tree.h" +#include "tensorflow/compiler/xla/shape_util.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/platform/logging.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/types.h" + +namespace xla { + +// Abstraction which identifies a specific point in the XLA graph. An +// HloPosition specifies a ShapeIndex within the output of a specific +// instruction. +struct HloPosition { + HloInstruction* instruction; + ShapeIndex index; + + // Returns the shape at this position. + const Shape& shape() const; + + string ToString() const; + + bool operator==(const HloPosition& other) const { + return instruction == other.instruction && index == other.index; + } + bool operator!=(const HloPosition& other) const { return !(*this == other); } + + // Stable less-than operator using instruction id and index. + bool operator<(const HloPosition& other) const { + return instruction->unique_id() < other.instruction->unique_id() || + (instruction->unique_id() == other.instruction->unique_id() && + index < other.index); + } +}; + +std::ostream& operator<<(std::ostream& out, const HloPosition& position); + +// Defines a single use of an HLO value. +struct HloUse { + // Instruction at which the value is used. + HloInstruction* instruction; + + // The operand number in which the value is appears. + int64 operand_number; + + // The shape index within the operand in which the value appears. + ShapeIndex operand_index; + + string ToString() const; + + bool operator==(const HloUse& other) const { + return instruction == other.instruction && + operand_number == other.operand_number && + operand_index == other.operand_index; + } + + bool operator!=(const HloUse& other) const { return !(*this == other); } +}; + +std::ostream& operator<<(std::ostream& out, const HloUse& use); + +// HloDataflowAnalysis uses this subclass of BufferValue. +class HloValue : public BufferValue { + public: + // Predicate comparing HloValues by increasing id, useful for std::sort. + static bool IdLessThan(const HloValue* a, const HloValue* b) { + return a->id() < b->id(); + } + + // Predicate comparing HloValues by equal id, useful for std::unique. + static bool IdEqual(const HloValue* a, const HloValue* b) { + return a->id() == b->id(); + } + + // Construct an HloValue defined by 'instruction' at shape index 'index'. If + // is_phi is true, then this value is a phi value, for example, at the + // parameter of a while body computation. Phi values are only used in the SSA + // dataflow analysis (HloDataflowAnalysis::ssa_form_ is true). + HloValue(Id id, HloInstruction* instruction, const ShapeIndex& index, + bool is_phi = false); + ~HloValue() override {} + + // Sets the positions in the module at which the HloValue appears. Updates + // uses. Should be called once and only once. The defining position should not + // be included in 'positions' as this is set at construction time. + void SetPositionsAndComputeUses(absl::Span positions); + + // Returns whether this value is a phi value. + bool is_phi() const { return is_phi_; } + + // Return the position where this value is defined. + const HloPosition& defining_position() const { return positions_[0]; } + + // Return the instruction which defines this HloValue. + HloInstruction* defining_instruction() const { + return defining_position().instruction; + } + + HloInstruction* instruction() const override { + return defining_instruction(); + } + + // Return the shape index at which this HloValue is defined in the output of + // its defining instruction. + const ShapeIndex& defining_index() const { return defining_position().index; } + + const ShapeIndex& index() const override { return defining_index(); } + + // Return the shape of this HloValue. + const Shape& shape() const override { return defining_position().shape(); } + + // Return all positions of the HloValue in the module. + const std::vector& positions() const { return positions_; } + + // Return all uses of the HloValue. + const std::vector& uses() const { return uses_; } + + // Get whether this HloValue is live out of the module. + bool live_out_of_module() const { return live_out_of_module_; } + + bool operator==(const HloValue& other) const; + bool operator!=(const HloValue& other) const; + + // Return a single-line string representation of the value. + string ToShortString() const; + + string ToString(int indent) const; + + string ToString() const override { return ToString(0); } + + private: + // Whether this instruction is a phi value. + const bool is_phi_; + + // The set of positions of this HloValue. The first element is always the + // position of the definition. + std::vector positions_; + + // The set of uses of this HloValue. + std::vector uses_; + + // Whether this value is live out of the HLO module. + bool live_out_of_module_ = false; + + // Whether this value is live out of its computation. + bool live_out_of_computation_ = false; +}; + +std::ostream& operator<<(std::ostream& out, const HloValue& hlo_value); + +// A class representing the possible set of HloValues at a particular point +// (shape index in the output of an instruction) in the XLA graph. This set +// contains the set of reaching HloValue definitions. For a simple array-shaped +// instruction like Add, the HloValueSet of the top-level of the instruction's +// output trivially contains only the HloValue defined by the instruction. For +// instructions which have non-trivial dataflow such as Tuple or Select, the +// HloValueSets of the instruction's output contains one or more HloValues +// defined by the instruction's operands or defined further up in the XLA graph. +class HloValueSet { + public: + HloValueSet() = default; + + explicit HloValueSet(absl::Span values) + : values_(values.begin(), values.end()) { + SortAndUniquifyValues(); + } + + // Sets this value set to the union of the given value sets. Returns whether + // this value set changed. + bool AssignUnionOf(absl::Span inputs); + + // Return the vector of HloValues in the set. Values in the vector are unique + // and stably sorted by value id. + const std::vector& values() const { return values_; } + + // Adds the value to the set. Returns true iff the value was added and didn't + // already exist in the set. + bool AddValue(const HloValue* value); + + // Clear all values from the set. + void Clear() { values_.clear(); } + + // Return the unique HLO value in the set. CHECKs if the set does not contain + // exactly one value. + const HloValue& GetUniqueValue() const { + CHECK_EQ(values_.size(), 1); + return *values_[0]; + } + + bool operator==(const HloValueSet& other) const { + if (values_.size() != other.values_.size()) return false; + for (size_t i = 0; i < values_.size(); ++i) { + if (values_[i]->id() != other.values_[i]->id()) { + return false; + } + } + return true; + } + bool operator!=(const HloValueSet& other) const { return !(*this == other); } + + string ToString() const; + + private: + // Sorts value_ and removes duplicates. This should be called after adding any + // elements to values_. + void SortAndUniquifyValues(); + + // HloValues sorted by HloValue::Id. + std::vector values_; +}; + +std::ostream& operator<<(std::ostream& out, const HloValueSet& hlo_value); + +// A class collecting the HloValues which might be contained in the output of +// an HLO instruction. For array-shaped instructions, an InstructionValueSet +// trivially holds a single HloValueSet. Tuple-shaped InstructionValueSets +// hold multiple HloValueSets. +class InstructionValueSet : public ShapeTree { + public: + InstructionValueSet(const Shape& shape) : ShapeTree(shape) {} + + // Sets this value set to the union of the given value sets. Returns whether + // this value set changed. + bool AssignUnionOf(absl::Span inputs); + + string ToString() const; +}; + +std::ostream& operator<<(std::ostream& out, + const InstructionValueSet& instruction_value_set); + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_HLO_VALUE_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/hlo_verifier.h b/linux/include/tensorflow/compiler/xla/service/hlo_verifier.h new file mode 100644 index 000000000..6d16586c2 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/hlo_verifier.h @@ -0,0 +1,184 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_HLO_VERIFIER_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_HLO_VERIFIER_H_ + +#include "tensorflow/compiler/xla/service/hlo_pass_interface.h" + +#include "absl/memory/memory.h" +#include "tensorflow/compiler/xla/service/shape_inference.h" + +namespace xla { + +// Visitor which verifies that the output shape is correctly set. Verifies +// against the inferred shape for the instruction. +// TODO(b/26024837): Check output shape for all instruction types. +class ShapeVerifier : public DfsHloVisitor { + public: + explicit ShapeVerifier(bool layout_sensitive, bool allow_mixed_precision) + : layout_sensitive_(layout_sensitive), + allow_mixed_precision_(allow_mixed_precision) {} + + Status HandleElementwiseUnary(HloInstruction* hlo) override; + Status HandleElementwiseBinary(HloInstruction* hlo) override; + Status HandleClamp(HloInstruction* clamp) override; + Status HandleSelect(HloInstruction* select) override; + Status HandleTupleSelect(HloInstruction* tuple_select) override; + Status HandleConcatenate(HloInstruction* concatenate) override; + Status HandleIota(HloInstruction* iota) override; + Status HandleConvert(HloInstruction* convert) override; + Status HandleBitcastConvert(HloInstruction* convert) override; + Status HandleCopy(HloInstruction* copy) override; + Status HandleDot(HloInstruction* dot) override; + Status HandleConvolution(HloInstruction* convolution) override; + Status HandleFft(HloInstruction* fft) override; + Status HandleCrossReplicaSum(HloInstruction* crs) override; + Status HandleAllToAll(HloInstruction* hlo) override; + Status HandleCollectivePermute(HloInstruction* hlo) override; + Status HandleReducePrecision(HloInstruction* reduce_precision) override; + Status HandleInfeed(HloInstruction*) override; + Status HandleOutfeed(HloInstruction*) override; + Status HandleRng(HloInstruction*) override; + Status HandleReverse(HloInstruction* reverse) override; + Status HandleSort(HloInstruction* sort) override; + Status HandleConstant(HloInstruction* constant) override; + Status HandleGetTupleElement(HloInstruction* get_tuple_element) override; + Status HandleReduce(HloInstruction* reduce) override; + Status HandleBitcast(HloInstruction* bitcast) override; + Status HandleBroadcast(HloInstruction* broadcast) override; + Status HandleReshape(HloInstruction* reshape) override; + Status HandleTranspose(HloInstruction* transpose) override; + Status HandleParameter(HloInstruction*) override; + Status HandleFusion(HloInstruction*) override; + Status HandleCall(HloInstruction* call) override; + Status HandleCustomCall(HloInstruction*) override; + Status HandleSlice(HloInstruction* slice) override; + Status HandleDynamicSlice(HloInstruction* dynamic_slice) override; + Status HandleDynamicUpdateSlice( + HloInstruction* dynamic_update_slice) override; + Status HandleTuple(HloInstruction* tuple) override; + Status HandleMap(HloInstruction* map) override; + Status HandleReduceWindow(HloInstruction* reduce_window) override; + Status HandleSelectAndScatter(HloInstruction* instruction) override; + Status HandleWhile(HloInstruction* xla_while) override; + Status HandleConditional(HloInstruction* conditional) override; + Status HandlePad(HloInstruction* pad) override; + Status HandleSend(HloInstruction* send) override; + Status HandleSendDone(HloInstruction* send_done) override; + Status HandleRecv(HloInstruction* recv) override; + Status HandleRecvDone(HloInstruction* recv_done) override; + Status HandleBatchNormTraining(HloInstruction* batch_norm_training) override; + Status HandleBatchNormInference( + HloInstruction* batch_norm_inference) override; + Status HandleBatchNormGrad(HloInstruction* batch_norm_grad) override; + Status HandleGather(HloInstruction* gather) override; + Status HandleScatter(HloInstruction* scatter) override; + Status HandleAfterAll(HloInstruction* token) override; + + Status FinishVisit(HloInstruction*) override { return Status::OK(); } + + protected: + // Check the instruction's shape against the shape given by ShapeInference + // and return an appropriate error if there is a mismatch. + Status CheckShape(const HloInstruction* instruction, + const Shape& inferred_shape); + + // Overload which takes a StatusOr to reduce boilerplate in the caller. + Status CheckShape(const HloInstruction* instruction, + const StatusOr& inferred_shape_status); + + // Check a unary (binary, etc) instruction's shape against the inferred shape. + Status CheckUnaryShape(const HloInstruction* instruction); + Status CheckBinaryShape(const HloInstruction* instruction); + Status CheckTernaryShape(const HloInstruction* instruction); + Status CheckVariadicShape(const HloInstruction* instruction); + + private: + // Helpers that switch on layout_sensitive_. + bool ShapesSame(const Shape& a, const Shape& b) { + return layout_sensitive_ ? ShapeUtil::Equal(a, b) + : ShapeUtil::Compatible(a, b); + } + bool ShapesSameIgnoringFpPrecision(const Shape& a, const Shape& b) { + return layout_sensitive_ ? ShapeUtil::EqualIgnoringFpPrecision(a, b) + : ShapeUtil::CompatibleIgnoringFpPrecision(a, b); + } + string StringifyShape(const Shape& s) { + return layout_sensitive_ ? ShapeUtil::HumanStringWithLayout(s) + : ShapeUtil::HumanString(s); + } + + // Checks that the given operand of the given instruction is of type TOKEN. + Status CheckIsTokenOperand(const HloInstruction* instruction, + int64 operand_no); + + // Checks that the shape of the given operand of the given instruction matches + // the given parameter of the given computation. + Status CheckOperandAndParameter(const HloInstruction* instruction, + int64 operand_number, + const HloComputation* computation, + int64 parameter_number); + + // Returns true if the shapes of the two operands have the same element type, + // and the result shape either has the same element type as the operand shapes + // or mixed precision is allowed and the result shape and the operand shapes + // have floating point element types. + bool HasCompatibleElementTypes(const Shape& shape_0, const Shape& shape_1, + const Shape& result_shape); + + // If the verifier is layout-sensitive, shapes must be equal to what's + // expected. Otherwise, the shapes must simply be compatible. + bool layout_sensitive_; + + // Whether the inputs and output of an instruction can contain both F32s and + // BF16s. Tuples that include both F32s and BF16s are allowed regardless of + // this flag. + bool allow_mixed_precision_; +}; + +// HLO pass that verifies invariants of HLO instructions for each computation in +// the module. +class HloVerifier : public HloModulePass { + public: + using ShapeVerifierFactory = std::function()>; + + explicit HloVerifier(bool layout_sensitive, bool allow_mixed_precision) + : shape_verifier_factory_([layout_sensitive, allow_mixed_precision] { + return absl::make_unique(layout_sensitive, + allow_mixed_precision); + }) {} + + // Uses custom shape verification. + explicit HloVerifier(ShapeVerifierFactory shape_verifier_factory) + : shape_verifier_factory_(std::move(shape_verifier_factory)) {} + + ~HloVerifier() override = default; + absl::string_view name() const override { return "verifier"; } + + // Never returns true; no instructions are ever modified by this pass. + StatusOr Run(HloModule* module) override; + + private: + // Creates a ShapeVerifier that checks that shapes match inferred + // expectations. This is a factory function because ShapeVerifier, + // being a DfsHloVisitor, is stateful. We want a clean object + // for each run of the verifier. + ShapeVerifierFactory shape_verifier_factory_; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_HLO_VERIFIER_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/human_readable_profile_builder.h b/linux/include/tensorflow/compiler/xla/service/human_readable_profile_builder.h new file mode 100644 index 000000000..925111fa1 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/human_readable_profile_builder.h @@ -0,0 +1,85 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_HUMAN_READABLE_PROFILE_BUILDER_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_HUMAN_READABLE_PROFILE_BUILDER_H_ + +#include + +#include "absl/strings/string_view.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/core/platform/logging.h" +#include "tensorflow/core/platform/types.h" + +namespace xla { + +// HumanReadableProfileBuilder helps you create a textual profile of a +// computation, suitable for consumption by humans. +class HumanReadableProfileBuilder { + public: + explicit HumanReadableProfileBuilder(absl::string_view computation_name, + int64 total_cycles, + double clock_rate_ghz) + : computation_name_(computation_name), + total_cycles_(total_cycles), + clock_rate_ghz_(clock_rate_ghz) { + CHECK_GE(clock_rate_ghz, 1e-9); + } + + int64 total_cycles() const { return total_cycles_; } + + // Adds an operation to the profile. If you don't know the number of + // floating-point ops or bytes touched by the op, or if you don't know how + // fast it would run optimally, pass -1 for that param. + void AddOp(absl::string_view op_name, absl::string_view short_name, + absl::string_view category, int64 cycles, int64 flop_count, + int64 transcendental_count, int64 bytes_accessed, + float optimal_seconds) { + op_infos_.push_back({string(op_name), string(short_name), string(category), + cycles, flop_count, transcendental_count, + bytes_accessed, optimal_seconds}); + } + + // Gets the human-readable profile. + string ToString() const; + + private: + struct OpInfo { + string name; + string short_name; + string category; + int64 cycles; + int64 flop_count; // -1 if unknown + int64 transcendental_count; + int64 bytes_accessed; // -1 if unknown + float optimal_seconds; // -1 if unknown + }; + + double CyclesToSeconds(int64 cycles) const { + return cycles / clock_rate_ghz_ / 1e9; + } + double CyclesToMicroseconds(int64 cycles) const { + return cycles / clock_rate_ghz_ / 1000.0; + } + + string computation_name_; + int64 total_cycles_; + double clock_rate_ghz_; + std::vector op_infos_; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_HUMAN_READABLE_PROFILE_BUILDER_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/indexed_array_analysis.h b/linux/include/tensorflow/compiler/xla/service/indexed_array_analysis.h new file mode 100644 index 000000000..e5aa67fd8 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/indexed_array_analysis.h @@ -0,0 +1,377 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_INDEXED_ARRAY_ANALYSIS_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_INDEXED_ARRAY_ANALYSIS_H_ + +#include + +#include "absl/container/flat_hash_map.h" +#include "tensorflow/compiler/xla/service/hlo_instruction.h" +#include "tensorflow/compiler/xla/service/hlo_module.h" +#include "tensorflow/compiler/xla/service/hlo_pass_interface.h" +#include "tensorflow/core/util/ptr_util.h" + +namespace xla { + +// IndexedArrayAnalysis decides if an HLO instruction can be rewritten as a +// gather from another array. It does this by mapping HLO instructions to +// instances of IndexedArrayAnalysis::Array, which can be inspected to discover +// whether said HLO is equivalent to a gather. +class IndexedArrayAnalysis { + public: + // IndexedArrayAnalysis maps each HLO instruction to an instance of a Array. + // Array really just a sum type of the classes that inherit from it. The + // meaning of each of the subtypes is documented on the subtype declaration. + // + // Array instances are immutable once created. + class Array { + public: + enum Kind { + kUnknown, + kConstant, + kReshaped, + kScalarIndexedConstant, + kScalarIndexed + }; + + virtual Kind kind() const = 0; + virtual const Shape& shape() const = 0; + + // Does a checked downcast from `Array` to `T` which must be one of its + // subtypes. + template + T* as() { + static_assert((std::is_base_of::value), + "target type not derived from source type"); + // We skip the CHECK and hence the dynamic_cast if RTTI is disabled. +#if !defined(__GNUC__) || defined(__GXX_RTTI) + CHECK_NE(dynamic_cast(this), nullptr); +#endif // !defined(__GNUC__) || defined(__GXX_RTTI) + + return static_cast(this); + } + + virtual ~Array() = default; + + Array& operator=(const Array& other) = delete; + }; + + // Represents an HLO instruction that was not analyzable by this + // IndexedArrayAnalysis. Instances of UnknownArray just wrap an existing + // HloInstruction. + class UnknownArray : public Array { + public: + Kind kind() const override { return kUnknown; } + const Shape& shape() const override { return instruction().shape(); } + const HloInstruction& instruction() const { return instruction_; } + + private: + explicit UnknownArray(const HloInstruction* instr) : instruction_(*instr) {} + + const HloInstruction& instruction_; + + friend class IndexedArrayAnalysis; + }; + + // Represents a constant value. This constant value may be present in the HLO + // module being analyzed, or it could have been created on the fly by the + // analysis. + class ConstantArray : public Array { + public: + Kind kind() const override { return kConstant; } + const Shape& shape() const override { return literal()->shape(); } + const Literal* literal() const { return literal_; } + + private: + explicit ConstantArray(const Literal* literal) : literal_(literal) {} + const Literal* literal_; + + friend class IndexedArrayAnalysis; + }; + + // Represents an Array that is a reshape of another Array. + class ReshapedArray : public Array { + public: + Kind kind() const override { return kReshaped; } + + // The array to reshape. + Array* operand() const { return operand_; } + + // The output shape. + const Shape& shape() const override { return shape_; } + + private: + explicit ReshapedArray(Array* operand, Shape shape) + : operand_(operand), shape_(shape) {} + + Array* operand_; + const Shape shape_; + + friend class IndexedArrayAnalysis; + }; + + // --------------------------------------------------------------------------- + // Indexed Array Overview + // --------------------------------------------------------------------------- + // + // ScalarIndexedArray and ScalarIndexedConstantArray form the core of this + // analysis. ScalarIndexedConstantArray is just a specialization of + // ScalarIndexedArray so we will only discuss ScalarIndexedArray in this + // overview. + // + // A ScalarIndexedArray represents an array that can be computed by indexing + // into a "source" array using an "indices" tensor. A simple example is a + // gather operation gathering 12 rows out of a [100,100] matrix -- such an + // operation will be represented by an instance of a ScalarIndexedArray with + // the [100,100] matrix as the "source" array and the [12]-shaped indices + // array as the "indices" tensor. The ScalarIndexedArray operation itself + // will be of shape [12,100] (assuming we were gathering with axis=0). + // + // Gather operations are not the only operation that maps to + // ScalarIndexedArray instances (if that were true there would be little point + // in having a separate analysis). We can often infer ScalarIndexedArrays for + // other operations too. For instance, consider: + // + // %source = f32[100,100] constant + // %indices = s32[12] ... + // %gather = f32[12,100] ... gather from %source using %indices at axis 0 + // %dot = dot(%gather, other_constant) [canonical contracting dims] + // + // The dot operation itself is also a ScalarIndexedArray with source = + // dot(constant, other_constant) and indices = %indices. A reshape of %gather + // to [12,5,20] too is a ScalarIndexedArray with source = an appropriately + // reshaped constant and indices = %indices. + + // Represents the result of a gather operation. This gather operation may + // explicitly be present in the HLO module being analyzed, or it could have + // been created on the fly by the analysis. + // + // An instance of ScalarIndexedArray represents a array whose I'th element can + // be mapped to the J'th element of the `source` array (where I and J are + // multidimensional indices) in this way: + // + // I' = remove components at positions `output_dims` from I + // G' = remove components not at positions `output_dims` from I + // T = indices[G'] + // J = I' with T inserted at position `source_dim` + // + // For example, if source is of shape [11,13,17,19], indices is of shape + // [23,29], output_dims is [0,2] and source_dim is 2 then the output is of + // shape [23,11,29,13,19] and the output index [A,B,C,D,E] is mapped to the + // input index [B,D,indices[A,C],E]. + class ScalarIndexedArray : public Array { + public: + Kind kind() const override { return kScalarIndexed; } + const Shape& shape() const override { return shape_; } + + Array* source() const { return source_; } + Array* indices() const { return indices_; } + + // `source_dim` is the dimension in the source array that is being indexed + // over using indices from the `indices` array. See the class documentation + // and the overview for more details. + int64 source_dim() const { return source_dim_; } + + // `output_dims` are the dimensions in the output array that are being used + // to compute an index into the `indices` array. See the class + // documentation and the overview for more details. + absl::Span output_dims() const { return output_dims_; } + + private: + explicit ScalarIndexedArray(Array* source, Array* indices, int64 source_dim, + std::vector output_dims, Shape shape) + : source_(source), + indices_(indices), + source_dim_(source_dim), + output_dims_(std::move(output_dims)), + shape_(std::move(shape)) {} + + Array* source_; + Array* indices_; + int64 source_dim_; + std::vector output_dims_; + Shape shape_; + + friend class IndexedArrayAnalysis; + }; + + // A ScalarIndexedConstantArray is just a ScalarIndexedArray constrained to + // have a ConstantArray instance as the source. This is an ergonomic + // concession -- in theory it is possible to just keep ScalarIndexedArray and + // check source()->kind(). + class ScalarIndexedConstantArray : public ScalarIndexedArray { + public: + Kind kind() const override { return kScalarIndexedConstant; } + + const Literal& literal() const { + return *source()->as()->literal(); + } + + private: + explicit ScalarIndexedConstantArray(Array* source, Array* indices, + int64 source_dim, + std::vector output_dims, + Shape shape) + : ScalarIndexedArray(source, indices, source_dim, + std::move(output_dims), std::move(shape)) { + CHECK(dynamic_cast(source)); + } + + friend class IndexedArrayAnalysis; + }; + + // Returns an Array instance for `instr`. The IndexedArrayAnalysis instance + // keeps ownership of the returned Array instance. + // + // Caching Behavior: IndexedArrayAnalysis has a cache mapping HLO + // instructions to IndexedArrayAnalysis::Array instances. This entire cache + // becomes stale and may cause the analysis to return incorrect results if any + // transitive operand (stopping at the containing computation) is modified for + // any HLO instruction on which GetArrayFor has been invoked. + // + // NB! By inspecting the implementation, you may be able to infer a stronger + // caching guarantee than what is mentioned above. Nevertheless, what is + // stated above is the contract. + StatusOr GetArrayFor(const HloInstruction* instr); + + // Pretty-prints the expression rooted at `root`. + string ToString(Array* root, bool print_constants = false); + + private: + // Helper function that ensures that every HLO instruction that is + // transitively used by `root` has an entry in `cache_`. + Status TraverseAndPopulateCache(const HloInstruction* root); + + // Creates an Array instance for `instr` under the assumption that all + // operations of `instr` are present in `cache_`. + StatusOr ComputeArrayFor(const HloInstruction* instr); + + StatusOr ComputeArrayForConstant(const Literal& literal); + + StatusOr ComputeArrayForGather( + const Shape& shape, const GatherDimensionNumbers& dim_numbers, + absl::Span slice_sizes, Array* source, Array* indices); + + StatusOr ComputeArrayForDotWithIndexedLhs( + const Shape& shape, const DotDimensionNumbers& dim_numbers, + const PrecisionConfig& precision_config, ScalarIndexedConstantArray* lhs, + ConstantArray* rhs); + + StatusOr ComputeArrayForDotWithIndexedRhs( + const Shape& shape, const DotDimensionNumbers& dim_numbers, + const PrecisionConfig& precision_config, ConstantArray* lhs, + ScalarIndexedConstantArray* rhs); + + StatusOr ComputeArrayForDot(const Shape& shape, + const DotDimensionNumbers& dim_numbers, + const PrecisionConfig& precision_config, + Array* lhs, Array* rhs); + + // This tries to fold a ScalarIndexedArray which has another + // ScalarIndexedArray as a source into a ScalarIndexedArray that instead has a + // ScalarIndexedArray as indices. If `source` happened to be a + // ScalarIndexedConstantArray this can result in an expression that is more + // canonical. + // + // As an example, consider a gather operation, G0, gathering 7 elements from + // an array "Arr" of shape [100] resulting in an array of shape [7], and a + // second gather operation, G1, which gathers 3 elements out of the result of + // G0 resulting in an array of shape [3]. Let the indices uses by G0 be I0 + // (of shape [7]) and the indices used by G1 be I1 (of shape [3]). We can + // instead rewrite G1 to gather directly from "Arr" with the three indices + // from I0 as per I1. In other words, we can rewrite: + // + // G0 = [Arr[i] for i in I0] + // G1 = [G0[i] for i in I1] + // + // into + // + // I2 = [I0[i] for i in I1] + // G1 = [Arr[i] for i in I2] + StatusOr FoldGatherOfGather( + ScalarIndexedArray* source, Array* indices, int64 source_dim, + absl::Span output_dims, Shape shape); + + // Reshapes a scalar-indexed node to remove the degenerate dimensions in its + // output. The result is always a scalar-indexed node. + StatusOr ReshapeToRemoveDegenerateDims( + ScalarIndexedArray* operand); + + // Reshapes a scalar-indexed node such that the result has the degenerate + // dimensions `degenerate_dims`. The result is always a scalar-indexed node. + StatusOr ReshapeToAddDegenerateDims( + ScalarIndexedArray* operand, absl::Span degenerate_dims); + + StatusOr FoldReshapeOfGather( + const Shape& shape, ScalarIndexedConstantArray* operand); + StatusOr FoldReshapeOfGatherNoDegenerateDims( + const Shape& shape, ScalarIndexedConstantArray* scalar_indexed); + StatusOr ComputeArrayForReshape(const Shape& shape, Array* operand); + + StatusOr ComputeArrayForElementwiseBinaryOp(HloOpcode opcode, + Array* lhs, Array* rhs); + StatusOr ComputeArrayForElementwiseUnaryOp(HloOpcode opcode, + Array* operand); + + template + T* Construct(Args&&... args) { + T* new_tensor = new T(std::forward(args)...); + owned_tensors_.push_back(std::unique_ptr(new_tensor)); + return new_tensor; + } + + ScalarIndexedArray* ConstructScalarIndexedArray( + Array* source, Array* indices, int64 source_dim, + std::vector output_dims, Shape shape) { + if (source->kind() == Array::kConstant) { + return Construct(source, indices, source_dim, + std::move(output_dims), + std::move(shape)); + } else { + return Construct(source, indices, source_dim, + std::move(output_dims), + std::move(shape)); + } + } + + Literal* TakeOwnership(Literal literal) { + owned_literals_.push_back(std::move(literal)); + return &owned_literals_.back(); + } + + StatusOr TakeOwnership(StatusOr literal_or_error) { + TF_ASSIGN_OR_RETURN(Literal literal, std::move(literal_or_error)); + owned_literals_.push_back(std::move(literal)); + return &owned_literals_.back(); + } + + std::vector> owned_tensors_; + std::vector owned_literals_; + absl::flat_hash_map cache_; +}; + +// A pass that prints all non-trivial results returned by IndexedArrayAnalysis. +// This pass is a no-op if !VLOG_IS_ON(2) so it should be fine to +// unconditionally add to the regular HLO pass pipeline. +class IndexedArrayAnalysisPrinterPass : public HloModulePass { + public: + absl::string_view name() const override; + StatusOr Run(HloModule* module) override; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_INDEXED_ARRAY_ANALYSIS_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/inliner.h b/linux/include/tensorflow/compiler/xla/service/inliner.h new file mode 100644 index 000000000..e20af08fb --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/inliner.h @@ -0,0 +1,39 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_INLINER_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_INLINER_H_ + +#include "tensorflow/compiler/xla/service/hlo_module.h" +#include "tensorflow/compiler/xla/service/hlo_pass_interface.h" + +namespace xla { + +// A pass which performs inlining. Which can result, for example, in functions +// that were previously being mapped by Map instead directly applied to the +// forwarded operands (i.e., map({X, Y}, max) -> max(X, Y)). +class Inliner : public HloModulePass { + public: + ~Inliner() override = default; + absl::string_view name() const override { return "inline"; } + + // Run inlining on the given computation. Returns whether the computation was + // changed. + StatusOr Run(HloModule* module) override; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_INLINER_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/instruction_fusion.h b/linux/include/tensorflow/compiler/xla/service/instruction_fusion.h new file mode 100644 index 000000000..da2032f6c --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/instruction_fusion.h @@ -0,0 +1,186 @@ +#include "absl/container/flat_hash_map.h" +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_INSTRUCTION_FUSION_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_INSTRUCTION_FUSION_H_ + +#include "tensorflow/compiler/xla/service/hlo_computation.h" +#include "tensorflow/compiler/xla/service/hlo_instruction.h" +#include "tensorflow/compiler/xla/service/hlo_module.h" +#include "tensorflow/compiler/xla/service/hlo_pass_interface.h" +#include "tensorflow/core/platform/macros.h" + +namespace xla { + +// A queue interface that allows implementations to choose fusion candidates in +// custom order. +class FusionQueue { + public: + FusionQueue() = default; + virtual ~FusionQueue() = default; + + // Dequeues the next fusion candidates: a consumer and the list of producers + // as operand indices. + virtual std::pair> + DequeueNextInstructionAndOperandsToFuseInOrder() = 0; + + // A callback passed to the queue implementation right before the producer is + // fused into the consumer. + virtual void PreFusion(HloInstruction* producer, HloInstruction* consumer) {} + + // A callback passed to the queue implementation right after the fusion is + // created. Note that original_producer could have been destroyed. + virtual void OnFusingInstruction(HloInstruction* fusion, + HloInstruction* original_producer, + HloInstruction* original_consumer) {} + + // A callback passed to the queue implementation to notify the removal of an + // instruction. + virtual void RemoveInstruction(HloInstruction* instruction) = 0; +}; + +// HLO pass which performs instruction fusion. Instructions are fused +// "vertically", meaning producing instructions are fused into their consumers +// with the intent that the loops which compute their values will be fused in +// code generation. Derived classes define ShouldFuse method to select which +// instructions to fuse. +class InstructionFusion : public HloModulePass { + public: + explicit InstructionFusion( + std::function is_expensive, + bool may_duplicate = true) + : is_expensive_(is_expensive), may_duplicate_(may_duplicate) {} + ~InstructionFusion() override = default; + absl::string_view name() const override { return "fusion"; } + + // Run instruction fusion on the given computation. Returns whether the + // computation was changed (instructions were fused). + StatusOr Run(HloModule* module) override; + + // Returns true if the computation of the given instruction is significantly + // more expensive than just writing all the values of the instructions' result + // array. Expensive operations will not be duplicated. + static bool IsExpensive(const HloInstruction& instruction); + + protected: + // Returns a FusionQueue that implements custom order of instructions being + // fused. The default implementation processes consumers in reverse post + // order. + virtual std::unique_ptr GetFusionQueue( + HloComputation* computation, + const std::function& skip_producer); + + // Returns whether the given producer instruction should be fused into the + // given consumer instruction. producer is necessarily an operand of consumer. + // Derived classes should define this method to specify which instructions + // should be fused. `operand_index` is which operand of the consumer the + // producer is. + // + // Instructions are traversed in reverse post order (computation root to + // leaves). This method is called for each operand of the instruction (where + // the operand is 'producer' and the instruction is 'consumer') + // + // Subtypes can override this with target-specific heuristics. + virtual bool ShouldFuse(HloInstruction* consumer, int64 operand_index); + + // Returns whether multi-output fusion can be applied to fuse `producer` into + // `consumer`. In contrast to "regular" fusion, the `producer` is not + // duplicated by multi-output fusion. + virtual bool ShouldFuseIntoMultiOutput(HloInstruction* consumer, + int64 operand_index) { + return false; + } + + // Chooses a fusion kind for `producer` and `consumer`. + // Default method chooses `kLoop`. + virtual HloInstruction::FusionKind ChooseKind(const HloInstruction* producer, + const HloInstruction* consumer); + + // Fuses producer into consumer. + virtual HloInstruction* Fuse(HloInstruction* producer, + HloInstruction* consumer); + + // Creates a new fusion instruction containing `producer` and `consumer`. A + // tuple is added as the fusion instruction's root, which consumes from both, + // `producer` and `consumer`. This style of fusion is referred to as + // multi-output fusion. + virtual HloInstruction* FuseIntoMultiOutput(HloInstruction* producer, + HloInstruction* consumer); + + // An "effectively unary" operation is one that has at most one "large" + // input with the others being negligible in terms of memory usage. + // We use "has a smaller true rank than the output" as a heuristic + // for "negligible" memory usage. + bool EffectivelyAtMostUnary(HloInstruction* hlo); + + // Returns true if fusing producer into consumer would cause producer to be + // duplicated. This is the case if producer has uses other than consumer. + bool FusionWouldDuplicate(const HloInstruction& producer, + const HloInstruction& consumer) { + return !(producer.users().size() == 1 && consumer.IsUserOf(&producer)); + } + + bool is_expensive(const HloInstruction& instruction) { + return is_expensive_(instruction); + } + + // Current HloComputation instance the loop fuser is traversing. + HloComputation* computation_; + HloModule* module_; + // Reachability information for the current computation. + std::unique_ptr reachability_; + + private: + // The set of producers whose consumers we cannot fuse into. + using HloInstructionSet = std::unordered_set; + + HloInstruction* AddFusionInstruction(HloInstruction* producer, + HloInstruction* consumer); + + // Whether or not we can fuse producer into consumer on all paths + // from the producer to the consumer where nodes are HLOs and edges are uses. + // + // A map from to a bool is required as the result cache + // to store and query the results of calls to this function, in order to avoid + // repeated computations. + bool CanFuseOnAllPaths( + HloInstruction* producer, HloInstruction* consumer, + const HloInstructionSet& do_not_fuse, + absl::flat_hash_map, bool>* + result_cache); + + // Computes the set of nodes that we do not want to fuse into any of their + // consumers based on a global analysis of the HLO graph. + HloInstructionSet ComputeGloballyUnfusible( + absl::Span post_order); + + // Used to determine if an HLO is expensive. Expensive operations will not be + // duplicated. + std::function is_expensive_; + + // Whether multi-output fusion would introduce a cycle into the HLO graph. + bool MultiOutputFusionCreatesCycle(HloInstruction* producer, + HloInstruction* consumer); + + // Returns whether we may duplicate an instruction if we want to fuse it. + bool may_duplicate_; + + TF_DISALLOW_COPY_AND_ASSIGN(InstructionFusion); +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_INSTRUCTION_FUSION_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/interpreter/platform_id.h b/linux/include/tensorflow/compiler/xla/service/interpreter/platform_id.h new file mode 100644 index 000000000..a6cc10bcc --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/interpreter/platform_id.h @@ -0,0 +1,29 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_INTERPRETER_PLATFORM_ID_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_INTERPRETER_PLATFORM_ID_H_ + +#include "tensorflow/stream_executor/platform.h" + +namespace stream_executor { +namespace interpreter { + +extern const Platform::Id kXlaInterpreterPlatformId; + +} // namespace interpreter +} // namespace stream_executor + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_INTERPRETER_PLATFORM_ID_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/layout_assignment.h b/linux/include/tensorflow/compiler/xla/service/layout_assignment.h new file mode 100644 index 000000000..2d48e1226 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/layout_assignment.h @@ -0,0 +1,539 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_LAYOUT_ASSIGNMENT_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_LAYOUT_ASSIGNMENT_H_ + +#include +#include +#include +#include +#include +#include +#include +#include + +#include "absl/container/flat_hash_map.h" +#include "absl/container/flat_hash_set.h" +#include "tensorflow/compiler/xla/service/computation_layout.h" +#include "tensorflow/compiler/xla/service/hlo_computation.h" +#include "tensorflow/compiler/xla/service/hlo_instruction.h" +#include "tensorflow/compiler/xla/service/hlo_module.h" +#include "tensorflow/compiler/xla/service/hlo_pass_interface.h" +#include "tensorflow/compiler/xla/service/logical_buffer.h" +#include "tensorflow/compiler/xla/service/tuple_points_to_analysis.h" +#include "tensorflow/compiler/xla/shape_layout.h" +#include "tensorflow/compiler/xla/shape_util.h" +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/platform/types.h" + +namespace xla { + +// Abstract base class for layout constraints. These constraint objects are +// gathered together in LayoutConstraints object. +class LayoutConstraint { + public: + LayoutConstraint(bool mandatory, bool dfs) + : mandatory_(mandatory), dfs_(dfs) {} + virtual ~LayoutConstraint() = default; + + virtual string ToString() const = 0; + + // True if this constraint cannot be overwritten by a different constraint. + bool mandatory() const { return mandatory_; } + + // When true, propagate in DFS. When false, constraint will propagate in BFS. + bool dfs() const { return dfs_; } + + private: + bool mandatory_; + bool dfs_; +}; + +std::ostream& operator<<(std::ostream& out, const LayoutConstraint& constraint); + +// Layout constraint on a single LogicalBuffer. This constrains the layout of an +// array produced by a particular instruction. +class BufferLayoutConstraint : public LayoutConstraint { + public: + BufferLayoutConstraint(const Layout& layout, const LogicalBuffer& buffer, + bool mandatory, bool dfs); + + const LogicalBuffer& buffer() const { return *buffer_; } + const Layout& layout() const { return layout_; } + + string ToString() const override; + + private: + Layout layout_; + const LogicalBuffer* buffer_; +}; + +// Constraint on the layout of the operand of an instruction. The constrained +// shape can be arbitrarily shaped (array or tuple). This is a constraint on the +// use of a shaped value and is not a hard constraint on the instruction(s) +// which define the value as copies may be inserted between the definition and +// use. +class OperandLayoutConstraint : public LayoutConstraint { + public: + OperandLayoutConstraint(const ShapeLayout& shape_layout, + const HloInstruction* instruction, int64 operand_no, + bool mandatory, bool dfs); + + const ShapeLayout& shape_layout() const { return shape_layout_; } + const HloInstruction* instruction() const { return instruction_; } + const int64 operand_no() const { return operand_no_; } + const HloInstruction* operand() const { + return instruction_->operand(operand_no_); + } + + string ToString() const override; + + private: + ShapeLayout shape_layout_; + const HloInstruction* instruction_; + int64 operand_no_; +}; + +// Constraint on the layout of the result of the entry computation. +class ResultLayoutConstraint : public LayoutConstraint { + public: + explicit ResultLayoutConstraint(const ShapeLayout& shape_layout, + bool dfs = false) + : LayoutConstraint(/*mandatory=*/true, dfs), + shape_layout_(shape_layout) {} + + const ShapeLayout& shape_layout() const { return shape_layout_; } + string ToString() const override; + + private: + const ShapeLayout shape_layout_; +}; + +// Class encapsulating the layout constraints of the values in a HLO +// computation. +class LayoutConstraints { + public: + LayoutConstraints(const TuplePointsToAnalysis& points_to_analysis, + HloComputation* computation); + ~LayoutConstraints() = default; + + const HloComputation* computation() const { return computation_; } + HloComputation* computation() { return computation_; } + const TuplePointsToAnalysis& points_to_analysis() const { + return points_to_analysis_; + } + + // Return a vector containing the constraints which have been added to the + // LayoutConstraints object since the construction of the object or since the + // last time ConsumeAddedConstraints() has been called. This is used to + // identify newly added constraints when propagating layouts. + std::vector ConsumeAddedConstraints() { + std::vector ret_vec(std::move(added_constraints_)); + added_constraints_.clear(); + return ret_vec; + } + void ClearAddedConstraints() { added_constraints_.clear(); } + + // Returns the layout of a LogicalBuffer, the layout of the operand of the + // instruction, or the layout of the result of the computation, respectively, + // if it has been constrained. Otherwise return nullptr. + const Layout* BufferLayout(const LogicalBuffer& buffer) const; + const BufferLayoutConstraint* GetBufferLayoutConstraint( + const LogicalBuffer& buffer) const; + const ShapeLayout* OperandLayout(const HloInstruction* instruction, + int64 operand_no) const; + const OperandLayoutConstraint* GetOperandLayoutConstraint( + const HloInstruction* instruction, int64 operand_no) const; + const ShapeLayout* ResultLayout() const; + + // Add a constraint on the layout of a LogicalBuffer, the layout of the + // operand of the instruction, or the layout of the result of the computation, + // respectively. + Status SetBufferLayout(const Layout& layout, const LogicalBuffer& buffer, + bool mandatory = true, bool dfs = true); + Status SetOperandLayout(const Shape& shape_with_layout, + const HloInstruction* instruction, int64 operand_no, + bool mandatory = true, bool dfs = true); + Status SetResultLayout(const Shape& shape_with_layout, bool dfs = true); + + // Convenience wrapper around SetOperandLayout for setting the layout of a + // operand using a Layout object. The operand must be array-shaped. + Status SetArrayOperandLayout(const Layout& layout, + const HloInstruction* instruction, + int64 operand_no, bool mandatory = true, + bool dfs = true); + + // Convenience wrapper around SetBufferLayout. Sets the layouts of all buffers + // created by the instruction to the layouts in the given shape. The + // instruction must define every logical buffer in its output. + Status SetInstructionLayout(const Shape& shape_with_layout, + const HloInstruction* instruction, + bool mandatory = true, bool dfs = true); + + // Returns true if any buffer in the given operand is forwarded to the output + // of the given instruction. For example, the Tuple instruction forwards the + // buffers of its operands and would return true for each of its operands. + bool OperandBufferForwarded(const HloInstruction* instruction, + int64 operand_no) const; + + // Returns the set of logical buffers (by LogicalBuffer:Id) which do not + // yet have a layout constraint + const std::set& unconstrained_buffer_ids() const { + return unconstrained_buffer_ids_; + } + + string ToString() const; + + private: + // Find a bufferset in the bufferset cache. This is useful since we can + // currently create the flattened buffer set for the same instruction many + // times, which is often slow. + PointsToSet::BufferSet* GetBufferSet(const HloInstruction* instruction) const; + + // The set of BufferLayoutConstraints applied to the computation. + std::unordered_map + buffer_constraints_; + + // The set of OperandLayoutConstraints applied to the computation. + using OperandConstraintKey = std::pair; + std::map operand_constraints_; + + // The result constraint for the computation (can be null). + std::unique_ptr result_constraint_; + + // A vector which holds constraints as they are added. Can be cleared with + // ClearAddedConstraints. + std::vector added_constraints_; + + // Points-to analysis for the module. Used to propagate constraints through + // the HLO graph. + const TuplePointsToAnalysis& points_to_analysis_; + + // Array-shaped buffers which have not yet been constrained. + std::set unconstrained_buffer_ids_; + + mutable absl::flat_hash_map> + buffer_sets_cache_; + + HloComputation* computation_; +}; + +// Contains constraints on the layout of channels; sends and recvs. +class ChannelLayoutConstraints { + public: + // Construct an empty constraint set. + ChannelLayoutConstraints() {} + + // Returns true if channel_id has a layout constraint. + bool IsChannelConstrained(int64 channel_id) const { + return constraints_.count(channel_id) > 0; + } + + // Given `shape`, apply the layout for `channel_id`. `channel_id` must already + // be constrained. + Shape LayoutShapeForChannel(Shape shape, int64 channel_id) const { + auto it = constraints_.find(channel_id); + CHECK(it != constraints_.end()) << "Channel " << channel_id; + *shape.mutable_layout() = it->second; + return shape; + } + + // Returns the layout constraint for `channel_id`, which must already be + // constrained. + const Layout& LayoutForChannel(int64 channel_id) const { + auto it = constraints_.find(channel_id); + CHECK(it != constraints_.end()) << "Channel " << channel_id; + return it->second; + } + + // Adds a new layout constraint for `channel_id`. If a constraint for + // `channel_id` has been added, this API returns nullptr, otherwise returns + // the layout which has already been set for the channel. + const Layout* ConstrainChannel(int64 channel_id, const Layout& layout) { + auto it = constraints_.emplace(std::make_pair(channel_id, layout)); + if (it.second) { + return nullptr; + } + return LayoutUtil::Equal(layout, it.first->second) ? nullptr + : &it.first->second; + } + + private: + std::unordered_map constraints_; +}; + +// HLO pass which assigns layouts to all instructions in the HLO module while +// satisfying all necessary invariants and minimizing cost. +class LayoutAssignment : public HloModulePass { + public: + // entry_computation_layout is modified to populate a layout for the result in + // the case that no particular layout is requested. + // + // instruction_can_change_layout_func is a function object that determines + // whether an instruction can change layouts. An instruction not being able to + // change layout means that it requires operands with the same rank as the + // output to have the same layout as the output. + // + // channel_constraints is both an input and output. Any sends or recvs that + // are present in channel_constraints will be laid out as constrained. Any + // unconstrained sends or recvs will be laid out as locally optimal and their + // layout will be added as a constraint to channel_constraints. + // + // If channel_constraints is nullptr, no kSend or kRecvs must be contained + // within any module passed to `Run`. + explicit LayoutAssignment( + ComputationLayout* entry_computation_layout, + std::function + instruction_can_change_layout_func = InstructionCanChangeLayout, + ChannelLayoutConstraints* channel_constraints = nullptr); + ~LayoutAssignment() override {} + absl::string_view name() const override { return "layout-assignment"; } + + // Assign layouts to the given module. Returns whether the module was changed + // (any layouts were changed). + StatusOr Run(HloModule* module) override; + + // Determines whether an instruction can change layouts. An instruction not + // being able to change layout means that it requires operands with the same + // rank as the output to have the same layout as the output. + static bool InstructionCanChangeLayout(const HloInstruction* instruction); + + protected: + // These methods, invoked by PropagateConstraints, propagate a layout + // constraint to its neighbors (i.e. operands and users) in order to minimize + // the cost of the instructions being constrainted on. New constraints are + // added to the given constraint set. + // + // Backends can override these methods with backend-specific propagation + // rules. + virtual Status PropagateBufferConstraint( + const BufferLayoutConstraint& layout_constraint, + LayoutConstraints* constraints); + virtual Status PropagateOperandConstraint( + const OperandLayoutConstraint& layout_constraint, + LayoutConstraints* constraints); + virtual Status PropagateResultConstraint( + const ResultLayoutConstraint& layout_constraint, + LayoutConstraints* constraints); + + // By default LayoutAssignment ensures that inputs and outputs of CustomCalls + // have the "major-first" layout (i.e. {n, n-1, ..., 0}). + // + // If this function returns true, LayoutAssignment does not set a layout for + // the given CustomCall. It's up to the backend to set one in + // AddBackendConstraints, if necessary. + // + // Precondition: instruction->opcode() == HloOpcode::kCustomCall. + virtual bool CustomCallRequiresMajorFirstLayout( + const HloInstruction* /*instruction*/) { + return true; + } + + // Called after layouts of an instruction have been finalized to allow + // subclasses to check for platform specific assumptions. + virtual Status Verify(const HloInstruction* instruction) { + return Status::OK(); + } + + // Propagates a buffer layout constraint into the operands that use it. + Status PropagateBufferConstraintToUses( + const BufferLayoutConstraint& layout_constraint, + LayoutConstraints* constraints); + + // Propagates a layout constraint on the use of the result of the given + // instruction to the definitions of the LogicalBuffers which make up the + // result. + Status PropagateUseConstraintToDefs(const ShapeLayout& shape_layout, + const HloInstruction* instruction, + LayoutConstraints* constraints); + + // Chooses a layout of operand `operand_no` of `instruction` that minimizes + // the cost of `instruction`. `output_layout` is the layout of `instruction`. + // Returns null if it can't decide the best layout. + // Precondition: `instruction` and the operand are array-shaped. + std::unique_ptr ChooseOperandLayoutFromOutputLayout( + const Layout& output_layout, const HloInstruction* instruction, + int64 operand_no); + // Given the layout of `user`'s `operand_no`-th operand, chooses a layout of + // `user` that minimizes its cost on that operand. Returns null if it can't + // decide the best layout. + // Precondition: `user` and the operand are array-shaped. + std::unique_ptr ChooseOutputLayoutFromOperandLayout( + const Layout& operand_layout, const HloInstruction* user, + int64 operand_no); + + private: + // Initializes the layout assignment object for a new Run() call. + Status Init(); + + // Adds constraints which must be satisfied for correctness on all + // backends. Called once prior to propagating constraints. + Status AddMandatoryConstraints(const ComputationLayout* computation_layout, + ChannelLayoutConstraints* channel_constraints, + HloComputation* computation, + LayoutConstraints* constraints); + + // This method can be overridden to add backend-specific constraints to the + // layout of the instructions of a computation. This method is called after + // all mandatory constraints have been added via AddMandatoryConstraints + // and before propagating constraints. + virtual Status AddBackendConstraints(LayoutConstraints* constraints) { + return Status::OK(); + } + + // Construct contraints and assign layouts to all instructions in the + // computation satisfying the given ComputationLayout, if not nullptr. + // Otherwise the ComputationLayout will be calculated by propagating the + // computation instruction contraints. + // Layouts constraints are added, then propagated until all LogicalBuffers in + // the computation are constrained. + Status RunOnComputation(ComputationLayout* computation_layout, + const TuplePointsToAnalysis& points_to_analysis, + HloComputation* computation, + ChannelLayoutConstraints* channel_constraints); + + // Assign layouts to the instructions of a computation which satisfy the given + // layout constraints. Copies may be added to satisfy the constraints. The + // given LayoutConstraints must have layout constraints every logical buffer + // in the computation. + Status AssignLayouts(const LayoutConstraints& constraints, + HloComputation* computation); + + // Propagates layout constraints from a set of initial constraints in order to + // minimize the local cost of the computation. This propagation is *not* + // required for correctness. + Status PropagateConstraints(LayoutConstraints* constraints); + + // Check that all layouts in the module have been set and satisfy all + // necessary conditions. + Status CheckLayouts(HloModule* module); + + // Computes the ComputationLayout of the given computation based of the + // layouts assigned to parameters and root instruction, and inserts it to the + // computation_layouts_ map. + Status CalculateComputationLayout(HloComputation* computation); + + // Clears all the layouts which can be cleared within a computation. + Status ClearComputationLayouts(HloComputation* computation); + + // Clears the side effects of a previous pass, like added copy instructions. + Status ClearPreviousPassSideEffects(HloModule* module); + + // Propagates the layouts computed by the layout assignment pass on the given + // computation, to the computation layout passed in to this API. + // This API propagates missing layout, and also checks that the caller + // specified have been respected, by comparing those with the parameters and + // root computation instruction. + Status PropagateComputationLayouts(HloComputation* computation, + ComputationLayout* computation_layout); + + // The pointer to the ComputationLayout passed as constructor parameter. + ComputationLayout* entry_computation_layout_; + + // A copy of entry_computation_layout_ used to reset it to the initial values + // during the multiple passes done by the layout assignment operation. + ComputationLayout saved_entry_computation_layout_; + + protected: + // Sets up the copy instruction according to the characteristic (sharding, + // metadata, ...) of the reference instruction. The index argument is used + // when the instruction is a tuple, and in such case the index represents + // the location from where the copy instruction was created from. + // If the index is empty, the whole sharding will be propagated, even in case + // the intruction has a tuple sharding. + static void SetupCopiedInstruction(const HloInstruction& instruction, + HloInstruction* copy, + const ShapeIndex& index); + + // Creates and returns a copy of the given instruction with a different + // layout. Tuple-shaped instructions will be deep-copied, and the last Tuple + // instruction producing the copy is returned. + StatusOr CreateCopyWithNewLayout( + const Shape& shape_with_layout, HloInstruction* instruction); + + // Creates a copy of the given operand if the operand's layout does not match + // the given layout. This copy replaces the use in the given instruction. + // Tuple operands will be deep-copied. + Status CopyOperandIfLayoutsDiffer(const ShapeLayout& operand_layout, + HloInstruction* instruction, + int64 operand_no); + + // Registers a copy instruction added by the layout assignment pass. + void RegisterAddedCopy(HloInstruction* copy) { + CHECK_EQ(copy->opcode(), HloOpcode::kCopy); + added_copies_.insert(copy); + } + + // Adds a copy for the operand of an instruction, unless such operand is + // already a copy, and has a single user (which is forcibly the instruction + // itself). + Status AddCopyForOperand(HloInstruction* instruction, int64 operand_number); + + // Apply the channel layout constraints by populating the channel_constraints + // data structure passed in at constructor time. Eventually adds copies in + // case two ends of a channel ended up with a different leyout. + Status ConstrainChannelLayouts(HloComputation* computation, + ChannelLayoutConstraints* channel_constraints); + + // Resets the input ChannelLayoutConstraints to the original copy received + // from the constructor input. + void ResetChannelConstraints() { + if (channel_layout_constraints_ != nullptr) { + *channel_layout_constraints_ = channel_constraints_; + } + } + + // Adds constraints related to host Send/Recv instructions. + Status BuildHostChannelConstraints(HloComputation* computation); + + // Map containing the layouts of all computations assigned so + // far. Computations are handled in a topological sort where computations are + // handled before their caller instructions so the layouts of caller + // instructions can be set to match the computation. + std::map computation_layouts_; + + // Every copy added to the module by the layout assignment pass is registered + // here. + absl::flat_hash_set added_copies_; + + // The pointer to the channel layout constraints passed in with the + // constructor. If not nullptr, this is an input/output argument. + ChannelLayoutConstraints* channel_layout_constraints_ = nullptr; + + // A copy of the input layout constraints used to reset the above pointer in + // case we have to undo operations due to the multiple passes over the + // computations/instructions. + ChannelLayoutConstraints channel_constraints_; + + // Layout constraints for send/recv instructions which communicate with the + // host. + ChannelLayoutConstraints host_channel_constraints_; + + // The set of HLO instructions which lacked any layout constraint, thus + // receiving propagated default layouts. + absl::flat_hash_set unconstrained_layout_instructions_; + + std::function + instruction_can_change_layout_func_; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_LAYOUT_ASSIGNMENT_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/llvm_compiler.h b/linux/include/tensorflow/compiler/xla/service/llvm_compiler.h new file mode 100644 index 000000000..f1c623508 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/llvm_compiler.h @@ -0,0 +1,84 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_LLVM_COMPILER_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_LLVM_COMPILER_H_ + +#include "llvm/IR/Module.h" +#include "tensorflow/compiler/xla/service/compiler.h" + +namespace xla { + +// Interface for an LLVM-based compiler. This provides the ability to register +// hooks to inspect the LLVM IR during compilation, both before and after +// optimizations are applied. +// +// Hooks get called once per HLO module being compiled. The following should not +// be relied on: +// * The order in which hooks get called. +// * Whether or not a hook gets called if a compilation exits with a non-OK +// status. +class LLVMCompiler : public Compiler { + public: + ~LLVMCompiler() override {} + + // A callback of this type can be run before and/or after IR-level + // optimization to e.g. dump out the generated IR to disk or gather some + // statistics. + using ModuleHook = std::function; + + void SetPreOptimizationHook(ModuleHook hook) { + CHECK(!user_pre_optimization_hook_) + << "Pre-optimization hook is already set"; + CHECK(hook) << "hook cannot be null"; + user_pre_optimization_hook_ = hook; + } + + void RemovePreOptimizationHook() { user_pre_optimization_hook_ = nullptr; } + + void SetPostOptimizationHook(ModuleHook hook) { + CHECK(!user_post_optimization_hook_) + << "Post-optimization hook is already set"; + CHECK(hook) << "hook cannot be null"; + user_post_optimization_hook_ = hook; + } + + void RemovePostOptimizationHook() { user_post_optimization_hook_ = nullptr; } + + // Bring in + // StatusOr> RunBackend( + // std::unique_ptr module, + // se::StreamExecutor* stream_exec, + // DeviceMemoryAllocator* device_allocator) + // StatusOr> RunHloPasses( + // std::unique_ptr module, + // se::StreamExecutor* stream_exec, + // DeviceMemoryAllocator* device_allocator) + using Compiler::RunBackend; + using Compiler::RunHloPasses; + + StatusOr>> Compile( + std::vector> modules, + std::vector> stream_execs, + DeviceMemoryAllocator* device_allocator) override; + + protected: + ModuleHook user_pre_optimization_hook_; + ModuleHook user_post_optimization_hook_; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_LLVM_COMPILER_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/llvm_ir/alias_analysis.h b/linux/include/tensorflow/compiler/xla/service/llvm_ir/alias_analysis.h new file mode 100644 index 000000000..2b46b3c39 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/llvm_ir/alias_analysis.h @@ -0,0 +1,93 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_LLVM_IR_ALIAS_ANALYSIS_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_LLVM_IR_ALIAS_ANALYSIS_H_ + +#include "absl/container/flat_hash_map.h" +#include "absl/strings/str_cat.h" +#include "llvm/IR/Module.h" +#include "tensorflow/compiler/xla/service/buffer_assignment.h" +#include "tensorflow/compiler/xla/service/hlo_instruction.h" +#include "tensorflow/compiler/xla/service/llvm_ir/ir_array.h" +#include "tensorflow/compiler/xla/types.h" + +namespace xla { +namespace llvm_ir { + +// Helper functionality used to augment the LLVM IR emitted with alias-scope +// metadata. +class AliasAnalysis { + public: + AliasAnalysis(const HloModule& module, const BufferAssignment& assignment, + llvm::LLVMContext* context) + : module_(module), assignment_(assignment), context_(context) {} + + // Augments IrArray with aliasing information. + void AddAliasingInformationToIrArray(const HloInstruction& hlo, + llvm_ir::IrArray* array, + const ShapeIndex& index = {}); + + private: + // Returns a unique alias domain for this emitter. + llvm::MDNode* GetAliasDomain(); + + // Returns an alias.scope metadata node corresponding to a given buffer slice. + llvm::MDNode* GetAliasScopeMetadataForBuffer( + const BufferAllocation::Slice& buffer_slice, llvm::MDNode* domain); + + // Returns a noalias metadata node corresponding to a given buffer slice. + // + // |buffer_slice| is the buffer slice. + // + // |domain| corresponds to the alias scope domain as documented at + // http://llvm.org/docs/LangRef.html#noalias-and-alias-scope-metadata + // + // |hlo| is the instruction we are computing a noalias set for. + llvm::MDNode* GetNoaliasMetadataForBuffer( + const BufferAllocation::Slice& buffer_slice, llvm::MDNode* domain, + const BufferAssignment& assignment, const HloInstruction& hlo); + + // The HLO module we are compiling for. + const HloModule& module_; + + // Assignment of the temporary buffers needed by the computation and their + // shape information. + const BufferAssignment& assignment_; + + // The LLVM context which we are using for IR emission. + llvm::LLVMContext* context_; + + // Holds the alias domain for this computation. + llvm::MDNode* alias_domain_ = nullptr; + + // A map from a buffer slice to metadata corresponding to its alias.scope + // metadata. The index kParameterAliasSet is used to hold aliasing + // information for parameters. + absl::flat_hash_map + alias_scope_metadata_; + + // A map from a buffer slice to metadata corresponding to its noalias + // metadata. + absl::flat_hash_map + noalias_metadata_; +}; + +} // namespace llvm_ir +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_LLVM_IR_ALIAS_ANALYSIS_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/llvm_ir/buffer_assignment_util.h b/linux/include/tensorflow/compiler/xla/service/llvm_ir/buffer_assignment_util.h new file mode 100644 index 000000000..bfb6eecb8 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/llvm_ir/buffer_assignment_util.h @@ -0,0 +1,34 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_LLVM_IR_BUFFER_ASSIGNMENT_UTIL_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_LLVM_IR_BUFFER_ASSIGNMENT_UTIL_H_ + +#include "tensorflow/compiler/xla/service/buffer_assignment.h" + +namespace xla { +namespace llvm_ir { +// In XLA:GPU we map constant buffer allocations to globals in the generated +// LLVM IR. This function gives us the name of the global variable a constant +// buffer is mapped to. Not used on XLA:CPU. +string ConstantBufferAllocationToGlobalName(const BufferAllocation& allocation); + +// Returns the Literal corresponding to `allocation`, which must be a constant +// allocation. +const Literal& LiteralForConstantAllocation(const BufferAllocation& allocation); +} // namespace llvm_ir +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_LLVM_IR_BUFFER_ASSIGNMENT_UTIL_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/llvm_ir/dynamic_update_slice_util.h b/linux/include/tensorflow/compiler/xla/service/llvm_ir/dynamic_update_slice_util.h new file mode 100644 index 000000000..fb3e4eb97 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/llvm_ir/dynamic_update_slice_util.h @@ -0,0 +1,90 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_LLVM_IR_DYNAMIC_UPDATE_SLICE_UTIL_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_LLVM_IR_DYNAMIC_UPDATE_SLICE_UTIL_H_ + +#include "tensorflow/compiler/xla/service/buffer_assignment.h" +#include "tensorflow/compiler/xla/service/elemental_ir_emitter.h" +#include "tensorflow/compiler/xla/service/gpu/partition_assignment.h" +#include "tensorflow/compiler/xla/service/hlo_instruction.h" +#include "tensorflow/compiler/xla/service/llvm_ir/ir_array.h" + +// Utilities related to emitting LLVM IR for various HLO ops. + +namespace xla { +namespace llvm_ir { + +// Checks if we can emit code for the given DynamicUpdateSlice node that updates +// its input in place. Returns true if the dynamic-update-slice's +// array-to-be-updated and output share the same BufferAllocation::Slice. +// +// dynamic_update_slice must be a DynamicUpdateSlice op. +bool CanUpdateDynamicSliceInPlace(HloInstruction* dynamic_update_slice, + const BufferAssignment& assignment); + +// Checks if the given fusion node is amenable to being implemented by +// EmitFusedDynamicUpdateSliceInPlace. +inline bool CanEmitFusedDynamicUpdateSliceInPlace( + HloInstruction* fusion, const BufferAssignment& assignment) { + CHECK_EQ(fusion->opcode(), HloOpcode::kFusion); + HloInstruction* fused_root = fusion->fused_expression_root(); + if (fused_root->opcode() != HloOpcode::kDynamicUpdateSlice || + fusion->fusion_kind() != HloInstruction::FusionKind::kLoop) { + return false; + } + // Walk DynamicUpdateSlice operand(0) to fused parameter and get its + // associated operand. See if it shares an allocation with this operand. + HloInstruction* fusion_operand; + ShapeIndex index; + std::tie(fusion_operand, index) = + fused_root->mutable_operand(0)->LatestNonGteAncestorAndIndex(); + if (fusion_operand->opcode() != HloOpcode::kParameter) { + return false; + } + auto* operand = fusion->operand(fusion_operand->parameter_number()); + return assignment.HasAllocationAt(operand, index) && + assignment.HasAllocationAt(fusion, {}) && + assignment.SharesSliceAtIndex(fusion, {}, operand, index); +} + +// Emits IR for running the given dynamic-update-slice op in-place -- that is, +// where the input and output buffers share the same slice, so we can simply +// modify the input/output buffer without touching any of the other elements. +Status EmitDynamicUpdateSliceInPlace(absl::Span operand_arrays, + const IrArray& output_array, + absl::string_view name, + llvm::IRBuilder<>* b); + +// Given a loop-fusion node whose root is a dynamic-update-slice op whose +// array-to-be-updated and output share the same buffer slice, emits +// (sequential) code for a fusion node that does the dynamic-update-slice in +// place. +Status EmitFusedDynamicUpdateSliceInPlace( + HloInstruction* fusion, absl::Span fusion_operand_arrays, + const IrArray& fusion_output_array, ElementalIrEmitter* elemental_emitter, + llvm::IRBuilder<>* b); + +// Same as EmitFusedDynamicUpdateSliceInPlace, except emits a parallel loop with +// the given launch dimensions. +Status EmitParallelFusedDynamicUpdateSliceInPlace( + HloInstruction* fusion, absl::Span fusion_operand_arrays, + const IrArray& fusion_output_array, ElementalIrEmitter* elemental_emitter, + const gpu::LaunchDimensions& launch_dimensions, llvm::IRBuilder<>* b); + +} // namespace llvm_ir +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_LLVM_IR_DYNAMIC_UPDATE_SLICE_UTIL_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/llvm_ir/fused_ir_emitter.h b/linux/include/tensorflow/compiler/xla/service/llvm_ir/fused_ir_emitter.h new file mode 100644 index 000000000..44d21fa75 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/llvm_ir/fused_ir_emitter.h @@ -0,0 +1,126 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_LLVM_IR_FUSED_IR_EMITTER_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_LLVM_IR_FUSED_IR_EMITTER_H_ + +#include +#include + +#include "absl/types/span.h" +#include "llvm/IR/IRBuilder.h" +#include "llvm/IR/Value.h" +#include "tensorflow/compiler/xla/service/dfs_hlo_visitor_with_default.h" +#include "tensorflow/compiler/xla/service/elemental_ir_emitter.h" +#include "tensorflow/compiler/xla/service/hlo_instruction.h" +#include "tensorflow/compiler/xla/service/llvm_ir/ir_array.h" +#include "tensorflow/compiler/xla/service/llvm_ir/kernel_tiling.h" +#include "tensorflow/compiler/xla/service/llvm_ir/loop_emitter.h" +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" + +namespace xla { + +// FusedIrEmitter is used to generate code for fusion nodes. +// +// Unlike IrEmitter and its ilk, which directly create LLVM IR in an LLVM +// Module, FusedIrEmitter is better understood as "IR generator generator". +// FusedIrEmitter recursively creates a generator (a host function) which the +// compiler can invoke at a later time. Invoking the generator emits LLVM IR +// that, when run, produces the value at a particular index of the output. +// +// After building this generator, the compiler creates a loop (or its moral +// equivalent, e.g. a GPU kernel) and calls the generator from within the loop. +// This generates code that produces each element of the output. +// +// This class handles both vanilla fusion and multi-output fusion. In the MOF +// case, the fusion node ends with a kTuple instruction, and the generator +// created produces an LLVM struct with N elements, one for each element of the +// arrays in the tuple. It follows that the arrays in the tuple must have the +// same length. +class FusedIrEmitter : public DfsHloVisitorWithDefault { + public: + using Generator = llvm_ir::ElementGenerator; + + FusedIrEmitter(absl::Span parameter_arrays, + ElementalIrEmitter* elemental_emitter) + : parameter_arrays_(parameter_arrays), + tiled_parameter_info_(nullptr), + elemental_emitter_(elemental_emitter), + b_(elemental_emitter->b()), + module_(elemental_emitter->module()) {} + + Status DefaultAction(HloInstruction* hlo) override; + + Status HandleConstant(HloInstruction* constant) override; + + Status HandleGetTupleElement(HloInstruction* get_tuple_element) override; + + Status HandleParameter(HloInstruction* parameter) override; + + // Emits the ir value for each element in the tuple. + Status HandleTuple(HloInstruction* tuple) override; + + Status FinishVisit(HloInstruction* root) override; + + // Returns the generator function for the root of the fused computation. + Generator GetRootGenerator() const; + + // Returns the generator function for the given instruction. + Generator GetGenerator(const HloInstruction* instruction) const; + + // Returns the ir value for instruction 'hlo'. + llvm::Value* GetIrValueForGTE(const HloInstruction* hlo) const { + auto it = gte_values_.find(hlo); + CHECK(it != gte_values_.end()); + return it->second; + } + + void SetTiledParameterInfo(const llvm_ir::TiledParameterInfo* info) { + tiled_parameter_info_ = info; + } + + private: + // Arrays of parameters of fusion instruction + absl::Span parameter_arrays_; + const llvm_ir::TiledParameterInfo* tiled_parameter_info_; + + ElementalIrEmitter* elemental_emitter_; + + // This member will be set by FinishVisit and used in GetRootGenerator. + const HloInstruction* fused_root_ = nullptr; + + // Borrowed + llvm::IRBuilder<>* b_; + llvm::Module* module_; + + // Map from instruction pointers to functions to generate elements of their + // outputs + std::unordered_map generators_; + + // Cache of generated values, lest we regenerate an element of a node with + // multiple outgoing edges + std::unordered_map, llvm::Value*>> + generated_value_cache_; + + // Stores ir values required to emit fused (and possibly nested) + // GetTupleElement instructions. + std::unordered_map gte_values_; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_LLVM_IR_FUSED_IR_EMITTER_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/llvm_ir/ir_array.h b/linux/include/tensorflow/compiler/xla/service/llvm_ir/ir_array.h new file mode 100644 index 000000000..f4b05f29c --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/llvm_ir/ir_array.h @@ -0,0 +1,338 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_LLVM_IR_IR_ARRAY_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_LLVM_IR_IR_ARRAY_H_ + +#include +#include + +#include "absl/algorithm/container.h" +#include "absl/strings/string_view.h" +#include "absl/types/span.h" +#include "llvm/IR/IRBuilder.h" +#include "llvm/IR/Value.h" +#include "tensorflow/compiler/xla/map_util.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/platform/logging.h" +#include "tensorflow/core/platform/types.h" + +namespace xla { +namespace llvm_ir { + +// IrArray represents an XLA array at the LLVM IR level. This class +// encapsulates a base pointer to the buffer holding the array (as an LLVM +// Value) and the shape of the array. The class includes methods for emitting +// LLVM IR sequences which access elements of the array at a multidimensional +// index (eg, [x, y, z] in a 3-dimensional array). Arbitrary shape and layouts +// are supported. +class IrArray { + public: + // A multidimensional index into an IrArray. The index for dimension zero is + // first in the vector. This is the reverse order of the notation used for + // describing the dimensions of an array. That is, for a [4 x 3 x 2] array + // dimension zero has size 2, dimension one has size 3, and dimension two has + // size 4. Thus the index {1, 2, 3} indexes the last element of this [4 x 3 x + // 2] array. + // + // This may also keep a linear index and the layout and dimensions it was + // emitted for; if the shape where this `Index` is used matches, the linear + // index may be used, potentially sparing the cost of computing the + // multidimensional index, which LLVM DCE can delete. + class Index { + public: + // Constructs an index of rank "size". Each dimension of the index is + // initialized to "value". + explicit Index(size_t size, llvm::Value* value) + : multidim_(size, value), index_type_(value->getType()) { + CHECK_NE(index_type_, nullptr); + } + + // Constructs an index of rank "size". Each dimension of the index is + // initialized to nullptr. + explicit Index(llvm::Type* index_ty, size_t size = 0) + : multidim_(size, nullptr), index_type_(index_ty) { + CHECK(index_ty->isIntegerTy()); + } + + // Constructs an index from multi-dimensional index "multidim". The linear + // index is set to nullptr. + explicit Index(absl::Span multidim, + llvm::Type* index_ty = nullptr) + : multidim_(multidim.begin(), multidim.end()) { + if (size() == 0) { + index_type_ = index_ty; + } else { + index_type_ = (*this)[0]->getType(); + if (index_ty != nullptr) { + CHECK_EQ(index_type_, index_ty); + } + } + CHECK_NE(index_type_, nullptr); + CHECK(absl::c_all_of(multidim, [&](llvm::Value* v) { + return index_type_ == v->getType(); + })); + } + + // Constructs an index from linear index "linear" and computes the + // multi-dimensional index from "linear" and "shape". "b" is the IR + // builder to emit the index of each dimension in the multi-dimensional + // index. + // + // Precondition: "shape" has a layout. + Index(llvm::Value* linear, const Shape& shape, llvm::IRBuilder<>* b); + + // Constructs an index from the given multi-dimensional index and the shape + // that it indexes into. + // + // Precondition: "shape" has a layout. + Index(absl::Span multidim, const Shape& shape, + llvm::IRBuilder<>* b); + + // Constructs an index from both a multi-dimensional index and a linear + // index. "shape" has the same meaning as that in the constructor that takes + // only a linear index. + Index(absl::Span multidim, llvm::Value* linear, + const Shape& shape); + + const std::vector& multidim() const { return multidim_; } + llvm::Value* linear() const { return linear_; } + + size_t size() const { return multidim().size(); } + + llvm::Value* operator[](size_t i) const { return multidim()[i]; } + llvm::Value*& operator[](size_t i) { return mutable_multidim()[i]; } + + void push_back(llvm::Value* value) { mutable_multidim().push_back(value); } + void InsertAt(int64 index, llvm::Value* value) { + CHECK_LE(index, size()); + mutable_multidim().insert(mutable_multidim().begin() + index, value); + } + + using iterator = std::vector::iterator; + using const_iterator = std::vector::const_iterator; + + iterator begin() { return mutable_multidim().begin(); } + iterator end() { return mutable_multidim().end(); } + + const_iterator begin() const { return multidim().begin(); } + const_iterator end() const { return multidim().end(); } + + llvm::Value* back() const { return multidim().back(); } + + bool LinearValidOnShape(const Shape& a) const; + + // Given that "this" is the target index of a reshape from `operand_shape` + // to `shape`, returns the source index. + Index SourceIndexOfReshape(const Shape& shape, const Shape& operand_shape, + llvm::IRBuilder<>* builder) const; + + // Returns the index into the source operand from which a slice operation + // selects a value to be placed into index "this". The slice is described + // by starting indices `starts` and stride values `strides`. + // + // Precondition: "this" is an index into a slice whose shape is `shape`. + Index SourceIndexOfSlice(const Shape& shape, absl::Span starts, + absl::Span strides, + llvm::IRBuilder<>* builder) const; + + // Given that "this" is the target index of a transpose from `operand_shape` + // to `shape` with the given dimension mapping, returns the source index. + Index SourceIndexOfTranspose(const Shape& shape, const Shape& operand_shape, + absl::Span dimension_mapping, + llvm::IRBuilder<>* builder) const; + + // Given that "this" is the target index of a bitcast from `operand_shape` + // to `shape`, returns the source index. + Index SourceIndexOfBitcast(const Shape& shape, const Shape& operand_shape, + llvm::IRBuilder<>* builder) const; + + // Given that "this" is the target index of a broadcast from `operand_shape` + // to `shape` with the given dimension mapping, returns the source index. + Index SourceIndexOfBroadcast(const Shape& shape, const Shape& operand_shape, + absl::Span dimension_mapping, + llvm::IRBuilder<>* builder) const; + + // Linearizes the index into the given shape, i.e. reshapes it to rank-1 and + // returns the index into the sole dimension 0 of the new shape. + llvm::Value* Linearize(absl::Span dimensions, + llvm::IRBuilder<>* builder) const; + + llvm::Type* GetType() const { return index_type_; } + + llvm::Constant* GetConstantWithIndexType(int64 c) const { + // The LLVM function makes sure that the value can be represented by the + // specified type, see ConstantInt::ConstantInt(IntegerType *Ty, const + // APInt &V). + return llvm::ConstantInt::get(index_type_, c); + } + + private: + // Changing the multi-dimensional index invalidates the linear index. + std::vector& mutable_multidim() { + linear_ = nullptr; + return multidim_; + } + + void Delinearize(std::vector* multidim, llvm::Value* linear, + const Shape& shape, llvm::IRBuilder<>* b) const; + + std::vector multidim_; + + // These values are purely for efficiency; `multidim_` is enough to find the + // element at a given `Index`, but if a loop is emitted with a linear index + // space, that linear index can be saved in `linear_`, and the layout and + // dimensions of the shape the loop was emitted for in `layout_` and + // `dims_`, and if the `Index` is used in another array, and its layout and + // dimensions match, the linear index can be used, sparing the cost of + // computing `multidim_`, which LLVM DCE could potentially so delete. + // Modifying `multidim_` after construction nullifies `linear_`, lest it + // be used wrongly, as it would be valid no more. + // If a loop is emitted with a multidimensional index space, `linear_` would + // be null and `layout_` and `dims_` would be ignored. + llvm::Value* linear_ = nullptr; + Layout layout_; + std::vector dims_; + + llvm::Type* index_type_; + }; + + // Default constructor. Constructs an IrArray in a null status. + IrArray() : base_ptr_(nullptr), shape_(nullptr) {} + + // Construct an IrArray with the given base pointer and shape. base_ptr is a + // pointer type pointing to the first element(lowest address) of the array. + IrArray(llvm::Value* base_ptr, const Shape& shape); + + // Default implementations of copying and moving. + IrArray(IrArray&& other) = default; + IrArray(const IrArray& other) = default; + IrArray& operator=(IrArray&& other) = default; + IrArray& operator=(const IrArray& other) = default; + + llvm::Value* GetBasePointer() const { return base_ptr_; } + llvm::Type* GetElementLlvmType() const { return element_type_; } + + const Shape& GetShape() const { + CHECK(shape_ != nullptr); + return *shape_; + } + + // Emit a sequence of instructions to compute the address of the element in + // the given array at the given index. Returns the address of the element as + // an LLVM Value. + // + // The optional name is useful for debugging when looking at + // the emitted LLVM IR. + llvm::Value* EmitArrayElementAddress(const Index& index, llvm::IRBuilder<>* b, + absl::string_view name = "") const; + + // Attach metadata this IrArray instance knows about to "instruction". + void AnnotateLoadStoreInstructionWithMetadata( + llvm::Instruction* instruction) const; + + // Emit IR to read an array element at the given index. Returns the read + // result (effectively, a Value loaded from memory). This method seamlessly + // handles scalar shapes by broadcasting their value to all indices (index is + // ignored). + // + // The optional name is useful for debugging when looking at + // the emitted LLVM IR. + llvm::Value* EmitReadArrayElement(const Index& index, llvm::IRBuilder<>* b, + absl::string_view name = "") const; + + // Emit IR to write the given value to the array element at the given index. + void EmitWriteArrayElement(const Index& index, llvm::Value* value, + llvm::IRBuilder<>* b) const; + + // Returns a new IrArray whose shape is "new_shape" and base pointer is a + // bitcast of the base pointer of "this" IrArray. + IrArray CastToShape(const Shape& new_shape, llvm::IRBuilder<>* b) const; + + void AddAliasScopeMetadata(llvm::MDNode* alias_scope) { + CHECK_NE(alias_scope, nullptr); + AddMetadata(llvm::LLVMContext::MD_alias_scope, alias_scope); + } + + void AddNoaliasMetadata(llvm::MDNode* noalias) { + CHECK_NE(noalias, nullptr); + AddMetadata(llvm::LLVMContext::MD_noalias, noalias); + } + + // Promises LLVM that the data pointed to by this IrArray never changes after + // it's first loaded. + // + // The temporal scope of this promise is the "whole program" from LLVM's point + // of view, but how this translates to HLOs differs between backends. + // + // In the single-threaded CPU backend, we emit one function that + // runs all the HLOs in sequence, so the whole program is the whole HLO + // module. + // + // In the GPU backend, we emit one GPU kernel per top-level HLO (i.e. per HLO + // in the entry computation). From LLVM's perspective, launching a new kernel + // is like launching a new program, and so the whole program is one top-level + // HLO. Since the scope of the promise is smaller than in the CPU backend, we + // can mark more things as invariant in the GPU backend. + // + // Marking loads as invariant is particularly helpful on GPUs because + // invariant loads can be lowered to PTX ld.global.nc (equivalent to CUDA's + // __ldg intrinsic). These loads use a special cache, and can be + // significantly faster than regular loads. + void MarkInvariantOverWholeProgram(llvm::LLVMContext* context) { + if (is_invariant_) { + return; + } + is_invariant_ = true; + AddMetadata(llvm::LLVMContext::MD_invariant_load, + llvm::MDNode::get(*context, {})); + } + + const std::map& metadata() const { return metadata_; } + + // Bumps the "which_dimension" value within the provided index by the provided + // addend. + static Index BumpIndex(const Index& index, int64 which_dimension, + int64 addend, llvm::IRBuilder<>* b); + + private: + // Add the specified LLVM IR metadata to loads/stores associated with this + // IrArray. + void AddMetadata(int kind, llvm::MDNode* md) { + InsertOrDie(&metadata_, kind, md); + } + + // Address of the base of the array as an LLVM Value. + llvm::Value* base_ptr_; + + // The LLVM type of the elements in the array. + llvm::Type* element_type_; + + // Shape of the XLA array. + const Shape* shape_; + + // The list of key/value pairs used when attaching metadata to emitted + // loads/stores for this array. They keys are the metadata kinds and the + // values are the metadata nodes. + std::map metadata_; + + bool is_invariant_ = false; +}; + +} // namespace llvm_ir +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_LLVM_IR_IR_ARRAY_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/llvm_ir/ir_builder_mixin.h b/linux/include/tensorflow/compiler/xla/service/llvm_ir/ir_builder_mixin.h new file mode 100644 index 000000000..abc06fb7b --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/llvm_ir/ir_builder_mixin.h @@ -0,0 +1,400 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_LLVM_IR_IR_BUILDER_MIXIN_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_LLVM_IR_IR_BUILDER_MIXIN_H_ + +#include "llvm/IR/IRBuilder.h" + +namespace xla { + +// Mixin class that injects more ergonomic versions of llvm::IRBuilder methods +// into a class. Intended to be used as a CRTP base class, like: +// +// class MyIrEmitter : public IrBuilderMixin { +// llvm::IRBuilder<>* builder() { return builder_; } +// +// void EmitFoo(HloInstruction* foo) { +// Add(Mul(...), FPToUI(...)); +// } +// }; + +template +class IrBuilderMixin { + protected: + template + llvm::Value* Add(Args&&... args) { + return mixin_builder()->CreateAdd(std::forward(args)...); + } + + template + llvm::LoadInst* AlignedLoad(Args&&... args) { + return mixin_builder()->CreateAlignedLoad(std::forward(args)...); + } + + template + llvm::StoreInst* AlignedStore(Args&&... args) { + return mixin_builder()->CreateAlignedStore(std::forward(args)...); + } + + template + llvm::AllocaInst* Alloca(Args&&... args) { + return mixin_builder()->CreateAlloca(std::forward(args)...); + } + + template + llvm::Value* And(Args&&... args) { + return mixin_builder()->CreateAnd(std::forward(args)...); + } + + template + llvm::Value* AtomicCmpXchg(Args&&... args) { + return mixin_builder()->CreateAtomicCmpXchg(std::forward(args)...); + } + + template + llvm::Value* AtomicRMW(Args&&... args) { + return mixin_builder()->CreateAtomicRMW(std::forward(args)...); + } + + template + llvm::Value* BitCast(Args&&... args) { + return mixin_builder()->CreateBitCast(std::forward(args)...); + } + + template + llvm::Value* Br(Args&&... args) { + return mixin_builder()->CreateBr(std::forward(args)...); + } + + llvm::CallInst* Call(llvm::Value* callee, + llvm::ArrayRef args = llvm::None, + const llvm::Twine& name = "", + llvm::MDNode* fp_math_tag = nullptr) { + return mixin_builder()->CreateCall(callee, args, name, fp_math_tag); + } + + template + llvm::BranchInst* CondBr(Args&&... args) { + return mixin_builder()->CreateCondBr(std::forward(args)...); + } + + template + llvm::Value* ConstInBoundsGEP1_32(Args&&... args) { + return mixin_builder()->CreateConstInBoundsGEP1_32( + std::forward(args)...); + } + + template + llvm::Value* FAdd(Args&&... args) { + return mixin_builder()->CreateFAdd(std::forward(args)...); + } + + template + llvm::Value* FMul(Args&&... args) { + return mixin_builder()->CreateFMul(std::forward(args)...); + } + + llvm::Value* GEP(llvm::Value* ptr, llvm::ArrayRef idx_list, + const llvm::Twine& name = "") { + return mixin_builder()->CreateGEP(ptr, idx_list, name); + } + + template + llvm::Value* ICmpEQ(Args&&... args) { + return mixin_builder()->CreateICmpEQ(std::forward(args)...); + } + + template + llvm::Value* ICmpNE(Args&&... args) { + return mixin_builder()->CreateICmpNE(std::forward(args)...); + } + + template + llvm::Value* ICmpULE(Args&&... args) { + return mixin_builder()->CreateICmpULE(std::forward(args)...); + } + + template + llvm::Value* ICmpULT(Args&&... args) { + return mixin_builder()->CreateICmpULT(std::forward(args)...); + } + + llvm::Value* InBoundsGEP(llvm::Value* ptr, + llvm::ArrayRef idx_list, + const llvm::Twine& name = "") { + return mixin_builder()->CreateInBoundsGEP(ptr, idx_list, name); + } + + llvm::Value* ExtractValue(llvm::Value* agg, llvm::ArrayRef idxs, + const llvm::Twine& name = "") { + return mixin_builder()->CreateExtractValue(agg, idxs, name); + } + + llvm::Value* InsertValue(llvm::Value* agg, llvm::Value* val, + llvm::ArrayRef idxs, + const llvm::Twine& name = "") { + return mixin_builder()->CreateInsertValue(agg, val, idxs, name); + } + + template + llvm::Value* IntToPtr(Args&&... args) { + return mixin_builder()->CreateIntToPtr(std::forward(args)...); + } + + template + llvm::LoadInst* Load(Args&&... args) { + return mixin_builder()->CreateLoad(std::forward(args)...); + } + + template + llvm::CallInst* MemCpy(Args&&... args) { + return mixin_builder()->CreateMemCpy(std::forward(args)...); + } + + template + llvm::Value* Mul(Args&&... args) { + return mixin_builder()->CreateMul(std::forward(args)...); + } + + template + llvm::Value* NSWAdd(Args&&... args) { + return mixin_builder()->CreateNSWAdd(std::forward(args)...); + } + + template + llvm::Value* NSWMul(Args&&... args) { + return mixin_builder()->CreateNSWMul(std::forward(args)...); + } + + template + llvm::Value* NSWSub(Args&&... args) { + return mixin_builder()->CreateNSWSub(std::forward(args)...); + } + + template + llvm::Value* Or(Args&&... args) { + return mixin_builder()->CreateOr(std::forward(args)...); + } + + template + llvm::Value* PointerCast(Args&&... args) { + return mixin_builder()->CreatePointerCast(std::forward(args)...); + } + + template + llvm::Value* PtrToInt(Args&&... args) { + return mixin_builder()->CreatePtrToInt(std::forward(args)...); + } + + template + llvm::Value* SDiv(Args&&... args) { + return mixin_builder()->CreateSDiv(std::forward(args)...); + } + + template + llvm::Value* Select(Args&&... args) { + return mixin_builder()->CreateSelect(std::forward(args)...); + } + + template + llvm::Value* SRem(Args&&... args) { + return mixin_builder()->CreateSRem(std::forward(args)...); + } + + template + llvm::StoreInst* Store(Args&&... args) { + return mixin_builder()->CreateStore(std::forward(args)...); + } + + template + llvm::Value* UDiv(Args&&... args) { + return mixin_builder()->CreateUDiv(std::forward(args)...); + } + + template + llvm::Value* URem(Args&&... args) { + return mixin_builder()->CreateURem(std::forward(args)...); + } + + template + llvm::Value* VectorSplat(Args&&... args) { + return mixin_builder()->CreateVectorSplat(std::forward(args)...); + } + + template + llvm::Value* ZExtOrTrunc(Args&&... args) { + return mixin_builder()->CreateZExtOrTrunc(std::forward(args)...); + } + + template + llvm::Value* AShr(Args&&... args) { + return mixin_builder()->CreateAShr(std::forward(args)...); + } + + template + llvm::Value* FCmpOEQ(Args&&... args) { + return mixin_builder()->CreateFCmpOEQ(std::forward(args)...); + } + + template + llvm::Value* FCmpOLT(Args&&... args) { + return mixin_builder()->CreateFCmpOLT(std::forward(args)...); + } + + template + llvm::Value* FCmpONE(Args&&... args) { + return mixin_builder()->CreateFCmpONE(std::forward(args)...); + } + + template + llvm::Value* FCmpUNE(Args&&... args) { + return mixin_builder()->CreateFCmpUNE(std::forward(args)...); + } + + template + llvm::Value* FDiv(Args&&... args) { + return mixin_builder()->CreateFDiv(std::forward(args)...); + } + + template + llvm::Value* FNeg(Args&&... args) { + return mixin_builder()->CreateFNeg(std::forward(args)...); + } + + template + llvm::Value* FPCast(Args&&... args) { + return mixin_builder()->CreateFPCast(std::forward(args)...); + } + + template + llvm::Value* FPToSI(Args&&... args) { + return mixin_builder()->CreateFPToSI(std::forward(args)...); + } + + template + llvm::Value* FPToUI(Args&&... args) { + return mixin_builder()->CreateFPToUI(std::forward(args)...); + } + + template + llvm::Value* FPTrunc(Args&&... args) { + return mixin_builder()->CreateFPTrunc(std::forward(args)...); + } + + template + llvm::Value* FRem(Args&&... args) { + return mixin_builder()->CreateFRem(std::forward(args)...); + } + + template + llvm::Value* FSub(Args&&... args) { + return mixin_builder()->CreateFSub(std::forward(args)...); + } + + template + llvm::Value* ICmpSGE(Args&&... args) { + return mixin_builder()->CreateICmpSGE(std::forward(args)...); + } + + template + llvm::Value* ICmpSLT(Args&&... args) { + return mixin_builder()->CreateICmpSLT(std::forward(args)...); + } + + template + llvm::Value* IntCast(Args&&... args) { + return mixin_builder()->CreateIntCast(std::forward(args)...); + } + + template + llvm::Value* LShr(Args&&... args) { + return mixin_builder()->CreateLShr(std::forward(args)...); + } + + template + llvm::Value* MemSet(Args&&... args) { + return mixin_builder()->CreateMemSet(std::forward(args)...); + } + + template + llvm::Value* Neg(Args&&... args) { + return mixin_builder()->CreateNeg(std::forward(args)...); + } + + template + llvm::Value* Not(Args&&... args) { + return mixin_builder()->CreateNot(std::forward(args)...); + } + + template + llvm::PHINode* PHI(Args&&... args) { + return mixin_builder()->CreatePHI(std::forward(args)...); + } + + template + llvm::Value* RetVoid(Args&&... args) { + return mixin_builder()->CreateRetVoid(std::forward(args)...); + } + + template + llvm::Value* SExtOrTrunc(Args&&... args) { + return mixin_builder()->CreateSExtOrTrunc(std::forward(args)...); + } + + template + llvm::Value* Shl(Args&&... args) { + return mixin_builder()->CreateShl(std::forward(args)...); + } + + template + llvm::Value* SIToFP(Args&&... args) { + return mixin_builder()->CreateSIToFP(std::forward(args)...); + } + + template + llvm::Value* Sub(Args&&... args) { + return mixin_builder()->CreateSub(std::forward(args)...); + } + + template + llvm::Value* Trunc(Args&&... args) { + return mixin_builder()->CreateTrunc(std::forward(args)...); + } + + template + llvm::Value* UIToFP(Args&&... args) { + return mixin_builder()->CreateUIToFP(std::forward(args)...); + } + + template + llvm::Value* Unreachable(Args&&... args) { + return mixin_builder()->CreateUnreachable(std::forward(args)...); + } + + template + llvm::Value* Xor(Args&&... args) { + return mixin_builder()->CreateXor(std::forward(args)...); + } + + private: + llvm::IRBuilder<>* mixin_builder() { + return static_cast(this)->builder(); + } +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_LLVM_IR_IR_BUILDER_MIXIN_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/llvm_ir/kernel_support_library.h b/linux/include/tensorflow/compiler/xla/service/llvm_ir/kernel_support_library.h new file mode 100644 index 000000000..43fec311f --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/llvm_ir/kernel_support_library.h @@ -0,0 +1,297 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_LLVM_IR_KERNEL_SUPPORT_LIBRARY_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_LLVM_IR_KERNEL_SUPPORT_LIBRARY_H_ + +#include + +#include "absl/strings/string_view.h" +#include "llvm/IR/BasicBlock.h" +#include "llvm/IR/IRBuilder.h" +#include "llvm/IR/Value.h" +#include "tensorflow/compiler/xla/service/llvm_ir/llvm_loop.h" +#include "tensorflow/compiler/xla/service/llvm_ir/llvm_util.h" + +namespace xla { +// A thin wrapper around llvm_loop.h to make code generating structured control +// flow more readable. +class KernelSupportLibrary { + public: + // `b` is the llvm::IRBuilder instance used to generate LLVM IR. + // `unroll_mode` specifies the desired LLVM unrolling behavior for every loop + // generated by this instance of KernelSupportLibrary. + explicit KernelSupportLibrary( + llvm::IRBuilder<>* b, + llvm_ir::UnrollMode unroll_mode = llvm_ir::UnrollMode::kNoUnroll, + bool prevent_vectorization = true) + : b_(b), + unroll_mode_(unroll_mode), + prevent_vectorization_(prevent_vectorization) {} + + // Generates the following control flow structure: + // + // if (`start` < `end`) { + // `for_body_generator(/*ind_var=*/start, /*is_first_iteration=*/true)`; + // for (i64 i = `start` + `step`; i s< `end`; i += `step`) + // `for_body_generator(/*ind_var=*/,i, /*is_first_iteration=*/false)`; + // } + Status For( + absl::string_view name, llvm::Value* start, llvm::Value* end, + llvm::Value* step, + const std::function& for_body_generator); + + void ForReturnVoid( + absl::string_view name, llvm::Value* start, llvm::Value* end, + llvm::Value* step, + const std::function& + for_body_generator) { + CHECK_EQ(Status::OK(), + For(name, start, end, step, + [&](llvm::Value* ind_var, bool is_first_iteration) -> Status { + for_body_generator(ind_var, is_first_iteration); + return Status::OK(); + })); + } + + Status For(absl::string_view name, int64 start, int64 end, int64 step, + const std::function& + for_body_generator) { + return For(name, /*start=*/b_->getInt64(start), + /*end=*/b_->getInt64(end), + /*step=*/b_->getInt64(step), for_body_generator); + } + + void ForReturnVoid( + absl::string_view name, int64 start, int64 end, int64 step, + const std::function& + for_body_generator) { + ForReturnVoid(name, /*start=*/b_->getInt64(start), + /*end=*/b_->getInt64(end), + /*step=*/b_->getInt64(step), for_body_generator); + } + + // Generates the following control flow structure if `peel_first_iteration` is + // true: + // + // if (`start` < `end`) { + // `for_body_generator(/*ind_var=*/start, /*is_first_iteration=*/,true)`; + // for (i64 i = `start` + `step`; i s< `end`; i += `step`) + // `for_body_generator(/*ind_var=*/,i, /*is_first_iteration=*/,false)`; + // } + // + // and the following if `peel_first_iteration` is false: + // + // for (i64 i = `start`; i s< `end`; i += `step`) + // `for_body_generator(/*ind_var=*/,i, + // /*is_first_iteration=*/,(i != `start`))`; + Status For(absl::string_view name, llvm::Value* start, llvm::Value* end, + llvm::Value* step, bool peel_first_iteration, + const std::function& + for_body_generator); + + void ForReturnVoid(absl::string_view name, llvm::Value* start, + llvm::Value* end, llvm::Value* step, + bool peel_first_iteration, + const std::function& + for_body_generator) { + TF_CHECK_OK(For( + name, start, end, step, peel_first_iteration, + [&](llvm::Value* ind_var, llvm::Value* is_first_iteration) -> Status { + for_body_generator(ind_var, is_first_iteration); + return Status::OK(); + })); + } + + Status For(absl::string_view name, llvm::Value* start, llvm::Value* end, + int64 step, bool peel_first_iteration, + const std::function& + for_body_generator) { + return For(name, /*start=*/start, /*end=*/end, + /*step=*/llvm::ConstantInt::get(start->getType(), step), + peel_first_iteration, for_body_generator); + } + + void ForReturnVoid(absl::string_view name, llvm::Value* start, + llvm::Value* end, int64 step, bool peel_first_iteration, + const std::function& + for_body_generator) { + ForReturnVoid(name, /*start=*/start, /*end=*/end, + /*step=*/llvm::ConstantInt::get(start->getType(), step), + peel_first_iteration, for_body_generator); + } + + Status For( + absl::string_view name, llvm::Value* start, llvm::Value* end, + llvm::Value* step, + const std::function& for_body_generator) { + return For(name, start, end, step, + /*peel_first_iteration=*/false, + [&](llvm::Value* indvar, llvm::Value*) -> Status { + return for_body_generator(indvar); + }); + } + + void ForReturnVoid( + absl::string_view name, llvm::Value* start, llvm::Value* end, + llvm::Value* step, + const std::function& for_body_generator) { + ForReturnVoid(name, start, end, step, + /*peel_first_iteration=*/false, + [&](llvm::Value* indvar, llvm::Value*) { + return for_body_generator(indvar); + }); + } + + Status For( + absl::string_view name, llvm::Value* start, llvm::Value* end, int64 step, + const std::function& for_body_generator) { + return For(name, start, end, llvm::ConstantInt::get(start->getType(), step), + /*peel_first_iteration=*/false, + [&](llvm::Value* indvar, llvm::Value*) -> Status { + return for_body_generator(indvar); + }); + } + + void ForReturnVoid( + absl::string_view name, llvm::Value* start, llvm::Value* end, int64 step, + const std::function& for_body_generator) { + ForReturnVoid(name, start, end, + llvm::ConstantInt::get(start->getType(), step), + for_body_generator); + } + + Status For( + absl::string_view name, int64 start, int64 end, int64 step, + const std::function& for_body_generator) { + return For(name, /*start=*/b_->getInt64(start), + /*end=*/b_->getInt64(end), + /*step=*/b_->getInt64(step), for_body_generator); + } + + void ForReturnVoid( + absl::string_view name, int64 start, int64 end, int64 step, + const std::function& for_body_generator) { + ForReturnVoid(name, /*start=*/b_->getInt64(start), + /*end=*/b_->getInt64(end), + /*step=*/b_->getInt64(step), for_body_generator); + } + + // Generates the following control flow structure: + // + // if (`condition`) + // `true_block_generator()`; + // else + // `false_block_generator()`; + Status If(absl::string_view name, llvm::Value* condition, + const std::function& true_block_generator, + const std::function& false_block_generator = + []() -> Status { return Status::OK(); }); + + Status If(llvm::Value* condition, + const std::function& true_block_generator, + const std::function& false_block_generator = + []() -> Status { return Status::OK(); }) { + return If("", condition, true_block_generator, false_block_generator); + } + + void IfReturnVoid(llvm::Value* condition, + const std::function& true_block_generator, + const std::function& false_block_generator = []() { + }) { + IfReturnVoid("", condition, true_block_generator, false_block_generator); + } + + void IfReturnVoid(absl::string_view name, llvm::Value* condition, + const std::function& true_block_generator, + const std::function& false_block_generator = []() { + }) { + TF_CHECK_OK(If(name, condition, + [&]() { + true_block_generator(); + return Status::OK(); + }, + [&]() { + false_block_generator(); + return Status::OK(); + })); + } + + using ArgumentVector = absl::Span; + + // Generates the following control flow structure: + // + // define @`kernel_name`(arg0, arg1, ... arg`arguments.size()`) { + // kernel_body_generator({arg0, arg1, ... arg`arguments.size()`}); + // } + // + // ... + // call @`kernel_name`(arguments[0], arguments[1] ...) + // ... + // + // If a function called `kernel_name` is already present in the module then + // that function is re-used. In that sense we're using the llvm::Module as a + // cache of outlined kernels, keyed by function name. + // + // If any of the values in `arguments` is nullptr (i.e. a nullptr + // llvm::Value*) then we ignore it when generating LLVM IR, and instead pass + // in a nullptr llvm::Value* in its position to `kernel_body_generator`. + // Currently we only support at most one nullptr value in `arguments`. + static void EmitAndCallOutlinedKernel( + bool enable_fast_math, bool optimize_for_size, llvm::IRBuilder<>* b, + absl::string_view kernel_name, ArgumentVector arguments, + const std::function& kernel_body_generator); + + // Thin wrappers around the more general EmitAndCallOutlinedKernel above. + static void EmitAndCallOutlinedKernel( + bool enable_fast_math, bool optimize_for_size, llvm::IRBuilder<>* b, + absl::string_view kernel_name, llvm::Value* arg0, llvm::Value* arg1, + llvm::Value* arg2, + const std::function& + kernel_body_generator) { + EmitAndCallOutlinedKernel( + enable_fast_math, optimize_for_size, b, kernel_name, {arg0, arg1, arg2}, + [&](ArgumentVector args) { + kernel_body_generator(args[0], args[1], args[2]); + }); + } + + static void EmitAndCallOutlinedKernel( + bool enable_fast_math, bool optimize_for_size, llvm::IRBuilder<>* b, + absl::string_view kernel_name, llvm::Value* arg0, llvm::Value* arg1, + llvm::Value* arg2, llvm::Value* arg3, + const std::function& kernel_body_generator) { + EmitAndCallOutlinedKernel( + enable_fast_math, optimize_for_size, b, kernel_name, + {arg0, arg1, arg2, arg3}, [&](ArgumentVector args) { + kernel_body_generator(args[0], args[1], args[2], args[3]); + }); + } + + private: + llvm::IRBuilder<>* b_; + llvm_ir::UnrollMode unroll_mode_; + bool prevent_vectorization_; +}; +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_LLVM_IR_KERNEL_SUPPORT_LIBRARY_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/llvm_ir/kernel_tiling.h b/linux/include/tensorflow/compiler/xla/service/llvm_ir/kernel_tiling.h new file mode 100644 index 000000000..5ea05b318 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/llvm_ir/kernel_tiling.h @@ -0,0 +1,80 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_LLVM_IR_KERNEL_TILING_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_LLVM_IR_KERNEL_TILING_H_ + +#include "llvm/IR/Value.h" +#include "tensorflow/compiler/xla/service/hlo_instruction.h" +#include "tensorflow/compiler/xla/service/llvm_ir/ir_array.h" + +namespace xla { +namespace llvm_ir { + +// About 0-2-1 transpose: +// +// If a shape can be viewed as three logical components 0-1-2 in the order of +// major to minor, a 0-2-1-transpose changes the order of such logical +// components to 0-2-1. We call the shape being transposed the input shape and +// the transposed shape the output shape. The logical view of the input and +// output shapes for the transpose are called the 0-1-2 shape or reduced input +// shape and the 0-2-1 shape or the reduced output shape respectively. The +// original input and output shapes are called the unreduced input and output +// shapes. + +// If `b` is a 0-2-1 transpose of `a` in 0-1-2, return the dimensions for the +// reduced shape of `b` or the 0-2-1 shape. +absl::optional > FindTranspose021(const Shape& a, + const Shape& b); + +// Return the unreduced output index corresponding to the given reduced output +// index. +IrArray::Index GetUnreducedOutputIndex( + const IrArray::Index& reduced_output_index, + const Shape& reduced_output_shape, const Shape& unreduced_output_shape, + llvm::IRBuilder<>* b); + +// A class to represent information for tiled parameters to support IR emission +// for 021 transpose. +class TiledParameterInfo { + public: + TiledParameterInfo(absl::Span param_buffers, + llvm::Value* y, llvm::Value* x) + : param_buffers_(param_buffers), y_(y), x_(x) {} + + llvm::Value* x() const { return x_; } + llvm::Value* y() const { return y_; } + + void set_x(llvm::Value* x) { x_ = x; } + void set_y(llvm::Value* y) { y_ = y; } + + llvm::Value* GetBufferForParameter(int64 index) const { + return param_buffers_[index]; + } + + private: + // Param_buffers_[i] stores the tile buffer for the ith parameter or nullptr + // if the parameter is not tiled. + absl::Span param_buffers_; + // The y coordinate within a tile. + llvm::Value* y_; + // The x coordinate within a tile. + llvm::Value* x_; +}; + +} // namespace llvm_ir +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_LLVM_IR_KERNEL_TILING_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/llvm_ir/llvm_loop.h b/linux/include/tensorflow/compiler/xla/service/llvm_ir/llvm_loop.h new file mode 100644 index 000000000..ac3bba3c9 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/llvm_ir/llvm_loop.h @@ -0,0 +1,299 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_LLVM_IR_LLVM_LOOP_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_LLVM_IR_LLVM_LOOP_H_ + +#include +#include + +#include "absl/strings/str_cat.h" +#include "absl/strings/string_view.h" +#include "absl/types/span.h" +#include "llvm/IR/BasicBlock.h" +#include "llvm/IR/IRBuilder.h" +#include "llvm/IR/Value.h" +#include "tensorflow/compiler/xla/service/llvm_ir/ir_array.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/types.h" + +namespace xla { +namespace llvm_ir { + +enum class UnrollMode { + kDefaultUnroll, + kFullyUnroll, + kNoUnroll, +}; + +// A class for constructing a for-loop in LLVM IR. +class ForLoop { + public: + // Emit a for-loop at the current insert point of the given IRBuilder. + // + // start_index and end_index are the loop bounds (end_index is not inclusive). + // `step` is the increment of the loop index after each iteration. + // + // The current insert basic block of the builder is the preheader to the loop + // (see below for definition of basic block names). All instructions (if any) + // at or after the insert point in the insert basic block are moved to a newly + // created exit basic block. Instructions before the insert point remain in + // the insert BB: + // + // +--------------+ +----------------+ + // | insert BB | | insert BB | + // | ... | | (preheader BB) | + // | %foo = ... | | ... | + // insert point ->| %bar = ... | ===> | %foo = ... | + // | ... | +----------------+ + // +--------------+ | + // V + // [[ LOOP BBs ]] + // | + // V + // +--------------+ + // | exit BB | + // | %bar = ... | + // | ... | + // +--------------+ + // + // `prefix` is used to disambiguate variable and basic block names emitted in + // LLVM IR. If non-empty, it is prepended to the name of the induction + // variable value and each basic block created for the loop. + // + // `unroll_mode` specifies the desired LLVM unrolling behavior for generated + // loop. + static std::unique_ptr EmitForLoop( + absl::string_view prefix, llvm::Value* start_index, + llvm::Value* end_index, llvm::Value* step, llvm::IRBuilder<>* b, + UnrollMode unroll_mode = llvm_ir::UnrollMode::kDefaultUnroll, + bool prevent_vectorization = false); + + // The names of the blocks follow LLVM's conventions. Control flow amongst the + // blocks for the example C code looks like: + // + // for (int i = 0; i < n; ++i) { + // do_stuff(i); + // } + // + // +--------------+ + // | preheader BB | + // | i = 0 | + // +--------------+ + // | + // V + // +-------------+ + // | header BB |<-+ + // | if i < n: | | + // | goto body | | + // | else: | | + // | goto exit | | + // +-------------+ | + // | | | + // +--------+ | | + // | V | + // | +-------------+ | + // | | body BB | | + // | | dostuff(i) |--+ + // | | ++i | + // | +-------------+ + // | + // | +-------------+ + // +->| exit BB | + // +-------------+ + // + // Caller-emitted code to execute within the loop should be placed within the + // "body" basic block. + // + // Return pointers to various blocks in the loop. + llvm::BasicBlock* GetPreheaderBasicBlock() const { return preheader_bb_; } + llvm::BasicBlock* GetHeaderBasicBlock() const { return header_bb_; } + llvm::BasicBlock* GetBodyBasicBlock() const { return body_bb_; } + llvm::BasicBlock* GetExitBasicBlock() const { return exit_bb_; } + + // Return the Value representing the induction variable in the body basic + // block of the loop. + llvm::Value* GetIndVarValue() const { return indvar_; } + + private: + // Allow ForLoopNest to call this private constructor. + friend class ForLoopNest; + + ForLoop(absl::string_view prefix, absl::string_view suffix, + llvm::Value* start_index, llvm::Value* end_index, llvm::Value* step, + UnrollMode unroll_mode, bool prevent_vectorization); + + // Emit the loop at the insert point of the builder. + void Emit(llvm::IRBuilder<>* b); + + llvm::BasicBlock* CreateLoopBB(absl::string_view name, llvm::IRBuilder<>* b); + + // Creates a name for an LLVM construct, appending prefix_ and suffix_, if + // they are set. + string GetQualifiedName(absl::string_view name); + + // Return a list of metadata nodes that should be associated with the + // llvm::Loop for this `ForLoop`. + std::vector GetLoopMetadata(llvm::IRBuilder<>* b); + + string prefix_; + string suffix_; + llvm::Value* start_index_; + llvm::Value* end_index_; + llvm::Value* step_; + + // To improve readability of the IR, we want the basic blocks to appear + // consecutively in the following order: preheader, header, body, loop, + // exit. The member insert_before_bb_ points to where the next basic block + // should be created to ensure this ordering. + llvm::BasicBlock* insert_before_bb_; + + llvm::BasicBlock* preheader_bb_; + llvm::BasicBlock* header_bb_; + llvm::BasicBlock* body_bb_; + llvm::BasicBlock* exit_bb_; + llvm::Value* indvar_; + UnrollMode unroll_mode_; + bool prevent_vectorization_; + + TF_DISALLOW_COPY_AND_ASSIGN(ForLoop); +}; + +// A simple class for constructing nested for-loops. +class ForLoopNest { + public: + explicit ForLoopNest(llvm::IRBuilder<>* b, llvm::Type* index_ty = nullptr) + : ForLoopNest(/*name=*/"", b) { + SetIndexType(index_ty); + } + + ForLoopNest(absl::string_view name, llvm::IRBuilder<>* b, + llvm::Type* index_ty = nullptr) + : name_(name), + outer_loop_preheader_bb_(nullptr), + outer_loop_exit_bb_(nullptr), + inner_loop_body_bb_(nullptr), + b_(b) { + SetIndexType(index_ty); + } + + // Adds a loop to the nest. If no loop has been added yet then emit a loop at + // the current insert point of the given builder. If one or more loops have + // been added then emit loop inside the body of the last added loop. + // unroll_mode is used to emit metadata that controls LLVM unrolling. + std::unique_ptr AddLoop( + absl::string_view suffix, llvm::Value* start_index, + llvm::Value* end_index, llvm::Value* stride, + UnrollMode unroll_mode = xla::llvm_ir::UnrollMode::kDefaultUnroll, + bool prevent_vectorization = false); + + // Like the above, except that it defaults to a stride of one. + std::unique_ptr AddLoop( + absl::string_view suffix, llvm::Value* start_index, + llvm::Value* end_index, + UnrollMode unroll_mode = xla::llvm_ir::UnrollMode::kDefaultUnroll, + bool prevent_vectorization = false); + + // A convenient wrapper of the other flavor of AddLoop. The given start and + // end index are constant. + std::unique_ptr AddLoop( + int64 start_index, int64 end_index, int64 stride, + absl::string_view suffix, + UnrollMode unroll_mode = xla::llvm_ir::UnrollMode::kDefaultUnroll, + bool prevent_vectorization = false); + + // Like the above, except that it defaults to a stride of one. + std::unique_ptr AddLoop( + int64 start_index, int64 end_index, absl::string_view suffix, + UnrollMode unroll_mode = xla::llvm_ir::UnrollMode::kDefaultUnroll, + bool prevent_vectorization = false); + + // Add loops to iterate through the indices within the specified + // shape. The returned index collects the induction variables of the + // loops so that it will iterate through all coordinates within the + // specified shape. + // + // E.g. if you pass in a 2x3 shape, you will get back an index with + // two entries that are induction variables of the two loops that + // will be added. That index will iterate through the 6 coordinates + // within the shape. One possible order for that sequence would be: + // + // (0,0), (0,1), (0,2), (1,0), (1,1), (1,2) + IrArray::Index AddLoopsForShape(const Shape& shape, absl::string_view suffix); + + // Add a loop for each dimension in "dimensions". "suffix" is the + // name suffix of the indvar and basic blocks in this new loop nest. + // + // The return value is an index with the induction variables. The + // size equals the rank of shape and there is a null for each + // dimension that is not in "dimensions". + IrArray::Index AddLoopsForShapeOnDimensions( + const Shape& shape, absl::Span dimensions, + absl::string_view suffix); + + // Emits a series of nested loops for iterating over an operand array. Loops + // are constructed in major to minor dimension layout order. No loop is + // emitted for the given 'dimension_to_skip'. The function returns an IrArray + // index for the given operand_array containing the indvars of the loops. All + // dimensions of the index are filled except for 'dimension_to_skip'. + // name_suffix is the string to append to the names of LLVM constructs (eg, + // basic blocks) constructed by this method. + IrArray::Index EmitOperandArrayLoopNest(const llvm_ir::IrArray& operand_array, + int64 dimension_to_skip, + absl::string_view name_suffix); + + // Convenience methods which return particular basic blocks of the outermost + // or innermost loops. These methods return nullptr if no loops have been + // added yet. + llvm::BasicBlock* GetOuterLoopPreheaderBasicBlock() { + return outer_loop_preheader_bb_; + } + llvm::BasicBlock* GetOuterLoopExitBasicBlock() { return outer_loop_exit_bb_; } + llvm::BasicBlock* GetInnerLoopBodyBasicBlock() { return inner_loop_body_bb_; } + + private: + void SetIndexType(llvm::Type* index_ty) { + index_type_ = index_ty == nullptr ? b_->getInt64Ty() : index_ty; + } + + llvm::Constant* GetConstantWithIndexType(int64 c) const { + return llvm::ConstantInt::get(index_type_, c); + } + + // Human-friendly name of the loop nest. + string name_; + + // The preheader and exit basic block of the outermost loop, or nullptr if no + // loop has been added yet. + llvm::BasicBlock* outer_loop_preheader_bb_; + llvm::BasicBlock* outer_loop_exit_bb_; + + // The body basic block of the most-recently added loop, or nullptr if no loop + // has been added yet. + llvm::BasicBlock* inner_loop_body_bb_; + + llvm::IRBuilder<>* b_; + + llvm::Type* index_type_; + + TF_DISALLOW_COPY_AND_ASSIGN(ForLoopNest); +}; + +} // namespace llvm_ir +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_LLVM_IR_LLVM_LOOP_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/llvm_ir/llvm_util.h b/linux/include/tensorflow/compiler/xla/service/llvm_ir/llvm_util.h new file mode 100644 index 000000000..f59baff26 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/llvm_ir/llvm_util.h @@ -0,0 +1,319 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_LLVM_IR_LLVM_UTIL_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_LLVM_IR_LLVM_UTIL_H_ + +#include +#include +#include + +#include "absl/strings/string_view.h" +#include "absl/types/span.h" +#include "llvm/ADT/StringRef.h" +#include "llvm/IR/BasicBlock.h" +#include "llvm/IR/IRBuilder.h" +#include "llvm/IR/Instructions.h" +#include "llvm/IR/Module.h" +#include "llvm/IR/Value.h" +#include "llvm/Support/raw_ostream.h" +#include "tensorflow/compiler/xla/literal.h" +#include "tensorflow/compiler/xla/service/hlo_instruction.h" +#include "tensorflow/compiler/xla/service/hlo_module_config.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/platform/types.h" + +namespace llvm { +class FastMathFlags; +class TargetOptions; +}; + +namespace xla { +namespace llvm_ir { + +// Convert a std::string (used by LLVM's interfaces) to string. +string AsString(const std::string& str); + +// Convert a absl::string_view to a llvm::StringRef. Note: both +// absl::string_view and llvm::StringRef are non-owning pointers into a +// string in memory. This method is used to feed strings to LLVM +// & Clang APIs that expect llvm::StringRef. +llvm::StringRef AsStringRef(absl::string_view str); + +template +llvm::ArrayRef AsArrayRef(const std::vector& vec) { + return llvm::ArrayRef(vec.data(), vec.size()); +} + +template +llvm::ArrayRef AsArrayRef(const absl::Span& slice) { + return llvm::ArrayRef(slice.data(), slice.size()); +} + +// Dump the given LLVM entity to a string. This works for Types and Values. +template +string DumpToString(const T& entity) { + std::string buffer_string; + llvm::raw_string_ostream ostream(buffer_string); + entity.print(ostream); + ostream.flush(); + return AsString(buffer_string); +} + +// Dump the given LLVM module to a string. This requires a function distinct +// from DumpToString because the signatures of the print() methods for Values +// and Modules are slightly different. +string DumpModuleToString(const llvm::Module& module); + +// Constructs a human-friendly name from the given inputs. The result is +// suitable for use as an llvm::Value's name. +// +// This is equivalent to +// +// - changing the HloInstruction* to its name() (if we called that overload), +// - joining all of the nonempty inputs by '.', and then +// - removing all '%'s. +// +string IrName(string a); +string IrName(absl::string_view a, absl::string_view b); +string IrName(const HloInstruction* a, absl::string_view b = ""); + +// Removes special characters from a function name. +// +// Note that this can cause different inputs to map to the same output, so after +// sanitizing a function name, you must run it through a uniquer. +string SanitizeFunctionName(string function_name); + +// Emits a call to the specified intrinsic with the given operands. Overloaded +// intrinsics (for example, "minnum") must include a type in overloaded_types +// for each overloaded type. Typically, overloaded intrinsics have only a single +// overloaded type. +llvm::Value* EmitCallToIntrinsic(llvm::Intrinsic::ID intrinsic_id, + absl::Span operands, + absl::Span overloaded_types, + llvm::IRBuilder<>* b); + +// Emit float max. Emit maxnum intrinsic is fast math is disabled, or +// fcmp+select otherwise +llvm::Value* EmitFloatMax(llvm::Value* lhs_value, llvm::Value* rhs_value, + llvm::IRBuilder<>* b); + +// Emit float min. Emit minnum intrinsic is fast math is disabled, or +// fcmp+select otherwise +llvm::Value* EmitFloatMin(llvm::Value* lhs_value, llvm::Value* rhs_value, + llvm::IRBuilder<>* b); + +// Convenience methods for emitting a GEP instruction that indexes into a buffer +// (1-dimensional array), equivalent to array[index]. The type is automatically +// determined from the element type of the array. The int64 index overload +// wraps the index in a i64 llvm::Value. +llvm::Value* EmitBufferIndexingGEP(llvm::Value* array, llvm::Value* index, + llvm::IRBuilder<>* b); +llvm::Value* EmitBufferIndexingGEP(llvm::Value* array, int64 index, + llvm::IRBuilder<>* b); + +// Returns the LLVM type which represents the given XLA primitive type. +llvm::Type* PrimitiveTypeToIrType(PrimitiveType element_type, + llvm::Module* module); + +// Returns the type size in bits. If "type" is a struct, it must be packed. +int GetSizeInBits(llvm::Type* type); + +// Returns the LLVM type which represents the given XLA shape. For example, +// if "shape" is [5 x [10 x f32]], the function returns [5 x [10 x float]]. +llvm::Type* ShapeToIrType(const Shape& shape, llvm::Module* module); + +// Returns a value that represents a pointer to a global string constant that +// encodes the shape as a serialized protobuf. +StatusOr EncodeSelfDescribingShapeConstant(const Shape& shape, + int32* shape_size, + llvm::IRBuilder<>* b); + +// Inverses the encoding of a Shape protobuf into an LLVM global variable. +// +// This is intended to be called from the runtime to decode the llvm::Constants +// that are created via ConvertShapeToSelfDescribingConstant and subsequently +// embedded into the program. +StatusOr DecodeSelfDescribingShapeConstant(const void* shape_ptr, + int32 size_bytes); + +// Converts a given literal to an IR Constant. Literals have known constant +// values at IR emission time. +llvm::Constant* ConvertLiteralToIrConstant(const Literal& literal, + llvm::Module* module); + +// Inserts an allocate of the requested type at the entry point of the +// function that the builder is currently building. The insert point +// of the builder is set to the same place after calling this function +// as before. +// +// This can be useful to avoid e.g. executing an alloca every time +// through a loop. +llvm::AllocaInst* EmitAllocaAtFunctionEntry(llvm::Type* type, + absl::string_view name, + llvm::IRBuilder<>* b, + int alignment = 0); + +// As EmitAllocaAtFunctionEntry, but allocates element_count entries +// instead of a single element. +llvm::AllocaInst* EmitAllocaAtFunctionEntryWithCount(llvm::Type* type, + llvm::Value* element_count, + absl::string_view name, + llvm::IRBuilder<>* b, + int alignment = 0); + +// Creates a basic block with the same context and function as for the +// builder. Inserts at the end of the function if insert_before is +// null. +llvm::BasicBlock* CreateBasicBlock(llvm::BasicBlock* insert_before, + absl::string_view name, + llvm::IRBuilder<>* b); + +// Struct with data on a conditional branch in a diamond shape created +// via EmitIfThenElse. +struct LlvmIfData { + // The block that has the conditional branch. + llvm::BasicBlock* if_block; + + // The block that is executed if the condition is true. + llvm::BasicBlock* true_block; + + // The block that is executed if the condition is false. + llvm::BasicBlock* false_block; + + // The block that follows after both the true_block and the + // false_block. + llvm::BasicBlock* after_block; +}; + +// Inserts a diamond-shaped if-then-else construct at the current +// insertion point of the builder. This involves splitting the current +// block into two blocks, at the insertion point, and introducing a +// true-block and a false-block that connect the two split pieces. The +// true-block is executed if the condition parameter evaluates to true +// and otherwise the false-block is executed. If `emit_else` is false, +// it jumps to the after-block rather than the false-block if the +// condition is false, and the returned `false_block` is null. +// +// Currently the insertion point of the builder must be a well-formed +// block with a terminator. If you need to use this for a +// non-terminated block, just make the function able to do that too. +LlvmIfData EmitIfThenElse(llvm::Value* condition, absl::string_view name, + llvm::IRBuilder<>* b, bool emit_else = true); + +// Emits a compare operation between "lhs" and "rhs" with the given predicate, +// and then converts the result to i8 so that it is addressable. +llvm::Value* EmitComparison(llvm::CmpInst::Predicate predicate, + llvm::Value* lhs, llvm::Value* rhs, + llvm::IRBuilder<>* b); + +// Emits a call that logs the given value with the given tag as a prefix. +// The provided tag and value are passed to a runtime logging call that is +// embedded in this translation unit when the emitted code is executed. +// +// This can be very useful for debugging generated programs in short order when +// developing new generated routines. +// +// Precondition: value must be an int64. +// Precondition: tag must be a stable pointer for the lifetime of the generated +// program (the constant pointer is burned in to the program). +void EmitLogging(const char* tag, llvm::Value* value, llvm::IRBuilder<>* b); + +// Adds alignment metadata to a load instruction using the given alignment. +// The alignment refers to the result of the load, not the load itself. +void SetAlignmentMetadataForLoad(llvm::LoadInst* load, uint64_t alignment); + +// Adds dereferenceable metadata to a load instruction using the given +// the number of dereferenceable bytes. +// Dereferenceable refers to the result of the load, not the load itself. +void SetDereferenceableMetadataForLoad(llvm::LoadInst* load, + uint64_t dereferenceable_bytes); + +// Tells LLVM `inst >= lower && inst < upper`. Returns `inst` for convenience. +llvm::Instruction* AddRangeMetadata(int64 lower, int64 upper, + llvm::Instruction* inst); + +void SetToFirstInsertPoint(llvm::BasicBlock* blk, llvm::IRBuilder<>* builder); + +void SetToLastInsertPoint(llvm::BasicBlock* blk, llvm::IRBuilder<>* builder); + +// Create a bitwise rotation of `rotand` by `rotor`. +llvm::Value* CreateRor(llvm::Value* rotand, llvm::Value* rotor, + llvm::IRBuilder<>* builder); + +// Returns the number of bytes within the shape. +int64 ByteSizeOf(const Shape& shape, const llvm::DataLayout& data_layout); + +// Gets an llvm::FastMathFlags that reflects the settings in the given +// module config. +llvm::FastMathFlags GetFastMathFlags(bool fast_math_enabled); + +// Sets values in the given TargetOptions struct according to the given +// compilation options. +void SetTargetOptions(bool fast_math_enabled, + llvm::TargetOptions* target_options); + +// Computes a conservative union of the metadata in "a" and "b". For +// aliasing-related metadata, this means the result can be applied to +// instructions whose aliasing relationship can be described either by "a" *or* +// by "b". +std::map MergeMetadata( + llvm::LLVMContext* context, const std::map& a, + const std::map& b); + +// Dumps out `llvm_module` to a file in the directory named `directory_name`, +// creating the directory if necessary. A sanitized version of +// `hlo_module_name` is incorporated into the file name. If `optimized` is true +// then a suffix of "-with-opt.ll" is used, else a suffix of "-no-opt.ll" is +// used. +Status DumpIRToDirectory(const string& directory_name, + const string& hlo_module_name, + const llvm::Module& llvm_module, bool optimized); + +llvm::Function* CreateFunction(llvm::FunctionType* function_type, + llvm::GlobalValue::LinkageTypes linkage, + bool enable_fast_math, bool optimize_for_size, + absl::string_view name, llvm::Module* module); + +// Extracts the xla_backend_extra_options from `config` and passes those that +// don't start with xla_ to LLVM. +void InitializeLLVMCommandLineOptions(const HloModuleConfig& config); + +// Zero-extends two 32-bit values to 64 bits, multiplies them, and returns the +// result as a pair of (low 32 bits, high 32 bits). +std::pair UMulLowHigh32(llvm::IRBuilder<>* b, + llvm::Value* src0, + llvm::Value* src1); +// Splits the 64-bit integer value into its high and low 32 bits. +std::pair SplitInt64ToInt32s( + llvm::IRBuilder<>* b, llvm::Value* value_64bits); + +// Checks whether a global variable is already created to represent a +// state passed between RNG calls implemented with Philox algorithm. If not, +// creates such a variable. Returns the global variable. +llvm::GlobalVariable* GetOrCreateVariableForPhiloxRngState( + llvm::Module* module, llvm::IRBuilder<>* b); + +// Adds a value to the global state variable each time when a RNG hlo is +// executed. The value of this global state variable is added to the seed +// of the Philox RNG algorithm so that calling the same RNG Hlo multiple times +// should rarely produce the same result. +void IncrementVariableForPhiloxRngState(int64 value, llvm::Module* module, + llvm::IRBuilder<>* b); +} // namespace llvm_ir +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_LLVM_IR_LLVM_UTIL_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/llvm_ir/loop_emitter.h b/linux/include/tensorflow/compiler/xla/service/llvm_ir/loop_emitter.h new file mode 100644 index 000000000..a537c0006 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/llvm_ir/loop_emitter.h @@ -0,0 +1,94 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_LLVM_IR_LOOP_EMITTER_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_LLVM_IR_LOOP_EMITTER_H_ + +#include + +#include "llvm/IR/BasicBlock.h" +#include "llvm/IR/IRBuilder.h" +#include "llvm/IR/Value.h" +#include "tensorflow/compiler/xla/service/llvm_ir/ir_array.h" +#include "tensorflow/compiler/xla/statusor.h" + +namespace xla { +namespace llvm_ir { + +// A function type for emitting code that generates an element in the target +// array. The function gets a multi-dimensional index as its only input. This +// index specifies the target element for which a value needs to be computed. +// The function has to emit code to compute this value and return the resulting +// llvm::Value*. +using ElementGenerator = + std::function(const IrArray::Index& index)>; + +// Emits a loop for every element in the given shape. +class LoopEmitter { + public: + using BodyEmitter = std::function; + + LoopEmitter(const BodyEmitter& body_emitter, const Shape& shape, + llvm::IRBuilder<>* b); + // Constructs a LoopEmitter from an element generator that generates each + // element of the given target array. + LoopEmitter(const ElementGenerator& target_element_generator, + const IrArray& target_array, llvm::IRBuilder<>* b); + + // Constructs a LoopEmitter that emits one element into each of N separate + // arrays on each iteration of the loop. + // + // This is used for multi-output fusion. target_element_generator must + // produce an LLVM struct with N elements. + LoopEmitter(const ElementGenerator& target_element_generator, + absl::Span target_arrays, llvm::IRBuilder<>* b); + + LoopEmitter(const LoopEmitter&) = delete; + LoopEmitter& operator=(const LoopEmitter&) = delete; + virtual ~LoopEmitter() = default; + + // Emits a loop nest (with a yet-to-be-filled loop body) that iterates through + // every element in the given shape. Returns the multi-dimensional index that + // specifies the element, will return multiple indices if the loop is + // unrolled. + std::vector EmitIndexAndSetExitBasicBlock() { + return EmitIndexAndSetExitBasicBlock(/*loop_name=*/"", b_->getInt64Ty()); + } + + virtual std::vector EmitIndexAndSetExitBasicBlock( + absl::string_view loop_name, llvm::Type* index_type); + + // Emits a complete loop nest for every element in the given shape. + Status EmitLoop(absl::string_view loop_name = "", + llvm::Type* index_type = nullptr); + + protected: + // An IR emitter that generates the loop body. + BodyEmitter body_emitter_; + + // The shape that the emitted loop iterates through. + Shape shape_; + + // Points to the exit block of the emitted loop. If the given shape is + // scalar, no loops are emitted and exit_bb_ is nullptr in that case. + llvm::BasicBlock* exit_bb_; + + llvm::IRBuilder<>* b_; +}; + +} // namespace llvm_ir +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_LLVM_IR_LOOP_EMITTER_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/llvm_ir/math_ops.h b/linux/include/tensorflow/compiler/xla/service/llvm_ir/math_ops.h new file mode 100644 index 000000000..6c8bc3a07 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/llvm_ir/math_ops.h @@ -0,0 +1,32 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_LLVM_IR_MATH_OPS_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_LLVM_IR_MATH_OPS_H_ + +#include "llvm/IR/IRBuilder.h" +#include "llvm/IR/Value.h" + +namespace xla { +namespace llvm_ir { + +// Emits an approximation of tanh. The implementation uses the same rational +// interpolant as implemented in Eigen3. +llvm::Value* EmitFastTanh(llvm::IRBuilder<>* b, llvm::Value* input); + +} // namespace llvm_ir +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_LLVM_IR_MATH_OPS_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/llvm_ir/tuple_ops.h b/linux/include/tensorflow/compiler/xla/service/llvm_ir/tuple_ops.h new file mode 100644 index 000000000..94340b91d --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/llvm_ir/tuple_ops.h @@ -0,0 +1,86 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_LLVM_IR_TUPLE_OPS_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_LLVM_IR_TUPLE_OPS_H_ + +#include "absl/types/span.h" +#include "llvm/IR/IRBuilder.h" +#include "llvm/IR/Value.h" +#include "tensorflow/compiler/xla/service/llvm_ir/ir_array.h" +#include "tensorflow/core/platform/types.h" + +// Utilities for emitting LLVM IR related to HLO tuples. + +namespace xla { +namespace llvm_ir { + +// Selection among tuples is special in how it's lowered, because a tuple is not +// an HLO array. +// +// tuple_on_true tuple_on_false +// | | +// V V +// ------------------------ ------------------------ +// | address of element 0 | | address of element 0 | +// |----------------------| |----------------------| +// | address of element 1 | | address of element 1 | +// |----------------------| |----------------------| +// | address of element 2 | | address of element 2 | +// ------------------------ ------------------------ +// \ / +// \ / +// ---------- +// pred ---------> | select | +// ---------- +// | +// V +// output ----> ------------------------ +// | address of element 0 | +// |----------------------| +// | address of element 1 | +// |----------------------| +// | address of element 2 | +// ------------------------ +// +// Only the addresses are copied to the output. For each element, we emit a copy +// of the address from the corresponding element in either +// tuple_on_true or tuple_on_false: +// output[i] = pred ? tuple_on_true[i] : tuple_on_false[i] +void EmitTupleSelect(const IrArray& select, const IrArray& pred, + llvm::Value* on_true, llvm::Value* on_false, + llvm::IRBuilder<>* b, llvm::Module* module); + +// A tuple is an array of pointers, one for each operand. Each pointer points to +// the output buffer of its corresponding operand. +void EmitTuple(const IrArray& tuple, absl::Span operands, + llvm::IRBuilder<>* b, llvm::Module* module); + +// Similar to EmitTuple above, except that the output buffers are provided in +// the form of IrArray. +void EmitTuple(const IrArray& tuple, absl::Span buffers, + llvm::IRBuilder<>* b, llvm::Module* module); + +// A tuple is an array of pointers, one for each operand. Each pointer points to +// the output buffer of its corresponding operand. A GetTupleElement instruction +// forwards the pointer to underlying tuple element buffer at the given index. +// Returns an llvm value representing a pointer to the tuple element buffer. +llvm::Value* EmitGetTupleElement(const Shape& target_shape, int64 index, + int alignment, llvm::Value* operand, + llvm::IRBuilder<>* b, llvm::Module* module); +} // namespace llvm_ir +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_LLVM_IR_TUPLE_OPS_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/local_service.h b/linux/include/tensorflow/compiler/xla/service/local_service.h new file mode 100644 index 000000000..3b4f0b508 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/local_service.h @@ -0,0 +1,75 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_LOCAL_SERVICE_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_LOCAL_SERVICE_H_ + +#include + +#include "absl/types/span.h" +#include "tensorflow/compiler/xla/client/executable_build_options.h" +#include "tensorflow/compiler/xla/client/xla_computation.h" +#include "tensorflow/compiler/xla/service/backend.h" +#include "tensorflow/compiler/xla/service/compiler.h" +#include "tensorflow/compiler/xla/service/device_memory_allocator.h" +#include "tensorflow/compiler/xla/service/executable.h" +#include "tensorflow/compiler/xla/service/service.h" +#include "tensorflow/compiler/xla/service/shaped_buffer.h" +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/platform/stream_executor_no_cuda.h" + +namespace xla { + +// Service implementation that extends the XLA Service to leverage running +// in the same process as the client. +class LocalService : public Service { + public: + // Factory for creating a LocalService. + static StatusOr> NewService( + const ServiceOptions& options); + + // Builds an Executable with the given XlaComputation, argument layouts and + // options. If result_layout is non-null, then the executable is compiled to + // produce a result of the given layout. If device_allocator is non-null, + // then the compiler may use it to allocate temp space on the device. The + // compiler is responsible for freeing any memory it allocates this way. + StatusOr> CompileExecutable( + const XlaComputation& computation, + const absl::Span argument_layouts, + const ExecutableBuildOptions& build_options); + + // Returns the device ordinal that corresponds to the given replica number. + // + // This returns an error if there is not a one-to-one correspondence of + // replicas to device ordinals, but is useful as a short term mechanism for + // the "easy" case where a single replica is a single device. + StatusOr ReplicaNumberToDeviceOrdinal(int replica_number); + + // Converts a GlobalDataHandle into a pointer to a ShapedBuffer that's valid + // as long as the handle is valid. + StatusOr GlobalDataToShapedBuffer( + const GlobalDataHandle& data, int replica_number); + + private: + explicit LocalService(const ServiceOptions& options, + std::unique_ptr backend); + LocalService(const LocalService&) = delete; + void operator=(const LocalService&) = delete; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_LOCAL_SERVICE_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/logical_buffer.h b/linux/include/tensorflow/compiler/xla/service/logical_buffer.h new file mode 100644 index 000000000..ceacab4ed --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/logical_buffer.h @@ -0,0 +1,67 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_LOGICAL_BUFFER_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_LOGICAL_BUFFER_H_ + +#include + +#include "absl/types/span.h" +#include "tensorflow/compiler/xla/service/buffer_value.h" +#include "tensorflow/compiler/xla/service/hlo.pb.h" +#include "tensorflow/compiler/xla/service/hlo_instruction.h" +#include "tensorflow/compiler/xla/shape_util.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/lib/gtl/int_type.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/types.h" + +namespace xla { + +// TuplePointsToAnalysis uses this subclass of BufferValue. +class LogicalBuffer : public BufferValue { + public: + LogicalBuffer(HloInstruction* instruction, const ShapeIndex& index, Id id); + ~LogicalBuffer() override; + + // Return the instruction that defines the buffer. + HloInstruction* instruction() const override { return instruction_; } + + // Return the index within the output of the instruction where the buffer is + // defined. Index used defined as in ShapeUtil::GetSubshape() + const ShapeIndex& index() const override { return index_; } + + // Return the shape of the buffer. This reference points into the shape field + // of the instruction defining the buffer. Therefore, the returned shape will + // contain the layout of instruction, if any. + const Shape& shape() const override { + return ShapeUtil::GetSubshape(instruction_->shape(), index_); + } + + string ToString() const override; + + private: + HloInstruction* instruction_; + ShapeIndex index_; + + // Similar to HLO constructs (HloInstruction, etc), pointers are used for + // comparison to equality, so disable all copying. + TF_DISALLOW_COPY_AND_ASSIGN(LogicalBuffer); +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_LOGICAL_BUFFER_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/logical_buffer_analysis.h b/linux/include/tensorflow/compiler/xla/service/logical_buffer_analysis.h new file mode 100644 index 000000000..81f524d84 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/logical_buffer_analysis.h @@ -0,0 +1,94 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_LOGICAL_BUFFER_ANALYSIS_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_LOGICAL_BUFFER_ANALYSIS_H_ + +#include "tensorflow/compiler/xla/service/hlo_instruction.h" +#include "tensorflow/compiler/xla/service/hlo_module.h" +#include "tensorflow/compiler/xla/service/logical_buffer.h" +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/core/lib/hash/hash.h" + +namespace xla { +// A class to create all the logical buffers defined by the HLO ops in a module. +class LogicalBufferAnalysis : public DfsHloVisitorWithDefault { + public: + // Runs points-to analysis on 'module'. + static StatusOr> Run( + const HloModule* module); + + // Returns the logical buffer with the given ID. + LogicalBuffer& GetBuffer(LogicalBuffer::Id id) const; + + // Returns the logical buffer that represents the output of a given HLO + // at a given index. + LogicalBuffer& GetBuffer(HloInstruction* instruction, + const ShapeIndex& index) const; + + const std::vector>& logical_buffers() const { + return logical_buffers_; + } + LogicalBuffer::Id num_logical_buffers() const { return next_buffer_id_; } + + private: + explicit LogicalBufferAnalysis(const HloModule* module) : module_(module) {} + Status Analyze(); + + // The module this analysis is performed on. + const HloModule* module_; + + // Create a new logical buffer and return a reference to it. The newly created + // buffer is stored in an internal vector of LogicalBuffers and can be + // accessed with GetBuffer. + void NewLogicalBuffer(HloInstruction* instruction, const ShapeIndex& index); + + Status DefaultAction(HloInstruction* hlo_instruction) override; + Status HandleTuple(HloInstruction* tuple) override; + Status HandleGetTupleElement(HloInstruction* get_tuple_element) override; + Status HandleBitcast(HloInstruction* bitcast) override; + Status HandleDomain(HloInstruction* domain) override; + Status HandleCopy(HloInstruction* copy) override; + Status HandleRecvDone(HloInstruction* recv_done) override; + Status HandleSend(HloInstruction* send) override; + Status HandleTupleSelect(HloInstruction* tuple_select) override; + + // A map from the buffer ID to the logical buffer + std::vector> logical_buffers_; + + struct Hasher { + size_t operator()( + std::pair p) const { + size_t inst_hash = tensorflow::hash()(p.first); + for (auto index = p.second.begin(); index != p.second.end(); ++index) { + inst_hash = tensorflow::Hash64Combine(*index, inst_hash); + } + return inst_hash; + } + }; + + // A map from an hlo + shape index to the logical buffer representing + // the appropriate output. + std::unordered_map, + LogicalBuffer*, Hasher> + output_buffers_; + + // The ID of the next logical buffer created. + LogicalBuffer::Id next_buffer_id_ = 0; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_LOGICAL_BUFFER_ANALYSIS_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/maybe_owning_device_memory.h b/linux/include/tensorflow/compiler/xla/service/maybe_owning_device_memory.h new file mode 100644 index 000000000..82e7f1183 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/maybe_owning_device_memory.h @@ -0,0 +1,70 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_MAYBE_OWNING_DEVICE_MEMORY_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_MAYBE_OWNING_DEVICE_MEMORY_H_ + +#include "absl/types/optional.h" +#include "absl/types/variant.h" +#include "tensorflow/compiler/xla/service/device_memory_allocator.h" +#include "tensorflow/compiler/xla/service/owning_device_memory.h" + +namespace xla { + +// MaybeOwningDeviceMemory represents either an owned or unowned device memory. +// Like std::variant. When the object goes +// output of scope, it will free the underlying memory if it owns it. +class MaybeOwningDeviceMemory { + public: + MaybeOwningDeviceMemory() = default; + explicit MaybeOwningDeviceMemory(OwningDeviceMemory owned) + : mem_(std::move(owned)) {} + explicit MaybeOwningDeviceMemory(se::DeviceMemoryBase unowned) + : mem_(unowned) {} + MaybeOwningDeviceMemory(MaybeOwningDeviceMemory&&) = default; + ~MaybeOwningDeviceMemory() = default; + + MaybeOwningDeviceMemory& operator=(se::DeviceMemoryBase unowned) { + mem_ = unowned; + return *this; + } + + MaybeOwningDeviceMemory& operator=(OwningDeviceMemory owned) { + mem_ = std::move(owned); + return *this; + } + + MaybeOwningDeviceMemory& operator=(MaybeOwningDeviceMemory&&) = default; + + // Fetches the underlying DeviceMemoryBase from a MaybeOwningDeviceMemory. The + // caller of this function is *not* responsible for freeing the memory. + se::DeviceMemoryBase AsDeviceMemoryBase(); + + // Release the OwningDeviceMemory without freeing it, and moves the ownership + // of the memory buffer from the object to the caller. + // + // A nullopt is returned if the HasOwnership() == false; + absl::optional Release(); + + // Returns true if the device_memory has ownership over underlying memory. + bool HasOwnership() const; + + private: + absl::variant mem_; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_MAYBE_OWNING_DEVICE_MEMORY_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/name_uniquer.h b/linux/include/tensorflow/compiler/xla/service/name_uniquer.h new file mode 100644 index 000000000..8909d0f4f --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/name_uniquer.h @@ -0,0 +1,88 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_NAME_UNIQUER_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_NAME_UNIQUER_H_ + +#include + +#include "absl/container/flat_hash_map.h" +#include "absl/container/flat_hash_set.h" +#include "absl/strings/string_view.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/core/platform/macros.h" + +namespace xla { + +// Simple stateful class that helps generate "unique" names. To use it, simply +// call GetUniqueName as many times as needed. The names returned by +// GetUniqueName are guaranteed to be distinct for this instance of the class. +// Note that the names will be sanitized to match regexp +// "[a-zA-Z_][a-zA-Z0-9_.-]*". +class NameUniquer { + public: + // The separator must contain allowed characters only: "[a-zA-Z0-9_.-]". + explicit NameUniquer(const string& separator = "__"); + + // Get a sanitized unique name in a string, with an optional prefix for + // convenience. + string GetUniqueName(absl::string_view prefix = ""); + + // Sanitizes and returns the name. Unallowed characters will be replaced with + // '_'. The result will match the regexp "[a-zA-Z_][a-zA-Z0-9_.-]*". + static string GetSanitizedName(const string& name); + + private: + // Used to track and generate new identifiers for the same instruction name + // root. + class SequentialIdGenerator { + public: + SequentialIdGenerator() = default; + + // Tries to register id as used identifier. If id is not already used, the + // id itself will be returned. Otherwise a new one will be generated, and + // returned. + int64 RegisterId(int64 id) { + if (used_.insert(id).second) { + return id; + } + while (!used_.insert(next_).second) { + ++next_; + } + return next_++; + } + + private: + // The next identifier to be tried. + int64 next_ = 0; + + // Set of all the identifiers which has been used. + absl::flat_hash_set used_; + }; + + // The string to use to separate the prefix of the name from the uniquing + // integer value. + string separator_; + + // Map from name prefix to the generator data structure which tracks used + // identifiers and generates new ones. + absl::flat_hash_map generated_names_; + + TF_DISALLOW_COPY_AND_ASSIGN(NameUniquer); +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_NAME_UNIQUER_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/owning_device_memory.h b/linux/include/tensorflow/compiler/xla/service/owning_device_memory.h new file mode 100644 index 000000000..9cf071f0d --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/owning_device_memory.h @@ -0,0 +1,131 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_OWNING_DEVICE_MEMORY_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_OWNING_DEVICE_MEMORY_H_ + +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/stream_executor_no_cuda.h" + +namespace xla { + +// Break circular dependency between this file and device_memory_allocator.h. +class DeviceMemoryAllocator; + +// Owning pointer for memory on a device. +// +// OwningDeviceMemory is an owning pointer like std::unique_ptr, but it can +// point to memory that resides on a "device" (e.g. a GPU). When an +// OwningDeviceMemory goes out of scope, it frees the memory it owns. +// +// We say that an instance of OwningDeviceMemory is "active" if it currently +// owns a (possibly empty) slice of memory on the device. Moving, Forget()'ing, +// Free()'ing, and other actions can deactive an active object. +// +// Note that we can't simply use stream_executor::ScopedDeviceMemory instead of +// OwningDeviceMemory, because ScopedDeviceMemory frees its pointer via a +// StreamExecutor. This class needs to free via a xla::DeviceMemoryAllocator. +class OwningDeviceMemory { + public: + OwningDeviceMemory() : device_ordinal_(-1), allocator_(nullptr) {} + + explicit OwningDeviceMemory(se::DeviceMemoryBase mem, int device_ordinal, + DeviceMemoryAllocator* allocator) + : mem_(mem), device_ordinal_(device_ordinal), allocator_(allocator) { + CHECK(allocator != nullptr) << "allocator cannot be null."; + } + + OwningDeviceMemory(OwningDeviceMemory&& other) + : mem_(other.mem_), + device_ordinal_(other.device_ordinal_), + allocator_(other.allocator_) { + other.mem_ = se::DeviceMemoryBase(); + other.allocator_ = nullptr; + } + + OwningDeviceMemory& operator=(OwningDeviceMemory&& other) { + if (allocator_ != nullptr) { + Free(); + } + mem_ = other.mem_; + device_ordinal_ = other.device_ordinal_; + allocator_ = other.allocator_; + + other.mem_ = se::DeviceMemoryBase(); + other.allocator_ = nullptr; + return *this; + } + + // Deactivates this instance if it's active. Nop if it's not active. + OwningDeviceMemory& operator=(std::nullptr_t) { + if (allocator_ != nullptr) { + Free(); + } + return *this; + } + + ~OwningDeviceMemory() { + if (allocator_ != nullptr) { + Free(); + } + } + + // The returned allocator is nonnull iff this object is active. + DeviceMemoryAllocator* allocator() const { return allocator_; } + + int device_ordinal() const { return device_ordinal_; } + + // Gets the device memory pointer. + const void* opaque() const { return mem_.opaque(); } + void* opaque() { return mem_.opaque(); } + + uint64 size() const { return mem_.size(); } + + // Determines whether this wraps a null pointer. + // + // !is_null() is sufficient but not necessary to imply `this` is active. + bool is_null() const { return mem_.is_null(); } + + se::DeviceMemoryBase AsDeviceMemoryBase() { + return se::DeviceMemoryBase(opaque(), size(), /*is_sub_buffer=*/false); + } + + // Returns the wrapped DeviceMemoryBase without freeing it, and deactivates + // this object. Precondition: `this` is active. + TF_MUST_USE_RESULT se::DeviceMemoryBase Forget() { + CHECK(allocator_ != nullptr) + << "Can't call Forget() on an inactive (i.e. moved from, Forget()'ten, " + "or Free()'ed) instance."; + allocator_ = nullptr; + se::DeviceMemoryBase mem(mem_); + mem_ = se::DeviceMemoryBase(); + return mem; + } + + // Frees the wrapped DeviceMemoryBase and deactivates this object. + // Precondition: `this` is active. + void Free(); + + private: + se::DeviceMemoryBase mem_; + int device_ordinal_; + DeviceMemoryAllocator* allocator_; // Null if this object is inactive. +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_OWNING_DEVICE_MEMORY_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/pattern_matcher.h b/linux/include/tensorflow/compiler/xla/service/pattern_matcher.h new file mode 100644 index 000000000..380cde0e6 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/pattern_matcher.h @@ -0,0 +1,1260 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_PATTERN_MATCHER_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_PATTERN_MATCHER_H_ + +#include "absl/strings/string_view.h" +#include "absl/utility/utility.h" +#include "tensorflow/compiler/xla/layout_util.h" +#include "tensorflow/compiler/xla/literal_util.h" +#include "tensorflow/compiler/xla/service/hlo_casting_utils.h" +#include "tensorflow/compiler/xla/service/hlo_instruction.h" +#include "tensorflow/compiler/xla/service/hlo_instructions.h" +#include "tensorflow/compiler/xla/service/hlo_opcode.h" +#include "tensorflow/compiler/xla/shape_util.h" + +namespace xla { + +// A pattern matcher for HloInstructions, Shapes, and Layouts. +// +// The Match function's first argument must be HloInstruction*, Shape*, or +// Layout*. The second argument is a pattern that will be matched against the +// first argument, as described below. +// +// Patterns are constructed using the match::Op, match::Shape, or match::Layout +// functions. By default, the returned patterns will match any HloInstruction, +// Shape, or Layout, respectively. However the match can be made more specific +// by using the pattern's modifier methods, for example: +// +// match::Op().WithOpcode(HloOpcode::kAdd).WithOperand( +// 0, match::Op().WithOpcode(HloOpcode::kConstant)) +// +// This pattern will match Add instructions whose first operand is a constant. +// +// Each pattern type has the following modifiers: +// +// Op(): +// - WithName: match operations with the given name +// - WithOpcode: match operations with the given opcode +// - WithShape: match operations whose shape matches the given pattern +// - WithOperand: match operations whose operand matches the given pattern +// +// Shape(): +// - EqualTo: matches shapes that are equal to the argument +// - CompatibleTo: matches shapes that are compatible to the argument +// - IsScalar/IsArray/IsTuple: matches scalar/array/tuple shapes +// - IsDenseArray/IsSparseArray: matches arrays with dense/sparse format +// - WithLayout: match shapes whose layout matches the given pattern +// - WithLayoutEqualTo: matches shapes whose layouts equal the argument +// - WithSubshape: matches tuple shapes whose subshape matches the given +// pattern +// - WithSubshapeEqualTo: matches shapes with a subshape equal the argument +// - WithElementType: matches array/scalar shapes with the given element +// type +// - WithRank: matches array/scalar types with the given rank +// +// Layout(): +// - EqualTo: matches layouts that are equal to the argument +// - WithDenseFormat/WithSparseFormat: matches layouts with dense/sparse +// format +// +// Op(), Shape(), and Layout() may be passed an argument of type +// HloInstruction**, Shape**, or Layout**, respectively, or const versions of +// these pointers. If the pattern is matched, the address of the matched value +// will be "captured" and stored at this location. +// +// For example: +// HloInstruction* foo = ...; +// HloInstruction* matched_operand; +// CHECK(Match(foo, +// match::Op().WithOperand(0, match::Op(&matched_operand)))); +// +// Helpers are provided for common nullary, unary, binary, and ternary +// instructions. These helpers can be called with no arguments, in which case +// they will match any instruction matching the opcode. They may also be called +// with matches for the operands and with an optional capture. (The capture must +// be the first argument.) Some examples of these helpers and their equivalents +// are provided below. +// +// Example nullary instruction: +// Param() == Op().WithOpcode(HloOpcode::kParam) +// Param(&a) == Op(&a).WithOpcode(HloOpcode::kParam) +// +// Example unary instruction: +// Abs() == Op().WithOpcode(HloOpcode::kAbs) +// Abs(Op(&a)) == Op().WithOpcode(HloOpcode::kAbs) +// .WithOperand(0, Op(&a))) +// Abs(&a, Op(&b)) == Op(&a).WithOpcode(HloOpcode::kAbs) +// .WithOperand(0, Op(&b)) +// +// Example binary instruction: +// Add() == Op().WithOpcode(HloOpcode::kAdd) +// Add(Op(&a), Op(&b)) == Op().WithOpcode(HloOpcode::kAdd) +// .WithOperand(0, Op(&a)) +// .WithOperand(1, Op(&b)) +// Add(&a, Op(&b), Op(&c)) == Op(&a).WithOpcode(HloOpcode::kAdd) +// .WithOperand(0, Op(&b)) +// .WithOperand(1, Op(&c)) +// +// Example ternary instruction: +// Clamp() == Op().WithOpcode(HloOpcode::kClamp) +// Clamp(Op(&a), Op(&b), Op(&c)) == Op().WithOpcode(HloOpcode::kClamp) +// .WithOperand(0, Op(&a)) +// .WithOperand(1, Op(&b)) +// .WithOperand(2, Op(&c)) +// Clamp(&a, Op(&b), Op(&c), Op(&d)) == Op(&a).WithOpcode(HloOpcode::kClamp) +// .WithOperand(0, Op(&b)) +// .WithOperand(1, Op(&c)) +// .WithOperand(2, Op(&d)) +// + +struct MatchOption { + // If true, actually capture matched item into the user pointer. + bool capture; +}; + +template +bool Match(Value* value, const Pattern& pattern, + MatchOption option = {/*.capture=*/true}) { + if (option.capture) { + auto new_option = option; + new_option.capture = false; + if (!pattern.Match(value, new_option)) { + return false; + } + } + return pattern.Match(value, option); +} + +namespace match { + +namespace detail { + +template +class AllOfPattern { + public: + explicit AllOfPattern(const Patterns&... patterns) : patterns_(patterns...) {} + + bool Match(const Item* item, MatchOption option) const { + bool matched = MatchImpl(item, option, std::integral_constant()); + // This invariant is guaranteed by the top-level Match and AnyOf. + DCHECK(matched || !option.capture); + return matched; + } + + bool Match(Item* item, MatchOption option) const { + bool matched = MatchImpl(item, option, std::integral_constant()); + // This invariant is guaranteed by the top-level Match and AnyOf. + DCHECK(matched || !option.capture); + return matched; + } + + private: + template + bool MatchImpl(ItemType* item, MatchOption option, + std::integral_constant) const { + return std::get(patterns_).Match(item, option) && + MatchImpl(item, option, std::integral_constant()); + } + + template + bool MatchImpl(ItemType* item, MatchOption option, + std::integral_constant) const { + return true; + } + + std::tuple patterns_; +}; + +} // namespace detail + +// Returns a pattern that represents the conjunction of all input patterns. All +// patterns need to match in order to have the AllOf pattern match. +// +// TODO(timshen): Currently AllOf is still nested, e.g. AllOf, B> is +// not AllOf. We might want to flatten the AllOf type structure if the +// C++ compile error message gets annoying. +template +detail::AllOfPattern::type, Patterns...> AllOf( + const Patterns&... patterns) { + return detail::AllOfPattern::type, + Patterns...>(patterns...); +} + +namespace detail { + +template +class LayoutPattern; + +// The base LayoutPattern implementation. Matches only if the layout is not +// nullptr. +class LayoutPatternBaseImpl { + public: + bool Match(const ::xla::Layout* layout, MatchOption option) const { + return layout != nullptr; + } +}; + +// A LayoutPattern implementation that matches only if the layout equals a +// Layout proto. +class LayoutPatternEqualImpl { + public: + explicit constexpr LayoutPatternEqualImpl(const ::xla::Layout* layout) + : layout_(layout) {} + + bool Match(const ::xla::Layout* layout, MatchOption option) const { + return LayoutUtil::Equal(*layout_, *layout); + } + + private: + const ::xla::Layout* layout_; +}; + +// A LayoutPattern implementation that matches only if the layout has a given +// format. +class LayoutPatternFormatImpl { + public: + explicit constexpr LayoutPatternFormatImpl(Format format) : format_(format) {} + + bool Match(const ::xla::Layout* layout, MatchOption option) const { + return layout->format() == format_; + } + + private: + Format format_; +}; + +// A pattern that matches Layouts. +template +class LayoutPattern { + private: + template + LayoutPattern> + AppendImpl(NewImpl new_impl) const { + return LayoutPattern>( + AllOf(impl_, std::move(new_impl)), matched_layout_); + } + + public: + explicit constexpr LayoutPattern(const Impl& impl, + LayoutType** matched_layout) + : impl_(impl), matched_layout_(matched_layout) {} + + // Returns true and captures the layout iff it matches the pattern. + bool Match(const ::xla::Layout* layout, MatchOption option) const { + if (impl_.Match(layout, option)) { + if (option.capture && matched_layout_) { + *matched_layout_ = layout; + } + return true; + } + return false; + } + + // Returns true and captures the layout iff it matches the pattern. + bool Match(::xla::Layout* layout, MatchOption option) const { + if (impl_.Match(layout, option)) { + if (option.capture && matched_layout_) { + *matched_layout_ = layout; + } + return true; + } + return false; + } + + // Modifies the pattern to match only if the layout equals the given proto. + // The layout must outlive the returned pattern. + constexpr auto EqualTo(const ::xla::Layout* layout) const + -> decltype(this->AppendImpl(LayoutPatternEqualImpl(layout))) { + return AppendImpl(LayoutPatternEqualImpl(layout)); + } + + // Modifies the pattern to match only if the layout has a dense format. + constexpr auto WithDenseFormat() const + -> decltype(this->AppendImpl(LayoutPatternFormatImpl(DENSE))) { + return AppendImpl(LayoutPatternFormatImpl(DENSE)); + } + + // Modifies the pattern to match only if the layout has a sparse format. + constexpr auto WithSparseFormat() const + -> decltype(this->AppendImpl(LayoutPatternFormatImpl(SPARSE))) { + return AppendImpl(LayoutPatternFormatImpl(SPARSE)); + } + + private: + Impl impl_; + LayoutType** matched_layout_; +}; + +template +class AnyOfPattern { + public: + explicit AnyOfPattern(const Patterns&... patterns) : patterns_(patterns...) {} + + bool Match(const Item* item, MatchOption option) const { + return MatchImpl(item, option, std::integral_constant()); + } + + bool Match(Item* item, MatchOption option) const { + return MatchImpl(item, option, std::integral_constant()); + } + + private: + template + bool MatchImpl(ItemType* item, MatchOption option, + std::integral_constant) const { + auto new_option = option; + new_option.capture = false; + // Try to match the sub-pattern without capturing behavior. + if (std::get(patterns_).Match(item, new_option)) { + // Capture the branch. + if (option.capture) { + // TODO(timshen): Currently the behavior can be exponential. Optimize it + // with memoization or recording the matched sub-pattern index, if it + // takes too long to run. + // + // Specifically, the "memoization" approach is to create an empty + // container with the key (pattern, instruction), and value as whether + // matched or not. + // + // Alternatively, we may run the pattern matching with captures off, but + // instead record a "trace" somewhere, indicating how exactly the + // pattern matches the input. For example, the trace information for + // AnyOf will be a runtime number indicate which sub-pattern is matched. + // Then we run another pass to do captures only with the help of the + // trace. + bool ret = std::get(patterns_).Match(item, option); + DCHECK(ret); + } + return true; + } + return MatchImpl(item, option, std::integral_constant()); + } + + template + bool MatchImpl(ItemType* item, MatchOption option, + std::integral_constant) const { + return false; + } + + std::tuple patterns_; +}; + +} // namespace detail + +// Returns a pattern that represents the logical disjunction of the input +// patterns. The returned pattern matches from left to right, and stops on the +// first match. +template +detail::AnyOfPattern::type, Patterns...> AnyOf( + const Patterns&... patterns) { + return detail::AnyOfPattern::type, + Patterns...>(patterns...); +} + +// Creates a layout pattern that will capture the matched layout in the +// argument. +inline constexpr detail::LayoutPattern +Layout(const ::xla::Layout** matched_layout = nullptr) { + return detail::LayoutPattern( + detail::LayoutPatternBaseImpl(), matched_layout); +} + +// Creates a layout pattern that will capture the matched layout in the +// argument. +inline constexpr detail::LayoutPattern<::xla::Layout, + detail::LayoutPatternBaseImpl> +Layout(::xla::Layout** matched_layout) { + return detail::LayoutPattern<::xla::Layout, detail::LayoutPatternBaseImpl>( + detail::LayoutPatternBaseImpl(), matched_layout); +} + +namespace detail { + +template +class ShapePattern; + +// The base ShapePattern implementation. Matches only if the shape is not +// nullptr. +class ShapePatternBaseImpl { + public: + bool Match(const ::xla::Shape* shape, MatchOption option) const { + return shape != nullptr; + } +}; + +// A ShapePattern implementation that matches only if the shape equals a Shape +// proto. +class ShapePatternEqualImpl { + public: + explicit constexpr ShapePatternEqualImpl(const ::xla::Shape* shape) + : shape_(shape) {} + + bool Match(const ::xla::Shape* shape, MatchOption option) const { + return ShapeUtil::Equal(*shape_, *shape); + } + + private: + const ::xla::Shape* shape_; +}; + +// A ShapePattern implementation that matches only if the shape is compatible to +// a Shape proto. +class ShapePatternCompatibleImpl { + public: + explicit constexpr ShapePatternCompatibleImpl(const ::xla::Shape* shape) + : shape_(shape) {} + + bool Match(const ::xla::Shape* shape, MatchOption option) const { + return ShapeUtil::Compatible(*shape_, *shape); + } + + private: + const ::xla::Shape* shape_; +}; + +// A ShapePattern implementation that matches only if the shape has a given +// element type. +class ShapePatternElementTypeImpl { + public: + explicit constexpr ShapePatternElementTypeImpl(PrimitiveType element_type) + : element_type_(element_type) {} + + bool Match(const ::xla::Shape* shape, MatchOption option) const { + return shape->element_type() == element_type_; + } + + private: + PrimitiveType element_type_; +}; + +// A ShapePattern implementation that matches only if the shape is scalar. +class ShapePatternIsScalarImpl { + public: + explicit constexpr ShapePatternIsScalarImpl() {} + + bool Match(const ::xla::Shape* shape, MatchOption option) const { + return ShapeUtil::IsScalar(*shape); + } +}; + +// A ShapePattern implementation that matches only if the shape is an array +class ShapePatternIsArrayImpl { + public: + explicit constexpr ShapePatternIsArrayImpl() {} + + bool Match(const ::xla::Shape* shape, MatchOption option) const { + return ShapeUtil::IsArray(*shape); + } +}; + +// A ShapePattern implementation that matches only if the shape is a tuple. +class ShapePatternIsTupleImpl { + public: + explicit constexpr ShapePatternIsTupleImpl() {} + + bool Match(const ::xla::Shape* shape, MatchOption option) const { + return ShapeUtil::IsTuple(*shape); + } +}; + +// A ShapePattern implementation that matches only if the shape has a given +// rank. +class ShapePatternRankImpl { + public: + explicit constexpr ShapePatternRankImpl(int64 rank) : rank_(rank) {} + + bool Match(const ::xla::Shape* shape, MatchOption option) const { + return ShapeUtil::Rank(*shape) == rank_; + } + + private: + int64 rank_; +}; + +// A ShapePattern implementation that matches only if the shape has a layout +// that matches a given pattern. +template +class ShapePatternLayoutImpl { + public: + explicit constexpr ShapePatternLayoutImpl( + const LayoutPattern& layout) + : layout_(layout) {} + + bool Match(const ::xla::Shape* shape, MatchOption option) const { + return LayoutUtil::HasLayout(*shape) && + layout_.Match(&shape->layout(), option); + } + + bool Match(Shape* shape, MatchOption option) const { + return LayoutUtil::HasLayout(*shape) && + layout_.Match(shape->mutable_layout(), option); + } + + private: + LayoutPattern layout_; +}; + +// A ShapePattern implementation that matches only if the shape has a subshape +// that matches a given pattern. +template +class ShapePatternSubshapeImpl { + public: + explicit ShapePatternSubshapeImpl( + ShapeIndexView index, + const ShapePattern& subshape) + : index_(index), subshape_(subshape) {} + + bool Match(const ::xla::Shape* shape, MatchOption option) const { + return ShapeUtil::IndexIsValid(*shape, index_) && + subshape_.Match(&ShapeUtil::GetSubshape(*shape, index_), option); + } + + bool Match(::xla::Shape* shape, MatchOption option) const { + return ShapeUtil::IndexIsValid(*shape, index_) && + subshape_.Match(ShapeUtil::GetMutableSubshape(shape, index_), + option); + } + + private: + ShapeIndexView index_; + ShapePattern subshape_; +}; + +// A pattern that matches Shapes. +template +class ShapePattern { + private: + template + ShapePattern> AppendImpl( + NewImpl new_impl) const { + return ShapePattern>( + AllOf(impl_, std::move(new_impl)), matched_shape_); + } + + public: + explicit constexpr ShapePattern(const Impl& impl, ShapeType** matched_shape) + : impl_(impl), matched_shape_(matched_shape) {} + + // Returns true and captures the shape iff it matches the pattern. + bool Match(const ::xla::Shape* shape, MatchOption option) const { + if (impl_.Match(shape, option)) { + if (option.capture && matched_shape_) { + *matched_shape_ = shape; + } + return true; + } + return false; + } + + // Returns true and captures the shape iff it matches the pattern. + bool Match(::xla::Shape* shape, MatchOption option) const { + if (impl_.Match(shape, option)) { + if (option.capture && matched_shape_) { + *matched_shape_ = shape; + } + return true; + } + return false; + } + + // Modifies the pattern to match only if the shape equals the given proto. + // The layout must outlive the returned pattern. + constexpr auto EqualTo(const ::xla::Shape* shape) const + -> decltype(this->AppendImpl(ShapePatternEqualImpl(shape))) { + return AppendImpl(ShapePatternEqualImpl(shape)); + } + + // Modifies the pattern to match only if the shape is compatible to the given + // proto. The layout must outlive the returned pattern. + constexpr auto CompatibleTo(const ::xla::Shape* shape) const + -> decltype(this->AppendImpl(ShapePatternCompatibleImpl(shape))) { + return AppendImpl(ShapePatternCompatibleImpl(shape)); + } + + // Modifies the pattern to match only if the shape has the given element type. + constexpr auto WithElementType(PrimitiveType element_type) const + -> decltype(this->AppendImpl(ShapePatternElementTypeImpl(element_type))) { + return AppendImpl(ShapePatternElementTypeImpl(element_type)); + } + + // Modifies the pattern to match only if the shape is scalar. + constexpr auto IsScalar() const + -> decltype(this->AppendImpl(ShapePatternIsScalarImpl())) { + return AppendImpl(ShapePatternIsScalarImpl()); + } + + // Modifies the pattern to match only if the shape is an array. + constexpr auto IsArray() const + -> decltype(this->AppendImpl(ShapePatternIsArrayImpl())) { + return AppendImpl(ShapePatternIsArrayImpl()); + } + + // Modifies the pattern to match only if the shape is a tuple. + constexpr auto IsTuple() const + -> decltype(this->AppendImpl(ShapePatternIsTupleImpl())) { + return AppendImpl(ShapePatternIsTupleImpl()); + } + + // Modifies the pattern to match only if the shape has the given rank. + constexpr auto WithRank(int64 rank) const + -> decltype(this->AppendImpl(ShapePatternRankImpl(rank))) { + return AppendImpl(ShapePatternRankImpl(rank)); + } + + // Modifies the pattern to match only if the shape has a layout that matches + // the given pattern. + template + auto WithLayout(const LayoutPattern& layout) const + -> decltype(this->AppendImpl( + ShapePatternLayoutImpl(layout))) { + return AppendImpl(ShapePatternLayoutImpl(layout)); + } + + constexpr auto WithLayoutEqualTo(const ::xla::Layout* layout) const + -> decltype(this->WithLayout(Layout().EqualTo(layout))) { + return WithLayout(Layout().EqualTo(layout)); + } + + constexpr auto IsDenseArray() const + -> decltype(this->WithLayout(Layout().WithDenseFormat())) { + return WithLayout(Layout().WithDenseFormat()); + } + + constexpr auto IsSparseArray() const + -> decltype(this->WithLayout(Layout().WithSparseFormat())) { + return WithLayout(Layout().WithSparseFormat()); + } + + // Modifies the pattern to match only if the shape has a subshape that matches + // the given pattern. + template + auto WithSubshape(ShapeIndexView index, + const ShapePattern& subshape) + const -> decltype(this->AppendImpl( + ShapePatternSubshapeImpl(index, + subshape))) { + return AppendImpl( + ShapePatternSubshapeImpl(index, subshape)); + } + + ShapePattern>>> + WithSubshapeEqualTo(ShapeIndexView index, const ::xla::Shape* shape) const { + return WithSubshape(index, + ShapePattern( + ShapePatternBaseImpl(), nullptr) + .EqualTo(shape)); + } + + ShapePattern>>> + WithSubshapeCompatibleTo(ShapeIndexView index, + const ::xla::Shape* shape) const { + return WithSubshape(index, + ShapePattern( + ShapePatternBaseImpl(), nullptr) + .CompatibleTo(shape)); + } + + private: + Impl impl_; + ShapeType** matched_shape_; +}; + +} // namespace detail + +// Creates a shape pattern that will capture the matched layout in the argument. +inline constexpr detail::ShapePattern +Shape(const ::xla::Shape** matched_shape = nullptr) { + return detail::ShapePattern( + detail::ShapePatternBaseImpl(), matched_shape); +} + +// Creates a shape pattern that will capture the matched layout in the argument. +inline constexpr detail::ShapePattern<::xla::Shape, + detail::ShapePatternBaseImpl> +Shape(::xla::Shape** matched_shape) { + return detail::ShapePattern<::xla::Shape, detail::ShapePatternBaseImpl>( + detail::ShapePatternBaseImpl(), matched_shape); +} + +namespace detail { + +template +class HloInstructionPattern; + +// The base HloInstructionPattern implementation. Matches only if the +// instruction is not nullptr. +class HloInstructionPatternBaseImpl { + public: + bool Match(const ::xla::HloInstruction* inst, MatchOption option) const { + return inst != nullptr; + } +}; + +// An HloInstructionPattern implementation that matches only if the instruction +// has a given name. +class HloInstructionPatternNameImpl { + public: + explicit HloInstructionPatternNameImpl(absl::string_view name) + : name_(name) {} + + bool Match(const ::xla::HloInstruction* inst, MatchOption option) const { + return inst->name() == name_; + } + + private: + absl::string_view name_; +}; + +// An HloInstructionPattern implementation that matches only if the instruction +// has a given opcode. +class HloInstructionPatternOpcodeImpl { + public: + explicit constexpr HloInstructionPatternOpcodeImpl(HloOpcode opcode, + bool invert) + : opcode_(opcode), invert_(invert) {} + + bool Match(const ::xla::HloInstruction* inst, MatchOption option) const { + return (invert_ ^ (inst->opcode() == opcode_)); + } + + private: + HloOpcode opcode_; + bool invert_; +}; + +// An HloInstructionPattern implementation that matches only if the instruction +// has a shape that matches a given pattern. +template +class HloInstructionPatternShapeImpl { + public: + explicit constexpr HloInstructionPatternShapeImpl( + const ShapePattern& shape) + : shape_(shape) {} + + bool Match(const ::xla::HloInstruction* inst, MatchOption option) const { + return shape_.Match(&inst->shape(), option); + } + + bool Match(::xla::HloInstruction* inst, MatchOption option) const { + return shape_.Match(inst->mutable_shape(), option); + } + + private: + ShapePattern shape_; +}; + +// An HloInstructionPattern implementation that matches only if the instruction +// has an operand that matches a given pattern. +template +class HloInstructionPatternOperandImpl { + public: + explicit constexpr HloInstructionPatternOperandImpl( + int64 operand_index, + const HloInstructionPattern& operand) + : operand_index_(operand_index), operand_(operand) {} + + bool Match(const ::xla::HloInstruction* inst, MatchOption option) const { + return operand_index_ < inst->operand_count() && + operand_.Match(inst->operand(operand_index_), option); + } + + bool Match(::xla::HloInstruction* inst, MatchOption option) const { + return operand_index_ < inst->operand_count() && + operand_.Match(inst->mutable_operand(operand_index_), option); + } + + private: + int64 operand_index_; + HloInstructionPattern operand_; +}; + +// An HloInstructionPattern implementation that matches only if the instruction +// is a fusion node with a particular kind. +class HloInstructionPatternFusionKindImpl { + public: + explicit constexpr HloInstructionPatternFusionKindImpl( + ::xla::HloInstruction::FusionKind kind) + : kind_(kind) {} + + bool Match(const ::xla::HloInstruction* inst, MatchOption option) const { + return inst->opcode() == HloOpcode::kFusion && inst->fusion_kind() == kind_; + } + + bool Match(::xla::HloInstruction* inst, MatchOption option) const { + return inst->opcode() == HloOpcode::kFusion && inst->fusion_kind() == kind_; + } + + private: + ::xla::HloInstruction::FusionKind kind_; +}; + +// An HloInstructionPattern implementation that matches only if the instruction +// is a kGetTupleElement with a particular tuple index. +class HloInstructionPatternTupleIndexImpl { + public: + explicit constexpr HloInstructionPatternTupleIndexImpl(int64 tuple_index) + : tuple_index_(tuple_index) {} + + bool Match(const ::xla::HloInstruction* inst, MatchOption option) const { + return inst->opcode() == HloOpcode::kGetTupleElement && + inst->tuple_index() == tuple_index_; + } + + bool Match(::xla::HloInstruction* inst, MatchOption option) const { + return inst->opcode() == HloOpcode::kGetTupleElement && + inst->tuple_index() == tuple_index_; + } + + private: + int64 tuple_index_; +}; + +template +class HloPredicatePatternImpl { + public: + explicit HloPredicatePatternImpl(Predicate pred) : pred_(std::move(pred)) {} + + bool Match(const ItemType* item, MatchOption option) const { + return pred_(item); + } + + bool Match(ItemType* item, MatchOption option) const { return pred_(item); } + + private: + Predicate pred_; +}; + +struct PatternFriend; + +// A pattern that matches HloInstructions. +template +class HloInstructionPattern { + private: + template + HloInstructionPattern> + AppendImpl(NewImpl new_impl) const { + return HloInstructionPattern< + HloInstructionType, AllOfPattern<::xla::HloInstruction, Impl, NewImpl>>( + AllOf(impl_, std::move(new_impl)), matched_inst_); + } + + public: + explicit constexpr HloInstructionPattern(const Impl& impl, + HloInstructionType** matched_inst) + : impl_(impl), matched_inst_(matched_inst) {} + + // Returns true and captures the instruction iff it matches the pattern. + bool Match(const ::xla::HloInstruction* inst, MatchOption option) const { + if (impl_.Match(inst, option)) { + if (option.capture && matched_inst_) { + *matched_inst_ = inst; + } + return true; + } + return false; + } + + // Returns true and captures the instruction iff it matches the pattern. + bool Match(::xla::HloInstruction* inst, MatchOption option) const { + if (impl_.Match(inst, option)) { + if (option.capture && matched_inst_) { + *matched_inst_ = inst; + } + return true; + } + return false; + } + + // Modifies the pattern to match only if the instruction has the given name. + auto WithName(absl::string_view name) const + -> decltype(this->AppendImpl(HloInstructionPatternNameImpl(name))) { + return AppendImpl(HloInstructionPatternNameImpl(name)); + } + + // Modifies the pattern to match only if the instruction has the given opcode. + auto WithOpcode(HloOpcode opcode) const + -> decltype(this->AppendImpl(HloInstructionPatternOpcodeImpl(opcode, + false))) { + return AppendImpl(HloInstructionPatternOpcodeImpl(opcode, false)); + } + + // Modifies the pattern to match only if the instruction does not have the + // given opcode. + auto WithoutOpcode(HloOpcode opcode) const + -> decltype(this->AppendImpl(HloInstructionPatternOpcodeImpl(opcode, + true))) { + return AppendImpl(HloInstructionPatternOpcodeImpl(opcode, true)); + } + + // Modifies the pattern to match only if the instruction is a constant. + constexpr auto IsConstant() const + -> decltype(this->WithOpcode(HloOpcode::kConstant)) { + return WithOpcode(HloOpcode::kConstant); + } + + // Modifies the pattern to match only if the instruction is not a constant. + constexpr auto IsNonConstant() const + -> decltype(this->WithoutOpcode(HloOpcode::kConstant)) { + return WithoutOpcode(HloOpcode::kConstant); + } + + // Modifies the pattern to match only if the instruction has a shape that + // matches the given pattern. + template + constexpr auto WithShape(const ShapePattern& shape) + const -> decltype(this->AppendImpl( + HloInstructionPatternShapeImpl(shape))) { + return AppendImpl( + HloInstructionPatternShapeImpl(shape)); + } + + // Modifies the pattern to match only if the instruction has an operand that + // matches the given pattern. + template + constexpr auto WithOperand( + int64 operand_index, + const HloInstructionPattern& operand) const + -> decltype(this->AppendImpl( + HloInstructionPatternOperandImpl( + operand_index, operand))) { + return AppendImpl( + HloInstructionPatternOperandImpl( + operand_index, operand)); + } + + // Modifies the pattern to match only if the instruction is a fusion node with + // the given kind. + constexpr auto WithFusionKind(HloInstruction::FusionKind kind) const + -> decltype(this->AppendImpl(HloInstructionPatternFusionKindImpl(kind))) { + return AppendImpl(HloInstructionPatternFusionKindImpl(kind)); + } + + // Modifies the pattern to match only if the instruction is a + // get-tuple-element with the given tuple index. + constexpr auto WithTupleIndex(int64 tuple_index) const -> decltype( + this->AppendImpl(HloInstructionPatternTupleIndexImpl(tuple_index))) { + return AppendImpl(HloInstructionPatternTupleIndexImpl(tuple_index)); + } + + private: + template + constexpr auto WithPredicate(Predicate pred) const -> decltype( + this->AppendImpl(HloPredicatePatternImpl( + std::move(pred)))) { + return AppendImpl( + HloPredicatePatternImpl(std::move(pred))); + } + + friend struct PatternFriend; + + Impl impl_; + HloInstructionType** matched_inst_; +}; + +} // namespace detail + +// Creates an instruction pattern that will capture the matched instruction in +// the argument. +inline constexpr detail::HloInstructionPattern< + const ::xla::HloInstruction, detail::HloInstructionPatternBaseImpl> +Op(const ::xla::HloInstruction** matched_inst = nullptr) { + return detail::HloInstructionPattern( + detail::HloInstructionPatternBaseImpl(), matched_inst); +} + +// Creates an instruction pattern that will capture the matched instruction in +// the argument. +inline constexpr detail::HloInstructionPattern< + ::xla::HloInstruction, detail::HloInstructionPatternBaseImpl> +Op(::xla::HloInstruction** matched_inst) { + return detail::HloInstructionPattern<::xla::HloInstruction, + detail::HloInstructionPatternBaseImpl>( + detail::HloInstructionPatternBaseImpl(), matched_inst); +} + +// Helpers for nullary instructions. +#define XLA_NULLOP_PATTERN(NAME) \ + inline auto NAME()->decltype(Op().WithOpcode(HloOpcode::k##NAME)) { \ + return Op().WithOpcode(HloOpcode::k##NAME); \ + } \ + \ + template \ + inline auto NAME(HloInstructionType** matched_inst) \ + ->decltype(Op(matched_inst).WithOpcode(HloOpcode::k##NAME)) { \ + return Op(matched_inst).WithOpcode(HloOpcode::k##NAME); \ + } +XLA_NULLOP_PATTERN(Constant) +XLA_NULLOP_PATTERN(Parameter) +XLA_NULLOP_PATTERN(Iota) +#undef XLA_NULLOP_PATTERN + +// Helpers for unary instructions. +#define XLA_UNOP_PATTERN(NAME) \ + inline auto NAME()->decltype(Op().WithOpcode(HloOpcode::k##NAME)) { \ + return Op().WithOpcode(HloOpcode::k##NAME); \ + } \ + \ + template \ + inline auto NAME(Arg&& arg)->decltype( \ + Op().WithOpcode(HloOpcode::k##NAME) \ + .WithOperand(0, std::forward(arg))) { \ + return Op() \ + .WithOpcode(HloOpcode::k##NAME) \ + .WithOperand(0, std::forward(arg)); \ + } \ + \ + template \ + inline auto NAME(HloInstructionType** matched_inst, Arg&& arg) \ + ->decltype(Op(matched_inst) \ + .WithOpcode(HloOpcode::k##NAME) \ + .WithOperand(0, std::forward(arg))) { \ + return Op(matched_inst) \ + .WithOpcode(HloOpcode::k##NAME) \ + .WithOperand(0, std::forward(arg)); \ + } +XLA_UNOP_PATTERN(Abs) +XLA_UNOP_PATTERN(RoundNearestAfz) +XLA_UNOP_PATTERN(Bitcast) +XLA_UNOP_PATTERN(Broadcast) +XLA_UNOP_PATTERN(Ceil) +XLA_UNOP_PATTERN(Copy) +XLA_UNOP_PATTERN(Cos) +XLA_UNOP_PATTERN(Exp) +XLA_UNOP_PATTERN(Fft) +XLA_UNOP_PATTERN(Floor) +XLA_UNOP_PATTERN(GetTupleElement) +XLA_UNOP_PATTERN(Imag) +XLA_UNOP_PATTERN(Infeed) +XLA_UNOP_PATTERN(IsFinite) +XLA_UNOP_PATTERN(Log) +XLA_UNOP_PATTERN(Not) +XLA_UNOP_PATTERN(Negate) +XLA_UNOP_PATTERN(Real) +XLA_UNOP_PATTERN(Recv) +XLA_UNOP_PATTERN(RecvDone) +XLA_UNOP_PATTERN(Reduce) +XLA_UNOP_PATTERN(ReducePrecision) +XLA_UNOP_PATTERN(Reshape) +XLA_UNOP_PATTERN(Reverse) +XLA_UNOP_PATTERN(SendDone) +XLA_UNOP_PATTERN(Sign) +XLA_UNOP_PATTERN(Sin) +XLA_UNOP_PATTERN(Sort) +XLA_UNOP_PATTERN(Tanh) +XLA_UNOP_PATTERN(Transpose) +#undef XLA_UNOP_PATTERN + +// Helpers for binary instructions. +#define XLA_BINOP_PATTERN(NAME) \ + inline auto NAME()->decltype(Op().WithOpcode(HloOpcode::k##NAME)) { \ + return Op().WithOpcode(HloOpcode::k##NAME); \ + } \ + \ + template \ + inline auto NAME(Lhs&& lhs, Rhs&& rhs) \ + ->decltype(Op().WithOpcode(HloOpcode::k##NAME) \ + .WithOperand(0, std::forward(lhs)) \ + .WithOperand(1, std::forward(rhs))) { \ + return Op() \ + .WithOpcode(HloOpcode::k##NAME) \ + .WithOperand(0, std::forward(lhs)) \ + .WithOperand(1, std::forward(rhs)); \ + } \ + \ + template \ + inline auto NAME(HloInstructionType** matched_inst, Lhs&& lhs, Rhs&& rhs) \ + ->decltype(Op(matched_inst) \ + .WithOpcode(HloOpcode::k##NAME) \ + .WithOperand(0, std::forward(lhs)) \ + .WithOperand(1, std::forward(rhs))) { \ + return Op(matched_inst) \ + .WithOpcode(HloOpcode::k##NAME) \ + .WithOperand(0, std::forward(lhs)) \ + .WithOperand(1, std::forward(rhs)); \ + } + +#define XLA_COMMUTATIVE_BINOP_PATTERN(NAME) \ + XLA_BINOP_PATTERN(NAME) \ + \ + template \ + inline auto NAME##AnyOrder(Lhs&& lhs, Rhs&& rhs) \ + ->decltype(AnyOf(NAME(lhs, rhs), NAME(rhs, lhs))) { \ + return AnyOf(NAME(lhs, rhs), NAME(rhs, lhs)); \ + } \ + \ + template \ + inline auto NAME##AnyOrder(HloInstructionType** matched_inst, Lhs&& lhs, \ + Rhs&& rhs) \ + ->decltype(AnyOf(NAME(matched_inst, lhs, rhs), \ + NAME(matched_inst, rhs, lhs))) { \ + return AnyOf(NAME(matched_inst, lhs, rhs), \ + NAME(matched_inst, rhs, lhs)); \ + } +XLA_COMMUTATIVE_BINOP_PATTERN(Add) +XLA_BINOP_PATTERN(Atan2) +XLA_BINOP_PATTERN(Divide) +XLA_BINOP_PATTERN(Complex) +XLA_BINOP_PATTERN(Dot) +XLA_COMMUTATIVE_BINOP_PATTERN(Eq) +XLA_BINOP_PATTERN(Gather) +XLA_BINOP_PATTERN(Ge) +XLA_BINOP_PATTERN(Gt) +XLA_BINOP_PATTERN(Le) +XLA_BINOP_PATTERN(Lt) +XLA_COMMUTATIVE_BINOP_PATTERN(Maximum) +XLA_COMMUTATIVE_BINOP_PATTERN(Minimum) +XLA_COMMUTATIVE_BINOP_PATTERN(Multiply) +XLA_COMMUTATIVE_BINOP_PATTERN(Ne) +XLA_BINOP_PATTERN(Outfeed) +XLA_BINOP_PATTERN(Power) +XLA_BINOP_PATTERN(Remainder) +XLA_BINOP_PATTERN(Send) +XLA_BINOP_PATTERN(Subtract) +XLA_COMMUTATIVE_BINOP_PATTERN(And) +XLA_COMMUTATIVE_BINOP_PATTERN(Or) +XLA_BINOP_PATTERN(ShiftLeft) +XLA_BINOP_PATTERN(ShiftRightArithmetic) +XLA_BINOP_PATTERN(ShiftRightLogical) +#undef XLA_COMMUTATIVE_BINOP_PATTERN +#undef XLA_BINOP_PATTERN + +// Helpers for ternary instructions. +#define XLA_TERNOP_PATTERN(NAME) \ + inline auto NAME()->decltype(Op().WithOpcode(HloOpcode::k##NAME)) { \ + return Op().WithOpcode(HloOpcode::k##NAME); \ + } \ + \ + template \ + inline auto NAME(Arg0&& arg0, Arg1&& arg1, Arg2&& arg2) \ + ->decltype(Op().WithOpcode(HloOpcode::k##NAME) \ + .WithOperand(0, std::forward(arg0)) \ + .WithOperand(1, std::forward(arg1)) \ + .WithOperand(2, std::forward(arg2))) { \ + return Op() \ + .WithOpcode(HloOpcode::k##NAME) \ + .WithOperand(0, std::forward(arg0)) \ + .WithOperand(1, std::forward(arg1)) \ + .WithOperand(2, std::forward(arg2)); \ + } \ + \ + template \ + inline auto NAME(HloInstructionType** matched_inst, Arg0&& arg0, \ + Arg1&& arg1, Arg2&& arg2) \ + ->decltype(Op(matched_inst) \ + .WithOpcode(HloOpcode::k##NAME) \ + .WithOperand(0, std::forward(arg0)) \ + .WithOperand(1, std::forward(arg1)) \ + .WithOperand(2, std::forward(arg2))) { \ + return Op(matched_inst) \ + .WithOpcode(HloOpcode::k##NAME) \ + .WithOperand(0, std::forward(arg0)) \ + .WithOperand(1, std::forward(arg1)) \ + .WithOperand(2, std::forward(arg2)); \ + } +XLA_TERNOP_PATTERN(Clamp); +XLA_TERNOP_PATTERN(Select); +#undef XLA_TERNOP_PATTERN + +namespace detail { +struct PatternFriend { + template + static auto ConstantScalar(T constant) -> decltype( + Constant() + .WithShape(match::Shape().IsScalar()) + .WithPredicate( + std::declval>())) { + std::function pred = + [constant](const HloInstruction* instr) { + const auto& literal = Cast(instr)->literal(); + auto status_or_const = LiteralUtil::CreateR0(constant).Convert( + literal.shape().element_type()); + return status_or_const.ok() && + literal == status_or_const.ConsumeValueOrDie(); + }; + + return Constant() + .WithShape(match::Shape().IsScalar()) + .WithPredicate(std::move(pred)); + } +}; +} // namespace detail + +// Helpers for matching non-constant instructions. +inline auto NonConstant() -> decltype(Op().IsNonConstant()) { + return Op().IsNonConstant(); +} + +template +inline auto NonConstant(HloInstructionType** matched_inst) + -> decltype(Op(matched_inst).IsNonConstant()) { + return Op(matched_inst).IsNonConstant(); +} + +// Add overloads for GetTupleElement which take a int64 specifying which tuple +// element is selected. +template +inline auto GetTupleElement(Arg&& arg, int64 tuple_index) + -> decltype(Op().WithOpcode(HloOpcode::kGetTupleElement) + .WithOperand(0, std::forward(arg)) + .WithTupleIndex(tuple_index)) { + return Op() + .WithOpcode(HloOpcode::kGetTupleElement) + .WithOperand(0, std::forward(arg)) + .WithTupleIndex(tuple_index); +} + +template +inline auto GetTupleElement(HloInstructionType** matched_inst, Arg&& arg, + int64 tuple_index) + -> decltype(Op(matched_inst) + .WithOpcode(HloOpcode::kGetTupleElement) + .WithOperand(0, std::forward(arg)) + .WithTupleIndex(tuple_index)) { + return Op(matched_inst) + .WithOpcode(HloOpcode::kGetTupleElement) + .WithOperand(0, std::forward(arg)) + .WithTupleIndex(tuple_index); +} + +template +inline auto ConstantScalar(T constant) + -> decltype(detail::PatternFriend::ConstantScalar(constant)) { + return detail::PatternFriend::ConstantScalar(constant); +} + +} // namespace match + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_PATTERN_MATCHER_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/platform_util.h b/linux/include/tensorflow/compiler/xla/service/platform_util.h new file mode 100644 index 000000000..571451ba4 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/platform_util.h @@ -0,0 +1,74 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_PLATFORM_UTIL_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_PLATFORM_UTIL_H_ + +#include +#include + +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/stream_executor_no_cuda.h" +#include "tensorflow/core/platform/types.h" + +namespace xla { + +// Utilities for querying platforms and devices used by XLA. +class PlatformUtil { + public: + // Returns the platforms present on the system and supported by XLA. + // + // Note that, even if a platform is present with zero devices, if we *do* have + // compilation support for it, it will be returned in this sequence. + static StatusOr> GetSupportedPlatforms(); + + // Convenience function which returns the default supported platform for + // tests. If exactly one supported platform is present, then this platform is + // the default platform. If exactly two platforms are present and one of them + // is the interpreter platform, then the other platform is the default + // platform. Otherwise returns an error. + static StatusOr GetDefaultPlatform(); + + // Convenience function which returns the sole supported platform. If + // exactly one supported platform is present, then this platform is the + // default platform. Otherwise returns an error. + static StatusOr GetSolePlatform(); + + // Returns the platform according to the given name. Returns error if there is + // no such platform. + static StatusOr GetPlatform(const string& platform_name); + + // Returns exactly one platform that does not have given name. Returns error + // if there is no such platform, or there are multiple such platforms. + static StatusOr GetPlatformExceptFor( + const string& platform_name); + + // Returns a vector of StreamExecutors for the given platform. The vector is + // indexed by device ordinal (device numbering used by StreamExecutor). If an + // element is nullptr, then the device is present by not supported by XLA. + // + // If the platform has no visible devices, a not-found error is returned. + static StatusOr> GetStreamExecutors( + se::Platform* platform); + + private: + TF_DISALLOW_COPY_AND_ASSIGN(PlatformUtil); +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_PLATFORM_UTIL_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/reduce_precision_insertion.h b/linux/include/tensorflow/compiler/xla/service/reduce_precision_insertion.h new file mode 100644 index 000000000..0b4e82e8d --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/reduce_precision_insertion.h @@ -0,0 +1,153 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_REDUCE_PRECISION_INSERTION_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_REDUCE_PRECISION_INSERTION_H_ + +#include "tensorflow/compiler/xla/service/buffer_liveness.h" +#include "tensorflow/compiler/xla/service/hlo_computation.h" +#include "tensorflow/compiler/xla/service/hlo_instruction.h" +#include "tensorflow/compiler/xla/service/hlo_module.h" +#include "tensorflow/compiler/xla/service/hlo_pass_interface.h" +#include "tensorflow/compiler/xla/service/hlo_pass_pipeline.h" + +namespace xla { + +// HLO pass which inserts reduce-precision instructions into the HLO graph, for +// purposes of experimenting with the effects of reduced-precision storage of +// intermediate values. +class ReducePrecisionInsertion : public HloModulePass { + using InstructionFilterFunction = std::function; + + public: + // The exponent_bits and mantissa_bits arguments specify the parameters of + // the instructions to insert. The instructions will be inserted after each + // instruction with an opcode for which the instruction_filter_function + // function returns true and the output type is F32. + explicit ReducePrecisionInsertion( + const int exponent_bits, const int mantissa_bits, + const HloReducePrecisionOptions::Location location, + const InstructionFilterFunction& instruction_filter_function) + : exponent_bits_(exponent_bits), + mantissa_bits_(mantissa_bits), + location_(location), + instruction_filter_function_(instruction_filter_function) {} + + // Version of the constructor that takes an HloReducePrecisionOptions proto + // rather than explicitly-enumerated parameters, for convenience when + // creating passes based on DebugOptions. + explicit ReducePrecisionInsertion( + const HloReducePrecisionOptions& reduce_precision_options) + : exponent_bits_(reduce_precision_options.exponent_bits()), + mantissa_bits_(reduce_precision_options.mantissa_bits()), + location_(reduce_precision_options.location()), + instruction_filter_function_( + make_filter_function(reduce_precision_options)) {} + + ~ReducePrecisionInsertion() override{}; + + absl::string_view name() const override { + return "reduce-precision-insertion"; + } + + // Run the pass on the given module. Returns whether the module was changed + // (reduce-precision instructions were inserted). + StatusOr Run(HloModule* module) override; + + // Convert between the (inconvenient) xla.proto HloReducePrecisionOptions + // representation and InstructionFilterFunction functions. + static InstructionFilterFunction make_filter_function( + const HloReducePrecisionOptions& reduce_precision_options); + static HloReducePrecisionOptions make_options_proto( + const HloReducePrecisionOptions::Location location, + const int exponent_bits, const int mantissa_bits, + const std::function& opcode_filter_function, + const std::vector& opname_substring_list = {}); + + // Enumeration to control which passes should be added. + enum class PassTiming { BEFORE_OPTIMIZATION, AFTER_FUSION }; + + // Add ReducePrecisionInsertion passes to an HloPassPipeline based on the list + // of HloReducePrecisionOptions in a DebugOptions proto. Returns true if any + // passes were added. + static bool AddPasses(HloPassPipeline* pipeline, + const DebugOptions& debug_options, + const PassTiming pass_timing); + + private: + // Select the instructions that should have reduce-precision operations + // attached to them. + std::vector instructions_to_modify( + const HloComputation* computation); + + // Insert a reduce-precision operation into the graph on the output of the + // given instruction. + StatusOr insert_after(HloInstruction* instruction); + + // Insert reduce-precision operations into the graph on the inputs of the + // given instructions. (For fusion instructions, the operations will be + // inserted inside the fusion computation, on the outputs of the relevant + // input parameters.) + StatusOr insert_on_inputs( + const std::vector& instructions); + + // Insert reduce-precision operations into the graph on the outputs of the + // given instructions. (For fusion instructions, the operations will be + // inserted inside the fusion computation as a new root.) + StatusOr insert_on_outputs( + const std::vector& instructions); + + // Is this shape valid for inserting a reduce-precision operation? + bool is_valid_shape(const Shape& shape) { + // For now, ReducePrecision is only implemented for F32 arrays, so this + // ignores instructions that produce other data. In particular, this + // currently ignores instructions producing tuples, even if those tuples + // contain F32 arrays inside them. The assumption is that in most cases + // equivalent behavior can be obtained by adding ReducePrecision + // instructions after the instructions that pull the F32 arrays out of + // the tuples. + // + // TODO(b/64093391): Remove the IsScalar check once this won't cause + // failures on the GPU backend if the ReducePrecision instruction ends up + // inserted between a scalar constant and the init_value argument of a + // Reduce operation. + return shape.element_type() == PrimitiveType::F32 && + !ShapeUtil::IsScalar(shape); + } + + // Is this instruction one such that following or preceding it with a new + // reduce-precision operation will be redundant? + bool is_redundant(const HloInstruction* instruction) { + return instruction->opcode() == HloOpcode::kReducePrecision && + instruction->exponent_bits() <= exponent_bits_ && + instruction->mantissa_bits() <= mantissa_bits_; + } + + // Parameters for the precision reduction to be added. + const int exponent_bits_; + const int mantissa_bits_; + + // Pass "timing" parameter. This also controls aspects of how the pass + // selects locations to insert instructions. + const HloReducePrecisionOptions::Location location_; + + // User-provided Function to determine whether a given instruction should + // have a reduce-precision instruction inserted in its output stream. + const InstructionFilterFunction instruction_filter_function_; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_REDUCE_PRECISION_INSERTION_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/reshape_mover.h b/linux/include/tensorflow/compiler/xla/service/reshape_mover.h new file mode 100644 index 000000000..a3db439e3 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/reshape_mover.h @@ -0,0 +1,36 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_RESHAPE_MOVER_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_RESHAPE_MOVER_H_ + +#include "tensorflow/compiler/xla/service/hlo_pass_interface.h" + +namespace xla { + +// A pass which moves Reshapes and Transposes to let later passes combine them. +// This now only moves them outputward across elementwise ops all whose operands +// are equivalent Reshapes or Transposes, but in future could potentially move +// them inputward also. +class ReshapeMover : public HloModulePass { + public: + absl::string_view name() const override { return "reshape-mover"; } + + StatusOr Run(HloModule* module) override; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_RESHAPE_MOVER_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/scatter_expander.h b/linux/include/tensorflow/compiler/xla/service/scatter_expander.h new file mode 100644 index 000000000..559a85dcc --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/scatter_expander.h @@ -0,0 +1,34 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_SCATTER_EXPANDER_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_SCATTER_EXPANDER_H_ + +#include "tensorflow/compiler/xla/service/hlo_pass_interface.h" + +namespace xla { + +class ScatterExpander : public HloModulePass { + public: + absl::string_view name() const override { return "scatter_expander"; } + StatusOr Run(HloModule* module) override; + + private: + StatusOr ExpandScatter(HloInstruction* scatter); +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_SCATTER_EXPANDER_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/service.h b/linux/include/tensorflow/compiler/xla/service/service.h new file mode 100644 index 000000000..1f62fad4c --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/service.h @@ -0,0 +1,301 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_SERVICE_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_SERVICE_H_ + +#include +#include +#include +#include + +#include "absl/types/span.h" +#include "tensorflow/compiler/xla/executable_run_options.h" +#include "tensorflow/compiler/xla/legacy_flags/debug_options_flags.h" +#include "tensorflow/compiler/xla/service/allocation_tracker.h" +#include "tensorflow/compiler/xla/service/backend.h" +#include "tensorflow/compiler/xla/service/channel_tracker.h" +#include "tensorflow/compiler/xla/service/device_memory_allocator.h" +#include "tensorflow/compiler/xla/service/executable.h" +#include "tensorflow/compiler/xla/service/execution_tracker.h" +#include "tensorflow/compiler/xla/service/hlo_execution_profile.h" +#include "tensorflow/compiler/xla/service/hlo_module.h" +#include "tensorflow/compiler/xla/service/hlo_module_config.h" +#include "tensorflow/compiler/xla/service_interface.h" +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/compiler/xla/xla.pb.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/platform/logging.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/stream_executor_no_cuda.h" + +namespace xla { + +// Options to configure the service when it is created. +class ServiceOptions { + public: + // Set the platform backing the service, or nullptr for the default platform. + ServiceOptions& set_platform(se::Platform* platform); + se::Platform* platform() const; + + // Set the number of replicas to use when compiling replicated + // programs. + ServiceOptions& set_number_of_replicas(int number_of_replicas); + int number_of_replicas() const; + + // Sets the thread pool size for parallel execution of an individual operator. + ServiceOptions& set_intra_op_parallelism_threads(int num_threads); + int intra_op_parallelism_threads() const; + + private: + se::Platform* platform_ = nullptr; + int number_of_replicas_ = 1; + int intra_op_parallelism_threads_ = -1; +}; + +// The XLA service object, which is the same across all platforms. It maintains +// the service state of computations and allocations, and delegates +// target-specific requests to the target-specific infrastructure +// (target-specific compiler, StreamExecutor). +class Service : public ServiceInterface { + public: + // Factory method for creating a new Service. + static StatusOr> NewService( + se::Platform* platform = nullptr); + static StatusOr> NewService( + const ServiceOptions& options); + + // Unregisters a previously-allocated global handle. + // + // If the handle given is not currently allocated, a NOT_FOUND status is + // returned. + Status Unregister(const UnregisterRequest* arg, + UnregisterResponse* result) override; + + // Deconstructs a tuple. Returns a newly created GlobalDataHandle for each + // element in the tuple. + Status DeconstructTuple(const DeconstructTupleRequest* arg, + DeconstructTupleResponse* result) override; + + // Executes a computation with the provided global data passed as + // immutable arguments. The request contains the whole computation graph. + // Returns global data output and execution timing. + Status ExecuteGraph(const ExecuteGraphRequest* arg, + ExecuteResponse* result) override; + + // Executes one or more computations in parallel with the provided global data + // passed as immutable arguments. Returns global data output for each + // computation. + Status ExecuteGraphParallel(const ExecuteGraphParallelRequest* arg, + ExecuteParallelResponse* result) override; + + // Requests one or more device handles from the target. + // + // When N device handles are requested and the number of replicas is R, at + // least N * R devices must be available. The devices are assigned based on + // the device ordinals such that the first R available devices are assigned to + // the first set of replicas, and the next R devices to the second set of + // replicas, etc. Each returned device handle represents the device with the + // replica id 0. + Status GetDeviceHandles(const GetDeviceHandlesRequest* arg, + GetDeviceHandlesResponse* result) override; + + // Waits until the specified execution is complete and returns the result. + // Calling this API multiple times with the same execution handle returns the + // method with an error since the execution handle is destroyed after the + // first call. + Status WaitForExecution(const WaitForExecutionRequest* arg, + WaitForExecutionResponse* result) override; + + // Requests that global data be transferred to the client in literal form. + Status TransferToClient(const TransferToClientRequest* arg, + TransferToClientResponse* result) override; + + // Transfers data from a literal provided by the client, into device memory. + Status TransferToServer(const TransferToServerRequest* arg, + TransferToServerResponse* result) override; + + // Transfers data from a literal provided by the client, into the Infeed + // buffer of the device. + Status TransferToInfeed(const TransferToInfeedRequest* arg, + TransferToInfeedResponse* result) override; + + // Transfers data from the Outfeed othe device to the literal provided by the + // client. + Status TransferFromOutfeed(const TransferFromOutfeedRequest* arg, + TransferFromOutfeedResponse* result) override; + + // Resets devices, clearing all existing state on all the devices associated + // with this service (including memory allocated on the devices). + // + // ResetDevice may only be called where no previous Execution state on the + // device is used by the next Execution. + // + // ResetDevice should be called before an Execution that expect the device to + // be in the reset state. For example, if the prior Execution modifies device + // state (e.g., architectural state) that the next Execution depends on. + Status ResetDevice(const ResetDeviceRequest* arg, + ResetDeviceResponse* result) override; + + Status ComputeConstantGraph(const ComputeConstantGraphRequest* arg, + ComputeConstantResponse* result) override; + + // Returns the shape (with layout) of an array associated with a given data + // handle. + Status GetShape(const GetShapeRequest* arg, + GetShapeResponse* result) override; + + // Retrieves the statistics of a computation. + Status GetComputationGraphStats(const ComputationGraphStatsRequest* arg, + ComputationStatsResponse* result) override; + + // Creates a unique channel handle that can be used for Send/Recv + // instructions. + Status CreateChannelHandle(const CreateChannelHandleRequest* arg, + CreateChannelHandleResponse* result) override; + + // Returns the backend used to execute computations. + const Backend& backend() const { return *execute_backend_; } + Backend* mutable_backend() { return execute_backend_.get(); } + + private: + // A private overload for Service itself, used by other methods within this + // class. + StatusOr> CreateModuleConfig( + const ProgramShape& program_shape, + absl::Span arguments, + const ExecutionOptions& execution_options); + + // Picks a parallel response and fills the result. + Status PickParallelResponse(const ExecuteParallelResponse& parallel_result, + ExecuteResponse* result); + + // Prepare the executors for executing parallel. + StatusOr> GetExecutors( + const ExecutionOptions& execution_options, int64 requests_size, + int64 request_index) const; + + // Prepare the arguments for executing parallel. + StatusOr>> GetArguments( + const ExecutionOptions& execution_options, + absl::Span arguments); + + protected: + friend class LocalExecutable; + + // The constructor is private. Use the NewService factory to create new + // service objects. + Service(const ServiceOptions& options, + std::unique_ptr execute_backend); + + // Resolves the given argument handles in the allocation tracker and returns + // the corresponding allocations for every replica. The function also verifies + // that each allocation matches the execution platform and device ordinal of + // the corresponding replica. + StatusOr>> + ResolveAndValidateArguments( + absl::Span arguments, + absl::Span stream_executors); + + // Create a Hlo module config for the given program shape and arguments. + // execution_options is optional; if not given a default is used. + StatusOr> CreateModuleConfig( + const ProgramShape& program_shape, + absl::Span argument_shapes, + const ExecutionOptions* execution_options); + + // Builds an Executable for the given parameters. + // + // If device_allocator is not null, the compiler may use it to allocate temp + // buffers, which the compiler is responsible for freeing. The allocator + // given here need not match the allocator used when running the executable. + StatusOr> BuildExecutable( + const HloModuleProto& module_proto, + std::unique_ptr module_config, Backend* backend, + se::StreamExecutor* executor, + DeviceMemoryAllocator* device_allocator = nullptr); + + // Same as BuildExecutable() above, but builds a list of Executables for the + // given computations that may interact with each other. + StatusOr>> BuildExecutables( + const std::vector& module_protos, + std::vector> module_configs, + Backend* backend, std::vector> executors, + DeviceMemoryAllocator* device_allocator); + + // Runs the given executable with the given arguments and register the result + // in the allocation tracker. The handle of the result from the tracker is + // returned. If the parameter "profile" is not null, it points to an + // ExecutionProfile object which will be filled in with profile data. + StatusOr ExecuteAndRegisterResult( + Executable* executable, + const absl::Span> arguments, + Backend* backend, const string& result_tag, ExecutionProfile* profile); + + // Runs the given executables with the given arguments and register the result + // from each executable in the allocation tracker. The handles of the result + // from the tracker are returned. + StatusOr> ExecuteParallelAndRegisterResult( + absl::Span executables, + absl::Span>> arguments, + Backend* backend, absl::Span device_handles, + absl::Span result_tags, ExecutionProfile* profile); + + // Executes a single computation which has more than one target device. + // The N devices are expected to all return an empty tuple, but one, which + // will be the result of this computation. + Status ExecuteOneToN(const ExecuteGraphRequest* arg, ExecuteResponse* result); + + // Convenience function which checks whether the given client_shape + // (presumably passed by the client to set the result layout) is valid for the + // given computation result shape. + Status ValidateResultShape(const Shape& client_shape, + const Shape& result_shape) const; + + // Returns the stream executors assigned to the replicas represented by the + // given device handle. Each device_handle is a virtual replicated device that + // represents a set of physical devices for the replicas. + StatusOr> Replicas( + const Backend& backend, const DeviceHandle& device_handle) const; + + // Dumps the (unoptimized) module given if the corresponding DebugOptions + // field has been set. + Status MaybeDumpUnoptimizedHloModule(const HloModule& module) const; + + // Returns the device handle that represents the replicated device for a + // single computation that is not model-parallelized. + DeviceHandle SingleComputationDeviceHandle() const; + + ServiceOptions options_; + + // Tracks channels created via the API. + ChannelTracker channel_tracker_; + + // Tracks allocations made via the API and computation execution. + AllocationTracker allocation_tracker_; + + // Tracks asynchronously launched executions via the API. + ExecutionTracker execution_tracker_; + + // Backend to compile and execute computations on. + std::unique_ptr execute_backend_; + + TF_DISALLOW_COPY_AND_ASSIGN(Service); +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_SERVICE_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/service_executable_run_options.h b/linux/include/tensorflow/compiler/xla/service/service_executable_run_options.h new file mode 100644 index 000000000..dbfed628b --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/service_executable_run_options.h @@ -0,0 +1,72 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_SERVICE_EXECUTABLE_RUN_OPTIONS_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_SERVICE_EXECUTABLE_RUN_OPTIONS_H_ + +#include "tensorflow/compiler/xla/executable_run_options.h" +#include "tensorflow/compiler/xla/service/stream_pool.h" +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/stream_executor/stream_executor.h" + +namespace xla { + +// Class containing options for running a LocalExecutable and other auxiliary +// data, now only a stream cache for GPU backend. +class ServiceExecutableRunOptions { + public: + using StreamBorrower = std::function(int)>; + + ServiceExecutableRunOptions() + : ServiceExecutableRunOptions(ExecutableRunOptions()) {} + + explicit ServiceExecutableRunOptions( + ExecutableRunOptions run_options, StreamBorrower borrow_stream = nullptr, + tensorflow::thread::ThreadPool* xla_intra_op_thread_pool = nullptr) + : run_options_(std::move(run_options)), + borrow_stream_(std::move(borrow_stream)), + xla_intra_op_thread_pool_(xla_intra_op_thread_pool) {} + + // Returns reference or pointer to `ExecutableRunOptions` member. + const ExecutableRunOptions& run_options() const { return run_options_; } + ExecutableRunOptions* mutable_run_options() { return &run_options_; } + + // Delegate to `ExecutableRunOptions` member. + se::Stream* stream() const { return run_options_.stream(); } + DeviceMemoryAllocator* allocator() const { return run_options_.allocator(); } + int device_ordinal() const { return run_options_.device_ordinal(); } + + // Borrows a stream and returns a smart pointer which returns the stream on + // destruction. + StatusOr BorrowStream(int device_ordinal) const { + return borrow_stream_ + ? borrow_stream_(device_ordinal) + : Status(tensorflow::error::UNIMPLEMENTED, "No stream cache"); + } + + // Returns reference to thread pool for execution of XLA ops on CPU backend. + tensorflow::thread::ThreadPool* xla_intra_op_thread_pool() const { + return xla_intra_op_thread_pool_; + } + + private: + ExecutableRunOptions run_options_; + StreamBorrower borrow_stream_; + tensorflow::thread::ThreadPool* xla_intra_op_thread_pool_; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_SERVICE_EXECUTABLE_RUN_OPTIONS_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/shape_inference.h b/linux/include/tensorflow/compiler/xla/service/shape_inference.h new file mode 100644 index 000000000..96a0ee165 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/shape_inference.h @@ -0,0 +1,331 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// Shape inference is used by the XLA service as the user builds up +// computation requests. + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_SHAPE_INFERENCE_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_SHAPE_INFERENCE_H_ + +#include + +#include "absl/types/span.h" +#include "tensorflow/compiler/xla/service/hlo_instruction.h" +#include "tensorflow/compiler/xla/service/hlo_opcode.h" +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/types.h" + +namespace xla { + +// For a given operation and input shapes, infers what the resulting shape is +// for the operation. With this functionality, the user does not need to specify +// the expected result type for computations that are built up via the API -- +// the shape that results from an operation is inferred. Some methods have +// overloads for inferring shape at the HLO level. +// +// TODO(b/73352135): Shape inference does not issue very good error messages, in +// part because HloInstruction::ToString() is not available since shape +// inference runs before the HloInstruction object is created. We need a +// solution for this. +class ShapeInference { + public: + // Infers the shape produced by applying the given unary operation to the + // given input shape. + static StatusOr InferUnaryOpShape(HloOpcode opcode, + const Shape& shape); + static StatusOr InferUnaryOpShape(HloOpcode opcode, + const HloInstruction* operand); + + // Infers the shape produced by applying the given binary operation to the + // given input shapes. + static StatusOr InferBinaryOpShape( + HloOpcode opcode, const Shape& lhs, const Shape& rhs, + absl::Span broadcast_dimensions); + static StatusOr InferBinaryOpShape(HloOpcode opcode, + const HloInstruction* lhs, + const HloInstruction* rhs); + + // Infers the shape produced by applying the given ternary operation to the + // given input shapes. + static StatusOr InferTernaryOpShape(HloOpcode opcode, const Shape& lhs, + const Shape& rhs, + const Shape& ehs); + static StatusOr InferTernaryOpShape(HloOpcode opcode, + const HloInstruction* lhs, + const HloInstruction* rhs, + const HloInstruction* ehs); + + // Infers the shape produced by applying the given variadic operation to the + // given input operand shapes. + static StatusOr InferVariadicOpShape( + HloOpcode opcode, absl::Span operand_shapes); + static StatusOr InferVariadicOpShape( + HloOpcode opcode, absl::Span operands); + + // Infers the shape produced by applying the given mapping computation shape + // to the given operand shapes. + static StatusOr InferMapShape( + absl::Span arg_shapes, const ProgramShape& to_apply, + absl::Span dimensions); + + // Infers the shape produced by InferBatchNormTraining with the given + // operands. + static StatusOr InferBatchNormTrainingShape(const Shape& operand_shape, + const Shape& scale_shape, + const Shape& offset_shape, + int64 feature_index); + + // Infers the shape produced by InferBatchNormInference with the given + // operands. + static StatusOr InferBatchNormInferenceShape( + const Shape& operand_shape, const Shape& scale_shape, + const Shape& offset_shape, const Shape& mean_shape, + const Shape& variance_shape, int64 feature_index); + + // Infers the shape produced by InferBatchNormGrad with the given operands. + static StatusOr InferBatchNormGradShape(const Shape& operand_shape, + const Shape& scale_shape, + const Shape& mean_shape, + const Shape& var_shape, + const Shape& output_grad_shape, + int64 feature_index); + + // Infers the shape produced by applying the given convolutional + // filter (rhs) to lhs in the way specified by the fields on window. + static StatusOr InferConvolveShape( + const Shape& lhs, const Shape& rhs, int64 feature_group_count, + const Window& window, + const ConvolutionDimensionNumbers& dimension_numbers); + + // Infers the shape produced by the given FFT type on the given operand. + static StatusOr InferFftShape(const Shape& in, FftType fft_type, + absl::Span fft_length); + + // Infers the shape produced by a cross replica sum with the given operand + // shapes. + static StatusOr InferCrossReplicaSumShape( + absl::Span operand_shapes); + + // Infers final shape of an Alltoall operation that is created by the xla + // builder. + static StatusOr InferAllToAllShape(const Shape& shape, + int64 split_dimension, + int64 concat_dimension, + int64 split_count); + + // Infers the shape of an HLO all-to-all instruction. + static StatusOr InferAllToAllTupleShape( + absl::Span operand_shapes); + + // Infers the shape of a collective permute operation. + static StatusOr InferCollectivePermuteShape(const Shape& shape); + + // Infers the shape produced by applying the given reduction computation + // shape to the given input operand shape. + // + // If pass_index is true, the reduce function is invoked with the element + // index as the leading parameter, and the program shape should match + // accordingly (or an error will result). + static StatusOr InferReduceShape( + absl::Span arg_shapes, + absl::Span dimensions_to_reduce, + const ProgramShape& to_apply); + + // Infers the shape produced by applying the given computation to the operand + // shape with the given window and stride dimensions. + static StatusOr InferReduceWindowShape( + const Shape& operand_shape, const Shape& init_value, const Window& window, + const ProgramShape& to_apply_shape); + + // Infers the shape produced by scattering the given source shape to the + // selected indices of each window on the operand shape. + static StatusOr InferSelectAndScatterShape( + const Shape& operand_shape, const ProgramShape& select_shape, + const Window& window, const Shape& source_shape, + const Shape& init_value_shape, const ProgramShape& scatter_shape); + + // Infers the shape produced by a reverse operation that reverses the order + // of the elements in the given dimensions. + static StatusOr InferReverseShape(const Shape& operand_shape, + absl::Span dimensions); + + // Infers the shape produced by a slice operation spanning from the starts to + // the limits in the original shape's dimensions. + // + // e.g. slice f32[32x32] 0:16 0:16 -> f32[16x16] + static StatusOr InferSliceShape(const Shape& arg, + absl::Span starts, + absl::Span limits, + absl::Span strides); + + // Infers the shape produced by a dynamic slice operation of size specified + // in 'slice_sizes', with dynamic start indices shape 'start_indices_shape'. + static StatusOr InferDynamicSliceShape( + const Shape& operand_shape, const Shape& start_indices_shape, + absl::Span slice_sizes); + + // Infers the shape produced by a dynamic update slice operation based + // on the shape of operand and update. + static StatusOr InferDynamicUpdateSliceShape( + const Shape& operand_shape, const Shape& update_shape, + const Shape& start_indices_shape); + + // Infers the shape produced by doing a compile-time-constant indexing into + // the given input shape. This is essential for operations on tuples, because + // it is impossible to infer the type that comes out of the tuple indexing if + // it is not a compile time constant. + static StatusOr InferGetTupleElementShape(const Shape& arg, + int64 index); + + // Infers the shape produced from a while node. condition and body are the + // shapes of computations for the condition and the body of a while node, and + // init is the shape of data initially passed in to the body as an argument. + // The shapes must match; condition: T -> PRED, body: T -> T, init: T + static StatusOr InferWhileShape(const ProgramShape& condition, + const ProgramShape& body, + const Shape& init); + + // Infers the shape produced by a conditional operation. + static StatusOr InferConditionalShape( + const Shape& predicate, const Shape& true_operand, + const Shape& false_operand, const ProgramShape& true_computation, + const ProgramShape& false_computation); + + // Infers the shape produced by a broadcast operation. + static StatusOr InferBroadcastShape( + const Shape& operand, absl::Span broadcast_sizes); + + // Infers the shape produced by a reshape operation from the element type of + // its operand and the new dimension sizes specified. + static StatusOr InferReshapeShape(const Shape& operand, + absl::Span dimensions, + absl::Span new_sizes); + + // Infers the shape produced by a transpose operation from the element type of + // its operand and its dimensions field. + static StatusOr InferTransposeShape( + const Shape& operand, absl::Span dimensions); + + // Helper that infers the shape produced by performing a concatenate operation + // with the given operand shapes. + static StatusOr InferConcatOpShape( + absl::Span arg_shapes, int64 dimension); + + // Infers the shape produced by a kAfterAll. Trivially this shape is always a + // TOKEN shape. However, ShapeInference serves two purposes: inferring shapes + // and checking operand shapes. This method verifies that the operand shapes + // are all TOKENs. + static StatusOr InferAfterAllShape( + absl::Span arg_shapes); + + // Helper that validates the given operand shape can be converted to the + // target output_shape via a convert instruction -- the requirement is that + // the shape is identical except for the element type. + static StatusOr InferConvertShape(const Shape& operand_shape, + PrimitiveType new_element_type); + + // Helper that validates the given operand shape can be bitcast converted to + // the target output_shape via a bitcast convert instruction -- the + // requirement is that the shape is identical except for the element type and + // the element types have identical bit-widths. + static StatusOr InferBitcastConvertShape( + const Shape& operand_shape, PrimitiveType new_element_type); + + // Helper that validates the input data type for a reduce-precision operation, + // and returns the result shape. + static StatusOr InferReducePrecisionShape(const Shape& operand_shape, + const int exponent_bits, + const int mantissa_bits); + + // Helper that infers the shape produced by a pad operation based on the + // padding configuration. + static StatusOr InferPadShape(const Shape& operand_shape, + const Shape& padding_value_shape, + const PaddingConfig& padding_config); + + // Helper that validates the given arg_shapes are compatible with the shape of + // the to_apply parameters, and returns the to_apply result shape. + static StatusOr InferCallShape( + absl::Span arg_shapes, const ProgramShape& to_apply); + + // Helper that infers the shape produced by performing a dot operation with + // the given LHS and RHS shapes. + static StatusOr InferDotOpShape( + const Shape& lhs, const Shape& rhs, + const DotDimensionNumbers& dimension_numbers); + + // Helper that infers the shape of the tensor produced by a gather operation + // with the given input shape, gather indices shape and gather dimension + // numbers. + static StatusOr InferGatherShape( + const Shape& input_shape, const Shape& start_indices_shape, + const GatherDimensionNumbers& gather_dim_numbers, + absl::Span slice_sizes); + + // Helper that validates the given input shape, scatter indices shape, updates + // shape, and scatter dimension numbers that constitute a scatter operation, + // and returns the result shape of the scatter operation. + static StatusOr InferScatterShape( + const Shape& operand_shape, const Shape& scatter_indices_shape, + const Shape& updates_shape, const ProgramShape& to_apply_shape, + const ScatterDimensionNumbers& scatter_dim_numbers); + + private: + // Helper that infers the shape produced by performing an element-wise binary + // operation with the given LHS and RHS shapes. + // Note: By "element-wise" we mean operations that look at a single element in + // the LHS and a single element in the RHS to produce a single output element, + // even in the presence of broadcasting of one of the operands over the other. + static StatusOr InferElementwiseBinaryOpShape( + HloOpcode operation, const Shape& lhs, const Shape& rhs, + absl::Span broadcast_dimensions); + + // Helper for inferring the shape of Clamp ops. + static StatusOr InferClampShape(const Shape& min, const Shape& operand, + const Shape& max); + + // Helper for inferring the shape of Select ops. + static StatusOr InferSelectShape(const Shape& pred, + const Shape& on_true, + const Shape& on_false); + // Helper for inferring the shape of TupleSelect ops. + static StatusOr InferTupleSelectShape(const Shape& pred, + const Shape& on_true, + const Shape& on_false); + + // Helper for inferring shapes of binary operations which use degenerate + // dimension broadcasting (a dimension of size 1 in one operand is broadcast + // up to match the size of the dimension in the other operand). + static StatusOr InferDegenerateDimensionBroadcastShape( + HloOpcode operation, const Shape& lhs, const Shape& rhs); + + // Helper for inferring shapes of binary operations using "InDim" + // broadcasting. This is the broadcasting used in the *InDim binary operations + // (for example ComputationBuilder::AddInDim). smaller_shape must be a + // lower-rank shape than larger_shape. Returns the shape that the + // smaller_shape is broadcast to. + static StatusOr InferInDimBroadcastShape( + const Shape& smaller_shape, const Shape& larger_shape, + absl::Span broadcast_dimensions); + + TF_DISALLOW_COPY_AND_ASSIGN(ShapeInference); +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_SHAPE_INFERENCE_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/shaped_buffer.h b/linux/include/tensorflow/compiler/xla/service/shaped_buffer.h new file mode 100644 index 000000000..e1d26da4a --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/shaped_buffer.h @@ -0,0 +1,187 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_SHAPED_BUFFER_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_SHAPED_BUFFER_H_ + +#include +#include +#include + +#include "absl/types/span.h" +#include "tensorflow/compiler/xla/service/device_memory_allocator.h" +#include "tensorflow/compiler/xla/shape_tree.h" +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/platform/stream_executor_no_cuda.h" +#include "tensorflow/core/platform/types.h" + +namespace xla { + +class ScopedShapedBuffer; + +// Class which encapsulates a buffer or set of buffers containing data of a +// particular XLA shape. +class ShapedBuffer { + public: + // Construct a ShapedBuffer with null DeviceMemoryBases at each index. The + // shape of the data on the host and the device may differ because the device + // may have a different representation for different data types. Therefore, + // both the on-host and on-device shape are required. The on-device shape + // determines the number of device allocations (DeviceMemoryBase) held by the + // ShapedBuffer. + ShapedBuffer(const Shape& on_host_shape, const Shape& on_device_shape, + const se::Platform* platform, int device_ordinal); + + // Movable, but not copyable. + ShapedBuffer(ShapedBuffer&& s); + ShapedBuffer& operator=(ShapedBuffer&&); + ShapedBuffer(const ShapedBuffer&) = delete; + ShapedBuffer& operator=(const ShapedBuffer&) = delete; + + // Prevent (some forms of) accidental object slicing. + ShapedBuffer(const ScopedShapedBuffer&) = delete; + ShapedBuffer& operator=(const ScopedShapedBuffer&) = delete; + + virtual ~ShapedBuffer(); + + // Returns the shape of the on-host representation of the data held by this + // ShapedBuffer. + const Shape& on_host_shape() const { return on_host_shape_; } + + // Returns the shape of the on-device representation of the data held by this + // ShapedBuffer. + const Shape& on_device_shape() const { return on_device_shape_; } + + const se::Platform* platform() const { return platform_; } + int device_ordinal() const { return device_ordinal_; } + + // Return the root buffer of the shape (shape index {}). + const se::DeviceMemoryBase& root_buffer() const { + return buffer(/*index=*/{}); + } + + // Returns the buffer at the given shape index where index is defined as in + // ShapeUtil::GetSubshape. + const se::DeviceMemoryBase& buffer(const ShapeIndex& index) const { + return buffers_.element(index); + } + + // Sets the device memory buffer at the given index. + void set_buffer(const se::DeviceMemoryBase& buffer, const ShapeIndex& index) { + *buffers_.mutable_element(index) = buffer; + } + + // Sets all buffers. + // + // Precondition: buffers.shape == on_device_shape_ + void set_buffers(ShapeTree buffers) { + CHECK(ShapeUtil::Equal(buffers.shape(), on_device_shape_)); + buffers_ = std::move(buffers); + } + + // Returns the underlying ShapeTree containing all the device addresses in the + // ShapedBuffer. + const ShapeTree& buffers() const { return buffers_; } + ShapeTree& buffers() { return buffers_; } + + // Set all device memory pointers in the object to null. + void clear(); + + string ToString() const; + + protected: + // The shape of the data when represented on the host. + Shape on_host_shape_; + + // The shape of the data on the device. + Shape on_device_shape_; + + // The platform the memory is allocated on. + const se::Platform* platform_; + + // The device the memory is allocated on. + int device_ordinal_; + + // The tree of device buffers. Its shape is on_device_shape(). + ShapeTree buffers_; +}; + +std::ostream& operator<<(std::ostream& out, const ShapedBuffer& buffer); + +// ShapedBuffer derived class which allocates all internal buffers on +// construction and deallocates the memory when the object is +// destructed. +// +// TODO(timshen): Remove inheritance between ScopedShapedBuffer and +// ShapedBuffer. There should never be a need to consider a ScopedShapedBuffer +// as a ShapedBuffer, because in that case we should just be able to pass around +// our ShapeTree. Inheritance only adds complexity. See +// discussion in cl/192849370. +class ScopedShapedBuffer : public ShapedBuffer { + public: + // Creates a ScopedShapedBuffer with null DeviceMemoryBases at each index. + explicit ScopedShapedBuffer(const Shape& on_host_shape, + const Shape& on_device_shape, + DeviceMemoryAllocator* allocator, + int device_ordinal); + + // Create a ScopedShapedBuffer by taking over the memory from the incoming + // ShapedBuffer. + explicit ScopedShapedBuffer(ShapedBuffer shaped_buffer, + DeviceMemoryAllocator* allocator); + + // Movable, but not copyable. + ScopedShapedBuffer(ScopedShapedBuffer&& s); + ScopedShapedBuffer& operator=(ScopedShapedBuffer&&); + ScopedShapedBuffer(const ScopedShapedBuffer&) = delete; + ScopedShapedBuffer& operator=(const ScopedShapedBuffer&) = delete; + + // All buffers in the shape are deallocated on destruction. + ~ScopedShapedBuffer() override; + + // Return the allocator used to allocate the device memory held in this + // ScopedShapedBuffer. + DeviceMemoryAllocator* memory_allocator() const { return allocator_; } + + // Sets the device memory buffer at the given index. + // + // If the given buffer's device memory is non-null, its device_ordinal and + // allocator must match those in `this`. + void set_buffer(OwningDeviceMemory buffer, const ShapeIndex& index) { + if (!buffer.is_null()) { + CHECK_EQ(buffer.device_ordinal(), device_ordinal()); + CHECK_EQ(buffer.allocator(), allocator_); + *buffers_.mutable_element(index) = buffer.Forget(); + } else { + *buffers_.mutable_element(index) = se::DeviceMemoryBase(); + } + } + + // Like unique_ptr::release(), creates and returns a regular ShapedBuffer from + // this ScopedShapedBuffer, without freeing any of the associated memory. + // + // It's the caller's job to ensure that the memory contained therein is freed. + TF_MUST_USE_RESULT ShapedBuffer release(); + + protected: + void Deallocate(); + + DeviceMemoryAllocator* allocator_; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_SHAPED_BUFFER_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/source_map_util.h b/linux/include/tensorflow/compiler/xla/service/source_map_util.h new file mode 100644 index 000000000..c5a7e17cb --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/source_map_util.h @@ -0,0 +1,64 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_SOURCE_MAP_UTIL_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_SOURCE_MAP_UTIL_H_ + +#include "absl/strings/str_format.h" +#include "tensorflow/compiler/xla/service/executable.h" +#include "tensorflow/compiler/xla/status.h" +#include "tensorflow/core/platform/macros.h" + +namespace xla { +namespace source_map_util { + +// Creates an INVALID_ARGUMENT status with the given format string. +template +Status InvalidParameterArgument(const OpMetadata& op_metadata, + const absl::FormatSpec& format, + const Args&... args) { + string message = absl::StrFormat(format, args...); + if (!op_metadata.source_file().empty()) { + absl::StrAppendFormat(&message, " (%s:%d)", op_metadata.source_file(), + op_metadata.source_line()); + } + return InvalidArgument("%s", message); +} + +// Creates an INVALID_ARGUMENT status with the given format string. +// +// Also, attempts to extract the OpMetadata for parameter_number on executable +// and append it to the status message for source mapping to user code. +// +// executable may be nullptr, but parameter_number should not be out of bounds +// or a CHECK-failure may occur. +template +Status InvalidParameterArgument(Executable* executable, int parameter_number, + const absl::FormatSpec& format, + const Args&... args) { + if (executable != nullptr && executable->has_module()) { + const HloModule& module = executable->module(); + const HloComputation& computation = *module.entry_computation(); + HloInstruction* param = computation.parameter_instruction(parameter_number); + const OpMetadata& metadata = param->metadata(); + return InvalidParameterArgument(metadata, format, args...); + } + return InvalidArgument(format, args...); +} + +} // namespace source_map_util +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_SOURCE_MAP_UTIL_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/stream_pool.h b/linux/include/tensorflow/compiler/xla/service/stream_pool.h new file mode 100644 index 000000000..7221d323a --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/stream_pool.h @@ -0,0 +1,64 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_STREAM_POOL_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_STREAM_POOL_H_ + +#include +#include + +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/core/platform/mutex.h" +#include "tensorflow/core/platform/stream_executor_no_cuda.h" + +namespace xla { + +// Pool of stream_executor::Streams, which are created as needed and +// destroyed when the pool is destroyed. +class StreamPool { + public: + struct PtrDeleter { + void operator()(se::Stream* stream) { pool->ReturnStream(stream); } + StreamPool* pool; + }; + + // Stream pointer type returned by BorrowStream, which returns the + // stream to the pool on destruction. + using Ptr = std::unique_ptr; + + StreamPool() {} + + // Returns a pointer to a stream in the pool, creating a new stream + // if none are available in the pool. The returned smart pointer + // returns the stream to the pool on destruction. + // + // This method is thread-safe. + Ptr BorrowStream(se::StreamExecutor* executor); + + private: + // Puts a pointer to a stream back into the pool, leaving it free + // for future use. Streams that have previously encountered errors + // are deleted, and not returned to the pool. + // + // This method is thread-safe. + void ReturnStream(se::Stream* stream); + + tensorflow::mutex mu_; + std::vector> streams_ GUARDED_BY(mu_); +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_STREAM_POOL_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/transfer_manager.h b/linux/include/tensorflow/compiler/xla/service/transfer_manager.h new file mode 100644 index 000000000..f952e64af --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/transfer_manager.h @@ -0,0 +1,235 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_TRANSFER_MANAGER_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_TRANSFER_MANAGER_H_ + +#include +#include +#include + +#include "absl/types/span.h" +#include "tensorflow/compiler/xla/literal.h" +#include "tensorflow/compiler/xla/service/shaped_buffer.h" +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/platform/mutex.h" +#include "tensorflow/core/platform/stream_executor_no_cuda.h" +#include "tensorflow/core/platform/thread_annotations.h" +#include "tensorflow/core/platform/types.h" + +namespace xla { + +// The TransferManager interface lets backends provide platform-specific +// mechanisms for constructing literals from given device memory handles. +// This lets each platform customize how literals are transferred to/from the +// device in terms of padding, leading dimension, etc. +class TransferManager { + public: + virtual ~TransferManager() {} + + // Returns the ID of the platform that this transfer manager acts on. + virtual se::Platform::Id PlatformId() const = 0; + + // Returns the shape of the on-device representation for the given shape on + // the host. This is intended for use with ShapedBuffer where buffers are + // pre-allocated by the host, e.g. TransferLiteralToDevice, without the user + // needing to consider device-specific behaviors. + virtual Shape HostShapeToDeviceShape(const Shape& host_shape) const { + return host_shape; + } + + // Returns a literal containing the data held in the given ShapedBuffer + // using the provided executor. This operation is performed synchronously + // without waiting for any other operation on a stream to complete. + // + // This function should be avoided in favor of the asynchronous version below. + virtual StatusOr TransferLiteralFromDevice( + se::Stream* stream, const ShapedBuffer& device_buffer); + virtual Status TransferLiteralFromDevice( + se::Stream* stream, const ShapedBuffer& device_buffer, + const MutableBorrowingLiteral& literal); + + // Begins transferring a literal containing the data held in the given + // ShapedBuffer using the provided executor. + // + // This operation is performed asynchronously on the given stream. It returns + // once the transfer is enqueued. 'done' is invoked with the result when + // complete. + // + // device_buffer is copied by reference and must live at least until done() is + // invoked. + virtual void TransferLiteralFromDevice(se::Stream* stream, + const ShapedBuffer& device_buffer, + MutableBorrowingLiteral literal, + std::function done) = 0; + + // Transfers the given literal into the previously allocated device memory + // represented by the given ShapedBuffer using the given executor. The shape + // of the ShapedBuffer and DeviceShape(literal.shape()) must be compatible, + // but need not have the same layout. + // + // This operation is performed synchronously without waiting for any other + // operation on a stream to complete. This function should be avoided in favor + // of the asynchronous version below. + virtual Status TransferLiteralToDevice(se::Stream* stream, + const LiteralSlice& literal, + const ShapedBuffer& device_buffer); + + // Transfers the given literal into the previously allocated device memory + // represented by the given ShapedBuffer using the given executor. The shape + // of the ShapedBuffer and DeviceShape(literal.shape()) must be compatible, + // but need not have the same layout. + // + // This operation is performed asynchronously on the given stream. It returns + // once the transfer is enqueued. + virtual Status TransferLiteralToDeviceAsync( + se::Stream* stream, const LiteralSlice& literal, + const ShapedBuffer& device_buffer) = 0; + + // Convenience methods for transferring an array to or from the device at a + // known address. This avoids having to construct a ShapedBuffer just to + // transfer an array at a known address. + Status TransferArrayToDevice(se::Stream* stream, const LiteralSlice& literal, + const se::DeviceMemoryBase& dest); + void TransferArrayFromDevice(se::Stream* stream, const Shape& shape, + const se::DeviceMemoryBase& source, + const MutableBorrowingLiteral& literal, + std::function done); + + Status TransferArrayToDeviceAsync(se::Stream* stream, + const LiteralSlice& literal, + const se::DeviceMemoryBase& dest); + StatusOr TransferArrayFromDevice(se::Stream* stream, + const Shape& shape, + const se::DeviceMemoryBase& source); + + // Transfers the given literal into the Infeed interface of the device, + // using the given executor. + virtual Status TransferLiteralToInfeed(se::StreamExecutor* executor, + const LiteralSlice& literal) = 0; + + // Transfers the given literal from the Outfeed interface of the device, + // using the given executor. + virtual Status TransferLiteralFromOutfeed( + se::StreamExecutor* executor, const Shape& literal_shape, + MutableBorrowingLiteral literal) = 0; + + // Resets the devices associated with this transfer manager. + virtual Status ResetDevices( + absl::Span executor) = 0; + + // Given an allocated ShapedBuffer, constructs the tuple index table(s) in + // each buffer of the given ShapedBuffer corresponding to tuple shapes. If the + // ShapedBuffer is array-shaped this method does nothing. + Status WriteTupleIndexTables(se::Stream* stream, + const ShapedBuffer& device_buffer); + Status WriteTupleIndexTablesAsync(se::Stream* stream, + const ShapedBuffer& device_buffer); + + // Determines the byte size requirement for the given shape on the underlying + // architecture. This will be used to allocate an appropriately sized memory + // region for a host-to-device transfer. + virtual int64 GetByteSizeRequirement(const Shape& shape) const = 0; + + // Allocates a ScopedShapedBuffer which can hold data with the given on-host + // shape. The on-device shape may be different as indicated by + // HostShapeToDeviceShape. + StatusOr AllocateScopedShapedBuffer( + const Shape& on_host_shape, DeviceMemoryAllocator* allocator, + int device_ordinal); + + // The given ShapedBuffer holds a handle to allocated memory, but it is not + // in the general case legal to immediately copy or access that allocated + // memory because queued operations on the device may alias that memory. + // Memory ordering is enforced by the Stream's happens-before relationship + // which allows eager deallocation and reallocation of buffers host-side even + // if the device hasn't finished with them. + // + // In certain cases, it can be known that a ShapedBuffer does not have any + // conflicting accesses on the device and thus is eligible to be accessed at + // any time from the host. + // + // This function returns true if device_buffer can be accessed immediately + // without waiting for the Stream's previously enqueued items. This only + // returns true if all subbuffers in device_buffer can be accessed + // immediately. + virtual bool CanShapedBufferBeAccessedNow( + se::StreamExecutor* executor, const ShapedBuffer& device_buffer) const { + return false; + } + + ///// + // The TransferManager class also serves as a point to register objects for + // the various platforms. + + // Registers the TransferManager singleton for the platform kind. This is + // assumed to be a singleton, so no ownership is transferred. + // + // Precondition: a platform kind must not be registered more than once. + typedef std::unique_ptr (*TransferManagerCreationFunction)(); + static void RegisterTransferManager( + se::Platform::Id platform_id, + TransferManagerCreationFunction transfer_manager); + + // Returns the transfer manager singleton pointer if it is available for the + // given platform, or an error status if it is not. + static StatusOr GetForPlatform( + const se::Platform* platform); + + protected: + // Transfer a memory block of the given size from the device source into the + // 'destination' buffer. + // + // size is the size to transfer to destination in bytes. + virtual Status TransferBufferFromDevice(se::Stream* stream, + const se::DeviceMemoryBase& source, + int64 size, void* destination); + + // Transfer a memory block of the given size from 'source' buffer to the given + // destination of the device. + // + // size is the size to transfer from source in bytes. + virtual Status TransferBufferToDevice(se::Stream* stream, int64 size, + const void* source, + se::DeviceMemoryBase* destination); + + // Writes the given device-memory pointers in 'elements' to the given region + // to construct a tuple index table in the platform-specific tuple + // representation. + virtual Status WriteSingleTupleIndexTable( + se::Stream* stream, absl::Span elements, + const Shape& shape, se::DeviceMemoryBase* region) = 0; + + private: + // The mutex that guards the platform-to-transfer manager map. + static tensorflow::mutex platform_transfer_manager_mutex_; + + // State kept for each kind of TransferManager. Registration functions + // set up creation_function, and then we use that to lazily create + // "manager" the first time GetForPlatform is invoked for a particular id. + struct State { + std::unique_ptr manager; + TransferManagerCreationFunction creation_function = nullptr; + }; + + // Map from platform kind to transfer manager singleton. + static std::map* GetPlatformTransferManagers(); +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_TRANSFER_MANAGER_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/transpose_folding.h b/linux/include/tensorflow/compiler/xla/service/transpose_folding.h new file mode 100644 index 000000000..f95f982eb --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/transpose_folding.h @@ -0,0 +1,63 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_TRANSPOSE_FOLDING_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_TRANSPOSE_FOLDING_H_ + +#include "tensorflow/compiler/xla/service/hlo_module.h" +#include "tensorflow/compiler/xla/service/hlo_pass_interface.h" + +namespace xla { + +// HLO pass that folds transpose operators into Dot operators, where the Dot +// operator is implemented by a GEMM kernel that can transpose its inputs. +class TransposeFolding : public HloModulePass { + public: + using OperandIndices = std::vector; + + // Returns the set of foldable operands for a given HLO and some candidate + // operands. + using FoldableOperands = std::function; + using TransposableGemmOperandsFn = FoldableOperands; + using TransposableConvOperandsFn = FoldableOperands; + + // Helper function to explicitly not fold transposes. + static OperandIndices NeverFoldTranspose(const HloInstruction&, + const OperandIndices&) { + return {}; + } + // transposable_gemm_operands returns the set of operands it wants to fold if + // the instruction argument is implemented as a GEMM kernel that supports + // transposing its arguments. + // + // transposable_conv_operands returns the set of operands it wants to fold if + // the instruction argument is implemented as a convolution that supports + // transposing its arguments. + explicit TransposeFolding( + TransposableGemmOperandsFn transposable_gemm_operands, + TransposableConvOperandsFn transposable_conv_operands); + absl::string_view name() const override { return "transpose-folding"; } + + StatusOr Run(HloModule* module) override; + + private: + TransposableGemmOperandsFn transposable_gemm_operands_; + TransposableConvOperandsFn transposable_conv_operands_; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_TRANSPOSE_FOLDING_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/tuple_points_to_analysis.h b/linux/include/tensorflow/compiler/xla/service/tuple_points_to_analysis.h new file mode 100644 index 000000000..64ad1dc80 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/tuple_points_to_analysis.h @@ -0,0 +1,354 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_TUPLE_POINTS_TO_ANALYSIS_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_TUPLE_POINTS_TO_ANALYSIS_H_ + +#include +#include +#include +#include +#include +#include + +#include "absl/container/inlined_vector.h" +#include "absl/types/span.h" +#include "tensorflow/compiler/xla/service/dfs_hlo_visitor_with_default.h" +#include "tensorflow/compiler/xla/service/hlo_instruction.h" +#include "tensorflow/compiler/xla/service/hlo_module.h" +#include "tensorflow/compiler/xla/service/logical_buffer.h" +#include "tensorflow/compiler/xla/service/logical_buffer_analysis.h" +#include "tensorflow/compiler/xla/shape_tree.h" +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/lib/gtl/compactptrset.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/types.h" + +namespace xla { + +// A class describing the source(s) of the Buffer(s) contained in the output of +// a particular HLO instruction. The structure of PointsToSet mirrors the +// structure of the instruction's shape, which may be an arbitrary tree (eg, a +// nested tuple). Each node in this tree corresponds to a single buffer in the +// instruction's output and contains the set of Buffers which might define +// the corresponding buffer. +class PointsToSet { + public: + // Construct our ShapeTree with a pointer rather than a reference to a Shape + // because this is very hot code, and copying (and then destroying) all these + // Shapes is slow. + explicit PointsToSet(const Shape* shape) : tree_(shape) {} + + // Returns true if any points-to sets for any subshape element is not a + // singleton. + bool IsAmbiguous() const; + + // Returns true if no LogicalBuffer appears in more than one points-to set of + // the shape nodes. + bool IsDistinct() const; + + // Returns the total number of different LogicalBuffers contained in this + // object. This is equal to CreateFlattenedSet().size(). + size_t size() const; + + // Creates a set containing the union of all LogicalBuffers contained in the + // PointsToSet. + using BufferSet = tensorflow::gtl::CompactPointerSet; + BufferSet CreateFlattenedSet() const; + + // Returns true if the given buffer is in the points-to set at the given + // index. + bool ContainsBufferAtIndex(const LogicalBuffer& buffer, + const ShapeIndex& index) const; + + // Returns true if the given buffer is in the points-to set at any index. + bool ContainsBuffer(const LogicalBuffer& buffer) const; + + // Adds the given buffer to the points-to set at the given index. This is a + // nop if the buffer already is in the set at that index. + void AddPointedToBuffer(const LogicalBuffer& buffer, const ShapeIndex& index); + + // For the subshape at the given index (where index is defined as in + // ShapeUtil::GetSubshape) this method returns the set of HLO instructions + // which may produce the tuple subshape at that index. For example, given: + // + // %tuple1 = tuple(...) + // %tuple2 = tuple(...) + // %select = select(%tuple1, %tuple2) + // %nested_tuple = tuple(%select, %tuple1) + // + // These are the values for tuple_sources() for the PointsToSet of + // %nested_tuple: + // + // tuple_sources({}) = {%nested_tuple} + // tuple_sources({0}) = {%tuple1, %tuple2} + // tuple_sources({1}) = {%tuple1} + // + // tuple_sources() at the index of an array shape (not a tuple) returns the + // empty set. The instructions in the set returned by tuple_sources + // necessarily are either Tuple instructions, constants, or parameters. + using SourceSet = tensorflow::gtl::CompactPointerSet; + const SourceSet& tuple_sources(const ShapeIndex& index) const; + + // Add a tuple source instruction for the given index. + void add_tuple_source(const ShapeIndex& index, HloInstruction* tuple); + + using BufferList = absl::InlinedVector; + + // Return the list of logical buffers for the subshape at index. + const BufferList& element(const ShapeIndex& index) const { + return tree_.element(index).buffers; + } + BufferList* mutable_element(const ShapeIndex& index) { + return &tree_.mutable_element(index)->buffers; + } + + // Call fn(index, buflist) for every subshape index. + template + void ForEachElement(const Fn& fn) const { + tree_.ForEachElement([&fn](const ShapeIndex& index, const Elem& elem) { + fn(index, elem.buffers); + }); + } + template + void ForEachMutableElement(const Fn& fn) { + tree_.ForEachMutableElement([&fn](const ShapeIndex& index, Elem* elem) { + fn(index, &elem->buffers); + }); + } + template + Status ForEachElementWithStatus(const Fn& fn) const { + return tree_.ForEachElementWithStatus( + [&fn](const ShapeIndex& index, const Elem& elem) { + return fn(index, elem.buffers); + }); + } + + private: + struct Elem { + BufferList buffers; + SourceSet tuple_sources; + }; + ShapeTree tree_; + + // PointsToSet contains references (const LogicalBuffer*) to elements within + // TuplePointsToAnalysis, so disable copying. + TF_DISALLOW_COPY_AND_ASSIGN(PointsToSet); +}; + +// This class describes a particular subshape in a computation (instruction and +// shape index) and the logical buffer which may be a source of the subshape +// value. +class BufferAlias { + public: + BufferAlias(HloInstruction* instruction, const ShapeIndex& index) + : instruction_(instruction), index_(index) {} + + // Return the instruction/index of the subshape. + HloInstruction* instruction() const { return instruction_; } + const ShapeIndex& index() const { return index_; } + + bool operator==(const BufferAlias& other) const { + return instruction_ == other.instruction_ && index_ == other.index_; + } + bool operator!=(const BufferAlias& other) const { return !(*this == other); } + + string ToString() const; + + private: + HloInstruction* instruction_; + ShapeIndex index_; +}; + +std::ostream& operator<<(std::ostream& out, const BufferAlias& buffer_alias); + +// DFS visitor that performs tuple points-to analysis. This analysis determines +// the potential sources of each buffer in each instruction's output. +class TuplePointsToAnalysis : public DfsHloVisitorWithDefault { + public: + // Runs points-to analysis on 'module'. + static StatusOr> Run( + const HloModule* module); + + // Return the points-to set of an instruction. This describes the potential + // sources of each buffer in the instruction's output. + const PointsToSet& GetPointsToSet( + const HloInstruction* hlo_instruction) const; + + // Returns the logical buffer with the given ID. + const LogicalBuffer& GetBuffer(LogicalBuffer::Id id) const; + + // Returns the buffer defined at the given instruction and index. An error is + // returned if no buffer is defined at that point. + StatusOr GetBufferDefinedAt( + const HloInstruction* instruction, const ShapeIndex& index) const; + + // Return a (possibly empty) vector containing all BufferAliases of the given + // logical buffer The buffer alias set is the inverse of the points-to set. + // That is, LogicalBuffer B is in the points-to set of instruction I at index + // N iff instruction I, index N is a BufferAlias of B. + using BufferAliasVector = absl::InlinedVector; + const BufferAliasVector& GetBufferAliases(const LogicalBuffer& buffer) const; + + // Returns the number of logical buffers in the module + LogicalBuffer::Id num_logical_buffers() const { + return logical_buffer_analysis_->num_logical_buffers(); + } + + // Return a the logical buffer with id "id" in the module. Iteration + // over all logical buffers is usually done with something like: + // + // for (LogicalBuffer:Id id = 0; id < points_to.num_logical_buffers(); id++){ + // const auto& buffer = points_to.logical_buffer(id); + // ... do something with buffer ... + // } + LogicalBuffer& logical_buffer(LogicalBuffer::Id id) const { + return logical_buffer_analysis_->GetBuffer(id); + } + + // Returns a vector of buffers that the instruction produces. Most + // instructions produce a single buffer (the top-level buffer), some produce + // no buffers (eg bitcast), and some produce more than one buffer (eg, + // tuple-shaped parameters). + using BufferDefinitionVector = absl::InlinedVector; + const BufferDefinitionVector& GetBuffersDefinedByInstruction( + const HloInstruction* instruction) const; + + // Returns true if the given instruction defines a buffer at the given index. + bool InstructionDefinesBufferAtIndex(const HloInstruction* instruction, + const ShapeIndex& index) const; + + // Returns an OK status if the given buffer is defined by instruction + // 'buffer.instruction()' at index 'buffer.index()' and if the given buffer + // matches the TuplePointsToAnalysis' LogicalBuffer with 'buffer.id'. Returns + // an FailedPrecondition error status otherwise. An example of a LogicalBuffer + // which is not defined is a tuple element in a Tuple instruction. In this + // case, the Tuple instruction does not define the LogicalBuffer, rather that + // index aliases one of its operands. + Status VerifyBuffer(const LogicalBuffer& buffer) const; + + Status DefaultAction(HloInstruction* hlo_instruction) override; + Status HandleTuple(HloInstruction* tuple) override; + Status HandleGetTupleElement(HloInstruction* get_tuple_element) override; + Status HandleBitcast(HloInstruction* bitcast) override; + Status HandleDomain(HloInstruction* domain) override; + Status HandleSlice(HloInstruction* slice) override; + Status HandleCopy(HloInstruction* copy) override; + Status HandleRecvDone(HloInstruction* recv_done) override; + Status HandleSend(HloInstruction* send) override; + Status HandleTupleSelect(HloInstruction* tuple_select) override; + + string ToString() const; + + // Returns true if 'user' cannot possibly use the buffer at 'index' in + // 'operand'. Returns false otherwise. + // + // REQUIRES: 'operand' is an operand of 'user'. + bool DoesNotUseOperandBuffer(const HloInstruction* operand, + const ShapeIndex& index, + const HloInstruction* user) const; + + // Returns true if 'user' (at 'user_index') can share a buffer with its + // operand 'operand' (at 'operand_index'). Returns false otherwise. + // + // REQUIRES: 'operand' is an operand of 'user'. + bool CanShareOperandBufferWithUser(HloInstruction* operand, + const ShapeIndex& operand_index, + HloInstruction* user, + const ShapeIndex& user_index) const; + + private: + explicit TuplePointsToAnalysis( + const HloModule* module, + std::unique_ptr logical_buffer_analysis) + : module_(module), + logical_buffer_analysis_(std::move(logical_buffer_analysis)) {} + + // Perform the analysis. Should be called immediately after constructing the + // object and before calling GetPointsToSet. + Status Analyze(); + + // Populates instruction-defined buffers and aliases for each instruction + // in 'instructions'. + Status PopulateDefinedBuffersAndAliases(const decltype( + std::declval().instructions())& instructions); + + // Creates an empty PointsToSet in the points_to_ map for the given + // instruction. + PointsToSet& CreateEmptyPointsToSet(const HloInstruction* instruction); + + // Creates a PointsToSet in the points_to_ map for 'instruction' which is a + // copy of the existing PointsToSet for 'src'. + PointsToSet& CreateCopiedPointsToSet(const HloInstruction* instruction, + const HloInstruction* src); + + // Adds the buffers defined by the given instruction to the given vector. + Status GatherBuffersDefinedByInstruction(const HloInstruction* instruction, + BufferDefinitionVector* buffers); + + // Print points-to set for 'instruction' to 'output'. + void InstructionToString(const HloInstruction* instruction, + string* output) const; + + // Information kept per instruction + struct PerInstruction { + std::unique_ptr points_to_set; + // Empircally, ~92% of instructions have 1 + // instruction_defined_buffer, and 99% have 0 or 1 + BufferDefinitionVector instruction_defined_buffers; + }; + + const PerInstruction* PerInst(const HloInstruction* inst) const { + int id = inst->unique_id(); + DCHECK_GE(id, 0); + DCHECK_LT(id, per_instruction_.size()); + return &per_instruction_[id]; + } + PerInstruction* PerInst(const HloInstruction* inst) { + int id = inst->unique_id(); + DCHECK_GE(id, 0); + DCHECK_LT(id, per_instruction_.size()); + return &per_instruction_[id]; + } + + std::vector> GetAllUsesOfInstructionAtIndex( + HloInstruction* instruction, const ShapeIndex& index) const; + bool HasUniqueFusedUseOfOperandAt(HloInstruction* operand, + const ShapeIndex& operand_index, + HloInstruction* fusion, + const int64 use_operand_index) const; + + // The module this analysis is performed on. + const HloModule* module_; + + // The logical buffers for this module. + const std::unique_ptr logical_buffer_analysis_; + + // A map from instruction->unique_id() to + std::vector per_instruction_; + + // A map from LogicalBuffer->id() to alias information about that logical + // buffer + std::vector logical_buffer_aliases_; + + TF_DISALLOW_COPY_AND_ASSIGN(TuplePointsToAnalysis); +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_TUPLE_POINTS_TO_ANALYSIS_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/tuple_simplifier.h b/linux/include/tensorflow/compiler/xla/service/tuple_simplifier.h new file mode 100644 index 000000000..e126a5302 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/tuple_simplifier.h @@ -0,0 +1,48 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_TUPLE_SIMPLIFIER_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_TUPLE_SIMPLIFIER_H_ + +#include + +#include "tensorflow/compiler/xla/service/hlo_module.h" +#include "tensorflow/compiler/xla/service/hlo_pass_interface.h" + +namespace xla { + +// A pass which simplifies patterns of Tuple and GetTupleElement instructions in +// the module. +class TupleSimplifier : public HloModulePass { + public: + TupleSimplifier() : TupleSimplifier(/*exclude_entry_computation=*/false) {} + explicit TupleSimplifier(bool exclude_entry_computation); + ~TupleSimplifier() override {} + absl::string_view name() const override { return "tuple-simplifier"; } + + // Run tuple simplification on the given computation. Returns whether the + // computation was changed. + StatusOr Run(HloModule* module) override; + + private: + // When set, this pipeline stage will perform optimization of all computations + // apart from the module's entry computation. This is used by Graphcore's + // backend. + bool exclude_entry_computation_; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_TUPLE_SIMPLIFIER_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/tuple_util.h b/linux/include/tensorflow/compiler/xla/service/tuple_util.h new file mode 100644 index 000000000..bc5aac09f --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/tuple_util.h @@ -0,0 +1,45 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_TUPLE_UTIL_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_TUPLE_UTIL_H_ + +#include "tensorflow/compiler/xla/service/hlo_instruction.h" + +namespace xla { +class TupleUtil { + public: + // Generates HLO instructions to get a prefix tuple from `input_tuple` (which + // must be of tuple shape) of length `elements`. Returns the root of the + // graph of instructions generated. + // + // The instructions are generated into the computation containing + // `input_tuple`. + static HloInstruction* ExtractPrefix(HloInstruction* input_tuple, + int64 elements); + + // Generates HLO instructions to create a tuple that consists of the values in + // `trailing_values` appended to `input_tuple` (which must be of tuple shape). + // Returns the root of the graph of instructions generated. + // + // The instructions are generated into the computation containing + // `input_tuple`. + static HloInstruction* AppendSuffix( + HloInstruction* input_tuple, + absl::Span trailing_values); +}; +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_TUPLE_UTIL_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/while_loop_analysis.h b/linux/include/tensorflow/compiler/xla/service/while_loop_analysis.h new file mode 100644 index 000000000..bf497f489 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/while_loop_analysis.h @@ -0,0 +1,33 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_WHILE_LOOP_ANALYSIS_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_WHILE_LOOP_ANALYSIS_H_ + +#include "absl/types/optional.h" +#include "tensorflow/compiler/xla/service/hlo_instruction.h" + +namespace xla { + +// Returns the precise trip count of the loop if it's statically known, +// nullopt otherwise. max_value_returned limits the number of steps that are +// evaluated while trying to brute force a loop trip count, trip counts larger +// than max_value_returned result in nullopt. +absl::optional ComputeWhileLoopTripCount(HloInstruction *while_op, + int64 max_value_returned = 128); + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_WHILE_LOOP_ANALYSIS_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/while_loop_constant_sinking.h b/linux/include/tensorflow/compiler/xla/service/while_loop_constant_sinking.h new file mode 100644 index 000000000..577bad6c7 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/while_loop_constant_sinking.h @@ -0,0 +1,68 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_WHILE_LOOP_CONSTANT_SINKING_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_WHILE_LOOP_CONSTANT_SINKING_H_ + +#include "tensorflow/compiler/xla/service/hlo_module.h" +#include "tensorflow/compiler/xla/service/hlo_pass_interface.h" +#include "tensorflow/compiler/xla/statusor.h" + +namespace xla { + +// Sinks while loop invariant values that happen to be constants into the while +// loop body. This is probably not a win in isolation but may unlock further +// optimizations like constant folding. +// +// state = (..., const, ...) +// while (pred(state)) { +// (..., v, ...) = state +// use(v) +// state = (..., v, ...) +// } +// +// => +// +// state = (..., const, ...) +// while (pred(state)) { +// (..., v, ...) = state +// use(const) +// state = (..., v, ...) +// } +// +// Note that it leaves the `v` in place to keep that component of the state +// tuple trivially loop invariant. WhileLoopSimplifier will later get rid of +// `v`. +// +// We only sink into while loop bodies, but this can be extended to transform +// conditions as well. +// +// TODO(b/79121449): We should also sink broadcasts of constants. +class WhileLoopConstantSinking : public HloModulePass { + public: + ~WhileLoopConstantSinking() override = default; + + absl::string_view name() const override { + return "while-loop-invariant-code-motion"; + } + + StatusOr Run(HloModule* module) override; + + private: + StatusOr TrySinkingConstantsIntoWhileBody(HloInstruction* while_instr); +}; +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_WHILE_LOOP_CONSTANT_SINKING_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/while_loop_invariant_code_motion.h b/linux/include/tensorflow/compiler/xla/service/while_loop_invariant_code_motion.h new file mode 100644 index 000000000..3031899f7 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/while_loop_invariant_code_motion.h @@ -0,0 +1,55 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_WHILE_LOOP_INVARIANT_CODE_MOTION_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_WHILE_LOOP_INVARIANT_CODE_MOTION_H_ + +#include "tensorflow/compiler/xla/service/hlo_module.h" +#include "tensorflow/compiler/xla/service/hlo_pass_interface.h" +#include "tensorflow/compiler/xla/statusor.h" + +namespace xla { + +// HLO pass that rewrites while loops to hoist loop invariant instructions in +// the while body into the computation that contains the while instruction. + +class WhileLoopInvariantCodeMotion : public HloModulePass { + public: + // If `hoist_constants` is true then constants are always hoisted out of while + // loop bodies. Otherwise they are only hoisted out if they enable other + // non-trivial computations to be hoisted out. + // + // Setting `hoist_constants` to false can be help if LICM is run in the mid + // level HLO pipeline because hoisting constants out of while loop bodies can + // break optimizations like constant folding. + explicit WhileLoopInvariantCodeMotion(bool hoist_constants = false) + : hoist_constants_(hoist_constants) {} + ~WhileLoopInvariantCodeMotion() override = default; + + absl::string_view name() const override { + return "while-loop-invariant-code-motion"; + } + StatusOr Run(HloModule* module) override; + + private: + bool NotWorthHoistingIndividually(const HloInstruction& instruction); + StatusOr TryHoistingInvariantInstructionsFromWhileBody( + HloInstruction* while_instr); + + bool hoist_constants_; +}; +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_WHILE_LOOP_INVARIANT_CODE_MOTION_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/while_loop_simplifier.h b/linux/include/tensorflow/compiler/xla/service/while_loop_simplifier.h new file mode 100644 index 000000000..0bc5a0107 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/while_loop_simplifier.h @@ -0,0 +1,42 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_WHILE_LOOP_SIMPLIFIER_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_WHILE_LOOP_SIMPLIFIER_H_ + +#include "tensorflow/compiler/xla/service/hlo_module.h" +#include "tensorflow/compiler/xla/service/hlo_pass_interface.h" +#include "tensorflow/compiler/xla/statusor.h" + +namespace xla { + +// HLO pass that makes the following transformations on while loops: +// +// - A while loop with static trip count of 0 is deleted. +// - A while loop with static trip count of 1 is replaced by its body (sans +// loop). +// - Elements of a while loop's tuple that the loop doesn't use are removed +// from the tuple. +// +class WhileLoopSimplifier : public HloModulePass { + public: + ~WhileLoopSimplifier() override {} + absl::string_view name() const override { return "simplify-while-loops"; } + StatusOr Run(HloModule* module) override; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_WHILE_LOOP_SIMPLIFIER_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/while_util.h b/linux/include/tensorflow/compiler/xla/service/while_util.h new file mode 100644 index 000000000..b1c448688 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/while_util.h @@ -0,0 +1,90 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_WHILE_UTIL_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_WHILE_UTIL_H_ + +#include "tensorflow/compiler/xla/service/call_inliner.h" +#include "tensorflow/compiler/xla/service/hlo_instruction.h" + +namespace xla { +class WhileUtil { + public: + // Holds a return value from MakeInstructionsLiveIn. + struct MakeInstructionsLiveInResult { + // The new while operation that has the requested values live in. + HloInstruction* new_while_instr; + + // The i'th element of `while_body_live_in_values` is an instruction in the + // while body that holds the i'th *newly added* live in value at runtime. + std::vector while_body_live_in_values; + + // `while_body_instruction_map` maps instructions in the original while body + // to the corresponding instructions in the body for the newly created while + // operation. + CallInliner::InlinedInstructionMap while_body_instruction_map; + }; + + // Replaces `while_instr` with a new while instruction that is equivalent to + // `while_instr` except that it has all of the HLO instructions in + // `instructions` as live-in, loop invariant values. These new live in values + // are represented as new elements appended to the parameter of the while + // loop, which must be of tuple shape. GetTupleElement instructions computing + // each new live in value is returned in the `while_body_live_in_values` + // vector. + // + // Deletes `while_instr` after replacing it. + // + // Preconditions: + // + // `while_instr` must have a tuple shaped state. + // + // Every instruction in `instructions` must be contained in the computation + // that contains `while_instr`. + static StatusOr MakeInstructionsLiveIn( + HloInstruction* while_instr, + absl::Span instructions); + + using LoopStateTy = std::vector; + using LoopBodyGeneratorTy = std::function( + HloInstruction* /*induction_var*/, + const LoopStateTy& /*current_values*/)>; + + // Creates a while loop in `computation` that runs for `trip_count` + // iterations. The structure of the while loop is as follows, in pseudocode: + // + // loop_state while_loop() { + // indvar = 0; + // loop_state = init_values + // while (indvar < trip_count) { + // loop_state = loop_body_generator(loop_state) + // indvar++; + // } + // return loop_state; + // } + static StatusOr MakeCountedLoop( + HloComputation* computation, int32 trip_count, + const LoopStateTy& init_values, + const LoopBodyGeneratorTy& loop_body_generator); + + // Returns the GetTupleElement instructions in `while_body` that access + // elements in the parameter tuple that don't change across iterations. + // Assumes `while_body` is the body computation of the while loop in question. + static std::vector GetInvariantGTEsForWhileBody( + const HloComputation& while_body); +}; +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_WHILE_UTIL_H_ diff --git a/linux/include/tensorflow/compiler/xla/service/zero_sized_hlo_elimination.h b/linux/include/tensorflow/compiler/xla/service/zero_sized_hlo_elimination.h new file mode 100644 index 000000000..87294120d --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service/zero_sized_hlo_elimination.h @@ -0,0 +1,32 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_ZERO_SIZED_HLO_ELIMINATION_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_ZERO_SIZED_HLO_ELIMINATION_H_ + +#include "tensorflow/compiler/xla/service/hlo_module.h" +#include "tensorflow/compiler/xla/service/hlo_pass_interface.h" + +// HLO pass that replaces zero sized Hlos with a zero sized constant literal. +namespace xla { +class ZeroSizedHloElimination : public HloModulePass { + public: + StatusOr Run(HloModule* module) override; + absl::string_view name() const override { + return "zero_sized_hlo_elimination"; + } +}; +} // namespace xla +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_ZERO_SIZED_HLO_ELIMINATION_H_ diff --git a/linux/include/tensorflow/compiler/xla/service_interface.h b/linux/include/tensorflow/compiler/xla/service_interface.h new file mode 100644 index 000000000..14c35e7b8 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/service_interface.h @@ -0,0 +1,85 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SERVICE_INTERFACE_H_ +#define TENSORFLOW_COMPILER_XLA_SERVICE_INTERFACE_H_ + +#include "tensorflow/compiler/xla/status.h" +#include "tensorflow/compiler/xla/xla.pb.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" + +namespace xla { + +// Defines the interface for an XLA service on the client side. This service +// helps abstract around the actual implementation of a service - the service +// can be local (running in the same process), or remote - in which case an RPC +// stub is used as the implementation. +class ServiceInterface { + public: + ServiceInterface() {} + virtual ~ServiceInterface() = default; + + // TODO(b/31824348): Convert to use StatusOr. + virtual Status TransferToClient(const TransferToClientRequest* arg, + TransferToClientResponse* result) = 0; + + virtual Status TransferToServer(const TransferToServerRequest* arg, + TransferToServerResponse* result) = 0; + + virtual Status TransferToInfeed(const TransferToInfeedRequest* arg, + TransferToInfeedResponse* result) = 0; + + virtual Status TransferFromOutfeed(const TransferFromOutfeedRequest* arg, + TransferFromOutfeedResponse* result) = 0; + + virtual Status ResetDevice(const ResetDeviceRequest* arg, + ResetDeviceResponse* result) = 0; + + virtual Status ExecuteGraph(const ExecuteGraphRequest* arg, + ExecuteResponse* result) = 0; + + virtual Status ExecuteGraphParallel(const ExecuteGraphParallelRequest* arg, + ExecuteParallelResponse* result) = 0; + + virtual Status WaitForExecution(const WaitForExecutionRequest* arg, + WaitForExecutionResponse* result) = 0; + + virtual Status DeconstructTuple(const DeconstructTupleRequest* arg, + DeconstructTupleResponse* result) = 0; + + virtual Status GetComputationGraphStats( + const ComputationGraphStatsRequest* arg, + ComputationStatsResponse* result) = 0; + + virtual Status GetShape(const GetShapeRequest* arg, + GetShapeResponse* result) = 0; + + virtual Status CreateChannelHandle(const CreateChannelHandleRequest* arg, + CreateChannelHandleResponse* result) = 0; + + virtual Status GetDeviceHandles(const GetDeviceHandlesRequest* arg, + GetDeviceHandlesResponse* result) = 0; + + virtual Status ComputeConstantGraph(const ComputeConstantGraphRequest* arg, + ComputeConstantResponse* result) = 0; + + // Methods used by GlobalData. + virtual Status Unregister(const UnregisterRequest* arg, + UnregisterResponse* result) = 0; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SERVICE_INTERFACE_H_ diff --git a/linux/include/tensorflow/compiler/xla/shape_layout.h b/linux/include/tensorflow/compiler/xla/shape_layout.h new file mode 100644 index 000000000..214cf9885 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/shape_layout.h @@ -0,0 +1,92 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SHAPE_LAYOUT_H_ +#define TENSORFLOW_COMPILER_XLA_SHAPE_LAYOUT_H_ + +#include + +#include "tensorflow/compiler/xla/shape_util.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/lib/core/status.h" + +namespace xla { + +// A ShapeLayout object encapsulates the layout of a particular shape (including +// tuples). This differs from the Layout proto which describes the layout of a +// single array. ShapeLayout contains a Layout proto for each array in the shape +// (a tuple can have more than one array). For array shapes, this object +// trivially holds a single Layout. Logically, ShapeLayout holds a nonmutable +// shape with mutable layouts. +class ShapeLayout { + public: + // Constructs a ShapeLayout of the given shape. Layouts are copied from the + // shape parameter. + explicit ShapeLayout(const Shape& shape) : shape_(shape) {} + + // Assigns the layouts in this ShapeLayout to the Layout fields of the given + // shape. 'to_shape' and the shape of the ShapeLayout object must be + // compatible. + Status AssignLayoutToShape(Shape* to_shape) const; + + // Returns true if the Layouts in this ShapeLayout match the layouts in the + // given shape. Returns false otherwise. If the given shape is not compatible + // with the ShapeLayout's shape, then false is returned. + bool MatchesLayoutInShape(const Shape& shape) const; + + // Copies the layout from the given shape into this ShapeLayout. 'other_shape' + // must be compatible with the ShapeLayout's shape. + Status CopyLayoutFromShape(const Shape& other_shape); + + // Clears (Layout::Clear) all the Layouts stored in this object. + void Clear(); + + // Sets all Layouts stored in this object to the default layout. + void SetToDefaultLayout(); + + // Returns the shape (with layouts). + const Shape& shape() const { return shape_; } + + // Checks that a layout is set for the shape, and returns a reference to the + // layout directly on the shape. Shape must not be a tuple. + const Layout& layout() const; + + // Returns true if all layouts have been set for this ShapeLayout object. That + // is, every array has a layout. + bool LayoutIsSet() const; + + // Resets the layout on the shape to the provided layout. Shape must not be a + // tuple. + void ResetLayout(const Layout& layout); + + // Resets the layout on the shape at the provided ShapeIndex to the provided + // layout. Shape must be a tuple. + void ResetLayout(const Layout& layout, ShapeIndexView shape_index); + + // Returns a string representation of this object. + string ToString() const { return ShapeUtil::HumanStringWithLayout(shape_); } + + // Tests for equality of both shape and layout (ShapeUtil::Equal). + bool operator==(const ShapeLayout& other) const; + bool operator!=(const ShapeLayout& other) const; + + private: + Shape shape_; +}; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SHAPE_LAYOUT_H_ diff --git a/linux/include/tensorflow/compiler/xla/shape_tree.h b/linux/include/tensorflow/compiler/xla/shape_tree.h new file mode 100644 index 000000000..df610102b --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/shape_tree.h @@ -0,0 +1,681 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_SHAPE_TREE_H_ +#define TENSORFLOW_COMPILER_XLA_SHAPE_TREE_H_ + +#include +#include +#include +#include + +#include "absl/memory/memory.h" +#include "absl/types/optional.h" +#include "absl/types/span.h" +#include "tensorflow/compiler/xla/layout_util.h" +#include "tensorflow/compiler/xla/shape_util.h" +#include "tensorflow/compiler/xla/status_macros.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/lib/core/errors.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/lib/gtl/iterator_range.h" +#include "tensorflow/core/platform/logging.h" +#include "tensorflow/core/platform/types.h" + +namespace xla { + +namespace internal { + +// Internal representation of each node in a ShapeTree. +template +struct ShapeTreeNode { + // Data corresponding to this node. + std::pair data; + + bool is_leaf = true; + + explicit ShapeTreeNode(ShapeIndex index) + : ShapeTreeNode(std::move(index), T()) {} + ShapeTreeNode(ShapeIndex index, T data) + : data(std::move(index), std::move(data)) {} +}; + +// Internal representation of an index table entry. +struct IndexTableEntry { + // Index of the node in the ShapeTreeNode vector. + uint32 index; + // Index of the first child in a IndexTableEntry vector. In the index + // table all children entries for a given node will be placed next to each + // other. This allows us to use a single field to index them. + uint32 children_start; +#ifndef NDEBUG + // Number of children, used for bounds checking. + uint32 children_count; +#endif +}; + +} // namespace internal + +template +class ShapeTreeIterator; + +// A ShapeTree is a recursive data structure which mirrors the structure of a +// XLA shape and holds a value of type T for each subshape (i.e. tuple or array) +// in the shape. For array shapes, a ShapeTree trivially holds a single value of +// type T. +// +// For tuple shapes which can be an arbitrary tree with arrays at the leaves, a +// ShapeTree is an identically structured tree with data elements of type T at +// every node. I.e. the root is a tuple by definition, all interior nodes are +// also tuples, and all leaves are arrays. +// +// Like the Shape data structure, this is a tree and tuple elements cannot be +// duplicated. That is, every distinct ShapeIndex in the Shape has a unique T +// object. +// +// Normally a ShapeTree owns its Shape, but for efficiency reasons, sometimes +// it's helpful not to copy a Shape just to make a ShapeTree. In these cases, +// you can pass a Shape* instead of a Shape& to the ShapeTree constructor. It's +// then up to you to ensure that the pointed-to Shape doesn't die or mutate +// before its ShapeTree goes away. +template +class ShapeTree { + public: + using Node = internal::ShapeTreeNode; + using Index = internal::IndexTableEntry; + + // Default constructor creates a tree with a nil shape (i.e. an empty tuple). + ShapeTree() : ShapeTree(ShapeUtil::MakeNil()) {} + + // Create ShapeTree with the given shape, and default-constructed T values for + // all nodes. + // + // The version that takes a pointer may be cheaper because it doesn't require + // any Shape copies, but then it's up to you to ensure that the pointer stays + // alive longer than this ShapeTree. + explicit ShapeTree(Shape shape); + explicit ShapeTree(const Shape* shape); + explicit ShapeTree(const std::shared_ptr& shape); + + // Create ShapeTree with the given shape, and init_value for all nodes. + ShapeTree(Shape shape, const T& init_value); + ShapeTree(const Shape* shape, const T& init_value); + ShapeTree(const std::shared_ptr& shape, const T& init_value); + + // Returns the data element associated with the array in the shape at the + // given index (see ShapeUtil::GetSubshape for how indexes are defined). + const T& element(ShapeIndexView index) const; + T* mutable_element(ShapeIndexView index); + + // Return the shape represented with this ShapeTree. + const Shape& shape() const { return *shape_; } + + // Replaces *only* the underlying shape of this ShapeTree. The caller must own + // the Shape object and hence shape_storage_ is not updated. + // + // Only safe to use this if the ShapeTree was constructed with 'explicit + // ShapeTree(const Shape* shape)' or is moved from one such ShapeTree. The + // caller must ensure that the input shape is consistent with the underlying + // tree. + void replace_shape_ptr(const Shape* shape) { + CHECK(shape_storage_.get() == nullptr); + shape_ = shape; + } + + // Returns true if the node at the given index is a leaf node (an array + // shape). + bool IsLeaf(ShapeIndexView index) const { return Lookup(index)->is_leaf; } + + ShapeTree(const ShapeTree&) = default; + ShapeTree& operator=(const ShapeTree&) = default; + ShapeTree(ShapeTree&&) = default; + ShapeTree& operator=(ShapeTree&& other) = default; + + // iterator implements a bidirectional_iterator with + // value_type = std::pair. + // + // The iteration order is guaranteed to be a pre-order walk of the ShapeTree. + using iterator = + ShapeTreeIterator, typename std::vector::iterator, + std::pair>; + using const_iterator = + ShapeTreeIterator, + typename std::vector::const_iterator, + const std::pair>; + using reverse_iterator = std::reverse_iterator; + using const_reverse_iterator = std::reverse_iterator; + + // begin/end for iterating over all nodes. + iterator begin() { + return iterator(&nodes_, nodes_.begin(), + /*iterate_leaves_only=*/false); + } + iterator end() { + return iterator(&nodes_, nodes_.end(), + /*iterate_leaves_only=*/false); + } + const_iterator begin() const { + return const_iterator(&nodes_, nodes_.begin(), + /*iterate_leaves_only=*/false); + } + const_iterator end() const { + return const_iterator(&nodes_, nodes_.end(), + /*iterate_leaves_only=*/false); + } + + // rbegin/rend for iterating over all nodes in reverse. + reverse_iterator rbegin() { return reverse_iterator(end()); } + reverse_iterator rend() { return reverse_iterator(begin()); } + const_reverse_iterator rbegin() const { + return const_reverse_iterator(end()); + } + const_reverse_iterator rend() const { + return const_reverse_iterator(begin()); + } + + // leaf_begin()/leaf_end() iterates over all leaf nodes (nodes with no + // children). + iterator leaf_begin() { + return iterator(&nodes_, nodes_.begin(), + /*iterate_leaves_only=*/true); + } + iterator leaf_end() { + return iterator(&nodes_, nodes_.end(), + /*iterate_leaves_only=*/true); + } + const_iterator leaf_begin() const { + return const_iterator(&nodes_, nodes_.begin(), + /*iterate_leaves_only=*/true); + } + const_iterator leaf_end() const { + return const_iterator(&nodes_, nodes_.end(), + /*iterate_leaves_only=*/true); + } + // range-based iterator for leaf_begin()/leaf_end(). + tensorflow::gtl::iterator_range leaves() { + return tensorflow::gtl::make_range(leaf_begin(), leaf_end()); + } + tensorflow::gtl::iterator_range leaves() const { + return tensorflow::gtl::make_range(leaf_begin(), leaf_end()); + } + + reverse_iterator leaf_rbegin() { return reverse_iterator(leaf_end()); } + reverse_iterator leaf_rend() { return reverse_iterator(leaf_begin()); } + const_reverse_iterator leaf_rbegin() const { + return const_reverse_iterator(leaf_end()); + } + const_reverse_iterator leaf_rend() const { + return const_reverse_iterator(leaf_begin()); + } + + // Returns an iterator pointing to the given ShapeIndex. + // REQUIRES: index must exist in the ShapeTree. + iterator find(ShapeIndexView index) { + Node* element = Lookup(index); + auto element_iter = nodes_.begin() + (element - &nodes_[0]); + return iterator(&nodes_, element_iter, /*iterate_leaves_only=*/false); + } + const_iterator find(ShapeIndexView index) const { + Node* element = Lookup(index); + auto element_iter = nodes_.cbegin() + (element - &nodes_[0]); + return const_iterator(&nodes_, element_iter, /*iterate_leaves_only=*/false); + } + + // Returns the number of leaf nodes in the tree. + int64 leaf_count() const { return std::distance(leaf_begin(), leaf_end()); } + + // Recursively traverses the shape and calls the given function at each + // element. The function has the following arguments: + // + // Fn : A callable of type void(const ShapeIndex& index, const T& data) + // (or compatible). + // index : the index of the element in the shape. See ShapeUtil::GetSubshape + // for definition of index. + // data : The data value at this element. + template + void ForEachElement(const Fn& func) const; + + // Like ForEachElement, but the callable has type + // + // void (const ShapeIndex& index, T* data). + // + template + void ForEachMutableElement(const Fn& func); + + // Like ForEach(Mutable)Element, but the callable returns a Status instead of + // void. The first non-OK return value is returned by the ForEach* function. + template + Status ForEachElementWithStatus(const Fn& func) const; + template + Status ForEachMutableElementWithStatus(const Fn& func); + + // Maps each element to generate a new tree with the same shape. + template + ShapeTree Map(const std::function& func) { + ShapeTree result(shape_storage_); + ForEachElement([&](const ShapeIndex& index, const T& t) { + *result.mutable_element(index) = func(t); + }); + return result; + } + + template + ShapeTree Map(const std::function& func) { + ShapeTree result(shape_storage_); + ForEachMutableElement([&](const ShapeIndex& index, T* t) { + *result.mutable_element(index) = func(t); + }); + return result; + } + + // Copy the subtree of values from 'other' rooted at ShapeIndex + // 'source_base_index' into the subtree of value in this ShapeTree rooted at + // 'target_base_index'. + // + // Precondition: The subshape of other.shape() at index source_base_index must + // be compatible with the subshape of shape() at index target_base_index. + void CopySubtreeFrom(const ShapeTree& other, + const ShapeIndex& source_base_index, + const ShapeIndex& target_base_index); + + bool operator==(const ShapeTree& other) const; + bool operator!=(const ShapeTree& other) const { return !(*this == other); } + + private: + // Initialize node->children based on 'shape'. All children are assigned the + // the given 'init_value'. + void InitChildren(const Shape& shape, const T& init_value, Node* node, + Index* index); + + // Initialize node->children based on 'shape'. All children have + // default-constructed data values. + void InitChildren(const Shape& shape, Node* node, Index* index); + + // Returns the number of subshapes, including interior nodes, in shape. + int64 CountSubshapes(const Shape& shape); + + // Helpers for traversing the shape via ForEachElement. The helpers + // recursively traverse the subtree rooted at "index" (defined as in + // ShapeUtil::GetSubshape). + template + static Status ForEachHelper(const Fn& func, const std::vector& nodes); + template + static Status ForEachMutableHelper(const Fn& func, std::vector* nodes); + + // Return the tree node at the given index. + Node* Lookup(ShapeIndexView index); + const Node* Lookup(ShapeIndexView index) const; + + // The nodes in this shape tree. + std::vector nodes_; + + // Index table for node lookups. + std::vector index_table_; + + // If we own our Shape, this field contains it, and shape_ is a pointer into + // here. Otherwise if we don't own our shape, this is nullptr. + std::shared_ptr shape_storage_; + + // The XLA shape mirrored in this ShapeTree. This is either + // shape_storage_.get() or the Shape pointer passed to our constructor. + const Shape* shape_; +}; + +// Internal iterator that performs a pre-order walk. This is cheap to copy. +// The iterator value_type is equivalent to a +// std::pair&, similar to std::map. +template +class ShapeTreeIterator + : public std::iterator { + public: + ShapeTreeIterator(ContainerType* nodes, IteratorType node, + bool iterate_leaves_only) + : nodes_(nodes), + node_(std::move(node)), + iterate_leaves_only_(iterate_leaves_only) { + while (iterate_leaves_only && node_ != nodes_->end() && !node_->is_leaf) { + ++node_; + } + } + + ShapeTreeIterator& operator++() { + ++node_; + while (iterate_leaves_only_ && node_ != nodes_->end() && !node_->is_leaf) { + ++node_; + } + return *this; + } + ShapeTreeIterator operator++(int) { + auto i = *this; + ++(*this); + return i; + } + + ShapeTreeIterator& operator--() { + --node_; + while (iterate_leaves_only_ && node_ > nodes_->begin() && !node_->is_leaf) { + --node_; + } + return *this; + } + ShapeTreeIterator operator--(int) { + auto i = *this; + --(*this); + return i; + } + + bool operator==(const ShapeTreeIterator& other) const { + return node_ == other.node_; + } + bool operator!=(const ShapeTreeIterator& other) const { + return node_ != other.node_; + } + ValueType& operator*() { return node_->data; } + ValueType* operator->() { return &node_->data; } + + private: + ContainerType* nodes_; + IteratorType node_; + // True if we should not include interior nodes in our walk. + const bool iterate_leaves_only_; +}; + +template +int64 ShapeTree::CountSubshapes(const Shape& shape) { + int64 current_count = 1; + if (ShapeUtil::IsTuple(shape)) { + int64 count = ShapeUtil::TupleElementCount(shape); + for (int i = 0; i < count; ++i) { + current_count += CountSubshapes(shape.tuple_shapes(i)); + } + } + return current_count; +} + +template +void ShapeTree::InitChildren(const Shape& shape, const T& init_value, + Node* node, Index* index) { + if (ShapeUtil::IsTuple(shape)) { + const int64 size = ShapeUtil::TupleElementCount(shape); +#ifndef NDEBUG + index->children_count = size; +#endif + node->is_leaf = false; + ShapeIndex shape_index = node->data.first; + shape_index.push_back(0); + + // At the end of the index_table, reserve a continuous space to hold the + // children of current node. In order to enforce the invariant that all + // children of a given node are placed together, we need to do the + // reservation before we recurse into any of its children. + int64 children_start_position = index_table_.size(); + index_table_.resize(index_table_.size() + size); + + for (int i = 0; i < size; ++i) { + shape_index[shape_index.size() - 1] = i; + index_table_[children_start_position + i].index = nodes_.size(); + // The first child of the node in the index table is placed at the end of + // the table. + index_table_[children_start_position + i].children_start = + index_table_.size(); + nodes_.emplace_back(shape_index, init_value); + InitChildren(shape.tuple_shapes(i), init_value, &nodes_.back(), + &index_table_[children_start_position + i]); + } + } else { +#ifndef NDEBUG + index->children_count = 0; +#endif + } +} + +template +void ShapeTree::InitChildren(const Shape& shape, Node* node, Index* index) { + if (ShapeUtil::IsTuple(shape)) { + const int64 size = ShapeUtil::TupleElementCount(shape); +#ifndef NDEBUG + index->children_count = size; +#endif + node->is_leaf = false; + ShapeIndex shape_index = node->data.first; + shape_index.push_back(0); + + // At the end of the index_table, reserve a continuous space to hold the + // children of current node. In order to enforce the invariant that all + // children of a given node are placed together, we need to do the + // reservation before we recurse into any of its children. + int64 children_start_position = index_table_.size(); + index_table_.resize(index_table_.size() + size); + + for (int i = 0; i < size; ++i) { + shape_index[shape_index.size() - 1] = i; + index_table_[children_start_position + i].index = nodes_.size(); + // The first child of the node in the index table is placed at the end of + // the table. + index_table_[children_start_position + i].children_start = + index_table_.size(); + nodes_.emplace_back(shape_index); + InitChildren(shape.tuple_shapes(i), &nodes_.back(), + &index_table_[children_start_position + i]); + } + } else { +#ifndef NDEBUG + index->children_count = 0; +#endif + } +} + +template +ShapeTree::ShapeTree(Shape shape) + : shape_storage_(std::make_shared(std::move(shape))), + shape_(shape_storage_.get()) { + const int64 count = CountSubshapes(*shape_); + nodes_.reserve(count); + nodes_.emplace_back(ShapeIndex{}); + + index_table_.reserve(count); + index_table_.emplace_back(Index{0, 1}); + InitChildren(*shape_, &nodes_[0], &index_table_[0]); +} + +template +ShapeTree::ShapeTree(const Shape* shape) : shape_(shape) { + const int64 count = CountSubshapes(*shape_); + nodes_.reserve(count); + nodes_.emplace_back(ShapeIndex{}); + + index_table_.reserve(count); + index_table_.emplace_back(Index{0, 1}); + InitChildren(*shape_, &nodes_[0], &index_table_[0]); +} + +template +ShapeTree::ShapeTree(const std::shared_ptr& shape) + : shape_storage_(shape), shape_(shape_storage_.get()) { + const int64 count = CountSubshapes(*shape_); + nodes_.reserve(count); + nodes_.emplace_back(ShapeIndex{}); + + index_table_.reserve(count); + index_table_.emplace_back(Index{0, 1}); + InitChildren(*shape_, &nodes_[0], &index_table_[0]); +} + +template +ShapeTree::ShapeTree(Shape shape, const T& init_value) + : shape_storage_(std::make_shared(std::move(shape))), + shape_(shape_storage_.get()) { + const int64 count = CountSubshapes(*shape_); + nodes_.reserve(count); + nodes_.emplace_back(ShapeIndex{}, init_value); + + index_table_.reserve(count); + index_table_.emplace_back(Index{0, 1}); + InitChildren(*shape_, init_value, &nodes_[0], &index_table_[0]); +} + +template +ShapeTree::ShapeTree(const Shape* shape, const T& init_value) + : shape_(shape) { + const int64 count = CountSubshapes(*shape_); + nodes_.reserve(count); + nodes_.emplace_back(ShapeIndex{}, init_value); + + index_table_.reserve(count); + index_table_.emplace_back(Index{0, 1}); + InitChildren(*shape_, init_value, &nodes_[0], &index_table_[0]); +} + +template +ShapeTree::ShapeTree(const std::shared_ptr& shape, + const T& init_value) + : shape_storage_(shape), shape_(shape_storage_.get()) { + const int64 count = CountSubshapes(*shape_); + nodes_.reserve(count); + nodes_.emplace_back(ShapeIndex{}, init_value); + + index_table_.reserve(count); + index_table_.emplace_back(Index{0, 1}); + InitChildren(*shape_, init_value, &nodes_[0], &index_table_[0]); +} + +template +const T& ShapeTree::element(ShapeIndexView index) const { + return Lookup(index)->data.second; +} + +template +T* ShapeTree::mutable_element(ShapeIndexView index) { + return &Lookup(index)->data.second; +} + +template +internal::ShapeTreeNode* ShapeTree::Lookup(ShapeIndexView index) { + Index* iter = &index_table_[0]; + for (const int64 i : index) { + CHECK_GE(i, 0); +#ifndef NDEBUG + CHECK_LT(i, iter->children_count); +#endif + iter = &index_table_[iter->children_start + i]; + } + + return &nodes_[iter->index]; +} + +template +const internal::ShapeTreeNode* ShapeTree::Lookup( + ShapeIndexView index) const { + return const_cast(this)->Lookup(index); +} + +/* static */ +template +template +Status ShapeTree::ForEachHelper(const Fn& func, + const std::vector& nodes) { + for (const auto& node : nodes) { + TF_RETURN_IF_ERROR(func(node.data.first, node.data.second)); + } + return Status::OK(); +} + +/* static */ +template +template +Status ShapeTree::ForEachMutableHelper(const Fn& func, + std::vector* nodes) { + for (auto& node : *nodes) { + TF_RETURN_IF_ERROR(func(node.data.first, &node.data.second)); + } + return Status::OK(); +} + +template +template +Status ShapeTree::ForEachElementWithStatus(const Fn& func) const { + return ForEachHelper(func, nodes_); +} + +template +template +Status ShapeTree::ForEachMutableElementWithStatus(const Fn& func) { + return ForEachMutableHelper(func, &nodes_); +} + +template +template +void ShapeTree::ForEachElement(const Fn& func) const { + return ForEachHelper( + [&func](const ShapeIndex& index, const T& data) { + func(index, data); + return Status::OK(); + }, + nodes_) + .IgnoreError(); +} + +template +template +void ShapeTree::ForEachMutableElement(const Fn& func) { + return ForEachMutableHelper( + [&func](const ShapeIndex& index, T* data) { + func(index, data); + return Status::OK(); + }, + &nodes_) + .IgnoreError(); +} + +template +void ShapeTree::CopySubtreeFrom(const ShapeTree& other, + const ShapeIndex& source_base_index, + const ShapeIndex& target_base_index) { + CHECK(ShapeUtil::Compatible( + ShapeUtil::GetSubshape(shape(), target_base_index), + ShapeUtil::GetSubshape(other.shape(), source_base_index))); + ForEachMutableElement([this, &other, &source_base_index, &target_base_index]( + const ShapeIndex& index, T* data) { + // Copy the data element only if index is in the + // subtree rooted at target_base_index. + for (int i = 0; i < target_base_index.size(); ++i) { + if (i >= index.size() || index[i] != target_base_index[i]) { + return; + } + } + // Construct source element index to copy from. + ShapeIndex source_index = source_base_index; + for (int i = target_base_index.size(); i < index.size(); ++i) { + source_index.push_back(index[i]); + } + *data = other.element(source_index); + }); +} + +template +bool ShapeTree::operator==(const ShapeTree& other) const { + bool equal = true; + ForEachElement( + [this, &other, &equal](const ShapeIndex& index, const T& data) { + if (data != other.element(index)) { + equal = false; + } + }); + return equal; +} + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SHAPE_TREE_H_ diff --git a/linux/include/tensorflow/compiler/xla/shape_util.h b/linux/include/tensorflow/compiler/xla/shape_util.h new file mode 100644 index 000000000..d8bb27bea --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/shape_util.h @@ -0,0 +1,776 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// Shapes are protobuf messages, so this utility header offers a bunch of +// functionality for querying / poking at them. + +#ifndef TENSORFLOW_COMPILER_XLA_SHAPE_UTIL_H_ +#define TENSORFLOW_COMPILER_XLA_SHAPE_UTIL_H_ + +#include +#include + +#include "absl/base/macros.h" +#include "absl/container/inlined_vector.h" +#include "absl/types/optional.h" +#include "absl/types/span.h" +#include "tensorflow/compiler/xla/layout_util.h" +#include "tensorflow/compiler/xla/primitive_util.h" +#include "tensorflow/compiler/xla/status_macros.h" +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/compiler/xla/util.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/lib/core/threadpool.h" +#include "tensorflow/core/platform/cpu_info.h" +#include "tensorflow/core/platform/env.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/types.h" + +namespace xla { + +// An index for specifying a particular nested subshape within a shape. Used in +// ShapeUtil::GetSubshape and other interfaces. Shapes are recursive data +// structures (trees) and ShapeIndex defines a path through the tree where each +// element of ShapeIndex indexes into a tuple (or nested tuple) within the +// shape. For a non-nested tuple, an index has a single element. For example, +// given a 3-element tuple (a, b, c) containing arrays a, b, and c, the index +// {1} corresponds to array b. For a nested tuple, the index can have more than +// one element. For the nested tuple (a, (b, c, d), e) below are the values +// corresponding to the given indices: +// +// index {0} : array a +// index {1, 2} : array d +// index {2} : array e +// index {0, 0} : invalid index (element at {0} is an array not a tuple) +// +// For indexing into array shapes, the index is always trivially empty, ie {}. +// +// ShapeIndex is a trivial wrapper around std::vector with a minimum number of +// methods implemented. +class ShapeIndex { + public: + ShapeIndex() = default; + ShapeIndex(std::initializer_list init) : indices_(init) {} + template + ShapeIndex(InputIt start, InputIt end) : indices_(start, end) {} + + bool empty() const { return indices_.empty(); } + size_t size() const { return indices_.size(); } + void push_back(int64 value) { indices_.push_back(value); } + void pop_back() { indices_.pop_back(); } + + // push_front is O(n^2), but shapes don't usually have a ton of dimensions. + void push_front(int64 value) { indices_.insert(indices_.begin(), value); } + + using container_type = absl::InlinedVector; + + container_type::const_iterator begin() const { return indices_.begin(); } + container_type::const_iterator end() const { return indices_.end(); } + container_type::iterator begin() { return indices_.begin(); } + container_type::iterator end() { return indices_.end(); } + + const int64* data() const { return indices_.data(); } + + int64 back() const { return indices_.back(); } + int64& back() { return indices_.back(); } + + const int64& operator[](size_t i) const { return indices_[i]; } + int64& operator[](size_t i) { return indices_[i]; } + + bool operator==(const ShapeIndex& other) const { + return indices_ == other.indices_; + } + bool operator!=(const ShapeIndex& other) const { return !(*this == other); } + bool operator<(const ShapeIndex& other) const { + return indices_ < other.indices_; + } + + string ToString() const; + + private: + container_type indices_; +}; + +// A view into a ShapeIndex as above, with the cheap/easy ability to consume the +// value at the front of the view. +// +// NB! ShapeIndexView does not own the memory backing the index array. +// The memory backing the index array should be owned by an object +// that lives longer than the ShapeIndexView instances pointing into +// it. +class ShapeIndexView { + public: + ShapeIndexView(const ShapeIndex& shape_index, int64 offset = 0) + : indices_(shape_index.data() + offset, shape_index.size() - offset) { + CHECK_LE(offset, shape_index.size()); + } + ShapeIndexView(std::initializer_list indices) : indices_(indices) {} + ShapeIndexView(const ShapeIndexView& other) = default; + + using iterator = const int64*; + + iterator begin() const { return indices_.begin(); } + iterator end() const { return indices_.end(); } + int64 size() const { return indices_.size(); } + bool empty() const { return indices_.empty(); } + int64 front() const { + CHECK(!empty()); + return indices_.front(); + } + ShapeIndexView ConsumeFront() const { + ShapeIndexView result = *this; + result.indices_.remove_prefix(1); + return result; + } + ShapeIndexView ConsumeBack() const { + ShapeIndexView result = *this; + result.indices_.remove_suffix(1); + return result; + } + ShapeIndex ToShapeIndex() const { return ShapeIndex(begin(), end()); } + + bool operator==(const ShapeIndexView& other) const; + bool operator!=(const ShapeIndexView& other) const; + + string ToString() const; + + private: + absl::Span indices_; +}; + +std::ostream& operator<<(std::ostream& out, const ShapeIndex& shape_index); +std::ostream& operator<<(std::ostream& out, const ShapeIndexView& shape_index); + +// Namespaced collection of (static) shape utilities. +// +// These are all effectively convenience functions for testing/tweaking proto +// properties, which do invariant checks before / after the operation. +class ShapeUtil { + public: + // Data structure which describes the coordinates and the shape, of a tuple + // shaped sub-shape. + struct IndexedShape { + IndexedShape() = default; + IndexedShape(ShapeIndex index, Shape shape) + : index(std::move(index)), shape(std::move(shape)) {} + ShapeIndex index; + Shape shape; + }; + + // Returns the number of elements are contained within the provided shape; + // e.g. for rank 0 (scalars) the result is always 1. Note that sparse shapes + // may not actually be able to store this number of elements. See + // LayoutUtil::MaxSparseElements(shape) to obtain the maximum number of + // elements that can be stored in a sparse shape. + // Precondition: IsArray(shape) + static int64 ElementsIn(const Shape& shape); + + // As ElementsIn(), but recurses through tuples. + static int64 ElementsInRecursive(const Shape& shape); + + // Returns true if shape has the primitive type, recurses through tuples. + static bool HasPrimitiveType(const Shape& shape, + PrimitiveType primitive_type); + + // Returns true if 'shape' is an array with zero elements. + static bool IsZeroElementArray(const Shape& shape); + + // Returns the number of bytes required for an allocation of shape. The + // |pointer_size| parameter is used for calculating the size of tuple + // shapes. This includes only the size of the top-level buffer. For example, a + // tuple is stored as an array of pointers to other buffers. In this case, + // this method only returns the size of the pointer array. + static int64 ByteSizeOf(const Shape& shape, int64 pointer_size = -1); + + // Returns the number of bytes used to store the primitive_type. + // + // Precondition: ShapeUtil::IsArray(shape) + static int64 ByteSizeOfPrimitiveType(PrimitiveType primitive_type); + + // Returns the number of bytes required to store the tuple member pointers for + // a allocation of shape. The `shape` must be a TUPLE shape, and + // `pointer_size` must be larger than zero. + static int64 ByteSizeOfTupleIndexTable(const Shape& shape, + int64 pointer_size); + + // Returns the number of bytes required for the elements in an allocation of + // `shape`, which must be an array shape. The return value does not include + // the bytes needed to store sparse indices. Dense shapes use a separate + // memory location for each element, and so for these shapes, + // `ByteSizeOf(shape) == ByteSizeOfElements(shape)`. For dense shapes, this + // size also includes padding if present in the layout. For sparse shapes, + // `ByteSizeOf(shape) == ByteSizeOfElements(shape) + + // ByteSizeOfSparseindices(shape)`. + static int64 ByteSizeOfElements(const Shape& shape); + + // Returns the number of bytes required for the sparse indices in an + // allocation of shape. The shape must be an array shape. The return value + // does not include the bytes needed to store sparse indices. + static int64 ByteSizeOfSparseIndices(const Shape& shape); + + // Returns a human-readable string that represents the given shape, with or + // without layout. e.g. "f32[42x12] {0, 1}" or "f32[64]". + static string HumanString(const Shape& shape); + static string HumanStringWithLayout(const Shape& shape); + + // As above, but for program shapes, returns a string for the form: + // + // (param_name: f32[42x12], ...) -> f32[24x42] + static string HumanString(const ProgramShape& program_shape); + + // Parses a ShapeUtil::HumanString-format shape string back into a shape + // object. + static StatusOr ParseShapeString(absl::string_view s); + + // Returns whether the LHS and RHS shapes have the same dimensions; note: does + // not check element type. + // Precondition: IsArray(lhs) && IsArray(rhs) + static bool SameDimensions(const Shape& lhs, const Shape& rhs); + + // Returns whether the lhs and rhs shapes have the same element type. + static bool SameElementType(const Shape& lhs, const Shape& rhs) { + return lhs.element_type() == rhs.element_type(); + } + + // As SameElementType, but allows floating point types to have different + // precisions. + static bool SameElementTypeIgnoringFpPrecision(const Shape& a, + const Shape& b) { + if (ElementIsFloating(a) && ElementIsFloating(b)) { + return true; + } + return ShapeUtil::SameElementType(a, b); + } + + // Returns the higher-precision element type if a and b are both floating + // point types; otherwise, checks that that they have the same element type + // and returns it. + static PrimitiveType HigherPrecisionElementType(const Shape& a, + const Shape& b) { + if (SameElementType(a, b)) { + return a.element_type(); + } + CHECK(SameElementTypeIgnoringFpPrecision(a, b)); + return primitive_util::BitWidth(a.element_type()) < + primitive_util::BitWidth(b.element_type()) + ? b.element_type() + : a.element_type(); + } + + // Returns true if the rank, dimension sizes, and element type are + // identical. Layout is ignored. Tuple elements are compared recursively for + // compatibility. + static bool Compatible(const Shape& lhs, const Shape& rhs); + + // Returns true if the rank and dimension sizes are identical. Element type + // and layout are ignored. Tuple elements are compared recursively for + // compatibility. + static bool CompatibleIgnoringElementType(const Shape& lhs, const Shape& rhs); + + // As Compatible, but allow one of lhs and rhs to be BF16 while the other + // being F32. Tuple elements are compared recursively for compatibility. + static bool CompatibleIgnoringFpPrecision(const Shape& lhs, const Shape& rhs); + + // Returns whether the lhs and rhs shapes are identical protobufs. + static bool Equal(const Shape& lhs, const Shape& rhs); + + // As Equal, but allow one of lhs and rhs to be F16 while the other is F32. + static bool EqualIgnoringFpPrecision(const Shape& lhs, const Shape& rhs); + + // Returns the rank (number of dimensions) of the given shape. + // Precondition: !IsTuple(shape) + static int64 Rank(const Shape& shape); + + // Returns the number of dimensions for which the dimension is not (trivially) + // 1. e.g., f32[2x1x1] has a true rank of 1D, the other dimensions are just + // fluff. Note that zero dimensions are included in the true rank, e.g., + // f32[3,0,1] has a true rank of 2D. + static int64 TrueRank(const Shape& shape); + + static ProgramShape MakeProgramShape(std::initializer_list parameters, + Shape result); + + //////////////////// + // Scalar-specific + + static bool IsScalar(const Shape& shape) { + return IsArray(shape) && Rank(shape) == 0; + } + static bool IsEffectiveScalar(const Shape& shape) { + return IsArray(shape) && TrueRank(shape) == 0; + } + static bool IsScalarF32(const Shape& shape); + + // Extracts the size of the shape's dimension at dimension number + // GetDimensionNumber(dimension_number). + static int64 GetDimension(const Shape& shape, int64 dimension_number); + + // Resolves a dimension number, supporting negative indexing. + // + // Negative indexing has similar semantics to Python. For an N-dimensional + // array, dimension -1 is equivalent to dimension N-1, -2 is equivalent to + // N-2, and so on. + // + // This function always returns a positive dimension number for any given + // dimension_number (which itself can be negative). + static int64 GetDimensionNumber(const Shape& shape, int64 dimension_number); + + // Returns a shape with the same dimensions as the original, but with the + // element type changed to type. + static Shape ChangeElementType(const Shape& original, PrimitiveType type); + + // Creates a tuple shape from a slice of element shapes within the tuple. + static Shape MakeTupleShape(absl::Span shapes); + + // Creates an opaque shape. These are generally used for threading a context + // into a custom operation. + static Shape MakeOpaqueShape(); + + // Creates a token shape. Values of this shape are used for ordering + // side-effecting operations. + static Shape MakeTokenShape(); + + // Appends a shape to the given tuple. + static void AppendShapeToTuple(const Shape& shape, Shape* tuple_shape); + + // Appends a major dimension to the shape with the given bound. + static void AppendMajorDimension(int bound, Shape* shape); + + // Returns an empty tuple shape. Can be used as a sentinel Shape value. + static Shape MakeNil() { return MakeTupleShape({}); } + + // Checks whether the shape is initialized. + static bool IsInitialized(const Shape& shape) { + return shape.element_type() != PRIMITIVE_TYPE_INVALID; + } + + // Constructs a new shape with the given element type and sequence of + // dimensions. + static Shape MakeShape(PrimitiveType element_type, + absl::Span dimensions); + + // Creates a Shape with element type corresponding to T and the given + // dimensions + template + static Shape MakeShapeWithType(absl::Span dimensions) { + return ShapeUtil::MakeShape(primitive_util::NativeToPrimitiveType(), + dimensions); + } + + // Constructs a new shape with the given minor_to_major order in its Layout. + // Returns a value shape such that shape.has_layout(). + static Shape MakeShapeWithLayout(PrimitiveType element_type, + absl::Span dimensions, + absl::Span minor_to_major); + + static Shape MakeShapeWithSparseLayout(PrimitiveType element_type, + absl::Span dimensions, + int64 max_sparse_elements); + + // Constructs a new shape with major-first layout (i.e. {n, n-1, ..., 0}). + static Shape MakeShapeWithDescendingLayout( + PrimitiveType element_type, absl::Span dimensions); + + // Returns a new Shape based on the given Shape with low-dimension-major + // layout (i.e. {n, n-1, ..., 0}, like Fortran), and with the dimensions + // rearranged so that it has the same in-memory layout as the given shape. + // + // For example, transforms f32[B,H,W,C]{0,3,2,1} to f32[H,W,C,B]{3,2,1,0}. + static Shape MakeShapeWithDescendingLayoutAndSamePhysicalLayout( + const Shape& shape); + + // As MakeShape, but the object to write to is passed in. + static void PopulateShape(PrimitiveType element_type, + absl::Span dimensions, Shape* shape); + + // Validates that the provided shape satisfies invariants. + static Status ValidateShape(const Shape& shape); + + // Validates the provided shape satisfies invariants, except those that + // pertain to layout. + // + // Layout is optional for client-provided shapes, so that the compiler may + // determine and assign an optimized layout. + static Status ValidateShapeWithOptionalLayout(const Shape& shape); + + // Returns whether the element type of the shape is integral (signed or + // unsigned). Note that predicates are not considered integral here, since + // they are logical values. + static bool ElementIsIntegral(const Shape& shape); + + // Returns whether the element type of the shape is floating point. + static bool ElementIsFloating(const Shape& shape); + + // Returns whether the element type of the shape is complex. + static bool ElementIsComplex(const Shape& shape); + + // Returns whether the element type has the given bit width. + static bool ElementHasBitWidth(const Shape& shape, int bits); + + // Returns whether the element type of the shape is integral and has + // the specified number of bits. + static bool ElementIsIntegralWithBits(const Shape& shape, int bits); + + // Returns whether the element type of the shape is signed. Note + // that floating point numbers are signed. + static bool ElementIsSigned(const Shape& shape); + + // Returns whether the shape is a tuple. + static bool IsTuple(const Shape& shape) { + return shape.element_type() == TUPLE; + } + + // Returns whether the shape is an opaque value (i.e. an 'existential' typed + // value that is passed to CustomCall operations). + static bool IsOpaque(const Shape& shape) { + return shape.element_type() == OPAQUE; + } + + // Returns whether the shape is an token value used for ordering + // side-effecting operations. + static bool IsToken(const Shape& shape) { + return shape.element_type() == TOKEN; + } + + // Returns whether the shape is an array. Note that scalars are considered + // arrays. + static bool IsArray(const Shape& shape); + + // Returns whether the shape is a tuple with at least one element which is + // also a tuple. + static bool IsNestedTuple(const Shape& shape); + + // Returns true if shape is an empty tuple. + static bool IsEmptyTuple(const Shape& shape); + + // Returns true if shape is the nil shape (an empty tuple). + static bool IsNil(const Shape& shape); + + // Returns the number of elements in the given tuple shape. + // Precondition: IsTuple(shape) + static int64 TupleElementCount(const Shape& shape); + + // Returns the tuple element shape at given index. + // Precondition: IsTuple(shape) && TupleElementCount(shape) > index + static const Shape& GetTupleElementShape(const Shape& shape, int64 index); + + // Returns the number of elements, recursively, in the given shape. + static int64 SubshapeCount(const Shape& shape); + + // Slices tuple elements in the range [start, limit) and returns a new tuple + // shape. E.g. a tuple like (f32, s32, u32) would slice via 1,3 to (s32, u32). + static Shape SliceTuple(const Shape& tuple, int64 start, int64 limit); + + // Returns the shape of the real/imaginary components of the given complex + // shape. + static Shape ComplexComponentShape(const Shape& complex_shape); + + // Shorthand for testing whether a shape is of a given element type and + // sequence of dimensions. + ABSL_DEPRECATED("Use Equal() instead.") + static bool ShapeIs(const Shape& shape, PrimitiveType element_type, + std::initializer_list dimensions); + + // Returns true if the given shape has a subshape at the given index. + static bool IndexIsValid(const Shape& shape, ShapeIndexView index); + + // GetSubshape and GetMutableSubshape return a particular nested Shape within + // the given Shape argument. The non-Try variants check fail if index is + // invalid. + static const Shape& GetSubshape(const Shape& shape, ShapeIndexView index); + static StatusOr TryGetSubshape(const Shape& shape, + ShapeIndexView index); + static Shape* GetMutableSubshape(Shape* shape, ShapeIndexView index); + + // Returns whether the given index in the given shape is a leaf element of the + // shape. + static bool IsLeafIndex(const Shape& shape, const ShapeIndex& index); + + // Returns the number of leaves in the shape. + static int64 GetLeafCount(const Shape& shape); + + // Retrieves all the leaf shapes and their indexes, in the order walked by + // the ForEachSubshape() API. + static std::vector GetLeafShapes(const Shape& shape); + + // Calls the given visitor function for each subshape of the given shape. + // Subshapes are visited in DFS pre-order starting with the entire shape + // (index {}). + using VisitorFunction = std::function; + static void ForEachSubshape(const Shape& shape, const VisitorFunction& func); + using MutatingVisitorFunction = + std::function; + static void ForEachMutableSubshape(Shape* shape, + const MutatingVisitorFunction& func); + + // Variants of ForEach(Mutable)Subshape which propagate Status from the + // visitor function. + using StatusVisitorFunction = std::function; + static Status ForEachSubshapeWithStatus(const Shape& shape, + const StatusVisitorFunction& func); + using MutatingStatusVisitorFunction = + std::function; + static Status ForEachMutableSubshapeWithStatus( + Shape* shape, const MutatingStatusVisitorFunction& func); + + // Returns true if `shape` (which must be an array) with degenerate dimensions + // (dimensions with bound 1). + static bool HasDegenerateDimensions(const Shape& shape); + + // Permutes the dimensions by the given permutation, so + // return_value.dimensions[permutation[i]] = argument.dimensions[i]. + // + // Postcondition: For any valid permutation, + // + // !HasLayout(shape) || + // TransposeIsBitcast(shape, PermuteDimensions(permutation, shape), + // InversePermutation(permutation)). + static Shape PermuteDimensions(absl::Span permutation, + const Shape& shape); + + // If we can go from `shape_pre` to `shape_post` by merely inserting or + // deleting 1-sized dimensions, return the indices in `shape_pre` of the + // deleted dimensions and the indices in `dims_post` of the inserted + // dimensions. + // For example, if `shape_pre = {a_1, a_2, ..., a_m}` and + // `shape_post = {b_1, b_2, ..., b_n}` where we can find some sequence of `i`s + // and some sequence of `j`s so `a_i = 1` for each `i` and `b_j = 1` for each + // `j` and `a_(k-s) = b_(k-t)` where `s` and `t` are the number of `i`s and + // `j`s less than `k` for all other `k`, we return the `i`s and `j`s. + // For another example, if `shape_pre = shape_post = {}`, we return `{}`. + static std::tuple, std::vector> + InsertedOrDeleted1SizedDimensions(const Shape& shape_pre, + const Shape& shape_post); + + // Suppose a reshape transforms input_shape to output shape. Returns a vector + // of pairs that indicate the input and output dimensions that this reshape + // doesn't logically (i.e. ignoring the layout) modify. For each pair (I,O) in + // the returned vector, the reshape transforms any input index whose I-th + // dimension is x to an output index whose O-th dimension is x too. + // + // Post-condition: the returned vector is sorted (by both input and output + // dimensions because input and output dimensions have the same order). + // + // Example: + // input shape = T[a, b, x, y, cd] + // output shape = T[ab, x, 1, y, c, d] + // return value = {{2, 1}, {3, 3}} + // + // The two pairs represent the input and output dimension of size x and + // those of size y. + static std::vector> DimensionsUnmodifiedByReshape( + const Shape& input_shape, const Shape& output_shape); + + // Returns whether a transpose from input_shape to output_shape with dimension + // mapping "dimension_mapping" produces a result which is bit-wise identical + // to its input and thus may be replaced with a bitcast. + // + // Precondition: Both input_shape and output_shape have explicit layouts. + static bool TransposeIsBitcast(const Shape& input_shape, + const Shape& output_shape, + absl::Span dimension_mapping); + + // Returns whether a reshape from "input_shape" to "output_shape" is a + // bitcast. + // + // Precondition: Both input_shape and output_shape have explicit layouts. + static bool ReshapeIsBitcast(const Shape& input_shape, + const Shape& output_shape); + + // Find a physical layout for 'output_shape' such that + // ShapeUtil::ReshapeIsBitcast(input_shape, output_shape_with_layout) returns + // true (where 'output_shape_with_layout' is 'output_shape' with the found + // layout). The layout of 'input_shape' is kept fixed. Returns + // 'output_shape_with_layout' if such a layout can be found, and an error + // otherwise. + static absl::optional AlignLayouts(const Shape& input_shape, + const Shape& output_shape); + + // Returns a shape with the given dimension deleted. + // For example: + // • `DeleteDimension(1, T[m, n, k]) = T[m, k]` + static Shape DeleteDimension(int64 dim_to_delete, Shape shape); + + // Returns a shape with all the dimensions of the input shape for which `p` + // returns true. + // For examples: + // • `FilterDimensions((< 2), T[m, n, k]) = T[m, n]` + // • `FilterDimensions(is_even_number, T[m, n, k]) = T[m, k]` + static Shape FilterDimensions(const std::function& p, + Shape shape); + + // Iterates through all the shape indexes, in minor to major order, starting + // from the base indexes, incrementing by the incr steps, up to count + // (index[i] < base[i] + count[i]), and calls the visitor_function with the + // current index. + // The visitor_function visitor function should return true if it wants to + // continue, or false otherwise. + // + // visitor_function must be a callable of type + // StatusOr(Span) or compatible. + template + static Status ForEachIndexWithStatus(const Shape& shape, + absl::Span base, + absl::Span count, + absl::Span incr, + const FnType& visitor_function) { + return ForEachIndexInternal(shape, base, count, incr, visitor_function); + } + + // Simple ergonomic wrapper around ShapeUtil::ForEachIndexWithStatus. + struct IndexIterationSpace { + std::vector index_base; + std::vector index_count; + std::vector index_incr; + }; + + template + static Status ForEachIndexWithStatus( + const Shape& shape, const IndexIterationSpace& iteration_space, + FnTy&& function) { + return ShapeUtil::ForEachIndexWithStatus( + shape, iteration_space.index_base, iteration_space.index_count, + iteration_space.index_incr, std::forward(function)); + } + + template + static void ForEachIndex(const Shape& shape, absl::Span base, + absl::Span count, + absl::Span incr, + const FnType& visitor_function) { + ForEachIndexWithStatus(shape, base, count, incr, + [&](absl::Span indices) { + return StatusOr(visitor_function(indices)); + }) + .IgnoreError(); + } + + // These convenience wrappers don't take `base`, `count` and `incr` + // explicitly, but iterate over every element in `shape` instead. + + template + static Status ForEachIndexWithStatus(const Shape& shape, + const FnType& visitor_function) { + std::vector base(shape.dimensions_size()); + std::vector incr(shape.dimensions_size(), 1); + return ForEachIndexWithStatus(shape, base, + /*count=*/AsInt64Slice(shape.dimensions()), + incr, visitor_function); + } + + template + static void ForEachIndex(const Shape& shape, const FnType& visitor_function) { + ForEachIndexWithStatus(shape, + [&](absl::Span indices) { + return StatusOr(visitor_function(indices)); + }) + .IgnoreError(); + } + + // A parallel version of ForEachIndex(WithStatus). This can only be used if + // the visitor_function is thread-safe and the order of iteration does not + // matter. + // + // visitor_function must be a callable of type + // void(Span) or compatible. + template + static void ForEachIndexParallel(const Shape& shape, + absl::Span base, + absl::Span count, + absl::Span incr, + const FnType& visitor_function) { + // The parallel version of ForEachIndexInternal can never fail. + CHECK(ForEachIndexInternal( + shape, base, count, incr, + [&visitor_function]( + absl::Span indexes) -> StatusOr { + visitor_function(indexes); + return true; + }, + /*parallel=*/true) + .ok()); + } + + // Compute a hash for `shape`. + static size_t Hash(const Shape& shape); + + private: + // Validates the shape size is sane. This makes sure it's safe to do + // calculations in int64 without overflowing. + static Status ValidateShapeSize(const Shape& shape); + + // Validates all of the non-layout properties of the shape -- this is a helper + // used by both the layout-optional and layout-required public method. + static Status ValidateShapeWithOptionalLayoutInternal(const Shape& shape); + + template + static Status ForEachIndexInternal(const Shape& shape, + absl::Span base, + absl::Span count, + absl::Span incr, + const FnType& visitor_function, + bool parallel = false) { + if (ShapeUtil::IsZeroElementArray(shape)) { + return Status::OK(); + } + CHECK_EQ(Rank(shape), base.size()); + CHECK_EQ(incr.size(), base.size()); + CHECK_EQ(count.size(), base.size()); + const int64 rank = LayoutUtil::MinorToMajor(shape).size(); + // Allows handling R0 arrays, such that the visitor function will be called + // once with the proper empty indexes. + int64 n = -1; + std::vector indexes(base.begin(), base.end()); + const int kNumThreads = tensorflow::port::NumSchedulableCPUs(); + absl::optional pool; + if (parallel) { + pool.emplace(tensorflow::Env::Default(), "foreach", kNumThreads); + } + + while (n < rank) { + if (pool != absl::nullopt) { + pool->Schedule( + [indexes, &visitor_function] { visitor_function(indexes); }); + } else { + TF_ASSIGN_OR_RETURN(bool should_continue, visitor_function(indexes)); + if (!should_continue) { + break; + } + } + // Increments dimensions in minor to major order. + for (n = 0; n < rank; ++n) { + int64 dim = LayoutUtil::Minor(shape.layout(), n); + indexes[dim] += incr[dim]; + if (indexes[dim] < base[dim] + count[dim]) { + break; + } + indexes[dim] = base[dim]; + } + } + + return Status::OK(); + } + + TF_DISALLOW_COPY_AND_ASSIGN(ShapeUtil); +}; + +std::ostream& operator<<(std::ostream& out, const Shape& shape); + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SHAPE_UTIL_H_ diff --git a/linux/include/tensorflow/compiler/xla/sparse_index_array.h b/linux/include/tensorflow/compiler/xla/sparse_index_array.h new file mode 100644 index 000000000..a96d48346 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/sparse_index_array.h @@ -0,0 +1,176 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// Utility class for managing sparse array indices. + +#ifndef TENSORFLOW_COMPILER_XLA_SPARSE_INDEX_ARRAY_H_ +#define TENSORFLOW_COMPILER_XLA_SPARSE_INDEX_ARRAY_H_ + +#include + +#include "absl/container/inlined_vector.h" +#include "absl/types/span.h" +#include "tensorflow/compiler/xla/array2d.h" +#include "tensorflow/compiler/xla/index_util.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" + +namespace xla { + +// Encapsulates the array of indices for a sparse array. A SparseIndexArray +// contain indices for up to `max_indices` elements of a sparse array. Each +// sparse index is an array of `rank` int64 value that gives the location of a +// value within a sparse array. Note that the dimensions of the array are not +// checked (except for the rank). To avoid confusion, we refer to the position +// of an index within a SparseIndexArray as a sparse index number. +class SparseIndexArray { + public: + SparseIndexArray(); + SparseIndexArray(const SparseIndexArray&) = default; + SparseIndexArray(SparseIndexArray&&) = default; + SparseIndexArray& operator=(const SparseIndexArray&) = default; + SparseIndexArray& operator=(SparseIndexArray&&) = default; + + // Constructs a SparseIndexArray that can hold up to `max_indices` sparse + // indices, with an initial contents obtained from the given array. The rank + // is taken from the minor dimension of the array. The major dimension of the + // array must not exceed `max_indices`. + SparseIndexArray(int64 max_indices, const Array2D& indices); + + // Like above, but the array is flattened. For example, the following are + // equivalent: + // + // SparseIndexArray(10, 3, + // Array2D{ + // {0, 1, 2}, + // {3, 4, 5}, + // {6, 7, 8}, + // {9, 10, 11}, + // }) + // + // SparseIndexArray(10, 3, + // {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11}) + // + SparseIndexArray(int64 max_indices, int64 rank, + std::vector indices = {}); + SparseIndexArray(int64 max_indices, int64 rank, + absl::Span indices); + + // Returns the number of elements represented by the indices stored in the + // array. + int64 index_count() const; + + // Returns a slice that refers to the given sparse index number. The argument + // must be in the range [0, element_count()). + absl::Span At(int64 sparse_element_number) const; + absl::Span At(int64 sparse_element_number); + + // Adds the given index at the end of the array. The new size of the + // SparseIndexArray must not exceed `max_indices`. + void Append(absl::Span index); + + // Removes all indices from the array. + void Clear(); + + // Resizes the array to contain the given number of sparse indices. The new + // size must be smaller than `max_indices`. If the new size is larger than + // the old size, the value of the new indices is not specified. + void Resize(int64 num_indices); + + // Returns true iff all indices are unique and occur in sorted order, and are + // valid for the given shape. + bool Validate(const Shape& shape) const; + + int64 rank() const { return rank_; } + int64 max_indices() const { return max_indices_; } + + // Returns a pointer to the int64 array that holds the sparse indices. + absl::Span mutable_data() { return absl::MakeSpan(indices_); } + absl::Span data() const { return indices_; } + + // Sorts this sparse index array along with the set of corresponding values. + // The indices and values are sorted in the lexicographic order of the + // indices, from smallest to largest. + // + // For example: + // + // std::vector v{10.0, 11.0, 12.0}; + // SparseIndexArray a(10, 3, + // {{3, 4, 5}, + // {1, 2, 3}, + // {2, 3, 4}}); + // a.SortWithValues(&v); + // // Prints "11.0, 12.0, 10.0": + // std::cout << v[0] << ", " << v[1] << ", " << v[2] << std::endl; + // + template + void SortWithValues(absl::Span values); + + private: + std::vector indices_; + int64 rank_; + int64 max_indices_; +}; + +template +void SparseIndexArray::SortWithValues(absl::Span values) { + int64 num_elements = index_count(); + CHECK_EQ(values.size(), num_elements); + std::vector sort_order; + sort_order.reserve(num_elements); + for (int64 i = 0; i < num_elements; ++i) { + sort_order.push_back(i); + } + auto sort_order_less = [this](int64 lhs, int64 rhs) { + return IndexUtil::CompareIndices(At(lhs), At(rhs)) < 0; + }; + std::sort(sort_order.begin(), sort_order.end(), sort_order_less); + + // Reorder the array elements according to sort_order. Work through the array + // and follow cycles so we can do the reorder in-place. + absl::InlinedVector saved_index(rank()); + for (int64 i = 0; i < num_elements; ++i) { + // sort_order[i] == -1 indicates the element has already been copied. + if (sort_order[i] < 0) { + continue; + } else if (i == sort_order[i]) { + // The element is already in sorted order. + sort_order[i] = -1; + continue; + } + + std::copy_n(At(i).begin(), rank(), saved_index.begin()); + NativeT saved_value = values[i]; + int64 j = i; + for (;;) { + if (sort_order[j] == i) { + std::copy_n(saved_index.begin(), rank(), At(j).begin()); + values[j] = saved_value; + sort_order[j] = -1; + break; + } + + std::copy_n(At(sort_order[j]).begin(), rank(), At(j).begin()); + values[j] = values[sort_order[j]]; + + int64 k = sort_order[j]; + sort_order[j] = -1; + j = k; + } + } +} + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_SPARSE_INDEX_ARRAY_H_ diff --git a/linux/include/tensorflow/compiler/xla/status.h b/linux/include/tensorflow/compiler/xla/status.h new file mode 100644 index 000000000..69abb5185 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/status.h @@ -0,0 +1,28 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_STATUS_H_ +#define TENSORFLOW_COMPILER_XLA_STATUS_H_ + +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/platform/macros.h" + +namespace xla { + +using tensorflow::Status; // TENSORFLOW_STATUS_OK + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_STATUS_H_ diff --git a/linux/include/tensorflow/compiler/xla/status_macros.h b/linux/include/tensorflow/compiler/xla/status_macros.h new file mode 100644 index 000000000..e51dd64e2 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/status_macros.h @@ -0,0 +1,210 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_STATUS_MACROS_H_ +#define TENSORFLOW_COMPILER_XLA_STATUS_MACROS_H_ + +#include +#include // NOLINT +#include +#include + +#include "tensorflow/compiler/xla/statusor.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/platform/logging.h" +#include "tensorflow/core/platform/macros.h" + +namespace xla { +namespace status_macros { + +// Stream object used to collect error messages in MAKE_ERROR macros +// or append error messages with APPEND_ERROR. It accepts any +// arguments with operator<< to build an error string, and then has an +// implicit cast operator to Status, which converts the +// logged string to a Status object and returns it, after logging the +// error. At least one call to operator<< is required; a compile time +// error will be generated if none are given. Errors will only be +// logged by default for certain status codes, as defined in +// IsLoggedByDefault. This class will give ERROR errors if you don't +// retrieve a Status exactly once before destruction. +// +// The class converts into an intermediate wrapper object +// MakeErrorStreamWithOutput to check that the error stream gets at least one +// item of input. +class MakeErrorStream { + public: + // Wrapper around MakeErrorStream that only allows for output. This + // is created as output of the first operator<< call on + // MakeErrorStream. The bare MakeErrorStream does not have a + // Status operator. The net effect of that is that you + // have to call operator<< at least once or else you'll get a + // compile time error. + class MakeErrorStreamWithOutput { + public: + explicit MakeErrorStreamWithOutput(MakeErrorStream* error_stream) + : wrapped_error_stream_(error_stream) {} + + template + MakeErrorStreamWithOutput& operator<<(const T& value) { + *wrapped_error_stream_ << value; + return *this; + } + + // Implicit cast operators to Status and StatusOr. + // Exactly one of these must be called exactly once before destruction. + operator Status() { return wrapped_error_stream_->GetStatus(); } + template + operator xla::StatusOr() { + return wrapped_error_stream_->GetStatus(); + } + + private: + MakeErrorStream* wrapped_error_stream_; + + TF_DISALLOW_COPY_AND_ASSIGN(MakeErrorStreamWithOutput); + }; + + // When starting from an existing error status, this determines whether we'll + // append or prepend to that status's error message. + enum PriorMessageHandling { kAppendToPriorMessage, kPrependToPriorMessage }; + + // Make an error with the given code. + template + MakeErrorStream(const char* file, int line, ERROR_CODE_TYPE code) + : impl_(new Impl(file, line, code, this, true)) {} + + template + MakeErrorStreamWithOutput& operator<<(const T& value) { + CheckNotDone(); + impl_->stream_ << value; + return impl_->make_error_stream_with_output_wrapper_; + } + + // When this message is logged (see with_logging()), include the stack trace. + MakeErrorStream& with_log_stack_trace() { + impl_->should_log_stack_trace_ = true; + return *this; + } + + // Adds RET_CHECK failure text to error message. + MakeErrorStreamWithOutput& add_ret_check_failure(const char* condition) { + return *this << "RET_CHECK failure (" << impl_->file_ << ":" << impl_->line_ + << ") " << condition << " "; + } + + private: + class Impl { + public: + Impl(const char* file, int line, tensorflow::error::Code code, + MakeErrorStream* error_stream, bool is_logged_by_default = true); + Impl(const Status& status, PriorMessageHandling prior_message_handling, + const char* file, int line, MakeErrorStream* error_stream); + + ~Impl(); + + // This must be called exactly once before destruction. + Status GetStatus(); + + void CheckNotDone() const; + + private: + const char* file_; + int line_; + tensorflow::error::Code code_; + + PriorMessageHandling prior_message_handling_ = kAppendToPriorMessage; + string prior_message_; + bool is_done_; // true after Status object has been returned + std::ostringstream stream_; + bool should_log_; + int log_severity_; + bool should_log_stack_trace_; + + // Wrapper around the MakeErrorStream object that has a + // Status conversion. The first << operator called on + // MakeErrorStream will return this object, and only this object + // can implicitly convert to Status. The net effect of + // this is that you'll get a compile time error if you call + // MAKE_ERROR etc. without adding any output. + MakeErrorStreamWithOutput make_error_stream_with_output_wrapper_; + + friend class MakeErrorStream; + TF_DISALLOW_COPY_AND_ASSIGN(Impl); + }; + + void CheckNotDone() const; + + // Returns the status. Used by MakeErrorStreamWithOutput. + Status GetStatus() const { return impl_->GetStatus(); } + + // Store the actual data on the heap to reduce stack frame sizes. + std::unique_ptr impl_; + + TF_DISALLOW_COPY_AND_ASSIGN(MakeErrorStream); +}; + +// Provides a conversion to bool so that it can be used inside an if statement +// that declares a variable. +class StatusAdaptorForMacros { + public: + explicit StatusAdaptorForMacros(Status status) : status_(std::move(status)) {} + + StatusAdaptorForMacros(const StatusAdaptorForMacros&) = delete; + StatusAdaptorForMacros& operator=(const StatusAdaptorForMacros&) = delete; + + explicit operator bool() const { return TF_PREDICT_TRUE(status_.ok()); } + + Status&& Consume() { return std::move(status_); } + + private: + Status status_; +}; + +} // namespace status_macros +} // namespace xla + +#define TF_RET_CHECK(condition) \ + while (TF_PREDICT_FALSE(!(condition))) \ + return xla::status_macros::MakeErrorStream(__FILE__, __LINE__, \ + tensorflow::error::INTERNAL) \ + .with_log_stack_trace() \ + .add_ret_check_failure(#condition) + +#define TF_ASSERT_OK_AND_ASSIGN(lhs, rexpr) \ + TF_ASSERT_OK_AND_ASSIGN_IMPL( \ + TF_STATUS_MACROS_CONCAT_NAME(_status_or_value, __COUNTER__), lhs, \ + rexpr); + +#define TF_ASSERT_OK_AND_ASSIGN_IMPL(statusor, lhs, rexpr) \ + auto statusor = (rexpr); \ + ASSERT_TRUE(statusor.status().ok()) << statusor.status(); \ + lhs = std::move(statusor.ValueOrDie()) + +#define TF_STATUS_MACROS_CONCAT_NAME(x, y) TF_STATUS_MACROS_CONCAT_IMPL(x, y) +#define TF_STATUS_MACROS_CONCAT_IMPL(x, y) x##y + +#define TF_ASSIGN_OR_RETURN(lhs, rexpr) \ + TF_ASSIGN_OR_RETURN_IMPL( \ + TF_STATUS_MACROS_CONCAT_NAME(_status_or_value, __COUNTER__), lhs, rexpr) + +#define TF_ASSIGN_OR_RETURN_IMPL(statusor, lhs, rexpr) \ + auto statusor = (rexpr); \ + if (TF_PREDICT_FALSE(!statusor.ok())) { \ + return statusor.status(); \ + } \ + lhs = std::move(statusor.ValueOrDie()) + +#endif // TENSORFLOW_COMPILER_XLA_STATUS_MACROS_H_ diff --git a/linux/include/tensorflow/compiler/xla/statusor.h b/linux/include/tensorflow/compiler/xla/statusor.h new file mode 100644 index 000000000..a32e2ad98 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/statusor.h @@ -0,0 +1,29 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_COMPILER_XLA_STATUSOR_H_ +#define TENSORFLOW_COMPILER_XLA_STATUSOR_H_ + +#include "tensorflow/compiler/xla/status.h" +#include "tensorflow/stream_executor/lib/statusor.h" + +namespace xla { + +// Use steam_executor's StatusOr so we don't duplicate code. +template +using StatusOr = ::stream_executor::port::StatusOr; + +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_STATUSOR_H_ diff --git a/linux/include/tensorflow/compiler/xla/types.h b/linux/include/tensorflow/compiler/xla/types.h new file mode 100644 index 000000000..b645acb70 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/types.h @@ -0,0 +1,55 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_TYPES_H_ +#define TENSORFLOW_COMPILER_XLA_TYPES_H_ + +#include + +#include "third_party/eigen3/Eigen/Core" +#include "tensorflow/core/framework/numeric_types.h" +#include "tensorflow/core/platform/types.h" + +#include + +namespace xla { + +using ::tensorflow::string; + +using ::tensorflow::int8; +using ::tensorflow::int16; +using ::tensorflow::int32; +using ::tensorflow::int64; + +using ::tensorflow::bfloat16; + +using ::tensorflow::uint8; +using ::tensorflow::uint16; +using ::tensorflow::uint32; +using ::tensorflow::uint64; + +using complex64 = std::complex; + +using ::Eigen::half; + +} // namespace xla + +// Alias namespace ::stream_executor as ::xla::se. +namespace stream_executor {} +namespace xla { +namespace se = ::stream_executor; +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_TYPES_H_ diff --git a/linux/include/tensorflow/compiler/xla/util.h b/linux/include/tensorflow/compiler/xla/util.h new file mode 100644 index 000000000..8ce741647 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/util.h @@ -0,0 +1,518 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// Generally useful utility functions that are common to (not specific to any +// given part of) the XLA code base. + +#ifndef TENSORFLOW_COMPILER_XLA_UTIL_H_ +#define TENSORFLOW_COMPILER_XLA_UTIL_H_ + +#include +#include +#include +#include + +#include "absl/algorithm/container.h" +#include "absl/container/inlined_vector.h" +#include "absl/strings/str_cat.h" +#include "absl/strings/str_format.h" +#include "absl/strings/string_view.h" +#include "absl/types/span.h" +#include "tensorflow/compiler/xla/status.h" +#include "tensorflow/compiler/xla/status_macros.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/core/lib/core/errors.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/lib/math/math_util.h" +#include "tensorflow/core/lib/strings/numbers.h" +#include "tensorflow/core/platform/logging.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/protobuf.h" +#include "tensorflow/core/platform/types.h" + +namespace xla { + +// Logs the provided status message with a backtrace. +// +// For use by Status-factories, logs a backtrace at the point where the status +// is created, such that we can use --vmodule=util=1 to see all status +// creation backtraces. +Status WithLogBacktrace(const Status& status); + +// Ranks greater than 8 are very rare, so use InlinedVector to store +// the bounds and indices. And for the rare cases of ranks greater than 8, +// the InlinedVector will just behave like an std::vector<> and allocate the +// memory to store its values. +static constexpr int kInlineRank = 8; +using DimensionVector = absl::InlinedVector; + +// RAII timer that logs with a given label the wall clock time duration in human +// readable form. This differs from base's ElapsedTimer primarily in that it +// spits out the human-readable duration form. +// +// By default, the timing traces are only printed at VLOG(1) and above: +// +// XLA_SCOPED_LOGGING_TIMER("fooing bar"); // nop if !VLOG_IS_ON(1). +// +// but you can control this via: +// +// XLA_SCOPED_LOGGING_TIMER_LEVEL("fooing bar", 2); // nop if !VLOG_IS_ON(2) +// +#define XLA_SCOPED_LOGGING_TIMER(label) \ + XLA_SCOPED_LOGGING_TIMER_HELPER(label, 1, __COUNTER__) +#define XLA_SCOPED_LOGGING_TIMER_LEVEL(label, level) \ + XLA_SCOPED_LOGGING_TIMER_HELPER(label, level, __COUNTER__) + +// Helper for implementing macros above. Do not use directly. +// +// Forces the evaluation of "counter", which we expect is equal to __COUNTER__. +#define XLA_SCOPED_LOGGING_TIMER_HELPER(label, level, counter) \ + XLA_SCOPED_LOGGING_TIMER_HELPER2(label, level, counter) + +// Helper for macros above. Don't use directly. +#define XLA_SCOPED_LOGGING_TIMER_HELPER2(label, level, counter) \ + ::xla::ScopedLoggingTimer XLA_ScopedLoggingTimerInstance##counter( \ + label, VLOG_IS_ON(level)) + +// RAII timer for XLA_SCOPED_LOGGING_TIMER and XLA_SCOPED_LOGGING_TIMER_LEVEL +// macros above. Recommended usage is via the macros so you don't have to give +// the timer a name or worry about calling VLOG_IS_ON yourself. +struct ScopedLoggingTimer { + // The timer does nothing if enabled is false. This lets you pass in your + // file's VLOG_IS_ON value. + ScopedLoggingTimer(const string& label, bool enabled); + ~ScopedLoggingTimer(); + + bool enabled; + string label; + uint64 start_micros; +}; + +// Given a vector, returns a Span that points at its +// internals. +// +// Warning: if the vector is updated its storage pointer may change, so use this +// with caution (ideally in limited scopes with temporary lifetimes). +template +absl::Span MutableByteSlice(std::vector* v) { + return absl::Span(reinterpret_cast(v->data()), + v->size() * sizeof(T)); +} + +// Turns an immutable slice of type T into an immutable slice of bytes with the +// same byte size. +template +absl::Span CastToByteSlice(absl::Span slice) { + return absl::Span(reinterpret_cast(slice.data()), + slice.size() * sizeof(T)); +} + +// Casts a byte slice to a non-byte type T, checking that the original slice +// length is a multiple of sizeof(T). +template +absl::Span CastByteSlice(absl::Span slice) { + CHECK_EQ(0, slice.size() % sizeof(T)); + return absl::Span(reinterpret_cast(slice.data()), + slice.size() / sizeof(T)); +} + +// Convenience function to force a vector to convert to an immutable slice. +template +absl::Span AsSlice(const std::vector& v) { + return absl::Span(v); +} + +// Converts a mutable vector pointer into a Span of the same +// type. +template +absl::Span AsMutableSlice(std::vector* v) { + return absl::Span(v->data(), v->size()); +} + +// xla::int64 is not the same type as tensorflow::protobuf_int64 in open-source. +// Wrapper function that gives an int64 array slice view of a repeated int64 +// protobuf field. +static inline absl::Span AsInt64Slice( + const tensorflow::protobuf::RepeatedField& v) { + absl::Span slice(v); + return absl::Span(reinterpret_cast(slice.data()), + slice.size()); +} + +// As above, but for uint64 types. +static inline absl::Span AsUInt64Slice( + const tensorflow::protobuf::RepeatedField& v) { + absl::Span slice(v); + return absl::Span(reinterpret_cast(slice.data()), + slice.size()); +} + +// Compares two containers for equality. Returns true iff the two containers +// have the same size and all their elements compare equal using their +// operator==. Like std::equal, but forces size equality. +template +bool ContainersEqual(const Container1T& c1, const Container2T& c2) { + return ((c1.size() == c2.size()) && + std::equal(std::begin(c1), std::end(c1), std::begin(c2))); +} + +template +bool ContainersEqual(const Container1T& c1, + std::initializer_list il) { + absl::Span c2{il}; + return ContainersEqual(c1, c2); +} + +// Compares two containers for equality. Returns true iff the two containers +// have the same size and all their elements compare equal using the predicate +// p. Like std::equal, but forces size equality. +template +bool ContainersEqual(const Container1T& c1, const Container2T& c2, + PredicateT p) { + return ((c1.size() == c2.size()) && + std::equal(std::begin(c1), std::end(c1), std::begin(c2), p)); +} + +// Performs a copy of count values from src to dest, using different strides for +// source and destination. The source starting index is src_base, while the +// destination one is dest_base. +template +void StridedCopy(absl::Span dest, int64 dest_base, int64 dest_stride, + absl::Span src, int64 src_base, int64 src_stride, + int64 count) { + for (; count > 0; --count, dest_base += dest_stride, src_base += src_stride) { + dest[dest_base] = static_cast(src[src_base]); + } +} + +// Adds some context information to the error message in a +// Status. This is useful as Statuses are +// propagated upwards. +Status AddStatus(Status prior, absl::string_view context); +Status AppendStatus(Status prior, absl::string_view context); + +// Status error shorthands -- StrFormat's the arguments to be used as an error +// message and returns a status in the canonical error space. +template +Status InvalidArgument(const absl::FormatSpec& format, + const Args&... args) { + return WithLogBacktrace( + tensorflow::errors::InvalidArgument(absl::StrFormat(format, args...))); +} +template +Status Unimplemented(const absl::FormatSpec& format, + const Args&... args) { + return WithLogBacktrace( + tensorflow::errors::Unimplemented(absl::StrFormat(format, args...))); +} +template +Status InternalError(const absl::FormatSpec& format, + const Args&... args) { + return WithLogBacktrace( + tensorflow::errors::Internal(absl::StrFormat(format, args...))); +} +template +Status FailedPrecondition(const absl::FormatSpec& format, + const Args&... args) { + return WithLogBacktrace( + tensorflow::errors::FailedPrecondition(absl::StrFormat(format, args...))); +} +template +Status Cancelled(const absl::FormatSpec& format, const Args&... args) { + return WithLogBacktrace( + tensorflow::errors::Cancelled(absl::StrFormat(format, args...))); +} +template +Status ResourceExhausted(const absl::FormatSpec& format, + const Args&... args) { + return WithLogBacktrace( + tensorflow::errors::ResourceExhausted(absl::StrFormat(format, args...))); +} +template +Status NotFound(const absl::FormatSpec& format, const Args&... args) { + return WithLogBacktrace( + tensorflow::errors::NotFound(absl::StrFormat(format, args...))); +} +template +Status Unavailable(const absl::FormatSpec& format, + const Args&... args) { + return WithLogBacktrace( + tensorflow::errors::Unavailable(absl::StrFormat(format, args...))); +} + +template +Status InvalidArgumentStrCat(Args&&... concat) { + return InvalidArgument("%s", absl::StrCat(std::forward(concat)...)); +} + +template +Status UnimplementedStrCat(Args&&... concat) { + return Unimplemented("%s", absl::StrCat(std::forward(concat)...)); +} + +template +Status InternalErrorStrCat(Args&&... concat) { + return InternalError("%s", absl::StrCat(std::forward(concat)...)); +} + +template +Status ResourceExhaustedStrCat(Args&&... concat) { + return ResourceExhausted("%s", absl::StrCat(std::forward(concat)...)); +} + +// Splits the lines of the original, replaces leading whitespace with the prefix +// given by "indentation", and returns the string joined by newlines again. As a +// side effect, any additional trailing whitespace is removed. +// +// Note: even different amounts of leading whitespace on different lines will be +// uniformly replaced with "indentation". +string Reindent(absl::string_view original, absl::string_view indentation); + +// Checks whether permutation is a permutation of the [0, rank) integer range. +bool IsPermutation(absl::Span permutation, int64 rank); + +// Applies `permutation` on `input` and returns the permuted array. +// For each i, output[permutation[i]] = input[i]. +// +// Precondition: +// 1. `permutation` is a permutation of 0..permutation.size()-1. +// 2. permutation.size() == input.size(). +template +std::vector Permute( + absl::Span permutation, const Container& input) { + using T = typename Container::value_type; + absl::Span data(input); + CHECK(IsPermutation(permutation, data.size())); + std::vector output(data.size()); + for (size_t i = 0; i < permutation.size(); ++i) { + output[permutation[i]] = data[i]; + } + return output; +} + +// Inverts a permutation, i.e., output_permutation[input_permutation[i]] = i. +std::vector InversePermutation( + absl::Span input_permutation); + +// Composes two permutations: output[i] = p1[p2[i]]. +std::vector ComposePermutations(absl::Span p1, + absl::Span p2); + +// Returns true iff permutation == {0, 1, 2, ...}. +bool IsIdentityPermutation(absl::Span permutation); + +template +int64 PositionInContainer(const Container& container, int64 value) { + return std::distance(container.begin(), + std::find(container.begin(), container.end(), value)); +} + +// Formats the container as a comma-separated string. StrAppend must support +// appending the elements of the container. Prefix is prepended and suffix is +// appended to the returned string. +template +string CommaSeparatedString(const Container& c, const char* prefix = "", + const char* suffix = "") { + // Not using Join() since the implementation here is simple anyway and this + // avoids copying the string to append prefix. + string comma_separated = prefix; + const char* separator = ""; + for (const auto& entry : c) { + absl::StrAppend(&comma_separated, separator, entry); + separator = ", "; + } + comma_separated += suffix; + return comma_separated; +} + +// Overload needed to allow the container to be an initializer list. The default +// type for T makes an empty initializer list work as well. +template +string CommaSeparatedString(const std::initializer_list& c, + const char* prefix = "", const char* suffix = "") { + return CommaSeparatedString>(c, prefix, suffix); +} + +// Formats the container in the mathematical notation for a vector, e.g. (1, 3, +// 7). StrAppend must support appending the elements of c. +template +string VectorString(const Container& c) { + return CommaSeparatedString(c, "(", ")"); +} + +// Overload needed to allow the container to be an initializer list. The default +// type for T makes an empty initializer list work as well. +template +string VectorString(const std::initializer_list& c) { + return VectorString>(c); +} + +// Returns a PaddingConfig object that represents no padding for the given rank. +PaddingConfig MakeNoPaddingConfig(int64 rank); + +// Returns a PaddingConfig object where 'padding' contains +// (low edge padding, high edge padding) pairs for each dimension. +PaddingConfig MakeEdgePaddingConfig( + absl::Span> padding); + +// Returns true if the padding configuration has at least one dimension with +// non-zero interior padding. +bool HasInteriorPadding(const PaddingConfig& config); + +// Imports the templated FloorOfRatio math function from the TensorFlow +// namespace, as it is very commonly used. +template +T FloorOfRatio(T dividend, T divisor) { + return tensorflow::MathUtil::FloorOfRatio(dividend, divisor); +} + +// Imports the templated CeilOfRatio math function from the TensorFlow +// namespace, as it is very commonly used. +template +T CeilOfRatio(T dividend, T divisor) { + return tensorflow::MathUtil::CeilOfRatio(dividend, divisor); +} + +// Rounds the value up to a multiple of the divisor by first calling CeilOfRatio +// then multiplying by the divisor. For example: RoundUpToNearest(13, 8) => 16 +template +T RoundUpToNearest(T value, T divisor) { + return CeilOfRatio(value, divisor) * divisor; +} + +// Rounds the value down to a multiple of the divisor by first calling +// FloorOfRatio then multiplying by the divisor. For example: +// RoundDownToNearest(13, 8) => 8 +template +T RoundDownToNearest(T value, T divisor) { + return FloorOfRatio(value, divisor) * divisor; +} + +// Given a number of flops executed in an amount of time, produces a string that +// represents the throughput; +// e.g. HumanReadableNumFlops(1e9, 1e9) => 1.00GFLOP/s. +string HumanReadableNumFlops(double flops, double nanoseconds); + +// Given a number of transcendental ops executed in an amount of time, produces +// a string that represents the throughput; +// e.g. HumanReadableNumTranscendentalOps(1e9, 1e9) => 1.00GTROP/s. +string HumanReadableNumTranscendentalOps(double trops, double nanoseconds); + +// Split the text into multiple lines and log each line with the given +// severity, filename, and line number. +void LogLines(int sev, absl::string_view text, const char* fname, int lineno); + +template +inline bool IsPowerOfTwo(T x) { + static_assert(!std::numeric_limits::is_signed, "unsigned types only"); + return x != 0 && (x & (x - 1)) == 0; +} + +// Returns a mask with "bits" number of least significant bits set. +inline uint32 LsbMaskU32(int bits) { + CHECK_GE(bits, 0); + return (1U << bits) - 1; +} + +// Utility for performing a static_cast<> on a std::unique_ptr<>. +template +std::unique_ptr unique_ptr_static_cast(std::unique_ptr ptr) { + return std::unique_ptr(static_cast(ptr.release())); +} + +int64 Product(absl::Span xs); + +// Returns the start indices of consecutive non-overlapping subsequences of `a` +// and `b` with the same product, i.e. `(i, j)` so +// • a = {a[0 = i_0], ..., a[i_1 - 1], a[i_1], ... , a[i_2 - 1], ...} +// • b = {b[0 = j_0], ..., b[j_1 - 1], b[j_1], ... , b[j_2 - 1], ...} +// • ∀ k . 0 <= k < CommonFactors(a, b).size - 1 => +// a[i_k] × a[i_k + 1] × ... × a[i_(k+1) - 1] = +// b[j_k] × b[j_k + 1] × ... × b[j_(k+1) - 1] +// where `CommonFactors(a, b)[CommonFactors(a, b).size - 1] = (a.size, b.size)` +// +// If the given shapes have non-zero size, returns the bounds of the shortest +// possible such subsequences; else, returns `{(0, 0), (a.size, b.size)}`. +std::vector> CommonFactors(absl::Span a, + absl::Span b); + +// Removes illegal characters from filenames. +string SanitizeFileName(string file_name); + +template +int64 FindIndex(const C& c, Value&& value) { + auto it = absl::c_find(c, std::forward(value)); + return std::distance(c.begin(), it); +} + +template +void InsertAt(C* c, int64 index, Value&& value) { + c->insert(c->begin() + index, std::forward(value)); +} + +template +void EraseAt(C* c, int64 index) { + c->erase(c->begin() + index); +} + +template +std::vector ArraySliceToVector(absl::Span slice) { + return std::vector(slice.begin(), slice.end()); +} + +template +std::vector InlinedVectorToVector( + const absl::InlinedVector& inlined_vector) { + return std::vector(inlined_vector.begin(), inlined_vector.end()); +} + +// Returns true if `x` fits in 32-bits. +template +bool IsInt32(T x) { + // Following conversion rules: "the value is unchanged if it can be + // represented in the destination type (and bit-field width); otherwise, the + // value is implementation-defined." + return static_cast(x) == x; +} + +template +Status EraseElementFromVector(std::vector* container, const T& value) { + // absl::c_find returns a const_iterator which does not seem to work on + // gcc 4.8.4, and this breaks the ubuntu/xla_gpu build bot. + auto it = std::find(container->begin(), container->end(), value); + TF_RET_CHECK(it != container->end()); + container->erase(it); + return Status::OK(); +} +} // namespace xla + +#define XLA_LOG_LINES(SEV, STRING) \ + ::xla::LogLines(SEV, STRING, __FILE__, __LINE__) + +#define XLA_VLOG_LINES(LEVEL, STRING) \ + do { \ + if (VLOG_IS_ON(LEVEL)) XLA_LOG_LINES(::tensorflow::INFO, STRING); \ + } while (false); + +// Utility macro that performs the equivalent of what one would expect +// LOG_LINES(FATAL, X) to do but can be used at the end of a function that +// returns a value without getting a compiler warning that no value is returned. +#define XLA_FATAL_LOG(X) \ + XLA_LOG_LINES(::tensorflow::ERROR, X); \ + LOG(FATAL) << "Aborting in " << __FUNCTION__ << " due to previous errors."; + +#endif // TENSORFLOW_COMPILER_XLA_UTIL_H_ diff --git a/linux/include/tensorflow/compiler/xla/window_util.h b/linux/include/tensorflow/compiler/xla/window_util.h new file mode 100644 index 000000000..1fb9e855f --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/window_util.h @@ -0,0 +1,88 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMPILER_XLA_WINDOW_UTIL_H_ +#define TENSORFLOW_COMPILER_XLA_WINDOW_UTIL_H_ + +#include "absl/types/span.h" +#include "tensorflow/compiler/xla/types.h" +#include "tensorflow/compiler/xla/xla_data.pb.h" + +namespace xla { +namespace window_util { + +// Creates a window with the given sizes in the dimensions and all strides set +// to 1. +Window MakeWindow(absl::Span sizes); + +// Creates a padding config with symmetrical padding in each dimension, of value +// given by sizes; e.g. {0, 1, 2} would create a R3 padding config that had zero +// pixels of padding in dimension 0, one pixel of padding symmetrically, on each +// side of dimension 1, and two pixels of padding symmetrically on dimension 2. +PaddingConfig MakeSymmetricPadding(absl::Span sizes); + +string ToString(const WindowDimension& dim); +string ToString(const Window& window); + +// The below functions return true if the given field is set to have a +// non-trivial effect, e.g. having a stride means that the stride of some +// dimension is not one. Whether the proto field is populated is not a +// consideration. + +bool HasStride(const Window& window); +bool HasPadding(const Window& window); +bool HasSymmetricPadding(const Window& window); +bool HasNegativePadding(const Window& window); + +// As with HasSymmetricPadding(Window) above, returns whether the "padding low" +// is equivalent to the "padding high" for all dimensions, but works on a +// padding configuration. +bool HasSymmetricPadding(const PaddingConfig& padding_config); + +bool HasBaseDilation(const Window& window); +bool HasWindowDilation(const Window& window); +bool HasDilation(const Window& window); + +bool HasWindowReversal(const Window& window); + +// Returns true if the given logical dimension is inactive in the sense that it +// has window bound 1, no striding and no padding. +bool IsInactiveWindowDimension(const Window& window, int64 logical_dim); + +// Returns the new bound after dilation. +// +// If a window with the given bound in some dimension is dilated with the given +// dilation factor in that dimension, then the value returned is the bound for +// the array in that dimension after dilation. +// +// For a 1D array with 3 entries 1, 2, 3, a dilation factor of 2 yields a new +// window with values 1, x, 2, x, 3, where x indicates holes left by the +// dilation. So DilatedBound(3, 2) == 5. +int64 DilatedBound(int64 bound, int64 dilation); + +// Returns the number of valid positions of a window with the given size and +// stride within an array with the given bound. This is the bound of an output +// array with one element per valid position of the window. +// +// For example, for arguments of (bound=5, window_size=2, stride=2), the +// returned value is 2. There are valid positions at offset 0 and offset 2, +// while offset 4 is not valid since the window's last entry would be at 5, +// which is beyond the bound of 5. +int64 StridedBound(int64 bound, int64 window_size, int64 stride); + +} // namespace window_util +} // namespace xla + +#endif // TENSORFLOW_COMPILER_XLA_WINDOW_UTIL_H_ diff --git a/linux/include/tensorflow/compiler/xla/xla.pb.h b/linux/include/tensorflow/compiler/xla/xla.pb.h new file mode 100644 index 000000000..7e5586806 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/xla.pb.h @@ -0,0 +1,8616 @@ +// Generated by the protocol buffer compiler. DO NOT EDIT! +// source: tensorflow/compiler/xla/xla.proto + +#ifndef PROTOBUF_INCLUDED_tensorflow_2fcompiler_2fxla_2fxla_2eproto +#define PROTOBUF_INCLUDED_tensorflow_2fcompiler_2fxla_2fxla_2eproto + +#include + +#include + +#if GOOGLE_PROTOBUF_VERSION < 3006000 +#error This file was generated by a newer version of protoc which is +#error incompatible with your Protocol Buffer headers. Please update +#error your headers. +#endif +#if 3006000 < GOOGLE_PROTOBUF_MIN_PROTOC_VERSION +#error This file was generated by an older version of protoc which is +#error incompatible with your Protocol Buffer headers. Please +#error regenerate this file with a newer version of protoc. +#endif + +#include +#include +#include +#include +#include +#include +#include +#include +#include // IWYU pragma: export +#include // IWYU pragma: export +#include // IWYU pragma: export +#include +#include +#include +#include +#include "tensorflow/compiler/xla/xla_data.pb.h" +#include "tensorflow/compiler/xla/service/hlo.pb.h" +// @@protoc_insertion_point(includes) +#define PROTOBUF_INTERNAL_EXPORT_protobuf_tensorflow_2fcompiler_2fxla_2fxla_2eproto + +namespace protobuf_tensorflow_2fcompiler_2fxla_2fxla_2eproto { +// Internal implementation detail -- do not use these members. +struct TableStruct { + static const ::google::protobuf::internal::ParseTableField entries[]; + static const ::google::protobuf::internal::AuxillaryParseTableField aux[]; + static const ::google::protobuf::internal::ParseTable schema[38]; + static const ::google::protobuf::internal::FieldMetadata field_metadata[]; + static const ::google::protobuf::internal::SerializationTable serialization_table[]; + static const ::google::protobuf::uint32 offsets[]; +}; +void AddDescriptors(); +} // namespace protobuf_tensorflow_2fcompiler_2fxla_2fxla_2eproto +namespace xla { +class ComputationGraphStatsRequest; +class ComputationGraphStatsRequestDefaultTypeInternal; +extern ComputationGraphStatsRequestDefaultTypeInternal _ComputationGraphStatsRequest_default_instance_; +class ComputationStatsResponse; +class ComputationStatsResponseDefaultTypeInternal; +extern ComputationStatsResponseDefaultTypeInternal _ComputationStatsResponse_default_instance_; +class ComputeConstantGraphRequest; +class ComputeConstantGraphRequestDefaultTypeInternal; +extern ComputeConstantGraphRequestDefaultTypeInternal _ComputeConstantGraphRequest_default_instance_; +class ComputeConstantResponse; +class ComputeConstantResponseDefaultTypeInternal; +extern ComputeConstantResponseDefaultTypeInternal _ComputeConstantResponse_default_instance_; +class CreateChannelHandleRequest; +class CreateChannelHandleRequestDefaultTypeInternal; +extern CreateChannelHandleRequestDefaultTypeInternal _CreateChannelHandleRequest_default_instance_; +class CreateChannelHandleResponse; +class CreateChannelHandleResponseDefaultTypeInternal; +extern CreateChannelHandleResponseDefaultTypeInternal _CreateChannelHandleResponse_default_instance_; +class DebugOptions; +class DebugOptionsDefaultTypeInternal; +extern DebugOptionsDefaultTypeInternal _DebugOptions_default_instance_; +class DebugOptions_XlaBackendExtraOptionsEntry_DoNotUse; +class DebugOptions_XlaBackendExtraOptionsEntry_DoNotUseDefaultTypeInternal; +extern DebugOptions_XlaBackendExtraOptionsEntry_DoNotUseDefaultTypeInternal _DebugOptions_XlaBackendExtraOptionsEntry_DoNotUse_default_instance_; +class DeconstructTupleRequest; +class DeconstructTupleRequestDefaultTypeInternal; +extern DeconstructTupleRequestDefaultTypeInternal _DeconstructTupleRequest_default_instance_; +class DeconstructTupleResponse; +class DeconstructTupleResponseDefaultTypeInternal; +extern DeconstructTupleResponseDefaultTypeInternal _DeconstructTupleResponse_default_instance_; +class ExecuteGraphParallelRequest; +class ExecuteGraphParallelRequestDefaultTypeInternal; +extern ExecuteGraphParallelRequestDefaultTypeInternal _ExecuteGraphParallelRequest_default_instance_; +class ExecuteGraphRequest; +class ExecuteGraphRequestDefaultTypeInternal; +extern ExecuteGraphRequestDefaultTypeInternal _ExecuteGraphRequest_default_instance_; +class ExecuteParallelResponse; +class ExecuteParallelResponseDefaultTypeInternal; +extern ExecuteParallelResponseDefaultTypeInternal _ExecuteParallelResponse_default_instance_; +class ExecuteResponse; +class ExecuteResponseDefaultTypeInternal; +extern ExecuteResponseDefaultTypeInternal _ExecuteResponse_default_instance_; +class ExecutionOptions; +class ExecutionOptionsDefaultTypeInternal; +extern ExecutionOptionsDefaultTypeInternal _ExecutionOptions_default_instance_; +class GetDeviceHandlesRequest; +class GetDeviceHandlesRequestDefaultTypeInternal; +extern GetDeviceHandlesRequestDefaultTypeInternal _GetDeviceHandlesRequest_default_instance_; +class GetDeviceHandlesResponse; +class GetDeviceHandlesResponseDefaultTypeInternal; +extern GetDeviceHandlesResponseDefaultTypeInternal _GetDeviceHandlesResponse_default_instance_; +class GetShapeRequest; +class GetShapeRequestDefaultTypeInternal; +extern GetShapeRequestDefaultTypeInternal _GetShapeRequest_default_instance_; +class GetShapeResponse; +class GetShapeResponseDefaultTypeInternal; +extern GetShapeResponseDefaultTypeInternal _GetShapeResponse_default_instance_; +class HloReducePrecisionOptions; +class HloReducePrecisionOptionsDefaultTypeInternal; +extern HloReducePrecisionOptionsDefaultTypeInternal _HloReducePrecisionOptions_default_instance_; +class LoadDataRequest; +class LoadDataRequestDefaultTypeInternal; +extern LoadDataRequestDefaultTypeInternal _LoadDataRequest_default_instance_; +class LoadDataResponse; +class LoadDataResponseDefaultTypeInternal; +extern LoadDataResponseDefaultTypeInternal _LoadDataResponse_default_instance_; +class ResetDeviceRequest; +class ResetDeviceRequestDefaultTypeInternal; +extern ResetDeviceRequestDefaultTypeInternal _ResetDeviceRequest_default_instance_; +class ResetDeviceResponse; +class ResetDeviceResponseDefaultTypeInternal; +extern ResetDeviceResponseDefaultTypeInternal _ResetDeviceResponse_default_instance_; +class TransferFromOutfeedRequest; +class TransferFromOutfeedRequestDefaultTypeInternal; +extern TransferFromOutfeedRequestDefaultTypeInternal _TransferFromOutfeedRequest_default_instance_; +class TransferFromOutfeedResponse; +class TransferFromOutfeedResponseDefaultTypeInternal; +extern TransferFromOutfeedResponseDefaultTypeInternal _TransferFromOutfeedResponse_default_instance_; +class TransferToClientRequest; +class TransferToClientRequestDefaultTypeInternal; +extern TransferToClientRequestDefaultTypeInternal _TransferToClientRequest_default_instance_; +class TransferToClientResponse; +class TransferToClientResponseDefaultTypeInternal; +extern TransferToClientResponseDefaultTypeInternal _TransferToClientResponse_default_instance_; +class TransferToInfeedRequest; +class TransferToInfeedRequestDefaultTypeInternal; +extern TransferToInfeedRequestDefaultTypeInternal _TransferToInfeedRequest_default_instance_; +class TransferToInfeedResponse; +class TransferToInfeedResponseDefaultTypeInternal; +extern TransferToInfeedResponseDefaultTypeInternal _TransferToInfeedResponse_default_instance_; +class TransferToServerRequest; +class TransferToServerRequestDefaultTypeInternal; +extern TransferToServerRequestDefaultTypeInternal _TransferToServerRequest_default_instance_; +class TransferToServerResponse; +class TransferToServerResponseDefaultTypeInternal; +extern TransferToServerResponseDefaultTypeInternal _TransferToServerResponse_default_instance_; +class UnpackRequest; +class UnpackRequestDefaultTypeInternal; +extern UnpackRequestDefaultTypeInternal _UnpackRequest_default_instance_; +class UnpackResponse; +class UnpackResponseDefaultTypeInternal; +extern UnpackResponseDefaultTypeInternal _UnpackResponse_default_instance_; +class UnregisterRequest; +class UnregisterRequestDefaultTypeInternal; +extern UnregisterRequestDefaultTypeInternal _UnregisterRequest_default_instance_; +class UnregisterResponse; +class UnregisterResponseDefaultTypeInternal; +extern UnregisterResponseDefaultTypeInternal _UnregisterResponse_default_instance_; +class WaitForExecutionRequest; +class WaitForExecutionRequestDefaultTypeInternal; +extern WaitForExecutionRequestDefaultTypeInternal _WaitForExecutionRequest_default_instance_; +class WaitForExecutionResponse; +class WaitForExecutionResponseDefaultTypeInternal; +extern WaitForExecutionResponseDefaultTypeInternal _WaitForExecutionResponse_default_instance_; +} // namespace xla +namespace google { +namespace protobuf { +template<> ::xla::ComputationGraphStatsRequest* Arena::CreateMaybeMessage<::xla::ComputationGraphStatsRequest>(Arena*); +template<> ::xla::ComputationStatsResponse* Arena::CreateMaybeMessage<::xla::ComputationStatsResponse>(Arena*); +template<> ::xla::ComputeConstantGraphRequest* Arena::CreateMaybeMessage<::xla::ComputeConstantGraphRequest>(Arena*); +template<> ::xla::ComputeConstantResponse* Arena::CreateMaybeMessage<::xla::ComputeConstantResponse>(Arena*); +template<> ::xla::CreateChannelHandleRequest* Arena::CreateMaybeMessage<::xla::CreateChannelHandleRequest>(Arena*); +template<> ::xla::CreateChannelHandleResponse* Arena::CreateMaybeMessage<::xla::CreateChannelHandleResponse>(Arena*); +template<> ::xla::DebugOptions* Arena::CreateMaybeMessage<::xla::DebugOptions>(Arena*); +template<> ::xla::DebugOptions_XlaBackendExtraOptionsEntry_DoNotUse* Arena::CreateMaybeMessage<::xla::DebugOptions_XlaBackendExtraOptionsEntry_DoNotUse>(Arena*); +template<> ::xla::DeconstructTupleRequest* Arena::CreateMaybeMessage<::xla::DeconstructTupleRequest>(Arena*); +template<> ::xla::DeconstructTupleResponse* Arena::CreateMaybeMessage<::xla::DeconstructTupleResponse>(Arena*); +template<> ::xla::ExecuteGraphParallelRequest* Arena::CreateMaybeMessage<::xla::ExecuteGraphParallelRequest>(Arena*); +template<> ::xla::ExecuteGraphRequest* Arena::CreateMaybeMessage<::xla::ExecuteGraphRequest>(Arena*); +template<> ::xla::ExecuteParallelResponse* Arena::CreateMaybeMessage<::xla::ExecuteParallelResponse>(Arena*); +template<> ::xla::ExecuteResponse* Arena::CreateMaybeMessage<::xla::ExecuteResponse>(Arena*); +template<> ::xla::ExecutionOptions* Arena::CreateMaybeMessage<::xla::ExecutionOptions>(Arena*); +template<> ::xla::GetDeviceHandlesRequest* Arena::CreateMaybeMessage<::xla::GetDeviceHandlesRequest>(Arena*); +template<> ::xla::GetDeviceHandlesResponse* Arena::CreateMaybeMessage<::xla::GetDeviceHandlesResponse>(Arena*); +template<> ::xla::GetShapeRequest* Arena::CreateMaybeMessage<::xla::GetShapeRequest>(Arena*); +template<> ::xla::GetShapeResponse* Arena::CreateMaybeMessage<::xla::GetShapeResponse>(Arena*); +template<> ::xla::HloReducePrecisionOptions* Arena::CreateMaybeMessage<::xla::HloReducePrecisionOptions>(Arena*); +template<> ::xla::LoadDataRequest* Arena::CreateMaybeMessage<::xla::LoadDataRequest>(Arena*); +template<> ::xla::LoadDataResponse* Arena::CreateMaybeMessage<::xla::LoadDataResponse>(Arena*); +template<> ::xla::ResetDeviceRequest* Arena::CreateMaybeMessage<::xla::ResetDeviceRequest>(Arena*); +template<> ::xla::ResetDeviceResponse* Arena::CreateMaybeMessage<::xla::ResetDeviceResponse>(Arena*); +template<> ::xla::TransferFromOutfeedRequest* Arena::CreateMaybeMessage<::xla::TransferFromOutfeedRequest>(Arena*); +template<> ::xla::TransferFromOutfeedResponse* Arena::CreateMaybeMessage<::xla::TransferFromOutfeedResponse>(Arena*); +template<> ::xla::TransferToClientRequest* Arena::CreateMaybeMessage<::xla::TransferToClientRequest>(Arena*); +template<> ::xla::TransferToClientResponse* Arena::CreateMaybeMessage<::xla::TransferToClientResponse>(Arena*); +template<> ::xla::TransferToInfeedRequest* Arena::CreateMaybeMessage<::xla::TransferToInfeedRequest>(Arena*); +template<> ::xla::TransferToInfeedResponse* Arena::CreateMaybeMessage<::xla::TransferToInfeedResponse>(Arena*); +template<> ::xla::TransferToServerRequest* Arena::CreateMaybeMessage<::xla::TransferToServerRequest>(Arena*); +template<> ::xla::TransferToServerResponse* Arena::CreateMaybeMessage<::xla::TransferToServerResponse>(Arena*); +template<> ::xla::UnpackRequest* Arena::CreateMaybeMessage<::xla::UnpackRequest>(Arena*); +template<> ::xla::UnpackResponse* Arena::CreateMaybeMessage<::xla::UnpackResponse>(Arena*); +template<> ::xla::UnregisterRequest* Arena::CreateMaybeMessage<::xla::UnregisterRequest>(Arena*); +template<> ::xla::UnregisterResponse* Arena::CreateMaybeMessage<::xla::UnregisterResponse>(Arena*); +template<> ::xla::WaitForExecutionRequest* Arena::CreateMaybeMessage<::xla::WaitForExecutionRequest>(Arena*); +template<> ::xla::WaitForExecutionResponse* Arena::CreateMaybeMessage<::xla::WaitForExecutionResponse>(Arena*); +} // namespace protobuf +} // namespace google +namespace xla { + +enum HloReducePrecisionOptions_Location { + HloReducePrecisionOptions_Location_OP_INPUTS = 0, + HloReducePrecisionOptions_Location_OP_OUTPUTS = 1, + HloReducePrecisionOptions_Location_UNFUSED_OP_OUTPUTS = 2, + HloReducePrecisionOptions_Location_FUSION_INPUTS_BY_CONTENT = 3, + HloReducePrecisionOptions_Location_FUSION_OUTPUTS_BY_CONTENT = 4, + HloReducePrecisionOptions_Location_HloReducePrecisionOptions_Location_INT_MIN_SENTINEL_DO_NOT_USE_ = ::google::protobuf::kint32min, + HloReducePrecisionOptions_Location_HloReducePrecisionOptions_Location_INT_MAX_SENTINEL_DO_NOT_USE_ = ::google::protobuf::kint32max +}; +bool HloReducePrecisionOptions_Location_IsValid(int value); +const HloReducePrecisionOptions_Location HloReducePrecisionOptions_Location_Location_MIN = HloReducePrecisionOptions_Location_OP_INPUTS; +const HloReducePrecisionOptions_Location HloReducePrecisionOptions_Location_Location_MAX = HloReducePrecisionOptions_Location_FUSION_OUTPUTS_BY_CONTENT; +const int HloReducePrecisionOptions_Location_Location_ARRAYSIZE = HloReducePrecisionOptions_Location_Location_MAX + 1; + +const ::google::protobuf::EnumDescriptor* HloReducePrecisionOptions_Location_descriptor(); +inline const ::std::string& HloReducePrecisionOptions_Location_Name(HloReducePrecisionOptions_Location value) { + return ::google::protobuf::internal::NameOfEnum( + HloReducePrecisionOptions_Location_descriptor(), value); +} +inline bool HloReducePrecisionOptions_Location_Parse( + const ::std::string& name, HloReducePrecisionOptions_Location* value) { + return ::google::protobuf::internal::ParseNamedEnum( + HloReducePrecisionOptions_Location_descriptor(), name, value); +} +// =================================================================== + +class HloReducePrecisionOptions : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.HloReducePrecisionOptions) */ { + public: + HloReducePrecisionOptions(); + virtual ~HloReducePrecisionOptions(); + + HloReducePrecisionOptions(const HloReducePrecisionOptions& from); + + inline HloReducePrecisionOptions& operator=(const HloReducePrecisionOptions& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + HloReducePrecisionOptions(HloReducePrecisionOptions&& from) noexcept + : HloReducePrecisionOptions() { + *this = ::std::move(from); + } + + inline HloReducePrecisionOptions& operator=(HloReducePrecisionOptions&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + static const ::google::protobuf::Descriptor* descriptor(); + static const HloReducePrecisionOptions& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const HloReducePrecisionOptions* internal_default_instance() { + return reinterpret_cast( + &_HloReducePrecisionOptions_default_instance_); + } + static constexpr int kIndexInFileMessages = + 0; + + void Swap(HloReducePrecisionOptions* other); + friend void swap(HloReducePrecisionOptions& a, HloReducePrecisionOptions& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline HloReducePrecisionOptions* New() const final { + return CreateMaybeMessage(NULL); + } + + HloReducePrecisionOptions* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const HloReducePrecisionOptions& from); + void MergeFrom(const HloReducePrecisionOptions& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(HloReducePrecisionOptions* other); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return NULL; + } + inline void* MaybeArenaPtr() const { + return NULL; + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + typedef HloReducePrecisionOptions_Location Location; + static const Location OP_INPUTS = + HloReducePrecisionOptions_Location_OP_INPUTS; + static const Location OP_OUTPUTS = + HloReducePrecisionOptions_Location_OP_OUTPUTS; + static const Location UNFUSED_OP_OUTPUTS = + HloReducePrecisionOptions_Location_UNFUSED_OP_OUTPUTS; + static const Location FUSION_INPUTS_BY_CONTENT = + HloReducePrecisionOptions_Location_FUSION_INPUTS_BY_CONTENT; + static const Location FUSION_OUTPUTS_BY_CONTENT = + HloReducePrecisionOptions_Location_FUSION_OUTPUTS_BY_CONTENT; + static inline bool Location_IsValid(int value) { + return HloReducePrecisionOptions_Location_IsValid(value); + } + static const Location Location_MIN = + HloReducePrecisionOptions_Location_Location_MIN; + static const Location Location_MAX = + HloReducePrecisionOptions_Location_Location_MAX; + static const int Location_ARRAYSIZE = + HloReducePrecisionOptions_Location_Location_ARRAYSIZE; + static inline const ::google::protobuf::EnumDescriptor* + Location_descriptor() { + return HloReducePrecisionOptions_Location_descriptor(); + } + static inline const ::std::string& Location_Name(Location value) { + return HloReducePrecisionOptions_Location_Name(value); + } + static inline bool Location_Parse(const ::std::string& name, + Location* value) { + return HloReducePrecisionOptions_Location_Parse(name, value); + } + + // accessors ------------------------------------------------------- + + // repeated uint32 opcodes_to_suffix = 4; + int opcodes_to_suffix_size() const; + void clear_opcodes_to_suffix(); + static const int kOpcodesToSuffixFieldNumber = 4; + ::google::protobuf::uint32 opcodes_to_suffix(int index) const; + void set_opcodes_to_suffix(int index, ::google::protobuf::uint32 value); + void add_opcodes_to_suffix(::google::protobuf::uint32 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::uint32 >& + opcodes_to_suffix() const; + ::google::protobuf::RepeatedField< ::google::protobuf::uint32 >* + mutable_opcodes_to_suffix(); + + // repeated string opname_substrings_to_suffix = 5; + int opname_substrings_to_suffix_size() const; + void clear_opname_substrings_to_suffix(); + static const int kOpnameSubstringsToSuffixFieldNumber = 5; + const ::std::string& opname_substrings_to_suffix(int index) const; + ::std::string* mutable_opname_substrings_to_suffix(int index); + void set_opname_substrings_to_suffix(int index, const ::std::string& value); + #if LANG_CXX11 + void set_opname_substrings_to_suffix(int index, ::std::string&& value); + #endif + void set_opname_substrings_to_suffix(int index, const char* value); + void set_opname_substrings_to_suffix(int index, const char* value, size_t size); + ::std::string* add_opname_substrings_to_suffix(); + void add_opname_substrings_to_suffix(const ::std::string& value); + #if LANG_CXX11 + void add_opname_substrings_to_suffix(::std::string&& value); + #endif + void add_opname_substrings_to_suffix(const char* value); + void add_opname_substrings_to_suffix(const char* value, size_t size); + const ::google::protobuf::RepeatedPtrField< ::std::string>& opname_substrings_to_suffix() const; + ::google::protobuf::RepeatedPtrField< ::std::string>* mutable_opname_substrings_to_suffix(); + + // .xla.HloReducePrecisionOptions.Location location = 1; + void clear_location(); + static const int kLocationFieldNumber = 1; + ::xla::HloReducePrecisionOptions_Location location() const; + void set_location(::xla::HloReducePrecisionOptions_Location value); + + // uint32 exponent_bits = 2; + void clear_exponent_bits(); + static const int kExponentBitsFieldNumber = 2; + ::google::protobuf::uint32 exponent_bits() const; + void set_exponent_bits(::google::protobuf::uint32 value); + + // uint32 mantissa_bits = 3; + void clear_mantissa_bits(); + static const int kMantissaBitsFieldNumber = 3; + ::google::protobuf::uint32 mantissa_bits() const; + void set_mantissa_bits(::google::protobuf::uint32 value); + + // @@protoc_insertion_point(class_scope:xla.HloReducePrecisionOptions) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + ::google::protobuf::RepeatedField< ::google::protobuf::uint32 > opcodes_to_suffix_; + mutable int _opcodes_to_suffix_cached_byte_size_; + ::google::protobuf::RepeatedPtrField< ::std::string> opname_substrings_to_suffix_; + int location_; + ::google::protobuf::uint32 exponent_bits_; + ::google::protobuf::uint32 mantissa_bits_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class DebugOptions_XlaBackendExtraOptionsEntry_DoNotUse : public ::google::protobuf::internal::MapEntry { +public: + typedef ::google::protobuf::internal::MapEntry SuperType; + DebugOptions_XlaBackendExtraOptionsEntry_DoNotUse(); + DebugOptions_XlaBackendExtraOptionsEntry_DoNotUse(::google::protobuf::Arena* arena); + void MergeFrom(const DebugOptions_XlaBackendExtraOptionsEntry_DoNotUse& other); + static const DebugOptions_XlaBackendExtraOptionsEntry_DoNotUse* internal_default_instance() { return reinterpret_cast(&_DebugOptions_XlaBackendExtraOptionsEntry_DoNotUse_default_instance_); } + void MergeFrom(const ::google::protobuf::Message& other) final; + ::google::protobuf::Metadata GetMetadata() const; +}; + +// ------------------------------------------------------------------- + +class DebugOptions : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.DebugOptions) */ { + public: + DebugOptions(); + virtual ~DebugOptions(); + + DebugOptions(const DebugOptions& from); + + inline DebugOptions& operator=(const DebugOptions& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + DebugOptions(DebugOptions&& from) noexcept + : DebugOptions() { + *this = ::std::move(from); + } + + inline DebugOptions& operator=(DebugOptions&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + static const ::google::protobuf::Descriptor* descriptor(); + static const DebugOptions& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const DebugOptions* internal_default_instance() { + return reinterpret_cast( + &_DebugOptions_default_instance_); + } + static constexpr int kIndexInFileMessages = + 2; + + void Swap(DebugOptions* other); + friend void swap(DebugOptions& a, DebugOptions& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline DebugOptions* New() const final { + return CreateMaybeMessage(NULL); + } + + DebugOptions* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const DebugOptions& from); + void MergeFrom(const DebugOptions& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(DebugOptions* other); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return NULL; + } + inline void* MaybeArenaPtr() const { + return NULL; + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + + // accessors ------------------------------------------------------- + + // repeated string xla_disable_hlo_passes = 30; + int xla_disable_hlo_passes_size() const; + void clear_xla_disable_hlo_passes(); + static const int kXlaDisableHloPassesFieldNumber = 30; + const ::std::string& xla_disable_hlo_passes(int index) const; + ::std::string* mutable_xla_disable_hlo_passes(int index); + void set_xla_disable_hlo_passes(int index, const ::std::string& value); + #if LANG_CXX11 + void set_xla_disable_hlo_passes(int index, ::std::string&& value); + #endif + void set_xla_disable_hlo_passes(int index, const char* value); + void set_xla_disable_hlo_passes(int index, const char* value, size_t size); + ::std::string* add_xla_disable_hlo_passes(); + void add_xla_disable_hlo_passes(const ::std::string& value); + #if LANG_CXX11 + void add_xla_disable_hlo_passes(::std::string&& value); + #endif + void add_xla_disable_hlo_passes(const char* value); + void add_xla_disable_hlo_passes(const char* value, size_t size); + const ::google::protobuf::RepeatedPtrField< ::std::string>& xla_disable_hlo_passes() const; + ::google::protobuf::RepeatedPtrField< ::std::string>* mutable_xla_disable_hlo_passes(); + + // repeated .xla.HloReducePrecisionOptions hlo_reduce_precision_options = 80; + int hlo_reduce_precision_options_size() const; + void clear_hlo_reduce_precision_options(); + static const int kHloReducePrecisionOptionsFieldNumber = 80; + ::xla::HloReducePrecisionOptions* mutable_hlo_reduce_precision_options(int index); + ::google::protobuf::RepeatedPtrField< ::xla::HloReducePrecisionOptions >* + mutable_hlo_reduce_precision_options(); + const ::xla::HloReducePrecisionOptions& hlo_reduce_precision_options(int index) const; + ::xla::HloReducePrecisionOptions* add_hlo_reduce_precision_options(); + const ::google::protobuf::RepeatedPtrField< ::xla::HloReducePrecisionOptions >& + hlo_reduce_precision_options() const; + + // map xla_backend_extra_options = 500; + int xla_backend_extra_options_size() const; + void clear_xla_backend_extra_options(); + static const int kXlaBackendExtraOptionsFieldNumber = 500; + const ::google::protobuf::Map< ::std::string, ::std::string >& + xla_backend_extra_options() const; + ::google::protobuf::Map< ::std::string, ::std::string >* + mutable_xla_backend_extra_options(); + + // string xla_generate_hlo_graph = 1; + void clear_xla_generate_hlo_graph(); + static const int kXlaGenerateHloGraphFieldNumber = 1; + const ::std::string& xla_generate_hlo_graph() const; + void set_xla_generate_hlo_graph(const ::std::string& value); + #if LANG_CXX11 + void set_xla_generate_hlo_graph(::std::string&& value); + #endif + void set_xla_generate_hlo_graph(const char* value); + void set_xla_generate_hlo_graph(const char* value, size_t size); + ::std::string* mutable_xla_generate_hlo_graph(); + ::std::string* release_xla_generate_hlo_graph(); + void set_allocated_xla_generate_hlo_graph(::std::string* xla_generate_hlo_graph); + + // string xla_hlo_graph_path = 4; + void clear_xla_hlo_graph_path(); + static const int kXlaHloGraphPathFieldNumber = 4; + const ::std::string& xla_hlo_graph_path() const; + void set_xla_hlo_graph_path(const ::std::string& value); + #if LANG_CXX11 + void set_xla_hlo_graph_path(::std::string&& value); + #endif + void set_xla_hlo_graph_path(const char* value); + void set_xla_hlo_graph_path(const char* value, size_t size); + ::std::string* mutable_xla_hlo_graph_path(); + ::std::string* release_xla_hlo_graph_path(); + void set_allocated_xla_hlo_graph_path(::std::string* xla_hlo_graph_path); + + // string xla_log_hlo_text = 6; + void clear_xla_log_hlo_text(); + static const int kXlaLogHloTextFieldNumber = 6; + const ::std::string& xla_log_hlo_text() const; + void set_xla_log_hlo_text(const ::std::string& value); + #if LANG_CXX11 + void set_xla_log_hlo_text(::std::string&& value); + #endif + void set_xla_log_hlo_text(const char* value); + void set_xla_log_hlo_text(const char* value, size_t size); + ::std::string* mutable_xla_log_hlo_text(); + ::std::string* release_xla_log_hlo_text(); + void set_allocated_xla_log_hlo_text(::std::string* xla_log_hlo_text); + + // string xla_generate_hlo_text_to = 7; + void clear_xla_generate_hlo_text_to(); + static const int kXlaGenerateHloTextToFieldNumber = 7; + const ::std::string& xla_generate_hlo_text_to() const; + void set_xla_generate_hlo_text_to(const ::std::string& value); + #if LANG_CXX11 + void set_xla_generate_hlo_text_to(::std::string&& value); + #endif + void set_xla_generate_hlo_text_to(const char* value); + void set_xla_generate_hlo_text_to(const char* value, size_t size); + ::std::string* mutable_xla_generate_hlo_text_to(); + ::std::string* release_xla_generate_hlo_text_to(); + void set_allocated_xla_generate_hlo_text_to(::std::string* xla_generate_hlo_text_to); + + // string xla_dump_optimized_hlo_proto_to = 8; + void clear_xla_dump_optimized_hlo_proto_to(); + static const int kXlaDumpOptimizedHloProtoToFieldNumber = 8; + const ::std::string& xla_dump_optimized_hlo_proto_to() const; + void set_xla_dump_optimized_hlo_proto_to(const ::std::string& value); + #if LANG_CXX11 + void set_xla_dump_optimized_hlo_proto_to(::std::string&& value); + #endif + void set_xla_dump_optimized_hlo_proto_to(const char* value); + void set_xla_dump_optimized_hlo_proto_to(const char* value, size_t size); + ::std::string* mutable_xla_dump_optimized_hlo_proto_to(); + ::std::string* release_xla_dump_optimized_hlo_proto_to(); + void set_allocated_xla_dump_optimized_hlo_proto_to(::std::string* xla_dump_optimized_hlo_proto_to); + + // string xla_dump_computations_to = 10; + void clear_xla_dump_computations_to(); + static const int kXlaDumpComputationsToFieldNumber = 10; + const ::std::string& xla_dump_computations_to() const; + void set_xla_dump_computations_to(const ::std::string& value); + #if LANG_CXX11 + void set_xla_dump_computations_to(::std::string&& value); + #endif + void set_xla_dump_computations_to(const char* value); + void set_xla_dump_computations_to(const char* value, size_t size); + ::std::string* mutable_xla_dump_computations_to(); + ::std::string* release_xla_dump_computations_to(); + void set_allocated_xla_dump_computations_to(::std::string* xla_dump_computations_to); + + // string xla_dump_executions_to = 11; + void clear_xla_dump_executions_to(); + static const int kXlaDumpExecutionsToFieldNumber = 11; + const ::std::string& xla_dump_executions_to() const; + void set_xla_dump_executions_to(const ::std::string& value); + #if LANG_CXX11 + void set_xla_dump_executions_to(::std::string&& value); + #endif + void set_xla_dump_executions_to(const char* value); + void set_xla_dump_executions_to(const char* value, size_t size); + ::std::string* mutable_xla_dump_executions_to(); + ::std::string* release_xla_dump_executions_to(); + void set_allocated_xla_dump_executions_to(::std::string* xla_dump_executions_to); + + // string xla_dump_ir_to = 34; + void clear_xla_dump_ir_to(); + static const int kXlaDumpIrToFieldNumber = 34; + const ::std::string& xla_dump_ir_to() const; + void set_xla_dump_ir_to(const ::std::string& value); + #if LANG_CXX11 + void set_xla_dump_ir_to(::std::string&& value); + #endif + void set_xla_dump_ir_to(const char* value); + void set_xla_dump_ir_to(const char* value, size_t size); + ::std::string* mutable_xla_dump_ir_to(); + ::std::string* release_xla_dump_ir_to(); + void set_allocated_xla_dump_ir_to(::std::string* xla_dump_ir_to); + + // string xla_gpu_cuda_data_dir = 61; + void clear_xla_gpu_cuda_data_dir(); + static const int kXlaGpuCudaDataDirFieldNumber = 61; + const ::std::string& xla_gpu_cuda_data_dir() const; + void set_xla_gpu_cuda_data_dir(const ::std::string& value); + #if LANG_CXX11 + void set_xla_gpu_cuda_data_dir(::std::string&& value); + #endif + void set_xla_gpu_cuda_data_dir(const char* value); + void set_xla_gpu_cuda_data_dir(const char* value, size_t size); + ::std::string* mutable_xla_gpu_cuda_data_dir(); + ::std::string* release_xla_gpu_cuda_data_dir(); + void set_allocated_xla_gpu_cuda_data_dir(::std::string* xla_gpu_cuda_data_dir); + + // string xla_dump_unoptimized_hlo_proto_to = 95; + void clear_xla_dump_unoptimized_hlo_proto_to(); + static const int kXlaDumpUnoptimizedHloProtoToFieldNumber = 95; + const ::std::string& xla_dump_unoptimized_hlo_proto_to() const; + void set_xla_dump_unoptimized_hlo_proto_to(const ::std::string& value); + #if LANG_CXX11 + void set_xla_dump_unoptimized_hlo_proto_to(::std::string&& value); + #endif + void set_xla_dump_unoptimized_hlo_proto_to(const char* value); + void set_xla_dump_unoptimized_hlo_proto_to(const char* value, size_t size); + ::std::string* mutable_xla_dump_unoptimized_hlo_proto_to(); + ::std::string* release_xla_dump_unoptimized_hlo_proto_to(); + void set_allocated_xla_dump_unoptimized_hlo_proto_to(::std::string* xla_dump_unoptimized_hlo_proto_to); + + // string xla_dump_per_pass_hlo_proto_to = 96; + void clear_xla_dump_per_pass_hlo_proto_to(); + static const int kXlaDumpPerPassHloProtoToFieldNumber = 96; + const ::std::string& xla_dump_per_pass_hlo_proto_to() const; + void set_xla_dump_per_pass_hlo_proto_to(const ::std::string& value); + #if LANG_CXX11 + void set_xla_dump_per_pass_hlo_proto_to(::std::string&& value); + #endif + void set_xla_dump_per_pass_hlo_proto_to(const char* value); + void set_xla_dump_per_pass_hlo_proto_to(const char* value, size_t size); + ::std::string* mutable_xla_dump_per_pass_hlo_proto_to(); + ::std::string* release_xla_dump_per_pass_hlo_proto_to(); + void set_allocated_xla_dump_per_pass_hlo_proto_to(::std::string* xla_dump_per_pass_hlo_proto_to); + + // bool xla_hlo_graph_addresses = 2; + void clear_xla_hlo_graph_addresses(); + static const int kXlaHloGraphAddressesFieldNumber = 2; + bool xla_hlo_graph_addresses() const; + void set_xla_hlo_graph_addresses(bool value); + + // bool xla_hlo_dump_as_graphdef = 5; + void clear_xla_hlo_dump_as_graphdef(); + static const int kXlaHloDumpAsGraphdefFieldNumber = 5; + bool xla_hlo_dump_as_graphdef() const; + void set_xla_hlo_dump_as_graphdef(bool value); + + // bool xla_hlo_profile = 9; + void clear_xla_hlo_profile(); + static const int kXlaHloProfileFieldNumber = 9; + bool xla_hlo_profile() const; + void set_xla_hlo_profile(bool value); + + // bool xla_embed_ir_in_executable = 33; + void clear_xla_embed_ir_in_executable(); + static const int kXlaEmbedIrInExecutableFieldNumber = 33; + bool xla_embed_ir_in_executable() const; + void set_xla_embed_ir_in_executable(bool value); + + // int32 xla_backend_optimization_level = 31; + void clear_xla_backend_optimization_level(); + static const int kXlaBackendOptimizationLevelFieldNumber = 31; + ::google::protobuf::int32 xla_backend_optimization_level() const; + void set_xla_backend_optimization_level(::google::protobuf::int32 value); + + // int32 xla_force_host_platform_device_count = 102; + void clear_xla_force_host_platform_device_count(); + static const int kXlaForceHostPlatformDeviceCountFieldNumber = 102; + ::google::protobuf::int32 xla_force_host_platform_device_count() const; + void set_xla_force_host_platform_device_count(::google::protobuf::int32 value); + + // bool xla_eliminate_hlo_implicit_broadcast = 35; + void clear_xla_eliminate_hlo_implicit_broadcast(); + static const int kXlaEliminateHloImplicitBroadcastFieldNumber = 35; + bool xla_eliminate_hlo_implicit_broadcast() const; + void set_xla_eliminate_hlo_implicit_broadcast(bool value); + + // bool xla_cpu_multi_thread_eigen = 60; + void clear_xla_cpu_multi_thread_eigen(); + static const int kXlaCpuMultiThreadEigenFieldNumber = 60; + bool xla_cpu_multi_thread_eigen() const; + void set_xla_cpu_multi_thread_eigen(bool value); + + // bool xla_gpu_ftz = 62; + void clear_xla_gpu_ftz(); + static const int kXlaGpuFtzFieldNumber = 62; + bool xla_gpu_ftz() const; + void set_xla_gpu_ftz(bool value); + + // bool xla_gpu_disable_multi_streaming = 63; + void clear_xla_gpu_disable_multi_streaming(); + static const int kXlaGpuDisableMultiStreamingFieldNumber = 63; + bool xla_gpu_disable_multi_streaming() const; + void set_xla_gpu_disable_multi_streaming(bool value); + + // bool xla_llvm_enable_alias_scope_metadata = 70; + void clear_xla_llvm_enable_alias_scope_metadata(); + static const int kXlaLlvmEnableAliasScopeMetadataFieldNumber = 70; + bool xla_llvm_enable_alias_scope_metadata() const; + void set_xla_llvm_enable_alias_scope_metadata(bool value); + + // bool xla_llvm_enable_noalias_metadata = 71; + void clear_xla_llvm_enable_noalias_metadata(); + static const int kXlaLlvmEnableNoaliasMetadataFieldNumber = 71; + bool xla_llvm_enable_noalias_metadata() const; + void set_xla_llvm_enable_noalias_metadata(bool value); + + // bool xla_llvm_enable_invariant_load_metadata = 72; + void clear_xla_llvm_enable_invariant_load_metadata(); + static const int kXlaLlvmEnableInvariantLoadMetadataFieldNumber = 72; + bool xla_llvm_enable_invariant_load_metadata() const; + void set_xla_llvm_enable_invariant_load_metadata(bool value); + + // bool xla_llvm_disable_expensive_passes = 73; + void clear_xla_llvm_disable_expensive_passes(); + static const int kXlaLlvmDisableExpensivePassesFieldNumber = 73; + bool xla_llvm_disable_expensive_passes() const; + void set_xla_llvm_disable_expensive_passes(bool value); + + // bool xla_test_all_output_layouts = 90; + void clear_xla_test_all_output_layouts(); + static const int kXlaTestAllOutputLayoutsFieldNumber = 90; + bool xla_test_all_output_layouts() const; + void set_xla_test_all_output_layouts(bool value); + + // bool xla_test_all_input_layouts = 91; + void clear_xla_test_all_input_layouts(); + static const int kXlaTestAllInputLayoutsFieldNumber = 91; + bool xla_test_all_input_layouts() const; + void set_xla_test_all_input_layouts(bool value); + + // bool xla_hlo_graph_sharding_color = 92; + void clear_xla_hlo_graph_sharding_color(); + static const int kXlaHloGraphShardingColorFieldNumber = 92; + bool xla_hlo_graph_sharding_color() const; + void set_xla_hlo_graph_sharding_color(bool value); + + // bool xla_hlo_tfgraph_device_scopes = 93; + void clear_xla_hlo_tfgraph_device_scopes(); + static const int kXlaHloTfgraphDeviceScopesFieldNumber = 93; + bool xla_hlo_tfgraph_device_scopes() const; + void set_xla_hlo_tfgraph_device_scopes(bool value); + + // bool xla_gpu_use_cudnn_batchnorm = 94; + void clear_xla_gpu_use_cudnn_batchnorm(); + static const int kXlaGpuUseCudnnBatchnormFieldNumber = 94; + bool xla_gpu_use_cudnn_batchnorm() const; + void set_xla_gpu_use_cudnn_batchnorm(bool value); + + // bool xla_cpu_use_mkl_dnn = 97; + void clear_xla_cpu_use_mkl_dnn(); + static const int kXlaCpuUseMklDnnFieldNumber = 97; + bool xla_cpu_use_mkl_dnn() const; + void set_xla_cpu_use_mkl_dnn(bool value); + + // bool xla_cpu_enable_fast_math = 99; + void clear_xla_cpu_enable_fast_math(); + static const int kXlaCpuEnableFastMathFieldNumber = 99; + bool xla_cpu_enable_fast_math() const; + void set_xla_cpu_enable_fast_math(bool value); + + // bool xla_gpu_enable_fast_math = 100; + void clear_xla_gpu_enable_fast_math(); + static const int kXlaGpuEnableFastMathFieldNumber = 100; + bool xla_gpu_enable_fast_math() const; + void set_xla_gpu_enable_fast_math(bool value); + + // int32 xla_gpu_max_kernel_unroll_factor = 98; + void clear_xla_gpu_max_kernel_unroll_factor(); + static const int kXlaGpuMaxKernelUnrollFactorFieldNumber = 98; + ::google::protobuf::int32 xla_gpu_max_kernel_unroll_factor() const; + void set_xla_gpu_max_kernel_unroll_factor(::google::protobuf::int32 value); + + // bool xla_gpu_crash_on_verification_failures = 101; + void clear_xla_gpu_crash_on_verification_failures(); + static const int kXlaGpuCrashOnVerificationFailuresFieldNumber = 101; + bool xla_gpu_crash_on_verification_failures() const; + void set_xla_gpu_crash_on_verification_failures(bool value); + + // @@protoc_insertion_point(class_scope:xla.DebugOptions) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + ::google::protobuf::RepeatedPtrField< ::std::string> xla_disable_hlo_passes_; + ::google::protobuf::RepeatedPtrField< ::xla::HloReducePrecisionOptions > hlo_reduce_precision_options_; + ::google::protobuf::internal::MapField< + DebugOptions_XlaBackendExtraOptionsEntry_DoNotUse, + ::std::string, ::std::string, + ::google::protobuf::internal::WireFormatLite::TYPE_STRING, + ::google::protobuf::internal::WireFormatLite::TYPE_STRING, + 0 > xla_backend_extra_options_; + ::google::protobuf::internal::ArenaStringPtr xla_generate_hlo_graph_; + ::google::protobuf::internal::ArenaStringPtr xla_hlo_graph_path_; + ::google::protobuf::internal::ArenaStringPtr xla_log_hlo_text_; + ::google::protobuf::internal::ArenaStringPtr xla_generate_hlo_text_to_; + ::google::protobuf::internal::ArenaStringPtr xla_dump_optimized_hlo_proto_to_; + ::google::protobuf::internal::ArenaStringPtr xla_dump_computations_to_; + ::google::protobuf::internal::ArenaStringPtr xla_dump_executions_to_; + ::google::protobuf::internal::ArenaStringPtr xla_dump_ir_to_; + ::google::protobuf::internal::ArenaStringPtr xla_gpu_cuda_data_dir_; + ::google::protobuf::internal::ArenaStringPtr xla_dump_unoptimized_hlo_proto_to_; + ::google::protobuf::internal::ArenaStringPtr xla_dump_per_pass_hlo_proto_to_; + bool xla_hlo_graph_addresses_; + bool xla_hlo_dump_as_graphdef_; + bool xla_hlo_profile_; + bool xla_embed_ir_in_executable_; + ::google::protobuf::int32 xla_backend_optimization_level_; + ::google::protobuf::int32 xla_force_host_platform_device_count_; + bool xla_eliminate_hlo_implicit_broadcast_; + bool xla_cpu_multi_thread_eigen_; + bool xla_gpu_ftz_; + bool xla_gpu_disable_multi_streaming_; + bool xla_llvm_enable_alias_scope_metadata_; + bool xla_llvm_enable_noalias_metadata_; + bool xla_llvm_enable_invariant_load_metadata_; + bool xla_llvm_disable_expensive_passes_; + bool xla_test_all_output_layouts_; + bool xla_test_all_input_layouts_; + bool xla_hlo_graph_sharding_color_; + bool xla_hlo_tfgraph_device_scopes_; + bool xla_gpu_use_cudnn_batchnorm_; + bool xla_cpu_use_mkl_dnn_; + bool xla_cpu_enable_fast_math_; + bool xla_gpu_enable_fast_math_; + ::google::protobuf::int32 xla_gpu_max_kernel_unroll_factor_; + bool xla_gpu_crash_on_verification_failures_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class ExecutionOptions : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.ExecutionOptions) */ { + public: + ExecutionOptions(); + virtual ~ExecutionOptions(); + + ExecutionOptions(const ExecutionOptions& from); + + inline ExecutionOptions& operator=(const ExecutionOptions& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + ExecutionOptions(ExecutionOptions&& from) noexcept + : ExecutionOptions() { + *this = ::std::move(from); + } + + inline ExecutionOptions& operator=(ExecutionOptions&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + static const ::google::protobuf::Descriptor* descriptor(); + static const ExecutionOptions& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const ExecutionOptions* internal_default_instance() { + return reinterpret_cast( + &_ExecutionOptions_default_instance_); + } + static constexpr int kIndexInFileMessages = + 3; + + void Swap(ExecutionOptions* other); + friend void swap(ExecutionOptions& a, ExecutionOptions& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline ExecutionOptions* New() const final { + return CreateMaybeMessage(NULL); + } + + ExecutionOptions* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const ExecutionOptions& from); + void MergeFrom(const ExecutionOptions& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(ExecutionOptions* other); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return NULL; + } + inline void* MaybeArenaPtr() const { + return NULL; + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated .xla.DeviceHandle device_handles = 5; + int device_handles_size() const; + void clear_device_handles(); + static const int kDeviceHandlesFieldNumber = 5; + ::xla::DeviceHandle* mutable_device_handles(int index); + ::google::protobuf::RepeatedPtrField< ::xla::DeviceHandle >* + mutable_device_handles(); + const ::xla::DeviceHandle& device_handles(int index) const; + ::xla::DeviceHandle* add_device_handles(); + const ::google::protobuf::RepeatedPtrField< ::xla::DeviceHandle >& + device_handles() const; + + // .xla.Shape shape_with_output_layout = 2; + bool has_shape_with_output_layout() const; + void clear_shape_with_output_layout(); + static const int kShapeWithOutputLayoutFieldNumber = 2; + private: + const ::xla::Shape& _internal_shape_with_output_layout() const; + public: + const ::xla::Shape& shape_with_output_layout() const; + ::xla::Shape* release_shape_with_output_layout(); + ::xla::Shape* mutable_shape_with_output_layout(); + void set_allocated_shape_with_output_layout(::xla::Shape* shape_with_output_layout); + + // .xla.DebugOptions debug_options = 4; + bool has_debug_options() const; + void clear_debug_options(); + static const int kDebugOptionsFieldNumber = 4; + private: + const ::xla::DebugOptions& _internal_debug_options() const; + public: + const ::xla::DebugOptions& debug_options() const; + ::xla::DebugOptions* release_debug_options(); + ::xla::DebugOptions* mutable_debug_options(); + void set_allocated_debug_options(::xla::DebugOptions* debug_options); + + // uint64 seed = 3; + void clear_seed(); + static const int kSeedFieldNumber = 3; + ::google::protobuf::uint64 seed() const; + void set_seed(::google::protobuf::uint64 value); + + // @@protoc_insertion_point(class_scope:xla.ExecutionOptions) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + ::google::protobuf::RepeatedPtrField< ::xla::DeviceHandle > device_handles_; + ::xla::Shape* shape_with_output_layout_; + ::xla::DebugOptions* debug_options_; + ::google::protobuf::uint64 seed_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class GetDeviceHandlesRequest : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.GetDeviceHandlesRequest) */ { + public: + GetDeviceHandlesRequest(); + virtual ~GetDeviceHandlesRequest(); + + GetDeviceHandlesRequest(const GetDeviceHandlesRequest& from); + + inline GetDeviceHandlesRequest& operator=(const GetDeviceHandlesRequest& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + GetDeviceHandlesRequest(GetDeviceHandlesRequest&& from) noexcept + : GetDeviceHandlesRequest() { + *this = ::std::move(from); + } + + inline GetDeviceHandlesRequest& operator=(GetDeviceHandlesRequest&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + static const ::google::protobuf::Descriptor* descriptor(); + static const GetDeviceHandlesRequest& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const GetDeviceHandlesRequest* internal_default_instance() { + return reinterpret_cast( + &_GetDeviceHandlesRequest_default_instance_); + } + static constexpr int kIndexInFileMessages = + 4; + + void Swap(GetDeviceHandlesRequest* other); + friend void swap(GetDeviceHandlesRequest& a, GetDeviceHandlesRequest& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline GetDeviceHandlesRequest* New() const final { + return CreateMaybeMessage(NULL); + } + + GetDeviceHandlesRequest* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const GetDeviceHandlesRequest& from); + void MergeFrom(const GetDeviceHandlesRequest& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(GetDeviceHandlesRequest* other); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return NULL; + } + inline void* MaybeArenaPtr() const { + return NULL; + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // int64 device_count = 1; + void clear_device_count(); + static const int kDeviceCountFieldNumber = 1; + ::google::protobuf::int64 device_count() const; + void set_device_count(::google::protobuf::int64 value); + + // @@protoc_insertion_point(class_scope:xla.GetDeviceHandlesRequest) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + ::google::protobuf::int64 device_count_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class GetDeviceHandlesResponse : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.GetDeviceHandlesResponse) */ { + public: + GetDeviceHandlesResponse(); + virtual ~GetDeviceHandlesResponse(); + + GetDeviceHandlesResponse(const GetDeviceHandlesResponse& from); + + inline GetDeviceHandlesResponse& operator=(const GetDeviceHandlesResponse& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + GetDeviceHandlesResponse(GetDeviceHandlesResponse&& from) noexcept + : GetDeviceHandlesResponse() { + *this = ::std::move(from); + } + + inline GetDeviceHandlesResponse& operator=(GetDeviceHandlesResponse&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + static const ::google::protobuf::Descriptor* descriptor(); + static const GetDeviceHandlesResponse& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const GetDeviceHandlesResponse* internal_default_instance() { + return reinterpret_cast( + &_GetDeviceHandlesResponse_default_instance_); + } + static constexpr int kIndexInFileMessages = + 5; + + void Swap(GetDeviceHandlesResponse* other); + friend void swap(GetDeviceHandlesResponse& a, GetDeviceHandlesResponse& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline GetDeviceHandlesResponse* New() const final { + return CreateMaybeMessage(NULL); + } + + GetDeviceHandlesResponse* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const GetDeviceHandlesResponse& from); + void MergeFrom(const GetDeviceHandlesResponse& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(GetDeviceHandlesResponse* other); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return NULL; + } + inline void* MaybeArenaPtr() const { + return NULL; + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated .xla.DeviceHandle device_handles = 1; + int device_handles_size() const; + void clear_device_handles(); + static const int kDeviceHandlesFieldNumber = 1; + ::xla::DeviceHandle* mutable_device_handles(int index); + ::google::protobuf::RepeatedPtrField< ::xla::DeviceHandle >* + mutable_device_handles(); + const ::xla::DeviceHandle& device_handles(int index) const; + ::xla::DeviceHandle* add_device_handles(); + const ::google::protobuf::RepeatedPtrField< ::xla::DeviceHandle >& + device_handles() const; + + // @@protoc_insertion_point(class_scope:xla.GetDeviceHandlesResponse) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + ::google::protobuf::RepeatedPtrField< ::xla::DeviceHandle > device_handles_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class TransferToClientRequest : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.TransferToClientRequest) */ { + public: + TransferToClientRequest(); + virtual ~TransferToClientRequest(); + + TransferToClientRequest(const TransferToClientRequest& from); + + inline TransferToClientRequest& operator=(const TransferToClientRequest& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + TransferToClientRequest(TransferToClientRequest&& from) noexcept + : TransferToClientRequest() { + *this = ::std::move(from); + } + + inline TransferToClientRequest& operator=(TransferToClientRequest&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + static const ::google::protobuf::Descriptor* descriptor(); + static const TransferToClientRequest& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const TransferToClientRequest* internal_default_instance() { + return reinterpret_cast( + &_TransferToClientRequest_default_instance_); + } + static constexpr int kIndexInFileMessages = + 6; + + void Swap(TransferToClientRequest* other); + friend void swap(TransferToClientRequest& a, TransferToClientRequest& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline TransferToClientRequest* New() const final { + return CreateMaybeMessage(NULL); + } + + TransferToClientRequest* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const TransferToClientRequest& from); + void MergeFrom(const TransferToClientRequest& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(TransferToClientRequest* other); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return NULL; + } + inline void* MaybeArenaPtr() const { + return NULL; + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // .xla.GlobalDataHandle data = 1; + bool has_data() const; + void clear_data(); + static const int kDataFieldNumber = 1; + private: + const ::xla::GlobalDataHandle& _internal_data() const; + public: + const ::xla::GlobalDataHandle& data() const; + ::xla::GlobalDataHandle* release_data(); + ::xla::GlobalDataHandle* mutable_data(); + void set_allocated_data(::xla::GlobalDataHandle* data); + + // .xla.Shape shape_with_layout = 2; + bool has_shape_with_layout() const; + void clear_shape_with_layout(); + static const int kShapeWithLayoutFieldNumber = 2; + private: + const ::xla::Shape& _internal_shape_with_layout() const; + public: + const ::xla::Shape& shape_with_layout() const; + ::xla::Shape* release_shape_with_layout(); + ::xla::Shape* mutable_shape_with_layout(); + void set_allocated_shape_with_layout(::xla::Shape* shape_with_layout); + + // @@protoc_insertion_point(class_scope:xla.TransferToClientRequest) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + ::xla::GlobalDataHandle* data_; + ::xla::Shape* shape_with_layout_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class TransferToClientResponse : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.TransferToClientResponse) */ { + public: + TransferToClientResponse(); + virtual ~TransferToClientResponse(); + + TransferToClientResponse(const TransferToClientResponse& from); + + inline TransferToClientResponse& operator=(const TransferToClientResponse& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + TransferToClientResponse(TransferToClientResponse&& from) noexcept + : TransferToClientResponse() { + *this = ::std::move(from); + } + + inline TransferToClientResponse& operator=(TransferToClientResponse&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + static const ::google::protobuf::Descriptor* descriptor(); + static const TransferToClientResponse& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const TransferToClientResponse* internal_default_instance() { + return reinterpret_cast( + &_TransferToClientResponse_default_instance_); + } + static constexpr int kIndexInFileMessages = + 7; + + void Swap(TransferToClientResponse* other); + friend void swap(TransferToClientResponse& a, TransferToClientResponse& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline TransferToClientResponse* New() const final { + return CreateMaybeMessage(NULL); + } + + TransferToClientResponse* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const TransferToClientResponse& from); + void MergeFrom(const TransferToClientResponse& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(TransferToClientResponse* other); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return NULL; + } + inline void* MaybeArenaPtr() const { + return NULL; + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // .xla.LiteralProto literal = 1; + bool has_literal() const; + void clear_literal(); + static const int kLiteralFieldNumber = 1; + private: + const ::xla::LiteralProto& _internal_literal() const; + public: + const ::xla::LiteralProto& literal() const; + ::xla::LiteralProto* release_literal(); + ::xla::LiteralProto* mutable_literal(); + void set_allocated_literal(::xla::LiteralProto* literal); + + // @@protoc_insertion_point(class_scope:xla.TransferToClientResponse) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + ::xla::LiteralProto* literal_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class TransferToServerRequest : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.TransferToServerRequest) */ { + public: + TransferToServerRequest(); + virtual ~TransferToServerRequest(); + + TransferToServerRequest(const TransferToServerRequest& from); + + inline TransferToServerRequest& operator=(const TransferToServerRequest& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + TransferToServerRequest(TransferToServerRequest&& from) noexcept + : TransferToServerRequest() { + *this = ::std::move(from); + } + + inline TransferToServerRequest& operator=(TransferToServerRequest&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + static const ::google::protobuf::Descriptor* descriptor(); + static const TransferToServerRequest& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const TransferToServerRequest* internal_default_instance() { + return reinterpret_cast( + &_TransferToServerRequest_default_instance_); + } + static constexpr int kIndexInFileMessages = + 8; + + void Swap(TransferToServerRequest* other); + friend void swap(TransferToServerRequest& a, TransferToServerRequest& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline TransferToServerRequest* New() const final { + return CreateMaybeMessage(NULL); + } + + TransferToServerRequest* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const TransferToServerRequest& from); + void MergeFrom(const TransferToServerRequest& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(TransferToServerRequest* other); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return NULL; + } + inline void* MaybeArenaPtr() const { + return NULL; + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // .xla.LiteralProto literal = 1; + bool has_literal() const; + void clear_literal(); + static const int kLiteralFieldNumber = 1; + private: + const ::xla::LiteralProto& _internal_literal() const; + public: + const ::xla::LiteralProto& literal() const; + ::xla::LiteralProto* release_literal(); + ::xla::LiteralProto* mutable_literal(); + void set_allocated_literal(::xla::LiteralProto* literal); + + // .xla.DeviceHandle device_handle = 2; + bool has_device_handle() const; + void clear_device_handle(); + static const int kDeviceHandleFieldNumber = 2; + private: + const ::xla::DeviceHandle& _internal_device_handle() const; + public: + const ::xla::DeviceHandle& device_handle() const; + ::xla::DeviceHandle* release_device_handle(); + ::xla::DeviceHandle* mutable_device_handle(); + void set_allocated_device_handle(::xla::DeviceHandle* device_handle); + + // @@protoc_insertion_point(class_scope:xla.TransferToServerRequest) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + ::xla::LiteralProto* literal_; + ::xla::DeviceHandle* device_handle_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class TransferToServerResponse : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.TransferToServerResponse) */ { + public: + TransferToServerResponse(); + virtual ~TransferToServerResponse(); + + TransferToServerResponse(const TransferToServerResponse& from); + + inline TransferToServerResponse& operator=(const TransferToServerResponse& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + TransferToServerResponse(TransferToServerResponse&& from) noexcept + : TransferToServerResponse() { + *this = ::std::move(from); + } + + inline TransferToServerResponse& operator=(TransferToServerResponse&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + static const ::google::protobuf::Descriptor* descriptor(); + static const TransferToServerResponse& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const TransferToServerResponse* internal_default_instance() { + return reinterpret_cast( + &_TransferToServerResponse_default_instance_); + } + static constexpr int kIndexInFileMessages = + 9; + + void Swap(TransferToServerResponse* other); + friend void swap(TransferToServerResponse& a, TransferToServerResponse& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline TransferToServerResponse* New() const final { + return CreateMaybeMessage(NULL); + } + + TransferToServerResponse* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const TransferToServerResponse& from); + void MergeFrom(const TransferToServerResponse& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(TransferToServerResponse* other); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return NULL; + } + inline void* MaybeArenaPtr() const { + return NULL; + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // .xla.GlobalDataHandle data = 1; + bool has_data() const; + void clear_data(); + static const int kDataFieldNumber = 1; + private: + const ::xla::GlobalDataHandle& _internal_data() const; + public: + const ::xla::GlobalDataHandle& data() const; + ::xla::GlobalDataHandle* release_data(); + ::xla::GlobalDataHandle* mutable_data(); + void set_allocated_data(::xla::GlobalDataHandle* data); + + // @@protoc_insertion_point(class_scope:xla.TransferToServerResponse) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + ::xla::GlobalDataHandle* data_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class TransferToInfeedRequest : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.TransferToInfeedRequest) */ { + public: + TransferToInfeedRequest(); + virtual ~TransferToInfeedRequest(); + + TransferToInfeedRequest(const TransferToInfeedRequest& from); + + inline TransferToInfeedRequest& operator=(const TransferToInfeedRequest& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + TransferToInfeedRequest(TransferToInfeedRequest&& from) noexcept + : TransferToInfeedRequest() { + *this = ::std::move(from); + } + + inline TransferToInfeedRequest& operator=(TransferToInfeedRequest&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + static const ::google::protobuf::Descriptor* descriptor(); + static const TransferToInfeedRequest& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const TransferToInfeedRequest* internal_default_instance() { + return reinterpret_cast( + &_TransferToInfeedRequest_default_instance_); + } + static constexpr int kIndexInFileMessages = + 10; + + void Swap(TransferToInfeedRequest* other); + friend void swap(TransferToInfeedRequest& a, TransferToInfeedRequest& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline TransferToInfeedRequest* New() const final { + return CreateMaybeMessage(NULL); + } + + TransferToInfeedRequest* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const TransferToInfeedRequest& from); + void MergeFrom(const TransferToInfeedRequest& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(TransferToInfeedRequest* other); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return NULL; + } + inline void* MaybeArenaPtr() const { + return NULL; + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // .xla.LiteralProto literal = 1; + bool has_literal() const; + void clear_literal(); + static const int kLiteralFieldNumber = 1; + private: + const ::xla::LiteralProto& _internal_literal() const; + public: + const ::xla::LiteralProto& literal() const; + ::xla::LiteralProto* release_literal(); + ::xla::LiteralProto* mutable_literal(); + void set_allocated_literal(::xla::LiteralProto* literal); + + // .xla.DeviceHandle device_handle = 3; + bool has_device_handle() const; + void clear_device_handle(); + static const int kDeviceHandleFieldNumber = 3; + private: + const ::xla::DeviceHandle& _internal_device_handle() const; + public: + const ::xla::DeviceHandle& device_handle() const; + ::xla::DeviceHandle* release_device_handle(); + ::xla::DeviceHandle* mutable_device_handle(); + void set_allocated_device_handle(::xla::DeviceHandle* device_handle); + + // int64 replica_id = 2; + void clear_replica_id(); + static const int kReplicaIdFieldNumber = 2; + ::google::protobuf::int64 replica_id() const; + void set_replica_id(::google::protobuf::int64 value); + + // @@protoc_insertion_point(class_scope:xla.TransferToInfeedRequest) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + ::xla::LiteralProto* literal_; + ::xla::DeviceHandle* device_handle_; + ::google::protobuf::int64 replica_id_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class TransferToInfeedResponse : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.TransferToInfeedResponse) */ { + public: + TransferToInfeedResponse(); + virtual ~TransferToInfeedResponse(); + + TransferToInfeedResponse(const TransferToInfeedResponse& from); + + inline TransferToInfeedResponse& operator=(const TransferToInfeedResponse& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + TransferToInfeedResponse(TransferToInfeedResponse&& from) noexcept + : TransferToInfeedResponse() { + *this = ::std::move(from); + } + + inline TransferToInfeedResponse& operator=(TransferToInfeedResponse&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + static const ::google::protobuf::Descriptor* descriptor(); + static const TransferToInfeedResponse& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const TransferToInfeedResponse* internal_default_instance() { + return reinterpret_cast( + &_TransferToInfeedResponse_default_instance_); + } + static constexpr int kIndexInFileMessages = + 11; + + void Swap(TransferToInfeedResponse* other); + friend void swap(TransferToInfeedResponse& a, TransferToInfeedResponse& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline TransferToInfeedResponse* New() const final { + return CreateMaybeMessage(NULL); + } + + TransferToInfeedResponse* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const TransferToInfeedResponse& from); + void MergeFrom(const TransferToInfeedResponse& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(TransferToInfeedResponse* other); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return NULL; + } + inline void* MaybeArenaPtr() const { + return NULL; + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // @@protoc_insertion_point(class_scope:xla.TransferToInfeedResponse) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class TransferFromOutfeedRequest : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.TransferFromOutfeedRequest) */ { + public: + TransferFromOutfeedRequest(); + virtual ~TransferFromOutfeedRequest(); + + TransferFromOutfeedRequest(const TransferFromOutfeedRequest& from); + + inline TransferFromOutfeedRequest& operator=(const TransferFromOutfeedRequest& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + TransferFromOutfeedRequest(TransferFromOutfeedRequest&& from) noexcept + : TransferFromOutfeedRequest() { + *this = ::std::move(from); + } + + inline TransferFromOutfeedRequest& operator=(TransferFromOutfeedRequest&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + static const ::google::protobuf::Descriptor* descriptor(); + static const TransferFromOutfeedRequest& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const TransferFromOutfeedRequest* internal_default_instance() { + return reinterpret_cast( + &_TransferFromOutfeedRequest_default_instance_); + } + static constexpr int kIndexInFileMessages = + 12; + + void Swap(TransferFromOutfeedRequest* other); + friend void swap(TransferFromOutfeedRequest& a, TransferFromOutfeedRequest& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline TransferFromOutfeedRequest* New() const final { + return CreateMaybeMessage(NULL); + } + + TransferFromOutfeedRequest* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const TransferFromOutfeedRequest& from); + void MergeFrom(const TransferFromOutfeedRequest& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(TransferFromOutfeedRequest* other); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return NULL; + } + inline void* MaybeArenaPtr() const { + return NULL; + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // .xla.Shape shape_with_layout = 1; + bool has_shape_with_layout() const; + void clear_shape_with_layout(); + static const int kShapeWithLayoutFieldNumber = 1; + private: + const ::xla::Shape& _internal_shape_with_layout() const; + public: + const ::xla::Shape& shape_with_layout() const; + ::xla::Shape* release_shape_with_layout(); + ::xla::Shape* mutable_shape_with_layout(); + void set_allocated_shape_with_layout(::xla::Shape* shape_with_layout); + + // .xla.DeviceHandle device_handle = 3; + bool has_device_handle() const; + void clear_device_handle(); + static const int kDeviceHandleFieldNumber = 3; + private: + const ::xla::DeviceHandle& _internal_device_handle() const; + public: + const ::xla::DeviceHandle& device_handle() const; + ::xla::DeviceHandle* release_device_handle(); + ::xla::DeviceHandle* mutable_device_handle(); + void set_allocated_device_handle(::xla::DeviceHandle* device_handle); + + // int64 replica_id = 2; + void clear_replica_id(); + static const int kReplicaIdFieldNumber = 2; + ::google::protobuf::int64 replica_id() const; + void set_replica_id(::google::protobuf::int64 value); + + // @@protoc_insertion_point(class_scope:xla.TransferFromOutfeedRequest) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + ::xla::Shape* shape_with_layout_; + ::xla::DeviceHandle* device_handle_; + ::google::protobuf::int64 replica_id_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class TransferFromOutfeedResponse : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.TransferFromOutfeedResponse) */ { + public: + TransferFromOutfeedResponse(); + virtual ~TransferFromOutfeedResponse(); + + TransferFromOutfeedResponse(const TransferFromOutfeedResponse& from); + + inline TransferFromOutfeedResponse& operator=(const TransferFromOutfeedResponse& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + TransferFromOutfeedResponse(TransferFromOutfeedResponse&& from) noexcept + : TransferFromOutfeedResponse() { + *this = ::std::move(from); + } + + inline TransferFromOutfeedResponse& operator=(TransferFromOutfeedResponse&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + static const ::google::protobuf::Descriptor* descriptor(); + static const TransferFromOutfeedResponse& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const TransferFromOutfeedResponse* internal_default_instance() { + return reinterpret_cast( + &_TransferFromOutfeedResponse_default_instance_); + } + static constexpr int kIndexInFileMessages = + 13; + + void Swap(TransferFromOutfeedResponse* other); + friend void swap(TransferFromOutfeedResponse& a, TransferFromOutfeedResponse& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline TransferFromOutfeedResponse* New() const final { + return CreateMaybeMessage(NULL); + } + + TransferFromOutfeedResponse* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const TransferFromOutfeedResponse& from); + void MergeFrom(const TransferFromOutfeedResponse& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(TransferFromOutfeedResponse* other); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return NULL; + } + inline void* MaybeArenaPtr() const { + return NULL; + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // .xla.LiteralProto literal = 1; + bool has_literal() const; + void clear_literal(); + static const int kLiteralFieldNumber = 1; + private: + const ::xla::LiteralProto& _internal_literal() const; + public: + const ::xla::LiteralProto& literal() const; + ::xla::LiteralProto* release_literal(); + ::xla::LiteralProto* mutable_literal(); + void set_allocated_literal(::xla::LiteralProto* literal); + + // @@protoc_insertion_point(class_scope:xla.TransferFromOutfeedResponse) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + ::xla::LiteralProto* literal_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class ResetDeviceRequest : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.ResetDeviceRequest) */ { + public: + ResetDeviceRequest(); + virtual ~ResetDeviceRequest(); + + ResetDeviceRequest(const ResetDeviceRequest& from); + + inline ResetDeviceRequest& operator=(const ResetDeviceRequest& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + ResetDeviceRequest(ResetDeviceRequest&& from) noexcept + : ResetDeviceRequest() { + *this = ::std::move(from); + } + + inline ResetDeviceRequest& operator=(ResetDeviceRequest&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + static const ::google::protobuf::Descriptor* descriptor(); + static const ResetDeviceRequest& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const ResetDeviceRequest* internal_default_instance() { + return reinterpret_cast( + &_ResetDeviceRequest_default_instance_); + } + static constexpr int kIndexInFileMessages = + 14; + + void Swap(ResetDeviceRequest* other); + friend void swap(ResetDeviceRequest& a, ResetDeviceRequest& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline ResetDeviceRequest* New() const final { + return CreateMaybeMessage(NULL); + } + + ResetDeviceRequest* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const ResetDeviceRequest& from); + void MergeFrom(const ResetDeviceRequest& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(ResetDeviceRequest* other); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return NULL; + } + inline void* MaybeArenaPtr() const { + return NULL; + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // .xla.DeviceHandle device_handle = 1; + bool has_device_handle() const; + void clear_device_handle(); + static const int kDeviceHandleFieldNumber = 1; + private: + const ::xla::DeviceHandle& _internal_device_handle() const; + public: + const ::xla::DeviceHandle& device_handle() const; + ::xla::DeviceHandle* release_device_handle(); + ::xla::DeviceHandle* mutable_device_handle(); + void set_allocated_device_handle(::xla::DeviceHandle* device_handle); + + // @@protoc_insertion_point(class_scope:xla.ResetDeviceRequest) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + ::xla::DeviceHandle* device_handle_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class ResetDeviceResponse : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.ResetDeviceResponse) */ { + public: + ResetDeviceResponse(); + virtual ~ResetDeviceResponse(); + + ResetDeviceResponse(const ResetDeviceResponse& from); + + inline ResetDeviceResponse& operator=(const ResetDeviceResponse& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + ResetDeviceResponse(ResetDeviceResponse&& from) noexcept + : ResetDeviceResponse() { + *this = ::std::move(from); + } + + inline ResetDeviceResponse& operator=(ResetDeviceResponse&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + static const ::google::protobuf::Descriptor* descriptor(); + static const ResetDeviceResponse& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const ResetDeviceResponse* internal_default_instance() { + return reinterpret_cast( + &_ResetDeviceResponse_default_instance_); + } + static constexpr int kIndexInFileMessages = + 15; + + void Swap(ResetDeviceResponse* other); + friend void swap(ResetDeviceResponse& a, ResetDeviceResponse& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline ResetDeviceResponse* New() const final { + return CreateMaybeMessage(NULL); + } + + ResetDeviceResponse* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const ResetDeviceResponse& from); + void MergeFrom(const ResetDeviceResponse& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(ResetDeviceResponse* other); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return NULL; + } + inline void* MaybeArenaPtr() const { + return NULL; + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // @@protoc_insertion_point(class_scope:xla.ResetDeviceResponse) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class ComputationGraphStatsRequest : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.ComputationGraphStatsRequest) */ { + public: + ComputationGraphStatsRequest(); + virtual ~ComputationGraphStatsRequest(); + + ComputationGraphStatsRequest(const ComputationGraphStatsRequest& from); + + inline ComputationGraphStatsRequest& operator=(const ComputationGraphStatsRequest& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + ComputationGraphStatsRequest(ComputationGraphStatsRequest&& from) noexcept + : ComputationGraphStatsRequest() { + *this = ::std::move(from); + } + + inline ComputationGraphStatsRequest& operator=(ComputationGraphStatsRequest&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + static const ::google::protobuf::Descriptor* descriptor(); + static const ComputationGraphStatsRequest& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const ComputationGraphStatsRequest* internal_default_instance() { + return reinterpret_cast( + &_ComputationGraphStatsRequest_default_instance_); + } + static constexpr int kIndexInFileMessages = + 16; + + void Swap(ComputationGraphStatsRequest* other); + friend void swap(ComputationGraphStatsRequest& a, ComputationGraphStatsRequest& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline ComputationGraphStatsRequest* New() const final { + return CreateMaybeMessage(NULL); + } + + ComputationGraphStatsRequest* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const ComputationGraphStatsRequest& from); + void MergeFrom(const ComputationGraphStatsRequest& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(ComputationGraphStatsRequest* other); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return NULL; + } + inline void* MaybeArenaPtr() const { + return NULL; + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // .xla.HloModuleProto computation = 1; + bool has_computation() const; + void clear_computation(); + static const int kComputationFieldNumber = 1; + private: + const ::xla::HloModuleProto& _internal_computation() const; + public: + const ::xla::HloModuleProto& computation() const; + ::xla::HloModuleProto* release_computation(); + ::xla::HloModuleProto* mutable_computation(); + void set_allocated_computation(::xla::HloModuleProto* computation); + + // .xla.DebugOptions debug_options = 2; + bool has_debug_options() const; + void clear_debug_options(); + static const int kDebugOptionsFieldNumber = 2; + private: + const ::xla::DebugOptions& _internal_debug_options() const; + public: + const ::xla::DebugOptions& debug_options() const; + ::xla::DebugOptions* release_debug_options(); + ::xla::DebugOptions* mutable_debug_options(); + void set_allocated_debug_options(::xla::DebugOptions* debug_options); + + // @@protoc_insertion_point(class_scope:xla.ComputationGraphStatsRequest) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + ::xla::HloModuleProto* computation_; + ::xla::DebugOptions* debug_options_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class ComputationStatsResponse : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.ComputationStatsResponse) */ { + public: + ComputationStatsResponse(); + virtual ~ComputationStatsResponse(); + + ComputationStatsResponse(const ComputationStatsResponse& from); + + inline ComputationStatsResponse& operator=(const ComputationStatsResponse& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + ComputationStatsResponse(ComputationStatsResponse&& from) noexcept + : ComputationStatsResponse() { + *this = ::std::move(from); + } + + inline ComputationStatsResponse& operator=(ComputationStatsResponse&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + static const ::google::protobuf::Descriptor* descriptor(); + static const ComputationStatsResponse& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const ComputationStatsResponse* internal_default_instance() { + return reinterpret_cast( + &_ComputationStatsResponse_default_instance_); + } + static constexpr int kIndexInFileMessages = + 17; + + void Swap(ComputationStatsResponse* other); + friend void swap(ComputationStatsResponse& a, ComputationStatsResponse& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline ComputationStatsResponse* New() const final { + return CreateMaybeMessage(NULL); + } + + ComputationStatsResponse* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const ComputationStatsResponse& from); + void MergeFrom(const ComputationStatsResponse& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(ComputationStatsResponse* other); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return NULL; + } + inline void* MaybeArenaPtr() const { + return NULL; + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // .xla.ComputationStats stats = 1; + bool has_stats() const; + void clear_stats(); + static const int kStatsFieldNumber = 1; + private: + const ::xla::ComputationStats& _internal_stats() const; + public: + const ::xla::ComputationStats& stats() const; + ::xla::ComputationStats* release_stats(); + ::xla::ComputationStats* mutable_stats(); + void set_allocated_stats(::xla::ComputationStats* stats); + + // @@protoc_insertion_point(class_scope:xla.ComputationStatsResponse) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + ::xla::ComputationStats* stats_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class CreateChannelHandleRequest : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.CreateChannelHandleRequest) */ { + public: + CreateChannelHandleRequest(); + virtual ~CreateChannelHandleRequest(); + + CreateChannelHandleRequest(const CreateChannelHandleRequest& from); + + inline CreateChannelHandleRequest& operator=(const CreateChannelHandleRequest& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + CreateChannelHandleRequest(CreateChannelHandleRequest&& from) noexcept + : CreateChannelHandleRequest() { + *this = ::std::move(from); + } + + inline CreateChannelHandleRequest& operator=(CreateChannelHandleRequest&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + static const ::google::protobuf::Descriptor* descriptor(); + static const CreateChannelHandleRequest& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const CreateChannelHandleRequest* internal_default_instance() { + return reinterpret_cast( + &_CreateChannelHandleRequest_default_instance_); + } + static constexpr int kIndexInFileMessages = + 18; + + void Swap(CreateChannelHandleRequest* other); + friend void swap(CreateChannelHandleRequest& a, CreateChannelHandleRequest& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline CreateChannelHandleRequest* New() const final { + return CreateMaybeMessage(NULL); + } + + CreateChannelHandleRequest* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const CreateChannelHandleRequest& from); + void MergeFrom(const CreateChannelHandleRequest& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(CreateChannelHandleRequest* other); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return NULL; + } + inline void* MaybeArenaPtr() const { + return NULL; + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // .xla.ChannelHandle.ChannelType channel_type = 1; + void clear_channel_type(); + static const int kChannelTypeFieldNumber = 1; + ::xla::ChannelHandle_ChannelType channel_type() const; + void set_channel_type(::xla::ChannelHandle_ChannelType value); + + // @@protoc_insertion_point(class_scope:xla.CreateChannelHandleRequest) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + int channel_type_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class CreateChannelHandleResponse : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.CreateChannelHandleResponse) */ { + public: + CreateChannelHandleResponse(); + virtual ~CreateChannelHandleResponse(); + + CreateChannelHandleResponse(const CreateChannelHandleResponse& from); + + inline CreateChannelHandleResponse& operator=(const CreateChannelHandleResponse& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + CreateChannelHandleResponse(CreateChannelHandleResponse&& from) noexcept + : CreateChannelHandleResponse() { + *this = ::std::move(from); + } + + inline CreateChannelHandleResponse& operator=(CreateChannelHandleResponse&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + static const ::google::protobuf::Descriptor* descriptor(); + static const CreateChannelHandleResponse& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const CreateChannelHandleResponse* internal_default_instance() { + return reinterpret_cast( + &_CreateChannelHandleResponse_default_instance_); + } + static constexpr int kIndexInFileMessages = + 19; + + void Swap(CreateChannelHandleResponse* other); + friend void swap(CreateChannelHandleResponse& a, CreateChannelHandleResponse& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline CreateChannelHandleResponse* New() const final { + return CreateMaybeMessage(NULL); + } + + CreateChannelHandleResponse* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const CreateChannelHandleResponse& from); + void MergeFrom(const CreateChannelHandleResponse& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(CreateChannelHandleResponse* other); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return NULL; + } + inline void* MaybeArenaPtr() const { + return NULL; + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // .xla.ChannelHandle channel = 1; + bool has_channel() const; + void clear_channel(); + static const int kChannelFieldNumber = 1; + private: + const ::xla::ChannelHandle& _internal_channel() const; + public: + const ::xla::ChannelHandle& channel() const; + ::xla::ChannelHandle* release_channel(); + ::xla::ChannelHandle* mutable_channel(); + void set_allocated_channel(::xla::ChannelHandle* channel); + + // @@protoc_insertion_point(class_scope:xla.CreateChannelHandleResponse) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + ::xla::ChannelHandle* channel_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class UnregisterRequest : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.UnregisterRequest) */ { + public: + UnregisterRequest(); + virtual ~UnregisterRequest(); + + UnregisterRequest(const UnregisterRequest& from); + + inline UnregisterRequest& operator=(const UnregisterRequest& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + UnregisterRequest(UnregisterRequest&& from) noexcept + : UnregisterRequest() { + *this = ::std::move(from); + } + + inline UnregisterRequest& operator=(UnregisterRequest&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + static const ::google::protobuf::Descriptor* descriptor(); + static const UnregisterRequest& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const UnregisterRequest* internal_default_instance() { + return reinterpret_cast( + &_UnregisterRequest_default_instance_); + } + static constexpr int kIndexInFileMessages = + 20; + + void Swap(UnregisterRequest* other); + friend void swap(UnregisterRequest& a, UnregisterRequest& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline UnregisterRequest* New() const final { + return CreateMaybeMessage(NULL); + } + + UnregisterRequest* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const UnregisterRequest& from); + void MergeFrom(const UnregisterRequest& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(UnregisterRequest* other); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return NULL; + } + inline void* MaybeArenaPtr() const { + return NULL; + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // .xla.GlobalDataHandle data = 1; + bool has_data() const; + void clear_data(); + static const int kDataFieldNumber = 1; + private: + const ::xla::GlobalDataHandle& _internal_data() const; + public: + const ::xla::GlobalDataHandle& data() const; + ::xla::GlobalDataHandle* release_data(); + ::xla::GlobalDataHandle* mutable_data(); + void set_allocated_data(::xla::GlobalDataHandle* data); + + // @@protoc_insertion_point(class_scope:xla.UnregisterRequest) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + ::xla::GlobalDataHandle* data_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class UnregisterResponse : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.UnregisterResponse) */ { + public: + UnregisterResponse(); + virtual ~UnregisterResponse(); + + UnregisterResponse(const UnregisterResponse& from); + + inline UnregisterResponse& operator=(const UnregisterResponse& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + UnregisterResponse(UnregisterResponse&& from) noexcept + : UnregisterResponse() { + *this = ::std::move(from); + } + + inline UnregisterResponse& operator=(UnregisterResponse&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + static const ::google::protobuf::Descriptor* descriptor(); + static const UnregisterResponse& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const UnregisterResponse* internal_default_instance() { + return reinterpret_cast( + &_UnregisterResponse_default_instance_); + } + static constexpr int kIndexInFileMessages = + 21; + + void Swap(UnregisterResponse* other); + friend void swap(UnregisterResponse& a, UnregisterResponse& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline UnregisterResponse* New() const final { + return CreateMaybeMessage(NULL); + } + + UnregisterResponse* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const UnregisterResponse& from); + void MergeFrom(const UnregisterResponse& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(UnregisterResponse* other); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return NULL; + } + inline void* MaybeArenaPtr() const { + return NULL; + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // @@protoc_insertion_point(class_scope:xla.UnregisterResponse) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class ExecuteGraphRequest : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.ExecuteGraphRequest) */ { + public: + ExecuteGraphRequest(); + virtual ~ExecuteGraphRequest(); + + ExecuteGraphRequest(const ExecuteGraphRequest& from); + + inline ExecuteGraphRequest& operator=(const ExecuteGraphRequest& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + ExecuteGraphRequest(ExecuteGraphRequest&& from) noexcept + : ExecuteGraphRequest() { + *this = ::std::move(from); + } + + inline ExecuteGraphRequest& operator=(ExecuteGraphRequest&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + static const ::google::protobuf::Descriptor* descriptor(); + static const ExecuteGraphRequest& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const ExecuteGraphRequest* internal_default_instance() { + return reinterpret_cast( + &_ExecuteGraphRequest_default_instance_); + } + static constexpr int kIndexInFileMessages = + 22; + + void Swap(ExecuteGraphRequest* other); + friend void swap(ExecuteGraphRequest& a, ExecuteGraphRequest& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline ExecuteGraphRequest* New() const final { + return CreateMaybeMessage(NULL); + } + + ExecuteGraphRequest* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const ExecuteGraphRequest& from); + void MergeFrom(const ExecuteGraphRequest& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(ExecuteGraphRequest* other); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return NULL; + } + inline void* MaybeArenaPtr() const { + return NULL; + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated .xla.GlobalDataHandle arguments = 2; + int arguments_size() const; + void clear_arguments(); + static const int kArgumentsFieldNumber = 2; + ::xla::GlobalDataHandle* mutable_arguments(int index); + ::google::protobuf::RepeatedPtrField< ::xla::GlobalDataHandle >* + mutable_arguments(); + const ::xla::GlobalDataHandle& arguments(int index) const; + ::xla::GlobalDataHandle* add_arguments(); + const ::google::protobuf::RepeatedPtrField< ::xla::GlobalDataHandle >& + arguments() const; + + // .xla.HloModuleProto computation = 1; + bool has_computation() const; + void clear_computation(); + static const int kComputationFieldNumber = 1; + private: + const ::xla::HloModuleProto& _internal_computation() const; + public: + const ::xla::HloModuleProto& computation() const; + ::xla::HloModuleProto* release_computation(); + ::xla::HloModuleProto* mutable_computation(); + void set_allocated_computation(::xla::HloModuleProto* computation); + + // .xla.ExecutionOptions execution_options = 3; + bool has_execution_options() const; + void clear_execution_options(); + static const int kExecutionOptionsFieldNumber = 3; + private: + const ::xla::ExecutionOptions& _internal_execution_options() const; + public: + const ::xla::ExecutionOptions& execution_options() const; + ::xla::ExecutionOptions* release_execution_options(); + ::xla::ExecutionOptions* mutable_execution_options(); + void set_allocated_execution_options(::xla::ExecutionOptions* execution_options); + + // @@protoc_insertion_point(class_scope:xla.ExecuteGraphRequest) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + ::google::protobuf::RepeatedPtrField< ::xla::GlobalDataHandle > arguments_; + ::xla::HloModuleProto* computation_; + ::xla::ExecutionOptions* execution_options_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class ExecuteGraphParallelRequest : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.ExecuteGraphParallelRequest) */ { + public: + ExecuteGraphParallelRequest(); + virtual ~ExecuteGraphParallelRequest(); + + ExecuteGraphParallelRequest(const ExecuteGraphParallelRequest& from); + + inline ExecuteGraphParallelRequest& operator=(const ExecuteGraphParallelRequest& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + ExecuteGraphParallelRequest(ExecuteGraphParallelRequest&& from) noexcept + : ExecuteGraphParallelRequest() { + *this = ::std::move(from); + } + + inline ExecuteGraphParallelRequest& operator=(ExecuteGraphParallelRequest&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + static const ::google::protobuf::Descriptor* descriptor(); + static const ExecuteGraphParallelRequest& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const ExecuteGraphParallelRequest* internal_default_instance() { + return reinterpret_cast( + &_ExecuteGraphParallelRequest_default_instance_); + } + static constexpr int kIndexInFileMessages = + 23; + + void Swap(ExecuteGraphParallelRequest* other); + friend void swap(ExecuteGraphParallelRequest& a, ExecuteGraphParallelRequest& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline ExecuteGraphParallelRequest* New() const final { + return CreateMaybeMessage(NULL); + } + + ExecuteGraphParallelRequest* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const ExecuteGraphParallelRequest& from); + void MergeFrom(const ExecuteGraphParallelRequest& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(ExecuteGraphParallelRequest* other); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return NULL; + } + inline void* MaybeArenaPtr() const { + return NULL; + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated .xla.ExecuteGraphRequest requests = 1; + int requests_size() const; + void clear_requests(); + static const int kRequestsFieldNumber = 1; + ::xla::ExecuteGraphRequest* mutable_requests(int index); + ::google::protobuf::RepeatedPtrField< ::xla::ExecuteGraphRequest >* + mutable_requests(); + const ::xla::ExecuteGraphRequest& requests(int index) const; + ::xla::ExecuteGraphRequest* add_requests(); + const ::google::protobuf::RepeatedPtrField< ::xla::ExecuteGraphRequest >& + requests() const; + + // @@protoc_insertion_point(class_scope:xla.ExecuteGraphParallelRequest) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + ::google::protobuf::RepeatedPtrField< ::xla::ExecuteGraphRequest > requests_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class ExecuteResponse : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.ExecuteResponse) */ { + public: + ExecuteResponse(); + virtual ~ExecuteResponse(); + + ExecuteResponse(const ExecuteResponse& from); + + inline ExecuteResponse& operator=(const ExecuteResponse& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + ExecuteResponse(ExecuteResponse&& from) noexcept + : ExecuteResponse() { + *this = ::std::move(from); + } + + inline ExecuteResponse& operator=(ExecuteResponse&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + static const ::google::protobuf::Descriptor* descriptor(); + static const ExecuteResponse& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const ExecuteResponse* internal_default_instance() { + return reinterpret_cast( + &_ExecuteResponse_default_instance_); + } + static constexpr int kIndexInFileMessages = + 24; + + void Swap(ExecuteResponse* other); + friend void swap(ExecuteResponse& a, ExecuteResponse& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline ExecuteResponse* New() const final { + return CreateMaybeMessage(NULL); + } + + ExecuteResponse* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const ExecuteResponse& from); + void MergeFrom(const ExecuteResponse& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(ExecuteResponse* other); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return NULL; + } + inline void* MaybeArenaPtr() const { + return NULL; + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // .xla.GlobalDataHandle output = 1; + bool has_output() const; + void clear_output(); + static const int kOutputFieldNumber = 1; + private: + const ::xla::GlobalDataHandle& _internal_output() const; + public: + const ::xla::GlobalDataHandle& output() const; + ::xla::GlobalDataHandle* release_output(); + ::xla::GlobalDataHandle* mutable_output(); + void set_allocated_output(::xla::GlobalDataHandle* output); + + // .xla.ExecutionProfile profile = 2; + bool has_profile() const; + void clear_profile(); + static const int kProfileFieldNumber = 2; + private: + const ::xla::ExecutionProfile& _internal_profile() const; + public: + const ::xla::ExecutionProfile& profile() const; + ::xla::ExecutionProfile* release_profile(); + ::xla::ExecutionProfile* mutable_profile(); + void set_allocated_profile(::xla::ExecutionProfile* profile); + + // @@protoc_insertion_point(class_scope:xla.ExecuteResponse) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + ::xla::GlobalDataHandle* output_; + ::xla::ExecutionProfile* profile_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class ExecuteParallelResponse : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.ExecuteParallelResponse) */ { + public: + ExecuteParallelResponse(); + virtual ~ExecuteParallelResponse(); + + ExecuteParallelResponse(const ExecuteParallelResponse& from); + + inline ExecuteParallelResponse& operator=(const ExecuteParallelResponse& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + ExecuteParallelResponse(ExecuteParallelResponse&& from) noexcept + : ExecuteParallelResponse() { + *this = ::std::move(from); + } + + inline ExecuteParallelResponse& operator=(ExecuteParallelResponse&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + static const ::google::protobuf::Descriptor* descriptor(); + static const ExecuteParallelResponse& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const ExecuteParallelResponse* internal_default_instance() { + return reinterpret_cast( + &_ExecuteParallelResponse_default_instance_); + } + static constexpr int kIndexInFileMessages = + 25; + + void Swap(ExecuteParallelResponse* other); + friend void swap(ExecuteParallelResponse& a, ExecuteParallelResponse& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline ExecuteParallelResponse* New() const final { + return CreateMaybeMessage(NULL); + } + + ExecuteParallelResponse* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const ExecuteParallelResponse& from); + void MergeFrom(const ExecuteParallelResponse& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(ExecuteParallelResponse* other); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return NULL; + } + inline void* MaybeArenaPtr() const { + return NULL; + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated .xla.ExecuteResponse responses = 1; + int responses_size() const; + void clear_responses(); + static const int kResponsesFieldNumber = 1; + ::xla::ExecuteResponse* mutable_responses(int index); + ::google::protobuf::RepeatedPtrField< ::xla::ExecuteResponse >* + mutable_responses(); + const ::xla::ExecuteResponse& responses(int index) const; + ::xla::ExecuteResponse* add_responses(); + const ::google::protobuf::RepeatedPtrField< ::xla::ExecuteResponse >& + responses() const; + + // @@protoc_insertion_point(class_scope:xla.ExecuteParallelResponse) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + ::google::protobuf::RepeatedPtrField< ::xla::ExecuteResponse > responses_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class WaitForExecutionRequest : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.WaitForExecutionRequest) */ { + public: + WaitForExecutionRequest(); + virtual ~WaitForExecutionRequest(); + + WaitForExecutionRequest(const WaitForExecutionRequest& from); + + inline WaitForExecutionRequest& operator=(const WaitForExecutionRequest& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + WaitForExecutionRequest(WaitForExecutionRequest&& from) noexcept + : WaitForExecutionRequest() { + *this = ::std::move(from); + } + + inline WaitForExecutionRequest& operator=(WaitForExecutionRequest&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + static const ::google::protobuf::Descriptor* descriptor(); + static const WaitForExecutionRequest& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const WaitForExecutionRequest* internal_default_instance() { + return reinterpret_cast( + &_WaitForExecutionRequest_default_instance_); + } + static constexpr int kIndexInFileMessages = + 26; + + void Swap(WaitForExecutionRequest* other); + friend void swap(WaitForExecutionRequest& a, WaitForExecutionRequest& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline WaitForExecutionRequest* New() const final { + return CreateMaybeMessage(NULL); + } + + WaitForExecutionRequest* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const WaitForExecutionRequest& from); + void MergeFrom(const WaitForExecutionRequest& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(WaitForExecutionRequest* other); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return NULL; + } + inline void* MaybeArenaPtr() const { + return NULL; + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // .xla.ExecutionHandle execution = 1; + bool has_execution() const; + void clear_execution(); + static const int kExecutionFieldNumber = 1; + private: + const ::xla::ExecutionHandle& _internal_execution() const; + public: + const ::xla::ExecutionHandle& execution() const; + ::xla::ExecutionHandle* release_execution(); + ::xla::ExecutionHandle* mutable_execution(); + void set_allocated_execution(::xla::ExecutionHandle* execution); + + // @@protoc_insertion_point(class_scope:xla.WaitForExecutionRequest) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + ::xla::ExecutionHandle* execution_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class WaitForExecutionResponse : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.WaitForExecutionResponse) */ { + public: + WaitForExecutionResponse(); + virtual ~WaitForExecutionResponse(); + + WaitForExecutionResponse(const WaitForExecutionResponse& from); + + inline WaitForExecutionResponse& operator=(const WaitForExecutionResponse& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + WaitForExecutionResponse(WaitForExecutionResponse&& from) noexcept + : WaitForExecutionResponse() { + *this = ::std::move(from); + } + + inline WaitForExecutionResponse& operator=(WaitForExecutionResponse&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + static const ::google::protobuf::Descriptor* descriptor(); + static const WaitForExecutionResponse& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const WaitForExecutionResponse* internal_default_instance() { + return reinterpret_cast( + &_WaitForExecutionResponse_default_instance_); + } + static constexpr int kIndexInFileMessages = + 27; + + void Swap(WaitForExecutionResponse* other); + friend void swap(WaitForExecutionResponse& a, WaitForExecutionResponse& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline WaitForExecutionResponse* New() const final { + return CreateMaybeMessage(NULL); + } + + WaitForExecutionResponse* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const WaitForExecutionResponse& from); + void MergeFrom(const WaitForExecutionResponse& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(WaitForExecutionResponse* other); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return NULL; + } + inline void* MaybeArenaPtr() const { + return NULL; + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // .xla.GlobalDataHandle output = 1; + bool has_output() const; + void clear_output(); + static const int kOutputFieldNumber = 1; + private: + const ::xla::GlobalDataHandle& _internal_output() const; + public: + const ::xla::GlobalDataHandle& output() const; + ::xla::GlobalDataHandle* release_output(); + ::xla::GlobalDataHandle* mutable_output(); + void set_allocated_output(::xla::GlobalDataHandle* output); + + // .xla.ExecutionProfile profile = 2; + bool has_profile() const; + void clear_profile(); + static const int kProfileFieldNumber = 2; + private: + const ::xla::ExecutionProfile& _internal_profile() const; + public: + const ::xla::ExecutionProfile& profile() const; + ::xla::ExecutionProfile* release_profile(); + ::xla::ExecutionProfile* mutable_profile(); + void set_allocated_profile(::xla::ExecutionProfile* profile); + + // @@protoc_insertion_point(class_scope:xla.WaitForExecutionResponse) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + ::xla::GlobalDataHandle* output_; + ::xla::ExecutionProfile* profile_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class ComputeConstantGraphRequest : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.ComputeConstantGraphRequest) */ { + public: + ComputeConstantGraphRequest(); + virtual ~ComputeConstantGraphRequest(); + + ComputeConstantGraphRequest(const ComputeConstantGraphRequest& from); + + inline ComputeConstantGraphRequest& operator=(const ComputeConstantGraphRequest& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + ComputeConstantGraphRequest(ComputeConstantGraphRequest&& from) noexcept + : ComputeConstantGraphRequest() { + *this = ::std::move(from); + } + + inline ComputeConstantGraphRequest& operator=(ComputeConstantGraphRequest&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + static const ::google::protobuf::Descriptor* descriptor(); + static const ComputeConstantGraphRequest& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const ComputeConstantGraphRequest* internal_default_instance() { + return reinterpret_cast( + &_ComputeConstantGraphRequest_default_instance_); + } + static constexpr int kIndexInFileMessages = + 28; + + void Swap(ComputeConstantGraphRequest* other); + friend void swap(ComputeConstantGraphRequest& a, ComputeConstantGraphRequest& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline ComputeConstantGraphRequest* New() const final { + return CreateMaybeMessage(NULL); + } + + ComputeConstantGraphRequest* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const ComputeConstantGraphRequest& from); + void MergeFrom(const ComputeConstantGraphRequest& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(ComputeConstantGraphRequest* other); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return NULL; + } + inline void* MaybeArenaPtr() const { + return NULL; + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // .xla.HloModuleProto computation = 1; + bool has_computation() const; + void clear_computation(); + static const int kComputationFieldNumber = 1; + private: + const ::xla::HloModuleProto& _internal_computation() const; + public: + const ::xla::HloModuleProto& computation() const; + ::xla::HloModuleProto* release_computation(); + ::xla::HloModuleProto* mutable_computation(); + void set_allocated_computation(::xla::HloModuleProto* computation); + + // .xla.Layout output_layout = 2; + bool has_output_layout() const; + void clear_output_layout(); + static const int kOutputLayoutFieldNumber = 2; + private: + const ::xla::Layout& _internal_output_layout() const; + public: + const ::xla::Layout& output_layout() const; + ::xla::Layout* release_output_layout(); + ::xla::Layout* mutable_output_layout(); + void set_allocated_output_layout(::xla::Layout* output_layout); + + // @@protoc_insertion_point(class_scope:xla.ComputeConstantGraphRequest) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + ::xla::HloModuleProto* computation_; + ::xla::Layout* output_layout_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class ComputeConstantResponse : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.ComputeConstantResponse) */ { + public: + ComputeConstantResponse(); + virtual ~ComputeConstantResponse(); + + ComputeConstantResponse(const ComputeConstantResponse& from); + + inline ComputeConstantResponse& operator=(const ComputeConstantResponse& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + ComputeConstantResponse(ComputeConstantResponse&& from) noexcept + : ComputeConstantResponse() { + *this = ::std::move(from); + } + + inline ComputeConstantResponse& operator=(ComputeConstantResponse&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + static const ::google::protobuf::Descriptor* descriptor(); + static const ComputeConstantResponse& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const ComputeConstantResponse* internal_default_instance() { + return reinterpret_cast( + &_ComputeConstantResponse_default_instance_); + } + static constexpr int kIndexInFileMessages = + 29; + + void Swap(ComputeConstantResponse* other); + friend void swap(ComputeConstantResponse& a, ComputeConstantResponse& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline ComputeConstantResponse* New() const final { + return CreateMaybeMessage(NULL); + } + + ComputeConstantResponse* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const ComputeConstantResponse& from); + void MergeFrom(const ComputeConstantResponse& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(ComputeConstantResponse* other); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return NULL; + } + inline void* MaybeArenaPtr() const { + return NULL; + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // .xla.LiteralProto literal = 1; + bool has_literal() const; + void clear_literal(); + static const int kLiteralFieldNumber = 1; + private: + const ::xla::LiteralProto& _internal_literal() const; + public: + const ::xla::LiteralProto& literal() const; + ::xla::LiteralProto* release_literal(); + ::xla::LiteralProto* mutable_literal(); + void set_allocated_literal(::xla::LiteralProto* literal); + + // @@protoc_insertion_point(class_scope:xla.ComputeConstantResponse) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + ::xla::LiteralProto* literal_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class DeconstructTupleRequest : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.DeconstructTupleRequest) */ { + public: + DeconstructTupleRequest(); + virtual ~DeconstructTupleRequest(); + + DeconstructTupleRequest(const DeconstructTupleRequest& from); + + inline DeconstructTupleRequest& operator=(const DeconstructTupleRequest& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + DeconstructTupleRequest(DeconstructTupleRequest&& from) noexcept + : DeconstructTupleRequest() { + *this = ::std::move(from); + } + + inline DeconstructTupleRequest& operator=(DeconstructTupleRequest&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + static const ::google::protobuf::Descriptor* descriptor(); + static const DeconstructTupleRequest& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const DeconstructTupleRequest* internal_default_instance() { + return reinterpret_cast( + &_DeconstructTupleRequest_default_instance_); + } + static constexpr int kIndexInFileMessages = + 30; + + void Swap(DeconstructTupleRequest* other); + friend void swap(DeconstructTupleRequest& a, DeconstructTupleRequest& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline DeconstructTupleRequest* New() const final { + return CreateMaybeMessage(NULL); + } + + DeconstructTupleRequest* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const DeconstructTupleRequest& from); + void MergeFrom(const DeconstructTupleRequest& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(DeconstructTupleRequest* other); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return NULL; + } + inline void* MaybeArenaPtr() const { + return NULL; + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // .xla.GlobalDataHandle tuple_handle = 2; + bool has_tuple_handle() const; + void clear_tuple_handle(); + static const int kTupleHandleFieldNumber = 2; + private: + const ::xla::GlobalDataHandle& _internal_tuple_handle() const; + public: + const ::xla::GlobalDataHandle& tuple_handle() const; + ::xla::GlobalDataHandle* release_tuple_handle(); + ::xla::GlobalDataHandle* mutable_tuple_handle(); + void set_allocated_tuple_handle(::xla::GlobalDataHandle* tuple_handle); + + // @@protoc_insertion_point(class_scope:xla.DeconstructTupleRequest) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + ::xla::GlobalDataHandle* tuple_handle_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class DeconstructTupleResponse : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.DeconstructTupleResponse) */ { + public: + DeconstructTupleResponse(); + virtual ~DeconstructTupleResponse(); + + DeconstructTupleResponse(const DeconstructTupleResponse& from); + + inline DeconstructTupleResponse& operator=(const DeconstructTupleResponse& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + DeconstructTupleResponse(DeconstructTupleResponse&& from) noexcept + : DeconstructTupleResponse() { + *this = ::std::move(from); + } + + inline DeconstructTupleResponse& operator=(DeconstructTupleResponse&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + static const ::google::protobuf::Descriptor* descriptor(); + static const DeconstructTupleResponse& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const DeconstructTupleResponse* internal_default_instance() { + return reinterpret_cast( + &_DeconstructTupleResponse_default_instance_); + } + static constexpr int kIndexInFileMessages = + 31; + + void Swap(DeconstructTupleResponse* other); + friend void swap(DeconstructTupleResponse& a, DeconstructTupleResponse& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline DeconstructTupleResponse* New() const final { + return CreateMaybeMessage(NULL); + } + + DeconstructTupleResponse* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const DeconstructTupleResponse& from); + void MergeFrom(const DeconstructTupleResponse& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(DeconstructTupleResponse* other); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return NULL; + } + inline void* MaybeArenaPtr() const { + return NULL; + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated .xla.GlobalDataHandle element_handles = 1; + int element_handles_size() const; + void clear_element_handles(); + static const int kElementHandlesFieldNumber = 1; + ::xla::GlobalDataHandle* mutable_element_handles(int index); + ::google::protobuf::RepeatedPtrField< ::xla::GlobalDataHandle >* + mutable_element_handles(); + const ::xla::GlobalDataHandle& element_handles(int index) const; + ::xla::GlobalDataHandle* add_element_handles(); + const ::google::protobuf::RepeatedPtrField< ::xla::GlobalDataHandle >& + element_handles() const; + + // @@protoc_insertion_point(class_scope:xla.DeconstructTupleResponse) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + ::google::protobuf::RepeatedPtrField< ::xla::GlobalDataHandle > element_handles_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class LoadDataRequest : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.LoadDataRequest) */ { + public: + LoadDataRequest(); + virtual ~LoadDataRequest(); + + LoadDataRequest(const LoadDataRequest& from); + + inline LoadDataRequest& operator=(const LoadDataRequest& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + LoadDataRequest(LoadDataRequest&& from) noexcept + : LoadDataRequest() { + *this = ::std::move(from); + } + + inline LoadDataRequest& operator=(LoadDataRequest&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + static const ::google::protobuf::Descriptor* descriptor(); + static const LoadDataRequest& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const LoadDataRequest* internal_default_instance() { + return reinterpret_cast( + &_LoadDataRequest_default_instance_); + } + static constexpr int kIndexInFileMessages = + 32; + + void Swap(LoadDataRequest* other); + friend void swap(LoadDataRequest& a, LoadDataRequest& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline LoadDataRequest* New() const final { + return CreateMaybeMessage(NULL); + } + + LoadDataRequest* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const LoadDataRequest& from); + void MergeFrom(const LoadDataRequest& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(LoadDataRequest* other); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return NULL; + } + inline void* MaybeArenaPtr() const { + return NULL; + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // string columnio_tablet_path = 1; + void clear_columnio_tablet_path(); + static const int kColumnioTabletPathFieldNumber = 1; + const ::std::string& columnio_tablet_path() const; + void set_columnio_tablet_path(const ::std::string& value); + #if LANG_CXX11 + void set_columnio_tablet_path(::std::string&& value); + #endif + void set_columnio_tablet_path(const char* value); + void set_columnio_tablet_path(const char* value, size_t size); + ::std::string* mutable_columnio_tablet_path(); + ::std::string* release_columnio_tablet_path(); + void set_allocated_columnio_tablet_path(::std::string* columnio_tablet_path); + + // string columnio_field = 2; + void clear_columnio_field(); + static const int kColumnioFieldFieldNumber = 2; + const ::std::string& columnio_field() const; + void set_columnio_field(const ::std::string& value); + #if LANG_CXX11 + void set_columnio_field(::std::string&& value); + #endif + void set_columnio_field(const char* value); + void set_columnio_field(const char* value, size_t size); + ::std::string* mutable_columnio_field(); + ::std::string* release_columnio_field(); + void set_allocated_columnio_field(::std::string* columnio_field); + + // .xla.Shape element_shape = 3; + bool has_element_shape() const; + void clear_element_shape(); + static const int kElementShapeFieldNumber = 3; + private: + const ::xla::Shape& _internal_element_shape() const; + public: + const ::xla::Shape& element_shape() const; + ::xla::Shape* release_element_shape(); + ::xla::Shape* mutable_element_shape(); + void set_allocated_element_shape(::xla::Shape* element_shape); + + // int64 offset = 4; + void clear_offset(); + static const int kOffsetFieldNumber = 4; + ::google::protobuf::int64 offset() const; + void set_offset(::google::protobuf::int64 value); + + // int64 limit = 5; + void clear_limit(); + static const int kLimitFieldNumber = 5; + ::google::protobuf::int64 limit() const; + void set_limit(::google::protobuf::int64 value); + + // bool zip = 6; + void clear_zip(); + static const int kZipFieldNumber = 6; + bool zip() const; + void set_zip(bool value); + + // @@protoc_insertion_point(class_scope:xla.LoadDataRequest) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + ::google::protobuf::internal::ArenaStringPtr columnio_tablet_path_; + ::google::protobuf::internal::ArenaStringPtr columnio_field_; + ::xla::Shape* element_shape_; + ::google::protobuf::int64 offset_; + ::google::protobuf::int64 limit_; + bool zip_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class LoadDataResponse : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.LoadDataResponse) */ { + public: + LoadDataResponse(); + virtual ~LoadDataResponse(); + + LoadDataResponse(const LoadDataResponse& from); + + inline LoadDataResponse& operator=(const LoadDataResponse& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + LoadDataResponse(LoadDataResponse&& from) noexcept + : LoadDataResponse() { + *this = ::std::move(from); + } + + inline LoadDataResponse& operator=(LoadDataResponse&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + static const ::google::protobuf::Descriptor* descriptor(); + static const LoadDataResponse& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const LoadDataResponse* internal_default_instance() { + return reinterpret_cast( + &_LoadDataResponse_default_instance_); + } + static constexpr int kIndexInFileMessages = + 33; + + void Swap(LoadDataResponse* other); + friend void swap(LoadDataResponse& a, LoadDataResponse& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline LoadDataResponse* New() const final { + return CreateMaybeMessage(NULL); + } + + LoadDataResponse* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const LoadDataResponse& from); + void MergeFrom(const LoadDataResponse& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(LoadDataResponse* other); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return NULL; + } + inline void* MaybeArenaPtr() const { + return NULL; + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // .xla.GlobalDataHandle data = 1; + bool has_data() const; + void clear_data(); + static const int kDataFieldNumber = 1; + private: + const ::xla::GlobalDataHandle& _internal_data() const; + public: + const ::xla::GlobalDataHandle& data() const; + ::xla::GlobalDataHandle* release_data(); + ::xla::GlobalDataHandle* mutable_data(); + void set_allocated_data(::xla::GlobalDataHandle* data); + + // .xla.Shape data_shape = 2; + bool has_data_shape() const; + void clear_data_shape(); + static const int kDataShapeFieldNumber = 2; + private: + const ::xla::Shape& _internal_data_shape() const; + public: + const ::xla::Shape& data_shape() const; + ::xla::Shape* release_data_shape(); + ::xla::Shape* mutable_data_shape(); + void set_allocated_data_shape(::xla::Shape* data_shape); + + // int64 available_rows = 3; + void clear_available_rows(); + static const int kAvailableRowsFieldNumber = 3; + ::google::protobuf::int64 available_rows() const; + void set_available_rows(::google::protobuf::int64 value); + + // int64 rows_loaded = 4; + void clear_rows_loaded(); + static const int kRowsLoadedFieldNumber = 4; + ::google::protobuf::int64 rows_loaded() const; + void set_rows_loaded(::google::protobuf::int64 value); + + // int64 nanoseconds = 5; + void clear_nanoseconds(); + static const int kNanosecondsFieldNumber = 5; + ::google::protobuf::int64 nanoseconds() const; + void set_nanoseconds(::google::protobuf::int64 value); + + // @@protoc_insertion_point(class_scope:xla.LoadDataResponse) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + ::xla::GlobalDataHandle* data_; + ::xla::Shape* data_shape_; + ::google::protobuf::int64 available_rows_; + ::google::protobuf::int64 rows_loaded_; + ::google::protobuf::int64 nanoseconds_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class GetShapeRequest : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.GetShapeRequest) */ { + public: + GetShapeRequest(); + virtual ~GetShapeRequest(); + + GetShapeRequest(const GetShapeRequest& from); + + inline GetShapeRequest& operator=(const GetShapeRequest& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + GetShapeRequest(GetShapeRequest&& from) noexcept + : GetShapeRequest() { + *this = ::std::move(from); + } + + inline GetShapeRequest& operator=(GetShapeRequest&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + static const ::google::protobuf::Descriptor* descriptor(); + static const GetShapeRequest& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const GetShapeRequest* internal_default_instance() { + return reinterpret_cast( + &_GetShapeRequest_default_instance_); + } + static constexpr int kIndexInFileMessages = + 34; + + void Swap(GetShapeRequest* other); + friend void swap(GetShapeRequest& a, GetShapeRequest& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline GetShapeRequest* New() const final { + return CreateMaybeMessage(NULL); + } + + GetShapeRequest* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const GetShapeRequest& from); + void MergeFrom(const GetShapeRequest& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(GetShapeRequest* other); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return NULL; + } + inline void* MaybeArenaPtr() const { + return NULL; + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // .xla.GlobalDataHandle data = 1; + bool has_data() const; + void clear_data(); + static const int kDataFieldNumber = 1; + private: + const ::xla::GlobalDataHandle& _internal_data() const; + public: + const ::xla::GlobalDataHandle& data() const; + ::xla::GlobalDataHandle* release_data(); + ::xla::GlobalDataHandle* mutable_data(); + void set_allocated_data(::xla::GlobalDataHandle* data); + + // @@protoc_insertion_point(class_scope:xla.GetShapeRequest) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + ::xla::GlobalDataHandle* data_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class GetShapeResponse : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.GetShapeResponse) */ { + public: + GetShapeResponse(); + virtual ~GetShapeResponse(); + + GetShapeResponse(const GetShapeResponse& from); + + inline GetShapeResponse& operator=(const GetShapeResponse& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + GetShapeResponse(GetShapeResponse&& from) noexcept + : GetShapeResponse() { + *this = ::std::move(from); + } + + inline GetShapeResponse& operator=(GetShapeResponse&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + static const ::google::protobuf::Descriptor* descriptor(); + static const GetShapeResponse& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const GetShapeResponse* internal_default_instance() { + return reinterpret_cast( + &_GetShapeResponse_default_instance_); + } + static constexpr int kIndexInFileMessages = + 35; + + void Swap(GetShapeResponse* other); + friend void swap(GetShapeResponse& a, GetShapeResponse& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline GetShapeResponse* New() const final { + return CreateMaybeMessage(NULL); + } + + GetShapeResponse* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const GetShapeResponse& from); + void MergeFrom(const GetShapeResponse& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(GetShapeResponse* other); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return NULL; + } + inline void* MaybeArenaPtr() const { + return NULL; + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // .xla.Shape shape = 1; + bool has_shape() const; + void clear_shape(); + static const int kShapeFieldNumber = 1; + private: + const ::xla::Shape& _internal_shape() const; + public: + const ::xla::Shape& shape() const; + ::xla::Shape* release_shape(); + ::xla::Shape* mutable_shape(); + void set_allocated_shape(::xla::Shape* shape); + + // @@protoc_insertion_point(class_scope:xla.GetShapeResponse) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + ::xla::Shape* shape_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class UnpackRequest : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.UnpackRequest) */ { + public: + UnpackRequest(); + virtual ~UnpackRequest(); + + UnpackRequest(const UnpackRequest& from); + + inline UnpackRequest& operator=(const UnpackRequest& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + UnpackRequest(UnpackRequest&& from) noexcept + : UnpackRequest() { + *this = ::std::move(from); + } + + inline UnpackRequest& operator=(UnpackRequest&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + static const ::google::protobuf::Descriptor* descriptor(); + static const UnpackRequest& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const UnpackRequest* internal_default_instance() { + return reinterpret_cast( + &_UnpackRequest_default_instance_); + } + static constexpr int kIndexInFileMessages = + 36; + + void Swap(UnpackRequest* other); + friend void swap(UnpackRequest& a, UnpackRequest& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline UnpackRequest* New() const final { + return CreateMaybeMessage(NULL); + } + + UnpackRequest* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const UnpackRequest& from); + void MergeFrom(const UnpackRequest& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(UnpackRequest* other); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return NULL; + } + inline void* MaybeArenaPtr() const { + return NULL; + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // .xla.GlobalDataHandle data = 1; + bool has_data() const; + void clear_data(); + static const int kDataFieldNumber = 1; + private: + const ::xla::GlobalDataHandle& _internal_data() const; + public: + const ::xla::GlobalDataHandle& data() const; + ::xla::GlobalDataHandle* release_data(); + ::xla::GlobalDataHandle* mutable_data(); + void set_allocated_data(::xla::GlobalDataHandle* data); + + // @@protoc_insertion_point(class_scope:xla.UnpackRequest) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + ::xla::GlobalDataHandle* data_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class UnpackResponse : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.UnpackResponse) */ { + public: + UnpackResponse(); + virtual ~UnpackResponse(); + + UnpackResponse(const UnpackResponse& from); + + inline UnpackResponse& operator=(const UnpackResponse& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + UnpackResponse(UnpackResponse&& from) noexcept + : UnpackResponse() { + *this = ::std::move(from); + } + + inline UnpackResponse& operator=(UnpackResponse&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + static const ::google::protobuf::Descriptor* descriptor(); + static const UnpackResponse& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const UnpackResponse* internal_default_instance() { + return reinterpret_cast( + &_UnpackResponse_default_instance_); + } + static constexpr int kIndexInFileMessages = + 37; + + void Swap(UnpackResponse* other); + friend void swap(UnpackResponse& a, UnpackResponse& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline UnpackResponse* New() const final { + return CreateMaybeMessage(NULL); + } + + UnpackResponse* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const UnpackResponse& from); + void MergeFrom(const UnpackResponse& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(UnpackResponse* other); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return NULL; + } + inline void* MaybeArenaPtr() const { + return NULL; + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated .xla.GlobalDataHandle tied_data = 1; + int tied_data_size() const; + void clear_tied_data(); + static const int kTiedDataFieldNumber = 1; + ::xla::GlobalDataHandle* mutable_tied_data(int index); + ::google::protobuf::RepeatedPtrField< ::xla::GlobalDataHandle >* + mutable_tied_data(); + const ::xla::GlobalDataHandle& tied_data(int index) const; + ::xla::GlobalDataHandle* add_tied_data(); + const ::google::protobuf::RepeatedPtrField< ::xla::GlobalDataHandle >& + tied_data() const; + + // @@protoc_insertion_point(class_scope:xla.UnpackResponse) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + ::google::protobuf::RepeatedPtrField< ::xla::GlobalDataHandle > tied_data_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_2eproto::TableStruct; +}; +// =================================================================== + + +// =================================================================== + +#ifdef __GNUC__ + #pragma GCC diagnostic push + #pragma GCC diagnostic ignored "-Wstrict-aliasing" +#endif // __GNUC__ +// HloReducePrecisionOptions + +// .xla.HloReducePrecisionOptions.Location location = 1; +inline void HloReducePrecisionOptions::clear_location() { + location_ = 0; +} +inline ::xla::HloReducePrecisionOptions_Location HloReducePrecisionOptions::location() const { + // @@protoc_insertion_point(field_get:xla.HloReducePrecisionOptions.location) + return static_cast< ::xla::HloReducePrecisionOptions_Location >(location_); +} +inline void HloReducePrecisionOptions::set_location(::xla::HloReducePrecisionOptions_Location value) { + + location_ = value; + // @@protoc_insertion_point(field_set:xla.HloReducePrecisionOptions.location) +} + +// uint32 exponent_bits = 2; +inline void HloReducePrecisionOptions::clear_exponent_bits() { + exponent_bits_ = 0u; +} +inline ::google::protobuf::uint32 HloReducePrecisionOptions::exponent_bits() const { + // @@protoc_insertion_point(field_get:xla.HloReducePrecisionOptions.exponent_bits) + return exponent_bits_; +} +inline void HloReducePrecisionOptions::set_exponent_bits(::google::protobuf::uint32 value) { + + exponent_bits_ = value; + // @@protoc_insertion_point(field_set:xla.HloReducePrecisionOptions.exponent_bits) +} + +// uint32 mantissa_bits = 3; +inline void HloReducePrecisionOptions::clear_mantissa_bits() { + mantissa_bits_ = 0u; +} +inline ::google::protobuf::uint32 HloReducePrecisionOptions::mantissa_bits() const { + // @@protoc_insertion_point(field_get:xla.HloReducePrecisionOptions.mantissa_bits) + return mantissa_bits_; +} +inline void HloReducePrecisionOptions::set_mantissa_bits(::google::protobuf::uint32 value) { + + mantissa_bits_ = value; + // @@protoc_insertion_point(field_set:xla.HloReducePrecisionOptions.mantissa_bits) +} + +// repeated uint32 opcodes_to_suffix = 4; +inline int HloReducePrecisionOptions::opcodes_to_suffix_size() const { + return opcodes_to_suffix_.size(); +} +inline void HloReducePrecisionOptions::clear_opcodes_to_suffix() { + opcodes_to_suffix_.Clear(); +} +inline ::google::protobuf::uint32 HloReducePrecisionOptions::opcodes_to_suffix(int index) const { + // @@protoc_insertion_point(field_get:xla.HloReducePrecisionOptions.opcodes_to_suffix) + return opcodes_to_suffix_.Get(index); +} +inline void HloReducePrecisionOptions::set_opcodes_to_suffix(int index, ::google::protobuf::uint32 value) { + opcodes_to_suffix_.Set(index, value); + // @@protoc_insertion_point(field_set:xla.HloReducePrecisionOptions.opcodes_to_suffix) +} +inline void HloReducePrecisionOptions::add_opcodes_to_suffix(::google::protobuf::uint32 value) { + opcodes_to_suffix_.Add(value); + // @@protoc_insertion_point(field_add:xla.HloReducePrecisionOptions.opcodes_to_suffix) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::uint32 >& +HloReducePrecisionOptions::opcodes_to_suffix() const { + // @@protoc_insertion_point(field_list:xla.HloReducePrecisionOptions.opcodes_to_suffix) + return opcodes_to_suffix_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::uint32 >* +HloReducePrecisionOptions::mutable_opcodes_to_suffix() { + // @@protoc_insertion_point(field_mutable_list:xla.HloReducePrecisionOptions.opcodes_to_suffix) + return &opcodes_to_suffix_; +} + +// repeated string opname_substrings_to_suffix = 5; +inline int HloReducePrecisionOptions::opname_substrings_to_suffix_size() const { + return opname_substrings_to_suffix_.size(); +} +inline void HloReducePrecisionOptions::clear_opname_substrings_to_suffix() { + opname_substrings_to_suffix_.Clear(); +} +inline const ::std::string& HloReducePrecisionOptions::opname_substrings_to_suffix(int index) const { + // @@protoc_insertion_point(field_get:xla.HloReducePrecisionOptions.opname_substrings_to_suffix) + return opname_substrings_to_suffix_.Get(index); +} +inline ::std::string* HloReducePrecisionOptions::mutable_opname_substrings_to_suffix(int index) { + // @@protoc_insertion_point(field_mutable:xla.HloReducePrecisionOptions.opname_substrings_to_suffix) + return opname_substrings_to_suffix_.Mutable(index); +} +inline void HloReducePrecisionOptions::set_opname_substrings_to_suffix(int index, const ::std::string& value) { + // @@protoc_insertion_point(field_set:xla.HloReducePrecisionOptions.opname_substrings_to_suffix) + opname_substrings_to_suffix_.Mutable(index)->assign(value); +} +#if LANG_CXX11 +inline void HloReducePrecisionOptions::set_opname_substrings_to_suffix(int index, ::std::string&& value) { + // @@protoc_insertion_point(field_set:xla.HloReducePrecisionOptions.opname_substrings_to_suffix) + opname_substrings_to_suffix_.Mutable(index)->assign(std::move(value)); +} +#endif +inline void HloReducePrecisionOptions::set_opname_substrings_to_suffix(int index, const char* value) { + GOOGLE_DCHECK(value != NULL); + opname_substrings_to_suffix_.Mutable(index)->assign(value); + // @@protoc_insertion_point(field_set_char:xla.HloReducePrecisionOptions.opname_substrings_to_suffix) +} +inline void HloReducePrecisionOptions::set_opname_substrings_to_suffix(int index, const char* value, size_t size) { + opname_substrings_to_suffix_.Mutable(index)->assign( + reinterpret_cast(value), size); + // @@protoc_insertion_point(field_set_pointer:xla.HloReducePrecisionOptions.opname_substrings_to_suffix) +} +inline ::std::string* HloReducePrecisionOptions::add_opname_substrings_to_suffix() { + // @@protoc_insertion_point(field_add_mutable:xla.HloReducePrecisionOptions.opname_substrings_to_suffix) + return opname_substrings_to_suffix_.Add(); +} +inline void HloReducePrecisionOptions::add_opname_substrings_to_suffix(const ::std::string& value) { + opname_substrings_to_suffix_.Add()->assign(value); + // @@protoc_insertion_point(field_add:xla.HloReducePrecisionOptions.opname_substrings_to_suffix) +} +#if LANG_CXX11 +inline void HloReducePrecisionOptions::add_opname_substrings_to_suffix(::std::string&& value) { + opname_substrings_to_suffix_.Add(std::move(value)); + // @@protoc_insertion_point(field_add:xla.HloReducePrecisionOptions.opname_substrings_to_suffix) +} +#endif +inline void HloReducePrecisionOptions::add_opname_substrings_to_suffix(const char* value) { + GOOGLE_DCHECK(value != NULL); + opname_substrings_to_suffix_.Add()->assign(value); + // @@protoc_insertion_point(field_add_char:xla.HloReducePrecisionOptions.opname_substrings_to_suffix) +} +inline void HloReducePrecisionOptions::add_opname_substrings_to_suffix(const char* value, size_t size) { + opname_substrings_to_suffix_.Add()->assign(reinterpret_cast(value), size); + // @@protoc_insertion_point(field_add_pointer:xla.HloReducePrecisionOptions.opname_substrings_to_suffix) +} +inline const ::google::protobuf::RepeatedPtrField< ::std::string>& +HloReducePrecisionOptions::opname_substrings_to_suffix() const { + // @@protoc_insertion_point(field_list:xla.HloReducePrecisionOptions.opname_substrings_to_suffix) + return opname_substrings_to_suffix_; +} +inline ::google::protobuf::RepeatedPtrField< ::std::string>* +HloReducePrecisionOptions::mutable_opname_substrings_to_suffix() { + // @@protoc_insertion_point(field_mutable_list:xla.HloReducePrecisionOptions.opname_substrings_to_suffix) + return &opname_substrings_to_suffix_; +} + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// DebugOptions + +// string xla_generate_hlo_graph = 1; +inline void DebugOptions::clear_xla_generate_hlo_graph() { + xla_generate_hlo_graph_.ClearToEmptyNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); +} +inline const ::std::string& DebugOptions::xla_generate_hlo_graph() const { + // @@protoc_insertion_point(field_get:xla.DebugOptions.xla_generate_hlo_graph) + return xla_generate_hlo_graph_.GetNoArena(); +} +inline void DebugOptions::set_xla_generate_hlo_graph(const ::std::string& value) { + + xla_generate_hlo_graph_.SetNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value); + // @@protoc_insertion_point(field_set:xla.DebugOptions.xla_generate_hlo_graph) +} +#if LANG_CXX11 +inline void DebugOptions::set_xla_generate_hlo_graph(::std::string&& value) { + + xla_generate_hlo_graph_.SetNoArena( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value)); + // @@protoc_insertion_point(field_set_rvalue:xla.DebugOptions.xla_generate_hlo_graph) +} +#endif +inline void DebugOptions::set_xla_generate_hlo_graph(const char* value) { + GOOGLE_DCHECK(value != NULL); + + xla_generate_hlo_graph_.SetNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value)); + // @@protoc_insertion_point(field_set_char:xla.DebugOptions.xla_generate_hlo_graph) +} +inline void DebugOptions::set_xla_generate_hlo_graph(const char* value, size_t size) { + + xla_generate_hlo_graph_.SetNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + ::std::string(reinterpret_cast(value), size)); + // @@protoc_insertion_point(field_set_pointer:xla.DebugOptions.xla_generate_hlo_graph) +} +inline ::std::string* DebugOptions::mutable_xla_generate_hlo_graph() { + + // @@protoc_insertion_point(field_mutable:xla.DebugOptions.xla_generate_hlo_graph) + return xla_generate_hlo_graph_.MutableNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); +} +inline ::std::string* DebugOptions::release_xla_generate_hlo_graph() { + // @@protoc_insertion_point(field_release:xla.DebugOptions.xla_generate_hlo_graph) + + return xla_generate_hlo_graph_.ReleaseNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); +} +inline void DebugOptions::set_allocated_xla_generate_hlo_graph(::std::string* xla_generate_hlo_graph) { + if (xla_generate_hlo_graph != NULL) { + + } else { + + } + xla_generate_hlo_graph_.SetAllocatedNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), xla_generate_hlo_graph); + // @@protoc_insertion_point(field_set_allocated:xla.DebugOptions.xla_generate_hlo_graph) +} + +// bool xla_hlo_graph_addresses = 2; +inline void DebugOptions::clear_xla_hlo_graph_addresses() { + xla_hlo_graph_addresses_ = false; +} +inline bool DebugOptions::xla_hlo_graph_addresses() const { + // @@protoc_insertion_point(field_get:xla.DebugOptions.xla_hlo_graph_addresses) + return xla_hlo_graph_addresses_; +} +inline void DebugOptions::set_xla_hlo_graph_addresses(bool value) { + + xla_hlo_graph_addresses_ = value; + // @@protoc_insertion_point(field_set:xla.DebugOptions.xla_hlo_graph_addresses) +} + +// string xla_hlo_graph_path = 4; +inline void DebugOptions::clear_xla_hlo_graph_path() { + xla_hlo_graph_path_.ClearToEmptyNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); +} +inline const ::std::string& DebugOptions::xla_hlo_graph_path() const { + // @@protoc_insertion_point(field_get:xla.DebugOptions.xla_hlo_graph_path) + return xla_hlo_graph_path_.GetNoArena(); +} +inline void DebugOptions::set_xla_hlo_graph_path(const ::std::string& value) { + + xla_hlo_graph_path_.SetNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value); + // @@protoc_insertion_point(field_set:xla.DebugOptions.xla_hlo_graph_path) +} +#if LANG_CXX11 +inline void DebugOptions::set_xla_hlo_graph_path(::std::string&& value) { + + xla_hlo_graph_path_.SetNoArena( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value)); + // @@protoc_insertion_point(field_set_rvalue:xla.DebugOptions.xla_hlo_graph_path) +} +#endif +inline void DebugOptions::set_xla_hlo_graph_path(const char* value) { + GOOGLE_DCHECK(value != NULL); + + xla_hlo_graph_path_.SetNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value)); + // @@protoc_insertion_point(field_set_char:xla.DebugOptions.xla_hlo_graph_path) +} +inline void DebugOptions::set_xla_hlo_graph_path(const char* value, size_t size) { + + xla_hlo_graph_path_.SetNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + ::std::string(reinterpret_cast(value), size)); + // @@protoc_insertion_point(field_set_pointer:xla.DebugOptions.xla_hlo_graph_path) +} +inline ::std::string* DebugOptions::mutable_xla_hlo_graph_path() { + + // @@protoc_insertion_point(field_mutable:xla.DebugOptions.xla_hlo_graph_path) + return xla_hlo_graph_path_.MutableNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); +} +inline ::std::string* DebugOptions::release_xla_hlo_graph_path() { + // @@protoc_insertion_point(field_release:xla.DebugOptions.xla_hlo_graph_path) + + return xla_hlo_graph_path_.ReleaseNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); +} +inline void DebugOptions::set_allocated_xla_hlo_graph_path(::std::string* xla_hlo_graph_path) { + if (xla_hlo_graph_path != NULL) { + + } else { + + } + xla_hlo_graph_path_.SetAllocatedNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), xla_hlo_graph_path); + // @@protoc_insertion_point(field_set_allocated:xla.DebugOptions.xla_hlo_graph_path) +} + +// bool xla_hlo_dump_as_graphdef = 5; +inline void DebugOptions::clear_xla_hlo_dump_as_graphdef() { + xla_hlo_dump_as_graphdef_ = false; +} +inline bool DebugOptions::xla_hlo_dump_as_graphdef() const { + // @@protoc_insertion_point(field_get:xla.DebugOptions.xla_hlo_dump_as_graphdef) + return xla_hlo_dump_as_graphdef_; +} +inline void DebugOptions::set_xla_hlo_dump_as_graphdef(bool value) { + + xla_hlo_dump_as_graphdef_ = value; + // @@protoc_insertion_point(field_set:xla.DebugOptions.xla_hlo_dump_as_graphdef) +} + +// string xla_log_hlo_text = 6; +inline void DebugOptions::clear_xla_log_hlo_text() { + xla_log_hlo_text_.ClearToEmptyNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); +} +inline const ::std::string& DebugOptions::xla_log_hlo_text() const { + // @@protoc_insertion_point(field_get:xla.DebugOptions.xla_log_hlo_text) + return xla_log_hlo_text_.GetNoArena(); +} +inline void DebugOptions::set_xla_log_hlo_text(const ::std::string& value) { + + xla_log_hlo_text_.SetNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value); + // @@protoc_insertion_point(field_set:xla.DebugOptions.xla_log_hlo_text) +} +#if LANG_CXX11 +inline void DebugOptions::set_xla_log_hlo_text(::std::string&& value) { + + xla_log_hlo_text_.SetNoArena( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value)); + // @@protoc_insertion_point(field_set_rvalue:xla.DebugOptions.xla_log_hlo_text) +} +#endif +inline void DebugOptions::set_xla_log_hlo_text(const char* value) { + GOOGLE_DCHECK(value != NULL); + + xla_log_hlo_text_.SetNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value)); + // @@protoc_insertion_point(field_set_char:xla.DebugOptions.xla_log_hlo_text) +} +inline void DebugOptions::set_xla_log_hlo_text(const char* value, size_t size) { + + xla_log_hlo_text_.SetNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + ::std::string(reinterpret_cast(value), size)); + // @@protoc_insertion_point(field_set_pointer:xla.DebugOptions.xla_log_hlo_text) +} +inline ::std::string* DebugOptions::mutable_xla_log_hlo_text() { + + // @@protoc_insertion_point(field_mutable:xla.DebugOptions.xla_log_hlo_text) + return xla_log_hlo_text_.MutableNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); +} +inline ::std::string* DebugOptions::release_xla_log_hlo_text() { + // @@protoc_insertion_point(field_release:xla.DebugOptions.xla_log_hlo_text) + + return xla_log_hlo_text_.ReleaseNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); +} +inline void DebugOptions::set_allocated_xla_log_hlo_text(::std::string* xla_log_hlo_text) { + if (xla_log_hlo_text != NULL) { + + } else { + + } + xla_log_hlo_text_.SetAllocatedNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), xla_log_hlo_text); + // @@protoc_insertion_point(field_set_allocated:xla.DebugOptions.xla_log_hlo_text) +} + +// string xla_generate_hlo_text_to = 7; +inline void DebugOptions::clear_xla_generate_hlo_text_to() { + xla_generate_hlo_text_to_.ClearToEmptyNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); +} +inline const ::std::string& DebugOptions::xla_generate_hlo_text_to() const { + // @@protoc_insertion_point(field_get:xla.DebugOptions.xla_generate_hlo_text_to) + return xla_generate_hlo_text_to_.GetNoArena(); +} +inline void DebugOptions::set_xla_generate_hlo_text_to(const ::std::string& value) { + + xla_generate_hlo_text_to_.SetNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value); + // @@protoc_insertion_point(field_set:xla.DebugOptions.xla_generate_hlo_text_to) +} +#if LANG_CXX11 +inline void DebugOptions::set_xla_generate_hlo_text_to(::std::string&& value) { + + xla_generate_hlo_text_to_.SetNoArena( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value)); + // @@protoc_insertion_point(field_set_rvalue:xla.DebugOptions.xla_generate_hlo_text_to) +} +#endif +inline void DebugOptions::set_xla_generate_hlo_text_to(const char* value) { + GOOGLE_DCHECK(value != NULL); + + xla_generate_hlo_text_to_.SetNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value)); + // @@protoc_insertion_point(field_set_char:xla.DebugOptions.xla_generate_hlo_text_to) +} +inline void DebugOptions::set_xla_generate_hlo_text_to(const char* value, size_t size) { + + xla_generate_hlo_text_to_.SetNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + ::std::string(reinterpret_cast(value), size)); + // @@protoc_insertion_point(field_set_pointer:xla.DebugOptions.xla_generate_hlo_text_to) +} +inline ::std::string* DebugOptions::mutable_xla_generate_hlo_text_to() { + + // @@protoc_insertion_point(field_mutable:xla.DebugOptions.xla_generate_hlo_text_to) + return xla_generate_hlo_text_to_.MutableNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); +} +inline ::std::string* DebugOptions::release_xla_generate_hlo_text_to() { + // @@protoc_insertion_point(field_release:xla.DebugOptions.xla_generate_hlo_text_to) + + return xla_generate_hlo_text_to_.ReleaseNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); +} +inline void DebugOptions::set_allocated_xla_generate_hlo_text_to(::std::string* xla_generate_hlo_text_to) { + if (xla_generate_hlo_text_to != NULL) { + + } else { + + } + xla_generate_hlo_text_to_.SetAllocatedNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), xla_generate_hlo_text_to); + // @@protoc_insertion_point(field_set_allocated:xla.DebugOptions.xla_generate_hlo_text_to) +} + +// string xla_dump_optimized_hlo_proto_to = 8; +inline void DebugOptions::clear_xla_dump_optimized_hlo_proto_to() { + xla_dump_optimized_hlo_proto_to_.ClearToEmptyNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); +} +inline const ::std::string& DebugOptions::xla_dump_optimized_hlo_proto_to() const { + // @@protoc_insertion_point(field_get:xla.DebugOptions.xla_dump_optimized_hlo_proto_to) + return xla_dump_optimized_hlo_proto_to_.GetNoArena(); +} +inline void DebugOptions::set_xla_dump_optimized_hlo_proto_to(const ::std::string& value) { + + xla_dump_optimized_hlo_proto_to_.SetNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value); + // @@protoc_insertion_point(field_set:xla.DebugOptions.xla_dump_optimized_hlo_proto_to) +} +#if LANG_CXX11 +inline void DebugOptions::set_xla_dump_optimized_hlo_proto_to(::std::string&& value) { + + xla_dump_optimized_hlo_proto_to_.SetNoArena( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value)); + // @@protoc_insertion_point(field_set_rvalue:xla.DebugOptions.xla_dump_optimized_hlo_proto_to) +} +#endif +inline void DebugOptions::set_xla_dump_optimized_hlo_proto_to(const char* value) { + GOOGLE_DCHECK(value != NULL); + + xla_dump_optimized_hlo_proto_to_.SetNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value)); + // @@protoc_insertion_point(field_set_char:xla.DebugOptions.xla_dump_optimized_hlo_proto_to) +} +inline void DebugOptions::set_xla_dump_optimized_hlo_proto_to(const char* value, size_t size) { + + xla_dump_optimized_hlo_proto_to_.SetNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + ::std::string(reinterpret_cast(value), size)); + // @@protoc_insertion_point(field_set_pointer:xla.DebugOptions.xla_dump_optimized_hlo_proto_to) +} +inline ::std::string* DebugOptions::mutable_xla_dump_optimized_hlo_proto_to() { + + // @@protoc_insertion_point(field_mutable:xla.DebugOptions.xla_dump_optimized_hlo_proto_to) + return xla_dump_optimized_hlo_proto_to_.MutableNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); +} +inline ::std::string* DebugOptions::release_xla_dump_optimized_hlo_proto_to() { + // @@protoc_insertion_point(field_release:xla.DebugOptions.xla_dump_optimized_hlo_proto_to) + + return xla_dump_optimized_hlo_proto_to_.ReleaseNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); +} +inline void DebugOptions::set_allocated_xla_dump_optimized_hlo_proto_to(::std::string* xla_dump_optimized_hlo_proto_to) { + if (xla_dump_optimized_hlo_proto_to != NULL) { + + } else { + + } + xla_dump_optimized_hlo_proto_to_.SetAllocatedNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), xla_dump_optimized_hlo_proto_to); + // @@protoc_insertion_point(field_set_allocated:xla.DebugOptions.xla_dump_optimized_hlo_proto_to) +} + +// bool xla_hlo_profile = 9; +inline void DebugOptions::clear_xla_hlo_profile() { + xla_hlo_profile_ = false; +} +inline bool DebugOptions::xla_hlo_profile() const { + // @@protoc_insertion_point(field_get:xla.DebugOptions.xla_hlo_profile) + return xla_hlo_profile_; +} +inline void DebugOptions::set_xla_hlo_profile(bool value) { + + xla_hlo_profile_ = value; + // @@protoc_insertion_point(field_set:xla.DebugOptions.xla_hlo_profile) +} + +// string xla_dump_computations_to = 10; +inline void DebugOptions::clear_xla_dump_computations_to() { + xla_dump_computations_to_.ClearToEmptyNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); +} +inline const ::std::string& DebugOptions::xla_dump_computations_to() const { + // @@protoc_insertion_point(field_get:xla.DebugOptions.xla_dump_computations_to) + return xla_dump_computations_to_.GetNoArena(); +} +inline void DebugOptions::set_xla_dump_computations_to(const ::std::string& value) { + + xla_dump_computations_to_.SetNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value); + // @@protoc_insertion_point(field_set:xla.DebugOptions.xla_dump_computations_to) +} +#if LANG_CXX11 +inline void DebugOptions::set_xla_dump_computations_to(::std::string&& value) { + + xla_dump_computations_to_.SetNoArena( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value)); + // @@protoc_insertion_point(field_set_rvalue:xla.DebugOptions.xla_dump_computations_to) +} +#endif +inline void DebugOptions::set_xla_dump_computations_to(const char* value) { + GOOGLE_DCHECK(value != NULL); + + xla_dump_computations_to_.SetNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value)); + // @@protoc_insertion_point(field_set_char:xla.DebugOptions.xla_dump_computations_to) +} +inline void DebugOptions::set_xla_dump_computations_to(const char* value, size_t size) { + + xla_dump_computations_to_.SetNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + ::std::string(reinterpret_cast(value), size)); + // @@protoc_insertion_point(field_set_pointer:xla.DebugOptions.xla_dump_computations_to) +} +inline ::std::string* DebugOptions::mutable_xla_dump_computations_to() { + + // @@protoc_insertion_point(field_mutable:xla.DebugOptions.xla_dump_computations_to) + return xla_dump_computations_to_.MutableNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); +} +inline ::std::string* DebugOptions::release_xla_dump_computations_to() { + // @@protoc_insertion_point(field_release:xla.DebugOptions.xla_dump_computations_to) + + return xla_dump_computations_to_.ReleaseNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); +} +inline void DebugOptions::set_allocated_xla_dump_computations_to(::std::string* xla_dump_computations_to) { + if (xla_dump_computations_to != NULL) { + + } else { + + } + xla_dump_computations_to_.SetAllocatedNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), xla_dump_computations_to); + // @@protoc_insertion_point(field_set_allocated:xla.DebugOptions.xla_dump_computations_to) +} + +// string xla_dump_executions_to = 11; +inline void DebugOptions::clear_xla_dump_executions_to() { + xla_dump_executions_to_.ClearToEmptyNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); +} +inline const ::std::string& DebugOptions::xla_dump_executions_to() const { + // @@protoc_insertion_point(field_get:xla.DebugOptions.xla_dump_executions_to) + return xla_dump_executions_to_.GetNoArena(); +} +inline void DebugOptions::set_xla_dump_executions_to(const ::std::string& value) { + + xla_dump_executions_to_.SetNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value); + // @@protoc_insertion_point(field_set:xla.DebugOptions.xla_dump_executions_to) +} +#if LANG_CXX11 +inline void DebugOptions::set_xla_dump_executions_to(::std::string&& value) { + + xla_dump_executions_to_.SetNoArena( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value)); + // @@protoc_insertion_point(field_set_rvalue:xla.DebugOptions.xla_dump_executions_to) +} +#endif +inline void DebugOptions::set_xla_dump_executions_to(const char* value) { + GOOGLE_DCHECK(value != NULL); + + xla_dump_executions_to_.SetNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value)); + // @@protoc_insertion_point(field_set_char:xla.DebugOptions.xla_dump_executions_to) +} +inline void DebugOptions::set_xla_dump_executions_to(const char* value, size_t size) { + + xla_dump_executions_to_.SetNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + ::std::string(reinterpret_cast(value), size)); + // @@protoc_insertion_point(field_set_pointer:xla.DebugOptions.xla_dump_executions_to) +} +inline ::std::string* DebugOptions::mutable_xla_dump_executions_to() { + + // @@protoc_insertion_point(field_mutable:xla.DebugOptions.xla_dump_executions_to) + return xla_dump_executions_to_.MutableNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); +} +inline ::std::string* DebugOptions::release_xla_dump_executions_to() { + // @@protoc_insertion_point(field_release:xla.DebugOptions.xla_dump_executions_to) + + return xla_dump_executions_to_.ReleaseNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); +} +inline void DebugOptions::set_allocated_xla_dump_executions_to(::std::string* xla_dump_executions_to) { + if (xla_dump_executions_to != NULL) { + + } else { + + } + xla_dump_executions_to_.SetAllocatedNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), xla_dump_executions_to); + // @@protoc_insertion_point(field_set_allocated:xla.DebugOptions.xla_dump_executions_to) +} + +// repeated string xla_disable_hlo_passes = 30; +inline int DebugOptions::xla_disable_hlo_passes_size() const { + return xla_disable_hlo_passes_.size(); +} +inline void DebugOptions::clear_xla_disable_hlo_passes() { + xla_disable_hlo_passes_.Clear(); +} +inline const ::std::string& DebugOptions::xla_disable_hlo_passes(int index) const { + // @@protoc_insertion_point(field_get:xla.DebugOptions.xla_disable_hlo_passes) + return xla_disable_hlo_passes_.Get(index); +} +inline ::std::string* DebugOptions::mutable_xla_disable_hlo_passes(int index) { + // @@protoc_insertion_point(field_mutable:xla.DebugOptions.xla_disable_hlo_passes) + return xla_disable_hlo_passes_.Mutable(index); +} +inline void DebugOptions::set_xla_disable_hlo_passes(int index, const ::std::string& value) { + // @@protoc_insertion_point(field_set:xla.DebugOptions.xla_disable_hlo_passes) + xla_disable_hlo_passes_.Mutable(index)->assign(value); +} +#if LANG_CXX11 +inline void DebugOptions::set_xla_disable_hlo_passes(int index, ::std::string&& value) { + // @@protoc_insertion_point(field_set:xla.DebugOptions.xla_disable_hlo_passes) + xla_disable_hlo_passes_.Mutable(index)->assign(std::move(value)); +} +#endif +inline void DebugOptions::set_xla_disable_hlo_passes(int index, const char* value) { + GOOGLE_DCHECK(value != NULL); + xla_disable_hlo_passes_.Mutable(index)->assign(value); + // @@protoc_insertion_point(field_set_char:xla.DebugOptions.xla_disable_hlo_passes) +} +inline void DebugOptions::set_xla_disable_hlo_passes(int index, const char* value, size_t size) { + xla_disable_hlo_passes_.Mutable(index)->assign( + reinterpret_cast(value), size); + // @@protoc_insertion_point(field_set_pointer:xla.DebugOptions.xla_disable_hlo_passes) +} +inline ::std::string* DebugOptions::add_xla_disable_hlo_passes() { + // @@protoc_insertion_point(field_add_mutable:xla.DebugOptions.xla_disable_hlo_passes) + return xla_disable_hlo_passes_.Add(); +} +inline void DebugOptions::add_xla_disable_hlo_passes(const ::std::string& value) { + xla_disable_hlo_passes_.Add()->assign(value); + // @@protoc_insertion_point(field_add:xla.DebugOptions.xla_disable_hlo_passes) +} +#if LANG_CXX11 +inline void DebugOptions::add_xla_disable_hlo_passes(::std::string&& value) { + xla_disable_hlo_passes_.Add(std::move(value)); + // @@protoc_insertion_point(field_add:xla.DebugOptions.xla_disable_hlo_passes) +} +#endif +inline void DebugOptions::add_xla_disable_hlo_passes(const char* value) { + GOOGLE_DCHECK(value != NULL); + xla_disable_hlo_passes_.Add()->assign(value); + // @@protoc_insertion_point(field_add_char:xla.DebugOptions.xla_disable_hlo_passes) +} +inline void DebugOptions::add_xla_disable_hlo_passes(const char* value, size_t size) { + xla_disable_hlo_passes_.Add()->assign(reinterpret_cast(value), size); + // @@protoc_insertion_point(field_add_pointer:xla.DebugOptions.xla_disable_hlo_passes) +} +inline const ::google::protobuf::RepeatedPtrField< ::std::string>& +DebugOptions::xla_disable_hlo_passes() const { + // @@protoc_insertion_point(field_list:xla.DebugOptions.xla_disable_hlo_passes) + return xla_disable_hlo_passes_; +} +inline ::google::protobuf::RepeatedPtrField< ::std::string>* +DebugOptions::mutable_xla_disable_hlo_passes() { + // @@protoc_insertion_point(field_mutable_list:xla.DebugOptions.xla_disable_hlo_passes) + return &xla_disable_hlo_passes_; +} + +// int32 xla_backend_optimization_level = 31; +inline void DebugOptions::clear_xla_backend_optimization_level() { + xla_backend_optimization_level_ = 0; +} +inline ::google::protobuf::int32 DebugOptions::xla_backend_optimization_level() const { + // @@protoc_insertion_point(field_get:xla.DebugOptions.xla_backend_optimization_level) + return xla_backend_optimization_level_; +} +inline void DebugOptions::set_xla_backend_optimization_level(::google::protobuf::int32 value) { + + xla_backend_optimization_level_ = value; + // @@protoc_insertion_point(field_set:xla.DebugOptions.xla_backend_optimization_level) +} + +// bool xla_embed_ir_in_executable = 33; +inline void DebugOptions::clear_xla_embed_ir_in_executable() { + xla_embed_ir_in_executable_ = false; +} +inline bool DebugOptions::xla_embed_ir_in_executable() const { + // @@protoc_insertion_point(field_get:xla.DebugOptions.xla_embed_ir_in_executable) + return xla_embed_ir_in_executable_; +} +inline void DebugOptions::set_xla_embed_ir_in_executable(bool value) { + + xla_embed_ir_in_executable_ = value; + // @@protoc_insertion_point(field_set:xla.DebugOptions.xla_embed_ir_in_executable) +} + +// string xla_dump_ir_to = 34; +inline void DebugOptions::clear_xla_dump_ir_to() { + xla_dump_ir_to_.ClearToEmptyNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); +} +inline const ::std::string& DebugOptions::xla_dump_ir_to() const { + // @@protoc_insertion_point(field_get:xla.DebugOptions.xla_dump_ir_to) + return xla_dump_ir_to_.GetNoArena(); +} +inline void DebugOptions::set_xla_dump_ir_to(const ::std::string& value) { + + xla_dump_ir_to_.SetNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value); + // @@protoc_insertion_point(field_set:xla.DebugOptions.xla_dump_ir_to) +} +#if LANG_CXX11 +inline void DebugOptions::set_xla_dump_ir_to(::std::string&& value) { + + xla_dump_ir_to_.SetNoArena( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value)); + // @@protoc_insertion_point(field_set_rvalue:xla.DebugOptions.xla_dump_ir_to) +} +#endif +inline void DebugOptions::set_xla_dump_ir_to(const char* value) { + GOOGLE_DCHECK(value != NULL); + + xla_dump_ir_to_.SetNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value)); + // @@protoc_insertion_point(field_set_char:xla.DebugOptions.xla_dump_ir_to) +} +inline void DebugOptions::set_xla_dump_ir_to(const char* value, size_t size) { + + xla_dump_ir_to_.SetNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + ::std::string(reinterpret_cast(value), size)); + // @@protoc_insertion_point(field_set_pointer:xla.DebugOptions.xla_dump_ir_to) +} +inline ::std::string* DebugOptions::mutable_xla_dump_ir_to() { + + // @@protoc_insertion_point(field_mutable:xla.DebugOptions.xla_dump_ir_to) + return xla_dump_ir_to_.MutableNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); +} +inline ::std::string* DebugOptions::release_xla_dump_ir_to() { + // @@protoc_insertion_point(field_release:xla.DebugOptions.xla_dump_ir_to) + + return xla_dump_ir_to_.ReleaseNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); +} +inline void DebugOptions::set_allocated_xla_dump_ir_to(::std::string* xla_dump_ir_to) { + if (xla_dump_ir_to != NULL) { + + } else { + + } + xla_dump_ir_to_.SetAllocatedNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), xla_dump_ir_to); + // @@protoc_insertion_point(field_set_allocated:xla.DebugOptions.xla_dump_ir_to) +} + +// bool xla_eliminate_hlo_implicit_broadcast = 35; +inline void DebugOptions::clear_xla_eliminate_hlo_implicit_broadcast() { + xla_eliminate_hlo_implicit_broadcast_ = false; +} +inline bool DebugOptions::xla_eliminate_hlo_implicit_broadcast() const { + // @@protoc_insertion_point(field_get:xla.DebugOptions.xla_eliminate_hlo_implicit_broadcast) + return xla_eliminate_hlo_implicit_broadcast_; +} +inline void DebugOptions::set_xla_eliminate_hlo_implicit_broadcast(bool value) { + + xla_eliminate_hlo_implicit_broadcast_ = value; + // @@protoc_insertion_point(field_set:xla.DebugOptions.xla_eliminate_hlo_implicit_broadcast) +} + +// bool xla_cpu_multi_thread_eigen = 60; +inline void DebugOptions::clear_xla_cpu_multi_thread_eigen() { + xla_cpu_multi_thread_eigen_ = false; +} +inline bool DebugOptions::xla_cpu_multi_thread_eigen() const { + // @@protoc_insertion_point(field_get:xla.DebugOptions.xla_cpu_multi_thread_eigen) + return xla_cpu_multi_thread_eigen_; +} +inline void DebugOptions::set_xla_cpu_multi_thread_eigen(bool value) { + + xla_cpu_multi_thread_eigen_ = value; + // @@protoc_insertion_point(field_set:xla.DebugOptions.xla_cpu_multi_thread_eigen) +} + +// string xla_gpu_cuda_data_dir = 61; +inline void DebugOptions::clear_xla_gpu_cuda_data_dir() { + xla_gpu_cuda_data_dir_.ClearToEmptyNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); +} +inline const ::std::string& DebugOptions::xla_gpu_cuda_data_dir() const { + // @@protoc_insertion_point(field_get:xla.DebugOptions.xla_gpu_cuda_data_dir) + return xla_gpu_cuda_data_dir_.GetNoArena(); +} +inline void DebugOptions::set_xla_gpu_cuda_data_dir(const ::std::string& value) { + + xla_gpu_cuda_data_dir_.SetNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value); + // @@protoc_insertion_point(field_set:xla.DebugOptions.xla_gpu_cuda_data_dir) +} +#if LANG_CXX11 +inline void DebugOptions::set_xla_gpu_cuda_data_dir(::std::string&& value) { + + xla_gpu_cuda_data_dir_.SetNoArena( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value)); + // @@protoc_insertion_point(field_set_rvalue:xla.DebugOptions.xla_gpu_cuda_data_dir) +} +#endif +inline void DebugOptions::set_xla_gpu_cuda_data_dir(const char* value) { + GOOGLE_DCHECK(value != NULL); + + xla_gpu_cuda_data_dir_.SetNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value)); + // @@protoc_insertion_point(field_set_char:xla.DebugOptions.xla_gpu_cuda_data_dir) +} +inline void DebugOptions::set_xla_gpu_cuda_data_dir(const char* value, size_t size) { + + xla_gpu_cuda_data_dir_.SetNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + ::std::string(reinterpret_cast(value), size)); + // @@protoc_insertion_point(field_set_pointer:xla.DebugOptions.xla_gpu_cuda_data_dir) +} +inline ::std::string* DebugOptions::mutable_xla_gpu_cuda_data_dir() { + + // @@protoc_insertion_point(field_mutable:xla.DebugOptions.xla_gpu_cuda_data_dir) + return xla_gpu_cuda_data_dir_.MutableNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); +} +inline ::std::string* DebugOptions::release_xla_gpu_cuda_data_dir() { + // @@protoc_insertion_point(field_release:xla.DebugOptions.xla_gpu_cuda_data_dir) + + return xla_gpu_cuda_data_dir_.ReleaseNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); +} +inline void DebugOptions::set_allocated_xla_gpu_cuda_data_dir(::std::string* xla_gpu_cuda_data_dir) { + if (xla_gpu_cuda_data_dir != NULL) { + + } else { + + } + xla_gpu_cuda_data_dir_.SetAllocatedNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), xla_gpu_cuda_data_dir); + // @@protoc_insertion_point(field_set_allocated:xla.DebugOptions.xla_gpu_cuda_data_dir) +} + +// bool xla_gpu_ftz = 62; +inline void DebugOptions::clear_xla_gpu_ftz() { + xla_gpu_ftz_ = false; +} +inline bool DebugOptions::xla_gpu_ftz() const { + // @@protoc_insertion_point(field_get:xla.DebugOptions.xla_gpu_ftz) + return xla_gpu_ftz_; +} +inline void DebugOptions::set_xla_gpu_ftz(bool value) { + + xla_gpu_ftz_ = value; + // @@protoc_insertion_point(field_set:xla.DebugOptions.xla_gpu_ftz) +} + +// bool xla_gpu_disable_multi_streaming = 63; +inline void DebugOptions::clear_xla_gpu_disable_multi_streaming() { + xla_gpu_disable_multi_streaming_ = false; +} +inline bool DebugOptions::xla_gpu_disable_multi_streaming() const { + // @@protoc_insertion_point(field_get:xla.DebugOptions.xla_gpu_disable_multi_streaming) + return xla_gpu_disable_multi_streaming_; +} +inline void DebugOptions::set_xla_gpu_disable_multi_streaming(bool value) { + + xla_gpu_disable_multi_streaming_ = value; + // @@protoc_insertion_point(field_set:xla.DebugOptions.xla_gpu_disable_multi_streaming) +} + +// bool xla_llvm_enable_alias_scope_metadata = 70; +inline void DebugOptions::clear_xla_llvm_enable_alias_scope_metadata() { + xla_llvm_enable_alias_scope_metadata_ = false; +} +inline bool DebugOptions::xla_llvm_enable_alias_scope_metadata() const { + // @@protoc_insertion_point(field_get:xla.DebugOptions.xla_llvm_enable_alias_scope_metadata) + return xla_llvm_enable_alias_scope_metadata_; +} +inline void DebugOptions::set_xla_llvm_enable_alias_scope_metadata(bool value) { + + xla_llvm_enable_alias_scope_metadata_ = value; + // @@protoc_insertion_point(field_set:xla.DebugOptions.xla_llvm_enable_alias_scope_metadata) +} + +// bool xla_llvm_enable_noalias_metadata = 71; +inline void DebugOptions::clear_xla_llvm_enable_noalias_metadata() { + xla_llvm_enable_noalias_metadata_ = false; +} +inline bool DebugOptions::xla_llvm_enable_noalias_metadata() const { + // @@protoc_insertion_point(field_get:xla.DebugOptions.xla_llvm_enable_noalias_metadata) + return xla_llvm_enable_noalias_metadata_; +} +inline void DebugOptions::set_xla_llvm_enable_noalias_metadata(bool value) { + + xla_llvm_enable_noalias_metadata_ = value; + // @@protoc_insertion_point(field_set:xla.DebugOptions.xla_llvm_enable_noalias_metadata) +} + +// bool xla_llvm_enable_invariant_load_metadata = 72; +inline void DebugOptions::clear_xla_llvm_enable_invariant_load_metadata() { + xla_llvm_enable_invariant_load_metadata_ = false; +} +inline bool DebugOptions::xla_llvm_enable_invariant_load_metadata() const { + // @@protoc_insertion_point(field_get:xla.DebugOptions.xla_llvm_enable_invariant_load_metadata) + return xla_llvm_enable_invariant_load_metadata_; +} +inline void DebugOptions::set_xla_llvm_enable_invariant_load_metadata(bool value) { + + xla_llvm_enable_invariant_load_metadata_ = value; + // @@protoc_insertion_point(field_set:xla.DebugOptions.xla_llvm_enable_invariant_load_metadata) +} + +// bool xla_llvm_disable_expensive_passes = 73; +inline void DebugOptions::clear_xla_llvm_disable_expensive_passes() { + xla_llvm_disable_expensive_passes_ = false; +} +inline bool DebugOptions::xla_llvm_disable_expensive_passes() const { + // @@protoc_insertion_point(field_get:xla.DebugOptions.xla_llvm_disable_expensive_passes) + return xla_llvm_disable_expensive_passes_; +} +inline void DebugOptions::set_xla_llvm_disable_expensive_passes(bool value) { + + xla_llvm_disable_expensive_passes_ = value; + // @@protoc_insertion_point(field_set:xla.DebugOptions.xla_llvm_disable_expensive_passes) +} + +// repeated .xla.HloReducePrecisionOptions hlo_reduce_precision_options = 80; +inline int DebugOptions::hlo_reduce_precision_options_size() const { + return hlo_reduce_precision_options_.size(); +} +inline void DebugOptions::clear_hlo_reduce_precision_options() { + hlo_reduce_precision_options_.Clear(); +} +inline ::xla::HloReducePrecisionOptions* DebugOptions::mutable_hlo_reduce_precision_options(int index) { + // @@protoc_insertion_point(field_mutable:xla.DebugOptions.hlo_reduce_precision_options) + return hlo_reduce_precision_options_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::xla::HloReducePrecisionOptions >* +DebugOptions::mutable_hlo_reduce_precision_options() { + // @@protoc_insertion_point(field_mutable_list:xla.DebugOptions.hlo_reduce_precision_options) + return &hlo_reduce_precision_options_; +} +inline const ::xla::HloReducePrecisionOptions& DebugOptions::hlo_reduce_precision_options(int index) const { + // @@protoc_insertion_point(field_get:xla.DebugOptions.hlo_reduce_precision_options) + return hlo_reduce_precision_options_.Get(index); +} +inline ::xla::HloReducePrecisionOptions* DebugOptions::add_hlo_reduce_precision_options() { + // @@protoc_insertion_point(field_add:xla.DebugOptions.hlo_reduce_precision_options) + return hlo_reduce_precision_options_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::xla::HloReducePrecisionOptions >& +DebugOptions::hlo_reduce_precision_options() const { + // @@protoc_insertion_point(field_list:xla.DebugOptions.hlo_reduce_precision_options) + return hlo_reduce_precision_options_; +} + +// bool xla_test_all_output_layouts = 90; +inline void DebugOptions::clear_xla_test_all_output_layouts() { + xla_test_all_output_layouts_ = false; +} +inline bool DebugOptions::xla_test_all_output_layouts() const { + // @@protoc_insertion_point(field_get:xla.DebugOptions.xla_test_all_output_layouts) + return xla_test_all_output_layouts_; +} +inline void DebugOptions::set_xla_test_all_output_layouts(bool value) { + + xla_test_all_output_layouts_ = value; + // @@protoc_insertion_point(field_set:xla.DebugOptions.xla_test_all_output_layouts) +} + +// bool xla_test_all_input_layouts = 91; +inline void DebugOptions::clear_xla_test_all_input_layouts() { + xla_test_all_input_layouts_ = false; +} +inline bool DebugOptions::xla_test_all_input_layouts() const { + // @@protoc_insertion_point(field_get:xla.DebugOptions.xla_test_all_input_layouts) + return xla_test_all_input_layouts_; +} +inline void DebugOptions::set_xla_test_all_input_layouts(bool value) { + + xla_test_all_input_layouts_ = value; + // @@protoc_insertion_point(field_set:xla.DebugOptions.xla_test_all_input_layouts) +} + +// bool xla_hlo_graph_sharding_color = 92; +inline void DebugOptions::clear_xla_hlo_graph_sharding_color() { + xla_hlo_graph_sharding_color_ = false; +} +inline bool DebugOptions::xla_hlo_graph_sharding_color() const { + // @@protoc_insertion_point(field_get:xla.DebugOptions.xla_hlo_graph_sharding_color) + return xla_hlo_graph_sharding_color_; +} +inline void DebugOptions::set_xla_hlo_graph_sharding_color(bool value) { + + xla_hlo_graph_sharding_color_ = value; + // @@protoc_insertion_point(field_set:xla.DebugOptions.xla_hlo_graph_sharding_color) +} + +// bool xla_hlo_tfgraph_device_scopes = 93; +inline void DebugOptions::clear_xla_hlo_tfgraph_device_scopes() { + xla_hlo_tfgraph_device_scopes_ = false; +} +inline bool DebugOptions::xla_hlo_tfgraph_device_scopes() const { + // @@protoc_insertion_point(field_get:xla.DebugOptions.xla_hlo_tfgraph_device_scopes) + return xla_hlo_tfgraph_device_scopes_; +} +inline void DebugOptions::set_xla_hlo_tfgraph_device_scopes(bool value) { + + xla_hlo_tfgraph_device_scopes_ = value; + // @@protoc_insertion_point(field_set:xla.DebugOptions.xla_hlo_tfgraph_device_scopes) +} + +// bool xla_gpu_use_cudnn_batchnorm = 94; +inline void DebugOptions::clear_xla_gpu_use_cudnn_batchnorm() { + xla_gpu_use_cudnn_batchnorm_ = false; +} +inline bool DebugOptions::xla_gpu_use_cudnn_batchnorm() const { + // @@protoc_insertion_point(field_get:xla.DebugOptions.xla_gpu_use_cudnn_batchnorm) + return xla_gpu_use_cudnn_batchnorm_; +} +inline void DebugOptions::set_xla_gpu_use_cudnn_batchnorm(bool value) { + + xla_gpu_use_cudnn_batchnorm_ = value; + // @@protoc_insertion_point(field_set:xla.DebugOptions.xla_gpu_use_cudnn_batchnorm) +} + +// string xla_dump_unoptimized_hlo_proto_to = 95; +inline void DebugOptions::clear_xla_dump_unoptimized_hlo_proto_to() { + xla_dump_unoptimized_hlo_proto_to_.ClearToEmptyNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); +} +inline const ::std::string& DebugOptions::xla_dump_unoptimized_hlo_proto_to() const { + // @@protoc_insertion_point(field_get:xla.DebugOptions.xla_dump_unoptimized_hlo_proto_to) + return xla_dump_unoptimized_hlo_proto_to_.GetNoArena(); +} +inline void DebugOptions::set_xla_dump_unoptimized_hlo_proto_to(const ::std::string& value) { + + xla_dump_unoptimized_hlo_proto_to_.SetNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value); + // @@protoc_insertion_point(field_set:xla.DebugOptions.xla_dump_unoptimized_hlo_proto_to) +} +#if LANG_CXX11 +inline void DebugOptions::set_xla_dump_unoptimized_hlo_proto_to(::std::string&& value) { + + xla_dump_unoptimized_hlo_proto_to_.SetNoArena( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value)); + // @@protoc_insertion_point(field_set_rvalue:xla.DebugOptions.xla_dump_unoptimized_hlo_proto_to) +} +#endif +inline void DebugOptions::set_xla_dump_unoptimized_hlo_proto_to(const char* value) { + GOOGLE_DCHECK(value != NULL); + + xla_dump_unoptimized_hlo_proto_to_.SetNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value)); + // @@protoc_insertion_point(field_set_char:xla.DebugOptions.xla_dump_unoptimized_hlo_proto_to) +} +inline void DebugOptions::set_xla_dump_unoptimized_hlo_proto_to(const char* value, size_t size) { + + xla_dump_unoptimized_hlo_proto_to_.SetNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + ::std::string(reinterpret_cast(value), size)); + // @@protoc_insertion_point(field_set_pointer:xla.DebugOptions.xla_dump_unoptimized_hlo_proto_to) +} +inline ::std::string* DebugOptions::mutable_xla_dump_unoptimized_hlo_proto_to() { + + // @@protoc_insertion_point(field_mutable:xla.DebugOptions.xla_dump_unoptimized_hlo_proto_to) + return xla_dump_unoptimized_hlo_proto_to_.MutableNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); +} +inline ::std::string* DebugOptions::release_xla_dump_unoptimized_hlo_proto_to() { + // @@protoc_insertion_point(field_release:xla.DebugOptions.xla_dump_unoptimized_hlo_proto_to) + + return xla_dump_unoptimized_hlo_proto_to_.ReleaseNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); +} +inline void DebugOptions::set_allocated_xla_dump_unoptimized_hlo_proto_to(::std::string* xla_dump_unoptimized_hlo_proto_to) { + if (xla_dump_unoptimized_hlo_proto_to != NULL) { + + } else { + + } + xla_dump_unoptimized_hlo_proto_to_.SetAllocatedNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), xla_dump_unoptimized_hlo_proto_to); + // @@protoc_insertion_point(field_set_allocated:xla.DebugOptions.xla_dump_unoptimized_hlo_proto_to) +} + +// string xla_dump_per_pass_hlo_proto_to = 96; +inline void DebugOptions::clear_xla_dump_per_pass_hlo_proto_to() { + xla_dump_per_pass_hlo_proto_to_.ClearToEmptyNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); +} +inline const ::std::string& DebugOptions::xla_dump_per_pass_hlo_proto_to() const { + // @@protoc_insertion_point(field_get:xla.DebugOptions.xla_dump_per_pass_hlo_proto_to) + return xla_dump_per_pass_hlo_proto_to_.GetNoArena(); +} +inline void DebugOptions::set_xla_dump_per_pass_hlo_proto_to(const ::std::string& value) { + + xla_dump_per_pass_hlo_proto_to_.SetNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value); + // @@protoc_insertion_point(field_set:xla.DebugOptions.xla_dump_per_pass_hlo_proto_to) +} +#if LANG_CXX11 +inline void DebugOptions::set_xla_dump_per_pass_hlo_proto_to(::std::string&& value) { + + xla_dump_per_pass_hlo_proto_to_.SetNoArena( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value)); + // @@protoc_insertion_point(field_set_rvalue:xla.DebugOptions.xla_dump_per_pass_hlo_proto_to) +} +#endif +inline void DebugOptions::set_xla_dump_per_pass_hlo_proto_to(const char* value) { + GOOGLE_DCHECK(value != NULL); + + xla_dump_per_pass_hlo_proto_to_.SetNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value)); + // @@protoc_insertion_point(field_set_char:xla.DebugOptions.xla_dump_per_pass_hlo_proto_to) +} +inline void DebugOptions::set_xla_dump_per_pass_hlo_proto_to(const char* value, size_t size) { + + xla_dump_per_pass_hlo_proto_to_.SetNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + ::std::string(reinterpret_cast(value), size)); + // @@protoc_insertion_point(field_set_pointer:xla.DebugOptions.xla_dump_per_pass_hlo_proto_to) +} +inline ::std::string* DebugOptions::mutable_xla_dump_per_pass_hlo_proto_to() { + + // @@protoc_insertion_point(field_mutable:xla.DebugOptions.xla_dump_per_pass_hlo_proto_to) + return xla_dump_per_pass_hlo_proto_to_.MutableNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); +} +inline ::std::string* DebugOptions::release_xla_dump_per_pass_hlo_proto_to() { + // @@protoc_insertion_point(field_release:xla.DebugOptions.xla_dump_per_pass_hlo_proto_to) + + return xla_dump_per_pass_hlo_proto_to_.ReleaseNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); +} +inline void DebugOptions::set_allocated_xla_dump_per_pass_hlo_proto_to(::std::string* xla_dump_per_pass_hlo_proto_to) { + if (xla_dump_per_pass_hlo_proto_to != NULL) { + + } else { + + } + xla_dump_per_pass_hlo_proto_to_.SetAllocatedNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), xla_dump_per_pass_hlo_proto_to); + // @@protoc_insertion_point(field_set_allocated:xla.DebugOptions.xla_dump_per_pass_hlo_proto_to) +} + +// bool xla_cpu_use_mkl_dnn = 97; +inline void DebugOptions::clear_xla_cpu_use_mkl_dnn() { + xla_cpu_use_mkl_dnn_ = false; +} +inline bool DebugOptions::xla_cpu_use_mkl_dnn() const { + // @@protoc_insertion_point(field_get:xla.DebugOptions.xla_cpu_use_mkl_dnn) + return xla_cpu_use_mkl_dnn_; +} +inline void DebugOptions::set_xla_cpu_use_mkl_dnn(bool value) { + + xla_cpu_use_mkl_dnn_ = value; + // @@protoc_insertion_point(field_set:xla.DebugOptions.xla_cpu_use_mkl_dnn) +} + +// int32 xla_gpu_max_kernel_unroll_factor = 98; +inline void DebugOptions::clear_xla_gpu_max_kernel_unroll_factor() { + xla_gpu_max_kernel_unroll_factor_ = 0; +} +inline ::google::protobuf::int32 DebugOptions::xla_gpu_max_kernel_unroll_factor() const { + // @@protoc_insertion_point(field_get:xla.DebugOptions.xla_gpu_max_kernel_unroll_factor) + return xla_gpu_max_kernel_unroll_factor_; +} +inline void DebugOptions::set_xla_gpu_max_kernel_unroll_factor(::google::protobuf::int32 value) { + + xla_gpu_max_kernel_unroll_factor_ = value; + // @@protoc_insertion_point(field_set:xla.DebugOptions.xla_gpu_max_kernel_unroll_factor) +} + +// bool xla_cpu_enable_fast_math = 99; +inline void DebugOptions::clear_xla_cpu_enable_fast_math() { + xla_cpu_enable_fast_math_ = false; +} +inline bool DebugOptions::xla_cpu_enable_fast_math() const { + // @@protoc_insertion_point(field_get:xla.DebugOptions.xla_cpu_enable_fast_math) + return xla_cpu_enable_fast_math_; +} +inline void DebugOptions::set_xla_cpu_enable_fast_math(bool value) { + + xla_cpu_enable_fast_math_ = value; + // @@protoc_insertion_point(field_set:xla.DebugOptions.xla_cpu_enable_fast_math) +} + +// bool xla_gpu_enable_fast_math = 100; +inline void DebugOptions::clear_xla_gpu_enable_fast_math() { + xla_gpu_enable_fast_math_ = false; +} +inline bool DebugOptions::xla_gpu_enable_fast_math() const { + // @@protoc_insertion_point(field_get:xla.DebugOptions.xla_gpu_enable_fast_math) + return xla_gpu_enable_fast_math_; +} +inline void DebugOptions::set_xla_gpu_enable_fast_math(bool value) { + + xla_gpu_enable_fast_math_ = value; + // @@protoc_insertion_point(field_set:xla.DebugOptions.xla_gpu_enable_fast_math) +} + +// bool xla_gpu_crash_on_verification_failures = 101; +inline void DebugOptions::clear_xla_gpu_crash_on_verification_failures() { + xla_gpu_crash_on_verification_failures_ = false; +} +inline bool DebugOptions::xla_gpu_crash_on_verification_failures() const { + // @@protoc_insertion_point(field_get:xla.DebugOptions.xla_gpu_crash_on_verification_failures) + return xla_gpu_crash_on_verification_failures_; +} +inline void DebugOptions::set_xla_gpu_crash_on_verification_failures(bool value) { + + xla_gpu_crash_on_verification_failures_ = value; + // @@protoc_insertion_point(field_set:xla.DebugOptions.xla_gpu_crash_on_verification_failures) +} + +// int32 xla_force_host_platform_device_count = 102; +inline void DebugOptions::clear_xla_force_host_platform_device_count() { + xla_force_host_platform_device_count_ = 0; +} +inline ::google::protobuf::int32 DebugOptions::xla_force_host_platform_device_count() const { + // @@protoc_insertion_point(field_get:xla.DebugOptions.xla_force_host_platform_device_count) + return xla_force_host_platform_device_count_; +} +inline void DebugOptions::set_xla_force_host_platform_device_count(::google::protobuf::int32 value) { + + xla_force_host_platform_device_count_ = value; + // @@protoc_insertion_point(field_set:xla.DebugOptions.xla_force_host_platform_device_count) +} + +// map xla_backend_extra_options = 500; +inline int DebugOptions::xla_backend_extra_options_size() const { + return xla_backend_extra_options_.size(); +} +inline void DebugOptions::clear_xla_backend_extra_options() { + xla_backend_extra_options_.Clear(); +} +inline const ::google::protobuf::Map< ::std::string, ::std::string >& +DebugOptions::xla_backend_extra_options() const { + // @@protoc_insertion_point(field_map:xla.DebugOptions.xla_backend_extra_options) + return xla_backend_extra_options_.GetMap(); +} +inline ::google::protobuf::Map< ::std::string, ::std::string >* +DebugOptions::mutable_xla_backend_extra_options() { + // @@protoc_insertion_point(field_mutable_map:xla.DebugOptions.xla_backend_extra_options) + return xla_backend_extra_options_.MutableMap(); +} + +// ------------------------------------------------------------------- + +// ExecutionOptions + +// .xla.Shape shape_with_output_layout = 2; +inline bool ExecutionOptions::has_shape_with_output_layout() const { + return this != internal_default_instance() && shape_with_output_layout_ != NULL; +} +inline const ::xla::Shape& ExecutionOptions::_internal_shape_with_output_layout() const { + return *shape_with_output_layout_; +} +inline const ::xla::Shape& ExecutionOptions::shape_with_output_layout() const { + const ::xla::Shape* p = shape_with_output_layout_; + // @@protoc_insertion_point(field_get:xla.ExecutionOptions.shape_with_output_layout) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_Shape_default_instance_); +} +inline ::xla::Shape* ExecutionOptions::release_shape_with_output_layout() { + // @@protoc_insertion_point(field_release:xla.ExecutionOptions.shape_with_output_layout) + + ::xla::Shape* temp = shape_with_output_layout_; + shape_with_output_layout_ = NULL; + return temp; +} +inline ::xla::Shape* ExecutionOptions::mutable_shape_with_output_layout() { + + if (shape_with_output_layout_ == NULL) { + auto* p = CreateMaybeMessage<::xla::Shape>(GetArenaNoVirtual()); + shape_with_output_layout_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.ExecutionOptions.shape_with_output_layout) + return shape_with_output_layout_; +} +inline void ExecutionOptions::set_allocated_shape_with_output_layout(::xla::Shape* shape_with_output_layout) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(shape_with_output_layout_); + } + if (shape_with_output_layout) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(shape_with_output_layout)->GetArena(); + if (message_arena != submessage_arena) { + shape_with_output_layout = ::google::protobuf::internal::GetOwnedMessage( + message_arena, shape_with_output_layout, submessage_arena); + } + + } else { + + } + shape_with_output_layout_ = shape_with_output_layout; + // @@protoc_insertion_point(field_set_allocated:xla.ExecutionOptions.shape_with_output_layout) +} + +// uint64 seed = 3; +inline void ExecutionOptions::clear_seed() { + seed_ = GOOGLE_ULONGLONG(0); +} +inline ::google::protobuf::uint64 ExecutionOptions::seed() const { + // @@protoc_insertion_point(field_get:xla.ExecutionOptions.seed) + return seed_; +} +inline void ExecutionOptions::set_seed(::google::protobuf::uint64 value) { + + seed_ = value; + // @@protoc_insertion_point(field_set:xla.ExecutionOptions.seed) +} + +// .xla.DebugOptions debug_options = 4; +inline bool ExecutionOptions::has_debug_options() const { + return this != internal_default_instance() && debug_options_ != NULL; +} +inline void ExecutionOptions::clear_debug_options() { + if (GetArenaNoVirtual() == NULL && debug_options_ != NULL) { + delete debug_options_; + } + debug_options_ = NULL; +} +inline const ::xla::DebugOptions& ExecutionOptions::_internal_debug_options() const { + return *debug_options_; +} +inline const ::xla::DebugOptions& ExecutionOptions::debug_options() const { + const ::xla::DebugOptions* p = debug_options_; + // @@protoc_insertion_point(field_get:xla.ExecutionOptions.debug_options) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_DebugOptions_default_instance_); +} +inline ::xla::DebugOptions* ExecutionOptions::release_debug_options() { + // @@protoc_insertion_point(field_release:xla.ExecutionOptions.debug_options) + + ::xla::DebugOptions* temp = debug_options_; + debug_options_ = NULL; + return temp; +} +inline ::xla::DebugOptions* ExecutionOptions::mutable_debug_options() { + + if (debug_options_ == NULL) { + auto* p = CreateMaybeMessage<::xla::DebugOptions>(GetArenaNoVirtual()); + debug_options_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.ExecutionOptions.debug_options) + return debug_options_; +} +inline void ExecutionOptions::set_allocated_debug_options(::xla::DebugOptions* debug_options) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete debug_options_; + } + if (debug_options) { + ::google::protobuf::Arena* submessage_arena = NULL; + if (message_arena != submessage_arena) { + debug_options = ::google::protobuf::internal::GetOwnedMessage( + message_arena, debug_options, submessage_arena); + } + + } else { + + } + debug_options_ = debug_options; + // @@protoc_insertion_point(field_set_allocated:xla.ExecutionOptions.debug_options) +} + +// repeated .xla.DeviceHandle device_handles = 5; +inline int ExecutionOptions::device_handles_size() const { + return device_handles_.size(); +} +inline ::xla::DeviceHandle* ExecutionOptions::mutable_device_handles(int index) { + // @@protoc_insertion_point(field_mutable:xla.ExecutionOptions.device_handles) + return device_handles_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::xla::DeviceHandle >* +ExecutionOptions::mutable_device_handles() { + // @@protoc_insertion_point(field_mutable_list:xla.ExecutionOptions.device_handles) + return &device_handles_; +} +inline const ::xla::DeviceHandle& ExecutionOptions::device_handles(int index) const { + // @@protoc_insertion_point(field_get:xla.ExecutionOptions.device_handles) + return device_handles_.Get(index); +} +inline ::xla::DeviceHandle* ExecutionOptions::add_device_handles() { + // @@protoc_insertion_point(field_add:xla.ExecutionOptions.device_handles) + return device_handles_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::xla::DeviceHandle >& +ExecutionOptions::device_handles() const { + // @@protoc_insertion_point(field_list:xla.ExecutionOptions.device_handles) + return device_handles_; +} + +// ------------------------------------------------------------------- + +// GetDeviceHandlesRequest + +// int64 device_count = 1; +inline void GetDeviceHandlesRequest::clear_device_count() { + device_count_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 GetDeviceHandlesRequest::device_count() const { + // @@protoc_insertion_point(field_get:xla.GetDeviceHandlesRequest.device_count) + return device_count_; +} +inline void GetDeviceHandlesRequest::set_device_count(::google::protobuf::int64 value) { + + device_count_ = value; + // @@protoc_insertion_point(field_set:xla.GetDeviceHandlesRequest.device_count) +} + +// ------------------------------------------------------------------- + +// GetDeviceHandlesResponse + +// repeated .xla.DeviceHandle device_handles = 1; +inline int GetDeviceHandlesResponse::device_handles_size() const { + return device_handles_.size(); +} +inline ::xla::DeviceHandle* GetDeviceHandlesResponse::mutable_device_handles(int index) { + // @@protoc_insertion_point(field_mutable:xla.GetDeviceHandlesResponse.device_handles) + return device_handles_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::xla::DeviceHandle >* +GetDeviceHandlesResponse::mutable_device_handles() { + // @@protoc_insertion_point(field_mutable_list:xla.GetDeviceHandlesResponse.device_handles) + return &device_handles_; +} +inline const ::xla::DeviceHandle& GetDeviceHandlesResponse::device_handles(int index) const { + // @@protoc_insertion_point(field_get:xla.GetDeviceHandlesResponse.device_handles) + return device_handles_.Get(index); +} +inline ::xla::DeviceHandle* GetDeviceHandlesResponse::add_device_handles() { + // @@protoc_insertion_point(field_add:xla.GetDeviceHandlesResponse.device_handles) + return device_handles_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::xla::DeviceHandle >& +GetDeviceHandlesResponse::device_handles() const { + // @@protoc_insertion_point(field_list:xla.GetDeviceHandlesResponse.device_handles) + return device_handles_; +} + +// ------------------------------------------------------------------- + +// TransferToClientRequest + +// .xla.GlobalDataHandle data = 1; +inline bool TransferToClientRequest::has_data() const { + return this != internal_default_instance() && data_ != NULL; +} +inline const ::xla::GlobalDataHandle& TransferToClientRequest::_internal_data() const { + return *data_; +} +inline const ::xla::GlobalDataHandle& TransferToClientRequest::data() const { + const ::xla::GlobalDataHandle* p = data_; + // @@protoc_insertion_point(field_get:xla.TransferToClientRequest.data) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_GlobalDataHandle_default_instance_); +} +inline ::xla::GlobalDataHandle* TransferToClientRequest::release_data() { + // @@protoc_insertion_point(field_release:xla.TransferToClientRequest.data) + + ::xla::GlobalDataHandle* temp = data_; + data_ = NULL; + return temp; +} +inline ::xla::GlobalDataHandle* TransferToClientRequest::mutable_data() { + + if (data_ == NULL) { + auto* p = CreateMaybeMessage<::xla::GlobalDataHandle>(GetArenaNoVirtual()); + data_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.TransferToClientRequest.data) + return data_; +} +inline void TransferToClientRequest::set_allocated_data(::xla::GlobalDataHandle* data) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(data_); + } + if (data) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(data)->GetArena(); + if (message_arena != submessage_arena) { + data = ::google::protobuf::internal::GetOwnedMessage( + message_arena, data, submessage_arena); + } + + } else { + + } + data_ = data; + // @@protoc_insertion_point(field_set_allocated:xla.TransferToClientRequest.data) +} + +// .xla.Shape shape_with_layout = 2; +inline bool TransferToClientRequest::has_shape_with_layout() const { + return this != internal_default_instance() && shape_with_layout_ != NULL; +} +inline const ::xla::Shape& TransferToClientRequest::_internal_shape_with_layout() const { + return *shape_with_layout_; +} +inline const ::xla::Shape& TransferToClientRequest::shape_with_layout() const { + const ::xla::Shape* p = shape_with_layout_; + // @@protoc_insertion_point(field_get:xla.TransferToClientRequest.shape_with_layout) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_Shape_default_instance_); +} +inline ::xla::Shape* TransferToClientRequest::release_shape_with_layout() { + // @@protoc_insertion_point(field_release:xla.TransferToClientRequest.shape_with_layout) + + ::xla::Shape* temp = shape_with_layout_; + shape_with_layout_ = NULL; + return temp; +} +inline ::xla::Shape* TransferToClientRequest::mutable_shape_with_layout() { + + if (shape_with_layout_ == NULL) { + auto* p = CreateMaybeMessage<::xla::Shape>(GetArenaNoVirtual()); + shape_with_layout_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.TransferToClientRequest.shape_with_layout) + return shape_with_layout_; +} +inline void TransferToClientRequest::set_allocated_shape_with_layout(::xla::Shape* shape_with_layout) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(shape_with_layout_); + } + if (shape_with_layout) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(shape_with_layout)->GetArena(); + if (message_arena != submessage_arena) { + shape_with_layout = ::google::protobuf::internal::GetOwnedMessage( + message_arena, shape_with_layout, submessage_arena); + } + + } else { + + } + shape_with_layout_ = shape_with_layout; + // @@protoc_insertion_point(field_set_allocated:xla.TransferToClientRequest.shape_with_layout) +} + +// ------------------------------------------------------------------- + +// TransferToClientResponse + +// .xla.LiteralProto literal = 1; +inline bool TransferToClientResponse::has_literal() const { + return this != internal_default_instance() && literal_ != NULL; +} +inline const ::xla::LiteralProto& TransferToClientResponse::_internal_literal() const { + return *literal_; +} +inline const ::xla::LiteralProto& TransferToClientResponse::literal() const { + const ::xla::LiteralProto* p = literal_; + // @@protoc_insertion_point(field_get:xla.TransferToClientResponse.literal) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_LiteralProto_default_instance_); +} +inline ::xla::LiteralProto* TransferToClientResponse::release_literal() { + // @@protoc_insertion_point(field_release:xla.TransferToClientResponse.literal) + + ::xla::LiteralProto* temp = literal_; + literal_ = NULL; + return temp; +} +inline ::xla::LiteralProto* TransferToClientResponse::mutable_literal() { + + if (literal_ == NULL) { + auto* p = CreateMaybeMessage<::xla::LiteralProto>(GetArenaNoVirtual()); + literal_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.TransferToClientResponse.literal) + return literal_; +} +inline void TransferToClientResponse::set_allocated_literal(::xla::LiteralProto* literal) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(literal_); + } + if (literal) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(literal)->GetArena(); + if (message_arena != submessage_arena) { + literal = ::google::protobuf::internal::GetOwnedMessage( + message_arena, literal, submessage_arena); + } + + } else { + + } + literal_ = literal; + // @@protoc_insertion_point(field_set_allocated:xla.TransferToClientResponse.literal) +} + +// ------------------------------------------------------------------- + +// TransferToServerRequest + +// .xla.LiteralProto literal = 1; +inline bool TransferToServerRequest::has_literal() const { + return this != internal_default_instance() && literal_ != NULL; +} +inline const ::xla::LiteralProto& TransferToServerRequest::_internal_literal() const { + return *literal_; +} +inline const ::xla::LiteralProto& TransferToServerRequest::literal() const { + const ::xla::LiteralProto* p = literal_; + // @@protoc_insertion_point(field_get:xla.TransferToServerRequest.literal) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_LiteralProto_default_instance_); +} +inline ::xla::LiteralProto* TransferToServerRequest::release_literal() { + // @@protoc_insertion_point(field_release:xla.TransferToServerRequest.literal) + + ::xla::LiteralProto* temp = literal_; + literal_ = NULL; + return temp; +} +inline ::xla::LiteralProto* TransferToServerRequest::mutable_literal() { + + if (literal_ == NULL) { + auto* p = CreateMaybeMessage<::xla::LiteralProto>(GetArenaNoVirtual()); + literal_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.TransferToServerRequest.literal) + return literal_; +} +inline void TransferToServerRequest::set_allocated_literal(::xla::LiteralProto* literal) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(literal_); + } + if (literal) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(literal)->GetArena(); + if (message_arena != submessage_arena) { + literal = ::google::protobuf::internal::GetOwnedMessage( + message_arena, literal, submessage_arena); + } + + } else { + + } + literal_ = literal; + // @@protoc_insertion_point(field_set_allocated:xla.TransferToServerRequest.literal) +} + +// .xla.DeviceHandle device_handle = 2; +inline bool TransferToServerRequest::has_device_handle() const { + return this != internal_default_instance() && device_handle_ != NULL; +} +inline const ::xla::DeviceHandle& TransferToServerRequest::_internal_device_handle() const { + return *device_handle_; +} +inline const ::xla::DeviceHandle& TransferToServerRequest::device_handle() const { + const ::xla::DeviceHandle* p = device_handle_; + // @@protoc_insertion_point(field_get:xla.TransferToServerRequest.device_handle) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_DeviceHandle_default_instance_); +} +inline ::xla::DeviceHandle* TransferToServerRequest::release_device_handle() { + // @@protoc_insertion_point(field_release:xla.TransferToServerRequest.device_handle) + + ::xla::DeviceHandle* temp = device_handle_; + device_handle_ = NULL; + return temp; +} +inline ::xla::DeviceHandle* TransferToServerRequest::mutable_device_handle() { + + if (device_handle_ == NULL) { + auto* p = CreateMaybeMessage<::xla::DeviceHandle>(GetArenaNoVirtual()); + device_handle_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.TransferToServerRequest.device_handle) + return device_handle_; +} +inline void TransferToServerRequest::set_allocated_device_handle(::xla::DeviceHandle* device_handle) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(device_handle_); + } + if (device_handle) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(device_handle)->GetArena(); + if (message_arena != submessage_arena) { + device_handle = ::google::protobuf::internal::GetOwnedMessage( + message_arena, device_handle, submessage_arena); + } + + } else { + + } + device_handle_ = device_handle; + // @@protoc_insertion_point(field_set_allocated:xla.TransferToServerRequest.device_handle) +} + +// ------------------------------------------------------------------- + +// TransferToServerResponse + +// .xla.GlobalDataHandle data = 1; +inline bool TransferToServerResponse::has_data() const { + return this != internal_default_instance() && data_ != NULL; +} +inline const ::xla::GlobalDataHandle& TransferToServerResponse::_internal_data() const { + return *data_; +} +inline const ::xla::GlobalDataHandle& TransferToServerResponse::data() const { + const ::xla::GlobalDataHandle* p = data_; + // @@protoc_insertion_point(field_get:xla.TransferToServerResponse.data) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_GlobalDataHandle_default_instance_); +} +inline ::xla::GlobalDataHandle* TransferToServerResponse::release_data() { + // @@protoc_insertion_point(field_release:xla.TransferToServerResponse.data) + + ::xla::GlobalDataHandle* temp = data_; + data_ = NULL; + return temp; +} +inline ::xla::GlobalDataHandle* TransferToServerResponse::mutable_data() { + + if (data_ == NULL) { + auto* p = CreateMaybeMessage<::xla::GlobalDataHandle>(GetArenaNoVirtual()); + data_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.TransferToServerResponse.data) + return data_; +} +inline void TransferToServerResponse::set_allocated_data(::xla::GlobalDataHandle* data) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(data_); + } + if (data) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(data)->GetArena(); + if (message_arena != submessage_arena) { + data = ::google::protobuf::internal::GetOwnedMessage( + message_arena, data, submessage_arena); + } + + } else { + + } + data_ = data; + // @@protoc_insertion_point(field_set_allocated:xla.TransferToServerResponse.data) +} + +// ------------------------------------------------------------------- + +// TransferToInfeedRequest + +// .xla.LiteralProto literal = 1; +inline bool TransferToInfeedRequest::has_literal() const { + return this != internal_default_instance() && literal_ != NULL; +} +inline const ::xla::LiteralProto& TransferToInfeedRequest::_internal_literal() const { + return *literal_; +} +inline const ::xla::LiteralProto& TransferToInfeedRequest::literal() const { + const ::xla::LiteralProto* p = literal_; + // @@protoc_insertion_point(field_get:xla.TransferToInfeedRequest.literal) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_LiteralProto_default_instance_); +} +inline ::xla::LiteralProto* TransferToInfeedRequest::release_literal() { + // @@protoc_insertion_point(field_release:xla.TransferToInfeedRequest.literal) + + ::xla::LiteralProto* temp = literal_; + literal_ = NULL; + return temp; +} +inline ::xla::LiteralProto* TransferToInfeedRequest::mutable_literal() { + + if (literal_ == NULL) { + auto* p = CreateMaybeMessage<::xla::LiteralProto>(GetArenaNoVirtual()); + literal_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.TransferToInfeedRequest.literal) + return literal_; +} +inline void TransferToInfeedRequest::set_allocated_literal(::xla::LiteralProto* literal) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(literal_); + } + if (literal) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(literal)->GetArena(); + if (message_arena != submessage_arena) { + literal = ::google::protobuf::internal::GetOwnedMessage( + message_arena, literal, submessage_arena); + } + + } else { + + } + literal_ = literal; + // @@protoc_insertion_point(field_set_allocated:xla.TransferToInfeedRequest.literal) +} + +// int64 replica_id = 2; +inline void TransferToInfeedRequest::clear_replica_id() { + replica_id_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 TransferToInfeedRequest::replica_id() const { + // @@protoc_insertion_point(field_get:xla.TransferToInfeedRequest.replica_id) + return replica_id_; +} +inline void TransferToInfeedRequest::set_replica_id(::google::protobuf::int64 value) { + + replica_id_ = value; + // @@protoc_insertion_point(field_set:xla.TransferToInfeedRequest.replica_id) +} + +// .xla.DeviceHandle device_handle = 3; +inline bool TransferToInfeedRequest::has_device_handle() const { + return this != internal_default_instance() && device_handle_ != NULL; +} +inline const ::xla::DeviceHandle& TransferToInfeedRequest::_internal_device_handle() const { + return *device_handle_; +} +inline const ::xla::DeviceHandle& TransferToInfeedRequest::device_handle() const { + const ::xla::DeviceHandle* p = device_handle_; + // @@protoc_insertion_point(field_get:xla.TransferToInfeedRequest.device_handle) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_DeviceHandle_default_instance_); +} +inline ::xla::DeviceHandle* TransferToInfeedRequest::release_device_handle() { + // @@protoc_insertion_point(field_release:xla.TransferToInfeedRequest.device_handle) + + ::xla::DeviceHandle* temp = device_handle_; + device_handle_ = NULL; + return temp; +} +inline ::xla::DeviceHandle* TransferToInfeedRequest::mutable_device_handle() { + + if (device_handle_ == NULL) { + auto* p = CreateMaybeMessage<::xla::DeviceHandle>(GetArenaNoVirtual()); + device_handle_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.TransferToInfeedRequest.device_handle) + return device_handle_; +} +inline void TransferToInfeedRequest::set_allocated_device_handle(::xla::DeviceHandle* device_handle) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(device_handle_); + } + if (device_handle) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(device_handle)->GetArena(); + if (message_arena != submessage_arena) { + device_handle = ::google::protobuf::internal::GetOwnedMessage( + message_arena, device_handle, submessage_arena); + } + + } else { + + } + device_handle_ = device_handle; + // @@protoc_insertion_point(field_set_allocated:xla.TransferToInfeedRequest.device_handle) +} + +// ------------------------------------------------------------------- + +// TransferToInfeedResponse + +// ------------------------------------------------------------------- + +// TransferFromOutfeedRequest + +// .xla.Shape shape_with_layout = 1; +inline bool TransferFromOutfeedRequest::has_shape_with_layout() const { + return this != internal_default_instance() && shape_with_layout_ != NULL; +} +inline const ::xla::Shape& TransferFromOutfeedRequest::_internal_shape_with_layout() const { + return *shape_with_layout_; +} +inline const ::xla::Shape& TransferFromOutfeedRequest::shape_with_layout() const { + const ::xla::Shape* p = shape_with_layout_; + // @@protoc_insertion_point(field_get:xla.TransferFromOutfeedRequest.shape_with_layout) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_Shape_default_instance_); +} +inline ::xla::Shape* TransferFromOutfeedRequest::release_shape_with_layout() { + // @@protoc_insertion_point(field_release:xla.TransferFromOutfeedRequest.shape_with_layout) + + ::xla::Shape* temp = shape_with_layout_; + shape_with_layout_ = NULL; + return temp; +} +inline ::xla::Shape* TransferFromOutfeedRequest::mutable_shape_with_layout() { + + if (shape_with_layout_ == NULL) { + auto* p = CreateMaybeMessage<::xla::Shape>(GetArenaNoVirtual()); + shape_with_layout_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.TransferFromOutfeedRequest.shape_with_layout) + return shape_with_layout_; +} +inline void TransferFromOutfeedRequest::set_allocated_shape_with_layout(::xla::Shape* shape_with_layout) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(shape_with_layout_); + } + if (shape_with_layout) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(shape_with_layout)->GetArena(); + if (message_arena != submessage_arena) { + shape_with_layout = ::google::protobuf::internal::GetOwnedMessage( + message_arena, shape_with_layout, submessage_arena); + } + + } else { + + } + shape_with_layout_ = shape_with_layout; + // @@protoc_insertion_point(field_set_allocated:xla.TransferFromOutfeedRequest.shape_with_layout) +} + +// int64 replica_id = 2; +inline void TransferFromOutfeedRequest::clear_replica_id() { + replica_id_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 TransferFromOutfeedRequest::replica_id() const { + // @@protoc_insertion_point(field_get:xla.TransferFromOutfeedRequest.replica_id) + return replica_id_; +} +inline void TransferFromOutfeedRequest::set_replica_id(::google::protobuf::int64 value) { + + replica_id_ = value; + // @@protoc_insertion_point(field_set:xla.TransferFromOutfeedRequest.replica_id) +} + +// .xla.DeviceHandle device_handle = 3; +inline bool TransferFromOutfeedRequest::has_device_handle() const { + return this != internal_default_instance() && device_handle_ != NULL; +} +inline const ::xla::DeviceHandle& TransferFromOutfeedRequest::_internal_device_handle() const { + return *device_handle_; +} +inline const ::xla::DeviceHandle& TransferFromOutfeedRequest::device_handle() const { + const ::xla::DeviceHandle* p = device_handle_; + // @@protoc_insertion_point(field_get:xla.TransferFromOutfeedRequest.device_handle) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_DeviceHandle_default_instance_); +} +inline ::xla::DeviceHandle* TransferFromOutfeedRequest::release_device_handle() { + // @@protoc_insertion_point(field_release:xla.TransferFromOutfeedRequest.device_handle) + + ::xla::DeviceHandle* temp = device_handle_; + device_handle_ = NULL; + return temp; +} +inline ::xla::DeviceHandle* TransferFromOutfeedRequest::mutable_device_handle() { + + if (device_handle_ == NULL) { + auto* p = CreateMaybeMessage<::xla::DeviceHandle>(GetArenaNoVirtual()); + device_handle_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.TransferFromOutfeedRequest.device_handle) + return device_handle_; +} +inline void TransferFromOutfeedRequest::set_allocated_device_handle(::xla::DeviceHandle* device_handle) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(device_handle_); + } + if (device_handle) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(device_handle)->GetArena(); + if (message_arena != submessage_arena) { + device_handle = ::google::protobuf::internal::GetOwnedMessage( + message_arena, device_handle, submessage_arena); + } + + } else { + + } + device_handle_ = device_handle; + // @@protoc_insertion_point(field_set_allocated:xla.TransferFromOutfeedRequest.device_handle) +} + +// ------------------------------------------------------------------- + +// TransferFromOutfeedResponse + +// .xla.LiteralProto literal = 1; +inline bool TransferFromOutfeedResponse::has_literal() const { + return this != internal_default_instance() && literal_ != NULL; +} +inline const ::xla::LiteralProto& TransferFromOutfeedResponse::_internal_literal() const { + return *literal_; +} +inline const ::xla::LiteralProto& TransferFromOutfeedResponse::literal() const { + const ::xla::LiteralProto* p = literal_; + // @@protoc_insertion_point(field_get:xla.TransferFromOutfeedResponse.literal) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_LiteralProto_default_instance_); +} +inline ::xla::LiteralProto* TransferFromOutfeedResponse::release_literal() { + // @@protoc_insertion_point(field_release:xla.TransferFromOutfeedResponse.literal) + + ::xla::LiteralProto* temp = literal_; + literal_ = NULL; + return temp; +} +inline ::xla::LiteralProto* TransferFromOutfeedResponse::mutable_literal() { + + if (literal_ == NULL) { + auto* p = CreateMaybeMessage<::xla::LiteralProto>(GetArenaNoVirtual()); + literal_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.TransferFromOutfeedResponse.literal) + return literal_; +} +inline void TransferFromOutfeedResponse::set_allocated_literal(::xla::LiteralProto* literal) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(literal_); + } + if (literal) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(literal)->GetArena(); + if (message_arena != submessage_arena) { + literal = ::google::protobuf::internal::GetOwnedMessage( + message_arena, literal, submessage_arena); + } + + } else { + + } + literal_ = literal; + // @@protoc_insertion_point(field_set_allocated:xla.TransferFromOutfeedResponse.literal) +} + +// ------------------------------------------------------------------- + +// ResetDeviceRequest + +// .xla.DeviceHandle device_handle = 1; +inline bool ResetDeviceRequest::has_device_handle() const { + return this != internal_default_instance() && device_handle_ != NULL; +} +inline const ::xla::DeviceHandle& ResetDeviceRequest::_internal_device_handle() const { + return *device_handle_; +} +inline const ::xla::DeviceHandle& ResetDeviceRequest::device_handle() const { + const ::xla::DeviceHandle* p = device_handle_; + // @@protoc_insertion_point(field_get:xla.ResetDeviceRequest.device_handle) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_DeviceHandle_default_instance_); +} +inline ::xla::DeviceHandle* ResetDeviceRequest::release_device_handle() { + // @@protoc_insertion_point(field_release:xla.ResetDeviceRequest.device_handle) + + ::xla::DeviceHandle* temp = device_handle_; + device_handle_ = NULL; + return temp; +} +inline ::xla::DeviceHandle* ResetDeviceRequest::mutable_device_handle() { + + if (device_handle_ == NULL) { + auto* p = CreateMaybeMessage<::xla::DeviceHandle>(GetArenaNoVirtual()); + device_handle_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.ResetDeviceRequest.device_handle) + return device_handle_; +} +inline void ResetDeviceRequest::set_allocated_device_handle(::xla::DeviceHandle* device_handle) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(device_handle_); + } + if (device_handle) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(device_handle)->GetArena(); + if (message_arena != submessage_arena) { + device_handle = ::google::protobuf::internal::GetOwnedMessage( + message_arena, device_handle, submessage_arena); + } + + } else { + + } + device_handle_ = device_handle; + // @@protoc_insertion_point(field_set_allocated:xla.ResetDeviceRequest.device_handle) +} + +// ------------------------------------------------------------------- + +// ResetDeviceResponse + +// ------------------------------------------------------------------- + +// ComputationGraphStatsRequest + +// .xla.HloModuleProto computation = 1; +inline bool ComputationGraphStatsRequest::has_computation() const { + return this != internal_default_instance() && computation_ != NULL; +} +inline const ::xla::HloModuleProto& ComputationGraphStatsRequest::_internal_computation() const { + return *computation_; +} +inline const ::xla::HloModuleProto& ComputationGraphStatsRequest::computation() const { + const ::xla::HloModuleProto* p = computation_; + // @@protoc_insertion_point(field_get:xla.ComputationGraphStatsRequest.computation) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_HloModuleProto_default_instance_); +} +inline ::xla::HloModuleProto* ComputationGraphStatsRequest::release_computation() { + // @@protoc_insertion_point(field_release:xla.ComputationGraphStatsRequest.computation) + + ::xla::HloModuleProto* temp = computation_; + computation_ = NULL; + return temp; +} +inline ::xla::HloModuleProto* ComputationGraphStatsRequest::mutable_computation() { + + if (computation_ == NULL) { + auto* p = CreateMaybeMessage<::xla::HloModuleProto>(GetArenaNoVirtual()); + computation_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.ComputationGraphStatsRequest.computation) + return computation_; +} +inline void ComputationGraphStatsRequest::set_allocated_computation(::xla::HloModuleProto* computation) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(computation_); + } + if (computation) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(computation)->GetArena(); + if (message_arena != submessage_arena) { + computation = ::google::protobuf::internal::GetOwnedMessage( + message_arena, computation, submessage_arena); + } + + } else { + + } + computation_ = computation; + // @@protoc_insertion_point(field_set_allocated:xla.ComputationGraphStatsRequest.computation) +} + +// .xla.DebugOptions debug_options = 2; +inline bool ComputationGraphStatsRequest::has_debug_options() const { + return this != internal_default_instance() && debug_options_ != NULL; +} +inline void ComputationGraphStatsRequest::clear_debug_options() { + if (GetArenaNoVirtual() == NULL && debug_options_ != NULL) { + delete debug_options_; + } + debug_options_ = NULL; +} +inline const ::xla::DebugOptions& ComputationGraphStatsRequest::_internal_debug_options() const { + return *debug_options_; +} +inline const ::xla::DebugOptions& ComputationGraphStatsRequest::debug_options() const { + const ::xla::DebugOptions* p = debug_options_; + // @@protoc_insertion_point(field_get:xla.ComputationGraphStatsRequest.debug_options) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_DebugOptions_default_instance_); +} +inline ::xla::DebugOptions* ComputationGraphStatsRequest::release_debug_options() { + // @@protoc_insertion_point(field_release:xla.ComputationGraphStatsRequest.debug_options) + + ::xla::DebugOptions* temp = debug_options_; + debug_options_ = NULL; + return temp; +} +inline ::xla::DebugOptions* ComputationGraphStatsRequest::mutable_debug_options() { + + if (debug_options_ == NULL) { + auto* p = CreateMaybeMessage<::xla::DebugOptions>(GetArenaNoVirtual()); + debug_options_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.ComputationGraphStatsRequest.debug_options) + return debug_options_; +} +inline void ComputationGraphStatsRequest::set_allocated_debug_options(::xla::DebugOptions* debug_options) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete debug_options_; + } + if (debug_options) { + ::google::protobuf::Arena* submessage_arena = NULL; + if (message_arena != submessage_arena) { + debug_options = ::google::protobuf::internal::GetOwnedMessage( + message_arena, debug_options, submessage_arena); + } + + } else { + + } + debug_options_ = debug_options; + // @@protoc_insertion_point(field_set_allocated:xla.ComputationGraphStatsRequest.debug_options) +} + +// ------------------------------------------------------------------- + +// ComputationStatsResponse + +// .xla.ComputationStats stats = 1; +inline bool ComputationStatsResponse::has_stats() const { + return this != internal_default_instance() && stats_ != NULL; +} +inline const ::xla::ComputationStats& ComputationStatsResponse::_internal_stats() const { + return *stats_; +} +inline const ::xla::ComputationStats& ComputationStatsResponse::stats() const { + const ::xla::ComputationStats* p = stats_; + // @@protoc_insertion_point(field_get:xla.ComputationStatsResponse.stats) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_ComputationStats_default_instance_); +} +inline ::xla::ComputationStats* ComputationStatsResponse::release_stats() { + // @@protoc_insertion_point(field_release:xla.ComputationStatsResponse.stats) + + ::xla::ComputationStats* temp = stats_; + stats_ = NULL; + return temp; +} +inline ::xla::ComputationStats* ComputationStatsResponse::mutable_stats() { + + if (stats_ == NULL) { + auto* p = CreateMaybeMessage<::xla::ComputationStats>(GetArenaNoVirtual()); + stats_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.ComputationStatsResponse.stats) + return stats_; +} +inline void ComputationStatsResponse::set_allocated_stats(::xla::ComputationStats* stats) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(stats_); + } + if (stats) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(stats)->GetArena(); + if (message_arena != submessage_arena) { + stats = ::google::protobuf::internal::GetOwnedMessage( + message_arena, stats, submessage_arena); + } + + } else { + + } + stats_ = stats; + // @@protoc_insertion_point(field_set_allocated:xla.ComputationStatsResponse.stats) +} + +// ------------------------------------------------------------------- + +// CreateChannelHandleRequest + +// .xla.ChannelHandle.ChannelType channel_type = 1; +inline void CreateChannelHandleRequest::clear_channel_type() { + channel_type_ = 0; +} +inline ::xla::ChannelHandle_ChannelType CreateChannelHandleRequest::channel_type() const { + // @@protoc_insertion_point(field_get:xla.CreateChannelHandleRequest.channel_type) + return static_cast< ::xla::ChannelHandle_ChannelType >(channel_type_); +} +inline void CreateChannelHandleRequest::set_channel_type(::xla::ChannelHandle_ChannelType value) { + + channel_type_ = value; + // @@protoc_insertion_point(field_set:xla.CreateChannelHandleRequest.channel_type) +} + +// ------------------------------------------------------------------- + +// CreateChannelHandleResponse + +// .xla.ChannelHandle channel = 1; +inline bool CreateChannelHandleResponse::has_channel() const { + return this != internal_default_instance() && channel_ != NULL; +} +inline const ::xla::ChannelHandle& CreateChannelHandleResponse::_internal_channel() const { + return *channel_; +} +inline const ::xla::ChannelHandle& CreateChannelHandleResponse::channel() const { + const ::xla::ChannelHandle* p = channel_; + // @@protoc_insertion_point(field_get:xla.CreateChannelHandleResponse.channel) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_ChannelHandle_default_instance_); +} +inline ::xla::ChannelHandle* CreateChannelHandleResponse::release_channel() { + // @@protoc_insertion_point(field_release:xla.CreateChannelHandleResponse.channel) + + ::xla::ChannelHandle* temp = channel_; + channel_ = NULL; + return temp; +} +inline ::xla::ChannelHandle* CreateChannelHandleResponse::mutable_channel() { + + if (channel_ == NULL) { + auto* p = CreateMaybeMessage<::xla::ChannelHandle>(GetArenaNoVirtual()); + channel_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.CreateChannelHandleResponse.channel) + return channel_; +} +inline void CreateChannelHandleResponse::set_allocated_channel(::xla::ChannelHandle* channel) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(channel_); + } + if (channel) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(channel)->GetArena(); + if (message_arena != submessage_arena) { + channel = ::google::protobuf::internal::GetOwnedMessage( + message_arena, channel, submessage_arena); + } + + } else { + + } + channel_ = channel; + // @@protoc_insertion_point(field_set_allocated:xla.CreateChannelHandleResponse.channel) +} + +// ------------------------------------------------------------------- + +// UnregisterRequest + +// .xla.GlobalDataHandle data = 1; +inline bool UnregisterRequest::has_data() const { + return this != internal_default_instance() && data_ != NULL; +} +inline const ::xla::GlobalDataHandle& UnregisterRequest::_internal_data() const { + return *data_; +} +inline const ::xla::GlobalDataHandle& UnregisterRequest::data() const { + const ::xla::GlobalDataHandle* p = data_; + // @@protoc_insertion_point(field_get:xla.UnregisterRequest.data) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_GlobalDataHandle_default_instance_); +} +inline ::xla::GlobalDataHandle* UnregisterRequest::release_data() { + // @@protoc_insertion_point(field_release:xla.UnregisterRequest.data) + + ::xla::GlobalDataHandle* temp = data_; + data_ = NULL; + return temp; +} +inline ::xla::GlobalDataHandle* UnregisterRequest::mutable_data() { + + if (data_ == NULL) { + auto* p = CreateMaybeMessage<::xla::GlobalDataHandle>(GetArenaNoVirtual()); + data_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.UnregisterRequest.data) + return data_; +} +inline void UnregisterRequest::set_allocated_data(::xla::GlobalDataHandle* data) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(data_); + } + if (data) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(data)->GetArena(); + if (message_arena != submessage_arena) { + data = ::google::protobuf::internal::GetOwnedMessage( + message_arena, data, submessage_arena); + } + + } else { + + } + data_ = data; + // @@protoc_insertion_point(field_set_allocated:xla.UnregisterRequest.data) +} + +// ------------------------------------------------------------------- + +// UnregisterResponse + +// ------------------------------------------------------------------- + +// ExecuteGraphRequest + +// .xla.HloModuleProto computation = 1; +inline bool ExecuteGraphRequest::has_computation() const { + return this != internal_default_instance() && computation_ != NULL; +} +inline const ::xla::HloModuleProto& ExecuteGraphRequest::_internal_computation() const { + return *computation_; +} +inline const ::xla::HloModuleProto& ExecuteGraphRequest::computation() const { + const ::xla::HloModuleProto* p = computation_; + // @@protoc_insertion_point(field_get:xla.ExecuteGraphRequest.computation) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_HloModuleProto_default_instance_); +} +inline ::xla::HloModuleProto* ExecuteGraphRequest::release_computation() { + // @@protoc_insertion_point(field_release:xla.ExecuteGraphRequest.computation) + + ::xla::HloModuleProto* temp = computation_; + computation_ = NULL; + return temp; +} +inline ::xla::HloModuleProto* ExecuteGraphRequest::mutable_computation() { + + if (computation_ == NULL) { + auto* p = CreateMaybeMessage<::xla::HloModuleProto>(GetArenaNoVirtual()); + computation_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.ExecuteGraphRequest.computation) + return computation_; +} +inline void ExecuteGraphRequest::set_allocated_computation(::xla::HloModuleProto* computation) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(computation_); + } + if (computation) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(computation)->GetArena(); + if (message_arena != submessage_arena) { + computation = ::google::protobuf::internal::GetOwnedMessage( + message_arena, computation, submessage_arena); + } + + } else { + + } + computation_ = computation; + // @@protoc_insertion_point(field_set_allocated:xla.ExecuteGraphRequest.computation) +} + +// repeated .xla.GlobalDataHandle arguments = 2; +inline int ExecuteGraphRequest::arguments_size() const { + return arguments_.size(); +} +inline ::xla::GlobalDataHandle* ExecuteGraphRequest::mutable_arguments(int index) { + // @@protoc_insertion_point(field_mutable:xla.ExecuteGraphRequest.arguments) + return arguments_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::xla::GlobalDataHandle >* +ExecuteGraphRequest::mutable_arguments() { + // @@protoc_insertion_point(field_mutable_list:xla.ExecuteGraphRequest.arguments) + return &arguments_; +} +inline const ::xla::GlobalDataHandle& ExecuteGraphRequest::arguments(int index) const { + // @@protoc_insertion_point(field_get:xla.ExecuteGraphRequest.arguments) + return arguments_.Get(index); +} +inline ::xla::GlobalDataHandle* ExecuteGraphRequest::add_arguments() { + // @@protoc_insertion_point(field_add:xla.ExecuteGraphRequest.arguments) + return arguments_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::xla::GlobalDataHandle >& +ExecuteGraphRequest::arguments() const { + // @@protoc_insertion_point(field_list:xla.ExecuteGraphRequest.arguments) + return arguments_; +} + +// .xla.ExecutionOptions execution_options = 3; +inline bool ExecuteGraphRequest::has_execution_options() const { + return this != internal_default_instance() && execution_options_ != NULL; +} +inline void ExecuteGraphRequest::clear_execution_options() { + if (GetArenaNoVirtual() == NULL && execution_options_ != NULL) { + delete execution_options_; + } + execution_options_ = NULL; +} +inline const ::xla::ExecutionOptions& ExecuteGraphRequest::_internal_execution_options() const { + return *execution_options_; +} +inline const ::xla::ExecutionOptions& ExecuteGraphRequest::execution_options() const { + const ::xla::ExecutionOptions* p = execution_options_; + // @@protoc_insertion_point(field_get:xla.ExecuteGraphRequest.execution_options) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_ExecutionOptions_default_instance_); +} +inline ::xla::ExecutionOptions* ExecuteGraphRequest::release_execution_options() { + // @@protoc_insertion_point(field_release:xla.ExecuteGraphRequest.execution_options) + + ::xla::ExecutionOptions* temp = execution_options_; + execution_options_ = NULL; + return temp; +} +inline ::xla::ExecutionOptions* ExecuteGraphRequest::mutable_execution_options() { + + if (execution_options_ == NULL) { + auto* p = CreateMaybeMessage<::xla::ExecutionOptions>(GetArenaNoVirtual()); + execution_options_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.ExecuteGraphRequest.execution_options) + return execution_options_; +} +inline void ExecuteGraphRequest::set_allocated_execution_options(::xla::ExecutionOptions* execution_options) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete execution_options_; + } + if (execution_options) { + ::google::protobuf::Arena* submessage_arena = NULL; + if (message_arena != submessage_arena) { + execution_options = ::google::protobuf::internal::GetOwnedMessage( + message_arena, execution_options, submessage_arena); + } + + } else { + + } + execution_options_ = execution_options; + // @@protoc_insertion_point(field_set_allocated:xla.ExecuteGraphRequest.execution_options) +} + +// ------------------------------------------------------------------- + +// ExecuteGraphParallelRequest + +// repeated .xla.ExecuteGraphRequest requests = 1; +inline int ExecuteGraphParallelRequest::requests_size() const { + return requests_.size(); +} +inline void ExecuteGraphParallelRequest::clear_requests() { + requests_.Clear(); +} +inline ::xla::ExecuteGraphRequest* ExecuteGraphParallelRequest::mutable_requests(int index) { + // @@protoc_insertion_point(field_mutable:xla.ExecuteGraphParallelRequest.requests) + return requests_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::xla::ExecuteGraphRequest >* +ExecuteGraphParallelRequest::mutable_requests() { + // @@protoc_insertion_point(field_mutable_list:xla.ExecuteGraphParallelRequest.requests) + return &requests_; +} +inline const ::xla::ExecuteGraphRequest& ExecuteGraphParallelRequest::requests(int index) const { + // @@protoc_insertion_point(field_get:xla.ExecuteGraphParallelRequest.requests) + return requests_.Get(index); +} +inline ::xla::ExecuteGraphRequest* ExecuteGraphParallelRequest::add_requests() { + // @@protoc_insertion_point(field_add:xla.ExecuteGraphParallelRequest.requests) + return requests_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::xla::ExecuteGraphRequest >& +ExecuteGraphParallelRequest::requests() const { + // @@protoc_insertion_point(field_list:xla.ExecuteGraphParallelRequest.requests) + return requests_; +} + +// ------------------------------------------------------------------- + +// ExecuteResponse + +// .xla.GlobalDataHandle output = 1; +inline bool ExecuteResponse::has_output() const { + return this != internal_default_instance() && output_ != NULL; +} +inline const ::xla::GlobalDataHandle& ExecuteResponse::_internal_output() const { + return *output_; +} +inline const ::xla::GlobalDataHandle& ExecuteResponse::output() const { + const ::xla::GlobalDataHandle* p = output_; + // @@protoc_insertion_point(field_get:xla.ExecuteResponse.output) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_GlobalDataHandle_default_instance_); +} +inline ::xla::GlobalDataHandle* ExecuteResponse::release_output() { + // @@protoc_insertion_point(field_release:xla.ExecuteResponse.output) + + ::xla::GlobalDataHandle* temp = output_; + output_ = NULL; + return temp; +} +inline ::xla::GlobalDataHandle* ExecuteResponse::mutable_output() { + + if (output_ == NULL) { + auto* p = CreateMaybeMessage<::xla::GlobalDataHandle>(GetArenaNoVirtual()); + output_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.ExecuteResponse.output) + return output_; +} +inline void ExecuteResponse::set_allocated_output(::xla::GlobalDataHandle* output) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(output_); + } + if (output) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(output)->GetArena(); + if (message_arena != submessage_arena) { + output = ::google::protobuf::internal::GetOwnedMessage( + message_arena, output, submessage_arena); + } + + } else { + + } + output_ = output; + // @@protoc_insertion_point(field_set_allocated:xla.ExecuteResponse.output) +} + +// .xla.ExecutionProfile profile = 2; +inline bool ExecuteResponse::has_profile() const { + return this != internal_default_instance() && profile_ != NULL; +} +inline const ::xla::ExecutionProfile& ExecuteResponse::_internal_profile() const { + return *profile_; +} +inline const ::xla::ExecutionProfile& ExecuteResponse::profile() const { + const ::xla::ExecutionProfile* p = profile_; + // @@protoc_insertion_point(field_get:xla.ExecuteResponse.profile) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_ExecutionProfile_default_instance_); +} +inline ::xla::ExecutionProfile* ExecuteResponse::release_profile() { + // @@protoc_insertion_point(field_release:xla.ExecuteResponse.profile) + + ::xla::ExecutionProfile* temp = profile_; + profile_ = NULL; + return temp; +} +inline ::xla::ExecutionProfile* ExecuteResponse::mutable_profile() { + + if (profile_ == NULL) { + auto* p = CreateMaybeMessage<::xla::ExecutionProfile>(GetArenaNoVirtual()); + profile_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.ExecuteResponse.profile) + return profile_; +} +inline void ExecuteResponse::set_allocated_profile(::xla::ExecutionProfile* profile) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(profile_); + } + if (profile) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(profile)->GetArena(); + if (message_arena != submessage_arena) { + profile = ::google::protobuf::internal::GetOwnedMessage( + message_arena, profile, submessage_arena); + } + + } else { + + } + profile_ = profile; + // @@protoc_insertion_point(field_set_allocated:xla.ExecuteResponse.profile) +} + +// ------------------------------------------------------------------- + +// ExecuteParallelResponse + +// repeated .xla.ExecuteResponse responses = 1; +inline int ExecuteParallelResponse::responses_size() const { + return responses_.size(); +} +inline void ExecuteParallelResponse::clear_responses() { + responses_.Clear(); +} +inline ::xla::ExecuteResponse* ExecuteParallelResponse::mutable_responses(int index) { + // @@protoc_insertion_point(field_mutable:xla.ExecuteParallelResponse.responses) + return responses_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::xla::ExecuteResponse >* +ExecuteParallelResponse::mutable_responses() { + // @@protoc_insertion_point(field_mutable_list:xla.ExecuteParallelResponse.responses) + return &responses_; +} +inline const ::xla::ExecuteResponse& ExecuteParallelResponse::responses(int index) const { + // @@protoc_insertion_point(field_get:xla.ExecuteParallelResponse.responses) + return responses_.Get(index); +} +inline ::xla::ExecuteResponse* ExecuteParallelResponse::add_responses() { + // @@protoc_insertion_point(field_add:xla.ExecuteParallelResponse.responses) + return responses_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::xla::ExecuteResponse >& +ExecuteParallelResponse::responses() const { + // @@protoc_insertion_point(field_list:xla.ExecuteParallelResponse.responses) + return responses_; +} + +// ------------------------------------------------------------------- + +// WaitForExecutionRequest + +// .xla.ExecutionHandle execution = 1; +inline bool WaitForExecutionRequest::has_execution() const { + return this != internal_default_instance() && execution_ != NULL; +} +inline const ::xla::ExecutionHandle& WaitForExecutionRequest::_internal_execution() const { + return *execution_; +} +inline const ::xla::ExecutionHandle& WaitForExecutionRequest::execution() const { + const ::xla::ExecutionHandle* p = execution_; + // @@protoc_insertion_point(field_get:xla.WaitForExecutionRequest.execution) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_ExecutionHandle_default_instance_); +} +inline ::xla::ExecutionHandle* WaitForExecutionRequest::release_execution() { + // @@protoc_insertion_point(field_release:xla.WaitForExecutionRequest.execution) + + ::xla::ExecutionHandle* temp = execution_; + execution_ = NULL; + return temp; +} +inline ::xla::ExecutionHandle* WaitForExecutionRequest::mutable_execution() { + + if (execution_ == NULL) { + auto* p = CreateMaybeMessage<::xla::ExecutionHandle>(GetArenaNoVirtual()); + execution_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.WaitForExecutionRequest.execution) + return execution_; +} +inline void WaitForExecutionRequest::set_allocated_execution(::xla::ExecutionHandle* execution) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(execution_); + } + if (execution) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(execution)->GetArena(); + if (message_arena != submessage_arena) { + execution = ::google::protobuf::internal::GetOwnedMessage( + message_arena, execution, submessage_arena); + } + + } else { + + } + execution_ = execution; + // @@protoc_insertion_point(field_set_allocated:xla.WaitForExecutionRequest.execution) +} + +// ------------------------------------------------------------------- + +// WaitForExecutionResponse + +// .xla.GlobalDataHandle output = 1; +inline bool WaitForExecutionResponse::has_output() const { + return this != internal_default_instance() && output_ != NULL; +} +inline const ::xla::GlobalDataHandle& WaitForExecutionResponse::_internal_output() const { + return *output_; +} +inline const ::xla::GlobalDataHandle& WaitForExecutionResponse::output() const { + const ::xla::GlobalDataHandle* p = output_; + // @@protoc_insertion_point(field_get:xla.WaitForExecutionResponse.output) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_GlobalDataHandle_default_instance_); +} +inline ::xla::GlobalDataHandle* WaitForExecutionResponse::release_output() { + // @@protoc_insertion_point(field_release:xla.WaitForExecutionResponse.output) + + ::xla::GlobalDataHandle* temp = output_; + output_ = NULL; + return temp; +} +inline ::xla::GlobalDataHandle* WaitForExecutionResponse::mutable_output() { + + if (output_ == NULL) { + auto* p = CreateMaybeMessage<::xla::GlobalDataHandle>(GetArenaNoVirtual()); + output_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.WaitForExecutionResponse.output) + return output_; +} +inline void WaitForExecutionResponse::set_allocated_output(::xla::GlobalDataHandle* output) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(output_); + } + if (output) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(output)->GetArena(); + if (message_arena != submessage_arena) { + output = ::google::protobuf::internal::GetOwnedMessage( + message_arena, output, submessage_arena); + } + + } else { + + } + output_ = output; + // @@protoc_insertion_point(field_set_allocated:xla.WaitForExecutionResponse.output) +} + +// .xla.ExecutionProfile profile = 2; +inline bool WaitForExecutionResponse::has_profile() const { + return this != internal_default_instance() && profile_ != NULL; +} +inline const ::xla::ExecutionProfile& WaitForExecutionResponse::_internal_profile() const { + return *profile_; +} +inline const ::xla::ExecutionProfile& WaitForExecutionResponse::profile() const { + const ::xla::ExecutionProfile* p = profile_; + // @@protoc_insertion_point(field_get:xla.WaitForExecutionResponse.profile) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_ExecutionProfile_default_instance_); +} +inline ::xla::ExecutionProfile* WaitForExecutionResponse::release_profile() { + // @@protoc_insertion_point(field_release:xla.WaitForExecutionResponse.profile) + + ::xla::ExecutionProfile* temp = profile_; + profile_ = NULL; + return temp; +} +inline ::xla::ExecutionProfile* WaitForExecutionResponse::mutable_profile() { + + if (profile_ == NULL) { + auto* p = CreateMaybeMessage<::xla::ExecutionProfile>(GetArenaNoVirtual()); + profile_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.WaitForExecutionResponse.profile) + return profile_; +} +inline void WaitForExecutionResponse::set_allocated_profile(::xla::ExecutionProfile* profile) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(profile_); + } + if (profile) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(profile)->GetArena(); + if (message_arena != submessage_arena) { + profile = ::google::protobuf::internal::GetOwnedMessage( + message_arena, profile, submessage_arena); + } + + } else { + + } + profile_ = profile; + // @@protoc_insertion_point(field_set_allocated:xla.WaitForExecutionResponse.profile) +} + +// ------------------------------------------------------------------- + +// ComputeConstantGraphRequest + +// .xla.HloModuleProto computation = 1; +inline bool ComputeConstantGraphRequest::has_computation() const { + return this != internal_default_instance() && computation_ != NULL; +} +inline const ::xla::HloModuleProto& ComputeConstantGraphRequest::_internal_computation() const { + return *computation_; +} +inline const ::xla::HloModuleProto& ComputeConstantGraphRequest::computation() const { + const ::xla::HloModuleProto* p = computation_; + // @@protoc_insertion_point(field_get:xla.ComputeConstantGraphRequest.computation) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_HloModuleProto_default_instance_); +} +inline ::xla::HloModuleProto* ComputeConstantGraphRequest::release_computation() { + // @@protoc_insertion_point(field_release:xla.ComputeConstantGraphRequest.computation) + + ::xla::HloModuleProto* temp = computation_; + computation_ = NULL; + return temp; +} +inline ::xla::HloModuleProto* ComputeConstantGraphRequest::mutable_computation() { + + if (computation_ == NULL) { + auto* p = CreateMaybeMessage<::xla::HloModuleProto>(GetArenaNoVirtual()); + computation_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.ComputeConstantGraphRequest.computation) + return computation_; +} +inline void ComputeConstantGraphRequest::set_allocated_computation(::xla::HloModuleProto* computation) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(computation_); + } + if (computation) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(computation)->GetArena(); + if (message_arena != submessage_arena) { + computation = ::google::protobuf::internal::GetOwnedMessage( + message_arena, computation, submessage_arena); + } + + } else { + + } + computation_ = computation; + // @@protoc_insertion_point(field_set_allocated:xla.ComputeConstantGraphRequest.computation) +} + +// .xla.Layout output_layout = 2; +inline bool ComputeConstantGraphRequest::has_output_layout() const { + return this != internal_default_instance() && output_layout_ != NULL; +} +inline const ::xla::Layout& ComputeConstantGraphRequest::_internal_output_layout() const { + return *output_layout_; +} +inline const ::xla::Layout& ComputeConstantGraphRequest::output_layout() const { + const ::xla::Layout* p = output_layout_; + // @@protoc_insertion_point(field_get:xla.ComputeConstantGraphRequest.output_layout) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_Layout_default_instance_); +} +inline ::xla::Layout* ComputeConstantGraphRequest::release_output_layout() { + // @@protoc_insertion_point(field_release:xla.ComputeConstantGraphRequest.output_layout) + + ::xla::Layout* temp = output_layout_; + output_layout_ = NULL; + return temp; +} +inline ::xla::Layout* ComputeConstantGraphRequest::mutable_output_layout() { + + if (output_layout_ == NULL) { + auto* p = CreateMaybeMessage<::xla::Layout>(GetArenaNoVirtual()); + output_layout_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.ComputeConstantGraphRequest.output_layout) + return output_layout_; +} +inline void ComputeConstantGraphRequest::set_allocated_output_layout(::xla::Layout* output_layout) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(output_layout_); + } + if (output_layout) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(output_layout)->GetArena(); + if (message_arena != submessage_arena) { + output_layout = ::google::protobuf::internal::GetOwnedMessage( + message_arena, output_layout, submessage_arena); + } + + } else { + + } + output_layout_ = output_layout; + // @@protoc_insertion_point(field_set_allocated:xla.ComputeConstantGraphRequest.output_layout) +} + +// ------------------------------------------------------------------- + +// ComputeConstantResponse + +// .xla.LiteralProto literal = 1; +inline bool ComputeConstantResponse::has_literal() const { + return this != internal_default_instance() && literal_ != NULL; +} +inline const ::xla::LiteralProto& ComputeConstantResponse::_internal_literal() const { + return *literal_; +} +inline const ::xla::LiteralProto& ComputeConstantResponse::literal() const { + const ::xla::LiteralProto* p = literal_; + // @@protoc_insertion_point(field_get:xla.ComputeConstantResponse.literal) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_LiteralProto_default_instance_); +} +inline ::xla::LiteralProto* ComputeConstantResponse::release_literal() { + // @@protoc_insertion_point(field_release:xla.ComputeConstantResponse.literal) + + ::xla::LiteralProto* temp = literal_; + literal_ = NULL; + return temp; +} +inline ::xla::LiteralProto* ComputeConstantResponse::mutable_literal() { + + if (literal_ == NULL) { + auto* p = CreateMaybeMessage<::xla::LiteralProto>(GetArenaNoVirtual()); + literal_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.ComputeConstantResponse.literal) + return literal_; +} +inline void ComputeConstantResponse::set_allocated_literal(::xla::LiteralProto* literal) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(literal_); + } + if (literal) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(literal)->GetArena(); + if (message_arena != submessage_arena) { + literal = ::google::protobuf::internal::GetOwnedMessage( + message_arena, literal, submessage_arena); + } + + } else { + + } + literal_ = literal; + // @@protoc_insertion_point(field_set_allocated:xla.ComputeConstantResponse.literal) +} + +// ------------------------------------------------------------------- + +// DeconstructTupleRequest + +// .xla.GlobalDataHandle tuple_handle = 2; +inline bool DeconstructTupleRequest::has_tuple_handle() const { + return this != internal_default_instance() && tuple_handle_ != NULL; +} +inline const ::xla::GlobalDataHandle& DeconstructTupleRequest::_internal_tuple_handle() const { + return *tuple_handle_; +} +inline const ::xla::GlobalDataHandle& DeconstructTupleRequest::tuple_handle() const { + const ::xla::GlobalDataHandle* p = tuple_handle_; + // @@protoc_insertion_point(field_get:xla.DeconstructTupleRequest.tuple_handle) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_GlobalDataHandle_default_instance_); +} +inline ::xla::GlobalDataHandle* DeconstructTupleRequest::release_tuple_handle() { + // @@protoc_insertion_point(field_release:xla.DeconstructTupleRequest.tuple_handle) + + ::xla::GlobalDataHandle* temp = tuple_handle_; + tuple_handle_ = NULL; + return temp; +} +inline ::xla::GlobalDataHandle* DeconstructTupleRequest::mutable_tuple_handle() { + + if (tuple_handle_ == NULL) { + auto* p = CreateMaybeMessage<::xla::GlobalDataHandle>(GetArenaNoVirtual()); + tuple_handle_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.DeconstructTupleRequest.tuple_handle) + return tuple_handle_; +} +inline void DeconstructTupleRequest::set_allocated_tuple_handle(::xla::GlobalDataHandle* tuple_handle) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(tuple_handle_); + } + if (tuple_handle) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(tuple_handle)->GetArena(); + if (message_arena != submessage_arena) { + tuple_handle = ::google::protobuf::internal::GetOwnedMessage( + message_arena, tuple_handle, submessage_arena); + } + + } else { + + } + tuple_handle_ = tuple_handle; + // @@protoc_insertion_point(field_set_allocated:xla.DeconstructTupleRequest.tuple_handle) +} + +// ------------------------------------------------------------------- + +// DeconstructTupleResponse + +// repeated .xla.GlobalDataHandle element_handles = 1; +inline int DeconstructTupleResponse::element_handles_size() const { + return element_handles_.size(); +} +inline ::xla::GlobalDataHandle* DeconstructTupleResponse::mutable_element_handles(int index) { + // @@protoc_insertion_point(field_mutable:xla.DeconstructTupleResponse.element_handles) + return element_handles_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::xla::GlobalDataHandle >* +DeconstructTupleResponse::mutable_element_handles() { + // @@protoc_insertion_point(field_mutable_list:xla.DeconstructTupleResponse.element_handles) + return &element_handles_; +} +inline const ::xla::GlobalDataHandle& DeconstructTupleResponse::element_handles(int index) const { + // @@protoc_insertion_point(field_get:xla.DeconstructTupleResponse.element_handles) + return element_handles_.Get(index); +} +inline ::xla::GlobalDataHandle* DeconstructTupleResponse::add_element_handles() { + // @@protoc_insertion_point(field_add:xla.DeconstructTupleResponse.element_handles) + return element_handles_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::xla::GlobalDataHandle >& +DeconstructTupleResponse::element_handles() const { + // @@protoc_insertion_point(field_list:xla.DeconstructTupleResponse.element_handles) + return element_handles_; +} + +// ------------------------------------------------------------------- + +// LoadDataRequest + +// string columnio_tablet_path = 1; +inline void LoadDataRequest::clear_columnio_tablet_path() { + columnio_tablet_path_.ClearToEmptyNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); +} +inline const ::std::string& LoadDataRequest::columnio_tablet_path() const { + // @@protoc_insertion_point(field_get:xla.LoadDataRequest.columnio_tablet_path) + return columnio_tablet_path_.GetNoArena(); +} +inline void LoadDataRequest::set_columnio_tablet_path(const ::std::string& value) { + + columnio_tablet_path_.SetNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value); + // @@protoc_insertion_point(field_set:xla.LoadDataRequest.columnio_tablet_path) +} +#if LANG_CXX11 +inline void LoadDataRequest::set_columnio_tablet_path(::std::string&& value) { + + columnio_tablet_path_.SetNoArena( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value)); + // @@protoc_insertion_point(field_set_rvalue:xla.LoadDataRequest.columnio_tablet_path) +} +#endif +inline void LoadDataRequest::set_columnio_tablet_path(const char* value) { + GOOGLE_DCHECK(value != NULL); + + columnio_tablet_path_.SetNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value)); + // @@protoc_insertion_point(field_set_char:xla.LoadDataRequest.columnio_tablet_path) +} +inline void LoadDataRequest::set_columnio_tablet_path(const char* value, size_t size) { + + columnio_tablet_path_.SetNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + ::std::string(reinterpret_cast(value), size)); + // @@protoc_insertion_point(field_set_pointer:xla.LoadDataRequest.columnio_tablet_path) +} +inline ::std::string* LoadDataRequest::mutable_columnio_tablet_path() { + + // @@protoc_insertion_point(field_mutable:xla.LoadDataRequest.columnio_tablet_path) + return columnio_tablet_path_.MutableNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); +} +inline ::std::string* LoadDataRequest::release_columnio_tablet_path() { + // @@protoc_insertion_point(field_release:xla.LoadDataRequest.columnio_tablet_path) + + return columnio_tablet_path_.ReleaseNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); +} +inline void LoadDataRequest::set_allocated_columnio_tablet_path(::std::string* columnio_tablet_path) { + if (columnio_tablet_path != NULL) { + + } else { + + } + columnio_tablet_path_.SetAllocatedNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), columnio_tablet_path); + // @@protoc_insertion_point(field_set_allocated:xla.LoadDataRequest.columnio_tablet_path) +} + +// string columnio_field = 2; +inline void LoadDataRequest::clear_columnio_field() { + columnio_field_.ClearToEmptyNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); +} +inline const ::std::string& LoadDataRequest::columnio_field() const { + // @@protoc_insertion_point(field_get:xla.LoadDataRequest.columnio_field) + return columnio_field_.GetNoArena(); +} +inline void LoadDataRequest::set_columnio_field(const ::std::string& value) { + + columnio_field_.SetNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value); + // @@protoc_insertion_point(field_set:xla.LoadDataRequest.columnio_field) +} +#if LANG_CXX11 +inline void LoadDataRequest::set_columnio_field(::std::string&& value) { + + columnio_field_.SetNoArena( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value)); + // @@protoc_insertion_point(field_set_rvalue:xla.LoadDataRequest.columnio_field) +} +#endif +inline void LoadDataRequest::set_columnio_field(const char* value) { + GOOGLE_DCHECK(value != NULL); + + columnio_field_.SetNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value)); + // @@protoc_insertion_point(field_set_char:xla.LoadDataRequest.columnio_field) +} +inline void LoadDataRequest::set_columnio_field(const char* value, size_t size) { + + columnio_field_.SetNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + ::std::string(reinterpret_cast(value), size)); + // @@protoc_insertion_point(field_set_pointer:xla.LoadDataRequest.columnio_field) +} +inline ::std::string* LoadDataRequest::mutable_columnio_field() { + + // @@protoc_insertion_point(field_mutable:xla.LoadDataRequest.columnio_field) + return columnio_field_.MutableNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); +} +inline ::std::string* LoadDataRequest::release_columnio_field() { + // @@protoc_insertion_point(field_release:xla.LoadDataRequest.columnio_field) + + return columnio_field_.ReleaseNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); +} +inline void LoadDataRequest::set_allocated_columnio_field(::std::string* columnio_field) { + if (columnio_field != NULL) { + + } else { + + } + columnio_field_.SetAllocatedNoArena(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), columnio_field); + // @@protoc_insertion_point(field_set_allocated:xla.LoadDataRequest.columnio_field) +} + +// .xla.Shape element_shape = 3; +inline bool LoadDataRequest::has_element_shape() const { + return this != internal_default_instance() && element_shape_ != NULL; +} +inline const ::xla::Shape& LoadDataRequest::_internal_element_shape() const { + return *element_shape_; +} +inline const ::xla::Shape& LoadDataRequest::element_shape() const { + const ::xla::Shape* p = element_shape_; + // @@protoc_insertion_point(field_get:xla.LoadDataRequest.element_shape) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_Shape_default_instance_); +} +inline ::xla::Shape* LoadDataRequest::release_element_shape() { + // @@protoc_insertion_point(field_release:xla.LoadDataRequest.element_shape) + + ::xla::Shape* temp = element_shape_; + element_shape_ = NULL; + return temp; +} +inline ::xla::Shape* LoadDataRequest::mutable_element_shape() { + + if (element_shape_ == NULL) { + auto* p = CreateMaybeMessage<::xla::Shape>(GetArenaNoVirtual()); + element_shape_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.LoadDataRequest.element_shape) + return element_shape_; +} +inline void LoadDataRequest::set_allocated_element_shape(::xla::Shape* element_shape) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(element_shape_); + } + if (element_shape) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(element_shape)->GetArena(); + if (message_arena != submessage_arena) { + element_shape = ::google::protobuf::internal::GetOwnedMessage( + message_arena, element_shape, submessage_arena); + } + + } else { + + } + element_shape_ = element_shape; + // @@protoc_insertion_point(field_set_allocated:xla.LoadDataRequest.element_shape) +} + +// int64 offset = 4; +inline void LoadDataRequest::clear_offset() { + offset_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 LoadDataRequest::offset() const { + // @@protoc_insertion_point(field_get:xla.LoadDataRequest.offset) + return offset_; +} +inline void LoadDataRequest::set_offset(::google::protobuf::int64 value) { + + offset_ = value; + // @@protoc_insertion_point(field_set:xla.LoadDataRequest.offset) +} + +// int64 limit = 5; +inline void LoadDataRequest::clear_limit() { + limit_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 LoadDataRequest::limit() const { + // @@protoc_insertion_point(field_get:xla.LoadDataRequest.limit) + return limit_; +} +inline void LoadDataRequest::set_limit(::google::protobuf::int64 value) { + + limit_ = value; + // @@protoc_insertion_point(field_set:xla.LoadDataRequest.limit) +} + +// bool zip = 6; +inline void LoadDataRequest::clear_zip() { + zip_ = false; +} +inline bool LoadDataRequest::zip() const { + // @@protoc_insertion_point(field_get:xla.LoadDataRequest.zip) + return zip_; +} +inline void LoadDataRequest::set_zip(bool value) { + + zip_ = value; + // @@protoc_insertion_point(field_set:xla.LoadDataRequest.zip) +} + +// ------------------------------------------------------------------- + +// LoadDataResponse + +// .xla.GlobalDataHandle data = 1; +inline bool LoadDataResponse::has_data() const { + return this != internal_default_instance() && data_ != NULL; +} +inline const ::xla::GlobalDataHandle& LoadDataResponse::_internal_data() const { + return *data_; +} +inline const ::xla::GlobalDataHandle& LoadDataResponse::data() const { + const ::xla::GlobalDataHandle* p = data_; + // @@protoc_insertion_point(field_get:xla.LoadDataResponse.data) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_GlobalDataHandle_default_instance_); +} +inline ::xla::GlobalDataHandle* LoadDataResponse::release_data() { + // @@protoc_insertion_point(field_release:xla.LoadDataResponse.data) + + ::xla::GlobalDataHandle* temp = data_; + data_ = NULL; + return temp; +} +inline ::xla::GlobalDataHandle* LoadDataResponse::mutable_data() { + + if (data_ == NULL) { + auto* p = CreateMaybeMessage<::xla::GlobalDataHandle>(GetArenaNoVirtual()); + data_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.LoadDataResponse.data) + return data_; +} +inline void LoadDataResponse::set_allocated_data(::xla::GlobalDataHandle* data) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(data_); + } + if (data) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(data)->GetArena(); + if (message_arena != submessage_arena) { + data = ::google::protobuf::internal::GetOwnedMessage( + message_arena, data, submessage_arena); + } + + } else { + + } + data_ = data; + // @@protoc_insertion_point(field_set_allocated:xla.LoadDataResponse.data) +} + +// .xla.Shape data_shape = 2; +inline bool LoadDataResponse::has_data_shape() const { + return this != internal_default_instance() && data_shape_ != NULL; +} +inline const ::xla::Shape& LoadDataResponse::_internal_data_shape() const { + return *data_shape_; +} +inline const ::xla::Shape& LoadDataResponse::data_shape() const { + const ::xla::Shape* p = data_shape_; + // @@protoc_insertion_point(field_get:xla.LoadDataResponse.data_shape) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_Shape_default_instance_); +} +inline ::xla::Shape* LoadDataResponse::release_data_shape() { + // @@protoc_insertion_point(field_release:xla.LoadDataResponse.data_shape) + + ::xla::Shape* temp = data_shape_; + data_shape_ = NULL; + return temp; +} +inline ::xla::Shape* LoadDataResponse::mutable_data_shape() { + + if (data_shape_ == NULL) { + auto* p = CreateMaybeMessage<::xla::Shape>(GetArenaNoVirtual()); + data_shape_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.LoadDataResponse.data_shape) + return data_shape_; +} +inline void LoadDataResponse::set_allocated_data_shape(::xla::Shape* data_shape) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(data_shape_); + } + if (data_shape) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(data_shape)->GetArena(); + if (message_arena != submessage_arena) { + data_shape = ::google::protobuf::internal::GetOwnedMessage( + message_arena, data_shape, submessage_arena); + } + + } else { + + } + data_shape_ = data_shape; + // @@protoc_insertion_point(field_set_allocated:xla.LoadDataResponse.data_shape) +} + +// int64 available_rows = 3; +inline void LoadDataResponse::clear_available_rows() { + available_rows_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 LoadDataResponse::available_rows() const { + // @@protoc_insertion_point(field_get:xla.LoadDataResponse.available_rows) + return available_rows_; +} +inline void LoadDataResponse::set_available_rows(::google::protobuf::int64 value) { + + available_rows_ = value; + // @@protoc_insertion_point(field_set:xla.LoadDataResponse.available_rows) +} + +// int64 rows_loaded = 4; +inline void LoadDataResponse::clear_rows_loaded() { + rows_loaded_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 LoadDataResponse::rows_loaded() const { + // @@protoc_insertion_point(field_get:xla.LoadDataResponse.rows_loaded) + return rows_loaded_; +} +inline void LoadDataResponse::set_rows_loaded(::google::protobuf::int64 value) { + + rows_loaded_ = value; + // @@protoc_insertion_point(field_set:xla.LoadDataResponse.rows_loaded) +} + +// int64 nanoseconds = 5; +inline void LoadDataResponse::clear_nanoseconds() { + nanoseconds_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 LoadDataResponse::nanoseconds() const { + // @@protoc_insertion_point(field_get:xla.LoadDataResponse.nanoseconds) + return nanoseconds_; +} +inline void LoadDataResponse::set_nanoseconds(::google::protobuf::int64 value) { + + nanoseconds_ = value; + // @@protoc_insertion_point(field_set:xla.LoadDataResponse.nanoseconds) +} + +// ------------------------------------------------------------------- + +// GetShapeRequest + +// .xla.GlobalDataHandle data = 1; +inline bool GetShapeRequest::has_data() const { + return this != internal_default_instance() && data_ != NULL; +} +inline const ::xla::GlobalDataHandle& GetShapeRequest::_internal_data() const { + return *data_; +} +inline const ::xla::GlobalDataHandle& GetShapeRequest::data() const { + const ::xla::GlobalDataHandle* p = data_; + // @@protoc_insertion_point(field_get:xla.GetShapeRequest.data) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_GlobalDataHandle_default_instance_); +} +inline ::xla::GlobalDataHandle* GetShapeRequest::release_data() { + // @@protoc_insertion_point(field_release:xla.GetShapeRequest.data) + + ::xla::GlobalDataHandle* temp = data_; + data_ = NULL; + return temp; +} +inline ::xla::GlobalDataHandle* GetShapeRequest::mutable_data() { + + if (data_ == NULL) { + auto* p = CreateMaybeMessage<::xla::GlobalDataHandle>(GetArenaNoVirtual()); + data_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.GetShapeRequest.data) + return data_; +} +inline void GetShapeRequest::set_allocated_data(::xla::GlobalDataHandle* data) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(data_); + } + if (data) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(data)->GetArena(); + if (message_arena != submessage_arena) { + data = ::google::protobuf::internal::GetOwnedMessage( + message_arena, data, submessage_arena); + } + + } else { + + } + data_ = data; + // @@protoc_insertion_point(field_set_allocated:xla.GetShapeRequest.data) +} + +// ------------------------------------------------------------------- + +// GetShapeResponse + +// .xla.Shape shape = 1; +inline bool GetShapeResponse::has_shape() const { + return this != internal_default_instance() && shape_ != NULL; +} +inline const ::xla::Shape& GetShapeResponse::_internal_shape() const { + return *shape_; +} +inline const ::xla::Shape& GetShapeResponse::shape() const { + const ::xla::Shape* p = shape_; + // @@protoc_insertion_point(field_get:xla.GetShapeResponse.shape) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_Shape_default_instance_); +} +inline ::xla::Shape* GetShapeResponse::release_shape() { + // @@protoc_insertion_point(field_release:xla.GetShapeResponse.shape) + + ::xla::Shape* temp = shape_; + shape_ = NULL; + return temp; +} +inline ::xla::Shape* GetShapeResponse::mutable_shape() { + + if (shape_ == NULL) { + auto* p = CreateMaybeMessage<::xla::Shape>(GetArenaNoVirtual()); + shape_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.GetShapeResponse.shape) + return shape_; +} +inline void GetShapeResponse::set_allocated_shape(::xla::Shape* shape) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(shape_); + } + if (shape) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(shape)->GetArena(); + if (message_arena != submessage_arena) { + shape = ::google::protobuf::internal::GetOwnedMessage( + message_arena, shape, submessage_arena); + } + + } else { + + } + shape_ = shape; + // @@protoc_insertion_point(field_set_allocated:xla.GetShapeResponse.shape) +} + +// ------------------------------------------------------------------- + +// UnpackRequest + +// .xla.GlobalDataHandle data = 1; +inline bool UnpackRequest::has_data() const { + return this != internal_default_instance() && data_ != NULL; +} +inline const ::xla::GlobalDataHandle& UnpackRequest::_internal_data() const { + return *data_; +} +inline const ::xla::GlobalDataHandle& UnpackRequest::data() const { + const ::xla::GlobalDataHandle* p = data_; + // @@protoc_insertion_point(field_get:xla.UnpackRequest.data) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_GlobalDataHandle_default_instance_); +} +inline ::xla::GlobalDataHandle* UnpackRequest::release_data() { + // @@protoc_insertion_point(field_release:xla.UnpackRequest.data) + + ::xla::GlobalDataHandle* temp = data_; + data_ = NULL; + return temp; +} +inline ::xla::GlobalDataHandle* UnpackRequest::mutable_data() { + + if (data_ == NULL) { + auto* p = CreateMaybeMessage<::xla::GlobalDataHandle>(GetArenaNoVirtual()); + data_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.UnpackRequest.data) + return data_; +} +inline void UnpackRequest::set_allocated_data(::xla::GlobalDataHandle* data) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(data_); + } + if (data) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(data)->GetArena(); + if (message_arena != submessage_arena) { + data = ::google::protobuf::internal::GetOwnedMessage( + message_arena, data, submessage_arena); + } + + } else { + + } + data_ = data; + // @@protoc_insertion_point(field_set_allocated:xla.UnpackRequest.data) +} + +// ------------------------------------------------------------------- + +// UnpackResponse + +// repeated .xla.GlobalDataHandle tied_data = 1; +inline int UnpackResponse::tied_data_size() const { + return tied_data_.size(); +} +inline ::xla::GlobalDataHandle* UnpackResponse::mutable_tied_data(int index) { + // @@protoc_insertion_point(field_mutable:xla.UnpackResponse.tied_data) + return tied_data_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::xla::GlobalDataHandle >* +UnpackResponse::mutable_tied_data() { + // @@protoc_insertion_point(field_mutable_list:xla.UnpackResponse.tied_data) + return &tied_data_; +} +inline const ::xla::GlobalDataHandle& UnpackResponse::tied_data(int index) const { + // @@protoc_insertion_point(field_get:xla.UnpackResponse.tied_data) + return tied_data_.Get(index); +} +inline ::xla::GlobalDataHandle* UnpackResponse::add_tied_data() { + // @@protoc_insertion_point(field_add:xla.UnpackResponse.tied_data) + return tied_data_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::xla::GlobalDataHandle >& +UnpackResponse::tied_data() const { + // @@protoc_insertion_point(field_list:xla.UnpackResponse.tied_data) + return tied_data_; +} + +#ifdef __GNUC__ + #pragma GCC diagnostic pop +#endif // __GNUC__ +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + + +// @@protoc_insertion_point(namespace_scope) + +} // namespace xla + +namespace google { +namespace protobuf { + +template <> struct is_proto_enum< ::xla::HloReducePrecisionOptions_Location> : ::std::true_type {}; +template <> +inline const EnumDescriptor* GetEnumDescriptor< ::xla::HloReducePrecisionOptions_Location>() { + return ::xla::HloReducePrecisionOptions_Location_descriptor(); +} + +} // namespace protobuf +} // namespace google + +// @@protoc_insertion_point(global_scope) + +#endif // PROTOBUF_INCLUDED_tensorflow_2fcompiler_2fxla_2fxla_2eproto diff --git a/linux/include/tensorflow/compiler/xla/xla_data.pb.h b/linux/include/tensorflow/compiler/xla/xla_data.pb.h new file mode 100644 index 000000000..a7be5e768 --- /dev/null +++ b/linux/include/tensorflow/compiler/xla/xla_data.pb.h @@ -0,0 +1,7043 @@ +// Generated by the protocol buffer compiler. DO NOT EDIT! +// source: tensorflow/compiler/xla/xla_data.proto + +#ifndef PROTOBUF_INCLUDED_tensorflow_2fcompiler_2fxla_2fxla_5fdata_2eproto +#define PROTOBUF_INCLUDED_tensorflow_2fcompiler_2fxla_2fxla_5fdata_2eproto + +#include + +#include + +#if GOOGLE_PROTOBUF_VERSION < 3006000 +#error This file was generated by a newer version of protoc which is +#error incompatible with your Protocol Buffer headers. Please update +#error your headers. +#endif +#if 3006000 < GOOGLE_PROTOBUF_MIN_PROTOC_VERSION +#error This file was generated by an older version of protoc which is +#error incompatible with your Protocol Buffer headers. Please +#error regenerate this file with a newer version of protoc. +#endif + +#include +#include +#include +#include +#include +#include +#include +#include +#include // IWYU pragma: export +#include // IWYU pragma: export +#include +#include +// @@protoc_insertion_point(includes) +#define PROTOBUF_INTERNAL_EXPORT_protobuf_tensorflow_2fcompiler_2fxla_2fxla_5fdata_2eproto + +namespace protobuf_tensorflow_2fcompiler_2fxla_2fxla_5fdata_2eproto { +// Internal implementation detail -- do not use these members. +struct TableStruct { + static const ::google::protobuf::internal::ParseTableField entries[]; + static const ::google::protobuf::internal::AuxillaryParseTableField aux[]; + static const ::google::protobuf::internal::ParseTable schema[25]; + static const ::google::protobuf::internal::FieldMetadata field_metadata[]; + static const ::google::protobuf::internal::SerializationTable serialization_table[]; + static const ::google::protobuf::uint32 offsets[]; +}; +void AddDescriptors(); +} // namespace protobuf_tensorflow_2fcompiler_2fxla_2fxla_5fdata_2eproto +namespace xla { +class ChannelHandle; +class ChannelHandleDefaultTypeInternal; +extern ChannelHandleDefaultTypeInternal _ChannelHandle_default_instance_; +class ComputationStats; +class ComputationStatsDefaultTypeInternal; +extern ComputationStatsDefaultTypeInternal _ComputationStats_default_instance_; +class ConvolutionDimensionNumbers; +class ConvolutionDimensionNumbersDefaultTypeInternal; +extern ConvolutionDimensionNumbersDefaultTypeInternal _ConvolutionDimensionNumbers_default_instance_; +class DeviceAssignmentProto; +class DeviceAssignmentProtoDefaultTypeInternal; +extern DeviceAssignmentProtoDefaultTypeInternal _DeviceAssignmentProto_default_instance_; +class DeviceAssignmentProto_ComputationDevice; +class DeviceAssignmentProto_ComputationDeviceDefaultTypeInternal; +extern DeviceAssignmentProto_ComputationDeviceDefaultTypeInternal _DeviceAssignmentProto_ComputationDevice_default_instance_; +class DeviceHandle; +class DeviceHandleDefaultTypeInternal; +extern DeviceHandleDefaultTypeInternal _DeviceHandle_default_instance_; +class DotDimensionNumbers; +class DotDimensionNumbersDefaultTypeInternal; +extern DotDimensionNumbersDefaultTypeInternal _DotDimensionNumbers_default_instance_; +class ExecutionHandle; +class ExecutionHandleDefaultTypeInternal; +extern ExecutionHandleDefaultTypeInternal _ExecutionHandle_default_instance_; +class ExecutionProfile; +class ExecutionProfileDefaultTypeInternal; +extern ExecutionProfileDefaultTypeInternal _ExecutionProfile_default_instance_; +class GatherDimensionNumbers; +class GatherDimensionNumbersDefaultTypeInternal; +extern GatherDimensionNumbersDefaultTypeInternal _GatherDimensionNumbers_default_instance_; +class GlobalDataHandle; +class GlobalDataHandleDefaultTypeInternal; +extern GlobalDataHandleDefaultTypeInternal _GlobalDataHandle_default_instance_; +class Layout; +class LayoutDefaultTypeInternal; +extern LayoutDefaultTypeInternal _Layout_default_instance_; +class LiteralProto; +class LiteralProtoDefaultTypeInternal; +extern LiteralProtoDefaultTypeInternal _LiteralProto_default_instance_; +class OpMetadata; +class OpMetadataDefaultTypeInternal; +extern OpMetadataDefaultTypeInternal _OpMetadata_default_instance_; +class OpSharding; +class OpShardingDefaultTypeInternal; +extern OpShardingDefaultTypeInternal _OpSharding_default_instance_; +class PaddingConfig; +class PaddingConfigDefaultTypeInternal; +extern PaddingConfigDefaultTypeInternal _PaddingConfig_default_instance_; +class PaddingConfig_PaddingConfigDimension; +class PaddingConfig_PaddingConfigDimensionDefaultTypeInternal; +extern PaddingConfig_PaddingConfigDimensionDefaultTypeInternal _PaddingConfig_PaddingConfigDimension_default_instance_; +class PrecisionConfig; +class PrecisionConfigDefaultTypeInternal; +extern PrecisionConfigDefaultTypeInternal _PrecisionConfig_default_instance_; +class ProgramShape; +class ProgramShapeDefaultTypeInternal; +extern ProgramShapeDefaultTypeInternal _ProgramShape_default_instance_; +class ReplicaGroup; +class ReplicaGroupDefaultTypeInternal; +extern ReplicaGroupDefaultTypeInternal _ReplicaGroup_default_instance_; +class ScatterDimensionNumbers; +class ScatterDimensionNumbersDefaultTypeInternal; +extern ScatterDimensionNumbersDefaultTypeInternal _ScatterDimensionNumbers_default_instance_; +class Shape; +class ShapeDefaultTypeInternal; +extern ShapeDefaultTypeInternal _Shape_default_instance_; +class SourceTarget; +class SourceTargetDefaultTypeInternal; +extern SourceTargetDefaultTypeInternal _SourceTarget_default_instance_; +class Window; +class WindowDefaultTypeInternal; +extern WindowDefaultTypeInternal _Window_default_instance_; +class WindowDimension; +class WindowDimensionDefaultTypeInternal; +extern WindowDimensionDefaultTypeInternal _WindowDimension_default_instance_; +} // namespace xla +namespace google { +namespace protobuf { +template<> ::xla::ChannelHandle* Arena::CreateMaybeMessage<::xla::ChannelHandle>(Arena*); +template<> ::xla::ComputationStats* Arena::CreateMaybeMessage<::xla::ComputationStats>(Arena*); +template<> ::xla::ConvolutionDimensionNumbers* Arena::CreateMaybeMessage<::xla::ConvolutionDimensionNumbers>(Arena*); +template<> ::xla::DeviceAssignmentProto* Arena::CreateMaybeMessage<::xla::DeviceAssignmentProto>(Arena*); +template<> ::xla::DeviceAssignmentProto_ComputationDevice* Arena::CreateMaybeMessage<::xla::DeviceAssignmentProto_ComputationDevice>(Arena*); +template<> ::xla::DeviceHandle* Arena::CreateMaybeMessage<::xla::DeviceHandle>(Arena*); +template<> ::xla::DotDimensionNumbers* Arena::CreateMaybeMessage<::xla::DotDimensionNumbers>(Arena*); +template<> ::xla::ExecutionHandle* Arena::CreateMaybeMessage<::xla::ExecutionHandle>(Arena*); +template<> ::xla::ExecutionProfile* Arena::CreateMaybeMessage<::xla::ExecutionProfile>(Arena*); +template<> ::xla::GatherDimensionNumbers* Arena::CreateMaybeMessage<::xla::GatherDimensionNumbers>(Arena*); +template<> ::xla::GlobalDataHandle* Arena::CreateMaybeMessage<::xla::GlobalDataHandle>(Arena*); +template<> ::xla::Layout* Arena::CreateMaybeMessage<::xla::Layout>(Arena*); +template<> ::xla::LiteralProto* Arena::CreateMaybeMessage<::xla::LiteralProto>(Arena*); +template<> ::xla::OpMetadata* Arena::CreateMaybeMessage<::xla::OpMetadata>(Arena*); +template<> ::xla::OpSharding* Arena::CreateMaybeMessage<::xla::OpSharding>(Arena*); +template<> ::xla::PaddingConfig* Arena::CreateMaybeMessage<::xla::PaddingConfig>(Arena*); +template<> ::xla::PaddingConfig_PaddingConfigDimension* Arena::CreateMaybeMessage<::xla::PaddingConfig_PaddingConfigDimension>(Arena*); +template<> ::xla::PrecisionConfig* Arena::CreateMaybeMessage<::xla::PrecisionConfig>(Arena*); +template<> ::xla::ProgramShape* Arena::CreateMaybeMessage<::xla::ProgramShape>(Arena*); +template<> ::xla::ReplicaGroup* Arena::CreateMaybeMessage<::xla::ReplicaGroup>(Arena*); +template<> ::xla::ScatterDimensionNumbers* Arena::CreateMaybeMessage<::xla::ScatterDimensionNumbers>(Arena*); +template<> ::xla::Shape* Arena::CreateMaybeMessage<::xla::Shape>(Arena*); +template<> ::xla::SourceTarget* Arena::CreateMaybeMessage<::xla::SourceTarget>(Arena*); +template<> ::xla::Window* Arena::CreateMaybeMessage<::xla::Window>(Arena*); +template<> ::xla::WindowDimension* Arena::CreateMaybeMessage<::xla::WindowDimension>(Arena*); +} // namespace protobuf +} // namespace google +namespace xla { + +enum ChannelHandle_ChannelType { + ChannelHandle_ChannelType_CHANNEL_TYPE_INVALID = 0, + ChannelHandle_ChannelType_DEVICE_TO_DEVICE = 1, + ChannelHandle_ChannelType_DEVICE_TO_HOST = 2, + ChannelHandle_ChannelType_HOST_TO_DEVICE = 3, + ChannelHandle_ChannelType_ChannelHandle_ChannelType_INT_MIN_SENTINEL_DO_NOT_USE_ = ::google::protobuf::kint32min, + ChannelHandle_ChannelType_ChannelHandle_ChannelType_INT_MAX_SENTINEL_DO_NOT_USE_ = ::google::protobuf::kint32max +}; +bool ChannelHandle_ChannelType_IsValid(int value); +const ChannelHandle_ChannelType ChannelHandle_ChannelType_ChannelType_MIN = ChannelHandle_ChannelType_CHANNEL_TYPE_INVALID; +const ChannelHandle_ChannelType ChannelHandle_ChannelType_ChannelType_MAX = ChannelHandle_ChannelType_HOST_TO_DEVICE; +const int ChannelHandle_ChannelType_ChannelType_ARRAYSIZE = ChannelHandle_ChannelType_ChannelType_MAX + 1; + +const ::google::protobuf::EnumDescriptor* ChannelHandle_ChannelType_descriptor(); +inline const ::std::string& ChannelHandle_ChannelType_Name(ChannelHandle_ChannelType value) { + return ::google::protobuf::internal::NameOfEnum( + ChannelHandle_ChannelType_descriptor(), value); +} +inline bool ChannelHandle_ChannelType_Parse( + const ::std::string& name, ChannelHandle_ChannelType* value) { + return ::google::protobuf::internal::ParseNamedEnum( + ChannelHandle_ChannelType_descriptor(), name, value); +} +enum OpSharding_Type { + OpSharding_Type_REPLICATED = 0, + OpSharding_Type_MAXIMAL = 1, + OpSharding_Type_TUPLE = 2, + OpSharding_Type_OTHER = 3, + OpSharding_Type_OpSharding_Type_INT_MIN_SENTINEL_DO_NOT_USE_ = ::google::protobuf::kint32min, + OpSharding_Type_OpSharding_Type_INT_MAX_SENTINEL_DO_NOT_USE_ = ::google::protobuf::kint32max +}; +bool OpSharding_Type_IsValid(int value); +const OpSharding_Type OpSharding_Type_Type_MIN = OpSharding_Type_REPLICATED; +const OpSharding_Type OpSharding_Type_Type_MAX = OpSharding_Type_OTHER; +const int OpSharding_Type_Type_ARRAYSIZE = OpSharding_Type_Type_MAX + 1; + +const ::google::protobuf::EnumDescriptor* OpSharding_Type_descriptor(); +inline const ::std::string& OpSharding_Type_Name(OpSharding_Type value) { + return ::google::protobuf::internal::NameOfEnum( + OpSharding_Type_descriptor(), value); +} +inline bool OpSharding_Type_Parse( + const ::std::string& name, OpSharding_Type* value) { + return ::google::protobuf::internal::ParseNamedEnum( + OpSharding_Type_descriptor(), name, value); +} +enum PrecisionConfig_Precision { + PrecisionConfig_Precision_DEFAULT = 0, + PrecisionConfig_Precision_HIGH = 1, + PrecisionConfig_Precision_HIGHEST = 2, + PrecisionConfig_Precision_PrecisionConfig_Precision_INT_MIN_SENTINEL_DO_NOT_USE_ = ::google::protobuf::kint32min, + PrecisionConfig_Precision_PrecisionConfig_Precision_INT_MAX_SENTINEL_DO_NOT_USE_ = ::google::protobuf::kint32max +}; +bool PrecisionConfig_Precision_IsValid(int value); +const PrecisionConfig_Precision PrecisionConfig_Precision_Precision_MIN = PrecisionConfig_Precision_DEFAULT; +const PrecisionConfig_Precision PrecisionConfig_Precision_Precision_MAX = PrecisionConfig_Precision_HIGHEST; +const int PrecisionConfig_Precision_Precision_ARRAYSIZE = PrecisionConfig_Precision_Precision_MAX + 1; + +const ::google::protobuf::EnumDescriptor* PrecisionConfig_Precision_descriptor(); +inline const ::std::string& PrecisionConfig_Precision_Name(PrecisionConfig_Precision value) { + return ::google::protobuf::internal::NameOfEnum( + PrecisionConfig_Precision_descriptor(), value); +} +inline bool PrecisionConfig_Precision_Parse( + const ::std::string& name, PrecisionConfig_Precision* value) { + return ::google::protobuf::internal::ParseNamedEnum( + PrecisionConfig_Precision_descriptor(), name, value); +} +enum PrimitiveType { + PRIMITIVE_TYPE_INVALID = 0, + PRED = 1, + S8 = 2, + S16 = 3, + S32 = 4, + S64 = 5, + U8 = 6, + U16 = 7, + U32 = 8, + U64 = 9, + F16 = 10, + F32 = 11, + BF16 = 16, + F64 = 12, + C64 = 15, + TUPLE = 13, + OPAQUE = 14, + TOKEN = 17, + PrimitiveType_INT_MIN_SENTINEL_DO_NOT_USE_ = ::google::protobuf::kint32min, + PrimitiveType_INT_MAX_SENTINEL_DO_NOT_USE_ = ::google::protobuf::kint32max +}; +bool PrimitiveType_IsValid(int value); +const PrimitiveType PrimitiveType_MIN = PRIMITIVE_TYPE_INVALID; +const PrimitiveType PrimitiveType_MAX = TOKEN; +const int PrimitiveType_ARRAYSIZE = PrimitiveType_MAX + 1; + +const ::google::protobuf::EnumDescriptor* PrimitiveType_descriptor(); +inline const ::std::string& PrimitiveType_Name(PrimitiveType value) { + return ::google::protobuf::internal::NameOfEnum( + PrimitiveType_descriptor(), value); +} +inline bool PrimitiveType_Parse( + const ::std::string& name, PrimitiveType* value) { + return ::google::protobuf::internal::ParseNamedEnum( + PrimitiveType_descriptor(), name, value); +} +enum PaddingValue { + INVALID_PAD = 0, + ZERO_PAD = 1, + ONE_PAD = 2, + LOWEST_PAD = 3, + HIGHEST_PAD = 4, + UNKNOWN_PAD = 5, + PaddingValue_INT_MIN_SENTINEL_DO_NOT_USE_ = ::google::protobuf::kint32min, + PaddingValue_INT_MAX_SENTINEL_DO_NOT_USE_ = ::google::protobuf::kint32max +}; +bool PaddingValue_IsValid(int value); +const PaddingValue PaddingValue_MIN = INVALID_PAD; +const PaddingValue PaddingValue_MAX = UNKNOWN_PAD; +const int PaddingValue_ARRAYSIZE = PaddingValue_MAX + 1; + +const ::google::protobuf::EnumDescriptor* PaddingValue_descriptor(); +inline const ::std::string& PaddingValue_Name(PaddingValue value) { + return ::google::protobuf::internal::NameOfEnum( + PaddingValue_descriptor(), value); +} +inline bool PaddingValue_Parse( + const ::std::string& name, PaddingValue* value) { + return ::google::protobuf::internal::ParseNamedEnum( + PaddingValue_descriptor(), name, value); +} +enum Format { + INVALID_FORMAT = 0, + DENSE = 1, + SPARSE = 2, + Format_INT_MIN_SENTINEL_DO_NOT_USE_ = ::google::protobuf::kint32min, + Format_INT_MAX_SENTINEL_DO_NOT_USE_ = ::google::protobuf::kint32max +}; +bool Format_IsValid(int value); +const Format Format_MIN = INVALID_FORMAT; +const Format Format_MAX = SPARSE; +const int Format_ARRAYSIZE = Format_MAX + 1; + +const ::google::protobuf::EnumDescriptor* Format_descriptor(); +inline const ::std::string& Format_Name(Format value) { + return ::google::protobuf::internal::NameOfEnum( + Format_descriptor(), value); +} +inline bool Format_Parse( + const ::std::string& name, Format* value) { + return ::google::protobuf::internal::ParseNamedEnum( + Format_descriptor(), name, value); +} +enum FftType { + FFT = 0, + IFFT = 1, + RFFT = 2, + IRFFT = 3, + FftType_INT_MIN_SENTINEL_DO_NOT_USE_ = ::google::protobuf::kint32min, + FftType_INT_MAX_SENTINEL_DO_NOT_USE_ = ::google::protobuf::kint32max +}; +bool FftType_IsValid(int value); +const FftType FftType_MIN = FFT; +const FftType FftType_MAX = IRFFT; +const int FftType_ARRAYSIZE = FftType_MAX + 1; + +const ::google::protobuf::EnumDescriptor* FftType_descriptor(); +inline const ::std::string& FftType_Name(FftType value) { + return ::google::protobuf::internal::NameOfEnum( + FftType_descriptor(), value); +} +inline bool FftType_Parse( + const ::std::string& name, FftType* value) { + return ::google::protobuf::internal::ParseNamedEnum( + FftType_descriptor(), name, value); +} +enum RandomDistribution { + RNG_INVALID = 0, + RNG_UNIFORM = 1, + RNG_NORMAL = 2, + RandomDistribution_INT_MIN_SENTINEL_DO_NOT_USE_ = ::google::protobuf::kint32min, + RandomDistribution_INT_MAX_SENTINEL_DO_NOT_USE_ = ::google::protobuf::kint32max +}; +bool RandomDistribution_IsValid(int value); +const RandomDistribution RandomDistribution_MIN = RNG_INVALID; +const RandomDistribution RandomDistribution_MAX = RNG_NORMAL; +const int RandomDistribution_ARRAYSIZE = RandomDistribution_MAX + 1; + +const ::google::protobuf::EnumDescriptor* RandomDistribution_descriptor(); +inline const ::std::string& RandomDistribution_Name(RandomDistribution value) { + return ::google::protobuf::internal::NameOfEnum( + RandomDistribution_descriptor(), value); +} +inline bool RandomDistribution_Parse( + const ::std::string& name, RandomDistribution* value) { + return ::google::protobuf::internal::ParseNamedEnum( + RandomDistribution_descriptor(), name, value); +} +// =================================================================== + +class PaddingConfig_PaddingConfigDimension : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.PaddingConfig.PaddingConfigDimension) */ { + public: + PaddingConfig_PaddingConfigDimension(); + virtual ~PaddingConfig_PaddingConfigDimension(); + + PaddingConfig_PaddingConfigDimension(const PaddingConfig_PaddingConfigDimension& from); + + inline PaddingConfig_PaddingConfigDimension& operator=(const PaddingConfig_PaddingConfigDimension& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + PaddingConfig_PaddingConfigDimension(PaddingConfig_PaddingConfigDimension&& from) noexcept + : PaddingConfig_PaddingConfigDimension() { + *this = ::std::move(from); + } + + inline PaddingConfig_PaddingConfigDimension& operator=(PaddingConfig_PaddingConfigDimension&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const PaddingConfig_PaddingConfigDimension& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const PaddingConfig_PaddingConfigDimension* internal_default_instance() { + return reinterpret_cast( + &_PaddingConfig_PaddingConfigDimension_default_instance_); + } + static constexpr int kIndexInFileMessages = + 0; + + void UnsafeArenaSwap(PaddingConfig_PaddingConfigDimension* other); + void Swap(PaddingConfig_PaddingConfigDimension* other); + friend void swap(PaddingConfig_PaddingConfigDimension& a, PaddingConfig_PaddingConfigDimension& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline PaddingConfig_PaddingConfigDimension* New() const final { + return CreateMaybeMessage(NULL); + } + + PaddingConfig_PaddingConfigDimension* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const PaddingConfig_PaddingConfigDimension& from); + void MergeFrom(const PaddingConfig_PaddingConfigDimension& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(PaddingConfig_PaddingConfigDimension* other); + protected: + explicit PaddingConfig_PaddingConfigDimension(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // int64 edge_padding_low = 1; + void clear_edge_padding_low(); + static const int kEdgePaddingLowFieldNumber = 1; + ::google::protobuf::int64 edge_padding_low() const; + void set_edge_padding_low(::google::protobuf::int64 value); + + // int64 edge_padding_high = 2; + void clear_edge_padding_high(); + static const int kEdgePaddingHighFieldNumber = 2; + ::google::protobuf::int64 edge_padding_high() const; + void set_edge_padding_high(::google::protobuf::int64 value); + + // int64 interior_padding = 3; + void clear_interior_padding(); + static const int kInteriorPaddingFieldNumber = 3; + ::google::protobuf::int64 interior_padding() const; + void set_interior_padding(::google::protobuf::int64 value); + + // @@protoc_insertion_point(class_scope:xla.PaddingConfig.PaddingConfigDimension) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::int64 edge_padding_low_; + ::google::protobuf::int64 edge_padding_high_; + ::google::protobuf::int64 interior_padding_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_5fdata_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class PaddingConfig : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.PaddingConfig) */ { + public: + PaddingConfig(); + virtual ~PaddingConfig(); + + PaddingConfig(const PaddingConfig& from); + + inline PaddingConfig& operator=(const PaddingConfig& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + PaddingConfig(PaddingConfig&& from) noexcept + : PaddingConfig() { + *this = ::std::move(from); + } + + inline PaddingConfig& operator=(PaddingConfig&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const PaddingConfig& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const PaddingConfig* internal_default_instance() { + return reinterpret_cast( + &_PaddingConfig_default_instance_); + } + static constexpr int kIndexInFileMessages = + 1; + + void UnsafeArenaSwap(PaddingConfig* other); + void Swap(PaddingConfig* other); + friend void swap(PaddingConfig& a, PaddingConfig& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline PaddingConfig* New() const final { + return CreateMaybeMessage(NULL); + } + + PaddingConfig* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const PaddingConfig& from); + void MergeFrom(const PaddingConfig& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(PaddingConfig* other); + protected: + explicit PaddingConfig(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + typedef PaddingConfig_PaddingConfigDimension PaddingConfigDimension; + + // accessors ------------------------------------------------------- + + // repeated .xla.PaddingConfig.PaddingConfigDimension dimensions = 1; + int dimensions_size() const; + void clear_dimensions(); + static const int kDimensionsFieldNumber = 1; + ::xla::PaddingConfig_PaddingConfigDimension* mutable_dimensions(int index); + ::google::protobuf::RepeatedPtrField< ::xla::PaddingConfig_PaddingConfigDimension >* + mutable_dimensions(); + const ::xla::PaddingConfig_PaddingConfigDimension& dimensions(int index) const; + ::xla::PaddingConfig_PaddingConfigDimension* add_dimensions(); + const ::google::protobuf::RepeatedPtrField< ::xla::PaddingConfig_PaddingConfigDimension >& + dimensions() const; + + // @@protoc_insertion_point(class_scope:xla.PaddingConfig) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedPtrField< ::xla::PaddingConfig_PaddingConfigDimension > dimensions_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_5fdata_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class Layout : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.Layout) */ { + public: + Layout(); + virtual ~Layout(); + + Layout(const Layout& from); + + inline Layout& operator=(const Layout& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + Layout(Layout&& from) noexcept + : Layout() { + *this = ::std::move(from); + } + + inline Layout& operator=(Layout&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const Layout& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const Layout* internal_default_instance() { + return reinterpret_cast( + &_Layout_default_instance_); + } + static constexpr int kIndexInFileMessages = + 2; + + void UnsafeArenaSwap(Layout* other); + void Swap(Layout* other); + friend void swap(Layout& a, Layout& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline Layout* New() const final { + return CreateMaybeMessage(NULL); + } + + Layout* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const Layout& from); + void MergeFrom(const Layout& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(Layout* other); + protected: + explicit Layout(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated int64 minor_to_major = 1; + int minor_to_major_size() const; + void clear_minor_to_major(); + static const int kMinorToMajorFieldNumber = 1; + ::google::protobuf::int64 minor_to_major(int index) const; + void set_minor_to_major(int index, ::google::protobuf::int64 value); + void add_minor_to_major(::google::protobuf::int64 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& + minor_to_major() const; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* + mutable_minor_to_major(); + + // repeated int64 padded_dimensions = 2; + int padded_dimensions_size() const; + void clear_padded_dimensions(); + static const int kPaddedDimensionsFieldNumber = 2; + ::google::protobuf::int64 padded_dimensions(int index) const; + void set_padded_dimensions(int index, ::google::protobuf::int64 value); + void add_padded_dimensions(::google::protobuf::int64 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& + padded_dimensions() const; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* + mutable_padded_dimensions(); + + // .xla.PaddingValue padding_value = 3; + void clear_padding_value(); + static const int kPaddingValueFieldNumber = 3; + ::xla::PaddingValue padding_value() const; + void set_padding_value(::xla::PaddingValue value); + + // .xla.Format format = 4; + void clear_format(); + static const int kFormatFieldNumber = 4; + ::xla::Format format() const; + void set_format(::xla::Format value); + + // int64 max_sparse_elements = 5; + void clear_max_sparse_elements(); + static const int kMaxSparseElementsFieldNumber = 5; + ::google::protobuf::int64 max_sparse_elements() const; + void set_max_sparse_elements(::google::protobuf::int64 value); + + // @@protoc_insertion_point(class_scope:xla.Layout) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 > minor_to_major_; + mutable int _minor_to_major_cached_byte_size_; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 > padded_dimensions_; + mutable int _padded_dimensions_cached_byte_size_; + int padding_value_; + int format_; + ::google::protobuf::int64 max_sparse_elements_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_5fdata_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class Shape : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.Shape) */ { + public: + Shape(); + virtual ~Shape(); + + Shape(const Shape& from); + + inline Shape& operator=(const Shape& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + Shape(Shape&& from) noexcept + : Shape() { + *this = ::std::move(from); + } + + inline Shape& operator=(Shape&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const Shape& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const Shape* internal_default_instance() { + return reinterpret_cast( + &_Shape_default_instance_); + } + static constexpr int kIndexInFileMessages = + 3; + + void UnsafeArenaSwap(Shape* other); + void Swap(Shape* other); + friend void swap(Shape& a, Shape& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline Shape* New() const final { + return CreateMaybeMessage(NULL); + } + + Shape* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const Shape& from); + void MergeFrom(const Shape& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(Shape* other); + protected: + explicit Shape(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated int64 dimensions = 3; + int dimensions_size() const; + void clear_dimensions(); + static const int kDimensionsFieldNumber = 3; + ::google::protobuf::int64 dimensions(int index) const; + void set_dimensions(int index, ::google::protobuf::int64 value); + void add_dimensions(::google::protobuf::int64 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& + dimensions() const; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* + mutable_dimensions(); + + // repeated .xla.Shape tuple_shapes = 4; + int tuple_shapes_size() const; + void clear_tuple_shapes(); + static const int kTupleShapesFieldNumber = 4; + ::xla::Shape* mutable_tuple_shapes(int index); + ::google::protobuf::RepeatedPtrField< ::xla::Shape >* + mutable_tuple_shapes(); + const ::xla::Shape& tuple_shapes(int index) const; + ::xla::Shape* add_tuple_shapes(); + const ::google::protobuf::RepeatedPtrField< ::xla::Shape >& + tuple_shapes() const; + + // .xla.Layout layout = 5; + bool has_layout() const; + void clear_layout(); + static const int kLayoutFieldNumber = 5; + private: + const ::xla::Layout& _internal_layout() const; + public: + const ::xla::Layout& layout() const; + ::xla::Layout* release_layout(); + ::xla::Layout* mutable_layout(); + void set_allocated_layout(::xla::Layout* layout); + void unsafe_arena_set_allocated_layout( + ::xla::Layout* layout); + ::xla::Layout* unsafe_arena_release_layout(); + + // .xla.PrimitiveType element_type = 2; + void clear_element_type(); + static const int kElementTypeFieldNumber = 2; + ::xla::PrimitiveType element_type() const; + void set_element_type(::xla::PrimitiveType value); + + // @@protoc_insertion_point(class_scope:xla.Shape) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 > dimensions_; + mutable int _dimensions_cached_byte_size_; + ::google::protobuf::RepeatedPtrField< ::xla::Shape > tuple_shapes_; + ::xla::Layout* layout_; + int element_type_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_5fdata_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class ProgramShape : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.ProgramShape) */ { + public: + ProgramShape(); + virtual ~ProgramShape(); + + ProgramShape(const ProgramShape& from); + + inline ProgramShape& operator=(const ProgramShape& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + ProgramShape(ProgramShape&& from) noexcept + : ProgramShape() { + *this = ::std::move(from); + } + + inline ProgramShape& operator=(ProgramShape&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const ProgramShape& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const ProgramShape* internal_default_instance() { + return reinterpret_cast( + &_ProgramShape_default_instance_); + } + static constexpr int kIndexInFileMessages = + 4; + + void UnsafeArenaSwap(ProgramShape* other); + void Swap(ProgramShape* other); + friend void swap(ProgramShape& a, ProgramShape& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline ProgramShape* New() const final { + return CreateMaybeMessage(NULL); + } + + ProgramShape* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const ProgramShape& from); + void MergeFrom(const ProgramShape& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(ProgramShape* other); + protected: + explicit ProgramShape(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated .xla.Shape parameters = 1; + int parameters_size() const; + void clear_parameters(); + static const int kParametersFieldNumber = 1; + ::xla::Shape* mutable_parameters(int index); + ::google::protobuf::RepeatedPtrField< ::xla::Shape >* + mutable_parameters(); + const ::xla::Shape& parameters(int index) const; + ::xla::Shape* add_parameters(); + const ::google::protobuf::RepeatedPtrField< ::xla::Shape >& + parameters() const; + + // repeated string parameter_names = 3; + int parameter_names_size() const; + void clear_parameter_names(); + static const int kParameterNamesFieldNumber = 3; + const ::std::string& parameter_names(int index) const; + ::std::string* mutable_parameter_names(int index); + void set_parameter_names(int index, const ::std::string& value); + #if LANG_CXX11 + void set_parameter_names(int index, ::std::string&& value); + #endif + void set_parameter_names(int index, const char* value); + void set_parameter_names(int index, const char* value, size_t size); + ::std::string* add_parameter_names(); + void add_parameter_names(const ::std::string& value); + #if LANG_CXX11 + void add_parameter_names(::std::string&& value); + #endif + void add_parameter_names(const char* value); + void add_parameter_names(const char* value, size_t size); + const ::google::protobuf::RepeatedPtrField< ::std::string>& parameter_names() const; + ::google::protobuf::RepeatedPtrField< ::std::string>* mutable_parameter_names(); + + // .xla.Shape result = 2; + bool has_result() const; + void clear_result(); + static const int kResultFieldNumber = 2; + private: + const ::xla::Shape& _internal_result() const; + public: + const ::xla::Shape& result() const; + ::xla::Shape* release_result(); + ::xla::Shape* mutable_result(); + void set_allocated_result(::xla::Shape* result); + void unsafe_arena_set_allocated_result( + ::xla::Shape* result); + ::xla::Shape* unsafe_arena_release_result(); + + // @@protoc_insertion_point(class_scope:xla.ProgramShape) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedPtrField< ::xla::Shape > parameters_; + ::google::protobuf::RepeatedPtrField< ::std::string> parameter_names_; + ::xla::Shape* result_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_5fdata_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class ComputationStats : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.ComputationStats) */ { + public: + ComputationStats(); + virtual ~ComputationStats(); + + ComputationStats(const ComputationStats& from); + + inline ComputationStats& operator=(const ComputationStats& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + ComputationStats(ComputationStats&& from) noexcept + : ComputationStats() { + *this = ::std::move(from); + } + + inline ComputationStats& operator=(ComputationStats&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const ComputationStats& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const ComputationStats* internal_default_instance() { + return reinterpret_cast( + &_ComputationStats_default_instance_); + } + static constexpr int kIndexInFileMessages = + 5; + + void UnsafeArenaSwap(ComputationStats* other); + void Swap(ComputationStats* other); + friend void swap(ComputationStats& a, ComputationStats& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline ComputationStats* New() const final { + return CreateMaybeMessage(NULL); + } + + ComputationStats* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const ComputationStats& from); + void MergeFrom(const ComputationStats& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(ComputationStats* other); + protected: + explicit ComputationStats(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // double flop_count = 1; + void clear_flop_count(); + static const int kFlopCountFieldNumber = 1; + double flop_count() const; + void set_flop_count(double value); + + // double transcendental_count = 2; + void clear_transcendental_count(); + static const int kTranscendentalCountFieldNumber = 2; + double transcendental_count() const; + void set_transcendental_count(double value); + + // @@protoc_insertion_point(class_scope:xla.ComputationStats) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + double flop_count_; + double transcendental_count_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_5fdata_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class OpMetadata : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.OpMetadata) */ { + public: + OpMetadata(); + virtual ~OpMetadata(); + + OpMetadata(const OpMetadata& from); + + inline OpMetadata& operator=(const OpMetadata& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + OpMetadata(OpMetadata&& from) noexcept + : OpMetadata() { + *this = ::std::move(from); + } + + inline OpMetadata& operator=(OpMetadata&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const OpMetadata& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const OpMetadata* internal_default_instance() { + return reinterpret_cast( + &_OpMetadata_default_instance_); + } + static constexpr int kIndexInFileMessages = + 6; + + void UnsafeArenaSwap(OpMetadata* other); + void Swap(OpMetadata* other); + friend void swap(OpMetadata& a, OpMetadata& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline OpMetadata* New() const final { + return CreateMaybeMessage(NULL); + } + + OpMetadata* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const OpMetadata& from); + void MergeFrom(const OpMetadata& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(OpMetadata* other); + protected: + explicit OpMetadata(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // string op_type = 1; + void clear_op_type(); + static const int kOpTypeFieldNumber = 1; + const ::std::string& op_type() const; + void set_op_type(const ::std::string& value); + #if LANG_CXX11 + void set_op_type(::std::string&& value); + #endif + void set_op_type(const char* value); + void set_op_type(const char* value, size_t size); + ::std::string* mutable_op_type(); + ::std::string* release_op_type(); + void set_allocated_op_type(::std::string* op_type); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_op_type(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_op_type( + ::std::string* op_type); + + // string op_name = 2; + void clear_op_name(); + static const int kOpNameFieldNumber = 2; + const ::std::string& op_name() const; + void set_op_name(const ::std::string& value); + #if LANG_CXX11 + void set_op_name(::std::string&& value); + #endif + void set_op_name(const char* value); + void set_op_name(const char* value, size_t size); + ::std::string* mutable_op_name(); + ::std::string* release_op_name(); + void set_allocated_op_name(::std::string* op_name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_op_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_op_name( + ::std::string* op_name); + + // string source_file = 3; + void clear_source_file(); + static const int kSourceFileFieldNumber = 3; + const ::std::string& source_file() const; + void set_source_file(const ::std::string& value); + #if LANG_CXX11 + void set_source_file(::std::string&& value); + #endif + void set_source_file(const char* value); + void set_source_file(const char* value, size_t size); + ::std::string* mutable_source_file(); + ::std::string* release_source_file(); + void set_allocated_source_file(::std::string* source_file); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_source_file(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_source_file( + ::std::string* source_file); + + // int32 source_line = 4; + void clear_source_line(); + static const int kSourceLineFieldNumber = 4; + ::google::protobuf::int32 source_line() const; + void set_source_line(::google::protobuf::int32 value); + + // @@protoc_insertion_point(class_scope:xla.OpMetadata) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::internal::ArenaStringPtr op_type_; + ::google::protobuf::internal::ArenaStringPtr op_name_; + ::google::protobuf::internal::ArenaStringPtr source_file_; + ::google::protobuf::int32 source_line_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_5fdata_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class ExecutionProfile : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.ExecutionProfile) */ { + public: + ExecutionProfile(); + virtual ~ExecutionProfile(); + + ExecutionProfile(const ExecutionProfile& from); + + inline ExecutionProfile& operator=(const ExecutionProfile& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + ExecutionProfile(ExecutionProfile&& from) noexcept + : ExecutionProfile() { + *this = ::std::move(from); + } + + inline ExecutionProfile& operator=(ExecutionProfile&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const ExecutionProfile& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const ExecutionProfile* internal_default_instance() { + return reinterpret_cast( + &_ExecutionProfile_default_instance_); + } + static constexpr int kIndexInFileMessages = + 7; + + void UnsafeArenaSwap(ExecutionProfile* other); + void Swap(ExecutionProfile* other); + friend void swap(ExecutionProfile& a, ExecutionProfile& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline ExecutionProfile* New() const final { + return CreateMaybeMessage(NULL); + } + + ExecutionProfile* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const ExecutionProfile& from); + void MergeFrom(const ExecutionProfile& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(ExecutionProfile* other); + protected: + explicit ExecutionProfile(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // int64 compile_time_ms = 2; + void clear_compile_time_ms(); + static const int kCompileTimeMsFieldNumber = 2; + ::google::protobuf::int64 compile_time_ms() const; + void set_compile_time_ms(::google::protobuf::int64 value); + + // int64 compute_cycle_count = 3; + void clear_compute_cycle_count(); + static const int kComputeCycleCountFieldNumber = 3; + ::google::protobuf::int64 compute_cycle_count() const; + void set_compute_cycle_count(::google::protobuf::int64 value); + + // int64 compute_time_ns = 4; + void clear_compute_time_ns(); + static const int kComputeTimeNsFieldNumber = 4; + ::google::protobuf::int64 compute_time_ns() const; + void set_compute_time_ns(::google::protobuf::int64 value); + + // int64 compute_and_transfer_time_ns = 5; + void clear_compute_and_transfer_time_ns(); + static const int kComputeAndTransferTimeNsFieldNumber = 5; + ::google::protobuf::int64 compute_and_transfer_time_ns() const; + void set_compute_and_transfer_time_ns(::google::protobuf::int64 value); + + // int64 executable_size_in_bytes = 6; + void clear_executable_size_in_bytes(); + static const int kExecutableSizeInBytesFieldNumber = 6; + ::google::protobuf::int64 executable_size_in_bytes() const; + void set_executable_size_in_bytes(::google::protobuf::int64 value); + + // bool compilation_cache_hit = 1; + void clear_compilation_cache_hit(); + static const int kCompilationCacheHitFieldNumber = 1; + bool compilation_cache_hit() const; + void set_compilation_cache_hit(bool value); + + // @@protoc_insertion_point(class_scope:xla.ExecutionProfile) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::int64 compile_time_ms_; + ::google::protobuf::int64 compute_cycle_count_; + ::google::protobuf::int64 compute_time_ns_; + ::google::protobuf::int64 compute_and_transfer_time_ns_; + ::google::protobuf::int64 executable_size_in_bytes_; + bool compilation_cache_hit_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_5fdata_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class ExecutionHandle : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.ExecutionHandle) */ { + public: + ExecutionHandle(); + virtual ~ExecutionHandle(); + + ExecutionHandle(const ExecutionHandle& from); + + inline ExecutionHandle& operator=(const ExecutionHandle& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + ExecutionHandle(ExecutionHandle&& from) noexcept + : ExecutionHandle() { + *this = ::std::move(from); + } + + inline ExecutionHandle& operator=(ExecutionHandle&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const ExecutionHandle& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const ExecutionHandle* internal_default_instance() { + return reinterpret_cast( + &_ExecutionHandle_default_instance_); + } + static constexpr int kIndexInFileMessages = + 8; + + void UnsafeArenaSwap(ExecutionHandle* other); + void Swap(ExecutionHandle* other); + friend void swap(ExecutionHandle& a, ExecutionHandle& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline ExecutionHandle* New() const final { + return CreateMaybeMessage(NULL); + } + + ExecutionHandle* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const ExecutionHandle& from); + void MergeFrom(const ExecutionHandle& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(ExecutionHandle* other); + protected: + explicit ExecutionHandle(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // int64 handle = 1; + void clear_handle(); + static const int kHandleFieldNumber = 1; + ::google::protobuf::int64 handle() const; + void set_handle(::google::protobuf::int64 value); + + // @@protoc_insertion_point(class_scope:xla.ExecutionHandle) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::int64 handle_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_5fdata_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class GlobalDataHandle : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.GlobalDataHandle) */ { + public: + GlobalDataHandle(); + virtual ~GlobalDataHandle(); + + GlobalDataHandle(const GlobalDataHandle& from); + + inline GlobalDataHandle& operator=(const GlobalDataHandle& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + GlobalDataHandle(GlobalDataHandle&& from) noexcept + : GlobalDataHandle() { + *this = ::std::move(from); + } + + inline GlobalDataHandle& operator=(GlobalDataHandle&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const GlobalDataHandle& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const GlobalDataHandle* internal_default_instance() { + return reinterpret_cast( + &_GlobalDataHandle_default_instance_); + } + static constexpr int kIndexInFileMessages = + 9; + + void UnsafeArenaSwap(GlobalDataHandle* other); + void Swap(GlobalDataHandle* other); + friend void swap(GlobalDataHandle& a, GlobalDataHandle& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline GlobalDataHandle* New() const final { + return CreateMaybeMessage(NULL); + } + + GlobalDataHandle* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const GlobalDataHandle& from); + void MergeFrom(const GlobalDataHandle& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(GlobalDataHandle* other); + protected: + explicit GlobalDataHandle(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // int64 handle = 1; + void clear_handle(); + static const int kHandleFieldNumber = 1; + ::google::protobuf::int64 handle() const; + void set_handle(::google::protobuf::int64 value); + + // @@protoc_insertion_point(class_scope:xla.GlobalDataHandle) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::int64 handle_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_5fdata_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class DeviceHandle : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.DeviceHandle) */ { + public: + DeviceHandle(); + virtual ~DeviceHandle(); + + DeviceHandle(const DeviceHandle& from); + + inline DeviceHandle& operator=(const DeviceHandle& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + DeviceHandle(DeviceHandle&& from) noexcept + : DeviceHandle() { + *this = ::std::move(from); + } + + inline DeviceHandle& operator=(DeviceHandle&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const DeviceHandle& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const DeviceHandle* internal_default_instance() { + return reinterpret_cast( + &_DeviceHandle_default_instance_); + } + static constexpr int kIndexInFileMessages = + 10; + + void UnsafeArenaSwap(DeviceHandle* other); + void Swap(DeviceHandle* other); + friend void swap(DeviceHandle& a, DeviceHandle& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline DeviceHandle* New() const final { + return CreateMaybeMessage(NULL); + } + + DeviceHandle* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const DeviceHandle& from); + void MergeFrom(const DeviceHandle& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(DeviceHandle* other); + protected: + explicit DeviceHandle(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // int64 handle = 1; + void clear_handle(); + static const int kHandleFieldNumber = 1; + ::google::protobuf::int64 handle() const; + void set_handle(::google::protobuf::int64 value); + + // int64 device_count = 2; + void clear_device_count(); + static const int kDeviceCountFieldNumber = 2; + ::google::protobuf::int64 device_count() const; + void set_device_count(::google::protobuf::int64 value); + + // @@protoc_insertion_point(class_scope:xla.DeviceHandle) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::int64 handle_; + ::google::protobuf::int64 device_count_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_5fdata_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class ChannelHandle : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.ChannelHandle) */ { + public: + ChannelHandle(); + virtual ~ChannelHandle(); + + ChannelHandle(const ChannelHandle& from); + + inline ChannelHandle& operator=(const ChannelHandle& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + ChannelHandle(ChannelHandle&& from) noexcept + : ChannelHandle() { + *this = ::std::move(from); + } + + inline ChannelHandle& operator=(ChannelHandle&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const ChannelHandle& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const ChannelHandle* internal_default_instance() { + return reinterpret_cast( + &_ChannelHandle_default_instance_); + } + static constexpr int kIndexInFileMessages = + 11; + + void UnsafeArenaSwap(ChannelHandle* other); + void Swap(ChannelHandle* other); + friend void swap(ChannelHandle& a, ChannelHandle& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline ChannelHandle* New() const final { + return CreateMaybeMessage(NULL); + } + + ChannelHandle* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const ChannelHandle& from); + void MergeFrom(const ChannelHandle& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(ChannelHandle* other); + protected: + explicit ChannelHandle(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + typedef ChannelHandle_ChannelType ChannelType; + static const ChannelType CHANNEL_TYPE_INVALID = + ChannelHandle_ChannelType_CHANNEL_TYPE_INVALID; + static const ChannelType DEVICE_TO_DEVICE = + ChannelHandle_ChannelType_DEVICE_TO_DEVICE; + static const ChannelType DEVICE_TO_HOST = + ChannelHandle_ChannelType_DEVICE_TO_HOST; + static const ChannelType HOST_TO_DEVICE = + ChannelHandle_ChannelType_HOST_TO_DEVICE; + static inline bool ChannelType_IsValid(int value) { + return ChannelHandle_ChannelType_IsValid(value); + } + static const ChannelType ChannelType_MIN = + ChannelHandle_ChannelType_ChannelType_MIN; + static const ChannelType ChannelType_MAX = + ChannelHandle_ChannelType_ChannelType_MAX; + static const int ChannelType_ARRAYSIZE = + ChannelHandle_ChannelType_ChannelType_ARRAYSIZE; + static inline const ::google::protobuf::EnumDescriptor* + ChannelType_descriptor() { + return ChannelHandle_ChannelType_descriptor(); + } + static inline const ::std::string& ChannelType_Name(ChannelType value) { + return ChannelHandle_ChannelType_Name(value); + } + static inline bool ChannelType_Parse(const ::std::string& name, + ChannelType* value) { + return ChannelHandle_ChannelType_Parse(name, value); + } + + // accessors ------------------------------------------------------- + + // int64 handle = 1; + void clear_handle(); + static const int kHandleFieldNumber = 1; + ::google::protobuf::int64 handle() const; + void set_handle(::google::protobuf::int64 value); + + // .xla.ChannelHandle.ChannelType type = 2; + void clear_type(); + static const int kTypeFieldNumber = 2; + ::xla::ChannelHandle_ChannelType type() const; + void set_type(::xla::ChannelHandle_ChannelType value); + + // @@protoc_insertion_point(class_scope:xla.ChannelHandle) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::int64 handle_; + int type_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_5fdata_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class DeviceAssignmentProto_ComputationDevice : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.DeviceAssignmentProto.ComputationDevice) */ { + public: + DeviceAssignmentProto_ComputationDevice(); + virtual ~DeviceAssignmentProto_ComputationDevice(); + + DeviceAssignmentProto_ComputationDevice(const DeviceAssignmentProto_ComputationDevice& from); + + inline DeviceAssignmentProto_ComputationDevice& operator=(const DeviceAssignmentProto_ComputationDevice& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + DeviceAssignmentProto_ComputationDevice(DeviceAssignmentProto_ComputationDevice&& from) noexcept + : DeviceAssignmentProto_ComputationDevice() { + *this = ::std::move(from); + } + + inline DeviceAssignmentProto_ComputationDevice& operator=(DeviceAssignmentProto_ComputationDevice&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const DeviceAssignmentProto_ComputationDevice& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const DeviceAssignmentProto_ComputationDevice* internal_default_instance() { + return reinterpret_cast( + &_DeviceAssignmentProto_ComputationDevice_default_instance_); + } + static constexpr int kIndexInFileMessages = + 12; + + void UnsafeArenaSwap(DeviceAssignmentProto_ComputationDevice* other); + void Swap(DeviceAssignmentProto_ComputationDevice* other); + friend void swap(DeviceAssignmentProto_ComputationDevice& a, DeviceAssignmentProto_ComputationDevice& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline DeviceAssignmentProto_ComputationDevice* New() const final { + return CreateMaybeMessage(NULL); + } + + DeviceAssignmentProto_ComputationDevice* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const DeviceAssignmentProto_ComputationDevice& from); + void MergeFrom(const DeviceAssignmentProto_ComputationDevice& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(DeviceAssignmentProto_ComputationDevice* other); + protected: + explicit DeviceAssignmentProto_ComputationDevice(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated int32 replica_device_ids = 1; + int replica_device_ids_size() const; + void clear_replica_device_ids(); + static const int kReplicaDeviceIdsFieldNumber = 1; + ::google::protobuf::int32 replica_device_ids(int index) const; + void set_replica_device_ids(int index, ::google::protobuf::int32 value); + void add_replica_device_ids(::google::protobuf::int32 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::int32 >& + replica_device_ids() const; + ::google::protobuf::RepeatedField< ::google::protobuf::int32 >* + mutable_replica_device_ids(); + + // @@protoc_insertion_point(class_scope:xla.DeviceAssignmentProto.ComputationDevice) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedField< ::google::protobuf::int32 > replica_device_ids_; + mutable int _replica_device_ids_cached_byte_size_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_5fdata_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class DeviceAssignmentProto : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.DeviceAssignmentProto) */ { + public: + DeviceAssignmentProto(); + virtual ~DeviceAssignmentProto(); + + DeviceAssignmentProto(const DeviceAssignmentProto& from); + + inline DeviceAssignmentProto& operator=(const DeviceAssignmentProto& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + DeviceAssignmentProto(DeviceAssignmentProto&& from) noexcept + : DeviceAssignmentProto() { + *this = ::std::move(from); + } + + inline DeviceAssignmentProto& operator=(DeviceAssignmentProto&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const DeviceAssignmentProto& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const DeviceAssignmentProto* internal_default_instance() { + return reinterpret_cast( + &_DeviceAssignmentProto_default_instance_); + } + static constexpr int kIndexInFileMessages = + 13; + + void UnsafeArenaSwap(DeviceAssignmentProto* other); + void Swap(DeviceAssignmentProto* other); + friend void swap(DeviceAssignmentProto& a, DeviceAssignmentProto& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline DeviceAssignmentProto* New() const final { + return CreateMaybeMessage(NULL); + } + + DeviceAssignmentProto* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const DeviceAssignmentProto& from); + void MergeFrom(const DeviceAssignmentProto& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(DeviceAssignmentProto* other); + protected: + explicit DeviceAssignmentProto(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + typedef DeviceAssignmentProto_ComputationDevice ComputationDevice; + + // accessors ------------------------------------------------------- + + // repeated .xla.DeviceAssignmentProto.ComputationDevice computation_devices = 3; + int computation_devices_size() const; + void clear_computation_devices(); + static const int kComputationDevicesFieldNumber = 3; + ::xla::DeviceAssignmentProto_ComputationDevice* mutable_computation_devices(int index); + ::google::protobuf::RepeatedPtrField< ::xla::DeviceAssignmentProto_ComputationDevice >* + mutable_computation_devices(); + const ::xla::DeviceAssignmentProto_ComputationDevice& computation_devices(int index) const; + ::xla::DeviceAssignmentProto_ComputationDevice* add_computation_devices(); + const ::google::protobuf::RepeatedPtrField< ::xla::DeviceAssignmentProto_ComputationDevice >& + computation_devices() const; + + // int32 replica_count = 1; + void clear_replica_count(); + static const int kReplicaCountFieldNumber = 1; + ::google::protobuf::int32 replica_count() const; + void set_replica_count(::google::protobuf::int32 value); + + // int32 computation_count = 2; + void clear_computation_count(); + static const int kComputationCountFieldNumber = 2; + ::google::protobuf::int32 computation_count() const; + void set_computation_count(::google::protobuf::int32 value); + + // @@protoc_insertion_point(class_scope:xla.DeviceAssignmentProto) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedPtrField< ::xla::DeviceAssignmentProto_ComputationDevice > computation_devices_; + ::google::protobuf::int32 replica_count_; + ::google::protobuf::int32 computation_count_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_5fdata_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class LiteralProto : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.LiteralProto) */ { + public: + LiteralProto(); + virtual ~LiteralProto(); + + LiteralProto(const LiteralProto& from); + + inline LiteralProto& operator=(const LiteralProto& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + LiteralProto(LiteralProto&& from) noexcept + : LiteralProto() { + *this = ::std::move(from); + } + + inline LiteralProto& operator=(LiteralProto&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const LiteralProto& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const LiteralProto* internal_default_instance() { + return reinterpret_cast( + &_LiteralProto_default_instance_); + } + static constexpr int kIndexInFileMessages = + 14; + + void UnsafeArenaSwap(LiteralProto* other); + void Swap(LiteralProto* other); + friend void swap(LiteralProto& a, LiteralProto& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline LiteralProto* New() const final { + return CreateMaybeMessage(NULL); + } + + LiteralProto* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const LiteralProto& from); + void MergeFrom(const LiteralProto& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(LiteralProto* other); + protected: + explicit LiteralProto(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated bool preds = 2; + int preds_size() const; + void clear_preds(); + static const int kPredsFieldNumber = 2; + bool preds(int index) const; + void set_preds(int index, bool value); + void add_preds(bool value); + const ::google::protobuf::RepeatedField< bool >& + preds() const; + ::google::protobuf::RepeatedField< bool >* + mutable_preds(); + + // repeated int32 s32s = 4; + int s32s_size() const; + void clear_s32s(); + static const int kS32SFieldNumber = 4; + ::google::protobuf::int32 s32s(int index) const; + void set_s32s(int index, ::google::protobuf::int32 value); + void add_s32s(::google::protobuf::int32 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::int32 >& + s32s() const; + ::google::protobuf::RepeatedField< ::google::protobuf::int32 >* + mutable_s32s(); + + // repeated int64 s64s = 5; + int s64s_size() const; + void clear_s64s(); + static const int kS64SFieldNumber = 5; + ::google::protobuf::int64 s64s(int index) const; + void set_s64s(int index, ::google::protobuf::int64 value); + void add_s64s(::google::protobuf::int64 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& + s64s() const; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* + mutable_s64s(); + + // repeated uint32 u32s = 6; + int u32s_size() const; + void clear_u32s(); + static const int kU32SFieldNumber = 6; + ::google::protobuf::uint32 u32s(int index) const; + void set_u32s(int index, ::google::protobuf::uint32 value); + void add_u32s(::google::protobuf::uint32 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::uint32 >& + u32s() const; + ::google::protobuf::RepeatedField< ::google::protobuf::uint32 >* + mutable_u32s(); + + // repeated uint64 u64s = 7; + int u64s_size() const; + void clear_u64s(); + static const int kU64SFieldNumber = 7; + ::google::protobuf::uint64 u64s(int index) const; + void set_u64s(int index, ::google::protobuf::uint64 value); + void add_u64s(::google::protobuf::uint64 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::uint64 >& + u64s() const; + ::google::protobuf::RepeatedField< ::google::protobuf::uint64 >* + mutable_u64s(); + + // repeated float f32s = 8; + int f32s_size() const; + void clear_f32s(); + static const int kF32SFieldNumber = 8; + float f32s(int index) const; + void set_f32s(int index, float value); + void add_f32s(float value); + const ::google::protobuf::RepeatedField< float >& + f32s() const; + ::google::protobuf::RepeatedField< float >* + mutable_f32s(); + + // repeated double f64s = 9; + int f64s_size() const; + void clear_f64s(); + static const int kF64SFieldNumber = 9; + double f64s(int index) const; + void set_f64s(int index, double value); + void add_f64s(double value); + const ::google::protobuf::RepeatedField< double >& + f64s() const; + ::google::protobuf::RepeatedField< double >* + mutable_f64s(); + + // repeated .xla.LiteralProto tuple_literals = 10; + int tuple_literals_size() const; + void clear_tuple_literals(); + static const int kTupleLiteralsFieldNumber = 10; + ::xla::LiteralProto* mutable_tuple_literals(int index); + ::google::protobuf::RepeatedPtrField< ::xla::LiteralProto >* + mutable_tuple_literals(); + const ::xla::LiteralProto& tuple_literals(int index) const; + ::xla::LiteralProto* add_tuple_literals(); + const ::google::protobuf::RepeatedPtrField< ::xla::LiteralProto >& + tuple_literals() const; + + // repeated float c64s = 12; + int c64s_size() const; + void clear_c64s(); + static const int kC64SFieldNumber = 12; + float c64s(int index) const; + void set_c64s(int index, float value); + void add_c64s(float value); + const ::google::protobuf::RepeatedField< float >& + c64s() const; + ::google::protobuf::RepeatedField< float >* + mutable_c64s(); + + // repeated int64 sparse_indices = 14; + int sparse_indices_size() const; + void clear_sparse_indices(); + static const int kSparseIndicesFieldNumber = 14; + ::google::protobuf::int64 sparse_indices(int index) const; + void set_sparse_indices(int index, ::google::protobuf::int64 value); + void add_sparse_indices(::google::protobuf::int64 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& + sparse_indices() const; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* + mutable_sparse_indices(); + + // bytes u8s = 3; + void clear_u8s(); + static const int kU8SFieldNumber = 3; + const ::std::string& u8s() const; + void set_u8s(const ::std::string& value); + #if LANG_CXX11 + void set_u8s(::std::string&& value); + #endif + void set_u8s(const char* value); + void set_u8s(const void* value, size_t size); + ::std::string* mutable_u8s(); + ::std::string* release_u8s(); + void set_allocated_u8s(::std::string* u8s); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_u8s(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_u8s( + ::std::string* u8s); + + // bytes f16s = 11; + void clear_f16s(); + static const int kF16SFieldNumber = 11; + const ::std::string& f16s() const; + void set_f16s(const ::std::string& value); + #if LANG_CXX11 + void set_f16s(::std::string&& value); + #endif + void set_f16s(const char* value); + void set_f16s(const void* value, size_t size); + ::std::string* mutable_f16s(); + ::std::string* release_f16s(); + void set_allocated_f16s(::std::string* f16s); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_f16s(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_f16s( + ::std::string* f16s); + + // bytes bf16s = 13; + void clear_bf16s(); + static const int kBf16SFieldNumber = 13; + const ::std::string& bf16s() const; + void set_bf16s(const ::std::string& value); + #if LANG_CXX11 + void set_bf16s(::std::string&& value); + #endif + void set_bf16s(const char* value); + void set_bf16s(const void* value, size_t size); + ::std::string* mutable_bf16s(); + ::std::string* release_bf16s(); + void set_allocated_bf16s(::std::string* bf16s); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_bf16s(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_bf16s( + ::std::string* bf16s); + + // bytes s8s = 15; + void clear_s8s(); + static const int kS8SFieldNumber = 15; + const ::std::string& s8s() const; + void set_s8s(const ::std::string& value); + #if LANG_CXX11 + void set_s8s(::std::string&& value); + #endif + void set_s8s(const char* value); + void set_s8s(const void* value, size_t size); + ::std::string* mutable_s8s(); + ::std::string* release_s8s(); + void set_allocated_s8s(::std::string* s8s); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_s8s(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_s8s( + ::std::string* s8s); + + // .xla.Shape shape = 1; + bool has_shape() const; + void clear_shape(); + static const int kShapeFieldNumber = 1; + private: + const ::xla::Shape& _internal_shape() const; + public: + const ::xla::Shape& shape() const; + ::xla::Shape* release_shape(); + ::xla::Shape* mutable_shape(); + void set_allocated_shape(::xla::Shape* shape); + void unsafe_arena_set_allocated_shape( + ::xla::Shape* shape); + ::xla::Shape* unsafe_arena_release_shape(); + + // @@protoc_insertion_point(class_scope:xla.LiteralProto) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedField< bool > preds_; + mutable int _preds_cached_byte_size_; + ::google::protobuf::RepeatedField< ::google::protobuf::int32 > s32s_; + mutable int _s32s_cached_byte_size_; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 > s64s_; + mutable int _s64s_cached_byte_size_; + ::google::protobuf::RepeatedField< ::google::protobuf::uint32 > u32s_; + mutable int _u32s_cached_byte_size_; + ::google::protobuf::RepeatedField< ::google::protobuf::uint64 > u64s_; + mutable int _u64s_cached_byte_size_; + ::google::protobuf::RepeatedField< float > f32s_; + mutable int _f32s_cached_byte_size_; + ::google::protobuf::RepeatedField< double > f64s_; + mutable int _f64s_cached_byte_size_; + ::google::protobuf::RepeatedPtrField< ::xla::LiteralProto > tuple_literals_; + ::google::protobuf::RepeatedField< float > c64s_; + mutable int _c64s_cached_byte_size_; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 > sparse_indices_; + mutable int _sparse_indices_cached_byte_size_; + ::google::protobuf::internal::ArenaStringPtr u8s_; + ::google::protobuf::internal::ArenaStringPtr f16s_; + ::google::protobuf::internal::ArenaStringPtr bf16s_; + ::google::protobuf::internal::ArenaStringPtr s8s_; + ::xla::Shape* shape_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_5fdata_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class WindowDimension : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.WindowDimension) */ { + public: + WindowDimension(); + virtual ~WindowDimension(); + + WindowDimension(const WindowDimension& from); + + inline WindowDimension& operator=(const WindowDimension& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + WindowDimension(WindowDimension&& from) noexcept + : WindowDimension() { + *this = ::std::move(from); + } + + inline WindowDimension& operator=(WindowDimension&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const WindowDimension& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const WindowDimension* internal_default_instance() { + return reinterpret_cast( + &_WindowDimension_default_instance_); + } + static constexpr int kIndexInFileMessages = + 15; + + void UnsafeArenaSwap(WindowDimension* other); + void Swap(WindowDimension* other); + friend void swap(WindowDimension& a, WindowDimension& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline WindowDimension* New() const final { + return CreateMaybeMessage(NULL); + } + + WindowDimension* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const WindowDimension& from); + void MergeFrom(const WindowDimension& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(WindowDimension* other); + protected: + explicit WindowDimension(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // int64 size = 1; + void clear_size(); + static const int kSizeFieldNumber = 1; + ::google::protobuf::int64 size() const; + void set_size(::google::protobuf::int64 value); + + // int64 stride = 2; + void clear_stride(); + static const int kStrideFieldNumber = 2; + ::google::protobuf::int64 stride() const; + void set_stride(::google::protobuf::int64 value); + + // int64 padding_low = 3; + void clear_padding_low(); + static const int kPaddingLowFieldNumber = 3; + ::google::protobuf::int64 padding_low() const; + void set_padding_low(::google::protobuf::int64 value); + + // int64 padding_high = 4; + void clear_padding_high(); + static const int kPaddingHighFieldNumber = 4; + ::google::protobuf::int64 padding_high() const; + void set_padding_high(::google::protobuf::int64 value); + + // int64 window_dilation = 5; + void clear_window_dilation(); + static const int kWindowDilationFieldNumber = 5; + ::google::protobuf::int64 window_dilation() const; + void set_window_dilation(::google::protobuf::int64 value); + + // int64 base_dilation = 6; + void clear_base_dilation(); + static const int kBaseDilationFieldNumber = 6; + ::google::protobuf::int64 base_dilation() const; + void set_base_dilation(::google::protobuf::int64 value); + + // bool window_reversal = 7; + void clear_window_reversal(); + static const int kWindowReversalFieldNumber = 7; + bool window_reversal() const; + void set_window_reversal(bool value); + + // @@protoc_insertion_point(class_scope:xla.WindowDimension) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::int64 size_; + ::google::protobuf::int64 stride_; + ::google::protobuf::int64 padding_low_; + ::google::protobuf::int64 padding_high_; + ::google::protobuf::int64 window_dilation_; + ::google::protobuf::int64 base_dilation_; + bool window_reversal_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_5fdata_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class Window : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.Window) */ { + public: + Window(); + virtual ~Window(); + + Window(const Window& from); + + inline Window& operator=(const Window& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + Window(Window&& from) noexcept + : Window() { + *this = ::std::move(from); + } + + inline Window& operator=(Window&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const Window& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const Window* internal_default_instance() { + return reinterpret_cast( + &_Window_default_instance_); + } + static constexpr int kIndexInFileMessages = + 16; + + void UnsafeArenaSwap(Window* other); + void Swap(Window* other); + friend void swap(Window& a, Window& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline Window* New() const final { + return CreateMaybeMessage(NULL); + } + + Window* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const Window& from); + void MergeFrom(const Window& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(Window* other); + protected: + explicit Window(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated .xla.WindowDimension dimensions = 1; + int dimensions_size() const; + void clear_dimensions(); + static const int kDimensionsFieldNumber = 1; + ::xla::WindowDimension* mutable_dimensions(int index); + ::google::protobuf::RepeatedPtrField< ::xla::WindowDimension >* + mutable_dimensions(); + const ::xla::WindowDimension& dimensions(int index) const; + ::xla::WindowDimension* add_dimensions(); + const ::google::protobuf::RepeatedPtrField< ::xla::WindowDimension >& + dimensions() const; + + // @@protoc_insertion_point(class_scope:xla.Window) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedPtrField< ::xla::WindowDimension > dimensions_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_5fdata_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class GatherDimensionNumbers : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.GatherDimensionNumbers) */ { + public: + GatherDimensionNumbers(); + virtual ~GatherDimensionNumbers(); + + GatherDimensionNumbers(const GatherDimensionNumbers& from); + + inline GatherDimensionNumbers& operator=(const GatherDimensionNumbers& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + GatherDimensionNumbers(GatherDimensionNumbers&& from) noexcept + : GatherDimensionNumbers() { + *this = ::std::move(from); + } + + inline GatherDimensionNumbers& operator=(GatherDimensionNumbers&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const GatherDimensionNumbers& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const GatherDimensionNumbers* internal_default_instance() { + return reinterpret_cast( + &_GatherDimensionNumbers_default_instance_); + } + static constexpr int kIndexInFileMessages = + 17; + + void UnsafeArenaSwap(GatherDimensionNumbers* other); + void Swap(GatherDimensionNumbers* other); + friend void swap(GatherDimensionNumbers& a, GatherDimensionNumbers& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline GatherDimensionNumbers* New() const final { + return CreateMaybeMessage(NULL); + } + + GatherDimensionNumbers* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const GatherDimensionNumbers& from); + void MergeFrom(const GatherDimensionNumbers& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(GatherDimensionNumbers* other); + protected: + explicit GatherDimensionNumbers(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated int64 offset_dims = 1; + int offset_dims_size() const; + void clear_offset_dims(); + static const int kOffsetDimsFieldNumber = 1; + ::google::protobuf::int64 offset_dims(int index) const; + void set_offset_dims(int index, ::google::protobuf::int64 value); + void add_offset_dims(::google::protobuf::int64 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& + offset_dims() const; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* + mutable_offset_dims(); + + // repeated int64 collapsed_slice_dims = 2; + int collapsed_slice_dims_size() const; + void clear_collapsed_slice_dims(); + static const int kCollapsedSliceDimsFieldNumber = 2; + ::google::protobuf::int64 collapsed_slice_dims(int index) const; + void set_collapsed_slice_dims(int index, ::google::protobuf::int64 value); + void add_collapsed_slice_dims(::google::protobuf::int64 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& + collapsed_slice_dims() const; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* + mutable_collapsed_slice_dims(); + + // repeated int64 start_index_map = 3; + int start_index_map_size() const; + void clear_start_index_map(); + static const int kStartIndexMapFieldNumber = 3; + ::google::protobuf::int64 start_index_map(int index) const; + void set_start_index_map(int index, ::google::protobuf::int64 value); + void add_start_index_map(::google::protobuf::int64 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& + start_index_map() const; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* + mutable_start_index_map(); + + // int64 index_vector_dim = 4; + void clear_index_vector_dim(); + static const int kIndexVectorDimFieldNumber = 4; + ::google::protobuf::int64 index_vector_dim() const; + void set_index_vector_dim(::google::protobuf::int64 value); + + // @@protoc_insertion_point(class_scope:xla.GatherDimensionNumbers) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 > offset_dims_; + mutable int _offset_dims_cached_byte_size_; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 > collapsed_slice_dims_; + mutable int _collapsed_slice_dims_cached_byte_size_; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 > start_index_map_; + mutable int _start_index_map_cached_byte_size_; + ::google::protobuf::int64 index_vector_dim_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_5fdata_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class ScatterDimensionNumbers : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.ScatterDimensionNumbers) */ { + public: + ScatterDimensionNumbers(); + virtual ~ScatterDimensionNumbers(); + + ScatterDimensionNumbers(const ScatterDimensionNumbers& from); + + inline ScatterDimensionNumbers& operator=(const ScatterDimensionNumbers& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + ScatterDimensionNumbers(ScatterDimensionNumbers&& from) noexcept + : ScatterDimensionNumbers() { + *this = ::std::move(from); + } + + inline ScatterDimensionNumbers& operator=(ScatterDimensionNumbers&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const ScatterDimensionNumbers& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const ScatterDimensionNumbers* internal_default_instance() { + return reinterpret_cast( + &_ScatterDimensionNumbers_default_instance_); + } + static constexpr int kIndexInFileMessages = + 18; + + void UnsafeArenaSwap(ScatterDimensionNumbers* other); + void Swap(ScatterDimensionNumbers* other); + friend void swap(ScatterDimensionNumbers& a, ScatterDimensionNumbers& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline ScatterDimensionNumbers* New() const final { + return CreateMaybeMessage(NULL); + } + + ScatterDimensionNumbers* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const ScatterDimensionNumbers& from); + void MergeFrom(const ScatterDimensionNumbers& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(ScatterDimensionNumbers* other); + protected: + explicit ScatterDimensionNumbers(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated int64 update_window_dims = 1; + int update_window_dims_size() const; + void clear_update_window_dims(); + static const int kUpdateWindowDimsFieldNumber = 1; + ::google::protobuf::int64 update_window_dims(int index) const; + void set_update_window_dims(int index, ::google::protobuf::int64 value); + void add_update_window_dims(::google::protobuf::int64 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& + update_window_dims() const; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* + mutable_update_window_dims(); + + // repeated int64 inserted_window_dims = 2; + int inserted_window_dims_size() const; + void clear_inserted_window_dims(); + static const int kInsertedWindowDimsFieldNumber = 2; + ::google::protobuf::int64 inserted_window_dims(int index) const; + void set_inserted_window_dims(int index, ::google::protobuf::int64 value); + void add_inserted_window_dims(::google::protobuf::int64 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& + inserted_window_dims() const; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* + mutable_inserted_window_dims(); + + // repeated int64 scatter_dims_to_operand_dims = 3; + int scatter_dims_to_operand_dims_size() const; + void clear_scatter_dims_to_operand_dims(); + static const int kScatterDimsToOperandDimsFieldNumber = 3; + ::google::protobuf::int64 scatter_dims_to_operand_dims(int index) const; + void set_scatter_dims_to_operand_dims(int index, ::google::protobuf::int64 value); + void add_scatter_dims_to_operand_dims(::google::protobuf::int64 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& + scatter_dims_to_operand_dims() const; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* + mutable_scatter_dims_to_operand_dims(); + + // int64 index_vector_dim = 4; + void clear_index_vector_dim(); + static const int kIndexVectorDimFieldNumber = 4; + ::google::protobuf::int64 index_vector_dim() const; + void set_index_vector_dim(::google::protobuf::int64 value); + + // @@protoc_insertion_point(class_scope:xla.ScatterDimensionNumbers) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 > update_window_dims_; + mutable int _update_window_dims_cached_byte_size_; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 > inserted_window_dims_; + mutable int _inserted_window_dims_cached_byte_size_; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 > scatter_dims_to_operand_dims_; + mutable int _scatter_dims_to_operand_dims_cached_byte_size_; + ::google::protobuf::int64 index_vector_dim_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_5fdata_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class ConvolutionDimensionNumbers : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.ConvolutionDimensionNumbers) */ { + public: + ConvolutionDimensionNumbers(); + virtual ~ConvolutionDimensionNumbers(); + + ConvolutionDimensionNumbers(const ConvolutionDimensionNumbers& from); + + inline ConvolutionDimensionNumbers& operator=(const ConvolutionDimensionNumbers& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + ConvolutionDimensionNumbers(ConvolutionDimensionNumbers&& from) noexcept + : ConvolutionDimensionNumbers() { + *this = ::std::move(from); + } + + inline ConvolutionDimensionNumbers& operator=(ConvolutionDimensionNumbers&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const ConvolutionDimensionNumbers& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const ConvolutionDimensionNumbers* internal_default_instance() { + return reinterpret_cast( + &_ConvolutionDimensionNumbers_default_instance_); + } + static constexpr int kIndexInFileMessages = + 19; + + void UnsafeArenaSwap(ConvolutionDimensionNumbers* other); + void Swap(ConvolutionDimensionNumbers* other); + friend void swap(ConvolutionDimensionNumbers& a, ConvolutionDimensionNumbers& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline ConvolutionDimensionNumbers* New() const final { + return CreateMaybeMessage(NULL); + } + + ConvolutionDimensionNumbers* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const ConvolutionDimensionNumbers& from); + void MergeFrom(const ConvolutionDimensionNumbers& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(ConvolutionDimensionNumbers* other); + protected: + explicit ConvolutionDimensionNumbers(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated int64 kernel_spatial_dimensions = 6; + int kernel_spatial_dimensions_size() const; + void clear_kernel_spatial_dimensions(); + static const int kKernelSpatialDimensionsFieldNumber = 6; + ::google::protobuf::int64 kernel_spatial_dimensions(int index) const; + void set_kernel_spatial_dimensions(int index, ::google::protobuf::int64 value); + void add_kernel_spatial_dimensions(::google::protobuf::int64 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& + kernel_spatial_dimensions() const; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* + mutable_kernel_spatial_dimensions(); + + // repeated int64 input_spatial_dimensions = 11; + int input_spatial_dimensions_size() const; + void clear_input_spatial_dimensions(); + static const int kInputSpatialDimensionsFieldNumber = 11; + ::google::protobuf::int64 input_spatial_dimensions(int index) const; + void set_input_spatial_dimensions(int index, ::google::protobuf::int64 value); + void add_input_spatial_dimensions(::google::protobuf::int64 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& + input_spatial_dimensions() const; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* + mutable_input_spatial_dimensions(); + + // repeated int64 output_spatial_dimensions = 12; + int output_spatial_dimensions_size() const; + void clear_output_spatial_dimensions(); + static const int kOutputSpatialDimensionsFieldNumber = 12; + ::google::protobuf::int64 output_spatial_dimensions(int index) const; + void set_output_spatial_dimensions(int index, ::google::protobuf::int64 value); + void add_output_spatial_dimensions(::google::protobuf::int64 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& + output_spatial_dimensions() const; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* + mutable_output_spatial_dimensions(); + + // int64 kernel_input_feature_dimension = 3; + void clear_kernel_input_feature_dimension(); + static const int kKernelInputFeatureDimensionFieldNumber = 3; + ::google::protobuf::int64 kernel_input_feature_dimension() const; + void set_kernel_input_feature_dimension(::google::protobuf::int64 value); + + // int64 kernel_output_feature_dimension = 4; + void clear_kernel_output_feature_dimension(); + static const int kKernelOutputFeatureDimensionFieldNumber = 4; + ::google::protobuf::int64 kernel_output_feature_dimension() const; + void set_kernel_output_feature_dimension(::google::protobuf::int64 value); + + // int64 input_batch_dimension = 7; + void clear_input_batch_dimension(); + static const int kInputBatchDimensionFieldNumber = 7; + ::google::protobuf::int64 input_batch_dimension() const; + void set_input_batch_dimension(::google::protobuf::int64 value); + + // int64 input_feature_dimension = 8; + void clear_input_feature_dimension(); + static const int kInputFeatureDimensionFieldNumber = 8; + ::google::protobuf::int64 input_feature_dimension() const; + void set_input_feature_dimension(::google::protobuf::int64 value); + + // int64 output_batch_dimension = 9; + void clear_output_batch_dimension(); + static const int kOutputBatchDimensionFieldNumber = 9; + ::google::protobuf::int64 output_batch_dimension() const; + void set_output_batch_dimension(::google::protobuf::int64 value); + + // int64 output_feature_dimension = 10; + void clear_output_feature_dimension(); + static const int kOutputFeatureDimensionFieldNumber = 10; + ::google::protobuf::int64 output_feature_dimension() const; + void set_output_feature_dimension(::google::protobuf::int64 value); + + // @@protoc_insertion_point(class_scope:xla.ConvolutionDimensionNumbers) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 > kernel_spatial_dimensions_; + mutable int _kernel_spatial_dimensions_cached_byte_size_; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 > input_spatial_dimensions_; + mutable int _input_spatial_dimensions_cached_byte_size_; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 > output_spatial_dimensions_; + mutable int _output_spatial_dimensions_cached_byte_size_; + ::google::protobuf::int64 kernel_input_feature_dimension_; + ::google::protobuf::int64 kernel_output_feature_dimension_; + ::google::protobuf::int64 input_batch_dimension_; + ::google::protobuf::int64 input_feature_dimension_; + ::google::protobuf::int64 output_batch_dimension_; + ::google::protobuf::int64 output_feature_dimension_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_5fdata_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class DotDimensionNumbers : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.DotDimensionNumbers) */ { + public: + DotDimensionNumbers(); + virtual ~DotDimensionNumbers(); + + DotDimensionNumbers(const DotDimensionNumbers& from); + + inline DotDimensionNumbers& operator=(const DotDimensionNumbers& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + DotDimensionNumbers(DotDimensionNumbers&& from) noexcept + : DotDimensionNumbers() { + *this = ::std::move(from); + } + + inline DotDimensionNumbers& operator=(DotDimensionNumbers&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const DotDimensionNumbers& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const DotDimensionNumbers* internal_default_instance() { + return reinterpret_cast( + &_DotDimensionNumbers_default_instance_); + } + static constexpr int kIndexInFileMessages = + 20; + + void UnsafeArenaSwap(DotDimensionNumbers* other); + void Swap(DotDimensionNumbers* other); + friend void swap(DotDimensionNumbers& a, DotDimensionNumbers& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline DotDimensionNumbers* New() const final { + return CreateMaybeMessage(NULL); + } + + DotDimensionNumbers* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const DotDimensionNumbers& from); + void MergeFrom(const DotDimensionNumbers& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(DotDimensionNumbers* other); + protected: + explicit DotDimensionNumbers(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated int64 lhs_contracting_dimensions = 1; + int lhs_contracting_dimensions_size() const; + void clear_lhs_contracting_dimensions(); + static const int kLhsContractingDimensionsFieldNumber = 1; + ::google::protobuf::int64 lhs_contracting_dimensions(int index) const; + void set_lhs_contracting_dimensions(int index, ::google::protobuf::int64 value); + void add_lhs_contracting_dimensions(::google::protobuf::int64 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& + lhs_contracting_dimensions() const; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* + mutable_lhs_contracting_dimensions(); + + // repeated int64 rhs_contracting_dimensions = 2; + int rhs_contracting_dimensions_size() const; + void clear_rhs_contracting_dimensions(); + static const int kRhsContractingDimensionsFieldNumber = 2; + ::google::protobuf::int64 rhs_contracting_dimensions(int index) const; + void set_rhs_contracting_dimensions(int index, ::google::protobuf::int64 value); + void add_rhs_contracting_dimensions(::google::protobuf::int64 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& + rhs_contracting_dimensions() const; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* + mutable_rhs_contracting_dimensions(); + + // repeated int64 lhs_batch_dimensions = 3; + int lhs_batch_dimensions_size() const; + void clear_lhs_batch_dimensions(); + static const int kLhsBatchDimensionsFieldNumber = 3; + ::google::protobuf::int64 lhs_batch_dimensions(int index) const; + void set_lhs_batch_dimensions(int index, ::google::protobuf::int64 value); + void add_lhs_batch_dimensions(::google::protobuf::int64 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& + lhs_batch_dimensions() const; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* + mutable_lhs_batch_dimensions(); + + // repeated int64 rhs_batch_dimensions = 4; + int rhs_batch_dimensions_size() const; + void clear_rhs_batch_dimensions(); + static const int kRhsBatchDimensionsFieldNumber = 4; + ::google::protobuf::int64 rhs_batch_dimensions(int index) const; + void set_rhs_batch_dimensions(int index, ::google::protobuf::int64 value); + void add_rhs_batch_dimensions(::google::protobuf::int64 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& + rhs_batch_dimensions() const; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* + mutable_rhs_batch_dimensions(); + + // @@protoc_insertion_point(class_scope:xla.DotDimensionNumbers) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 > lhs_contracting_dimensions_; + mutable int _lhs_contracting_dimensions_cached_byte_size_; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 > rhs_contracting_dimensions_; + mutable int _rhs_contracting_dimensions_cached_byte_size_; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 > lhs_batch_dimensions_; + mutable int _lhs_batch_dimensions_cached_byte_size_; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 > rhs_batch_dimensions_; + mutable int _rhs_batch_dimensions_cached_byte_size_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_5fdata_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class OpSharding : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.OpSharding) */ { + public: + OpSharding(); + virtual ~OpSharding(); + + OpSharding(const OpSharding& from); + + inline OpSharding& operator=(const OpSharding& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + OpSharding(OpSharding&& from) noexcept + : OpSharding() { + *this = ::std::move(from); + } + + inline OpSharding& operator=(OpSharding&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const OpSharding& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const OpSharding* internal_default_instance() { + return reinterpret_cast( + &_OpSharding_default_instance_); + } + static constexpr int kIndexInFileMessages = + 21; + + void UnsafeArenaSwap(OpSharding* other); + void Swap(OpSharding* other); + friend void swap(OpSharding& a, OpSharding& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline OpSharding* New() const final { + return CreateMaybeMessage(NULL); + } + + OpSharding* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const OpSharding& from); + void MergeFrom(const OpSharding& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(OpSharding* other); + protected: + explicit OpSharding(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + typedef OpSharding_Type Type; + static const Type REPLICATED = + OpSharding_Type_REPLICATED; + static const Type MAXIMAL = + OpSharding_Type_MAXIMAL; + static const Type TUPLE = + OpSharding_Type_TUPLE; + static const Type OTHER = + OpSharding_Type_OTHER; + static inline bool Type_IsValid(int value) { + return OpSharding_Type_IsValid(value); + } + static const Type Type_MIN = + OpSharding_Type_Type_MIN; + static const Type Type_MAX = + OpSharding_Type_Type_MAX; + static const int Type_ARRAYSIZE = + OpSharding_Type_Type_ARRAYSIZE; + static inline const ::google::protobuf::EnumDescriptor* + Type_descriptor() { + return OpSharding_Type_descriptor(); + } + static inline const ::std::string& Type_Name(Type value) { + return OpSharding_Type_Name(value); + } + static inline bool Type_Parse(const ::std::string& name, + Type* value) { + return OpSharding_Type_Parse(name, value); + } + + // accessors ------------------------------------------------------- + + // repeated int64 tile_assignment_dimensions = 3; + int tile_assignment_dimensions_size() const; + void clear_tile_assignment_dimensions(); + static const int kTileAssignmentDimensionsFieldNumber = 3; + ::google::protobuf::int64 tile_assignment_dimensions(int index) const; + void set_tile_assignment_dimensions(int index, ::google::protobuf::int64 value); + void add_tile_assignment_dimensions(::google::protobuf::int64 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& + tile_assignment_dimensions() const; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* + mutable_tile_assignment_dimensions(); + + // repeated int64 tile_assignment_devices = 4; + int tile_assignment_devices_size() const; + void clear_tile_assignment_devices(); + static const int kTileAssignmentDevicesFieldNumber = 4; + ::google::protobuf::int64 tile_assignment_devices(int index) const; + void set_tile_assignment_devices(int index, ::google::protobuf::int64 value); + void add_tile_assignment_devices(::google::protobuf::int64 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& + tile_assignment_devices() const; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* + mutable_tile_assignment_devices(); + + // repeated .xla.OpSharding tuple_shardings = 5; + int tuple_shardings_size() const; + void clear_tuple_shardings(); + static const int kTupleShardingsFieldNumber = 5; + ::xla::OpSharding* mutable_tuple_shardings(int index); + ::google::protobuf::RepeatedPtrField< ::xla::OpSharding >* + mutable_tuple_shardings(); + const ::xla::OpSharding& tuple_shardings(int index) const; + ::xla::OpSharding* add_tuple_shardings(); + const ::google::protobuf::RepeatedPtrField< ::xla::OpSharding >& + tuple_shardings() const; + + // .xla.Shape tile_shape = 2; + bool has_tile_shape() const; + void clear_tile_shape(); + static const int kTileShapeFieldNumber = 2; + private: + const ::xla::Shape& _internal_tile_shape() const; + public: + const ::xla::Shape& tile_shape() const; + ::xla::Shape* release_tile_shape(); + ::xla::Shape* mutable_tile_shape(); + void set_allocated_tile_shape(::xla::Shape* tile_shape); + void unsafe_arena_set_allocated_tile_shape( + ::xla::Shape* tile_shape); + ::xla::Shape* unsafe_arena_release_tile_shape(); + + // .xla.OpSharding.Type type = 1; + void clear_type(); + static const int kTypeFieldNumber = 1; + ::xla::OpSharding_Type type() const; + void set_type(::xla::OpSharding_Type value); + + // @@protoc_insertion_point(class_scope:xla.OpSharding) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 > tile_assignment_dimensions_; + mutable int _tile_assignment_dimensions_cached_byte_size_; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 > tile_assignment_devices_; + mutable int _tile_assignment_devices_cached_byte_size_; + ::google::protobuf::RepeatedPtrField< ::xla::OpSharding > tuple_shardings_; + ::xla::Shape* tile_shape_; + int type_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_5fdata_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class ReplicaGroup : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.ReplicaGroup) */ { + public: + ReplicaGroup(); + virtual ~ReplicaGroup(); + + ReplicaGroup(const ReplicaGroup& from); + + inline ReplicaGroup& operator=(const ReplicaGroup& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + ReplicaGroup(ReplicaGroup&& from) noexcept + : ReplicaGroup() { + *this = ::std::move(from); + } + + inline ReplicaGroup& operator=(ReplicaGroup&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const ReplicaGroup& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const ReplicaGroup* internal_default_instance() { + return reinterpret_cast( + &_ReplicaGroup_default_instance_); + } + static constexpr int kIndexInFileMessages = + 22; + + void UnsafeArenaSwap(ReplicaGroup* other); + void Swap(ReplicaGroup* other); + friend void swap(ReplicaGroup& a, ReplicaGroup& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline ReplicaGroup* New() const final { + return CreateMaybeMessage(NULL); + } + + ReplicaGroup* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const ReplicaGroup& from); + void MergeFrom(const ReplicaGroup& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(ReplicaGroup* other); + protected: + explicit ReplicaGroup(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated int64 replica_ids = 1; + int replica_ids_size() const; + void clear_replica_ids(); + static const int kReplicaIdsFieldNumber = 1; + ::google::protobuf::int64 replica_ids(int index) const; + void set_replica_ids(int index, ::google::protobuf::int64 value); + void add_replica_ids(::google::protobuf::int64 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& + replica_ids() const; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* + mutable_replica_ids(); + + // @@protoc_insertion_point(class_scope:xla.ReplicaGroup) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 > replica_ids_; + mutable int _replica_ids_cached_byte_size_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_5fdata_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class SourceTarget : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.SourceTarget) */ { + public: + SourceTarget(); + virtual ~SourceTarget(); + + SourceTarget(const SourceTarget& from); + + inline SourceTarget& operator=(const SourceTarget& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + SourceTarget(SourceTarget&& from) noexcept + : SourceTarget() { + *this = ::std::move(from); + } + + inline SourceTarget& operator=(SourceTarget&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const SourceTarget& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const SourceTarget* internal_default_instance() { + return reinterpret_cast( + &_SourceTarget_default_instance_); + } + static constexpr int kIndexInFileMessages = + 23; + + void UnsafeArenaSwap(SourceTarget* other); + void Swap(SourceTarget* other); + friend void swap(SourceTarget& a, SourceTarget& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline SourceTarget* New() const final { + return CreateMaybeMessage(NULL); + } + + SourceTarget* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const SourceTarget& from); + void MergeFrom(const SourceTarget& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(SourceTarget* other); + protected: + explicit SourceTarget(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // int64 source = 1; + void clear_source(); + static const int kSourceFieldNumber = 1; + ::google::protobuf::int64 source() const; + void set_source(::google::protobuf::int64 value); + + // int64 target = 2; + void clear_target(); + static const int kTargetFieldNumber = 2; + ::google::protobuf::int64 target() const; + void set_target(::google::protobuf::int64 value); + + // @@protoc_insertion_point(class_scope:xla.SourceTarget) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::int64 source_; + ::google::protobuf::int64 target_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_5fdata_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class PrecisionConfig : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:xla.PrecisionConfig) */ { + public: + PrecisionConfig(); + virtual ~PrecisionConfig(); + + PrecisionConfig(const PrecisionConfig& from); + + inline PrecisionConfig& operator=(const PrecisionConfig& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + PrecisionConfig(PrecisionConfig&& from) noexcept + : PrecisionConfig() { + *this = ::std::move(from); + } + + inline PrecisionConfig& operator=(PrecisionConfig&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const PrecisionConfig& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const PrecisionConfig* internal_default_instance() { + return reinterpret_cast( + &_PrecisionConfig_default_instance_); + } + static constexpr int kIndexInFileMessages = + 24; + + void UnsafeArenaSwap(PrecisionConfig* other); + void Swap(PrecisionConfig* other); + friend void swap(PrecisionConfig& a, PrecisionConfig& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline PrecisionConfig* New() const final { + return CreateMaybeMessage(NULL); + } + + PrecisionConfig* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const PrecisionConfig& from); + void MergeFrom(const PrecisionConfig& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(PrecisionConfig* other); + protected: + explicit PrecisionConfig(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + typedef PrecisionConfig_Precision Precision; + static const Precision DEFAULT = + PrecisionConfig_Precision_DEFAULT; + static const Precision HIGH = + PrecisionConfig_Precision_HIGH; + static const Precision HIGHEST = + PrecisionConfig_Precision_HIGHEST; + static inline bool Precision_IsValid(int value) { + return PrecisionConfig_Precision_IsValid(value); + } + static const Precision Precision_MIN = + PrecisionConfig_Precision_Precision_MIN; + static const Precision Precision_MAX = + PrecisionConfig_Precision_Precision_MAX; + static const int Precision_ARRAYSIZE = + PrecisionConfig_Precision_Precision_ARRAYSIZE; + static inline const ::google::protobuf::EnumDescriptor* + Precision_descriptor() { + return PrecisionConfig_Precision_descriptor(); + } + static inline const ::std::string& Precision_Name(Precision value) { + return PrecisionConfig_Precision_Name(value); + } + static inline bool Precision_Parse(const ::std::string& name, + Precision* value) { + return PrecisionConfig_Precision_Parse(name, value); + } + + // accessors ------------------------------------------------------- + + // repeated .xla.PrecisionConfig.Precision operand_precision = 1; + int operand_precision_size() const; + void clear_operand_precision(); + static const int kOperandPrecisionFieldNumber = 1; + ::xla::PrecisionConfig_Precision operand_precision(int index) const; + void set_operand_precision(int index, ::xla::PrecisionConfig_Precision value); + void add_operand_precision(::xla::PrecisionConfig_Precision value); + const ::google::protobuf::RepeatedField& operand_precision() const; + ::google::protobuf::RepeatedField* mutable_operand_precision(); + + // @@protoc_insertion_point(class_scope:xla.PrecisionConfig) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedField operand_precision_; + mutable int _operand_precision_cached_byte_size_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcompiler_2fxla_2fxla_5fdata_2eproto::TableStruct; +}; +// =================================================================== + + +// =================================================================== + +#ifdef __GNUC__ + #pragma GCC diagnostic push + #pragma GCC diagnostic ignored "-Wstrict-aliasing" +#endif // __GNUC__ +// PaddingConfig_PaddingConfigDimension + +// int64 edge_padding_low = 1; +inline void PaddingConfig_PaddingConfigDimension::clear_edge_padding_low() { + edge_padding_low_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 PaddingConfig_PaddingConfigDimension::edge_padding_low() const { + // @@protoc_insertion_point(field_get:xla.PaddingConfig.PaddingConfigDimension.edge_padding_low) + return edge_padding_low_; +} +inline void PaddingConfig_PaddingConfigDimension::set_edge_padding_low(::google::protobuf::int64 value) { + + edge_padding_low_ = value; + // @@protoc_insertion_point(field_set:xla.PaddingConfig.PaddingConfigDimension.edge_padding_low) +} + +// int64 edge_padding_high = 2; +inline void PaddingConfig_PaddingConfigDimension::clear_edge_padding_high() { + edge_padding_high_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 PaddingConfig_PaddingConfigDimension::edge_padding_high() const { + // @@protoc_insertion_point(field_get:xla.PaddingConfig.PaddingConfigDimension.edge_padding_high) + return edge_padding_high_; +} +inline void PaddingConfig_PaddingConfigDimension::set_edge_padding_high(::google::protobuf::int64 value) { + + edge_padding_high_ = value; + // @@protoc_insertion_point(field_set:xla.PaddingConfig.PaddingConfigDimension.edge_padding_high) +} + +// int64 interior_padding = 3; +inline void PaddingConfig_PaddingConfigDimension::clear_interior_padding() { + interior_padding_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 PaddingConfig_PaddingConfigDimension::interior_padding() const { + // @@protoc_insertion_point(field_get:xla.PaddingConfig.PaddingConfigDimension.interior_padding) + return interior_padding_; +} +inline void PaddingConfig_PaddingConfigDimension::set_interior_padding(::google::protobuf::int64 value) { + + interior_padding_ = value; + // @@protoc_insertion_point(field_set:xla.PaddingConfig.PaddingConfigDimension.interior_padding) +} + +// ------------------------------------------------------------------- + +// PaddingConfig + +// repeated .xla.PaddingConfig.PaddingConfigDimension dimensions = 1; +inline int PaddingConfig::dimensions_size() const { + return dimensions_.size(); +} +inline void PaddingConfig::clear_dimensions() { + dimensions_.Clear(); +} +inline ::xla::PaddingConfig_PaddingConfigDimension* PaddingConfig::mutable_dimensions(int index) { + // @@protoc_insertion_point(field_mutable:xla.PaddingConfig.dimensions) + return dimensions_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::xla::PaddingConfig_PaddingConfigDimension >* +PaddingConfig::mutable_dimensions() { + // @@protoc_insertion_point(field_mutable_list:xla.PaddingConfig.dimensions) + return &dimensions_; +} +inline const ::xla::PaddingConfig_PaddingConfigDimension& PaddingConfig::dimensions(int index) const { + // @@protoc_insertion_point(field_get:xla.PaddingConfig.dimensions) + return dimensions_.Get(index); +} +inline ::xla::PaddingConfig_PaddingConfigDimension* PaddingConfig::add_dimensions() { + // @@protoc_insertion_point(field_add:xla.PaddingConfig.dimensions) + return dimensions_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::xla::PaddingConfig_PaddingConfigDimension >& +PaddingConfig::dimensions() const { + // @@protoc_insertion_point(field_list:xla.PaddingConfig.dimensions) + return dimensions_; +} + +// ------------------------------------------------------------------- + +// Layout + +// .xla.Format format = 4; +inline void Layout::clear_format() { + format_ = 0; +} +inline ::xla::Format Layout::format() const { + // @@protoc_insertion_point(field_get:xla.Layout.format) + return static_cast< ::xla::Format >(format_); +} +inline void Layout::set_format(::xla::Format value) { + + format_ = value; + // @@protoc_insertion_point(field_set:xla.Layout.format) +} + +// repeated int64 minor_to_major = 1; +inline int Layout::minor_to_major_size() const { + return minor_to_major_.size(); +} +inline void Layout::clear_minor_to_major() { + minor_to_major_.Clear(); +} +inline ::google::protobuf::int64 Layout::minor_to_major(int index) const { + // @@protoc_insertion_point(field_get:xla.Layout.minor_to_major) + return minor_to_major_.Get(index); +} +inline void Layout::set_minor_to_major(int index, ::google::protobuf::int64 value) { + minor_to_major_.Set(index, value); + // @@protoc_insertion_point(field_set:xla.Layout.minor_to_major) +} +inline void Layout::add_minor_to_major(::google::protobuf::int64 value) { + minor_to_major_.Add(value); + // @@protoc_insertion_point(field_add:xla.Layout.minor_to_major) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& +Layout::minor_to_major() const { + // @@protoc_insertion_point(field_list:xla.Layout.minor_to_major) + return minor_to_major_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* +Layout::mutable_minor_to_major() { + // @@protoc_insertion_point(field_mutable_list:xla.Layout.minor_to_major) + return &minor_to_major_; +} + +// repeated int64 padded_dimensions = 2; +inline int Layout::padded_dimensions_size() const { + return padded_dimensions_.size(); +} +inline void Layout::clear_padded_dimensions() { + padded_dimensions_.Clear(); +} +inline ::google::protobuf::int64 Layout::padded_dimensions(int index) const { + // @@protoc_insertion_point(field_get:xla.Layout.padded_dimensions) + return padded_dimensions_.Get(index); +} +inline void Layout::set_padded_dimensions(int index, ::google::protobuf::int64 value) { + padded_dimensions_.Set(index, value); + // @@protoc_insertion_point(field_set:xla.Layout.padded_dimensions) +} +inline void Layout::add_padded_dimensions(::google::protobuf::int64 value) { + padded_dimensions_.Add(value); + // @@protoc_insertion_point(field_add:xla.Layout.padded_dimensions) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& +Layout::padded_dimensions() const { + // @@protoc_insertion_point(field_list:xla.Layout.padded_dimensions) + return padded_dimensions_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* +Layout::mutable_padded_dimensions() { + // @@protoc_insertion_point(field_mutable_list:xla.Layout.padded_dimensions) + return &padded_dimensions_; +} + +// .xla.PaddingValue padding_value = 3; +inline void Layout::clear_padding_value() { + padding_value_ = 0; +} +inline ::xla::PaddingValue Layout::padding_value() const { + // @@protoc_insertion_point(field_get:xla.Layout.padding_value) + return static_cast< ::xla::PaddingValue >(padding_value_); +} +inline void Layout::set_padding_value(::xla::PaddingValue value) { + + padding_value_ = value; + // @@protoc_insertion_point(field_set:xla.Layout.padding_value) +} + +// int64 max_sparse_elements = 5; +inline void Layout::clear_max_sparse_elements() { + max_sparse_elements_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 Layout::max_sparse_elements() const { + // @@protoc_insertion_point(field_get:xla.Layout.max_sparse_elements) + return max_sparse_elements_; +} +inline void Layout::set_max_sparse_elements(::google::protobuf::int64 value) { + + max_sparse_elements_ = value; + // @@protoc_insertion_point(field_set:xla.Layout.max_sparse_elements) +} + +// ------------------------------------------------------------------- + +// Shape + +// .xla.PrimitiveType element_type = 2; +inline void Shape::clear_element_type() { + element_type_ = 0; +} +inline ::xla::PrimitiveType Shape::element_type() const { + // @@protoc_insertion_point(field_get:xla.Shape.element_type) + return static_cast< ::xla::PrimitiveType >(element_type_); +} +inline void Shape::set_element_type(::xla::PrimitiveType value) { + + element_type_ = value; + // @@protoc_insertion_point(field_set:xla.Shape.element_type) +} + +// repeated int64 dimensions = 3; +inline int Shape::dimensions_size() const { + return dimensions_.size(); +} +inline void Shape::clear_dimensions() { + dimensions_.Clear(); +} +inline ::google::protobuf::int64 Shape::dimensions(int index) const { + // @@protoc_insertion_point(field_get:xla.Shape.dimensions) + return dimensions_.Get(index); +} +inline void Shape::set_dimensions(int index, ::google::protobuf::int64 value) { + dimensions_.Set(index, value); + // @@protoc_insertion_point(field_set:xla.Shape.dimensions) +} +inline void Shape::add_dimensions(::google::protobuf::int64 value) { + dimensions_.Add(value); + // @@protoc_insertion_point(field_add:xla.Shape.dimensions) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& +Shape::dimensions() const { + // @@protoc_insertion_point(field_list:xla.Shape.dimensions) + return dimensions_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* +Shape::mutable_dimensions() { + // @@protoc_insertion_point(field_mutable_list:xla.Shape.dimensions) + return &dimensions_; +} + +// repeated .xla.Shape tuple_shapes = 4; +inline int Shape::tuple_shapes_size() const { + return tuple_shapes_.size(); +} +inline void Shape::clear_tuple_shapes() { + tuple_shapes_.Clear(); +} +inline ::xla::Shape* Shape::mutable_tuple_shapes(int index) { + // @@protoc_insertion_point(field_mutable:xla.Shape.tuple_shapes) + return tuple_shapes_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::xla::Shape >* +Shape::mutable_tuple_shapes() { + // @@protoc_insertion_point(field_mutable_list:xla.Shape.tuple_shapes) + return &tuple_shapes_; +} +inline const ::xla::Shape& Shape::tuple_shapes(int index) const { + // @@protoc_insertion_point(field_get:xla.Shape.tuple_shapes) + return tuple_shapes_.Get(index); +} +inline ::xla::Shape* Shape::add_tuple_shapes() { + // @@protoc_insertion_point(field_add:xla.Shape.tuple_shapes) + return tuple_shapes_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::xla::Shape >& +Shape::tuple_shapes() const { + // @@protoc_insertion_point(field_list:xla.Shape.tuple_shapes) + return tuple_shapes_; +} + +// .xla.Layout layout = 5; +inline bool Shape::has_layout() const { + return this != internal_default_instance() && layout_ != NULL; +} +inline void Shape::clear_layout() { + if (GetArenaNoVirtual() == NULL && layout_ != NULL) { + delete layout_; + } + layout_ = NULL; +} +inline const ::xla::Layout& Shape::_internal_layout() const { + return *layout_; +} +inline const ::xla::Layout& Shape::layout() const { + const ::xla::Layout* p = layout_; + // @@protoc_insertion_point(field_get:xla.Shape.layout) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_Layout_default_instance_); +} +inline ::xla::Layout* Shape::release_layout() { + // @@protoc_insertion_point(field_release:xla.Shape.layout) + + ::xla::Layout* temp = layout_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + layout_ = NULL; + return temp; +} +inline ::xla::Layout* Shape::unsafe_arena_release_layout() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.Shape.layout) + + ::xla::Layout* temp = layout_; + layout_ = NULL; + return temp; +} +inline ::xla::Layout* Shape::mutable_layout() { + + if (layout_ == NULL) { + auto* p = CreateMaybeMessage<::xla::Layout>(GetArenaNoVirtual()); + layout_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.Shape.layout) + return layout_; +} +inline void Shape::set_allocated_layout(::xla::Layout* layout) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete layout_; + } + if (layout) { + ::google::protobuf::Arena* submessage_arena = + ::google::protobuf::Arena::GetArena(layout); + if (message_arena != submessage_arena) { + layout = ::google::protobuf::internal::GetOwnedMessage( + message_arena, layout, submessage_arena); + } + + } else { + + } + layout_ = layout; + // @@protoc_insertion_point(field_set_allocated:xla.Shape.layout) +} + +// ------------------------------------------------------------------- + +// ProgramShape + +// repeated .xla.Shape parameters = 1; +inline int ProgramShape::parameters_size() const { + return parameters_.size(); +} +inline void ProgramShape::clear_parameters() { + parameters_.Clear(); +} +inline ::xla::Shape* ProgramShape::mutable_parameters(int index) { + // @@protoc_insertion_point(field_mutable:xla.ProgramShape.parameters) + return parameters_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::xla::Shape >* +ProgramShape::mutable_parameters() { + // @@protoc_insertion_point(field_mutable_list:xla.ProgramShape.parameters) + return ¶meters_; +} +inline const ::xla::Shape& ProgramShape::parameters(int index) const { + // @@protoc_insertion_point(field_get:xla.ProgramShape.parameters) + return parameters_.Get(index); +} +inline ::xla::Shape* ProgramShape::add_parameters() { + // @@protoc_insertion_point(field_add:xla.ProgramShape.parameters) + return parameters_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::xla::Shape >& +ProgramShape::parameters() const { + // @@protoc_insertion_point(field_list:xla.ProgramShape.parameters) + return parameters_; +} + +// .xla.Shape result = 2; +inline bool ProgramShape::has_result() const { + return this != internal_default_instance() && result_ != NULL; +} +inline void ProgramShape::clear_result() { + if (GetArenaNoVirtual() == NULL && result_ != NULL) { + delete result_; + } + result_ = NULL; +} +inline const ::xla::Shape& ProgramShape::_internal_result() const { + return *result_; +} +inline const ::xla::Shape& ProgramShape::result() const { + const ::xla::Shape* p = result_; + // @@protoc_insertion_point(field_get:xla.ProgramShape.result) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_Shape_default_instance_); +} +inline ::xla::Shape* ProgramShape::release_result() { + // @@protoc_insertion_point(field_release:xla.ProgramShape.result) + + ::xla::Shape* temp = result_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + result_ = NULL; + return temp; +} +inline ::xla::Shape* ProgramShape::unsafe_arena_release_result() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.ProgramShape.result) + + ::xla::Shape* temp = result_; + result_ = NULL; + return temp; +} +inline ::xla::Shape* ProgramShape::mutable_result() { + + if (result_ == NULL) { + auto* p = CreateMaybeMessage<::xla::Shape>(GetArenaNoVirtual()); + result_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.ProgramShape.result) + return result_; +} +inline void ProgramShape::set_allocated_result(::xla::Shape* result) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete result_; + } + if (result) { + ::google::protobuf::Arena* submessage_arena = + ::google::protobuf::Arena::GetArena(result); + if (message_arena != submessage_arena) { + result = ::google::protobuf::internal::GetOwnedMessage( + message_arena, result, submessage_arena); + } + + } else { + + } + result_ = result; + // @@protoc_insertion_point(field_set_allocated:xla.ProgramShape.result) +} + +// repeated string parameter_names = 3; +inline int ProgramShape::parameter_names_size() const { + return parameter_names_.size(); +} +inline void ProgramShape::clear_parameter_names() { + parameter_names_.Clear(); +} +inline const ::std::string& ProgramShape::parameter_names(int index) const { + // @@protoc_insertion_point(field_get:xla.ProgramShape.parameter_names) + return parameter_names_.Get(index); +} +inline ::std::string* ProgramShape::mutable_parameter_names(int index) { + // @@protoc_insertion_point(field_mutable:xla.ProgramShape.parameter_names) + return parameter_names_.Mutable(index); +} +inline void ProgramShape::set_parameter_names(int index, const ::std::string& value) { + // @@protoc_insertion_point(field_set:xla.ProgramShape.parameter_names) + parameter_names_.Mutable(index)->assign(value); +} +#if LANG_CXX11 +inline void ProgramShape::set_parameter_names(int index, ::std::string&& value) { + // @@protoc_insertion_point(field_set:xla.ProgramShape.parameter_names) + parameter_names_.Mutable(index)->assign(std::move(value)); +} +#endif +inline void ProgramShape::set_parameter_names(int index, const char* value) { + GOOGLE_DCHECK(value != NULL); + parameter_names_.Mutable(index)->assign(value); + // @@protoc_insertion_point(field_set_char:xla.ProgramShape.parameter_names) +} +inline void ProgramShape::set_parameter_names(int index, const char* value, size_t size) { + parameter_names_.Mutable(index)->assign( + reinterpret_cast(value), size); + // @@protoc_insertion_point(field_set_pointer:xla.ProgramShape.parameter_names) +} +inline ::std::string* ProgramShape::add_parameter_names() { + // @@protoc_insertion_point(field_add_mutable:xla.ProgramShape.parameter_names) + return parameter_names_.Add(); +} +inline void ProgramShape::add_parameter_names(const ::std::string& value) { + parameter_names_.Add()->assign(value); + // @@protoc_insertion_point(field_add:xla.ProgramShape.parameter_names) +} +#if LANG_CXX11 +inline void ProgramShape::add_parameter_names(::std::string&& value) { + parameter_names_.Add(std::move(value)); + // @@protoc_insertion_point(field_add:xla.ProgramShape.parameter_names) +} +#endif +inline void ProgramShape::add_parameter_names(const char* value) { + GOOGLE_DCHECK(value != NULL); + parameter_names_.Add()->assign(value); + // @@protoc_insertion_point(field_add_char:xla.ProgramShape.parameter_names) +} +inline void ProgramShape::add_parameter_names(const char* value, size_t size) { + parameter_names_.Add()->assign(reinterpret_cast(value), size); + // @@protoc_insertion_point(field_add_pointer:xla.ProgramShape.parameter_names) +} +inline const ::google::protobuf::RepeatedPtrField< ::std::string>& +ProgramShape::parameter_names() const { + // @@protoc_insertion_point(field_list:xla.ProgramShape.parameter_names) + return parameter_names_; +} +inline ::google::protobuf::RepeatedPtrField< ::std::string>* +ProgramShape::mutable_parameter_names() { + // @@protoc_insertion_point(field_mutable_list:xla.ProgramShape.parameter_names) + return ¶meter_names_; +} + +// ------------------------------------------------------------------- + +// ComputationStats + +// double flop_count = 1; +inline void ComputationStats::clear_flop_count() { + flop_count_ = 0; +} +inline double ComputationStats::flop_count() const { + // @@protoc_insertion_point(field_get:xla.ComputationStats.flop_count) + return flop_count_; +} +inline void ComputationStats::set_flop_count(double value) { + + flop_count_ = value; + // @@protoc_insertion_point(field_set:xla.ComputationStats.flop_count) +} + +// double transcendental_count = 2; +inline void ComputationStats::clear_transcendental_count() { + transcendental_count_ = 0; +} +inline double ComputationStats::transcendental_count() const { + // @@protoc_insertion_point(field_get:xla.ComputationStats.transcendental_count) + return transcendental_count_; +} +inline void ComputationStats::set_transcendental_count(double value) { + + transcendental_count_ = value; + // @@protoc_insertion_point(field_set:xla.ComputationStats.transcendental_count) +} + +// ------------------------------------------------------------------- + +// OpMetadata + +// string op_type = 1; +inline void OpMetadata::clear_op_type() { + op_type_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& OpMetadata::op_type() const { + // @@protoc_insertion_point(field_get:xla.OpMetadata.op_type) + return op_type_.Get(); +} +inline void OpMetadata::set_op_type(const ::std::string& value) { + + op_type_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:xla.OpMetadata.op_type) +} +#if LANG_CXX11 +inline void OpMetadata::set_op_type(::std::string&& value) { + + op_type_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:xla.OpMetadata.op_type) +} +#endif +inline void OpMetadata::set_op_type(const char* value) { + GOOGLE_DCHECK(value != NULL); + + op_type_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:xla.OpMetadata.op_type) +} +inline void OpMetadata::set_op_type(const char* value, + size_t size) { + + op_type_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:xla.OpMetadata.op_type) +} +inline ::std::string* OpMetadata::mutable_op_type() { + + // @@protoc_insertion_point(field_mutable:xla.OpMetadata.op_type) + return op_type_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* OpMetadata::release_op_type() { + // @@protoc_insertion_point(field_release:xla.OpMetadata.op_type) + + return op_type_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void OpMetadata::set_allocated_op_type(::std::string* op_type) { + if (op_type != NULL) { + + } else { + + } + op_type_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), op_type, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:xla.OpMetadata.op_type) +} +inline ::std::string* OpMetadata::unsafe_arena_release_op_type() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.OpMetadata.op_type) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return op_type_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void OpMetadata::unsafe_arena_set_allocated_op_type( + ::std::string* op_type) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (op_type != NULL) { + + } else { + + } + op_type_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + op_type, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:xla.OpMetadata.op_type) +} + +// string op_name = 2; +inline void OpMetadata::clear_op_name() { + op_name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& OpMetadata::op_name() const { + // @@protoc_insertion_point(field_get:xla.OpMetadata.op_name) + return op_name_.Get(); +} +inline void OpMetadata::set_op_name(const ::std::string& value) { + + op_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:xla.OpMetadata.op_name) +} +#if LANG_CXX11 +inline void OpMetadata::set_op_name(::std::string&& value) { + + op_name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:xla.OpMetadata.op_name) +} +#endif +inline void OpMetadata::set_op_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + op_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:xla.OpMetadata.op_name) +} +inline void OpMetadata::set_op_name(const char* value, + size_t size) { + + op_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:xla.OpMetadata.op_name) +} +inline ::std::string* OpMetadata::mutable_op_name() { + + // @@protoc_insertion_point(field_mutable:xla.OpMetadata.op_name) + return op_name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* OpMetadata::release_op_name() { + // @@protoc_insertion_point(field_release:xla.OpMetadata.op_name) + + return op_name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void OpMetadata::set_allocated_op_name(::std::string* op_name) { + if (op_name != NULL) { + + } else { + + } + op_name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), op_name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:xla.OpMetadata.op_name) +} +inline ::std::string* OpMetadata::unsafe_arena_release_op_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.OpMetadata.op_name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return op_name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void OpMetadata::unsafe_arena_set_allocated_op_name( + ::std::string* op_name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (op_name != NULL) { + + } else { + + } + op_name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + op_name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:xla.OpMetadata.op_name) +} + +// string source_file = 3; +inline void OpMetadata::clear_source_file() { + source_file_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& OpMetadata::source_file() const { + // @@protoc_insertion_point(field_get:xla.OpMetadata.source_file) + return source_file_.Get(); +} +inline void OpMetadata::set_source_file(const ::std::string& value) { + + source_file_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:xla.OpMetadata.source_file) +} +#if LANG_CXX11 +inline void OpMetadata::set_source_file(::std::string&& value) { + + source_file_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:xla.OpMetadata.source_file) +} +#endif +inline void OpMetadata::set_source_file(const char* value) { + GOOGLE_DCHECK(value != NULL); + + source_file_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:xla.OpMetadata.source_file) +} +inline void OpMetadata::set_source_file(const char* value, + size_t size) { + + source_file_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:xla.OpMetadata.source_file) +} +inline ::std::string* OpMetadata::mutable_source_file() { + + // @@protoc_insertion_point(field_mutable:xla.OpMetadata.source_file) + return source_file_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* OpMetadata::release_source_file() { + // @@protoc_insertion_point(field_release:xla.OpMetadata.source_file) + + return source_file_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void OpMetadata::set_allocated_source_file(::std::string* source_file) { + if (source_file != NULL) { + + } else { + + } + source_file_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), source_file, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:xla.OpMetadata.source_file) +} +inline ::std::string* OpMetadata::unsafe_arena_release_source_file() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.OpMetadata.source_file) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return source_file_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void OpMetadata::unsafe_arena_set_allocated_source_file( + ::std::string* source_file) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (source_file != NULL) { + + } else { + + } + source_file_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + source_file, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:xla.OpMetadata.source_file) +} + +// int32 source_line = 4; +inline void OpMetadata::clear_source_line() { + source_line_ = 0; +} +inline ::google::protobuf::int32 OpMetadata::source_line() const { + // @@protoc_insertion_point(field_get:xla.OpMetadata.source_line) + return source_line_; +} +inline void OpMetadata::set_source_line(::google::protobuf::int32 value) { + + source_line_ = value; + // @@protoc_insertion_point(field_set:xla.OpMetadata.source_line) +} + +// ------------------------------------------------------------------- + +// ExecutionProfile + +// bool compilation_cache_hit = 1; +inline void ExecutionProfile::clear_compilation_cache_hit() { + compilation_cache_hit_ = false; +} +inline bool ExecutionProfile::compilation_cache_hit() const { + // @@protoc_insertion_point(field_get:xla.ExecutionProfile.compilation_cache_hit) + return compilation_cache_hit_; +} +inline void ExecutionProfile::set_compilation_cache_hit(bool value) { + + compilation_cache_hit_ = value; + // @@protoc_insertion_point(field_set:xla.ExecutionProfile.compilation_cache_hit) +} + +// int64 compile_time_ms = 2; +inline void ExecutionProfile::clear_compile_time_ms() { + compile_time_ms_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 ExecutionProfile::compile_time_ms() const { + // @@protoc_insertion_point(field_get:xla.ExecutionProfile.compile_time_ms) + return compile_time_ms_; +} +inline void ExecutionProfile::set_compile_time_ms(::google::protobuf::int64 value) { + + compile_time_ms_ = value; + // @@protoc_insertion_point(field_set:xla.ExecutionProfile.compile_time_ms) +} + +// int64 compute_cycle_count = 3; +inline void ExecutionProfile::clear_compute_cycle_count() { + compute_cycle_count_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 ExecutionProfile::compute_cycle_count() const { + // @@protoc_insertion_point(field_get:xla.ExecutionProfile.compute_cycle_count) + return compute_cycle_count_; +} +inline void ExecutionProfile::set_compute_cycle_count(::google::protobuf::int64 value) { + + compute_cycle_count_ = value; + // @@protoc_insertion_point(field_set:xla.ExecutionProfile.compute_cycle_count) +} + +// int64 compute_time_ns = 4; +inline void ExecutionProfile::clear_compute_time_ns() { + compute_time_ns_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 ExecutionProfile::compute_time_ns() const { + // @@protoc_insertion_point(field_get:xla.ExecutionProfile.compute_time_ns) + return compute_time_ns_; +} +inline void ExecutionProfile::set_compute_time_ns(::google::protobuf::int64 value) { + + compute_time_ns_ = value; + // @@protoc_insertion_point(field_set:xla.ExecutionProfile.compute_time_ns) +} + +// int64 compute_and_transfer_time_ns = 5; +inline void ExecutionProfile::clear_compute_and_transfer_time_ns() { + compute_and_transfer_time_ns_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 ExecutionProfile::compute_and_transfer_time_ns() const { + // @@protoc_insertion_point(field_get:xla.ExecutionProfile.compute_and_transfer_time_ns) + return compute_and_transfer_time_ns_; +} +inline void ExecutionProfile::set_compute_and_transfer_time_ns(::google::protobuf::int64 value) { + + compute_and_transfer_time_ns_ = value; + // @@protoc_insertion_point(field_set:xla.ExecutionProfile.compute_and_transfer_time_ns) +} + +// int64 executable_size_in_bytes = 6; +inline void ExecutionProfile::clear_executable_size_in_bytes() { + executable_size_in_bytes_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 ExecutionProfile::executable_size_in_bytes() const { + // @@protoc_insertion_point(field_get:xla.ExecutionProfile.executable_size_in_bytes) + return executable_size_in_bytes_; +} +inline void ExecutionProfile::set_executable_size_in_bytes(::google::protobuf::int64 value) { + + executable_size_in_bytes_ = value; + // @@protoc_insertion_point(field_set:xla.ExecutionProfile.executable_size_in_bytes) +} + +// ------------------------------------------------------------------- + +// ExecutionHandle + +// int64 handle = 1; +inline void ExecutionHandle::clear_handle() { + handle_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 ExecutionHandle::handle() const { + // @@protoc_insertion_point(field_get:xla.ExecutionHandle.handle) + return handle_; +} +inline void ExecutionHandle::set_handle(::google::protobuf::int64 value) { + + handle_ = value; + // @@protoc_insertion_point(field_set:xla.ExecutionHandle.handle) +} + +// ------------------------------------------------------------------- + +// GlobalDataHandle + +// int64 handle = 1; +inline void GlobalDataHandle::clear_handle() { + handle_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 GlobalDataHandle::handle() const { + // @@protoc_insertion_point(field_get:xla.GlobalDataHandle.handle) + return handle_; +} +inline void GlobalDataHandle::set_handle(::google::protobuf::int64 value) { + + handle_ = value; + // @@protoc_insertion_point(field_set:xla.GlobalDataHandle.handle) +} + +// ------------------------------------------------------------------- + +// DeviceHandle + +// int64 handle = 1; +inline void DeviceHandle::clear_handle() { + handle_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 DeviceHandle::handle() const { + // @@protoc_insertion_point(field_get:xla.DeviceHandle.handle) + return handle_; +} +inline void DeviceHandle::set_handle(::google::protobuf::int64 value) { + + handle_ = value; + // @@protoc_insertion_point(field_set:xla.DeviceHandle.handle) +} + +// int64 device_count = 2; +inline void DeviceHandle::clear_device_count() { + device_count_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 DeviceHandle::device_count() const { + // @@protoc_insertion_point(field_get:xla.DeviceHandle.device_count) + return device_count_; +} +inline void DeviceHandle::set_device_count(::google::protobuf::int64 value) { + + device_count_ = value; + // @@protoc_insertion_point(field_set:xla.DeviceHandle.device_count) +} + +// ------------------------------------------------------------------- + +// ChannelHandle + +// int64 handle = 1; +inline void ChannelHandle::clear_handle() { + handle_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 ChannelHandle::handle() const { + // @@protoc_insertion_point(field_get:xla.ChannelHandle.handle) + return handle_; +} +inline void ChannelHandle::set_handle(::google::protobuf::int64 value) { + + handle_ = value; + // @@protoc_insertion_point(field_set:xla.ChannelHandle.handle) +} + +// .xla.ChannelHandle.ChannelType type = 2; +inline void ChannelHandle::clear_type() { + type_ = 0; +} +inline ::xla::ChannelHandle_ChannelType ChannelHandle::type() const { + // @@protoc_insertion_point(field_get:xla.ChannelHandle.type) + return static_cast< ::xla::ChannelHandle_ChannelType >(type_); +} +inline void ChannelHandle::set_type(::xla::ChannelHandle_ChannelType value) { + + type_ = value; + // @@protoc_insertion_point(field_set:xla.ChannelHandle.type) +} + +// ------------------------------------------------------------------- + +// DeviceAssignmentProto_ComputationDevice + +// repeated int32 replica_device_ids = 1; +inline int DeviceAssignmentProto_ComputationDevice::replica_device_ids_size() const { + return replica_device_ids_.size(); +} +inline void DeviceAssignmentProto_ComputationDevice::clear_replica_device_ids() { + replica_device_ids_.Clear(); +} +inline ::google::protobuf::int32 DeviceAssignmentProto_ComputationDevice::replica_device_ids(int index) const { + // @@protoc_insertion_point(field_get:xla.DeviceAssignmentProto.ComputationDevice.replica_device_ids) + return replica_device_ids_.Get(index); +} +inline void DeviceAssignmentProto_ComputationDevice::set_replica_device_ids(int index, ::google::protobuf::int32 value) { + replica_device_ids_.Set(index, value); + // @@protoc_insertion_point(field_set:xla.DeviceAssignmentProto.ComputationDevice.replica_device_ids) +} +inline void DeviceAssignmentProto_ComputationDevice::add_replica_device_ids(::google::protobuf::int32 value) { + replica_device_ids_.Add(value); + // @@protoc_insertion_point(field_add:xla.DeviceAssignmentProto.ComputationDevice.replica_device_ids) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int32 >& +DeviceAssignmentProto_ComputationDevice::replica_device_ids() const { + // @@protoc_insertion_point(field_list:xla.DeviceAssignmentProto.ComputationDevice.replica_device_ids) + return replica_device_ids_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int32 >* +DeviceAssignmentProto_ComputationDevice::mutable_replica_device_ids() { + // @@protoc_insertion_point(field_mutable_list:xla.DeviceAssignmentProto.ComputationDevice.replica_device_ids) + return &replica_device_ids_; +} + +// ------------------------------------------------------------------- + +// DeviceAssignmentProto + +// int32 replica_count = 1; +inline void DeviceAssignmentProto::clear_replica_count() { + replica_count_ = 0; +} +inline ::google::protobuf::int32 DeviceAssignmentProto::replica_count() const { + // @@protoc_insertion_point(field_get:xla.DeviceAssignmentProto.replica_count) + return replica_count_; +} +inline void DeviceAssignmentProto::set_replica_count(::google::protobuf::int32 value) { + + replica_count_ = value; + // @@protoc_insertion_point(field_set:xla.DeviceAssignmentProto.replica_count) +} + +// int32 computation_count = 2; +inline void DeviceAssignmentProto::clear_computation_count() { + computation_count_ = 0; +} +inline ::google::protobuf::int32 DeviceAssignmentProto::computation_count() const { + // @@protoc_insertion_point(field_get:xla.DeviceAssignmentProto.computation_count) + return computation_count_; +} +inline void DeviceAssignmentProto::set_computation_count(::google::protobuf::int32 value) { + + computation_count_ = value; + // @@protoc_insertion_point(field_set:xla.DeviceAssignmentProto.computation_count) +} + +// repeated .xla.DeviceAssignmentProto.ComputationDevice computation_devices = 3; +inline int DeviceAssignmentProto::computation_devices_size() const { + return computation_devices_.size(); +} +inline void DeviceAssignmentProto::clear_computation_devices() { + computation_devices_.Clear(); +} +inline ::xla::DeviceAssignmentProto_ComputationDevice* DeviceAssignmentProto::mutable_computation_devices(int index) { + // @@protoc_insertion_point(field_mutable:xla.DeviceAssignmentProto.computation_devices) + return computation_devices_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::xla::DeviceAssignmentProto_ComputationDevice >* +DeviceAssignmentProto::mutable_computation_devices() { + // @@protoc_insertion_point(field_mutable_list:xla.DeviceAssignmentProto.computation_devices) + return &computation_devices_; +} +inline const ::xla::DeviceAssignmentProto_ComputationDevice& DeviceAssignmentProto::computation_devices(int index) const { + // @@protoc_insertion_point(field_get:xla.DeviceAssignmentProto.computation_devices) + return computation_devices_.Get(index); +} +inline ::xla::DeviceAssignmentProto_ComputationDevice* DeviceAssignmentProto::add_computation_devices() { + // @@protoc_insertion_point(field_add:xla.DeviceAssignmentProto.computation_devices) + return computation_devices_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::xla::DeviceAssignmentProto_ComputationDevice >& +DeviceAssignmentProto::computation_devices() const { + // @@protoc_insertion_point(field_list:xla.DeviceAssignmentProto.computation_devices) + return computation_devices_; +} + +// ------------------------------------------------------------------- + +// LiteralProto + +// .xla.Shape shape = 1; +inline bool LiteralProto::has_shape() const { + return this != internal_default_instance() && shape_ != NULL; +} +inline void LiteralProto::clear_shape() { + if (GetArenaNoVirtual() == NULL && shape_ != NULL) { + delete shape_; + } + shape_ = NULL; +} +inline const ::xla::Shape& LiteralProto::_internal_shape() const { + return *shape_; +} +inline const ::xla::Shape& LiteralProto::shape() const { + const ::xla::Shape* p = shape_; + // @@protoc_insertion_point(field_get:xla.LiteralProto.shape) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_Shape_default_instance_); +} +inline ::xla::Shape* LiteralProto::release_shape() { + // @@protoc_insertion_point(field_release:xla.LiteralProto.shape) + + ::xla::Shape* temp = shape_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + shape_ = NULL; + return temp; +} +inline ::xla::Shape* LiteralProto::unsafe_arena_release_shape() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.LiteralProto.shape) + + ::xla::Shape* temp = shape_; + shape_ = NULL; + return temp; +} +inline ::xla::Shape* LiteralProto::mutable_shape() { + + if (shape_ == NULL) { + auto* p = CreateMaybeMessage<::xla::Shape>(GetArenaNoVirtual()); + shape_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.LiteralProto.shape) + return shape_; +} +inline void LiteralProto::set_allocated_shape(::xla::Shape* shape) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete shape_; + } + if (shape) { + ::google::protobuf::Arena* submessage_arena = + ::google::protobuf::Arena::GetArena(shape); + if (message_arena != submessage_arena) { + shape = ::google::protobuf::internal::GetOwnedMessage( + message_arena, shape, submessage_arena); + } + + } else { + + } + shape_ = shape; + // @@protoc_insertion_point(field_set_allocated:xla.LiteralProto.shape) +} + +// repeated bool preds = 2; +inline int LiteralProto::preds_size() const { + return preds_.size(); +} +inline void LiteralProto::clear_preds() { + preds_.Clear(); +} +inline bool LiteralProto::preds(int index) const { + // @@protoc_insertion_point(field_get:xla.LiteralProto.preds) + return preds_.Get(index); +} +inline void LiteralProto::set_preds(int index, bool value) { + preds_.Set(index, value); + // @@protoc_insertion_point(field_set:xla.LiteralProto.preds) +} +inline void LiteralProto::add_preds(bool value) { + preds_.Add(value); + // @@protoc_insertion_point(field_add:xla.LiteralProto.preds) +} +inline const ::google::protobuf::RepeatedField< bool >& +LiteralProto::preds() const { + // @@protoc_insertion_point(field_list:xla.LiteralProto.preds) + return preds_; +} +inline ::google::protobuf::RepeatedField< bool >* +LiteralProto::mutable_preds() { + // @@protoc_insertion_point(field_mutable_list:xla.LiteralProto.preds) + return &preds_; +} + +// bytes s8s = 15; +inline void LiteralProto::clear_s8s() { + s8s_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& LiteralProto::s8s() const { + // @@protoc_insertion_point(field_get:xla.LiteralProto.s8s) + return s8s_.Get(); +} +inline void LiteralProto::set_s8s(const ::std::string& value) { + + s8s_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:xla.LiteralProto.s8s) +} +#if LANG_CXX11 +inline void LiteralProto::set_s8s(::std::string&& value) { + + s8s_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:xla.LiteralProto.s8s) +} +#endif +inline void LiteralProto::set_s8s(const char* value) { + GOOGLE_DCHECK(value != NULL); + + s8s_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:xla.LiteralProto.s8s) +} +inline void LiteralProto::set_s8s(const void* value, + size_t size) { + + s8s_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:xla.LiteralProto.s8s) +} +inline ::std::string* LiteralProto::mutable_s8s() { + + // @@protoc_insertion_point(field_mutable:xla.LiteralProto.s8s) + return s8s_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* LiteralProto::release_s8s() { + // @@protoc_insertion_point(field_release:xla.LiteralProto.s8s) + + return s8s_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void LiteralProto::set_allocated_s8s(::std::string* s8s) { + if (s8s != NULL) { + + } else { + + } + s8s_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), s8s, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:xla.LiteralProto.s8s) +} +inline ::std::string* LiteralProto::unsafe_arena_release_s8s() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.LiteralProto.s8s) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return s8s_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void LiteralProto::unsafe_arena_set_allocated_s8s( + ::std::string* s8s) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (s8s != NULL) { + + } else { + + } + s8s_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + s8s, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:xla.LiteralProto.s8s) +} + +// bytes u8s = 3; +inline void LiteralProto::clear_u8s() { + u8s_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& LiteralProto::u8s() const { + // @@protoc_insertion_point(field_get:xla.LiteralProto.u8s) + return u8s_.Get(); +} +inline void LiteralProto::set_u8s(const ::std::string& value) { + + u8s_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:xla.LiteralProto.u8s) +} +#if LANG_CXX11 +inline void LiteralProto::set_u8s(::std::string&& value) { + + u8s_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:xla.LiteralProto.u8s) +} +#endif +inline void LiteralProto::set_u8s(const char* value) { + GOOGLE_DCHECK(value != NULL); + + u8s_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:xla.LiteralProto.u8s) +} +inline void LiteralProto::set_u8s(const void* value, + size_t size) { + + u8s_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:xla.LiteralProto.u8s) +} +inline ::std::string* LiteralProto::mutable_u8s() { + + // @@protoc_insertion_point(field_mutable:xla.LiteralProto.u8s) + return u8s_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* LiteralProto::release_u8s() { + // @@protoc_insertion_point(field_release:xla.LiteralProto.u8s) + + return u8s_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void LiteralProto::set_allocated_u8s(::std::string* u8s) { + if (u8s != NULL) { + + } else { + + } + u8s_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), u8s, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:xla.LiteralProto.u8s) +} +inline ::std::string* LiteralProto::unsafe_arena_release_u8s() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.LiteralProto.u8s) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return u8s_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void LiteralProto::unsafe_arena_set_allocated_u8s( + ::std::string* u8s) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (u8s != NULL) { + + } else { + + } + u8s_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + u8s, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:xla.LiteralProto.u8s) +} + +// repeated int32 s32s = 4; +inline int LiteralProto::s32s_size() const { + return s32s_.size(); +} +inline void LiteralProto::clear_s32s() { + s32s_.Clear(); +} +inline ::google::protobuf::int32 LiteralProto::s32s(int index) const { + // @@protoc_insertion_point(field_get:xla.LiteralProto.s32s) + return s32s_.Get(index); +} +inline void LiteralProto::set_s32s(int index, ::google::protobuf::int32 value) { + s32s_.Set(index, value); + // @@protoc_insertion_point(field_set:xla.LiteralProto.s32s) +} +inline void LiteralProto::add_s32s(::google::protobuf::int32 value) { + s32s_.Add(value); + // @@protoc_insertion_point(field_add:xla.LiteralProto.s32s) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int32 >& +LiteralProto::s32s() const { + // @@protoc_insertion_point(field_list:xla.LiteralProto.s32s) + return s32s_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int32 >* +LiteralProto::mutable_s32s() { + // @@protoc_insertion_point(field_mutable_list:xla.LiteralProto.s32s) + return &s32s_; +} + +// repeated int64 s64s = 5; +inline int LiteralProto::s64s_size() const { + return s64s_.size(); +} +inline void LiteralProto::clear_s64s() { + s64s_.Clear(); +} +inline ::google::protobuf::int64 LiteralProto::s64s(int index) const { + // @@protoc_insertion_point(field_get:xla.LiteralProto.s64s) + return s64s_.Get(index); +} +inline void LiteralProto::set_s64s(int index, ::google::protobuf::int64 value) { + s64s_.Set(index, value); + // @@protoc_insertion_point(field_set:xla.LiteralProto.s64s) +} +inline void LiteralProto::add_s64s(::google::protobuf::int64 value) { + s64s_.Add(value); + // @@protoc_insertion_point(field_add:xla.LiteralProto.s64s) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& +LiteralProto::s64s() const { + // @@protoc_insertion_point(field_list:xla.LiteralProto.s64s) + return s64s_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* +LiteralProto::mutable_s64s() { + // @@protoc_insertion_point(field_mutable_list:xla.LiteralProto.s64s) + return &s64s_; +} + +// repeated uint32 u32s = 6; +inline int LiteralProto::u32s_size() const { + return u32s_.size(); +} +inline void LiteralProto::clear_u32s() { + u32s_.Clear(); +} +inline ::google::protobuf::uint32 LiteralProto::u32s(int index) const { + // @@protoc_insertion_point(field_get:xla.LiteralProto.u32s) + return u32s_.Get(index); +} +inline void LiteralProto::set_u32s(int index, ::google::protobuf::uint32 value) { + u32s_.Set(index, value); + // @@protoc_insertion_point(field_set:xla.LiteralProto.u32s) +} +inline void LiteralProto::add_u32s(::google::protobuf::uint32 value) { + u32s_.Add(value); + // @@protoc_insertion_point(field_add:xla.LiteralProto.u32s) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::uint32 >& +LiteralProto::u32s() const { + // @@protoc_insertion_point(field_list:xla.LiteralProto.u32s) + return u32s_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::uint32 >* +LiteralProto::mutable_u32s() { + // @@protoc_insertion_point(field_mutable_list:xla.LiteralProto.u32s) + return &u32s_; +} + +// repeated uint64 u64s = 7; +inline int LiteralProto::u64s_size() const { + return u64s_.size(); +} +inline void LiteralProto::clear_u64s() { + u64s_.Clear(); +} +inline ::google::protobuf::uint64 LiteralProto::u64s(int index) const { + // @@protoc_insertion_point(field_get:xla.LiteralProto.u64s) + return u64s_.Get(index); +} +inline void LiteralProto::set_u64s(int index, ::google::protobuf::uint64 value) { + u64s_.Set(index, value); + // @@protoc_insertion_point(field_set:xla.LiteralProto.u64s) +} +inline void LiteralProto::add_u64s(::google::protobuf::uint64 value) { + u64s_.Add(value); + // @@protoc_insertion_point(field_add:xla.LiteralProto.u64s) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::uint64 >& +LiteralProto::u64s() const { + // @@protoc_insertion_point(field_list:xla.LiteralProto.u64s) + return u64s_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::uint64 >* +LiteralProto::mutable_u64s() { + // @@protoc_insertion_point(field_mutable_list:xla.LiteralProto.u64s) + return &u64s_; +} + +// repeated float f32s = 8; +inline int LiteralProto::f32s_size() const { + return f32s_.size(); +} +inline void LiteralProto::clear_f32s() { + f32s_.Clear(); +} +inline float LiteralProto::f32s(int index) const { + // @@protoc_insertion_point(field_get:xla.LiteralProto.f32s) + return f32s_.Get(index); +} +inline void LiteralProto::set_f32s(int index, float value) { + f32s_.Set(index, value); + // @@protoc_insertion_point(field_set:xla.LiteralProto.f32s) +} +inline void LiteralProto::add_f32s(float value) { + f32s_.Add(value); + // @@protoc_insertion_point(field_add:xla.LiteralProto.f32s) +} +inline const ::google::protobuf::RepeatedField< float >& +LiteralProto::f32s() const { + // @@protoc_insertion_point(field_list:xla.LiteralProto.f32s) + return f32s_; +} +inline ::google::protobuf::RepeatedField< float >* +LiteralProto::mutable_f32s() { + // @@protoc_insertion_point(field_mutable_list:xla.LiteralProto.f32s) + return &f32s_; +} + +// repeated double f64s = 9; +inline int LiteralProto::f64s_size() const { + return f64s_.size(); +} +inline void LiteralProto::clear_f64s() { + f64s_.Clear(); +} +inline double LiteralProto::f64s(int index) const { + // @@protoc_insertion_point(field_get:xla.LiteralProto.f64s) + return f64s_.Get(index); +} +inline void LiteralProto::set_f64s(int index, double value) { + f64s_.Set(index, value); + // @@protoc_insertion_point(field_set:xla.LiteralProto.f64s) +} +inline void LiteralProto::add_f64s(double value) { + f64s_.Add(value); + // @@protoc_insertion_point(field_add:xla.LiteralProto.f64s) +} +inline const ::google::protobuf::RepeatedField< double >& +LiteralProto::f64s() const { + // @@protoc_insertion_point(field_list:xla.LiteralProto.f64s) + return f64s_; +} +inline ::google::protobuf::RepeatedField< double >* +LiteralProto::mutable_f64s() { + // @@protoc_insertion_point(field_mutable_list:xla.LiteralProto.f64s) + return &f64s_; +} + +// repeated float c64s = 12; +inline int LiteralProto::c64s_size() const { + return c64s_.size(); +} +inline void LiteralProto::clear_c64s() { + c64s_.Clear(); +} +inline float LiteralProto::c64s(int index) const { + // @@protoc_insertion_point(field_get:xla.LiteralProto.c64s) + return c64s_.Get(index); +} +inline void LiteralProto::set_c64s(int index, float value) { + c64s_.Set(index, value); + // @@protoc_insertion_point(field_set:xla.LiteralProto.c64s) +} +inline void LiteralProto::add_c64s(float value) { + c64s_.Add(value); + // @@protoc_insertion_point(field_add:xla.LiteralProto.c64s) +} +inline const ::google::protobuf::RepeatedField< float >& +LiteralProto::c64s() const { + // @@protoc_insertion_point(field_list:xla.LiteralProto.c64s) + return c64s_; +} +inline ::google::protobuf::RepeatedField< float >* +LiteralProto::mutable_c64s() { + // @@protoc_insertion_point(field_mutable_list:xla.LiteralProto.c64s) + return &c64s_; +} + +// repeated .xla.LiteralProto tuple_literals = 10; +inline int LiteralProto::tuple_literals_size() const { + return tuple_literals_.size(); +} +inline void LiteralProto::clear_tuple_literals() { + tuple_literals_.Clear(); +} +inline ::xla::LiteralProto* LiteralProto::mutable_tuple_literals(int index) { + // @@protoc_insertion_point(field_mutable:xla.LiteralProto.tuple_literals) + return tuple_literals_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::xla::LiteralProto >* +LiteralProto::mutable_tuple_literals() { + // @@protoc_insertion_point(field_mutable_list:xla.LiteralProto.tuple_literals) + return &tuple_literals_; +} +inline const ::xla::LiteralProto& LiteralProto::tuple_literals(int index) const { + // @@protoc_insertion_point(field_get:xla.LiteralProto.tuple_literals) + return tuple_literals_.Get(index); +} +inline ::xla::LiteralProto* LiteralProto::add_tuple_literals() { + // @@protoc_insertion_point(field_add:xla.LiteralProto.tuple_literals) + return tuple_literals_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::xla::LiteralProto >& +LiteralProto::tuple_literals() const { + // @@protoc_insertion_point(field_list:xla.LiteralProto.tuple_literals) + return tuple_literals_; +} + +// bytes f16s = 11; +inline void LiteralProto::clear_f16s() { + f16s_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& LiteralProto::f16s() const { + // @@protoc_insertion_point(field_get:xla.LiteralProto.f16s) + return f16s_.Get(); +} +inline void LiteralProto::set_f16s(const ::std::string& value) { + + f16s_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:xla.LiteralProto.f16s) +} +#if LANG_CXX11 +inline void LiteralProto::set_f16s(::std::string&& value) { + + f16s_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:xla.LiteralProto.f16s) +} +#endif +inline void LiteralProto::set_f16s(const char* value) { + GOOGLE_DCHECK(value != NULL); + + f16s_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:xla.LiteralProto.f16s) +} +inline void LiteralProto::set_f16s(const void* value, + size_t size) { + + f16s_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:xla.LiteralProto.f16s) +} +inline ::std::string* LiteralProto::mutable_f16s() { + + // @@protoc_insertion_point(field_mutable:xla.LiteralProto.f16s) + return f16s_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* LiteralProto::release_f16s() { + // @@protoc_insertion_point(field_release:xla.LiteralProto.f16s) + + return f16s_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void LiteralProto::set_allocated_f16s(::std::string* f16s) { + if (f16s != NULL) { + + } else { + + } + f16s_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), f16s, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:xla.LiteralProto.f16s) +} +inline ::std::string* LiteralProto::unsafe_arena_release_f16s() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.LiteralProto.f16s) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return f16s_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void LiteralProto::unsafe_arena_set_allocated_f16s( + ::std::string* f16s) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (f16s != NULL) { + + } else { + + } + f16s_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + f16s, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:xla.LiteralProto.f16s) +} + +// bytes bf16s = 13; +inline void LiteralProto::clear_bf16s() { + bf16s_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& LiteralProto::bf16s() const { + // @@protoc_insertion_point(field_get:xla.LiteralProto.bf16s) + return bf16s_.Get(); +} +inline void LiteralProto::set_bf16s(const ::std::string& value) { + + bf16s_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:xla.LiteralProto.bf16s) +} +#if LANG_CXX11 +inline void LiteralProto::set_bf16s(::std::string&& value) { + + bf16s_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:xla.LiteralProto.bf16s) +} +#endif +inline void LiteralProto::set_bf16s(const char* value) { + GOOGLE_DCHECK(value != NULL); + + bf16s_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:xla.LiteralProto.bf16s) +} +inline void LiteralProto::set_bf16s(const void* value, + size_t size) { + + bf16s_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:xla.LiteralProto.bf16s) +} +inline ::std::string* LiteralProto::mutable_bf16s() { + + // @@protoc_insertion_point(field_mutable:xla.LiteralProto.bf16s) + return bf16s_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* LiteralProto::release_bf16s() { + // @@protoc_insertion_point(field_release:xla.LiteralProto.bf16s) + + return bf16s_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void LiteralProto::set_allocated_bf16s(::std::string* bf16s) { + if (bf16s != NULL) { + + } else { + + } + bf16s_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), bf16s, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:xla.LiteralProto.bf16s) +} +inline ::std::string* LiteralProto::unsafe_arena_release_bf16s() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.LiteralProto.bf16s) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return bf16s_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void LiteralProto::unsafe_arena_set_allocated_bf16s( + ::std::string* bf16s) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (bf16s != NULL) { + + } else { + + } + bf16s_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + bf16s, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:xla.LiteralProto.bf16s) +} + +// repeated int64 sparse_indices = 14; +inline int LiteralProto::sparse_indices_size() const { + return sparse_indices_.size(); +} +inline void LiteralProto::clear_sparse_indices() { + sparse_indices_.Clear(); +} +inline ::google::protobuf::int64 LiteralProto::sparse_indices(int index) const { + // @@protoc_insertion_point(field_get:xla.LiteralProto.sparse_indices) + return sparse_indices_.Get(index); +} +inline void LiteralProto::set_sparse_indices(int index, ::google::protobuf::int64 value) { + sparse_indices_.Set(index, value); + // @@protoc_insertion_point(field_set:xla.LiteralProto.sparse_indices) +} +inline void LiteralProto::add_sparse_indices(::google::protobuf::int64 value) { + sparse_indices_.Add(value); + // @@protoc_insertion_point(field_add:xla.LiteralProto.sparse_indices) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& +LiteralProto::sparse_indices() const { + // @@protoc_insertion_point(field_list:xla.LiteralProto.sparse_indices) + return sparse_indices_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* +LiteralProto::mutable_sparse_indices() { + // @@protoc_insertion_point(field_mutable_list:xla.LiteralProto.sparse_indices) + return &sparse_indices_; +} + +// ------------------------------------------------------------------- + +// WindowDimension + +// int64 size = 1; +inline void WindowDimension::clear_size() { + size_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 WindowDimension::size() const { + // @@protoc_insertion_point(field_get:xla.WindowDimension.size) + return size_; +} +inline void WindowDimension::set_size(::google::protobuf::int64 value) { + + size_ = value; + // @@protoc_insertion_point(field_set:xla.WindowDimension.size) +} + +// int64 stride = 2; +inline void WindowDimension::clear_stride() { + stride_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 WindowDimension::stride() const { + // @@protoc_insertion_point(field_get:xla.WindowDimension.stride) + return stride_; +} +inline void WindowDimension::set_stride(::google::protobuf::int64 value) { + + stride_ = value; + // @@protoc_insertion_point(field_set:xla.WindowDimension.stride) +} + +// int64 padding_low = 3; +inline void WindowDimension::clear_padding_low() { + padding_low_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 WindowDimension::padding_low() const { + // @@protoc_insertion_point(field_get:xla.WindowDimension.padding_low) + return padding_low_; +} +inline void WindowDimension::set_padding_low(::google::protobuf::int64 value) { + + padding_low_ = value; + // @@protoc_insertion_point(field_set:xla.WindowDimension.padding_low) +} + +// int64 padding_high = 4; +inline void WindowDimension::clear_padding_high() { + padding_high_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 WindowDimension::padding_high() const { + // @@protoc_insertion_point(field_get:xla.WindowDimension.padding_high) + return padding_high_; +} +inline void WindowDimension::set_padding_high(::google::protobuf::int64 value) { + + padding_high_ = value; + // @@protoc_insertion_point(field_set:xla.WindowDimension.padding_high) +} + +// int64 window_dilation = 5; +inline void WindowDimension::clear_window_dilation() { + window_dilation_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 WindowDimension::window_dilation() const { + // @@protoc_insertion_point(field_get:xla.WindowDimension.window_dilation) + return window_dilation_; +} +inline void WindowDimension::set_window_dilation(::google::protobuf::int64 value) { + + window_dilation_ = value; + // @@protoc_insertion_point(field_set:xla.WindowDimension.window_dilation) +} + +// int64 base_dilation = 6; +inline void WindowDimension::clear_base_dilation() { + base_dilation_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 WindowDimension::base_dilation() const { + // @@protoc_insertion_point(field_get:xla.WindowDimension.base_dilation) + return base_dilation_; +} +inline void WindowDimension::set_base_dilation(::google::protobuf::int64 value) { + + base_dilation_ = value; + // @@protoc_insertion_point(field_set:xla.WindowDimension.base_dilation) +} + +// bool window_reversal = 7; +inline void WindowDimension::clear_window_reversal() { + window_reversal_ = false; +} +inline bool WindowDimension::window_reversal() const { + // @@protoc_insertion_point(field_get:xla.WindowDimension.window_reversal) + return window_reversal_; +} +inline void WindowDimension::set_window_reversal(bool value) { + + window_reversal_ = value; + // @@protoc_insertion_point(field_set:xla.WindowDimension.window_reversal) +} + +// ------------------------------------------------------------------- + +// Window + +// repeated .xla.WindowDimension dimensions = 1; +inline int Window::dimensions_size() const { + return dimensions_.size(); +} +inline void Window::clear_dimensions() { + dimensions_.Clear(); +} +inline ::xla::WindowDimension* Window::mutable_dimensions(int index) { + // @@protoc_insertion_point(field_mutable:xla.Window.dimensions) + return dimensions_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::xla::WindowDimension >* +Window::mutable_dimensions() { + // @@protoc_insertion_point(field_mutable_list:xla.Window.dimensions) + return &dimensions_; +} +inline const ::xla::WindowDimension& Window::dimensions(int index) const { + // @@protoc_insertion_point(field_get:xla.Window.dimensions) + return dimensions_.Get(index); +} +inline ::xla::WindowDimension* Window::add_dimensions() { + // @@protoc_insertion_point(field_add:xla.Window.dimensions) + return dimensions_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::xla::WindowDimension >& +Window::dimensions() const { + // @@protoc_insertion_point(field_list:xla.Window.dimensions) + return dimensions_; +} + +// ------------------------------------------------------------------- + +// GatherDimensionNumbers + +// repeated int64 offset_dims = 1; +inline int GatherDimensionNumbers::offset_dims_size() const { + return offset_dims_.size(); +} +inline void GatherDimensionNumbers::clear_offset_dims() { + offset_dims_.Clear(); +} +inline ::google::protobuf::int64 GatherDimensionNumbers::offset_dims(int index) const { + // @@protoc_insertion_point(field_get:xla.GatherDimensionNumbers.offset_dims) + return offset_dims_.Get(index); +} +inline void GatherDimensionNumbers::set_offset_dims(int index, ::google::protobuf::int64 value) { + offset_dims_.Set(index, value); + // @@protoc_insertion_point(field_set:xla.GatherDimensionNumbers.offset_dims) +} +inline void GatherDimensionNumbers::add_offset_dims(::google::protobuf::int64 value) { + offset_dims_.Add(value); + // @@protoc_insertion_point(field_add:xla.GatherDimensionNumbers.offset_dims) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& +GatherDimensionNumbers::offset_dims() const { + // @@protoc_insertion_point(field_list:xla.GatherDimensionNumbers.offset_dims) + return offset_dims_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* +GatherDimensionNumbers::mutable_offset_dims() { + // @@protoc_insertion_point(field_mutable_list:xla.GatherDimensionNumbers.offset_dims) + return &offset_dims_; +} + +// repeated int64 collapsed_slice_dims = 2; +inline int GatherDimensionNumbers::collapsed_slice_dims_size() const { + return collapsed_slice_dims_.size(); +} +inline void GatherDimensionNumbers::clear_collapsed_slice_dims() { + collapsed_slice_dims_.Clear(); +} +inline ::google::protobuf::int64 GatherDimensionNumbers::collapsed_slice_dims(int index) const { + // @@protoc_insertion_point(field_get:xla.GatherDimensionNumbers.collapsed_slice_dims) + return collapsed_slice_dims_.Get(index); +} +inline void GatherDimensionNumbers::set_collapsed_slice_dims(int index, ::google::protobuf::int64 value) { + collapsed_slice_dims_.Set(index, value); + // @@protoc_insertion_point(field_set:xla.GatherDimensionNumbers.collapsed_slice_dims) +} +inline void GatherDimensionNumbers::add_collapsed_slice_dims(::google::protobuf::int64 value) { + collapsed_slice_dims_.Add(value); + // @@protoc_insertion_point(field_add:xla.GatherDimensionNumbers.collapsed_slice_dims) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& +GatherDimensionNumbers::collapsed_slice_dims() const { + // @@protoc_insertion_point(field_list:xla.GatherDimensionNumbers.collapsed_slice_dims) + return collapsed_slice_dims_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* +GatherDimensionNumbers::mutable_collapsed_slice_dims() { + // @@protoc_insertion_point(field_mutable_list:xla.GatherDimensionNumbers.collapsed_slice_dims) + return &collapsed_slice_dims_; +} + +// repeated int64 start_index_map = 3; +inline int GatherDimensionNumbers::start_index_map_size() const { + return start_index_map_.size(); +} +inline void GatherDimensionNumbers::clear_start_index_map() { + start_index_map_.Clear(); +} +inline ::google::protobuf::int64 GatherDimensionNumbers::start_index_map(int index) const { + // @@protoc_insertion_point(field_get:xla.GatherDimensionNumbers.start_index_map) + return start_index_map_.Get(index); +} +inline void GatherDimensionNumbers::set_start_index_map(int index, ::google::protobuf::int64 value) { + start_index_map_.Set(index, value); + // @@protoc_insertion_point(field_set:xla.GatherDimensionNumbers.start_index_map) +} +inline void GatherDimensionNumbers::add_start_index_map(::google::protobuf::int64 value) { + start_index_map_.Add(value); + // @@protoc_insertion_point(field_add:xla.GatherDimensionNumbers.start_index_map) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& +GatherDimensionNumbers::start_index_map() const { + // @@protoc_insertion_point(field_list:xla.GatherDimensionNumbers.start_index_map) + return start_index_map_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* +GatherDimensionNumbers::mutable_start_index_map() { + // @@protoc_insertion_point(field_mutable_list:xla.GatherDimensionNumbers.start_index_map) + return &start_index_map_; +} + +// int64 index_vector_dim = 4; +inline void GatherDimensionNumbers::clear_index_vector_dim() { + index_vector_dim_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 GatherDimensionNumbers::index_vector_dim() const { + // @@protoc_insertion_point(field_get:xla.GatherDimensionNumbers.index_vector_dim) + return index_vector_dim_; +} +inline void GatherDimensionNumbers::set_index_vector_dim(::google::protobuf::int64 value) { + + index_vector_dim_ = value; + // @@protoc_insertion_point(field_set:xla.GatherDimensionNumbers.index_vector_dim) +} + +// ------------------------------------------------------------------- + +// ScatterDimensionNumbers + +// repeated int64 update_window_dims = 1; +inline int ScatterDimensionNumbers::update_window_dims_size() const { + return update_window_dims_.size(); +} +inline void ScatterDimensionNumbers::clear_update_window_dims() { + update_window_dims_.Clear(); +} +inline ::google::protobuf::int64 ScatterDimensionNumbers::update_window_dims(int index) const { + // @@protoc_insertion_point(field_get:xla.ScatterDimensionNumbers.update_window_dims) + return update_window_dims_.Get(index); +} +inline void ScatterDimensionNumbers::set_update_window_dims(int index, ::google::protobuf::int64 value) { + update_window_dims_.Set(index, value); + // @@protoc_insertion_point(field_set:xla.ScatterDimensionNumbers.update_window_dims) +} +inline void ScatterDimensionNumbers::add_update_window_dims(::google::protobuf::int64 value) { + update_window_dims_.Add(value); + // @@protoc_insertion_point(field_add:xla.ScatterDimensionNumbers.update_window_dims) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& +ScatterDimensionNumbers::update_window_dims() const { + // @@protoc_insertion_point(field_list:xla.ScatterDimensionNumbers.update_window_dims) + return update_window_dims_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* +ScatterDimensionNumbers::mutable_update_window_dims() { + // @@protoc_insertion_point(field_mutable_list:xla.ScatterDimensionNumbers.update_window_dims) + return &update_window_dims_; +} + +// repeated int64 inserted_window_dims = 2; +inline int ScatterDimensionNumbers::inserted_window_dims_size() const { + return inserted_window_dims_.size(); +} +inline void ScatterDimensionNumbers::clear_inserted_window_dims() { + inserted_window_dims_.Clear(); +} +inline ::google::protobuf::int64 ScatterDimensionNumbers::inserted_window_dims(int index) const { + // @@protoc_insertion_point(field_get:xla.ScatterDimensionNumbers.inserted_window_dims) + return inserted_window_dims_.Get(index); +} +inline void ScatterDimensionNumbers::set_inserted_window_dims(int index, ::google::protobuf::int64 value) { + inserted_window_dims_.Set(index, value); + // @@protoc_insertion_point(field_set:xla.ScatterDimensionNumbers.inserted_window_dims) +} +inline void ScatterDimensionNumbers::add_inserted_window_dims(::google::protobuf::int64 value) { + inserted_window_dims_.Add(value); + // @@protoc_insertion_point(field_add:xla.ScatterDimensionNumbers.inserted_window_dims) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& +ScatterDimensionNumbers::inserted_window_dims() const { + // @@protoc_insertion_point(field_list:xla.ScatterDimensionNumbers.inserted_window_dims) + return inserted_window_dims_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* +ScatterDimensionNumbers::mutable_inserted_window_dims() { + // @@protoc_insertion_point(field_mutable_list:xla.ScatterDimensionNumbers.inserted_window_dims) + return &inserted_window_dims_; +} + +// repeated int64 scatter_dims_to_operand_dims = 3; +inline int ScatterDimensionNumbers::scatter_dims_to_operand_dims_size() const { + return scatter_dims_to_operand_dims_.size(); +} +inline void ScatterDimensionNumbers::clear_scatter_dims_to_operand_dims() { + scatter_dims_to_operand_dims_.Clear(); +} +inline ::google::protobuf::int64 ScatterDimensionNumbers::scatter_dims_to_operand_dims(int index) const { + // @@protoc_insertion_point(field_get:xla.ScatterDimensionNumbers.scatter_dims_to_operand_dims) + return scatter_dims_to_operand_dims_.Get(index); +} +inline void ScatterDimensionNumbers::set_scatter_dims_to_operand_dims(int index, ::google::protobuf::int64 value) { + scatter_dims_to_operand_dims_.Set(index, value); + // @@protoc_insertion_point(field_set:xla.ScatterDimensionNumbers.scatter_dims_to_operand_dims) +} +inline void ScatterDimensionNumbers::add_scatter_dims_to_operand_dims(::google::protobuf::int64 value) { + scatter_dims_to_operand_dims_.Add(value); + // @@protoc_insertion_point(field_add:xla.ScatterDimensionNumbers.scatter_dims_to_operand_dims) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& +ScatterDimensionNumbers::scatter_dims_to_operand_dims() const { + // @@protoc_insertion_point(field_list:xla.ScatterDimensionNumbers.scatter_dims_to_operand_dims) + return scatter_dims_to_operand_dims_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* +ScatterDimensionNumbers::mutable_scatter_dims_to_operand_dims() { + // @@protoc_insertion_point(field_mutable_list:xla.ScatterDimensionNumbers.scatter_dims_to_operand_dims) + return &scatter_dims_to_operand_dims_; +} + +// int64 index_vector_dim = 4; +inline void ScatterDimensionNumbers::clear_index_vector_dim() { + index_vector_dim_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 ScatterDimensionNumbers::index_vector_dim() const { + // @@protoc_insertion_point(field_get:xla.ScatterDimensionNumbers.index_vector_dim) + return index_vector_dim_; +} +inline void ScatterDimensionNumbers::set_index_vector_dim(::google::protobuf::int64 value) { + + index_vector_dim_ = value; + // @@protoc_insertion_point(field_set:xla.ScatterDimensionNumbers.index_vector_dim) +} + +// ------------------------------------------------------------------- + +// ConvolutionDimensionNumbers + +// int64 input_batch_dimension = 7; +inline void ConvolutionDimensionNumbers::clear_input_batch_dimension() { + input_batch_dimension_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 ConvolutionDimensionNumbers::input_batch_dimension() const { + // @@protoc_insertion_point(field_get:xla.ConvolutionDimensionNumbers.input_batch_dimension) + return input_batch_dimension_; +} +inline void ConvolutionDimensionNumbers::set_input_batch_dimension(::google::protobuf::int64 value) { + + input_batch_dimension_ = value; + // @@protoc_insertion_point(field_set:xla.ConvolutionDimensionNumbers.input_batch_dimension) +} + +// int64 input_feature_dimension = 8; +inline void ConvolutionDimensionNumbers::clear_input_feature_dimension() { + input_feature_dimension_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 ConvolutionDimensionNumbers::input_feature_dimension() const { + // @@protoc_insertion_point(field_get:xla.ConvolutionDimensionNumbers.input_feature_dimension) + return input_feature_dimension_; +} +inline void ConvolutionDimensionNumbers::set_input_feature_dimension(::google::protobuf::int64 value) { + + input_feature_dimension_ = value; + // @@protoc_insertion_point(field_set:xla.ConvolutionDimensionNumbers.input_feature_dimension) +} + +// repeated int64 input_spatial_dimensions = 11; +inline int ConvolutionDimensionNumbers::input_spatial_dimensions_size() const { + return input_spatial_dimensions_.size(); +} +inline void ConvolutionDimensionNumbers::clear_input_spatial_dimensions() { + input_spatial_dimensions_.Clear(); +} +inline ::google::protobuf::int64 ConvolutionDimensionNumbers::input_spatial_dimensions(int index) const { + // @@protoc_insertion_point(field_get:xla.ConvolutionDimensionNumbers.input_spatial_dimensions) + return input_spatial_dimensions_.Get(index); +} +inline void ConvolutionDimensionNumbers::set_input_spatial_dimensions(int index, ::google::protobuf::int64 value) { + input_spatial_dimensions_.Set(index, value); + // @@protoc_insertion_point(field_set:xla.ConvolutionDimensionNumbers.input_spatial_dimensions) +} +inline void ConvolutionDimensionNumbers::add_input_spatial_dimensions(::google::protobuf::int64 value) { + input_spatial_dimensions_.Add(value); + // @@protoc_insertion_point(field_add:xla.ConvolutionDimensionNumbers.input_spatial_dimensions) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& +ConvolutionDimensionNumbers::input_spatial_dimensions() const { + // @@protoc_insertion_point(field_list:xla.ConvolutionDimensionNumbers.input_spatial_dimensions) + return input_spatial_dimensions_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* +ConvolutionDimensionNumbers::mutable_input_spatial_dimensions() { + // @@protoc_insertion_point(field_mutable_list:xla.ConvolutionDimensionNumbers.input_spatial_dimensions) + return &input_spatial_dimensions_; +} + +// int64 kernel_input_feature_dimension = 3; +inline void ConvolutionDimensionNumbers::clear_kernel_input_feature_dimension() { + kernel_input_feature_dimension_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 ConvolutionDimensionNumbers::kernel_input_feature_dimension() const { + // @@protoc_insertion_point(field_get:xla.ConvolutionDimensionNumbers.kernel_input_feature_dimension) + return kernel_input_feature_dimension_; +} +inline void ConvolutionDimensionNumbers::set_kernel_input_feature_dimension(::google::protobuf::int64 value) { + + kernel_input_feature_dimension_ = value; + // @@protoc_insertion_point(field_set:xla.ConvolutionDimensionNumbers.kernel_input_feature_dimension) +} + +// int64 kernel_output_feature_dimension = 4; +inline void ConvolutionDimensionNumbers::clear_kernel_output_feature_dimension() { + kernel_output_feature_dimension_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 ConvolutionDimensionNumbers::kernel_output_feature_dimension() const { + // @@protoc_insertion_point(field_get:xla.ConvolutionDimensionNumbers.kernel_output_feature_dimension) + return kernel_output_feature_dimension_; +} +inline void ConvolutionDimensionNumbers::set_kernel_output_feature_dimension(::google::protobuf::int64 value) { + + kernel_output_feature_dimension_ = value; + // @@protoc_insertion_point(field_set:xla.ConvolutionDimensionNumbers.kernel_output_feature_dimension) +} + +// repeated int64 kernel_spatial_dimensions = 6; +inline int ConvolutionDimensionNumbers::kernel_spatial_dimensions_size() const { + return kernel_spatial_dimensions_.size(); +} +inline void ConvolutionDimensionNumbers::clear_kernel_spatial_dimensions() { + kernel_spatial_dimensions_.Clear(); +} +inline ::google::protobuf::int64 ConvolutionDimensionNumbers::kernel_spatial_dimensions(int index) const { + // @@protoc_insertion_point(field_get:xla.ConvolutionDimensionNumbers.kernel_spatial_dimensions) + return kernel_spatial_dimensions_.Get(index); +} +inline void ConvolutionDimensionNumbers::set_kernel_spatial_dimensions(int index, ::google::protobuf::int64 value) { + kernel_spatial_dimensions_.Set(index, value); + // @@protoc_insertion_point(field_set:xla.ConvolutionDimensionNumbers.kernel_spatial_dimensions) +} +inline void ConvolutionDimensionNumbers::add_kernel_spatial_dimensions(::google::protobuf::int64 value) { + kernel_spatial_dimensions_.Add(value); + // @@protoc_insertion_point(field_add:xla.ConvolutionDimensionNumbers.kernel_spatial_dimensions) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& +ConvolutionDimensionNumbers::kernel_spatial_dimensions() const { + // @@protoc_insertion_point(field_list:xla.ConvolutionDimensionNumbers.kernel_spatial_dimensions) + return kernel_spatial_dimensions_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* +ConvolutionDimensionNumbers::mutable_kernel_spatial_dimensions() { + // @@protoc_insertion_point(field_mutable_list:xla.ConvolutionDimensionNumbers.kernel_spatial_dimensions) + return &kernel_spatial_dimensions_; +} + +// int64 output_batch_dimension = 9; +inline void ConvolutionDimensionNumbers::clear_output_batch_dimension() { + output_batch_dimension_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 ConvolutionDimensionNumbers::output_batch_dimension() const { + // @@protoc_insertion_point(field_get:xla.ConvolutionDimensionNumbers.output_batch_dimension) + return output_batch_dimension_; +} +inline void ConvolutionDimensionNumbers::set_output_batch_dimension(::google::protobuf::int64 value) { + + output_batch_dimension_ = value; + // @@protoc_insertion_point(field_set:xla.ConvolutionDimensionNumbers.output_batch_dimension) +} + +// int64 output_feature_dimension = 10; +inline void ConvolutionDimensionNumbers::clear_output_feature_dimension() { + output_feature_dimension_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 ConvolutionDimensionNumbers::output_feature_dimension() const { + // @@protoc_insertion_point(field_get:xla.ConvolutionDimensionNumbers.output_feature_dimension) + return output_feature_dimension_; +} +inline void ConvolutionDimensionNumbers::set_output_feature_dimension(::google::protobuf::int64 value) { + + output_feature_dimension_ = value; + // @@protoc_insertion_point(field_set:xla.ConvolutionDimensionNumbers.output_feature_dimension) +} + +// repeated int64 output_spatial_dimensions = 12; +inline int ConvolutionDimensionNumbers::output_spatial_dimensions_size() const { + return output_spatial_dimensions_.size(); +} +inline void ConvolutionDimensionNumbers::clear_output_spatial_dimensions() { + output_spatial_dimensions_.Clear(); +} +inline ::google::protobuf::int64 ConvolutionDimensionNumbers::output_spatial_dimensions(int index) const { + // @@protoc_insertion_point(field_get:xla.ConvolutionDimensionNumbers.output_spatial_dimensions) + return output_spatial_dimensions_.Get(index); +} +inline void ConvolutionDimensionNumbers::set_output_spatial_dimensions(int index, ::google::protobuf::int64 value) { + output_spatial_dimensions_.Set(index, value); + // @@protoc_insertion_point(field_set:xla.ConvolutionDimensionNumbers.output_spatial_dimensions) +} +inline void ConvolutionDimensionNumbers::add_output_spatial_dimensions(::google::protobuf::int64 value) { + output_spatial_dimensions_.Add(value); + // @@protoc_insertion_point(field_add:xla.ConvolutionDimensionNumbers.output_spatial_dimensions) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& +ConvolutionDimensionNumbers::output_spatial_dimensions() const { + // @@protoc_insertion_point(field_list:xla.ConvolutionDimensionNumbers.output_spatial_dimensions) + return output_spatial_dimensions_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* +ConvolutionDimensionNumbers::mutable_output_spatial_dimensions() { + // @@protoc_insertion_point(field_mutable_list:xla.ConvolutionDimensionNumbers.output_spatial_dimensions) + return &output_spatial_dimensions_; +} + +// ------------------------------------------------------------------- + +// DotDimensionNumbers + +// repeated int64 lhs_contracting_dimensions = 1; +inline int DotDimensionNumbers::lhs_contracting_dimensions_size() const { + return lhs_contracting_dimensions_.size(); +} +inline void DotDimensionNumbers::clear_lhs_contracting_dimensions() { + lhs_contracting_dimensions_.Clear(); +} +inline ::google::protobuf::int64 DotDimensionNumbers::lhs_contracting_dimensions(int index) const { + // @@protoc_insertion_point(field_get:xla.DotDimensionNumbers.lhs_contracting_dimensions) + return lhs_contracting_dimensions_.Get(index); +} +inline void DotDimensionNumbers::set_lhs_contracting_dimensions(int index, ::google::protobuf::int64 value) { + lhs_contracting_dimensions_.Set(index, value); + // @@protoc_insertion_point(field_set:xla.DotDimensionNumbers.lhs_contracting_dimensions) +} +inline void DotDimensionNumbers::add_lhs_contracting_dimensions(::google::protobuf::int64 value) { + lhs_contracting_dimensions_.Add(value); + // @@protoc_insertion_point(field_add:xla.DotDimensionNumbers.lhs_contracting_dimensions) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& +DotDimensionNumbers::lhs_contracting_dimensions() const { + // @@protoc_insertion_point(field_list:xla.DotDimensionNumbers.lhs_contracting_dimensions) + return lhs_contracting_dimensions_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* +DotDimensionNumbers::mutable_lhs_contracting_dimensions() { + // @@protoc_insertion_point(field_mutable_list:xla.DotDimensionNumbers.lhs_contracting_dimensions) + return &lhs_contracting_dimensions_; +} + +// repeated int64 rhs_contracting_dimensions = 2; +inline int DotDimensionNumbers::rhs_contracting_dimensions_size() const { + return rhs_contracting_dimensions_.size(); +} +inline void DotDimensionNumbers::clear_rhs_contracting_dimensions() { + rhs_contracting_dimensions_.Clear(); +} +inline ::google::protobuf::int64 DotDimensionNumbers::rhs_contracting_dimensions(int index) const { + // @@protoc_insertion_point(field_get:xla.DotDimensionNumbers.rhs_contracting_dimensions) + return rhs_contracting_dimensions_.Get(index); +} +inline void DotDimensionNumbers::set_rhs_contracting_dimensions(int index, ::google::protobuf::int64 value) { + rhs_contracting_dimensions_.Set(index, value); + // @@protoc_insertion_point(field_set:xla.DotDimensionNumbers.rhs_contracting_dimensions) +} +inline void DotDimensionNumbers::add_rhs_contracting_dimensions(::google::protobuf::int64 value) { + rhs_contracting_dimensions_.Add(value); + // @@protoc_insertion_point(field_add:xla.DotDimensionNumbers.rhs_contracting_dimensions) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& +DotDimensionNumbers::rhs_contracting_dimensions() const { + // @@protoc_insertion_point(field_list:xla.DotDimensionNumbers.rhs_contracting_dimensions) + return rhs_contracting_dimensions_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* +DotDimensionNumbers::mutable_rhs_contracting_dimensions() { + // @@protoc_insertion_point(field_mutable_list:xla.DotDimensionNumbers.rhs_contracting_dimensions) + return &rhs_contracting_dimensions_; +} + +// repeated int64 lhs_batch_dimensions = 3; +inline int DotDimensionNumbers::lhs_batch_dimensions_size() const { + return lhs_batch_dimensions_.size(); +} +inline void DotDimensionNumbers::clear_lhs_batch_dimensions() { + lhs_batch_dimensions_.Clear(); +} +inline ::google::protobuf::int64 DotDimensionNumbers::lhs_batch_dimensions(int index) const { + // @@protoc_insertion_point(field_get:xla.DotDimensionNumbers.lhs_batch_dimensions) + return lhs_batch_dimensions_.Get(index); +} +inline void DotDimensionNumbers::set_lhs_batch_dimensions(int index, ::google::protobuf::int64 value) { + lhs_batch_dimensions_.Set(index, value); + // @@protoc_insertion_point(field_set:xla.DotDimensionNumbers.lhs_batch_dimensions) +} +inline void DotDimensionNumbers::add_lhs_batch_dimensions(::google::protobuf::int64 value) { + lhs_batch_dimensions_.Add(value); + // @@protoc_insertion_point(field_add:xla.DotDimensionNumbers.lhs_batch_dimensions) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& +DotDimensionNumbers::lhs_batch_dimensions() const { + // @@protoc_insertion_point(field_list:xla.DotDimensionNumbers.lhs_batch_dimensions) + return lhs_batch_dimensions_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* +DotDimensionNumbers::mutable_lhs_batch_dimensions() { + // @@protoc_insertion_point(field_mutable_list:xla.DotDimensionNumbers.lhs_batch_dimensions) + return &lhs_batch_dimensions_; +} + +// repeated int64 rhs_batch_dimensions = 4; +inline int DotDimensionNumbers::rhs_batch_dimensions_size() const { + return rhs_batch_dimensions_.size(); +} +inline void DotDimensionNumbers::clear_rhs_batch_dimensions() { + rhs_batch_dimensions_.Clear(); +} +inline ::google::protobuf::int64 DotDimensionNumbers::rhs_batch_dimensions(int index) const { + // @@protoc_insertion_point(field_get:xla.DotDimensionNumbers.rhs_batch_dimensions) + return rhs_batch_dimensions_.Get(index); +} +inline void DotDimensionNumbers::set_rhs_batch_dimensions(int index, ::google::protobuf::int64 value) { + rhs_batch_dimensions_.Set(index, value); + // @@protoc_insertion_point(field_set:xla.DotDimensionNumbers.rhs_batch_dimensions) +} +inline void DotDimensionNumbers::add_rhs_batch_dimensions(::google::protobuf::int64 value) { + rhs_batch_dimensions_.Add(value); + // @@protoc_insertion_point(field_add:xla.DotDimensionNumbers.rhs_batch_dimensions) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& +DotDimensionNumbers::rhs_batch_dimensions() const { + // @@protoc_insertion_point(field_list:xla.DotDimensionNumbers.rhs_batch_dimensions) + return rhs_batch_dimensions_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* +DotDimensionNumbers::mutable_rhs_batch_dimensions() { + // @@protoc_insertion_point(field_mutable_list:xla.DotDimensionNumbers.rhs_batch_dimensions) + return &rhs_batch_dimensions_; +} + +// ------------------------------------------------------------------- + +// OpSharding + +// .xla.OpSharding.Type type = 1; +inline void OpSharding::clear_type() { + type_ = 0; +} +inline ::xla::OpSharding_Type OpSharding::type() const { + // @@protoc_insertion_point(field_get:xla.OpSharding.type) + return static_cast< ::xla::OpSharding_Type >(type_); +} +inline void OpSharding::set_type(::xla::OpSharding_Type value) { + + type_ = value; + // @@protoc_insertion_point(field_set:xla.OpSharding.type) +} + +// .xla.Shape tile_shape = 2; +inline bool OpSharding::has_tile_shape() const { + return this != internal_default_instance() && tile_shape_ != NULL; +} +inline void OpSharding::clear_tile_shape() { + if (GetArenaNoVirtual() == NULL && tile_shape_ != NULL) { + delete tile_shape_; + } + tile_shape_ = NULL; +} +inline const ::xla::Shape& OpSharding::_internal_tile_shape() const { + return *tile_shape_; +} +inline const ::xla::Shape& OpSharding::tile_shape() const { + const ::xla::Shape* p = tile_shape_; + // @@protoc_insertion_point(field_get:xla.OpSharding.tile_shape) + return p != NULL ? *p : *reinterpret_cast( + &::xla::_Shape_default_instance_); +} +inline ::xla::Shape* OpSharding::release_tile_shape() { + // @@protoc_insertion_point(field_release:xla.OpSharding.tile_shape) + + ::xla::Shape* temp = tile_shape_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + tile_shape_ = NULL; + return temp; +} +inline ::xla::Shape* OpSharding::unsafe_arena_release_tile_shape() { + // @@protoc_insertion_point(field_unsafe_arena_release:xla.OpSharding.tile_shape) + + ::xla::Shape* temp = tile_shape_; + tile_shape_ = NULL; + return temp; +} +inline ::xla::Shape* OpSharding::mutable_tile_shape() { + + if (tile_shape_ == NULL) { + auto* p = CreateMaybeMessage<::xla::Shape>(GetArenaNoVirtual()); + tile_shape_ = p; + } + // @@protoc_insertion_point(field_mutable:xla.OpSharding.tile_shape) + return tile_shape_; +} +inline void OpSharding::set_allocated_tile_shape(::xla::Shape* tile_shape) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete tile_shape_; + } + if (tile_shape) { + ::google::protobuf::Arena* submessage_arena = + ::google::protobuf::Arena::GetArena(tile_shape); + if (message_arena != submessage_arena) { + tile_shape = ::google::protobuf::internal::GetOwnedMessage( + message_arena, tile_shape, submessage_arena); + } + + } else { + + } + tile_shape_ = tile_shape; + // @@protoc_insertion_point(field_set_allocated:xla.OpSharding.tile_shape) +} + +// repeated int64 tile_assignment_dimensions = 3; +inline int OpSharding::tile_assignment_dimensions_size() const { + return tile_assignment_dimensions_.size(); +} +inline void OpSharding::clear_tile_assignment_dimensions() { + tile_assignment_dimensions_.Clear(); +} +inline ::google::protobuf::int64 OpSharding::tile_assignment_dimensions(int index) const { + // @@protoc_insertion_point(field_get:xla.OpSharding.tile_assignment_dimensions) + return tile_assignment_dimensions_.Get(index); +} +inline void OpSharding::set_tile_assignment_dimensions(int index, ::google::protobuf::int64 value) { + tile_assignment_dimensions_.Set(index, value); + // @@protoc_insertion_point(field_set:xla.OpSharding.tile_assignment_dimensions) +} +inline void OpSharding::add_tile_assignment_dimensions(::google::protobuf::int64 value) { + tile_assignment_dimensions_.Add(value); + // @@protoc_insertion_point(field_add:xla.OpSharding.tile_assignment_dimensions) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& +OpSharding::tile_assignment_dimensions() const { + // @@protoc_insertion_point(field_list:xla.OpSharding.tile_assignment_dimensions) + return tile_assignment_dimensions_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* +OpSharding::mutable_tile_assignment_dimensions() { + // @@protoc_insertion_point(field_mutable_list:xla.OpSharding.tile_assignment_dimensions) + return &tile_assignment_dimensions_; +} + +// repeated int64 tile_assignment_devices = 4; +inline int OpSharding::tile_assignment_devices_size() const { + return tile_assignment_devices_.size(); +} +inline void OpSharding::clear_tile_assignment_devices() { + tile_assignment_devices_.Clear(); +} +inline ::google::protobuf::int64 OpSharding::tile_assignment_devices(int index) const { + // @@protoc_insertion_point(field_get:xla.OpSharding.tile_assignment_devices) + return tile_assignment_devices_.Get(index); +} +inline void OpSharding::set_tile_assignment_devices(int index, ::google::protobuf::int64 value) { + tile_assignment_devices_.Set(index, value); + // @@protoc_insertion_point(field_set:xla.OpSharding.tile_assignment_devices) +} +inline void OpSharding::add_tile_assignment_devices(::google::protobuf::int64 value) { + tile_assignment_devices_.Add(value); + // @@protoc_insertion_point(field_add:xla.OpSharding.tile_assignment_devices) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& +OpSharding::tile_assignment_devices() const { + // @@protoc_insertion_point(field_list:xla.OpSharding.tile_assignment_devices) + return tile_assignment_devices_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* +OpSharding::mutable_tile_assignment_devices() { + // @@protoc_insertion_point(field_mutable_list:xla.OpSharding.tile_assignment_devices) + return &tile_assignment_devices_; +} + +// repeated .xla.OpSharding tuple_shardings = 5; +inline int OpSharding::tuple_shardings_size() const { + return tuple_shardings_.size(); +} +inline void OpSharding::clear_tuple_shardings() { + tuple_shardings_.Clear(); +} +inline ::xla::OpSharding* OpSharding::mutable_tuple_shardings(int index) { + // @@protoc_insertion_point(field_mutable:xla.OpSharding.tuple_shardings) + return tuple_shardings_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::xla::OpSharding >* +OpSharding::mutable_tuple_shardings() { + // @@protoc_insertion_point(field_mutable_list:xla.OpSharding.tuple_shardings) + return &tuple_shardings_; +} +inline const ::xla::OpSharding& OpSharding::tuple_shardings(int index) const { + // @@protoc_insertion_point(field_get:xla.OpSharding.tuple_shardings) + return tuple_shardings_.Get(index); +} +inline ::xla::OpSharding* OpSharding::add_tuple_shardings() { + // @@protoc_insertion_point(field_add:xla.OpSharding.tuple_shardings) + return tuple_shardings_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::xla::OpSharding >& +OpSharding::tuple_shardings() const { + // @@protoc_insertion_point(field_list:xla.OpSharding.tuple_shardings) + return tuple_shardings_; +} + +// ------------------------------------------------------------------- + +// ReplicaGroup + +// repeated int64 replica_ids = 1; +inline int ReplicaGroup::replica_ids_size() const { + return replica_ids_.size(); +} +inline void ReplicaGroup::clear_replica_ids() { + replica_ids_.Clear(); +} +inline ::google::protobuf::int64 ReplicaGroup::replica_ids(int index) const { + // @@protoc_insertion_point(field_get:xla.ReplicaGroup.replica_ids) + return replica_ids_.Get(index); +} +inline void ReplicaGroup::set_replica_ids(int index, ::google::protobuf::int64 value) { + replica_ids_.Set(index, value); + // @@protoc_insertion_point(field_set:xla.ReplicaGroup.replica_ids) +} +inline void ReplicaGroup::add_replica_ids(::google::protobuf::int64 value) { + replica_ids_.Add(value); + // @@protoc_insertion_point(field_add:xla.ReplicaGroup.replica_ids) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& +ReplicaGroup::replica_ids() const { + // @@protoc_insertion_point(field_list:xla.ReplicaGroup.replica_ids) + return replica_ids_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* +ReplicaGroup::mutable_replica_ids() { + // @@protoc_insertion_point(field_mutable_list:xla.ReplicaGroup.replica_ids) + return &replica_ids_; +} + +// ------------------------------------------------------------------- + +// SourceTarget + +// int64 source = 1; +inline void SourceTarget::clear_source() { + source_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 SourceTarget::source() const { + // @@protoc_insertion_point(field_get:xla.SourceTarget.source) + return source_; +} +inline void SourceTarget::set_source(::google::protobuf::int64 value) { + + source_ = value; + // @@protoc_insertion_point(field_set:xla.SourceTarget.source) +} + +// int64 target = 2; +inline void SourceTarget::clear_target() { + target_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 SourceTarget::target() const { + // @@protoc_insertion_point(field_get:xla.SourceTarget.target) + return target_; +} +inline void SourceTarget::set_target(::google::protobuf::int64 value) { + + target_ = value; + // @@protoc_insertion_point(field_set:xla.SourceTarget.target) +} + +// ------------------------------------------------------------------- + +// PrecisionConfig + +// repeated .xla.PrecisionConfig.Precision operand_precision = 1; +inline int PrecisionConfig::operand_precision_size() const { + return operand_precision_.size(); +} +inline void PrecisionConfig::clear_operand_precision() { + operand_precision_.Clear(); +} +inline ::xla::PrecisionConfig_Precision PrecisionConfig::operand_precision(int index) const { + // @@protoc_insertion_point(field_get:xla.PrecisionConfig.operand_precision) + return static_cast< ::xla::PrecisionConfig_Precision >(operand_precision_.Get(index)); +} +inline void PrecisionConfig::set_operand_precision(int index, ::xla::PrecisionConfig_Precision value) { + operand_precision_.Set(index, value); + // @@protoc_insertion_point(field_set:xla.PrecisionConfig.operand_precision) +} +inline void PrecisionConfig::add_operand_precision(::xla::PrecisionConfig_Precision value) { + operand_precision_.Add(value); + // @@protoc_insertion_point(field_add:xla.PrecisionConfig.operand_precision) +} +inline const ::google::protobuf::RepeatedField& +PrecisionConfig::operand_precision() const { + // @@protoc_insertion_point(field_list:xla.PrecisionConfig.operand_precision) + return operand_precision_; +} +inline ::google::protobuf::RepeatedField* +PrecisionConfig::mutable_operand_precision() { + // @@protoc_insertion_point(field_mutable_list:xla.PrecisionConfig.operand_precision) + return &operand_precision_; +} + +#ifdef __GNUC__ + #pragma GCC diagnostic pop +#endif // __GNUC__ +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + + +// @@protoc_insertion_point(namespace_scope) + +} // namespace xla + +namespace google { +namespace protobuf { + +template <> struct is_proto_enum< ::xla::ChannelHandle_ChannelType> : ::std::true_type {}; +template <> +inline const EnumDescriptor* GetEnumDescriptor< ::xla::ChannelHandle_ChannelType>() { + return ::xla::ChannelHandle_ChannelType_descriptor(); +} +template <> struct is_proto_enum< ::xla::OpSharding_Type> : ::std::true_type {}; +template <> +inline const EnumDescriptor* GetEnumDescriptor< ::xla::OpSharding_Type>() { + return ::xla::OpSharding_Type_descriptor(); +} +template <> struct is_proto_enum< ::xla::PrecisionConfig_Precision> : ::std::true_type {}; +template <> +inline const EnumDescriptor* GetEnumDescriptor< ::xla::PrecisionConfig_Precision>() { + return ::xla::PrecisionConfig_Precision_descriptor(); +} +template <> struct is_proto_enum< ::xla::PrimitiveType> : ::std::true_type {}; +template <> +inline const EnumDescriptor* GetEnumDescriptor< ::xla::PrimitiveType>() { + return ::xla::PrimitiveType_descriptor(); +} +template <> struct is_proto_enum< ::xla::PaddingValue> : ::std::true_type {}; +template <> +inline const EnumDescriptor* GetEnumDescriptor< ::xla::PaddingValue>() { + return ::xla::PaddingValue_descriptor(); +} +template <> struct is_proto_enum< ::xla::Format> : ::std::true_type {}; +template <> +inline const EnumDescriptor* GetEnumDescriptor< ::xla::Format>() { + return ::xla::Format_descriptor(); +} +template <> struct is_proto_enum< ::xla::FftType> : ::std::true_type {}; +template <> +inline const EnumDescriptor* GetEnumDescriptor< ::xla::FftType>() { + return ::xla::FftType_descriptor(); +} +template <> struct is_proto_enum< ::xla::RandomDistribution> : ::std::true_type {}; +template <> +inline const EnumDescriptor* GetEnumDescriptor< ::xla::RandomDistribution>() { + return ::xla::RandomDistribution_descriptor(); +} + +} // namespace protobuf +} // namespace google + +// @@protoc_insertion_point(global_scope) + +#endif // PROTOBUF_INCLUDED_tensorflow_2fcompiler_2fxla_2fxla_5fdata_2eproto diff --git a/linux/include/tensorflow/contrib/cloud/kernels/bigquery_table_accessor.h b/linux/include/tensorflow/contrib/cloud/kernels/bigquery_table_accessor.h new file mode 100644 index 000000000..f1fcaff73 --- /dev/null +++ b/linux/include/tensorflow/contrib/cloud/kernels/bigquery_table_accessor.h @@ -0,0 +1,201 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CONTRIB_CLOUD_KERNELS_BIGQUERY_TABLE_ACCESSOR_H_ +#define TENSORFLOW_CONTRIB_CLOUD_KERNELS_BIGQUERY_TABLE_ACCESSOR_H_ + +#include +#include +#include + +#include "tensorflow/contrib/cloud/kernels/bigquery_table_partition.pb.h" +#include "tensorflow/core/example/example.pb.h" +#include "tensorflow/core/lib/core/errors.h" +#include "tensorflow/core/platform/cloud/curl_http_request.h" +#include "tensorflow/core/platform/cloud/google_auth_provider.h" + +namespace tensorflow { + +/// This class facilitates accessing BigQuery tables. +/// +/// Notes: +/// - Nested fields are not supported. +/// - BigQuery 'Record's are automatically flattened, +/// - BigQuery float type is a double but is converted to a C++ float in this +/// class. +/// - It is possible for a table snapshot to go out-of-scope in the BigQuery +/// service while accessing the table if a very old timestamp is used. For +/// exact details, see 'Table Decorators' in BigQuery docs. +class BigQueryTableAccessor { + public: + // Column types supported by BigQuery. + enum class ColumnType { + kString = 0, + kBytes, + kInteger, + kFloat, + kBoolean, + kTimestamp, + kDate, + kTime, + kDatetime, + kRecord, + kNone + }; + + /// \brief Creates a new BigQueryTableAccessor object. + // + // We do not allow relative (negative or zero) snapshot times here since we + // want to have a consistent snapshot of the table for the lifetime of this + // object. + // Use end_point if you want to connect to a different end point than the + // official BigQuery end point. Otherwise send an empty string. + static Status New(const string& project_id, const string& dataset_id, + const string& table_id, int64 timestamp_millis, + int64 row_buffer_size, const string& end_point, + const std::vector& columns, + const BigQueryTablePartition& partition, + std::unique_ptr* accessor); + + /// \brief Starts reading a new partition. + Status SetPartition(const BigQueryTablePartition& partition); + + /// \brief Returns true if there are more rows available in the current + /// partition. + bool Done(); + + /// \brief Returns a single row as example proto. + /// + /// This function will return an error if the table snapshot goes out of scope + /// in the BigQuery service. + Status ReadRow(int64* row_id, Example* example); + + /// \brief Returns total number of rows in the table. + int64 total_num_rows() { return total_num_rows_; } + + virtual ~BigQueryTableAccessor() {} + + private: + friend class BigQueryTableAccessorTest; + + // This struct encapsulates schema nodes for a BigQuery table. + struct SchemaNode { + SchemaNode() {} + SchemaNode(const string& name, ColumnType type) : name(name), type(type) {} + + string name; + ColumnType type; + std::vector schema_nodes; + }; + + /// If nullptr is passed for http_request_factory and auth_provider the + /// default production ones are used. This can be used by tests to override + /// these two variables. + static Status New(const string& project_id, const string& dataset_id, + const string& table_id, int64 timestamp_millis, + int64 row_buffer_size, const string& end_point, + const std::vector& columns, + const BigQueryTablePartition& partition, + std::unique_ptr auth_provider, + std::shared_ptr http_request_factory, + std::unique_ptr* accessor); + + /// \brief Constructs an object for a given table and partition. + BigQueryTableAccessor( + const string& project_id, const string& dataset_id, + const string& table_id, int64 timestamp_millis, int64 row_buffer_size, + const string& end_point, const std::vector& columns, + const BigQueryTablePartition& partition, + std::unique_ptr auth_provider, + std::shared_ptr http_request_factory); + + /// \brief Parses column values for a given row. + Status ParseColumnValues(const Json::Value& value, + const SchemaNode& root_schema_node, + Example* example); + + /// \brief Reads the table schema and stores it. + Status ReadSchema(); + + /// \brief Extracts column type from a column in schema. + Status ExtractColumnType(const Json::Value& columns, + const string& column_name_prefix, SchemaNode* root); + + /// \brief Appends a single BigQuery column Value to 'example' for a given + /// column. + Status AppendValueToExample(const string& column_name, + const Json::Value& column_value, + const BigQueryTableAccessor::ColumnType type, + Example* example); + + /// \brief Resets internal counters for reading a partition. + void Reset(); + + /// \brief Helper function that returns BigQuery http endpoint prefix. + string BigQueryUriPrefix(); + + /// \brief Computes the maxResults arg to send to BigQuery. + int64 ComputeMaxResultsArg(); + + /// \brief Returns full name of the underlying table name. + string FullTableName() { + return strings::StrCat(project_id_, ":", dataset_id_, ".", table_id_, "@", + timestamp_millis_); + } + + const string project_id_; + const string dataset_id_; + const string table_id_; + + // Snapshot timestamp. + const int64 timestamp_millis_; + + // Columns that should be read. Empty means all columns. + const std::set columns_; + + // HTTP address of BigQuery end point to use. + const string bigquery_end_point_; + + // Describes the portion of the table that we are currently accessing. + BigQueryTablePartition partition_; + + // Total number of rows in the underlying table. + int64 total_num_rows_ = 0; + + // Offset of the first row in the underlying row_buffer_. + int64 first_buffered_row_index_ = 0; + + // Offset of the next row in the row_buffer_. -1 indicates that this index + // is invalid. + int next_row_in_buffer_ = -1; + + // This buffer holds next rows to improve performance. Its size will be + // based on how much buffering was requested. + std::vector row_buffer_; + + // If next_page is set, it will used to read next batch of data. + string next_page_token_; + + // A tree representing the schema for the underlying table. + SchemaNode schema_root_; + + std::unique_ptr auth_provider_; + std::shared_ptr http_request_factory_; + + TF_DISALLOW_COPY_AND_ASSIGN(BigQueryTableAccessor); +}; + +} // namespace tensorflow +#endif // TENSORFLOW_CONTRIB_CLOUD_KERNELS_BIGQUERY_TABLE_ACCESSOR_H_ diff --git a/linux/include/tensorflow/contrib/cloud/kernels/bigquery_table_partition.pb.h b/linux/include/tensorflow/contrib/cloud/kernels/bigquery_table_partition.pb.h new file mode 100644 index 000000000..e066e9f56 --- /dev/null +++ b/linux/include/tensorflow/contrib/cloud/kernels/bigquery_table_partition.pb.h @@ -0,0 +1,219 @@ +// Generated by the protocol buffer compiler. DO NOT EDIT! +// source: tensorflow/contrib/cloud/kernels/bigquery_table_partition.proto + +#ifndef PROTOBUF_INCLUDED_tensorflow_2fcontrib_2fcloud_2fkernels_2fbigquery_5ftable_5fpartition_2eproto +#define PROTOBUF_INCLUDED_tensorflow_2fcontrib_2fcloud_2fkernels_2fbigquery_5ftable_5fpartition_2eproto + +#include + +#include + +#if GOOGLE_PROTOBUF_VERSION < 3006000 +#error This file was generated by a newer version of protoc which is +#error incompatible with your Protocol Buffer headers. Please update +#error your headers. +#endif +#if 3006000 < GOOGLE_PROTOBUF_MIN_PROTOC_VERSION +#error This file was generated by an older version of protoc which is +#error incompatible with your Protocol Buffer headers. Please +#error regenerate this file with a newer version of protoc. +#endif + +#include +#include +#include +#include +#include +#include +#include +#include +#include // IWYU pragma: export +#include // IWYU pragma: export +#include +// @@protoc_insertion_point(includes) +#define PROTOBUF_INTERNAL_EXPORT_protobuf_tensorflow_2fcontrib_2fcloud_2fkernels_2fbigquery_5ftable_5fpartition_2eproto + +namespace protobuf_tensorflow_2fcontrib_2fcloud_2fkernels_2fbigquery_5ftable_5fpartition_2eproto { +// Internal implementation detail -- do not use these members. +struct TableStruct { + static const ::google::protobuf::internal::ParseTableField entries[]; + static const ::google::protobuf::internal::AuxillaryParseTableField aux[]; + static const ::google::protobuf::internal::ParseTable schema[1]; + static const ::google::protobuf::internal::FieldMetadata field_metadata[]; + static const ::google::protobuf::internal::SerializationTable serialization_table[]; + static const ::google::protobuf::uint32 offsets[]; +}; +void AddDescriptors(); +} // namespace protobuf_tensorflow_2fcontrib_2fcloud_2fkernels_2fbigquery_5ftable_5fpartition_2eproto +namespace tensorflow { +class BigQueryTablePartition; +class BigQueryTablePartitionDefaultTypeInternal; +extern BigQueryTablePartitionDefaultTypeInternal _BigQueryTablePartition_default_instance_; +} // namespace tensorflow +namespace google { +namespace protobuf { +template<> ::tensorflow::BigQueryTablePartition* Arena::CreateMaybeMessage<::tensorflow::BigQueryTablePartition>(Arena*); +} // namespace protobuf +} // namespace google +namespace tensorflow { + +// =================================================================== + +class BigQueryTablePartition : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.BigQueryTablePartition) */ { + public: + BigQueryTablePartition(); + virtual ~BigQueryTablePartition(); + + BigQueryTablePartition(const BigQueryTablePartition& from); + + inline BigQueryTablePartition& operator=(const BigQueryTablePartition& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + BigQueryTablePartition(BigQueryTablePartition&& from) noexcept + : BigQueryTablePartition() { + *this = ::std::move(from); + } + + inline BigQueryTablePartition& operator=(BigQueryTablePartition&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + static const ::google::protobuf::Descriptor* descriptor(); + static const BigQueryTablePartition& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const BigQueryTablePartition* internal_default_instance() { + return reinterpret_cast( + &_BigQueryTablePartition_default_instance_); + } + static constexpr int kIndexInFileMessages = + 0; + + void Swap(BigQueryTablePartition* other); + friend void swap(BigQueryTablePartition& a, BigQueryTablePartition& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline BigQueryTablePartition* New() const final { + return CreateMaybeMessage(NULL); + } + + BigQueryTablePartition* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const BigQueryTablePartition& from); + void MergeFrom(const BigQueryTablePartition& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(BigQueryTablePartition* other); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return NULL; + } + inline void* MaybeArenaPtr() const { + return NULL; + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // int64 start_index = 1; + void clear_start_index(); + static const int kStartIndexFieldNumber = 1; + ::google::protobuf::int64 start_index() const; + void set_start_index(::google::protobuf::int64 value); + + // int64 end_index = 2; + void clear_end_index(); + static const int kEndIndexFieldNumber = 2; + ::google::protobuf::int64 end_index() const; + void set_end_index(::google::protobuf::int64 value); + + // @@protoc_insertion_point(class_scope:tensorflow.BigQueryTablePartition) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + ::google::protobuf::int64 start_index_; + ::google::protobuf::int64 end_index_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcontrib_2fcloud_2fkernels_2fbigquery_5ftable_5fpartition_2eproto::TableStruct; +}; +// =================================================================== + + +// =================================================================== + +#ifdef __GNUC__ + #pragma GCC diagnostic push + #pragma GCC diagnostic ignored "-Wstrict-aliasing" +#endif // __GNUC__ +// BigQueryTablePartition + +// int64 start_index = 1; +inline void BigQueryTablePartition::clear_start_index() { + start_index_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 BigQueryTablePartition::start_index() const { + // @@protoc_insertion_point(field_get:tensorflow.BigQueryTablePartition.start_index) + return start_index_; +} +inline void BigQueryTablePartition::set_start_index(::google::protobuf::int64 value) { + + start_index_ = value; + // @@protoc_insertion_point(field_set:tensorflow.BigQueryTablePartition.start_index) +} + +// int64 end_index = 2; +inline void BigQueryTablePartition::clear_end_index() { + end_index_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 BigQueryTablePartition::end_index() const { + // @@protoc_insertion_point(field_get:tensorflow.BigQueryTablePartition.end_index) + return end_index_; +} +inline void BigQueryTablePartition::set_end_index(::google::protobuf::int64 value) { + + end_index_ = value; + // @@protoc_insertion_point(field_set:tensorflow.BigQueryTablePartition.end_index) +} + +#ifdef __GNUC__ + #pragma GCC diagnostic pop +#endif // __GNUC__ + +// @@protoc_insertion_point(namespace_scope) + +} // namespace tensorflow + +// @@protoc_insertion_point(global_scope) + +#endif // PROTOBUF_INCLUDED_tensorflow_2fcontrib_2fcloud_2fkernels_2fbigquery_5ftable_5fpartition_2eproto diff --git a/linux/include/tensorflow/contrib/tensorboard/db/schema.h b/linux/include/tensorflow/contrib/tensorboard/db/schema.h new file mode 100644 index 000000000..3da450422 --- /dev/null +++ b/linux/include/tensorflow/contrib/tensorboard/db/schema.h @@ -0,0 +1,33 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CONTRIB_TENSORBOARD_DB_SCHEMA_H_ +#define TENSORFLOW_CONTRIB_TENSORBOARD_DB_SCHEMA_H_ + +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/lib/db/sqlite.h" + +namespace tensorflow { + +constexpr uint32 kTensorboardSqliteApplicationId = 0xfeedabee; + +/// \brief Creates TensorBoard SQLite tables and indexes. +/// +/// If they are already created, this has no effect. If schema +/// migrations are necessary, they will be performed with logging. +Status SetupTensorboardSqliteDb(Sqlite* db); + +} // namespace tensorflow + +#endif // TENSORFLOW_CONTRIB_TENSORBOARD_DB_SCHEMA_H_ diff --git a/linux/include/tensorflow/contrib/tensorboard/db/summary_converter.h b/linux/include/tensorflow/contrib/tensorboard/db/summary_converter.h new file mode 100644 index 000000000..329c7f9f2 --- /dev/null +++ b/linux/include/tensorflow/contrib/tensorboard/db/summary_converter.h @@ -0,0 +1,38 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CONTRIB_TENSORBOARD_DB_SUMMARY_CONVERTER_H_ +#define TENSORFLOW_CONTRIB_TENSORBOARD_DB_SUMMARY_CONVERTER_H_ + +#include "tensorflow/core/framework/summary.pb.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/lib/core/status.h" + +namespace tensorflow { + +// TODO(jart): Delete these methods in favor of new Python implementation. +Status AddTensorAsScalarToSummary(const Tensor& t, const string& tag, + Summary* s); +Status AddTensorAsHistogramToSummary(const Tensor& t, const string& tag, + Summary* s); +Status AddTensorAsImageToSummary(const Tensor& tensor, const string& tag, + int max_images, const Tensor& bad_color, + Summary* s); +Status AddTensorAsAudioToSummary(const Tensor& tensor, const string& tag, + int max_outputs, float sample_rate, + Summary* s); + +} // namespace tensorflow + +#endif // TENSORFLOW_CONTRIB_TENSORBOARD_DB_SUMMARY_CONVERTER_H_ diff --git a/linux/include/tensorflow/contrib/tensorboard/db/summary_db_writer.h b/linux/include/tensorflow/contrib/tensorboard/db/summary_db_writer.h new file mode 100644 index 000000000..746da1533 --- /dev/null +++ b/linux/include/tensorflow/contrib/tensorboard/db/summary_db_writer.h @@ -0,0 +1,42 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CONTRIB_TENSORBOARD_DB_SUMMARY_DB_WRITER_H_ +#define TENSORFLOW_CONTRIB_TENSORBOARD_DB_SUMMARY_DB_WRITER_H_ + +#include "tensorflow/core/kernels/summary_interface.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/lib/db/sqlite.h" +#include "tensorflow/core/platform/env.h" + +namespace tensorflow { + +/// \brief Creates SQLite SummaryWriterInterface. +/// +/// This can be used to write tensors from the execution graph directly +/// to a database. The schema must be created beforehand. Entries in +/// Users, Experiments, and Runs tables will be created automatically +/// if they don't already exist. +/// +/// Please note that the type signature of this function may change in +/// the future if support for other DBs is added to core. +/// +/// The result holds a new reference to db. +Status CreateSummaryDbWriter(Sqlite* db, const string& experiment_name, + const string& run_name, const string& user_name, + Env* env, SummaryWriterInterface** result); + +} // namespace tensorflow + +#endif // TENSORFLOW_CONTRIB_TENSORBOARD_DB_SUMMARY_DB_WRITER_H_ diff --git a/linux/include/tensorflow/contrib/tensorboard/db/summary_file_writer.h b/linux/include/tensorflow/contrib/tensorboard/db/summary_file_writer.h new file mode 100644 index 000000000..73b0a5542 --- /dev/null +++ b/linux/include/tensorflow/contrib/tensorboard/db/summary_file_writer.h @@ -0,0 +1,43 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CONTRIB_TENSORBOARD_DB_SUMMARY_FILE_WRITER_H_ +#define TENSORFLOW_CONTRIB_TENSORBOARD_DB_SUMMARY_FILE_WRITER_H_ + +#include "tensorflow/core/kernels/summary_interface.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/platform/env.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +/// \brief Creates SummaryWriterInterface which writes to a file. +/// +/// The file is an append-only records file of tf.Event protos. That +/// makes this summary writer suitable for file systems like GCS. +/// +/// It will enqueue up to max_queue summaries, and flush at least every +/// flush_millis milliseconds. The summaries will be written to the +/// directory specified by logdir and with the filename suffixed by +/// filename_suffix. The caller owns a reference to result if the +/// returned status is ok. The Env object must not be destroyed until +/// after the returned writer. +Status CreateSummaryFileWriter(int max_queue, int flush_millis, + const string& logdir, + const string& filename_suffix, Env* env, + SummaryWriterInterface** result); + +} // namespace tensorflow + +#endif // TENSORFLOW_CONTRIB_TENSORBOARD_DB_SUMMARY_FILE_WRITER_H_ diff --git a/linux/include/tensorflow/core/common_runtime/allocator_retry.h b/linux/include/tensorflow/core/common_runtime/allocator_retry.h new file mode 100644 index 000000000..9ac88f968 --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/allocator_retry.h @@ -0,0 +1,60 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_ALLOCATOR_RETRY_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_ALLOCATOR_RETRY_H_ + +#include "tensorflow/core/platform/env.h" +#include "tensorflow/core/platform/mutex.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +// A retrying wrapper for a memory allocator. +class AllocatorRetry { + public: + AllocatorRetry(); + + // Call 'alloc_func' to obtain memory. On first call, + // 'verbose_failure' will be false. If return value is nullptr, + // then wait up to 'max_millis_to_wait' milliseconds, retrying each + // time a call to DeallocateRaw() is detected, until either a good + // pointer is returned or the deadline is exhausted. If the + // deadline is exhausted, try one more time with 'verbose_failure' + // set to true. The value returned is either the first good pointer + // obtained from 'alloc_func' or nullptr. + void* AllocateRaw(std::function + alloc_func, + int max_millis_to_wait, size_t alignment, size_t bytes); + + // Called to notify clients that some memory was returned. + void NotifyDealloc(); + + private: + Env* env_; + mutex mu_; + condition_variable memory_returned_; +}; + +// Implementation details below +inline void AllocatorRetry::NotifyDealloc() { + mutex_lock l(mu_); + memory_returned_.notify_all(); +} + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_ALLOCATOR_RETRY_H_ diff --git a/linux/include/tensorflow/core/common_runtime/base_collective_executor.h b/linux/include/tensorflow/core/common_runtime/base_collective_executor.h new file mode 100644 index 000000000..360ce4db7 --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/base_collective_executor.h @@ -0,0 +1,143 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_BASE_COLLECTIVE_EXECUTOR_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_BASE_COLLECTIVE_EXECUTOR_H_ + +#include +#include + +#include "tensorflow/core/common_runtime/buf_rendezvous.h" +#include "tensorflow/core/framework/collective.h" +#include "tensorflow/core/framework/device_attributes.pb.h" + +namespace tensorflow { +class CollectiveImplementation; +class DeviceMgr; +class Device; + +// Helper interface that aliases regular subfields of a Tensor as separate +// Tensors for in-place update. +class CollectiveAdapter { + public: + virtual ~CollectiveAdapter() {} + + // Move the backing tensor to 'output' with its original storage and + // shape. After this call this CollectiveAdapter object should be + // deleted immediately without calling any of its other methods. + virtual void ConsumeFinalValue(Tensor* output) = 0; + + // const access to entire intermediate value for debugging + virtual const Tensor& Value() const = 0; + + // Returns tensor for chunk i which aliases the backing buffer. + virtual Tensor ChunkAlias(int i) = 0; + + // Returns tensor allocated on the same device but with its own + // separate backing buffer. Will have same type and size as + // chunk i. + virtual Tensor TempChunk(int i) const = 0; + + // Bytes in chunk i + virtual int64 ChunkBytes(int i) const = 0; + + // Generate a CPU RAM scalar tensor of the same DataType as the + // backing tensor with the given integer value. + virtual Tensor Scalar(int v) const = 0; + + // Generate a scalar tensor of same DataType and on the same device + // as the backing tensor. + virtual Tensor Scalar(Allocator* a) const = 0; + + // Debugging string describing buffer location + virtual string TBounds(const Tensor& t) const = 0; + + virtual string DebugString() const = 0; + + // Computes the number of elements per alias chunk tensor. + // + // A CHECK in tensor.cc expects that the memory buffer backing a + // Tensor will be aligned according to EIGEN_MAX_ALIGN_BYTES. To + // ensure that all chunk aliasing Tensors maintain this alignment we + // need to pick a chunk size that preserves it. Note than in extreme + // cases (impractical, but possible with very small tensors) one or + // more tail chunks can end up emptby. + static int64 AlignedChunkElts(int64 elt_bytes, int64 total_elts, + int64 num_chunks); +}; + +// Create a CollectiveAdaptor wrapping 'output', specialized to its +// data-type and shape. +CollectiveAdapter* MakeCollectiveAdapter(Tensor* output, int num_chunks, + Allocator* allocator); + +// Default implementation of CollectiveExecutor. Delegates the actual +// work of moving data to a class specialized for the operation type, +// arguments and device+interconnect topology. +class BaseCollectiveExecutor : public CollectiveExecutor { + public: + BaseCollectiveExecutor(CollectiveExecutorMgrInterface* cem, + PerStepCollectiveRemoteAccess* remote_access, + int64 step_id, const DeviceMgr* dev_mgr) + : CollectiveExecutor(cem), + step_id_(step_id), + dev_mgr_(dev_mgr), + remote_access_(remote_access) {} + + ~BaseCollectiveExecutor() override; + + void StartAbort(const Status& s) override; + + void ExecuteAsync(OpKernelContext* ctx, const CollectiveParams& col_params, + const string& exec_key, StatusCallback done) override; + + PerStepCollectiveRemoteAccess* remote_access() override { + return remote_access_.get(); + } + + void RecvFromPeer(const string& peer_device, const string& peer_task, + bool peer_is_local, const string& key, Device* to_device, + DeviceContext* to_device_ctx, + const AllocatorAttributes& to_alloc_attr, Tensor* to_tensor, + const DeviceLocality& client_locality, int stream_index, + const StatusCallback& done) override { + remote_access_->RecvFromPeer( + peer_device, peer_task, peer_is_local, key, to_device, to_device_ctx, + to_alloc_attr, to_tensor, client_locality, stream_index, done); + } + + void PostToPeer(const string& peer_device, const string& peer_task, + const string& key, Device* from_device, + DeviceContext* from_device_ctx, + const AllocatorAttributes& from_alloc_attr, + const Tensor* from_tensor, + const DeviceLocality& client_locality, + const StatusCallback& done) override { + remote_access_->PostToPeer(peer_device, peer_task, key, from_device, + from_device_ctx, from_alloc_attr, from_tensor, + client_locality, done); + } + + protected: + const int64 step_id_; + const DeviceMgr* dev_mgr_; // Not owned. + std::unique_ptr remote_access_; + + private: + Status CreateCollective(const CollectiveParams& col_params, + CollectiveImplementationInterface** col_impl); +}; + +} // namespace tensorflow +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_BASE_COLLECTIVE_EXECUTOR_H_ diff --git a/linux/include/tensorflow/core/common_runtime/bfc_allocator.h b/linux/include/tensorflow/core/common_runtime/bfc_allocator.h new file mode 100644 index 000000000..2d74bf2b2 --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/bfc_allocator.h @@ -0,0 +1,446 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_BFC_ALLOCATOR_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_BFC_ALLOCATOR_H_ + +#include +#include +#include +#include +#include + +#include "tensorflow/core/common_runtime/allocator_retry.h" +#include "tensorflow/core/framework/allocator.h" +#include "tensorflow/core/lib/gtl/stl_util.h" +#include "tensorflow/core/lib/strings/strcat.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/mutex.h" +#include "tensorflow/core/platform/thread_annotations.h" +#include "tensorflow/core/platform/types.h" +#include "tensorflow/core/protobuf/config.pb.h" + +namespace tensorflow { + +// A memory allocator that implements a 'best-fit with coalescing' +// algorithm. This is essentially a very simple version of Doug Lea's +// malloc (dlmalloc). +// +// The goal of this allocator is to support defragmentation via +// coalescing. One assumption we make is that the process using this +// allocator owns pretty much all of the memory, and that nearly +// all requests to allocate memory go through this interface. +class BFCAllocator : public Allocator { + public: + // Takes ownership of sub_allocator. + BFCAllocator(SubAllocator* sub_allocator, size_t total_memory, + bool allow_growth, const string& name); + ~BFCAllocator() override; + + string Name() override { return name_; } + void* AllocateRaw(size_t alignment, size_t num_bytes) override; + void* AllocateRaw(size_t alignment, size_t num_bytes, + const AllocationAttributes& allocation_attr) override; + void DeallocateRaw(void* ptr) override; + + bool TracksAllocationSizes() override; + + size_t RequestedSize(const void* ptr) override; + + size_t AllocatedSize(const void* ptr) override; + + int64 AllocationId(const void* ptr) override; + + void GetStats(AllocatorStats* stats) override; + + void ClearStats() override; + + private: + struct Bin; + + void* AllocateRawInternal(size_t alignment, size_t num_bytes, + bool dump_log_on_failure); + void DeallocateRawInternal(void* ptr); + + // A ChunkHandle is an index into the chunks_ vector in BFCAllocator + // kInvalidChunkHandle means an invalid chunk + typedef size_t ChunkHandle; + static const int kInvalidChunkHandle = -1; + + typedef int BinNum; + static const int kInvalidBinNum = -1; + static const int kNumBins = 21; + + // A Chunk points to a piece of memory that's either entirely free or entirely + // in use by one user memory allocation. + // + // An AllocationRegion's memory is split up into one or more disjoint Chunks, + // which together cover the whole region without gaps. Chunks participate in + // a doubly-linked list, and the prev/next pointers point to the physically + // adjacent chunks. + // + // Since a chunk cannot be partially in use, we may need to split a free chunk + // in order to service a user allocation. We always merge adjacent free + // chunks. + // + // Chunks contain information about whether they are in use or whether they + // are free, and contain a pointer to the bin they are in. + struct Chunk { + size_t size = 0; // Full size of buffer. + + // We sometimes give chunks that are larger than needed to reduce + // fragmentation. requested_size keeps track of what the client + // actually wanted so we can understand whether our splitting + // strategy is efficient. + size_t requested_size = 0; + + // allocation_id is set to -1 when the chunk is not in use. It is assigned a + // value greater than zero before the chunk is returned from + // AllocateRaw, and this value is unique among values assigned by + // the parent allocator. + int64 allocation_id = -1; + void* ptr = nullptr; // pointer to granted subbuffer. + + // If not kInvalidChunkHandle, the memory referred to by 'prev' is directly + // preceding the memory used by this chunk. E.g., It should start + // at 'ptr - prev->size' + ChunkHandle prev = kInvalidChunkHandle; + + // If not kInvalidChunkHandle, the memory referred to by 'next' is directly + // following the memory used by this chunk. E.g., It should be at + // 'ptr + size' + ChunkHandle next = kInvalidChunkHandle; + + // What bin are we in? + BinNum bin_num = kInvalidBinNum; + + bool in_use() const { return allocation_id != -1; } + + string DebugString(BFCAllocator* a, + bool recurse) NO_THREAD_SAFETY_ANALYSIS { + string dbg; + strings::StrAppend( + &dbg, " Size: ", strings::HumanReadableNumBytes(size), + " | Requested Size: ", strings::HumanReadableNumBytes(requested_size), + " | in_use: ", in_use()); + if (recurse && prev != BFCAllocator::kInvalidChunkHandle) { + Chunk* p = a->ChunkFromHandle(prev); + strings::StrAppend(&dbg, ", prev: ", p->DebugString(a, false)); + } + if (recurse && next != BFCAllocator::kInvalidChunkHandle) { + Chunk* n = a->ChunkFromHandle(next); + strings::StrAppend(&dbg, ", next: ", n->DebugString(a, false)); + } + return dbg; + } + }; + + // A Bin is a collection of similar-sized free chunks. + struct Bin { + // All chunks in this bin have >= bin_size memory. + size_t bin_size = 0; + + struct ChunkComparator { + explicit ChunkComparator(BFCAllocator* allocator) + : allocator_(allocator) {} + // Sort first by size and then use pointer address as a tie breaker. + bool operator()(const ChunkHandle ha, + const ChunkHandle hb) const NO_THREAD_SAFETY_ANALYSIS { + const Chunk* a = allocator_->ChunkFromHandle(ha); + const Chunk* b = allocator_->ChunkFromHandle(hb); + if (a->size != b->size) { + return a->size < b->size; + } + return a->ptr < b->ptr; + } + + private: + BFCAllocator* allocator_; // The parent allocator + }; + + typedef std::set FreeChunkSet; + // List of free chunks within the bin, sorted by chunk size. + // Chunk * not owned. + FreeChunkSet free_chunks; + Bin(BFCAllocator* allocator, size_t bs) + : bin_size(bs), free_chunks(ChunkComparator(allocator)) {} + }; + + static const size_t kMinAllocationBits = 8; + static const size_t kMinAllocationSize = 1 << kMinAllocationBits; + + // BFCAllocator allocates memory into a collection of disjoint + // AllocationRegions. Each AllocationRegion corresponds to one call to + // SubAllocator::Alloc(). + // + // An AllocationRegion contains one or more Chunks, covering all of its + // memory. Its primary job is to map a pointers to ChunkHandles. + // + // This class is thread-compatible. + class AllocationRegion { + public: + AllocationRegion(void* ptr, size_t memory_size) + : ptr_(ptr), + memory_size_(memory_size), + end_ptr_( + static_cast(static_cast(ptr_) + memory_size_)) { + DCHECK_EQ(0, memory_size % kMinAllocationSize); + const size_t n_handles = + (memory_size + kMinAllocationSize - 1) / kMinAllocationSize; + handles_.reset(new ChunkHandle[n_handles]); + for (size_t i = 0; i < n_handles; i++) { + handles_[i] = kInvalidChunkHandle; + } + } + + AllocationRegion() = default; + AllocationRegion(AllocationRegion&& other) { Swap(other); } + AllocationRegion& operator=(AllocationRegion&& other) { + Swap(other); + return *this; + } + + void* ptr() const { return ptr_; } + void* end_ptr() const { return end_ptr_; } + size_t memory_size() const { return memory_size_; } + ChunkHandle get_handle(const void* p) const { + return handles_[IndexFor(p)]; + } + void set_handle(const void* p, ChunkHandle h) { handles_[IndexFor(p)] = h; } + void erase(const void* p) { set_handle(p, kInvalidChunkHandle); } + + private: + void Swap(AllocationRegion& other) { + std::swap(ptr_, other.ptr_); + std::swap(memory_size_, other.memory_size_); + std::swap(end_ptr_, other.end_ptr_); + std::swap(handles_, other.handles_); + } + + int IndexFor(const void* p) const { + std::uintptr_t p_int = reinterpret_cast(p); + std::uintptr_t base_int = reinterpret_cast(ptr_); + DCHECK_GE(p_int, base_int); + DCHECK_LT(p_int, base_int + memory_size_); + return static_cast(((p_int - base_int) >> kMinAllocationBits)); + } + + // Metadata about the allocation region. + void* ptr_ = nullptr; + size_t memory_size_ = 0; + void* end_ptr_ = nullptr; + + // Array of size "memory_size / kMinAllocationSize". It is + // indexed by (p-base) / kMinAllocationSize, contains ChunkHandle + // for the memory allocation represented by "p" + std::unique_ptr handles_; + + TF_DISALLOW_COPY_AND_ASSIGN(AllocationRegion); + }; + + // RegionManager aggregates one or more "AllocationRegions" and provides + // a layer of indirection from pointers to the underlying ChunkHandle, + // allowing allocation across multiple discontiguous memory regions. + // + // This class is thread-compatible. + class RegionManager { + public: + RegionManager() {} + ~RegionManager() {} + + void AddAllocationRegion(void* ptr, size_t memory_size) { + // Insert sorted by end_ptr + auto entry = + std::upper_bound(regions_.begin(), regions_.end(), ptr, &Comparator); + regions_.insert(entry, AllocationRegion(ptr, memory_size)); + } + + ChunkHandle get_handle(const void* p) const { + return RegionFor(p)->get_handle(p); + } + + void set_handle(const void* p, ChunkHandle h) { + return MutableRegionFor(p)->set_handle(p, h); + } + void erase(const void* p) { return MutableRegionFor(p)->erase(p); } + + const std::vector& regions() const { return regions_; } + + private: + static bool Comparator(const void* ptr, const AllocationRegion& other) { + return ptr < other.end_ptr(); + } + + AllocationRegion* MutableRegionFor(const void* p) { + return const_cast(RegionFor(p)); + } + + const AllocationRegion* RegionFor(const void* p) const { + auto entry = + std::upper_bound(regions_.begin(), regions_.end(), p, &Comparator); + + if (entry != regions_.end()) { + return &(*entry); + } + + LOG(FATAL) << "Could not find Region for " << p; + return nullptr; + } + + private: + std::vector regions_; + }; + + // Returns 'bytes' rounded up to the next highest kMinAllocationSize. + static size_t RoundedBytes(size_t bytes); + + // Try to add a new memory region that can satisfy an allocation of + // 'rounded_bytes' bytes. Returns true on success and false on + // failure. + bool Extend(size_t alignment, size_t rounded_bytes) + EXCLUSIVE_LOCKS_REQUIRED(lock_); + + // Returns a pointer to an underlying allocated chunk of size + // 'rounded_bytes'. + void* FindChunkPtr(BinNum bin_num, size_t rounded_bytes, size_t num_bytes) + EXCLUSIVE_LOCKS_REQUIRED(lock_); + + // Splits the chunk specified by 'h' into two chunks, one at least + // of size 'num_bytes'. + void SplitChunk(ChunkHandle h, size_t num_bytes) + EXCLUSIVE_LOCKS_REQUIRED(lock_); + + // Merges the two chunk handles. Requires that the chunks are + // contiguous in their allocation. + void Merge(ChunkHandle h, ChunkHandle h2) EXCLUSIVE_LOCKS_REQUIRED(lock_); + + // Frees the memory represented by 'h', coalescing the chunk if + // possible. + void FreeAndMaybeCoalesce(ChunkHandle h) EXCLUSIVE_LOCKS_REQUIRED(lock_); + + // Adds the chunk 'h' to the proper free bin. + void InsertFreeChunkIntoBin(ChunkHandle h) EXCLUSIVE_LOCKS_REQUIRED(lock_); + + // Removes the free chunk pointed to by 'c' from the set free_chunks. + void RemoveFreeChunkIterFromBin(Bin::FreeChunkSet* free_chunks, + const Bin::FreeChunkSet::iterator& c) + EXCLUSIVE_LOCKS_REQUIRED(lock_); + + // Removes a free chunk from the bin. + void RemoveFreeChunkFromBin(ChunkHandle h) EXCLUSIVE_LOCKS_REQUIRED(lock_); + + // Removes the chunk metadata represented by 'h'. + void DeleteChunk(ChunkHandle h) EXCLUSIVE_LOCKS_REQUIRED(lock_); + + string RenderOccupancy() EXCLUSIVE_LOCKS_REQUIRED(lock_); + void DumpMemoryLog(size_t num_bytes) EXCLUSIVE_LOCKS_REQUIRED(lock_); + + ChunkHandle AllocateChunk() EXCLUSIVE_LOCKS_REQUIRED(lock_); + void DeallocateChunk(ChunkHandle h) EXCLUSIVE_LOCKS_REQUIRED(lock_); + + Chunk* ChunkFromHandle(ChunkHandle h) EXCLUSIVE_LOCKS_REQUIRED(lock_); + + // Information about a Bin that is useful for debugging. + struct BinDebugInfo { + size_t total_bytes_in_use = 0; + size_t total_bytes_in_bin = 0; + size_t total_requested_bytes_in_use = 0; + size_t total_chunks_in_use = 0; + size_t total_chunks_in_bin = 0; + }; + + // Computes and returns a BinDebugInfo for each Bin. + std::array get_bin_debug_info() + EXCLUSIVE_LOCKS_REQUIRED(lock_); + + AllocatorRetry retry_helper_; + + // Structures immutable after construction + size_t memory_limit_ = 0; + + inline int Log2FloorNonZeroSlow(uint64 n) { + int r = 0; + while (n > 0) { + r++; + n >>= 1; + } + return r - 1; + } + + // Returns floor(log2(n)). + inline int Log2FloorNonZero(uint64 n) { +#if defined(__GNUC__) + return 63 ^ __builtin_clzll(n); +#elif defined(PLATFORM_WINDOWS) && (_WIN64) + unsigned long index; + _BitScanReverse64(&index, n); + return index; +#else + return Log2FloorNonZeroSlow(n); +#endif + } + + // Map from bin size to Bin + Bin* BinFromIndex(BinNum index) { + return reinterpret_cast(&(bins_space_[index * sizeof(Bin)])); + } + size_t BinNumToSize(BinNum index) { + return static_cast(256) << index; + } + BinNum BinNumForSize(size_t bytes) { + uint64 v = std::max(bytes, 256) >> kMinAllocationBits; + int b = std::min(kNumBins - 1, Log2FloorNonZero(v)); + return b; + } + Bin* BinForSize(size_t bytes) { return BinFromIndex(BinNumForSize(bytes)); } + + char bins_space_[sizeof(Bin) * kNumBins]; + + // The size of the current region allocation. + size_t curr_region_allocation_bytes_; + + // The total number of allocated bytes by the allocator. + size_t total_region_allocated_bytes_ = 0; + + // An indicator that expansion of a region has hit the limits + // of the available memory. + bool started_backpedal_ = false; + + std::unique_ptr sub_allocator_; + string name_; + + // Structures mutable after construction + mutable mutex lock_; + RegionManager region_manager_ GUARDED_BY(lock_); + + std::vector chunks_ GUARDED_BY(lock_); + + // Pointer to head of linked list of free Chunks + ChunkHandle free_chunks_list_ GUARDED_BY(lock_); + + // Counter containing the next unique identifier to assign to a + // newly-created chunk. + int64 next_allocation_id_ GUARDED_BY(lock_); + + // Stats. + AllocatorStats stats_ GUARDED_BY(lock_); + + friend class GPUBFCAllocatorPrivateMethodsTest; + TF_DISALLOW_COPY_AND_ASSIGN(BFCAllocator); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_BFC_ALLOCATOR_H_ diff --git a/linux/include/tensorflow/core/common_runtime/buf_rendezvous.h b/linux/include/tensorflow/core/common_runtime/buf_rendezvous.h new file mode 100644 index 000000000..065bbd008 --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/buf_rendezvous.h @@ -0,0 +1,103 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_BUF_RENDEZVOUS_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_BUF_RENDEZVOUS_H_ + +#include +#include + +#include "tensorflow/core/framework/allocator.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/lib/gtl/flatmap.h" +#include "tensorflow/core/platform/mutex.h" + +namespace tensorflow { +class Device; +class DeviceContext; +class Tensor; + +// EXPERIMENTAL: RDMA oriented producer/consumer rendezvous on a local +// Tensor value for which DMAHelper::CanUseDMA() is true, i.e. dense +// numeric types. Similar to Rendezvous but never owns a Ref on the +// tensor, instead it uses an explicit callback to the producer when +// the consumer side is finished with the value. This allows the +// producer to perform in-place updates on the source buffer or to take +// other actions that depend on knowing the consumer has passed a certain +// execution point. +class BufRendezvous { + public: + explicit BufRendezvous(uint64 step_id) : step_id_(step_id) {} + + ~BufRendezvous(); + + // Inform all all waiting parties that this BufRendezvous is defunct + // because of an error Status interrupting the Step. + void StartAbort(const Status& s); + + struct Hook; + // Provided by the consumer to be called when access to the buffer + // is available. If the Status arg is not OK, then hook will not + // be populated. Ownership of Hook passes to consumer with the + // callback. + typedef std::function ConsumerCallback; + // Provided by the producer to be called when the consumer has finished + // reading the buffer and will no longer access it. + typedef std::function ProducerCallback; + + struct Hook { + Device* prod_dev; + DeviceContext* prod_ctx; + const Tensor* prod_value; + AllocatorAttributes prod_attr; + ProducerCallback prod_cb; + ConsumerCallback cons_cb; + Hook() + : prod_dev(nullptr), + prod_ctx(nullptr), + prod_value(nullptr), + prod_cb(nullptr), + cons_cb(nullptr) {} + string DebugString() const; + }; + + // Called to advertise availability of a Tensor value corresponding + // to key. That value must stay valid until done is called. + void ProvideBuf(const string& key, Device* dev, DeviceContext* dev_ctx, + const Tensor* v, const AllocatorAttributes& attr, + const ProducerCallback& done); + + // Called to request access to a Tensor value corresponding to key. + // Consumer is provide with a Hook as soon as available. + void ConsumeBuf(const string& key, const ConsumerCallback& done); + + // Consumer must call this function when it's done reading the Hook provided + // by the ConsumerCallback. This function will invoke the producer callback + // and then delete h. + static void DoneWithHook(Hook* h); + + // Write the current contents of the table to the INFO log. + void LogContents(); + + protected: + const uint64 step_id_; + mutex mu_; + Status status_ GUARDED_BY(mu_); + typedef gtl::FlatMap HookTable; + HookTable hook_table_ GUARDED_BY(mu_); + + void PurgeTable(const Status& s, HookTable* table); +}; +} // namespace tensorflow +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_BUF_RENDEZVOUS_H_ diff --git a/linux/include/tensorflow/core/common_runtime/build_graph_options.h b/linux/include/tensorflow/core/common_runtime/build_graph_options.h new file mode 100644 index 000000000..3d0f242ea --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/build_graph_options.h @@ -0,0 +1,42 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_BUILD_GRAPH_OPTIONS_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_BUILD_GRAPH_OPTIONS_H_ + +#include + +#include "tensorflow/core/platform/types.h" +#include "tensorflow/core/protobuf/config.pb.h" + +namespace tensorflow { + +struct BuildGraphOptions { + CallableOptions callable_options; + + // If `true`, uses Arg/Retval to implement feeds/fetches; otherwise + // uses Recv/Send to implement feeds/fetches. + // TODO(mrry): Remove this when the distributed runtime supports Arg/Retval. + bool use_function_convention = false; + + static const int64 kNoCollectiveGraphKey = 0; + int64 collective_graph_key = kNoCollectiveGraphKey; + + string DebugString() const; +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_BUILD_GRAPH_OPTIONS_H_ diff --git a/linux/include/tensorflow/core/common_runtime/collective_executor_mgr.h b/linux/include/tensorflow/core/common_runtime/collective_executor_mgr.h new file mode 100644 index 000000000..d53aca85b --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/collective_executor_mgr.h @@ -0,0 +1,75 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_COLLECTIVE_EXECUTOR_MGR_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_COLLECTIVE_EXECUTOR_MGR_H_ + +#include "tensorflow/core/framework/collective.h" +#include "tensorflow/core/lib/gtl/flatmap.h" + +namespace tensorflow { +class ConfigProto; +class DeviceMgr; + +class CollectiveExecutorMgr : public CollectiveExecutorMgrInterface { + public: + CollectiveExecutorMgr(const ConfigProto& config, const DeviceMgr* dev_mgr, + std::unique_ptr dev_resolver, + std::unique_ptr param_resolver); + + virtual ~CollectiveExecutorMgr(); + + CollectiveExecutor* FindOrCreate(int64 step_id) override; + + void Cleanup(int64 step_id) override; + + ParamResolverInterface* GetParamResolver() const override { + return param_resolver_.get(); + } + + DeviceResolverInterface* GetDeviceResolver() const override { + return dev_resolver_.get(); + } + + void GetStepSequenceAsync(const GetStepSequenceRequest* request, + GetStepSequenceResponse* response, + const StatusCallback& done) override; + + void RefreshStepIdSequenceAsync(int64 graph_key, + const StatusCallback& done) override; + + int64 NextStepId(int64 graph_key) override { + return CollectiveExecutor::kInvalidId; + } + + void RetireStepId(int64 graph_key, int64 step_id) override {} + + protected: + // Called by FindOrCreate when table entry does not yet exist. + virtual CollectiveExecutor* Create(int64 step_id); + + const DeviceMgr* dev_mgr_; + std::unique_ptr dev_resolver_; + std::unique_ptr param_resolver_; + CollectiveRemoteAccess* remote_access_; + string task_name_; + + private: + mutex exec_mu_; + // Map from step_id to CollectiveExecutor + gtl::FlatMap executor_table_ GUARDED_BY(exec_mu_); +}; + +} // namespace tensorflow +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_COLLECTIVE_EXECUTOR_MGR_H_ diff --git a/linux/include/tensorflow/core/common_runtime/collective_param_resolver_local.h b/linux/include/tensorflow/core/common_runtime/collective_param_resolver_local.h new file mode 100644 index 000000000..c5c3497e2 --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/collective_param_resolver_local.h @@ -0,0 +1,233 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_COLLECTIVE_PARAM_RESOLVER_LOCAL_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_COLLECTIVE_PARAM_RESOLVER_LOCAL_H_ + +#include +#include +#include +#include +#include + +#include "tensorflow/core/framework/collective.h" +#include "tensorflow/core/lib/gtl/flatmap.h" + +namespace tensorflow { +class CompleteGroupRequest; +class CompleteGroupResponse; +class CompleteInstanceRequest; +class CompleteInstanceResponse; +class DeviceMgr; + +// Implements ParamResolverInterface for a single-task context. +// It also implements the functionality necessary to serve as the +// group leader for param resolution in a multi-task context. +class CollectiveParamResolverLocal : public ParamResolverInterface { + public: + CollectiveParamResolverLocal(const DeviceMgr* dev_mgr, + DeviceResolverInterface* dev_resolver, + const string& task_name); + + ~CollectiveParamResolverLocal() override {} + + void CompleteParamsAsync(const string& device, CollectiveParams* cp, + CancellationManager* cancel_mgr, + const StatusCallback& done) override; + + void CompleteGroupAsync(const CompleteGroupRequest* request, + CompleteGroupResponse* response, + CancellationManager* cancel_mgr, + const StatusCallback& done) override; + + void CompleteInstanceAsync(const CompleteInstanceRequest* request, + CompleteInstanceResponse* response, + CancellationManager* cancel_mgr, + const StatusCallback& done) override; + + protected: + // Used to complete/verify CollGroup. + struct GroupRec { + CollGroupParams group; + mutable mutex mu; + Status status GUARDED_BY(mu); + std::set device_set GUARDED_BY(mu); + std::vector device_list GUARDED_BY(mu); + std::set task_set GUARDED_BY(mu); + std::vector task_list GUARDED_BY(mu); + std::vector waiting GUARDED_BY(mu); + }; + + // Finds the GroupRec that corresponds to cp->group_key. + // Also populates cp->group from that group_rec. + // Will wait until GroupRec is fully populated or an error arises before + // calling done. Callback GroupRec* arg is only valid if status is ok. + // Ownership of GroupRec stays with this object and does not pass to the + // callback. + typedef std::function + GroupRecCallback; + void CompleteGroupLocal(const string& device, CollectiveParams* cp, + const GroupRecCallback& done) + LOCKS_EXCLUDED(group_mu_); + + // Used to complete/verify CollInstance. + struct InstanceRec; + + typedef std::function IRConsumer; + struct InstanceRec { + // This structure has two mutexes so that a possibly long + // initialization can be done without holding the instance_mu_ + // table lock the whole time (which can cause an excessive number + // of threads to block on it), and because the compiler may not + // permit mutex locks to be taken in more than one order. + // + // out_mu guards access to most of the fields. + // in_mu guards access to a queue of consumer callbacks wanting to + // read the fields guarded by out_mu. + // + // The in_mu should be locked only while holding instance_mu_; the + // out_mu should be locked only while not holding + // instance_mu_. + // + // When is_init is false (the initial value) any potential user + // other than the creator should queue a callback on init_waiters. + // As soon as the shared member of this structure is fully + // initialized is_init will be set true and those callbacks will + // be invoked. + // + // Once inserted in the table this structure will never be replaced + // so users can capture the pointer while holding instance_mu_, + // drop that lock, then take a lock on out_mu before + // reading/modifying its values. + mutex in_mu; + bool is_init GUARDED_BY(in_mu); + std::vector init_waiters GUARDED_BY(in_mu); + + // A thread that wishes to acquire out_mu must ensure that it is available + // by invoking WaitForOutMu(). + mutex out_mu; + condition_variable out_cv; + bool out_mu_available GUARDED_BY(out_mu); + // Values to be shared by all instances, constant after initialization. + CollectiveParams shared GUARDED_BY(out_mu); + // If an error occurs during initialization this structure stays in + // the table with a non-OK status. Purging the table and restarting + // needs to be done at a higher level. + Status status GUARDED_BY(out_mu); + + // These fields are used to count the instances that have called + // in and become known while resolving broadcast source identity. + int source_rank GUARDED_BY(out_mu); + int known_count GUARDED_BY(out_mu); + std::vector known GUARDED_BY(out_mu); + std::vector known_waiters GUARDED_BY(out_mu); + + InstanceRec() + : is_init(false), + out_mu_available(true), + source_rank(-1), + known_count(0) {} + + // If out_mu is unavailable during distributed device locality + // initialization, wait on out_cv until it is available again. + void WaitForOutMu(mutex_lock& lock) EXCLUSIVE_LOCKS_REQUIRED(out_mu); + }; + + // Find the InstanceRec with the same instance_key as cp. If it doesn't + // already exist, create and initialize from gr and cp. + // + // Precondition: *gr must be a complete GroupRec, i.e. the value set + // by CompleteGroupLocal. *cp must be populated with all the fields + // required by InitInstanceSharedParams. Ownership of InstanceRec stays + // with this object and does not pass to the callback. + typedef std::function + InstanceRecCallback; + void FindInstanceRec(const GroupRec* gr, CollectiveParams* cp, + const InstanceRecCallback& done) + LOCKS_EXCLUDED(instance_mu_, gr->mu, group_mu_); + + // Populate *ir with device membership from gr, then initialize to be specific + // to cp->instance_key, i.e. order the devices and tasks. + // + // Preconditions: + // cp is populated with all DeviceLocalities + void InitInstanceSharedParams(const GroupRec* gr, const CollectiveParams* cp, + InstanceRec* ir, const StatusCallback& done) + UNLOCK_FUNCTION(ir->out_mu) LOCKS_EXCLUDED(gr->mu); + + void CallInitInstanceSharedParams(const GroupRec* gr, + const CollectiveParams* cp, InstanceRec* ir, + const InstanceRecCallback& done) + LOCKS_EXCLUDED(ir->out_mu, gr->mu); + + // Establishes the final order of ir->shared.instance.device_names and + // ir->shared.instance.task_names by considering localities of all devices. + void CompleteDefaultRanking(const GroupRec* gr, const CollectiveParams* cp, + InstanceRec* ir, + const std::vector& localities) + EXCLUSIVE_LOCKS_REQUIRED(ir->out_mu); + + // Finish populating *cp. + // Precondition: *gr has been fully populated by CompleteGroupLocal. + void CompleteInstanceLocal(const string& device, const GroupRec* gr, + CollectiveParams* cp, bool is_source, + const StatusCallback& done) + LOCKS_EXCLUDED(instance_mu_, gr->mu, group_mu_); + + // Finish populating *cp from fully initialized *ir. + // Precondition: *gr and *ir are fully populated. + void CompleteInstanceFromInitializedIRec(const string& device, + const GroupRec* gr, + CollectiveParams* cp, + InstanceRec* ir, bool is_source, + const StatusCallback& done) + LOCKS_EXCLUDED(ir->out_mu); + + // Complete source data for a broadcast instance. + // Precondition: *cp has complete group data and default_rank. + void CompleteInstanceSource(InstanceRec* ir, CollectiveParams* cp, + bool is_source, const IRConsumer& f) + LOCKS_EXCLUDED(ir->out_mu); + + // If cp.device_names contains only devices local to this process + // populates *localities, else returns an error. + Status GetLocalDeviceLocalities(const CollectiveParams& cp, + std::vector* localities); + + // Sets CollTaskParams.is_local and CollectiveParams.default_rank. + // Precondition: cp->device_names is fully populated and in final order. + void CompleteTaskIsLocal(const string& task_name, CollectiveParams* cp); + + // Sets cp->instance_default_rank according to location of device in + // current ordering of cp->instance.device_names. + void SetDefaultRank(const string& device, CollectiveParams* cp); + + // Helper to grab status under lock, invoke callback out of lock. + void CallbackWithStatus(const InstanceRecCallback& done, InstanceRec* irec) + LOCKS_EXCLUDED(irec->out_mu); + + const DeviceMgr* dev_mgr_; + DeviceResolverInterface* dev_resolver_; // Not owned. + string task_name_; + mutex group_mu_; + gtl::FlatMap> group_table_ + GUARDED_BY(group_mu_); + mutex instance_mu_; + gtl::FlatMap> instance_table_ + GUARDED_BY(instance_mu_); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_COLLECTIVE_PARAM_RESOLVER_LOCAL_H_ diff --git a/linux/include/tensorflow/core/common_runtime/collective_rma_local.h b/linux/include/tensorflow/core/common_runtime/collective_rma_local.h new file mode 100644 index 000000000..218808795 --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/collective_rma_local.h @@ -0,0 +1,92 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_COLLECTIVE_RMA_LOCAL_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_COLLECTIVE_RMA_LOCAL_H_ +#include "tensorflow/core/common_runtime/buf_rendezvous.h" +#include "tensorflow/core/common_runtime/device_mgr.h" +#include "tensorflow/core/framework/collective.h" +#include "tensorflow/core/framework/rendezvous.h" + +namespace tensorflow { + +// Basic implementation of PerStepCollectiveRemoteAccess. +class CollectiveRemoteAccessLocal : public PerStepCollectiveRemoteAccess { + public: + CollectiveRemoteAccessLocal(const DeviceMgr* dev_mgr, + DeviceResolverInterface* dev_resolver, + int64 step_id) + : dev_mgr_(dev_mgr), + dev_resolver_(dev_resolver), + buf_rendezvous_(step_id), + step_id_(step_id) {} + + virtual ~CollectiveRemoteAccessLocal() {} + + void StartAbort(const Status& s) override; + + void RecvFromPeer(const string& peer_device, const string& peer_task, + bool peer_is_local, const string& key, Device* to_device, + DeviceContext* to_device_ctx, + const AllocatorAttributes& to_alloc_attr, Tensor* to_tensor, + const DeviceLocality& client_locality, + int dev_to_dev_stream_index, + const StatusCallback& done) override; + + void PostToPeer(const string& peer_device, const string& peer_task, + const string& key, Device* from_device, + DeviceContext* from_device_ctx, + const AllocatorAttributes& from_alloc_attr, + const Tensor* from_tensor, + const DeviceLocality& client_locality, + const StatusCallback& done) override; + + void GetDeviceLocalitiesAsync(const CollInstanceParams& ci_params, + std::vector* localities, + const StatusCallback& done) override { + dev_resolver_->GetDeviceLocalitiesAsync(ci_params, localities, done); + } + + void GetLocalityAsync(const string& device, const string& task, + DeviceLocality* locality, + const StatusCallback& done) override { + dev_resolver_->GetLocalityAsync(device, task, locality, done); + } + + void ClearTask(const string& task) override { + dev_resolver_->ClearTask(task); + } + + BufRendezvous* buf_rendezvous() override { return &buf_rendezvous_; } + + // Copy utility that always copies bytes from src to dst even if + // they are on the same device, unlike CopyTensor::ViaDMA which will + // just change the dst buffer pointer in that case. + static void MemCpyAsync(DeviceContext* src_dev_ctx, + DeviceContext* dst_dev_ctx, Device* src_dev, + Device* dst_dev, const AllocatorAttributes& src_attr, + const AllocatorAttributes& dst_attr, + const Tensor* src, Tensor* dst, + int dev_to_dev_stream_index, + const StatusCallback& done); + + protected: + const DeviceMgr* dev_mgr_; // not owned + DeviceResolverInterface* dev_resolver_; // not owned + BufRendezvous buf_rendezvous_; + int64 step_id_; +}; + +} // namespace tensorflow +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_COLLECTIVE_RMA_LOCAL_H_ diff --git a/linux/include/tensorflow/core/common_runtime/collective_util.h b/linux/include/tensorflow/core/common_runtime/collective_util.h new file mode 100644 index 000000000..ebb5731be --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/collective_util.h @@ -0,0 +1,38 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_COLLECTIVE_UTIL_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_COLLECTIVE_UTIL_H_ + +#include + +#include "tensorflow/core/common_runtime/device.h" +#include "tensorflow/core/common_runtime/device_mgr.h" +#include "tensorflow/core/framework/collective.h" +#include "tensorflow/core/framework/device_attributes.pb.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/lib/core/status.h" + +namespace tensorflow { +namespace collective_util { + +Status InitializeDeviceAndLocality(const DeviceMgr* dev_mgr, + const string& device_name, Device** device, + DeviceLocality* device_locality); +string SubdivPermDebugString(const CollectiveParams& col_params); + +} // namespace collective_util +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_COLLECTIVE_UTIL_H_ diff --git a/linux/include/tensorflow/core/common_runtime/constant_folding.h b/linux/include/tensorflow/core/common_runtime/constant_folding.h new file mode 100644 index 000000000..a9a84f761 --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/constant_folding.h @@ -0,0 +1,69 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_CONSTANT_FOLDING_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_CONSTANT_FOLDING_H_ + +#include "tensorflow/core/common_runtime/device.h" +#include "tensorflow/core/framework/function.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/graph/graph.h" +#include "tensorflow/core/platform/env.h" + +// TODO(skyewm): can this be combined with EvaluateConstantTensor? + +namespace tensorflow { + +// This generator type is used to generate a name for the newly folded node +// based on the node's old name. +using ConstantFoldNameGenerator = + std::function; + +// Options specific to constant folding optimizations. +struct ConstantFoldingOptions { + // If "consider" is not a nullptr, then only constant fold a node "n" if + // consider(n) returns true. + std::function consider = nullptr; + // If shape_map is not a nullptr, it is a map from node n to a + // vector of the (potentially partially-known) shapes of its + // outputs. + const std::unordered_map>* shape_map = + nullptr; // not owned + // The maximum size of each constant created during constant folding + // optimization. + int64 max_constant_size_in_bytes = 10 * 1024 * 1024; + + // A generator for the name suffix of constant folded nodes. A + // default id generator that monotonically increases is used if nullptr is + // passed. + ConstantFoldNameGenerator generate_new_name = nullptr; +}; + +// Perform constant folding optimization on "graph". +// Looks for nodes in "graph" that can be completely evaluated statically, i.e., +// that are only dependent on constants. Evaluates those nodes on a CPU device +// and replaces those nodes with the result of the evaluation. +// "partition_device", if non-null, is the device where all the graph nodes are +// assumed to execute. +// Sets `was_mutated` to true if and only if "graph" has been mutated. +// The status is only set to a non-OK state if an unexpected error is hit +// running the graph. +Status ConstantFold(const ConstantFoldingOptions& opts, + FunctionLibraryRuntime* function_library, Env* env, + Device* partition_device, Graph* graph, bool* was_mutated); + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_CONSTANT_FOLDING_H_ diff --git a/linux/include/tensorflow/core/common_runtime/copy_tensor.h b/linux/include/tensorflow/core/common_runtime/copy_tensor.h new file mode 100644 index 000000000..9cd5ac2a3 --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/copy_tensor.h @@ -0,0 +1,73 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_COPY_TENSOR_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_COPY_TENSOR_H_ + +#include "tensorflow/core/common_runtime/device.h" +#include "tensorflow/core/framework/allocator.h" +#include "tensorflow/core/framework/device_base.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +class CopyTensor { + public: + typedef void (*CopyFunction)( + DeviceContext* send_dev_context, DeviceContext* recv_dev_context, + Device* src, Device* dst, const AllocatorAttributes src_alloc_attr, + const AllocatorAttributes dst_alloc_attr, const Tensor* input, + Tensor* output, int dev_to_dev_stream_index, StatusCallback done); + + // Copies "input" to "output" between devices accessible to the + // local process via some DMA-like method. "edge_name" is the name + // of the tensor being copied, for debugging purposes. Depending on + // the type of devices and memory in use, the copy may be performed + // synchronously or asynchronously. 'done' will be invoked only + // after the copy is actually complete. + static void ViaDMA(StringPiece edge_name, DeviceContext* send_dev_context, + DeviceContext* recv_dev_context, Device* src, Device* dst, + const AllocatorAttributes src_alloc_attr, + const AllocatorAttributes dst_alloc_attr, + const Tensor* input, Tensor* output, + int dev_to_dev_stream_index, StatusCallback done); + + // Object used to call Register() at static-initialization time. + // Note: This should only ever be used as a global-static object; no stack + // or heap instances. + class Registration { + public: + Registration(DeviceType sender_device_type, DeviceType receiver_device_type, + CopyFunction copy_function) { + TF_QCHECK_OK( + Register(sender_device_type, receiver_device_type, copy_function)); + } + }; + + private: + // Register a function for copying between two specific DeviceTypes. + // Note: This should only be called via the constructor of + // CopyTensor::Registration. + static Status Register(DeviceType sender_device_type, + DeviceType receiver_device_type, + CopyFunction copy_function); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_COPY_TENSOR_H_ diff --git a/linux/include/tensorflow/core/common_runtime/costmodel_manager.h b/linux/include/tensorflow/core/common_runtime/costmodel_manager.h new file mode 100644 index 000000000..e8ea2498b --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/costmodel_manager.h @@ -0,0 +1,55 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_COSTMODEL_MANAGER_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_COSTMODEL_MANAGER_H_ + +#include + +#include "tensorflow/core/framework/cost_graph.pb.h" +#include "tensorflow/core/graph/costmodel.h" +#include "tensorflow/core/graph/graph.h" +#include "tensorflow/core/lib/core/errors.h" +#include "tensorflow/core/lib/gtl/iterator_range.h" + +namespace tensorflow { + +// Used to manage all the cost models for a session. +class CostModelManager { + public: + ~CostModelManager(); + + typedef std::unordered_map CostModelMap; + typedef CostModelMap::iterator CostModelMapIter; + + void ExportCostModels(CostModelMap* cost_models) { + mutex_lock l(mu_); + *cost_models = cost_models_; + } + + CostModel* FindOrCreateCostModel(const Graph* graph); + + bool RemoveCostModelForGraph(const Graph* graph); + + Status AddToCostGraphDef(const Graph* graph, CostGraphDef* cost_graph); + + private: + mutex mu_; + CostModelMap cost_models_ GUARDED_BY(mu_); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_COSTMODEL_MANAGER_H_ diff --git a/linux/include/tensorflow/core/common_runtime/debugger_state_interface.h b/linux/include/tensorflow/core/common_runtime/debugger_state_interface.h new file mode 100644 index 000000000..797a0ade5 --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/debugger_state_interface.h @@ -0,0 +1,120 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_DEBUGGER_STATE_INTERFACE_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_DEBUGGER_STATE_INTERFACE_H_ + +#include + +#include "tensorflow/core/common_runtime/device.h" +#include "tensorflow/core/graph/graph.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/protobuf.h" +#include "tensorflow/core/protobuf/debug.pb.h" + +namespace tensorflow { + +// Returns a summary string for the list of debug tensor watches. +const string SummarizeDebugTensorWatches( + const protobuf::RepeatedPtrField& watches); + +// An abstract interface for storing and retrieving debugging information. +class DebuggerStateInterface { + public: + virtual ~DebuggerStateInterface() {} + + // Publish metadata about the debugged Session::Run() call. + // + // Args: + // global_step: A global step count supplied by the caller of + // Session::Run(). + // session_run_index: A chronologically sorted index for calls to the Run() + // method of the Session object. + // executor_step_index: A chronologically sorted index of invocations of the + // executor charged to serve this Session::Run() call. + // input_names: Name of the input Tensors (feed keys). + // output_names: Names of the fetched Tensors. + // target_names: Names of the target nodes. + virtual Status PublishDebugMetadata( + const int64 global_step, const int64 session_run_index, + const int64 executor_step_index, const std::vector& input_names, + const std::vector& output_names, + const std::vector& target_nodes) = 0; +}; + +class DebugGraphDecoratorInterface { + public: + virtual ~DebugGraphDecoratorInterface() {} + + // Insert special-purpose debug nodes to graph and dump the graph for + // record. See the documentation of DebugNodeInserter::InsertNodes() for + // details. + virtual Status DecorateGraph(Graph* graph, Device* device) = 0; + + // Publish Graph to debug URLs. + virtual Status PublishGraph(const Graph& graph, + const string& device_name) = 0; +}; + +typedef std::function( + const DebugOptions& options)> + DebuggerStateFactory; + +// Contains only static methods for registering DebuggerStateFactory. +// We don't expect to create any instances of this class. +// Call DebuggerStateRegistry::RegisterFactory() at initialization time to +// define a global factory that creates instances of DebuggerState, then call +// DebuggerStateRegistry::CreateState() to create a single instance. +class DebuggerStateRegistry { + public: + // Registers a function that creates a concrete DebuggerStateInterface + // implementation based on DebugOptions. + static void RegisterFactory(const DebuggerStateFactory& factory); + + // If RegisterFactory() has been called, creates and supplies a concrete + // DebuggerStateInterface implementation using the registered factory, + // owned by the caller and return an OK Status. Otherwise returns an error + // Status. + static Status CreateState(const DebugOptions& debug_options, + std::unique_ptr* state); + + private: + static DebuggerStateFactory* factory_; + + TF_DISALLOW_COPY_AND_ASSIGN(DebuggerStateRegistry); +}; + +typedef std::function( + const DebugOptions& options)> + DebugGraphDecoratorFactory; + +class DebugGraphDecoratorRegistry { + public: + static void RegisterFactory(const DebugGraphDecoratorFactory& factory); + + static Status CreateDecorator( + const DebugOptions& options, + std::unique_ptr* decorator); + + private: + static DebugGraphDecoratorFactory* factory_; + + TF_DISALLOW_COPY_AND_ASSIGN(DebugGraphDecoratorRegistry); +}; + +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_DEBUGGER_STATE_INTERFACE_H_ diff --git a/linux/include/tensorflow/core/common_runtime/device.h b/linux/include/tensorflow/core/common_runtime/device.h new file mode 100644 index 000000000..2ef1547cd --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/device.h @@ -0,0 +1,196 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// A Device is a something that can perform computations as part of a +// model. Devices can be local (runs computation on this machine), or +// remote (contacts a device local to another machine using an RPC to +// do the work). Devices are registered in a DeviceSet, which is also +// responsible for the Device <-> id mapping. +// +// Device names +// * Every Device should have a unique name with the format: +// /job:___/replica:___/task:___/(gpu|cpu):___ +// An example name would be "/job:train/replica:0/task:3/device:GPU:2". +// * Task numbers are within the specified replica, so there are as +// many "task zeros" as replicas. + +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_DEVICE_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_DEVICE_H_ + +#include +#include + +#include "tensorflow/core/framework/allocator.h" +#include "tensorflow/core/framework/control_flow.h" +#include "tensorflow/core/framework/device_attributes.pb_text.h" +#include "tensorflow/core/framework/device_attributes.pb.h" +#include "tensorflow/core/framework/device_base.h" +#include "tensorflow/core/framework/graph.pb.h" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/op_segment.h" +#include "tensorflow/core/framework/resource_mgr.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/graph/graph.h" +#include "tensorflow/core/graph/types.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/types.h" +#include "tensorflow/core/util/device_name_utils.h" + +namespace tensorflow { + +class DeviceMgr; + +class Device : public DeviceBase { + public: + Device(Env* env, const DeviceAttributes& device_attributes); + ~Device() override; + + // Full name of this device (see top comment). + const string& name() const override { return device_attributes_.name(); } + + // Parsed name of this device + const DeviceNameUtils::ParsedName& parsed_name() const { + return parsed_name_; + } + + // Describes what kind of device this is. This is intended to be + // human-readable and not computer-parsed, except that two devices + // with the same device_type() are expected to perform similarly + // (both from a computation and communication perspective). + const string& device_type() const { return device_attributes_.device_type(); } + + // Returns an aggregation of device attributes. + const DeviceAttributes& attributes() const override { + return device_attributes_; + } + + // Performs the actual compute function. + // + // Subclasses may override this function if they wish to perform + // some initialization before each compute. + virtual void Compute(OpKernel* op_kernel, OpKernelContext* context) { + op_kernel->Compute(context); + } + + // Asynchronous kernel's compute. + virtual void ComputeAsync(AsyncOpKernel* op_kernel, OpKernelContext* context, + AsyncOpKernel::DoneCallback done) { + op_kernel->ComputeAsync(context, std::move(done)); + } + + // Takes ownership of the references in tensors. If necessary, a + // device may override this method to keep a reference to the + // accessed tensors until the async computation has completed. + virtual void ConsumeListOfAccessedTensors( + DeviceContext* context, const TensorReferenceVector& tensors) { + for (const auto& ref : tensors) { + ref.Unref(); + } + } + + // If true, and tracing is enabled, the `tracing::ScopedAnnotation()` tracing + // mechanism will be used instead of `tracing::ScopedActivity()`. Some devices + // may override this method to use annotations, which enable child activities + // (such as GPU kernel launches) to be related to the OpKernel invocation. + virtual bool TraceUsingAnnotations() const { return false; } + + // Blocks until all operations queued on the device at the time of + // the call have completed. Returns any error pending on the device + // at completion. + virtual Status Sync() = 0; + + // Override this to return true for devices that require a Sync() call before + // session completion. + virtual bool RequiresSyncOnCompletion() const { return false; } + + // Optionally modify the device's GraphDef before execution. + // + // This method should be considered experimental and is supplied to enable + // prototyping of TensorFlow device implementations that need to modify + // the GraphDef before execution. + // + // 'graph' supplies the partition of the graph assigned to this + // device. + virtual Status MaybeRewriteGraph(std::unique_ptr* /*graph*/) { + return Status::OK(); + } + + // Fill in the context map for the graph. Default behavior is to do + // nothing. + // + // The caller takes ownership over the DeviceContext objects given + // by the device. + virtual Status FillContextMap(const Graph* graph, + DeviceContextMap* device_context_map) { + return Status::OK(); + } + + // Returns the op segment of this device. The caller can reuse op + // kernels registered for the same session running on this device. + OpSegment* op_segment() { return &op_seg_; } + + // Returns the resource manager associated w/ this device. + virtual ResourceMgr* resource_manager() { return rmgr_; } + + // Returns the device manager that owns this device, or nullptr if this Device + // is not owned by a device manager. + DeviceMgr* device_mgr() const { return device_mgr_; } + + // Summarizes the status of this Device, for debugging. + string DebugString() const { return ProtoDebugString(device_attributes_); } + + // Assembles the parameter components into a complete DeviceAttributes value. + static DeviceAttributes BuildDeviceAttributes( + const string& name, DeviceType device, Bytes memory_limit, + const DeviceLocality& locality, const string& physical_device_desc); + + static DeviceAttributes BuildDeviceAttributes( + const string& name, DeviceType device, Bytes memory_limit, + const DeviceLocality& locality) { + // Pass in an empty string as physical device name. + return BuildDeviceAttributes(name, device, memory_limit, locality, ""); + } + + // Clears the resource manager associated with this device. + void ClearResourceMgr() { rmgr_->Clear(); } + + protected: + void DeleteResourceMgr() { + delete rmgr_; + rmgr_ = nullptr; + } + + private: + friend class DeviceMgr; + + // Pointer to the device manager that owns this device. Not owned. + DeviceMgr* device_mgr_ = nullptr; + + const DeviceAttributes device_attributes_; + DeviceNameUtils::ParsedName parsed_name_; + + // op_seg_ maps session handle and op name to OpKernel objects. + OpSegment op_seg_; + + // Resources associated w/ this device. E.g., shared variables, etc. + ResourceMgr* rmgr_ = nullptr; + + TF_DISALLOW_COPY_AND_ASSIGN(Device); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_DEVICE_H_ diff --git a/linux/include/tensorflow/core/common_runtime/device_factory.h b/linux/include/tensorflow/core/common_runtime/device_factory.h new file mode 100644 index 000000000..db50226fe --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/device_factory.h @@ -0,0 +1,129 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_DEVICE_FACTORY_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_DEVICE_FACTORY_H_ + +#include +#include + +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +class Device; +struct SessionOptions; + +class DeviceFactory { + public: + virtual ~DeviceFactory() {} + static void Register(const string& device_type, DeviceFactory* factory, + int priority); + static DeviceFactory* GetFactory(const string& device_type); + + // Append to "*devices" all suitable devices, respecting + // any device type specific properties/counts listed in "options". + // + // CPU devices are added first. + static Status AddDevices(const SessionOptions& options, + const string& name_prefix, + std::vector* devices); + + // Helper for tests. Create a single device of type "type". The + // returned device is always numbered zero, so if creating multiple + // devices of the same type, supply distinct name_prefix arguments. + static Device* NewDevice(const string& type, const SessionOptions& options, + const string& name_prefix); + + // Most clients should call AddDevices() instead. + virtual Status CreateDevices(const SessionOptions& options, + const string& name_prefix, + std::vector* devices) = 0; + + // Return the device priority number for a "device_type" string. + // + // Higher number implies higher priority. + // + // In standard TensorFlow distributions, GPU device types are + // preferred over CPU, and by default, custom devices that don't set + // a custom priority during registration will be prioritized lower + // than CPU. Custom devices that want a higher priority can set the + // 'priority' field when registering their device to something + // higher than the packaged devices. See calls to + // REGISTER_LOCAL_DEVICE_FACTORY to see the existing priorities used + // for built-in devices. + static int32 DevicePriority(const string& device_type); +}; + +namespace dfactory { + +template +class Registrar { + public: + // Multiple registrations for the same device type with different priorities + // are allowed. Priorities are used in two different ways: + // + // 1) When choosing which factory (that is, which device + // implementation) to use for a specific 'device_type', the + // factory registered with the highest priority will be chosen. + // For example, if there are two registrations: + // + // Registrar("CPU", 125); + // Registrar("CPU", 150); + // + // then CPUFactory2 will be chosen when + // DeviceFactory::GetFactory("CPU") is called. + // + // 2) When choosing which 'device_type' is preferred over other + // DeviceTypes in a DeviceSet, the ordering is determined + // by the 'priority' set during registration. For example, if there + // are two registrations: + // + // Registrar("CPU", 100); + // Registrar("GPU", 200); + // + // then DeviceType("GPU") will be prioritized higher than + // DeviceType("CPU"). + // + // The default priority values for built-in devices is: + // GPU: 210 + // SYCL: 200 + // GPUCompatibleCPU: 70 + // ThreadPoolDevice: 60 + // Default: 50 + explicit Registrar(const string& device_type, int priority = 50) { + DeviceFactory::Register(device_type, new Factory(), priority); + } +}; + +} // namespace dfactory + +#define REGISTER_LOCAL_DEVICE_FACTORY(device_type, device_factory, ...) \ + INTERNAL_REGISTER_LOCAL_DEVICE_FACTORY(device_type, device_factory, \ + __COUNTER__, ##__VA_ARGS__) + +#define INTERNAL_REGISTER_LOCAL_DEVICE_FACTORY(device_type, device_factory, \ + ctr, ...) \ + static ::tensorflow::dfactory::Registrar \ + INTERNAL_REGISTER_LOCAL_DEVICE_FACTORY_NAME(ctr)(device_type, \ + ##__VA_ARGS__) + +// __COUNTER__ must go through another macro to be properly expanded +#define INTERNAL_REGISTER_LOCAL_DEVICE_FACTORY_NAME(ctr) ___##ctr##__object_ + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_DEVICE_FACTORY_H_ diff --git a/linux/include/tensorflow/core/common_runtime/device_mgr.h b/linux/include/tensorflow/core/common_runtime/device_mgr.h new file mode 100644 index 000000000..c1ff10d9b --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/device_mgr.h @@ -0,0 +1,80 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_DEVICE_MGR_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_DEVICE_MGR_H_ + +#include +#include +#include +#include + +#include "tensorflow/core/common_runtime/device.h" +#include "tensorflow/core/lib/core/arena.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/lib/core/stringpiece.h" +#include "tensorflow/core/lib/gtl/inlined_vector.h" +#include "tensorflow/core/platform/macros.h" + +namespace tensorflow { + +class DeviceAttributes; + +class DeviceMgr { + public: + // Takes ownership of each device in 'devices'. + // TODO(zhifengc): Other initialization information. + // TODO(b/37437134): Use std::unique_ptr's to track ownership. + explicit DeviceMgr(const std::vector& devices); + ~DeviceMgr(); + + // Returns attributes of all devices. + void ListDeviceAttributes(std::vector* devices) const; + + std::vector ListDevices() const; + + // Returns a string listing all devices. + string DebugString() const; + + // Returns a string of all the device mapping. + string DeviceMappingString() const; + + // Assigns *device with pointer to Device of the given name. + // Accepts either a full device name, or just the replica-local suffix. + Status LookupDevice(StringPiece name, Device** device) const; + + // Clears given containers of all devices if 'container' is + // non-empty. Otherwise, clears default containers of all devices. + void ClearContainers(gtl::ArraySlice containers) const; + + int NumDeviceType(const string& type) const; + + private: + // TODO(b/37437134): Use std::unique_ptr's to track ownership. + typedef gtl::InlinedVector DeviceVec; + DeviceVec devices_; + + StringPiece CopyToBackingStore(StringPiece s); + + std::unordered_map device_map_; + core::Arena name_backing_store_; // Storage for keys in device_map_ + std::unordered_map device_type_counts_; + + TF_DISALLOW_COPY_AND_ASSIGN(DeviceMgr); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_DEVICE_MGR_H_ diff --git a/linux/include/tensorflow/core/common_runtime/device_resolver_local.h b/linux/include/tensorflow/core/common_runtime/device_resolver_local.h new file mode 100644 index 000000000..bb6ff2efa --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/device_resolver_local.h @@ -0,0 +1,48 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_DEVICE_RESOLVER_LOCAL_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_DEVICE_RESOLVER_LOCAL_H_ + +#include + +#include "tensorflow/core/framework/collective.h" +#include "tensorflow/core/framework/device_attributes.pb.h" + +namespace tensorflow { +class DeviceMgr; + +// Implements DeviceResolverInterface in a single-task context. +class DeviceResolverLocal : public DeviceResolverInterface { + public: + DeviceResolverLocal(const DeviceMgr* dev_mgr) : dev_mgr_(dev_mgr) {} + + virtual ~DeviceResolverLocal() {} + + void GetDeviceLocalitiesAsync(const CollInstanceParams& ci_params, + std::vector* localities, + const StatusCallback& done) override; + + void GetLocalityAsync(const string& device, const string& task, + DeviceLocality* locality, + const StatusCallback& done) override; + + void ClearTask(const string& task) override {} + + protected: + const DeviceMgr* dev_mgr_; +}; + +} // namespace tensorflow +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_DEVICE_RESOLVER_LOCAL_H_ diff --git a/linux/include/tensorflow/core/common_runtime/device_set.h b/linux/include/tensorflow/core/common_runtime/device_set.h new file mode 100644 index 000000000..c384d46e9 --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/device_set.h @@ -0,0 +1,89 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_DEVICE_SET_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_DEVICE_SET_H_ + +#include +#include +#include + +#include "tensorflow/core/common_runtime/device.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/types.h" +#include "tensorflow/core/util/device_name_utils.h" + +namespace tensorflow { + +// DeviceSet is a container class for managing the various types of +// devices used by a model. +class DeviceSet { + public: + DeviceSet(); + ~DeviceSet(); + + // Does not take ownership of 'device'. + void AddDevice(Device* device); + + // Set the device designated as the "client". This device + // must also be registered via AddDevice(). + void set_client_device(Device* device) { + DCHECK(client_device_ == nullptr); + client_device_ = device; + } + + // Returns a pointer to the device designated as the "client". + Device* client_device() const { return client_device_; } + + // Return the list of devices in this set. + const std::vector& devices() const { return devices_; } + + // Given a DeviceNameUtils::ParsedName (which may have some + // wildcards for different components), fills "*devices" with all + // devices in "*this" that match "spec". + void FindMatchingDevices(const DeviceNameUtils::ParsedName& spec, + std::vector* devices) const; + + // Finds the device with the given "fullname". Returns nullptr if + // not found. + Device* FindDeviceByName(const string& fullname) const; + + // Return the list of unique device types in this set, ordered + // with more preferable devices earlier. + std::vector PrioritizedDeviceTypeList() const; + + // An order to sort by device types according to system-determined + // priority. + // + // Higher result implies higher priority. + static int DeviceTypeOrder(const DeviceType& d); + + private: + // Not owned. + std::vector devices_; + + // Fullname -> device* for device in devices_. + std::unordered_map device_by_name_; + + // client_device_ points to an element of devices_ that we consider + // to be the client device (in this local process). + Device* client_device_ = nullptr; + + TF_DISALLOW_COPY_AND_ASSIGN(DeviceSet); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_DEVICE_SET_H_ diff --git a/linux/include/tensorflow/core/common_runtime/direct_session.h b/linux/include/tensorflow/core/common_runtime/direct_session.h new file mode 100644 index 000000000..3a168bbe3 --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/direct_session.h @@ -0,0 +1,411 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_DIRECT_SESSION_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_DIRECT_SESSION_H_ + +#include +#include +#include +#include +#include +#include + +#include "tensorflow/core/common_runtime/costmodel_manager.h" +#include "tensorflow/core/common_runtime/debugger_state_interface.h" +#include "tensorflow/core/common_runtime/device_mgr.h" +#include "tensorflow/core/common_runtime/device_set.h" +#include "tensorflow/core/common_runtime/executor.h" +#include "tensorflow/core/common_runtime/graph_execution_state.h" +#include "tensorflow/core/common_runtime/process_function_library_runtime.h" +#include "tensorflow/core/common_runtime/rendezvous_mgr.h" +#include "tensorflow/core/common_runtime/session_factory.h" +#include "tensorflow/core/framework/cancellation.h" +#include "tensorflow/core/framework/collective.h" +#include "tensorflow/core/framework/graph.pb.h" +#include "tensorflow/core/framework/session_state.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/lib/core/errors.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/mutex.h" +#include "tensorflow/core/platform/thread_annotations.h" +#include "tensorflow/core/platform/types.h" +#include "tensorflow/core/public/session.h" + +namespace tensorflow { + +class CostModel; +class DebugGateway; +class Device; +class DirectSessionFactory; + +class DirectSession : public Session { + public: + typedef std::function CloseCallback; + + // Takes ownership of 'device_mgr'. + // 'factory' is used to unregister the DirectSession with 'factory' when its + // closed. This ensures that Reset requests from the 'factory' don't get sent + // to sessions that are already closed. + DirectSession(const SessionOptions& options, const DeviceMgr* device_mgr, + DirectSessionFactory* factory); + ~DirectSession() override; + + typedef std::vector> NamedTensorList; + typedef std::unordered_map NameNodeMap; + + ::tensorflow::Status Create(const GraphDef& graph) override; + ::tensorflow::Status Extend(const GraphDef& graph) override; + ::tensorflow::Status Run(const NamedTensorList& inputs, + const std::vector& output_names, + const std::vector& target_nodes, + std::vector* outputs) override; + + // NOTE: Experimental and subject to change. + ::tensorflow::Status Run(const ::tensorflow::RunOptions& run_options, + const NamedTensorList& inputs, + const std::vector& output_names, + const std::vector& target_nodes, + std::vector* outputs, + RunMetadata* run_metadata) override; + + // NOTE: PRunSetup and PRun are added to support partial execution. This + // feature is experimental and subject to change. + ::tensorflow::Status PRunSetup(const std::vector& input_names, + const std::vector& output_names, + const std::vector& target_nodes, + string* handle) override; + ::tensorflow::Status PRun(const string& handle, const NamedTensorList& inputs, + const std::vector& output_names, + std::vector* outputs) override; + + // Reset clears 'containers' from the device_mgr of the DirectSession. + // If 'containers' is empty, then Reset clears the default container. + ::tensorflow::Status Reset(const std::vector& containers); + + ::tensorflow::Status ListDevices( + std::vector* response) override; + ::tensorflow::Status Close() override; + ::tensorflow::Status LocalDeviceManager(const DeviceMgr** output) override { + *output = device_mgr_.get(); + return ::tensorflow::Status::OK(); + } + + void ExportCostModels(CostModelManager::CostModelMap* cost_models) { + cost_model_manager_.ExportCostModels(cost_models); + } + + ::tensorflow::Status MakeCallable(const CallableOptions& callable_options, + CallableHandle* out_handle) override; + ::tensorflow::Status RunCallable(CallableHandle handle, + const std::vector& feed_tensors, + std::vector* fetch_tensors, + RunMetadata* run_metadata) override; + ::tensorflow::Status ReleaseCallable(CallableHandle handle) override; + + private: + // For access to collective_graph_key_. + friend class DirectSessionCollectiveTest; + + // We create one executor and its dependent library runtime for + // every partition. + struct PerPartitionExecutorsAndLib { + Graph* graph = nullptr; // not owned. + Device* device = nullptr; // not owned. + FunctionLibraryRuntime* flib = nullptr; // not owned. + std::unique_ptr executor; + }; + + // An ExecutorsAndKeys is created for a given set of feeds/fetches. + // 'step_count' is the number of times this graph is executed. + // 'graph' is the entire graph being executed. 'name_to_node' + // maps node name to node. We keep 'graph' and 'name_to_node' only in + // the case of partial runs. Each item in 'items' is the executor for + // a partition of the graph bundled with its dependent library runtime. + // 'input_keys' are the rendezvous keys for the feeds and 'output_keys' + // are rendezvous keys for the fetches. + struct ExecutorsAndKeys { + ExecutorsAndKeys() : step_count(0) {} + + std::atomic_int_fast64_t step_count; + std::unique_ptr graph; + NameNodeMap name_to_node; + std::vector items; + std::unordered_map input_name_to_index; + std::unordered_map input_name_to_rendezvous_key; + std::unordered_map output_name_to_index; + std::unordered_map output_name_to_rendezvous_key; + + DataTypeVector input_types; + DataTypeVector output_types; + + CallableOptions callable_options; + + int64 collective_graph_key = BuildGraphOptions::kNoCollectiveGraphKey; + }; + + // A FunctionInfo object is created for every unique set of feeds/fetches. + // This info could be folded into the ExecutorsAndKeys object but we would + // like to maintain a deletion order in which the OpKernels (owned by the + // executor) should be destroyed first, followed by the resources in the + // device and then followed by the function stuff. + // TODO(rohanj): Consolidate function library definitions so that we can + // instantiate only one ProcFLR and lib_def and make this just a member + // variable and not a vector. + // 'flib_def' is the function library used. + // 'proc_flr' is the collection of FunctionLibraryRuntime objects, one per + // device. + struct FunctionInfo { + std::unique_ptr flib_def; + std::unique_ptr proc_flr; + }; + + // For each live partial execution, the session maintains a RunState. + // 'status' is the current status of this partial execution. 'executor_done' + // is "notified" when all executors are done. 'pending_inputs' are the set + // of pending feeds and 'pending_outputs' are the set of pending fetches. + struct RunState { + mutex mu_; + Status status GUARDED_BY(mu_); + IntraProcessRendezvous* rendez = nullptr; + std::unique_ptr collective_executor; + std::unique_ptr collector; + Notification executors_done; + std::unordered_map pending_inputs; // true if fed + std::unordered_map pending_outputs; // true if fetched + TensorStore tensor_store; + ScopedStepContainer step_container; + + RunState(int64 step_id, const std::vector* devices); + + RunState(const std::vector& pending_input_names, + const std::vector& pending_output_names, int64 step_id, + const std::vector* devices); + + // Returns true if all pending inputs and outputs have been completed. + bool PendingDone() const; + + ~RunState(); + }; + + struct RunStateArgs { + RunStateArgs(const DebugOptions& options) : debug_options(options) {} + + bool is_partial_run = false; + string handle; + std::unique_ptr graph; + const DebugOptions& debug_options; + int64 collective_graph_key = BuildGraphOptions::kNoCollectiveGraphKey; + }; + + // Initializes the base execution state given the 'graph', + // if not already initialized. + Status MaybeInitializeExecutionState(const GraphDef& graph, + bool* out_already_initialized) + EXCLUSIVE_LOCKS_REQUIRED(graph_state_lock_); + + // Retrieves an already existing set of executors to run 'inputs' and + // 'outputs', or creates and caches them for future use. + ::tensorflow::Status GetOrCreateExecutors( + gtl::ArraySlice inputs, gtl::ArraySlice outputs, + gtl::ArraySlice target_nodes, + ExecutorsAndKeys** executors_and_keys, RunStateArgs* run_state_args); + + // Creates a set of executors to run the subgraph defined by + // `callable_options`. + ::tensorflow::Status CreateExecutors( + const CallableOptions& callable_options, + std::unique_ptr* out_executors_and_keys, + std::unique_ptr* out_func_info, + RunStateArgs* run_state_args); + + // Creates several graphs given the existing graph_def_ and the + // input feeds and fetches, given 'devices'. The graphs share a common + // function library 'flib_def'. + ::tensorflow::Status CreateGraphs( + const BuildGraphOptions& options, + std::unordered_map>* outputs, + std::unique_ptr* flib_def, + RunStateArgs* run_state_args, DataTypeVector* input_types, + DataTypeVector* output_types, int64* collective_graph_key); + + ::tensorflow::Status RunInternal(int64 step_id, const RunOptions& run_options, + CallFrameInterface* call_frame, + ExecutorsAndKeys* executors_and_keys, + RunMetadata* run_metadata); + + // Returns whether inter-op execution uses a global pool. + bool ShouldUseRunHandlerPool() const; + + ::tensorflow::Status ExtendLocked(const GraphDef& graph) + EXCLUSIVE_LOCKS_REQUIRED(graph_state_lock_); + + ::tensorflow::Status ResourceHandleToInputTensor( + const Tensor& resource_tensor, Tensor* retrieved_tensor); + + // Feeds more inputs to the executors, triggering further execution. + ::tensorflow::Status SendPRunInputs( + const std::vector>& inputs, + const ExecutorsAndKeys* executors_and_keys, + IntraProcessRendezvous* rendez); + + // Fetches more outputs from the executors. It waits until the output + // tensors are computed. + ::tensorflow::Status RecvPRunOutputs( + const std::vector& output_names, + const ExecutorsAndKeys* executors_and_keys, RunState* run_state, + std::vector* outputs); + + // Check if the specified fetches can be computed from the feeds + // that we have already provided. + ::tensorflow::Status CheckFetch( + const std::vector>& feeds, + const std::vector& fetches, + const ExecutorsAndKeys* executors_and_keys, const RunState* run_state); + + // Use the appropriate WaitForNotification function based on whether + // operation_timeout_in_ms is greater than 0. + // + // If the timeout expires, the `cm->StartCancel()` will be called. + ::tensorflow::Status WaitForNotification(Notification* n, + int64 timeout_in_ms); + void WaitForNotification(RunState* run_state, CancellationManager* cm, + int64 timeout_in_ms); + + ::tensorflow::Status CheckNotClosed() { + mutex_lock l(closed_lock_); + if (closed_) return errors::Cancelled("Session has been closed."); + return ::tensorflow::Status::OK(); + } + + ::tensorflow::Status CheckGraphCreated(const char* method) { + mutex_lock l(graph_state_lock_); + if (!graph_created_) { + return errors::InvalidArgument( + "Session was not created with a graph before ", method, "!"); + } + return ::tensorflow::Status::OK(); + } + + ::tensorflow::Status CreateDebuggerState( + const CallableOptions& options, int64 global_step, + int64 session_run_index, int64 executor_step_index, + std::unique_ptr* debugger_state); + + ::tensorflow::Status DecorateAndPublishGraphForDebug( + const DebugOptions& debug_options, Graph* graph, Device* device); + + const SessionOptions options_; + + // Device structures. + const std::unique_ptr device_mgr_; + std::vector devices_; // not owned + DeviceSet device_set_; + + string session_handle_; + mutex graph_state_lock_; + bool graph_created_ GUARDED_BY(graph_state_lock_) = false; + + // The thread-pools to use for running ops, with a bool indicating if the pool + // is owned. + std::vector> thread_pools_; + + Status init_error_; // Set to an error if construction failed. + + // If true, blocks until device has finished all queued operations in a step. + bool sync_on_finish_ = true; + // Schedules 'c' for execution on pool. + void SchedClosure(thread::ThreadPool* pool, std::function c); + + std::vector> functions_ + GUARDED_BY(executor_lock_); + + mutex executor_lock_; // protects executors_ + // Holds mappings from signature to the executors that process + // it. The reason for a level of indirection around mapped_type is + // to guarantee address stability. + // The map value is a shared_ptr since multiple map keys can point to the + // same ExecutorsAndKey object. + std::unordered_map> executors_ + GUARDED_BY(executor_lock_); + + class RunCallableCallFrame; + struct Callable { + std::shared_ptr executors_and_keys; + std::shared_ptr function_info; + ~Callable(); + }; + mutex callables_lock_; + int64 next_callable_handle_ GUARDED_BY(callables_lock_) = 0; + std::unordered_map callables_ GUARDED_BY(callables_lock_); + + // Holds mappings from handle to partial run state. + std::unordered_map> partial_runs_ + GUARDED_BY(executor_lock_); + + // This holds all the tensors that are currently alive in the session. + SessionState session_state_; + + DirectSessionFactory* const factory_; // not owned + CancellationManager* cancellation_manager_; + std::unique_ptr collective_executor_mgr_; + + // Map of placed stateful nodes, i.e. nodes for which is_stateful() + // is true, such as "params" and "queue" nodes. Once placed these + // nodes can not be moved to a different device. Maps node names to + // device names. + std::unordered_map stateful_placements_ + GUARDED_BY(graph_state_lock_); + + // Execution_state; used when placing the entire graph. + std::unique_ptr execution_state_ + GUARDED_BY(graph_state_lock_); + + // The function library, before any rewrites or optimizations have been + // performed. In particular, CreateGraphs() may need to modify the function + // library; it copies and modifies the function library. + std::unique_ptr flib_def_; + + // true if the Session has been Closed. + mutex closed_lock_; + bool closed_ GUARDED_BY(closed_lock_) = false; + + // For generating unique names for this session instance. + std::atomic edge_name_counter_ = {0}; + std::atomic handle_name_counter_ = {0}; + + // For generating step ids that are unique across this sessions. + static std::atomic_int_fast64_t step_id_counter_; + + // Global timeout for all blocking operations in this session. + const int64 operation_timeout_in_ms_ = 0; + + // Manages all the cost models for the graphs executed in this session. + CostModelManager cost_model_manager_; + + // For testing collective graph key generation. + mutex collective_graph_key_lock_; + int64 collective_graph_key_ GUARDED_BY(collective_graph_key_lock_) = -1; + + TF_DISALLOW_COPY_AND_ASSIGN(DirectSession); + + // EXPERIMENTAL: debugger (tfdbg) related + friend class DebugGateway; +}; + +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_DIRECT_SESSION_H_ diff --git a/linux/include/tensorflow/core/common_runtime/dma_helper.h b/linux/include/tensorflow/core/common_runtime/dma_helper.h new file mode 100644 index 000000000..4a76cff1e --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/dma_helper.h @@ -0,0 +1,38 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_DMA_HELPER_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_DMA_HELPER_H_ + +#include "tensorflow/core/framework/tensor.h" + +namespace tensorflow { + +// For TensorFlow internal use only. +class DMAHelper { + public: + static bool CanUseDMA(const Tensor* t) { return t->CanUseDMA(); } + static const void* base(const Tensor* t) { return t->base(); } + static void* base(Tensor* t) { return t->base(); } + static TensorBuffer* buffer(Tensor* t) { return t->buf_; } + static const TensorBuffer* buffer(const Tensor* t) { return t->buf_; } + static void UnsafeSetShape(Tensor* t, const TensorShape& s) { + t->set_shape(s); + } +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_DMA_HELPER_H_ diff --git a/linux/include/tensorflow/core/common_runtime/eigen_thread_pool.h b/linux/include/tensorflow/core/common_runtime/eigen_thread_pool.h new file mode 100644 index 000000000..c6f13c6a1 --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/eigen_thread_pool.h @@ -0,0 +1,41 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_COMMON_RUNTIME_EIGEN_THREAD_POOL_H_ +#define TENSORFLOW_COMMON_RUNTIME_EIGEN_THREAD_POOL_H_ + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/lib/core/threadpool.h" + +namespace tensorflow { + +class EigenThreadPoolWrapper : public Eigen::ThreadPoolInterface { + public: + explicit EigenThreadPoolWrapper(thread::ThreadPool* pool) : pool_(pool) {} + ~EigenThreadPoolWrapper() override {} + + void Schedule(std::function fn) override { + pool_->Schedule(std::move(fn)); + } + int NumThreads() const override { return pool_->NumThreads(); } + int CurrentThreadId() const override { return pool_->CurrentThreadId(); } + + private: + thread::ThreadPool* pool_ = nullptr; +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_COMMON_RUNTIME_EIGEN_THREAD_POOL_H_ diff --git a/linux/include/tensorflow/core/common_runtime/eval_const_tensor.h b/linux/include/tensorflow/core/common_runtime/eval_const_tensor.h new file mode 100644 index 000000000..fca5a2356 --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/eval_const_tensor.h @@ -0,0 +1,66 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_EVAL_CONST_TENSOR_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_EVAL_CONST_TENSOR_H_ + +#include "tensorflow/core/graph/graph.h" +#include "tensorflow/core/lib/core/status.h" + +// TODO(skyewm): can this be combined with ConstantFold? + +namespace tensorflow { + +class GraphRunner; +class OpRegistryInterface; +class ShapeRefiner; +class Tensor; + +// Attempts to evaluate `tensor`. This will only be possible if `tensor` doesn't +// depend on any graph inputs (this function is safe to call if this isn't the +// case though). +// +// If the evaluation is successful, `evaluated` will be set to true and +// `tensor`s value returned in `result`. Otherwise `evaluated` will be set to +// false. An error status is returned if something is wrong with the graph or +// input. Note that `evaluated` may set to false if Status::OK() is returned. +// +// Params: +// tensor - the tensor to be evaluated. +// refiner - used to fetch the InferenceContexts for nodes in the graph. +// ops - the OpRegistryInterface for the graph. +// graph_def_version - the producer version of the graph. +// evaluated - output param indicating whether evaluation was successful. +// result - output param containing the result if evaluated is true. +// graph_runner - optional. If not set, a GraphRunner will be created for +// evaluating tensor. This can be set to avoid creating a new GraphRunner +// for every call. +// cached_values - optional. This can be used to cache evaluated results +// across calls, to avoid evaluating the same parts of the graph multiple +// times. +// max_cached_value_size - optional. If `cached_values` is set, the maximum +// result size to cache. +// disable_constant_propagation - if true, only Const node values will be +// returned. +Status EvaluateConstantTensor( + OutputTensor tensor, const ShapeRefiner& refiner, + const OpRegistryInterface& ops, int32 graph_def_version, bool* evaluated, + Tensor* result, GraphRunner* graph_runner = nullptr, + std::unordered_map* cached_values = nullptr, + int64 max_cached_value_size = 1024, + bool disable_constant_propagation = false); + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_EVAL_CONST_TENSOR_H_ diff --git a/linux/include/tensorflow/core/common_runtime/executor.h b/linux/include/tensorflow/core/common_runtime/executor.h new file mode 100644 index 000000000..34bf73972 --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/executor.h @@ -0,0 +1,232 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_EXECUTOR_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_EXECUTOR_H_ + +#include "tensorflow/core/common_runtime/device.h" +#include "tensorflow/core/framework/rendezvous.h" +#include "tensorflow/core/framework/session_state.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/graph/graph.h" +#include "tensorflow/core/lib/core/notification.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/platform/logging.h" +#include "tensorflow/core/platform/macros.h" + +namespace tensorflow { + +class StepStatsCollector; + +// Executor runs a graph computation. +// Example: +// Graph* graph = ...; +// ... construct graph ... +// Executor* executor; +// TF_CHECK_OK(NewSimpleExecutor(my_device, graph, &executor)); +// Rendezvous* rendezvous = NewNaiveRendezvous(); +// TF_CHECK_OK(rendezvous->Send("input", some_input_tensor)); +// TF_CHECK_OK(executor->Run({ExecutorOpts, rendezvous, nullptr})); +// TF_CHECK_OK(rendezvous->Recv("output", &output_tensor)); +// ... ... +// +// Multiple threads can call Executor::Run concurrently. +class Executor { + public: + virtual ~Executor() {} + + // RunAsync() executes the graph computation. "done" is run when the + // graph computation completes. If any error happens during the + // computation, "done" is run and the error is passed to "done". + // + // RunAsync() is given a few arguments in Args. The caller must + // ensure objects passed in Args (rendezvous, stats_collector, etc.) + // are alive at least until done is invoked. All pointers to the + // argument objects can be nullptr. + // + // "step_id" is a process-wide unique identifier for the step being + // run. Executors on different devices may receive the same step_id + // in the case that a step runs Ops on more than one device. The + // step_id is used for tracking resource usage of a given step. + // + // RunAsync() uses the given "rendezvous", if not null, as the + // mechanism to communicate inputs and outputs of the underlying + // graph computation. + // + // RunAsync() calls "stats_collector", if not null, to keep track of + // stats. This allows us to collect statistics and traces on demand. + // + // RunAsync() is provided a "call_frame", if the executor is used + // for executing a function, is used to pass arguments and return + // values between the caller and the callee. + // + // RunAsync() uses "cancellation_manager", if not nullptr, to + // register callbacks that should be called if the graph computation + // is canceled. Note that the callbacks merely unblock any + // long-running computation, and a canceled step will terminate by + // returning/calling the DoneCallback as usual. + // + // RunAsync() dispatches closures to "runner". Typically, "runner" + // is backed up by a bounded threadpool. + struct Args { + int64 step_id = 0; + Rendezvous* rendezvous = nullptr; + StepStatsCollectorInterface* stats_collector = nullptr; + CallFrameInterface* call_frame = nullptr; + CancellationManager* cancellation_manager = nullptr; + SessionState* session_state = nullptr; + TensorStore* tensor_store = nullptr; + ScopedStepContainer* step_container = nullptr; + CollectiveExecutor* collective_executor = nullptr; + + // If true, calls Sync() on the device. + bool sync_on_finish = false; + + typedef std::function Closure; + typedef std::function Runner; + Runner runner = nullptr; + }; + typedef std::function DoneCallback; + virtual void RunAsync(const Args& args, DoneCallback done) = 0; + + // Synchronous wrapper for RunAsync(). + Status Run(const Args& args) { + Status ret; + Notification n; + RunAsync(args, [&ret, &n](const Status& s) { + ret = s; + n.Notify(); + }); + n.WaitForNotification(); + return ret; + } +}; + +// Creates an Executor that computes the given "graph". +// +// If successful, returns the constructed executor in "*executor". Otherwise, +// returns an error status. +// +// "params" provides a set of context for the executor. We expect that +// different context would provide different implementations. +struct LocalExecutorParams { + Device* device; + + // The library runtime support. + FunctionLibraryRuntime* function_library = nullptr; + + // create_kernel returns an instance of op kernel based on NodeDef. + // delete_kernel is called for every kernel used by the executor + // when the executor is deleted. + std::function create_kernel; + std::function delete_kernel; +}; +::tensorflow::Status NewLocalExecutor(const LocalExecutorParams& params, + std::unique_ptr graph, + Executor** executor); + +// A class to help run multiple executors in parallel and wait until +// all of them are complete. +// +// ExecutorBarrier deletes itself after the function returned by Get() +// is called. +class ExecutorBarrier { + public: + typedef std::function StatusCallback; + + // Create an ExecutorBarrier for 'num' different executors. + // + // 'r' is the shared Rendezvous object that is used to communicate + // state. If any of the executors experiences an error, the + // rendezvous object will be aborted exactly once. + // + // 'done' is called after the last executor completes, and + // ExecutorBarrier is deleted. + ExecutorBarrier(size_t num, Rendezvous* r, StatusCallback done) + : rendez_(r), done_cb_(done), pending_(num) {} + + ~ExecutorBarrier() {} + + // Returns a closure that Executors must call when they are done + // computing, passing the status of their execution as an argument. + StatusCallback Get() { + return std::bind(&ExecutorBarrier::WhenDone, this, std::placeholders::_1); + } + + private: + Rendezvous* rendez_ = nullptr; + StatusCallback done_cb_ = nullptr; + + mutable mutex mu_; + int pending_ GUARDED_BY(mu_) = 0; + Status status_ GUARDED_BY(mu_); + + void WhenDone(const Status& s) { + bool error = false; + Rendezvous* error_rendez = nullptr; + StatusCallback done = nullptr; + Status status; + { + mutex_lock l(mu_); + // If we are the first error encountered, mark the status + // appropriately and later trigger an abort of the Rendezvous + // object by this thread only. + if (status_.ok() && !s.ok()) { + error = true; + error_rendez = rendez_; + error_rendez->Ref(); + status_ = s; + } + + // If this is the last call to WhenDone, call the final callback + // below. + if (--pending_ == 0) { + CHECK(done_cb_ != nullptr); + std::swap(done, done_cb_); + } + + if (!status_.ok()) { + status = status_; + } + } + + if (error) { + error_rendez->StartAbort(status); + error_rendez->Unref(); + } + if (done != nullptr) { + delete this; + done(status); + } + } + + TF_DISALLOW_COPY_AND_ASSIGN(ExecutorBarrier); +}; + +// A few helpers to facilitate create/delete kernels. + +// Creates a kernel based on "ndef" on device "device". The kernel can +// access the functions in the "flib". The caller takes ownership of +// returned "*kernel". +Status CreateNonCachedKernel(Device* device, FunctionLibraryRuntime* flib, + const NodeDef& ndef, int graph_def_version, + OpKernel** kernel); + +// Deletes "kernel" returned by CreateKernel. +void DeleteNonCachedKernel(OpKernel* kernel); + +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_EXECUTOR_H_ diff --git a/linux/include/tensorflow/core/common_runtime/executor_factory.h b/linux/include/tensorflow/core/common_runtime/executor_factory.h new file mode 100644 index 000000000..f81bb080e --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/executor_factory.h @@ -0,0 +1,51 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_EXECUTOR_FACTORY_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_EXECUTOR_FACTORY_H_ + +#include + +#include "tensorflow/core/lib/core/errors.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/lib/gtl/array_slice.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +class Executor; +class Graph; +struct LocalExecutorParams; + +class ExecutorFactory { + public: + virtual Status NewExecutor(const LocalExecutorParams& params, + std::unique_ptr graph, + std::unique_ptr* out_executor) = 0; + virtual ~ExecutorFactory() {} + + static void Register(const string& executor_type, ExecutorFactory* factory); + static Status GetFactory(const string& executor_type, + ExecutorFactory** out_factory); +}; + +Status NewExecutor(const string& executor_type, + const LocalExecutorParams& params, + std::unique_ptr graph, + std::unique_ptr* out_executor); + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_EXECUTOR_FACTORY_H_ diff --git a/linux/include/tensorflow/core/common_runtime/function.h b/linux/include/tensorflow/core/common_runtime/function.h new file mode 100644 index 000000000..eeca66f5d --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/function.h @@ -0,0 +1,173 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_FUNCTION_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_FUNCTION_H_ + +#include +#include + +#include "tensorflow/core/common_runtime/device.h" +#include "tensorflow/core/common_runtime/device_mgr.h" +#include "tensorflow/core/common_runtime/process_function_library_runtime.h" +#include "tensorflow/core/framework/function.h" +#include "tensorflow/core/graph/graph.h" +#include "tensorflow/core/protobuf/config.pb.h" + +namespace tensorflow { + +static constexpr const char* const kNoInlineAttr = "_noinline"; + +// Registers a default customizable kernel creator for a function call. +// +// If 'cb()' returns a non-OK, we still fall back to an executor-based +// interpreter op kernel to execute a function. If 'cb()' returns OK, +// takes ownership of the returned OpKernel. +// +// TODO(zhifengc/phawkins): b/32379046 +void RegisterDefaultCustomKernelCreator(CustomKernelCreator cb); + +// Creates a FunctionLibraryRuntime, which instantiates functions +// defined in "lib_def" and executes functions on the "device". +// "device_mgr" must contain the "device". If not nullptr, +// "custom_kernel_creator" is consulted by the returned runtime to +// create kernels. +// +// The returned object does not take ownerships of "device" or +// "lib_def". The caller must ensure "device" and "lib_def" outlives +// the returned object. +// +// The "parent" is a pointer to the ProcessFunctionLibraryRuntime object that +// typically owns the created FunctionLibraryRuntime object. The parent pointer +// is not owned by the FunctionLibraryRuntime object. +std::unique_ptr NewFunctionLibraryRuntime( + const DeviceMgr* device_mgr, Env* env, Device* device, + int graph_def_version, const FunctionLibraryDefinition* lib_def, + thread::ThreadPool* thread_pool, const OptimizerOptions& optimizer_options, + CustomKernelCreator custom_kernel_creator, + ProcessFunctionLibraryRuntime* parent); + +// Same as above except that the returned runtime consults with the +// global default custom kernel creator registered by +// RegisterDefaultCustomKernelCreator. +std::unique_ptr NewFunctionLibraryRuntime( + const DeviceMgr* device_mgr, Env* env, Device* device, + int graph_def_version, const FunctionLibraryDefinition* lib_def, + thread::ThreadPool* thread_pool, const OptimizerOptions& optimizer_options, + ProcessFunctionLibraryRuntime* parent); + +// FunctionLibraryRuntime::GetFunctionBody returns a description of an +// instantiated function that is represented as a Graph with arg/ret +// nodes annotated. +struct FunctionBody { + FunctionDef fdef; + Graph* graph = nullptr; // owned. + DataTypeVector arg_types; + DataTypeVector ret_types; + gtl::InlinedVector arg_nodes; + gtl::InlinedVector ret_nodes; + + FunctionBody() {} + FunctionBody(const FunctionDef& f, DataTypeSlice arg_types, + DataTypeSlice ret_types, Graph* g); + ~FunctionBody(); +}; + +// Debugging facility. Returns a debug string for a graph +// representing an instantiated function. +string DebugString(const Graph* instantiated_func_graph); + +// A few hand-crafted optimization on the instantiated function body +// (a Graph*). + +// Removes nodes that are +// 1. not stateful; and +// 2. not _Arg; and +// 3. not reachable from _Retval. +// Returns true iff any node is removed from "g". +bool RemoveDeadNodes(Graph* g); + +// Find a pattern: +// src -(in)-> node -(out)-> dst, where +// 1) node is an identity node; +// 2) in is the only incoming data edge; +// 3) out is the only outgoing data edge; +// +// Rewrites the above pattern with src->dst and relevant data +// dependencies updated. Repeat the process until no such pattern +// left. +bool RemoveIdentityNodes(Graph* g); + +// Rewrites _ListToArray and _ArrayToList to a set of Identity nodes. +bool RemoveListArrayConverter(Graph* g); + +// For each node in "graph", if "lib" indicates that the node is a +// function call, inline the function body. Returns true if at least +// one node is inlined. +// +// This routine goes through "graph" nodes once and applies the +// inlining. The caller may decide to apply the inlining on "graph" +// multiple times by calling ExpandInlineFunctions a few times. +bool ExpandInlineFunctions(FunctionLibraryRuntime* lib, Graph* graph); + +// Dump the contents of the "graph" to log files if the logging level is +// sufficiently high. +void DumpGraph(StringPiece label, const Graph* g); + +// Applies graph rewrite optimization such as inlining, dead code +// removal, etc. +// +// **g is a graph constructed based on the runtime library 'lib'. +// OptimizeGraph mutates **g extensively and replaces '*g' with a +// complete copy. Therefore, the caller should not keep any references +// to nodes *g. +void OptimizeGraph(FunctionLibraryRuntime* lib, std::unique_ptr* g); + +// Convert the Graph of a function to a GraphDef. +// +// Handles renaming of nodes to avoid duplicate names which may +// be present after various rewriting operations. +void ToGraphDef(const Graph* g, GraphDef* gdef, bool pretty = false); + +// Given a numerical function "f", returns another numerical function +// "g", such that if "f" takes N inputs and produces M outputs, "g" +// takes N + M inputs and produces N outputs. I.e., if +// (y1, y2, ..., y_M) = f(x1, x2, ..., x_N), +// g is a function which is +// (dL/dx1, dL/dx2, ..., dL/dx_N) = g(x1, x2, ..., x_N, +// dL/dy1, dL/dy2, ..., dL/dy_M), +// where L is a scalar-value function of (...x_i...). +// +// TODO(zhifengc): Asks math expert to say the comment again. +FunctionBody* SymbolicGradient(const FunctionBody& f); + +// Given a "caller" in graph "g", which is a function call of a function +// to "fbody". Replaces the "caller" with fbody->graph and connects +// edges properly. "override_device" specifies whether inlining should replace +// explicitly specified devices inside fbody with the callee's device. +void InlineFunctionBody(const FunctionLibraryDefinition& flib_def, Graph* g, + Node* caller, const FunctionBody* fbody, + bool override_device = true); + +// Instantiates FunctionDef into a graph. Set *fbody to point to the +// FunctionBody that holds the instantiated FunctionDef. +Status FunctionDefToBodyHelper( + const FunctionDef& fdef, const AttrSlice& attrs, + const FunctionLibraryDefinition* const lib_def, + const std::function& get_func_sig, + FunctionBody** fbody); +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_FUNCTION_H_ diff --git a/linux/include/tensorflow/core/common_runtime/gpu/cuda_host_allocator.h b/linux/include/tensorflow/core/common_runtime/gpu/cuda_host_allocator.h new file mode 100644 index 000000000..6bd29ef77 --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/gpu/cuda_host_allocator.h @@ -0,0 +1,68 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_GPU_CUDA_HOST_ALLOCATOR_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_GPU_CUDA_HOST_ALLOCATOR_H_ + +#include "tensorflow/core/framework/allocator.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/stream_executor.h" + +namespace tensorflow { +// Allocator for pinned CPU RAM that is made known to CUDA for the +// purpose of efficient DMA with a GPU. +class CUDAHostAllocator : public SubAllocator { + public: + // Note: stream_exec cannot be null. + explicit CUDAHostAllocator(se::StreamExecutor* stream_exec, int numa_node, + const std::vector& alloc_visitors, + const std::vector& free_visitors) + : SubAllocator(alloc_visitors, free_visitors), + stream_exec_(stream_exec), + numa_node_(numa_node) { + CHECK(stream_exec_ != nullptr); + } + ~CUDAHostAllocator() override {} + + void* Alloc(size_t alignment, size_t num_bytes) override { + void* ptr = nullptr; + if (num_bytes > 0) { + ptr = stream_exec_->HostMemoryAllocate(num_bytes); + if (ptr == nullptr) { + LOG(WARNING) << "could not allocate pinned host memory of size: " + << num_bytes; + return ptr; + } + VisitAlloc(ptr, numa_node_, num_bytes); + } + return ptr; + } + + void Free(void* ptr, size_t num_bytes) override { + if (ptr != nullptr) { + VisitFree(ptr, numa_node_, num_bytes); + stream_exec_->HostMemoryDeallocate(ptr); + } + } + + private: + se::StreamExecutor* stream_exec_; // not owned, non-null + const int numa_node_; + + TF_DISALLOW_COPY_AND_ASSIGN(CUDAHostAllocator); +}; + +} // namespace tensorflow +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_GPU_CUDA_HOST_ALLOCATOR_H_ diff --git a/linux/include/tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.h b/linux/include/tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.h new file mode 100644 index 000000000..d4c9cee89 --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.h @@ -0,0 +1,103 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_GPU_GPU_BFC_ALLOCATOR_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_GPU_GPU_BFC_ALLOCATOR_H_ + +#include +#include +#include +#include + +#include "tensorflow/core/common_runtime/allocator_retry.h" +#include "tensorflow/core/common_runtime/bfc_allocator.h" +#include "tensorflow/core/common_runtime/gpu/gpu_id.h" +#include "tensorflow/core/platform/stream_executor.h" +#include "tensorflow/core/platform/thread_annotations.h" +#include "tensorflow/core/platform/types.h" +#include "tensorflow/core/protobuf/config.pb.h" + +namespace tensorflow { + +// Suballocator for GPU memory. +class GPUMemAllocator : public SubAllocator { + public: + // 'platform_gpu_id' refers to the ID of the GPU device within + // the process and must reference a valid ID in the process. + // Note: stream_exec cannot be null. + explicit GPUMemAllocator(se::StreamExecutor* stream_exec, + PlatformGpuId gpu_id, bool use_unified_memory, + const std::vector& alloc_visitors, + const std::vector& free_visitors) + : SubAllocator(alloc_visitors, free_visitors), + stream_exec_(stream_exec), + gpu_id_(gpu_id), + use_unified_memory_(use_unified_memory) { + CHECK(stream_exec_ != nullptr); + } + ~GPUMemAllocator() override {} + + void* Alloc(size_t alignment, size_t num_bytes) override { + void* ptr = nullptr; + if (num_bytes > 0) { + if (use_unified_memory_) { + ptr = stream_exec_->UnifiedMemoryAllocate(num_bytes); + } else { + ptr = stream_exec_->AllocateArray(num_bytes).opaque(); + } + VisitAlloc(ptr, gpu_id_.value(), num_bytes); + } + return ptr; + } + + void Free(void* ptr, size_t num_bytes) override { + if (ptr != nullptr) { + VisitFree(ptr, gpu_id_.value(), num_bytes); + if (use_unified_memory_) { + stream_exec_->UnifiedMemoryDeallocate(ptr); + } else { + se::DeviceMemoryBase gpu_ptr(ptr); + stream_exec_->Deallocate(&gpu_ptr); + } + } + } + + private: + se::StreamExecutor* stream_exec_; // not owned, non-null + const PlatformGpuId gpu_id_; + const bool use_unified_memory_ = false; + + TF_DISALLOW_COPY_AND_ASSIGN(GPUMemAllocator); +}; + +// A GPU memory allocator that implements a 'best-fit with coalescing' +// algorithm. +class GPUBFCAllocator : public BFCAllocator { + public: + GPUBFCAllocator(GPUMemAllocator* sub_allocator, size_t total_memory, + const string& name); + GPUBFCAllocator(GPUMemAllocator* sub_allocator, size_t total_memory, + const GPUOptions& gpu_options, const string& name); + ~GPUBFCAllocator() override {} + + TF_DISALLOW_COPY_AND_ASSIGN(GPUBFCAllocator); + + private: + static bool GetAllowGrowthValue(const GPUOptions& gpu_options); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_GPU_GPU_BFC_ALLOCATOR_H_ diff --git a/linux/include/tensorflow/core/common_runtime/gpu/gpu_cudamalloc_allocator.h b/linux/include/tensorflow/core/common_runtime/gpu/gpu_cudamalloc_allocator.h new file mode 100644 index 000000000..8df3724bc --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/gpu/gpu_cudamalloc_allocator.h @@ -0,0 +1,52 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_GPU_GPU_CUDAMALLOC_ALLOCATOR_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_GPU_GPU_CUDAMALLOC_ALLOCATOR_H_ + +#include + +#include "tensorflow/core/common_runtime/gpu/gpu_id.h" +#include "tensorflow/core/framework/allocator.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/stream_executor.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +// An allocator that wraps a GPU allocator and adds debugging +// functionality that verifies that users do not write outside their +// allocated memory. +class GPUcudaMallocAllocator : public Allocator { + public: + explicit GPUcudaMallocAllocator(Allocator* allocator, + PlatformGpuId platform_gpu_id); + ~GPUcudaMallocAllocator() override; + string Name() override { return "gpu_debug"; } + void* AllocateRaw(size_t alignment, size_t num_bytes) override; + void DeallocateRaw(void* ptr) override; + bool TracksAllocationSizes() override; + + private: + Allocator* base_allocator_ = nullptr; // owned + + se::StreamExecutor* stream_exec_; // Not owned. + + TF_DISALLOW_COPY_AND_ASSIGN(GPUcudaMallocAllocator); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_GPU_GPU_CUDAMALLOC_ALLOCATOR_H_ diff --git a/linux/include/tensorflow/core/common_runtime/gpu/gpu_debug_allocator.h b/linux/include/tensorflow/core/common_runtime/gpu/gpu_debug_allocator.h new file mode 100644 index 000000000..17757a106 --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/gpu/gpu_debug_allocator.h @@ -0,0 +1,87 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_GPU_GPU_DEBUG_ALLOCATOR_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_GPU_GPU_DEBUG_ALLOCATOR_H_ + +#include +#include +#include + +#include "tensorflow/core/common_runtime/gpu/gpu_id.h" +#include "tensorflow/core/framework/allocator.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/stream_executor.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +// An allocator that wraps a GPU allocator and adds debugging +// functionality that verifies that users do not write outside their +// allocated memory. +class GPUDebugAllocator : public Allocator { + public: + explicit GPUDebugAllocator(Allocator* allocator, + PlatformGpuId platform_gpu_id); + ~GPUDebugAllocator() override; + string Name() override { return "gpu_debug"; } + void* AllocateRaw(size_t alignment, size_t num_bytes) override; + void DeallocateRaw(void* ptr) override; + bool TracksAllocationSizes() override; + size_t RequestedSize(const void* ptr) override; + size_t AllocatedSize(const void* ptr) override; + int64 AllocationId(const void* ptr) override; + void GetStats(AllocatorStats* stats) override; + void ClearStats() override; + + // For testing. + bool CheckHeader(void* ptr); + bool CheckFooter(void* ptr); + + private: + Allocator* base_allocator_ = nullptr; // owned + + se::StreamExecutor* stream_exec_; // Not owned. + + TF_DISALLOW_COPY_AND_ASSIGN(GPUDebugAllocator); +}; + +// An allocator that wraps a GPU allocator and resets the memory on +// allocation and free to 'NaN', helping to identify cases where the +// user forgets to initialize the memory. +class GPUNanResetAllocator : public Allocator { + public: + explicit GPUNanResetAllocator(Allocator* allocator, + PlatformGpuId platform_gpu_id); + ~GPUNanResetAllocator() override; + string Name() override { return "gpu_nan_reset"; } + void* AllocateRaw(size_t alignment, size_t num_bytes) override; + void DeallocateRaw(void* ptr) override; + size_t RequestedSize(const void* ptr) override; + size_t AllocatedSize(const void* ptr) override; + void GetStats(AllocatorStats* stats) override; + void ClearStats() override; + + private: + Allocator* base_allocator_ = nullptr; // owned + + se::StreamExecutor* stream_exec_; // Not owned. + + TF_DISALLOW_COPY_AND_ASSIGN(GPUNanResetAllocator); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_GPU_GPU_DEBUG_ALLOCATOR_H_ diff --git a/linux/include/tensorflow/core/common_runtime/gpu/gpu_device.h b/linux/include/tensorflow/core/common_runtime/gpu/gpu_device.h new file mode 100644 index 000000000..674e8384d --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/gpu/gpu_device.h @@ -0,0 +1,235 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#if !GOOGLE_CUDA +#error This file must only be included when building with Cuda support +#endif + +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_GPU_GPU_DEVICE_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_GPU_GPU_DEVICE_H_ + +#include +#include +#include +#include + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/common_runtime/device_factory.h" +#include "tensorflow/core/common_runtime/gpu/gpu_event_mgr.h" +#include "tensorflow/core/common_runtime/gpu/gpu_id.h" +#include "tensorflow/core/common_runtime/gpu/gpu_id_manager.h" +#include "tensorflow/core/common_runtime/gpu/gpu_id_utils.h" +#include "tensorflow/core/common_runtime/gpu_device_context.h" +#include "tensorflow/core/common_runtime/local_device.h" +#include "tensorflow/core/common_runtime/scoped_allocator_mgr.h" +#include "tensorflow/core/framework/allocator.h" +#include "tensorflow/core/framework/device_base.h" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/lib/gtl/inlined_vector.h" +#include "tensorflow/core/platform/mutex.h" +#include "tensorflow/core/platform/stream_executor.h" +#include "tensorflow/core/platform/types.h" +#include "tensorflow/core/public/session_options.h" + +namespace tensorflow { + +class BaseGPUDevice : public LocalDevice { + public: + BaseGPUDevice(const SessionOptions& options, const string& name, + Bytes memory_limit, const DeviceLocality& locality, + TfGpuId tf_gpu_id, const string& physical_device_desc, + Allocator* gpu_allocator, Allocator* cpu_allocator, + bool sync_every_op, int32 max_streams); + + ~BaseGPUDevice() override; + + // Initialize the device and return the status of initialization. + Status Init(const SessionOptions& options); + + // GPU devices require the Op Compute method to save a reference to + // any temporary tensors that are allocated until the Op execution + // completes. + bool RequiresRecordingAccessedTensors() const override; + + // GPU kernel execution requires us to use `tracing::ScopedAnnotation()` + // rather than `tracing::ScopedActivity()`, in order to relate asynchronously + // launched GPU kernels to the OpKernel. + bool TraceUsingAnnotations() const { return true; } + + void ConsumeListOfAccessedTensors( + DeviceContext* device_context, + const TensorReferenceVector& tensor_refs) override; + + Status FillContextMap(const Graph* graph, + DeviceContextMap* device_context_map) override; + + void Compute(OpKernel* op_kernel, OpKernelContext* context) override; + + Status Sync() override; + + void ComputeAsync(AsyncOpKernel* op_kernel, OpKernelContext* context, + AsyncOpKernel::DoneCallback done) override; + + Status MakeTensorFromProto(const TensorProto& tensor_proto, + const AllocatorAttributes alloc_attrs, + Tensor* tensor) override; + + // The caller owns the returned device. + PerOpGpuDevice* MakeGpuDevice() override; + + Status ReinitializeGpuDevice(OpKernelContext* context, PerOpGpuDevice* device, + DeviceContext* dc, + Allocator* allocator) override; + + // Returns the platform GPU id of this device within the native driver system; + // e.g., for CUDA this is the ordinal of the GPU within the system. + int gpu_id() const { + PlatformGpuId platform_gpu_id; + TF_CHECK_OK(GpuIdManager::TfToPlatformGpuId(tf_gpu_id_, &platform_gpu_id)); + return platform_gpu_id.value(); + } + + // The executor that provides control for the device; e.g., for CUDA this + // corresponds to the cuda context. + se::StreamExecutor* executor() const { return executor_; } + + Allocator* GetScopedAllocator(AllocatorAttributes attr, + int64 step_id) override; + + ScopedAllocatorMgr* GetScopedAllocatorMgr() const override { + return scoped_allocator_mgr_.get(); + } + + protected: + Allocator* gpu_allocator_; // not owned + Allocator* cpu_allocator_; // not owned + + se::StreamExecutor* executor_; // not owned + std::unique_ptr scoped_allocator_mgr_; + + private: + struct StreamGroup { + se::Stream* compute = nullptr; + se::Stream* host_to_device = nullptr; + se::Stream* device_to_host = nullptr; + gtl::InlinedVector device_to_device; + }; + class StreamGroupFactory; + + gtl::InlinedVector streams_; + mutex scratch_init_mutex_; + gtl::InlinedVector scratch_; + std::vector device_contexts_; + GpuDeviceInfo* gpu_device_info_ = nullptr; + mutex trace_mu_; + TfGpuId tf_gpu_id_; + const bool sync_every_op_ = false; + const int32 max_streams_; + std::unique_ptr em_; + std::unique_ptr thread_pool_; + + // Initialize scractch buffers used by Eigen. + Status InitScratchBuffers(); + + void ReinitializeDevice(OpKernelContext* context, PerOpGpuDevice* device, + int stream_id, Allocator* allocator); + + void ComputeHelper(OpKernel* op_kernel, OpKernelContext* context); + + string ComputeOpKernelDebugString(const OpKernel& op_kernel, + const int& stream_id); + + // This method returns an initialization status, in addition to + // calling the "done" StatusCallback, if there is a failure to + // allocate memory or if the tensor "from" is not DMA-copyable. + // If there is no error prior to enqueueing the copy, an OK status + // is returned. + Status MaybeCopyTensorToGPU(const AllocatorAttributes& alloc_attrs, + const Tensor& from, Tensor* to, + StatusCallback done); +}; + +class BaseGPUDeviceFactory : public DeviceFactory { + public: + Status CreateDevices(const SessionOptions& options, const string& name_prefix, + std::vector* devices) override; + + struct InterconnectMap { + // Name of interconnect technology, if known. + string name; + // If possible, strength should approximate Gb/sec bandwidth rate. + // Where architecture-specific subclassing is not done that won't + // always be possible. The minimum expectation is that + // faster links should have a higher value than slower links. + int32 strength; + static const int kSameDeviceStrength; + static const int kStreamExecutorStrength; + std::set> directed_links; + }; + + protected: + // Populates *maps with interconnect maps for all local direct access + // pathways between GPUs. + virtual Status GetInterconnectMaps( + const std::vector& visible_gpu_order, + se::Platform* gpu_manager, std::vector* maps); + + struct TfGpuIdHash { + std::size_t operator()(const TfGpuId& id) const noexcept { + return std::hash{}(id.value()); + } + }; + typedef std::unordered_map LocalityMap; + // Populates *localities with the DeviceLocality descriptor for + // every TfGpuId. + virtual Status GetDeviceLocalities( + int num_tf_gpus, const std::vector& interconnects, + LocalityMap* localities); + + private: + // Creates a BaseGPUDevice associated with 'tf_gpu_id', allocates (strictly) + // 'memory_limit' bytes of GPU memory to it, and adds it to the 'devices' + // vector. + Status CreateGPUDevice(const SessionOptions& options, + const string& name_prefix, TfGpuId tf_gpu_id, + int64 memory_limit, const DeviceLocality& dev_locality, + std::vector* devices); + + virtual BaseGPUDevice* CreateGPUDevice(const SessionOptions& options, + const string& name, Bytes memory_limit, + const DeviceLocality& dev_locality, + TfGpuId tf_gpu_id, + const string& physical_device_desc, + Allocator* gpu_allocator, + Allocator* cpu_allocator) = 0; + + // Returns into 'ids' the list of valid platform GPU ids, in the order that + // they should map to TF GPU ids "/device:GPU:0", "/device:GPU:1", etc, + // based upon 'visible_gpu_order' which was generated by parsing + // GPUOptions::visible_device_list which is a comma-separated list of CUDA GPU + // ids. + Status GetValidDeviceIds(const std::vector& visible_gpu_order, + std::vector* ids); + + // visible_gpu_initialized_[platform_gpu_id] is true if visible GPU + // platform_gpu_id has been initialized by the process. + std::unordered_map visible_gpu_initialized_; +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_GPU_GPU_DEVICE_H_ diff --git a/linux/include/tensorflow/core/common_runtime/gpu/gpu_event_mgr.h b/linux/include/tensorflow/core/common_runtime/gpu/gpu_event_mgr.h new file mode 100644 index 000000000..2d406b676 --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/gpu/gpu_event_mgr.h @@ -0,0 +1,206 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_GPU_GPU_EVENT_MGR_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_GPU_GPU_EVENT_MGR_H_ + +#include +#include +#include "tensorflow/core/framework/log_memory.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_reference.h" +#include "tensorflow/core/lib/core/notification.h" +#include "tensorflow/core/lib/core/threadpool.h" +#include "tensorflow/core/lib/gtl/inlined_vector.h" +#include "tensorflow/core/platform/mutex.h" +#include "tensorflow/core/platform/stream_executor.h" +#include "tensorflow/core/platform/thread_annotations.h" +#include "tensorflow/core/platform/types.h" + +namespace stream_executor { +class Event; +class Stream; +class StreamExecutor; +} // namespace stream_executor + +namespace tensorflow { + +class GPUOptions; + +// The callback provided to EventMgr::ThenExecute must not block or take a long +// time. If it does, performance may be impacted and GPU memory may be +// exhausted. This macro is for checking that an EventMgr thread is not +// accidentally entering blocking parts of the code, e.g. the RPC subsystem. +// +// Intended use is something like +// +// void RespondToAnRPC(Params* params) { +// WARN_IF_IN_EVENT_MGR_THREAD; +// if (params->status.ok()) { ... +// +namespace gpu_event_mgr { +// Logs a stack trace if current execution thread belongs to this EventMgr +// object. If f is not nullptr, executes instead of logging the stack trace. +// trace. +void WarnIfInCallback(std::function f); +} // namespace gpu_event_mgr +#define WARN_IF_IN_EVENT_MGR_THREAD gpu_event_mgr::WarnIfInCallback(nullptr) + +// An object to keep track of pending Events in the StreamExecutor streams +// and associated Tensors that cannot safely be deleted until the associated +// Events are recorded. +class EventMgr { + public: + EventMgr(se::StreamExecutor* se, const GPUOptions& gpu_options); + + ~EventMgr(); + + // Releases the references on the elements of "tensors" as soon as + // all events currently enqueued on "stream" have completed. + void ThenDeleteTensors(se::Stream* stream, + const TensorReferenceVector& tensors); + + struct BufRec { + Allocator* alloc; + void* buf; + // operation and step_id are only populated when + // LogMemory::IsEnabled() is true. + string operation; + int64 step_id; + }; + + // Takes ownership of *bufrec.buf and calls bufrec.alloc->DeallocateRaw() + // on it as soon as all events currently enqueued on *stream have completed. + inline void ThenDeleteBuffer(se::Stream* stream, BufRec bufrec) { + ToFreeVector to_free; + { + mutex_lock l(mu_); + QueueBuffer(stream, bufrec); + PollEvents(false, &to_free); + } + FreeMemory(to_free); + } + + // Execute func when all pending stream actions have completed. + // func must be brief and non-blocking since it executes in the one + // thread used for all such callbacks and also buffer deletions. + inline void ThenExecute(se::Stream* stream, std::function func) { + ToFreeVector to_free; + { + mutex_lock l(mu_); + QueueFunc(stream, std::move(func)); + PollEvents(false, &to_free); + } + FreeMemory(to_free); + } + + private: + friend class TEST_EventMgrHelper; + se::StreamExecutor* const exec_; + const int64 deferred_bytes_threshold_; + const int32 polling_active_delay_usecs_; + mutex mu_; + condition_variable events_pending_ GUARDED_BY(mu_); + + void FlushAccumulatedTensors() EXCLUSIVE_LOCKS_REQUIRED(mu_); + + struct InUse { + se::Event* event; + TensorReferenceVector* mem; + BufRec bufrec; + std::function func; + }; + + typedef gtl::InlinedVector ToFreeVector; + + void FreeMemory(const ToFreeVector& to_free) { + for (const auto& iu : to_free) { + if (iu.mem != nullptr) { + for (auto& t : *(iu.mem)) { + t.Unref(); + } + delete iu.mem; + } + if (iu.bufrec.buf) { + if (LogMemory::IsEnabled()) { + LogMemory::RecordRawDeallocation(iu.bufrec.operation, + iu.bufrec.step_id, iu.bufrec.buf, + iu.bufrec.alloc, false); + } + iu.bufrec.alloc->DeallocateRaw(iu.bufrec.buf); + } + // The function must be called in another thread. + if (iu.func != nullptr) threadpool_.Schedule(iu.func); + } + } + + // Stream-enqueue an unused Event and save with it a collection of + // Tensors and/or a BufRec to be deleted only after the Event + // records. + void QueueInUse(se::Stream* stream, InUse in_use) + EXCLUSIVE_LOCKS_REQUIRED(mu_); + + void QueueTensors(se::Stream* stream, TensorReferenceVector* tensors) + EXCLUSIVE_LOCKS_REQUIRED(mu_) { + QueueInUse(stream, {nullptr, tensors, BufRec(), nullptr}); + } + + void QueueBuffer(se::Stream* stream, BufRec bufrec) + EXCLUSIVE_LOCKS_REQUIRED(mu_) { + QueueInUse(stream, {nullptr, nullptr, bufrec, nullptr}); + } + + void QueueFunc(se::Stream* stream, std::function func) + EXCLUSIVE_LOCKS_REQUIRED(mu_) { + QueueInUse(stream, {nullptr, nullptr, BufRec(), std::move(func)}); + } + + // This function should be called at roughly the same tempo as + // QueueTensors() to check whether pending events have recorded, + // and then retire them. It appends InUse elements that need cleanup + // to "*to_free". The caller should call FreeMemory(to_free) + // when this returns. + void PollEvents(bool is_dedicated_poller, ToFreeVector* to_free) + EXCLUSIVE_LOCKS_REQUIRED(mu_); + + // An internal polling loop that runs at a low frequency to clear + // straggler Events. + void PollLoop(); + + // Setup/Teardown functions for the polling loop. + void StartPollingLoop(); + void StopPollingLoop(); + + // A stack of unused events + std::vector free_events_ GUARDED_BY(mu_); + + // Buffered list of tensors waiting to have an event queued for deletion + se::Stream* accumulated_stream_ GUARDED_BY(mu_); + TensorReferenceVector* accumulated_tensors_ GUARDED_BY(mu_); + // Sum of the TotalBytes() of the tensors in "accumulated_tensors_" + int64 accumulated_tensor_bytes_ GUARDED_BY(mu_); + + // A FIFO queue of InUse events and associated tensors. + std::deque used_events_ GUARDED_BY(mu_); + + bool stop_polling_ GUARDED_BY(mu_); + std::unique_ptr polling_stopped_; + + // The main PollLoop for the event manager runs in this threadpool. + thread::ThreadPool threadpool_; +}; + +} // namespace tensorflow +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_GPU_GPU_EVENT_MGR_H_ diff --git a/linux/include/tensorflow/core/common_runtime/gpu/gpu_id.h b/linux/include/tensorflow/core/common_runtime/gpu/gpu_id.h new file mode 100644 index 000000000..f0d902282 --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/gpu/gpu_id.h @@ -0,0 +1,88 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_GPU_GPU_ID_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_GPU_GPU_ID_H_ + +#include "tensorflow/core/lib/gtl/int_type.h" + +namespace tensorflow { + +// There are three types of GPU ids: +// - *physical* GPU id: this is the integer index of a GPU hardware in the +// physical machine, it can be filtered by CUDA environment variable +// CUDA_VISIBLE_DEVICES. Note that this id is not visible to Tensorflow, but +// result after filtering by CUDA_VISIBLE_DEVICES is visible to TF and is +// called platform GPU id as below. See +// http://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#env-vars +// for more details. +// - *platform* GPU id (also called *visible* GPU id in +// third_party/tensorflow/core/protobuf/config.proto): this is the id that is +// visible to Tensorflow after filtering by CUDA_VISIBLE_DEVICES, and is +// generated by the CUDA GPU driver. It starts from 0 and is used for CUDA API +// calls like cuDeviceGet(). +// - TF GPU id (also called *virtual* GPU id in +// third_party/tensorflow/core/protobuf/config.proto): this is the id that +// Tensorflow generates and exposes to its users. It is the id in the +// field of the device name "/device:GPU:", and is also the identifier of +// a BaseGPUDevice. Note that the configuration allows us to create multiple +// BaseGPUDevice per GPU hardware in order to use multi CUDA streams on the +// hardware, so the mapping between TF GPU id and platform GPU id is not a 1:1 +// mapping, see the example below. +// +// For example, assuming that in the machine we have GPU device with index 0, 1, +// 2 and 3 (physical GPU id). Setting "CUDA_VISIBLE_DEVICES=1,2,3" will create +// the following mapping between platform GPU id and physical GPU id: +// +// platform GPU id -> physical GPU id +// 0 -> 1 +// 1 -> 2 +// 2 -> 3 +// +// Note that physical GPU id 0 is invisible to TF so there is no mapping entry +// for it. +// +// Assuming we configure the Session to create one BaseGPUDevice per GPU +// hardware, then setting GPUOptions::visible_device_list to "2,0" will create +// the following mappting between TF GPU id and platform GPU id: +// +// TF GPU id -> platform GPU ID +// 0 (i.e. /device:GPU:0) -> 2 +// 1 (i.e. /device:GPU:1) -> 0 +// +// Note that platform GPU id 1 is filtered out by +// GPUOptions::visible_device_list, so it won't be used by the TF process. +// +// On the other hand, if we configure it to create 2 BaseGPUDevice per GPU +// hardware, then setting GPUOptions::visible_device_list to "2,0" will create +// the following mappting between TF GPU id and platform GPU id: +// +// TF GPU id -> platform GPU ID +// 0 (i.e. /device:GPU:0) -> 2 +// 1 (i.e. /device:GPU:1) -> 2 +// 2 (i.e. /device:GPU:2) -> 0 +// 3 (i.e. /device:GPU:3) -> 0 +// +// We create strong-typed integer classes for both TF GPU id and platform GPU id +// to minimize programming errors and improve code readability. Except for the +// StreamExecutor interface (as we don't change its API), whenever we need a +// TF GPU id (or platform GPU id) we should use TfGpuId (or PlatformGpuId) +// instead of a raw integer. +TF_LIB_GTL_DEFINE_INT_TYPE(TfGpuId, int32); +TF_LIB_GTL_DEFINE_INT_TYPE(PlatformGpuId, int32); + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_GPU_GPU_ID_H_ diff --git a/linux/include/tensorflow/core/common_runtime/gpu/gpu_id_manager.h b/linux/include/tensorflow/core/common_runtime/gpu/gpu_id_manager.h new file mode 100644 index 000000000..62df4310c --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/gpu/gpu_id_manager.h @@ -0,0 +1,42 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_GPU_GPU_ID_MANAGER_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_GPU_GPU_ID_MANAGER_H_ + +#include "tensorflow/core/common_runtime/gpu/gpu_id.h" +#include "tensorflow/core/lib/core/status.h" + +namespace tensorflow { + +// Class that maintains a map from TfGpuId to PlatformGpuId, and manages the +// translation between them. +class GpuIdManager { + public: + // Adds a mapping from tf_gpu_id to platform_gpu_id. + static Status InsertTfPlatformGpuIdPair(TfGpuId tf_gpu_id, + PlatformGpuId platform_gpu_id); + + // Gets the platform_gpu_id associated with tf_gpu_id. Returns OK if found. + static Status TfToPlatformGpuId(TfGpuId tf_gpu_id, + PlatformGpuId* platform_gpu_id); + + // Clears the map. Used in unit tests only. + static void TestOnlyReset(); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_GPU_GPU_ID_MANAGER_H_ diff --git a/linux/include/tensorflow/core/common_runtime/gpu/gpu_id_utils.h b/linux/include/tensorflow/core/common_runtime/gpu/gpu_id_utils.h new file mode 100644 index 000000000..b1f10fb1d --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/gpu/gpu_id_utils.h @@ -0,0 +1,64 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_GPU_GPU_ID_UTILS_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_GPU_GPU_ID_UTILS_H_ + +#include "tensorflow/core/common_runtime/gpu/gpu_id.h" +#include "tensorflow/core/common_runtime/gpu/gpu_id_manager.h" +#include "tensorflow/core/common_runtime/gpu/gpu_init.h" +#include "tensorflow/core/lib/gtl/int_type.h" +#include "tensorflow/core/platform/stream_executor.h" + +namespace tensorflow { + +// Utility methods for translation between Tensorflow GPU ids and platform GPU +// ids. +class GpuIdUtil { + public: + // Convenient methods for getting the associated executor given a TfGpuId or + // PlatformGpuId. + static se::port::StatusOr ExecutorForPlatformGpuId( + se::Platform* gpu_manager, PlatformGpuId platform_gpu_id) { + return gpu_manager->ExecutorForDevice(platform_gpu_id.value()); + } + static se::port::StatusOr ExecutorForPlatformGpuId( + PlatformGpuId platform_gpu_id) { + return ExecutorForPlatformGpuId(GPUMachineManager(), platform_gpu_id); + } + static se::port::StatusOr ExecutorForTfGpuId( + TfGpuId tf_gpu_id) { + PlatformGpuId platform_gpu_id; + TF_RETURN_IF_ERROR( + GpuIdManager::TfToPlatformGpuId(tf_gpu_id, &platform_gpu_id)); + return ExecutorForPlatformGpuId(platform_gpu_id); + } + + // Verify that the platform_gpu_id associated with a TfGpuId is legitimate. + static void CheckValidTfGpuId(TfGpuId tf_gpu_id) { + PlatformGpuId platform_gpu_id; + TF_CHECK_OK(GpuIdManager::TfToPlatformGpuId(tf_gpu_id, &platform_gpu_id)); + const int visible_device_count = GPUMachineManager()->VisibleDeviceCount(); + CHECK_LT(platform_gpu_id.value(), visible_device_count) + << "platform_gpu_id is outside discovered device range." + << " TF GPU id: " << tf_gpu_id + << " platform GPU id: " << platform_gpu_id + << " visible device count: " << visible_device_count; + } +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_GPU_GPU_ID_UTILS_H_ diff --git a/linux/include/tensorflow/core/common_runtime/gpu/gpu_init.h b/linux/include/tensorflow/core/common_runtime/gpu/gpu_init.h new file mode 100644 index 000000000..4e1f06ac8 --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/gpu/gpu_init.h @@ -0,0 +1,39 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_GPU_GPU_INIT_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_GPU_GPU_INIT_H_ + +#include "tensorflow/core/lib/core/status.h" + +namespace stream_executor { +class Platform; +} // namespace stream_executor + +namespace tensorflow { + +// Initializes the CUDA platform and returns OK if the CUDA +// platform could be initialized. +Status ValidateGPUMachineManager(); + +// Returns the GPU machine manager singleton, creating it and +// initializing the GPUs on the machine if needed the first time it is +// called. Must only be called when there is a valid GPU environment +// in the process (e.g., ValidateGPUMachineManager() returns OK). +stream_executor::Platform* GPUMachineManager(); + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_GPU_GPU_INIT_H_ diff --git a/linux/include/tensorflow/core/common_runtime/gpu/gpu_managed_allocator.h b/linux/include/tensorflow/core/common_runtime/gpu/gpu_managed_allocator.h new file mode 100644 index 000000000..2d49a64c0 --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/gpu/gpu_managed_allocator.h @@ -0,0 +1,36 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_GPU_GPU_MANAGED_ALLOCATOR_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_GPU_GPU_MANAGED_ALLOCATOR_H_ + +#include "tensorflow/core/framework/allocator.h" + +namespace tensorflow { + +// An allocator for CUDA unified memory. Memory allocated with this allocator +// can be accessed from both host and device. CUDA transparently migrates dirty +// pages, which can be slow. Therefore, this allocator is intended for +// convenience in functional tests only. +class GpuManagedAllocator : public Allocator { + public: + string Name() override { return "GpuManagedAllocator"; } + void* AllocateRaw(size_t alignment, size_t num_bytes) override; + void DeallocateRaw(void* ptr) override; +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_GPU_GPU_MANAGED_ALLOCATOR_H_ diff --git a/linux/include/tensorflow/core/common_runtime/gpu/gpu_process_state.h b/linux/include/tensorflow/core/common_runtime/gpu/gpu_process_state.h new file mode 100644 index 000000000..43e9a3166 --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/gpu/gpu_process_state.h @@ -0,0 +1,137 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_GPU_GPU_PROCESS_STATE_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_GPU_GPU_PROCESS_STATE_H_ + +#include +#include +#include +#include + +#include "tensorflow/core/common_runtime/gpu/gpu_id.h" +#include "tensorflow/core/common_runtime/process_state.h" +#include "tensorflow/core/framework/allocator.h" +#include "tensorflow/core/platform/mutex.h" +#include "tensorflow/core/platform/thread_annotations.h" +#include "tensorflow/core/platform/types.h" +#include "tensorflow/core/protobuf/config.pb.h" + +namespace tensorflow { + +class Allocator; +class PoolAllocator; + +// Singleton that manages per-process state when GPUs are present. +class GPUProcessState { + public: + static GPUProcessState* singleton(); + + // Query whether any GPU device has been created so far. + // Disable thread safety analysis since a race is benign here. + bool HasGPUDevice() const NO_THREAD_SAFETY_ANALYSIS { + return gpu_device_enabled_; + } + + // Set the flag to indicate a GPU device has been created. + // Disable thread safety analysis since a race is benign here. + void EnableGPUDevice() NO_THREAD_SAFETY_ANALYSIS { + gpu_device_enabled_ = true; + } + + // Returns the one GPU allocator used for the indexed GPU. + // Note that this is a system GPU index, not (necessarily) a brain + // device index. + // + // 'total_bytes' is the total number of bytes that should be made + // available to the allocator. The first call to this function for + // a given tf_gpu_id creates the allocator, so only the total_bytes + // used on that first call is used. + // + // "Allocator type" describes the type of algorithm to use for the + // underlying allocator. REQUIRES: Must be a valid type (see + // config.proto for the list of supported strings.). + // + // REQUIRES: tf_gpu_id must be a valid id for a BaseGPUDevice available in the + // current system environment. Otherwise returns nullptr. + virtual Allocator* GetGPUAllocator(const GPUOptions& options, + TfGpuId tf_gpu_id, size_t total_bytes); + + virtual Allocator* GetCUDAHostAllocator(int numa_node); + + // Registers a Visitor to be invoked on new chunks of memory allocated by the + // SubAllocator of every GPU proximate to the specified bus. The AllocVisitor + // is provided with a memory pointer, a GPU id, and the size of the area it + // identifies. The pointer is not guaranteed to be valid after the call + // terminates. The intention is for this interface to be used for network + // device memory registration. "bus_id" is platform-specific. On many + // platforms it should be 0. On machines with multiple PCIe buses, it should + // be the index of one of the PCIe buses (maybe the NUMA node at which the + // PCIe is rooted). If the bus_id is invalid, results are undefined. + virtual void AddGPUAllocVisitor(int bus_id, + const SubAllocator::Visitor& visitor); + + // Registers a Visitor to be invoked on new chunks of memory allocated by + // the SubAllocator of the CUDAHostAllocator for the given numa_node. + virtual void AddCUDAHostAllocVisitor(int numa_node, + const SubAllocator::Visitor& visitor); + + // Registers a Visitor to be invoked on each chunk handed back for freeing to + // the SubAllocator of the CUDAHostAllocator for the given numa_node. + virtual void AddCUDAHostFreeVisitor(int numa_node, + const SubAllocator::Visitor& visitor); + + // Returns bus_id for the given GPU id. + virtual int BusIdForGPU(TfGpuId tf_gpu_id); + + protected: + GPUProcessState(); + + // Helper method for unit tests to reset the ProcessState singleton by + // cleaning up everything. Never use in production. + virtual void TestOnlyReset(); + + ProcessState::MDMap* mem_desc_map() { + if (process_state_) return &process_state_->mem_desc_map_; + return nullptr; + } + + static GPUProcessState* instance_; + ProcessState* process_state_; // Not owned. + bool gpu_device_enabled_; + + mutex mu_; + + struct AllocatorParts { + std::unique_ptr allocator; + SubAllocator* sub_allocator; // owned by allocator + std::unique_ptr recording_allocator; + }; + std::vector gpu_allocators_ GUARDED_BY(mu_); + std::vector> gpu_visitors_ GUARDED_BY(mu_); + + std::vector cuda_host_allocators_ GUARDED_BY(mu_); + std::vector> cuda_host_alloc_visitors_ + GUARDED_BY(mu_); + std::vector> cuda_host_free_visitors_ + GUARDED_BY(mu_); + + virtual ~GPUProcessState(); + + friend class GPUDeviceTest; +}; + +} // namespace tensorflow +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_GPU_GPU_PROCESS_STATE_H_ diff --git a/linux/include/tensorflow/core/common_runtime/gpu/gpu_stream_util.h b/linux/include/tensorflow/core/common_runtime/gpu/gpu_stream_util.h new file mode 100644 index 000000000..c61ada96e --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/gpu/gpu_stream_util.h @@ -0,0 +1,45 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_GPU_GPU_STREAM_UTIL_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_GPU_GPU_STREAM_UTIL_H_ + +#include + +#include "tensorflow/core/graph/graph.h" +#include "tensorflow/core/lib/core/status.h" + +namespace tensorflow { +namespace gpu_stream_util { + +struct AssignStreamsOpts { + int32 max_streams = 1; + // The following options specify a stream to use for specific op + // types. The value -1 allows ops to be assigned to any stream. + int32 send_stream = -1; + int32 recv_stream = -1; + int32 const_stream = -1; + int32 compute_stream = -1; +}; + +// Given the input graph, assigns every node in the graph with a +// stream_id that should be used. +Status AssignStreams(const Graph* graph, const AssignStreamsOpts& opts, + std::unordered_map* node_to_stream_id); + +} // namespace gpu_stream_util +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_GPU_GPU_STREAM_UTIL_H_ diff --git a/linux/include/tensorflow/core/common_runtime/gpu/gpu_util.h b/linux/include/tensorflow/core/common_runtime/gpu/gpu_util.h new file mode 100644 index 000000000..8ac3febb0 --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/gpu/gpu_util.h @@ -0,0 +1,111 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_GPU_GPU_UTIL_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_GPU_GPU_UTIL_H_ + +#include "tensorflow/core/common_runtime/device.h" +#include "tensorflow/core/common_runtime/dma_helper.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/platform/stream_executor.h" + +namespace tensorflow { + +class RecvTensorResponse; +class TensorProto; + +class GPUUtil { + public: + // "tensor" is GPU-local. "dev" is the hosting GPU. + // "device_context" should be the context of the GPU "_Send" op + // which provides the Tensor. + // Sets all necessary fields of "proto" by transferring value + // bytes from GPU to CPU RAM. "is_dead" indicates that the + // tensor is dead with an uninit value. + static void SetProtoFromGPU(const Tensor& tensor, Device* dev, + const DeviceContext* device_context, + TensorProto* proto, bool is_dead, + StatusCallback done); + + // Copies the data in 'gpu_tensor' into 'cpu_tensor'. + // 'gpu_tensor''s backing memory must be on 'gpu_device' and + // 'cpu_tensor' must be allocated to be of the same size as + // 'gpu_tensor'. Synchronous: may block. + static void CopyGPUTensorToCPU(Device* gpu_device, + const DeviceContext* device_context, + const Tensor* gpu_tensor, Tensor* cpu_tensor, + StatusCallback done); + + // Blocks until all operations queued on the stream associated with + // "gpu_device" at the time of the call have completed. Returns any + // error pending on the stream at completion. + static Status Sync(Device* gpu_device); + + // Blocks until all operations queued on all streams associated with the + // corresponding GPU device at the time of call have completed. + // Returns any error pending on the stream at completion. + static Status SyncAll(Device* gpu_device); + + // For debugging purpose, given a "device" and a "tensor" allocated + // on the device, return a string printing each byte in the tensor + // (up to a limit). "device" can be either a CPU or a GPU device. + static string MemoryDebugString(const Device* device, Tensor* tensor); + + // Map a Tensor as a DeviceMemory object wrapping the given typed + // buffer. + // + // NOTE: will be removed soon, see StreamExecutorUtil::AsDeviceMemory + // instead. + template + static se::DeviceMemory AsDeviceMemory(const Tensor& t) { + T* ptr = reinterpret_cast(const_cast(DMAHelper::base(&t))); + return se::DeviceMemory(se::DeviceMemoryBase(ptr, t.TotalBytes())); + } + + // Computes a checksum over the contents of "tensor", which is allocated + // on "gpu_device". + static uint64 Checksum(Device* gpu_device, + const DeviceContext* device_context, + const Tensor& tensor); + + // Computes a checksum over the contents of "tensor", which is allocated + // in local CPU RAM. + static uint64 Checksum(const Tensor& tensor); + + static void CopyCPUTensorToGPU(const Tensor* cpu_tensor, + const DeviceContext* device_context, + Device* gpu_device, Tensor* gpu_tensor, + StatusCallback done); + + static void DeviceToDeviceCopy( + DeviceContext* send_dev_context, DeviceContext* recv_dev_context, + Device* src, Device* dst, AllocatorAttributes src_alloc_attr, + AllocatorAttributes dst_alloc_attr, const Tensor* input, Tensor* output, + int dev_to_dev_stream_index, StatusCallback done); + + // Deep-copying of GPU tensor on the same device. + // 'src_gpu_tensor''s and 'dst_gpu_tensor''s backing memory must be on + // 'gpu_device' and 'dst_cpu_tensor' must be allocated to be of the same + // size as 'src_gpu_tensor'. + static void CopyGPUTensorToSameGPU(Device* gpu_device, + const DeviceContext* device_context, + const Tensor* src_gpu_tensor, + Tensor* dst_gpu_tensor, + StatusCallback done); +}; + +} // namespace tensorflow +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_GPU_GPU_UTIL_H_ diff --git a/linux/include/tensorflow/core/common_runtime/gpu_device_context.h b/linux/include/tensorflow/core/common_runtime/gpu_device_context.h new file mode 100644 index 000000000..360380815 --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/gpu_device_context.h @@ -0,0 +1,81 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_GPU_DEVICE_CONTEXT_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_GPU_DEVICE_CONTEXT_H_ + +#include "tensorflow/core/common_runtime/device.h" +#include "tensorflow/core/framework/device_base.h" +#include "tensorflow/core/lib/gtl/inlined_vector.h" + +namespace stream_executor { +class Stream; +} // namespace stream_executor + +namespace tensorflow { + +class GPUDeviceContext : public DeviceContext { + public: + // Does not take ownership of streams. + GPUDeviceContext(int stream_id, se::Stream* stream, + se::Stream* host_to_device_stream, + se::Stream* device_to_host_stream, + gtl::InlinedVector device_to_device_stream) + : stream_id_(stream_id), + stream_(stream), + host_to_device_stream_(host_to_device_stream), + device_to_host_stream_(device_to_host_stream), + device_to_device_stream_(device_to_device_stream) {} + + ~GPUDeviceContext() override {} + + se::Stream* stream() const override { return stream_; } + se::Stream* host_to_device_stream() const { return host_to_device_stream_; } + se::Stream* device_to_host_stream() const { return device_to_host_stream_; } + se::Stream* device_to_device_stream(int index) const { + return device_to_device_stream_[index % device_to_device_stream_.size()]; + } + int stream_id() const { return stream_id_; } + + void CopyCPUTensorToDevice(const Tensor* cpu_tensor, Device* device, + Tensor* device_tensor, + StatusCallback done) const override; + + void CopyDeviceTensorToCPU(const Tensor* device_tensor, StringPiece edge_name, + Device* device, Tensor* cpu_tensor, + StatusCallback done) override; + + void MaintainLifetimeOnStream(const Tensor* t, + se::Stream* stream) const override {} + + Status ThenExecute(Device* device, se::Stream* stream, + std::function func) override; + + private: + int stream_id_; + // The default primary stream to use for this context. + // All the memory belongs to this stream. + se::Stream* stream_; + // The stream to use for copying data from host into GPU. + se::Stream* host_to_device_stream_; + // The stream to use for copying data from GPU to host. + se::Stream* device_to_host_stream_; + // Streams to use for copying data between GPUs. + gtl::InlinedVector device_to_device_stream_; +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_GPU_DEVICE_CONTEXT_H_ diff --git a/linux/include/tensorflow/core/common_runtime/graph_execution_state.h b/linux/include/tensorflow/core/common_runtime/graph_execution_state.h new file mode 100644 index 000000000..9cabe478a --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/graph_execution_state.h @@ -0,0 +1,215 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_GRAPH_EXECUTION_STATE_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_GRAPH_EXECUTION_STATE_H_ + +#include +#include +#include +#include + +#include "tensorflow/core/common_runtime/build_graph_options.h" +#include "tensorflow/core/common_runtime/device.h" +#include "tensorflow/core/common_runtime/device_set.h" +#include "tensorflow/core/framework/graph.pb.h" +#include "tensorflow/core/graph/costmodel.h" +#include "tensorflow/core/graph/graph.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { +struct SessionOptions; + +namespace subgraph { +struct RewriteGraphMetadata; +} + +struct GraphExecutionStateOptions { + const DeviceSet* device_set = nullptr; + const SessionOptions* session_options = nullptr; + // A map from node name to device name, representing the unchangeable + // placement of stateful nodes. + std::unordered_map stateful_placements; +}; + +// A ClientGraph is simply a sub-graph of the full graph as induced by +// BuildGraphOptions. +struct ClientGraph { + explicit ClientGraph(std::unique_ptr flib, + DataTypeVector feed_types, DataTypeVector fetch_types, + int64 collective_graph_key) + : flib_def(std::move(flib)), + graph(flib_def.get()), + feed_types(std::move(feed_types)), + fetch_types(std::move(fetch_types)), + collective_graph_key(collective_graph_key) {} + // Each client-graph gets its own function library since optimization passes + // post rewrite for execution might want to introduce new functions. + std::unique_ptr flib_def; + Graph graph; + DataTypeVector feed_types; + DataTypeVector fetch_types; + int64 collective_graph_key; +}; + +// GraphExecutionState is responsible for generating an +// executable ClientGraph from the original GraphDef that specifies +// the complete graph and from BuildGraphOptions which specifies +// input/output nodes. +// +// An executable Graph differs from a GraphDef by being Placed, +// meaning that each Node is assigned to a single Device in the +// available set. +// +// When GraphExecutionState is first constructed it instantiates +// a full Graph from the provided GraphDef, and places it, using only +// the static device assignments from the GraphDef. Nodes without are +// currently placed in a very naive way. Since stateful Nodes cannot +// be moved after initial placement, it is important that stateful +// Nodes get sensible initial device assignments in the graph +// definition. +// +// Subsequently, GraphExecutionState generates a SimpleClientGraph on +// demand, which is a sub-graph of the latest placement of the full +// Graph. MasterSession uses such a ClientGraph to execute one or +// more similar client requests. +// +// GraphExecutionState is thread-safe. + +class GraphExecutionState { + public: + virtual ~GraphExecutionState(); + + // Creates a new `GraphExecutionState` for the given + // `graph_def`, which represents the entire graph for a session. + // + // N.B. This method uses `GraphDef::Swap()` and leaves `graph_def` + // in an undefined state. If it is necessary to use `*graph_def` + // after this call, make an explicit copy of the graph before + // calling this method. + static Status MakeForBaseGraph( + GraphDef* graph_def, const GraphExecutionStateOptions& options, + std::unique_ptr* out_state); + + // Creates a new `GraphExecutionState` and `SimpleClientGraph` + // for the subgraph of `original_graph_def` defined by + // `subgraph_options`. + static Status MakeForPrunedGraph( + const FunctionDefLibrary& func_def_lib, + const GraphExecutionStateOptions& options, + const GraphDef& original_graph_def, + const BuildGraphOptions& subgraph_options, + std::unique_ptr* out_state, + std::unique_ptr* out_client_graph); + + // Creates a new GraphExecutionState representing the + // concatenation of this graph, and the graph defined by + // "extension_def". The same name may not be used to define a node + // in both this graph and "extension_def". + // + // If successful, returns OK and the caller takes ownership of "*out". + // Otherwise returns an error and does not modify "*out". + // + // After calling `old_state->Extend()`, `old_state` may no longer be + // used. + // + // NOTE(mrry): This method respects the placement of stateful nodes in + // in *this, but currently does not transfer any other placement + // or cost model information to the new graph. + Status Extend(const GraphDef& extension_def, + std::unique_ptr* out) const; + + // Builds a ClientGraph (a sub-graph of the full graph as induced by + // the Node set specified in "options"). If successful, returns OK + // and the caller takes the ownership of "*out". Otherwise, returns + // an error. + Status BuildGraph(const BuildGraphOptions& options, + std::unique_ptr* out); + + // The graph returned by BuildGraph may contain only the pruned + // graph, whereas some clients may want access to the full graph. + const Graph* full_graph() { return graph_; } + + // Returns the node with the given name, or null if it does not exist. + const Node* get_node_by_name(const string& name) const { + NodeNameToCostIdMap::const_iterator iter = + node_name_to_cost_id_map_.find(name); + if (iter != node_name_to_cost_id_map_.end()) { + return graph_->FindNodeId(iter->second); + } else { + return nullptr; + } + } + + // Returns a reference to the current graph_def. Use must + // not extend beyond lifetime of GrahExecutionState object. + const GraphDef& original_graph_def() { return original_graph_def_; } + + // Returns the map of stateful placements as a map of + // node name to placement string. + std::unordered_map GetStatefulPlacements() const { + return stateful_placements_; + } + + private: + GraphExecutionState(GraphDef* graph_def, + const GraphExecutionStateOptions& options); + + Status InitBaseGraph(const BuildGraphOptions& options); + + // Map of placed stateful nodes, i.e. nodes for which is_stateful() + // is true, such as "params" and "queue" nodes. Once placed these + // nodes can not be moved to a different device. Maps node names to + // device names. + std::unordered_map stateful_placements_; // Immutable after + // ctor. + void SaveStatefulNodes(Graph* graph); + void RestoreStatefulNodes(Graph* graph); + + // Extract the subset of the graph that needs to be run, adding feed/fetch + // ops as needed. + Status PruneGraph(const BuildGraphOptions& options, Graph* graph, + subgraph::RewriteGraphMetadata* out_rewrite_metadata); + + Status OptimizeGraph( + const BuildGraphOptions& options, std::unique_ptr* optimized_graph, + std::unique_ptr* optimized_flib); + + GraphDef original_graph_def_; // Immutable after ctor. + const DeviceSet* device_set_; // Not owned + const SessionOptions* session_options_; // Not owned + + // Map from name to Node for the full graph in placed_. + NodeNameToCostIdMap node_name_to_cost_id_map_; + + // 'flib_def_' is initialized from the initial graph def's library, + // and may be updated by a graph optimization pass. + std::unique_ptr flib_def_; + + // `rewrite_metadata_` is only set for GraphExecutionState + // objects created by `MakeForPrunedGraph()`. + std::unique_ptr rewrite_metadata_; + + // The dataflow graph owned by this object. + Graph* graph_; + + TF_DISALLOW_COPY_AND_ASSIGN(GraphExecutionState); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_GRAPH_EXECUTION_STATE_H_ diff --git a/linux/include/tensorflow/core/common_runtime/graph_optimizer.h b/linux/include/tensorflow/core/common_runtime/graph_optimizer.h new file mode 100644 index 000000000..789cc5694 --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/graph_optimizer.h @@ -0,0 +1,68 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_GRAPH_OPTIMIZER_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_GRAPH_OPTIMIZER_H_ + +#include "tensorflow/core/framework/function.h" +#include "tensorflow/core/graph/graph.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/platform/env.h" +#include "tensorflow/core/protobuf/config.pb.h" + +namespace tensorflow { + +class GraphOptimizer { + public: + GraphOptimizer(const OptimizerOptions& opts); + ~GraphOptimizer(); + + // Applies optimization passes specified in 'opts' to 'graph'. + // Maybe replace *graph with a new graph object. 'device' is device + // on which the 'graph' will execute. It's passed to the optimizers + // so that they can respect constraints if any, that should be + // respected. + // + // If shape_map is not null it maps from nodes in graph to partially-known + // shapes of their outputs, and may be used, e.g., in the constant folding + // pass. The use of shape_map implies that the mapping from node name to the + // vector of partial shapes of its outputs is stable, i.e., no optimization + // pass may replace a node with a different node of the same name that has a + // different number of outputs, or outputs with different known shapes. + // TODO(b/65453533) introduce a unique way to name nodes in a graph. + // + // If cse_consider_fn is not null then only nodes for which cse_consider_fn + // returns true will be considered for CSE. + // If cf_consider_fn is not null then only nodes for which cf_consider_fn + // returns true will be considered for CF. + void Optimize( + FunctionLibraryRuntime* runtime, Env* env, Device* device, + std::unique_ptr* graph, + const std::unordered_map>* + shape_map, + const std::function& cse_consider_fn = nullptr, + const std::function& cf_consider_fn = nullptr); + + const OptimizerOptions& options() { return opts_; } + + private: + OptimizerOptions opts_; + + TF_DISALLOW_COPY_AND_ASSIGN(GraphOptimizer); +}; + +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_GRAPH_OPTIMIZER_H_ diff --git a/linux/include/tensorflow/core/common_runtime/graph_runner.h b/linux/include/tensorflow/core/common_runtime/graph_runner.h new file mode 100644 index 000000000..1c4b2b719 --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/graph_runner.h @@ -0,0 +1,70 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_GRAPH_RUNNER_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_GRAPH_RUNNER_H_ + +#include +#include +#include + +#include "tensorflow/core/common_runtime/device.h" +#include "tensorflow/core/framework/function.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/graph/graph.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/platform/env.h" + +namespace tensorflow { + +// GraphRunner takes a Graph, some inputs to feed, and some outputs +// to fetch and executes the graph required to feed and fetch the +// inputs and outputs. +// +// This class is only meant for internal use where one needs to +// partially evaluate inexpensive nodes in a graph, such as for shape +// inference or for constant folding. Because of its limited, simple +// use-cases, it executes all computation on the given device (CPU by default) +// and is not meant to be particularly lightweight, fast, or efficient. +class GraphRunner { + public: + // REQUIRES: `env` is not nullptr. + GraphRunner(Env* env); + // REQUIRES: 'device' is not nullptr. Not owned. + GraphRunner(Device* device); + ~GraphRunner(); + + // Function semantics for `inputs`, `output_names` and `outputs` + // matches those from Session::Run(). + // + // NOTE: The output tensors share lifetime with the GraphRunner, and could + // be destroyed once the GraphRunner is destroyed. + // + // REQUIRES: `graph`, `env`, and `outputs` are not nullptr. + // `function_library` may be nullptr. + typedef std::vector> NamedTensorList; + Status Run(Graph* graph, FunctionLibraryRuntime* function_library, + const NamedTensorList& inputs, + const std::vector& output_names, + std::vector* outputs); + + private: + std::unique_ptr device_deleter_; + Device* const device_; +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_GRAPH_RUNNER_H_ diff --git a/linux/include/tensorflow/core/common_runtime/hierarchical_tree_broadcaster.h b/linux/include/tensorflow/core/common_runtime/hierarchical_tree_broadcaster.h new file mode 100644 index 000000000..ceb9baad3 --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/hierarchical_tree_broadcaster.h @@ -0,0 +1,85 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_HIERARCHICAL_TREE_BROADCASTER_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_HIERARCHICAL_TREE_BROADCASTER_H_ + +#include + +#include "tensorflow/core/common_runtime/base_collective_executor.h" +#include "tensorflow/core/framework/collective.h" + +namespace tensorflow { + +// Hierarchical tree-algorithm implementation of collective broadcast. +class HierarchicalTreeBroadcaster : public CollectiveImplementationInterface { + public: + HierarchicalTreeBroadcaster(); + ~HierarchicalTreeBroadcaster() override = default; + + // Establishes the subdiv permutations needed for a hierarchical broadcast. + // If all devices are local, establishes a single subdiv comprising all + // devices. If any devices are on a different task, establishes n+1 subdivs + // for n tasks. + // The first subdiv comprises one device per task which gets the tensor on + // each task. Subdiv i+1 corresponds to a task-local tree-broadcast for task + // i. + Status InitializeCollectiveParams(CollectiveParams* col_params) override; + + // Initializes members of CollectiveContext not yet initialized, i.e. device + // and device_locality. Also saves the CollectiveContext in this object. + Status InitializeCollectiveContext(CollectiveContext* col_ctx) override; + + // Begins async execution of the hierarchical tree broadcast. + // Must be called in a blockable thread. + // TODO(b/80529858): remove the previous warning when we have a dedicated + // collective threadpool. + void Run(StatusCallback done) override; + + // Returns the rank of the device from which this device should receive + // its value, -1 if no value should be received. + static int TreeRecvFrom(const CollectiveParams& cp, int subdiv); + + // Populates targets with the ranks of the devices to which this device + // should forward the value. + static void TreeSendTo(const CollectiveParams& cp, int subdiv, + std::vector* targets); + + private: + // Get the task to which the device at `device_rank` belongs. + int GetDeviceTask(int device_rank, const std::vector& dev_per_task); + + // Sends `src_tensor` asynchronously from this device to device at `dst_rank` + // in `subdiv`. Calls `done` upon completion. + void DispatchSend(int subdiv, int dst_rank, int src_rank, + const Tensor* src_tensor, const StatusCallback& done); + + // Receives a tensor into the memory buffer owned by `dst_tensor` at this + // device from device at `src_rank` in `subdiv`. Calls `done` upon + // completion. + void DispatchRecv(int subdiv, int src_rank, int dst_rank, Tensor* dst_tensor, + const StatusCallback& done); + + // Executes the hierarchical broadcast defined by this op. + void RunTree(); + + CollectiveContext* col_ctx_; // Not owned + const CollectiveParams* col_params_; // Not owned + StatusCallback done_; + Status status_; + bool is_source_; +}; + +} // namespace tensorflow +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_HIERARCHICAL_TREE_BROADCASTER_H_ diff --git a/linux/include/tensorflow/core/common_runtime/local_device.h b/linux/include/tensorflow/core/common_runtime/local_device.h new file mode 100644 index 000000000..226f121bf --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/local_device.h @@ -0,0 +1,57 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_LOCAL_DEVICE_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_LOCAL_DEVICE_H_ + +#include "tensorflow/core/common_runtime/device.h" +#include "tensorflow/core/framework/device_attributes.pb.h" +#include "tensorflow/core/platform/macros.h" + +namespace tensorflow { + +namespace test { +class Benchmark; +} +struct SessionOptions; + +// This class is shared by ThreadPoolDevice and GPUDevice and +// initializes a shared Eigen compute device used by both. This +// should eventually be removed once we refactor ThreadPoolDevice and +// GPUDevice into more 'process-wide' abstractions. +class LocalDevice : public Device { + public: + LocalDevice(const SessionOptions& options, + const DeviceAttributes& attributes); + ~LocalDevice() override; + + private: + static bool use_global_threadpool_; + + static void set_use_global_threadpool(bool use_global_threadpool) { + use_global_threadpool_ = use_global_threadpool; + } + + struct EigenThreadPoolInfo; + std::unique_ptr owned_tp_info_; + + friend class test::Benchmark; + + TF_DISALLOW_COPY_AND_ASSIGN(LocalDevice); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_LOCAL_DEVICE_H_ diff --git a/linux/include/tensorflow/core/common_runtime/lower_if_op.h b/linux/include/tensorflow/core/common_runtime/lower_if_op.h new file mode 100644 index 000000000..5ab1123e3 --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/lower_if_op.h @@ -0,0 +1,39 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_LOWER_IF_OP_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_LOWER_IF_OP_H_ + +#include "tensorflow/core/common_runtime/optimization_registry.h" +#include "tensorflow/core/lib/core/status.h" + +namespace tensorflow { + +// Rewrite If ops to use switch and merge nodes instead. +class LowerIfOpPass : public GraphOptimizationPass { + public: + static const char* const kLowerUsingSwitchMergeAttr; + + Status Run(const GraphOptimizationPassOptions& options) override; + + private: + // Rewrite the given If node `n` in graph `g` to use the switch-merge + // form. `flib` should contain the branch functions referenced by `n`. + Status RewriteNode(Node* n, const FunctionLibraryDefinition& flib, Graph* g); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_LOWER_IF_OP_H_ diff --git a/linux/include/tensorflow/core/common_runtime/lower_while_op.h b/linux/include/tensorflow/core/common_runtime/lower_while_op.h new file mode 100644 index 000000000..eadafbeb9 --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/lower_while_op.h @@ -0,0 +1,37 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_LOWER_WHILE_OP_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_LOWER_WHILE_OP_H_ + +#include "tensorflow/core/common_runtime/optimization_registry.h" +#include "tensorflow/core/lib/core/status.h" + +namespace tensorflow { + +// Rewrite While ops to use lower level control flow primitives instead. +class LowerWhileOpPass : public GraphOptimizationPass { + public: + Status Run(const GraphOptimizationPassOptions& options) override; + + private: + // Rewrite the given While node `n` in graph `g` to use the lower level + // primitives Enter, Exit, Switch, Merge and NextIteration. + Status RewriteNode(Node* n, Graph* g); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_LOWER_WHILE_OP_H_ diff --git a/linux/include/tensorflow/core/common_runtime/memory_types.h b/linux/include/tensorflow/core/common_runtime/memory_types.h new file mode 100644 index 000000000..f854acfdc --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/memory_types.h @@ -0,0 +1,48 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_MEMORY_TYPES_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_MEMORY_TYPES_H_ + +#include "tensorflow/core/framework/memory_types.h" +#include "tensorflow/core/graph/graph.h" +#include "tensorflow/core/lib/core/status.h" + +namespace tensorflow { + +// Returns an error iff *g running on a single device of 'device_type' +// has memory type mismatch for any edge's source and destination. +Status ValidateMemoryTypes(const DeviceType& device_type, const Graph* g); + +// Updates '*g' so that every edge's source and destination has +// compatible memory types by inserting proper HostSend/Recv and +// Send/HostRecv nodes. 'device_type' specifies the type of device on +// which '*g' is going to run on and that device has the name +// 'device_name'. +// +// Returns OK if '*g' is updated properly (ValidateMemoryTypes(g) must +// be OK). Otherwise, returns an error and '*g' may be in an +// invalidate state and the caller should discard it. +Status EnsureMemoryTypes(const DeviceType& device_type, + const string& device_name, Graph* g); + +// Get the memory type for 'index'th output of node 'n' in graph 'g', when +// running on 'device_type'. +Status MemoryTypeForOutput(const DeviceType& device_type, const Graph* g, + const Node* n, int index, MemoryType* memory_type); + +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_MEMORY_TYPES_H_ diff --git a/linux/include/tensorflow/core/common_runtime/mkl_cpu_allocator.h b/linux/include/tensorflow/core/common_runtime/mkl_cpu_allocator.h new file mode 100644 index 000000000..429b19599 --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/mkl_cpu_allocator.h @@ -0,0 +1,315 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// A simple CPU allocator that intercepts malloc/free calls from MKL library +// and redirects them to Tensorflow allocator + +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_MKL_CPU_ALLOCATOR_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_MKL_CPU_ALLOCATOR_H_ + +#ifdef INTEL_MKL + +#include +#include "tensorflow/core/common_runtime/bfc_allocator.h" +#include "tensorflow/core/common_runtime/pool_allocator.h" +#include "tensorflow/core/lib/strings/numbers.h" +#include "tensorflow/core/lib/strings/str_util.h" +#include "tensorflow/core/platform/mem.h" +#include "tensorflow/core/platform/numa.h" + +#ifndef INTEL_MKL_DNN_ONLY +#include "i_malloc.h" +#endif + +#ifdef _WIN32 +typedef unsigned int uint; +#endif + +namespace tensorflow { + +class MklSubAllocator : public BasicCPUAllocator { + public: + MklSubAllocator() : BasicCPUAllocator(port::kNUMANoAffinity, {}, {}) {} + ~MklSubAllocator() override {} +}; + +// CPU allocator that handles small-size allocations by calling +// suballocator directly. Mostly, it is just a wrapper around a suballocator +// (that calls malloc and free directly) with support for bookkeeping. +class MklSmallSizeAllocator : public Allocator { + public: + MklSmallSizeAllocator(SubAllocator* sub_allocator, size_t total_memory, + const string& name) + : sub_allocator_(sub_allocator), name_(name) { + stats_.bytes_limit = total_memory; + } + ~MklSmallSizeAllocator() override {} + + TF_DISALLOW_COPY_AND_ASSIGN(MklSmallSizeAllocator); + + inline string Name() override { return name_; } + + void* AllocateRaw(size_t alignment, size_t num_bytes) override { + void* ptr = sub_allocator_->Alloc(alignment, num_bytes); + if (ptr != nullptr) { + std::pair map_val(ptr, num_bytes); + mutex_lock l(mutex_); + // Check that insertion in the hash map was successful. + CHECK(map_.insert(map_val).second); + // Increment statistics for small-size allocations. + IncrementStats(num_bytes); + } + return ptr; + } + + void DeallocateRaw(void* ptr) override { + if (ptr == nullptr) { + LOG(ERROR) << "tried to deallocate nullptr"; + return; + } + + mutex_lock l(mutex_); + auto map_iter = map_.find(ptr); + if (map_iter != map_.end()) { + // Call free visitors. + size_t dealloc_bytes = map_iter->second; + sub_allocator_->Free(ptr, dealloc_bytes); + DecrementStats(dealloc_bytes); + map_.erase(map_iter); + } else { + LOG(ERROR) << "tried to deallocate invalid pointer"; + return; + } + } + + inline bool IsSmallSizeAllocation(const void* ptr) const { + mutex_lock l(mutex_); + return map_.find(ptr) != map_.end(); + } + + void GetStats(AllocatorStats* stats) override { + mutex_lock l(mutex_); + *stats = stats_; + } + + void ClearStats() override { + mutex_lock l(mutex_); + stats_.Clear(); + } + + private: + // Increment statistics for the allocator handling small allocations. + inline void IncrementStats(size_t alloc_size) + EXCLUSIVE_LOCKS_REQUIRED(mutex_) { + ++stats_.num_allocs; + stats_.bytes_in_use += alloc_size; + stats_.max_bytes_in_use = + std::max(stats_.max_bytes_in_use, stats_.bytes_in_use); + stats_.max_alloc_size = + std::max(alloc_size, static_cast(stats_.max_alloc_size)); + } + + // Decrement statistics for the allocator handling small allocations. + inline void DecrementStats(size_t dealloc_size) + EXCLUSIVE_LOCKS_REQUIRED(mutex_) { + stats_.bytes_in_use -= dealloc_size; + } + + SubAllocator* sub_allocator_; // Not owned by this class. + + // Mutex for protecting updates to map of allocations. + mutable mutex mutex_; + + // Allocator name + string name_; + + // Hash map to keep track of "small" allocations + // We do not use BFC allocator for small allocations. + std::unordered_map map_ GUARDED_BY(mutex_); + + // Allocator stats for small allocs + AllocatorStats stats_ GUARDED_BY(mutex_); +}; + +/// CPU allocator for MKL that wraps BFC allocator and intercepts +/// and redirects memory allocation calls from MKL. +class MklCPUAllocator : public Allocator { + public: + // Constructor and other standard functions + + /// Environment variable that user can set to upper bound on memory allocation + static constexpr const char* kMaxLimitStr = "TF_MKL_ALLOC_MAX_BYTES"; + + /// Default upper limit on allocator size - 64GB + static constexpr size_t kDefaultMaxLimit = 64LL << 30; + + MklCPUAllocator() { TF_CHECK_OK(Initialize()); } + + ~MklCPUAllocator() override { + delete small_size_allocator_; + delete large_size_allocator_; + } + + Status Initialize() { + VLOG(2) << "MklCPUAllocator: In MklCPUAllocator"; + + // Set upper bound on memory allocation to physical RAM available on the + // CPU unless explicitly specified by user + uint64 max_mem_bytes = kDefaultMaxLimit; +#if defined(_SC_PHYS_PAGES) && defined(_SC_PAGESIZE) + max_mem_bytes = + (uint64)sysconf(_SC_PHYS_PAGES) * (uint64)sysconf(_SC_PAGESIZE); +#endif + char* user_mem_bytes = getenv(kMaxLimitStr); + + if (user_mem_bytes != NULL) { + uint64 user_val = 0; + if (!strings::safe_strtou64(user_mem_bytes, &user_val)) { + return errors::InvalidArgument("Invalid memory limit (", user_mem_bytes, + ") specified for MKL allocator through ", + kMaxLimitStr); + } +#if defined(_SC_PHYS_PAGES) && defined(_SC_PAGESIZE) + if (user_val > max_mem_bytes) { + LOG(WARNING) << "The user specified a memory limit " << kMaxLimitStr + << "=" << user_val + << " greater than available physical memory: " + << max_mem_bytes + << ". This could significantly reduce performance!"; + } +#endif + max_mem_bytes = user_val; + } + + VLOG(1) << "MklCPUAllocator: Setting max_mem_bytes: " << max_mem_bytes; + + sub_allocator_ = new MklSubAllocator(); + + // SubAllocator is owned by BFCAllocator, so we do not need to deallocate + // it in MklSmallSizeAllocator. + small_size_allocator_ = + new MklSmallSizeAllocator(sub_allocator_, max_mem_bytes, kName); + large_size_allocator_ = + new BFCAllocator(sub_allocator_, max_mem_bytes, kAllowGrowth, kName); +#ifndef INTEL_MKL_DNN_ONLY + // For redirecting all allocations from MKL to this allocator + // From: http://software.intel.com/en-us/node/528565 + i_malloc = MallocHook; + i_calloc = CallocHook; + i_realloc = ReallocHook; + i_free = FreeHook; +#endif + return Status::OK(); + } + + inline string Name() override { return kName; } + + inline void* AllocateRaw(size_t alignment, size_t num_bytes) override { + // If the allocation size is less than threshold, call small allocator, + // otherwise call large-size allocator (BFC). We found that BFC allocator + // does not deliver good performance for small allocations when + // inter_op_parallelism_threads is high. + return (num_bytes < kSmallAllocationsThreshold) + ? small_size_allocator_->AllocateRaw(alignment, num_bytes) + : large_size_allocator_->AllocateRaw(alignment, num_bytes); + } + + inline void DeallocateRaw(void* ptr) override { + // Check if ptr is for "small" allocation. If it is, then call Free + // directly. Otherwise, call BFC to handle free. + if (small_size_allocator_->IsSmallSizeAllocation(ptr)) { + small_size_allocator_->DeallocateRaw(ptr); + } else { + large_size_allocator_->DeallocateRaw(ptr); + } + } + + void GetStats(AllocatorStats* stats) override { + AllocatorStats l_stats, s_stats; + small_size_allocator_->GetStats(&s_stats); + large_size_allocator_->GetStats(&l_stats); + + // Combine statistics from small-size and large-size allocator. + stats->num_allocs = l_stats.num_allocs + s_stats.num_allocs; + stats->bytes_in_use = l_stats.bytes_in_use + s_stats.bytes_in_use; + stats->max_bytes_in_use = + l_stats.max_bytes_in_use + s_stats.max_bytes_in_use; + + // Since small-size allocations go to MklSmallSizeAllocator, + // max_alloc_size from large_size_allocator would be the maximum + // size allocated by MklCPUAllocator. + stats->max_alloc_size = l_stats.max_alloc_size; + stats->bytes_limit = std::max(s_stats.bytes_limit, l_stats.bytes_limit); + } + + void ClearStats() override { + small_size_allocator_->ClearStats(); + large_size_allocator_->ClearStats(); + } + + private: + // Hooks provided by this allocator for memory allocation routines from MKL + + static inline void* MallocHook(size_t size) { + VLOG(3) << "MklCPUAllocator: In MallocHook"; + return cpu_allocator()->AllocateRaw(kAlignment, size); + } + + static inline void FreeHook(void* ptr) { + VLOG(3) << "MklCPUAllocator: In FreeHook"; + cpu_allocator()->DeallocateRaw(ptr); + } + + static inline void* CallocHook(size_t num, size_t size) { + Status s = Status(error::Code::UNIMPLEMENTED, + "Unimplemented case for hooking MKL function."); + TF_CHECK_OK(s); // way to assert with an error message + return nullptr; // return a value and make static code analyzers happy + } + + static inline void* ReallocHook(void* ptr, size_t size) { + Status s = Status(error::Code::UNIMPLEMENTED, + "Unimplemented case for hooking MKL function."); + TF_CHECK_OK(s); // way to assert with an error message + return nullptr; // return a value and make static code analyzers happy + } + + // Do we allow growth in BFC Allocator + static const bool kAllowGrowth = true; + + // Name + static constexpr const char* kName = "mklcpu"; + + // The alignment that we need for the allocations + static constexpr const size_t kAlignment = 64; + + Allocator* large_size_allocator_; // owned by this class + MklSmallSizeAllocator* small_size_allocator_; // owned by this class. + + SubAllocator* sub_allocator_; // not owned by this class + + // Size in bytes that defines the upper-bound for "small" allocations. + // Any allocation below this threshold is "small" allocation. + static constexpr const size_t kSmallAllocationsThreshold = 4096; + + // Prevent copying and assignment + TF_DISALLOW_COPY_AND_ASSIGN(MklCPUAllocator); +}; + +} // namespace tensorflow + +#endif // INTEL_MKL + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_MKL_CPU_ALLOCATOR_H_ diff --git a/linux/include/tensorflow/core/common_runtime/optimization_registry.h b/linux/include/tensorflow/core/common_runtime/optimization_registry.h new file mode 100644 index 000000000..6fcd2afd2 --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/optimization_registry.h @@ -0,0 +1,145 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// Classes to maintain a static registry of whole-graph optimization +// passes to be applied by the Session when it initializes a graph. +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_OPTIMIZATION_REGISTRY_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_OPTIMIZATION_REGISTRY_H_ + +#include +#include +#include + +#include "tensorflow/core/common_runtime/device_set.h" +#include "tensorflow/core/framework/function.h" +#include "tensorflow/core/graph/costmodel.h" +#include "tensorflow/core/graph/graph.h" + +namespace tensorflow { +struct SessionOptions; + +// All the parameters used by an optimization pass are packaged in +// this struct. They should be enough for the optimization pass to use +// as a key into a state dictionary if it wants to keep state across +// calls. +struct GraphOptimizationPassOptions { + string session_handle; + const SessionOptions* session_options = nullptr; + const CostModel* cost_model = nullptr; + + FunctionLibraryDefinition* flib_def = nullptr; // Not owned. + // The DeviceSet contains all the devices known to the system and is + // filled in for optimizations run by the session master, i.e., + // PRE_PLACEMENT, POST_PLACEMENT, and POST_REWRITE_FOR_EXEC. It is + // nullptr for POST_PARTITIONING optimizations which are run at the + // workers. + const DeviceSet* device_set = nullptr; // Not owned. + + // The graph to optimize, for optimization passes that run before + // partitioning. Null for post-partitioning passes. + // An optimization pass may replace *graph with a new graph object. + std::unique_ptr* graph = nullptr; + + // Graphs for each partition, if running post-partitioning. Optimization + // passes may alter the graphs, but must not add or remove partitions. + // Null for pre-partitioning passes. + std::unordered_map>* partition_graphs = + nullptr; +}; + +// Optimization passes are implemented by inheriting from +// GraphOptimizationPass. +class GraphOptimizationPass { + public: + virtual ~GraphOptimizationPass() {} + virtual Status Run(const GraphOptimizationPassOptions& options) = 0; + void set_name(const string& name) { name_ = name; } + string name() const { return name_; } + + private: + // The name of the opitimization pass, which is the same as the inherited + // class name. + string name_; +}; + +// The key is a 'phase' number. Phases are executed in increasing +// order. Within each phase the order of passes is undefined. +typedef std::map>> + GraphOptimizationPasses; + +// A global OptimizationPassRegistry is used to hold all passes. +class OptimizationPassRegistry { + public: + // Groups of passes are run at different points in initialization. + enum Grouping { + PRE_PLACEMENT, // after cost model assignment, before placement. + POST_PLACEMENT, // after placement. + POST_REWRITE_FOR_EXEC, // after re-write using feed/fetch endpoints. + POST_PARTITIONING, // after partitioning + }; + + // Add an optimization pass to the registry. + void Register(Grouping grouping, int phase, + std::unique_ptr pass); + + // Run all passes in grouping, ordered by phase, with the same + // options. + Status RunGrouping(Grouping grouping, + const GraphOptimizationPassOptions& options); + + // Returns the global registry of optimization passes. + static OptimizationPassRegistry* Global(); + + // Prints registered optimization passes for debugging. + void LogGrouping(Grouping grouping, int vlog_level); + void LogAllGroupings(int vlog_level); + + private: + std::map groups_; +}; + +namespace optimization_registration { + +class OptimizationPassRegistration { + public: + OptimizationPassRegistration(OptimizationPassRegistry::Grouping grouping, + int phase, + std::unique_ptr pass, + string optimization_pass_name) { + pass->set_name(optimization_pass_name); + OptimizationPassRegistry::Global()->Register(grouping, phase, + std::move(pass)); + } +}; + +} // namespace optimization_registration + +#define REGISTER_OPTIMIZATION(grouping, phase, optimization) \ + REGISTER_OPTIMIZATION_UNIQ_HELPER(__COUNTER__, grouping, phase, optimization) + +#define REGISTER_OPTIMIZATION_UNIQ_HELPER(ctr, grouping, phase, optimization) \ + REGISTER_OPTIMIZATION_UNIQ(ctr, grouping, phase, optimization) + +#define REGISTER_OPTIMIZATION_UNIQ(ctr, grouping, phase, optimization) \ + static ::tensorflow::optimization_registration::OptimizationPassRegistration \ + register_optimization_##ctr( \ + grouping, phase, \ + ::std::unique_ptr<::tensorflow::GraphOptimizationPass>( \ + new optimization()), \ + #optimization) + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_OPTIMIZATION_REGISTRY_H_ diff --git a/linux/include/tensorflow/core/common_runtime/pending_counts.h b/linux/include/tensorflow/core/common_runtime/pending_counts.h new file mode 100644 index 000000000..5e1925c40 --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/pending_counts.h @@ -0,0 +1,331 @@ +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_PENDING_COUNTS_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_PENDING_COUNTS_H_ + +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#include "tensorflow/core/lib/gtl/flatmap.h" +#include "tensorflow/core/lib/hash/hash.h" +#include "tensorflow/core/platform/logging.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/util/port.h" + +namespace tensorflow { + +// PendingCounts is an internal helper class to keep track of pending and +// dead counts for nodes, for use in the ExecutorState module. It +// holds a map from Handles to various counts for that handle. This +// information is needed per frame iteration. The amount of memory +// needed for an iteration is the same across all executions of the +// iteration. The memory amount and handles are precomputed at startup +// using a Layout object. +// +// PendingCounts::Layout layout; +// std::vector h(C); +// for (int id = 0; id < C; id++) { +// h[id] = r.AddHandle(max_pending[id], max_dead[id]); +// } +// +// When we actually want to start an iteration we first create a +// PendingCounts object and then index into it using the precomputed +// handles: + +// PendingCounts counts(layout); +// ... +// counts.decrement_pending(h[id], 1); +class PendingCounts { + public: + // The state machine for a node's execution. + enum NodeState { + // The pending count for the node > 0. + PENDING_NOTREADY, + // The pending count for the node == 0, but the node has not + // started executing. + PENDING_READY, + // The node has started executing. + STARTED, + // The node has finished executing. + COMPLETED + }; + + // An opaque handle indicating where in the PendingCounts data structure + // the appropriate count information can be found. + class Handle; + // Given a node that needs to represent counts no larger than the + // specified "max_pending_count" and "max_dead_count", create a + // handle that can be passed to various PendingCounts routines + // to retrieve the count data for this node. + class Layout { + public: + Handle CreateHandle(size_t max_pending_count, size_t max_dead_count); + + private: + friend class PendingCounts; + int next_offset_ = 0; // Next byte offset to allocate + }; + + // Create a new PendingCounts object that can hold the state of + // all the Handles allocated from "final_allocator". + explicit PendingCounts(Layout layout) + : num_bytes_(layout.next_offset_), bytes_(new char[num_bytes_]) {} + + // Create a new PendingCounts object with the same layout and counts + // as "other". + explicit PendingCounts(const PendingCounts& other) + : num_bytes_(other.num_bytes_), bytes_(new char[num_bytes_]) { + CHECK_EQ(uintptr_t(bytes_) % alignof(LargeCounts), 0); + memcpy(bytes_, other.bytes_, other.num_bytes_); + } + + ~PendingCounts() { delete[] bytes_; } + + void set_initial_count(Handle h, size_t pending_count) { + if (h.is_large_) { + LargeCounts* c = Large(h); + c->pending = pending_count; + c->dead_count = 0; + c->has_started = 0; + } else { + PackedCounts* c = Packed(h); + DCHECK_LE(pending_count, kMaxCountForPackedCounts); + c->pending = pending_count; + c->dead_count = 0; + c->has_started = 0; + } + } + + NodeState node_state(Handle h) { + if (h.is_large_) { + return NodeStateForStruct(Large(h)); + } else { + return NodeStateForStruct(Packed(h)); + } + } + void mark_started(Handle h) { + DCHECK_EQ(pending(h), 0); + if (h.is_large_) { + LargeCounts* c = Large(h); + DCHECK_EQ(c->has_started, 0); + c->has_started = 1; + } else { + PackedCounts* c = Packed(h); + DCHECK_EQ(c->has_started, 0); + c->has_started = 1; + } + } + void mark_completed(Handle h) { + if (h.is_large_) { + LargeCounts* c = Large(h); + DCHECK_EQ(c->has_started, 1); + c->pending = 1; + } else { + PackedCounts* c = Packed(h); + DCHECK_EQ(c->has_started, 1); + c->pending = 1; + } + } + int pending(Handle h) { + if (h.is_large_) { + LargeCounts* c = Large(h); + if (PENDING_NOTREADY == NodeStateForStruct(c)) { + return c->pending; + } else { + // The pending count encodes the state once the node has + // started, so just return 0. + return 0; + } + } else { + PackedCounts* c = Packed(h); + if (PENDING_NOTREADY == NodeStateForStruct(c)) { + return c->pending; + } else { + // The pending count encodes the state once the node has + // started, so just return 0. + return 0; + } + } + } + int decrement_pending(Handle h, int v) { + DCHECK_GE(pending(h), v); + if (h.is_large_) { + LargeCounts* c = Large(h); + c->pending -= v; + return c->pending; + } else { + PackedCounts* c = Packed(h); + c->pending -= v; + return c->pending; + } + } + // Mark a merge node as live + // REQUIRES: Node corresponding to "h" is a merge node + void mark_live(Handle h) { + if (h.is_large_) { + LargeCounts* c = Large(h); + // Only do anything if the node hasn't already started executing. + if (PENDING_NOTREADY == NodeStateForStruct(c)) { + c->pending &= ~static_cast(0x1); + } + } else { + PackedCounts* c = Packed(h); + // Only do anything if the node hasn't already started executing. + if (PENDING_NOTREADY == NodeStateForStruct(c)) { + static_assert(7 == kMaxCountForPackedCounts, + "Live flag incorrect for max packed count"); + c->pending &= 0x6; + } + } + } + + int dead_count(Handle h) { + int r = h.is_large_ ? Large(h)->dead_count : Packed(h)->dead_count; + return r; + } + void increment_dead_count(Handle h) { + if (h.is_large_) { + LargeCounts* c = Large(h); + if (PENDING_NOTREADY == NodeStateForStruct(c)) { + c->dead_count++; + } + } else { + PackedCounts* c = Packed(h); + if (PENDING_NOTREADY == NodeStateForStruct(c)) { + DCHECK_LT(c->dead_count, kMaxCountForPackedCounts); + c->dead_count++; + } + } + } + + // A streamlined routine that does several pieces of bookkeeping at + // once. Equivalent to: + // if (increment_dead) increment_dead_count(h); + // decrement_pending(h, 1); + // *pending_result = pending(h); + // *dead_result = dead_count(h); + void adjust_for_activation(Handle h, bool increment_dead, int* pending_result, + int* dead_result) { + DCHECK_GE(pending(h), 1); + if (h.is_large_) { + adjust_for_activation_shared(Large(h), increment_dead, pending_result, + dead_result); + } else { + adjust_for_activation_shared(Packed(h), increment_dead, pending_result, + dead_result); + } + } + + class Handle { + public: + Handle() : byte_offset_(0), is_large_(0) {} + + private: + friend class PendingCounts; + int byte_offset_ : 31; // Byte offset of the rep in PendingCounts object + bool is_large_ : 1; // If true, rep is LargeCounts; otherwise PackedCounts + }; + + private: + template + inline void adjust_for_activation_shared(T* c, bool increment_dead, + int* pending_result, + int* dead_result) { + if (increment_dead) { + if (PENDING_NOTREADY == NodeStateForStruct(c)) { + c->dead_count++; + } + } + c->pending -= 1; + *dead_result = c->dead_count; + *pending_result = c->pending; + } + + // We keep track of the pending count and dead input count for each + // graph node. The representation used here is designed to be cache + // efficient for graphs with large numbers of nodes, where most + // nodes have relatively small maximum pending counts (e.g. for one + // LSTM model, 99% of 5000+ nodes had in-degrees of 3 or less). We + // use one byte to hold both the pending and dead count for a node + // where these together can fit in one byte, and we use a hash table + // to handle the rare node ids that need larger counts than this. + // Each frame in this subgraph has its own PendingCounts. + + // We use 3 bits each for dead_count and pending. + static const int kMaxCountForPackedCounts = 7; + + // Most counts are small, so we pack a pending count and a dead + // count into 3 bits each, use 1 bit to indicate that the node has + // started computing. + struct PackedCounts { + uint8 pending : 3; + uint8 dead_count : 3; + uint8 has_started : 1; + }; + + struct LargeCounts { + uint32 pending; + uint32 dead_count : 31; + uint8 has_started : 1; + }; + + template + NodeState NodeStateForStruct(T* c) const { + if (c->has_started) { + return (c->pending == 0) ? STARTED : COMPLETED; + } else { + return (c->pending == 0) ? PENDING_READY : PENDING_NOTREADY; + } + } + inline LargeCounts* Large(Handle h) { + DCHECK(h.is_large_); + DCHECK_LE(h.byte_offset_ + sizeof(LargeCounts), num_bytes_); + DCHECK_EQ(h.byte_offset_ % alignof(LargeCounts), 0); + return reinterpret_cast(bytes_ + h.byte_offset_); + } + inline PackedCounts* Packed(Handle h) { + DCHECK(!h.is_large_); + DCHECK_LE(h.byte_offset_ + sizeof(PackedCounts), num_bytes_); + return reinterpret_cast(bytes_ + h.byte_offset_); + } + + const int num_bytes_; // Just for bounds checking in debug mode + char* bytes_; // Array of num_bytes_ bytes + + void operator=(const PendingCounts&) = delete; +}; + +inline PendingCounts::Handle PendingCounts::Layout::CreateHandle( + size_t max_pending_count, size_t max_dead_count) { + Handle result; + if ((max_pending_count > kMaxCountForPackedCounts) || + (max_dead_count > kMaxCountForPackedCounts)) { + int B = sizeof(LargeCounts); + // Round byte offset to proper alignment + DCHECK_GE(sizeof(LargeCounts), alignof(LargeCounts)); + int64 offset = ((static_cast(next_offset_) + B - 1) / B) * B; + result.byte_offset_ = offset; + result.is_large_ = true; + next_offset_ = result.byte_offset_ + B; + } else { + result.byte_offset_ = next_offset_; + result.is_large_ = false; + DCHECK_EQ(sizeof(PackedCounts), 1); + next_offset_ += sizeof(PackedCounts); + } + return result; +} + +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_PENDING_COUNTS_H_ diff --git a/linux/include/tensorflow/core/common_runtime/placer.h b/linux/include/tensorflow/core/common_runtime/placer.h new file mode 100644 index 000000000..f97ffe737 --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/placer.h @@ -0,0 +1,101 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_PLACER_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_PLACER_H_ + +#include +#include + +#include "tensorflow/core/common_runtime/device_set.h" +#include "tensorflow/core/graph/graph.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/types.h" +#include "tensorflow/core/public/session_options.h" +#include "tensorflow/core/util/device_name_utils.h" + +namespace tensorflow { + +// A placement algorithm that assigns the nodes of the given Graph to +// devices the given DeviceSet, respecting the following constraints: +// +// 1. Existing device assignments remain unchanged. +// 2. Requested (partial or complete) device specifications given by device name +// for each node are granted. +// 3. Nodes connected by edges of a reference type are colocated on +// the same device. +// 4. Given nodes "A" and "B", if node "B" has a colocation group +// "@loc:A", nodes "A" and "B" will be colocated on the same device. +// +// The implementation builds a constraint graph with the same set of +// nodes, and edges that represent colocation constraints between +// nodes. Each connected component in the resulting constraint graph +// is then assigned to a set of valid devices. +// +// Run() will finally assign the device to each node given the list of +// possible devices. +// +// TODO(mrry): "Soft" constraints, such as "place node 'x' as close as +// possible to node 'y' while respecting the other constraints"? +// TODO(mrry): Create a common interface for this and the other +// placement algorithms so that they may be injected into the graph +// builder. +class Placer { + public: + // A map from graph node names to numerical IDs (in a Graph object). + typedef std::unordered_map NodeNameToIdMap; + + // Creates an instance of the Placer algorithm for the given + // Graph "graph" (nodes in which may or may not be assigned) on the + // given DeviceSet "devices". + // + // The "graph", and "devices" pointer arguments + // are borrowed by this Placer, and must outlive it. + Placer(Graph* graph, const DeviceSet* devices, const SessionOptions* options); + + Placer(Graph* graph, const DeviceSet* devices); + + ~Placer(); + + // Assigns each node in this Placer's graph to a device in its + // set of devices. + // + // This method is not thread-safe. + // Run() may be invoked at most once. + Status Run(); + + private: + // Returns true if the device type of 'candidate_device_name' is + // found in 'devices'. + bool CanAssignToDevice(const string& candidate_device_name, + const std::vector& devices) const; + + // Assigns 'node's devices to 'assigned_device', and logs the + // placement if the SessionOptions entry in 'options_' requests it. + void AssignAndLog(int assigned_device, Node* node) const; + void LogDeviceAssignment(const Node* node) const; + + Graph* const graph_; // Not owned. + const DeviceSet* const devices_; // Not owned. + const SessionOptions* options_; // Not owned. + const bool log_device_placement_; + + TF_DISALLOW_COPY_AND_ASSIGN(Placer); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_PLACER_H_ diff --git a/linux/include/tensorflow/core/common_runtime/pool_allocator.h b/linux/include/tensorflow/core/common_runtime/pool_allocator.h new file mode 100644 index 000000000..5b4623ba1 --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/pool_allocator.h @@ -0,0 +1,171 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_POOL_ALLOCATOR_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_POOL_ALLOCATOR_H_ + +// Simple LRU pool allocators for various flavors of CPU RAM. + +#include +#include +#include +#include +#include "tensorflow/core/framework/allocator.h" +#include "tensorflow/core/lib/core/bits.h" +#include "tensorflow/core/platform/logging.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/mutex.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +// Interface of an object that rounds up integers. +class RoundUpInterface { + public: + virtual ~RoundUpInterface() {} + virtual size_t RoundUp(size_t num_bytes) = 0; +}; + +// Size-limited pool of memory buffers obtained from a SubAllocator +// instance. Pool eviction policy is LRU. +class PoolAllocator : public Allocator { + public: + // "pool_size_limit" is the maximum number of returned, re-usable + // memory buffers to keep in the pool. If pool_size_limit == 0, the + // pool is effectively a thin wrapper around the allocator. + // If "auto_resize" is true, then the pool_size_limit will gradually + // be raised so that deallocations happen very rarely, if at all. + // Transitory start-up objects may deallocate, but the long-term + // working-set should not. Auto-resizing can raise pool_size_limit + // but will never lower it. + // "allocator" is the object that performs the underlying memory + // malloc/free operations. This object takes ownership of allocator. + PoolAllocator(size_t pool_size_limit, bool auto_resize, + SubAllocator* allocator, RoundUpInterface* size_rounder, + string name); + ~PoolAllocator() override; + + string Name() override { return name_; } + + void* AllocateRaw(size_t alignment, size_t num_bytes) override; + + void DeallocateRaw(void* ptr) override; + + // Allocate an unused memory region of size "num_bytes". Fetch from + // the pool if available, otherwise call allocator_. + void* Get(size_t num_bytes); + + // Return a no-longer needed memory region to the pool. It is an error + // to deference "ptr" after this call. If the pool is full, the least + // recently used region will be deallocated. + void Put(void* ptr, size_t num_bytes); + + // Reset the pool to empty. + void Clear(); + + // The following accessors permit monitoring the effectiveness of + // the pool at avoiding repeated malloc/frees on the underlying + // allocator. Read locks are not taken on the theory that value + // consistency with other threads is not important. + + // Number of Get() requests satisfied from pool. + int64 get_from_pool_count() const NO_THREAD_SAFETY_ANALYSIS { + return get_from_pool_count_; + } + // Number of Put() requests. + int64 put_count() const NO_THREAD_SAFETY_ANALYSIS { return put_count_; } + // Number of Get() requests requiring a fresh allocation. + int64 allocated_count() const NO_THREAD_SAFETY_ANALYSIS { + return allocated_count_; + } + // Number of pool evictions. + int64 evicted_count() const NO_THREAD_SAFETY_ANALYSIS { + return evicted_count_; + } + // Current size limit. + size_t size_limit() const NO_THREAD_SAFETY_ANALYSIS { + return pool_size_limit_; + } + + void GetStats(AllocatorStats* stats) override { stats->Clear(); } + + private: + struct PtrRecord { + void* ptr; + size_t num_bytes; + PtrRecord* prev; + PtrRecord* next; + }; + + // Remove "pr" from the double-linked LRU list. + void RemoveFromList(PtrRecord* pr) EXCLUSIVE_LOCKS_REQUIRED(mutex_); + + // Add "pr" to the head of the double-linked LRU list. + void AddToList(PtrRecord* pr) EXCLUSIVE_LOCKS_REQUIRED(mutex_); + + // Delete the least recently used record. + void EvictOne() EXCLUSIVE_LOCKS_REQUIRED(mutex_); + + const string name_; + const bool has_size_limit_; + const bool auto_resize_; + size_t pool_size_limit_; + std::unique_ptr allocator_; + std::unique_ptr size_rounder_; + mutex mutex_; + std::multimap pool_ GUARDED_BY(mutex_); + PtrRecord* lru_head_ GUARDED_BY(mutex_) = nullptr; + PtrRecord* lru_tail_ GUARDED_BY(mutex_) = nullptr; + int64 get_from_pool_count_ GUARDED_BY(mutex_) = 0; + int64 put_count_ GUARDED_BY(mutex_) = 0; + int64 allocated_count_ GUARDED_BY(mutex_) = 0; + int64 evicted_count_ GUARDED_BY(mutex_) = 0; +}; + +// Do-nothing rounder. Passes through sizes unchanged. +class NoopRounder : public RoundUpInterface { + public: + size_t RoundUp(size_t num_bytes) override { return num_bytes; } +}; + +// Power of 2 rounder: rounds up to nearest power of 2 size. +class Pow2Rounder : public RoundUpInterface { + public: + size_t RoundUp(size_t num_bytes) override { + return 1uLL << Log2Ceiling64(num_bytes); + } +}; + +class BasicCPUAllocator : public SubAllocator { + public: + // Argument numa_node is currently ignored. + BasicCPUAllocator(int numa_node, const std::vector& alloc_visitors, + const std::vector& free_visitors) + : SubAllocator(alloc_visitors, free_visitors), numa_node_(numa_node) {} + + ~BasicCPUAllocator() override {} + + void* Alloc(size_t alignment, size_t num_bytes) override; + + void Free(void* ptr, size_t num_bytes) override; + + private: + int numa_node_; + + TF_DISALLOW_COPY_AND_ASSIGN(BasicCPUAllocator); +}; + +} // namespace tensorflow +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_POOL_ALLOCATOR_H_ diff --git a/linux/include/tensorflow/core/common_runtime/process_function_library_runtime.h b/linux/include/tensorflow/core/common_runtime/process_function_library_runtime.h new file mode 100644 index 000000000..53815715d --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/process_function_library_runtime.h @@ -0,0 +1,200 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_PROCESS_FUNCTION_LIBRARY_RUNTIME_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_PROCESS_FUNCTION_LIBRARY_RUNTIME_H_ + +#include + +#include "tensorflow/core/common_runtime/device_mgr.h" +#include "tensorflow/core/framework/function.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/protobuf/config.pb.h" + +namespace tensorflow { + +// A class that stores all the FunctionLibraryRuntime objects, one per device. +class ProcessFunctionLibraryRuntime { + public: + // Creates FunctionLibraryRuntime objects for each device in the provided + // DeviceMgr. Caller needs to make sure that device_mgr, lib_def and parent + // (if provided) outlive this object. + ProcessFunctionLibraryRuntime( + const DeviceMgr* device_mgr, Env* env, int graph_def_version, + const FunctionLibraryDefinition* lib_def, + const OptimizerOptions& optimizer_options, + thread::ThreadPool* thread_pool = nullptr, + DistributedFunctionLibraryRuntime* parent = nullptr); + + // With `custom_kernel_creator`. + ProcessFunctionLibraryRuntime(const DeviceMgr* device_mgr, Env* env, + int graph_def_version, + const FunctionLibraryDefinition* lib_def, + const OptimizerOptions& optimizer_options, + CustomKernelCreator custom_kernel_creator, + thread::ThreadPool* thread_pool, + DistributedFunctionLibraryRuntime* parent); + + // Sends `tensors_to_send` from `source_device` to `target_device` using + // `rendezvous`. `key_prefix` is used as a prefix for the keys sent to the + // Rendezvous. `device_context` should be the DeviceContext of the device + // doing the sending. `alloc_attrs` should either be empty or be the size of + // `tensors_to_send` and indicates how the input tensors are allocated. Method + // takes references on each of the `tensors_to_send`. Method doesn't block. + static Status SendTensors(const string& source_device, + const string& target_device, + const string& key_prefix, int64 src_incarnation, + gtl::ArraySlice tensors_to_send, + DeviceContext* device_context, + const std::vector& alloc_attrs, + Rendezvous* rendezvous); + + // Receives `received_tensors` from `target_device` (originally sent from + // `source_device`) using `rendezvous`. Uses `key_prefix` to construct the + // keys to be retrieved. `device_context` should be for the device receiving + // the tensors. `alloc_attrs` indicates how to allocate the received + // tensors and should either be empty or `num_tensors` in size. Method doesn't + // block and calls `done` when `num_tensors` are fetched. + static void ReceiveTensorsAsync( + const string& source_device, const string& target_device, + const string& key_prefix, int64 src_incarnation, int64 num_tensors, + DeviceContext* device_context, + const std::vector& alloc_attrs, + Rendezvous* rendezvous, std::vector* received_tensors, + StatusCallback done); + + static const char kDefaultFLRDevice[]; + // Returns the FunctionLibraryRuntime for the corresponding device_name. + FunctionLibraryRuntime* GetFLR(const string& device_name) const; + + // Returns the device incarnation for the given device_name. + Status GetDeviceIncarnation(const string& device_name, int64* incarnation); + + // For a given canonicalized key signature of the function instantiated + // on device `device_name` and a `local_handle`, creates a handle and returns + // that value. Uses core/common_runtime/framework/function.h::Canonicalize + // to canonicalize the function signature. + FunctionLibraryRuntime::Handle AddHandle( + const string& function_key, const string& device_name, + FunctionLibraryRuntime::LocalHandle local_handle); + + // Returns a handle if found for the given key, else returns kInvalidHandle. + FunctionLibraryRuntime::Handle GetHandle(const string& function_key) const; + + // For the given handle instantiated on device `device_name` returns the local + // index of instantiation of that function. If the function was not + // instantiated on `device_name` returns kInvalidLocalHandle. + FunctionLibraryRuntime::LocalHandle GetHandleOnDevice( + const string& device_name, FunctionLibraryRuntime::Handle handle); + + // Returns true if function with handle `handle` was instantiated on device + // `device_name`. + bool IsInstantiatedOnDevice(const string& device_name, + FunctionLibraryRuntime::Handle handle); + + // Instantiates the function. See framework/function.h for more details. + // Allows for function_name to be instantiated on different devices + // as specified in attrs. + Status Instantiate(const string& function_name, AttrSlice attrs, + const FunctionLibraryRuntime::InstantiateOptions& options, + FunctionLibraryRuntime::Handle* handle); + + // Delegates to the local FLR that owns state corresponding to `handle` and + // tells it to release it. If the `handle` isnt' needed at all, the local FLR + // might call RemoveHandle on this to get rid of the state owned by the Proc + // FLR. + Status ReleaseHandle(FunctionLibraryRuntime::Handle handle); + + // Runs the function with given `handle`. Function could have been + // instantiated on any device. More details in framework/function.h + void Run(const FunctionLibraryRuntime::Options& opts, + FunctionLibraryRuntime::Handle handle, gtl::ArraySlice args, + std::vector* rets, + FunctionLibraryRuntime::DoneCallback done); + + private: + // For a given device_name, returns a DeviceContext for copying + // tensors to/from the device. + Status GetDeviceContext(const string& device_name, + DeviceContext** device_context); + + // Looks up the information for the given `handle` and returns the name + // of the device where the function is registered. + string GetDeviceName(FunctionLibraryRuntime::Handle handle); + + // Removes handle from the state owned by this object. + Status RemoveHandle(FunctionLibraryRuntime::Handle handle); + + Status Clone(Env* env, int graph_def_version, + const OptimizerOptions& optimizer_options, + CustomKernelCreator custom_kernel_creator, + std::unique_ptr* out_lib_def, + std::unique_ptr* out_pflr); + + friend class FunctionLibraryRuntimeImpl; + + mutable mutex mu_; + + class FunctionData { + public: + FunctionData(const string& target_device, + FunctionLibraryRuntime::LocalHandle local_handle, + const string& function_key) + : target_device_(target_device), + local_handle_(local_handle), + function_key_(function_key) {} + + string target_device() { return target_device_; } + const string& function_key() { return function_key_; } + + FunctionLibraryRuntime::LocalHandle local_handle() { + mutex_lock l(mu_); + return local_handle_; + } + + // Initializes the FunctionData object by potentially making an Initialize + // call to the DistributedFunctionLibraryRuntime. + Status DistributedInit( + DistributedFunctionLibraryRuntime* parent, const string& function_name, + const FunctionLibraryDefinition& lib_def, AttrSlice attrs, + const FunctionLibraryRuntime::InstantiateOptions& options); + + private: + mutex mu_; + + const string target_device_; + FunctionLibraryRuntime::LocalHandle local_handle_ GUARDED_BY(mu_); + const string function_key_; + bool init_started_ GUARDED_BY(mu_) = false; + Status init_result_ GUARDED_BY(mu_); + Notification init_done_; + }; + + const DeviceMgr* const device_mgr_; + const FunctionLibraryDefinition* lib_def_; + thread::ThreadPool* default_thread_pool_; + // Holds all the function invocations here. + std::unordered_map table_ + GUARDED_BY(mu_); + std::unordered_map> + function_data_ GUARDED_BY(mu_); + std::unordered_map> flr_map_; + int next_handle_ GUARDED_BY(mu_); + DistributedFunctionLibraryRuntime* const parent_; +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_PROCESS_FUNCTION_LIBRARY_RUNTIME_H_ diff --git a/linux/include/tensorflow/core/common_runtime/process_state.h b/linux/include/tensorflow/core/common_runtime/process_state.h new file mode 100644 index 000000000..cac312d84 --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/process_state.h @@ -0,0 +1,141 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_PROCESS_STATE_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_PROCESS_STATE_H_ + +#include +#include +#include +#include + +#include "tensorflow/core/framework/allocator.h" +#include "tensorflow/core/platform/mutex.h" +#include "tensorflow/core/platform/thread_annotations.h" +#include "tensorflow/core/platform/types.h" +#include "tensorflow/core/protobuf/config.pb.h" + +namespace tensorflow { + +class Allocator; +class PoolAllocator; + +// Singleton that manages per-process state, e.g. allocation of +// shared resources. +class ProcessState { + public: + static ProcessState* singleton(); + + // Descriptor for memory allocation attributes, used by optional + // runtime correctness analysis logic. + struct MemDesc { + enum MemLoc { CPU, GPU }; + MemLoc loc; + int dev_index; + bool gpu_registered; + bool nic_registered; + MemDesc() + : loc(CPU), + dev_index(0), + gpu_registered(false), + nic_registered(false) {} + string DebugString(); + }; + + // If NUMA Allocators are desired, call this before calling any + // Allocator accessor. + void EnableNUMA() { numa_enabled_ = true; } + + // Returns what we know about the memory at ptr. + // If we know nothing, it's called CPU 0 with no other attributes. + MemDesc PtrType(const void* ptr); + + // Returns the one CPUAllocator used for the given numa_node. + // TEMPORARY: ignores numa_node. + Allocator* GetCPUAllocator(int numa_node); + + // Registers alloc visitor for the CPU allocator(s). + // REQUIRES: must be called before GetCPUAllocator. + void AddCPUAllocVisitor(SubAllocator::Visitor v); + + // Registers free visitor for the CPU allocator(s). + // REQUIRES: must be called before GetCPUAllocator. + void AddCPUFreeVisitor(SubAllocator::Visitor v); + + typedef std::unordered_map MDMap; + + protected: + ProcessState(); + friend class GPUProcessState; + + // If these flags need to be runtime configurable consider adding + // them to ConfigProto. + static const bool FLAGS_brain_mem_reg_cuda_dma = true; + static const bool FLAGS_brain_gpu_record_mem_types = false; + + // Helper method for unit tests to reset the ProcessState singleton by + // cleaning up everything. Never use in production. + virtual void TestOnlyReset(); + + static ProcessState* instance_; + bool numa_enabled_; + + mutex mu_; + + std::vector cpu_allocators_ GUARDED_BY(mu_); + std::vector cpu_alloc_visitors_ GUARDED_BY(mu_); + std::vector cpu_free_visitors_ GUARDED_BY(mu_); + + virtual ~ProcessState(); + + // Optional RecordingAllocators that wrap the corresponding + // Allocators for runtime attribute use analysis. + MDMap mem_desc_map_; + std::vector cpu_al_ GUARDED_BY(mu_); +}; + +namespace internal { +class RecordingAllocator : public Allocator { + public: + RecordingAllocator(ProcessState::MDMap* mm, Allocator* a, + ProcessState::MemDesc md, mutex* mu) + : mm_(mm), a_(a), md_(md), mu_(mu) {} + + string Name() override { return a_->Name(); } + void* AllocateRaw(size_t alignment, size_t num_bytes) override { + void* p = a_->AllocateRaw(alignment, num_bytes); + mutex_lock l(*mu_); + (*mm_)[p] = md_; + return p; + } + void DeallocateRaw(void* p) override { + mutex_lock l(*mu_); + auto iter = mm_->find(p); + mm_->erase(iter); + a_->DeallocateRaw(p); + } + bool TracksAllocationSizes() override { return a_->TracksAllocationSizes(); } + size_t RequestedSize(const void* p) override { return a_->RequestedSize(p); } + size_t AllocatedSize(const void* p) override { return a_->AllocatedSize(p); } + void GetStats(AllocatorStats* stats) override { a_->GetStats(stats); } + void ClearStats() override { a_->ClearStats(); } + ProcessState::MDMap* mm_; // not owned + Allocator* a_; // not owned + ProcessState::MemDesc md_; + mutex* mu_; +}; +} // namespace internal +} // namespace tensorflow +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_PROCESS_STATE_H_ diff --git a/linux/include/tensorflow/core/common_runtime/process_util.h b/linux/include/tensorflow/core/common_runtime/process_util.h new file mode 100644 index 000000000..5d9266671 --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/process_util.h @@ -0,0 +1,49 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_PROCESS_UTIL_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_PROCESS_UTIL_H_ + +#include + +#include "tensorflow/core/lib/core/threadpool.h" +#include "tensorflow/core/public/session_options.h" + +// TODO(vrv, mrry): Remove this library: its interface circumvents the +// callers' Env and calls Env::Default() directly. + +namespace tensorflow { + +// Returns a process-wide ThreadPool for scheduling compute operations +// using 'options'. Caller does not take ownership over threadpool. +thread::ThreadPool* ComputePool(const SessionOptions& options); + +// Returns number of inter op threads. +int32 NumInterOpThreadsFromSessionOptions(const SessionOptions& options); + +// Creates a thread pool with number of inter op threads. +thread::ThreadPool* NewThreadPoolFromSessionOptions( + const SessionOptions& options); + +// Schedule "closure" in the default thread queue. +void SchedClosure(std::function closure); + +// Schedule "closure" after the given number of microseconds in the +// fixed-size ThreadPool used for non-blocking compute tasks. +void SchedNonBlockingClosureAfter(int64 micros, std::function closure); + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_PROCESS_UTIL_H_ diff --git a/linux/include/tensorflow/core/common_runtime/profile_handler.h b/linux/include/tensorflow/core/common_runtime/profile_handler.h new file mode 100644 index 000000000..391dc8c19 --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/profile_handler.h @@ -0,0 +1,67 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_PROFILE_HANDLER_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_PROFILE_HANDLER_H_ + +#include "tensorflow/core/framework/step_stats.pb.h" +#include "tensorflow/core/graph/types.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/lib/core/stringpiece.h" + +namespace tensorflow { + +// A profile handler collects event stats from a running step. +class ProfileHandler { + public: + ProfileHandler() {} + virtual ~ProfileHandler() {} + + // Records that a single Op was executed in the current step. + // + // Implementations of this method must be thread-safe. + // + // Args: + // - device: Device on which the Op was executed. + // - stats: Statistics of node execution timing. + // - is_copy: True if the op was a copy, send or recv. + // - label: Extra content for timeline click text. + // - op_type: String name of the Op. + // - details: Main content for timeline click text. + virtual void RecordOneOp(const string& device, const NodeExecStats& stats, + bool is_copy, StringPiece label, StringPiece op_type, + StringPiece details) = 0; + + // Records that the current step finished. + // + // Implementations of this method need not be thread-safe. + // + // Args: + // - start_time: The time at which the step started. + // - finish_time: The time at which the step finished. + // - cleanup_time: The time at which cleanup for the step finished. + // - total_runops: The number of ops that ran during this step. + // - final_status: The status that this step finished with. + virtual void StepDone(Microseconds start_time, Microseconds finish_time, + Microseconds cleanup_time, int total_runops, + Status final_status) = 0; + + // Returns true if the caller should collect rpc activity. + virtual bool should_collect_rpcs() = 0; +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_PROFILE_HANDLER_H_ diff --git a/linux/include/tensorflow/core/common_runtime/renamed_device.h b/linux/include/tensorflow/core/common_runtime/renamed_device.h new file mode 100644 index 000000000..c00789a55 --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/renamed_device.h @@ -0,0 +1,142 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_RENAMED_DEVICE_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_RENAMED_DEVICE_H_ + +#include "tensorflow/core/common_runtime/device.h" +#include "tensorflow/core/util/device_name_utils.h" + +namespace tensorflow { + +// Wraps a device with a new name, delegating work to the wrapped device. +// +// This class is used to wrap local devices when using clusterspec propagation +// where the name of a particular device may change in the context of a given +// session. +class RenamedDevice : public Device { + public: + static Device* NewRenamedDevice(const string& new_base, Device* underlying, + bool owns_underlying, + bool isolate_session_state); + + ~RenamedDevice() override; + + // Below are virtual methods defined on DeviceBase + bool RequiresRecordingAccessedTensors() const override { + return underlying_->RequiresRecordingAccessedTensors(); + } + + const DeviceBase* UnderlyingDevice() const override { + return underlying_->UnderlyingDevice(); + } + DeviceBase* UnderlyingDevice() override { + return underlying_->UnderlyingDevice(); + } + + const CpuWorkerThreads* tensorflow_cpu_worker_threads() const override { + return underlying_->tensorflow_cpu_worker_threads(); + } + + const GpuDeviceInfo* tensorflow_gpu_device_info() const override { + return underlying_->tensorflow_gpu_device_info(); + } + + Allocator* GetAllocator(AllocatorAttributes attr) override { + return underlying_->GetAllocator(attr); + } + + Allocator* GetScopedAllocator(AllocatorAttributes attr, + int64 step_id) override { + return underlying_->GetScopedAllocator(attr, step_id); + } + + ScopedAllocatorMgr* GetScopedAllocatorMgr() const override { + return underlying_->GetScopedAllocatorMgr(); + } + + const Eigen::ThreadPoolDevice* eigen_cpu_device() override { + return underlying_->eigen_cpu_device(); + } + +#ifdef TENSORFLOW_USE_SYCL + const Eigen::SyclDevice* eigen_sycl_device() const override { + return underlying_->eigen_sycl_device(); + } +#endif + + PerOpGpuDevice* MakeGpuDevice() override { + return underlying_->MakeGpuDevice(); + } + + Status ReinitializeGpuDevice(OpKernelContext* context, PerOpGpuDevice* device, + DeviceContext* dc, + Allocator* allocator) override { + return underlying_->ReinitializeGpuDevice(context, device, dc, allocator); + } + + Status MakeTensorFromProto(const TensorProto& tensor_proto, + const AllocatorAttributes alloc_attrs, + Tensor* tensor) override { + return underlying_->MakeTensorFromProto(tensor_proto, alloc_attrs, tensor); + } + + // Below are virtual methods defined on Device + + void Compute(OpKernel* op_kernel, OpKernelContext* context) override { + underlying_->Compute(op_kernel, context); + } + + void ComputeAsync(AsyncOpKernel* op_kernel, OpKernelContext* context, + AsyncOpKernel::DoneCallback done) override { + underlying_->ComputeAsync(op_kernel, context, std::move(done)); + } + + void ConsumeListOfAccessedTensors( + DeviceContext* context, const TensorReferenceVector& tensors) override { + underlying_->ConsumeListOfAccessedTensors(context, tensors); + } + + Status Sync() override { return underlying_->Sync(); } + + Status MaybeRewriteGraph(std::unique_ptr* graph) override { + return underlying_->MaybeRewriteGraph(graph); + } + + Status FillContextMap(const Graph* graph, + DeviceContextMap* device_context_map) override { + return underlying_->FillContextMap(graph, device_context_map); + } + + // Returns the resource manager associated w/ this device. + ResourceMgr* resource_manager() override { + if (isolate_session_state_) { + return Device::resource_manager(); + } else { + return underlying_->resource_manager(); + } + } + + private: + RenamedDevice(Device* underlying, const DeviceAttributes& attributes, + bool owns_underlying, bool isolate_session_state); + Device* const underlying_; + const bool owns_underlying_; + const bool isolate_session_state_; +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_RENAMED_DEVICE_H_ diff --git a/linux/include/tensorflow/core/common_runtime/rendezvous_mgr.h b/linux/include/tensorflow/core/common_runtime/rendezvous_mgr.h new file mode 100644 index 000000000..b4d8ab4eb --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/rendezvous_mgr.h @@ -0,0 +1,90 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_RENDEZVOUS_MGR_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_RENDEZVOUS_MGR_H_ + +#include +#include + +#include "tensorflow/core/common_runtime/device_mgr.h" +#include "tensorflow/core/framework/rendezvous.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/mutex.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +// IntraProcessRendezvous is a Rendezvous which expects all producers +// and consumers to be devices immediately accessible within the +// process. That is, it will never be necessary to perform an RPC to +// communicate with either. +// +// Buffering of Tensor values is delegated to a "local" Rendezvous +// obtained from NewLocalRendezvous(). This class just adds +// functionality to coordinate multiple process-local devices. +class IntraProcessRendezvous : public Rendezvous { + public: + explicit IntraProcessRendezvous(const DeviceMgr* device_mgr); + + // Forwards to local_, where the Tensor "val" will be buffered and + // any waiting callback stored. + Status Send(const ParsedKey& key, const Rendezvous::Args& args, + const Tensor& val, const bool is_dead) override; + + // This method is called only by the RecvOp. It tests to see + // whether the value will be produced by a local or remote device + // and handles accordingly. In the local case it forwards to + // local_, in the remote case it initiates an RPC request. + void RecvAsync(const ParsedKey& key, const Rendezvous::Args& args, + DoneCallback done) override; + + void StartAbort(const Status& status) override; + + private: + const DeviceMgr* device_mgr_; + Rendezvous* local_; // Owns a Ref on this object. + + mutable mutex mu_; + + // Status given by StartAbort() if any. + Status status_ GUARDED_BY(mu_); + + ~IntraProcessRendezvous() override; + + // Parses "key" into "parsed". If "is_src" is true, checks that the + // rendezvous key's source is in this process. If "is_src" is false, + // checks that the rendezvous key's destination is in this process. + Status ParseKey(const string& key, bool is_src, + Rendezvous::ParsedKey* parsed); + + // Callback handling the case when a rendezvous has been + // accomplished in local_ and the consumer is local to this process. + // Tensor "in" will be copied into "out". The key "parsed" encodes + // the src and dst devices. + typedef std::function StatusCallback; + void SameWorkerRecvDone(const Rendezvous::ParsedKey& parsed, + const Rendezvous::Args& send_args, + const Rendezvous::Args& recv_args, const Tensor& in, + Tensor* out, StatusCallback done); + + TF_DISALLOW_COPY_AND_ASSIGN(IntraProcessRendezvous); +}; + +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_RENDEZVOUS_MGR_H_ diff --git a/linux/include/tensorflow/core/common_runtime/rendezvous_util.h b/linux/include/tensorflow/core/common_runtime/rendezvous_util.h new file mode 100644 index 000000000..deb9a7c82 --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/rendezvous_util.h @@ -0,0 +1,53 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_RENDEZVOUS_UTIL_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_RENDEZVOUS_UTIL_H_ + +#include + +#include "tensorflow/core/framework/rendezvous.h" +#include "tensorflow/core/lib/core/status.h" + +namespace tensorflow { + +typedef std::map NamedTensors; +typedef std::function StatusCallback; + +// Uses `rendezvous` to send tensors in `tensors_to_send`. `device_context` +// should be the DeviceContext associated with the source of the tensors. +// `alloc_attrs` contains information about how the `tensors_to_send` are +// allocated. `alloc_attrs` should either be {} or should match the length of +// `keys`. +Status SendTensorsToRendezvous( + Rendezvous* rendezvous, DeviceContext* device_context, + const std::vector& alloc_attrs, + const std::vector& keys, gtl::ArraySlice tensors_to_send); + +// Uses `rendezvous` to obtain tensors. `device_context` should be the +// DeviceContext associated with the receiving device. `alloc_attrs` contains +// information as how to store the received tensors. Should be {} or match the +// length of `keys`. +void RecvOutputsFromRendezvousAsync( + Rendezvous* rendezvous, DeviceContext* device_context, + const std::vector& alloc_attrs, + const std::vector& keys, std::vector* received_tensors, + StatusCallback done); + +Status RecvOutputsFromRendezvous(Rendezvous* rendezvous, NamedTensors* out, + const Rendezvous::Args& args); + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_RENDEZVOUS_UTIL_H_ diff --git a/linux/include/tensorflow/core/common_runtime/ring_reducer.h b/linux/include/tensorflow/core/common_runtime/ring_reducer.h new file mode 100644 index 000000000..0848e37b5 --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/ring_reducer.h @@ -0,0 +1,145 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_RING_REDUCER_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_RING_REDUCER_H_ + +#include +#include +#include +#include + +#include "tensorflow/core/common_runtime/base_collective_executor.h" +#include "tensorflow/core/framework/collective.h" + +namespace tensorflow { +class Device; + +// Ring-algorithm implementation of collective all-reduce. +class RingReducer : public CollectiveImplementationInterface { + public: + RingReducer(); + ~RingReducer() override; + + // Establishes the requested number of subdivision permutations based on the + // ring order implicit in the device order. + Status InitializeCollectiveParams(CollectiveParams* col_params) override; + + // Initializes members of CollectiveContext not yet initialized, i.e. device + // and device_locality. Also saves the CollectiveContext in this object. + Status InitializeCollectiveContext(CollectiveContext* col_ctx) override; + + // Begins async execution of the ring reduce algorithm. + // Must be called in a blockable thread. + // TODO(b/80529858): remove the previous warning when we have a dedicated + // collective threadpool. + void Run(StatusCallback done) override; + + private: + // Called when a bad status is received that implies we should terminate + // execution and return a bad status. + void StartAbort(const Status& s); + void ContinueAfterInputCopy(); + void Finish(bool ok); + Status ComputeBinOp(Device* device, OpKernel* op, Tensor* output, + Tensor* input); + bool RunAsyncParts(); + + // Used for executing a sub-operation, e.g. a merge_op instance, with + // an OpKernelContext based on the one passed into this Op. + class SubContext { + public: + OpKernelContext::Params sub_params_; + gtl::InlinedVector sub_inputs_; + gtl::InlinedVector sub_input_attr_; + gtl::InlinedVector sub_input_dc_; + // Used only for Binary and Unary Ops for which we require + // the calculation to be in-place on the first input. + int forward_from_ = 0; + OpKernelContext* sub_ctx_; + SubContext(OpKernelContext* ctx, OpKernelContext::Params* params, + OpKernel* op, Tensor* output, Tensor* input); + ~SubContext() { delete sub_ctx_; } + }; + + // Current status of a RingField + enum RingFieldAction { + RF_INIT = 0, // Just initialized for a pass + RF_RECV, // Recv pending + RF_REDUCE, // Reduce pending + RF_FINALIZE, // FinalOp pending + RF_SEND_READY, // Ready to send + RF_SEND, // Send pending + RF_DONE, // No more work + }; + + // Tracks progress of actions on a single subfield of the entire tensor. + struct RingField { + int16 chunk_idx; // major division index + int16 subdiv_idx; // minor division index + int16 sc_idx; // subchunk index + int16 rank; // rank within subdiv permutation + int16 recv_dev_idx; // dev from which value should be recv'd + RingFieldAction action; + bool second_pass; + bool recv_is_remote = false; + bool send_is_remote = false; + bool do_send = false; // is the value sent in this pass? + bool do_recv = false; // is the value recv'd in this pass? + bool is_final = false; // is the last field in the pass for this rank + Tensor chunk; // alias to field values + Tensor tmp_chunk; + Status status; + string DebugString() const; + }; + void AdvanceToSecondPass(RingField* rf); + void InitRingField(RingField* rf, int chunk_idx, int subdiv_idx, + int field_idx); + void DispatchSend(RingField* rf, const StatusCallback& done); + void DispatchRecv(RingField* rf, const StatusCallback& done); + + // For constructing log messages for debugging. + string FieldState(); + string TensorDebugString(const Tensor& tensor); + + // Producer/Consumer Queue of RingField structs. + class PCQueue { + public: + void Enqueue(RingField* rf); + RingField* Dequeue(); + + private: + mutex pcq_mu_; + condition_variable cv_; + int waiter_count_ GUARDED_BY(pcq_mu_) = 0; + std::deque deque_ GUARDED_BY(pcq_mu_); + }; + + CollectiveContext* col_ctx_; // Not owned + const CollectiveParams* col_params_; // Not owned + StatusCallback done_; + int group_size_; + int num_subdivs_; + Tensor group_size_tensor_; + Notification group_size_tensor_ready_; + std::unique_ptr ca_; + mutex status_mu_; + Status status_ GUARDED_BY(status_mu_); + std::vector rfv_; + + friend class RingReducerTest; +}; + +} // namespace tensorflow +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_RING_REDUCER_H_ diff --git a/linux/include/tensorflow/core/common_runtime/scoped_allocator.h b/linux/include/tensorflow/core/common_runtime/scoped_allocator.h new file mode 100644 index 000000000..71a700121 --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/scoped_allocator.h @@ -0,0 +1,124 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_SCOPED_ALLOCATOR_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_SCOPED_ALLOCATOR_H_ + +#include "tensorflow/core/framework/allocator.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/lib/core/refcount.h" +#include "tensorflow/core/platform/mutex.h" + +namespace tensorflow { +class ScopedAllocatorContainer; +class ScopedAllocatorInstance; + +// Manages a single backing tensor and a collection of aliases. +class ScopedAllocator { + public: + static const int32 kInvalidId = 0; + static const size_t kMaxAlignment = 64; + + // A subrange of the TensorBuffer associated with this object that + // will be the backing memory for one aliased tensor. + struct Field { + int32 scope_id; + size_t offset; + size_t bytes; + }; + // Field index that refers to backing tensor, not any aliased field. + static const int32 kBackingIndex = -1; + + // backing_tensor is expected to be newly allocated by a ScopedAllocatorOp + // instance. It must be large enough to back all of the specified + // (offset, byte) ranges of the fields. + ScopedAllocator(const Tensor& backing_tensor, int32 scope_id, + const string& name, const gtl::ArraySlice& fields, + int32 expected_call_count, + ScopedAllocatorContainer* container); + + // Automatically deletes when last use expires, or when + // ScopedAllocatorContainer decides to delete. + ~ScopedAllocator() LOCKS_EXCLUDED(mu_); + + // For debugging: returns true iff p is a pointer that could have + // been returned by AllocateRaw. + bool VerifyPointer(const void* p); + bool VerifyTensor(const Tensor* t); + + const Tensor& tensor() const { return backing_tensor_; } + + const string& name() const { return name_; } + + private: + friend class ScopedAllocatorInstance; + // Only ScopedAllocatorInstances can call AllocateRaw and DeallocateRaw on a + // ScopedAllocator + void* AllocateRaw(int32 field_index, size_t num_bytes) LOCKS_EXCLUDED(mu_); + void DeallocateRaw(void* p) LOCKS_EXCLUDED(mu_); + Tensor backing_tensor_; + TensorBuffer* tbuf_; + int32 id_; + string name_; + ScopedAllocatorContainer* container_; + std::vector fields_; + mutex mu_; + int32 expected_call_count_ GUARDED_BY(mu_); + int32 live_alloc_count_ GUARDED_BY(mu_); +}; + +// An Allocator that will return a pointer into the backing buffer of +// a previously allocated tensor, allowing creation of an alias +// tensor. There is a one-to-one mapping between the fields of a +// ScopedAllocator and ScopedAllocatorInstances. There is also a one-to-one +// mapping between scope_ids and ScopedAllocatorInstances. It should be +// discarded immediately after a single use. +class ScopedAllocatorInstance : public Allocator { + public: + explicit ScopedAllocatorInstance(ScopedAllocator* sa, int32 field_index); + + private: + ~ScopedAllocatorInstance() { VLOG(1) << "~ScopedAllocatorInstance " << this; } + + public: + // When a ScopedAllocatorContainer "Drops" a scope_id, it calls DropFromTable + // on the underlying ScopedAllocatorInstance. If this instance has already + // deallocated the tensor slice, we can safely delete this. + void DropFromTable() LOCKS_EXCLUDED(mu_); + void* AllocateRaw(size_t alignment, size_t num_bytes) + LOCKS_EXCLUDED(mu_) override; + void* AllocateRaw(size_t alignment, size_t num_bytes, + const AllocationAttributes& allocator_attr) override { + return AllocateRaw(alignment, num_bytes); + } + void DeallocateRaw(void* p) LOCKS_EXCLUDED(mu_) override; + bool TracksAllocationSizes() override { return false; } + bool ShouldAllocateEmptyTensors() override { return false; } + size_t RequestedSize(const void* ptr) override { return 0; } + size_t AllocatedSize(const void* ptr) override { return 0; } + int64 AllocationId(const void* ptr) override { return 0; } + size_t AllocatedSizeSlow(const void* ptr) override { return 0; } + string Name() override; + + private: + mutex mu_; + ScopedAllocator* scoped_allocator_; + int32 field_index_; + bool allocated_ GUARDED_BY(mu_); + bool deallocated_ GUARDED_BY(mu_); + bool in_table_ GUARDED_BY(mu_); +}; + +} // namespace tensorflow +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_SCOPED_ALLOCATOR_H_ diff --git a/linux/include/tensorflow/core/common_runtime/scoped_allocator_mgr.h b/linux/include/tensorflow/core/common_runtime/scoped_allocator_mgr.h new file mode 100644 index 000000000..8c5e85347 --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/scoped_allocator_mgr.h @@ -0,0 +1,110 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_SCOPED_ALLOCATOR_MGR_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_SCOPED_ALLOCATOR_MGR_H_ + +#include +#include + +#include "tensorflow/core/common_runtime/scoped_allocator.h" +#include "tensorflow/core/lib/core/refcount.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/platform/mutex.h" + +namespace tensorflow { +class ScopedAllocatorMgr; + +// At most one of these exists per pair. +// A Ref is held by every ScopedAllocator and also by the ScopedAllocatorMgr. +class ScopedAllocatorContainer : public core::RefCounted { + public: + // Establishes a reachable ScopedAllocator. + Status AddScopedAllocator( + const Tensor& backing_tensor, int32 scope_id, const string& scope_name, + const gtl::ArraySlice& fields, + int32 expected_call_count); + + ScopedAllocatorInstance* GetInstance(int32 scope_id); + ScopedAllocator* GetAllocator(int32 scope_id); + + // Retire the scope_id. + void Drop(int32 scope_id, ScopedAllocator* sa); + + protected: + friend class ScopedAllocatorMgr; + ScopedAllocatorContainer(const ScopedAllocatorMgr* mgr, int64 step_id) + : mgr_(mgr), step_id_(step_id) {} + ~ScopedAllocatorContainer(); + + private: + const ScopedAllocatorMgr* mgr_; + int64 step_id_; + mutex mu_; + struct SAField { + int32 field_index; + union { + ScopedAllocator* scoped_allocator; + ScopedAllocatorInstance* instance; + }; + SAField(int32 fi, ScopedAllocatorInstance* sai) + : field_index(fi), instance(sai) {} + SAField(int32 fi, ScopedAllocator* sa) + : field_index(fi), scoped_allocator(sa) {} + SAField() + : field_index(ScopedAllocator::kBackingIndex), + scoped_allocator(nullptr) {} + }; + std::unordered_map allocators_ GUARDED_BY(mu_); +}; + +// At most one of these exists per device. +class ScopedAllocatorMgr { + public: + explicit ScopedAllocatorMgr(const string& device_name) + : device_name_(device_name) {} + ~ScopedAllocatorMgr(); + + ScopedAllocatorContainer* GetContainer(int64 step_id); + + // Establishes a reachable ScopedAllocator. + Status AddScopedAllocator( + const Tensor& backing_tensor, int64 step_id, int32 scope_id, + const string& scope_name, + const gtl::ArraySlice& fields, + int32 expected_call_count); + + void Cleanup(int64 step_id); + + // Populate the bytes and offset members of Field. Instance allocaters get + // consecutive scope_id values following that of the base ScopedAllocator. + // Returns the total number of bytes required to be allocated in the + // backing tensor, for convenience. (The same value can be obtained + // by summing offset and bytes in the last field.) + static size_t PopulateFields(int32 scope_id, + const gtl::ArraySlice& shapes, + const DataType dtype, + std::vector* fields); + + const string& device_name() const { return device_name_; } + + private: + string device_name_; + mutex mu_; + std::unordered_map per_step_map_ + GUARDED_BY(mu_); +}; + +} // namespace tensorflow +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_SCOPED_ALLOCATOR_MGR_H_ diff --git a/linux/include/tensorflow/core/common_runtime/session_factory.h b/linux/include/tensorflow/core/common_runtime/session_factory.h new file mode 100644 index 000000000..8565088af --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/session_factory.h @@ -0,0 +1,76 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_SESSION_FACTORY_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_SESSION_FACTORY_H_ + +#include + +#include "tensorflow/core/lib/core/errors.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/lib/gtl/array_slice.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +class Session; +struct SessionOptions; + +class SessionFactory { + public: + // Creates a new session and stores it in *out_session, or fails with an error + // status if the Session could not be created. Caller takes ownership of + // *out_session if this returns Status::OK(). + virtual Status NewSession(const SessionOptions& options, + Session** out_session) = 0; + + virtual bool AcceptsOptions(const SessionOptions& options) = 0; + + // Abort and close all existing sessions, disconnecting their resources from + // future sessions. + // + // Reset() allows misbehaving or slow sessions to be aborted and closed, and + // causes their resources eventually to be released. Reset() does not wait + // for the computations in old sessions to cease; it merely starts the + // process of tearing them down. However, if a new session is started after + // a Reset(), the new session is isolated from changes that old sessions + // (started prior to the Reset()) may continue to make to resources, provided + // all those resources are in containers listed in "containers". + // + // Old sessions may continue to have side-effects on resources not in + // containers listed in "containers", and thus may affect future + // sessions' results in ways that are hard to predict. Thus, if well-defined + // behavior is desired, is it recommended that all containers be listed in + // "containers". + // + // If the "containers" vector is empty, the default container is assumed. + // If the "containers" vector is non-empty, the default container should be + // listed explicitly. + // + // Sessions that support resource containers should override this function. + virtual Status Reset(const SessionOptions& options, + const std::vector& containers) { + return errors::Unimplemented("Reset()"); + } + + virtual ~SessionFactory() {} + static void Register(const string& runtime_type, SessionFactory* factory); + static Status GetFactory(const SessionOptions& options, + SessionFactory** out_factory); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_SESSION_FACTORY_H_ diff --git a/linux/include/tensorflow/core/common_runtime/shape_refiner.h b/linux/include/tensorflow/core/common_runtime/shape_refiner.h new file mode 100644 index 000000000..9c96dcbc2 --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/shape_refiner.h @@ -0,0 +1,306 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_SHAPE_REFINER_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_SHAPE_REFINER_H_ + +#include + +#include "tensorflow/core/common_runtime/graph_runner.h" +#include "tensorflow/core/framework/function.pb.h" +#include "tensorflow/core/framework/shape_inference.h" +#include "tensorflow/core/graph/graph.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/platform/macros.h" + +namespace tensorflow { +namespace grappler { +class GraphProperties; +} + +// This class stores extra inference information in addition to +// InferenceContext, such as inference tree for user-defined functions and node +// input and output types. +class ExtendedInferenceContext { + public: + ExtendedInferenceContext( + std::unique_ptr ic, const Node* node) + : inference_context_(std::move(ic)) { + input_types_.reserve(node->num_inputs()); + for (int i = 0; i < node->num_inputs(); i++) { + input_types_.push_back(node->input_type(i)); + } + output_types_.reserve(node->num_outputs()); + for (int i = 0; i < node->num_outputs(); i++) { + output_types_.push_back(node->output_type(i)); + } + } + + const std::unordered_map>& + nested_inferences() const { + return nested_inferences_; + } + DataType input_type(int64 idx) const { return input_types_[idx]; } + DataType output_type(int64 idx) const { return output_types_[idx]; } + + shape_inference::InferenceContext* get_context() { + return inference_context_.get(); + } + + // Sets nested inference info. + // For composite ops (user-defined functions) only. + // Inference for trivial ops must not call this setter. + void set_nested_inferences( + std::unordered_map> + inferences) { + nested_inferences_ = std::move(inferences); + } + + private: + std::unique_ptr inference_context_; + std::vector input_types_; + std::vector output_types_; + + // Nested inferences for composite ops (user-defined functions). + // Mapping key is nested node name. + // For trivial ops this map must be empty. + std::unordered_map> + nested_inferences_; + + TF_DISALLOW_COPY_AND_ASSIGN(ExtendedInferenceContext); +}; + +// ShapeRefiner performs shape inference for TensorFlow Graphs. It is +// responsible for instantiating InferenceContext objects for each +// Node in the Graph, and providing/storing the 'input_tensor' Tensors +// used by Shape Inference functions, when available at graph +// construction time. +class ShapeRefiner { + public: + ShapeRefiner(int graph_def_version, const OpRegistryInterface* ops); + + // Same as ShapeRefiner(versions.producer(), ops) + ShapeRefiner(const VersionDef& versions, const OpRegistryInterface* ops); + + ~ShapeRefiner(); + + // Performs validation of 'node' and runs 'node's shape function, + // storing its shape outputs. + // + // All inputs of 'node' must be added to ShapeRefiner prior to + // adding 'node'. + // + // Returns an error if: + // - the shape function for 'node' was not registered. + // - 'node' was added before its inputs. + // - The shape inference function returns an error. + Status AddNode(const Node* node); + + // Sets 'node's 'output_port' output to have shape 'shape'. + // + // Returns an error if 'node' was not previously added to this + // object, if 'output_port' is invalid, or if 'shape' is + // not compatible with the existing shape of the output. + Status SetShape(const Node* node, int output_port, + shape_inference::ShapeHandle shape); + + // Update the input shapes of node in case the shapes of the fan-ins of 'node' + // have themselves been modified (For example, in case of incremental shape + // refinement). If 'relax' is true, a new shape with the broadest set of + // information will be set as the new input (see InferenceContext::RelaxInput + // for full details and examples). Sets refined to true if any shapes have + // changed (in their string representations). Note that shapes may have been + // updated to newer versions (but with identical string representations) even + // if <*refined> is set to false. + Status UpdateNode(const Node* node, bool relax, bool* refined); + + // Returns the InferenceContext for 'node', if present. + shape_inference::InferenceContext* GetContext(const Node* node) const { + auto it = node_to_context_.find(node); + if (it == node_to_context_.end()) { + return nullptr; + } + return it->second->get_context(); + } + + // Returns the ExtendedInferenceContext for 'node', if present. + ExtendedInferenceContext* GetExtendedContext(const Node* node) const { + auto it = node_to_context_.find(node); + if (it == node_to_context_.end()) { + return nullptr; + } + return it->second.get(); + } + + // Getters and setters for graph_def_version_. + int32 graph_def_version() const { return graph_def_version_; } + void set_graph_def_version(int32 version) { graph_def_version_ = version; } + + void set_require_shape_inference_fns(bool require_shape_inference_fns) { + require_shape_inference_fns_ = require_shape_inference_fns; + } + void set_disable_constant_propagation(bool disable) { + disable_constant_propagation_ = disable; + } + + // Set function library to enable function shape inference. + // Without function library, function inference always yields unknown shapes. + // With this enabled, shape inference can take more time since it descends + // into all function calls. It doesn't do inference once for each function + // definition, but once for each function call. + // The function library must outlive the shape refiner. + void set_function_library_for_shape_inference( + const tensorflow::FunctionLibraryDefinition* lib) { + function_library_ = lib; + } + + bool function_shape_inference_supported() const { + return function_library_ != nullptr; + } + + // Call this to keep nested shapes information for user-defined functions: + // nested inferences will be available on the ExtendedInferenceContext for + // each function node, forming a tree of shape inferences corresponding to the + // tree of nested function calls. By default this setting is disabled, and + // only the shapes for the top-level function node will be reported on the + // InferenceContext for each function node, to reduce memory usage. + // + // This flag has no effect when the function inference is not enabled via + // set_function_library_for_shape_inference. + void set_keep_nested_shape_inferences() { + keep_nested_shape_inferences_ = true; + } + + private: + friend class ShapeRefinerTest; + friend class ::tensorflow::grappler::GraphProperties; + + // Returns true if the ranks and all dimensions of and are either + // equal in value or both unknown. + static bool SameDefinedShape(shape_inference::InferenceContext* c, + shape_inference::ShapeHandle s0, + shape_inference::ShapeHandle s1); + + // Returns true if the shapes and types stored in <*existing> are identical in + // value to the shapes and types in <*updated>. + static bool IsUpdatedShapesOrTypes( + shape_inference::InferenceContext* c, + const std::vector& existing, + const std::vector& updated); + + // Performs shape inference for the given function_def within the + // given outer_context. Internally it instantiates the function as a graph + // and runs shape inference recursively on it with the input shapes provided + // by the outer_context. + // + // Returns an error if: + // - number of inputs/outputs on outer_context doesn't match the function_def + // + // On success: + // - outer_context will contain output shapes inferred from input shapes + // - outer_context will contain nested inferences collection, iff + // keep_nested_shapes is true + Status InferShapesForFunction(const tensorflow::FunctionDef* function_def, + bool keep_nested_shapes, + ExtendedInferenceContext* outer_context); + + // Attempts to evaluate the 'dst_idx'-th input to 'node'. If the input edge + // value can be evaluated, 'evaluated' is set to true and the value returned + // in 'result'. Otherwise 'evaluated' is set to false. + Status EvaluateConstantTensorForEdge(const Node* node, int dst_idx, + bool* evaluated, Tensor* result); + + // Wrapper around EvaluateConstantTensorForEdge for scalar int32/int64 input + // tensors. The caller is responsible for checking that the specified edge is + // scalar and int32 or int64. + Status EvaluateConstantIntScalarEdge(const Node* node, int dst_idx, + bool* evaluated, int64* result); + + // This function tries to materialize as much information about the 'node''s + // dst_idx input as a statically computable shape, and the result may be + // partially known, depending on what is statically inferable. + // + // This is called when node.input[dst_idx] is a tensor that is used to define + // the shape of some other tensor (e.g., the second argument to Reshape is a + // tensor, where each element of the shape tensor is a dimension of + // the target tensor). It returns in a shape for that input. + // + // Unlike simply resolving node.input[dst_idx] to a constant and then + // converting that to a shape, this function can return a partial shape. This + // is useful for cases where the shape tensor is only partially defined, such + // as with calls for: reshape(x, shape(y)) where shape(y) is partially + // defined. + // + // The implementation has op implementations for ops commonly called on shape + // tensors, and the implementations are specialized to shape tensors (namely, + // the output is a vector). + // + // is used when creating new DimensionHandle and ShapeHandle + // objects. + Status ConstantPartialShape(shape_inference::InferenceContext* target_context, + const Node* node, int dst_idx, + shape_inference::ShapeHandle* result); + + // Implementation of ConstantPartialShape for StridedSlice nodes. + Status PartialStridedSliceShape(Node* slice_node, + shape_inference::InferenceContext* ctx, + shape_inference::ShapeHandle* result); + + Status RunShapeFn(const Node* node, const OpRegistrationData* op_reg_data, + ExtendedInferenceContext* ec); + + int32 graph_def_version_; + const OpRegistryInterface* const ops_registry_; + + // The lifetime of the tensors are bound to the runner, so it should be the + // deleted after the tensors. + GraphRunner graph_runner_; + + // Stores a map from a node to its ExtendedInferenceContext. + std::unordered_map> + node_to_context_; + + // Holds a cache from 'tensor name' to the tensor that is + // evaluatable as a constant expression. This reduces repeated + // execution of the entire constant subgraph as a graph is being + // built up. This could be changed to some kind of size-based LRU + // cache to avoid consuming too much memory, if that eventually + // becomes a concern. + // + // Only tensors less than 1KiB are currently stored in the cache. + static constexpr int64 kMaxTensorSize = 1024; + std::unordered_map const_tensor_map_; + + bool require_shape_inference_fns_ = true; + bool disable_constant_propagation_ = false; + + // Function library is optional, but has to be set to enable function + // shape inference. + const tensorflow::FunctionLibraryDefinition* function_library_ = nullptr; + + // Determines whether to keep the nested shape inference info for user- + // defined functions. By default that info is discarded to save memory. + bool keep_nested_shape_inferences_ = false; + + // Cache the graph corresponding to each functin definition for which shapes + // are refined. + std::unordered_map> + functions_; + + TF_DISALLOW_COPY_AND_ASSIGN(ShapeRefiner); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_SHAPE_REFINER_H_ diff --git a/linux/include/tensorflow/core/common_runtime/single_threaded_cpu_device.h b/linux/include/tensorflow/core/common_runtime/single_threaded_cpu_device.h new file mode 100644 index 000000000..22650b0d8 --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/single_threaded_cpu_device.h @@ -0,0 +1,83 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_SINGLE_THREADED_CPU_DEVICE_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_SINGLE_THREADED_CPU_DEVICE_H_ + +#define EIGEN_USE_THREADS + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/common_runtime/device.h" +#include "tensorflow/core/common_runtime/eigen_thread_pool.h" +#include "tensorflow/core/framework/allocator.h" +#include "tensorflow/core/framework/tensor.pb.h" +#include "tensorflow/core/lib/core/threadpool.h" + +namespace tensorflow { + +class Env; + +// A simple single-threaded CPU device. This can be used to run inexpensive +// computations. In particular, using this avoids initializing the global thread +// pools in LocalDevice. +class SingleThreadedCpuDevice : public Device { + public: + SingleThreadedCpuDevice(Env* env) + : Device(env, Device::BuildDeviceAttributes("/device:CPU:0", DEVICE_CPU, + Bytes(256 << 20), + DeviceLocality())) { + eigen_worker_threads_.num_threads = 1; + eigen_worker_threads_.workers = new thread::ThreadPool( + env, "graph_runner", eigen_worker_threads_.num_threads); + eigen_threadpool_wrapper_.reset( + new EigenThreadPoolWrapper(eigen_worker_threads_.workers)); + eigen_device_.reset(new Eigen::ThreadPoolDevice( + eigen_threadpool_wrapper_.get(), eigen_worker_threads_.num_threads)); + set_tensorflow_cpu_worker_threads(&eigen_worker_threads_); + set_eigen_cpu_device(eigen_device_.get()); + } + + ~SingleThreadedCpuDevice() override { + eigen_threadpool_wrapper_.reset(); + eigen_device_.reset(); + delete eigen_worker_threads_.workers; + } + + Status Sync() override { return Status::OK(); } + + Status MakeTensorFromProto(const TensorProto& tensor_proto, + const AllocatorAttributes alloc_attrs, + Tensor* tensor) override { + Tensor parsed(tensor_proto.dtype()); + if (!parsed.FromProto(cpu_allocator(), tensor_proto)) { + return errors::InvalidArgument("Cannot parse tensor from tensor_proto."); + } + *tensor = parsed; + return Status::OK(); + } + + Allocator* GetAllocator(AllocatorAttributes attr) override { + return cpu_allocator(); + } + + private: + DeviceBase::CpuWorkerThreads eigen_worker_threads_; + std::unique_ptr eigen_threadpool_wrapper_; + std::unique_ptr eigen_device_; +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_SINGLE_THREADED_CPU_DEVICE_H_ diff --git a/linux/include/tensorflow/core/common_runtime/stats_publisher_interface.h b/linux/include/tensorflow/core/common_runtime/stats_publisher_interface.h new file mode 100644 index 000000000..f063ee529 --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/stats_publisher_interface.h @@ -0,0 +1,64 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_STATS_PUBLISHER_INTERFACE_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_STATS_PUBLISHER_INTERFACE_H_ + +#include "tensorflow/core/common_runtime/build_graph_options.h" +#include "tensorflow/core/common_runtime/profile_handler.h" +#include "tensorflow/core/protobuf/config.pb.h" +#include "tensorflow/core/public/session_options.h" + +namespace tensorflow { + +// StatsPublisherInterface describes objects that publish information exported +// by Sessions. +// NOTE: This interface is experimental and subject to change. +// Implementations must be thread-safe. +class StatsPublisherInterface { + public: + // PublishStatsProto publishes step_stats. + // When PublishStatsProto is called multiple times, only the step_stats + // corresponding to the latest call will be published. + virtual void PublishStatsProto(const StepStats& step_stats) = 0; + + // PublishGraphProto publishes the graph_defs corresponding to each partition + // in the session. + // When PublishGraphProto is called multiple times, only the graph_defs + // corresponding to the latest call will be published. + virtual void PublishGraphProto( + const std::vector& graph_defs) = 0; + + // Returns a profile handler for the given step based on the execution_count + // and RunOptions. + // + // This method may return a null pointer, if no handler was created. + virtual std::unique_ptr GetProfileHandler( + uint64 step, int64 execution_count, const RunOptions& ropts) = 0; + + virtual ~StatsPublisherInterface() {} +}; + +typedef std::function( + const string&, const BuildGraphOptions&, const SessionOptions&)> + StatsPublisherFactory; + +std::unique_ptr CreateNoOpStatsPublisher( + const string& session, const BuildGraphOptions& bopts, + const SessionOptions& sopts); + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_STATS_PUBLISHER_INTERFACE_H_ diff --git a/linux/include/tensorflow/core/common_runtime/step_stats_collector.h b/linux/include/tensorflow/core/common_runtime/step_stats_collector.h new file mode 100644 index 000000000..4365b11b1 --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/step_stats_collector.h @@ -0,0 +1,200 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_STEP_STATS_COLLECTOR_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_STEP_STATS_COLLECTOR_H_ + +#include +#include +#include +#include "tensorflow/core/framework/step_stats.pb.h" +#include "tensorflow/core/framework/tensor_reference.h" +#include "tensorflow/core/lib/gtl/inlined_vector.h" +#include "tensorflow/core/platform/env.h" +#include "tensorflow/core/platform/mutex.h" +#include "tensorflow/core/platform/thread_annotations.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +class Allocator; +class AllocatorMemoryUsed; +class CostModelManager; +class Graph; +class Node; +class NodeExecStats; +class OpKernelContext; +class StepStats; +class StepStatsCollector; +class Tensor; +class TrackingAllocator; + +// Statistics collection interface for individual node execution. +// +// See `NodeExecStatsWrapper` for a concrete implementation of this interface +// that interfaces with the `Session` layer. +class NodeExecStatsInterface { + public: + virtual ~NodeExecStatsInterface() {} + + // Called when the statistics collection for the node has finished. Once this + // method is called, the caller should not make assumptions about the validity + // of this object. + virtual void Done(const string& device) = 0; + + // Called immediately after this node starts being processed by the executor. + virtual void RecordExecutorStarted() = 0; + + // Called immediately before this node's `Compute()` or `ComputeAsync()` + // method is called. + virtual void RecordComputeStarted() = 0; + + // Called immediately after this node's `Compute()` method returned (or, for + // asynchronous operations, the callback passed to its `ComputeAsync()` method + // was called). + virtual void RecordComputeEnded() = 0; + + // Called immediately after this executor finishes processing this node. + virtual void RecordExecutorEnded() = 0; + + // Records information about the memory allocated during the execution of this + // node. + virtual void SetMemory(OpKernelContext* ctx) = 0; + + // Records information about the tensor produced by this node at the given + // output slot. + virtual void SetOutput(int slot, const Tensor* tensor) = 0; + + // Records information about the tensors that were accessed during the + // execution of this node. + virtual void SetReferencedTensors(const TensorReferenceVector& tensors) = 0; + + // Records the absolute time in nanoseconds at which this node became + // runnable (i.e. was scheduled for execution). + virtual void SetScheduled(int64 nanos) = 0; +}; + +// Wraps NodeExecStats and adds allocation to it. +class NodeExecStatsWrapper : public NodeExecStatsInterface { + public: + // Does not take ownership of `node` or `step_stats_collector`. + NodeExecStatsWrapper(const Node* node, + StepStatsCollector* step_stats_collector); + + // Takes ownership of 'stats' but not `node` or `step_stats_collector`. + NodeExecStatsWrapper(std::unique_ptr stats, const Node* node, + StepStatsCollector* step_stats_collector); + + // Destructor calls Finalize() to release the TrackingAllocators. + ~NodeExecStatsWrapper() { Finalize(); } + + void Done(const string& device) override; + void RecordExecutorStarted() override; + void RecordComputeStarted() override; + void RecordComputeEnded() override; + void RecordExecutorEnded() override; + void SetMemory(OpKernelContext* ctx) override; + void SetOutput(int slot, const Tensor* tensor) override; + void SetReferencedTensors(const TensorReferenceVector& tensors) override; + void SetScheduled(int64 nanos) override; + + private: + friend class StepStatsCollector; + + NodeExecStats* stats() { return stats_.get(); } + + // Populates stats_ and releases TrackingAllocator. + void Finalize(); + + // Does not take ownership of the `allocator`. + // Takes ownership of `tracking_allocator`. + void AddAllocation(Allocator* allocator, + TrackingAllocator* tracking_allocator); + + gtl::InlinedVector, 2> + allocations_; + std::unique_ptr stats_; + const Node* const node_; // Not owned. + StepStatsCollector* const step_stats_collector_; // Not owned. +}; + +// Statistics collection interface for step execution. +// +// See `StepStatsCollector` for a concrete implementation of this interface +// that interfaces with the `Session` layer. +class StepStatsCollectorInterface { + public: + virtual ~StepStatsCollectorInterface() {} + + // Creates an instance of `NodeExecStatsInterface` that should be used for + // collecting statistics about individual node execution. + virtual NodeExecStatsInterface* CreateNodeExecStats(const Node* node) = 0; + + // Generates a string reporting the currently used memory based + // on ResourceExhausted OOM `err` message. + // `err` message needs to contain device name and allocator name, e.g.: + // "ResourceExhaustedError: OOM when allocating tensor ... + // on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc" + virtual string ReportAllocsOnResourceExhausted(const string& err) = 0; +}; + +// StepStatsCollector manages the collection of a StepStats object. +// The StepStats object holds multiple DeviceStats. +// Each DeviceStats object holds multiple NodeExecStats. +class StepStatsCollector : public StepStatsCollectorInterface { + public: + // Does not take ownership of `step_stats`. + explicit StepStatsCollector(StepStats* step_stats); + + // BuildCostModel builds or updates a CostModel managed by cost_model_manager, + // using the currently collected DeviceStats associated with the devices in + // device_map. + void BuildCostModel( + CostModelManager* cost_model_manager, + const std::unordered_map& device_map); + + // Saves node statistics to the DeviceStats object associated with device. + // Should be called before Finalize. + void Save(const string& device, NodeExecStats* node_stats_pb); + void Save(const string& device, NodeExecStatsWrapper* node_stats); + + NodeExecStatsInterface* CreateNodeExecStats(const Node* node) override; + string ReportAllocsOnResourceExhausted(const string& err) override; + + // The following 2 Finalize methods populate the StepStats passed + // from the constructor. Calling it more than once won't have any effect. + // User shouldn't call Save() methods after Finalize. + void Finalize(); + // swaps the content of StepStats* from constructor with 'ss'. + void FinalizeAndSwap(StepStats* step_stats); + + private: + // TODO(suharshs): Make this configurable if its not possible to find a value + // that works for all cases. + static const uint64 kMaxCollectedNodes = 1 << 20; + + typedef std::vector> NodeStatsVector; + + void FinalizeInternal() EXCLUSIVE_LOCKS_REQUIRED(mu_); + + mutex mu_; + bool finalized_ GUARDED_BY(mu_); + std::unordered_map dev_stats_ GUARDED_BY(mu_); + StepStats* step_stats_ GUARDED_BY(mu_); + uint64 collected_nodes_ GUARDED_BY(mu_) = 0; +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_STEP_STATS_COLLECTOR_H_ diff --git a/linux/include/tensorflow/core/common_runtime/sycl/sycl_allocator.h b/linux/include/tensorflow/core/common_runtime/sycl/sycl_allocator.h new file mode 100644 index 000000000..cc5909de1 --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/sycl/sycl_allocator.h @@ -0,0 +1,75 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#if !TENSORFLOW_USE_SYCL +#error This file must only be included when building TensorFlow with SYCL support +#endif + +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_SYCL_SYCL_ALLOCATOR_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_SYCL_SYCL_ALLOCATOR_H_ + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/allocator.h" +#include "tensorflow/core/platform/mutex.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +class SYCLAllocator : public Allocator { + public: + SYCLAllocator(Eigen::QueueInterface* queue); + virtual ~SYCLAllocator() override; + string Name() override; + void* AllocateRaw(size_t alignment, size_t num_bytes) override; + void DeallocateRaw(void* ptr) override; + + virtual bool ShouldAllocateEmptyTensors() override final { return true; } + void Synchronize() { + mutex_lock lock(mu_); + if (sycl_device_) { + sycl_device_->synchronize(); + } + } + bool Ok() { return sycl_device_ && sycl_device_->ok(); } + void GetStats(AllocatorStats* stats) override; + void ClearStats() override; + + // The SYCL buffers keep track of their size, so we already have tracking. + bool TracksAllocationSizes() override { return true; } + // Get the size of the corresponding SYCL buffer. + // Implementing this also provides an implementation of + // AllocatedSize(void* ptr) by default. + size_t RequestedSize(void* ptr) override; + Eigen::SyclDevice* getSyclDevice() { return sycl_device_; } + // Clear the SYCL device used by the Allocator + void ClearSYCLDevice() { + mutex_lock lock(mu_); + if (sycl_device_) { + delete sycl_device_; + sycl_device_ = nullptr; + } + } + + private: + mutable mutex mu_; + Eigen::SyclDevice* sycl_device_ GUARDED_BY(mu_); // owned + AllocatorStats stats_ GUARDED_BY(mu_); + + TF_DISALLOW_COPY_AND_ASSIGN(SYCLAllocator); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_SYCL_SYCL_ALLOCATOR_H_ diff --git a/linux/include/tensorflow/core/common_runtime/sycl/sycl_device.h b/linux/include/tensorflow/core/common_runtime/sycl/sycl_device.h new file mode 100644 index 000000000..7c09e0b8f --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/sycl/sycl_device.h @@ -0,0 +1,232 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#if !TENSORFLOW_USE_SYCL +#error This file must only be included when building TensorFlow with SYCL support +#endif + +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_SYCL_SYCL_DEVICE_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_SYCL_SYCL_DEVICE_H_ + +#include "tensorflow/core/common_runtime/local_device.h" +#include "tensorflow/core/common_runtime/sycl/sycl_allocator.h" +#include "tensorflow/core/common_runtime/sycl/sycl_device_context.h" +#include "tensorflow/core/public/session_options.h" + +namespace tensorflow { + +class GSYCLInterface { + std::vector m_queue_interface_; // owned + std::vector m_cpu_allocator_; // not owned + std::vector m_sycl_allocator_; // owned + std::vector m_sycl_context_; // ref counted + GSYCLInterface() { + bool found_device = false; + auto device_list = Eigen::get_sycl_supported_devices(); + // Obtain list of supported devices from Eigen + for (const auto& device : device_list) { + if (device.is_gpu()) { + // returns first found GPU + AddDevice(device); + found_device = true; + } + } + + if (!found_device) { + // Currently Intel GPU is not supported + LOG(WARNING) << "No OpenCL GPU found that is supported by " + << "ComputeCpp/triSYCL, trying OpenCL CPU"; + } + + for (const auto& device : device_list) { + if (device.is_cpu()) { + // returns first found CPU + AddDevice(device); + found_device = true; + } + } + + if (!found_device) { + LOG(WARNING) << "No OpenCL CPU found that is supported by " + << "ComputeCpp/triSYCL, checking for host sycl device"; + } + + for (const auto& device : device_list) { + // triSYCL only supports the host device for now + if (device.is_host()) { + LOG(WARNING) << "Found SYCL host device"; + AddDevice(device); + found_device = true; + } + } + + if (!found_device) { + // Currently Intel GPU is not supported + LOG(FATAL) << "No SYCL host and no OpenCL GPU nor CPU" + << " supported by ComputeCPP/triSYCL was found"; + } else { + LOG(INFO) << "Found following OpenCL devices:"; + for (int i = 0; i < device_list.size(); i++) { + LOG(INFO) << GetShortDeviceDescription(i); + } + } + } + + ~GSYCLInterface() { + m_cpu_allocator_.clear(); + + for (auto p : m_sycl_allocator_) { + p->Synchronize(); + p->ClearSYCLDevice(); + // Cannot delete the Allocator instances, as the Allocator lifetime + // needs to exceed any Tensor created by it. There is no way of + // knowing when all Tensors have been deallocated, as they are + // RefCounted and wait until all instances of a Tensor have been + // destroyed before calling Allocator.Deallocate. This could happen at + // program exit, which can set up a race condition between destroying + // Tensors and Allocators when the program is cleaning up. + } + m_sycl_allocator_.clear(); + + for (auto p : m_sycl_context_) { + p->Unref(); + } + m_sycl_context_.clear(); + + for (auto p : m_queue_interface_) { + p->deallocate_all(); + delete p; + } + m_queue_interface_.clear(); + } + + void AddDevice(const cl::sycl::device& d) { + m_queue_interface_.push_back(new Eigen::QueueInterface(d)); + m_cpu_allocator_.push_back(cpu_allocator()); + m_sycl_allocator_.push_back(new SYCLAllocator(m_queue_interface_.back())); + m_sycl_context_.push_back(new SYCLDeviceContext()); + } + + public: + static const GSYCLInterface* instance() { + // c++11 guarantees that this will be constructed in a thread safe way + static const GSYCLInterface instance; + return &instance; + } + + Eigen::QueueInterface* GetQueueInterface(size_t i = 0) const { + if (!m_queue_interface_.empty()) { + return m_queue_interface_[i]; + } else { + std::cerr << "No cl::sycl::device has been added" << std::endl; + return nullptr; + } + } + + SYCLAllocator* GetSYCLAllocator(size_t i = 0) const { + if (!m_sycl_allocator_.empty()) { + return m_sycl_allocator_[i]; + } else { + std::cerr << "No cl::sycl::device has been added" << std::endl; + return nullptr; + } + } + + Allocator* GetCPUAllocator(size_t i = 0) const { + if (!m_cpu_allocator_.empty()) { + return m_cpu_allocator_[i]; + } else { + std::cerr << "No cl::sycl::device has been added" << std::endl; + return nullptr; + } + } + + SYCLDeviceContext* GetSYCLContext(size_t i = 0) const { + if (!m_sycl_context_.empty()) { + return m_sycl_context_[i]; + } else { + std::cerr << "No cl::sycl::device has been added" << std::endl; + return nullptr; + } + } + + string GetShortDeviceDescription(int device_id = 0) const { + Eigen::QueueInterface* queue_ptr = GetQueueInterface(device_id); + if (!queue_ptr) { + LOG(ERROR) + << "Device name cannot be given after Eigen QueueInterface destroyed"; + return ""; + } + auto device = queue_ptr->sycl_queue().get_device(); + auto name = device.get_info(); + auto vendor = device.get_info(); + auto profile = device.get_info(); + + std::string type; + if (device.is_host()) { + type = "Host"; + } else if (device.is_cpu()) { + type = "CPU"; + } else if (device.is_gpu()) { + type = "GPU"; + } else if (device.is_accelerator()) { + type = "Accelerator"; + } else { + type = "Unknown"; + } + + return strings::StrCat( + "id: ", device_id, ", type: ", type, ", name: ", name.c_str(), + ", vendor: ", vendor.c_str(), ", profile: ", profile.c_str()); + } +}; + +class SYCLDevice : public LocalDevice { + public: + SYCLDevice(const SessionOptions& options, const string& name, + Bytes memory_limit, const DeviceLocality& locality, + const string& physical_device_desc, SYCLAllocator* sycl_allocator, + Allocator* cpu_allocator, SYCLDeviceContext* ctx) + : LocalDevice(options, Device::BuildDeviceAttributes( + name, DEVICE_SYCL, memory_limit, locality, + physical_device_desc)), + cpu_allocator_(cpu_allocator), + sycl_allocator_(sycl_allocator), + device_context_(ctx) { + set_eigen_sycl_device(sycl_allocator->getSyclDevice()); + } + + ~SYCLDevice() override; + + void Compute(OpKernel* op_kernel, OpKernelContext* context) override; + Allocator* GetAllocator(AllocatorAttributes attr) override; + Status MakeTensorFromProto(const TensorProto& tensor_proto, + const AllocatorAttributes alloc_attrs, + Tensor* tensor) override; + + Status FillContextMap(const Graph* graph, + DeviceContextMap* device_context_map) override; + + Status Sync() override; + + private: + Allocator* cpu_allocator_; // not owned + SYCLAllocator* sycl_allocator_; // not owned + SYCLDeviceContext* device_context_; // not owned +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_SYCL_SYCL_DEVICE_H_ diff --git a/linux/include/tensorflow/core/common_runtime/sycl/sycl_device_context.h b/linux/include/tensorflow/core/common_runtime/sycl/sycl_device_context.h new file mode 100644 index 000000000..0f8f17b80 --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/sycl/sycl_device_context.h @@ -0,0 +1,45 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#if !TENSORFLOW_USE_SYCL +#error This file must only be included when building TensorFlow with SYCL support +#endif + +#ifndef TENSORFLOW_COMMON_RUNTIME_SYCL_SYCL_DEVICE_CONTEXT_H_ +#define TENSORFLOW_COMMON_RUNTIME_SYCL_SYCL_DEVICE_CONTEXT_H_ + +#include "tensorflow/core/common_runtime/device.h" +#include "tensorflow/core/framework/device_base.h" + +namespace tensorflow { + +class SYCLDeviceContext : public DeviceContext { + public: + SYCLDeviceContext() {} + + ~SYCLDeviceContext() override {} + + void CopyCPUTensorToDevice(const Tensor *cpu_tensor, Device *device, + Tensor *device_tensor, + StatusCallback done) const override; + + void CopyDeviceTensorToCPU(const Tensor *device_tensor, StringPiece edge_name, + Device *device, Tensor *cpu_tensor, + StatusCallback done) override; +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_COMMON_RUNTIME_SYCL_SYCL_DEVICE_CONTEXT_H_ diff --git a/linux/include/tensorflow/core/common_runtime/sycl/sycl_util.h b/linux/include/tensorflow/core/common_runtime/sycl/sycl_util.h new file mode 100644 index 000000000..3124ed23c --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/sycl/sycl_util.h @@ -0,0 +1,80 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#if !TENSORFLOW_USE_SYCL +#error This file must only be included when building TensorFlow with SYCL support +#endif + +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_SYCL_SYCL_UTIL_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_SYCL_SYCL_UTIL_H_ + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/common_runtime/device.h" +// For DMA helper +#include "tensorflow/core/common_runtime/dma_helper.h" +#include "tensorflow/core/framework/tensor.h" + +namespace tensorflow { +inline void const* GetBase(const Tensor* src) { return DMAHelper::base(src); } +inline void* GetBase(Tensor* dst) { return DMAHelper::base(dst); } + +inline void SYCLmemcpy(Eigen::SyclDevice const& device, + Tensor const& src_tensor, Tensor* dst_tensor) { + const size_t size = src_tensor.TotalBytes(); + void* dst_ptr = GetBase(dst_tensor); + void const* src_ptr = GetBase(&src_tensor); + +#define COPY_WITH_TYPE(T) \ + device.memcpy(dst_ptr, static_cast(src_ptr), size); + switch (src_tensor.dtype()) { + case DT_COMPLEX128: + COPY_WITH_TYPE(cl::sycl::cl_ulong2); + break; + case DT_DOUBLE: + case DT_COMPLEX64: + case DT_INT64: + COPY_WITH_TYPE(cl::sycl::cl_ulong); + break; + case DT_FLOAT: + case DT_INT32: + case DT_QINT32: + COPY_WITH_TYPE(cl::sycl::cl_uint); + break; + case DT_INT16: + case DT_UINT16: + case DT_BFLOAT16: + case DT_QINT16: + case DT_QUINT16: + case DT_HALF: + COPY_WITH_TYPE(cl::sycl::cl_ushort); + break; + case DT_BOOL: + COPY_WITH_TYPE(bool); + break; + case DT_UINT8: + case DT_INT8: + case DT_QINT8: + case DT_QUINT8: + COPY_WITH_TYPE(cl::sycl::cl_uchar); + break; + default: + LOG(FATAL) << "Unknown data type " << src_tensor.dtype(); + break; + } +#undef COPY_WITH_TYPE +} +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_SYCL_SYCL_UTIL_H_ diff --git a/linux/include/tensorflow/core/common_runtime/threadpool_device.h b/linux/include/tensorflow/core/common_runtime/threadpool_device.h new file mode 100644 index 000000000..51bd038a1 --- /dev/null +++ b/linux/include/tensorflow/core/common_runtime/threadpool_device.h @@ -0,0 +1,51 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_COMMON_RUNTIME_THREADPOOL_DEVICE_H_ +#define TENSORFLOW_CORE_COMMON_RUNTIME_THREADPOOL_DEVICE_H_ + +#include "tensorflow/core/common_runtime/device_factory.h" +#include "tensorflow/core/common_runtime/local_device.h" + +namespace tensorflow { + +// CPU device implementation. +class ThreadPoolDevice : public LocalDevice { + public: + ThreadPoolDevice(const SessionOptions& options, const string& name, + Bytes memory_limit, const DeviceLocality& locality, + Allocator* allocator); + ~ThreadPoolDevice() override; + + Allocator* GetAllocator(AllocatorAttributes attr) override; + Allocator* GetScopedAllocator(AllocatorAttributes attr, + int64 step_id) override; + ScopedAllocatorMgr* GetScopedAllocatorMgr() const override { + return scoped_allocator_mgr_.get(); + } + Status MakeTensorFromProto(const TensorProto& tensor_proto, + const AllocatorAttributes alloc_attrs, + Tensor* tensor) override; + + Status Sync() override { return Status::OK(); } + + private: + Allocator* allocator_; // Not owned + std::unique_ptr scoped_allocator_mgr_; +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_COMMON_RUNTIME_THREADPOOL_DEVICE_H_ diff --git a/linux/include/tensorflow/core/debug/debug_graph_utils.h b/linux/include/tensorflow/core/debug/debug_graph_utils.h new file mode 100644 index 000000000..86dc90a13 --- /dev/null +++ b/linux/include/tensorflow/core/debug/debug_graph_utils.h @@ -0,0 +1,126 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_DEBUG_DEBUG_GRAPH_UTILS_H_ +#define TENSORFLOW_CORE_DEBUG_DEBUG_GRAPH_UTILS_H_ + +#include +#include + +#include "tensorflow/core/common_runtime/debugger_state_interface.h" +#include "tensorflow/core/common_runtime/device.h" +#include "tensorflow/core/graph/graph.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/platform/protobuf.h" +#include "tensorflow/core/protobuf/debug.pb.h" + +namespace tensorflow { + +class DebugNodeInserter { + public: + // EXPERIMENTAL: Insert special debug ops (e.g., DebugIdentity) to graph for + // debugging. Currently, such ops need to take exactly one input and has the + // string attribute "tensor_name" to indicate what tensor it watches. + // For example, before the node insertion, the graph may look like: + // + // A:0 -----------1----------> B + // | + // ---------2-----------> C + // + // wherein the output slot 0 of node A feeds as the input to nodes B through + // edge 1 and to node C through edge 2. + // After the node insertion, assuming both B and C have non-Ref input, the + // graph becomes: + // A:0 ---3---> Copy -----------4----------> B + // | + // ---------5--------> C + // | + // ---------6--------> X + // + // If a node (e.g., B) has Ref input, the graph becomes: + // + // --------------------------------> B + // | + // A:0 ---3-----> Copy -----------4----------> C + // | + // -----------5--------> X + // + // In other words, we do not feed Refs to deep-copies to downstream nodes. + // + // Copy is the inserted deep-copy node that copies the input tensor on-device + // (e.g., CPU-to-CPU or GPU-to-GPU deep copy) that reduces the likelihood of + // racy updates during the debug watches. X is the newly created debug node + // that transforms the input (copy of the watched tensor) into a debug signal. + // + // DebugIdentity is the simplest debugging paradigm, in which the debug signal + // (i.e., X:0) equals the tensor itself. More sophisticated debug ops can be + // used to transform the tensor into other debug signals. An example is the + // DebugNanCounter op. + // + // If the nodes (A, B and C) are located on GPU and the edges from A to B or C + // is HOST_MEMORY, then the CopyHost op will be used instead of the Copy op. + static Status InsertNodes( + const protobuf::RepeatedPtrField& watches, Graph* graph, + Device* device); + + // Set the parallel_iterations attribute of TensorFlow while loops + // (specifically the nodes for which IsEnter() returns true) to 1 to prevent + // any node from being executed multiple times concurrently and + // generating temporally-overlapping debug Tensor dumps. + static void DeparallelizeWhileLoops(Graph* graph, Device* device); + + // Get canonical name of a copy node. + static const string GetCopyNodeName(const string& node_name, + const int output_slot); + + // Get canonical name of a debug node. + static const string GetDebugNodeName(const string& tensor_name, + const int debug_op_num, + const string& debug_op_name); + + private: + static Status CreateCopyNode( + Graph* graph, const DeviceType device_type, const bool is_host_memory, + const string& src_node_name, const int src_output, const DataType src_dt, + const string& tensor_name, const std::vector& debug_ops, + const std::vector& debug_urls, Node** copy_node); + + // Parse the debug_op_name string to extract proper op name and attributes. + // debug_op_name can be the proper op name only, e.g., "DebugNumericSummary". + // It can also contain customizable keys and values. Each key-value pair is + // connected with an equal sign ("="). Multiple key-value pairs are separated + // with semicolons (";"), which optional whitespace in between, e.g., + // "DebugNumericSummary(mute_if_healthy=true, lower_bound=-100.0)". + static Status ParseDebugOpName( + const string& debug_op_name, string* debug_op_name_proper, + std::unordered_map* attributes); + + static Status SetDebugNodeAttributes( + Node* debug_node, const std::unordered_map& attributes); + + static Status CreateDebugNode(Graph* graph, const Device& device, + const string& src_copy_node_name, + const DataType src_dt, + const string& tensor_name, + const std::vector& debug_urls, + const int debug_op_num, + const string& debug_op_name, Node** debug_node); + // TODO(cais): Cut down the number of args to this method. + + friend class DebugGraphUtilsTest; +}; +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_DEBUG_DEBUG_GRAPH_UTILS_H_ diff --git a/linux/include/tensorflow/core/example/example.pb.h b/linux/include/tensorflow/core/example/example.pb.h new file mode 100644 index 000000000..0320ceb04 --- /dev/null +++ b/linux/include/tensorflow/core/example/example.pb.h @@ -0,0 +1,539 @@ +// Generated by the protocol buffer compiler. DO NOT EDIT! +// source: tensorflow/core/example/example.proto + +#ifndef PROTOBUF_INCLUDED_tensorflow_2fcore_2fexample_2fexample_2eproto +#define PROTOBUF_INCLUDED_tensorflow_2fcore_2fexample_2fexample_2eproto + +#include + +#include + +#if GOOGLE_PROTOBUF_VERSION < 3006000 +#error This file was generated by a newer version of protoc which is +#error incompatible with your Protocol Buffer headers. Please update +#error your headers. +#endif +#if 3006000 < GOOGLE_PROTOBUF_MIN_PROTOC_VERSION +#error This file was generated by an older version of protoc which is +#error incompatible with your Protocol Buffer headers. Please +#error regenerate this file with a newer version of protoc. +#endif + +#include +#include +#include +#include +#include +#include +#include +#include +#include // IWYU pragma: export +#include // IWYU pragma: export +#include +#include "tensorflow/core/example/feature.pb.h" +// @@protoc_insertion_point(includes) +#define PROTOBUF_INTERNAL_EXPORT_protobuf_tensorflow_2fcore_2fexample_2fexample_2eproto + +namespace protobuf_tensorflow_2fcore_2fexample_2fexample_2eproto { +// Internal implementation detail -- do not use these members. +struct TableStruct { + static const ::google::protobuf::internal::ParseTableField entries[]; + static const ::google::protobuf::internal::AuxillaryParseTableField aux[]; + static const ::google::protobuf::internal::ParseTable schema[2]; + static const ::google::protobuf::internal::FieldMetadata field_metadata[]; + static const ::google::protobuf::internal::SerializationTable serialization_table[]; + static const ::google::protobuf::uint32 offsets[]; +}; +void AddDescriptors(); +} // namespace protobuf_tensorflow_2fcore_2fexample_2fexample_2eproto +namespace tensorflow { +class Example; +class ExampleDefaultTypeInternal; +extern ExampleDefaultTypeInternal _Example_default_instance_; +class SequenceExample; +class SequenceExampleDefaultTypeInternal; +extern SequenceExampleDefaultTypeInternal _SequenceExample_default_instance_; +} // namespace tensorflow +namespace google { +namespace protobuf { +template<> ::tensorflow::Example* Arena::CreateMaybeMessage<::tensorflow::Example>(Arena*); +template<> ::tensorflow::SequenceExample* Arena::CreateMaybeMessage<::tensorflow::SequenceExample>(Arena*); +} // namespace protobuf +} // namespace google +namespace tensorflow { + +// =================================================================== + +class Example : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.Example) */ { + public: + Example(); + virtual ~Example(); + + Example(const Example& from); + + inline Example& operator=(const Example& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + Example(Example&& from) noexcept + : Example() { + *this = ::std::move(from); + } + + inline Example& operator=(Example&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const Example& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const Example* internal_default_instance() { + return reinterpret_cast( + &_Example_default_instance_); + } + static constexpr int kIndexInFileMessages = + 0; + + void UnsafeArenaSwap(Example* other); + void Swap(Example* other); + friend void swap(Example& a, Example& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline Example* New() const final { + return CreateMaybeMessage(NULL); + } + + Example* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const Example& from); + void MergeFrom(const Example& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(Example* other); + protected: + explicit Example(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // .tensorflow.Features features = 1; + bool has_features() const; + void clear_features(); + static const int kFeaturesFieldNumber = 1; + private: + const ::tensorflow::Features& _internal_features() const; + public: + const ::tensorflow::Features& features() const; + ::tensorflow::Features* release_features(); + ::tensorflow::Features* mutable_features(); + void set_allocated_features(::tensorflow::Features* features); + void unsafe_arena_set_allocated_features( + ::tensorflow::Features* features); + ::tensorflow::Features* unsafe_arena_release_features(); + + // @@protoc_insertion_point(class_scope:tensorflow.Example) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::tensorflow::Features* features_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fexample_2fexample_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class SequenceExample : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.SequenceExample) */ { + public: + SequenceExample(); + virtual ~SequenceExample(); + + SequenceExample(const SequenceExample& from); + + inline SequenceExample& operator=(const SequenceExample& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + SequenceExample(SequenceExample&& from) noexcept + : SequenceExample() { + *this = ::std::move(from); + } + + inline SequenceExample& operator=(SequenceExample&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const SequenceExample& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const SequenceExample* internal_default_instance() { + return reinterpret_cast( + &_SequenceExample_default_instance_); + } + static constexpr int kIndexInFileMessages = + 1; + + void UnsafeArenaSwap(SequenceExample* other); + void Swap(SequenceExample* other); + friend void swap(SequenceExample& a, SequenceExample& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline SequenceExample* New() const final { + return CreateMaybeMessage(NULL); + } + + SequenceExample* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const SequenceExample& from); + void MergeFrom(const SequenceExample& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(SequenceExample* other); + protected: + explicit SequenceExample(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // .tensorflow.Features context = 1; + bool has_context() const; + void clear_context(); + static const int kContextFieldNumber = 1; + private: + const ::tensorflow::Features& _internal_context() const; + public: + const ::tensorflow::Features& context() const; + ::tensorflow::Features* release_context(); + ::tensorflow::Features* mutable_context(); + void set_allocated_context(::tensorflow::Features* context); + void unsafe_arena_set_allocated_context( + ::tensorflow::Features* context); + ::tensorflow::Features* unsafe_arena_release_context(); + + // .tensorflow.FeatureLists feature_lists = 2; + bool has_feature_lists() const; + void clear_feature_lists(); + static const int kFeatureListsFieldNumber = 2; + private: + const ::tensorflow::FeatureLists& _internal_feature_lists() const; + public: + const ::tensorflow::FeatureLists& feature_lists() const; + ::tensorflow::FeatureLists* release_feature_lists(); + ::tensorflow::FeatureLists* mutable_feature_lists(); + void set_allocated_feature_lists(::tensorflow::FeatureLists* feature_lists); + void unsafe_arena_set_allocated_feature_lists( + ::tensorflow::FeatureLists* feature_lists); + ::tensorflow::FeatureLists* unsafe_arena_release_feature_lists(); + + // @@protoc_insertion_point(class_scope:tensorflow.SequenceExample) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::tensorflow::Features* context_; + ::tensorflow::FeatureLists* feature_lists_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fexample_2fexample_2eproto::TableStruct; +}; +// =================================================================== + + +// =================================================================== + +#ifdef __GNUC__ + #pragma GCC diagnostic push + #pragma GCC diagnostic ignored "-Wstrict-aliasing" +#endif // __GNUC__ +// Example + +// .tensorflow.Features features = 1; +inline bool Example::has_features() const { + return this != internal_default_instance() && features_ != NULL; +} +inline const ::tensorflow::Features& Example::_internal_features() const { + return *features_; +} +inline const ::tensorflow::Features& Example::features() const { + const ::tensorflow::Features* p = features_; + // @@protoc_insertion_point(field_get:tensorflow.Example.features) + return p != NULL ? *p : *reinterpret_cast( + &::tensorflow::_Features_default_instance_); +} +inline ::tensorflow::Features* Example::release_features() { + // @@protoc_insertion_point(field_release:tensorflow.Example.features) + + ::tensorflow::Features* temp = features_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + features_ = NULL; + return temp; +} +inline ::tensorflow::Features* Example::unsafe_arena_release_features() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.Example.features) + + ::tensorflow::Features* temp = features_; + features_ = NULL; + return temp; +} +inline ::tensorflow::Features* Example::mutable_features() { + + if (features_ == NULL) { + auto* p = CreateMaybeMessage<::tensorflow::Features>(GetArenaNoVirtual()); + features_ = p; + } + // @@protoc_insertion_point(field_mutable:tensorflow.Example.features) + return features_; +} +inline void Example::set_allocated_features(::tensorflow::Features* features) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(features_); + } + if (features) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(features)->GetArena(); + if (message_arena != submessage_arena) { + features = ::google::protobuf::internal::GetOwnedMessage( + message_arena, features, submessage_arena); + } + + } else { + + } + features_ = features; + // @@protoc_insertion_point(field_set_allocated:tensorflow.Example.features) +} + +// ------------------------------------------------------------------- + +// SequenceExample + +// .tensorflow.Features context = 1; +inline bool SequenceExample::has_context() const { + return this != internal_default_instance() && context_ != NULL; +} +inline const ::tensorflow::Features& SequenceExample::_internal_context() const { + return *context_; +} +inline const ::tensorflow::Features& SequenceExample::context() const { + const ::tensorflow::Features* p = context_; + // @@protoc_insertion_point(field_get:tensorflow.SequenceExample.context) + return p != NULL ? *p : *reinterpret_cast( + &::tensorflow::_Features_default_instance_); +} +inline ::tensorflow::Features* SequenceExample::release_context() { + // @@protoc_insertion_point(field_release:tensorflow.SequenceExample.context) + + ::tensorflow::Features* temp = context_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + context_ = NULL; + return temp; +} +inline ::tensorflow::Features* SequenceExample::unsafe_arena_release_context() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.SequenceExample.context) + + ::tensorflow::Features* temp = context_; + context_ = NULL; + return temp; +} +inline ::tensorflow::Features* SequenceExample::mutable_context() { + + if (context_ == NULL) { + auto* p = CreateMaybeMessage<::tensorflow::Features>(GetArenaNoVirtual()); + context_ = p; + } + // @@protoc_insertion_point(field_mutable:tensorflow.SequenceExample.context) + return context_; +} +inline void SequenceExample::set_allocated_context(::tensorflow::Features* context) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(context_); + } + if (context) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(context)->GetArena(); + if (message_arena != submessage_arena) { + context = ::google::protobuf::internal::GetOwnedMessage( + message_arena, context, submessage_arena); + } + + } else { + + } + context_ = context; + // @@protoc_insertion_point(field_set_allocated:tensorflow.SequenceExample.context) +} + +// .tensorflow.FeatureLists feature_lists = 2; +inline bool SequenceExample::has_feature_lists() const { + return this != internal_default_instance() && feature_lists_ != NULL; +} +inline const ::tensorflow::FeatureLists& SequenceExample::_internal_feature_lists() const { + return *feature_lists_; +} +inline const ::tensorflow::FeatureLists& SequenceExample::feature_lists() const { + const ::tensorflow::FeatureLists* p = feature_lists_; + // @@protoc_insertion_point(field_get:tensorflow.SequenceExample.feature_lists) + return p != NULL ? *p : *reinterpret_cast( + &::tensorflow::_FeatureLists_default_instance_); +} +inline ::tensorflow::FeatureLists* SequenceExample::release_feature_lists() { + // @@protoc_insertion_point(field_release:tensorflow.SequenceExample.feature_lists) + + ::tensorflow::FeatureLists* temp = feature_lists_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + feature_lists_ = NULL; + return temp; +} +inline ::tensorflow::FeatureLists* SequenceExample::unsafe_arena_release_feature_lists() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.SequenceExample.feature_lists) + + ::tensorflow::FeatureLists* temp = feature_lists_; + feature_lists_ = NULL; + return temp; +} +inline ::tensorflow::FeatureLists* SequenceExample::mutable_feature_lists() { + + if (feature_lists_ == NULL) { + auto* p = CreateMaybeMessage<::tensorflow::FeatureLists>(GetArenaNoVirtual()); + feature_lists_ = p; + } + // @@protoc_insertion_point(field_mutable:tensorflow.SequenceExample.feature_lists) + return feature_lists_; +} +inline void SequenceExample::set_allocated_feature_lists(::tensorflow::FeatureLists* feature_lists) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(feature_lists_); + } + if (feature_lists) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(feature_lists)->GetArena(); + if (message_arena != submessage_arena) { + feature_lists = ::google::protobuf::internal::GetOwnedMessage( + message_arena, feature_lists, submessage_arena); + } + + } else { + + } + feature_lists_ = feature_lists; + // @@protoc_insertion_point(field_set_allocated:tensorflow.SequenceExample.feature_lists) +} + +#ifdef __GNUC__ + #pragma GCC diagnostic pop +#endif // __GNUC__ +// ------------------------------------------------------------------- + + +// @@protoc_insertion_point(namespace_scope) + +} // namespace tensorflow + +// @@protoc_insertion_point(global_scope) + +#endif // PROTOBUF_INCLUDED_tensorflow_2fcore_2fexample_2fexample_2eproto diff --git a/linux/include/tensorflow/core/example/example.pb_text-impl.h b/linux/include/tensorflow/core/example/example.pb_text-impl.h new file mode 100644 index 000000000..68c5c8d00 --- /dev/null +++ b/linux/include/tensorflow/core/example/example.pb_text-impl.h @@ -0,0 +1,34 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_example_example_proto_IMPL_H_ +#define tensorflow_core_example_example_proto_IMPL_H_ + +#include "tensorflow/core/example/example.pb.h" +#include "tensorflow/core/example/example.pb_text.h" +#include "tensorflow/core/example/feature.pb.h" +#include "tensorflow/core/example/feature.pb_text-impl.h" +#include "tensorflow/core/lib/strings/proto_text_util.h" +#include "tensorflow/core/lib/strings/scanner.h" + +namespace tensorflow { + +namespace internal { + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::Example& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::Example* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::SequenceExample& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::SequenceExample* msg); + +} // namespace internal + +} // namespace tensorflow + +#endif // tensorflow_core_example_example_proto_IMPL_H_ diff --git a/linux/include/tensorflow/core/example/example.pb_text.h b/linux/include/tensorflow/core/example/example.pb_text.h new file mode 100644 index 000000000..873f923cf --- /dev/null +++ b/linux/include/tensorflow/core/example/example.pb_text.h @@ -0,0 +1,34 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_example_example_proto_H_ +#define tensorflow_core_example_example_proto_H_ + +#include "tensorflow/core/example/example.pb.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/protobuf.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +// Message-text conversion for tensorflow.Example +string ProtoDebugString( + const ::tensorflow::Example& msg); +string ProtoShortDebugString( + const ::tensorflow::Example& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::Example* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.SequenceExample +string ProtoDebugString( + const ::tensorflow::SequenceExample& msg); +string ProtoShortDebugString( + const ::tensorflow::SequenceExample& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::SequenceExample* msg) + TF_MUST_USE_RESULT; + +} // namespace tensorflow + +#endif // tensorflow_core_example_example_proto_H_ diff --git a/linux/include/tensorflow/core/example/example_parser_configuration.h b/linux/include/tensorflow/core/example/example_parser_configuration.h new file mode 100644 index 000000000..8bbed2847 --- /dev/null +++ b/linux/include/tensorflow/core/example/example_parser_configuration.h @@ -0,0 +1,56 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_EXAMPLE_EXAMPLE_PARSER_CONFIGURATION_H_ +#define TENSORFLOW_CORE_EXAMPLE_EXAMPLE_PARSER_CONFIGURATION_H_ + +#include +#include + +#include "tensorflow/core/example/example.pb.h" +#include "tensorflow/core/example/example_parser_configuration.pb.h" +#include "tensorflow/core/framework/allocator.h" +#include "tensorflow/core/framework/graph.pb.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/platform/types.h" +#include "tensorflow/core/public/session.h" +#include "tensorflow/core/util/example_proto_helper.h" +#include "tensorflow/core/util/sparse/sparse_tensor.h" + +// This is a set of helper methods that will make it possible to share +// tensorflow::Example proto Tensor conversion code inside the ExampleParserOp +// OpKernel as well as in external code. +namespace tensorflow { + +// Given a graph and the node_name of a ParseExample op, +// extract the FixedLenFeature/VarLenFeature configurations. +Status ExtractExampleParserConfiguration( + const tensorflow::GraphDef& graph, const string& node_name, + tensorflow::Session* session, + std::vector* fixed_len_features, + std::vector* var_len_features); + +// Given a config proto, ostensibly extracted via python, +// fill a vector of C++ structs suitable for calling +// the tensorflow.Example -> Tensor conversion code. +Status ExampleParserConfigurationProtoToFeatureVectors( + const ExampleParserConfiguration& config_proto, + std::vector* fixed_len_features, + std::vector* var_len_features); + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_EXAMPLE_EXAMPLE_PARSER_CONFIGURATION_H_ diff --git a/linux/include/tensorflow/core/example/example_parser_configuration.pb.h b/linux/include/tensorflow/core/example/example_parser_configuration.pb.h new file mode 100644 index 000000000..2b37f068c --- /dev/null +++ b/linux/include/tensorflow/core/example/example_parser_configuration.pb.h @@ -0,0 +1,1407 @@ +// Generated by the protocol buffer compiler. DO NOT EDIT! +// source: tensorflow/core/example/example_parser_configuration.proto + +#ifndef PROTOBUF_INCLUDED_tensorflow_2fcore_2fexample_2fexample_5fparser_5fconfiguration_2eproto +#define PROTOBUF_INCLUDED_tensorflow_2fcore_2fexample_2fexample_5fparser_5fconfiguration_2eproto + +#include + +#include + +#if GOOGLE_PROTOBUF_VERSION < 3006000 +#error This file was generated by a newer version of protoc which is +#error incompatible with your Protocol Buffer headers. Please update +#error your headers. +#endif +#if 3006000 < GOOGLE_PROTOBUF_MIN_PROTOC_VERSION +#error This file was generated by an older version of protoc which is +#error incompatible with your Protocol Buffer headers. Please +#error regenerate this file with a newer version of protoc. +#endif + +#include +#include +#include +#include +#include +#include +#include +#include +#include // IWYU pragma: export +#include // IWYU pragma: export +#include // IWYU pragma: export +#include +#include +#include +#include "tensorflow/core/framework/tensor_shape.pb.h" +#include "tensorflow/core/framework/tensor.pb.h" +#include "tensorflow/core/framework/types.pb.h" +// @@protoc_insertion_point(includes) +#define PROTOBUF_INTERNAL_EXPORT_protobuf_tensorflow_2fcore_2fexample_2fexample_5fparser_5fconfiguration_2eproto + +namespace protobuf_tensorflow_2fcore_2fexample_2fexample_5fparser_5fconfiguration_2eproto { +// Internal implementation detail -- do not use these members. +struct TableStruct { + static const ::google::protobuf::internal::ParseTableField entries[]; + static const ::google::protobuf::internal::AuxillaryParseTableField aux[]; + static const ::google::protobuf::internal::ParseTable schema[5]; + static const ::google::protobuf::internal::FieldMetadata field_metadata[]; + static const ::google::protobuf::internal::SerializationTable serialization_table[]; + static const ::google::protobuf::uint32 offsets[]; +}; +void AddDescriptors(); +} // namespace protobuf_tensorflow_2fcore_2fexample_2fexample_5fparser_5fconfiguration_2eproto +namespace tensorflow { +class ExampleParserConfiguration; +class ExampleParserConfigurationDefaultTypeInternal; +extern ExampleParserConfigurationDefaultTypeInternal _ExampleParserConfiguration_default_instance_; +class ExampleParserConfiguration_FeatureMapEntry_DoNotUse; +class ExampleParserConfiguration_FeatureMapEntry_DoNotUseDefaultTypeInternal; +extern ExampleParserConfiguration_FeatureMapEntry_DoNotUseDefaultTypeInternal _ExampleParserConfiguration_FeatureMapEntry_DoNotUse_default_instance_; +class FeatureConfiguration; +class FeatureConfigurationDefaultTypeInternal; +extern FeatureConfigurationDefaultTypeInternal _FeatureConfiguration_default_instance_; +class FixedLenFeatureProto; +class FixedLenFeatureProtoDefaultTypeInternal; +extern FixedLenFeatureProtoDefaultTypeInternal _FixedLenFeatureProto_default_instance_; +class VarLenFeatureProto; +class VarLenFeatureProtoDefaultTypeInternal; +extern VarLenFeatureProtoDefaultTypeInternal _VarLenFeatureProto_default_instance_; +} // namespace tensorflow +namespace google { +namespace protobuf { +template<> ::tensorflow::ExampleParserConfiguration* Arena::CreateMaybeMessage<::tensorflow::ExampleParserConfiguration>(Arena*); +template<> ::tensorflow::ExampleParserConfiguration_FeatureMapEntry_DoNotUse* Arena::CreateMaybeMessage<::tensorflow::ExampleParserConfiguration_FeatureMapEntry_DoNotUse>(Arena*); +template<> ::tensorflow::FeatureConfiguration* Arena::CreateMaybeMessage<::tensorflow::FeatureConfiguration>(Arena*); +template<> ::tensorflow::FixedLenFeatureProto* Arena::CreateMaybeMessage<::tensorflow::FixedLenFeatureProto>(Arena*); +template<> ::tensorflow::VarLenFeatureProto* Arena::CreateMaybeMessage<::tensorflow::VarLenFeatureProto>(Arena*); +} // namespace protobuf +} // namespace google +namespace tensorflow { + +// =================================================================== + +class VarLenFeatureProto : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.VarLenFeatureProto) */ { + public: + VarLenFeatureProto(); + virtual ~VarLenFeatureProto(); + + VarLenFeatureProto(const VarLenFeatureProto& from); + + inline VarLenFeatureProto& operator=(const VarLenFeatureProto& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + VarLenFeatureProto(VarLenFeatureProto&& from) noexcept + : VarLenFeatureProto() { + *this = ::std::move(from); + } + + inline VarLenFeatureProto& operator=(VarLenFeatureProto&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const VarLenFeatureProto& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const VarLenFeatureProto* internal_default_instance() { + return reinterpret_cast( + &_VarLenFeatureProto_default_instance_); + } + static constexpr int kIndexInFileMessages = + 0; + + void UnsafeArenaSwap(VarLenFeatureProto* other); + void Swap(VarLenFeatureProto* other); + friend void swap(VarLenFeatureProto& a, VarLenFeatureProto& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline VarLenFeatureProto* New() const final { + return CreateMaybeMessage(NULL); + } + + VarLenFeatureProto* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const VarLenFeatureProto& from); + void MergeFrom(const VarLenFeatureProto& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(VarLenFeatureProto* other); + protected: + explicit VarLenFeatureProto(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // string values_output_tensor_name = 2; + void clear_values_output_tensor_name(); + static const int kValuesOutputTensorNameFieldNumber = 2; + const ::std::string& values_output_tensor_name() const; + void set_values_output_tensor_name(const ::std::string& value); + #if LANG_CXX11 + void set_values_output_tensor_name(::std::string&& value); + #endif + void set_values_output_tensor_name(const char* value); + void set_values_output_tensor_name(const char* value, size_t size); + ::std::string* mutable_values_output_tensor_name(); + ::std::string* release_values_output_tensor_name(); + void set_allocated_values_output_tensor_name(::std::string* values_output_tensor_name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_values_output_tensor_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_values_output_tensor_name( + ::std::string* values_output_tensor_name); + + // string indices_output_tensor_name = 3; + void clear_indices_output_tensor_name(); + static const int kIndicesOutputTensorNameFieldNumber = 3; + const ::std::string& indices_output_tensor_name() const; + void set_indices_output_tensor_name(const ::std::string& value); + #if LANG_CXX11 + void set_indices_output_tensor_name(::std::string&& value); + #endif + void set_indices_output_tensor_name(const char* value); + void set_indices_output_tensor_name(const char* value, size_t size); + ::std::string* mutable_indices_output_tensor_name(); + ::std::string* release_indices_output_tensor_name(); + void set_allocated_indices_output_tensor_name(::std::string* indices_output_tensor_name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_indices_output_tensor_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_indices_output_tensor_name( + ::std::string* indices_output_tensor_name); + + // string shapes_output_tensor_name = 4; + void clear_shapes_output_tensor_name(); + static const int kShapesOutputTensorNameFieldNumber = 4; + const ::std::string& shapes_output_tensor_name() const; + void set_shapes_output_tensor_name(const ::std::string& value); + #if LANG_CXX11 + void set_shapes_output_tensor_name(::std::string&& value); + #endif + void set_shapes_output_tensor_name(const char* value); + void set_shapes_output_tensor_name(const char* value, size_t size); + ::std::string* mutable_shapes_output_tensor_name(); + ::std::string* release_shapes_output_tensor_name(); + void set_allocated_shapes_output_tensor_name(::std::string* shapes_output_tensor_name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_shapes_output_tensor_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_shapes_output_tensor_name( + ::std::string* shapes_output_tensor_name); + + // .tensorflow.DataType dtype = 1; + void clear_dtype(); + static const int kDtypeFieldNumber = 1; + ::tensorflow::DataType dtype() const; + void set_dtype(::tensorflow::DataType value); + + // @@protoc_insertion_point(class_scope:tensorflow.VarLenFeatureProto) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::internal::ArenaStringPtr values_output_tensor_name_; + ::google::protobuf::internal::ArenaStringPtr indices_output_tensor_name_; + ::google::protobuf::internal::ArenaStringPtr shapes_output_tensor_name_; + int dtype_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fexample_2fexample_5fparser_5fconfiguration_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class FixedLenFeatureProto : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.FixedLenFeatureProto) */ { + public: + FixedLenFeatureProto(); + virtual ~FixedLenFeatureProto(); + + FixedLenFeatureProto(const FixedLenFeatureProto& from); + + inline FixedLenFeatureProto& operator=(const FixedLenFeatureProto& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + FixedLenFeatureProto(FixedLenFeatureProto&& from) noexcept + : FixedLenFeatureProto() { + *this = ::std::move(from); + } + + inline FixedLenFeatureProto& operator=(FixedLenFeatureProto&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const FixedLenFeatureProto& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const FixedLenFeatureProto* internal_default_instance() { + return reinterpret_cast( + &_FixedLenFeatureProto_default_instance_); + } + static constexpr int kIndexInFileMessages = + 1; + + void UnsafeArenaSwap(FixedLenFeatureProto* other); + void Swap(FixedLenFeatureProto* other); + friend void swap(FixedLenFeatureProto& a, FixedLenFeatureProto& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline FixedLenFeatureProto* New() const final { + return CreateMaybeMessage(NULL); + } + + FixedLenFeatureProto* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const FixedLenFeatureProto& from); + void MergeFrom(const FixedLenFeatureProto& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(FixedLenFeatureProto* other); + protected: + explicit FixedLenFeatureProto(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // string values_output_tensor_name = 4; + void clear_values_output_tensor_name(); + static const int kValuesOutputTensorNameFieldNumber = 4; + const ::std::string& values_output_tensor_name() const; + void set_values_output_tensor_name(const ::std::string& value); + #if LANG_CXX11 + void set_values_output_tensor_name(::std::string&& value); + #endif + void set_values_output_tensor_name(const char* value); + void set_values_output_tensor_name(const char* value, size_t size); + ::std::string* mutable_values_output_tensor_name(); + ::std::string* release_values_output_tensor_name(); + void set_allocated_values_output_tensor_name(::std::string* values_output_tensor_name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_values_output_tensor_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_values_output_tensor_name( + ::std::string* values_output_tensor_name); + + // .tensorflow.TensorShapeProto shape = 2; + bool has_shape() const; + void clear_shape(); + static const int kShapeFieldNumber = 2; + private: + const ::tensorflow::TensorShapeProto& _internal_shape() const; + public: + const ::tensorflow::TensorShapeProto& shape() const; + ::tensorflow::TensorShapeProto* release_shape(); + ::tensorflow::TensorShapeProto* mutable_shape(); + void set_allocated_shape(::tensorflow::TensorShapeProto* shape); + void unsafe_arena_set_allocated_shape( + ::tensorflow::TensorShapeProto* shape); + ::tensorflow::TensorShapeProto* unsafe_arena_release_shape(); + + // .tensorflow.TensorProto default_value = 3; + bool has_default_value() const; + void clear_default_value(); + static const int kDefaultValueFieldNumber = 3; + private: + const ::tensorflow::TensorProto& _internal_default_value() const; + public: + const ::tensorflow::TensorProto& default_value() const; + ::tensorflow::TensorProto* release_default_value(); + ::tensorflow::TensorProto* mutable_default_value(); + void set_allocated_default_value(::tensorflow::TensorProto* default_value); + void unsafe_arena_set_allocated_default_value( + ::tensorflow::TensorProto* default_value); + ::tensorflow::TensorProto* unsafe_arena_release_default_value(); + + // .tensorflow.DataType dtype = 1; + void clear_dtype(); + static const int kDtypeFieldNumber = 1; + ::tensorflow::DataType dtype() const; + void set_dtype(::tensorflow::DataType value); + + // @@protoc_insertion_point(class_scope:tensorflow.FixedLenFeatureProto) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::internal::ArenaStringPtr values_output_tensor_name_; + ::tensorflow::TensorShapeProto* shape_; + ::tensorflow::TensorProto* default_value_; + int dtype_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fexample_2fexample_5fparser_5fconfiguration_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class FeatureConfiguration : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.FeatureConfiguration) */ { + public: + FeatureConfiguration(); + virtual ~FeatureConfiguration(); + + FeatureConfiguration(const FeatureConfiguration& from); + + inline FeatureConfiguration& operator=(const FeatureConfiguration& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + FeatureConfiguration(FeatureConfiguration&& from) noexcept + : FeatureConfiguration() { + *this = ::std::move(from); + } + + inline FeatureConfiguration& operator=(FeatureConfiguration&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const FeatureConfiguration& default_instance(); + + enum ConfigCase { + kFixedLenFeature = 1, + kVarLenFeature = 2, + CONFIG_NOT_SET = 0, + }; + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const FeatureConfiguration* internal_default_instance() { + return reinterpret_cast( + &_FeatureConfiguration_default_instance_); + } + static constexpr int kIndexInFileMessages = + 2; + + void UnsafeArenaSwap(FeatureConfiguration* other); + void Swap(FeatureConfiguration* other); + friend void swap(FeatureConfiguration& a, FeatureConfiguration& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline FeatureConfiguration* New() const final { + return CreateMaybeMessage(NULL); + } + + FeatureConfiguration* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const FeatureConfiguration& from); + void MergeFrom(const FeatureConfiguration& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(FeatureConfiguration* other); + protected: + explicit FeatureConfiguration(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // .tensorflow.FixedLenFeatureProto fixed_len_feature = 1; + bool has_fixed_len_feature() const; + void clear_fixed_len_feature(); + static const int kFixedLenFeatureFieldNumber = 1; + private: + const ::tensorflow::FixedLenFeatureProto& _internal_fixed_len_feature() const; + public: + const ::tensorflow::FixedLenFeatureProto& fixed_len_feature() const; + ::tensorflow::FixedLenFeatureProto* release_fixed_len_feature(); + ::tensorflow::FixedLenFeatureProto* mutable_fixed_len_feature(); + void set_allocated_fixed_len_feature(::tensorflow::FixedLenFeatureProto* fixed_len_feature); + void unsafe_arena_set_allocated_fixed_len_feature( + ::tensorflow::FixedLenFeatureProto* fixed_len_feature); + ::tensorflow::FixedLenFeatureProto* unsafe_arena_release_fixed_len_feature(); + + // .tensorflow.VarLenFeatureProto var_len_feature = 2; + bool has_var_len_feature() const; + void clear_var_len_feature(); + static const int kVarLenFeatureFieldNumber = 2; + private: + const ::tensorflow::VarLenFeatureProto& _internal_var_len_feature() const; + public: + const ::tensorflow::VarLenFeatureProto& var_len_feature() const; + ::tensorflow::VarLenFeatureProto* release_var_len_feature(); + ::tensorflow::VarLenFeatureProto* mutable_var_len_feature(); + void set_allocated_var_len_feature(::tensorflow::VarLenFeatureProto* var_len_feature); + void unsafe_arena_set_allocated_var_len_feature( + ::tensorflow::VarLenFeatureProto* var_len_feature); + ::tensorflow::VarLenFeatureProto* unsafe_arena_release_var_len_feature(); + + void clear_config(); + ConfigCase config_case() const; + // @@protoc_insertion_point(class_scope:tensorflow.FeatureConfiguration) + private: + void set_has_fixed_len_feature(); + void set_has_var_len_feature(); + + inline bool has_config() const; + inline void clear_has_config(); + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + union ConfigUnion { + ConfigUnion() {} + ::tensorflow::FixedLenFeatureProto* fixed_len_feature_; + ::tensorflow::VarLenFeatureProto* var_len_feature_; + } config_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + ::google::protobuf::uint32 _oneof_case_[1]; + + friend struct ::protobuf_tensorflow_2fcore_2fexample_2fexample_5fparser_5fconfiguration_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class ExampleParserConfiguration_FeatureMapEntry_DoNotUse : public ::google::protobuf::internal::MapEntry { +public: + typedef ::google::protobuf::internal::MapEntry SuperType; + ExampleParserConfiguration_FeatureMapEntry_DoNotUse(); + ExampleParserConfiguration_FeatureMapEntry_DoNotUse(::google::protobuf::Arena* arena); + void MergeFrom(const ExampleParserConfiguration_FeatureMapEntry_DoNotUse& other); + static const ExampleParserConfiguration_FeatureMapEntry_DoNotUse* internal_default_instance() { return reinterpret_cast(&_ExampleParserConfiguration_FeatureMapEntry_DoNotUse_default_instance_); } + void MergeFrom(const ::google::protobuf::Message& other) final; + ::google::protobuf::Metadata GetMetadata() const; +}; + +// ------------------------------------------------------------------- + +class ExampleParserConfiguration : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.ExampleParserConfiguration) */ { + public: + ExampleParserConfiguration(); + virtual ~ExampleParserConfiguration(); + + ExampleParserConfiguration(const ExampleParserConfiguration& from); + + inline ExampleParserConfiguration& operator=(const ExampleParserConfiguration& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + ExampleParserConfiguration(ExampleParserConfiguration&& from) noexcept + : ExampleParserConfiguration() { + *this = ::std::move(from); + } + + inline ExampleParserConfiguration& operator=(ExampleParserConfiguration&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const ExampleParserConfiguration& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const ExampleParserConfiguration* internal_default_instance() { + return reinterpret_cast( + &_ExampleParserConfiguration_default_instance_); + } + static constexpr int kIndexInFileMessages = + 4; + + void UnsafeArenaSwap(ExampleParserConfiguration* other); + void Swap(ExampleParserConfiguration* other); + friend void swap(ExampleParserConfiguration& a, ExampleParserConfiguration& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline ExampleParserConfiguration* New() const final { + return CreateMaybeMessage(NULL); + } + + ExampleParserConfiguration* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const ExampleParserConfiguration& from); + void MergeFrom(const ExampleParserConfiguration& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(ExampleParserConfiguration* other); + protected: + explicit ExampleParserConfiguration(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + + // accessors ------------------------------------------------------- + + // map feature_map = 1; + int feature_map_size() const; + void clear_feature_map(); + static const int kFeatureMapFieldNumber = 1; + const ::google::protobuf::Map< ::std::string, ::tensorflow::FeatureConfiguration >& + feature_map() const; + ::google::protobuf::Map< ::std::string, ::tensorflow::FeatureConfiguration >* + mutable_feature_map(); + + // @@protoc_insertion_point(class_scope:tensorflow.ExampleParserConfiguration) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::internal::MapField< + ExampleParserConfiguration_FeatureMapEntry_DoNotUse, + ::std::string, ::tensorflow::FeatureConfiguration, + ::google::protobuf::internal::WireFormatLite::TYPE_STRING, + ::google::protobuf::internal::WireFormatLite::TYPE_MESSAGE, + 0 > feature_map_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fexample_2fexample_5fparser_5fconfiguration_2eproto::TableStruct; +}; +// =================================================================== + + +// =================================================================== + +#ifdef __GNUC__ + #pragma GCC diagnostic push + #pragma GCC diagnostic ignored "-Wstrict-aliasing" +#endif // __GNUC__ +// VarLenFeatureProto + +// .tensorflow.DataType dtype = 1; +inline void VarLenFeatureProto::clear_dtype() { + dtype_ = 0; +} +inline ::tensorflow::DataType VarLenFeatureProto::dtype() const { + // @@protoc_insertion_point(field_get:tensorflow.VarLenFeatureProto.dtype) + return static_cast< ::tensorflow::DataType >(dtype_); +} +inline void VarLenFeatureProto::set_dtype(::tensorflow::DataType value) { + + dtype_ = value; + // @@protoc_insertion_point(field_set:tensorflow.VarLenFeatureProto.dtype) +} + +// string values_output_tensor_name = 2; +inline void VarLenFeatureProto::clear_values_output_tensor_name() { + values_output_tensor_name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& VarLenFeatureProto::values_output_tensor_name() const { + // @@protoc_insertion_point(field_get:tensorflow.VarLenFeatureProto.values_output_tensor_name) + return values_output_tensor_name_.Get(); +} +inline void VarLenFeatureProto::set_values_output_tensor_name(const ::std::string& value) { + + values_output_tensor_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.VarLenFeatureProto.values_output_tensor_name) +} +#if LANG_CXX11 +inline void VarLenFeatureProto::set_values_output_tensor_name(::std::string&& value) { + + values_output_tensor_name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.VarLenFeatureProto.values_output_tensor_name) +} +#endif +inline void VarLenFeatureProto::set_values_output_tensor_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + values_output_tensor_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.VarLenFeatureProto.values_output_tensor_name) +} +inline void VarLenFeatureProto::set_values_output_tensor_name(const char* value, + size_t size) { + + values_output_tensor_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.VarLenFeatureProto.values_output_tensor_name) +} +inline ::std::string* VarLenFeatureProto::mutable_values_output_tensor_name() { + + // @@protoc_insertion_point(field_mutable:tensorflow.VarLenFeatureProto.values_output_tensor_name) + return values_output_tensor_name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* VarLenFeatureProto::release_values_output_tensor_name() { + // @@protoc_insertion_point(field_release:tensorflow.VarLenFeatureProto.values_output_tensor_name) + + return values_output_tensor_name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void VarLenFeatureProto::set_allocated_values_output_tensor_name(::std::string* values_output_tensor_name) { + if (values_output_tensor_name != NULL) { + + } else { + + } + values_output_tensor_name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), values_output_tensor_name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.VarLenFeatureProto.values_output_tensor_name) +} +inline ::std::string* VarLenFeatureProto::unsafe_arena_release_values_output_tensor_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.VarLenFeatureProto.values_output_tensor_name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return values_output_tensor_name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void VarLenFeatureProto::unsafe_arena_set_allocated_values_output_tensor_name( + ::std::string* values_output_tensor_name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (values_output_tensor_name != NULL) { + + } else { + + } + values_output_tensor_name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + values_output_tensor_name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.VarLenFeatureProto.values_output_tensor_name) +} + +// string indices_output_tensor_name = 3; +inline void VarLenFeatureProto::clear_indices_output_tensor_name() { + indices_output_tensor_name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& VarLenFeatureProto::indices_output_tensor_name() const { + // @@protoc_insertion_point(field_get:tensorflow.VarLenFeatureProto.indices_output_tensor_name) + return indices_output_tensor_name_.Get(); +} +inline void VarLenFeatureProto::set_indices_output_tensor_name(const ::std::string& value) { + + indices_output_tensor_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.VarLenFeatureProto.indices_output_tensor_name) +} +#if LANG_CXX11 +inline void VarLenFeatureProto::set_indices_output_tensor_name(::std::string&& value) { + + indices_output_tensor_name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.VarLenFeatureProto.indices_output_tensor_name) +} +#endif +inline void VarLenFeatureProto::set_indices_output_tensor_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + indices_output_tensor_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.VarLenFeatureProto.indices_output_tensor_name) +} +inline void VarLenFeatureProto::set_indices_output_tensor_name(const char* value, + size_t size) { + + indices_output_tensor_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.VarLenFeatureProto.indices_output_tensor_name) +} +inline ::std::string* VarLenFeatureProto::mutable_indices_output_tensor_name() { + + // @@protoc_insertion_point(field_mutable:tensorflow.VarLenFeatureProto.indices_output_tensor_name) + return indices_output_tensor_name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* VarLenFeatureProto::release_indices_output_tensor_name() { + // @@protoc_insertion_point(field_release:tensorflow.VarLenFeatureProto.indices_output_tensor_name) + + return indices_output_tensor_name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void VarLenFeatureProto::set_allocated_indices_output_tensor_name(::std::string* indices_output_tensor_name) { + if (indices_output_tensor_name != NULL) { + + } else { + + } + indices_output_tensor_name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), indices_output_tensor_name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.VarLenFeatureProto.indices_output_tensor_name) +} +inline ::std::string* VarLenFeatureProto::unsafe_arena_release_indices_output_tensor_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.VarLenFeatureProto.indices_output_tensor_name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return indices_output_tensor_name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void VarLenFeatureProto::unsafe_arena_set_allocated_indices_output_tensor_name( + ::std::string* indices_output_tensor_name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (indices_output_tensor_name != NULL) { + + } else { + + } + indices_output_tensor_name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + indices_output_tensor_name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.VarLenFeatureProto.indices_output_tensor_name) +} + +// string shapes_output_tensor_name = 4; +inline void VarLenFeatureProto::clear_shapes_output_tensor_name() { + shapes_output_tensor_name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& VarLenFeatureProto::shapes_output_tensor_name() const { + // @@protoc_insertion_point(field_get:tensorflow.VarLenFeatureProto.shapes_output_tensor_name) + return shapes_output_tensor_name_.Get(); +} +inline void VarLenFeatureProto::set_shapes_output_tensor_name(const ::std::string& value) { + + shapes_output_tensor_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.VarLenFeatureProto.shapes_output_tensor_name) +} +#if LANG_CXX11 +inline void VarLenFeatureProto::set_shapes_output_tensor_name(::std::string&& value) { + + shapes_output_tensor_name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.VarLenFeatureProto.shapes_output_tensor_name) +} +#endif +inline void VarLenFeatureProto::set_shapes_output_tensor_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + shapes_output_tensor_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.VarLenFeatureProto.shapes_output_tensor_name) +} +inline void VarLenFeatureProto::set_shapes_output_tensor_name(const char* value, + size_t size) { + + shapes_output_tensor_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.VarLenFeatureProto.shapes_output_tensor_name) +} +inline ::std::string* VarLenFeatureProto::mutable_shapes_output_tensor_name() { + + // @@protoc_insertion_point(field_mutable:tensorflow.VarLenFeatureProto.shapes_output_tensor_name) + return shapes_output_tensor_name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* VarLenFeatureProto::release_shapes_output_tensor_name() { + // @@protoc_insertion_point(field_release:tensorflow.VarLenFeatureProto.shapes_output_tensor_name) + + return shapes_output_tensor_name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void VarLenFeatureProto::set_allocated_shapes_output_tensor_name(::std::string* shapes_output_tensor_name) { + if (shapes_output_tensor_name != NULL) { + + } else { + + } + shapes_output_tensor_name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), shapes_output_tensor_name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.VarLenFeatureProto.shapes_output_tensor_name) +} +inline ::std::string* VarLenFeatureProto::unsafe_arena_release_shapes_output_tensor_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.VarLenFeatureProto.shapes_output_tensor_name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return shapes_output_tensor_name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void VarLenFeatureProto::unsafe_arena_set_allocated_shapes_output_tensor_name( + ::std::string* shapes_output_tensor_name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (shapes_output_tensor_name != NULL) { + + } else { + + } + shapes_output_tensor_name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + shapes_output_tensor_name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.VarLenFeatureProto.shapes_output_tensor_name) +} + +// ------------------------------------------------------------------- + +// FixedLenFeatureProto + +// .tensorflow.DataType dtype = 1; +inline void FixedLenFeatureProto::clear_dtype() { + dtype_ = 0; +} +inline ::tensorflow::DataType FixedLenFeatureProto::dtype() const { + // @@protoc_insertion_point(field_get:tensorflow.FixedLenFeatureProto.dtype) + return static_cast< ::tensorflow::DataType >(dtype_); +} +inline void FixedLenFeatureProto::set_dtype(::tensorflow::DataType value) { + + dtype_ = value; + // @@protoc_insertion_point(field_set:tensorflow.FixedLenFeatureProto.dtype) +} + +// .tensorflow.TensorShapeProto shape = 2; +inline bool FixedLenFeatureProto::has_shape() const { + return this != internal_default_instance() && shape_ != NULL; +} +inline const ::tensorflow::TensorShapeProto& FixedLenFeatureProto::_internal_shape() const { + return *shape_; +} +inline const ::tensorflow::TensorShapeProto& FixedLenFeatureProto::shape() const { + const ::tensorflow::TensorShapeProto* p = shape_; + // @@protoc_insertion_point(field_get:tensorflow.FixedLenFeatureProto.shape) + return p != NULL ? *p : *reinterpret_cast( + &::tensorflow::_TensorShapeProto_default_instance_); +} +inline ::tensorflow::TensorShapeProto* FixedLenFeatureProto::release_shape() { + // @@protoc_insertion_point(field_release:tensorflow.FixedLenFeatureProto.shape) + + ::tensorflow::TensorShapeProto* temp = shape_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + shape_ = NULL; + return temp; +} +inline ::tensorflow::TensorShapeProto* FixedLenFeatureProto::unsafe_arena_release_shape() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.FixedLenFeatureProto.shape) + + ::tensorflow::TensorShapeProto* temp = shape_; + shape_ = NULL; + return temp; +} +inline ::tensorflow::TensorShapeProto* FixedLenFeatureProto::mutable_shape() { + + if (shape_ == NULL) { + auto* p = CreateMaybeMessage<::tensorflow::TensorShapeProto>(GetArenaNoVirtual()); + shape_ = p; + } + // @@protoc_insertion_point(field_mutable:tensorflow.FixedLenFeatureProto.shape) + return shape_; +} +inline void FixedLenFeatureProto::set_allocated_shape(::tensorflow::TensorShapeProto* shape) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(shape_); + } + if (shape) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(shape)->GetArena(); + if (message_arena != submessage_arena) { + shape = ::google::protobuf::internal::GetOwnedMessage( + message_arena, shape, submessage_arena); + } + + } else { + + } + shape_ = shape; + // @@protoc_insertion_point(field_set_allocated:tensorflow.FixedLenFeatureProto.shape) +} + +// .tensorflow.TensorProto default_value = 3; +inline bool FixedLenFeatureProto::has_default_value() const { + return this != internal_default_instance() && default_value_ != NULL; +} +inline const ::tensorflow::TensorProto& FixedLenFeatureProto::_internal_default_value() const { + return *default_value_; +} +inline const ::tensorflow::TensorProto& FixedLenFeatureProto::default_value() const { + const ::tensorflow::TensorProto* p = default_value_; + // @@protoc_insertion_point(field_get:tensorflow.FixedLenFeatureProto.default_value) + return p != NULL ? *p : *reinterpret_cast( + &::tensorflow::_TensorProto_default_instance_); +} +inline ::tensorflow::TensorProto* FixedLenFeatureProto::release_default_value() { + // @@protoc_insertion_point(field_release:tensorflow.FixedLenFeatureProto.default_value) + + ::tensorflow::TensorProto* temp = default_value_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + default_value_ = NULL; + return temp; +} +inline ::tensorflow::TensorProto* FixedLenFeatureProto::unsafe_arena_release_default_value() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.FixedLenFeatureProto.default_value) + + ::tensorflow::TensorProto* temp = default_value_; + default_value_ = NULL; + return temp; +} +inline ::tensorflow::TensorProto* FixedLenFeatureProto::mutable_default_value() { + + if (default_value_ == NULL) { + auto* p = CreateMaybeMessage<::tensorflow::TensorProto>(GetArenaNoVirtual()); + default_value_ = p; + } + // @@protoc_insertion_point(field_mutable:tensorflow.FixedLenFeatureProto.default_value) + return default_value_; +} +inline void FixedLenFeatureProto::set_allocated_default_value(::tensorflow::TensorProto* default_value) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(default_value_); + } + if (default_value) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(default_value)->GetArena(); + if (message_arena != submessage_arena) { + default_value = ::google::protobuf::internal::GetOwnedMessage( + message_arena, default_value, submessage_arena); + } + + } else { + + } + default_value_ = default_value; + // @@protoc_insertion_point(field_set_allocated:tensorflow.FixedLenFeatureProto.default_value) +} + +// string values_output_tensor_name = 4; +inline void FixedLenFeatureProto::clear_values_output_tensor_name() { + values_output_tensor_name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& FixedLenFeatureProto::values_output_tensor_name() const { + // @@protoc_insertion_point(field_get:tensorflow.FixedLenFeatureProto.values_output_tensor_name) + return values_output_tensor_name_.Get(); +} +inline void FixedLenFeatureProto::set_values_output_tensor_name(const ::std::string& value) { + + values_output_tensor_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.FixedLenFeatureProto.values_output_tensor_name) +} +#if LANG_CXX11 +inline void FixedLenFeatureProto::set_values_output_tensor_name(::std::string&& value) { + + values_output_tensor_name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.FixedLenFeatureProto.values_output_tensor_name) +} +#endif +inline void FixedLenFeatureProto::set_values_output_tensor_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + values_output_tensor_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.FixedLenFeatureProto.values_output_tensor_name) +} +inline void FixedLenFeatureProto::set_values_output_tensor_name(const char* value, + size_t size) { + + values_output_tensor_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.FixedLenFeatureProto.values_output_tensor_name) +} +inline ::std::string* FixedLenFeatureProto::mutable_values_output_tensor_name() { + + // @@protoc_insertion_point(field_mutable:tensorflow.FixedLenFeatureProto.values_output_tensor_name) + return values_output_tensor_name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* FixedLenFeatureProto::release_values_output_tensor_name() { + // @@protoc_insertion_point(field_release:tensorflow.FixedLenFeatureProto.values_output_tensor_name) + + return values_output_tensor_name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void FixedLenFeatureProto::set_allocated_values_output_tensor_name(::std::string* values_output_tensor_name) { + if (values_output_tensor_name != NULL) { + + } else { + + } + values_output_tensor_name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), values_output_tensor_name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.FixedLenFeatureProto.values_output_tensor_name) +} +inline ::std::string* FixedLenFeatureProto::unsafe_arena_release_values_output_tensor_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.FixedLenFeatureProto.values_output_tensor_name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return values_output_tensor_name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void FixedLenFeatureProto::unsafe_arena_set_allocated_values_output_tensor_name( + ::std::string* values_output_tensor_name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (values_output_tensor_name != NULL) { + + } else { + + } + values_output_tensor_name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + values_output_tensor_name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.FixedLenFeatureProto.values_output_tensor_name) +} + +// ------------------------------------------------------------------- + +// FeatureConfiguration + +// .tensorflow.FixedLenFeatureProto fixed_len_feature = 1; +inline bool FeatureConfiguration::has_fixed_len_feature() const { + return config_case() == kFixedLenFeature; +} +inline void FeatureConfiguration::set_has_fixed_len_feature() { + _oneof_case_[0] = kFixedLenFeature; +} +inline void FeatureConfiguration::clear_fixed_len_feature() { + if (has_fixed_len_feature()) { + if (GetArenaNoVirtual() == NULL) { + delete config_.fixed_len_feature_; + } + clear_has_config(); + } +} +inline const ::tensorflow::FixedLenFeatureProto& FeatureConfiguration::_internal_fixed_len_feature() const { + return *config_.fixed_len_feature_; +} +inline ::tensorflow::FixedLenFeatureProto* FeatureConfiguration::release_fixed_len_feature() { + // @@protoc_insertion_point(field_release:tensorflow.FeatureConfiguration.fixed_len_feature) + if (has_fixed_len_feature()) { + clear_has_config(); + ::tensorflow::FixedLenFeatureProto* temp = config_.fixed_len_feature_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + config_.fixed_len_feature_ = NULL; + return temp; + } else { + return NULL; + } +} +inline const ::tensorflow::FixedLenFeatureProto& FeatureConfiguration::fixed_len_feature() const { + // @@protoc_insertion_point(field_get:tensorflow.FeatureConfiguration.fixed_len_feature) + return has_fixed_len_feature() + ? *config_.fixed_len_feature_ + : *reinterpret_cast< ::tensorflow::FixedLenFeatureProto*>(&::tensorflow::_FixedLenFeatureProto_default_instance_); +} +inline ::tensorflow::FixedLenFeatureProto* FeatureConfiguration::unsafe_arena_release_fixed_len_feature() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.FeatureConfiguration.fixed_len_feature) + if (has_fixed_len_feature()) { + clear_has_config(); + ::tensorflow::FixedLenFeatureProto* temp = config_.fixed_len_feature_; + config_.fixed_len_feature_ = NULL; + return temp; + } else { + return NULL; + } +} +inline void FeatureConfiguration::unsafe_arena_set_allocated_fixed_len_feature(::tensorflow::FixedLenFeatureProto* fixed_len_feature) { + clear_config(); + if (fixed_len_feature) { + set_has_fixed_len_feature(); + config_.fixed_len_feature_ = fixed_len_feature; + } + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.FeatureConfiguration.fixed_len_feature) +} +inline ::tensorflow::FixedLenFeatureProto* FeatureConfiguration::mutable_fixed_len_feature() { + if (!has_fixed_len_feature()) { + clear_config(); + set_has_fixed_len_feature(); + config_.fixed_len_feature_ = CreateMaybeMessage< ::tensorflow::FixedLenFeatureProto >( + GetArenaNoVirtual()); + } + // @@protoc_insertion_point(field_mutable:tensorflow.FeatureConfiguration.fixed_len_feature) + return config_.fixed_len_feature_; +} + +// .tensorflow.VarLenFeatureProto var_len_feature = 2; +inline bool FeatureConfiguration::has_var_len_feature() const { + return config_case() == kVarLenFeature; +} +inline void FeatureConfiguration::set_has_var_len_feature() { + _oneof_case_[0] = kVarLenFeature; +} +inline void FeatureConfiguration::clear_var_len_feature() { + if (has_var_len_feature()) { + if (GetArenaNoVirtual() == NULL) { + delete config_.var_len_feature_; + } + clear_has_config(); + } +} +inline const ::tensorflow::VarLenFeatureProto& FeatureConfiguration::_internal_var_len_feature() const { + return *config_.var_len_feature_; +} +inline ::tensorflow::VarLenFeatureProto* FeatureConfiguration::release_var_len_feature() { + // @@protoc_insertion_point(field_release:tensorflow.FeatureConfiguration.var_len_feature) + if (has_var_len_feature()) { + clear_has_config(); + ::tensorflow::VarLenFeatureProto* temp = config_.var_len_feature_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + config_.var_len_feature_ = NULL; + return temp; + } else { + return NULL; + } +} +inline const ::tensorflow::VarLenFeatureProto& FeatureConfiguration::var_len_feature() const { + // @@protoc_insertion_point(field_get:tensorflow.FeatureConfiguration.var_len_feature) + return has_var_len_feature() + ? *config_.var_len_feature_ + : *reinterpret_cast< ::tensorflow::VarLenFeatureProto*>(&::tensorflow::_VarLenFeatureProto_default_instance_); +} +inline ::tensorflow::VarLenFeatureProto* FeatureConfiguration::unsafe_arena_release_var_len_feature() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.FeatureConfiguration.var_len_feature) + if (has_var_len_feature()) { + clear_has_config(); + ::tensorflow::VarLenFeatureProto* temp = config_.var_len_feature_; + config_.var_len_feature_ = NULL; + return temp; + } else { + return NULL; + } +} +inline void FeatureConfiguration::unsafe_arena_set_allocated_var_len_feature(::tensorflow::VarLenFeatureProto* var_len_feature) { + clear_config(); + if (var_len_feature) { + set_has_var_len_feature(); + config_.var_len_feature_ = var_len_feature; + } + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.FeatureConfiguration.var_len_feature) +} +inline ::tensorflow::VarLenFeatureProto* FeatureConfiguration::mutable_var_len_feature() { + if (!has_var_len_feature()) { + clear_config(); + set_has_var_len_feature(); + config_.var_len_feature_ = CreateMaybeMessage< ::tensorflow::VarLenFeatureProto >( + GetArenaNoVirtual()); + } + // @@protoc_insertion_point(field_mutable:tensorflow.FeatureConfiguration.var_len_feature) + return config_.var_len_feature_; +} + +inline bool FeatureConfiguration::has_config() const { + return config_case() != CONFIG_NOT_SET; +} +inline void FeatureConfiguration::clear_has_config() { + _oneof_case_[0] = CONFIG_NOT_SET; +} +inline FeatureConfiguration::ConfigCase FeatureConfiguration::config_case() const { + return FeatureConfiguration::ConfigCase(_oneof_case_[0]); +} +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ExampleParserConfiguration + +// map feature_map = 1; +inline int ExampleParserConfiguration::feature_map_size() const { + return feature_map_.size(); +} +inline void ExampleParserConfiguration::clear_feature_map() { + feature_map_.Clear(); +} +inline const ::google::protobuf::Map< ::std::string, ::tensorflow::FeatureConfiguration >& +ExampleParserConfiguration::feature_map() const { + // @@protoc_insertion_point(field_map:tensorflow.ExampleParserConfiguration.feature_map) + return feature_map_.GetMap(); +} +inline ::google::protobuf::Map< ::std::string, ::tensorflow::FeatureConfiguration >* +ExampleParserConfiguration::mutable_feature_map() { + // @@protoc_insertion_point(field_mutable_map:tensorflow.ExampleParserConfiguration.feature_map) + return feature_map_.MutableMap(); +} + +#ifdef __GNUC__ + #pragma GCC diagnostic pop +#endif // __GNUC__ +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + + +// @@protoc_insertion_point(namespace_scope) + +} // namespace tensorflow + +// @@protoc_insertion_point(global_scope) + +#endif // PROTOBUF_INCLUDED_tensorflow_2fcore_2fexample_2fexample_5fparser_5fconfiguration_2eproto diff --git a/linux/include/tensorflow/core/example/feature.pb.h b/linux/include/tensorflow/core/example/feature.pb.h new file mode 100644 index 000000000..d471d0b35 --- /dev/null +++ b/linux/include/tensorflow/core/example/feature.pb.h @@ -0,0 +1,1551 @@ +// Generated by the protocol buffer compiler. DO NOT EDIT! +// source: tensorflow/core/example/feature.proto + +#ifndef PROTOBUF_INCLUDED_tensorflow_2fcore_2fexample_2ffeature_2eproto +#define PROTOBUF_INCLUDED_tensorflow_2fcore_2fexample_2ffeature_2eproto + +#include + +#include + +#if GOOGLE_PROTOBUF_VERSION < 3006000 +#error This file was generated by a newer version of protoc which is +#error incompatible with your Protocol Buffer headers. Please update +#error your headers. +#endif +#if 3006000 < GOOGLE_PROTOBUF_MIN_PROTOC_VERSION +#error This file was generated by an older version of protoc which is +#error incompatible with your Protocol Buffer headers. Please +#error regenerate this file with a newer version of protoc. +#endif + +#include +#include +#include +#include +#include +#include +#include +#include +#include // IWYU pragma: export +#include // IWYU pragma: export +#include // IWYU pragma: export +#include +#include +#include +// @@protoc_insertion_point(includes) +#define PROTOBUF_INTERNAL_EXPORT_protobuf_tensorflow_2fcore_2fexample_2ffeature_2eproto + +namespace protobuf_tensorflow_2fcore_2fexample_2ffeature_2eproto { +// Internal implementation detail -- do not use these members. +struct TableStruct { + static const ::google::protobuf::internal::ParseTableField entries[]; + static const ::google::protobuf::internal::AuxillaryParseTableField aux[]; + static const ::google::protobuf::internal::ParseTable schema[9]; + static const ::google::protobuf::internal::FieldMetadata field_metadata[]; + static const ::google::protobuf::internal::SerializationTable serialization_table[]; + static const ::google::protobuf::uint32 offsets[]; +}; +void AddDescriptors(); +} // namespace protobuf_tensorflow_2fcore_2fexample_2ffeature_2eproto +namespace tensorflow { +class BytesList; +class BytesListDefaultTypeInternal; +extern BytesListDefaultTypeInternal _BytesList_default_instance_; +class Feature; +class FeatureDefaultTypeInternal; +extern FeatureDefaultTypeInternal _Feature_default_instance_; +class FeatureList; +class FeatureListDefaultTypeInternal; +extern FeatureListDefaultTypeInternal _FeatureList_default_instance_; +class FeatureLists; +class FeatureListsDefaultTypeInternal; +extern FeatureListsDefaultTypeInternal _FeatureLists_default_instance_; +class FeatureLists_FeatureListEntry_DoNotUse; +class FeatureLists_FeatureListEntry_DoNotUseDefaultTypeInternal; +extern FeatureLists_FeatureListEntry_DoNotUseDefaultTypeInternal _FeatureLists_FeatureListEntry_DoNotUse_default_instance_; +class Features; +class FeaturesDefaultTypeInternal; +extern FeaturesDefaultTypeInternal _Features_default_instance_; +class Features_FeatureEntry_DoNotUse; +class Features_FeatureEntry_DoNotUseDefaultTypeInternal; +extern Features_FeatureEntry_DoNotUseDefaultTypeInternal _Features_FeatureEntry_DoNotUse_default_instance_; +class FloatList; +class FloatListDefaultTypeInternal; +extern FloatListDefaultTypeInternal _FloatList_default_instance_; +class Int64List; +class Int64ListDefaultTypeInternal; +extern Int64ListDefaultTypeInternal _Int64List_default_instance_; +} // namespace tensorflow +namespace google { +namespace protobuf { +template<> ::tensorflow::BytesList* Arena::CreateMaybeMessage<::tensorflow::BytesList>(Arena*); +template<> ::tensorflow::Feature* Arena::CreateMaybeMessage<::tensorflow::Feature>(Arena*); +template<> ::tensorflow::FeatureList* Arena::CreateMaybeMessage<::tensorflow::FeatureList>(Arena*); +template<> ::tensorflow::FeatureLists* Arena::CreateMaybeMessage<::tensorflow::FeatureLists>(Arena*); +template<> ::tensorflow::FeatureLists_FeatureListEntry_DoNotUse* Arena::CreateMaybeMessage<::tensorflow::FeatureLists_FeatureListEntry_DoNotUse>(Arena*); +template<> ::tensorflow::Features* Arena::CreateMaybeMessage<::tensorflow::Features>(Arena*); +template<> ::tensorflow::Features_FeatureEntry_DoNotUse* Arena::CreateMaybeMessage<::tensorflow::Features_FeatureEntry_DoNotUse>(Arena*); +template<> ::tensorflow::FloatList* Arena::CreateMaybeMessage<::tensorflow::FloatList>(Arena*); +template<> ::tensorflow::Int64List* Arena::CreateMaybeMessage<::tensorflow::Int64List>(Arena*); +} // namespace protobuf +} // namespace google +namespace tensorflow { + +// =================================================================== + +class BytesList : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.BytesList) */ { + public: + BytesList(); + virtual ~BytesList(); + + BytesList(const BytesList& from); + + inline BytesList& operator=(const BytesList& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + BytesList(BytesList&& from) noexcept + : BytesList() { + *this = ::std::move(from); + } + + inline BytesList& operator=(BytesList&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const BytesList& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const BytesList* internal_default_instance() { + return reinterpret_cast( + &_BytesList_default_instance_); + } + static constexpr int kIndexInFileMessages = + 0; + + void UnsafeArenaSwap(BytesList* other); + void Swap(BytesList* other); + friend void swap(BytesList& a, BytesList& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline BytesList* New() const final { + return CreateMaybeMessage(NULL); + } + + BytesList* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const BytesList& from); + void MergeFrom(const BytesList& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(BytesList* other); + protected: + explicit BytesList(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated bytes value = 1; + int value_size() const; + void clear_value(); + static const int kValueFieldNumber = 1; + const ::std::string& value(int index) const; + ::std::string* mutable_value(int index); + void set_value(int index, const ::std::string& value); + #if LANG_CXX11 + void set_value(int index, ::std::string&& value); + #endif + void set_value(int index, const char* value); + void set_value(int index, const void* value, size_t size); + ::std::string* add_value(); + void add_value(const ::std::string& value); + #if LANG_CXX11 + void add_value(::std::string&& value); + #endif + void add_value(const char* value); + void add_value(const void* value, size_t size); + const ::google::protobuf::RepeatedPtrField< ::std::string>& value() const; + ::google::protobuf::RepeatedPtrField< ::std::string>* mutable_value(); + + // @@protoc_insertion_point(class_scope:tensorflow.BytesList) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedPtrField< ::std::string> value_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fexample_2ffeature_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class FloatList : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.FloatList) */ { + public: + FloatList(); + virtual ~FloatList(); + + FloatList(const FloatList& from); + + inline FloatList& operator=(const FloatList& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + FloatList(FloatList&& from) noexcept + : FloatList() { + *this = ::std::move(from); + } + + inline FloatList& operator=(FloatList&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const FloatList& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const FloatList* internal_default_instance() { + return reinterpret_cast( + &_FloatList_default_instance_); + } + static constexpr int kIndexInFileMessages = + 1; + + void UnsafeArenaSwap(FloatList* other); + void Swap(FloatList* other); + friend void swap(FloatList& a, FloatList& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline FloatList* New() const final { + return CreateMaybeMessage(NULL); + } + + FloatList* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const FloatList& from); + void MergeFrom(const FloatList& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(FloatList* other); + protected: + explicit FloatList(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated float value = 1 [packed = true]; + int value_size() const; + void clear_value(); + static const int kValueFieldNumber = 1; + float value(int index) const; + void set_value(int index, float value); + void add_value(float value); + const ::google::protobuf::RepeatedField< float >& + value() const; + ::google::protobuf::RepeatedField< float >* + mutable_value(); + + // @@protoc_insertion_point(class_scope:tensorflow.FloatList) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedField< float > value_; + mutable int _value_cached_byte_size_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fexample_2ffeature_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class Int64List : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.Int64List) */ { + public: + Int64List(); + virtual ~Int64List(); + + Int64List(const Int64List& from); + + inline Int64List& operator=(const Int64List& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + Int64List(Int64List&& from) noexcept + : Int64List() { + *this = ::std::move(from); + } + + inline Int64List& operator=(Int64List&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const Int64List& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const Int64List* internal_default_instance() { + return reinterpret_cast( + &_Int64List_default_instance_); + } + static constexpr int kIndexInFileMessages = + 2; + + void UnsafeArenaSwap(Int64List* other); + void Swap(Int64List* other); + friend void swap(Int64List& a, Int64List& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline Int64List* New() const final { + return CreateMaybeMessage(NULL); + } + + Int64List* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const Int64List& from); + void MergeFrom(const Int64List& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(Int64List* other); + protected: + explicit Int64List(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated int64 value = 1 [packed = true]; + int value_size() const; + void clear_value(); + static const int kValueFieldNumber = 1; + ::google::protobuf::int64 value(int index) const; + void set_value(int index, ::google::protobuf::int64 value); + void add_value(::google::protobuf::int64 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& + value() const; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* + mutable_value(); + + // @@protoc_insertion_point(class_scope:tensorflow.Int64List) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 > value_; + mutable int _value_cached_byte_size_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fexample_2ffeature_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class Feature : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.Feature) */ { + public: + Feature(); + virtual ~Feature(); + + Feature(const Feature& from); + + inline Feature& operator=(const Feature& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + Feature(Feature&& from) noexcept + : Feature() { + *this = ::std::move(from); + } + + inline Feature& operator=(Feature&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const Feature& default_instance(); + + enum KindCase { + kBytesList = 1, + kFloatList = 2, + kInt64List = 3, + KIND_NOT_SET = 0, + }; + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const Feature* internal_default_instance() { + return reinterpret_cast( + &_Feature_default_instance_); + } + static constexpr int kIndexInFileMessages = + 3; + + void UnsafeArenaSwap(Feature* other); + void Swap(Feature* other); + friend void swap(Feature& a, Feature& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline Feature* New() const final { + return CreateMaybeMessage(NULL); + } + + Feature* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const Feature& from); + void MergeFrom(const Feature& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(Feature* other); + protected: + explicit Feature(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // .tensorflow.BytesList bytes_list = 1; + bool has_bytes_list() const; + void clear_bytes_list(); + static const int kBytesListFieldNumber = 1; + private: + const ::tensorflow::BytesList& _internal_bytes_list() const; + public: + const ::tensorflow::BytesList& bytes_list() const; + ::tensorflow::BytesList* release_bytes_list(); + ::tensorflow::BytesList* mutable_bytes_list(); + void set_allocated_bytes_list(::tensorflow::BytesList* bytes_list); + void unsafe_arena_set_allocated_bytes_list( + ::tensorflow::BytesList* bytes_list); + ::tensorflow::BytesList* unsafe_arena_release_bytes_list(); + + // .tensorflow.FloatList float_list = 2; + bool has_float_list() const; + void clear_float_list(); + static const int kFloatListFieldNumber = 2; + private: + const ::tensorflow::FloatList& _internal_float_list() const; + public: + const ::tensorflow::FloatList& float_list() const; + ::tensorflow::FloatList* release_float_list(); + ::tensorflow::FloatList* mutable_float_list(); + void set_allocated_float_list(::tensorflow::FloatList* float_list); + void unsafe_arena_set_allocated_float_list( + ::tensorflow::FloatList* float_list); + ::tensorflow::FloatList* unsafe_arena_release_float_list(); + + // .tensorflow.Int64List int64_list = 3; + bool has_int64_list() const; + void clear_int64_list(); + static const int kInt64ListFieldNumber = 3; + private: + const ::tensorflow::Int64List& _internal_int64_list() const; + public: + const ::tensorflow::Int64List& int64_list() const; + ::tensorflow::Int64List* release_int64_list(); + ::tensorflow::Int64List* mutable_int64_list(); + void set_allocated_int64_list(::tensorflow::Int64List* int64_list); + void unsafe_arena_set_allocated_int64_list( + ::tensorflow::Int64List* int64_list); + ::tensorflow::Int64List* unsafe_arena_release_int64_list(); + + void clear_kind(); + KindCase kind_case() const; + // @@protoc_insertion_point(class_scope:tensorflow.Feature) + private: + void set_has_bytes_list(); + void set_has_float_list(); + void set_has_int64_list(); + + inline bool has_kind() const; + inline void clear_has_kind(); + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + union KindUnion { + KindUnion() {} + ::tensorflow::BytesList* bytes_list_; + ::tensorflow::FloatList* float_list_; + ::tensorflow::Int64List* int64_list_; + } kind_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + ::google::protobuf::uint32 _oneof_case_[1]; + + friend struct ::protobuf_tensorflow_2fcore_2fexample_2ffeature_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class Features_FeatureEntry_DoNotUse : public ::google::protobuf::internal::MapEntry { +public: + typedef ::google::protobuf::internal::MapEntry SuperType; + Features_FeatureEntry_DoNotUse(); + Features_FeatureEntry_DoNotUse(::google::protobuf::Arena* arena); + void MergeFrom(const Features_FeatureEntry_DoNotUse& other); + static const Features_FeatureEntry_DoNotUse* internal_default_instance() { return reinterpret_cast(&_Features_FeatureEntry_DoNotUse_default_instance_); } + void MergeFrom(const ::google::protobuf::Message& other) final; + ::google::protobuf::Metadata GetMetadata() const; +}; + +// ------------------------------------------------------------------- + +class Features : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.Features) */ { + public: + Features(); + virtual ~Features(); + + Features(const Features& from); + + inline Features& operator=(const Features& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + Features(Features&& from) noexcept + : Features() { + *this = ::std::move(from); + } + + inline Features& operator=(Features&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const Features& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const Features* internal_default_instance() { + return reinterpret_cast( + &_Features_default_instance_); + } + static constexpr int kIndexInFileMessages = + 5; + + void UnsafeArenaSwap(Features* other); + void Swap(Features* other); + friend void swap(Features& a, Features& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline Features* New() const final { + return CreateMaybeMessage(NULL); + } + + Features* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const Features& from); + void MergeFrom(const Features& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(Features* other); + protected: + explicit Features(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + + // accessors ------------------------------------------------------- + + // map feature = 1; + int feature_size() const; + void clear_feature(); + static const int kFeatureFieldNumber = 1; + const ::google::protobuf::Map< ::std::string, ::tensorflow::Feature >& + feature() const; + ::google::protobuf::Map< ::std::string, ::tensorflow::Feature >* + mutable_feature(); + + // @@protoc_insertion_point(class_scope:tensorflow.Features) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::internal::MapField< + Features_FeatureEntry_DoNotUse, + ::std::string, ::tensorflow::Feature, + ::google::protobuf::internal::WireFormatLite::TYPE_STRING, + ::google::protobuf::internal::WireFormatLite::TYPE_MESSAGE, + 0 > feature_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fexample_2ffeature_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class FeatureList : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.FeatureList) */ { + public: + FeatureList(); + virtual ~FeatureList(); + + FeatureList(const FeatureList& from); + + inline FeatureList& operator=(const FeatureList& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + FeatureList(FeatureList&& from) noexcept + : FeatureList() { + *this = ::std::move(from); + } + + inline FeatureList& operator=(FeatureList&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const FeatureList& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const FeatureList* internal_default_instance() { + return reinterpret_cast( + &_FeatureList_default_instance_); + } + static constexpr int kIndexInFileMessages = + 6; + + void UnsafeArenaSwap(FeatureList* other); + void Swap(FeatureList* other); + friend void swap(FeatureList& a, FeatureList& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline FeatureList* New() const final { + return CreateMaybeMessage(NULL); + } + + FeatureList* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const FeatureList& from); + void MergeFrom(const FeatureList& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(FeatureList* other); + protected: + explicit FeatureList(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated .tensorflow.Feature feature = 1; + int feature_size() const; + void clear_feature(); + static const int kFeatureFieldNumber = 1; + ::tensorflow::Feature* mutable_feature(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::Feature >* + mutable_feature(); + const ::tensorflow::Feature& feature(int index) const; + ::tensorflow::Feature* add_feature(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::Feature >& + feature() const; + + // @@protoc_insertion_point(class_scope:tensorflow.FeatureList) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::Feature > feature_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fexample_2ffeature_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class FeatureLists_FeatureListEntry_DoNotUse : public ::google::protobuf::internal::MapEntry { +public: + typedef ::google::protobuf::internal::MapEntry SuperType; + FeatureLists_FeatureListEntry_DoNotUse(); + FeatureLists_FeatureListEntry_DoNotUse(::google::protobuf::Arena* arena); + void MergeFrom(const FeatureLists_FeatureListEntry_DoNotUse& other); + static const FeatureLists_FeatureListEntry_DoNotUse* internal_default_instance() { return reinterpret_cast(&_FeatureLists_FeatureListEntry_DoNotUse_default_instance_); } + void MergeFrom(const ::google::protobuf::Message& other) final; + ::google::protobuf::Metadata GetMetadata() const; +}; + +// ------------------------------------------------------------------- + +class FeatureLists : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.FeatureLists) */ { + public: + FeatureLists(); + virtual ~FeatureLists(); + + FeatureLists(const FeatureLists& from); + + inline FeatureLists& operator=(const FeatureLists& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + FeatureLists(FeatureLists&& from) noexcept + : FeatureLists() { + *this = ::std::move(from); + } + + inline FeatureLists& operator=(FeatureLists&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const FeatureLists& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const FeatureLists* internal_default_instance() { + return reinterpret_cast( + &_FeatureLists_default_instance_); + } + static constexpr int kIndexInFileMessages = + 8; + + void UnsafeArenaSwap(FeatureLists* other); + void Swap(FeatureLists* other); + friend void swap(FeatureLists& a, FeatureLists& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline FeatureLists* New() const final { + return CreateMaybeMessage(NULL); + } + + FeatureLists* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const FeatureLists& from); + void MergeFrom(const FeatureLists& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(FeatureLists* other); + protected: + explicit FeatureLists(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + + // accessors ------------------------------------------------------- + + // map feature_list = 1; + int feature_list_size() const; + void clear_feature_list(); + static const int kFeatureListFieldNumber = 1; + const ::google::protobuf::Map< ::std::string, ::tensorflow::FeatureList >& + feature_list() const; + ::google::protobuf::Map< ::std::string, ::tensorflow::FeatureList >* + mutable_feature_list(); + + // @@protoc_insertion_point(class_scope:tensorflow.FeatureLists) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::internal::MapField< + FeatureLists_FeatureListEntry_DoNotUse, + ::std::string, ::tensorflow::FeatureList, + ::google::protobuf::internal::WireFormatLite::TYPE_STRING, + ::google::protobuf::internal::WireFormatLite::TYPE_MESSAGE, + 0 > feature_list_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fexample_2ffeature_2eproto::TableStruct; +}; +// =================================================================== + + +// =================================================================== + +#ifdef __GNUC__ + #pragma GCC diagnostic push + #pragma GCC diagnostic ignored "-Wstrict-aliasing" +#endif // __GNUC__ +// BytesList + +// repeated bytes value = 1; +inline int BytesList::value_size() const { + return value_.size(); +} +inline void BytesList::clear_value() { + value_.Clear(); +} +inline const ::std::string& BytesList::value(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.BytesList.value) + return value_.Get(index); +} +inline ::std::string* BytesList::mutable_value(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.BytesList.value) + return value_.Mutable(index); +} +inline void BytesList::set_value(int index, const ::std::string& value) { + // @@protoc_insertion_point(field_set:tensorflow.BytesList.value) + value_.Mutable(index)->assign(value); +} +#if LANG_CXX11 +inline void BytesList::set_value(int index, ::std::string&& value) { + // @@protoc_insertion_point(field_set:tensorflow.BytesList.value) + value_.Mutable(index)->assign(std::move(value)); +} +#endif +inline void BytesList::set_value(int index, const char* value) { + GOOGLE_DCHECK(value != NULL); + value_.Mutable(index)->assign(value); + // @@protoc_insertion_point(field_set_char:tensorflow.BytesList.value) +} +inline void BytesList::set_value(int index, const void* value, size_t size) { + value_.Mutable(index)->assign( + reinterpret_cast(value), size); + // @@protoc_insertion_point(field_set_pointer:tensorflow.BytesList.value) +} +inline ::std::string* BytesList::add_value() { + // @@protoc_insertion_point(field_add_mutable:tensorflow.BytesList.value) + return value_.Add(); +} +inline void BytesList::add_value(const ::std::string& value) { + value_.Add()->assign(value); + // @@protoc_insertion_point(field_add:tensorflow.BytesList.value) +} +#if LANG_CXX11 +inline void BytesList::add_value(::std::string&& value) { + value_.Add(std::move(value)); + // @@protoc_insertion_point(field_add:tensorflow.BytesList.value) +} +#endif +inline void BytesList::add_value(const char* value) { + GOOGLE_DCHECK(value != NULL); + value_.Add()->assign(value); + // @@protoc_insertion_point(field_add_char:tensorflow.BytesList.value) +} +inline void BytesList::add_value(const void* value, size_t size) { + value_.Add()->assign(reinterpret_cast(value), size); + // @@protoc_insertion_point(field_add_pointer:tensorflow.BytesList.value) +} +inline const ::google::protobuf::RepeatedPtrField< ::std::string>& +BytesList::value() const { + // @@protoc_insertion_point(field_list:tensorflow.BytesList.value) + return value_; +} +inline ::google::protobuf::RepeatedPtrField< ::std::string>* +BytesList::mutable_value() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.BytesList.value) + return &value_; +} + +// ------------------------------------------------------------------- + +// FloatList + +// repeated float value = 1 [packed = true]; +inline int FloatList::value_size() const { + return value_.size(); +} +inline void FloatList::clear_value() { + value_.Clear(); +} +inline float FloatList::value(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.FloatList.value) + return value_.Get(index); +} +inline void FloatList::set_value(int index, float value) { + value_.Set(index, value); + // @@protoc_insertion_point(field_set:tensorflow.FloatList.value) +} +inline void FloatList::add_value(float value) { + value_.Add(value); + // @@protoc_insertion_point(field_add:tensorflow.FloatList.value) +} +inline const ::google::protobuf::RepeatedField< float >& +FloatList::value() const { + // @@protoc_insertion_point(field_list:tensorflow.FloatList.value) + return value_; +} +inline ::google::protobuf::RepeatedField< float >* +FloatList::mutable_value() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.FloatList.value) + return &value_; +} + +// ------------------------------------------------------------------- + +// Int64List + +// repeated int64 value = 1 [packed = true]; +inline int Int64List::value_size() const { + return value_.size(); +} +inline void Int64List::clear_value() { + value_.Clear(); +} +inline ::google::protobuf::int64 Int64List::value(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.Int64List.value) + return value_.Get(index); +} +inline void Int64List::set_value(int index, ::google::protobuf::int64 value) { + value_.Set(index, value); + // @@protoc_insertion_point(field_set:tensorflow.Int64List.value) +} +inline void Int64List::add_value(::google::protobuf::int64 value) { + value_.Add(value); + // @@protoc_insertion_point(field_add:tensorflow.Int64List.value) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& +Int64List::value() const { + // @@protoc_insertion_point(field_list:tensorflow.Int64List.value) + return value_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* +Int64List::mutable_value() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.Int64List.value) + return &value_; +} + +// ------------------------------------------------------------------- + +// Feature + +// .tensorflow.BytesList bytes_list = 1; +inline bool Feature::has_bytes_list() const { + return kind_case() == kBytesList; +} +inline void Feature::set_has_bytes_list() { + _oneof_case_[0] = kBytesList; +} +inline void Feature::clear_bytes_list() { + if (has_bytes_list()) { + if (GetArenaNoVirtual() == NULL) { + delete kind_.bytes_list_; + } + clear_has_kind(); + } +} +inline const ::tensorflow::BytesList& Feature::_internal_bytes_list() const { + return *kind_.bytes_list_; +} +inline ::tensorflow::BytesList* Feature::release_bytes_list() { + // @@protoc_insertion_point(field_release:tensorflow.Feature.bytes_list) + if (has_bytes_list()) { + clear_has_kind(); + ::tensorflow::BytesList* temp = kind_.bytes_list_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + kind_.bytes_list_ = NULL; + return temp; + } else { + return NULL; + } +} +inline const ::tensorflow::BytesList& Feature::bytes_list() const { + // @@protoc_insertion_point(field_get:tensorflow.Feature.bytes_list) + return has_bytes_list() + ? *kind_.bytes_list_ + : *reinterpret_cast< ::tensorflow::BytesList*>(&::tensorflow::_BytesList_default_instance_); +} +inline ::tensorflow::BytesList* Feature::unsafe_arena_release_bytes_list() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.Feature.bytes_list) + if (has_bytes_list()) { + clear_has_kind(); + ::tensorflow::BytesList* temp = kind_.bytes_list_; + kind_.bytes_list_ = NULL; + return temp; + } else { + return NULL; + } +} +inline void Feature::unsafe_arena_set_allocated_bytes_list(::tensorflow::BytesList* bytes_list) { + clear_kind(); + if (bytes_list) { + set_has_bytes_list(); + kind_.bytes_list_ = bytes_list; + } + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.Feature.bytes_list) +} +inline ::tensorflow::BytesList* Feature::mutable_bytes_list() { + if (!has_bytes_list()) { + clear_kind(); + set_has_bytes_list(); + kind_.bytes_list_ = CreateMaybeMessage< ::tensorflow::BytesList >( + GetArenaNoVirtual()); + } + // @@protoc_insertion_point(field_mutable:tensorflow.Feature.bytes_list) + return kind_.bytes_list_; +} + +// .tensorflow.FloatList float_list = 2; +inline bool Feature::has_float_list() const { + return kind_case() == kFloatList; +} +inline void Feature::set_has_float_list() { + _oneof_case_[0] = kFloatList; +} +inline void Feature::clear_float_list() { + if (has_float_list()) { + if (GetArenaNoVirtual() == NULL) { + delete kind_.float_list_; + } + clear_has_kind(); + } +} +inline const ::tensorflow::FloatList& Feature::_internal_float_list() const { + return *kind_.float_list_; +} +inline ::tensorflow::FloatList* Feature::release_float_list() { + // @@protoc_insertion_point(field_release:tensorflow.Feature.float_list) + if (has_float_list()) { + clear_has_kind(); + ::tensorflow::FloatList* temp = kind_.float_list_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + kind_.float_list_ = NULL; + return temp; + } else { + return NULL; + } +} +inline const ::tensorflow::FloatList& Feature::float_list() const { + // @@protoc_insertion_point(field_get:tensorflow.Feature.float_list) + return has_float_list() + ? *kind_.float_list_ + : *reinterpret_cast< ::tensorflow::FloatList*>(&::tensorflow::_FloatList_default_instance_); +} +inline ::tensorflow::FloatList* Feature::unsafe_arena_release_float_list() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.Feature.float_list) + if (has_float_list()) { + clear_has_kind(); + ::tensorflow::FloatList* temp = kind_.float_list_; + kind_.float_list_ = NULL; + return temp; + } else { + return NULL; + } +} +inline void Feature::unsafe_arena_set_allocated_float_list(::tensorflow::FloatList* float_list) { + clear_kind(); + if (float_list) { + set_has_float_list(); + kind_.float_list_ = float_list; + } + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.Feature.float_list) +} +inline ::tensorflow::FloatList* Feature::mutable_float_list() { + if (!has_float_list()) { + clear_kind(); + set_has_float_list(); + kind_.float_list_ = CreateMaybeMessage< ::tensorflow::FloatList >( + GetArenaNoVirtual()); + } + // @@protoc_insertion_point(field_mutable:tensorflow.Feature.float_list) + return kind_.float_list_; +} + +// .tensorflow.Int64List int64_list = 3; +inline bool Feature::has_int64_list() const { + return kind_case() == kInt64List; +} +inline void Feature::set_has_int64_list() { + _oneof_case_[0] = kInt64List; +} +inline void Feature::clear_int64_list() { + if (has_int64_list()) { + if (GetArenaNoVirtual() == NULL) { + delete kind_.int64_list_; + } + clear_has_kind(); + } +} +inline const ::tensorflow::Int64List& Feature::_internal_int64_list() const { + return *kind_.int64_list_; +} +inline ::tensorflow::Int64List* Feature::release_int64_list() { + // @@protoc_insertion_point(field_release:tensorflow.Feature.int64_list) + if (has_int64_list()) { + clear_has_kind(); + ::tensorflow::Int64List* temp = kind_.int64_list_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + kind_.int64_list_ = NULL; + return temp; + } else { + return NULL; + } +} +inline const ::tensorflow::Int64List& Feature::int64_list() const { + // @@protoc_insertion_point(field_get:tensorflow.Feature.int64_list) + return has_int64_list() + ? *kind_.int64_list_ + : *reinterpret_cast< ::tensorflow::Int64List*>(&::tensorflow::_Int64List_default_instance_); +} +inline ::tensorflow::Int64List* Feature::unsafe_arena_release_int64_list() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.Feature.int64_list) + if (has_int64_list()) { + clear_has_kind(); + ::tensorflow::Int64List* temp = kind_.int64_list_; + kind_.int64_list_ = NULL; + return temp; + } else { + return NULL; + } +} +inline void Feature::unsafe_arena_set_allocated_int64_list(::tensorflow::Int64List* int64_list) { + clear_kind(); + if (int64_list) { + set_has_int64_list(); + kind_.int64_list_ = int64_list; + } + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.Feature.int64_list) +} +inline ::tensorflow::Int64List* Feature::mutable_int64_list() { + if (!has_int64_list()) { + clear_kind(); + set_has_int64_list(); + kind_.int64_list_ = CreateMaybeMessage< ::tensorflow::Int64List >( + GetArenaNoVirtual()); + } + // @@protoc_insertion_point(field_mutable:tensorflow.Feature.int64_list) + return kind_.int64_list_; +} + +inline bool Feature::has_kind() const { + return kind_case() != KIND_NOT_SET; +} +inline void Feature::clear_has_kind() { + _oneof_case_[0] = KIND_NOT_SET; +} +inline Feature::KindCase Feature::kind_case() const { + return Feature::KindCase(_oneof_case_[0]); +} +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// Features + +// map feature = 1; +inline int Features::feature_size() const { + return feature_.size(); +} +inline void Features::clear_feature() { + feature_.Clear(); +} +inline const ::google::protobuf::Map< ::std::string, ::tensorflow::Feature >& +Features::feature() const { + // @@protoc_insertion_point(field_map:tensorflow.Features.feature) + return feature_.GetMap(); +} +inline ::google::protobuf::Map< ::std::string, ::tensorflow::Feature >* +Features::mutable_feature() { + // @@protoc_insertion_point(field_mutable_map:tensorflow.Features.feature) + return feature_.MutableMap(); +} + +// ------------------------------------------------------------------- + +// FeatureList + +// repeated .tensorflow.Feature feature = 1; +inline int FeatureList::feature_size() const { + return feature_.size(); +} +inline void FeatureList::clear_feature() { + feature_.Clear(); +} +inline ::tensorflow::Feature* FeatureList::mutable_feature(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.FeatureList.feature) + return feature_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::Feature >* +FeatureList::mutable_feature() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.FeatureList.feature) + return &feature_; +} +inline const ::tensorflow::Feature& FeatureList::feature(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.FeatureList.feature) + return feature_.Get(index); +} +inline ::tensorflow::Feature* FeatureList::add_feature() { + // @@protoc_insertion_point(field_add:tensorflow.FeatureList.feature) + return feature_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::Feature >& +FeatureList::feature() const { + // @@protoc_insertion_point(field_list:tensorflow.FeatureList.feature) + return feature_; +} + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// FeatureLists + +// map feature_list = 1; +inline int FeatureLists::feature_list_size() const { + return feature_list_.size(); +} +inline void FeatureLists::clear_feature_list() { + feature_list_.Clear(); +} +inline const ::google::protobuf::Map< ::std::string, ::tensorflow::FeatureList >& +FeatureLists::feature_list() const { + // @@protoc_insertion_point(field_map:tensorflow.FeatureLists.feature_list) + return feature_list_.GetMap(); +} +inline ::google::protobuf::Map< ::std::string, ::tensorflow::FeatureList >* +FeatureLists::mutable_feature_list() { + // @@protoc_insertion_point(field_mutable_map:tensorflow.FeatureLists.feature_list) + return feature_list_.MutableMap(); +} + +#ifdef __GNUC__ + #pragma GCC diagnostic pop +#endif // __GNUC__ +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + + +// @@protoc_insertion_point(namespace_scope) + +} // namespace tensorflow + +// @@protoc_insertion_point(global_scope) + +#endif // PROTOBUF_INCLUDED_tensorflow_2fcore_2fexample_2ffeature_2eproto diff --git a/linux/include/tensorflow/core/example/feature.pb_text-impl.h b/linux/include/tensorflow/core/example/feature.pb_text-impl.h new file mode 100644 index 000000000..a28159551 --- /dev/null +++ b/linux/include/tensorflow/core/example/feature.pb_text-impl.h @@ -0,0 +1,67 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_example_feature_proto_IMPL_H_ +#define tensorflow_core_example_feature_proto_IMPL_H_ + +#include "tensorflow/core/example/feature.pb.h" +#include "tensorflow/core/example/feature.pb_text.h" +#include "tensorflow/core/lib/strings/proto_text_util.h" +#include "tensorflow/core/lib/strings/scanner.h" + +namespace tensorflow { + +namespace internal { + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::BytesList& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::BytesList* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::FloatList& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::FloatList* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::Int64List& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::Int64List* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::Feature& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::Feature* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::Features& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::Features* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::FeatureList& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::FeatureList* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::FeatureLists& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::FeatureLists* msg); + +} // namespace internal + +} // namespace tensorflow + +#endif // tensorflow_core_example_feature_proto_IMPL_H_ diff --git a/linux/include/tensorflow/core/example/feature.pb_text.h b/linux/include/tensorflow/core/example/feature.pb_text.h new file mode 100644 index 000000000..f280b6014 --- /dev/null +++ b/linux/include/tensorflow/core/example/feature.pb_text.h @@ -0,0 +1,84 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_example_feature_proto_H_ +#define tensorflow_core_example_feature_proto_H_ + +#include "tensorflow/core/example/feature.pb.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/protobuf.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +// Message-text conversion for tensorflow.BytesList +string ProtoDebugString( + const ::tensorflow::BytesList& msg); +string ProtoShortDebugString( + const ::tensorflow::BytesList& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::BytesList* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.FloatList +string ProtoDebugString( + const ::tensorflow::FloatList& msg); +string ProtoShortDebugString( + const ::tensorflow::FloatList& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::FloatList* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.Int64List +string ProtoDebugString( + const ::tensorflow::Int64List& msg); +string ProtoShortDebugString( + const ::tensorflow::Int64List& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::Int64List* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.Feature +string ProtoDebugString( + const ::tensorflow::Feature& msg); +string ProtoShortDebugString( + const ::tensorflow::Feature& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::Feature* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.Features +string ProtoDebugString( + const ::tensorflow::Features& msg); +string ProtoShortDebugString( + const ::tensorflow::Features& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::Features* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.FeatureList +string ProtoDebugString( + const ::tensorflow::FeatureList& msg); +string ProtoShortDebugString( + const ::tensorflow::FeatureList& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::FeatureList* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.FeatureLists +string ProtoDebugString( + const ::tensorflow::FeatureLists& msg); +string ProtoShortDebugString( + const ::tensorflow::FeatureLists& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::FeatureLists* msg) + TF_MUST_USE_RESULT; + +} // namespace tensorflow + +#endif // tensorflow_core_example_feature_proto_H_ diff --git a/linux/include/tensorflow/core/example/feature_util.h b/linux/include/tensorflow/core/example/feature_util.h new file mode 100644 index 000000000..016d1a92c --- /dev/null +++ b/linux/include/tensorflow/core/example/feature_util.h @@ -0,0 +1,326 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// A set of lightweight wrappers which simplify access to Feature protos. +// +// TensorFlow Example proto uses associative maps on top of oneof fields. +// SequenceExample proto uses associative map of FeatureList. +// So accessing feature values is not very convenient. +// +// For example, to read a first value of integer feature "tag": +// int id = example.features().feature().at("tag").int64_list().value(0) +// +// to add a value: +// auto features = example->mutable_features(); +// (*features->mutable_feature())["tag"].mutable_int64_list()->add_value(id) +// +// For float features you have to use float_list, for string - bytes_list. +// +// To do the same with this library: +// int id = GetFeatureValues("tag", example).Get(0); +// GetFeatureValues("tag", &example)->Add(id); +// +// Modification of bytes features is slightly different: +// auto tag = GetFeatureValues("tag", &example); +// *tag->Add() = "lorem ipsum"; +// +// To copy multiple values into a feature: +// AppendFeatureValues({1,2,3}, "tag", &example); +// +// GetFeatureValues gives you access to underlying data - RepeatedField object +// (RepeatedPtrField for byte list). So refer to its documentation of +// RepeatedField for full list of supported methods. +// +// NOTE: Due to the nature of oneof proto fields setting a feature of one type +// automatically clears all values stored as another type with the same feature +// key. +// +// This library also has tools to work with SequenceExample protos. +// +// To get a value from SequenceExample.context: +// int id = GetFeatureValues("tag", se.context()).Get(0); +// To add a value to the context: +// GetFeatureValues("tag", se.mutable_context())->Add(42); +// +// To add values to feature_lists: +// AppendFeatureValues({4.0}, +// GetFeatureList("images", &se)->Add()); +// AppendFeatureValues({5.0, 3.0}, +// GetFeatureList("images", &se)->Add()); +// This will create a feature list keyed as "images" with two features: +// feature_lists { +// feature_list { +// key: "images" +// value { +// feature { float_list { value: [4.0] } } +// feature { float_list { value: [5.0, 3.0] } } +// } +// } } +// +// Functions exposed by this library: +// HasFeature<[FeatureType]>(key, proto) -> bool +// Returns true if a feature with the specified key, and optionally +// FeatureType, belongs to the Features or Example proto. +// HasFeatureList(key, sequence_example) -> bool +// Returns true if SequenceExample has a feature_list with the key. +// GetFeatureValues(key, proto) -> RepeatedField +// Returns values for the specified key and the FeatureType. +// Supported types for the proto: Example, Features. +// GetFeatureList(key, sequence_example) -> RepeatedPtrField +// Returns Feature protos associated with a key. +// AppendFeatureValues(begin, end, feature) +// AppendFeatureValues(container or initializer_list, feature) +// Copies values into a Feature. +// AppendFeatureValues(begin, end, key, proto) +// AppendFeatureValues(container or initializer_list, key, proto) +// Copies values into Features and Example protos with the specified key. +// +// Auxiliary functions, it is unlikely you'll need to use them directly: +// GetFeatures(proto) -> Features +// A convenience function to get Features proto. +// Supported types for the proto: Example, Features. +// GetFeature(key, proto) -> Feature* +// Returns a Feature proto for the specified key, creates a new if +// necessary. Supported types for the proto: Example, Features. +// GetFeatureValues(feature) -> RepeatedField +// Returns values of the feature for the FeatureType. + +#ifndef TENSORFLOW_CORE_EXAMPLE_FEATURE_UTIL_H_ +#define TENSORFLOW_CORE_EXAMPLE_FEATURE_UTIL_H_ + +#include +#include + +#include "absl/base/macros.h" +#include "tensorflow/core/example/example.pb.h" +#include "tensorflow/core/example/feature.pb.h" +#include "tensorflow/core/lib/core/stringpiece.h" +#include "tensorflow/core/platform/protobuf.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +namespace internal { + +// TODO(gorban): Update all clients in a followup CL. +// Returns a reference to a feature corresponding to the name. +// Note: it will create a new Feature if it is missing in the example. +ABSL_DEPRECATED("Use GetFeature instead.") +Feature& ExampleFeature(const string& name, Example* example); + +// Specializations of RepeatedFieldTrait define a type of RepeatedField +// corresponding to a selected feature type. +template +struct RepeatedFieldTrait; + +template <> +struct RepeatedFieldTrait { + using Type = protobuf::RepeatedField; +}; + +template <> +struct RepeatedFieldTrait { + using Type = protobuf::RepeatedField; +}; + +template <> +struct RepeatedFieldTrait { + using Type = protobuf::RepeatedPtrField; +}; + +// Specializations of FeatureTrait define a type of feature corresponding to a +// selected value type. +template +struct FeatureTrait; + +template +struct FeatureTrait::value>::type> { + using Type = protobuf_int64; +}; + +template +struct FeatureTrait< + ValueType, + typename std::enable_if::value>::type> { + using Type = float; +}; + +template +struct is_string + : public std::integral_constant< + bool, + std::is_same::type>::value || + std::is_same::type>::value> { +}; + +template <> +struct is_string : std::true_type {}; + +template <> +struct is_string<::tensorflow::StringPiece> : std::true_type {}; + +template +struct FeatureTrait< + ValueType, typename std::enable_if::value>::type> { + using Type = string; +}; + +} // namespace internal + +// Returns true if sequence_example has a feature_list with the specified key. +bool HasFeatureList(const string& key, const SequenceExample& sequence_example); + +template +struct TypeHasFeatures : std::false_type {}; + +template <> +struct TypeHasFeatures : std::true_type {}; + +template <> +struct TypeHasFeatures : std::true_type {}; + +// A family of template functions to return mutable Features proto from a +// container proto. Supported ProtoTypes: Example, Features. +template +typename std::enable_if::value, Features*>::type +GetFeatures(ProtoType* proto); + +template +typename std::enable_if::value, + const Features&>::type +GetFeatures(const ProtoType& proto); + +// Base declaration of a family of template functions to return a read only +// repeated field of feature values. +template +const typename internal::RepeatedFieldTrait::Type& +GetFeatureValues(const Feature& feature); + +// Returns a read only repeated field corresponding to a feature with the +// specified name and FeatureType. Supported ProtoTypes: Example, Features. +template +const typename internal::RepeatedFieldTrait::Type& +GetFeatureValues(const string& key, const ProtoType& proto) { + return GetFeatureValues(GetFeatures(proto).feature().at(key)); +} + +// Returns a mutable repeated field of a feature values. +template +typename internal::RepeatedFieldTrait::Type* GetFeatureValues( + Feature* feature); + +// Returns a mutable repeated field corresponding to a feature with the +// specified name and FeatureType. Supported ProtoTypes: Example, Features. +template +typename internal::RepeatedFieldTrait::Type* GetFeatureValues( + const string& key, ProtoType* proto) { + ::tensorflow::Feature& feature = + (*GetFeatures(proto)->mutable_feature())[key]; + return GetFeatureValues(&feature); +} + +// Returns a Feature proto for the specified key, creates a new if necessary. +// Supported types for the proto: Example, Features. +template +Feature* GetFeature(const string& key, ProtoType* proto) { + return &(*GetFeatures(proto)->mutable_feature())[key]; +} + +// Returns a repeated field with features corresponding to a feature_list key. +const protobuf::RepeatedPtrField& GetFeatureList( + const string& key, const SequenceExample& sequence_example); + +// Returns a mutable repeated field with features corresponding to a +// feature_list key. It will create a new FeatureList if necessary. +protobuf::RepeatedPtrField* GetFeatureList( + const string& feature_list_key, SequenceExample* sequence_example); + +template +void AppendFeatureValues(IteratorType first, IteratorType last, + Feature* feature) { + using FeatureType = typename internal::FeatureTrait< + typename std::iterator_traits::value_type>::Type; + std::copy(first, last, + protobuf::RepeatedFieldBackInserter( + GetFeatureValues(feature))); +} + +template +void AppendFeatureValues(std::initializer_list container, + Feature* feature) { + AppendFeatureValues(container.begin(), container.end(), feature); +} + +template +void AppendFeatureValues(const ContainerType& container, Feature* feature) { + using IteratorType = typename ContainerType::const_iterator; + AppendFeatureValues(container.begin(), container.end(), + feature); +} + +// Copies elements from the range, defined by [first, last) into the feature +// obtainable from the (proto, key) combination. +template +void AppendFeatureValues(IteratorType first, IteratorType last, + const string& key, ProtoType* proto) { + AppendFeatureValues(first, last, GetFeature(key, GetFeatures(proto))); +} + +// Copies all elements from the container into a feature. +template +void AppendFeatureValues(const ContainerType& container, const string& key, + ProtoType* proto) { + using IteratorType = typename ContainerType::const_iterator; + AppendFeatureValues(container.begin(), container.end(), key, + proto); +} + +// Copies all elements from the initializer list into a Feature contained by +// Features or Example proto. +template +void AppendFeatureValues(std::initializer_list container, + const string& key, ProtoType* proto) { + using IteratorType = + typename std::initializer_list::const_iterator; + AppendFeatureValues(container.begin(), container.end(), key, + proto); +} + +// Returns true if a feature with the specified key belongs to the Features. +// The template parameter pack accepts zero or one template argument - which +// is FeatureType. If the FeatureType not specified (zero template arguments) +// the function will not check the feature type. Otherwise it will return false +// if the feature has a wrong type. +template +bool HasFeature(const string& key, const Features& features); + +// Returns true if a feature with the specified key belongs to the Example. +// Doesn't check feature type if used without FeatureType, otherwise the +// specialized versions return false if the feature has a wrong type. +template +bool HasFeature(const string& key, const Example& example) { + return HasFeature(key, GetFeatures(example)); +} + +// TODO(gorban): update all clients in a followup CL. +template +ABSL_DEPRECATED("Use HasFeature instead.") +bool ExampleHasFeature(const string& key, const Example& example) { + return HasFeature(key, example); +} + +} // namespace tensorflow +#endif // TENSORFLOW_CORE_EXAMPLE_FEATURE_UTIL_H_ diff --git a/linux/include/tensorflow/core/framework/allocation_description.pb.h b/linux/include/tensorflow/core/framework/allocation_description.pb.h new file mode 100644 index 000000000..dcfaf7fe8 --- /dev/null +++ b/linux/include/tensorflow/core/framework/allocation_description.pb.h @@ -0,0 +1,396 @@ +// Generated by the protocol buffer compiler. DO NOT EDIT! +// source: tensorflow/core/framework/allocation_description.proto + +#ifndef PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2fallocation_5fdescription_2eproto +#define PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2fallocation_5fdescription_2eproto + +#include + +#include + +#if GOOGLE_PROTOBUF_VERSION < 3006000 +#error This file was generated by a newer version of protoc which is +#error incompatible with your Protocol Buffer headers. Please update +#error your headers. +#endif +#if 3006000 < GOOGLE_PROTOBUF_MIN_PROTOC_VERSION +#error This file was generated by an older version of protoc which is +#error incompatible with your Protocol Buffer headers. Please +#error regenerate this file with a newer version of protoc. +#endif + +#include +#include +#include +#include +#include +#include +#include +#include +#include // IWYU pragma: export +#include // IWYU pragma: export +#include +// @@protoc_insertion_point(includes) +#define PROTOBUF_INTERNAL_EXPORT_protobuf_tensorflow_2fcore_2fframework_2fallocation_5fdescription_2eproto + +namespace protobuf_tensorflow_2fcore_2fframework_2fallocation_5fdescription_2eproto { +// Internal implementation detail -- do not use these members. +struct TableStruct { + static const ::google::protobuf::internal::ParseTableField entries[]; + static const ::google::protobuf::internal::AuxillaryParseTableField aux[]; + static const ::google::protobuf::internal::ParseTable schema[1]; + static const ::google::protobuf::internal::FieldMetadata field_metadata[]; + static const ::google::protobuf::internal::SerializationTable serialization_table[]; + static const ::google::protobuf::uint32 offsets[]; +}; +void AddDescriptors(); +} // namespace protobuf_tensorflow_2fcore_2fframework_2fallocation_5fdescription_2eproto +namespace tensorflow { +class AllocationDescription; +class AllocationDescriptionDefaultTypeInternal; +extern AllocationDescriptionDefaultTypeInternal _AllocationDescription_default_instance_; +} // namespace tensorflow +namespace google { +namespace protobuf { +template<> ::tensorflow::AllocationDescription* Arena::CreateMaybeMessage<::tensorflow::AllocationDescription>(Arena*); +} // namespace protobuf +} // namespace google +namespace tensorflow { + +// =================================================================== + +class AllocationDescription : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.AllocationDescription) */ { + public: + AllocationDescription(); + virtual ~AllocationDescription(); + + AllocationDescription(const AllocationDescription& from); + + inline AllocationDescription& operator=(const AllocationDescription& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + AllocationDescription(AllocationDescription&& from) noexcept + : AllocationDescription() { + *this = ::std::move(from); + } + + inline AllocationDescription& operator=(AllocationDescription&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const AllocationDescription& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const AllocationDescription* internal_default_instance() { + return reinterpret_cast( + &_AllocationDescription_default_instance_); + } + static constexpr int kIndexInFileMessages = + 0; + + void UnsafeArenaSwap(AllocationDescription* other); + void Swap(AllocationDescription* other); + friend void swap(AllocationDescription& a, AllocationDescription& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline AllocationDescription* New() const final { + return CreateMaybeMessage(NULL); + } + + AllocationDescription* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const AllocationDescription& from); + void MergeFrom(const AllocationDescription& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(AllocationDescription* other); + protected: + explicit AllocationDescription(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // string allocator_name = 3; + void clear_allocator_name(); + static const int kAllocatorNameFieldNumber = 3; + const ::std::string& allocator_name() const; + void set_allocator_name(const ::std::string& value); + #if LANG_CXX11 + void set_allocator_name(::std::string&& value); + #endif + void set_allocator_name(const char* value); + void set_allocator_name(const char* value, size_t size); + ::std::string* mutable_allocator_name(); + ::std::string* release_allocator_name(); + void set_allocated_allocator_name(::std::string* allocator_name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_allocator_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_allocator_name( + ::std::string* allocator_name); + + // int64 requested_bytes = 1; + void clear_requested_bytes(); + static const int kRequestedBytesFieldNumber = 1; + ::google::protobuf::int64 requested_bytes() const; + void set_requested_bytes(::google::protobuf::int64 value); + + // int64 allocated_bytes = 2; + void clear_allocated_bytes(); + static const int kAllocatedBytesFieldNumber = 2; + ::google::protobuf::int64 allocated_bytes() const; + void set_allocated_bytes(::google::protobuf::int64 value); + + // int64 allocation_id = 4; + void clear_allocation_id(); + static const int kAllocationIdFieldNumber = 4; + ::google::protobuf::int64 allocation_id() const; + void set_allocation_id(::google::protobuf::int64 value); + + // uint64 ptr = 6; + void clear_ptr(); + static const int kPtrFieldNumber = 6; + ::google::protobuf::uint64 ptr() const; + void set_ptr(::google::protobuf::uint64 value); + + // bool has_single_reference = 5; + void clear_has_single_reference(); + static const int kHasSingleReferenceFieldNumber = 5; + bool has_single_reference() const; + void set_has_single_reference(bool value); + + // @@protoc_insertion_point(class_scope:tensorflow.AllocationDescription) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::internal::ArenaStringPtr allocator_name_; + ::google::protobuf::int64 requested_bytes_; + ::google::protobuf::int64 allocated_bytes_; + ::google::protobuf::int64 allocation_id_; + ::google::protobuf::uint64 ptr_; + bool has_single_reference_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fallocation_5fdescription_2eproto::TableStruct; +}; +// =================================================================== + + +// =================================================================== + +#ifdef __GNUC__ + #pragma GCC diagnostic push + #pragma GCC diagnostic ignored "-Wstrict-aliasing" +#endif // __GNUC__ +// AllocationDescription + +// int64 requested_bytes = 1; +inline void AllocationDescription::clear_requested_bytes() { + requested_bytes_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 AllocationDescription::requested_bytes() const { + // @@protoc_insertion_point(field_get:tensorflow.AllocationDescription.requested_bytes) + return requested_bytes_; +} +inline void AllocationDescription::set_requested_bytes(::google::protobuf::int64 value) { + + requested_bytes_ = value; + // @@protoc_insertion_point(field_set:tensorflow.AllocationDescription.requested_bytes) +} + +// int64 allocated_bytes = 2; +inline void AllocationDescription::clear_allocated_bytes() { + allocated_bytes_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 AllocationDescription::allocated_bytes() const { + // @@protoc_insertion_point(field_get:tensorflow.AllocationDescription.allocated_bytes) + return allocated_bytes_; +} +inline void AllocationDescription::set_allocated_bytes(::google::protobuf::int64 value) { + + allocated_bytes_ = value; + // @@protoc_insertion_point(field_set:tensorflow.AllocationDescription.allocated_bytes) +} + +// string allocator_name = 3; +inline void AllocationDescription::clear_allocator_name() { + allocator_name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& AllocationDescription::allocator_name() const { + // @@protoc_insertion_point(field_get:tensorflow.AllocationDescription.allocator_name) + return allocator_name_.Get(); +} +inline void AllocationDescription::set_allocator_name(const ::std::string& value) { + + allocator_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.AllocationDescription.allocator_name) +} +#if LANG_CXX11 +inline void AllocationDescription::set_allocator_name(::std::string&& value) { + + allocator_name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.AllocationDescription.allocator_name) +} +#endif +inline void AllocationDescription::set_allocator_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + allocator_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.AllocationDescription.allocator_name) +} +inline void AllocationDescription::set_allocator_name(const char* value, + size_t size) { + + allocator_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.AllocationDescription.allocator_name) +} +inline ::std::string* AllocationDescription::mutable_allocator_name() { + + // @@protoc_insertion_point(field_mutable:tensorflow.AllocationDescription.allocator_name) + return allocator_name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* AllocationDescription::release_allocator_name() { + // @@protoc_insertion_point(field_release:tensorflow.AllocationDescription.allocator_name) + + return allocator_name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void AllocationDescription::set_allocated_allocator_name(::std::string* allocator_name) { + if (allocator_name != NULL) { + + } else { + + } + allocator_name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), allocator_name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.AllocationDescription.allocator_name) +} +inline ::std::string* AllocationDescription::unsafe_arena_release_allocator_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.AllocationDescription.allocator_name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return allocator_name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void AllocationDescription::unsafe_arena_set_allocated_allocator_name( + ::std::string* allocator_name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (allocator_name != NULL) { + + } else { + + } + allocator_name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + allocator_name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.AllocationDescription.allocator_name) +} + +// int64 allocation_id = 4; +inline void AllocationDescription::clear_allocation_id() { + allocation_id_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 AllocationDescription::allocation_id() const { + // @@protoc_insertion_point(field_get:tensorflow.AllocationDescription.allocation_id) + return allocation_id_; +} +inline void AllocationDescription::set_allocation_id(::google::protobuf::int64 value) { + + allocation_id_ = value; + // @@protoc_insertion_point(field_set:tensorflow.AllocationDescription.allocation_id) +} + +// bool has_single_reference = 5; +inline void AllocationDescription::clear_has_single_reference() { + has_single_reference_ = false; +} +inline bool AllocationDescription::has_single_reference() const { + // @@protoc_insertion_point(field_get:tensorflow.AllocationDescription.has_single_reference) + return has_single_reference_; +} +inline void AllocationDescription::set_has_single_reference(bool value) { + + has_single_reference_ = value; + // @@protoc_insertion_point(field_set:tensorflow.AllocationDescription.has_single_reference) +} + +// uint64 ptr = 6; +inline void AllocationDescription::clear_ptr() { + ptr_ = GOOGLE_ULONGLONG(0); +} +inline ::google::protobuf::uint64 AllocationDescription::ptr() const { + // @@protoc_insertion_point(field_get:tensorflow.AllocationDescription.ptr) + return ptr_; +} +inline void AllocationDescription::set_ptr(::google::protobuf::uint64 value) { + + ptr_ = value; + // @@protoc_insertion_point(field_set:tensorflow.AllocationDescription.ptr) +} + +#ifdef __GNUC__ + #pragma GCC diagnostic pop +#endif // __GNUC__ + +// @@protoc_insertion_point(namespace_scope) + +} // namespace tensorflow + +// @@protoc_insertion_point(global_scope) + +#endif // PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2fallocation_5fdescription_2eproto diff --git a/linux/include/tensorflow/core/framework/allocation_description.pb_text-impl.h b/linux/include/tensorflow/core/framework/allocation_description.pb_text-impl.h new file mode 100644 index 000000000..3992b7eae --- /dev/null +++ b/linux/include/tensorflow/core/framework/allocation_description.pb_text-impl.h @@ -0,0 +1,25 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_framework_allocation_description_proto_IMPL_H_ +#define tensorflow_core_framework_allocation_description_proto_IMPL_H_ + +#include "tensorflow/core/framework/allocation_description.pb.h" +#include "tensorflow/core/framework/allocation_description.pb_text.h" +#include "tensorflow/core/lib/strings/proto_text_util.h" +#include "tensorflow/core/lib/strings/scanner.h" + +namespace tensorflow { + +namespace internal { + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::AllocationDescription& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::AllocationDescription* msg); + +} // namespace internal + +} // namespace tensorflow + +#endif // tensorflow_core_framework_allocation_description_proto_IMPL_H_ diff --git a/linux/include/tensorflow/core/framework/allocation_description.pb_text.h b/linux/include/tensorflow/core/framework/allocation_description.pb_text.h new file mode 100644 index 000000000..ab0ad5c03 --- /dev/null +++ b/linux/include/tensorflow/core/framework/allocation_description.pb_text.h @@ -0,0 +1,24 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_framework_allocation_description_proto_H_ +#define tensorflow_core_framework_allocation_description_proto_H_ + +#include "tensorflow/core/framework/allocation_description.pb.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/protobuf.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +// Message-text conversion for tensorflow.AllocationDescription +string ProtoDebugString( + const ::tensorflow::AllocationDescription& msg); +string ProtoShortDebugString( + const ::tensorflow::AllocationDescription& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::AllocationDescription* msg) + TF_MUST_USE_RESULT; + +} // namespace tensorflow + +#endif // tensorflow_core_framework_allocation_description_proto_H_ diff --git a/linux/include/tensorflow/core/framework/allocator.h b/linux/include/tensorflow/core/framework/allocator.h new file mode 100644 index 000000000..8c2360462 --- /dev/null +++ b/linux/include/tensorflow/core/framework/allocator.h @@ -0,0 +1,425 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_FRAMEWORK_ALLOCATOR_H_ +#define TENSORFLOW_CORE_FRAMEWORK_ALLOCATOR_H_ + +#include + +#include + +#include "tensorflow/core/framework/numeric_types.h" +#include "tensorflow/core/framework/resource_handle.h" +#include "tensorflow/core/framework/type_traits.h" +#include "tensorflow/core/platform/logging.h" +#include "tensorflow/core/platform/mutex.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +class Variant; + +// Attributes for a single allocation call. Different calls to the same +// allocator could potentially have different allocation attributes. +struct AllocationAttributes { + // If the first attempt to allocate the memory fails, the allocation + // should return immediately without retrying. + // An example use case is optional scratch spaces where a failure + // has only performance impact. + bool no_retry_on_failure = false; + // If a Tensor is allocated without the following set to true, then + // it is logged as an unknown allocation. During execution Tensors + // should be allocated through the OpKernelContext which records + // which Op is performing the allocation, and sets this flag to + // true. + bool allocation_will_be_logged = false; +}; + +// Runtime statistics collected by an allocator. +struct AllocatorStats { + int64 num_allocs; // Number of allocations. + int64 bytes_in_use; // Number of bytes in use. + int64 max_bytes_in_use; // The maximum bytes in use. + int64 max_alloc_size; // The max single allocation seen. + + // The upper limit what the allocator can allocate, if such a limit + // is known. Certain allocator may return 0 to indicate the limit is + // unknown. + int64 bytes_limit; + + AllocatorStats() { Clear(); } + + void Clear(); + string DebugString() const; +}; + +// Allocator is an abstract interface for allocating and deallocating +// device memory. +class Allocator { + public: + // Align to 64 byte boundary. + static constexpr size_t kAllocatorAlignment = 64; + + virtual ~Allocator(); + + // Return a string identifying this allocator + virtual string Name() = 0; + + // Return an uninitialized block of memory that is "num_bytes" bytes + // in size. The returned pointer is guaranteed to be aligned to a + // multiple of "alignment" bytes. + // REQUIRES: "alignment" is a power of 2. + virtual void* AllocateRaw(size_t alignment, size_t num_bytes) = 0; + + // Return an uninitialized block of memory that is "num_bytes" bytes + // in size with specified allocation attributes. The returned pointer is + // guaranteed to be aligned to a multiple of "alignment" bytes. + // REQUIRES: "alignment" is a power of 2. + virtual void* AllocateRaw(size_t alignment, size_t num_bytes, + const AllocationAttributes& allocation_attr) { + // The default behavior is to use the implementation without any allocation + // attributes. + return AllocateRaw(alignment, num_bytes); + } + + // Deallocate a block of memory pointer to by "ptr" + // REQUIRES: "ptr" was previously returned by a call to AllocateRaw + virtual void DeallocateRaw(void* ptr) = 0; + + // Convenience functions to do typed allocation. C++ constructors + // and destructors are invoked for complex types if necessary, + // depending on the concrete Allocator implementation. May return + // NULL if the tensor has too many elements to represent in a single + // allocation. + template + T* Allocate(size_t num_elements) { + return Allocate(num_elements, AllocationAttributes()); + } + + template + T* Allocate(size_t num_elements, + const AllocationAttributes& allocation_attr) { + // TODO(jeff): Do we need to allow clients to pass in alignment + // requirements? + + if (num_elements > (std::numeric_limits::max() / sizeof(T))) { + return NULL; + } + + void* p = AllocateRaw(kAllocatorAlignment, sizeof(T) * num_elements, + allocation_attr); + T* typed_p = reinterpret_cast(p); + if (typed_p) RunCtor(typed_p, num_elements); + return typed_p; + } + + template + void Deallocate(T* ptr, size_t num_elements) { + if (ptr) { + RunDtor(ptr, num_elements); + DeallocateRaw(ptr); + } + } + + // Returns true if this allocator tracks the sizes of allocations. + // RequestedSize and AllocatedSize must be overridden if + // TracksAllocationSizes is overridden to return true. + virtual bool TracksAllocationSizes() { return false; } + + // Returns true if this allocator requires tensors with 0 elements + // to allocate buffers. This is false for most allocators, but may + // be used by special-case allocators that want to track tensor + // usage. + virtual bool ShouldAllocateEmptyTensors() { return false; } + + // Returns the user-requested size of the data allocated at + // 'ptr'. Note that the actual buffer allocated might be larger + // than requested, but this function returns the size requested by + // the user. + // + // REQUIRES: TracksAllocationSizes() is true. + // + // REQUIRES: 'ptr!=nullptr' and points to a buffer previously + // allocated by this allocator. + virtual size_t RequestedSize(const void* ptr) { + CHECK(false) << "allocator doesn't track sizes"; + return size_t(0); + } + + // Returns the allocated size of the buffer at 'ptr' if known, + // otherwise returns RequestedSize(ptr). AllocatedSize(ptr) is + // guaranteed to be >= RequestedSize(ptr). + // + // REQUIRES: TracksAllocationSizes() is true. + // + // REQUIRES: 'ptr!=nullptr' and points to a buffer previously + // allocated by this allocator. + virtual size_t AllocatedSize(const void* ptr) { return RequestedSize(ptr); } + + // Returns either 0 or an identifier assigned to the buffer at 'ptr' + // when the buffer was returned by AllocateRaw. If non-zero, the + // identifier differs from every other ID assigned by this + // allocator. + // + // REQUIRES: TracksAllocationSizes() is true. + // + // REQUIRES: 'ptr!=nullptr' and points to a buffer previously + // allocated by this allocator. + virtual int64 AllocationId(const void* ptr) { return 0; } + + // Returns the allocated size of the buffer at 'ptr' if known, + // otherwise returns 0. This method can be called when + // TracksAllocationSizes() is false, but can be extremely slow. + // + // REQUIRES: 'ptr!=nullptr' and points to a buffer previously + // allocated by this allocator. + virtual size_t AllocatedSizeSlow(const void* ptr) { + if (TracksAllocationSizes()) { + return AllocatedSize(ptr); + } + return 0; + } + + // Fills in 'stats' with statistics collected by this allocator. + virtual void GetStats(AllocatorStats* stats) { stats->Clear(); } + + // Clears the internal stats except for the `in_use` field. + virtual void ClearStats() {} + + private: + // No constructors or destructors are run for simple types + template + void RunCtor(T* p, size_t n) { + static_assert(is_simple_type::value, "T is not a simple type."); + } + + template + void RunDtor(T* p, size_t n) {} + + // custom constructors and destructors that can be overridden for + // non-standard allocators + + // Runs string's default constructor for p[0], p[1], ..., p[n-1]. + virtual void RunStringCtor(string* p, size_t n) { + for (size_t i = 0; i < n; ++p, ++i) new (p) string(); + } + + // Runs string's default destructor for p[0], p[1], ..., p[n-1]. + virtual void RunStringDtor(string* p, size_t n) { + for (size_t i = 0; i < n; ++p, ++i) p->~string(); + } + + virtual void RunResourceCtor(ResourceHandle* p, size_t n) { + for (size_t i = 0; i < n; ++p, ++i) new (p) ResourceHandle(); + } + + // Runs string's default destructor for p[0], p[1], ..., p[n-1]. + virtual void RunResourceDtor(ResourceHandle* p, size_t n) { + for (size_t i = 0; i < n; ++p, ++i) p->~ResourceHandle(); + } + + virtual void RunVariantCtor(Variant* p, size_t n); + + virtual void RunVariantDtor(Variant* p, size_t n); + + // TODO(jeff): Maybe provide some interface to give info about + // current allocation state (total number of bytes available for + // allocation, number of bytes free on device, etc.) +}; + +// Allocator-specific constructors and destructors are used for +// strings +template <> +inline void Allocator::RunCtor(string* p, size_t n) { + RunStringCtor(p, n); +} + +template <> +inline void Allocator::RunDtor(string* p, size_t n) { + RunStringDtor(p, n); +} + +template <> +inline void Allocator::RunCtor(ResourceHandle* p, size_t n) { + RunResourceCtor(p, n); +} + +template <> +inline void Allocator::RunDtor(ResourceHandle* p, size_t n) { + RunResourceDtor(p, n); +} + +template <> +inline void Allocator::RunCtor(Variant* p, size_t n) { + RunVariantCtor(p, n); +} + +template <> +inline void Allocator::RunDtor(Variant* p, size_t n) { + RunVariantDtor(p, n); +} + +// An implementation of Allocator that delegates all calls to another Allocator. +// +// Useful to clients who want to override part of the functionality of another +// allocator. +class AllocatorWrapper : public Allocator { + public: + explicit AllocatorWrapper(Allocator* wrapped) : wrapped_(wrapped) {} + + ~AllocatorWrapper() override {} + + // Returns the wrapped allocator to which all calls are delegated. + Allocator* wrapped() const { return wrapped_; } + + string Name() override { return wrapped_->Name(); } + + void* AllocateRaw(size_t alignment, size_t num_bytes) override { + return wrapped_->AllocateRaw(alignment, num_bytes); + } + + void* AllocateRaw(size_t alignment, size_t num_bytes, + const AllocationAttributes& allocation_attr) override { + return wrapped_->AllocateRaw(alignment, num_bytes, allocation_attr); + } + + void DeallocateRaw(void* ptr) override { wrapped_->DeallocateRaw(ptr); } + + bool TracksAllocationSizes() override { + return wrapped_->TracksAllocationSizes(); + } + + bool ShouldAllocateEmptyTensors() override { + return wrapped_->TracksAllocationSizes(); + } + + size_t RequestedSize(const void* ptr) override { + return wrapped_->RequestedSize(ptr); + } + + size_t AllocatedSize(const void* ptr) override { + return wrapped_->AllocatedSize(ptr); + } + + int64 AllocationId(const void* ptr) override { + return wrapped_->AllocationId(ptr); + } + + size_t AllocatedSizeSlow(const void* ptr) override { + return wrapped_->AllocatedSizeSlow(ptr); + } + + private: + Allocator* const wrapped_; +}; + +// A tensorflow Op may need access to different kinds of memory that +// are not simply a function of the device to which the Op has been +// assigned. For example, an Op executing on a GPU may still need +// to allocate CPU RAM for some purpose. Internal to the tensorflow +// runtime we may choose to allocate CPU ram from special regions +// that have been prepared for higher performance in some use +// contexts, e.g. doing DMA with particular devices. For these +// reasons, the Device interface does not expose just one memory +// Allocator, but instead provides an accessor that takes a +// specification of the desired memory attributes in order to select +// an Allocator. +// +// Example use: +// // Allocator for ordinary device memory: +// Allocator* a = allocator(AllocatorAttributes()); +// ... +// // Allocator for CPU RAM, regardless of where Op is executing: +// AllocatorAttributes attr; +// attr.set_on_host(true); +// Allocator* a = allocator(attr); +struct AllocatorAttributes { + void set_on_host(bool v) { value |= (static_cast(v)); } + bool on_host() const { return value & 0x1; } + void set_nic_compatible(bool v) { value |= (static_cast(v) << 1); } + bool nic_compatible() const { return value & (0x1 << 1); } + void set_gpu_compatible(bool v) { value |= (static_cast(v) << 2); } + bool gpu_compatible() const { return value & (0x1 << 2); } + void Merge(AllocatorAttributes other) { + value |= other.value; + scope_id = (scope_id > 0 && other.scope_id == 0) + ? scope_id + : ((scope_id == 0) ? other.scope_id : 0); + } + // Returns true if the fields set in *this is a subset of or equal to + // those set in other. + bool IsEqualOrLessRestrictiveThan(const AllocatorAttributes& other) const { + return (value | other.value) == other.value; + } + + // NOTE: The upper 8 bits of the value are reserved for + // device-specific uses. Implementors of a device can interpret these + // upper 8 bits in device-specific ways, and ops implemented for those + // devices are responsible for setting those 8 bits appropriately. + uint32 value = 0; + // EXPERIMENTAL: If this is greater than zero, then allocation is delegated to + // a named special-purpose allocator on the same device. + int32 scope_id = 0; +}; + +// Returns a trivial implementation of Allocator, which is a process singleton. +// Access through this function is only intended for use in tests and auxiliary +// processing. Performance sensitive uses should always obtain allocators from +// ProcessState. +Allocator* cpu_allocator(); + +// If 'enable' is true, the default CPU allocator implementation will collect +// AllocatorStats. By default, it's disabled. +void EnableCPUAllocatorStats(bool enable); + +// If 'enable' is true, the default CPU allocator implementation will collect +// full statistics. By default, it's disabled. +void EnableCPUAllocatorFullStats(bool enable); + +// An object that does the underlying suballoc/free of memory for a higher-level +// allocator. The expectation is that the higher-level allocator is doing some +// kind of cache or pool management so that it will call SubAllocator::Alloc and +// Free relatively infrequently, compared to the number of times its own +// AllocateRaw and Free methods are called. +class SubAllocator { + public: + // Visitor gets called with a pointer to a memory area and its + // size in bytes. The index value will be numa_node for a CPU + // allocator and GPU id for a GPU allocator. + typedef std::function Visitor; + + SubAllocator(const std::vector& alloc_visitors, + const std::vector& free_visitors); + + virtual ~SubAllocator() {} + virtual void* Alloc(size_t alignment, size_t num_bytes) = 0; + virtual void Free(void* ptr, size_t num_bytes) = 0; + + protected: + // Implementation of Alloc() method must call this on newly allocated + // value. + void VisitAlloc(void* ptr, int index, size_t num_bytes); + + // Implementation of Free() method must call this on value to be + // freed immediately before deallocation. + void VisitFree(void* ptr, int index, size_t num_bytes); + + const std::vector alloc_visitors_; + const std::vector free_visitors_; +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_FRAMEWORK_ALLOCATOR_H_ diff --git a/linux/include/tensorflow/core/framework/allocator_registry.h b/linux/include/tensorflow/core/framework/allocator_registry.h new file mode 100644 index 000000000..e907c52ba --- /dev/null +++ b/linux/include/tensorflow/core/framework/allocator_registry.h @@ -0,0 +1,122 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// Classes to maintain a static registry of memory allocator factories. +#ifndef TENSORFLOW_CORE_FRAMEWORK_ALLOCATOR_REGISTRY_H_ +#define TENSORFLOW_CORE_FRAMEWORK_ALLOCATOR_REGISTRY_H_ + +#include +#include + +#include "tensorflow/core/framework/allocator.h" +#include "tensorflow/core/platform/mutex.h" +#include "tensorflow/core/platform/numa.h" + +namespace tensorflow { + +class AllocatorFactory { + public: + virtual ~AllocatorFactory() {} + + // Returns true if the factory will create a functionally different + // SubAllocator for different (legal) values of numa_node. + virtual bool NumaEnabled() { return false; } + + // Create an Allocator. + virtual Allocator* CreateAllocator() = 0; + + // Create a SubAllocator. If NumaEnabled() is true, then returned SubAllocator + // will allocate memory local to numa_node. If numa_node == kNUMANoAffinity + // then allocated memory is not specific to any NUMA node. + virtual SubAllocator* CreateSubAllocator(int numa_node) = 0; +}; + +// A singleton registry of AllocatorFactories. +// +// Allocators should be obtained through ProcessState or cpu_allocator() +// (deprecated), not directly through this interface. The purpose of this +// registry is to allow link-time discovery of multiple AllocatorFactories among +// which ProcessState will obtain the best fit at startup. +class AllocatorFactoryRegistry { + public: + AllocatorFactoryRegistry() {} + ~AllocatorFactoryRegistry() {} + + void Register(const char* source_file, int source_line, const string& name, + int priority, AllocatorFactory* factory); + + // Returns 'best fit' Allocator. Find the factory with the highest priority + // and return an allocator constructed by it. If multiple factories have + // been registered with the same priority, picks one by unspecified criteria. + Allocator* GetAllocator(); + + // Returns 'best fit' SubAllocator. First look for the highest priority + // factory that is NUMA-enabled. If none is registered, fall back to the + // highest priority non-NUMA-enabled factory. If NUMA-enabled, return a + // SubAllocator specific to numa_node, otherwise return a NUMA-insensitive + // SubAllocator. + SubAllocator* GetSubAllocator(int numa_node); + + // Returns the singleton value. + static AllocatorFactoryRegistry* singleton(); + + private: + mutex mu_; + bool first_alloc_made_ = false; + struct FactoryEntry { + const char* source_file; + int source_line; + string name; + int priority; + std::unique_ptr factory; + std::unique_ptr allocator; + // Index 0 corresponds to kNUMANoAffinity, other indices are (numa_node + + // 1). + std::vector> sub_allocators; + }; + std::vector factories_ GUARDED_BY(mu_); + + // Returns any FactoryEntry registered under 'name' and 'priority', + // or 'nullptr' if none found. + const FactoryEntry* FindEntry(const string& name, int priority) const + EXCLUSIVE_LOCKS_REQUIRED(mu_); + + TF_DISALLOW_COPY_AND_ASSIGN(AllocatorFactoryRegistry); +}; + +class AllocatorFactoryRegistration { + public: + AllocatorFactoryRegistration(const char* file, int line, const string& name, + int priority, AllocatorFactory* factory) { + AllocatorFactoryRegistry::singleton()->Register(file, line, name, priority, + factory); + } +}; + +#define REGISTER_MEM_ALLOCATOR(name, priority, factory) \ + REGISTER_MEM_ALLOCATOR_UNIQ_HELPER(__COUNTER__, __FILE__, __LINE__, name, \ + priority, factory) + +#define REGISTER_MEM_ALLOCATOR_UNIQ_HELPER(ctr, file, line, name, priority, \ + factory) \ + REGISTER_MEM_ALLOCATOR_UNIQ(ctr, file, line, name, priority, factory) + +#define REGISTER_MEM_ALLOCATOR_UNIQ(ctr, file, line, name, priority, factory) \ + static AllocatorFactoryRegistration allocator_factory_reg_##ctr( \ + file, line, name, priority, new factory) + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_FRAMEWORK_ALLOCATOR_REGISTRY_H_ diff --git a/linux/include/tensorflow/core/framework/api_def.pb.h b/linux/include/tensorflow/core/framework/api_def.pb.h new file mode 100644 index 000000000..c053ea94f --- /dev/null +++ b/linux/include/tensorflow/core/framework/api_def.pb.h @@ -0,0 +1,2458 @@ +// Generated by the protocol buffer compiler. DO NOT EDIT! +// source: tensorflow/core/framework/api_def.proto + +#ifndef PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2fapi_5fdef_2eproto +#define PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2fapi_5fdef_2eproto + +#include + +#include + +#if GOOGLE_PROTOBUF_VERSION < 3006000 +#error This file was generated by a newer version of protoc which is +#error incompatible with your Protocol Buffer headers. Please update +#error your headers. +#endif +#if 3006000 < GOOGLE_PROTOBUF_MIN_PROTOC_VERSION +#error This file was generated by an older version of protoc which is +#error incompatible with your Protocol Buffer headers. Please +#error regenerate this file with a newer version of protoc. +#endif + +#include +#include +#include +#include +#include +#include +#include +#include +#include // IWYU pragma: export +#include // IWYU pragma: export +#include +#include +#include "tensorflow/core/framework/attr_value.pb.h" +// @@protoc_insertion_point(includes) +#define PROTOBUF_INTERNAL_EXPORT_protobuf_tensorflow_2fcore_2fframework_2fapi_5fdef_2eproto + +namespace protobuf_tensorflow_2fcore_2fframework_2fapi_5fdef_2eproto { +// Internal implementation detail -- do not use these members. +struct TableStruct { + static const ::google::protobuf::internal::ParseTableField entries[]; + static const ::google::protobuf::internal::AuxillaryParseTableField aux[]; + static const ::google::protobuf::internal::ParseTable schema[5]; + static const ::google::protobuf::internal::FieldMetadata field_metadata[]; + static const ::google::protobuf::internal::SerializationTable serialization_table[]; + static const ::google::protobuf::uint32 offsets[]; +}; +void AddDescriptors(); +} // namespace protobuf_tensorflow_2fcore_2fframework_2fapi_5fdef_2eproto +namespace tensorflow { +class ApiDef; +class ApiDefDefaultTypeInternal; +extern ApiDefDefaultTypeInternal _ApiDef_default_instance_; +class ApiDef_Arg; +class ApiDef_ArgDefaultTypeInternal; +extern ApiDef_ArgDefaultTypeInternal _ApiDef_Arg_default_instance_; +class ApiDef_Attr; +class ApiDef_AttrDefaultTypeInternal; +extern ApiDef_AttrDefaultTypeInternal _ApiDef_Attr_default_instance_; +class ApiDef_Endpoint; +class ApiDef_EndpointDefaultTypeInternal; +extern ApiDef_EndpointDefaultTypeInternal _ApiDef_Endpoint_default_instance_; +class ApiDefs; +class ApiDefsDefaultTypeInternal; +extern ApiDefsDefaultTypeInternal _ApiDefs_default_instance_; +} // namespace tensorflow +namespace google { +namespace protobuf { +template<> ::tensorflow::ApiDef* Arena::CreateMaybeMessage<::tensorflow::ApiDef>(Arena*); +template<> ::tensorflow::ApiDef_Arg* Arena::CreateMaybeMessage<::tensorflow::ApiDef_Arg>(Arena*); +template<> ::tensorflow::ApiDef_Attr* Arena::CreateMaybeMessage<::tensorflow::ApiDef_Attr>(Arena*); +template<> ::tensorflow::ApiDef_Endpoint* Arena::CreateMaybeMessage<::tensorflow::ApiDef_Endpoint>(Arena*); +template<> ::tensorflow::ApiDefs* Arena::CreateMaybeMessage<::tensorflow::ApiDefs>(Arena*); +} // namespace protobuf +} // namespace google +namespace tensorflow { + +enum ApiDef_Visibility { + ApiDef_Visibility_DEFAULT_VISIBILITY = 0, + ApiDef_Visibility_VISIBLE = 1, + ApiDef_Visibility_SKIP = 2, + ApiDef_Visibility_HIDDEN = 3, + ApiDef_Visibility_ApiDef_Visibility_INT_MIN_SENTINEL_DO_NOT_USE_ = ::google::protobuf::kint32min, + ApiDef_Visibility_ApiDef_Visibility_INT_MAX_SENTINEL_DO_NOT_USE_ = ::google::protobuf::kint32max +}; +bool ApiDef_Visibility_IsValid(int value); +const ApiDef_Visibility ApiDef_Visibility_Visibility_MIN = ApiDef_Visibility_DEFAULT_VISIBILITY; +const ApiDef_Visibility ApiDef_Visibility_Visibility_MAX = ApiDef_Visibility_HIDDEN; +const int ApiDef_Visibility_Visibility_ARRAYSIZE = ApiDef_Visibility_Visibility_MAX + 1; + +const ::google::protobuf::EnumDescriptor* ApiDef_Visibility_descriptor(); +inline const ::std::string& ApiDef_Visibility_Name(ApiDef_Visibility value) { + return ::google::protobuf::internal::NameOfEnum( + ApiDef_Visibility_descriptor(), value); +} +inline bool ApiDef_Visibility_Parse( + const ::std::string& name, ApiDef_Visibility* value) { + return ::google::protobuf::internal::ParseNamedEnum( + ApiDef_Visibility_descriptor(), name, value); +} +// =================================================================== + +class ApiDef_Endpoint : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.ApiDef.Endpoint) */ { + public: + ApiDef_Endpoint(); + virtual ~ApiDef_Endpoint(); + + ApiDef_Endpoint(const ApiDef_Endpoint& from); + + inline ApiDef_Endpoint& operator=(const ApiDef_Endpoint& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + ApiDef_Endpoint(ApiDef_Endpoint&& from) noexcept + : ApiDef_Endpoint() { + *this = ::std::move(from); + } + + inline ApiDef_Endpoint& operator=(ApiDef_Endpoint&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const ApiDef_Endpoint& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const ApiDef_Endpoint* internal_default_instance() { + return reinterpret_cast( + &_ApiDef_Endpoint_default_instance_); + } + static constexpr int kIndexInFileMessages = + 0; + + void UnsafeArenaSwap(ApiDef_Endpoint* other); + void Swap(ApiDef_Endpoint* other); + friend void swap(ApiDef_Endpoint& a, ApiDef_Endpoint& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline ApiDef_Endpoint* New() const final { + return CreateMaybeMessage(NULL); + } + + ApiDef_Endpoint* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const ApiDef_Endpoint& from); + void MergeFrom(const ApiDef_Endpoint& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(ApiDef_Endpoint* other); + protected: + explicit ApiDef_Endpoint(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // string name = 1; + void clear_name(); + static const int kNameFieldNumber = 1; + const ::std::string& name() const; + void set_name(const ::std::string& value); + #if LANG_CXX11 + void set_name(::std::string&& value); + #endif + void set_name(const char* value); + void set_name(const char* value, size_t size); + ::std::string* mutable_name(); + ::std::string* release_name(); + void set_allocated_name(::std::string* name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_name( + ::std::string* name); + + // bool deprecated = 3; + void clear_deprecated(); + static const int kDeprecatedFieldNumber = 3; + bool deprecated() const; + void set_deprecated(bool value); + + // @@protoc_insertion_point(class_scope:tensorflow.ApiDef.Endpoint) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::internal::ArenaStringPtr name_; + bool deprecated_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fapi_5fdef_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class ApiDef_Arg : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.ApiDef.Arg) */ { + public: + ApiDef_Arg(); + virtual ~ApiDef_Arg(); + + ApiDef_Arg(const ApiDef_Arg& from); + + inline ApiDef_Arg& operator=(const ApiDef_Arg& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + ApiDef_Arg(ApiDef_Arg&& from) noexcept + : ApiDef_Arg() { + *this = ::std::move(from); + } + + inline ApiDef_Arg& operator=(ApiDef_Arg&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const ApiDef_Arg& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const ApiDef_Arg* internal_default_instance() { + return reinterpret_cast( + &_ApiDef_Arg_default_instance_); + } + static constexpr int kIndexInFileMessages = + 1; + + void UnsafeArenaSwap(ApiDef_Arg* other); + void Swap(ApiDef_Arg* other); + friend void swap(ApiDef_Arg& a, ApiDef_Arg& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline ApiDef_Arg* New() const final { + return CreateMaybeMessage(NULL); + } + + ApiDef_Arg* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const ApiDef_Arg& from); + void MergeFrom(const ApiDef_Arg& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(ApiDef_Arg* other); + protected: + explicit ApiDef_Arg(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // string name = 1; + void clear_name(); + static const int kNameFieldNumber = 1; + const ::std::string& name() const; + void set_name(const ::std::string& value); + #if LANG_CXX11 + void set_name(::std::string&& value); + #endif + void set_name(const char* value); + void set_name(const char* value, size_t size); + ::std::string* mutable_name(); + ::std::string* release_name(); + void set_allocated_name(::std::string* name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_name( + ::std::string* name); + + // string rename_to = 2; + void clear_rename_to(); + static const int kRenameToFieldNumber = 2; + const ::std::string& rename_to() const; + void set_rename_to(const ::std::string& value); + #if LANG_CXX11 + void set_rename_to(::std::string&& value); + #endif + void set_rename_to(const char* value); + void set_rename_to(const char* value, size_t size); + ::std::string* mutable_rename_to(); + ::std::string* release_rename_to(); + void set_allocated_rename_to(::std::string* rename_to); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_rename_to(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_rename_to( + ::std::string* rename_to); + + // string description = 3; + void clear_description(); + static const int kDescriptionFieldNumber = 3; + const ::std::string& description() const; + void set_description(const ::std::string& value); + #if LANG_CXX11 + void set_description(::std::string&& value); + #endif + void set_description(const char* value); + void set_description(const char* value, size_t size); + ::std::string* mutable_description(); + ::std::string* release_description(); + void set_allocated_description(::std::string* description); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_description(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_description( + ::std::string* description); + + // @@protoc_insertion_point(class_scope:tensorflow.ApiDef.Arg) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::internal::ArenaStringPtr name_; + ::google::protobuf::internal::ArenaStringPtr rename_to_; + ::google::protobuf::internal::ArenaStringPtr description_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fapi_5fdef_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class ApiDef_Attr : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.ApiDef.Attr) */ { + public: + ApiDef_Attr(); + virtual ~ApiDef_Attr(); + + ApiDef_Attr(const ApiDef_Attr& from); + + inline ApiDef_Attr& operator=(const ApiDef_Attr& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + ApiDef_Attr(ApiDef_Attr&& from) noexcept + : ApiDef_Attr() { + *this = ::std::move(from); + } + + inline ApiDef_Attr& operator=(ApiDef_Attr&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const ApiDef_Attr& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const ApiDef_Attr* internal_default_instance() { + return reinterpret_cast( + &_ApiDef_Attr_default_instance_); + } + static constexpr int kIndexInFileMessages = + 2; + + void UnsafeArenaSwap(ApiDef_Attr* other); + void Swap(ApiDef_Attr* other); + friend void swap(ApiDef_Attr& a, ApiDef_Attr& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline ApiDef_Attr* New() const final { + return CreateMaybeMessage(NULL); + } + + ApiDef_Attr* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const ApiDef_Attr& from); + void MergeFrom(const ApiDef_Attr& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(ApiDef_Attr* other); + protected: + explicit ApiDef_Attr(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // string name = 1; + void clear_name(); + static const int kNameFieldNumber = 1; + const ::std::string& name() const; + void set_name(const ::std::string& value); + #if LANG_CXX11 + void set_name(::std::string&& value); + #endif + void set_name(const char* value); + void set_name(const char* value, size_t size); + ::std::string* mutable_name(); + ::std::string* release_name(); + void set_allocated_name(::std::string* name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_name( + ::std::string* name); + + // string rename_to = 2; + void clear_rename_to(); + static const int kRenameToFieldNumber = 2; + const ::std::string& rename_to() const; + void set_rename_to(const ::std::string& value); + #if LANG_CXX11 + void set_rename_to(::std::string&& value); + #endif + void set_rename_to(const char* value); + void set_rename_to(const char* value, size_t size); + ::std::string* mutable_rename_to(); + ::std::string* release_rename_to(); + void set_allocated_rename_to(::std::string* rename_to); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_rename_to(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_rename_to( + ::std::string* rename_to); + + // string description = 4; + void clear_description(); + static const int kDescriptionFieldNumber = 4; + const ::std::string& description() const; + void set_description(const ::std::string& value); + #if LANG_CXX11 + void set_description(::std::string&& value); + #endif + void set_description(const char* value); + void set_description(const char* value, size_t size); + ::std::string* mutable_description(); + ::std::string* release_description(); + void set_allocated_description(::std::string* description); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_description(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_description( + ::std::string* description); + + // .tensorflow.AttrValue default_value = 3; + bool has_default_value() const; + void clear_default_value(); + static const int kDefaultValueFieldNumber = 3; + private: + const ::tensorflow::AttrValue& _internal_default_value() const; + public: + const ::tensorflow::AttrValue& default_value() const; + ::tensorflow::AttrValue* release_default_value(); + ::tensorflow::AttrValue* mutable_default_value(); + void set_allocated_default_value(::tensorflow::AttrValue* default_value); + void unsafe_arena_set_allocated_default_value( + ::tensorflow::AttrValue* default_value); + ::tensorflow::AttrValue* unsafe_arena_release_default_value(); + + // @@protoc_insertion_point(class_scope:tensorflow.ApiDef.Attr) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::internal::ArenaStringPtr name_; + ::google::protobuf::internal::ArenaStringPtr rename_to_; + ::google::protobuf::internal::ArenaStringPtr description_; + ::tensorflow::AttrValue* default_value_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fapi_5fdef_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class ApiDef : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.ApiDef) */ { + public: + ApiDef(); + virtual ~ApiDef(); + + ApiDef(const ApiDef& from); + + inline ApiDef& operator=(const ApiDef& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + ApiDef(ApiDef&& from) noexcept + : ApiDef() { + *this = ::std::move(from); + } + + inline ApiDef& operator=(ApiDef&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const ApiDef& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const ApiDef* internal_default_instance() { + return reinterpret_cast( + &_ApiDef_default_instance_); + } + static constexpr int kIndexInFileMessages = + 3; + + void UnsafeArenaSwap(ApiDef* other); + void Swap(ApiDef* other); + friend void swap(ApiDef& a, ApiDef& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline ApiDef* New() const final { + return CreateMaybeMessage(NULL); + } + + ApiDef* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const ApiDef& from); + void MergeFrom(const ApiDef& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(ApiDef* other); + protected: + explicit ApiDef(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + typedef ApiDef_Endpoint Endpoint; + typedef ApiDef_Arg Arg; + typedef ApiDef_Attr Attr; + + typedef ApiDef_Visibility Visibility; + static const Visibility DEFAULT_VISIBILITY = + ApiDef_Visibility_DEFAULT_VISIBILITY; + static const Visibility VISIBLE = + ApiDef_Visibility_VISIBLE; + static const Visibility SKIP = + ApiDef_Visibility_SKIP; + static const Visibility HIDDEN = + ApiDef_Visibility_HIDDEN; + static inline bool Visibility_IsValid(int value) { + return ApiDef_Visibility_IsValid(value); + } + static const Visibility Visibility_MIN = + ApiDef_Visibility_Visibility_MIN; + static const Visibility Visibility_MAX = + ApiDef_Visibility_Visibility_MAX; + static const int Visibility_ARRAYSIZE = + ApiDef_Visibility_Visibility_ARRAYSIZE; + static inline const ::google::protobuf::EnumDescriptor* + Visibility_descriptor() { + return ApiDef_Visibility_descriptor(); + } + static inline const ::std::string& Visibility_Name(Visibility value) { + return ApiDef_Visibility_Name(value); + } + static inline bool Visibility_Parse(const ::std::string& name, + Visibility* value) { + return ApiDef_Visibility_Parse(name, value); + } + + // accessors ------------------------------------------------------- + + // repeated .tensorflow.ApiDef.Endpoint endpoint = 3; + int endpoint_size() const; + void clear_endpoint(); + static const int kEndpointFieldNumber = 3; + ::tensorflow::ApiDef_Endpoint* mutable_endpoint(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::ApiDef_Endpoint >* + mutable_endpoint(); + const ::tensorflow::ApiDef_Endpoint& endpoint(int index) const; + ::tensorflow::ApiDef_Endpoint* add_endpoint(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::ApiDef_Endpoint >& + endpoint() const; + + // repeated .tensorflow.ApiDef.Arg in_arg = 4; + int in_arg_size() const; + void clear_in_arg(); + static const int kInArgFieldNumber = 4; + ::tensorflow::ApiDef_Arg* mutable_in_arg(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::ApiDef_Arg >* + mutable_in_arg(); + const ::tensorflow::ApiDef_Arg& in_arg(int index) const; + ::tensorflow::ApiDef_Arg* add_in_arg(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::ApiDef_Arg >& + in_arg() const; + + // repeated .tensorflow.ApiDef.Arg out_arg = 5; + int out_arg_size() const; + void clear_out_arg(); + static const int kOutArgFieldNumber = 5; + ::tensorflow::ApiDef_Arg* mutable_out_arg(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::ApiDef_Arg >* + mutable_out_arg(); + const ::tensorflow::ApiDef_Arg& out_arg(int index) const; + ::tensorflow::ApiDef_Arg* add_out_arg(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::ApiDef_Arg >& + out_arg() const; + + // repeated .tensorflow.ApiDef.Attr attr = 6; + int attr_size() const; + void clear_attr(); + static const int kAttrFieldNumber = 6; + ::tensorflow::ApiDef_Attr* mutable_attr(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::ApiDef_Attr >* + mutable_attr(); + const ::tensorflow::ApiDef_Attr& attr(int index) const; + ::tensorflow::ApiDef_Attr* add_attr(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::ApiDef_Attr >& + attr() const; + + // repeated string arg_order = 11; + int arg_order_size() const; + void clear_arg_order(); + static const int kArgOrderFieldNumber = 11; + const ::std::string& arg_order(int index) const; + ::std::string* mutable_arg_order(int index); + void set_arg_order(int index, const ::std::string& value); + #if LANG_CXX11 + void set_arg_order(int index, ::std::string&& value); + #endif + void set_arg_order(int index, const char* value); + void set_arg_order(int index, const char* value, size_t size); + ::std::string* add_arg_order(); + void add_arg_order(const ::std::string& value); + #if LANG_CXX11 + void add_arg_order(::std::string&& value); + #endif + void add_arg_order(const char* value); + void add_arg_order(const char* value, size_t size); + const ::google::protobuf::RepeatedPtrField< ::std::string>& arg_order() const; + ::google::protobuf::RepeatedPtrField< ::std::string>* mutable_arg_order(); + + // string graph_op_name = 1; + void clear_graph_op_name(); + static const int kGraphOpNameFieldNumber = 1; + const ::std::string& graph_op_name() const; + void set_graph_op_name(const ::std::string& value); + #if LANG_CXX11 + void set_graph_op_name(::std::string&& value); + #endif + void set_graph_op_name(const char* value); + void set_graph_op_name(const char* value, size_t size); + ::std::string* mutable_graph_op_name(); + ::std::string* release_graph_op_name(); + void set_allocated_graph_op_name(::std::string* graph_op_name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_graph_op_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_graph_op_name( + ::std::string* graph_op_name); + + // string summary = 7; + void clear_summary(); + static const int kSummaryFieldNumber = 7; + const ::std::string& summary() const; + void set_summary(const ::std::string& value); + #if LANG_CXX11 + void set_summary(::std::string&& value); + #endif + void set_summary(const char* value); + void set_summary(const char* value, size_t size); + ::std::string* mutable_summary(); + ::std::string* release_summary(); + void set_allocated_summary(::std::string* summary); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_summary(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_summary( + ::std::string* summary); + + // string description = 8; + void clear_description(); + static const int kDescriptionFieldNumber = 8; + const ::std::string& description() const; + void set_description(const ::std::string& value); + #if LANG_CXX11 + void set_description(::std::string&& value); + #endif + void set_description(const char* value); + void set_description(const char* value, size_t size); + ::std::string* mutable_description(); + ::std::string* release_description(); + void set_allocated_description(::std::string* description); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_description(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_description( + ::std::string* description); + + // string description_prefix = 9; + void clear_description_prefix(); + static const int kDescriptionPrefixFieldNumber = 9; + const ::std::string& description_prefix() const; + void set_description_prefix(const ::std::string& value); + #if LANG_CXX11 + void set_description_prefix(::std::string&& value); + #endif + void set_description_prefix(const char* value); + void set_description_prefix(const char* value, size_t size); + ::std::string* mutable_description_prefix(); + ::std::string* release_description_prefix(); + void set_allocated_description_prefix(::std::string* description_prefix); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_description_prefix(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_description_prefix( + ::std::string* description_prefix); + + // string description_suffix = 10; + void clear_description_suffix(); + static const int kDescriptionSuffixFieldNumber = 10; + const ::std::string& description_suffix() const; + void set_description_suffix(const ::std::string& value); + #if LANG_CXX11 + void set_description_suffix(::std::string&& value); + #endif + void set_description_suffix(const char* value); + void set_description_suffix(const char* value, size_t size); + ::std::string* mutable_description_suffix(); + ::std::string* release_description_suffix(); + void set_allocated_description_suffix(::std::string* description_suffix); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_description_suffix(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_description_suffix( + ::std::string* description_suffix); + + // string deprecation_message = 12; + void clear_deprecation_message(); + static const int kDeprecationMessageFieldNumber = 12; + const ::std::string& deprecation_message() const; + void set_deprecation_message(const ::std::string& value); + #if LANG_CXX11 + void set_deprecation_message(::std::string&& value); + #endif + void set_deprecation_message(const char* value); + void set_deprecation_message(const char* value, size_t size); + ::std::string* mutable_deprecation_message(); + ::std::string* release_deprecation_message(); + void set_allocated_deprecation_message(::std::string* deprecation_message); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_deprecation_message(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_deprecation_message( + ::std::string* deprecation_message); + + // .tensorflow.ApiDef.Visibility visibility = 2; + void clear_visibility(); + static const int kVisibilityFieldNumber = 2; + ::tensorflow::ApiDef_Visibility visibility() const; + void set_visibility(::tensorflow::ApiDef_Visibility value); + + // @@protoc_insertion_point(class_scope:tensorflow.ApiDef) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::ApiDef_Endpoint > endpoint_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::ApiDef_Arg > in_arg_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::ApiDef_Arg > out_arg_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::ApiDef_Attr > attr_; + ::google::protobuf::RepeatedPtrField< ::std::string> arg_order_; + ::google::protobuf::internal::ArenaStringPtr graph_op_name_; + ::google::protobuf::internal::ArenaStringPtr summary_; + ::google::protobuf::internal::ArenaStringPtr description_; + ::google::protobuf::internal::ArenaStringPtr description_prefix_; + ::google::protobuf::internal::ArenaStringPtr description_suffix_; + ::google::protobuf::internal::ArenaStringPtr deprecation_message_; + int visibility_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fapi_5fdef_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class ApiDefs : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.ApiDefs) */ { + public: + ApiDefs(); + virtual ~ApiDefs(); + + ApiDefs(const ApiDefs& from); + + inline ApiDefs& operator=(const ApiDefs& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + ApiDefs(ApiDefs&& from) noexcept + : ApiDefs() { + *this = ::std::move(from); + } + + inline ApiDefs& operator=(ApiDefs&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const ApiDefs& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const ApiDefs* internal_default_instance() { + return reinterpret_cast( + &_ApiDefs_default_instance_); + } + static constexpr int kIndexInFileMessages = + 4; + + void UnsafeArenaSwap(ApiDefs* other); + void Swap(ApiDefs* other); + friend void swap(ApiDefs& a, ApiDefs& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline ApiDefs* New() const final { + return CreateMaybeMessage(NULL); + } + + ApiDefs* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const ApiDefs& from); + void MergeFrom(const ApiDefs& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(ApiDefs* other); + protected: + explicit ApiDefs(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated .tensorflow.ApiDef op = 1; + int op_size() const; + void clear_op(); + static const int kOpFieldNumber = 1; + ::tensorflow::ApiDef* mutable_op(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::ApiDef >* + mutable_op(); + const ::tensorflow::ApiDef& op(int index) const; + ::tensorflow::ApiDef* add_op(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::ApiDef >& + op() const; + + // @@protoc_insertion_point(class_scope:tensorflow.ApiDefs) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::ApiDef > op_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fapi_5fdef_2eproto::TableStruct; +}; +// =================================================================== + + +// =================================================================== + +#ifdef __GNUC__ + #pragma GCC diagnostic push + #pragma GCC diagnostic ignored "-Wstrict-aliasing" +#endif // __GNUC__ +// ApiDef_Endpoint + +// string name = 1; +inline void ApiDef_Endpoint::clear_name() { + name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& ApiDef_Endpoint::name() const { + // @@protoc_insertion_point(field_get:tensorflow.ApiDef.Endpoint.name) + return name_.Get(); +} +inline void ApiDef_Endpoint::set_name(const ::std::string& value) { + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.ApiDef.Endpoint.name) +} +#if LANG_CXX11 +inline void ApiDef_Endpoint::set_name(::std::string&& value) { + + name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.ApiDef.Endpoint.name) +} +#endif +inline void ApiDef_Endpoint::set_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.ApiDef.Endpoint.name) +} +inline void ApiDef_Endpoint::set_name(const char* value, + size_t size) { + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.ApiDef.Endpoint.name) +} +inline ::std::string* ApiDef_Endpoint::mutable_name() { + + // @@protoc_insertion_point(field_mutable:tensorflow.ApiDef.Endpoint.name) + return name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* ApiDef_Endpoint::release_name() { + // @@protoc_insertion_point(field_release:tensorflow.ApiDef.Endpoint.name) + + return name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void ApiDef_Endpoint::set_allocated_name(::std::string* name) { + if (name != NULL) { + + } else { + + } + name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.ApiDef.Endpoint.name) +} +inline ::std::string* ApiDef_Endpoint::unsafe_arena_release_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.ApiDef.Endpoint.name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void ApiDef_Endpoint::unsafe_arena_set_allocated_name( + ::std::string* name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (name != NULL) { + + } else { + + } + name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.ApiDef.Endpoint.name) +} + +// bool deprecated = 3; +inline void ApiDef_Endpoint::clear_deprecated() { + deprecated_ = false; +} +inline bool ApiDef_Endpoint::deprecated() const { + // @@protoc_insertion_point(field_get:tensorflow.ApiDef.Endpoint.deprecated) + return deprecated_; +} +inline void ApiDef_Endpoint::set_deprecated(bool value) { + + deprecated_ = value; + // @@protoc_insertion_point(field_set:tensorflow.ApiDef.Endpoint.deprecated) +} + +// ------------------------------------------------------------------- + +// ApiDef_Arg + +// string name = 1; +inline void ApiDef_Arg::clear_name() { + name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& ApiDef_Arg::name() const { + // @@protoc_insertion_point(field_get:tensorflow.ApiDef.Arg.name) + return name_.Get(); +} +inline void ApiDef_Arg::set_name(const ::std::string& value) { + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.ApiDef.Arg.name) +} +#if LANG_CXX11 +inline void ApiDef_Arg::set_name(::std::string&& value) { + + name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.ApiDef.Arg.name) +} +#endif +inline void ApiDef_Arg::set_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.ApiDef.Arg.name) +} +inline void ApiDef_Arg::set_name(const char* value, + size_t size) { + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.ApiDef.Arg.name) +} +inline ::std::string* ApiDef_Arg::mutable_name() { + + // @@protoc_insertion_point(field_mutable:tensorflow.ApiDef.Arg.name) + return name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* ApiDef_Arg::release_name() { + // @@protoc_insertion_point(field_release:tensorflow.ApiDef.Arg.name) + + return name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void ApiDef_Arg::set_allocated_name(::std::string* name) { + if (name != NULL) { + + } else { + + } + name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.ApiDef.Arg.name) +} +inline ::std::string* ApiDef_Arg::unsafe_arena_release_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.ApiDef.Arg.name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void ApiDef_Arg::unsafe_arena_set_allocated_name( + ::std::string* name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (name != NULL) { + + } else { + + } + name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.ApiDef.Arg.name) +} + +// string rename_to = 2; +inline void ApiDef_Arg::clear_rename_to() { + rename_to_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& ApiDef_Arg::rename_to() const { + // @@protoc_insertion_point(field_get:tensorflow.ApiDef.Arg.rename_to) + return rename_to_.Get(); +} +inline void ApiDef_Arg::set_rename_to(const ::std::string& value) { + + rename_to_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.ApiDef.Arg.rename_to) +} +#if LANG_CXX11 +inline void ApiDef_Arg::set_rename_to(::std::string&& value) { + + rename_to_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.ApiDef.Arg.rename_to) +} +#endif +inline void ApiDef_Arg::set_rename_to(const char* value) { + GOOGLE_DCHECK(value != NULL); + + rename_to_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.ApiDef.Arg.rename_to) +} +inline void ApiDef_Arg::set_rename_to(const char* value, + size_t size) { + + rename_to_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.ApiDef.Arg.rename_to) +} +inline ::std::string* ApiDef_Arg::mutable_rename_to() { + + // @@protoc_insertion_point(field_mutable:tensorflow.ApiDef.Arg.rename_to) + return rename_to_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* ApiDef_Arg::release_rename_to() { + // @@protoc_insertion_point(field_release:tensorflow.ApiDef.Arg.rename_to) + + return rename_to_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void ApiDef_Arg::set_allocated_rename_to(::std::string* rename_to) { + if (rename_to != NULL) { + + } else { + + } + rename_to_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), rename_to, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.ApiDef.Arg.rename_to) +} +inline ::std::string* ApiDef_Arg::unsafe_arena_release_rename_to() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.ApiDef.Arg.rename_to) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return rename_to_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void ApiDef_Arg::unsafe_arena_set_allocated_rename_to( + ::std::string* rename_to) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (rename_to != NULL) { + + } else { + + } + rename_to_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + rename_to, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.ApiDef.Arg.rename_to) +} + +// string description = 3; +inline void ApiDef_Arg::clear_description() { + description_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& ApiDef_Arg::description() const { + // @@protoc_insertion_point(field_get:tensorflow.ApiDef.Arg.description) + return description_.Get(); +} +inline void ApiDef_Arg::set_description(const ::std::string& value) { + + description_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.ApiDef.Arg.description) +} +#if LANG_CXX11 +inline void ApiDef_Arg::set_description(::std::string&& value) { + + description_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.ApiDef.Arg.description) +} +#endif +inline void ApiDef_Arg::set_description(const char* value) { + GOOGLE_DCHECK(value != NULL); + + description_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.ApiDef.Arg.description) +} +inline void ApiDef_Arg::set_description(const char* value, + size_t size) { + + description_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.ApiDef.Arg.description) +} +inline ::std::string* ApiDef_Arg::mutable_description() { + + // @@protoc_insertion_point(field_mutable:tensorflow.ApiDef.Arg.description) + return description_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* ApiDef_Arg::release_description() { + // @@protoc_insertion_point(field_release:tensorflow.ApiDef.Arg.description) + + return description_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void ApiDef_Arg::set_allocated_description(::std::string* description) { + if (description != NULL) { + + } else { + + } + description_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), description, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.ApiDef.Arg.description) +} +inline ::std::string* ApiDef_Arg::unsafe_arena_release_description() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.ApiDef.Arg.description) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return description_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void ApiDef_Arg::unsafe_arena_set_allocated_description( + ::std::string* description) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (description != NULL) { + + } else { + + } + description_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + description, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.ApiDef.Arg.description) +} + +// ------------------------------------------------------------------- + +// ApiDef_Attr + +// string name = 1; +inline void ApiDef_Attr::clear_name() { + name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& ApiDef_Attr::name() const { + // @@protoc_insertion_point(field_get:tensorflow.ApiDef.Attr.name) + return name_.Get(); +} +inline void ApiDef_Attr::set_name(const ::std::string& value) { + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.ApiDef.Attr.name) +} +#if LANG_CXX11 +inline void ApiDef_Attr::set_name(::std::string&& value) { + + name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.ApiDef.Attr.name) +} +#endif +inline void ApiDef_Attr::set_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.ApiDef.Attr.name) +} +inline void ApiDef_Attr::set_name(const char* value, + size_t size) { + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.ApiDef.Attr.name) +} +inline ::std::string* ApiDef_Attr::mutable_name() { + + // @@protoc_insertion_point(field_mutable:tensorflow.ApiDef.Attr.name) + return name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* ApiDef_Attr::release_name() { + // @@protoc_insertion_point(field_release:tensorflow.ApiDef.Attr.name) + + return name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void ApiDef_Attr::set_allocated_name(::std::string* name) { + if (name != NULL) { + + } else { + + } + name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.ApiDef.Attr.name) +} +inline ::std::string* ApiDef_Attr::unsafe_arena_release_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.ApiDef.Attr.name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void ApiDef_Attr::unsafe_arena_set_allocated_name( + ::std::string* name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (name != NULL) { + + } else { + + } + name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.ApiDef.Attr.name) +} + +// string rename_to = 2; +inline void ApiDef_Attr::clear_rename_to() { + rename_to_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& ApiDef_Attr::rename_to() const { + // @@protoc_insertion_point(field_get:tensorflow.ApiDef.Attr.rename_to) + return rename_to_.Get(); +} +inline void ApiDef_Attr::set_rename_to(const ::std::string& value) { + + rename_to_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.ApiDef.Attr.rename_to) +} +#if LANG_CXX11 +inline void ApiDef_Attr::set_rename_to(::std::string&& value) { + + rename_to_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.ApiDef.Attr.rename_to) +} +#endif +inline void ApiDef_Attr::set_rename_to(const char* value) { + GOOGLE_DCHECK(value != NULL); + + rename_to_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.ApiDef.Attr.rename_to) +} +inline void ApiDef_Attr::set_rename_to(const char* value, + size_t size) { + + rename_to_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.ApiDef.Attr.rename_to) +} +inline ::std::string* ApiDef_Attr::mutable_rename_to() { + + // @@protoc_insertion_point(field_mutable:tensorflow.ApiDef.Attr.rename_to) + return rename_to_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* ApiDef_Attr::release_rename_to() { + // @@protoc_insertion_point(field_release:tensorflow.ApiDef.Attr.rename_to) + + return rename_to_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void ApiDef_Attr::set_allocated_rename_to(::std::string* rename_to) { + if (rename_to != NULL) { + + } else { + + } + rename_to_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), rename_to, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.ApiDef.Attr.rename_to) +} +inline ::std::string* ApiDef_Attr::unsafe_arena_release_rename_to() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.ApiDef.Attr.rename_to) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return rename_to_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void ApiDef_Attr::unsafe_arena_set_allocated_rename_to( + ::std::string* rename_to) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (rename_to != NULL) { + + } else { + + } + rename_to_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + rename_to, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.ApiDef.Attr.rename_to) +} + +// .tensorflow.AttrValue default_value = 3; +inline bool ApiDef_Attr::has_default_value() const { + return this != internal_default_instance() && default_value_ != NULL; +} +inline const ::tensorflow::AttrValue& ApiDef_Attr::_internal_default_value() const { + return *default_value_; +} +inline const ::tensorflow::AttrValue& ApiDef_Attr::default_value() const { + const ::tensorflow::AttrValue* p = default_value_; + // @@protoc_insertion_point(field_get:tensorflow.ApiDef.Attr.default_value) + return p != NULL ? *p : *reinterpret_cast( + &::tensorflow::_AttrValue_default_instance_); +} +inline ::tensorflow::AttrValue* ApiDef_Attr::release_default_value() { + // @@protoc_insertion_point(field_release:tensorflow.ApiDef.Attr.default_value) + + ::tensorflow::AttrValue* temp = default_value_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + default_value_ = NULL; + return temp; +} +inline ::tensorflow::AttrValue* ApiDef_Attr::unsafe_arena_release_default_value() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.ApiDef.Attr.default_value) + + ::tensorflow::AttrValue* temp = default_value_; + default_value_ = NULL; + return temp; +} +inline ::tensorflow::AttrValue* ApiDef_Attr::mutable_default_value() { + + if (default_value_ == NULL) { + auto* p = CreateMaybeMessage<::tensorflow::AttrValue>(GetArenaNoVirtual()); + default_value_ = p; + } + // @@protoc_insertion_point(field_mutable:tensorflow.ApiDef.Attr.default_value) + return default_value_; +} +inline void ApiDef_Attr::set_allocated_default_value(::tensorflow::AttrValue* default_value) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(default_value_); + } + if (default_value) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(default_value)->GetArena(); + if (message_arena != submessage_arena) { + default_value = ::google::protobuf::internal::GetOwnedMessage( + message_arena, default_value, submessage_arena); + } + + } else { + + } + default_value_ = default_value; + // @@protoc_insertion_point(field_set_allocated:tensorflow.ApiDef.Attr.default_value) +} + +// string description = 4; +inline void ApiDef_Attr::clear_description() { + description_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& ApiDef_Attr::description() const { + // @@protoc_insertion_point(field_get:tensorflow.ApiDef.Attr.description) + return description_.Get(); +} +inline void ApiDef_Attr::set_description(const ::std::string& value) { + + description_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.ApiDef.Attr.description) +} +#if LANG_CXX11 +inline void ApiDef_Attr::set_description(::std::string&& value) { + + description_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.ApiDef.Attr.description) +} +#endif +inline void ApiDef_Attr::set_description(const char* value) { + GOOGLE_DCHECK(value != NULL); + + description_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.ApiDef.Attr.description) +} +inline void ApiDef_Attr::set_description(const char* value, + size_t size) { + + description_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.ApiDef.Attr.description) +} +inline ::std::string* ApiDef_Attr::mutable_description() { + + // @@protoc_insertion_point(field_mutable:tensorflow.ApiDef.Attr.description) + return description_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* ApiDef_Attr::release_description() { + // @@protoc_insertion_point(field_release:tensorflow.ApiDef.Attr.description) + + return description_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void ApiDef_Attr::set_allocated_description(::std::string* description) { + if (description != NULL) { + + } else { + + } + description_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), description, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.ApiDef.Attr.description) +} +inline ::std::string* ApiDef_Attr::unsafe_arena_release_description() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.ApiDef.Attr.description) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return description_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void ApiDef_Attr::unsafe_arena_set_allocated_description( + ::std::string* description) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (description != NULL) { + + } else { + + } + description_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + description, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.ApiDef.Attr.description) +} + +// ------------------------------------------------------------------- + +// ApiDef + +// string graph_op_name = 1; +inline void ApiDef::clear_graph_op_name() { + graph_op_name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& ApiDef::graph_op_name() const { + // @@protoc_insertion_point(field_get:tensorflow.ApiDef.graph_op_name) + return graph_op_name_.Get(); +} +inline void ApiDef::set_graph_op_name(const ::std::string& value) { + + graph_op_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.ApiDef.graph_op_name) +} +#if LANG_CXX11 +inline void ApiDef::set_graph_op_name(::std::string&& value) { + + graph_op_name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.ApiDef.graph_op_name) +} +#endif +inline void ApiDef::set_graph_op_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + graph_op_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.ApiDef.graph_op_name) +} +inline void ApiDef::set_graph_op_name(const char* value, + size_t size) { + + graph_op_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.ApiDef.graph_op_name) +} +inline ::std::string* ApiDef::mutable_graph_op_name() { + + // @@protoc_insertion_point(field_mutable:tensorflow.ApiDef.graph_op_name) + return graph_op_name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* ApiDef::release_graph_op_name() { + // @@protoc_insertion_point(field_release:tensorflow.ApiDef.graph_op_name) + + return graph_op_name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void ApiDef::set_allocated_graph_op_name(::std::string* graph_op_name) { + if (graph_op_name != NULL) { + + } else { + + } + graph_op_name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), graph_op_name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.ApiDef.graph_op_name) +} +inline ::std::string* ApiDef::unsafe_arena_release_graph_op_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.ApiDef.graph_op_name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return graph_op_name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void ApiDef::unsafe_arena_set_allocated_graph_op_name( + ::std::string* graph_op_name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (graph_op_name != NULL) { + + } else { + + } + graph_op_name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + graph_op_name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.ApiDef.graph_op_name) +} + +// string deprecation_message = 12; +inline void ApiDef::clear_deprecation_message() { + deprecation_message_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& ApiDef::deprecation_message() const { + // @@protoc_insertion_point(field_get:tensorflow.ApiDef.deprecation_message) + return deprecation_message_.Get(); +} +inline void ApiDef::set_deprecation_message(const ::std::string& value) { + + deprecation_message_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.ApiDef.deprecation_message) +} +#if LANG_CXX11 +inline void ApiDef::set_deprecation_message(::std::string&& value) { + + deprecation_message_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.ApiDef.deprecation_message) +} +#endif +inline void ApiDef::set_deprecation_message(const char* value) { + GOOGLE_DCHECK(value != NULL); + + deprecation_message_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.ApiDef.deprecation_message) +} +inline void ApiDef::set_deprecation_message(const char* value, + size_t size) { + + deprecation_message_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.ApiDef.deprecation_message) +} +inline ::std::string* ApiDef::mutable_deprecation_message() { + + // @@protoc_insertion_point(field_mutable:tensorflow.ApiDef.deprecation_message) + return deprecation_message_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* ApiDef::release_deprecation_message() { + // @@protoc_insertion_point(field_release:tensorflow.ApiDef.deprecation_message) + + return deprecation_message_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void ApiDef::set_allocated_deprecation_message(::std::string* deprecation_message) { + if (deprecation_message != NULL) { + + } else { + + } + deprecation_message_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), deprecation_message, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.ApiDef.deprecation_message) +} +inline ::std::string* ApiDef::unsafe_arena_release_deprecation_message() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.ApiDef.deprecation_message) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return deprecation_message_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void ApiDef::unsafe_arena_set_allocated_deprecation_message( + ::std::string* deprecation_message) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (deprecation_message != NULL) { + + } else { + + } + deprecation_message_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + deprecation_message, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.ApiDef.deprecation_message) +} + +// .tensorflow.ApiDef.Visibility visibility = 2; +inline void ApiDef::clear_visibility() { + visibility_ = 0; +} +inline ::tensorflow::ApiDef_Visibility ApiDef::visibility() const { + // @@protoc_insertion_point(field_get:tensorflow.ApiDef.visibility) + return static_cast< ::tensorflow::ApiDef_Visibility >(visibility_); +} +inline void ApiDef::set_visibility(::tensorflow::ApiDef_Visibility value) { + + visibility_ = value; + // @@protoc_insertion_point(field_set:tensorflow.ApiDef.visibility) +} + +// repeated .tensorflow.ApiDef.Endpoint endpoint = 3; +inline int ApiDef::endpoint_size() const { + return endpoint_.size(); +} +inline void ApiDef::clear_endpoint() { + endpoint_.Clear(); +} +inline ::tensorflow::ApiDef_Endpoint* ApiDef::mutable_endpoint(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.ApiDef.endpoint) + return endpoint_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::ApiDef_Endpoint >* +ApiDef::mutable_endpoint() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.ApiDef.endpoint) + return &endpoint_; +} +inline const ::tensorflow::ApiDef_Endpoint& ApiDef::endpoint(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.ApiDef.endpoint) + return endpoint_.Get(index); +} +inline ::tensorflow::ApiDef_Endpoint* ApiDef::add_endpoint() { + // @@protoc_insertion_point(field_add:tensorflow.ApiDef.endpoint) + return endpoint_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::ApiDef_Endpoint >& +ApiDef::endpoint() const { + // @@protoc_insertion_point(field_list:tensorflow.ApiDef.endpoint) + return endpoint_; +} + +// repeated .tensorflow.ApiDef.Arg in_arg = 4; +inline int ApiDef::in_arg_size() const { + return in_arg_.size(); +} +inline void ApiDef::clear_in_arg() { + in_arg_.Clear(); +} +inline ::tensorflow::ApiDef_Arg* ApiDef::mutable_in_arg(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.ApiDef.in_arg) + return in_arg_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::ApiDef_Arg >* +ApiDef::mutable_in_arg() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.ApiDef.in_arg) + return &in_arg_; +} +inline const ::tensorflow::ApiDef_Arg& ApiDef::in_arg(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.ApiDef.in_arg) + return in_arg_.Get(index); +} +inline ::tensorflow::ApiDef_Arg* ApiDef::add_in_arg() { + // @@protoc_insertion_point(field_add:tensorflow.ApiDef.in_arg) + return in_arg_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::ApiDef_Arg >& +ApiDef::in_arg() const { + // @@protoc_insertion_point(field_list:tensorflow.ApiDef.in_arg) + return in_arg_; +} + +// repeated .tensorflow.ApiDef.Arg out_arg = 5; +inline int ApiDef::out_arg_size() const { + return out_arg_.size(); +} +inline void ApiDef::clear_out_arg() { + out_arg_.Clear(); +} +inline ::tensorflow::ApiDef_Arg* ApiDef::mutable_out_arg(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.ApiDef.out_arg) + return out_arg_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::ApiDef_Arg >* +ApiDef::mutable_out_arg() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.ApiDef.out_arg) + return &out_arg_; +} +inline const ::tensorflow::ApiDef_Arg& ApiDef::out_arg(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.ApiDef.out_arg) + return out_arg_.Get(index); +} +inline ::tensorflow::ApiDef_Arg* ApiDef::add_out_arg() { + // @@protoc_insertion_point(field_add:tensorflow.ApiDef.out_arg) + return out_arg_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::ApiDef_Arg >& +ApiDef::out_arg() const { + // @@protoc_insertion_point(field_list:tensorflow.ApiDef.out_arg) + return out_arg_; +} + +// repeated string arg_order = 11; +inline int ApiDef::arg_order_size() const { + return arg_order_.size(); +} +inline void ApiDef::clear_arg_order() { + arg_order_.Clear(); +} +inline const ::std::string& ApiDef::arg_order(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.ApiDef.arg_order) + return arg_order_.Get(index); +} +inline ::std::string* ApiDef::mutable_arg_order(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.ApiDef.arg_order) + return arg_order_.Mutable(index); +} +inline void ApiDef::set_arg_order(int index, const ::std::string& value) { + // @@protoc_insertion_point(field_set:tensorflow.ApiDef.arg_order) + arg_order_.Mutable(index)->assign(value); +} +#if LANG_CXX11 +inline void ApiDef::set_arg_order(int index, ::std::string&& value) { + // @@protoc_insertion_point(field_set:tensorflow.ApiDef.arg_order) + arg_order_.Mutable(index)->assign(std::move(value)); +} +#endif +inline void ApiDef::set_arg_order(int index, const char* value) { + GOOGLE_DCHECK(value != NULL); + arg_order_.Mutable(index)->assign(value); + // @@protoc_insertion_point(field_set_char:tensorflow.ApiDef.arg_order) +} +inline void ApiDef::set_arg_order(int index, const char* value, size_t size) { + arg_order_.Mutable(index)->assign( + reinterpret_cast(value), size); + // @@protoc_insertion_point(field_set_pointer:tensorflow.ApiDef.arg_order) +} +inline ::std::string* ApiDef::add_arg_order() { + // @@protoc_insertion_point(field_add_mutable:tensorflow.ApiDef.arg_order) + return arg_order_.Add(); +} +inline void ApiDef::add_arg_order(const ::std::string& value) { + arg_order_.Add()->assign(value); + // @@protoc_insertion_point(field_add:tensorflow.ApiDef.arg_order) +} +#if LANG_CXX11 +inline void ApiDef::add_arg_order(::std::string&& value) { + arg_order_.Add(std::move(value)); + // @@protoc_insertion_point(field_add:tensorflow.ApiDef.arg_order) +} +#endif +inline void ApiDef::add_arg_order(const char* value) { + GOOGLE_DCHECK(value != NULL); + arg_order_.Add()->assign(value); + // @@protoc_insertion_point(field_add_char:tensorflow.ApiDef.arg_order) +} +inline void ApiDef::add_arg_order(const char* value, size_t size) { + arg_order_.Add()->assign(reinterpret_cast(value), size); + // @@protoc_insertion_point(field_add_pointer:tensorflow.ApiDef.arg_order) +} +inline const ::google::protobuf::RepeatedPtrField< ::std::string>& +ApiDef::arg_order() const { + // @@protoc_insertion_point(field_list:tensorflow.ApiDef.arg_order) + return arg_order_; +} +inline ::google::protobuf::RepeatedPtrField< ::std::string>* +ApiDef::mutable_arg_order() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.ApiDef.arg_order) + return &arg_order_; +} + +// repeated .tensorflow.ApiDef.Attr attr = 6; +inline int ApiDef::attr_size() const { + return attr_.size(); +} +inline void ApiDef::clear_attr() { + attr_.Clear(); +} +inline ::tensorflow::ApiDef_Attr* ApiDef::mutable_attr(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.ApiDef.attr) + return attr_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::ApiDef_Attr >* +ApiDef::mutable_attr() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.ApiDef.attr) + return &attr_; +} +inline const ::tensorflow::ApiDef_Attr& ApiDef::attr(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.ApiDef.attr) + return attr_.Get(index); +} +inline ::tensorflow::ApiDef_Attr* ApiDef::add_attr() { + // @@protoc_insertion_point(field_add:tensorflow.ApiDef.attr) + return attr_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::ApiDef_Attr >& +ApiDef::attr() const { + // @@protoc_insertion_point(field_list:tensorflow.ApiDef.attr) + return attr_; +} + +// string summary = 7; +inline void ApiDef::clear_summary() { + summary_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& ApiDef::summary() const { + // @@protoc_insertion_point(field_get:tensorflow.ApiDef.summary) + return summary_.Get(); +} +inline void ApiDef::set_summary(const ::std::string& value) { + + summary_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.ApiDef.summary) +} +#if LANG_CXX11 +inline void ApiDef::set_summary(::std::string&& value) { + + summary_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.ApiDef.summary) +} +#endif +inline void ApiDef::set_summary(const char* value) { + GOOGLE_DCHECK(value != NULL); + + summary_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.ApiDef.summary) +} +inline void ApiDef::set_summary(const char* value, + size_t size) { + + summary_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.ApiDef.summary) +} +inline ::std::string* ApiDef::mutable_summary() { + + // @@protoc_insertion_point(field_mutable:tensorflow.ApiDef.summary) + return summary_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* ApiDef::release_summary() { + // @@protoc_insertion_point(field_release:tensorflow.ApiDef.summary) + + return summary_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void ApiDef::set_allocated_summary(::std::string* summary) { + if (summary != NULL) { + + } else { + + } + summary_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), summary, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.ApiDef.summary) +} +inline ::std::string* ApiDef::unsafe_arena_release_summary() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.ApiDef.summary) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return summary_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void ApiDef::unsafe_arena_set_allocated_summary( + ::std::string* summary) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (summary != NULL) { + + } else { + + } + summary_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + summary, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.ApiDef.summary) +} + +// string description = 8; +inline void ApiDef::clear_description() { + description_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& ApiDef::description() const { + // @@protoc_insertion_point(field_get:tensorflow.ApiDef.description) + return description_.Get(); +} +inline void ApiDef::set_description(const ::std::string& value) { + + description_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.ApiDef.description) +} +#if LANG_CXX11 +inline void ApiDef::set_description(::std::string&& value) { + + description_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.ApiDef.description) +} +#endif +inline void ApiDef::set_description(const char* value) { + GOOGLE_DCHECK(value != NULL); + + description_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.ApiDef.description) +} +inline void ApiDef::set_description(const char* value, + size_t size) { + + description_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.ApiDef.description) +} +inline ::std::string* ApiDef::mutable_description() { + + // @@protoc_insertion_point(field_mutable:tensorflow.ApiDef.description) + return description_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* ApiDef::release_description() { + // @@protoc_insertion_point(field_release:tensorflow.ApiDef.description) + + return description_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void ApiDef::set_allocated_description(::std::string* description) { + if (description != NULL) { + + } else { + + } + description_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), description, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.ApiDef.description) +} +inline ::std::string* ApiDef::unsafe_arena_release_description() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.ApiDef.description) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return description_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void ApiDef::unsafe_arena_set_allocated_description( + ::std::string* description) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (description != NULL) { + + } else { + + } + description_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + description, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.ApiDef.description) +} + +// string description_prefix = 9; +inline void ApiDef::clear_description_prefix() { + description_prefix_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& ApiDef::description_prefix() const { + // @@protoc_insertion_point(field_get:tensorflow.ApiDef.description_prefix) + return description_prefix_.Get(); +} +inline void ApiDef::set_description_prefix(const ::std::string& value) { + + description_prefix_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.ApiDef.description_prefix) +} +#if LANG_CXX11 +inline void ApiDef::set_description_prefix(::std::string&& value) { + + description_prefix_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.ApiDef.description_prefix) +} +#endif +inline void ApiDef::set_description_prefix(const char* value) { + GOOGLE_DCHECK(value != NULL); + + description_prefix_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.ApiDef.description_prefix) +} +inline void ApiDef::set_description_prefix(const char* value, + size_t size) { + + description_prefix_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.ApiDef.description_prefix) +} +inline ::std::string* ApiDef::mutable_description_prefix() { + + // @@protoc_insertion_point(field_mutable:tensorflow.ApiDef.description_prefix) + return description_prefix_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* ApiDef::release_description_prefix() { + // @@protoc_insertion_point(field_release:tensorflow.ApiDef.description_prefix) + + return description_prefix_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void ApiDef::set_allocated_description_prefix(::std::string* description_prefix) { + if (description_prefix != NULL) { + + } else { + + } + description_prefix_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), description_prefix, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.ApiDef.description_prefix) +} +inline ::std::string* ApiDef::unsafe_arena_release_description_prefix() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.ApiDef.description_prefix) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return description_prefix_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void ApiDef::unsafe_arena_set_allocated_description_prefix( + ::std::string* description_prefix) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (description_prefix != NULL) { + + } else { + + } + description_prefix_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + description_prefix, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.ApiDef.description_prefix) +} + +// string description_suffix = 10; +inline void ApiDef::clear_description_suffix() { + description_suffix_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& ApiDef::description_suffix() const { + // @@protoc_insertion_point(field_get:tensorflow.ApiDef.description_suffix) + return description_suffix_.Get(); +} +inline void ApiDef::set_description_suffix(const ::std::string& value) { + + description_suffix_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.ApiDef.description_suffix) +} +#if LANG_CXX11 +inline void ApiDef::set_description_suffix(::std::string&& value) { + + description_suffix_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.ApiDef.description_suffix) +} +#endif +inline void ApiDef::set_description_suffix(const char* value) { + GOOGLE_DCHECK(value != NULL); + + description_suffix_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.ApiDef.description_suffix) +} +inline void ApiDef::set_description_suffix(const char* value, + size_t size) { + + description_suffix_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.ApiDef.description_suffix) +} +inline ::std::string* ApiDef::mutable_description_suffix() { + + // @@protoc_insertion_point(field_mutable:tensorflow.ApiDef.description_suffix) + return description_suffix_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* ApiDef::release_description_suffix() { + // @@protoc_insertion_point(field_release:tensorflow.ApiDef.description_suffix) + + return description_suffix_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void ApiDef::set_allocated_description_suffix(::std::string* description_suffix) { + if (description_suffix != NULL) { + + } else { + + } + description_suffix_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), description_suffix, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.ApiDef.description_suffix) +} +inline ::std::string* ApiDef::unsafe_arena_release_description_suffix() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.ApiDef.description_suffix) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return description_suffix_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void ApiDef::unsafe_arena_set_allocated_description_suffix( + ::std::string* description_suffix) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (description_suffix != NULL) { + + } else { + + } + description_suffix_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + description_suffix, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.ApiDef.description_suffix) +} + +// ------------------------------------------------------------------- + +// ApiDefs + +// repeated .tensorflow.ApiDef op = 1; +inline int ApiDefs::op_size() const { + return op_.size(); +} +inline void ApiDefs::clear_op() { + op_.Clear(); +} +inline ::tensorflow::ApiDef* ApiDefs::mutable_op(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.ApiDefs.op) + return op_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::ApiDef >* +ApiDefs::mutable_op() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.ApiDefs.op) + return &op_; +} +inline const ::tensorflow::ApiDef& ApiDefs::op(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.ApiDefs.op) + return op_.Get(index); +} +inline ::tensorflow::ApiDef* ApiDefs::add_op() { + // @@protoc_insertion_point(field_add:tensorflow.ApiDefs.op) + return op_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::ApiDef >& +ApiDefs::op() const { + // @@protoc_insertion_point(field_list:tensorflow.ApiDefs.op) + return op_; +} + +#ifdef __GNUC__ + #pragma GCC diagnostic pop +#endif // __GNUC__ +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + + +// @@protoc_insertion_point(namespace_scope) + +} // namespace tensorflow + +namespace google { +namespace protobuf { + +template <> struct is_proto_enum< ::tensorflow::ApiDef_Visibility> : ::std::true_type {}; +template <> +inline const EnumDescriptor* GetEnumDescriptor< ::tensorflow::ApiDef_Visibility>() { + return ::tensorflow::ApiDef_Visibility_descriptor(); +} + +} // namespace protobuf +} // namespace google + +// @@protoc_insertion_point(global_scope) + +#endif // PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2fapi_5fdef_2eproto diff --git a/linux/include/tensorflow/core/framework/api_def.pb_text-impl.h b/linux/include/tensorflow/core/framework/api_def.pb_text-impl.h new file mode 100644 index 000000000..dc7b2033f --- /dev/null +++ b/linux/include/tensorflow/core/framework/api_def.pb_text-impl.h @@ -0,0 +1,63 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_framework_api_def_proto_IMPL_H_ +#define tensorflow_core_framework_api_def_proto_IMPL_H_ + +#include "tensorflow/core/framework/api_def.pb.h" +#include "tensorflow/core/framework/api_def.pb_text.h" +#include "tensorflow/core/framework/attr_value.pb.h" +#include "tensorflow/core/framework/attr_value.pb_text-impl.h" +#include "tensorflow/core/framework/resource_handle.pb.h" +#include "tensorflow/core/framework/resource_handle.pb_text-impl.h" +#include "tensorflow/core/framework/tensor.pb.h" +#include "tensorflow/core/framework/tensor.pb_text-impl.h" +#include "tensorflow/core/framework/tensor_shape.pb.h" +#include "tensorflow/core/framework/tensor_shape.pb_text-impl.h" +#include "tensorflow/core/framework/types.pb.h" +#include "tensorflow/core/framework/types.pb_text-impl.h" +#include "tensorflow/core/lib/strings/proto_text_util.h" +#include "tensorflow/core/lib/strings/scanner.h" + +namespace tensorflow { + +namespace internal { + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::ApiDef_Endpoint& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::ApiDef_Endpoint* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::ApiDef_Arg& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::ApiDef_Arg* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::ApiDef_Attr& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::ApiDef_Attr* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::ApiDef& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::ApiDef* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::ApiDefs& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::ApiDefs* msg); + +} // namespace internal + +} // namespace tensorflow + +#endif // tensorflow_core_framework_api_def_proto_IMPL_H_ diff --git a/linux/include/tensorflow/core/framework/api_def.pb_text.h b/linux/include/tensorflow/core/framework/api_def.pb_text.h new file mode 100644 index 000000000..f0d880ebb --- /dev/null +++ b/linux/include/tensorflow/core/framework/api_def.pb_text.h @@ -0,0 +1,68 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_framework_api_def_proto_H_ +#define tensorflow_core_framework_api_def_proto_H_ + +#include "tensorflow/core/framework/api_def.pb.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/protobuf.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +// Enum text output for tensorflow.ApiDef.Visibility +const char* EnumName_ApiDef_Visibility( + ::tensorflow::ApiDef_Visibility value); + +// Message-text conversion for tensorflow.ApiDef.Endpoint +string ProtoDebugString( + const ::tensorflow::ApiDef_Endpoint& msg); +string ProtoShortDebugString( + const ::tensorflow::ApiDef_Endpoint& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::ApiDef_Endpoint* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.ApiDef.Arg +string ProtoDebugString( + const ::tensorflow::ApiDef_Arg& msg); +string ProtoShortDebugString( + const ::tensorflow::ApiDef_Arg& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::ApiDef_Arg* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.ApiDef.Attr +string ProtoDebugString( + const ::tensorflow::ApiDef_Attr& msg); +string ProtoShortDebugString( + const ::tensorflow::ApiDef_Attr& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::ApiDef_Attr* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.ApiDef +string ProtoDebugString( + const ::tensorflow::ApiDef& msg); +string ProtoShortDebugString( + const ::tensorflow::ApiDef& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::ApiDef* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.ApiDefs +string ProtoDebugString( + const ::tensorflow::ApiDefs& msg); +string ProtoShortDebugString( + const ::tensorflow::ApiDefs& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::ApiDefs* msg) + TF_MUST_USE_RESULT; + +} // namespace tensorflow + +#endif // tensorflow_core_framework_api_def_proto_H_ diff --git a/linux/include/tensorflow/core/framework/attr_value.pb.h b/linux/include/tensorflow/core/framework/attr_value.pb.h new file mode 100644 index 000000000..669f142b4 --- /dev/null +++ b/linux/include/tensorflow/core/framework/attr_value.pb.h @@ -0,0 +1,1814 @@ +// Generated by the protocol buffer compiler. DO NOT EDIT! +// source: tensorflow/core/framework/attr_value.proto + +#ifndef PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2fattr_5fvalue_2eproto +#define PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2fattr_5fvalue_2eproto + +#include + +#include + +#if GOOGLE_PROTOBUF_VERSION < 3006000 +#error This file was generated by a newer version of protoc which is +#error incompatible with your Protocol Buffer headers. Please update +#error your headers. +#endif +#if 3006000 < GOOGLE_PROTOBUF_MIN_PROTOC_VERSION +#error This file was generated by an older version of protoc which is +#error incompatible with your Protocol Buffer headers. Please +#error regenerate this file with a newer version of protoc. +#endif + +#include +#include +#include +#include +#include +#include +#include +#include +#include // IWYU pragma: export +#include // IWYU pragma: export +#include // IWYU pragma: export +#include +#include +#include +#include "tensorflow/core/framework/tensor.pb.h" +#include "tensorflow/core/framework/tensor_shape.pb.h" +#include "tensorflow/core/framework/types.pb.h" +// @@protoc_insertion_point(includes) +#define PROTOBUF_INTERNAL_EXPORT_protobuf_tensorflow_2fcore_2fframework_2fattr_5fvalue_2eproto + +namespace protobuf_tensorflow_2fcore_2fframework_2fattr_5fvalue_2eproto { +// Internal implementation detail -- do not use these members. +struct TableStruct { + static const ::google::protobuf::internal::ParseTableField entries[]; + static const ::google::protobuf::internal::AuxillaryParseTableField aux[]; + static const ::google::protobuf::internal::ParseTable schema[4]; + static const ::google::protobuf::internal::FieldMetadata field_metadata[]; + static const ::google::protobuf::internal::SerializationTable serialization_table[]; + static const ::google::protobuf::uint32 offsets[]; +}; +void AddDescriptors(); +} // namespace protobuf_tensorflow_2fcore_2fframework_2fattr_5fvalue_2eproto +namespace tensorflow { +class AttrValue; +class AttrValueDefaultTypeInternal; +extern AttrValueDefaultTypeInternal _AttrValue_default_instance_; +class AttrValue_ListValue; +class AttrValue_ListValueDefaultTypeInternal; +extern AttrValue_ListValueDefaultTypeInternal _AttrValue_ListValue_default_instance_; +class NameAttrList; +class NameAttrListDefaultTypeInternal; +extern NameAttrListDefaultTypeInternal _NameAttrList_default_instance_; +class NameAttrList_AttrEntry_DoNotUse; +class NameAttrList_AttrEntry_DoNotUseDefaultTypeInternal; +extern NameAttrList_AttrEntry_DoNotUseDefaultTypeInternal _NameAttrList_AttrEntry_DoNotUse_default_instance_; +} // namespace tensorflow +namespace google { +namespace protobuf { +template<> ::tensorflow::AttrValue* Arena::CreateMaybeMessage<::tensorflow::AttrValue>(Arena*); +template<> ::tensorflow::AttrValue_ListValue* Arena::CreateMaybeMessage<::tensorflow::AttrValue_ListValue>(Arena*); +template<> ::tensorflow::NameAttrList* Arena::CreateMaybeMessage<::tensorflow::NameAttrList>(Arena*); +template<> ::tensorflow::NameAttrList_AttrEntry_DoNotUse* Arena::CreateMaybeMessage<::tensorflow::NameAttrList_AttrEntry_DoNotUse>(Arena*); +} // namespace protobuf +} // namespace google +namespace tensorflow { + +// =================================================================== + +class AttrValue_ListValue : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.AttrValue.ListValue) */ { + public: + AttrValue_ListValue(); + virtual ~AttrValue_ListValue(); + + AttrValue_ListValue(const AttrValue_ListValue& from); + + inline AttrValue_ListValue& operator=(const AttrValue_ListValue& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + AttrValue_ListValue(AttrValue_ListValue&& from) noexcept + : AttrValue_ListValue() { + *this = ::std::move(from); + } + + inline AttrValue_ListValue& operator=(AttrValue_ListValue&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const AttrValue_ListValue& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const AttrValue_ListValue* internal_default_instance() { + return reinterpret_cast( + &_AttrValue_ListValue_default_instance_); + } + static constexpr int kIndexInFileMessages = + 0; + + void UnsafeArenaSwap(AttrValue_ListValue* other); + void Swap(AttrValue_ListValue* other); + friend void swap(AttrValue_ListValue& a, AttrValue_ListValue& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline AttrValue_ListValue* New() const final { + return CreateMaybeMessage(NULL); + } + + AttrValue_ListValue* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const AttrValue_ListValue& from); + void MergeFrom(const AttrValue_ListValue& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(AttrValue_ListValue* other); + protected: + explicit AttrValue_ListValue(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated bytes s = 2; + int s_size() const; + void clear_s(); + static const int kSFieldNumber = 2; + const ::std::string& s(int index) const; + ::std::string* mutable_s(int index); + void set_s(int index, const ::std::string& value); + #if LANG_CXX11 + void set_s(int index, ::std::string&& value); + #endif + void set_s(int index, const char* value); + void set_s(int index, const void* value, size_t size); + ::std::string* add_s(); + void add_s(const ::std::string& value); + #if LANG_CXX11 + void add_s(::std::string&& value); + #endif + void add_s(const char* value); + void add_s(const void* value, size_t size); + const ::google::protobuf::RepeatedPtrField< ::std::string>& s() const; + ::google::protobuf::RepeatedPtrField< ::std::string>* mutable_s(); + + // repeated int64 i = 3 [packed = true]; + int i_size() const; + void clear_i(); + static const int kIFieldNumber = 3; + ::google::protobuf::int64 i(int index) const; + void set_i(int index, ::google::protobuf::int64 value); + void add_i(::google::protobuf::int64 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& + i() const; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* + mutable_i(); + + // repeated float f = 4 [packed = true]; + int f_size() const; + void clear_f(); + static const int kFFieldNumber = 4; + float f(int index) const; + void set_f(int index, float value); + void add_f(float value); + const ::google::protobuf::RepeatedField< float >& + f() const; + ::google::protobuf::RepeatedField< float >* + mutable_f(); + + // repeated bool b = 5 [packed = true]; + int b_size() const; + void clear_b(); + static const int kBFieldNumber = 5; + bool b(int index) const; + void set_b(int index, bool value); + void add_b(bool value); + const ::google::protobuf::RepeatedField< bool >& + b() const; + ::google::protobuf::RepeatedField< bool >* + mutable_b(); + + // repeated .tensorflow.DataType type = 6 [packed = true]; + int type_size() const; + void clear_type(); + static const int kTypeFieldNumber = 6; + ::tensorflow::DataType type(int index) const; + void set_type(int index, ::tensorflow::DataType value); + void add_type(::tensorflow::DataType value); + const ::google::protobuf::RepeatedField& type() const; + ::google::protobuf::RepeatedField* mutable_type(); + + // repeated .tensorflow.TensorShapeProto shape = 7; + int shape_size() const; + void clear_shape(); + static const int kShapeFieldNumber = 7; + ::tensorflow::TensorShapeProto* mutable_shape(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::TensorShapeProto >* + mutable_shape(); + const ::tensorflow::TensorShapeProto& shape(int index) const; + ::tensorflow::TensorShapeProto* add_shape(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::TensorShapeProto >& + shape() const; + + // repeated .tensorflow.TensorProto tensor = 8; + int tensor_size() const; + void clear_tensor(); + static const int kTensorFieldNumber = 8; + ::tensorflow::TensorProto* mutable_tensor(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::TensorProto >* + mutable_tensor(); + const ::tensorflow::TensorProto& tensor(int index) const; + ::tensorflow::TensorProto* add_tensor(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::TensorProto >& + tensor() const; + + // repeated .tensorflow.NameAttrList func = 9; + int func_size() const; + void clear_func(); + static const int kFuncFieldNumber = 9; + ::tensorflow::NameAttrList* mutable_func(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::NameAttrList >* + mutable_func(); + const ::tensorflow::NameAttrList& func(int index) const; + ::tensorflow::NameAttrList* add_func(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::NameAttrList >& + func() const; + + // @@protoc_insertion_point(class_scope:tensorflow.AttrValue.ListValue) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedPtrField< ::std::string> s_; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 > i_; + mutable int _i_cached_byte_size_; + ::google::protobuf::RepeatedField< float > f_; + mutable int _f_cached_byte_size_; + ::google::protobuf::RepeatedField< bool > b_; + mutable int _b_cached_byte_size_; + ::google::protobuf::RepeatedField type_; + mutable int _type_cached_byte_size_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::TensorShapeProto > shape_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::TensorProto > tensor_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::NameAttrList > func_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fattr_5fvalue_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class AttrValue : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.AttrValue) */ { + public: + AttrValue(); + virtual ~AttrValue(); + + AttrValue(const AttrValue& from); + + inline AttrValue& operator=(const AttrValue& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + AttrValue(AttrValue&& from) noexcept + : AttrValue() { + *this = ::std::move(from); + } + + inline AttrValue& operator=(AttrValue&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const AttrValue& default_instance(); + + enum ValueCase { + kS = 2, + kI = 3, + kF = 4, + kB = 5, + kType = 6, + kShape = 7, + kTensor = 8, + kList = 1, + kFunc = 10, + kPlaceholder = 9, + VALUE_NOT_SET = 0, + }; + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const AttrValue* internal_default_instance() { + return reinterpret_cast( + &_AttrValue_default_instance_); + } + static constexpr int kIndexInFileMessages = + 1; + + void UnsafeArenaSwap(AttrValue* other); + void Swap(AttrValue* other); + friend void swap(AttrValue& a, AttrValue& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline AttrValue* New() const final { + return CreateMaybeMessage(NULL); + } + + AttrValue* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const AttrValue& from); + void MergeFrom(const AttrValue& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(AttrValue* other); + protected: + explicit AttrValue(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + typedef AttrValue_ListValue ListValue; + + // accessors ------------------------------------------------------- + + // bytes s = 2; + private: + bool has_s() const; + public: + void clear_s(); + static const int kSFieldNumber = 2; + const ::std::string& s() const; + void set_s(const ::std::string& value); + #if LANG_CXX11 + void set_s(::std::string&& value); + #endif + void set_s(const char* value); + void set_s(const void* value, size_t size); + ::std::string* mutable_s(); + ::std::string* release_s(); + void set_allocated_s(::std::string* s); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_s(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_s( + ::std::string* s); + + // int64 i = 3; + private: + bool has_i() const; + public: + void clear_i(); + static const int kIFieldNumber = 3; + ::google::protobuf::int64 i() const; + void set_i(::google::protobuf::int64 value); + + // float f = 4; + private: + bool has_f() const; + public: + void clear_f(); + static const int kFFieldNumber = 4; + float f() const; + void set_f(float value); + + // bool b = 5; + private: + bool has_b() const; + public: + void clear_b(); + static const int kBFieldNumber = 5; + bool b() const; + void set_b(bool value); + + // .tensorflow.DataType type = 6; + private: + bool has_type() const; + public: + void clear_type(); + static const int kTypeFieldNumber = 6; + ::tensorflow::DataType type() const; + void set_type(::tensorflow::DataType value); + + // .tensorflow.TensorShapeProto shape = 7; + bool has_shape() const; + void clear_shape(); + static const int kShapeFieldNumber = 7; + private: + const ::tensorflow::TensorShapeProto& _internal_shape() const; + public: + const ::tensorflow::TensorShapeProto& shape() const; + ::tensorflow::TensorShapeProto* release_shape(); + ::tensorflow::TensorShapeProto* mutable_shape(); + void set_allocated_shape(::tensorflow::TensorShapeProto* shape); + void unsafe_arena_set_allocated_shape( + ::tensorflow::TensorShapeProto* shape); + ::tensorflow::TensorShapeProto* unsafe_arena_release_shape(); + + // .tensorflow.TensorProto tensor = 8; + bool has_tensor() const; + void clear_tensor(); + static const int kTensorFieldNumber = 8; + private: + const ::tensorflow::TensorProto& _internal_tensor() const; + public: + const ::tensorflow::TensorProto& tensor() const; + ::tensorflow::TensorProto* release_tensor(); + ::tensorflow::TensorProto* mutable_tensor(); + void set_allocated_tensor(::tensorflow::TensorProto* tensor); + void unsafe_arena_set_allocated_tensor( + ::tensorflow::TensorProto* tensor); + ::tensorflow::TensorProto* unsafe_arena_release_tensor(); + + // .tensorflow.AttrValue.ListValue list = 1; + bool has_list() const; + void clear_list(); + static const int kListFieldNumber = 1; + private: + const ::tensorflow::AttrValue_ListValue& _internal_list() const; + public: + const ::tensorflow::AttrValue_ListValue& list() const; + ::tensorflow::AttrValue_ListValue* release_list(); + ::tensorflow::AttrValue_ListValue* mutable_list(); + void set_allocated_list(::tensorflow::AttrValue_ListValue* list); + void unsafe_arena_set_allocated_list( + ::tensorflow::AttrValue_ListValue* list); + ::tensorflow::AttrValue_ListValue* unsafe_arena_release_list(); + + // .tensorflow.NameAttrList func = 10; + bool has_func() const; + void clear_func(); + static const int kFuncFieldNumber = 10; + private: + const ::tensorflow::NameAttrList& _internal_func() const; + public: + const ::tensorflow::NameAttrList& func() const; + ::tensorflow::NameAttrList* release_func(); + ::tensorflow::NameAttrList* mutable_func(); + void set_allocated_func(::tensorflow::NameAttrList* func); + void unsafe_arena_set_allocated_func( + ::tensorflow::NameAttrList* func); + ::tensorflow::NameAttrList* unsafe_arena_release_func(); + + // string placeholder = 9; + private: + bool has_placeholder() const; + public: + void clear_placeholder(); + static const int kPlaceholderFieldNumber = 9; + const ::std::string& placeholder() const; + void set_placeholder(const ::std::string& value); + #if LANG_CXX11 + void set_placeholder(::std::string&& value); + #endif + void set_placeholder(const char* value); + void set_placeholder(const char* value, size_t size); + ::std::string* mutable_placeholder(); + ::std::string* release_placeholder(); + void set_allocated_placeholder(::std::string* placeholder); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_placeholder(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_placeholder( + ::std::string* placeholder); + + void clear_value(); + ValueCase value_case() const; + // @@protoc_insertion_point(class_scope:tensorflow.AttrValue) + private: + void set_has_s(); + void set_has_i(); + void set_has_f(); + void set_has_b(); + void set_has_type(); + void set_has_shape(); + void set_has_tensor(); + void set_has_list(); + void set_has_func(); + void set_has_placeholder(); + + inline bool has_value() const; + inline void clear_has_value(); + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + union ValueUnion { + ValueUnion() {} + ::google::protobuf::internal::ArenaStringPtr s_; + ::google::protobuf::int64 i_; + float f_; + bool b_; + int type_; + ::tensorflow::TensorShapeProto* shape_; + ::tensorflow::TensorProto* tensor_; + ::tensorflow::AttrValue_ListValue* list_; + ::tensorflow::NameAttrList* func_; + ::google::protobuf::internal::ArenaStringPtr placeholder_; + } value_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + ::google::protobuf::uint32 _oneof_case_[1]; + + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fattr_5fvalue_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class NameAttrList_AttrEntry_DoNotUse : public ::google::protobuf::internal::MapEntry { +public: + typedef ::google::protobuf::internal::MapEntry SuperType; + NameAttrList_AttrEntry_DoNotUse(); + NameAttrList_AttrEntry_DoNotUse(::google::protobuf::Arena* arena); + void MergeFrom(const NameAttrList_AttrEntry_DoNotUse& other); + static const NameAttrList_AttrEntry_DoNotUse* internal_default_instance() { return reinterpret_cast(&_NameAttrList_AttrEntry_DoNotUse_default_instance_); } + void MergeFrom(const ::google::protobuf::Message& other) final; + ::google::protobuf::Metadata GetMetadata() const; +}; + +// ------------------------------------------------------------------- + +class NameAttrList : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.NameAttrList) */ { + public: + NameAttrList(); + virtual ~NameAttrList(); + + NameAttrList(const NameAttrList& from); + + inline NameAttrList& operator=(const NameAttrList& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + NameAttrList(NameAttrList&& from) noexcept + : NameAttrList() { + *this = ::std::move(from); + } + + inline NameAttrList& operator=(NameAttrList&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const NameAttrList& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const NameAttrList* internal_default_instance() { + return reinterpret_cast( + &_NameAttrList_default_instance_); + } + static constexpr int kIndexInFileMessages = + 3; + + void UnsafeArenaSwap(NameAttrList* other); + void Swap(NameAttrList* other); + friend void swap(NameAttrList& a, NameAttrList& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline NameAttrList* New() const final { + return CreateMaybeMessage(NULL); + } + + NameAttrList* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const NameAttrList& from); + void MergeFrom(const NameAttrList& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(NameAttrList* other); + protected: + explicit NameAttrList(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + + // accessors ------------------------------------------------------- + + // map attr = 2; + int attr_size() const; + void clear_attr(); + static const int kAttrFieldNumber = 2; + const ::google::protobuf::Map< ::std::string, ::tensorflow::AttrValue >& + attr() const; + ::google::protobuf::Map< ::std::string, ::tensorflow::AttrValue >* + mutable_attr(); + + // string name = 1; + void clear_name(); + static const int kNameFieldNumber = 1; + const ::std::string& name() const; + void set_name(const ::std::string& value); + #if LANG_CXX11 + void set_name(::std::string&& value); + #endif + void set_name(const char* value); + void set_name(const char* value, size_t size); + ::std::string* mutable_name(); + ::std::string* release_name(); + void set_allocated_name(::std::string* name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_name( + ::std::string* name); + + // @@protoc_insertion_point(class_scope:tensorflow.NameAttrList) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::internal::MapField< + NameAttrList_AttrEntry_DoNotUse, + ::std::string, ::tensorflow::AttrValue, + ::google::protobuf::internal::WireFormatLite::TYPE_STRING, + ::google::protobuf::internal::WireFormatLite::TYPE_MESSAGE, + 0 > attr_; + ::google::protobuf::internal::ArenaStringPtr name_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fattr_5fvalue_2eproto::TableStruct; +}; +// =================================================================== + + +// =================================================================== + +#ifdef __GNUC__ + #pragma GCC diagnostic push + #pragma GCC diagnostic ignored "-Wstrict-aliasing" +#endif // __GNUC__ +// AttrValue_ListValue + +// repeated bytes s = 2; +inline int AttrValue_ListValue::s_size() const { + return s_.size(); +} +inline void AttrValue_ListValue::clear_s() { + s_.Clear(); +} +inline const ::std::string& AttrValue_ListValue::s(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.AttrValue.ListValue.s) + return s_.Get(index); +} +inline ::std::string* AttrValue_ListValue::mutable_s(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.AttrValue.ListValue.s) + return s_.Mutable(index); +} +inline void AttrValue_ListValue::set_s(int index, const ::std::string& value) { + // @@protoc_insertion_point(field_set:tensorflow.AttrValue.ListValue.s) + s_.Mutable(index)->assign(value); +} +#if LANG_CXX11 +inline void AttrValue_ListValue::set_s(int index, ::std::string&& value) { + // @@protoc_insertion_point(field_set:tensorflow.AttrValue.ListValue.s) + s_.Mutable(index)->assign(std::move(value)); +} +#endif +inline void AttrValue_ListValue::set_s(int index, const char* value) { + GOOGLE_DCHECK(value != NULL); + s_.Mutable(index)->assign(value); + // @@protoc_insertion_point(field_set_char:tensorflow.AttrValue.ListValue.s) +} +inline void AttrValue_ListValue::set_s(int index, const void* value, size_t size) { + s_.Mutable(index)->assign( + reinterpret_cast(value), size); + // @@protoc_insertion_point(field_set_pointer:tensorflow.AttrValue.ListValue.s) +} +inline ::std::string* AttrValue_ListValue::add_s() { + // @@protoc_insertion_point(field_add_mutable:tensorflow.AttrValue.ListValue.s) + return s_.Add(); +} +inline void AttrValue_ListValue::add_s(const ::std::string& value) { + s_.Add()->assign(value); + // @@protoc_insertion_point(field_add:tensorflow.AttrValue.ListValue.s) +} +#if LANG_CXX11 +inline void AttrValue_ListValue::add_s(::std::string&& value) { + s_.Add(std::move(value)); + // @@protoc_insertion_point(field_add:tensorflow.AttrValue.ListValue.s) +} +#endif +inline void AttrValue_ListValue::add_s(const char* value) { + GOOGLE_DCHECK(value != NULL); + s_.Add()->assign(value); + // @@protoc_insertion_point(field_add_char:tensorflow.AttrValue.ListValue.s) +} +inline void AttrValue_ListValue::add_s(const void* value, size_t size) { + s_.Add()->assign(reinterpret_cast(value), size); + // @@protoc_insertion_point(field_add_pointer:tensorflow.AttrValue.ListValue.s) +} +inline const ::google::protobuf::RepeatedPtrField< ::std::string>& +AttrValue_ListValue::s() const { + // @@protoc_insertion_point(field_list:tensorflow.AttrValue.ListValue.s) + return s_; +} +inline ::google::protobuf::RepeatedPtrField< ::std::string>* +AttrValue_ListValue::mutable_s() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.AttrValue.ListValue.s) + return &s_; +} + +// repeated int64 i = 3 [packed = true]; +inline int AttrValue_ListValue::i_size() const { + return i_.size(); +} +inline void AttrValue_ListValue::clear_i() { + i_.Clear(); +} +inline ::google::protobuf::int64 AttrValue_ListValue::i(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.AttrValue.ListValue.i) + return i_.Get(index); +} +inline void AttrValue_ListValue::set_i(int index, ::google::protobuf::int64 value) { + i_.Set(index, value); + // @@protoc_insertion_point(field_set:tensorflow.AttrValue.ListValue.i) +} +inline void AttrValue_ListValue::add_i(::google::protobuf::int64 value) { + i_.Add(value); + // @@protoc_insertion_point(field_add:tensorflow.AttrValue.ListValue.i) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& +AttrValue_ListValue::i() const { + // @@protoc_insertion_point(field_list:tensorflow.AttrValue.ListValue.i) + return i_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* +AttrValue_ListValue::mutable_i() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.AttrValue.ListValue.i) + return &i_; +} + +// repeated float f = 4 [packed = true]; +inline int AttrValue_ListValue::f_size() const { + return f_.size(); +} +inline void AttrValue_ListValue::clear_f() { + f_.Clear(); +} +inline float AttrValue_ListValue::f(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.AttrValue.ListValue.f) + return f_.Get(index); +} +inline void AttrValue_ListValue::set_f(int index, float value) { + f_.Set(index, value); + // @@protoc_insertion_point(field_set:tensorflow.AttrValue.ListValue.f) +} +inline void AttrValue_ListValue::add_f(float value) { + f_.Add(value); + // @@protoc_insertion_point(field_add:tensorflow.AttrValue.ListValue.f) +} +inline const ::google::protobuf::RepeatedField< float >& +AttrValue_ListValue::f() const { + // @@protoc_insertion_point(field_list:tensorflow.AttrValue.ListValue.f) + return f_; +} +inline ::google::protobuf::RepeatedField< float >* +AttrValue_ListValue::mutable_f() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.AttrValue.ListValue.f) + return &f_; +} + +// repeated bool b = 5 [packed = true]; +inline int AttrValue_ListValue::b_size() const { + return b_.size(); +} +inline void AttrValue_ListValue::clear_b() { + b_.Clear(); +} +inline bool AttrValue_ListValue::b(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.AttrValue.ListValue.b) + return b_.Get(index); +} +inline void AttrValue_ListValue::set_b(int index, bool value) { + b_.Set(index, value); + // @@protoc_insertion_point(field_set:tensorflow.AttrValue.ListValue.b) +} +inline void AttrValue_ListValue::add_b(bool value) { + b_.Add(value); + // @@protoc_insertion_point(field_add:tensorflow.AttrValue.ListValue.b) +} +inline const ::google::protobuf::RepeatedField< bool >& +AttrValue_ListValue::b() const { + // @@protoc_insertion_point(field_list:tensorflow.AttrValue.ListValue.b) + return b_; +} +inline ::google::protobuf::RepeatedField< bool >* +AttrValue_ListValue::mutable_b() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.AttrValue.ListValue.b) + return &b_; +} + +// repeated .tensorflow.DataType type = 6 [packed = true]; +inline int AttrValue_ListValue::type_size() const { + return type_.size(); +} +inline void AttrValue_ListValue::clear_type() { + type_.Clear(); +} +inline ::tensorflow::DataType AttrValue_ListValue::type(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.AttrValue.ListValue.type) + return static_cast< ::tensorflow::DataType >(type_.Get(index)); +} +inline void AttrValue_ListValue::set_type(int index, ::tensorflow::DataType value) { + type_.Set(index, value); + // @@protoc_insertion_point(field_set:tensorflow.AttrValue.ListValue.type) +} +inline void AttrValue_ListValue::add_type(::tensorflow::DataType value) { + type_.Add(value); + // @@protoc_insertion_point(field_add:tensorflow.AttrValue.ListValue.type) +} +inline const ::google::protobuf::RepeatedField& +AttrValue_ListValue::type() const { + // @@protoc_insertion_point(field_list:tensorflow.AttrValue.ListValue.type) + return type_; +} +inline ::google::protobuf::RepeatedField* +AttrValue_ListValue::mutable_type() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.AttrValue.ListValue.type) + return &type_; +} + +// repeated .tensorflow.TensorShapeProto shape = 7; +inline int AttrValue_ListValue::shape_size() const { + return shape_.size(); +} +inline ::tensorflow::TensorShapeProto* AttrValue_ListValue::mutable_shape(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.AttrValue.ListValue.shape) + return shape_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::TensorShapeProto >* +AttrValue_ListValue::mutable_shape() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.AttrValue.ListValue.shape) + return &shape_; +} +inline const ::tensorflow::TensorShapeProto& AttrValue_ListValue::shape(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.AttrValue.ListValue.shape) + return shape_.Get(index); +} +inline ::tensorflow::TensorShapeProto* AttrValue_ListValue::add_shape() { + // @@protoc_insertion_point(field_add:tensorflow.AttrValue.ListValue.shape) + return shape_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::TensorShapeProto >& +AttrValue_ListValue::shape() const { + // @@protoc_insertion_point(field_list:tensorflow.AttrValue.ListValue.shape) + return shape_; +} + +// repeated .tensorflow.TensorProto tensor = 8; +inline int AttrValue_ListValue::tensor_size() const { + return tensor_.size(); +} +inline ::tensorflow::TensorProto* AttrValue_ListValue::mutable_tensor(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.AttrValue.ListValue.tensor) + return tensor_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::TensorProto >* +AttrValue_ListValue::mutable_tensor() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.AttrValue.ListValue.tensor) + return &tensor_; +} +inline const ::tensorflow::TensorProto& AttrValue_ListValue::tensor(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.AttrValue.ListValue.tensor) + return tensor_.Get(index); +} +inline ::tensorflow::TensorProto* AttrValue_ListValue::add_tensor() { + // @@protoc_insertion_point(field_add:tensorflow.AttrValue.ListValue.tensor) + return tensor_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::TensorProto >& +AttrValue_ListValue::tensor() const { + // @@protoc_insertion_point(field_list:tensorflow.AttrValue.ListValue.tensor) + return tensor_; +} + +// repeated .tensorflow.NameAttrList func = 9; +inline int AttrValue_ListValue::func_size() const { + return func_.size(); +} +inline void AttrValue_ListValue::clear_func() { + func_.Clear(); +} +inline ::tensorflow::NameAttrList* AttrValue_ListValue::mutable_func(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.AttrValue.ListValue.func) + return func_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::NameAttrList >* +AttrValue_ListValue::mutable_func() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.AttrValue.ListValue.func) + return &func_; +} +inline const ::tensorflow::NameAttrList& AttrValue_ListValue::func(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.AttrValue.ListValue.func) + return func_.Get(index); +} +inline ::tensorflow::NameAttrList* AttrValue_ListValue::add_func() { + // @@protoc_insertion_point(field_add:tensorflow.AttrValue.ListValue.func) + return func_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::NameAttrList >& +AttrValue_ListValue::func() const { + // @@protoc_insertion_point(field_list:tensorflow.AttrValue.ListValue.func) + return func_; +} + +// ------------------------------------------------------------------- + +// AttrValue + +// bytes s = 2; +inline bool AttrValue::has_s() const { + return value_case() == kS; +} +inline void AttrValue::set_has_s() { + _oneof_case_[0] = kS; +} +inline void AttrValue::clear_s() { + if (has_s()) { + value_.s_.Destroy(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); + clear_has_value(); + } +} +inline const ::std::string& AttrValue::s() const { + // @@protoc_insertion_point(field_get:tensorflow.AttrValue.s) + if (has_s()) { + return value_.s_.Get(); + } + return *&::google::protobuf::internal::GetEmptyStringAlreadyInited(); +} +inline void AttrValue::set_s(const ::std::string& value) { + if (!has_s()) { + clear_value(); + set_has_s(); + value_.s_.UnsafeSetDefault(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); + } + value_.s_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.AttrValue.s) +} +#if LANG_CXX11 +inline void AttrValue::set_s(::std::string&& value) { + // @@protoc_insertion_point(field_set:tensorflow.AttrValue.s) + if (!has_s()) { + clear_value(); + set_has_s(); + value_.s_.UnsafeSetDefault(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); + } + value_.s_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.AttrValue.s) +} +#endif +inline void AttrValue::set_s(const char* value) { + GOOGLE_DCHECK(value != NULL); + if (!has_s()) { + clear_value(); + set_has_s(); + value_.s_.UnsafeSetDefault(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); + } + value_.s_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + ::std::string(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.AttrValue.s) +} +inline void AttrValue::set_s(const void* value, + size_t size) { + if (!has_s()) { + clear_value(); + set_has_s(); + value_.s_.UnsafeSetDefault(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); + } + value_.s_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.AttrValue.s) +} +inline ::std::string* AttrValue::mutable_s() { + if (!has_s()) { + clear_value(); + set_has_s(); + value_.s_.UnsafeSetDefault(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); + } + return value_.s_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_mutable:tensorflow.AttrValue.s) +} +inline ::std::string* AttrValue::release_s() { + // @@protoc_insertion_point(field_release:tensorflow.AttrValue.s) + if (has_s()) { + clear_has_value(); + return value_.s_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); + } else { + return NULL; + } +} +inline void AttrValue::set_allocated_s(::std::string* s) { + if (!has_s()) { + value_.s_.UnsafeSetDefault(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); + } + clear_value(); + if (s != NULL) { + set_has_s(); + value_.s_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), s, + GetArenaNoVirtual()); + } + // @@protoc_insertion_point(field_set_allocated:tensorflow.AttrValue.s) +} +inline ::std::string* AttrValue::unsafe_arena_release_s() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.AttrValue.s) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (has_s()) { + clear_has_value(); + return value_.s_.UnsafeArenaRelease( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); + } else { + return NULL; + } +} +inline void AttrValue::unsafe_arena_set_allocated_s(::std::string* s) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (!has_s()) { + value_.s_.UnsafeSetDefault(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); + } + clear_value(); + if (s) { + set_has_s(); + value_.s_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), s, GetArenaNoVirtual()); + } + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.AttrValue.s) +} + +// int64 i = 3; +inline bool AttrValue::has_i() const { + return value_case() == kI; +} +inline void AttrValue::set_has_i() { + _oneof_case_[0] = kI; +} +inline void AttrValue::clear_i() { + if (has_i()) { + value_.i_ = GOOGLE_LONGLONG(0); + clear_has_value(); + } +} +inline ::google::protobuf::int64 AttrValue::i() const { + // @@protoc_insertion_point(field_get:tensorflow.AttrValue.i) + if (has_i()) { + return value_.i_; + } + return GOOGLE_LONGLONG(0); +} +inline void AttrValue::set_i(::google::protobuf::int64 value) { + if (!has_i()) { + clear_value(); + set_has_i(); + } + value_.i_ = value; + // @@protoc_insertion_point(field_set:tensorflow.AttrValue.i) +} + +// float f = 4; +inline bool AttrValue::has_f() const { + return value_case() == kF; +} +inline void AttrValue::set_has_f() { + _oneof_case_[0] = kF; +} +inline void AttrValue::clear_f() { + if (has_f()) { + value_.f_ = 0; + clear_has_value(); + } +} +inline float AttrValue::f() const { + // @@protoc_insertion_point(field_get:tensorflow.AttrValue.f) + if (has_f()) { + return value_.f_; + } + return 0; +} +inline void AttrValue::set_f(float value) { + if (!has_f()) { + clear_value(); + set_has_f(); + } + value_.f_ = value; + // @@protoc_insertion_point(field_set:tensorflow.AttrValue.f) +} + +// bool b = 5; +inline bool AttrValue::has_b() const { + return value_case() == kB; +} +inline void AttrValue::set_has_b() { + _oneof_case_[0] = kB; +} +inline void AttrValue::clear_b() { + if (has_b()) { + value_.b_ = false; + clear_has_value(); + } +} +inline bool AttrValue::b() const { + // @@protoc_insertion_point(field_get:tensorflow.AttrValue.b) + if (has_b()) { + return value_.b_; + } + return false; +} +inline void AttrValue::set_b(bool value) { + if (!has_b()) { + clear_value(); + set_has_b(); + } + value_.b_ = value; + // @@protoc_insertion_point(field_set:tensorflow.AttrValue.b) +} + +// .tensorflow.DataType type = 6; +inline bool AttrValue::has_type() const { + return value_case() == kType; +} +inline void AttrValue::set_has_type() { + _oneof_case_[0] = kType; +} +inline void AttrValue::clear_type() { + if (has_type()) { + value_.type_ = 0; + clear_has_value(); + } +} +inline ::tensorflow::DataType AttrValue::type() const { + // @@protoc_insertion_point(field_get:tensorflow.AttrValue.type) + if (has_type()) { + return static_cast< ::tensorflow::DataType >(value_.type_); + } + return static_cast< ::tensorflow::DataType >(0); +} +inline void AttrValue::set_type(::tensorflow::DataType value) { + if (!has_type()) { + clear_value(); + set_has_type(); + } + value_.type_ = value; + // @@protoc_insertion_point(field_set:tensorflow.AttrValue.type) +} + +// .tensorflow.TensorShapeProto shape = 7; +inline bool AttrValue::has_shape() const { + return value_case() == kShape; +} +inline void AttrValue::set_has_shape() { + _oneof_case_[0] = kShape; +} +inline const ::tensorflow::TensorShapeProto& AttrValue::_internal_shape() const { + return *value_.shape_; +} +inline ::tensorflow::TensorShapeProto* AttrValue::release_shape() { + // @@protoc_insertion_point(field_release:tensorflow.AttrValue.shape) + if (has_shape()) { + clear_has_value(); + ::tensorflow::TensorShapeProto* temp = value_.shape_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + value_.shape_ = NULL; + return temp; + } else { + return NULL; + } +} +inline const ::tensorflow::TensorShapeProto& AttrValue::shape() const { + // @@protoc_insertion_point(field_get:tensorflow.AttrValue.shape) + return has_shape() + ? *value_.shape_ + : *reinterpret_cast< ::tensorflow::TensorShapeProto*>(&::tensorflow::_TensorShapeProto_default_instance_); +} +inline ::tensorflow::TensorShapeProto* AttrValue::unsafe_arena_release_shape() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.AttrValue.shape) + if (has_shape()) { + clear_has_value(); + ::tensorflow::TensorShapeProto* temp = value_.shape_; + value_.shape_ = NULL; + return temp; + } else { + return NULL; + } +} +inline void AttrValue::unsafe_arena_set_allocated_shape(::tensorflow::TensorShapeProto* shape) { + clear_value(); + if (shape) { + set_has_shape(); + value_.shape_ = shape; + } + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.AttrValue.shape) +} +inline ::tensorflow::TensorShapeProto* AttrValue::mutable_shape() { + if (!has_shape()) { + clear_value(); + set_has_shape(); + value_.shape_ = CreateMaybeMessage< ::tensorflow::TensorShapeProto >( + GetArenaNoVirtual()); + } + // @@protoc_insertion_point(field_mutable:tensorflow.AttrValue.shape) + return value_.shape_; +} + +// .tensorflow.TensorProto tensor = 8; +inline bool AttrValue::has_tensor() const { + return value_case() == kTensor; +} +inline void AttrValue::set_has_tensor() { + _oneof_case_[0] = kTensor; +} +inline const ::tensorflow::TensorProto& AttrValue::_internal_tensor() const { + return *value_.tensor_; +} +inline ::tensorflow::TensorProto* AttrValue::release_tensor() { + // @@protoc_insertion_point(field_release:tensorflow.AttrValue.tensor) + if (has_tensor()) { + clear_has_value(); + ::tensorflow::TensorProto* temp = value_.tensor_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + value_.tensor_ = NULL; + return temp; + } else { + return NULL; + } +} +inline const ::tensorflow::TensorProto& AttrValue::tensor() const { + // @@protoc_insertion_point(field_get:tensorflow.AttrValue.tensor) + return has_tensor() + ? *value_.tensor_ + : *reinterpret_cast< ::tensorflow::TensorProto*>(&::tensorflow::_TensorProto_default_instance_); +} +inline ::tensorflow::TensorProto* AttrValue::unsafe_arena_release_tensor() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.AttrValue.tensor) + if (has_tensor()) { + clear_has_value(); + ::tensorflow::TensorProto* temp = value_.tensor_; + value_.tensor_ = NULL; + return temp; + } else { + return NULL; + } +} +inline void AttrValue::unsafe_arena_set_allocated_tensor(::tensorflow::TensorProto* tensor) { + clear_value(); + if (tensor) { + set_has_tensor(); + value_.tensor_ = tensor; + } + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.AttrValue.tensor) +} +inline ::tensorflow::TensorProto* AttrValue::mutable_tensor() { + if (!has_tensor()) { + clear_value(); + set_has_tensor(); + value_.tensor_ = CreateMaybeMessage< ::tensorflow::TensorProto >( + GetArenaNoVirtual()); + } + // @@protoc_insertion_point(field_mutable:tensorflow.AttrValue.tensor) + return value_.tensor_; +} + +// .tensorflow.AttrValue.ListValue list = 1; +inline bool AttrValue::has_list() const { + return value_case() == kList; +} +inline void AttrValue::set_has_list() { + _oneof_case_[0] = kList; +} +inline void AttrValue::clear_list() { + if (has_list()) { + if (GetArenaNoVirtual() == NULL) { + delete value_.list_; + } + clear_has_value(); + } +} +inline const ::tensorflow::AttrValue_ListValue& AttrValue::_internal_list() const { + return *value_.list_; +} +inline ::tensorflow::AttrValue_ListValue* AttrValue::release_list() { + // @@protoc_insertion_point(field_release:tensorflow.AttrValue.list) + if (has_list()) { + clear_has_value(); + ::tensorflow::AttrValue_ListValue* temp = value_.list_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + value_.list_ = NULL; + return temp; + } else { + return NULL; + } +} +inline const ::tensorflow::AttrValue_ListValue& AttrValue::list() const { + // @@protoc_insertion_point(field_get:tensorflow.AttrValue.list) + return has_list() + ? *value_.list_ + : *reinterpret_cast< ::tensorflow::AttrValue_ListValue*>(&::tensorflow::_AttrValue_ListValue_default_instance_); +} +inline ::tensorflow::AttrValue_ListValue* AttrValue::unsafe_arena_release_list() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.AttrValue.list) + if (has_list()) { + clear_has_value(); + ::tensorflow::AttrValue_ListValue* temp = value_.list_; + value_.list_ = NULL; + return temp; + } else { + return NULL; + } +} +inline void AttrValue::unsafe_arena_set_allocated_list(::tensorflow::AttrValue_ListValue* list) { + clear_value(); + if (list) { + set_has_list(); + value_.list_ = list; + } + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.AttrValue.list) +} +inline ::tensorflow::AttrValue_ListValue* AttrValue::mutable_list() { + if (!has_list()) { + clear_value(); + set_has_list(); + value_.list_ = CreateMaybeMessage< ::tensorflow::AttrValue_ListValue >( + GetArenaNoVirtual()); + } + // @@protoc_insertion_point(field_mutable:tensorflow.AttrValue.list) + return value_.list_; +} + +// .tensorflow.NameAttrList func = 10; +inline bool AttrValue::has_func() const { + return value_case() == kFunc; +} +inline void AttrValue::set_has_func() { + _oneof_case_[0] = kFunc; +} +inline void AttrValue::clear_func() { + if (has_func()) { + if (GetArenaNoVirtual() == NULL) { + delete value_.func_; + } + clear_has_value(); + } +} +inline const ::tensorflow::NameAttrList& AttrValue::_internal_func() const { + return *value_.func_; +} +inline ::tensorflow::NameAttrList* AttrValue::release_func() { + // @@protoc_insertion_point(field_release:tensorflow.AttrValue.func) + if (has_func()) { + clear_has_value(); + ::tensorflow::NameAttrList* temp = value_.func_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + value_.func_ = NULL; + return temp; + } else { + return NULL; + } +} +inline const ::tensorflow::NameAttrList& AttrValue::func() const { + // @@protoc_insertion_point(field_get:tensorflow.AttrValue.func) + return has_func() + ? *value_.func_ + : *reinterpret_cast< ::tensorflow::NameAttrList*>(&::tensorflow::_NameAttrList_default_instance_); +} +inline ::tensorflow::NameAttrList* AttrValue::unsafe_arena_release_func() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.AttrValue.func) + if (has_func()) { + clear_has_value(); + ::tensorflow::NameAttrList* temp = value_.func_; + value_.func_ = NULL; + return temp; + } else { + return NULL; + } +} +inline void AttrValue::unsafe_arena_set_allocated_func(::tensorflow::NameAttrList* func) { + clear_value(); + if (func) { + set_has_func(); + value_.func_ = func; + } + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.AttrValue.func) +} +inline ::tensorflow::NameAttrList* AttrValue::mutable_func() { + if (!has_func()) { + clear_value(); + set_has_func(); + value_.func_ = CreateMaybeMessage< ::tensorflow::NameAttrList >( + GetArenaNoVirtual()); + } + // @@protoc_insertion_point(field_mutable:tensorflow.AttrValue.func) + return value_.func_; +} + +// string placeholder = 9; +inline bool AttrValue::has_placeholder() const { + return value_case() == kPlaceholder; +} +inline void AttrValue::set_has_placeholder() { + _oneof_case_[0] = kPlaceholder; +} +inline void AttrValue::clear_placeholder() { + if (has_placeholder()) { + value_.placeholder_.Destroy(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); + clear_has_value(); + } +} +inline const ::std::string& AttrValue::placeholder() const { + // @@protoc_insertion_point(field_get:tensorflow.AttrValue.placeholder) + if (has_placeholder()) { + return value_.placeholder_.Get(); + } + return *&::google::protobuf::internal::GetEmptyStringAlreadyInited(); +} +inline void AttrValue::set_placeholder(const ::std::string& value) { + if (!has_placeholder()) { + clear_value(); + set_has_placeholder(); + value_.placeholder_.UnsafeSetDefault(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); + } + value_.placeholder_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.AttrValue.placeholder) +} +#if LANG_CXX11 +inline void AttrValue::set_placeholder(::std::string&& value) { + // @@protoc_insertion_point(field_set:tensorflow.AttrValue.placeholder) + if (!has_placeholder()) { + clear_value(); + set_has_placeholder(); + value_.placeholder_.UnsafeSetDefault(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); + } + value_.placeholder_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.AttrValue.placeholder) +} +#endif +inline void AttrValue::set_placeholder(const char* value) { + GOOGLE_DCHECK(value != NULL); + if (!has_placeholder()) { + clear_value(); + set_has_placeholder(); + value_.placeholder_.UnsafeSetDefault(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); + } + value_.placeholder_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + ::std::string(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.AttrValue.placeholder) +} +inline void AttrValue::set_placeholder(const char* value, + size_t size) { + if (!has_placeholder()) { + clear_value(); + set_has_placeholder(); + value_.placeholder_.UnsafeSetDefault(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); + } + value_.placeholder_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.AttrValue.placeholder) +} +inline ::std::string* AttrValue::mutable_placeholder() { + if (!has_placeholder()) { + clear_value(); + set_has_placeholder(); + value_.placeholder_.UnsafeSetDefault(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); + } + return value_.placeholder_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_mutable:tensorflow.AttrValue.placeholder) +} +inline ::std::string* AttrValue::release_placeholder() { + // @@protoc_insertion_point(field_release:tensorflow.AttrValue.placeholder) + if (has_placeholder()) { + clear_has_value(); + return value_.placeholder_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); + } else { + return NULL; + } +} +inline void AttrValue::set_allocated_placeholder(::std::string* placeholder) { + if (!has_placeholder()) { + value_.placeholder_.UnsafeSetDefault(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); + } + clear_value(); + if (placeholder != NULL) { + set_has_placeholder(); + value_.placeholder_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), placeholder, + GetArenaNoVirtual()); + } + // @@protoc_insertion_point(field_set_allocated:tensorflow.AttrValue.placeholder) +} +inline ::std::string* AttrValue::unsafe_arena_release_placeholder() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.AttrValue.placeholder) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (has_placeholder()) { + clear_has_value(); + return value_.placeholder_.UnsafeArenaRelease( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); + } else { + return NULL; + } +} +inline void AttrValue::unsafe_arena_set_allocated_placeholder(::std::string* placeholder) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (!has_placeholder()) { + value_.placeholder_.UnsafeSetDefault(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); + } + clear_value(); + if (placeholder) { + set_has_placeholder(); + value_.placeholder_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), placeholder, GetArenaNoVirtual()); + } + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.AttrValue.placeholder) +} + +inline bool AttrValue::has_value() const { + return value_case() != VALUE_NOT_SET; +} +inline void AttrValue::clear_has_value() { + _oneof_case_[0] = VALUE_NOT_SET; +} +inline AttrValue::ValueCase AttrValue::value_case() const { + return AttrValue::ValueCase(_oneof_case_[0]); +} +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// NameAttrList + +// string name = 1; +inline void NameAttrList::clear_name() { + name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& NameAttrList::name() const { + // @@protoc_insertion_point(field_get:tensorflow.NameAttrList.name) + return name_.Get(); +} +inline void NameAttrList::set_name(const ::std::string& value) { + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.NameAttrList.name) +} +#if LANG_CXX11 +inline void NameAttrList::set_name(::std::string&& value) { + + name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.NameAttrList.name) +} +#endif +inline void NameAttrList::set_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.NameAttrList.name) +} +inline void NameAttrList::set_name(const char* value, + size_t size) { + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.NameAttrList.name) +} +inline ::std::string* NameAttrList::mutable_name() { + + // @@protoc_insertion_point(field_mutable:tensorflow.NameAttrList.name) + return name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* NameAttrList::release_name() { + // @@protoc_insertion_point(field_release:tensorflow.NameAttrList.name) + + return name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void NameAttrList::set_allocated_name(::std::string* name) { + if (name != NULL) { + + } else { + + } + name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.NameAttrList.name) +} +inline ::std::string* NameAttrList::unsafe_arena_release_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.NameAttrList.name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void NameAttrList::unsafe_arena_set_allocated_name( + ::std::string* name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (name != NULL) { + + } else { + + } + name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.NameAttrList.name) +} + +// map attr = 2; +inline int NameAttrList::attr_size() const { + return attr_.size(); +} +inline void NameAttrList::clear_attr() { + attr_.Clear(); +} +inline const ::google::protobuf::Map< ::std::string, ::tensorflow::AttrValue >& +NameAttrList::attr() const { + // @@protoc_insertion_point(field_map:tensorflow.NameAttrList.attr) + return attr_.GetMap(); +} +inline ::google::protobuf::Map< ::std::string, ::tensorflow::AttrValue >* +NameAttrList::mutable_attr() { + // @@protoc_insertion_point(field_mutable_map:tensorflow.NameAttrList.attr) + return attr_.MutableMap(); +} + +#ifdef __GNUC__ + #pragma GCC diagnostic pop +#endif // __GNUC__ +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + + +// @@protoc_insertion_point(namespace_scope) + +} // namespace tensorflow + +// @@protoc_insertion_point(global_scope) + +#endif // PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2fattr_5fvalue_2eproto diff --git a/linux/include/tensorflow/core/framework/attr_value.pb_text-impl.h b/linux/include/tensorflow/core/framework/attr_value.pb_text-impl.h new file mode 100644 index 000000000..5c5de009d --- /dev/null +++ b/linux/include/tensorflow/core/framework/attr_value.pb_text-impl.h @@ -0,0 +1,47 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_framework_attr_value_proto_IMPL_H_ +#define tensorflow_core_framework_attr_value_proto_IMPL_H_ + +#include "tensorflow/core/framework/attr_value.pb.h" +#include "tensorflow/core/framework/attr_value.pb_text.h" +#include "tensorflow/core/framework/resource_handle.pb.h" +#include "tensorflow/core/framework/resource_handle.pb_text-impl.h" +#include "tensorflow/core/framework/tensor.pb.h" +#include "tensorflow/core/framework/tensor.pb_text-impl.h" +#include "tensorflow/core/framework/tensor_shape.pb.h" +#include "tensorflow/core/framework/tensor_shape.pb_text-impl.h" +#include "tensorflow/core/framework/types.pb.h" +#include "tensorflow/core/framework/types.pb_text-impl.h" +#include "tensorflow/core/lib/strings/proto_text_util.h" +#include "tensorflow/core/lib/strings/scanner.h" + +namespace tensorflow { + +namespace internal { + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::AttrValue_ListValue& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::AttrValue_ListValue* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::AttrValue& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::AttrValue* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::NameAttrList& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::NameAttrList* msg); + +} // namespace internal + +} // namespace tensorflow + +#endif // tensorflow_core_framework_attr_value_proto_IMPL_H_ diff --git a/linux/include/tensorflow/core/framework/attr_value.pb_text.h b/linux/include/tensorflow/core/framework/attr_value.pb_text.h new file mode 100644 index 000000000..3e9f069e2 --- /dev/null +++ b/linux/include/tensorflow/core/framework/attr_value.pb_text.h @@ -0,0 +1,44 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_framework_attr_value_proto_H_ +#define tensorflow_core_framework_attr_value_proto_H_ + +#include "tensorflow/core/framework/attr_value.pb.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/protobuf.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +// Message-text conversion for tensorflow.AttrValue.ListValue +string ProtoDebugString( + const ::tensorflow::AttrValue_ListValue& msg); +string ProtoShortDebugString( + const ::tensorflow::AttrValue_ListValue& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::AttrValue_ListValue* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.AttrValue +string ProtoDebugString( + const ::tensorflow::AttrValue& msg); +string ProtoShortDebugString( + const ::tensorflow::AttrValue& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::AttrValue* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.NameAttrList +string ProtoDebugString( + const ::tensorflow::NameAttrList& msg); +string ProtoShortDebugString( + const ::tensorflow::NameAttrList& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::NameAttrList* msg) + TF_MUST_USE_RESULT; + +} // namespace tensorflow + +#endif // tensorflow_core_framework_attr_value_proto_H_ diff --git a/linux/include/tensorflow/core/framework/attr_value_util.h b/linux/include/tensorflow/core/framework/attr_value_util.h new file mode 100644 index 000000000..9fce48879 --- /dev/null +++ b/linux/include/tensorflow/core/framework/attr_value_util.h @@ -0,0 +1,129 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_FRAMEWORK_ATTR_VALUE_UTIL_H_ +#define TENSORFLOW_CORE_FRAMEWORK_ATTR_VALUE_UTIL_H_ + +#include +#include +#include + +#include "tensorflow/core/framework/partial_tensor_shape.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/lib/core/stringpiece.h" +#include "tensorflow/core/lib/gtl/array_slice.h" + +namespace tensorflow { + +// Forward declare protos so their symbols can be removed from .so exports +class AttrValue; +class NameAttrList; + +// A human-readable rendering of attr_value, that is more concise than a +// text-format proto. +string SummarizeAttrValue(const AttrValue& attr_value); + +// Generates an error if attr_value doesn't have the indicated attr type. +Status AttrValueHasType(const AttrValue& attr_value, StringPiece type); + +// Converts a text proto value from "text" into the field of *out +// indicated by "type" (e.g. from the type field of an AttrDef). +// Examples: +// * If type:"int" and text:"-14", then *out is set to "i: -14" +// * If type:"list(string)" and text:"['foo', 'bar']", +// then *out is set to "list { s: ['foo', 'bar'] }" +// Returns true on success. +bool ParseAttrValue(StringPiece type, StringPiece text, AttrValue* out); + +// Sets *out based on the type of value. +void SetAttrValue(const string& value, AttrValue* out); +void SetAttrValue(const char* value, AttrValue* out); +void SetAttrValue(StringPiece value, AttrValue* out); +void SetAttrValue(int64 value, AttrValue* out); +void SetAttrValue(int32 value, AttrValue* out); +void SetAttrValue(float value, AttrValue* out); +void SetAttrValue(double value, AttrValue* out); +void SetAttrValue(bool value, AttrValue* out); +void SetAttrValue(DataType value, AttrValue* out); +void SetAttrValue(const TensorShape& value, AttrValue* out); +void SetAttrValue(const TensorShapeProto& value, AttrValue* out); +void SetAttrValue(const PartialTensorShape& value, AttrValue* out); +void SetAttrValue(const Tensor& value, AttrValue* out); +void SetAttrValue(const TensorProto& value, AttrValue* out); +void SetAttrValue(const NameAttrList& value, AttrValue* out); + +void SetAttrValue(gtl::ArraySlice value, AttrValue* out); +void SetAttrValue(gtl::ArraySlice value, AttrValue* out); +void SetAttrValue(gtl::ArraySlice value, AttrValue* out); +void SetAttrValue(gtl::ArraySlice value, AttrValue* out); +void SetAttrValue(gtl::ArraySlice value, AttrValue* out); +void SetAttrValue(gtl::ArraySlice value, AttrValue* out); +void SetAttrValue(gtl::ArraySlice value, AttrValue* out); +void SetAttrValue(gtl::ArraySlice value, AttrValue* out); +void SetAttrValue(const std::vector& value, AttrValue* out); +void SetAttrValue(std::initializer_list value, AttrValue* out); +void SetAttrValue(DataTypeSlice value, AttrValue* out); +void SetAttrValue(gtl::ArraySlice value, AttrValue* out); +void SetAttrValue(gtl::ArraySlice value, AttrValue* out); +void SetAttrValue(gtl::ArraySlice value, AttrValue* out); +void SetAttrValue(gtl::ArraySlice value, AttrValue* out); +void SetAttrValue(gtl::ArraySlice value, AttrValue* out); +void SetAttrValue(gtl::ArraySlice value, AttrValue* out); + +void SetAttrValue(const AttrValue& value, AttrValue* out); + +// Returns true if a and b have the same value. +bool AreAttrValuesEqual(const AttrValue& a, const AttrValue& b); + +// Returns a hash of `a` that is consistent with AreAttrValuesEqual. In other +// words, if two AttrValues compare equal according to AreAttrValuesEqual, +// they will have the same hash value. +// Similarly to protobuf deterministic serialization, hash value is +// guaranteed to be stable only for a given binary. In particular, one should +// probably not persist the returned value. +uint64 AttrValueHash(const AttrValue& a); + +// WARNING: Equality check might return false-negative for large (> 32mb) +// tensors defined with different TensorProto representations. +// +// A pair of consistent hash and equals functions that are guaranteed to be fast +// with AttrValues that potentially can have very large Tensors (larger than +// 32mb) defined by TensorProto. If large identical Tensors are defined using +// different representations (e.g. one with tensor content, and second with +// bool_val), they will have different hash code and equals will return false. +// Small (less than 32mb) tensors with different TensorProto representations +// hashed/compared by their tensor content. +uint64 FastAttrValueHash(const AttrValue& a); +bool FastAreAttrValuesEqual(const AttrValue& a, const AttrValue& b); + +// Returns true if "val" has a placeholder. +bool HasPlaceHolder(const AttrValue& val); + +// SubstitutePlaceholders recursively replaces placeholders in 'value' +// with an attr value by calling SubstituteFunc. Returns true iff all +// placeholders in "value" are replaced with a value. +// +// SubstituteFunc is given a placeholder string. If the placeholder is +// unknown, SubstituteFunc returns false. Otherwise, overwrites the +// attr value and returns true. +using SubstituteFunc = std::function; +bool SubstitutePlaceholders(const SubstituteFunc& substitute, AttrValue* value); + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_FRAMEWORK_ATTR_VALUE_UTIL_H_ diff --git a/linux/include/tensorflow/core/framework/bfloat16.h b/linux/include/tensorflow/core/framework/bfloat16.h new file mode 100644 index 000000000..e9e94024f --- /dev/null +++ b/linux/include/tensorflow/core/framework/bfloat16.h @@ -0,0 +1,63 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_FRAMEWORK_BFLOAT16_H_ +#define TENSORFLOW_CORE_FRAMEWORK_BFLOAT16_H_ + +#include "tensorflow/core/framework/numeric_types.h" +#include "tensorflow/core/platform/byte_order.h" +#include "tensorflow/core/platform/types.h" + +#if defined(PLATFORM_WINDOWS) +#include "tensorflow/core/platform/windows/cpu_info.h" +#endif + +// Compact 16-bit encoding of floating point numbers. This representation uses +// 1 bit for the sign, 8 bits for the exponent and 7 bits for the mantissa. It +// is assumed that floats are in IEEE 754 format so the representation is just +// bits 16-31 of a single precision float. +// +// NOTE: The IEEE floating point standard defines a float16 format that +// is different than this format (it has fewer bits of exponent and more +// bits of mantissa). We don't use that format here because conversion +// to/from 32-bit floats is more complex for that format, and the +// conversion for this format is very simple. +// +// Because of the existing IEEE float16 type, we do not name our representation +// "float16" but just use "uint16". +// +// <-----our 16bits float-------> +// s e e e e e e e e f f f f f f f f f f f f f f f f f f f f f f f +// <------------------------------float--------------------------> +// 3 3 2 2 1 1 0 +// 1 0 3 2 5 4 0 +// +// +// This type only supports conversion back and forth with float. +// +// This file must be compilable by nvcc. +// +// The type is defined in framework/numeric_types.h. + +namespace tensorflow { + +// Conversion routines between an array of float and bfloat16 of +// "size". +void FloatToBFloat16(const float* src, bfloat16* dst, int64 size); +void BFloat16ToFloat(const bfloat16* src, float* dst, int64 size); + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_FRAMEWORK_BFLOAT16_H_ diff --git a/linux/include/tensorflow/core/framework/cancellation.h b/linux/include/tensorflow/core/framework/cancellation.h new file mode 100644 index 000000000..7a5d94248 --- /dev/null +++ b/linux/include/tensorflow/core/framework/cancellation.h @@ -0,0 +1,146 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_FRAMEWORK_CANCELLATION_H_ +#define TENSORFLOW_CORE_FRAMEWORK_CANCELLATION_H_ + +#include +#include + +#include "tensorflow/core/lib/core/notification.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/lib/gtl/flatmap.h" +#include "tensorflow/core/lib/hash/hash.h" +#include "tensorflow/core/platform/mutex.h" +#include "tensorflow/core/platform/thread_annotations.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +// A token that can be used to register and deregister a +// CancelCallback with a CancellationManager. +// +// CancellationToken values must be created by a call to +// CancellationManager::get_cancellation_token. +typedef int64 CancellationToken; + +// A callback that is invoked when a step is canceled. +// +// NOTE(mrry): See caveats about CancelCallback implementations in the +// comment for CancellationManager::RegisterCallback. +typedef std::function CancelCallback; + +class CancellationManager { + public: + // A value that won't be returned by get_cancellation_token(). + static const CancellationToken kInvalidToken; + + CancellationManager(); + ~CancellationManager(); + + // Run all callbacks associated with this manager. + void StartCancel(); + + // Returns true iff StartCancel() has been called. + bool IsCancelled() { return is_cancelled_.load(std::memory_order_acquire); } + + // Returns a token that must be used in calls to RegisterCallback + // and DeregisterCallback. + CancellationToken get_cancellation_token(); + + // Attempts to register the given callback to be invoked when this + // manager is cancelled. Returns true if the callback was + // registered; returns false if this manager was already cancelled, + // and the callback was not registered. + // + // If this method returns false, it is the caller's responsibility + // to perform any cancellation cleanup. + // + // This method is tricky to use correctly. The following usage pattern + // is recommended: + // + // class ObjectWithCancellableOperation { + // mutex mu_; + // void CancellableOperation(CancellationManager* cm, + // std::function callback) { + // bool already_cancelled; + // CancellationToken token = cm->get_cancellation_token(); + // { + // mutex_lock(mu_); + // already_cancelled = !cm->RegisterCallback( + // [this, token]() { Cancel(token); }); + // if (!already_cancelled) { + // // Issue asynchronous operation. Associate the pending operation + // // with `token` in some object state, or provide another way for + // // the Cancel method to look up the operation for cancellation. + // // Ensure that `cm->DeregisterCallback(token)` is called without + // // holding `mu_`, before `callback` is invoked. + // // ... + // } + // } + // if (already_cancelled) { + // callback(errors::Cancelled("Operation was cancelled")); + // } + // } + // + // void Cancel(CancellationToken token) { + // mutex_lock(mu_); + // // Take action to cancel the operation with the given cancellation + // // token. + // } + // + // NOTE(mrry): The caller should take care that (i) the calling code + // is robust to `callback` being invoked asynchronously (e.g. from + // another thread), (ii) `callback` is deregistered by a call to + // this->DeregisterCallback(token) when the operation completes + // successfully, and (iii) `callback` does not invoke any method + // on this cancellation manager. Furthermore, it is important that + // the eventual caller of the complementary DeregisterCallback does not + // hold any mutexes that are required by `callback`. + bool RegisterCallback(CancellationToken token, CancelCallback callback); + + // Deregister the callback that, when registered, was associated + // with the given cancellation token. Returns true iff the callback + // was deregistered and will not be invoked; otherwise returns false + // after the callback has been invoked, blocking if necessary. + // + // NOTE(mrry): This method may block if cancellation is in progress. + // The caller of this method must not hold any mutexes that are required + // to invoke any cancellation callback that has been registered with this + // cancellation manager. + bool DeregisterCallback(CancellationToken token); + + // Deregister the callback that, when registered, was associated + // with the given cancellation token. Returns true iff the callback + // was deregistered and will not be invoked; otherwise returns false + // immediately, with no guarantee that the callback has completed. + // + // This method is guaranteed to return true if StartCancel has not been + // called. + bool TryDeregisterCallback(CancellationToken token); + + private: + bool is_cancelling_; + std::atomic_bool is_cancelled_; + + mutex mu_; + Notification cancelled_notification_; + CancellationToken next_cancellation_token_ GUARDED_BY(mu_); + gtl::FlatMap callbacks_ GUARDED_BY(mu_); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_FRAMEWORK_CANCELLATION_H_ diff --git a/linux/include/tensorflow/core/framework/collective.h b/linux/include/tensorflow/core/framework/collective.h new file mode 100644 index 000000000..e35edb09d --- /dev/null +++ b/linux/include/tensorflow/core/framework/collective.h @@ -0,0 +1,418 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CORE_FRAMEWORK_COLLECTIVE_H_ +#define TENSORFLOW_CORE_FRAMEWORK_COLLECTIVE_H_ + +#include +#include + +#include "tensorflow/core/framework/device_attributes.pb.h" +#include "tensorflow/core/framework/device_base.h" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/lib/core/refcount.h" +#include "tensorflow/core/lib/core/status.h" + +namespace tensorflow { +class BufRendezvous; +class CancellationManager; +class CompleteGroupRequest; +class CompleteGroupResponse; +class CompleteInstanceRequest; +class CompleteInstanceResponse; +class Device; +class DeviceMgr; +class GetStepSequenceRequest; +class GetStepSequenceResponse; +class Op; +class Tensor; + +// Types of supported collective operations. +enum CollectiveType { + REDUCTION_COLLECTIVE = 0, + BROADCAST_COLLECTIVE, + UNDEFINED_COLLECTIVE, +}; + +// Data common to all members of a device group. +// All members share the same device set but its order is +// particular to an instance so it is stored there. +struct CollGroupParams { + int32 group_key; + int32 group_size; + DeviceType device_type; + int32 num_tasks; // number of distinct tasks in group + string ToString() const; + CollGroupParams() + : group_key(0), group_size(0), device_type(DEVICE_CPU), num_tasks(0) {} +}; + +// The best implementation of a collective op depends on many factors +// including the number of devices involved, the topology of +// interconnects between them and the sizes of inputs. This structure +// is used in generating and representing data movement choreography +// for each specific algorithm, hence it does not have a single, fixed +// interpretation. On first execution the runtime will update this +// structure with decisions that will guide all subsequent executions. +struct CollImplDetails { + string collective_name; + std::vector> subdiv_permutations; + std::vector subdiv_offsets; + std::vector subdiv_source_rank; // rank of source in each subdiv +}; + +// Data common to all members of a collective instance. +struct CollInstanceParams { + // Identifies all participating graph nodes. + int32 instance_key = -1; + CollectiveType type = UNDEFINED_COLLECTIVE; + DataType data_type = DT_FLOAT; + TensorShape shape = {0}; + // Fully qualified name of device for each member, in default rank order. + std::vector device_names; + // Task name prefix of corresponding device name. + std::vector task_names; + // True if every task has the same number of devices. + bool same_num_devices_per_task = false; + CollImplDetails impl_details; + string ToString() const; + CollInstanceParams& operator=(const struct CollInstanceParams& other); +}; + +// Data common to all instance members in the same task. +struct CollTaskParams { + // True for devices that are local to the process, i.e. no RPC needed. + std::vector is_local; + string ToString() const; +}; + +// Unique to a single CollectiveOp node. +struct CollectiveParams { + CollGroupParams group; + CollInstanceParams instance; + CollTaskParams task; + + string name = ""; // node name used only for log or error messages + int default_rank = -1; // index of this op within device_names + bool is_source = false; // broadcast only + int source_rank = -1; // broadcast only + // Rank of this device in each subdivision permutation. + std::vector subdiv_rank; + std::unique_ptr merge_op; // reduction only + std::unique_ptr final_op; // reduction only + string ToString() const; +}; + +class CollectiveExecutor; + +// Interface that provides resolution of device localities. +class DeviceResolverInterface { + public: + virtual ~DeviceResolverInterface() {} + + // Collects DeviceLocality protobufs from all of the devices identified + // in 'col_params'. + virtual void GetDeviceLocalitiesAsync(const CollInstanceParams& inst_params, + std::vector* localities, + const StatusCallback& done) = 0; + + // Populate *locality with the DeviceLocality of the specified + // device. + virtual void GetLocalityAsync(const string& device, const string& task, + DeviceLocality* locality, + const StatusCallback& done) = 0; + + // Clear the cache of device data belonging + // to the specified task. + virtual void ClearTask(const string& task) = 0; +}; + +// Interface that provides resolution of shared CollectiveParams fields. +class ParamResolverInterface { + public: + virtual ~ParamResolverInterface() {} + + // Called by each collective op at first execution in order to fill out + // the CollectiveParams structure with data gathered from the full + // (maybe distributed) collection of peer nodes. + virtual void CompleteParamsAsync(const string& device, CollectiveParams* cp, + CancellationManager* cancel_mgr, + const StatusCallback& done) = 0; + + // Used within a distributed implementation to discover/verify + // data shared across a device group. + virtual void CompleteGroupAsync(const CompleteGroupRequest* request, + CompleteGroupResponse* response, + CancellationManager* cancel_mgr, + const StatusCallback& done) = 0; + + // Used within a distributed implementation to discover/verify data + // shared across an instance group. + virtual void CompleteInstanceAsync(const CompleteInstanceRequest* request, + CompleteInstanceResponse* response, + CancellationManager* cancel_mgr, + const StatusCallback& done) = 0; +}; + +// Graphs which utilize Collective Ops in a common instance must +// execute with identical step_ids even if they are disjoint graphs +// run by otherwise independent tasks. This interface supplies +// coordinated step_ids to use in such cases. +class StepSequenceInterface { + public: + virtual ~StepSequenceInterface() {} + + // Used with a distributed implementation to coordinate step_id + // sequences across tasks. + virtual void GetStepSequenceAsync(const GetStepSequenceRequest* request, + GetStepSequenceResponse* response, + const StatusCallback& done) = 0; + + // Refresh the local per-graph_key step_id sequence from collective + // group leader, if applicable. + virtual void RefreshStepIdSequenceAsync(int64 graph_key, + const StatusCallback& done) = 0; + + // Returns the step_id that should be used for initiating a new execution + // on the specified graph. May return the same step_id multiple times if + // RetireStepId or RefreshStepIdReservation is not called. + virtual int64 NextStepId(int64 graph_key) = 0; + + // Reports that execution of the given step has completed successfully. + // Should be called immediately after a step completes with OK status, + // prior to calling NextStepId(). If the step fails, don't call. + virtual void RetireStepId(int64 graph_key, int64 step_id) = 0; +}; + +// Interface that provides access to per-step CollectiveExecutor +// instances and various distributed resolution capabilities. +class CollectiveExecutorMgrInterface : public StepSequenceInterface { + public: + virtual ~CollectiveExecutorMgrInterface() {} + + // Returns the step-specific CollectiveExecutor, creating if one does not + // already exist. The caller assumes ownership of one Ref on the object. + virtual CollectiveExecutor* FindOrCreate(int64 step_id) = 0; + + // If there is a CollectiveExecutor for step_id, remove it from the + // table. + virtual void Cleanup(int64 step_id) = 0; + + virtual ParamResolverInterface* GetParamResolver() const = 0; + + virtual DeviceResolverInterface* GetDeviceResolver() const = 0; +}; + +// Interface that a Collective Op implementation uses to exchange data +// with peers. Note that data exchange is currently limited to types +// for which DMAHelper::CanUseDMA() returns true, i.e. dense numeric +// types. +class PeerAccessInterface { + public: + virtual ~PeerAccessInterface() {} + + virtual void RecvFromPeer(const string& peer_device, const string& peer_task, + bool peer_is_local, const string& key, + Device* to_device, DeviceContext* to_device_ctx, + const AllocatorAttributes& to_alloc_attr, + Tensor* to_tensor, + const DeviceLocality& client_locality, + int dev_to_dev_stream_index, + const StatusCallback& done) = 0; + + virtual void PostToPeer(const string& peer_device, const string& peer_task, + const string& key, Device* from_device, + DeviceContext* from_device_ctx, + const AllocatorAttributes& from_alloc_attr, + const Tensor* from_tensor, + const DeviceLocality& client_locality, + const StatusCallback& done) = 0; +}; + +class PerStepCollectiveRemoteAccess; + +// A step-specific object that can execute a collective operation completely +// described by a CollectiveParams object. +class CollectiveExecutor : public PeerAccessInterface, public core::RefCounted { + public: + virtual void StartAbort(const Status& s) {} + + virtual void ExecuteAsync(OpKernelContext* ctx, + const CollectiveParams& col_params, + const string& exec_key, StatusCallback done) { + done(errors::Internal( + "A collective Op has been called in a context in which " + "a CollectiveExecutor has not been provided.")); + } + + virtual void CompleteParamsAsync(const string& device, CollectiveParams* cp, + CancellationManager* cancel_mgr, + StatusCallback done) { + cem_->GetParamResolver()->CompleteParamsAsync(device, cp, cancel_mgr, done); + } + + virtual PerStepCollectiveRemoteAccess* remote_access() { return nullptr; } + + // Used to designate an invalid group or instance key. + static int64 kInvalidId; + + // Lexically scoped handle for Ref. + class Handle { + public: + explicit Handle(CollectiveExecutor* ce, bool inherit_ref) : ce_(ce) { + if (!inherit_ref) ce->Ref(); + } + ~Handle() { ce_->Unref(); } + CollectiveExecutor* get() const { return ce_; } + + private: + CollectiveExecutor* ce_; + }; + + protected: + explicit CollectiveExecutor(CollectiveExecutorMgrInterface* cem) + : cem_(cem) {} + + // For use only by derived classes + static OpKernelContext::Params* CtxParams(OpKernelContext* ctx); + CollectiveExecutorMgrInterface* cem_; + + TF_DISALLOW_COPY_AND_ASSIGN(CollectiveExecutor); +}; + +// Interface of a helper object that provides a CollectiveExecutor with +// all of the remote access it needs. +class CollectiveRemoteAccess : public PeerAccessInterface, + public DeviceResolverInterface { + public: + virtual ~CollectiveRemoteAccess() {} + + virtual BufRendezvous* buf_rendezvous() = 0; +}; + +// A per-step version of CollectiveRemoteAccess that cleans up outstanding +// communications in case step execution is abandoned. +class PerStepCollectiveRemoteAccess : public CollectiveRemoteAccess { + public: + virtual ~PerStepCollectiveRemoteAccess() {} + virtual void StartAbort(const Status& s) = 0; +}; + +class CollectiveContext { + public: + CollectiveContext(CollectiveExecutor* col_exec, const DeviceMgr* dev_mgr, + OpKernelContext* ctx, OpKernelContext::Params* op_params, + const CollectiveParams& col_params, const string& exec_key, + int64 step_id, const Tensor* input, Tensor* output); + + virtual ~CollectiveContext() = default; + + CollectiveExecutor* col_exec; // Not owned + const DeviceMgr* dev_mgr; // Not owned + OpKernelContext* op_ctx; // Not owned + OpKernelContext::Params* op_params; // Not owned + const CollectiveParams& col_params; + const string exec_key; + const int64 step_id; + const Tensor* input; // Not owned + Tensor* output; // Not owned + Device* device; // The device for which this instance labors + const string device_name; + DeviceLocality device_locality; +}; + +// Interface of a Collective Op implementation. Each specific CollectiveOp will +// implement this interface and register the implementation via the +// CollectiveRegistry detailed below. See common_runtime/ring_reducer and +// common_runtime/hierarchical_tree_broadcaster for examples. +class CollectiveImplementationInterface { + public: + virtual ~CollectiveImplementationInterface() = default; + + // Initializes the portions of `col_params` specific to this + // implementation. Called exactly once for every Collective instance during + // the CollectiveParams resolution process when the graph is first executed. + // NOTE(ayushd): This is effectively a static function because it modifies the + // `col_params` passed in and should not manipulate any data members. However + // because it is virtual and needs to be implemented by every derived class we + // do not mark it as static. + virtual Status InitializeCollectiveParams(CollectiveParams* col_params) = 0; + + // Prepares the CollectiveContext for executing this CollectiveImplementation. + // Called from CollectiveExecutor right before calling Run(). The + // CollectiveContext passed in must outlive the CollectiveImplementation + // object. + virtual Status InitializeCollectiveContext(CollectiveContext* col_ctx) = 0; + + // Processes and moves data according to the logic of this Collective + // implementation. Relies on appropriate initialization of op-specific + // CollectiveParams in InitializeCollectiveParams(), as well as appropriate + // context initialization in InitializeCollectiveContext(). + virtual void Run(StatusCallback done) = 0; +}; + +// Static-methods only class for registering and looking up collective +// implementations. +class CollectiveRegistry { + public: + using Factory = std::function; + // Looks up a previously registered CollectiveImplementation under + // `collective_name`. If found, creates an instance of the implementation and + // assign to `implementation`. + static Status Lookup(const string& collective_name, + CollectiveImplementationInterface** implementation); + + // Looks up a previously registered CollectiveImplementation under + // `collective_name`. If found, returns the static instance of this + // implementation via `implementation`. This instance should only be used to + // call InitializateCollectiveParams. + static Status LookupParamResolverInstance( + const string& collective_name, + CollectiveImplementationInterface** implementation); + + // Returns all registered collective implementations. + static void GetAll( + std::vector* implementations); + + private: + friend class CollectiveRegistration; + // Registers a CollectiveImplementation with name `collective_name` and + // factory `factory`. The latter is a function used to create instances of + // the CollectiveImplementation. Also creates a static instance of the + // implementation - this instance is used during param resolution and should + // only be used to call InitializeCollectiveParams. + static Status Register(const string& collective_name, Factory factory); + + static Status LookupHelper(const string& collective_name, + CollectiveImplementationInterface** implementation, + bool param_resolver); +}; + +// Class used to call CollectiveRegistry::Register. This should only be used to +// create a global static object. +class CollectiveRegistration { + public: + CollectiveRegistration(const string& collective_name, + CollectiveRegistry::Factory factory) { + TF_CHECK_OK(CollectiveRegistry::Register(collective_name, factory)); + } +}; + +#define REGISTER_COLLECTIVE(name, implementation) \ + static CollectiveRegistration register_##name##_collective( \ + #name, []() { return new implementation; }); + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_FRAMEWORK_COLLECTIVE_H_ diff --git a/linux/include/tensorflow/core/framework/common_shape_fns.h b/linux/include/tensorflow/core/framework/common_shape_fns.h new file mode 100644 index 000000000..3a496e06a --- /dev/null +++ b/linux/include/tensorflow/core/framework/common_shape_fns.h @@ -0,0 +1,317 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CORE_FRAMEWORK_COMMON_SHAPE_FNS_H_ +#define TENSORFLOW_CORE_FRAMEWORK_COMMON_SHAPE_FNS_H_ + +#include + +#include "tensorflow/core/framework/shape_inference.h" +#include "tensorflow/core/util/padding.h" +#include "tensorflow/core/util/tensor_format.h" + +namespace tensorflow { + +// GetWindowedOutputSize(): Given an input tensor, kernel, stride and padding +// type, the function computes the output and padding dimensions. +// +// For example, ignoring batches or multiple features, a 1D convolution +// takes as input a 1D tensor of shape (H), and convolves it with a filter of +// shape (K). +// +// It also takes in a few additional parameters: +// +// Stride (S): the stride with which we apply the filters. This is the offset +// between locations where we apply the filters. A larger stride +// means that the output will be spatially smaller. +// +// Padding (P): the padding we apply to the input tensor along each +// dimension. This is usually used to make sure that the spatial dimensions +// do not shrink when we progress with convolutions. Two types of padding are +// often used: +// SAME: the pad value is computed so that the output will have size H/S. +// VALID: no padding is carried out. +// The padded area is zero-filled. +// +// The output dimensions for convolution and many other operations, when given +// all the parameters above, are as follows: +// - When Padding = SAME: the output size is (H'), where +// H' = ceil(float(H) / float(S)) +// where ceil is the ceiling function. The number of padded cells +// is computed as: +// Pc = ((H' - 1) * S + K - H) / 2 +// When the stride is 1, the expression simplifies to +// H' = H, Pc = (K-1)/2. +// This is where SAME comes from - the output has the same size as the input +// has. +// +// - When Padding = VALID: the output size is computed as +// H' = ceil(float(H - K + 1) / float(S)) +// and the number of padded cells is always zero. +// When the stride is 1, the expression simplifies to +// H' = H-K+1. +// +// For convolution, mathematically, the output value at location (r') +// is the inner product of two vectors: the chunk of input at +// ((r'*S-Pr) : (r'*S-Pr+K)), +// and the filter. +// +// For 2D and 3D convolutions, the spatial dimensions are orthogonal, so the +// size and padding of each spatial dimension can be computed by calling +// GetWindowedOutputSize separately for each dimension. +// +Status GetWindowedOutputSize(int64 input_size, int64 filter_size, int64 stride, + Padding padding_type, int64* output_size, + int64* padding_size); + +// The V2 version computes the same outputs with arbitrary dilation_rate. +// The output dimensions are computed as follows: +// - When adding dilation_rate (D), we compute an effective filter size (K'): +// K' = (K - 1) * D + 1 +// - When Padding = SAME: the output size is (H'), where +// H' = ceil(float(H) / float(S)) +// where ceil is the ceiling function. The number of padded cells +// is computed as: +// Pc = ((H' - 1) * S + K' - H) / 2 +// When the stride is 1, the expression simplifies to +// H' = H, Pc = (K'-1)/2. +// This is where SAME comes from - the output has the same size as the input +// has. +// +// - When Padding = VALID: the output size is computed as +// H' = ceil(float(H - K' + 1) / float(S)) +// and the number of padded cells is always zero. +// When the stride is 1, the expression simplifies to +// H' = H-K'+1. +// +// TODO(b/67112639): Merge V2 versions and the original versions eventually. +Status GetWindowedOutputSizeV2(int64 input_size, int64 filter_size, + int64 dilation_rate, int64 stride, + Padding padding_type, int64* output_size, + int64* padding_size); + +// Returns the same output dimensions as in GetWindowedOutputSize, but returns +// verbose padding dimensions (before/after). Any excess padding +// (caused by an odd padding size value) is added to the 'padding_after' +// dimension. +Status GetWindowedOutputSizeVerbose(int64 input_size, int64 filter_size, + int64 stride, Padding padding_type, + int64* output_size, int64* padding_before, + int64* padding_after); + +// The V2 version computes the same outputs with arbitrary dilation_rate. For +// detailed equations, refer to the comments for GetWindowedOutputSizeV2(). +Status GetWindowedOutputSizeVerboseV2(int64 input_size, int64 filter_size, + int64 dilation_rate, int64 stride, + Padding padding_type, int64* output_size, + int64* padding_before, + int64* padding_after); + +// Given an input tensor, kernel, stride and padding type, populates the 3D size +// of the output tensor and padding to be applied to the input tensor at the +// lower end of every dimension. Use for 3D convolutions, where the input data +// is padded with zeros, as well as for 3D avg/max pooling, where the input data +// is padded with invalid values that are not considered for pooling. +Status Get3dOutputSize(const std::array& input, + const std::array& window, + const std::array& strides, + Padding padding_type, std::array* output_ptr, + std::array* padding_ptr); + +// The V2 version computes the same outputs with arbitrary dilation_rate. For +// detailed equations, refer to the comments for GetWindowedOutputSizeV2(). +Status Get3dOutputSizeV2(const std::array& input, + const std::array& window, + const std::array& dilations, + const std::array& strides, + Padding padding_type, std::array* output_ptr, + std::array* padding_ptr); + +namespace shape_inference { + +// Like GetWindowedOutputSize, but deals with DimensionHandles. +Status GetWindowedOutputSizeFromDims(InferenceContext* c, + DimensionHandle input_size, + DimensionOrConstant filter_size, + int64 stride, Padding padding_type, + DimensionHandle* output_size); + +// The V2 version computes the same outputs with arbitrary dilation_rate. For +// detailed equations, refer to the comments for GetWindowedOutputSizeV2(). +Status GetWindowedOutputSizeFromDimsV2(InferenceContext* c, + DimensionHandle input_size, + DimensionOrConstant filter_size, + int64 dilation_rate, int64 stride, + Padding padding_type, + DimensionHandle* output_size); + +// Transfers shape of input(0) to output(0). +Status UnchangedShape(shape_inference::InferenceContext* c); + +// Transfers shape of input(0) to output(0), after asserting its rank is . +inline Status UnchangedShapeWithRank(shape_inference::InferenceContext* c, + int32 rank) { + ShapeHandle out; + TF_RETURN_IF_ERROR(c->WithRank(c->input(0), rank, &out)); + c->set_output(0, out); + return Status::OK(); +} + +// Transfers shape of input(0) to output(0), after asserting its rank >= . +inline Status UnchangedShapeWithRankAtLeast( + shape_inference::InferenceContext* c, int32 rank) { + ShapeHandle out; + TF_RETURN_IF_ERROR(c->WithRankAtLeast(c->input(0), rank, &out)); + c->set_output(0, out); + return Status::OK(); +} + +// Transfers shape of input(0) to output(0), after asserting its rank <= . +inline Status UnchangedShapeWithRankAtMost(shape_inference::InferenceContext* c, + int32 rank) { + ShapeHandle out; + TF_RETURN_IF_ERROR(c->WithRankAtMost(c->input(0), rank, &out)); + c->set_output(0, out); + return Status::OK(); +} + +// Shape function for use with ops no outputs. +inline Status NoOutputs(shape_inference::InferenceContext* c) { + return Status::OK(); +} + +// Shape function for ops that output a single scalar value. +inline Status ScalarShape(shape_inference::InferenceContext* c) { + c->set_output(0, c->Scalar()); + return Status::OK(); +} + +// Shape function for binary ops where both inputs and the output match. +inline Status MergeBothInputsShapeFn(InferenceContext* c) { + ShapeHandle out; + TF_RETURN_IF_ERROR(c->Merge(c->input(0), c->input(1), &out)); + c->set_output(0, out); + return Status::OK(); +} + +// Returns a new shape with the specified dims arranged in the specified +// format. The returned value is owned by this context. +// Note: if format = "FORMAT_NCHW_VECT_C" then C represents the outer_depth. +Status MakeShapeFromFormat(TensorFormat format, DimensionOrConstant N, + const std::vector& spatial, + DimensionOrConstant C, ShapeHandle* out, + shape_inference::InferenceContext* context); + +// Shape function for MatMul-like operations. +Status MatMulShape(shape_inference::InferenceContext* c); + +// Shape function for BiasAdd-like operations. +Status BiasAddShape(shape_inference::InferenceContext* c); + +// Shape function for BiasAddGrad-like operations. +Status BiasAddGradShape(shape_inference::InferenceContext* c); + +// Shape function for Conv2D-like operations. +Status Conv2DShape(shape_inference::InferenceContext* c); + +// Shape function for Conv3D-like operations. +Status Conv3DShape(shape_inference::InferenceContext* c); + +// Shape function for DepthwiseConv2D-like operations. +Status DepthwiseConv2DNativeShape(shape_inference::InferenceContext* c); + +// Shape function for AvgPool-like operations. +Status AvgPoolShape(shape_inference::InferenceContext* c); + +// Shape function for FusedBatchNorm and FusedBatchNormV2 operations. +Status FusedBatchNormShape(shape_inference::InferenceContext* c); + +// Shape function for FusedBatchNormGrad and FusedBatchNormGradV2 operations. +Status FusedBatchNormGradShape(shape_inference::InferenceContext* c); + +// Shape function for MaxPool-like operations. +Status MaxPoolShape(shape_inference::InferenceContext* c); + +// Shape function for MaxPoolV2-like operations. +Status MaxPoolV2Shape(shape_inference::InferenceContext* c, int num_inputs); + +// Shape function for 3D Pooling operations. +Status Pool3DShape(shape_inference::InferenceContext* c); + +// Shape function for use with ops whose output shapes are unknown. +Status UnknownShape(shape_inference::InferenceContext* c); + +// Shape function for reduction operations. +Status ReductionShape(shape_inference::InferenceContext* c); + +// Shape function for concat operations. +// is the number of inputs to concatenate and are taken +// from inputs +// [1,num_inputs_to_concat] of the op. Input 0 is the concat_dim input. +Status ConcatShape(shape_inference::InferenceContext* c, + int num_inputs_to_concat); + +// Shape function for concat operations. +Status ConcatV2Shape(shape_inference::InferenceContext* c); + +// Shape function for binary operators that broadcast their inputs +// and with output to output_index. +// Note: out cannot be NULL. +Status BroadcastBinaryOpOutputShapeFnHelper(InferenceContext* c, + ShapeHandle shape_x, + ShapeHandle shape_y, + ShapeHandle* out); + +// Shape function for binary operators that broadcast their inputs +// and with output to output_index. +inline Status BroadcastBinaryOpOutputShapeFn(InferenceContext* c, + int output_index) { + ShapeHandle out; + TF_RETURN_IF_ERROR( + BroadcastBinaryOpOutputShapeFnHelper(c, c->input(0), c->input(1), &out)); + c->set_output(output_index, out); + return Status::OK(); +} + +// Shape function for binary operators that broadcast their inputs. +// Tested by ops/math_ops_test.cc. +inline Status BroadcastBinaryOpShapeFn(InferenceContext* c) { + return BroadcastBinaryOpOutputShapeFn(c, 0); +} + +// Shape function for random operations. +Status RandomShape(shape_inference::InferenceContext* c); + +// Shape function for Slice opertaions. +Status SliceShape(shape_inference::InferenceContext* c); + +// Validates the 3 component tensors of a sparse tensor have the proper +// shapes. This mimics SparseTensor.__init__ in python/framework/ops.py. +Status ValidateSparseTensor(InferenceContext* c, ShapeHandle indices_shape, + ShapeHandle values_shape, ShapeHandle shape_shape); + +// Shape function for ScatterNd update/add/sub/... operations. +Status ScatterNdUpdateShape(InferenceContext* c); + +// Shape function for ops with an explicit "shape" attribute. +Status ExplicitShape(InferenceContext* c); + +// Shape function for multiple-output ops with an explicit "shapes" attribute. +Status ExplicitShapes(InferenceContext* c); + +} // namespace shape_inference + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_FRAMEWORK_COMMON_SHAPE_FNS_H_ diff --git a/linux/include/tensorflow/core/framework/control_flow.h b/linux/include/tensorflow/core/framework/control_flow.h new file mode 100644 index 000000000..4839e02e2 --- /dev/null +++ b/linux/include/tensorflow/core/framework/control_flow.h @@ -0,0 +1,58 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_FRAMEWORK_CONTROL_FLOW_H_ +#define TENSORFLOW_CORE_FRAMEWORK_CONTROL_FLOW_H_ + +#include "tensorflow/core/lib/hash/hash.h" +#include "tensorflow/core/platform/logging.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +const uint64 kIllegalFrameId = ~0uLL; +const int64 kIllegalIterId = -1; + +// For the purpose of control flow, every tensor produced by TensorFlow is +// conceptually tagged by a 'FrameAndIter'. FrameAndIter consists of a +// 'frame_id' and an 'iter_id'. The tensor value it represents is produced +// in the frame with frame_id at the iteration of iter_id. +struct FrameAndIter { + uint64 frame_id = kIllegalFrameId; + int64 iter_id = kIllegalIterId; + + FrameAndIter() {} + + FrameAndIter(uint64 frame, int64 iter) { + frame_id = frame; + iter_id = iter; + } + + bool operator==(const FrameAndIter& other) const { + return (frame_id == other.frame_id && iter_id == other.iter_id); + } +}; + +struct FrameAndIterHash { + size_t operator()(const FrameAndIter& key) const { + // Make sure there are no padding bytes that we don't want + CHECK_EQ(sizeof(uint64) + sizeof(int64), sizeof(FrameAndIter)); + return Hash64(reinterpret_cast(&key), sizeof(FrameAndIter)); + } +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_FRAMEWORK_CONTROL_FLOW_H_ diff --git a/linux/include/tensorflow/core/framework/cost_graph.pb.h b/linux/include/tensorflow/core/framework/cost_graph.pb.h new file mode 100644 index 000000000..f05b0ec71 --- /dev/null +++ b/linux/include/tensorflow/core/framework/cost_graph.pb.h @@ -0,0 +1,1344 @@ +// Generated by the protocol buffer compiler. DO NOT EDIT! +// source: tensorflow/core/framework/cost_graph.proto + +#ifndef PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2fcost_5fgraph_2eproto +#define PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2fcost_5fgraph_2eproto + +#include + +#include + +#if GOOGLE_PROTOBUF_VERSION < 3006000 +#error This file was generated by a newer version of protoc which is +#error incompatible with your Protocol Buffer headers. Please update +#error your headers. +#endif +#if 3006000 < GOOGLE_PROTOBUF_MIN_PROTOC_VERSION +#error This file was generated by an older version of protoc which is +#error incompatible with your Protocol Buffer headers. Please +#error regenerate this file with a newer version of protoc. +#endif + +#include +#include +#include +#include +#include +#include +#include +#include +#include // IWYU pragma: export +#include // IWYU pragma: export +#include +#include "tensorflow/core/framework/tensor_shape.pb.h" +#include "tensorflow/core/framework/types.pb.h" +// @@protoc_insertion_point(includes) +#define PROTOBUF_INTERNAL_EXPORT_protobuf_tensorflow_2fcore_2fframework_2fcost_5fgraph_2eproto + +namespace protobuf_tensorflow_2fcore_2fframework_2fcost_5fgraph_2eproto { +// Internal implementation detail -- do not use these members. +struct TableStruct { + static const ::google::protobuf::internal::ParseTableField entries[]; + static const ::google::protobuf::internal::AuxillaryParseTableField aux[]; + static const ::google::protobuf::internal::ParseTable schema[4]; + static const ::google::protobuf::internal::FieldMetadata field_metadata[]; + static const ::google::protobuf::internal::SerializationTable serialization_table[]; + static const ::google::protobuf::uint32 offsets[]; +}; +void AddDescriptors(); +} // namespace protobuf_tensorflow_2fcore_2fframework_2fcost_5fgraph_2eproto +namespace tensorflow { +class CostGraphDef; +class CostGraphDefDefaultTypeInternal; +extern CostGraphDefDefaultTypeInternal _CostGraphDef_default_instance_; +class CostGraphDef_Node; +class CostGraphDef_NodeDefaultTypeInternal; +extern CostGraphDef_NodeDefaultTypeInternal _CostGraphDef_Node_default_instance_; +class CostGraphDef_Node_InputInfo; +class CostGraphDef_Node_InputInfoDefaultTypeInternal; +extern CostGraphDef_Node_InputInfoDefaultTypeInternal _CostGraphDef_Node_InputInfo_default_instance_; +class CostGraphDef_Node_OutputInfo; +class CostGraphDef_Node_OutputInfoDefaultTypeInternal; +extern CostGraphDef_Node_OutputInfoDefaultTypeInternal _CostGraphDef_Node_OutputInfo_default_instance_; +} // namespace tensorflow +namespace google { +namespace protobuf { +template<> ::tensorflow::CostGraphDef* Arena::CreateMaybeMessage<::tensorflow::CostGraphDef>(Arena*); +template<> ::tensorflow::CostGraphDef_Node* Arena::CreateMaybeMessage<::tensorflow::CostGraphDef_Node>(Arena*); +template<> ::tensorflow::CostGraphDef_Node_InputInfo* Arena::CreateMaybeMessage<::tensorflow::CostGraphDef_Node_InputInfo>(Arena*); +template<> ::tensorflow::CostGraphDef_Node_OutputInfo* Arena::CreateMaybeMessage<::tensorflow::CostGraphDef_Node_OutputInfo>(Arena*); +} // namespace protobuf +} // namespace google +namespace tensorflow { + +// =================================================================== + +class CostGraphDef_Node_InputInfo : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.CostGraphDef.Node.InputInfo) */ { + public: + CostGraphDef_Node_InputInfo(); + virtual ~CostGraphDef_Node_InputInfo(); + + CostGraphDef_Node_InputInfo(const CostGraphDef_Node_InputInfo& from); + + inline CostGraphDef_Node_InputInfo& operator=(const CostGraphDef_Node_InputInfo& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + CostGraphDef_Node_InputInfo(CostGraphDef_Node_InputInfo&& from) noexcept + : CostGraphDef_Node_InputInfo() { + *this = ::std::move(from); + } + + inline CostGraphDef_Node_InputInfo& operator=(CostGraphDef_Node_InputInfo&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const CostGraphDef_Node_InputInfo& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const CostGraphDef_Node_InputInfo* internal_default_instance() { + return reinterpret_cast( + &_CostGraphDef_Node_InputInfo_default_instance_); + } + static constexpr int kIndexInFileMessages = + 0; + + void UnsafeArenaSwap(CostGraphDef_Node_InputInfo* other); + void Swap(CostGraphDef_Node_InputInfo* other); + friend void swap(CostGraphDef_Node_InputInfo& a, CostGraphDef_Node_InputInfo& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline CostGraphDef_Node_InputInfo* New() const final { + return CreateMaybeMessage(NULL); + } + + CostGraphDef_Node_InputInfo* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const CostGraphDef_Node_InputInfo& from); + void MergeFrom(const CostGraphDef_Node_InputInfo& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(CostGraphDef_Node_InputInfo* other); + protected: + explicit CostGraphDef_Node_InputInfo(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // int32 preceding_node = 1; + void clear_preceding_node(); + static const int kPrecedingNodeFieldNumber = 1; + ::google::protobuf::int32 preceding_node() const; + void set_preceding_node(::google::protobuf::int32 value); + + // int32 preceding_port = 2; + void clear_preceding_port(); + static const int kPrecedingPortFieldNumber = 2; + ::google::protobuf::int32 preceding_port() const; + void set_preceding_port(::google::protobuf::int32 value); + + // @@protoc_insertion_point(class_scope:tensorflow.CostGraphDef.Node.InputInfo) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::int32 preceding_node_; + ::google::protobuf::int32 preceding_port_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fcost_5fgraph_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class CostGraphDef_Node_OutputInfo : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.CostGraphDef.Node.OutputInfo) */ { + public: + CostGraphDef_Node_OutputInfo(); + virtual ~CostGraphDef_Node_OutputInfo(); + + CostGraphDef_Node_OutputInfo(const CostGraphDef_Node_OutputInfo& from); + + inline CostGraphDef_Node_OutputInfo& operator=(const CostGraphDef_Node_OutputInfo& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + CostGraphDef_Node_OutputInfo(CostGraphDef_Node_OutputInfo&& from) noexcept + : CostGraphDef_Node_OutputInfo() { + *this = ::std::move(from); + } + + inline CostGraphDef_Node_OutputInfo& operator=(CostGraphDef_Node_OutputInfo&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const CostGraphDef_Node_OutputInfo& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const CostGraphDef_Node_OutputInfo* internal_default_instance() { + return reinterpret_cast( + &_CostGraphDef_Node_OutputInfo_default_instance_); + } + static constexpr int kIndexInFileMessages = + 1; + + void UnsafeArenaSwap(CostGraphDef_Node_OutputInfo* other); + void Swap(CostGraphDef_Node_OutputInfo* other); + friend void swap(CostGraphDef_Node_OutputInfo& a, CostGraphDef_Node_OutputInfo& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline CostGraphDef_Node_OutputInfo* New() const final { + return CreateMaybeMessage(NULL); + } + + CostGraphDef_Node_OutputInfo* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const CostGraphDef_Node_OutputInfo& from); + void MergeFrom(const CostGraphDef_Node_OutputInfo& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(CostGraphDef_Node_OutputInfo* other); + protected: + explicit CostGraphDef_Node_OutputInfo(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // .tensorflow.TensorShapeProto shape = 3; + bool has_shape() const; + void clear_shape(); + static const int kShapeFieldNumber = 3; + private: + const ::tensorflow::TensorShapeProto& _internal_shape() const; + public: + const ::tensorflow::TensorShapeProto& shape() const; + ::tensorflow::TensorShapeProto* release_shape(); + ::tensorflow::TensorShapeProto* mutable_shape(); + void set_allocated_shape(::tensorflow::TensorShapeProto* shape); + void unsafe_arena_set_allocated_shape( + ::tensorflow::TensorShapeProto* shape); + ::tensorflow::TensorShapeProto* unsafe_arena_release_shape(); + + // int64 size = 1; + void clear_size(); + static const int kSizeFieldNumber = 1; + ::google::protobuf::int64 size() const; + void set_size(::google::protobuf::int64 value); + + // int64 alias_input_port = 2; + void clear_alias_input_port(); + static const int kAliasInputPortFieldNumber = 2; + ::google::protobuf::int64 alias_input_port() const; + void set_alias_input_port(::google::protobuf::int64 value); + + // .tensorflow.DataType dtype = 4; + void clear_dtype(); + static const int kDtypeFieldNumber = 4; + ::tensorflow::DataType dtype() const; + void set_dtype(::tensorflow::DataType value); + + // @@protoc_insertion_point(class_scope:tensorflow.CostGraphDef.Node.OutputInfo) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::tensorflow::TensorShapeProto* shape_; + ::google::protobuf::int64 size_; + ::google::protobuf::int64 alias_input_port_; + int dtype_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fcost_5fgraph_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class CostGraphDef_Node : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.CostGraphDef.Node) */ { + public: + CostGraphDef_Node(); + virtual ~CostGraphDef_Node(); + + CostGraphDef_Node(const CostGraphDef_Node& from); + + inline CostGraphDef_Node& operator=(const CostGraphDef_Node& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + CostGraphDef_Node(CostGraphDef_Node&& from) noexcept + : CostGraphDef_Node() { + *this = ::std::move(from); + } + + inline CostGraphDef_Node& operator=(CostGraphDef_Node&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const CostGraphDef_Node& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const CostGraphDef_Node* internal_default_instance() { + return reinterpret_cast( + &_CostGraphDef_Node_default_instance_); + } + static constexpr int kIndexInFileMessages = + 2; + + void UnsafeArenaSwap(CostGraphDef_Node* other); + void Swap(CostGraphDef_Node* other); + friend void swap(CostGraphDef_Node& a, CostGraphDef_Node& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline CostGraphDef_Node* New() const final { + return CreateMaybeMessage(NULL); + } + + CostGraphDef_Node* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const CostGraphDef_Node& from); + void MergeFrom(const CostGraphDef_Node& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(CostGraphDef_Node* other); + protected: + explicit CostGraphDef_Node(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + typedef CostGraphDef_Node_InputInfo InputInfo; + typedef CostGraphDef_Node_OutputInfo OutputInfo; + + // accessors ------------------------------------------------------- + + // repeated .tensorflow.CostGraphDef.Node.InputInfo input_info = 4; + int input_info_size() const; + void clear_input_info(); + static const int kInputInfoFieldNumber = 4; + ::tensorflow::CostGraphDef_Node_InputInfo* mutable_input_info(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::CostGraphDef_Node_InputInfo >* + mutable_input_info(); + const ::tensorflow::CostGraphDef_Node_InputInfo& input_info(int index) const; + ::tensorflow::CostGraphDef_Node_InputInfo* add_input_info(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::CostGraphDef_Node_InputInfo >& + input_info() const; + + // repeated .tensorflow.CostGraphDef.Node.OutputInfo output_info = 5; + int output_info_size() const; + void clear_output_info(); + static const int kOutputInfoFieldNumber = 5; + ::tensorflow::CostGraphDef_Node_OutputInfo* mutable_output_info(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::CostGraphDef_Node_OutputInfo >* + mutable_output_info(); + const ::tensorflow::CostGraphDef_Node_OutputInfo& output_info(int index) const; + ::tensorflow::CostGraphDef_Node_OutputInfo* add_output_info(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::CostGraphDef_Node_OutputInfo >& + output_info() const; + + // repeated int32 control_input = 8; + int control_input_size() const; + void clear_control_input(); + static const int kControlInputFieldNumber = 8; + ::google::protobuf::int32 control_input(int index) const; + void set_control_input(int index, ::google::protobuf::int32 value); + void add_control_input(::google::protobuf::int32 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::int32 >& + control_input() const; + ::google::protobuf::RepeatedField< ::google::protobuf::int32 >* + mutable_control_input(); + + // string name = 1; + void clear_name(); + static const int kNameFieldNumber = 1; + const ::std::string& name() const; + void set_name(const ::std::string& value); + #if LANG_CXX11 + void set_name(::std::string&& value); + #endif + void set_name(const char* value); + void set_name(const char* value, size_t size); + ::std::string* mutable_name(); + ::std::string* release_name(); + void set_allocated_name(::std::string* name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_name( + ::std::string* name); + + // string device = 2; + void clear_device(); + static const int kDeviceFieldNumber = 2; + const ::std::string& device() const; + void set_device(const ::std::string& value); + #if LANG_CXX11 + void set_device(::std::string&& value); + #endif + void set_device(const char* value); + void set_device(const char* value, size_t size); + ::std::string* mutable_device(); + ::std::string* release_device(); + void set_allocated_device(::std::string* device); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_device(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_device( + ::std::string* device); + + // int64 temporary_memory_size = 6; + void clear_temporary_memory_size(); + static const int kTemporaryMemorySizeFieldNumber = 6; + ::google::protobuf::int64 temporary_memory_size() const; + void set_temporary_memory_size(::google::protobuf::int64 value); + + // int64 compute_cost = 9; + void clear_compute_cost(); + static const int kComputeCostFieldNumber = 9; + ::google::protobuf::int64 compute_cost() const; + void set_compute_cost(::google::protobuf::int64 value); + + // int32 id = 3; + void clear_id(); + static const int kIdFieldNumber = 3; + ::google::protobuf::int32 id() const; + void set_id(::google::protobuf::int32 value); + + // bool is_final = 7; + void clear_is_final(); + static const int kIsFinalFieldNumber = 7; + bool is_final() const; + void set_is_final(bool value); + + // bool inaccurate = 17; + void clear_inaccurate(); + static const int kInaccurateFieldNumber = 17; + bool inaccurate() const; + void set_inaccurate(bool value); + + // int64 host_temp_memory_size = 10 [deprecated = true]; + GOOGLE_PROTOBUF_DEPRECATED_ATTR void clear_host_temp_memory_size(); + GOOGLE_PROTOBUF_DEPRECATED_ATTR static const int kHostTempMemorySizeFieldNumber = 10; + GOOGLE_PROTOBUF_DEPRECATED_ATTR ::google::protobuf::int64 host_temp_memory_size() const; + GOOGLE_PROTOBUF_DEPRECATED_ATTR void set_host_temp_memory_size(::google::protobuf::int64 value); + + // int64 device_temp_memory_size = 11 [deprecated = true]; + GOOGLE_PROTOBUF_DEPRECATED_ATTR void clear_device_temp_memory_size(); + GOOGLE_PROTOBUF_DEPRECATED_ATTR static const int kDeviceTempMemorySizeFieldNumber = 11; + GOOGLE_PROTOBUF_DEPRECATED_ATTR ::google::protobuf::int64 device_temp_memory_size() const; + GOOGLE_PROTOBUF_DEPRECATED_ATTR void set_device_temp_memory_size(::google::protobuf::int64 value); + + // int64 persistent_memory_size = 12; + void clear_persistent_memory_size(); + static const int kPersistentMemorySizeFieldNumber = 12; + ::google::protobuf::int64 persistent_memory_size() const; + void set_persistent_memory_size(::google::protobuf::int64 value); + + // int64 compute_time = 14; + void clear_compute_time(); + static const int kComputeTimeFieldNumber = 14; + ::google::protobuf::int64 compute_time() const; + void set_compute_time(::google::protobuf::int64 value); + + // int64 memory_time = 15; + void clear_memory_time(); + static const int kMemoryTimeFieldNumber = 15; + ::google::protobuf::int64 memory_time() const; + void set_memory_time(::google::protobuf::int64 value); + + // int64 device_persistent_memory_size = 16 [deprecated = true]; + GOOGLE_PROTOBUF_DEPRECATED_ATTR void clear_device_persistent_memory_size(); + GOOGLE_PROTOBUF_DEPRECATED_ATTR static const int kDevicePersistentMemorySizeFieldNumber = 16; + GOOGLE_PROTOBUF_DEPRECATED_ATTR ::google::protobuf::int64 device_persistent_memory_size() const; + GOOGLE_PROTOBUF_DEPRECATED_ATTR void set_device_persistent_memory_size(::google::protobuf::int64 value); + + // @@protoc_insertion_point(class_scope:tensorflow.CostGraphDef.Node) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::CostGraphDef_Node_InputInfo > input_info_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::CostGraphDef_Node_OutputInfo > output_info_; + ::google::protobuf::RepeatedField< ::google::protobuf::int32 > control_input_; + mutable int _control_input_cached_byte_size_; + ::google::protobuf::internal::ArenaStringPtr name_; + ::google::protobuf::internal::ArenaStringPtr device_; + ::google::protobuf::int64 temporary_memory_size_; + ::google::protobuf::int64 compute_cost_; + ::google::protobuf::int32 id_; + bool is_final_; + bool inaccurate_; + ::google::protobuf::int64 host_temp_memory_size_; + ::google::protobuf::int64 device_temp_memory_size_; + ::google::protobuf::int64 persistent_memory_size_; + ::google::protobuf::int64 compute_time_; + ::google::protobuf::int64 memory_time_; + ::google::protobuf::int64 device_persistent_memory_size_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fcost_5fgraph_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class CostGraphDef : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.CostGraphDef) */ { + public: + CostGraphDef(); + virtual ~CostGraphDef(); + + CostGraphDef(const CostGraphDef& from); + + inline CostGraphDef& operator=(const CostGraphDef& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + CostGraphDef(CostGraphDef&& from) noexcept + : CostGraphDef() { + *this = ::std::move(from); + } + + inline CostGraphDef& operator=(CostGraphDef&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const CostGraphDef& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const CostGraphDef* internal_default_instance() { + return reinterpret_cast( + &_CostGraphDef_default_instance_); + } + static constexpr int kIndexInFileMessages = + 3; + + void UnsafeArenaSwap(CostGraphDef* other); + void Swap(CostGraphDef* other); + friend void swap(CostGraphDef& a, CostGraphDef& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline CostGraphDef* New() const final { + return CreateMaybeMessage(NULL); + } + + CostGraphDef* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const CostGraphDef& from); + void MergeFrom(const CostGraphDef& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(CostGraphDef* other); + protected: + explicit CostGraphDef(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + typedef CostGraphDef_Node Node; + + // accessors ------------------------------------------------------- + + // repeated .tensorflow.CostGraphDef.Node node = 1; + int node_size() const; + void clear_node(); + static const int kNodeFieldNumber = 1; + ::tensorflow::CostGraphDef_Node* mutable_node(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::CostGraphDef_Node >* + mutable_node(); + const ::tensorflow::CostGraphDef_Node& node(int index) const; + ::tensorflow::CostGraphDef_Node* add_node(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::CostGraphDef_Node >& + node() const; + + // @@protoc_insertion_point(class_scope:tensorflow.CostGraphDef) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::CostGraphDef_Node > node_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fcost_5fgraph_2eproto::TableStruct; +}; +// =================================================================== + + +// =================================================================== + +#ifdef __GNUC__ + #pragma GCC diagnostic push + #pragma GCC diagnostic ignored "-Wstrict-aliasing" +#endif // __GNUC__ +// CostGraphDef_Node_InputInfo + +// int32 preceding_node = 1; +inline void CostGraphDef_Node_InputInfo::clear_preceding_node() { + preceding_node_ = 0; +} +inline ::google::protobuf::int32 CostGraphDef_Node_InputInfo::preceding_node() const { + // @@protoc_insertion_point(field_get:tensorflow.CostGraphDef.Node.InputInfo.preceding_node) + return preceding_node_; +} +inline void CostGraphDef_Node_InputInfo::set_preceding_node(::google::protobuf::int32 value) { + + preceding_node_ = value; + // @@protoc_insertion_point(field_set:tensorflow.CostGraphDef.Node.InputInfo.preceding_node) +} + +// int32 preceding_port = 2; +inline void CostGraphDef_Node_InputInfo::clear_preceding_port() { + preceding_port_ = 0; +} +inline ::google::protobuf::int32 CostGraphDef_Node_InputInfo::preceding_port() const { + // @@protoc_insertion_point(field_get:tensorflow.CostGraphDef.Node.InputInfo.preceding_port) + return preceding_port_; +} +inline void CostGraphDef_Node_InputInfo::set_preceding_port(::google::protobuf::int32 value) { + + preceding_port_ = value; + // @@protoc_insertion_point(field_set:tensorflow.CostGraphDef.Node.InputInfo.preceding_port) +} + +// ------------------------------------------------------------------- + +// CostGraphDef_Node_OutputInfo + +// int64 size = 1; +inline void CostGraphDef_Node_OutputInfo::clear_size() { + size_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 CostGraphDef_Node_OutputInfo::size() const { + // @@protoc_insertion_point(field_get:tensorflow.CostGraphDef.Node.OutputInfo.size) + return size_; +} +inline void CostGraphDef_Node_OutputInfo::set_size(::google::protobuf::int64 value) { + + size_ = value; + // @@protoc_insertion_point(field_set:tensorflow.CostGraphDef.Node.OutputInfo.size) +} + +// int64 alias_input_port = 2; +inline void CostGraphDef_Node_OutputInfo::clear_alias_input_port() { + alias_input_port_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 CostGraphDef_Node_OutputInfo::alias_input_port() const { + // @@protoc_insertion_point(field_get:tensorflow.CostGraphDef.Node.OutputInfo.alias_input_port) + return alias_input_port_; +} +inline void CostGraphDef_Node_OutputInfo::set_alias_input_port(::google::protobuf::int64 value) { + + alias_input_port_ = value; + // @@protoc_insertion_point(field_set:tensorflow.CostGraphDef.Node.OutputInfo.alias_input_port) +} + +// .tensorflow.TensorShapeProto shape = 3; +inline bool CostGraphDef_Node_OutputInfo::has_shape() const { + return this != internal_default_instance() && shape_ != NULL; +} +inline const ::tensorflow::TensorShapeProto& CostGraphDef_Node_OutputInfo::_internal_shape() const { + return *shape_; +} +inline const ::tensorflow::TensorShapeProto& CostGraphDef_Node_OutputInfo::shape() const { + const ::tensorflow::TensorShapeProto* p = shape_; + // @@protoc_insertion_point(field_get:tensorflow.CostGraphDef.Node.OutputInfo.shape) + return p != NULL ? *p : *reinterpret_cast( + &::tensorflow::_TensorShapeProto_default_instance_); +} +inline ::tensorflow::TensorShapeProto* CostGraphDef_Node_OutputInfo::release_shape() { + // @@protoc_insertion_point(field_release:tensorflow.CostGraphDef.Node.OutputInfo.shape) + + ::tensorflow::TensorShapeProto* temp = shape_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + shape_ = NULL; + return temp; +} +inline ::tensorflow::TensorShapeProto* CostGraphDef_Node_OutputInfo::unsafe_arena_release_shape() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.CostGraphDef.Node.OutputInfo.shape) + + ::tensorflow::TensorShapeProto* temp = shape_; + shape_ = NULL; + return temp; +} +inline ::tensorflow::TensorShapeProto* CostGraphDef_Node_OutputInfo::mutable_shape() { + + if (shape_ == NULL) { + auto* p = CreateMaybeMessage<::tensorflow::TensorShapeProto>(GetArenaNoVirtual()); + shape_ = p; + } + // @@protoc_insertion_point(field_mutable:tensorflow.CostGraphDef.Node.OutputInfo.shape) + return shape_; +} +inline void CostGraphDef_Node_OutputInfo::set_allocated_shape(::tensorflow::TensorShapeProto* shape) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(shape_); + } + if (shape) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(shape)->GetArena(); + if (message_arena != submessage_arena) { + shape = ::google::protobuf::internal::GetOwnedMessage( + message_arena, shape, submessage_arena); + } + + } else { + + } + shape_ = shape; + // @@protoc_insertion_point(field_set_allocated:tensorflow.CostGraphDef.Node.OutputInfo.shape) +} + +// .tensorflow.DataType dtype = 4; +inline void CostGraphDef_Node_OutputInfo::clear_dtype() { + dtype_ = 0; +} +inline ::tensorflow::DataType CostGraphDef_Node_OutputInfo::dtype() const { + // @@protoc_insertion_point(field_get:tensorflow.CostGraphDef.Node.OutputInfo.dtype) + return static_cast< ::tensorflow::DataType >(dtype_); +} +inline void CostGraphDef_Node_OutputInfo::set_dtype(::tensorflow::DataType value) { + + dtype_ = value; + // @@protoc_insertion_point(field_set:tensorflow.CostGraphDef.Node.OutputInfo.dtype) +} + +// ------------------------------------------------------------------- + +// CostGraphDef_Node + +// string name = 1; +inline void CostGraphDef_Node::clear_name() { + name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& CostGraphDef_Node::name() const { + // @@protoc_insertion_point(field_get:tensorflow.CostGraphDef.Node.name) + return name_.Get(); +} +inline void CostGraphDef_Node::set_name(const ::std::string& value) { + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.CostGraphDef.Node.name) +} +#if LANG_CXX11 +inline void CostGraphDef_Node::set_name(::std::string&& value) { + + name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.CostGraphDef.Node.name) +} +#endif +inline void CostGraphDef_Node::set_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.CostGraphDef.Node.name) +} +inline void CostGraphDef_Node::set_name(const char* value, + size_t size) { + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.CostGraphDef.Node.name) +} +inline ::std::string* CostGraphDef_Node::mutable_name() { + + // @@protoc_insertion_point(field_mutable:tensorflow.CostGraphDef.Node.name) + return name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* CostGraphDef_Node::release_name() { + // @@protoc_insertion_point(field_release:tensorflow.CostGraphDef.Node.name) + + return name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void CostGraphDef_Node::set_allocated_name(::std::string* name) { + if (name != NULL) { + + } else { + + } + name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.CostGraphDef.Node.name) +} +inline ::std::string* CostGraphDef_Node::unsafe_arena_release_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.CostGraphDef.Node.name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void CostGraphDef_Node::unsafe_arena_set_allocated_name( + ::std::string* name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (name != NULL) { + + } else { + + } + name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.CostGraphDef.Node.name) +} + +// string device = 2; +inline void CostGraphDef_Node::clear_device() { + device_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& CostGraphDef_Node::device() const { + // @@protoc_insertion_point(field_get:tensorflow.CostGraphDef.Node.device) + return device_.Get(); +} +inline void CostGraphDef_Node::set_device(const ::std::string& value) { + + device_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.CostGraphDef.Node.device) +} +#if LANG_CXX11 +inline void CostGraphDef_Node::set_device(::std::string&& value) { + + device_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.CostGraphDef.Node.device) +} +#endif +inline void CostGraphDef_Node::set_device(const char* value) { + GOOGLE_DCHECK(value != NULL); + + device_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.CostGraphDef.Node.device) +} +inline void CostGraphDef_Node::set_device(const char* value, + size_t size) { + + device_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.CostGraphDef.Node.device) +} +inline ::std::string* CostGraphDef_Node::mutable_device() { + + // @@protoc_insertion_point(field_mutable:tensorflow.CostGraphDef.Node.device) + return device_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* CostGraphDef_Node::release_device() { + // @@protoc_insertion_point(field_release:tensorflow.CostGraphDef.Node.device) + + return device_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void CostGraphDef_Node::set_allocated_device(::std::string* device) { + if (device != NULL) { + + } else { + + } + device_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), device, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.CostGraphDef.Node.device) +} +inline ::std::string* CostGraphDef_Node::unsafe_arena_release_device() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.CostGraphDef.Node.device) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return device_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void CostGraphDef_Node::unsafe_arena_set_allocated_device( + ::std::string* device) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (device != NULL) { + + } else { + + } + device_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + device, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.CostGraphDef.Node.device) +} + +// int32 id = 3; +inline void CostGraphDef_Node::clear_id() { + id_ = 0; +} +inline ::google::protobuf::int32 CostGraphDef_Node::id() const { + // @@protoc_insertion_point(field_get:tensorflow.CostGraphDef.Node.id) + return id_; +} +inline void CostGraphDef_Node::set_id(::google::protobuf::int32 value) { + + id_ = value; + // @@protoc_insertion_point(field_set:tensorflow.CostGraphDef.Node.id) +} + +// repeated .tensorflow.CostGraphDef.Node.InputInfo input_info = 4; +inline int CostGraphDef_Node::input_info_size() const { + return input_info_.size(); +} +inline void CostGraphDef_Node::clear_input_info() { + input_info_.Clear(); +} +inline ::tensorflow::CostGraphDef_Node_InputInfo* CostGraphDef_Node::mutable_input_info(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.CostGraphDef.Node.input_info) + return input_info_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::CostGraphDef_Node_InputInfo >* +CostGraphDef_Node::mutable_input_info() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.CostGraphDef.Node.input_info) + return &input_info_; +} +inline const ::tensorflow::CostGraphDef_Node_InputInfo& CostGraphDef_Node::input_info(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.CostGraphDef.Node.input_info) + return input_info_.Get(index); +} +inline ::tensorflow::CostGraphDef_Node_InputInfo* CostGraphDef_Node::add_input_info() { + // @@protoc_insertion_point(field_add:tensorflow.CostGraphDef.Node.input_info) + return input_info_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::CostGraphDef_Node_InputInfo >& +CostGraphDef_Node::input_info() const { + // @@protoc_insertion_point(field_list:tensorflow.CostGraphDef.Node.input_info) + return input_info_; +} + +// repeated .tensorflow.CostGraphDef.Node.OutputInfo output_info = 5; +inline int CostGraphDef_Node::output_info_size() const { + return output_info_.size(); +} +inline void CostGraphDef_Node::clear_output_info() { + output_info_.Clear(); +} +inline ::tensorflow::CostGraphDef_Node_OutputInfo* CostGraphDef_Node::mutable_output_info(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.CostGraphDef.Node.output_info) + return output_info_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::CostGraphDef_Node_OutputInfo >* +CostGraphDef_Node::mutable_output_info() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.CostGraphDef.Node.output_info) + return &output_info_; +} +inline const ::tensorflow::CostGraphDef_Node_OutputInfo& CostGraphDef_Node::output_info(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.CostGraphDef.Node.output_info) + return output_info_.Get(index); +} +inline ::tensorflow::CostGraphDef_Node_OutputInfo* CostGraphDef_Node::add_output_info() { + // @@protoc_insertion_point(field_add:tensorflow.CostGraphDef.Node.output_info) + return output_info_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::CostGraphDef_Node_OutputInfo >& +CostGraphDef_Node::output_info() const { + // @@protoc_insertion_point(field_list:tensorflow.CostGraphDef.Node.output_info) + return output_info_; +} + +// int64 temporary_memory_size = 6; +inline void CostGraphDef_Node::clear_temporary_memory_size() { + temporary_memory_size_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 CostGraphDef_Node::temporary_memory_size() const { + // @@protoc_insertion_point(field_get:tensorflow.CostGraphDef.Node.temporary_memory_size) + return temporary_memory_size_; +} +inline void CostGraphDef_Node::set_temporary_memory_size(::google::protobuf::int64 value) { + + temporary_memory_size_ = value; + // @@protoc_insertion_point(field_set:tensorflow.CostGraphDef.Node.temporary_memory_size) +} + +// int64 persistent_memory_size = 12; +inline void CostGraphDef_Node::clear_persistent_memory_size() { + persistent_memory_size_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 CostGraphDef_Node::persistent_memory_size() const { + // @@protoc_insertion_point(field_get:tensorflow.CostGraphDef.Node.persistent_memory_size) + return persistent_memory_size_; +} +inline void CostGraphDef_Node::set_persistent_memory_size(::google::protobuf::int64 value) { + + persistent_memory_size_ = value; + // @@protoc_insertion_point(field_set:tensorflow.CostGraphDef.Node.persistent_memory_size) +} + +// int64 host_temp_memory_size = 10 [deprecated = true]; +inline void CostGraphDef_Node::clear_host_temp_memory_size() { + host_temp_memory_size_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 CostGraphDef_Node::host_temp_memory_size() const { + // @@protoc_insertion_point(field_get:tensorflow.CostGraphDef.Node.host_temp_memory_size) + return host_temp_memory_size_; +} +inline void CostGraphDef_Node::set_host_temp_memory_size(::google::protobuf::int64 value) { + + host_temp_memory_size_ = value; + // @@protoc_insertion_point(field_set:tensorflow.CostGraphDef.Node.host_temp_memory_size) +} + +// int64 device_temp_memory_size = 11 [deprecated = true]; +inline void CostGraphDef_Node::clear_device_temp_memory_size() { + device_temp_memory_size_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 CostGraphDef_Node::device_temp_memory_size() const { + // @@protoc_insertion_point(field_get:tensorflow.CostGraphDef.Node.device_temp_memory_size) + return device_temp_memory_size_; +} +inline void CostGraphDef_Node::set_device_temp_memory_size(::google::protobuf::int64 value) { + + device_temp_memory_size_ = value; + // @@protoc_insertion_point(field_set:tensorflow.CostGraphDef.Node.device_temp_memory_size) +} + +// int64 device_persistent_memory_size = 16 [deprecated = true]; +inline void CostGraphDef_Node::clear_device_persistent_memory_size() { + device_persistent_memory_size_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 CostGraphDef_Node::device_persistent_memory_size() const { + // @@protoc_insertion_point(field_get:tensorflow.CostGraphDef.Node.device_persistent_memory_size) + return device_persistent_memory_size_; +} +inline void CostGraphDef_Node::set_device_persistent_memory_size(::google::protobuf::int64 value) { + + device_persistent_memory_size_ = value; + // @@protoc_insertion_point(field_set:tensorflow.CostGraphDef.Node.device_persistent_memory_size) +} + +// int64 compute_cost = 9; +inline void CostGraphDef_Node::clear_compute_cost() { + compute_cost_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 CostGraphDef_Node::compute_cost() const { + // @@protoc_insertion_point(field_get:tensorflow.CostGraphDef.Node.compute_cost) + return compute_cost_; +} +inline void CostGraphDef_Node::set_compute_cost(::google::protobuf::int64 value) { + + compute_cost_ = value; + // @@protoc_insertion_point(field_set:tensorflow.CostGraphDef.Node.compute_cost) +} + +// int64 compute_time = 14; +inline void CostGraphDef_Node::clear_compute_time() { + compute_time_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 CostGraphDef_Node::compute_time() const { + // @@protoc_insertion_point(field_get:tensorflow.CostGraphDef.Node.compute_time) + return compute_time_; +} +inline void CostGraphDef_Node::set_compute_time(::google::protobuf::int64 value) { + + compute_time_ = value; + // @@protoc_insertion_point(field_set:tensorflow.CostGraphDef.Node.compute_time) +} + +// int64 memory_time = 15; +inline void CostGraphDef_Node::clear_memory_time() { + memory_time_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 CostGraphDef_Node::memory_time() const { + // @@protoc_insertion_point(field_get:tensorflow.CostGraphDef.Node.memory_time) + return memory_time_; +} +inline void CostGraphDef_Node::set_memory_time(::google::protobuf::int64 value) { + + memory_time_ = value; + // @@protoc_insertion_point(field_set:tensorflow.CostGraphDef.Node.memory_time) +} + +// bool is_final = 7; +inline void CostGraphDef_Node::clear_is_final() { + is_final_ = false; +} +inline bool CostGraphDef_Node::is_final() const { + // @@protoc_insertion_point(field_get:tensorflow.CostGraphDef.Node.is_final) + return is_final_; +} +inline void CostGraphDef_Node::set_is_final(bool value) { + + is_final_ = value; + // @@protoc_insertion_point(field_set:tensorflow.CostGraphDef.Node.is_final) +} + +// repeated int32 control_input = 8; +inline int CostGraphDef_Node::control_input_size() const { + return control_input_.size(); +} +inline void CostGraphDef_Node::clear_control_input() { + control_input_.Clear(); +} +inline ::google::protobuf::int32 CostGraphDef_Node::control_input(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.CostGraphDef.Node.control_input) + return control_input_.Get(index); +} +inline void CostGraphDef_Node::set_control_input(int index, ::google::protobuf::int32 value) { + control_input_.Set(index, value); + // @@protoc_insertion_point(field_set:tensorflow.CostGraphDef.Node.control_input) +} +inline void CostGraphDef_Node::add_control_input(::google::protobuf::int32 value) { + control_input_.Add(value); + // @@protoc_insertion_point(field_add:tensorflow.CostGraphDef.Node.control_input) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int32 >& +CostGraphDef_Node::control_input() const { + // @@protoc_insertion_point(field_list:tensorflow.CostGraphDef.Node.control_input) + return control_input_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int32 >* +CostGraphDef_Node::mutable_control_input() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.CostGraphDef.Node.control_input) + return &control_input_; +} + +// bool inaccurate = 17; +inline void CostGraphDef_Node::clear_inaccurate() { + inaccurate_ = false; +} +inline bool CostGraphDef_Node::inaccurate() const { + // @@protoc_insertion_point(field_get:tensorflow.CostGraphDef.Node.inaccurate) + return inaccurate_; +} +inline void CostGraphDef_Node::set_inaccurate(bool value) { + + inaccurate_ = value; + // @@protoc_insertion_point(field_set:tensorflow.CostGraphDef.Node.inaccurate) +} + +// ------------------------------------------------------------------- + +// CostGraphDef + +// repeated .tensorflow.CostGraphDef.Node node = 1; +inline int CostGraphDef::node_size() const { + return node_.size(); +} +inline void CostGraphDef::clear_node() { + node_.Clear(); +} +inline ::tensorflow::CostGraphDef_Node* CostGraphDef::mutable_node(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.CostGraphDef.node) + return node_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::CostGraphDef_Node >* +CostGraphDef::mutable_node() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.CostGraphDef.node) + return &node_; +} +inline const ::tensorflow::CostGraphDef_Node& CostGraphDef::node(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.CostGraphDef.node) + return node_.Get(index); +} +inline ::tensorflow::CostGraphDef_Node* CostGraphDef::add_node() { + // @@protoc_insertion_point(field_add:tensorflow.CostGraphDef.node) + return node_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::CostGraphDef_Node >& +CostGraphDef::node() const { + // @@protoc_insertion_point(field_list:tensorflow.CostGraphDef.node) + return node_; +} + +#ifdef __GNUC__ + #pragma GCC diagnostic pop +#endif // __GNUC__ +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + + +// @@protoc_insertion_point(namespace_scope) + +} // namespace tensorflow + +// @@protoc_insertion_point(global_scope) + +#endif // PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2fcost_5fgraph_2eproto diff --git a/linux/include/tensorflow/core/framework/cost_graph.pb_text-impl.h b/linux/include/tensorflow/core/framework/cost_graph.pb_text-impl.h new file mode 100644 index 000000000..fb2e2f857 --- /dev/null +++ b/linux/include/tensorflow/core/framework/cost_graph.pb_text-impl.h @@ -0,0 +1,50 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_framework_cost_graph_proto_IMPL_H_ +#define tensorflow_core_framework_cost_graph_proto_IMPL_H_ + +#include "tensorflow/core/framework/cost_graph.pb.h" +#include "tensorflow/core/framework/cost_graph.pb_text.h" +#include "tensorflow/core/framework/tensor_shape.pb.h" +#include "tensorflow/core/framework/tensor_shape.pb_text-impl.h" +#include "tensorflow/core/framework/types.pb.h" +#include "tensorflow/core/framework/types.pb_text-impl.h" +#include "tensorflow/core/lib/strings/proto_text_util.h" +#include "tensorflow/core/lib/strings/scanner.h" + +namespace tensorflow { + +namespace internal { + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::CostGraphDef_Node_InputInfo& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::CostGraphDef_Node_InputInfo* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::CostGraphDef_Node_OutputInfo& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::CostGraphDef_Node_OutputInfo* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::CostGraphDef_Node& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::CostGraphDef_Node* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::CostGraphDef& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::CostGraphDef* msg); + +} // namespace internal + +} // namespace tensorflow + +#endif // tensorflow_core_framework_cost_graph_proto_IMPL_H_ diff --git a/linux/include/tensorflow/core/framework/cost_graph.pb_text.h b/linux/include/tensorflow/core/framework/cost_graph.pb_text.h new file mode 100644 index 000000000..2c1825331 --- /dev/null +++ b/linux/include/tensorflow/core/framework/cost_graph.pb_text.h @@ -0,0 +1,54 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_framework_cost_graph_proto_H_ +#define tensorflow_core_framework_cost_graph_proto_H_ + +#include "tensorflow/core/framework/cost_graph.pb.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/protobuf.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +// Message-text conversion for tensorflow.CostGraphDef.Node.InputInfo +string ProtoDebugString( + const ::tensorflow::CostGraphDef_Node_InputInfo& msg); +string ProtoShortDebugString( + const ::tensorflow::CostGraphDef_Node_InputInfo& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::CostGraphDef_Node_InputInfo* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.CostGraphDef.Node.OutputInfo +string ProtoDebugString( + const ::tensorflow::CostGraphDef_Node_OutputInfo& msg); +string ProtoShortDebugString( + const ::tensorflow::CostGraphDef_Node_OutputInfo& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::CostGraphDef_Node_OutputInfo* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.CostGraphDef.Node +string ProtoDebugString( + const ::tensorflow::CostGraphDef_Node& msg); +string ProtoShortDebugString( + const ::tensorflow::CostGraphDef_Node& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::CostGraphDef_Node* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.CostGraphDef +string ProtoDebugString( + const ::tensorflow::CostGraphDef& msg); +string ProtoShortDebugString( + const ::tensorflow::CostGraphDef& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::CostGraphDef* msg) + TF_MUST_USE_RESULT; + +} // namespace tensorflow + +#endif // tensorflow_core_framework_cost_graph_proto_H_ diff --git a/linux/include/tensorflow/core/framework/dataset.h b/linux/include/tensorflow/core/framework/dataset.h new file mode 100644 index 000000000..8c1151cb5 --- /dev/null +++ b/linux/include/tensorflow/core/framework/dataset.h @@ -0,0 +1,860 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CORE_FRAMEWORK_DATASET_H_ +#define TENSORFLOW_CORE_FRAMEWORK_DATASET_H_ + +#include +#include + +#include "tensorflow/core/framework/attr_value.pb.h" +#include "tensorflow/core/framework/attr_value_util.h" +#include "tensorflow/core/framework/dataset_stateful_op_whitelist.h" +#include "tensorflow/core/framework/function.h" +#include "tensorflow/core/framework/graph.pb.h" +#include "tensorflow/core/framework/model.h" +#include "tensorflow/core/framework/node_def.pb.h" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/register_types.h" +#include "tensorflow/core/framework/types.pb.h" +#include "tensorflow/core/framework/variant_encode_decode.h" +#include "tensorflow/core/framework/variant_tensor_data.h" +#include "tensorflow/core/lib/strings/str_util.h" +#include "tensorflow/core/lib/strings/strcat.h" +#include "tensorflow/core/platform/tracing.h" + +// Polymorphic datasets should support all primitive TensorFlow +// types. Use this macro to expand `m(T)` once for each primitive type +// `T`, e.g. to build a `switch` statement. +#define TF_CALL_DATASET_TYPES(m) TF_CALL_ALL_TYPES(m) TF_CALL_QUANTIZED_TYPES(m) + +namespace tensorflow { + +// Forward declarations to avoid introducing a dependency on headers in +// "tensorflow/core/graph/...". +class GraphDefBuilder; +class Node; + +namespace data { +// A constant that can be used to enable auto-tuning. +constexpr int kAutoTune = -1; + +class DatasetBase; +class SerializationContext; + +// Interface for reading values from a key-value store. +// Used for restoring iterator state. +class IteratorStateReader { + public: + virtual Status ReadScalar(StringPiece key, int64* val) = 0; + virtual Status ReadScalar(StringPiece key, string* val) = 0; + virtual Status ReadTensor(StringPiece key, Tensor* val) = 0; + virtual bool Contains(StringPiece key) = 0; + + virtual ~IteratorStateReader() {} +}; + +// Interface for writing values to a key-value store. +// Used for saving iterator state. +class IteratorStateWriter { + public: + virtual Status WriteScalar(StringPiece key, const int64 val) = 0; + virtual Status WriteScalar(StringPiece key, const string& val) = 0; + virtual Status WriteTensor(StringPiece key, const Tensor& val) = 0; + + virtual ~IteratorStateWriter() {} +}; + +// Wrapper around GraphDefBuilder. Used to serialize Dataset graph. +class GraphDefBuilderWrapper { + public: + explicit GraphDefBuilderWrapper(GraphDefBuilder* b) : b_(b) {} + + // Adds a Const node with scalar value to the Graph. + // `*output` contains a pointer to the output `Node`. It is guaranteed to be + // non-null if the method returns with an OK status. + // The returned Node pointer is owned by the backing Graph of GraphDefBuilder. + template + Status AddScalar(const T& val, Node** output) { + Tensor val_t = Tensor(DataTypeToEnum::v(), TensorShape({})); + val_t.scalar()() = val; + AddTensorInternal(val_t, output); + if (*output == nullptr) { + return errors::Internal("AddScalar: Failed to build Const op."); + } + return Status::OK(); + } + + // Adds a Const node with vector value to the Graph. + // `*output` contains a pointer to the output `Node`. It is guaranteed to be + // non-null if the method returns with an OK status. + // The returned Node pointer is owned by the backing Graph of GraphDefBuilder. + // TODO(shivaniagrawal): Consider changing to gtl::ArraySlice? + template + Status AddVector(const std::vector& val, Node** output) { + Tensor val_t = Tensor(DataTypeToEnum::v(), + TensorShape({static_cast(val.size())})); + for (int i = 0; i < val.size(); i++) { + val_t.flat()(i) = val[i]; + } + AddTensorInternal(val_t, output); + if (*output == nullptr) { + return errors::Internal("AddVector: Failed to build Const op."); + } + return Status::OK(); + } + + // Adds a `Const` node for the given tensor value to the graph. + // + // `*output` contains a pointer to the output `Node`. It is guaranteed to be + // non-null if the method returns with an OK status. The returned `Node` + // pointer is owned by the backing graph of `GraphDefBuilder`. + Status AddTensor(const Tensor& val, Node** output) { + AddTensorInternal(val, output); + if (*output == nullptr) { + return errors::Internal("AddTensor: Failed to build Const op."); + } + return Status::OK(); + } + + // Adds a `Placeholder` node for the given tensor value to the graph. + // + // `*output` contains a pointer to the output `Node`. It is guaranteed to be + // non-null if the method returns with an OK status. The returned `Node` + // pointer is owned by the backing graph of `GraphDefBuilder`. + Status AddPlaceholder(const Tensor& val, Node** output) { + AddPlaceholderInternal(val, output); + if (*output == nullptr) { + return errors::Internal( + "AddPlaceholder: Failed to build Placeholder op."); + } + return Status::OK(); + } + + Status AddDataset(const DatasetBase* dataset, + const std::vector& inputs, Node** output) { + return AddDataset(dataset, inputs, {}, output); + } + + // Adds a node corresponding to the `DatasetType` to the Graph. + // Return value of `DatasetType::op_name()` is used as the op type for the + // node. + // Values for the output_types and output_shapes node attributes are also + // written if those attributes are defined in the OpDef. + // `*output` contains a pointer to the output `Node`. It is guaranteed to be + // non-null if the method returns with an OK status. + // The returned Node pointer is owned by the backing Graph of GraphDefBuilder. + Status AddDataset(const DatasetBase* dataset, + const std::vector& inputs, + const std::vector>& attrs, + Node** output) { + std::vector> enumerated_inputs(inputs.size()); + for (int i = 0; i < inputs.size(); i++) { + enumerated_inputs[i] = std::make_pair(i, inputs[i]); + } + return AddDataset(dataset, enumerated_inputs, {}, attrs, output); + } + + Status AddDataset( + const DatasetBase* dataset, + const std::vector>& inputs, + const std::vector>>& list_inputs, + const std::vector>& attrs, + Node** output); + + // Adds a user-defined function with name `function_name` to the graph and + // recursively adds all functions it references. If a function with a matching + // name has already been added, returns with OK status. If a user-defined with + // name `function_name` is not found in the context's function library, + // returns an InvalidArgumentError. If the function with name `function_name` + // or any of its dependent functions are stateful, and the context does not + // explicitly permit stateful functions, returns an InvalidArgument error. + Status AddFunction(SerializationContext* ctx, const string& function_name); + + template + void BuildAttrValue(const T& value, AttrValue* attr) { + SetAttrValue(value, attr); + } + + private: + void AddPlaceholderInternal(const Tensor& val, Node** output); + void AddTensorInternal(const Tensor& val, Node** output); + + Status EnsureFunctionIsStateless(const FunctionLibraryDefinition& flib_def, + const string& function_name) const { + const FunctionDef* function_def = flib_def.Find(function_name); + if (!function_def) { + return errors::InvalidArgument("Unable to find FunctionDef for ", + function_name, " in registry."); + } + for (const NodeDef& node_def : function_def->node_def()) { + const OpDef* op_def; + TF_RETURN_IF_ERROR(flib_def.LookUpOpDef(node_def.op(), &op_def)); + // TODO(b/65524810): Hack to allow functions to capture Dataset op + // nodes needed for FlatMap. Currently, source datasets nodes have been + // marked stateful to avoid constant folding since we do not have a + // good way of serializing them. + if (IsOpWhitelisted(op_def)) { + continue; + } + if (op_def->is_stateful()) { + return errors::InvalidArgument( + "Op[name: ", node_def.name(), ", type: ", node_def.op(), "] ", + "in function ", function_name, " is stateful. ", + "Saving stateful functions is not supported yet."); + } + } + return Status::OK(); + } + + // Returns whether an op has been whitelisted for use inside map_fns. + // Uses a heuristic to whitelist source dataset ops which have been + // marked stateful due to b/65524810. + // Also looks up the `op_def->name` in the global + // `WhitelistedStatefulOpRegistry`. + bool IsOpWhitelisted(const OpDef* op_def) const { + return (str_util::EndsWith(op_def->name(), "Dataset") && + op_def->output_arg_size() == 1 && + op_def->output_arg(0).type() == DT_VARIANT) || + WhitelistedStatefulOpRegistry::Global()->Contains(op_def->name()); + } + + bool HasAttr(const string& op_type_name, const string& attr_name) const; + + bool HasAttr(const OpDef* op_def, const string& attr_name) const { + for (auto attr : op_def->attr()) { + if (attr.name() == attr_name) { + return true; + } + } + return false; + } + + Status AddAttrFunctions(SerializationContext* ctx, + const AttrValue& attr_value) { + if (attr_value.has_func()) { + TF_RETURN_IF_ERROR(AddFunction(ctx, attr_value.func().name())); + } else if (attr_value.has_list()) { + for (const NameAttrList& name_attr_list : attr_value.list().func()) { + TF_RETURN_IF_ERROR(AddFunction(ctx, name_attr_list.name())); + } + } + return Status::OK(); + } + + GraphDefBuilder* b_; +}; + +class StatsAggregator; + +// A cut-down version of `OpKernelContext` for running computations in +// iterators. Note that we cannot simply use `OpKernelContext` here because we +// might run computation in an iterator whose lifetime is not nested within the +// lifetime of a single `OpKernelContext` (e.g. asynchronous prefetching). +// +// TODO(mrry): We're making some daring assumptions about the lifetime of the +// runner passed in here. A runner will be deleted when the original step ends, +// but all existing runners only close over session-lifetime (or longer-lived) +// state, so we can make a copy of the function. There's nothing in the +// definition of the API from which we took the runner to guarantee that what we +// are doing is safe. We should formalize the properties here. +class IteratorContext { + public: + struct Params { + // Interface to operating system functionality. + Env* env; + + // Function call support. + std::function)> runner = nullptr; + + // A function that returns the current `StatsAggregator` instance to be + // used when recording statistics about the iterator. + // + // NOTE(mrry): This is somewhat awkward, because (i) the `StatsAggregator` + // is a property of the `IteratorResource` (which this class does not know + // about), and (ii) it can change after the `IteratorContext` has been + // created. Better suggestions are welcome! + std::function()> stats_aggregator_getter = + nullptr; + + // The FunctionLibraryRuntime object to be used to make function calls. + FunctionLibraryRuntime* lib = nullptr; + std::shared_ptr function_library = nullptr; + + // The Allocator to be used to allocate the output of an iterator. + std::function allocator_getter = nullptr; + + // If non-null, identifies the object used for performance modeling. + std::shared_ptr model = nullptr; + }; + + explicit IteratorContext(Params params) : params_(std::move(params)) {} + + explicit IteratorContext(OpKernelContext* ctx) { + params_.env = ctx->env(); + params_.runner = *(ctx->runner()); + params_.lib = ctx->function_library(); + // NOTE: must use reinterpret_cast because function.h forward-declares + // Device. + DeviceBase* device = + reinterpret_cast(ctx->function_library()->device()); + params_.allocator_getter = [device](AllocatorAttributes attrs) { + return device->GetAllocator(attrs); + }; + } + + Env* env() const { return params_.env; } + + std::function)>* runner() { + return ¶ms_.runner; + } + + std::shared_ptr stats_aggregator() { + if (params_.stats_aggregator_getter) { + return params_.stats_aggregator_getter(); + } else { + return nullptr; + } + } + + std::shared_ptr function_library() { + return params_.function_library; + } + + FunctionLibraryRuntime* lib() { return params_.lib; } + + void set_lib(FunctionLibraryRuntime* lib) { params_.lib = lib; } + + Allocator* allocator(AllocatorAttributes attrs) { + return params_.allocator_getter(attrs); + } + + std::function allocator_getter() { + return params_.allocator_getter; + } + + std::function()> stats_aggregator_getter() { + return params_.stats_aggregator_getter; + } + + std::shared_ptr model() { return params_.model; } + + Params params() { return params_; } + + private: + Params params_; +}; + +// Aggregates runtime support needed for dataset and iterator serialization. +class SerializationContext { + public: + struct Params { + bool allow_stateful_functions = false; + const FunctionLibraryDefinition* flib_def = nullptr; // Not owned. + std::vector>* input_list = nullptr; // Not owned. + }; + + explicit SerializationContext(Params params) : params_(std::move(params)) {} + + bool allow_stateful_functions() { return params_.allow_stateful_functions; } + + const FunctionLibraryDefinition& flib_def() { return *params_.flib_def; } + + std::vector>* input_list() { + return params_.input_list; + } + + private: + Params params_; + + TF_DISALLOW_COPY_AND_ASSIGN(SerializationContext); +}; + +// Represents the current position in a range of outputs, where the +// range of outputs is typically represented by an `DatasetBase`, +// defined below. +class IteratorBase { + public: + virtual ~IteratorBase() { + for (auto rit = cleanup_fns_.rbegin(); rit != cleanup_fns_.rend(); ++rit) { + (*rit)(); + } + } + + // Gets the next output from the range that this iterator is traversing. + // + // If at least one output remains in this iterator's range, that + // output will be stored in `*out_tensors` and `false` will be + // stored in `*end_of_sequence`. + // + // If no more outputs remain in this iterator's range, `true` will + // be stored in `*end_of_sequence`, and the content of + // `*out_tensors` will be undefined. + // + // This method is thread-safe. + // + // TODO(mrry): Define `GetNextAsync()` or `GetNextManyAsync()`, and + // potentially remove this method. + virtual Status GetNext(IteratorContext* ctx, std::vector* out_tensors, + bool* end_of_sequence) = 0; + + Status GetNext(IteratorContext&& ctx, std::vector* out_tensors, + bool* end_of_sequence) { + return GetNext(&ctx, out_tensors, end_of_sequence); + } + + // Returns a vector of DataType values, representing the respective + // element types of each tuple component in the outputs of this + // iterator. + virtual const DataTypeVector& output_dtypes() const = 0; + + // Returns a vector of tensor shapes, representing the respective + // (and possibly partially defined) shapes of each tuple component + // in the outputs of this iterator. + virtual const std::vector& output_shapes() const = 0; + + // Returns a string that identifies the sequence of iterators leading up to + // this iterator. + virtual const string& prefix() const = 0; + + // Performs initialization that needs to happen outside of a constructor to + // properly propagate errors. + virtual Status Initialize(IteratorContext* ctx) { return Status::OK(); } + + // Saves the state of this iterator. + virtual Status Save(SerializationContext* ctx, IteratorStateWriter* writer) { + return SaveInternal(writer); + } + + // Restores the state of this iterator. + virtual Status Restore(IteratorContext* ctx, IteratorStateReader* reader) { + return RestoreInternal(ctx, reader); + } + + protected: + // This is needed so that sub-classes of IteratorBase can call + // `SaveInternal` on their input iterators. + Status SaveInput(IteratorStateWriter* writer, + const std::unique_ptr& input) { + return input->SaveInternal(writer); + } + + // This is needed so that sub-classes of IteratorBase can call + // `RestoreInternal` on their input iterators. + Status RestoreInput(IteratorContext* ctx, IteratorStateReader* reader, + const std::unique_ptr& input) { + return input->RestoreInternal(ctx, reader); + } + + // Saves the state of this iterator recursively. + virtual Status SaveInternal(IteratorStateWriter* writer) { + return errors::Unimplemented("SaveInternal"); + } + + // Restores the state of this iterator recursively. + virtual Status RestoreInternal(IteratorContext* ctx, + IteratorStateReader* reader) { + return errors::Unimplemented("RestoreInternal"); + } + + private: + friend class DatasetBase; // for access to `AddCleanupFunction` + + // Registers a cleanup function to be called upon object destruction. + // + // Registered functions are invoked in the reserve order of registration. + void AddCleanupFunction(std::function&& cleanup_fn) { + cleanup_fns_.push_back(std::move(cleanup_fn)); + } + + std::vector> cleanup_fns_; +}; + +// Represents runtime information needed to construct a dataset. +class DatasetContext { + public: + struct Params { + string name; + }; + + explicit DatasetContext(Params params) : params_(std::move(params)) {} + + explicit DatasetContext(OpKernelContext* ctx) { + params_.name = ctx->op_kernel().type_string(); + } + + const string& name() const { return params_.name; } + + private: + Params params_; +}; + +// Represents a (potentially infinite) range of outputs, where each +// output is a tuple of tensors. +class DatasetBase : public core::RefCounted { + public: + // Key for storing the Dataset graph in the serialized format. + TF_EXPORT static const char kDatasetGraphKey[]; + + // Key for storing the output node of the Dataset graph in the serialized + // format. + TF_EXPORT static const char kDatasetGraphOutputNodeKey[]; + + explicit DatasetBase(DatasetContext&& ctx) : name_(ctx.name()) {} + + const string& name() const { return name_; } + + // Returns a new iterator for iterating over the range of elements in + // this dataset. + // + // This method may be called multiple times on the same instance, + // and the resulting iterators will have distinct state. Each + // iterator will traverse all elements in this dataset from the + // start. + // + // The prefix identifies the sequence of iterators leading up to the newly + // created iterator. + Status MakeIterator(IteratorContext* ctx, const string& prefix, + std::unique_ptr* iterator) const { + *iterator = MakeIteratorInternal(prefix); + if (ctx->model()) { + ctx->model()->AddNode((*iterator)->prefix(), prefix); + std::shared_ptr model = ctx->model(); + const string& prefix = (*iterator)->prefix(); + (*iterator)->AddCleanupFunction( + [model, prefix]() { model->RemoveNode(prefix); }); + } + return (*iterator)->Initialize(ctx); + } + + Status MakeIterator(IteratorContext&& ctx, const string& prefix, + std::unique_ptr* iterator) const { + return MakeIterator(&ctx, prefix, iterator); + } + + // Returns a vector of DataType values, representing the respective + // element types of each tuple component in the outputs of this + // dataset. + virtual const DataTypeVector& output_dtypes() const = 0; + + // Returns a vector of tensor shapes, representing the respective + // (and possibly partially defined) shapes of each tuple component + // in the outputs of this dataset. + virtual const std::vector& output_shapes() const = 0; + + // A human-readable debug string for this dataset. + virtual string DebugString() const = 0; + + // Serializes the dataset and writes it to the `writer`. + virtual Status Save(SerializationContext* ctx, + IteratorStateWriter* writer) const; + + protected: + friend class DatasetToGraphOp; // For access to graph related members. + + class DatasetGraphDefBuilder : public GraphDefBuilderWrapper { + public: + DatasetGraphDefBuilder(GraphDefBuilder* b) : GraphDefBuilderWrapper(b) {} + Status AddInputDataset(SerializationContext* ctx, + const DatasetBase* dataset, Node** output) { + return dataset->AsGraphDefInternal(ctx, this, output); + } + }; + + // TODO(jsimsa): Consolidate overloading into a single method. + virtual Status AsGraphDefInternal(SerializationContext* ctx, + DatasetGraphDefBuilder* b, + Node** node) const = 0; + + virtual std::unique_ptr MakeIteratorInternal( + const string& prefix) const = 0; + + private: + const string name_; +}; + +// Represents an iterator that is associated with a particular dataset. +class DatasetBaseIterator : public IteratorBase { + public: + struct BaseParams { + // Owns one reference on the shared dataset object. + const DatasetBase* dataset; + + // Identifies the sequence of iterators leading up to this iterator. + const string prefix; + }; + + explicit DatasetBaseIterator(const BaseParams& params) : params_(params) { + params_.dataset->Ref(); + } + + ~DatasetBaseIterator() override { params_.dataset->Unref(); } + + // The sequence of iterators leading up to this iterator. + const string& prefix() const override { return params_.prefix; } + + const DataTypeVector& output_dtypes() const override { + return params_.dataset->output_dtypes(); + } + + const std::vector& output_shapes() const override { + return params_.dataset->output_shapes(); + } + + Status GetNext(IteratorContext* ctx, std::vector* out_tensors, + bool* end_of_sequence) final { + tracing::ScopedActivity activity(params_.prefix); + RecordStart(ctx, true /* stop_output */); + Status s = GetNextInternal(ctx, out_tensors, end_of_sequence); + if (s.ok() && !*end_of_sequence) RecordElement(ctx); + RecordStop(ctx, true /* start_output */); + if (TF_PREDICT_FALSE(errors::IsOutOfRange(s) && !*end_of_sequence)) { + s = errors::Internal( + "Iterator \"", params_.prefix, + "\" returned OutOfRange without setting `*end_of_sequence`. This " + "indicates that an error may have occurred. Original message: ", + s.error_message()); + LOG(ERROR) << s; + } + return s; + } + + Status Save(SerializationContext* ctx, IteratorStateWriter* writer) final { + TF_RETURN_IF_ERROR(params_.dataset->Save(ctx, writer)); + return IteratorBase::Save(ctx, writer); + } + + protected: + // Internal implementation of GetNext that is wrapped in tracing logic. + virtual Status GetNextInternal(IteratorContext* ctx, + std::vector* out_tensors, + bool* end_of_sequence) = 0; + + string full_name(const string& name) const { + return strings::StrCat(params_.prefix, ":", name); + } + + // When performance modeling is enabled, this method adds a constant parameter + // to the model node corresponding to this iterator. + void AddConstantParameter(IteratorContext* ctx, const string& name, + int64 value) { + if (ctx->model()) { + ctx->model()->AddConstantParameter(prefix(), name, value); + } + } + + // When performance modeling is enabled, this method adds a tunable parameter + // to the model node corresponding to this iterator. + // + // The performance modeling logic may use `state` to set the value of the + // tunable parameter at any point during the lifetime of this iterator. When + // it does, it acquires `state->mu` and notifies `state->cond_var`. + void AddTunableParameter(IteratorContext* ctx, const string& name, + std::shared_ptr state, int64 min, + int64 max) { + if (ctx->model()) { + ctx->model()->AddTunableParameter(prefix(), name, std::move(state), min, + max); + } + } + + // When performance modeling is enabled, this method records the fact that + // this iterator has produced an element. + void RecordElement(IteratorContext* ctx) { + if (ctx->model()) { + ctx->model()->RecordElement(prefix()); + } + } + + // When performance modeling is enabled, this method records the fact that + // a thread of this iterator has started work. + void RecordStart(IteratorContext* ctx, bool stop_output = false) { + if (ctx->model()) { + ctx->model()->RecordStart(prefix(), stop_output); + } + } + + // When performance modeling is enabled, this method records the fact that + // a thread of this iterator has stopped work. + void RecordStop(IteratorContext* ctx, bool start_output = false) { + if (ctx->model()) { + ctx->model()->RecordStop(prefix(), start_output); + } + } + + private: + BaseParams params_; +}; + +// Represents an iterator that is associated with a particular dataset +// with a particular type. +template +class DatasetIterator : public DatasetBaseIterator { + public: + struct Params { + // Borrowed pointer to the dataset. + const DatasetType* dataset; + + // Identifies the sequence of iterators leading up to this iterator. + const string prefix; + }; + + explicit DatasetIterator(const Params& params) + : DatasetBaseIterator({params.dataset, params.prefix}), + typed_dataset_(params.dataset) {} + + // The dataset from which this iterator was created. + const DatasetType* dataset() const { return typed_dataset_; } + + protected: + virtual Status GetNextInternal(IteratorContext* ctx, + std::vector* out_tensors, + bool* end_of_sequence) = 0; + + private: + const DatasetType* const typed_dataset_; // Not owned. +}; + +// Encapsulates the work required to plug a DatasetBase into the core TensorFlow +// graph execution engine. +class DatasetOpKernel : public OpKernel { + public: + DatasetOpKernel(OpKernelConstruction* ctx) : OpKernel(ctx) {} + void Compute(OpKernelContext* ctx) final; + + protected: + // Subclasses should implement this method. It will be called during Compute + // execution. + virtual void MakeDataset(OpKernelContext* ctx, DatasetBase** output) = 0; + + template + Status ParseScalarArgument(OpKernelContext* ctx, + const StringPiece& argument_name, T* output) { + const Tensor* argument_t; + TF_RETURN_IF_ERROR(ctx->input(argument_name, &argument_t)); + if (!TensorShapeUtils::IsScalar(argument_t->shape())) { + return errors::InvalidArgument(argument_name, " must be a scalar"); + } + *output = argument_t->scalar()(); + return Status::OK(); + } + + template + Status ParseVectorArgument(OpKernelContext* ctx, + const StringPiece& argument_name, + std::vector* output) { + const Tensor* argument_t; + TF_RETURN_IF_ERROR(ctx->input(argument_name, &argument_t)); + if (!TensorShapeUtils::IsVector(argument_t->shape())) { + return errors::InvalidArgument(argument_name, " must be a vector"); + } + int size = argument_t->vec().size(); + output->reserve(size); + for (int i = 0; i < size; ++i) { + output->push_back(argument_t->vec()(i)); + } + return Status::OK(); + } +}; + +// Encapsulates the work required to plug unary Datasets into the core +// TensorFlow graph execution engine. +class UnaryDatasetOpKernel : public DatasetOpKernel { + public: + UnaryDatasetOpKernel(OpKernelConstruction* ctx) : DatasetOpKernel(ctx) {} + + protected: + void MakeDataset(OpKernelContext* ctx, DatasetBase** output) final; + virtual void MakeDataset(OpKernelContext* ctx, DatasetBase* input, + DatasetBase** output) = 0; +}; + +// Encapsulates the work required to plug binary Datasets into the core +// TensorFlow graph execution engine. +class BinaryDatasetOpKernel : public DatasetOpKernel { + public: + BinaryDatasetOpKernel(OpKernelConstruction* ctx) : DatasetOpKernel(ctx) {} + + protected: + void MakeDataset(OpKernelContext* ctx, DatasetBase** output) final; + virtual void MakeDataset(OpKernelContext* ctx, DatasetBase* input, + DatasetBase* another_input, + DatasetBase** output) = 0; +}; + +// Validates and extracts a `DatasetBase` object from `tensor`. +// +// `tensor` must have been written by a call to SetVariantTensorToDataset(). +// +// The retrieved pointer is a borrowed reference to the dataset, which is owned +// by the tensor. The consumer must either acquire its own reference to the +// dataset by calling `(*out_dataset)->Ref()`, or ensure that `tensor` is not +// destroyed or mutated while the retrieved pointer is in use. +Status GetDatasetFromVariantTensor(const Tensor& tensor, + DatasetBase** out_dataset); + +// Stores a `DatasetBase` object in `tensor`. +// +// The ownership of `dataset` is transferred to `tensor`. +Status StoreDatasetInVariantTensor(DatasetBase* dataset, Tensor* tensor); + +// A simple background worker that executes closures asynchronously and without +// blocking. +// +// A `BackgroundWorker` is used to offload blocking work from an `AsyncOpKernel` +// to avoid blocking an executor thread that may be required by the blocking +// work. +// +// NOTE(mrry): We do not use a regular `tensorflow::thread::ThreadPool` for this +// purpose because its current implementation (in Eigen) uses a finite-length +// queue and will block the caller when full. This can lead to deadlock under +// heavy load. Since the number of concurrent work items in each user of a +// `BackgroundWorker` is at most one per op invocation, the dynamic allocation +// overhead is tolerable. +class BackgroundWorker { + public: + BackgroundWorker(Env* env, const string& name); + + ~BackgroundWorker(); + + void Schedule(std::function work_item); + + private: + void WorkerLoop(); + + std::unique_ptr thread_; + mutex mu_; + condition_variable cond_var_; + bool cancelled_ GUARDED_BY(mu_) = false; + std::deque> work_queue_ GUARDED_BY(mu_); +}; + +} // namespace data + +// TODO(b/114112161): Remove these aliases when all users have moved over to the +// `tensorflow::data` namespace. +using data::DatasetBase; +using data::DatasetContext; +using data::DatasetIterator; +using data::DatasetOpKernel; +using data::IteratorBase; +using data::IteratorContext; +using data::IteratorStateReader; +using data::IteratorStateWriter; +using data::SerializationContext; +using data::UnaryDatasetOpKernel; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_FRAMEWORK_DATASET_H_ diff --git a/linux/include/tensorflow/core/framework/dataset_stateful_op_whitelist.h b/linux/include/tensorflow/core/framework/dataset_stateful_op_whitelist.h new file mode 100644 index 000000000..74bd39cb6 --- /dev/null +++ b/linux/include/tensorflow/core/framework/dataset_stateful_op_whitelist.h @@ -0,0 +1,76 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_FRAMEWORK_DATASET_STATEFUL_OP_WHITELIST_H_ +#define TENSORFLOW_CORE_FRAMEWORK_DATASET_STATEFUL_OP_WHITELIST_H_ + +#include +#include "tensorflow/core/lib/core/status.h" + +namespace tensorflow { +namespace data { +// Registry for stateful ops that need to be used in dataset functions. +// See below macro for usage details. +class WhitelistedStatefulOpRegistry { + public: + Status Add(string op_name) { + op_names_.insert(std::move(op_name)); + return Status::OK(); + } + + bool Contains(const string& op_name) { return op_names_.count(op_name); } + + static WhitelistedStatefulOpRegistry* Global() { + static auto* reg = new WhitelistedStatefulOpRegistry; + return reg; + } + + private: + WhitelistedStatefulOpRegistry() = default; + WhitelistedStatefulOpRegistry(WhitelistedStatefulOpRegistry const& copy) = + delete; + WhitelistedStatefulOpRegistry operator=( + WhitelistedStatefulOpRegistry const& copy) = delete; + + std::unordered_set op_names_; +}; + +} // namespace data + +// Use this macro to whitelist an op that is marked stateful but needs to be +// used inside a map_fn in an input pipeline. This is only needed if you wish +// to be able to checkpoint the state of the input pipeline. We currently +// do not allow stateful ops to be defined inside of map_fns since it is not +// possible to save their state. +// Note that the state of the whitelisted ops inside functions will not be +// saved during checkpointing, hence this should only be used if the op is +// marked stateful for reasons like to avoid constant folding during graph +// optimiztion but is not stateful. +// If possible, try to remove the stateful flag on the op first. +// Example usage: +// +// WHITELIST_STATEFUL_OP_FOR_DATASET_FUNCTIONS("LegacyStatefulReader"); +// +#define WHITELIST_STATEFUL_OP_FOR_DATASET_FUNCTIONS(name) \ + WHITELIST_STATEFUL_OP_FOR_DATASET_FUNCTIONS_UNIQ_HELPER(__COUNTER__, name) +#define WHITELIST_STATEFUL_OP_FOR_DATASET_FUNCTIONS_UNIQ_HELPER(ctr, name) \ + WHITELIST_STATEFUL_OP_FOR_DATASET_FUNCTIONS_UNIQ(ctr, name) +#define WHITELIST_STATEFUL_OP_FOR_DATASET_FUNCTIONS_UNIQ(ctr, name) \ + static ::tensorflow::Status whitelist_op##ctr TF_ATTRIBUTE_UNUSED = \ + ::tensorflow::data::WhitelistedStatefulOpRegistry::Global()->Add(name) + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_FRAMEWORK_DATASET_STATEFUL_OP_WHITELIST_H_ diff --git a/linux/include/tensorflow/core/framework/device_attributes.pb.h b/linux/include/tensorflow/core/framework/device_attributes.pb.h new file mode 100644 index 000000000..5f9559bf0 --- /dev/null +++ b/linux/include/tensorflow/core/framework/device_attributes.pb.h @@ -0,0 +1,1282 @@ +// Generated by the protocol buffer compiler. DO NOT EDIT! +// source: tensorflow/core/framework/device_attributes.proto + +#ifndef PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2fdevice_5fattributes_2eproto +#define PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2fdevice_5fattributes_2eproto + +#include + +#include + +#if GOOGLE_PROTOBUF_VERSION < 3006000 +#error This file was generated by a newer version of protoc which is +#error incompatible with your Protocol Buffer headers. Please update +#error your headers. +#endif +#if 3006000 < GOOGLE_PROTOBUF_MIN_PROTOC_VERSION +#error This file was generated by an older version of protoc which is +#error incompatible with your Protocol Buffer headers. Please +#error regenerate this file with a newer version of protoc. +#endif + +#include +#include +#include +#include +#include +#include +#include +#include +#include // IWYU pragma: export +#include // IWYU pragma: export +#include +// @@protoc_insertion_point(includes) +#define PROTOBUF_INTERNAL_EXPORT_protobuf_tensorflow_2fcore_2fframework_2fdevice_5fattributes_2eproto + +namespace protobuf_tensorflow_2fcore_2fframework_2fdevice_5fattributes_2eproto { +// Internal implementation detail -- do not use these members. +struct TableStruct { + static const ::google::protobuf::internal::ParseTableField entries[]; + static const ::google::protobuf::internal::AuxillaryParseTableField aux[]; + static const ::google::protobuf::internal::ParseTable schema[4]; + static const ::google::protobuf::internal::FieldMetadata field_metadata[]; + static const ::google::protobuf::internal::SerializationTable serialization_table[]; + static const ::google::protobuf::uint32 offsets[]; +}; +void AddDescriptors(); +} // namespace protobuf_tensorflow_2fcore_2fframework_2fdevice_5fattributes_2eproto +namespace tensorflow { +class DeviceAttributes; +class DeviceAttributesDefaultTypeInternal; +extern DeviceAttributesDefaultTypeInternal _DeviceAttributes_default_instance_; +class DeviceLocality; +class DeviceLocalityDefaultTypeInternal; +extern DeviceLocalityDefaultTypeInternal _DeviceLocality_default_instance_; +class InterconnectLink; +class InterconnectLinkDefaultTypeInternal; +extern InterconnectLinkDefaultTypeInternal _InterconnectLink_default_instance_; +class LocalLinks; +class LocalLinksDefaultTypeInternal; +extern LocalLinksDefaultTypeInternal _LocalLinks_default_instance_; +} // namespace tensorflow +namespace google { +namespace protobuf { +template<> ::tensorflow::DeviceAttributes* Arena::CreateMaybeMessage<::tensorflow::DeviceAttributes>(Arena*); +template<> ::tensorflow::DeviceLocality* Arena::CreateMaybeMessage<::tensorflow::DeviceLocality>(Arena*); +template<> ::tensorflow::InterconnectLink* Arena::CreateMaybeMessage<::tensorflow::InterconnectLink>(Arena*); +template<> ::tensorflow::LocalLinks* Arena::CreateMaybeMessage<::tensorflow::LocalLinks>(Arena*); +} // namespace protobuf +} // namespace google +namespace tensorflow { + +// =================================================================== + +class InterconnectLink : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.InterconnectLink) */ { + public: + InterconnectLink(); + virtual ~InterconnectLink(); + + InterconnectLink(const InterconnectLink& from); + + inline InterconnectLink& operator=(const InterconnectLink& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + InterconnectLink(InterconnectLink&& from) noexcept + : InterconnectLink() { + *this = ::std::move(from); + } + + inline InterconnectLink& operator=(InterconnectLink&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const InterconnectLink& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const InterconnectLink* internal_default_instance() { + return reinterpret_cast( + &_InterconnectLink_default_instance_); + } + static constexpr int kIndexInFileMessages = + 0; + + void UnsafeArenaSwap(InterconnectLink* other); + void Swap(InterconnectLink* other); + friend void swap(InterconnectLink& a, InterconnectLink& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline InterconnectLink* New() const final { + return CreateMaybeMessage(NULL); + } + + InterconnectLink* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const InterconnectLink& from); + void MergeFrom(const InterconnectLink& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(InterconnectLink* other); + protected: + explicit InterconnectLink(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // string type = 2; + void clear_type(); + static const int kTypeFieldNumber = 2; + const ::std::string& type() const; + void set_type(const ::std::string& value); + #if LANG_CXX11 + void set_type(::std::string&& value); + #endif + void set_type(const char* value); + void set_type(const char* value, size_t size); + ::std::string* mutable_type(); + ::std::string* release_type(); + void set_allocated_type(::std::string* type); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_type(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_type( + ::std::string* type); + + // int32 device_id = 1; + void clear_device_id(); + static const int kDeviceIdFieldNumber = 1; + ::google::protobuf::int32 device_id() const; + void set_device_id(::google::protobuf::int32 value); + + // int32 strength = 3; + void clear_strength(); + static const int kStrengthFieldNumber = 3; + ::google::protobuf::int32 strength() const; + void set_strength(::google::protobuf::int32 value); + + // @@protoc_insertion_point(class_scope:tensorflow.InterconnectLink) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::internal::ArenaStringPtr type_; + ::google::protobuf::int32 device_id_; + ::google::protobuf::int32 strength_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fdevice_5fattributes_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class LocalLinks : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.LocalLinks) */ { + public: + LocalLinks(); + virtual ~LocalLinks(); + + LocalLinks(const LocalLinks& from); + + inline LocalLinks& operator=(const LocalLinks& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + LocalLinks(LocalLinks&& from) noexcept + : LocalLinks() { + *this = ::std::move(from); + } + + inline LocalLinks& operator=(LocalLinks&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const LocalLinks& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const LocalLinks* internal_default_instance() { + return reinterpret_cast( + &_LocalLinks_default_instance_); + } + static constexpr int kIndexInFileMessages = + 1; + + void UnsafeArenaSwap(LocalLinks* other); + void Swap(LocalLinks* other); + friend void swap(LocalLinks& a, LocalLinks& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline LocalLinks* New() const final { + return CreateMaybeMessage(NULL); + } + + LocalLinks* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const LocalLinks& from); + void MergeFrom(const LocalLinks& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(LocalLinks* other); + protected: + explicit LocalLinks(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated .tensorflow.InterconnectLink link = 1; + int link_size() const; + void clear_link(); + static const int kLinkFieldNumber = 1; + ::tensorflow::InterconnectLink* mutable_link(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::InterconnectLink >* + mutable_link(); + const ::tensorflow::InterconnectLink& link(int index) const; + ::tensorflow::InterconnectLink* add_link(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::InterconnectLink >& + link() const; + + // @@protoc_insertion_point(class_scope:tensorflow.LocalLinks) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::InterconnectLink > link_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fdevice_5fattributes_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class DeviceLocality : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.DeviceLocality) */ { + public: + DeviceLocality(); + virtual ~DeviceLocality(); + + DeviceLocality(const DeviceLocality& from); + + inline DeviceLocality& operator=(const DeviceLocality& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + DeviceLocality(DeviceLocality&& from) noexcept + : DeviceLocality() { + *this = ::std::move(from); + } + + inline DeviceLocality& operator=(DeviceLocality&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const DeviceLocality& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const DeviceLocality* internal_default_instance() { + return reinterpret_cast( + &_DeviceLocality_default_instance_); + } + static constexpr int kIndexInFileMessages = + 2; + + void UnsafeArenaSwap(DeviceLocality* other); + void Swap(DeviceLocality* other); + friend void swap(DeviceLocality& a, DeviceLocality& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline DeviceLocality* New() const final { + return CreateMaybeMessage(NULL); + } + + DeviceLocality* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const DeviceLocality& from); + void MergeFrom(const DeviceLocality& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(DeviceLocality* other); + protected: + explicit DeviceLocality(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // .tensorflow.LocalLinks links = 3; + bool has_links() const; + void clear_links(); + static const int kLinksFieldNumber = 3; + private: + const ::tensorflow::LocalLinks& _internal_links() const; + public: + const ::tensorflow::LocalLinks& links() const; + ::tensorflow::LocalLinks* release_links(); + ::tensorflow::LocalLinks* mutable_links(); + void set_allocated_links(::tensorflow::LocalLinks* links); + void unsafe_arena_set_allocated_links( + ::tensorflow::LocalLinks* links); + ::tensorflow::LocalLinks* unsafe_arena_release_links(); + + // int32 bus_id = 1; + void clear_bus_id(); + static const int kBusIdFieldNumber = 1; + ::google::protobuf::int32 bus_id() const; + void set_bus_id(::google::protobuf::int32 value); + + // int32 numa_node = 2; + void clear_numa_node(); + static const int kNumaNodeFieldNumber = 2; + ::google::protobuf::int32 numa_node() const; + void set_numa_node(::google::protobuf::int32 value); + + // @@protoc_insertion_point(class_scope:tensorflow.DeviceLocality) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::tensorflow::LocalLinks* links_; + ::google::protobuf::int32 bus_id_; + ::google::protobuf::int32 numa_node_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fdevice_5fattributes_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class DeviceAttributes : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.DeviceAttributes) */ { + public: + DeviceAttributes(); + virtual ~DeviceAttributes(); + + DeviceAttributes(const DeviceAttributes& from); + + inline DeviceAttributes& operator=(const DeviceAttributes& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + DeviceAttributes(DeviceAttributes&& from) noexcept + : DeviceAttributes() { + *this = ::std::move(from); + } + + inline DeviceAttributes& operator=(DeviceAttributes&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const DeviceAttributes& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const DeviceAttributes* internal_default_instance() { + return reinterpret_cast( + &_DeviceAttributes_default_instance_); + } + static constexpr int kIndexInFileMessages = + 3; + + void UnsafeArenaSwap(DeviceAttributes* other); + void Swap(DeviceAttributes* other); + friend void swap(DeviceAttributes& a, DeviceAttributes& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline DeviceAttributes* New() const final { + return CreateMaybeMessage(NULL); + } + + DeviceAttributes* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const DeviceAttributes& from); + void MergeFrom(const DeviceAttributes& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(DeviceAttributes* other); + protected: + explicit DeviceAttributes(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // string name = 1; + void clear_name(); + static const int kNameFieldNumber = 1; + const ::std::string& name() const; + void set_name(const ::std::string& value); + #if LANG_CXX11 + void set_name(::std::string&& value); + #endif + void set_name(const char* value); + void set_name(const char* value, size_t size); + ::std::string* mutable_name(); + ::std::string* release_name(); + void set_allocated_name(::std::string* name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_name( + ::std::string* name); + + // string device_type = 2; + void clear_device_type(); + static const int kDeviceTypeFieldNumber = 2; + const ::std::string& device_type() const; + void set_device_type(const ::std::string& value); + #if LANG_CXX11 + void set_device_type(::std::string&& value); + #endif + void set_device_type(const char* value); + void set_device_type(const char* value, size_t size); + ::std::string* mutable_device_type(); + ::std::string* release_device_type(); + void set_allocated_device_type(::std::string* device_type); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_device_type(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_device_type( + ::std::string* device_type); + + // string physical_device_desc = 7; + void clear_physical_device_desc(); + static const int kPhysicalDeviceDescFieldNumber = 7; + const ::std::string& physical_device_desc() const; + void set_physical_device_desc(const ::std::string& value); + #if LANG_CXX11 + void set_physical_device_desc(::std::string&& value); + #endif + void set_physical_device_desc(const char* value); + void set_physical_device_desc(const char* value, size_t size); + ::std::string* mutable_physical_device_desc(); + ::std::string* release_physical_device_desc(); + void set_allocated_physical_device_desc(::std::string* physical_device_desc); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_physical_device_desc(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_physical_device_desc( + ::std::string* physical_device_desc); + + // .tensorflow.DeviceLocality locality = 5; + bool has_locality() const; + void clear_locality(); + static const int kLocalityFieldNumber = 5; + private: + const ::tensorflow::DeviceLocality& _internal_locality() const; + public: + const ::tensorflow::DeviceLocality& locality() const; + ::tensorflow::DeviceLocality* release_locality(); + ::tensorflow::DeviceLocality* mutable_locality(); + void set_allocated_locality(::tensorflow::DeviceLocality* locality); + void unsafe_arena_set_allocated_locality( + ::tensorflow::DeviceLocality* locality); + ::tensorflow::DeviceLocality* unsafe_arena_release_locality(); + + // int64 memory_limit = 4; + void clear_memory_limit(); + static const int kMemoryLimitFieldNumber = 4; + ::google::protobuf::int64 memory_limit() const; + void set_memory_limit(::google::protobuf::int64 value); + + // fixed64 incarnation = 6; + void clear_incarnation(); + static const int kIncarnationFieldNumber = 6; + ::google::protobuf::uint64 incarnation() const; + void set_incarnation(::google::protobuf::uint64 value); + + // @@protoc_insertion_point(class_scope:tensorflow.DeviceAttributes) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::internal::ArenaStringPtr name_; + ::google::protobuf::internal::ArenaStringPtr device_type_; + ::google::protobuf::internal::ArenaStringPtr physical_device_desc_; + ::tensorflow::DeviceLocality* locality_; + ::google::protobuf::int64 memory_limit_; + ::google::protobuf::uint64 incarnation_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fdevice_5fattributes_2eproto::TableStruct; +}; +// =================================================================== + + +// =================================================================== + +#ifdef __GNUC__ + #pragma GCC diagnostic push + #pragma GCC diagnostic ignored "-Wstrict-aliasing" +#endif // __GNUC__ +// InterconnectLink + +// int32 device_id = 1; +inline void InterconnectLink::clear_device_id() { + device_id_ = 0; +} +inline ::google::protobuf::int32 InterconnectLink::device_id() const { + // @@protoc_insertion_point(field_get:tensorflow.InterconnectLink.device_id) + return device_id_; +} +inline void InterconnectLink::set_device_id(::google::protobuf::int32 value) { + + device_id_ = value; + // @@protoc_insertion_point(field_set:tensorflow.InterconnectLink.device_id) +} + +// string type = 2; +inline void InterconnectLink::clear_type() { + type_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& InterconnectLink::type() const { + // @@protoc_insertion_point(field_get:tensorflow.InterconnectLink.type) + return type_.Get(); +} +inline void InterconnectLink::set_type(const ::std::string& value) { + + type_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.InterconnectLink.type) +} +#if LANG_CXX11 +inline void InterconnectLink::set_type(::std::string&& value) { + + type_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.InterconnectLink.type) +} +#endif +inline void InterconnectLink::set_type(const char* value) { + GOOGLE_DCHECK(value != NULL); + + type_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.InterconnectLink.type) +} +inline void InterconnectLink::set_type(const char* value, + size_t size) { + + type_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.InterconnectLink.type) +} +inline ::std::string* InterconnectLink::mutable_type() { + + // @@protoc_insertion_point(field_mutable:tensorflow.InterconnectLink.type) + return type_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* InterconnectLink::release_type() { + // @@protoc_insertion_point(field_release:tensorflow.InterconnectLink.type) + + return type_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void InterconnectLink::set_allocated_type(::std::string* type) { + if (type != NULL) { + + } else { + + } + type_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), type, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.InterconnectLink.type) +} +inline ::std::string* InterconnectLink::unsafe_arena_release_type() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.InterconnectLink.type) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return type_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void InterconnectLink::unsafe_arena_set_allocated_type( + ::std::string* type) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (type != NULL) { + + } else { + + } + type_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + type, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.InterconnectLink.type) +} + +// int32 strength = 3; +inline void InterconnectLink::clear_strength() { + strength_ = 0; +} +inline ::google::protobuf::int32 InterconnectLink::strength() const { + // @@protoc_insertion_point(field_get:tensorflow.InterconnectLink.strength) + return strength_; +} +inline void InterconnectLink::set_strength(::google::protobuf::int32 value) { + + strength_ = value; + // @@protoc_insertion_point(field_set:tensorflow.InterconnectLink.strength) +} + +// ------------------------------------------------------------------- + +// LocalLinks + +// repeated .tensorflow.InterconnectLink link = 1; +inline int LocalLinks::link_size() const { + return link_.size(); +} +inline void LocalLinks::clear_link() { + link_.Clear(); +} +inline ::tensorflow::InterconnectLink* LocalLinks::mutable_link(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.LocalLinks.link) + return link_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::InterconnectLink >* +LocalLinks::mutable_link() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.LocalLinks.link) + return &link_; +} +inline const ::tensorflow::InterconnectLink& LocalLinks::link(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.LocalLinks.link) + return link_.Get(index); +} +inline ::tensorflow::InterconnectLink* LocalLinks::add_link() { + // @@protoc_insertion_point(field_add:tensorflow.LocalLinks.link) + return link_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::InterconnectLink >& +LocalLinks::link() const { + // @@protoc_insertion_point(field_list:tensorflow.LocalLinks.link) + return link_; +} + +// ------------------------------------------------------------------- + +// DeviceLocality + +// int32 bus_id = 1; +inline void DeviceLocality::clear_bus_id() { + bus_id_ = 0; +} +inline ::google::protobuf::int32 DeviceLocality::bus_id() const { + // @@protoc_insertion_point(field_get:tensorflow.DeviceLocality.bus_id) + return bus_id_; +} +inline void DeviceLocality::set_bus_id(::google::protobuf::int32 value) { + + bus_id_ = value; + // @@protoc_insertion_point(field_set:tensorflow.DeviceLocality.bus_id) +} + +// int32 numa_node = 2; +inline void DeviceLocality::clear_numa_node() { + numa_node_ = 0; +} +inline ::google::protobuf::int32 DeviceLocality::numa_node() const { + // @@protoc_insertion_point(field_get:tensorflow.DeviceLocality.numa_node) + return numa_node_; +} +inline void DeviceLocality::set_numa_node(::google::protobuf::int32 value) { + + numa_node_ = value; + // @@protoc_insertion_point(field_set:tensorflow.DeviceLocality.numa_node) +} + +// .tensorflow.LocalLinks links = 3; +inline bool DeviceLocality::has_links() const { + return this != internal_default_instance() && links_ != NULL; +} +inline void DeviceLocality::clear_links() { + if (GetArenaNoVirtual() == NULL && links_ != NULL) { + delete links_; + } + links_ = NULL; +} +inline const ::tensorflow::LocalLinks& DeviceLocality::_internal_links() const { + return *links_; +} +inline const ::tensorflow::LocalLinks& DeviceLocality::links() const { + const ::tensorflow::LocalLinks* p = links_; + // @@protoc_insertion_point(field_get:tensorflow.DeviceLocality.links) + return p != NULL ? *p : *reinterpret_cast( + &::tensorflow::_LocalLinks_default_instance_); +} +inline ::tensorflow::LocalLinks* DeviceLocality::release_links() { + // @@protoc_insertion_point(field_release:tensorflow.DeviceLocality.links) + + ::tensorflow::LocalLinks* temp = links_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + links_ = NULL; + return temp; +} +inline ::tensorflow::LocalLinks* DeviceLocality::unsafe_arena_release_links() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.DeviceLocality.links) + + ::tensorflow::LocalLinks* temp = links_; + links_ = NULL; + return temp; +} +inline ::tensorflow::LocalLinks* DeviceLocality::mutable_links() { + + if (links_ == NULL) { + auto* p = CreateMaybeMessage<::tensorflow::LocalLinks>(GetArenaNoVirtual()); + links_ = p; + } + // @@protoc_insertion_point(field_mutable:tensorflow.DeviceLocality.links) + return links_; +} +inline void DeviceLocality::set_allocated_links(::tensorflow::LocalLinks* links) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete links_; + } + if (links) { + ::google::protobuf::Arena* submessage_arena = + ::google::protobuf::Arena::GetArena(links); + if (message_arena != submessage_arena) { + links = ::google::protobuf::internal::GetOwnedMessage( + message_arena, links, submessage_arena); + } + + } else { + + } + links_ = links; + // @@protoc_insertion_point(field_set_allocated:tensorflow.DeviceLocality.links) +} + +// ------------------------------------------------------------------- + +// DeviceAttributes + +// string name = 1; +inline void DeviceAttributes::clear_name() { + name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& DeviceAttributes::name() const { + // @@protoc_insertion_point(field_get:tensorflow.DeviceAttributes.name) + return name_.Get(); +} +inline void DeviceAttributes::set_name(const ::std::string& value) { + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.DeviceAttributes.name) +} +#if LANG_CXX11 +inline void DeviceAttributes::set_name(::std::string&& value) { + + name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.DeviceAttributes.name) +} +#endif +inline void DeviceAttributes::set_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.DeviceAttributes.name) +} +inline void DeviceAttributes::set_name(const char* value, + size_t size) { + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.DeviceAttributes.name) +} +inline ::std::string* DeviceAttributes::mutable_name() { + + // @@protoc_insertion_point(field_mutable:tensorflow.DeviceAttributes.name) + return name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* DeviceAttributes::release_name() { + // @@protoc_insertion_point(field_release:tensorflow.DeviceAttributes.name) + + return name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void DeviceAttributes::set_allocated_name(::std::string* name) { + if (name != NULL) { + + } else { + + } + name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.DeviceAttributes.name) +} +inline ::std::string* DeviceAttributes::unsafe_arena_release_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.DeviceAttributes.name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void DeviceAttributes::unsafe_arena_set_allocated_name( + ::std::string* name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (name != NULL) { + + } else { + + } + name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.DeviceAttributes.name) +} + +// string device_type = 2; +inline void DeviceAttributes::clear_device_type() { + device_type_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& DeviceAttributes::device_type() const { + // @@protoc_insertion_point(field_get:tensorflow.DeviceAttributes.device_type) + return device_type_.Get(); +} +inline void DeviceAttributes::set_device_type(const ::std::string& value) { + + device_type_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.DeviceAttributes.device_type) +} +#if LANG_CXX11 +inline void DeviceAttributes::set_device_type(::std::string&& value) { + + device_type_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.DeviceAttributes.device_type) +} +#endif +inline void DeviceAttributes::set_device_type(const char* value) { + GOOGLE_DCHECK(value != NULL); + + device_type_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.DeviceAttributes.device_type) +} +inline void DeviceAttributes::set_device_type(const char* value, + size_t size) { + + device_type_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.DeviceAttributes.device_type) +} +inline ::std::string* DeviceAttributes::mutable_device_type() { + + // @@protoc_insertion_point(field_mutable:tensorflow.DeviceAttributes.device_type) + return device_type_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* DeviceAttributes::release_device_type() { + // @@protoc_insertion_point(field_release:tensorflow.DeviceAttributes.device_type) + + return device_type_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void DeviceAttributes::set_allocated_device_type(::std::string* device_type) { + if (device_type != NULL) { + + } else { + + } + device_type_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), device_type, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.DeviceAttributes.device_type) +} +inline ::std::string* DeviceAttributes::unsafe_arena_release_device_type() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.DeviceAttributes.device_type) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return device_type_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void DeviceAttributes::unsafe_arena_set_allocated_device_type( + ::std::string* device_type) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (device_type != NULL) { + + } else { + + } + device_type_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + device_type, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.DeviceAttributes.device_type) +} + +// int64 memory_limit = 4; +inline void DeviceAttributes::clear_memory_limit() { + memory_limit_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 DeviceAttributes::memory_limit() const { + // @@protoc_insertion_point(field_get:tensorflow.DeviceAttributes.memory_limit) + return memory_limit_; +} +inline void DeviceAttributes::set_memory_limit(::google::protobuf::int64 value) { + + memory_limit_ = value; + // @@protoc_insertion_point(field_set:tensorflow.DeviceAttributes.memory_limit) +} + +// .tensorflow.DeviceLocality locality = 5; +inline bool DeviceAttributes::has_locality() const { + return this != internal_default_instance() && locality_ != NULL; +} +inline void DeviceAttributes::clear_locality() { + if (GetArenaNoVirtual() == NULL && locality_ != NULL) { + delete locality_; + } + locality_ = NULL; +} +inline const ::tensorflow::DeviceLocality& DeviceAttributes::_internal_locality() const { + return *locality_; +} +inline const ::tensorflow::DeviceLocality& DeviceAttributes::locality() const { + const ::tensorflow::DeviceLocality* p = locality_; + // @@protoc_insertion_point(field_get:tensorflow.DeviceAttributes.locality) + return p != NULL ? *p : *reinterpret_cast( + &::tensorflow::_DeviceLocality_default_instance_); +} +inline ::tensorflow::DeviceLocality* DeviceAttributes::release_locality() { + // @@protoc_insertion_point(field_release:tensorflow.DeviceAttributes.locality) + + ::tensorflow::DeviceLocality* temp = locality_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + locality_ = NULL; + return temp; +} +inline ::tensorflow::DeviceLocality* DeviceAttributes::unsafe_arena_release_locality() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.DeviceAttributes.locality) + + ::tensorflow::DeviceLocality* temp = locality_; + locality_ = NULL; + return temp; +} +inline ::tensorflow::DeviceLocality* DeviceAttributes::mutable_locality() { + + if (locality_ == NULL) { + auto* p = CreateMaybeMessage<::tensorflow::DeviceLocality>(GetArenaNoVirtual()); + locality_ = p; + } + // @@protoc_insertion_point(field_mutable:tensorflow.DeviceAttributes.locality) + return locality_; +} +inline void DeviceAttributes::set_allocated_locality(::tensorflow::DeviceLocality* locality) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete locality_; + } + if (locality) { + ::google::protobuf::Arena* submessage_arena = + ::google::protobuf::Arena::GetArena(locality); + if (message_arena != submessage_arena) { + locality = ::google::protobuf::internal::GetOwnedMessage( + message_arena, locality, submessage_arena); + } + + } else { + + } + locality_ = locality; + // @@protoc_insertion_point(field_set_allocated:tensorflow.DeviceAttributes.locality) +} + +// fixed64 incarnation = 6; +inline void DeviceAttributes::clear_incarnation() { + incarnation_ = GOOGLE_ULONGLONG(0); +} +inline ::google::protobuf::uint64 DeviceAttributes::incarnation() const { + // @@protoc_insertion_point(field_get:tensorflow.DeviceAttributes.incarnation) + return incarnation_; +} +inline void DeviceAttributes::set_incarnation(::google::protobuf::uint64 value) { + + incarnation_ = value; + // @@protoc_insertion_point(field_set:tensorflow.DeviceAttributes.incarnation) +} + +// string physical_device_desc = 7; +inline void DeviceAttributes::clear_physical_device_desc() { + physical_device_desc_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& DeviceAttributes::physical_device_desc() const { + // @@protoc_insertion_point(field_get:tensorflow.DeviceAttributes.physical_device_desc) + return physical_device_desc_.Get(); +} +inline void DeviceAttributes::set_physical_device_desc(const ::std::string& value) { + + physical_device_desc_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.DeviceAttributes.physical_device_desc) +} +#if LANG_CXX11 +inline void DeviceAttributes::set_physical_device_desc(::std::string&& value) { + + physical_device_desc_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.DeviceAttributes.physical_device_desc) +} +#endif +inline void DeviceAttributes::set_physical_device_desc(const char* value) { + GOOGLE_DCHECK(value != NULL); + + physical_device_desc_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.DeviceAttributes.physical_device_desc) +} +inline void DeviceAttributes::set_physical_device_desc(const char* value, + size_t size) { + + physical_device_desc_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.DeviceAttributes.physical_device_desc) +} +inline ::std::string* DeviceAttributes::mutable_physical_device_desc() { + + // @@protoc_insertion_point(field_mutable:tensorflow.DeviceAttributes.physical_device_desc) + return physical_device_desc_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* DeviceAttributes::release_physical_device_desc() { + // @@protoc_insertion_point(field_release:tensorflow.DeviceAttributes.physical_device_desc) + + return physical_device_desc_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void DeviceAttributes::set_allocated_physical_device_desc(::std::string* physical_device_desc) { + if (physical_device_desc != NULL) { + + } else { + + } + physical_device_desc_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), physical_device_desc, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.DeviceAttributes.physical_device_desc) +} +inline ::std::string* DeviceAttributes::unsafe_arena_release_physical_device_desc() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.DeviceAttributes.physical_device_desc) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return physical_device_desc_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void DeviceAttributes::unsafe_arena_set_allocated_physical_device_desc( + ::std::string* physical_device_desc) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (physical_device_desc != NULL) { + + } else { + + } + physical_device_desc_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + physical_device_desc, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.DeviceAttributes.physical_device_desc) +} + +#ifdef __GNUC__ + #pragma GCC diagnostic pop +#endif // __GNUC__ +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + + +// @@protoc_insertion_point(namespace_scope) + +} // namespace tensorflow + +// @@protoc_insertion_point(global_scope) + +#endif // PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2fdevice_5fattributes_2eproto diff --git a/linux/include/tensorflow/core/framework/device_attributes.pb_text-impl.h b/linux/include/tensorflow/core/framework/device_attributes.pb_text-impl.h new file mode 100644 index 000000000..f282c1465 --- /dev/null +++ b/linux/include/tensorflow/core/framework/device_attributes.pb_text-impl.h @@ -0,0 +1,46 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_framework_device_attributes_proto_IMPL_H_ +#define tensorflow_core_framework_device_attributes_proto_IMPL_H_ + +#include "tensorflow/core/framework/device_attributes.pb.h" +#include "tensorflow/core/framework/device_attributes.pb_text.h" +#include "tensorflow/core/lib/strings/proto_text_util.h" +#include "tensorflow/core/lib/strings/scanner.h" + +namespace tensorflow { + +namespace internal { + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::InterconnectLink& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::InterconnectLink* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::LocalLinks& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::LocalLinks* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::DeviceLocality& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::DeviceLocality* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::DeviceAttributes& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::DeviceAttributes* msg); + +} // namespace internal + +} // namespace tensorflow + +#endif // tensorflow_core_framework_device_attributes_proto_IMPL_H_ diff --git a/linux/include/tensorflow/core/framework/device_attributes.pb_text.h b/linux/include/tensorflow/core/framework/device_attributes.pb_text.h new file mode 100644 index 000000000..6d1326f82 --- /dev/null +++ b/linux/include/tensorflow/core/framework/device_attributes.pb_text.h @@ -0,0 +1,54 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_framework_device_attributes_proto_H_ +#define tensorflow_core_framework_device_attributes_proto_H_ + +#include "tensorflow/core/framework/device_attributes.pb.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/protobuf.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +// Message-text conversion for tensorflow.InterconnectLink +string ProtoDebugString( + const ::tensorflow::InterconnectLink& msg); +string ProtoShortDebugString( + const ::tensorflow::InterconnectLink& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::InterconnectLink* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.LocalLinks +string ProtoDebugString( + const ::tensorflow::LocalLinks& msg); +string ProtoShortDebugString( + const ::tensorflow::LocalLinks& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::LocalLinks* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.DeviceLocality +string ProtoDebugString( + const ::tensorflow::DeviceLocality& msg); +string ProtoShortDebugString( + const ::tensorflow::DeviceLocality& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::DeviceLocality* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.DeviceAttributes +string ProtoDebugString( + const ::tensorflow::DeviceAttributes& msg); +string ProtoShortDebugString( + const ::tensorflow::DeviceAttributes& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::DeviceAttributes* msg) + TF_MUST_USE_RESULT; + +} // namespace tensorflow + +#endif // tensorflow_core_framework_device_attributes_proto_H_ diff --git a/linux/include/tensorflow/core/framework/device_base.h b/linux/include/tensorflow/core/framework/device_base.h new file mode 100644 index 000000000..446c31b17 --- /dev/null +++ b/linux/include/tensorflow/core/framework/device_base.h @@ -0,0 +1,262 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_FRAMEWORK_DEVICE_BASE_H_ +#define TENSORFLOW_CORE_FRAMEWORK_DEVICE_BASE_H_ + +#include +#include +#include + +#include "absl/base/macros.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/lib/core/errors.h" +#include "tensorflow/core/lib/core/refcount.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/lib/core/stringpiece.h" +#include "tensorflow/core/platform/logging.h" + +namespace Eigen { +struct ThreadPoolDevice; +#ifdef TENSORFLOW_USE_SYCL +struct SyclDevice; +#endif +} // end namespace Eigen + +namespace stream_executor { +class Stream; +} // namespace stream_executor + +namespace tensorflow { + +class Device; +class DeviceAttributes; +class Env; +class EventMgr; +class OpKernelContext; +class ResourceMgr; +class ScopedAllocatorMgr; +class TensorProto; + +namespace thread { +class ThreadPool; +} + +// A wrapper for an Eigen Gpu Device that includes per-op state. The +// class is defined even for non-GPU devices since the +// OpKernelContext::Params structure wants to fill it in. +class PerOpGpuDevice { + public: + virtual ~PerOpGpuDevice() {} + virtual const Eigen::GpuDevice& device() const = 0; +}; + +// A class that devices can subclass to pass around +// Device-specific context to OpKernels. +class DeviceContext : public core::RefCounted { + public: + ~DeviceContext() override {} + virtual stream_executor::Stream* stream() const { return nullptr; } + virtual void MaintainLifetimeOnStream(const Tensor* t, + stream_executor::Stream* stream) const { + } + + // "cpu_tensor" is a tensor on a CPU. Copies "cpu_tensor" into + // "device_tensor" which is on a GPU device "device". "device_tensor" + // must be allocated to be of the same size as "cpu_tensor". + virtual void CopyCPUTensorToDevice(const Tensor* cpu_tensor, Device* device, + Tensor* device_tensor, + StatusCallback done) const { + done(errors::Internal("Unrecognized device type in CPU-to-device Copy")); + } + + // "device_tensor" is a tensor on a non-CPU device. Copies + // device_tensor into "cpu_tensor". "cpu_tensor" must be allocated + // to be of the same size as "device_tensor". + virtual void CopyDeviceTensorToCPU(const Tensor* device_tensor, + StringPiece tensor_name, Device* device, + Tensor* cpu_tensor, StatusCallback done) { + done(errors::Internal("Unrecognized device type in device-to-CPU Copy")); + } + + // If possible, wait for all events on *stream to complete then execute func. + // A non-OK Status is returned otherwise. The stream argument should be the + // one provided by GpuDeviceInfo. This function is not applicable to devices + // that don't provide such a value. + virtual Status ThenExecute(Device* device, stream_executor::Stream* stream, + std::function func) { + return errors::Internal("ThenExecute not supported by device"); + } +}; + +// map[i] is the DeviceContext* for the node with id i, if i < map.size(). +typedef std::vector DeviceContextMap; + +class DeviceBase { + public: + explicit DeviceBase(Env* env) : env_(env) {} + virtual ~DeviceBase(); + + Env* env() const { return env_; } + + // Override this to return true for devices that require an Op's + // compute method to save references to the temporary tensors it + // allocates until the Op execution completes + virtual bool RequiresRecordingAccessedTensors() const { return false; } + + struct CpuWorkerThreads { + int num_threads = 0; + thread::ThreadPool* workers = nullptr; + }; + + // Does not take ownership. + void set_tensorflow_cpu_worker_threads(CpuWorkerThreads* t) { + cpu_worker_threads_ = t; + } + + virtual const CpuWorkerThreads* tensorflow_cpu_worker_threads() const { + CHECK(cpu_worker_threads_ != nullptr); + return cpu_worker_threads_; + } + + // "stream" is used in special circumstances (such as the + // constructors of Ops) where there is no available OpKernelContext. + // "default_context" is used by OpKernelContext whenever a device does not + // supply a DeviceContext for an op in FillContextMap (e.g. when only + // using a single stream.) + // "event_mgr" is used to delay deallocation of temporary GPU buffers. + // TODO(pbar) Work out how to move this out of DeviceBase. + // GpuDeviceInfo name is an unfortunate legacy, it is used not only by GPUs + // but also by TPU devices (to provide default device context). + struct GpuDeviceInfo { + // Make sure all the defaults are NULL, so we can spot missing assignments. + stream_executor::Stream* stream = nullptr; + DeviceContext* default_context = nullptr; + EventMgr* event_mgr = nullptr; + int gpu_id = -1; + }; + + // Does not take ownership. + void set_tensorflow_gpu_device_info(GpuDeviceInfo* g) { + gpu_device_info_ = g; + } + + virtual const GpuDeviceInfo* tensorflow_gpu_device_info() const { + return gpu_device_info_; + } + + // The preferred thread pool for this device. If it is nullptr, the system + // automatically assigns a thread pool for execution. + virtual thread::ThreadPool* tensorflow_device_thread_pool() { + return device_thread_pool_; + } + + // Does not take ownership. + void set_eigen_cpu_device(Eigen::ThreadPoolDevice* d); + +#ifdef TENSORFLOW_USE_SYCL + void set_eigen_sycl_device(Eigen::SyclDevice* d) { eigen_sycl_device_ = d; } +#endif + + // Return the Allocator implementation to use based on the allocator + // attributes requested. See allocator.h for more details. + virtual Allocator* GetAllocator(AllocatorAttributes /*attr*/) { + LOG(FATAL) << "GetAllocator() is not implemented."; + return nullptr; + } + + // This method is provided for backwards compatibility, and will be removed + // in a future release. + ABSL_DEPRECATED("Use `this->GetAllocator()` or `this->GetScopedAllocator()`.") + Allocator* GetStepAllocator(AllocatorAttributes attr, ResourceMgr*) { + return GetAllocator(attr); + } + + // Return an Allocator prepared for use in particular places by graph + // optimization + virtual Allocator* GetScopedAllocator(AllocatorAttributes attr, + int64 step_id) { + LOG(FATAL) << "Device does not implement GetScopedAllocator()"; + return nullptr; + } + + virtual ScopedAllocatorMgr* GetScopedAllocatorMgr() const { return nullptr; } + + bool has_eigen_cpu_device() const { return !eigen_cpu_devices_.empty(); } + + virtual const Eigen::ThreadPoolDevice* eigen_cpu_device(); + +#ifdef TENSORFLOW_USE_SYCL + virtual const Eigen::SyclDevice* eigen_sycl_device() const { + CHECK(eigen_sycl_device_ != nullptr); + return eigen_sycl_device_; + } +#endif + + // Caller owns the return value. The OpKernelContext calls this even + // for devices that do not implement an eigen_gpu_device. Overridden + // by GPU devices to return a derived type. + virtual PerOpGpuDevice* MakeGpuDevice() { return nullptr; } + + virtual DeviceBase* UnderlyingDevice() { return this; } + virtual const DeviceBase* UnderlyingDevice() const { return this; } + + // This is overridden by GPU devices to reinitialize the derived + // type returned by MakeGpuDevice. + virtual Status ReinitializeGpuDevice(OpKernelContext* /*context*/, + PerOpGpuDevice* /*device*/, + DeviceContext* /*dc*/, + Allocator* /*allocator*/) { + return Status::OK(); + } + + // Unimplemented by default + virtual const DeviceAttributes& attributes() const; + virtual const string& name() const; + + // Materializes the given TensorProto into 'tensor' stored in Device + // memory. Most devices will want to override this. + // + // TODO(vrv): We should be able to put this function into + // OpKernelContext and handle the copies from device memory via send + // and receive nodes, instead of requiring that each device handle + // the copies here as well as in copy ops. + virtual Status MakeTensorFromProto(const TensorProto& tensor_proto, + const AllocatorAttributes alloc_attrs, + Tensor* tensor) { + return errors::Internal("Device does not implement MakeTensorFromProto()"); + } + + protected: + // Does not take ownership. + void set_tensorflow_device_thread_pool(thread::ThreadPool* thread_pool) { + device_thread_pool_ = thread_pool; + } + + private: + Env* const env_; + CpuWorkerThreads* cpu_worker_threads_ = nullptr; + // Set by GPUs as well as by TPU devices. + GpuDeviceInfo* gpu_device_info_ = nullptr; + thread::ThreadPool* device_thread_pool_ = nullptr; + std::vector eigen_cpu_devices_; +#ifdef TENSORFLOW_USE_SYCL + Eigen::SyclDevice* eigen_sycl_device_ = nullptr; +#endif +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_FRAMEWORK_DEVICE_BASE_H_ diff --git a/linux/include/tensorflow/core/framework/function.h b/linux/include/tensorflow/core/framework/function.h new file mode 100644 index 000000000..4d6d68e21 --- /dev/null +++ b/linux/include/tensorflow/core/framework/function.h @@ -0,0 +1,749 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_FRAMEWORK_FUNCTION_H_ +#define TENSORFLOW_CORE_FRAMEWORK_FUNCTION_H_ + +#include +#include "tensorflow/core/framework/attr_value.pb.h" +#include "tensorflow/core/framework/attr_value_util.h" +#include "tensorflow/core/framework/function.pb.h" +#include "tensorflow/core/framework/node_def_util.h" +#include "tensorflow/core/framework/op.h" +#include "tensorflow/core/framework/selective_registration.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/gtl/flatmap.h" +#include "tensorflow/core/lib/hash/hash.h" +#include "tensorflow/core/platform/env.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/mutex.h" +#include "tensorflow/core/platform/protobuf.h" + +namespace tensorflow { + +class CancellationManager; +class CollectiveExecutor; +class GraphDef; +class OpKernel; +class ProcessFunctionLibraryRuntime; +class ResourceMgr; +class Rendezvous; +class ScopedStepContainer; +class StepStatsCollectorInterface; +class Node; + +// FunctionDefHelper::Create is a convenient helper to construct a +// FunctionDef proto. +// E.g., +// FunctionDef my_func = FunctionDefHelper::Create( +// "my_func_name", +// {"x:T", "y:T" /* one string per argument */}, +// {"z:T" /* one string per return value */}, +// {"T: {float, double}" /* one string per attribute */}, +// { +// {{"o"}, "Mul", {"x", "y"}, {{"T", "$T"}}} +// /* one entry per function node */ +// }, +// /* Mapping between function returns and function node outputs. */ +// {{"z", "o:z"}}); +// +// For the old Function::Node approach, use FunctionDefHelper::Define() +// E.g., +// FunctionDef my_func = FunctionDefHelper::Define( +// "my_func_name", +// {"x:T", "y:T" /* one string per argument */}, +// {"z:T" /* one string per return value */}, +// {"T: {float, double}" /* one string per attribute */}, +// { +// {{"z"}, "Mul", {"x", "y"}, {{"T", "$T"}}} +// /* one entry per function node */ +// }); +class FunctionDefHelper { + public: + // AttrValueWrapper has copy constructors for the type T so that + // it's easy to construct a simple AttrValue proto. + // + // If T is a string type (const char*, string, or StringPiece), and + // it starts with "$", we construct a AttrValue of "placeholder". + // + // E.g., + // std:: x = {"T", "$T"} + // is a named attr value placeholder. + struct AttrValueWrapper { + AttrValue proto; + + AttrValueWrapper() {} + + template + AttrValueWrapper(T val) { // NOLINT(runtime/explicit) + SetAttrValue(val, &proto); + } + + private: + void InitFromString(StringPiece val); + }; + + // Constructs an AttrValue.func given the "name" and "attrs". + static AttrValueWrapper FunctionRef( + const string& name, + gtl::ArraySlice> attrs); + static AttrValueWrapper FunctionRef(const string& name) { + return FunctionRef(name, {}); + } + + // Node is used to construct FunctionDef.Node using initialization + // lists. E.g., + // Node n = {{"z"}, "Mul", {"x", "y"}, {{"T", "$T"}}}; // z = x * y + struct Node { + // When constructing a NodeDef, the first entry in ret is used as + // the node name, the remaining values are ignored. + std::vector ret; + string op; + std::vector arg; + std::vector> attr; + std::vector dep; + + NodeDef ToNodeDef() const; + }; + + // The Create() function uses the new NodeDef field. `ret_def` + // holds a mapping from the function output names from `out_def` to + // the node outputs from `node_def`. + static FunctionDef Create(const string& function_name, + gtl::ArraySlice in_def, + gtl::ArraySlice out_def, + gtl::ArraySlice attr_def, + gtl::ArraySlice node_def, + gtl::ArraySlice> ret_def); + + // The two Define() functions use the old FunctionDef::Node field. + // TODO(josh11b): Get rid of these and transition to the one above. + static FunctionDef Define(const string& function_name, + gtl::ArraySlice arg_def, + gtl::ArraySlice ret_def, + gtl::ArraySlice attr_def, + gtl::ArraySlice node_def); + + // Defines an anonymous function. I.e., its name is not relevant. + static FunctionDef Define(gtl::ArraySlice arg_def, + gtl::ArraySlice ret_def, + gtl::ArraySlice attr_def, + gtl::ArraySlice node_def); + + // Helpers to construct a constant scalar. + template + static Node Const(const string& name, const T& val) { + Node n = {{name}, "Const"}; + const DataType dtype = DataTypeToEnum::value; + n.attr.push_back({"dtype", dtype}); + Tensor t(dtype, TensorShape({})); + t.scalar()() = val; + n.attr.push_back({"value", t}); + return n; + } + + template + static Node Const(const string& name, gtl::ArraySlice vals) { + Node n = {{name}, "Const"}; + const DataType dtype = DataTypeToEnum::value; + n.attr.push_back({"dtype", dtype}); + int64 num = vals.size(); + Tensor t(dtype, TensorShape({num})); + for (size_t i = 0; i < vals.size(); ++i) { + t.flat()(i) = vals[i]; + } + n.attr.push_back({"value", t}); + return n; + } +}; + +template <> +inline FunctionDefHelper::AttrValueWrapper::AttrValueWrapper(const char* val) { + InitFromString(val); +} + +template <> +inline FunctionDefHelper::AttrValueWrapper::AttrValueWrapper( + const string& val) { + InitFromString(val); +} + +template <> +inline FunctionDefHelper::AttrValueWrapper::AttrValueWrapper(StringPiece val) { + InitFromString(val); +} + +// Instantiate a function. +// +// "fdef" encodes a TF function with some attrs in fdef.signature.attr +// containing placeholders. InstantiateFunction binds these +// placeholders and produces an instantiated function encoded in +// "result.gdef". The value to substitute a placeholder is given by +// "attr_values", which is a map from a placeholder name to an attr +// value. +// +// InstantiateFunction calls "get_function" to find signatures of other +// functions and primitive ops. + +// GetFunctionSignature(func name, opdef) returns OK if the func name is found +// and opdef is filled with a pointer to the corresponding signature +// (a OpDef proto). Otherwise, returns an error. +typedef std::function + GetFunctionSignature; + +struct InstantiationResult { + DataTypeVector arg_types; + DataTypeVector ret_types; + std::vector nodes; +}; +Status InstantiateFunction(const FunctionDef& fdef, AttrSlice attr_values, + GetFunctionSignature get_function, + InstantiationResult* result); + +// Returns a debug string for a function definition. +// +// The returned text is multiple-line. It is intended to be +// human-readable rather than being friendly to parsers. It is _NOT_ +// intended to be the canonical string representation of "func_def". +// Particularly, it may not include all information presented in +// "func_def" (e.g., comments, description of the function arguments, +// etc.) +string DebugString(const FunctionDef& func_def); +string DebugString(const GraphDef& instantiated_func_def); +string DebugString(gtl::ArraySlice instantiated_func_nodes); + +// Returns a debug string for a top level graph (the main program and +// its supporting functions defined in its library). +string DebugStringWhole(const GraphDef& gdef); + +// Returns true if f1 == f2. Compares all fields, including descriptions. Order +// of NodeDefs doesn't matter. +bool FunctionDefsEqual(const FunctionDef& f1, const FunctionDef& f2); + +// Return a hash of `fdef` that is consistent with FunctionDefsEqual method. +// In other words, if two fdefs compare equal, their hash values will be the +// same. +uint64 FunctionDefHash(const FunctionDef& fdef); + +class CallFrameInterface { + public: + virtual ~CallFrameInterface() {} + + virtual size_t num_args() const = 0; + virtual size_t num_retvals() const = 0; + + virtual Status GetArg(int index, Tensor* val) const = 0; + virtual Status SetRetval(int index, const Tensor& val) = 0; +}; + +// Represents a function call frame. I.e., the data structure used to +// pass arguments to a function and retrieve its results. +// +// Runtime must arrange accesses to one FunctionCallFrame s.t. +// 1. SetArgs() happens before any GetArg(); +// 2. GetRetvals happens after all SetRetval(); +class FunctionCallFrame : public CallFrameInterface { + public: + FunctionCallFrame(DataTypeSlice arg_types, DataTypeSlice ret_types); + ~FunctionCallFrame(); + + // Caller methods. + Status SetArgs(gtl::ArraySlice args); + Status GetRetvals(std::vector* rets) const; + + // Moves the return values from the frame to rets. If allow_dead_tensors is + // false it will fail if any of the retvals do not have a value. + Status ConsumeRetvals(std::vector* rets, bool allow_dead_tensors); + + size_t num_args() const override { return arg_types_.size(); } + size_t num_retvals() const override { return ret_types_.size(); } + + // Callee methods. + Status GetArg(int index, Tensor* val) const override; + Status SetRetval(int index, const Tensor& val) override; + + private: + DataTypeVector arg_types_; + DataTypeVector ret_types_; + gtl::InlinedVector args_; + struct Retval { + bool has_val = false; + Tensor val; + }; + gtl::InlinedVector rets_; + + TF_DISALLOW_COPY_AND_ASSIGN(FunctionCallFrame); +}; + +// Helper to maintain a map between function names in a given +// FunctionDefLibrary and function definitions. +// +// This class is thread-safe. +class FunctionLibraryDefinition : public OpRegistryInterface { + public: + // Note: This constructor grabs `lib_def`'s lock in shared mode. + explicit FunctionLibraryDefinition(const FunctionLibraryDefinition& lib_def); + FunctionLibraryDefinition(const OpRegistryInterface* default_registry, + const FunctionDefLibrary& lib_def); + ~FunctionLibraryDefinition() override; + + FunctionLibraryDefinition& operator=(const FunctionLibraryDefinition&) = + delete; + + // Returns True if the library contains `func`, False otherwise. + bool Contains(const string& func) const; + + // Returns nullptr if "func" is not defined in "lib_def". Otherwise, + // returns its definition proto. + // + // NB: This function returns a borrowed pointer, which can be invalidated by a + // subsequent call to `ReplaceFunction()` with the given name. + const FunctionDef* Find(const string& func) const LOCKS_EXCLUDED(mu_); + + // Adds function definition 'fdef' to this function library. + // Returns status 'ok' on success, or error otherwise. This is a no-op if + // 'fdef' already exists in this function library. + // If 'fdef' is successfully added to the library, it will be accessible + // from 'LookUp' and included in the proto returned by 'ToProto'. + // This operation is atomic. + Status AddFunctionDef(const FunctionDef& fdef) LOCKS_EXCLUDED(mu_); + + // Adds gradient definition 'grad' to this function library. + // This is a no-op if 'grad' already exists in this function library. + // If 'grad' is successfully added, it will be accessible via 'FindGradient' + // and included in the proto returned by 'ToProto'. + // This operation is atomic. + Status AddGradientDef(const GradientDef& grad) LOCKS_EXCLUDED(mu_); + + // Replaces the function corresponding to `func` with `fdef`. Returns + // a non-OK status if "func" was not found in the library, OK otherwise. + Status ReplaceFunction(const string& func, const FunctionDef& fdef); + + // Replaces the gradient corresponding to `grad.function_name()`. Returns + // a non-OK status if "grad.function_name()" was not found in the library, OK + // otherwise. + Status ReplaceGradient(const GradientDef& grad); + + // Adds the functions and gradients in 'other' to this function library. + // Duplicate functions and gradients are ignored. + // This operation is atomic. + Status AddLibrary(const FunctionLibraryDefinition& other) LOCKS_EXCLUDED(mu_); + + // Adds the functions and gradients in 'lib_def' to this function library. + // Duplicate functions and gradients are ignored. + // This operation is atomic. + Status AddLibrary(const FunctionDefLibrary& lib_def) LOCKS_EXCLUDED(mu_); + + // If the gradient function for 'func' is specified explicitly in + // the library, returns the gradient function name. Otherwise, + // returns an empty string. + string FindGradient(const string& func) const LOCKS_EXCLUDED(mu_); + + // OpRegistryInterface method. Useful for constructing a Graph. + // + // If "op" is defined in the library, returns its signature. + // Otherwise, assume "op" is a primitive op and returns its op + // signature and shape inference function. + // + // NB: This function outputs a borrowed pointer, which can be invalidated by a + // subsequent call to `ReplaceFunction()` with the given name. + Status LookUp(const string& op_type_name, + const OpRegistrationData** op_reg_data) const override + LOCKS_EXCLUDED(mu_); + + // Generates new function name with the specified prefix that is unique + // across this library. + string UniqueFunctionName(StringPiece prefix) const LOCKS_EXCLUDED(mu_); + + // Ops created for function arguments bear the name given by `kArgOp`; those + // created for return values bear the name given by `kRetOp`. + static constexpr const char* const kArgOp = "_Arg"; + static constexpr const char* const kRetOp = "_Retval"; + + static constexpr const char* const kGradientOp = "SymbolicGradient"; + static constexpr const char* const kFuncAttr = "f"; + + // Given a node def 'ndef', inspects attributes of the callee + // function to derive the attribute 'value' for 'attr'. Returns OK + // iff the attribute is given by the function's definition. + // TODO(irving): Remove; keep only the const Node& version. + template + Status GetAttr(const NodeDef& ndef, const string& attr, T* value) const; + + // Given a node, inspects attributes of the callee function to derive the + // attribute 'value' for 'attr'. Returns OK iff the attribute is given by the + // function's definition. + template + Status GetAttr(const Node& node, const string& attr, T* value) const; + + // Returns a proto representation of the state of this function library. + FunctionDefLibrary ToProto() const LOCKS_EXCLUDED(mu_); + + size_t num_functions() const { + tf_shared_lock l(mu_); + return function_defs_.size(); + } + + const OpRegistryInterface* default_registry() const { + return default_registry_; + } + + private: + // Shape inference for functions is handled separately by ShapeRefiner. + + struct FunctionDefAndOpRegistration { + FunctionDefAndOpRegistration(const FunctionDef& fdef_in); + + FunctionDef fdef; + OpRegistrationData op_registration_data; + }; + + const FunctionDef* FindHelper(const string& func) const + SHARED_LOCKS_REQUIRED(mu_); + string FindGradientHelper(const string& func) const + SHARED_LOCKS_REQUIRED(mu_); + + // Same as AddFunctionDef/AddGradientDef except these methods set + // `added` to true if the `fdef`/`grad` were actually added to this. + Status AddFunctionDefHelper(const FunctionDef& fdef, bool* added) + EXCLUSIVE_LOCKS_REQUIRED(mu_); + Status AddGradientDefHelper(const GradientDef& grad, bool* added) + EXCLUSIVE_LOCKS_REQUIRED(mu_); + + mutable mutex mu_; + const OpRegistryInterface* const default_registry_; + gtl::FlatMap> + function_defs_ GUARDED_BY(mu_); + gtl::FlatMap func_grad_ GUARDED_BY(mu_); + + // Helper function for GetAttr. Returns the FunctionDef* to get the + // attr from. + const FunctionDef* GetAttrImpl(const NodeDef& ndef) const LOCKS_EXCLUDED(mu_); + + // Remove all functions in `funcs` and all gradients of functions in + // `funcs_with_grads` from this library. + void Remove(const std::vector& funcs, + const std::vector& funcs_with_grads) + EXCLUSIVE_LOCKS_REQUIRED(mu_); + + // Remove `func` from the library. Returns non-OK Status unless `func` is in + // the library. This should only be called when there is a guarantee that the + // function being removed hasn't been retrieved with `Find`. + Status RemoveFunction(const string& func) EXCLUSIVE_LOCKS_REQUIRED(mu_); + + // Remove gradient of function `func` from the library. Returns non-OK Status + // unless `func` has a gradient. + Status RemoveGradient(const string& func) EXCLUSIVE_LOCKS_REQUIRED(mu_); +}; + +// Forward declare. Defined in common_runtime/function.h +struct FunctionBody; + +// Forward declare. Defined in common_runtime/device.h +class Device; +// Forward declare. Defined in common_runtime/device_mgr.h +class DeviceMgr; + +class FunctionLibraryRuntime { + public: + virtual ~FunctionLibraryRuntime() {} + + // Instantiate a function with the given "attrs". + // + // Returns OK and fills in "handle" if the instantiation succeeds. + // Otherwise returns an error and "handle" is undefined. + struct InstantiateOptions { + // The canonical device name of the device on which the function + // should be instantiated. If empty, the function will be + // instantiated on the local device. + string target; + + // This interface is EXPERIMENTAL and subject to change. + // + // If non-null, the runtime will use `overlay_lib` to resolve + // function(s) named in `function_name` and `attrs`. Otherwise, + // the runtime will use its internal library. + // NOTE(mrry): If provided, all functions defined in `overlay_lib` + // must be self-contained, and cannot refer to functions defined + // in other libraries. + // TODO(mrry): Provide a mechanism for sharing core functions + // between a set of libraries (e.g. by allowing a + // `FunctionLibraryDefinition` to store an `outer_scope` pointer + // and implementing name resolution across libraries). + const FunctionLibraryDefinition* overlay_lib = nullptr; + + // This interface is EXPERIMENTAL and subject to change. + // + // If non-empty, the runtime will use `state_handle` to identify + // cached state related the instantiated function. Two functions + // of the same name and attrs, instantiated with the same + // `state_handle` will have the same handle and share the same + // state (in stateful kernels); and two functions with different + // values for `state_handle` will have independent state. + string state_handle; + + // This interface is EXPERIMENTAL and subject to change. + // + // Instantiates the function using an executor of the given type. If empty, + // the default TensorFlow executor will be used. + string executor_type; + + // If true, the runtime will attempt to create kernels for the function at + // instantiation time, rather than on the first run. This can be used to + // surface errors earlier. + bool create_kernels_eagerly = false; + }; + typedef uint64 Handle; + virtual Status Instantiate(const string& function_name, AttrSlice attrs, + const InstantiateOptions& options, + Handle* handle) = 0; + Status Instantiate(const string& function_name, AttrSlice attrs, + Handle* handle) { + return Instantiate(function_name, attrs, {}, handle); + } + + // Releases state associated with the handle. + virtual Status ReleaseHandle(Handle handle) = 0; + + // Returns the function body for the instantiated function given its + // handle 'h'. Returns nullptr if "h" is not found. + // + // *this keeps the ownership of the returned object, which remains alive + // as long as *this. + virtual const FunctionBody* GetFunctionBody(Handle h) = 0; + + // Asynchronously invokes the instantiated function identified by + // "handle". + // + // If function execution succeeds, "done" is called with OK and + // "*rets" is filled with the function's return values. Otheriwse, + // "done" is called with an error status. + // + // Does not take ownership of "rets". + // In the cross-process scenario, runner isn't used for making the Async + // RPC calls. + struct Options { + // The id of the step that is calling this function. + int64 step_id = 0; + Rendezvous* rendezvous = nullptr; + CancellationManager* cancellation_manager = nullptr; + CollectiveExecutor* collective_executor = nullptr; + ScopedStepContainer* step_container = nullptr; + StepStatsCollectorInterface* stats_collector = nullptr; + + std::function)>* runner = nullptr; + + // Parameters for remote function execution. + bool remote_execution = false; + string source_device = ""; // Fully specified device name. + + // Allocator attributes specifying where the args are / rets should be put. + // These should either be {} or match the length of args / retvals. If {}, + // the default allocator attributes will be assumed for all args / retvals. + std::vector args_alloc_attrs; + std::vector rets_alloc_attrs; + + // If true, we create a new IntraProcessRendezvous, else use the existing + // one. + bool create_rendezvous = false; + + // If True, allow returning dead tensors. + bool allow_dead_tensors = false; + }; + typedef std::function DoneCallback; + virtual void Run(const Options& opts, Handle handle, + gtl::ArraySlice args, std::vector* rets, + DoneCallback done) = 0; + virtual void Run(const Options& opts, Handle handle, + CallFrameInterface* call_frame, DoneCallback done) = 0; + + // Creates a "kernel" for the given node def "ndef". + // + // If succeeds, returns OK and the caller takes the ownership of the + // returned "*kernel". Otherwise, returns an error. + virtual Status CreateKernel(const NodeDef& ndef, OpKernel** kernel) = 0; + + // Returns true iff the function named `function_name` is stateful. + // NOTE(mrry): This method assumes that the runtime is associated with a + // default function library, and looks up `function_name` in that library. + // It does not support overlay libraries. + virtual bool IsStateful(const string& function_name) = 0; + + // Returns the device on which the function executes. + virtual Device* device() = 0; + + // Get the DeviceMgr from which the device was obtained. + virtual const DeviceMgr* device_mgr() const = 0; + + // Returns the function library definition that backs this runtime. + // NOTE(mrry): The returned library definition is the default function library + // for this runtime. The runtime may instantiate functions from separate + // overlay libraries, which are not returned by this function. + virtual const FunctionLibraryDefinition* GetFunctionLibraryDefinition() + const = 0; + + // Returns the environment on which the function executes. + virtual Env* env() = 0; + + // Returns a debug string showing the definition of the function of + // 'handle'. + virtual string DebugString(Handle handle) = 0; + + // Returns the graph version number. + virtual int graph_def_version() = 0; + + typedef uint64 LocalHandle; + + virtual Status Clone(std::unique_ptr* out_lib_def, + std::unique_ptr* out_pflr, + FunctionLibraryRuntime** out_flr) = 0; +}; + +// Returns a canonicalized string for the instantiation of the +// function of the given "name", attributes "attrs", and "options". +// +// The returned string is guaranteed to be stable within one address +// space. But it may be change as the implementation +// evolves. Therefore, it should not be persisted or compared across +// address spaces. +string Canonicalize(const string& funcname, AttrSlice attrs, + const FunctionLibraryRuntime::InstantiateOptions& options); +inline string Canonicalize(const string& funcname, AttrSlice attrs) { + return Canonicalize(funcname, attrs, {}); +} + +const FunctionLibraryRuntime::Handle kInvalidHandle = -1; +const FunctionLibraryRuntime::LocalHandle kInvalidLocalHandle = -1; +typedef std::function*)> + CustomKernelCreator; + +// Used to instantiate and run functions in a distributed system. +class DistributedFunctionLibraryRuntime { + public: + virtual ~DistributedFunctionLibraryRuntime() {} + + // The _target attr in attrs determines where the function is instantiated. + virtual Status Instantiate( + const string& function_name, const FunctionLibraryDefinition& lib_def, + AttrSlice attrs, + const FunctionLibraryRuntime::InstantiateOptions& options, + FunctionLibraryRuntime::LocalHandle* handle) = 0; + + // opts.runner isn't used for execution. + virtual void Run(const FunctionLibraryRuntime::Options& opts, + FunctionLibraryRuntime::LocalHandle handle, + gtl::ArraySlice args, std::vector* rets, + FunctionLibraryRuntime::DoneCallback done) = 0; +}; + +// Extracts the actual type from "attr_values" based on its definition +// "arg_def". +// +// If "arg_def" is a N*T type, *is_type_list is set to false, and +// *dtypes is set to be a vector of size N and each element is T. +// +// If "arg_def" is a list(type), *is_type_list is set to true, and +// *dtypes is set to be a vector of types specified in attrs for +// arg_def. +// +// Otherwise (arg_def is a simple type T), *is_type_list is set to +// false, and *dtypes is set to a single element vector, whose only +// element is T. +Status ArgNumType(AttrSlice attrs, const OpDef::ArgDef& arg_def, + bool* is_type_list, DataTypeVector* dtypes); + +// To register a gradient function for a builtin op, one should use +// REGISTER_OP_GRADIENT(, ); +// +// Typically, the c++ grad factory is a plan function that can be +// converted into ::tensorflow::gradient::Creator, which is +// std::function. +// +// A ::tensorflow::gradient::Creator should populate in FunctionDef* with a +// definition of a brain function which compute the gradient for the +// when the is instantiated with the given attrs. +// +// E.g., +// +// Status MatMulGrad(const AttrSlice& attrs, FunctionDef* g) { +// bool transpose_a; +// TF_RETURN_IF_ERROR(attrs.Get("transpose_a", &transpose_a)); +// bool transpose_b; +// TF_RETURN_IF_ERROR(attrs.Get("transpose_b", &transpose_b)); +// DataType dtype; +// TF_RETURN_IF_ERROR(attrs.Get("dtype", &dtype)); +// if (!transpose_a && !transpose_b) { +// *g = FunctionDefHelper::Define( +// "MatMulGrad", +// {"x:T ", "y:T", "dz:T"}, // Inputs to this function +// {"dx:T", "dy:T"}, // Outputs from this function +// {"T: {float, double}"}, // Attributes needed by this function +// { +// {{"x_t"}, "Transpose", {"x"}, {{"T", "$T"}}}, +// {{"y_t"}, "Transpose", {"y"}, {{"T", "$T"}}}, +// {{"dx"}, "MatMul", {"dz", "y_t"}, {{"T", "$T"}}}, +// {{"dy"}, "MatMul", {"x_", "dz"}, {{"T", "$T"}}}, +// }); +// } else { +// ... ... +// } +// return Status::OK(); +// } +// +// NOTE: $T is substituted with the type variable "T" when the +// gradient function MatMul is instantiated. +// +// TODO(zhifengc): Better documentation somewhere. + +// Macros to define a gradient function factory for a primitive +// operation. +#define REGISTER_OP_GRADIENT(name, fn) \ + REGISTER_OP_GRADIENT_UNIQ_HELPER(__COUNTER__, name, fn) + +#define REGISTER_OP_NO_GRADIENT(name) \ + REGISTER_OP_GRADIENT_UNIQ_HELPER(__COUNTER__, name, nullptr) + +#define REGISTER_OP_GRADIENT_UNIQ_HELPER(ctr, name, fn) \ + REGISTER_OP_GRADIENT_UNIQ(ctr, name, fn) + +#define REGISTER_OP_GRADIENT_UNIQ(ctr, name, fn) \ + static bool unused_grad_##ctr TF_ATTRIBUTE_UNUSED = \ + SHOULD_REGISTER_OP_GRADIENT && \ + ::tensorflow::gradient::RegisterOp(name, fn) + +namespace gradient { +// Register a gradient creator for the "op". +typedef std::function Creator; +bool RegisterOp(const string& op, Creator func); + +// Returns OK the gradient creator for the "op" is found (may be +// nullptr if REGISTER_OP_NO_GRADIENT is used. +Status GetOpGradientCreator(const string& op, Creator* creator); +}; // namespace gradient + +// Declare explicit instantiations of GetAttr +#define GET_ATTR(T) \ + extern template Status FunctionLibraryDefinition::GetAttr( \ + const Node&, const string&, T*) const; \ + extern template Status FunctionLibraryDefinition::GetAttr( \ + const NodeDef&, const string&, T*) const; +GET_ATTR(string) +GET_ATTR(bool) +#undef GET_ATTR + +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_FRAMEWORK_FUNCTION_H_ diff --git a/linux/include/tensorflow/core/framework/function.pb.h b/linux/include/tensorflow/core/framework/function.pb.h new file mode 100644 index 000000000..24710985e --- /dev/null +++ b/linux/include/tensorflow/core/framework/function.pb.h @@ -0,0 +1,961 @@ +// Generated by the protocol buffer compiler. DO NOT EDIT! +// source: tensorflow/core/framework/function.proto + +#ifndef PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2ffunction_2eproto +#define PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2ffunction_2eproto + +#include + +#include + +#if GOOGLE_PROTOBUF_VERSION < 3006000 +#error This file was generated by a newer version of protoc which is +#error incompatible with your Protocol Buffer headers. Please update +#error your headers. +#endif +#if 3006000 < GOOGLE_PROTOBUF_MIN_PROTOC_VERSION +#error This file was generated by an older version of protoc which is +#error incompatible with your Protocol Buffer headers. Please +#error regenerate this file with a newer version of protoc. +#endif + +#include +#include +#include +#include +#include +#include +#include +#include +#include // IWYU pragma: export +#include // IWYU pragma: export +#include // IWYU pragma: export +#include +#include +#include +#include "tensorflow/core/framework/attr_value.pb.h" +#include "tensorflow/core/framework/node_def.pb.h" +#include "tensorflow/core/framework/op_def.pb.h" +// @@protoc_insertion_point(includes) +#define PROTOBUF_INTERNAL_EXPORT_protobuf_tensorflow_2fcore_2fframework_2ffunction_2eproto + +namespace protobuf_tensorflow_2fcore_2fframework_2ffunction_2eproto { +// Internal implementation detail -- do not use these members. +struct TableStruct { + static const ::google::protobuf::internal::ParseTableField entries[]; + static const ::google::protobuf::internal::AuxillaryParseTableField aux[]; + static const ::google::protobuf::internal::ParseTable schema[5]; + static const ::google::protobuf::internal::FieldMetadata field_metadata[]; + static const ::google::protobuf::internal::SerializationTable serialization_table[]; + static const ::google::protobuf::uint32 offsets[]; +}; +void AddDescriptors(); +} // namespace protobuf_tensorflow_2fcore_2fframework_2ffunction_2eproto +namespace tensorflow { +class FunctionDef; +class FunctionDefDefaultTypeInternal; +extern FunctionDefDefaultTypeInternal _FunctionDef_default_instance_; +class FunctionDefLibrary; +class FunctionDefLibraryDefaultTypeInternal; +extern FunctionDefLibraryDefaultTypeInternal _FunctionDefLibrary_default_instance_; +class FunctionDef_AttrEntry_DoNotUse; +class FunctionDef_AttrEntry_DoNotUseDefaultTypeInternal; +extern FunctionDef_AttrEntry_DoNotUseDefaultTypeInternal _FunctionDef_AttrEntry_DoNotUse_default_instance_; +class FunctionDef_RetEntry_DoNotUse; +class FunctionDef_RetEntry_DoNotUseDefaultTypeInternal; +extern FunctionDef_RetEntry_DoNotUseDefaultTypeInternal _FunctionDef_RetEntry_DoNotUse_default_instance_; +class GradientDef; +class GradientDefDefaultTypeInternal; +extern GradientDefDefaultTypeInternal _GradientDef_default_instance_; +} // namespace tensorflow +namespace google { +namespace protobuf { +template<> ::tensorflow::FunctionDef* Arena::CreateMaybeMessage<::tensorflow::FunctionDef>(Arena*); +template<> ::tensorflow::FunctionDefLibrary* Arena::CreateMaybeMessage<::tensorflow::FunctionDefLibrary>(Arena*); +template<> ::tensorflow::FunctionDef_AttrEntry_DoNotUse* Arena::CreateMaybeMessage<::tensorflow::FunctionDef_AttrEntry_DoNotUse>(Arena*); +template<> ::tensorflow::FunctionDef_RetEntry_DoNotUse* Arena::CreateMaybeMessage<::tensorflow::FunctionDef_RetEntry_DoNotUse>(Arena*); +template<> ::tensorflow::GradientDef* Arena::CreateMaybeMessage<::tensorflow::GradientDef>(Arena*); +} // namespace protobuf +} // namespace google +namespace tensorflow { + +// =================================================================== + +class FunctionDefLibrary : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.FunctionDefLibrary) */ { + public: + FunctionDefLibrary(); + virtual ~FunctionDefLibrary(); + + FunctionDefLibrary(const FunctionDefLibrary& from); + + inline FunctionDefLibrary& operator=(const FunctionDefLibrary& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + FunctionDefLibrary(FunctionDefLibrary&& from) noexcept + : FunctionDefLibrary() { + *this = ::std::move(from); + } + + inline FunctionDefLibrary& operator=(FunctionDefLibrary&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const FunctionDefLibrary& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const FunctionDefLibrary* internal_default_instance() { + return reinterpret_cast( + &_FunctionDefLibrary_default_instance_); + } + static constexpr int kIndexInFileMessages = + 0; + + void UnsafeArenaSwap(FunctionDefLibrary* other); + void Swap(FunctionDefLibrary* other); + friend void swap(FunctionDefLibrary& a, FunctionDefLibrary& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline FunctionDefLibrary* New() const final { + return CreateMaybeMessage(NULL); + } + + FunctionDefLibrary* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const FunctionDefLibrary& from); + void MergeFrom(const FunctionDefLibrary& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(FunctionDefLibrary* other); + protected: + explicit FunctionDefLibrary(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated .tensorflow.FunctionDef function = 1; + int function_size() const; + void clear_function(); + static const int kFunctionFieldNumber = 1; + ::tensorflow::FunctionDef* mutable_function(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::FunctionDef >* + mutable_function(); + const ::tensorflow::FunctionDef& function(int index) const; + ::tensorflow::FunctionDef* add_function(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::FunctionDef >& + function() const; + + // repeated .tensorflow.GradientDef gradient = 2; + int gradient_size() const; + void clear_gradient(); + static const int kGradientFieldNumber = 2; + ::tensorflow::GradientDef* mutable_gradient(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::GradientDef >* + mutable_gradient(); + const ::tensorflow::GradientDef& gradient(int index) const; + ::tensorflow::GradientDef* add_gradient(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::GradientDef >& + gradient() const; + + // @@protoc_insertion_point(class_scope:tensorflow.FunctionDefLibrary) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::FunctionDef > function_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::GradientDef > gradient_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2ffunction_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class FunctionDef_AttrEntry_DoNotUse : public ::google::protobuf::internal::MapEntry { +public: + typedef ::google::protobuf::internal::MapEntry SuperType; + FunctionDef_AttrEntry_DoNotUse(); + FunctionDef_AttrEntry_DoNotUse(::google::protobuf::Arena* arena); + void MergeFrom(const FunctionDef_AttrEntry_DoNotUse& other); + static const FunctionDef_AttrEntry_DoNotUse* internal_default_instance() { return reinterpret_cast(&_FunctionDef_AttrEntry_DoNotUse_default_instance_); } + void MergeFrom(const ::google::protobuf::Message& other) final; + ::google::protobuf::Metadata GetMetadata() const; +}; + +// ------------------------------------------------------------------- + +class FunctionDef_RetEntry_DoNotUse : public ::google::protobuf::internal::MapEntry { +public: + typedef ::google::protobuf::internal::MapEntry SuperType; + FunctionDef_RetEntry_DoNotUse(); + FunctionDef_RetEntry_DoNotUse(::google::protobuf::Arena* arena); + void MergeFrom(const FunctionDef_RetEntry_DoNotUse& other); + static const FunctionDef_RetEntry_DoNotUse* internal_default_instance() { return reinterpret_cast(&_FunctionDef_RetEntry_DoNotUse_default_instance_); } + void MergeFrom(const ::google::protobuf::Message& other) final; + ::google::protobuf::Metadata GetMetadata() const; +}; + +// ------------------------------------------------------------------- + +class FunctionDef : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.FunctionDef) */ { + public: + FunctionDef(); + virtual ~FunctionDef(); + + FunctionDef(const FunctionDef& from); + + inline FunctionDef& operator=(const FunctionDef& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + FunctionDef(FunctionDef&& from) noexcept + : FunctionDef() { + *this = ::std::move(from); + } + + inline FunctionDef& operator=(FunctionDef&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const FunctionDef& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const FunctionDef* internal_default_instance() { + return reinterpret_cast( + &_FunctionDef_default_instance_); + } + static constexpr int kIndexInFileMessages = + 3; + + void UnsafeArenaSwap(FunctionDef* other); + void Swap(FunctionDef* other); + friend void swap(FunctionDef& a, FunctionDef& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline FunctionDef* New() const final { + return CreateMaybeMessage(NULL); + } + + FunctionDef* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const FunctionDef& from); + void MergeFrom(const FunctionDef& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(FunctionDef* other); + protected: + explicit FunctionDef(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + + // accessors ------------------------------------------------------- + + // repeated .tensorflow.NodeDef node_def = 3; + int node_def_size() const; + void clear_node_def(); + static const int kNodeDefFieldNumber = 3; + ::tensorflow::NodeDef* mutable_node_def(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::NodeDef >* + mutable_node_def(); + const ::tensorflow::NodeDef& node_def(int index) const; + ::tensorflow::NodeDef* add_node_def(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::NodeDef >& + node_def() const; + + // map ret = 4; + int ret_size() const; + void clear_ret(); + static const int kRetFieldNumber = 4; + const ::google::protobuf::Map< ::std::string, ::std::string >& + ret() const; + ::google::protobuf::Map< ::std::string, ::std::string >* + mutable_ret(); + + // map attr = 5; + int attr_size() const; + void clear_attr(); + static const int kAttrFieldNumber = 5; + const ::google::protobuf::Map< ::std::string, ::tensorflow::AttrValue >& + attr() const; + ::google::protobuf::Map< ::std::string, ::tensorflow::AttrValue >* + mutable_attr(); + + // .tensorflow.OpDef signature = 1; + bool has_signature() const; + void clear_signature(); + static const int kSignatureFieldNumber = 1; + private: + const ::tensorflow::OpDef& _internal_signature() const; + public: + const ::tensorflow::OpDef& signature() const; + ::tensorflow::OpDef* release_signature(); + ::tensorflow::OpDef* mutable_signature(); + void set_allocated_signature(::tensorflow::OpDef* signature); + void unsafe_arena_set_allocated_signature( + ::tensorflow::OpDef* signature); + ::tensorflow::OpDef* unsafe_arena_release_signature(); + + // @@protoc_insertion_point(class_scope:tensorflow.FunctionDef) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::NodeDef > node_def_; + ::google::protobuf::internal::MapField< + FunctionDef_RetEntry_DoNotUse, + ::std::string, ::std::string, + ::google::protobuf::internal::WireFormatLite::TYPE_STRING, + ::google::protobuf::internal::WireFormatLite::TYPE_STRING, + 0 > ret_; + ::google::protobuf::internal::MapField< + FunctionDef_AttrEntry_DoNotUse, + ::std::string, ::tensorflow::AttrValue, + ::google::protobuf::internal::WireFormatLite::TYPE_STRING, + ::google::protobuf::internal::WireFormatLite::TYPE_MESSAGE, + 0 > attr_; + ::tensorflow::OpDef* signature_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2ffunction_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class GradientDef : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.GradientDef) */ { + public: + GradientDef(); + virtual ~GradientDef(); + + GradientDef(const GradientDef& from); + + inline GradientDef& operator=(const GradientDef& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + GradientDef(GradientDef&& from) noexcept + : GradientDef() { + *this = ::std::move(from); + } + + inline GradientDef& operator=(GradientDef&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const GradientDef& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const GradientDef* internal_default_instance() { + return reinterpret_cast( + &_GradientDef_default_instance_); + } + static constexpr int kIndexInFileMessages = + 4; + + void UnsafeArenaSwap(GradientDef* other); + void Swap(GradientDef* other); + friend void swap(GradientDef& a, GradientDef& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline GradientDef* New() const final { + return CreateMaybeMessage(NULL); + } + + GradientDef* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const GradientDef& from); + void MergeFrom(const GradientDef& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(GradientDef* other); + protected: + explicit GradientDef(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // string function_name = 1; + void clear_function_name(); + static const int kFunctionNameFieldNumber = 1; + const ::std::string& function_name() const; + void set_function_name(const ::std::string& value); + #if LANG_CXX11 + void set_function_name(::std::string&& value); + #endif + void set_function_name(const char* value); + void set_function_name(const char* value, size_t size); + ::std::string* mutable_function_name(); + ::std::string* release_function_name(); + void set_allocated_function_name(::std::string* function_name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_function_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_function_name( + ::std::string* function_name); + + // string gradient_func = 2; + void clear_gradient_func(); + static const int kGradientFuncFieldNumber = 2; + const ::std::string& gradient_func() const; + void set_gradient_func(const ::std::string& value); + #if LANG_CXX11 + void set_gradient_func(::std::string&& value); + #endif + void set_gradient_func(const char* value); + void set_gradient_func(const char* value, size_t size); + ::std::string* mutable_gradient_func(); + ::std::string* release_gradient_func(); + void set_allocated_gradient_func(::std::string* gradient_func); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_gradient_func(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_gradient_func( + ::std::string* gradient_func); + + // @@protoc_insertion_point(class_scope:tensorflow.GradientDef) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::internal::ArenaStringPtr function_name_; + ::google::protobuf::internal::ArenaStringPtr gradient_func_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2ffunction_2eproto::TableStruct; +}; +// =================================================================== + + +// =================================================================== + +#ifdef __GNUC__ + #pragma GCC diagnostic push + #pragma GCC diagnostic ignored "-Wstrict-aliasing" +#endif // __GNUC__ +// FunctionDefLibrary + +// repeated .tensorflow.FunctionDef function = 1; +inline int FunctionDefLibrary::function_size() const { + return function_.size(); +} +inline void FunctionDefLibrary::clear_function() { + function_.Clear(); +} +inline ::tensorflow::FunctionDef* FunctionDefLibrary::mutable_function(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.FunctionDefLibrary.function) + return function_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::FunctionDef >* +FunctionDefLibrary::mutable_function() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.FunctionDefLibrary.function) + return &function_; +} +inline const ::tensorflow::FunctionDef& FunctionDefLibrary::function(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.FunctionDefLibrary.function) + return function_.Get(index); +} +inline ::tensorflow::FunctionDef* FunctionDefLibrary::add_function() { + // @@protoc_insertion_point(field_add:tensorflow.FunctionDefLibrary.function) + return function_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::FunctionDef >& +FunctionDefLibrary::function() const { + // @@protoc_insertion_point(field_list:tensorflow.FunctionDefLibrary.function) + return function_; +} + +// repeated .tensorflow.GradientDef gradient = 2; +inline int FunctionDefLibrary::gradient_size() const { + return gradient_.size(); +} +inline void FunctionDefLibrary::clear_gradient() { + gradient_.Clear(); +} +inline ::tensorflow::GradientDef* FunctionDefLibrary::mutable_gradient(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.FunctionDefLibrary.gradient) + return gradient_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::GradientDef >* +FunctionDefLibrary::mutable_gradient() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.FunctionDefLibrary.gradient) + return &gradient_; +} +inline const ::tensorflow::GradientDef& FunctionDefLibrary::gradient(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.FunctionDefLibrary.gradient) + return gradient_.Get(index); +} +inline ::tensorflow::GradientDef* FunctionDefLibrary::add_gradient() { + // @@protoc_insertion_point(field_add:tensorflow.FunctionDefLibrary.gradient) + return gradient_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::GradientDef >& +FunctionDefLibrary::gradient() const { + // @@protoc_insertion_point(field_list:tensorflow.FunctionDefLibrary.gradient) + return gradient_; +} + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// FunctionDef + +// .tensorflow.OpDef signature = 1; +inline bool FunctionDef::has_signature() const { + return this != internal_default_instance() && signature_ != NULL; +} +inline const ::tensorflow::OpDef& FunctionDef::_internal_signature() const { + return *signature_; +} +inline const ::tensorflow::OpDef& FunctionDef::signature() const { + const ::tensorflow::OpDef* p = signature_; + // @@protoc_insertion_point(field_get:tensorflow.FunctionDef.signature) + return p != NULL ? *p : *reinterpret_cast( + &::tensorflow::_OpDef_default_instance_); +} +inline ::tensorflow::OpDef* FunctionDef::release_signature() { + // @@protoc_insertion_point(field_release:tensorflow.FunctionDef.signature) + + ::tensorflow::OpDef* temp = signature_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + signature_ = NULL; + return temp; +} +inline ::tensorflow::OpDef* FunctionDef::unsafe_arena_release_signature() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.FunctionDef.signature) + + ::tensorflow::OpDef* temp = signature_; + signature_ = NULL; + return temp; +} +inline ::tensorflow::OpDef* FunctionDef::mutable_signature() { + + if (signature_ == NULL) { + auto* p = CreateMaybeMessage<::tensorflow::OpDef>(GetArenaNoVirtual()); + signature_ = p; + } + // @@protoc_insertion_point(field_mutable:tensorflow.FunctionDef.signature) + return signature_; +} +inline void FunctionDef::set_allocated_signature(::tensorflow::OpDef* signature) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(signature_); + } + if (signature) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(signature)->GetArena(); + if (message_arena != submessage_arena) { + signature = ::google::protobuf::internal::GetOwnedMessage( + message_arena, signature, submessage_arena); + } + + } else { + + } + signature_ = signature; + // @@protoc_insertion_point(field_set_allocated:tensorflow.FunctionDef.signature) +} + +// map attr = 5; +inline int FunctionDef::attr_size() const { + return attr_.size(); +} +inline const ::google::protobuf::Map< ::std::string, ::tensorflow::AttrValue >& +FunctionDef::attr() const { + // @@protoc_insertion_point(field_map:tensorflow.FunctionDef.attr) + return attr_.GetMap(); +} +inline ::google::protobuf::Map< ::std::string, ::tensorflow::AttrValue >* +FunctionDef::mutable_attr() { + // @@protoc_insertion_point(field_mutable_map:tensorflow.FunctionDef.attr) + return attr_.MutableMap(); +} + +// repeated .tensorflow.NodeDef node_def = 3; +inline int FunctionDef::node_def_size() const { + return node_def_.size(); +} +inline ::tensorflow::NodeDef* FunctionDef::mutable_node_def(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.FunctionDef.node_def) + return node_def_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::NodeDef >* +FunctionDef::mutable_node_def() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.FunctionDef.node_def) + return &node_def_; +} +inline const ::tensorflow::NodeDef& FunctionDef::node_def(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.FunctionDef.node_def) + return node_def_.Get(index); +} +inline ::tensorflow::NodeDef* FunctionDef::add_node_def() { + // @@protoc_insertion_point(field_add:tensorflow.FunctionDef.node_def) + return node_def_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::NodeDef >& +FunctionDef::node_def() const { + // @@protoc_insertion_point(field_list:tensorflow.FunctionDef.node_def) + return node_def_; +} + +// map ret = 4; +inline int FunctionDef::ret_size() const { + return ret_.size(); +} +inline void FunctionDef::clear_ret() { + ret_.Clear(); +} +inline const ::google::protobuf::Map< ::std::string, ::std::string >& +FunctionDef::ret() const { + // @@protoc_insertion_point(field_map:tensorflow.FunctionDef.ret) + return ret_.GetMap(); +} +inline ::google::protobuf::Map< ::std::string, ::std::string >* +FunctionDef::mutable_ret() { + // @@protoc_insertion_point(field_mutable_map:tensorflow.FunctionDef.ret) + return ret_.MutableMap(); +} + +// ------------------------------------------------------------------- + +// GradientDef + +// string function_name = 1; +inline void GradientDef::clear_function_name() { + function_name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& GradientDef::function_name() const { + // @@protoc_insertion_point(field_get:tensorflow.GradientDef.function_name) + return function_name_.Get(); +} +inline void GradientDef::set_function_name(const ::std::string& value) { + + function_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.GradientDef.function_name) +} +#if LANG_CXX11 +inline void GradientDef::set_function_name(::std::string&& value) { + + function_name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.GradientDef.function_name) +} +#endif +inline void GradientDef::set_function_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + function_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.GradientDef.function_name) +} +inline void GradientDef::set_function_name(const char* value, + size_t size) { + + function_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.GradientDef.function_name) +} +inline ::std::string* GradientDef::mutable_function_name() { + + // @@protoc_insertion_point(field_mutable:tensorflow.GradientDef.function_name) + return function_name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* GradientDef::release_function_name() { + // @@protoc_insertion_point(field_release:tensorflow.GradientDef.function_name) + + return function_name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void GradientDef::set_allocated_function_name(::std::string* function_name) { + if (function_name != NULL) { + + } else { + + } + function_name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), function_name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.GradientDef.function_name) +} +inline ::std::string* GradientDef::unsafe_arena_release_function_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.GradientDef.function_name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return function_name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void GradientDef::unsafe_arena_set_allocated_function_name( + ::std::string* function_name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (function_name != NULL) { + + } else { + + } + function_name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + function_name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.GradientDef.function_name) +} + +// string gradient_func = 2; +inline void GradientDef::clear_gradient_func() { + gradient_func_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& GradientDef::gradient_func() const { + // @@protoc_insertion_point(field_get:tensorflow.GradientDef.gradient_func) + return gradient_func_.Get(); +} +inline void GradientDef::set_gradient_func(const ::std::string& value) { + + gradient_func_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.GradientDef.gradient_func) +} +#if LANG_CXX11 +inline void GradientDef::set_gradient_func(::std::string&& value) { + + gradient_func_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.GradientDef.gradient_func) +} +#endif +inline void GradientDef::set_gradient_func(const char* value) { + GOOGLE_DCHECK(value != NULL); + + gradient_func_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.GradientDef.gradient_func) +} +inline void GradientDef::set_gradient_func(const char* value, + size_t size) { + + gradient_func_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.GradientDef.gradient_func) +} +inline ::std::string* GradientDef::mutable_gradient_func() { + + // @@protoc_insertion_point(field_mutable:tensorflow.GradientDef.gradient_func) + return gradient_func_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* GradientDef::release_gradient_func() { + // @@protoc_insertion_point(field_release:tensorflow.GradientDef.gradient_func) + + return gradient_func_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void GradientDef::set_allocated_gradient_func(::std::string* gradient_func) { + if (gradient_func != NULL) { + + } else { + + } + gradient_func_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), gradient_func, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.GradientDef.gradient_func) +} +inline ::std::string* GradientDef::unsafe_arena_release_gradient_func() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.GradientDef.gradient_func) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return gradient_func_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void GradientDef::unsafe_arena_set_allocated_gradient_func( + ::std::string* gradient_func) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (gradient_func != NULL) { + + } else { + + } + gradient_func_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + gradient_func, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.GradientDef.gradient_func) +} + +#ifdef __GNUC__ + #pragma GCC diagnostic pop +#endif // __GNUC__ +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + + +// @@protoc_insertion_point(namespace_scope) + +} // namespace tensorflow + +// @@protoc_insertion_point(global_scope) + +#endif // PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2ffunction_2eproto diff --git a/linux/include/tensorflow/core/framework/function.pb_text-impl.h b/linux/include/tensorflow/core/framework/function.pb_text-impl.h new file mode 100644 index 000000000..77f643364 --- /dev/null +++ b/linux/include/tensorflow/core/framework/function.pb_text-impl.h @@ -0,0 +1,53 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_framework_function_proto_IMPL_H_ +#define tensorflow_core_framework_function_proto_IMPL_H_ + +#include "tensorflow/core/framework/attr_value.pb.h" +#include "tensorflow/core/framework/attr_value.pb_text-impl.h" +#include "tensorflow/core/framework/function.pb.h" +#include "tensorflow/core/framework/function.pb_text.h" +#include "tensorflow/core/framework/node_def.pb.h" +#include "tensorflow/core/framework/node_def.pb_text-impl.h" +#include "tensorflow/core/framework/op_def.pb.h" +#include "tensorflow/core/framework/op_def.pb_text-impl.h" +#include "tensorflow/core/framework/resource_handle.pb.h" +#include "tensorflow/core/framework/resource_handle.pb_text-impl.h" +#include "tensorflow/core/framework/tensor.pb.h" +#include "tensorflow/core/framework/tensor.pb_text-impl.h" +#include "tensorflow/core/framework/tensor_shape.pb.h" +#include "tensorflow/core/framework/tensor_shape.pb_text-impl.h" +#include "tensorflow/core/framework/types.pb.h" +#include "tensorflow/core/framework/types.pb_text-impl.h" +#include "tensorflow/core/lib/strings/proto_text_util.h" +#include "tensorflow/core/lib/strings/scanner.h" + +namespace tensorflow { + +namespace internal { + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::FunctionDefLibrary& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::FunctionDefLibrary* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::FunctionDef& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::FunctionDef* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::GradientDef& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::GradientDef* msg); + +} // namespace internal + +} // namespace tensorflow + +#endif // tensorflow_core_framework_function_proto_IMPL_H_ diff --git a/linux/include/tensorflow/core/framework/function.pb_text.h b/linux/include/tensorflow/core/framework/function.pb_text.h new file mode 100644 index 000000000..ca49f7ed4 --- /dev/null +++ b/linux/include/tensorflow/core/framework/function.pb_text.h @@ -0,0 +1,44 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_framework_function_proto_H_ +#define tensorflow_core_framework_function_proto_H_ + +#include "tensorflow/core/framework/function.pb.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/protobuf.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +// Message-text conversion for tensorflow.FunctionDefLibrary +string ProtoDebugString( + const ::tensorflow::FunctionDefLibrary& msg); +string ProtoShortDebugString( + const ::tensorflow::FunctionDefLibrary& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::FunctionDefLibrary* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.FunctionDef +string ProtoDebugString( + const ::tensorflow::FunctionDef& msg); +string ProtoShortDebugString( + const ::tensorflow::FunctionDef& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::FunctionDef* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.GradientDef +string ProtoDebugString( + const ::tensorflow::GradientDef& msg); +string ProtoShortDebugString( + const ::tensorflow::GradientDef& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::GradientDef* msg) + TF_MUST_USE_RESULT; + +} // namespace tensorflow + +#endif // tensorflow_core_framework_function_proto_H_ diff --git a/linux/include/tensorflow/core/framework/graph.pb.h b/linux/include/tensorflow/core/framework/graph.pb.h new file mode 100644 index 000000000..8e980f25d --- /dev/null +++ b/linux/include/tensorflow/core/framework/graph.pb.h @@ -0,0 +1,406 @@ +// Generated by the protocol buffer compiler. DO NOT EDIT! +// source: tensorflow/core/framework/graph.proto + +#ifndef PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2fgraph_2eproto +#define PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2fgraph_2eproto + +#include + +#include + +#if GOOGLE_PROTOBUF_VERSION < 3006000 +#error This file was generated by a newer version of protoc which is +#error incompatible with your Protocol Buffer headers. Please update +#error your headers. +#endif +#if 3006000 < GOOGLE_PROTOBUF_MIN_PROTOC_VERSION +#error This file was generated by an older version of protoc which is +#error incompatible with your Protocol Buffer headers. Please +#error regenerate this file with a newer version of protoc. +#endif + +#include +#include +#include +#include +#include +#include +#include +#include +#include // IWYU pragma: export +#include // IWYU pragma: export +#include +#include "tensorflow/core/framework/node_def.pb.h" +#include "tensorflow/core/framework/function.pb.h" +#include "tensorflow/core/framework/versions.pb.h" +// @@protoc_insertion_point(includes) +#define PROTOBUF_INTERNAL_EXPORT_protobuf_tensorflow_2fcore_2fframework_2fgraph_2eproto + +namespace protobuf_tensorflow_2fcore_2fframework_2fgraph_2eproto { +// Internal implementation detail -- do not use these members. +struct TableStruct { + static const ::google::protobuf::internal::ParseTableField entries[]; + static const ::google::protobuf::internal::AuxillaryParseTableField aux[]; + static const ::google::protobuf::internal::ParseTable schema[1]; + static const ::google::protobuf::internal::FieldMetadata field_metadata[]; + static const ::google::protobuf::internal::SerializationTable serialization_table[]; + static const ::google::protobuf::uint32 offsets[]; +}; +void AddDescriptors(); +} // namespace protobuf_tensorflow_2fcore_2fframework_2fgraph_2eproto +namespace tensorflow { +class GraphDef; +class GraphDefDefaultTypeInternal; +extern GraphDefDefaultTypeInternal _GraphDef_default_instance_; +} // namespace tensorflow +namespace google { +namespace protobuf { +template<> ::tensorflow::GraphDef* Arena::CreateMaybeMessage<::tensorflow::GraphDef>(Arena*); +} // namespace protobuf +} // namespace google +namespace tensorflow { + +// =================================================================== + +class GraphDef : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.GraphDef) */ { + public: + GraphDef(); + virtual ~GraphDef(); + + GraphDef(const GraphDef& from); + + inline GraphDef& operator=(const GraphDef& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + GraphDef(GraphDef&& from) noexcept + : GraphDef() { + *this = ::std::move(from); + } + + inline GraphDef& operator=(GraphDef&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const GraphDef& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const GraphDef* internal_default_instance() { + return reinterpret_cast( + &_GraphDef_default_instance_); + } + static constexpr int kIndexInFileMessages = + 0; + + void UnsafeArenaSwap(GraphDef* other); + void Swap(GraphDef* other); + friend void swap(GraphDef& a, GraphDef& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline GraphDef* New() const final { + return CreateMaybeMessage(NULL); + } + + GraphDef* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const GraphDef& from); + void MergeFrom(const GraphDef& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(GraphDef* other); + protected: + explicit GraphDef(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated .tensorflow.NodeDef node = 1; + int node_size() const; + void clear_node(); + static const int kNodeFieldNumber = 1; + ::tensorflow::NodeDef* mutable_node(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::NodeDef >* + mutable_node(); + const ::tensorflow::NodeDef& node(int index) const; + ::tensorflow::NodeDef* add_node(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::NodeDef >& + node() const; + + // .tensorflow.FunctionDefLibrary library = 2; + bool has_library() const; + void clear_library(); + static const int kLibraryFieldNumber = 2; + private: + const ::tensorflow::FunctionDefLibrary& _internal_library() const; + public: + const ::tensorflow::FunctionDefLibrary& library() const; + ::tensorflow::FunctionDefLibrary* release_library(); + ::tensorflow::FunctionDefLibrary* mutable_library(); + void set_allocated_library(::tensorflow::FunctionDefLibrary* library); + void unsafe_arena_set_allocated_library( + ::tensorflow::FunctionDefLibrary* library); + ::tensorflow::FunctionDefLibrary* unsafe_arena_release_library(); + + // .tensorflow.VersionDef versions = 4; + bool has_versions() const; + void clear_versions(); + static const int kVersionsFieldNumber = 4; + private: + const ::tensorflow::VersionDef& _internal_versions() const; + public: + const ::tensorflow::VersionDef& versions() const; + ::tensorflow::VersionDef* release_versions(); + ::tensorflow::VersionDef* mutable_versions(); + void set_allocated_versions(::tensorflow::VersionDef* versions); + void unsafe_arena_set_allocated_versions( + ::tensorflow::VersionDef* versions); + ::tensorflow::VersionDef* unsafe_arena_release_versions(); + + // int32 version = 3 [deprecated = true]; + GOOGLE_PROTOBUF_DEPRECATED_ATTR void clear_version(); + GOOGLE_PROTOBUF_DEPRECATED_ATTR static const int kVersionFieldNumber = 3; + GOOGLE_PROTOBUF_DEPRECATED_ATTR ::google::protobuf::int32 version() const; + GOOGLE_PROTOBUF_DEPRECATED_ATTR void set_version(::google::protobuf::int32 value); + + // @@protoc_insertion_point(class_scope:tensorflow.GraphDef) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::NodeDef > node_; + ::tensorflow::FunctionDefLibrary* library_; + ::tensorflow::VersionDef* versions_; + ::google::protobuf::int32 version_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fgraph_2eproto::TableStruct; +}; +// =================================================================== + + +// =================================================================== + +#ifdef __GNUC__ + #pragma GCC diagnostic push + #pragma GCC diagnostic ignored "-Wstrict-aliasing" +#endif // __GNUC__ +// GraphDef + +// repeated .tensorflow.NodeDef node = 1; +inline int GraphDef::node_size() const { + return node_.size(); +} +inline ::tensorflow::NodeDef* GraphDef::mutable_node(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.GraphDef.node) + return node_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::NodeDef >* +GraphDef::mutable_node() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.GraphDef.node) + return &node_; +} +inline const ::tensorflow::NodeDef& GraphDef::node(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.GraphDef.node) + return node_.Get(index); +} +inline ::tensorflow::NodeDef* GraphDef::add_node() { + // @@protoc_insertion_point(field_add:tensorflow.GraphDef.node) + return node_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::NodeDef >& +GraphDef::node() const { + // @@protoc_insertion_point(field_list:tensorflow.GraphDef.node) + return node_; +} + +// .tensorflow.VersionDef versions = 4; +inline bool GraphDef::has_versions() const { + return this != internal_default_instance() && versions_ != NULL; +} +inline const ::tensorflow::VersionDef& GraphDef::_internal_versions() const { + return *versions_; +} +inline const ::tensorflow::VersionDef& GraphDef::versions() const { + const ::tensorflow::VersionDef* p = versions_; + // @@protoc_insertion_point(field_get:tensorflow.GraphDef.versions) + return p != NULL ? *p : *reinterpret_cast( + &::tensorflow::_VersionDef_default_instance_); +} +inline ::tensorflow::VersionDef* GraphDef::release_versions() { + // @@protoc_insertion_point(field_release:tensorflow.GraphDef.versions) + + ::tensorflow::VersionDef* temp = versions_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + versions_ = NULL; + return temp; +} +inline ::tensorflow::VersionDef* GraphDef::unsafe_arena_release_versions() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.GraphDef.versions) + + ::tensorflow::VersionDef* temp = versions_; + versions_ = NULL; + return temp; +} +inline ::tensorflow::VersionDef* GraphDef::mutable_versions() { + + if (versions_ == NULL) { + auto* p = CreateMaybeMessage<::tensorflow::VersionDef>(GetArenaNoVirtual()); + versions_ = p; + } + // @@protoc_insertion_point(field_mutable:tensorflow.GraphDef.versions) + return versions_; +} +inline void GraphDef::set_allocated_versions(::tensorflow::VersionDef* versions) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(versions_); + } + if (versions) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(versions)->GetArena(); + if (message_arena != submessage_arena) { + versions = ::google::protobuf::internal::GetOwnedMessage( + message_arena, versions, submessage_arena); + } + + } else { + + } + versions_ = versions; + // @@protoc_insertion_point(field_set_allocated:tensorflow.GraphDef.versions) +} + +// int32 version = 3 [deprecated = true]; +inline void GraphDef::clear_version() { + version_ = 0; +} +inline ::google::protobuf::int32 GraphDef::version() const { + // @@protoc_insertion_point(field_get:tensorflow.GraphDef.version) + return version_; +} +inline void GraphDef::set_version(::google::protobuf::int32 value) { + + version_ = value; + // @@protoc_insertion_point(field_set:tensorflow.GraphDef.version) +} + +// .tensorflow.FunctionDefLibrary library = 2; +inline bool GraphDef::has_library() const { + return this != internal_default_instance() && library_ != NULL; +} +inline const ::tensorflow::FunctionDefLibrary& GraphDef::_internal_library() const { + return *library_; +} +inline const ::tensorflow::FunctionDefLibrary& GraphDef::library() const { + const ::tensorflow::FunctionDefLibrary* p = library_; + // @@protoc_insertion_point(field_get:tensorflow.GraphDef.library) + return p != NULL ? *p : *reinterpret_cast( + &::tensorflow::_FunctionDefLibrary_default_instance_); +} +inline ::tensorflow::FunctionDefLibrary* GraphDef::release_library() { + // @@protoc_insertion_point(field_release:tensorflow.GraphDef.library) + + ::tensorflow::FunctionDefLibrary* temp = library_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + library_ = NULL; + return temp; +} +inline ::tensorflow::FunctionDefLibrary* GraphDef::unsafe_arena_release_library() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.GraphDef.library) + + ::tensorflow::FunctionDefLibrary* temp = library_; + library_ = NULL; + return temp; +} +inline ::tensorflow::FunctionDefLibrary* GraphDef::mutable_library() { + + if (library_ == NULL) { + auto* p = CreateMaybeMessage<::tensorflow::FunctionDefLibrary>(GetArenaNoVirtual()); + library_ = p; + } + // @@protoc_insertion_point(field_mutable:tensorflow.GraphDef.library) + return library_; +} +inline void GraphDef::set_allocated_library(::tensorflow::FunctionDefLibrary* library) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(library_); + } + if (library) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(library)->GetArena(); + if (message_arena != submessage_arena) { + library = ::google::protobuf::internal::GetOwnedMessage( + message_arena, library, submessage_arena); + } + + } else { + + } + library_ = library; + // @@protoc_insertion_point(field_set_allocated:tensorflow.GraphDef.library) +} + +#ifdef __GNUC__ + #pragma GCC diagnostic pop +#endif // __GNUC__ + +// @@protoc_insertion_point(namespace_scope) + +} // namespace tensorflow + +// @@protoc_insertion_point(global_scope) + +#endif // PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2fgraph_2eproto diff --git a/linux/include/tensorflow/core/framework/graph.pb_text-impl.h b/linux/include/tensorflow/core/framework/graph.pb_text-impl.h new file mode 100644 index 000000000..acab99edd --- /dev/null +++ b/linux/include/tensorflow/core/framework/graph.pb_text-impl.h @@ -0,0 +1,43 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_framework_graph_proto_IMPL_H_ +#define tensorflow_core_framework_graph_proto_IMPL_H_ + +#include "tensorflow/core/framework/attr_value.pb.h" +#include "tensorflow/core/framework/attr_value.pb_text-impl.h" +#include "tensorflow/core/framework/function.pb.h" +#include "tensorflow/core/framework/function.pb_text-impl.h" +#include "tensorflow/core/framework/graph.pb.h" +#include "tensorflow/core/framework/graph.pb_text.h" +#include "tensorflow/core/framework/node_def.pb.h" +#include "tensorflow/core/framework/node_def.pb_text-impl.h" +#include "tensorflow/core/framework/op_def.pb.h" +#include "tensorflow/core/framework/op_def.pb_text-impl.h" +#include "tensorflow/core/framework/resource_handle.pb.h" +#include "tensorflow/core/framework/resource_handle.pb_text-impl.h" +#include "tensorflow/core/framework/tensor.pb.h" +#include "tensorflow/core/framework/tensor.pb_text-impl.h" +#include "tensorflow/core/framework/tensor_shape.pb.h" +#include "tensorflow/core/framework/tensor_shape.pb_text-impl.h" +#include "tensorflow/core/framework/types.pb.h" +#include "tensorflow/core/framework/types.pb_text-impl.h" +#include "tensorflow/core/framework/versions.pb.h" +#include "tensorflow/core/framework/versions.pb_text-impl.h" +#include "tensorflow/core/lib/strings/proto_text_util.h" +#include "tensorflow/core/lib/strings/scanner.h" + +namespace tensorflow { + +namespace internal { + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::GraphDef& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::GraphDef* msg); + +} // namespace internal + +} // namespace tensorflow + +#endif // tensorflow_core_framework_graph_proto_IMPL_H_ diff --git a/linux/include/tensorflow/core/framework/graph.pb_text.h b/linux/include/tensorflow/core/framework/graph.pb_text.h new file mode 100644 index 000000000..decf17f0a --- /dev/null +++ b/linux/include/tensorflow/core/framework/graph.pb_text.h @@ -0,0 +1,24 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_framework_graph_proto_H_ +#define tensorflow_core_framework_graph_proto_H_ + +#include "tensorflow/core/framework/graph.pb.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/protobuf.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +// Message-text conversion for tensorflow.GraphDef +string ProtoDebugString( + const ::tensorflow::GraphDef& msg); +string ProtoShortDebugString( + const ::tensorflow::GraphDef& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::GraphDef* msg) + TF_MUST_USE_RESULT; + +} // namespace tensorflow + +#endif // tensorflow_core_framework_graph_proto_H_ diff --git a/linux/include/tensorflow/core/framework/graph_def_util.h b/linux/include/tensorflow/core/framework/graph_def_util.h new file mode 100644 index 000000000..2f8d5e8f5 --- /dev/null +++ b/linux/include/tensorflow/core/framework/graph_def_util.h @@ -0,0 +1,121 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_FRAMEWORK_GRAPH_DEF_UTIL_H_ +#define TENSORFLOW_CORE_FRAMEWORK_GRAPH_DEF_UTIL_H_ + +#include +#include "tensorflow/core/framework/op.h" +#include "tensorflow/core/lib/core/status.h" + +namespace tensorflow { + +// Forward declare proto so that it's symbols can be removed from .so exports +class GraphDef; + +// Produce a human-readable version of a GraphDef that is more concise +// than a text-format proto. +string SummarizeGraphDef(const GraphDef& graph_def); + +// Validates the syntax of a GraphDef provided externally. +// +// The following is an EBNF-style syntax for GraphDef objects. Note that +// Node objects are actually specified as tensorflow::NodeDef protocol buffers, +// which contain many other fields that are not (currently) validated. +// +// Graph = Node * +// Node = NodeName, Inputs +// Inputs = ( DataInput * ), ( ControlInput * ) +// DataInput = NodeName, ( ":", [1-9], [0-9] * ) ? +// ControlInput = "^", NodeName +// NodeName = [A-Za-z0-9.], [A-Za-z0-9_./] * +Status ValidateExternalGraphDefSyntax(const GraphDef& graph_def); + +// Adds default attributes to NodeDefs in 'graph_def' starting +// from the 'node_offset' node in 'graph_def'. +// +// Default attributes are defined by 'op_registry'. +// +// Returns OK on success, an error if 'graph_def' has a NodeDef +// that cannot be found in 'op_registry'. +// +// REQUIRES: 'graph_def' and 'op_registry' are not nullptr. +Status AddDefaultAttrsToGraphDef(GraphDef* graph_def, + const OpRegistryInterface& op_registry, + int node_offset); + +// Same as above, except for the fact that it skips nodes that aren't found in +// op_registry if skip_unknown_ops is true. +Status AddDefaultAttrsToGraphDef(GraphDef* graph_def, + const OpRegistryInterface& op_registry, + int node_offset, bool skip_unknown_ops); + +// Remove attrs from 'graph_def' that have the default value according +// to 'producer_op_registry', but don't exist according to +// 'consumer_op_registry'. This can allow 'graph_def' to run on the +// consumer even if consumer was built at an earlier CL (before an +// attr with a default was added). Note that this will not affect +// attrs with non-default values, so you must run a +// ValidateGraphDef...() function to see if the result is in fact +// compatible. If not nullptr, the op/attr pairs that were removed +// are added to '*op_attr_removed'. +// +// Expected usage, for a producer that wants to prepare a graph for +// a consumer: +// // For each consumer, update 'graph_def': +// OpListOpRegistry consumer_op_registry(consumer_server_op_list); +// std::unordered_set> op_attr_removed; +// TF_RETURN_IF_ERROR(RemoveNewDefaultAttrsFromGraphDef( +// &graph_def, consumer_op_registry, *OpRegistry::Global(), +// &op_attr_removed)); +// // Validate that each consumer can understand the resulting 'graph_def' +// TF_RETURN_IF_ERROR(graph::ValidateGraphDefAgainstOpRegistry( +// graph_def, consumer_op_registry)); +// // Consumer can use 'graph_def', and 'op_attr_removed' summarizes +// // what changes had to be made to 'graph_def' for it to work. +// +// Expected usage, for a consumer that has a graph and a +// (optionally-stripped) op_list from a producer (say from a call to +// StrippedOpListForGraph(), or in the MetaGraphDef): +// OpListOpRegistry producer_op_registry(producer_stripped_op_list); +// TF_RETURN_IF_ERROR(RemoveNewDefaultAttrsFromGraphDef( +// &graph_def, *OpRegistry::Global(), producer_op_registry, nullptr)); +Status RemoveNewDefaultAttrsFromGraphDef( + GraphDef* graph_def, const OpRegistryInterface& consumer_op_registry, + const OpRegistryInterface& producer_op_registry, + std::set>* op_attr_removed); + +// Two functions that collect the ops used by a graph. +// +// This returns the ops used as a set of strings. +void OpsUsedByGraph(const GraphDef& graph_def, + std::set* ops_used_in_graph); + +// This function computes the stripped_op_list field of MetaGraphDef +// and similar protos. The op_registry should contain the ops used to +// produce graph_def. The resulting stripped_op_list can be +// communicated from the producer to the consumer, which can use +// RemoveNewDefaultAttrsFromGraphDef() to improve forwards compatibility +// (using an OpListOpRegistry as indicated in the example above). +// +// Most users will pass *OpRegistry::Global() for op_registry to strip against +// the list of ops registered in this process. +Status StrippedOpListForGraph(const GraphDef& graph_def, + const OpRegistryInterface& op_registry, + OpList* stripped_op_list); + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_FRAMEWORK_GRAPH_DEF_UTIL_H_ diff --git a/linux/include/tensorflow/core/framework/graph_to_functiondef.h b/linux/include/tensorflow/core/framework/graph_to_functiondef.h new file mode 100644 index 000000000..cb0e2b2fb --- /dev/null +++ b/linux/include/tensorflow/core/framework/graph_to_functiondef.h @@ -0,0 +1,32 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_FRAMEWORK_GRAPH_TO_FUNCTIONDEF_H_ +#define TENSORFLOW_CORE_FRAMEWORK_GRAPH_TO_FUNCTIONDEF_H_ + +#include "tensorflow/core/framework/function.h" +#include "tensorflow/core/graph/graph.h" +#include "tensorflow/core/lib/core/status.h" + +namespace tensorflow { + +// Converts 'graph' to a FunctionDef 'fdef', with name 'name'. +// Closely modeled on the Python code in tensorflow/python/framework/function.py +Status GraphToFunctionDef(const Graph& graph, const string& name, + FunctionDef* fdef); + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_FRAMEWORK_GRAPH_TO_FUNCTIONDEF_H_ diff --git a/linux/include/tensorflow/core/framework/graph_transfer_info.pb.h b/linux/include/tensorflow/core/framework/graph_transfer_info.pb.h new file mode 100644 index 000000000..35269bfec --- /dev/null +++ b/linux/include/tensorflow/core/framework/graph_transfer_info.pb.h @@ -0,0 +1,2465 @@ +// Generated by the protocol buffer compiler. DO NOT EDIT! +// source: tensorflow/core/framework/graph_transfer_info.proto + +#ifndef PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2fgraph_5ftransfer_5finfo_2eproto +#define PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2fgraph_5ftransfer_5finfo_2eproto + +#include + +#include + +#if GOOGLE_PROTOBUF_VERSION < 3006000 +#error This file was generated by a newer version of protoc which is +#error incompatible with your Protocol Buffer headers. Please update +#error your headers. +#endif +#if 3006000 < GOOGLE_PROTOBUF_MIN_PROTOC_VERSION +#error This file was generated by an older version of protoc which is +#error incompatible with your Protocol Buffer headers. Please +#error regenerate this file with a newer version of protoc. +#endif + +#include +#include +#include +#include +#include +#include +#include +#include +#include // IWYU pragma: export +#include // IWYU pragma: export +#include +#include +#include "tensorflow/core/framework/types.pb.h" +// @@protoc_insertion_point(includes) +#define PROTOBUF_INTERNAL_EXPORT_protobuf_tensorflow_2fcore_2fframework_2fgraph_5ftransfer_5finfo_2eproto + +namespace protobuf_tensorflow_2fcore_2fframework_2fgraph_5ftransfer_5finfo_2eproto { +// Internal implementation detail -- do not use these members. +struct TableStruct { + static const ::google::protobuf::internal::ParseTableField entries[]; + static const ::google::protobuf::internal::AuxillaryParseTableField aux[]; + static const ::google::protobuf::internal::ParseTable schema[8]; + static const ::google::protobuf::internal::FieldMetadata field_metadata[]; + static const ::google::protobuf::internal::SerializationTable serialization_table[]; + static const ::google::protobuf::uint32 offsets[]; +}; +void AddDescriptors(); +} // namespace protobuf_tensorflow_2fcore_2fframework_2fgraph_5ftransfer_5finfo_2eproto +namespace tensorflow { +class GraphTransferConstNodeInfo; +class GraphTransferConstNodeInfoDefaultTypeInternal; +extern GraphTransferConstNodeInfoDefaultTypeInternal _GraphTransferConstNodeInfo_default_instance_; +class GraphTransferGraphInputNodeInfo; +class GraphTransferGraphInputNodeInfoDefaultTypeInternal; +extern GraphTransferGraphInputNodeInfoDefaultTypeInternal _GraphTransferGraphInputNodeInfo_default_instance_; +class GraphTransferGraphOutputNodeInfo; +class GraphTransferGraphOutputNodeInfoDefaultTypeInternal; +extern GraphTransferGraphOutputNodeInfoDefaultTypeInternal _GraphTransferGraphOutputNodeInfo_default_instance_; +class GraphTransferInfo; +class GraphTransferInfoDefaultTypeInternal; +extern GraphTransferInfoDefaultTypeInternal _GraphTransferInfo_default_instance_; +class GraphTransferNodeInfo; +class GraphTransferNodeInfoDefaultTypeInternal; +extern GraphTransferNodeInfoDefaultTypeInternal _GraphTransferNodeInfo_default_instance_; +class GraphTransferNodeInput; +class GraphTransferNodeInputDefaultTypeInternal; +extern GraphTransferNodeInputDefaultTypeInternal _GraphTransferNodeInput_default_instance_; +class GraphTransferNodeInputInfo; +class GraphTransferNodeInputInfoDefaultTypeInternal; +extern GraphTransferNodeInputInfoDefaultTypeInternal _GraphTransferNodeInputInfo_default_instance_; +class GraphTransferNodeOutputInfo; +class GraphTransferNodeOutputInfoDefaultTypeInternal; +extern GraphTransferNodeOutputInfoDefaultTypeInternal _GraphTransferNodeOutputInfo_default_instance_; +} // namespace tensorflow +namespace google { +namespace protobuf { +template<> ::tensorflow::GraphTransferConstNodeInfo* Arena::CreateMaybeMessage<::tensorflow::GraphTransferConstNodeInfo>(Arena*); +template<> ::tensorflow::GraphTransferGraphInputNodeInfo* Arena::CreateMaybeMessage<::tensorflow::GraphTransferGraphInputNodeInfo>(Arena*); +template<> ::tensorflow::GraphTransferGraphOutputNodeInfo* Arena::CreateMaybeMessage<::tensorflow::GraphTransferGraphOutputNodeInfo>(Arena*); +template<> ::tensorflow::GraphTransferInfo* Arena::CreateMaybeMessage<::tensorflow::GraphTransferInfo>(Arena*); +template<> ::tensorflow::GraphTransferNodeInfo* Arena::CreateMaybeMessage<::tensorflow::GraphTransferNodeInfo>(Arena*); +template<> ::tensorflow::GraphTransferNodeInput* Arena::CreateMaybeMessage<::tensorflow::GraphTransferNodeInput>(Arena*); +template<> ::tensorflow::GraphTransferNodeInputInfo* Arena::CreateMaybeMessage<::tensorflow::GraphTransferNodeInputInfo>(Arena*); +template<> ::tensorflow::GraphTransferNodeOutputInfo* Arena::CreateMaybeMessage<::tensorflow::GraphTransferNodeOutputInfo>(Arena*); +} // namespace protobuf +} // namespace google +namespace tensorflow { + +enum GraphTransferInfo_Destination { + GraphTransferInfo_Destination_NOP = 0, + GraphTransferInfo_Destination_HEXAGON = 1, + GraphTransferInfo_Destination_GraphTransferInfo_Destination_INT_MIN_SENTINEL_DO_NOT_USE_ = ::google::protobuf::kint32min, + GraphTransferInfo_Destination_GraphTransferInfo_Destination_INT_MAX_SENTINEL_DO_NOT_USE_ = ::google::protobuf::kint32max +}; +bool GraphTransferInfo_Destination_IsValid(int value); +const GraphTransferInfo_Destination GraphTransferInfo_Destination_Destination_MIN = GraphTransferInfo_Destination_NOP; +const GraphTransferInfo_Destination GraphTransferInfo_Destination_Destination_MAX = GraphTransferInfo_Destination_HEXAGON; +const int GraphTransferInfo_Destination_Destination_ARRAYSIZE = GraphTransferInfo_Destination_Destination_MAX + 1; + +const ::google::protobuf::EnumDescriptor* GraphTransferInfo_Destination_descriptor(); +inline const ::std::string& GraphTransferInfo_Destination_Name(GraphTransferInfo_Destination value) { + return ::google::protobuf::internal::NameOfEnum( + GraphTransferInfo_Destination_descriptor(), value); +} +inline bool GraphTransferInfo_Destination_Parse( + const ::std::string& name, GraphTransferInfo_Destination* value) { + return ::google::protobuf::internal::ParseNamedEnum( + GraphTransferInfo_Destination_descriptor(), name, value); +} +// =================================================================== + +class GraphTransferNodeInput : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.GraphTransferNodeInput) */ { + public: + GraphTransferNodeInput(); + virtual ~GraphTransferNodeInput(); + + GraphTransferNodeInput(const GraphTransferNodeInput& from); + + inline GraphTransferNodeInput& operator=(const GraphTransferNodeInput& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + GraphTransferNodeInput(GraphTransferNodeInput&& from) noexcept + : GraphTransferNodeInput() { + *this = ::std::move(from); + } + + inline GraphTransferNodeInput& operator=(GraphTransferNodeInput&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const GraphTransferNodeInput& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const GraphTransferNodeInput* internal_default_instance() { + return reinterpret_cast( + &_GraphTransferNodeInput_default_instance_); + } + static constexpr int kIndexInFileMessages = + 0; + + void UnsafeArenaSwap(GraphTransferNodeInput* other); + void Swap(GraphTransferNodeInput* other); + friend void swap(GraphTransferNodeInput& a, GraphTransferNodeInput& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline GraphTransferNodeInput* New() const final { + return CreateMaybeMessage(NULL); + } + + GraphTransferNodeInput* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const GraphTransferNodeInput& from); + void MergeFrom(const GraphTransferNodeInput& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(GraphTransferNodeInput* other); + protected: + explicit GraphTransferNodeInput(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // int32 node_id = 1; + void clear_node_id(); + static const int kNodeIdFieldNumber = 1; + ::google::protobuf::int32 node_id() const; + void set_node_id(::google::protobuf::int32 value); + + // int32 output_port = 2; + void clear_output_port(); + static const int kOutputPortFieldNumber = 2; + ::google::protobuf::int32 output_port() const; + void set_output_port(::google::protobuf::int32 value); + + // @@protoc_insertion_point(class_scope:tensorflow.GraphTransferNodeInput) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::int32 node_id_; + ::google::protobuf::int32 output_port_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fgraph_5ftransfer_5finfo_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class GraphTransferNodeInfo : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.GraphTransferNodeInfo) */ { + public: + GraphTransferNodeInfo(); + virtual ~GraphTransferNodeInfo(); + + GraphTransferNodeInfo(const GraphTransferNodeInfo& from); + + inline GraphTransferNodeInfo& operator=(const GraphTransferNodeInfo& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + GraphTransferNodeInfo(GraphTransferNodeInfo&& from) noexcept + : GraphTransferNodeInfo() { + *this = ::std::move(from); + } + + inline GraphTransferNodeInfo& operator=(GraphTransferNodeInfo&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const GraphTransferNodeInfo& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const GraphTransferNodeInfo* internal_default_instance() { + return reinterpret_cast( + &_GraphTransferNodeInfo_default_instance_); + } + static constexpr int kIndexInFileMessages = + 1; + + void UnsafeArenaSwap(GraphTransferNodeInfo* other); + void Swap(GraphTransferNodeInfo* other); + friend void swap(GraphTransferNodeInfo& a, GraphTransferNodeInfo& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline GraphTransferNodeInfo* New() const final { + return CreateMaybeMessage(NULL); + } + + GraphTransferNodeInfo* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const GraphTransferNodeInfo& from); + void MergeFrom(const GraphTransferNodeInfo& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(GraphTransferNodeInfo* other); + protected: + explicit GraphTransferNodeInfo(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // string name = 1; + void clear_name(); + static const int kNameFieldNumber = 1; + const ::std::string& name() const; + void set_name(const ::std::string& value); + #if LANG_CXX11 + void set_name(::std::string&& value); + #endif + void set_name(const char* value); + void set_name(const char* value, size_t size); + ::std::string* mutable_name(); + ::std::string* release_name(); + void set_allocated_name(::std::string* name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_name( + ::std::string* name); + + // string type_name = 3; + void clear_type_name(); + static const int kTypeNameFieldNumber = 3; + const ::std::string& type_name() const; + void set_type_name(const ::std::string& value); + #if LANG_CXX11 + void set_type_name(::std::string&& value); + #endif + void set_type_name(const char* value); + void set_type_name(const char* value, size_t size); + ::std::string* mutable_type_name(); + ::std::string* release_type_name(); + void set_allocated_type_name(::std::string* type_name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_type_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_type_name( + ::std::string* type_name); + + // int32 node_id = 2; + void clear_node_id(); + static const int kNodeIdFieldNumber = 2; + ::google::protobuf::int32 node_id() const; + void set_node_id(::google::protobuf::int32 value); + + // int32 soc_op_id = 4; + void clear_soc_op_id(); + static const int kSocOpIdFieldNumber = 4; + ::google::protobuf::int32 soc_op_id() const; + void set_soc_op_id(::google::protobuf::int32 value); + + // int32 padding_id = 5; + void clear_padding_id(); + static const int kPaddingIdFieldNumber = 5; + ::google::protobuf::int32 padding_id() const; + void set_padding_id(::google::protobuf::int32 value); + + // int32 input_count = 6; + void clear_input_count(); + static const int kInputCountFieldNumber = 6; + ::google::protobuf::int32 input_count() const; + void set_input_count(::google::protobuf::int32 value); + + // int32 output_count = 7; + void clear_output_count(); + static const int kOutputCountFieldNumber = 7; + ::google::protobuf::int32 output_count() const; + void set_output_count(::google::protobuf::int32 value); + + // @@protoc_insertion_point(class_scope:tensorflow.GraphTransferNodeInfo) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::internal::ArenaStringPtr name_; + ::google::protobuf::internal::ArenaStringPtr type_name_; + ::google::protobuf::int32 node_id_; + ::google::protobuf::int32 soc_op_id_; + ::google::protobuf::int32 padding_id_; + ::google::protobuf::int32 input_count_; + ::google::protobuf::int32 output_count_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fgraph_5ftransfer_5finfo_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class GraphTransferConstNodeInfo : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.GraphTransferConstNodeInfo) */ { + public: + GraphTransferConstNodeInfo(); + virtual ~GraphTransferConstNodeInfo(); + + GraphTransferConstNodeInfo(const GraphTransferConstNodeInfo& from); + + inline GraphTransferConstNodeInfo& operator=(const GraphTransferConstNodeInfo& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + GraphTransferConstNodeInfo(GraphTransferConstNodeInfo&& from) noexcept + : GraphTransferConstNodeInfo() { + *this = ::std::move(from); + } + + inline GraphTransferConstNodeInfo& operator=(GraphTransferConstNodeInfo&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const GraphTransferConstNodeInfo& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const GraphTransferConstNodeInfo* internal_default_instance() { + return reinterpret_cast( + &_GraphTransferConstNodeInfo_default_instance_); + } + static constexpr int kIndexInFileMessages = + 2; + + void UnsafeArenaSwap(GraphTransferConstNodeInfo* other); + void Swap(GraphTransferConstNodeInfo* other); + friend void swap(GraphTransferConstNodeInfo& a, GraphTransferConstNodeInfo& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline GraphTransferConstNodeInfo* New() const final { + return CreateMaybeMessage(NULL); + } + + GraphTransferConstNodeInfo* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const GraphTransferConstNodeInfo& from); + void MergeFrom(const GraphTransferConstNodeInfo& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(GraphTransferConstNodeInfo* other); + protected: + explicit GraphTransferConstNodeInfo(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated int64 shape = 3; + int shape_size() const; + void clear_shape(); + static const int kShapeFieldNumber = 3; + ::google::protobuf::int64 shape(int index) const; + void set_shape(int index, ::google::protobuf::int64 value); + void add_shape(::google::protobuf::int64 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& + shape() const; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* + mutable_shape(); + + // string name = 1; + void clear_name(); + static const int kNameFieldNumber = 1; + const ::std::string& name() const; + void set_name(const ::std::string& value); + #if LANG_CXX11 + void set_name(::std::string&& value); + #endif + void set_name(const char* value); + void set_name(const char* value, size_t size); + ::std::string* mutable_name(); + ::std::string* release_name(); + void set_allocated_name(::std::string* name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_name( + ::std::string* name); + + // bytes data = 4; + void clear_data(); + static const int kDataFieldNumber = 4; + const ::std::string& data() const; + void set_data(const ::std::string& value); + #if LANG_CXX11 + void set_data(::std::string&& value); + #endif + void set_data(const char* value); + void set_data(const void* value, size_t size); + ::std::string* mutable_data(); + ::std::string* release_data(); + void set_allocated_data(::std::string* data); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_data(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_data( + ::std::string* data); + + // int32 node_id = 2; + void clear_node_id(); + static const int kNodeIdFieldNumber = 2; + ::google::protobuf::int32 node_id() const; + void set_node_id(::google::protobuf::int32 value); + + // .tensorflow.DataType dtype = 5; + void clear_dtype(); + static const int kDtypeFieldNumber = 5; + ::tensorflow::DataType dtype() const; + void set_dtype(::tensorflow::DataType value); + + // @@protoc_insertion_point(class_scope:tensorflow.GraphTransferConstNodeInfo) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 > shape_; + mutable int _shape_cached_byte_size_; + ::google::protobuf::internal::ArenaStringPtr name_; + ::google::protobuf::internal::ArenaStringPtr data_; + ::google::protobuf::int32 node_id_; + int dtype_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fgraph_5ftransfer_5finfo_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class GraphTransferNodeInputInfo : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.GraphTransferNodeInputInfo) */ { + public: + GraphTransferNodeInputInfo(); + virtual ~GraphTransferNodeInputInfo(); + + GraphTransferNodeInputInfo(const GraphTransferNodeInputInfo& from); + + inline GraphTransferNodeInputInfo& operator=(const GraphTransferNodeInputInfo& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + GraphTransferNodeInputInfo(GraphTransferNodeInputInfo&& from) noexcept + : GraphTransferNodeInputInfo() { + *this = ::std::move(from); + } + + inline GraphTransferNodeInputInfo& operator=(GraphTransferNodeInputInfo&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const GraphTransferNodeInputInfo& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const GraphTransferNodeInputInfo* internal_default_instance() { + return reinterpret_cast( + &_GraphTransferNodeInputInfo_default_instance_); + } + static constexpr int kIndexInFileMessages = + 3; + + void UnsafeArenaSwap(GraphTransferNodeInputInfo* other); + void Swap(GraphTransferNodeInputInfo* other); + friend void swap(GraphTransferNodeInputInfo& a, GraphTransferNodeInputInfo& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline GraphTransferNodeInputInfo* New() const final { + return CreateMaybeMessage(NULL); + } + + GraphTransferNodeInputInfo* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const GraphTransferNodeInputInfo& from); + void MergeFrom(const GraphTransferNodeInputInfo& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(GraphTransferNodeInputInfo* other); + protected: + explicit GraphTransferNodeInputInfo(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated .tensorflow.GraphTransferNodeInput node_input = 2; + int node_input_size() const; + void clear_node_input(); + static const int kNodeInputFieldNumber = 2; + ::tensorflow::GraphTransferNodeInput* mutable_node_input(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::GraphTransferNodeInput >* + mutable_node_input(); + const ::tensorflow::GraphTransferNodeInput& node_input(int index) const; + ::tensorflow::GraphTransferNodeInput* add_node_input(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::GraphTransferNodeInput >& + node_input() const; + + // int32 node_id = 1; + void clear_node_id(); + static const int kNodeIdFieldNumber = 1; + ::google::protobuf::int32 node_id() const; + void set_node_id(::google::protobuf::int32 value); + + // @@protoc_insertion_point(class_scope:tensorflow.GraphTransferNodeInputInfo) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::GraphTransferNodeInput > node_input_; + ::google::protobuf::int32 node_id_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fgraph_5ftransfer_5finfo_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class GraphTransferNodeOutputInfo : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.GraphTransferNodeOutputInfo) */ { + public: + GraphTransferNodeOutputInfo(); + virtual ~GraphTransferNodeOutputInfo(); + + GraphTransferNodeOutputInfo(const GraphTransferNodeOutputInfo& from); + + inline GraphTransferNodeOutputInfo& operator=(const GraphTransferNodeOutputInfo& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + GraphTransferNodeOutputInfo(GraphTransferNodeOutputInfo&& from) noexcept + : GraphTransferNodeOutputInfo() { + *this = ::std::move(from); + } + + inline GraphTransferNodeOutputInfo& operator=(GraphTransferNodeOutputInfo&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const GraphTransferNodeOutputInfo& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const GraphTransferNodeOutputInfo* internal_default_instance() { + return reinterpret_cast( + &_GraphTransferNodeOutputInfo_default_instance_); + } + static constexpr int kIndexInFileMessages = + 4; + + void UnsafeArenaSwap(GraphTransferNodeOutputInfo* other); + void Swap(GraphTransferNodeOutputInfo* other); + friend void swap(GraphTransferNodeOutputInfo& a, GraphTransferNodeOutputInfo& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline GraphTransferNodeOutputInfo* New() const final { + return CreateMaybeMessage(NULL); + } + + GraphTransferNodeOutputInfo* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const GraphTransferNodeOutputInfo& from); + void MergeFrom(const GraphTransferNodeOutputInfo& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(GraphTransferNodeOutputInfo* other); + protected: + explicit GraphTransferNodeOutputInfo(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated int32 max_byte_size = 2; + int max_byte_size_size() const; + void clear_max_byte_size(); + static const int kMaxByteSizeFieldNumber = 2; + ::google::protobuf::int32 max_byte_size(int index) const; + void set_max_byte_size(int index, ::google::protobuf::int32 value); + void add_max_byte_size(::google::protobuf::int32 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::int32 >& + max_byte_size() const; + ::google::protobuf::RepeatedField< ::google::protobuf::int32 >* + mutable_max_byte_size(); + + // int32 node_id = 1; + void clear_node_id(); + static const int kNodeIdFieldNumber = 1; + ::google::protobuf::int32 node_id() const; + void set_node_id(::google::protobuf::int32 value); + + // @@protoc_insertion_point(class_scope:tensorflow.GraphTransferNodeOutputInfo) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedField< ::google::protobuf::int32 > max_byte_size_; + mutable int _max_byte_size_cached_byte_size_; + ::google::protobuf::int32 node_id_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fgraph_5ftransfer_5finfo_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class GraphTransferGraphInputNodeInfo : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.GraphTransferGraphInputNodeInfo) */ { + public: + GraphTransferGraphInputNodeInfo(); + virtual ~GraphTransferGraphInputNodeInfo(); + + GraphTransferGraphInputNodeInfo(const GraphTransferGraphInputNodeInfo& from); + + inline GraphTransferGraphInputNodeInfo& operator=(const GraphTransferGraphInputNodeInfo& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + GraphTransferGraphInputNodeInfo(GraphTransferGraphInputNodeInfo&& from) noexcept + : GraphTransferGraphInputNodeInfo() { + *this = ::std::move(from); + } + + inline GraphTransferGraphInputNodeInfo& operator=(GraphTransferGraphInputNodeInfo&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const GraphTransferGraphInputNodeInfo& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const GraphTransferGraphInputNodeInfo* internal_default_instance() { + return reinterpret_cast( + &_GraphTransferGraphInputNodeInfo_default_instance_); + } + static constexpr int kIndexInFileMessages = + 5; + + void UnsafeArenaSwap(GraphTransferGraphInputNodeInfo* other); + void Swap(GraphTransferGraphInputNodeInfo* other); + friend void swap(GraphTransferGraphInputNodeInfo& a, GraphTransferGraphInputNodeInfo& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline GraphTransferGraphInputNodeInfo* New() const final { + return CreateMaybeMessage(NULL); + } + + GraphTransferGraphInputNodeInfo* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const GraphTransferGraphInputNodeInfo& from); + void MergeFrom(const GraphTransferGraphInputNodeInfo& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(GraphTransferGraphInputNodeInfo* other); + protected: + explicit GraphTransferGraphInputNodeInfo(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated int64 shape = 2; + int shape_size() const; + void clear_shape(); + static const int kShapeFieldNumber = 2; + ::google::protobuf::int64 shape(int index) const; + void set_shape(int index, ::google::protobuf::int64 value); + void add_shape(::google::protobuf::int64 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& + shape() const; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* + mutable_shape(); + + // string name = 1; + void clear_name(); + static const int kNameFieldNumber = 1; + const ::std::string& name() const; + void set_name(const ::std::string& value); + #if LANG_CXX11 + void set_name(::std::string&& value); + #endif + void set_name(const char* value); + void set_name(const char* value, size_t size); + ::std::string* mutable_name(); + ::std::string* release_name(); + void set_allocated_name(::std::string* name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_name( + ::std::string* name); + + // .tensorflow.DataType dtype = 3; + void clear_dtype(); + static const int kDtypeFieldNumber = 3; + ::tensorflow::DataType dtype() const; + void set_dtype(::tensorflow::DataType value); + + // @@protoc_insertion_point(class_scope:tensorflow.GraphTransferGraphInputNodeInfo) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 > shape_; + mutable int _shape_cached_byte_size_; + ::google::protobuf::internal::ArenaStringPtr name_; + int dtype_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fgraph_5ftransfer_5finfo_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class GraphTransferGraphOutputNodeInfo : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.GraphTransferGraphOutputNodeInfo) */ { + public: + GraphTransferGraphOutputNodeInfo(); + virtual ~GraphTransferGraphOutputNodeInfo(); + + GraphTransferGraphOutputNodeInfo(const GraphTransferGraphOutputNodeInfo& from); + + inline GraphTransferGraphOutputNodeInfo& operator=(const GraphTransferGraphOutputNodeInfo& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + GraphTransferGraphOutputNodeInfo(GraphTransferGraphOutputNodeInfo&& from) noexcept + : GraphTransferGraphOutputNodeInfo() { + *this = ::std::move(from); + } + + inline GraphTransferGraphOutputNodeInfo& operator=(GraphTransferGraphOutputNodeInfo&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const GraphTransferGraphOutputNodeInfo& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const GraphTransferGraphOutputNodeInfo* internal_default_instance() { + return reinterpret_cast( + &_GraphTransferGraphOutputNodeInfo_default_instance_); + } + static constexpr int kIndexInFileMessages = + 6; + + void UnsafeArenaSwap(GraphTransferGraphOutputNodeInfo* other); + void Swap(GraphTransferGraphOutputNodeInfo* other); + friend void swap(GraphTransferGraphOutputNodeInfo& a, GraphTransferGraphOutputNodeInfo& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline GraphTransferGraphOutputNodeInfo* New() const final { + return CreateMaybeMessage(NULL); + } + + GraphTransferGraphOutputNodeInfo* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const GraphTransferGraphOutputNodeInfo& from); + void MergeFrom(const GraphTransferGraphOutputNodeInfo& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(GraphTransferGraphOutputNodeInfo* other); + protected: + explicit GraphTransferGraphOutputNodeInfo(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated int64 shape = 2; + int shape_size() const; + void clear_shape(); + static const int kShapeFieldNumber = 2; + ::google::protobuf::int64 shape(int index) const; + void set_shape(int index, ::google::protobuf::int64 value); + void add_shape(::google::protobuf::int64 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& + shape() const; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* + mutable_shape(); + + // string name = 1; + void clear_name(); + static const int kNameFieldNumber = 1; + const ::std::string& name() const; + void set_name(const ::std::string& value); + #if LANG_CXX11 + void set_name(::std::string&& value); + #endif + void set_name(const char* value); + void set_name(const char* value, size_t size); + ::std::string* mutable_name(); + ::std::string* release_name(); + void set_allocated_name(::std::string* name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_name( + ::std::string* name); + + // .tensorflow.DataType dtype = 3; + void clear_dtype(); + static const int kDtypeFieldNumber = 3; + ::tensorflow::DataType dtype() const; + void set_dtype(::tensorflow::DataType value); + + // @@protoc_insertion_point(class_scope:tensorflow.GraphTransferGraphOutputNodeInfo) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 > shape_; + mutable int _shape_cached_byte_size_; + ::google::protobuf::internal::ArenaStringPtr name_; + int dtype_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fgraph_5ftransfer_5finfo_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class GraphTransferInfo : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.GraphTransferInfo) */ { + public: + GraphTransferInfo(); + virtual ~GraphTransferInfo(); + + GraphTransferInfo(const GraphTransferInfo& from); + + inline GraphTransferInfo& operator=(const GraphTransferInfo& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + GraphTransferInfo(GraphTransferInfo&& from) noexcept + : GraphTransferInfo() { + *this = ::std::move(from); + } + + inline GraphTransferInfo& operator=(GraphTransferInfo&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const GraphTransferInfo& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const GraphTransferInfo* internal_default_instance() { + return reinterpret_cast( + &_GraphTransferInfo_default_instance_); + } + static constexpr int kIndexInFileMessages = + 7; + + void UnsafeArenaSwap(GraphTransferInfo* other); + void Swap(GraphTransferInfo* other); + friend void swap(GraphTransferInfo& a, GraphTransferInfo& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline GraphTransferInfo* New() const final { + return CreateMaybeMessage(NULL); + } + + GraphTransferInfo* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const GraphTransferInfo& from); + void MergeFrom(const GraphTransferInfo& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(GraphTransferInfo* other); + protected: + explicit GraphTransferInfo(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + typedef GraphTransferInfo_Destination Destination; + static const Destination NOP = + GraphTransferInfo_Destination_NOP; + static const Destination HEXAGON = + GraphTransferInfo_Destination_HEXAGON; + static inline bool Destination_IsValid(int value) { + return GraphTransferInfo_Destination_IsValid(value); + } + static const Destination Destination_MIN = + GraphTransferInfo_Destination_Destination_MIN; + static const Destination Destination_MAX = + GraphTransferInfo_Destination_Destination_MAX; + static const int Destination_ARRAYSIZE = + GraphTransferInfo_Destination_Destination_ARRAYSIZE; + static inline const ::google::protobuf::EnumDescriptor* + Destination_descriptor() { + return GraphTransferInfo_Destination_descriptor(); + } + static inline const ::std::string& Destination_Name(Destination value) { + return GraphTransferInfo_Destination_Name(value); + } + static inline bool Destination_Parse(const ::std::string& name, + Destination* value) { + return GraphTransferInfo_Destination_Parse(name, value); + } + + // accessors ------------------------------------------------------- + + // repeated .tensorflow.GraphTransferNodeInfo node_info = 1; + int node_info_size() const; + void clear_node_info(); + static const int kNodeInfoFieldNumber = 1; + ::tensorflow::GraphTransferNodeInfo* mutable_node_info(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::GraphTransferNodeInfo >* + mutable_node_info(); + const ::tensorflow::GraphTransferNodeInfo& node_info(int index) const; + ::tensorflow::GraphTransferNodeInfo* add_node_info(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::GraphTransferNodeInfo >& + node_info() const; + + // repeated .tensorflow.GraphTransferConstNodeInfo const_node_info = 2; + int const_node_info_size() const; + void clear_const_node_info(); + static const int kConstNodeInfoFieldNumber = 2; + ::tensorflow::GraphTransferConstNodeInfo* mutable_const_node_info(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::GraphTransferConstNodeInfo >* + mutable_const_node_info(); + const ::tensorflow::GraphTransferConstNodeInfo& const_node_info(int index) const; + ::tensorflow::GraphTransferConstNodeInfo* add_const_node_info(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::GraphTransferConstNodeInfo >& + const_node_info() const; + + // repeated .tensorflow.GraphTransferNodeInputInfo node_input_info = 3; + int node_input_info_size() const; + void clear_node_input_info(); + static const int kNodeInputInfoFieldNumber = 3; + ::tensorflow::GraphTransferNodeInputInfo* mutable_node_input_info(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::GraphTransferNodeInputInfo >* + mutable_node_input_info(); + const ::tensorflow::GraphTransferNodeInputInfo& node_input_info(int index) const; + ::tensorflow::GraphTransferNodeInputInfo* add_node_input_info(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::GraphTransferNodeInputInfo >& + node_input_info() const; + + // repeated .tensorflow.GraphTransferNodeOutputInfo node_output_info = 4; + int node_output_info_size() const; + void clear_node_output_info(); + static const int kNodeOutputInfoFieldNumber = 4; + ::tensorflow::GraphTransferNodeOutputInfo* mutable_node_output_info(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::GraphTransferNodeOutputInfo >* + mutable_node_output_info(); + const ::tensorflow::GraphTransferNodeOutputInfo& node_output_info(int index) const; + ::tensorflow::GraphTransferNodeOutputInfo* add_node_output_info(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::GraphTransferNodeOutputInfo >& + node_output_info() const; + + // repeated .tensorflow.GraphTransferGraphInputNodeInfo graph_input_node_info = 5; + int graph_input_node_info_size() const; + void clear_graph_input_node_info(); + static const int kGraphInputNodeInfoFieldNumber = 5; + ::tensorflow::GraphTransferGraphInputNodeInfo* mutable_graph_input_node_info(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::GraphTransferGraphInputNodeInfo >* + mutable_graph_input_node_info(); + const ::tensorflow::GraphTransferGraphInputNodeInfo& graph_input_node_info(int index) const; + ::tensorflow::GraphTransferGraphInputNodeInfo* add_graph_input_node_info(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::GraphTransferGraphInputNodeInfo >& + graph_input_node_info() const; + + // repeated .tensorflow.GraphTransferGraphOutputNodeInfo graph_output_node_info = 6; + int graph_output_node_info_size() const; + void clear_graph_output_node_info(); + static const int kGraphOutputNodeInfoFieldNumber = 6; + ::tensorflow::GraphTransferGraphOutputNodeInfo* mutable_graph_output_node_info(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::GraphTransferGraphOutputNodeInfo >* + mutable_graph_output_node_info(); + const ::tensorflow::GraphTransferGraphOutputNodeInfo& graph_output_node_info(int index) const; + ::tensorflow::GraphTransferGraphOutputNodeInfo* add_graph_output_node_info(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::GraphTransferGraphOutputNodeInfo >& + graph_output_node_info() const; + + // .tensorflow.GraphTransferInfo.Destination destination = 7; + void clear_destination(); + static const int kDestinationFieldNumber = 7; + ::tensorflow::GraphTransferInfo_Destination destination() const; + void set_destination(::tensorflow::GraphTransferInfo_Destination value); + + // @@protoc_insertion_point(class_scope:tensorflow.GraphTransferInfo) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::GraphTransferNodeInfo > node_info_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::GraphTransferConstNodeInfo > const_node_info_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::GraphTransferNodeInputInfo > node_input_info_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::GraphTransferNodeOutputInfo > node_output_info_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::GraphTransferGraphInputNodeInfo > graph_input_node_info_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::GraphTransferGraphOutputNodeInfo > graph_output_node_info_; + int destination_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fgraph_5ftransfer_5finfo_2eproto::TableStruct; +}; +// =================================================================== + + +// =================================================================== + +#ifdef __GNUC__ + #pragma GCC diagnostic push + #pragma GCC diagnostic ignored "-Wstrict-aliasing" +#endif // __GNUC__ +// GraphTransferNodeInput + +// int32 node_id = 1; +inline void GraphTransferNodeInput::clear_node_id() { + node_id_ = 0; +} +inline ::google::protobuf::int32 GraphTransferNodeInput::node_id() const { + // @@protoc_insertion_point(field_get:tensorflow.GraphTransferNodeInput.node_id) + return node_id_; +} +inline void GraphTransferNodeInput::set_node_id(::google::protobuf::int32 value) { + + node_id_ = value; + // @@protoc_insertion_point(field_set:tensorflow.GraphTransferNodeInput.node_id) +} + +// int32 output_port = 2; +inline void GraphTransferNodeInput::clear_output_port() { + output_port_ = 0; +} +inline ::google::protobuf::int32 GraphTransferNodeInput::output_port() const { + // @@protoc_insertion_point(field_get:tensorflow.GraphTransferNodeInput.output_port) + return output_port_; +} +inline void GraphTransferNodeInput::set_output_port(::google::protobuf::int32 value) { + + output_port_ = value; + // @@protoc_insertion_point(field_set:tensorflow.GraphTransferNodeInput.output_port) +} + +// ------------------------------------------------------------------- + +// GraphTransferNodeInfo + +// string name = 1; +inline void GraphTransferNodeInfo::clear_name() { + name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& GraphTransferNodeInfo::name() const { + // @@protoc_insertion_point(field_get:tensorflow.GraphTransferNodeInfo.name) + return name_.Get(); +} +inline void GraphTransferNodeInfo::set_name(const ::std::string& value) { + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.GraphTransferNodeInfo.name) +} +#if LANG_CXX11 +inline void GraphTransferNodeInfo::set_name(::std::string&& value) { + + name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.GraphTransferNodeInfo.name) +} +#endif +inline void GraphTransferNodeInfo::set_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.GraphTransferNodeInfo.name) +} +inline void GraphTransferNodeInfo::set_name(const char* value, + size_t size) { + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.GraphTransferNodeInfo.name) +} +inline ::std::string* GraphTransferNodeInfo::mutable_name() { + + // @@protoc_insertion_point(field_mutable:tensorflow.GraphTransferNodeInfo.name) + return name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* GraphTransferNodeInfo::release_name() { + // @@protoc_insertion_point(field_release:tensorflow.GraphTransferNodeInfo.name) + + return name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void GraphTransferNodeInfo::set_allocated_name(::std::string* name) { + if (name != NULL) { + + } else { + + } + name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.GraphTransferNodeInfo.name) +} +inline ::std::string* GraphTransferNodeInfo::unsafe_arena_release_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.GraphTransferNodeInfo.name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void GraphTransferNodeInfo::unsafe_arena_set_allocated_name( + ::std::string* name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (name != NULL) { + + } else { + + } + name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.GraphTransferNodeInfo.name) +} + +// int32 node_id = 2; +inline void GraphTransferNodeInfo::clear_node_id() { + node_id_ = 0; +} +inline ::google::protobuf::int32 GraphTransferNodeInfo::node_id() const { + // @@protoc_insertion_point(field_get:tensorflow.GraphTransferNodeInfo.node_id) + return node_id_; +} +inline void GraphTransferNodeInfo::set_node_id(::google::protobuf::int32 value) { + + node_id_ = value; + // @@protoc_insertion_point(field_set:tensorflow.GraphTransferNodeInfo.node_id) +} + +// string type_name = 3; +inline void GraphTransferNodeInfo::clear_type_name() { + type_name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& GraphTransferNodeInfo::type_name() const { + // @@protoc_insertion_point(field_get:tensorflow.GraphTransferNodeInfo.type_name) + return type_name_.Get(); +} +inline void GraphTransferNodeInfo::set_type_name(const ::std::string& value) { + + type_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.GraphTransferNodeInfo.type_name) +} +#if LANG_CXX11 +inline void GraphTransferNodeInfo::set_type_name(::std::string&& value) { + + type_name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.GraphTransferNodeInfo.type_name) +} +#endif +inline void GraphTransferNodeInfo::set_type_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + type_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.GraphTransferNodeInfo.type_name) +} +inline void GraphTransferNodeInfo::set_type_name(const char* value, + size_t size) { + + type_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.GraphTransferNodeInfo.type_name) +} +inline ::std::string* GraphTransferNodeInfo::mutable_type_name() { + + // @@protoc_insertion_point(field_mutable:tensorflow.GraphTransferNodeInfo.type_name) + return type_name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* GraphTransferNodeInfo::release_type_name() { + // @@protoc_insertion_point(field_release:tensorflow.GraphTransferNodeInfo.type_name) + + return type_name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void GraphTransferNodeInfo::set_allocated_type_name(::std::string* type_name) { + if (type_name != NULL) { + + } else { + + } + type_name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), type_name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.GraphTransferNodeInfo.type_name) +} +inline ::std::string* GraphTransferNodeInfo::unsafe_arena_release_type_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.GraphTransferNodeInfo.type_name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return type_name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void GraphTransferNodeInfo::unsafe_arena_set_allocated_type_name( + ::std::string* type_name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (type_name != NULL) { + + } else { + + } + type_name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + type_name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.GraphTransferNodeInfo.type_name) +} + +// int32 soc_op_id = 4; +inline void GraphTransferNodeInfo::clear_soc_op_id() { + soc_op_id_ = 0; +} +inline ::google::protobuf::int32 GraphTransferNodeInfo::soc_op_id() const { + // @@protoc_insertion_point(field_get:tensorflow.GraphTransferNodeInfo.soc_op_id) + return soc_op_id_; +} +inline void GraphTransferNodeInfo::set_soc_op_id(::google::protobuf::int32 value) { + + soc_op_id_ = value; + // @@protoc_insertion_point(field_set:tensorflow.GraphTransferNodeInfo.soc_op_id) +} + +// int32 padding_id = 5; +inline void GraphTransferNodeInfo::clear_padding_id() { + padding_id_ = 0; +} +inline ::google::protobuf::int32 GraphTransferNodeInfo::padding_id() const { + // @@protoc_insertion_point(field_get:tensorflow.GraphTransferNodeInfo.padding_id) + return padding_id_; +} +inline void GraphTransferNodeInfo::set_padding_id(::google::protobuf::int32 value) { + + padding_id_ = value; + // @@protoc_insertion_point(field_set:tensorflow.GraphTransferNodeInfo.padding_id) +} + +// int32 input_count = 6; +inline void GraphTransferNodeInfo::clear_input_count() { + input_count_ = 0; +} +inline ::google::protobuf::int32 GraphTransferNodeInfo::input_count() const { + // @@protoc_insertion_point(field_get:tensorflow.GraphTransferNodeInfo.input_count) + return input_count_; +} +inline void GraphTransferNodeInfo::set_input_count(::google::protobuf::int32 value) { + + input_count_ = value; + // @@protoc_insertion_point(field_set:tensorflow.GraphTransferNodeInfo.input_count) +} + +// int32 output_count = 7; +inline void GraphTransferNodeInfo::clear_output_count() { + output_count_ = 0; +} +inline ::google::protobuf::int32 GraphTransferNodeInfo::output_count() const { + // @@protoc_insertion_point(field_get:tensorflow.GraphTransferNodeInfo.output_count) + return output_count_; +} +inline void GraphTransferNodeInfo::set_output_count(::google::protobuf::int32 value) { + + output_count_ = value; + // @@protoc_insertion_point(field_set:tensorflow.GraphTransferNodeInfo.output_count) +} + +// ------------------------------------------------------------------- + +// GraphTransferConstNodeInfo + +// string name = 1; +inline void GraphTransferConstNodeInfo::clear_name() { + name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& GraphTransferConstNodeInfo::name() const { + // @@protoc_insertion_point(field_get:tensorflow.GraphTransferConstNodeInfo.name) + return name_.Get(); +} +inline void GraphTransferConstNodeInfo::set_name(const ::std::string& value) { + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.GraphTransferConstNodeInfo.name) +} +#if LANG_CXX11 +inline void GraphTransferConstNodeInfo::set_name(::std::string&& value) { + + name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.GraphTransferConstNodeInfo.name) +} +#endif +inline void GraphTransferConstNodeInfo::set_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.GraphTransferConstNodeInfo.name) +} +inline void GraphTransferConstNodeInfo::set_name(const char* value, + size_t size) { + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.GraphTransferConstNodeInfo.name) +} +inline ::std::string* GraphTransferConstNodeInfo::mutable_name() { + + // @@protoc_insertion_point(field_mutable:tensorflow.GraphTransferConstNodeInfo.name) + return name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* GraphTransferConstNodeInfo::release_name() { + // @@protoc_insertion_point(field_release:tensorflow.GraphTransferConstNodeInfo.name) + + return name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void GraphTransferConstNodeInfo::set_allocated_name(::std::string* name) { + if (name != NULL) { + + } else { + + } + name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.GraphTransferConstNodeInfo.name) +} +inline ::std::string* GraphTransferConstNodeInfo::unsafe_arena_release_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.GraphTransferConstNodeInfo.name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void GraphTransferConstNodeInfo::unsafe_arena_set_allocated_name( + ::std::string* name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (name != NULL) { + + } else { + + } + name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.GraphTransferConstNodeInfo.name) +} + +// int32 node_id = 2; +inline void GraphTransferConstNodeInfo::clear_node_id() { + node_id_ = 0; +} +inline ::google::protobuf::int32 GraphTransferConstNodeInfo::node_id() const { + // @@protoc_insertion_point(field_get:tensorflow.GraphTransferConstNodeInfo.node_id) + return node_id_; +} +inline void GraphTransferConstNodeInfo::set_node_id(::google::protobuf::int32 value) { + + node_id_ = value; + // @@protoc_insertion_point(field_set:tensorflow.GraphTransferConstNodeInfo.node_id) +} + +// repeated int64 shape = 3; +inline int GraphTransferConstNodeInfo::shape_size() const { + return shape_.size(); +} +inline void GraphTransferConstNodeInfo::clear_shape() { + shape_.Clear(); +} +inline ::google::protobuf::int64 GraphTransferConstNodeInfo::shape(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.GraphTransferConstNodeInfo.shape) + return shape_.Get(index); +} +inline void GraphTransferConstNodeInfo::set_shape(int index, ::google::protobuf::int64 value) { + shape_.Set(index, value); + // @@protoc_insertion_point(field_set:tensorflow.GraphTransferConstNodeInfo.shape) +} +inline void GraphTransferConstNodeInfo::add_shape(::google::protobuf::int64 value) { + shape_.Add(value); + // @@protoc_insertion_point(field_add:tensorflow.GraphTransferConstNodeInfo.shape) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& +GraphTransferConstNodeInfo::shape() const { + // @@protoc_insertion_point(field_list:tensorflow.GraphTransferConstNodeInfo.shape) + return shape_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* +GraphTransferConstNodeInfo::mutable_shape() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.GraphTransferConstNodeInfo.shape) + return &shape_; +} + +// bytes data = 4; +inline void GraphTransferConstNodeInfo::clear_data() { + data_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& GraphTransferConstNodeInfo::data() const { + // @@protoc_insertion_point(field_get:tensorflow.GraphTransferConstNodeInfo.data) + return data_.Get(); +} +inline void GraphTransferConstNodeInfo::set_data(const ::std::string& value) { + + data_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.GraphTransferConstNodeInfo.data) +} +#if LANG_CXX11 +inline void GraphTransferConstNodeInfo::set_data(::std::string&& value) { + + data_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.GraphTransferConstNodeInfo.data) +} +#endif +inline void GraphTransferConstNodeInfo::set_data(const char* value) { + GOOGLE_DCHECK(value != NULL); + + data_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.GraphTransferConstNodeInfo.data) +} +inline void GraphTransferConstNodeInfo::set_data(const void* value, + size_t size) { + + data_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.GraphTransferConstNodeInfo.data) +} +inline ::std::string* GraphTransferConstNodeInfo::mutable_data() { + + // @@protoc_insertion_point(field_mutable:tensorflow.GraphTransferConstNodeInfo.data) + return data_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* GraphTransferConstNodeInfo::release_data() { + // @@protoc_insertion_point(field_release:tensorflow.GraphTransferConstNodeInfo.data) + + return data_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void GraphTransferConstNodeInfo::set_allocated_data(::std::string* data) { + if (data != NULL) { + + } else { + + } + data_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), data, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.GraphTransferConstNodeInfo.data) +} +inline ::std::string* GraphTransferConstNodeInfo::unsafe_arena_release_data() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.GraphTransferConstNodeInfo.data) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return data_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void GraphTransferConstNodeInfo::unsafe_arena_set_allocated_data( + ::std::string* data) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (data != NULL) { + + } else { + + } + data_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + data, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.GraphTransferConstNodeInfo.data) +} + +// .tensorflow.DataType dtype = 5; +inline void GraphTransferConstNodeInfo::clear_dtype() { + dtype_ = 0; +} +inline ::tensorflow::DataType GraphTransferConstNodeInfo::dtype() const { + // @@protoc_insertion_point(field_get:tensorflow.GraphTransferConstNodeInfo.dtype) + return static_cast< ::tensorflow::DataType >(dtype_); +} +inline void GraphTransferConstNodeInfo::set_dtype(::tensorflow::DataType value) { + + dtype_ = value; + // @@protoc_insertion_point(field_set:tensorflow.GraphTransferConstNodeInfo.dtype) +} + +// ------------------------------------------------------------------- + +// GraphTransferNodeInputInfo + +// int32 node_id = 1; +inline void GraphTransferNodeInputInfo::clear_node_id() { + node_id_ = 0; +} +inline ::google::protobuf::int32 GraphTransferNodeInputInfo::node_id() const { + // @@protoc_insertion_point(field_get:tensorflow.GraphTransferNodeInputInfo.node_id) + return node_id_; +} +inline void GraphTransferNodeInputInfo::set_node_id(::google::protobuf::int32 value) { + + node_id_ = value; + // @@protoc_insertion_point(field_set:tensorflow.GraphTransferNodeInputInfo.node_id) +} + +// repeated .tensorflow.GraphTransferNodeInput node_input = 2; +inline int GraphTransferNodeInputInfo::node_input_size() const { + return node_input_.size(); +} +inline void GraphTransferNodeInputInfo::clear_node_input() { + node_input_.Clear(); +} +inline ::tensorflow::GraphTransferNodeInput* GraphTransferNodeInputInfo::mutable_node_input(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.GraphTransferNodeInputInfo.node_input) + return node_input_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::GraphTransferNodeInput >* +GraphTransferNodeInputInfo::mutable_node_input() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.GraphTransferNodeInputInfo.node_input) + return &node_input_; +} +inline const ::tensorflow::GraphTransferNodeInput& GraphTransferNodeInputInfo::node_input(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.GraphTransferNodeInputInfo.node_input) + return node_input_.Get(index); +} +inline ::tensorflow::GraphTransferNodeInput* GraphTransferNodeInputInfo::add_node_input() { + // @@protoc_insertion_point(field_add:tensorflow.GraphTransferNodeInputInfo.node_input) + return node_input_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::GraphTransferNodeInput >& +GraphTransferNodeInputInfo::node_input() const { + // @@protoc_insertion_point(field_list:tensorflow.GraphTransferNodeInputInfo.node_input) + return node_input_; +} + +// ------------------------------------------------------------------- + +// GraphTransferNodeOutputInfo + +// int32 node_id = 1; +inline void GraphTransferNodeOutputInfo::clear_node_id() { + node_id_ = 0; +} +inline ::google::protobuf::int32 GraphTransferNodeOutputInfo::node_id() const { + // @@protoc_insertion_point(field_get:tensorflow.GraphTransferNodeOutputInfo.node_id) + return node_id_; +} +inline void GraphTransferNodeOutputInfo::set_node_id(::google::protobuf::int32 value) { + + node_id_ = value; + // @@protoc_insertion_point(field_set:tensorflow.GraphTransferNodeOutputInfo.node_id) +} + +// repeated int32 max_byte_size = 2; +inline int GraphTransferNodeOutputInfo::max_byte_size_size() const { + return max_byte_size_.size(); +} +inline void GraphTransferNodeOutputInfo::clear_max_byte_size() { + max_byte_size_.Clear(); +} +inline ::google::protobuf::int32 GraphTransferNodeOutputInfo::max_byte_size(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.GraphTransferNodeOutputInfo.max_byte_size) + return max_byte_size_.Get(index); +} +inline void GraphTransferNodeOutputInfo::set_max_byte_size(int index, ::google::protobuf::int32 value) { + max_byte_size_.Set(index, value); + // @@protoc_insertion_point(field_set:tensorflow.GraphTransferNodeOutputInfo.max_byte_size) +} +inline void GraphTransferNodeOutputInfo::add_max_byte_size(::google::protobuf::int32 value) { + max_byte_size_.Add(value); + // @@protoc_insertion_point(field_add:tensorflow.GraphTransferNodeOutputInfo.max_byte_size) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int32 >& +GraphTransferNodeOutputInfo::max_byte_size() const { + // @@protoc_insertion_point(field_list:tensorflow.GraphTransferNodeOutputInfo.max_byte_size) + return max_byte_size_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int32 >* +GraphTransferNodeOutputInfo::mutable_max_byte_size() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.GraphTransferNodeOutputInfo.max_byte_size) + return &max_byte_size_; +} + +// ------------------------------------------------------------------- + +// GraphTransferGraphInputNodeInfo + +// string name = 1; +inline void GraphTransferGraphInputNodeInfo::clear_name() { + name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& GraphTransferGraphInputNodeInfo::name() const { + // @@protoc_insertion_point(field_get:tensorflow.GraphTransferGraphInputNodeInfo.name) + return name_.Get(); +} +inline void GraphTransferGraphInputNodeInfo::set_name(const ::std::string& value) { + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.GraphTransferGraphInputNodeInfo.name) +} +#if LANG_CXX11 +inline void GraphTransferGraphInputNodeInfo::set_name(::std::string&& value) { + + name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.GraphTransferGraphInputNodeInfo.name) +} +#endif +inline void GraphTransferGraphInputNodeInfo::set_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.GraphTransferGraphInputNodeInfo.name) +} +inline void GraphTransferGraphInputNodeInfo::set_name(const char* value, + size_t size) { + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.GraphTransferGraphInputNodeInfo.name) +} +inline ::std::string* GraphTransferGraphInputNodeInfo::mutable_name() { + + // @@protoc_insertion_point(field_mutable:tensorflow.GraphTransferGraphInputNodeInfo.name) + return name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* GraphTransferGraphInputNodeInfo::release_name() { + // @@protoc_insertion_point(field_release:tensorflow.GraphTransferGraphInputNodeInfo.name) + + return name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void GraphTransferGraphInputNodeInfo::set_allocated_name(::std::string* name) { + if (name != NULL) { + + } else { + + } + name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.GraphTransferGraphInputNodeInfo.name) +} +inline ::std::string* GraphTransferGraphInputNodeInfo::unsafe_arena_release_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.GraphTransferGraphInputNodeInfo.name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void GraphTransferGraphInputNodeInfo::unsafe_arena_set_allocated_name( + ::std::string* name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (name != NULL) { + + } else { + + } + name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.GraphTransferGraphInputNodeInfo.name) +} + +// repeated int64 shape = 2; +inline int GraphTransferGraphInputNodeInfo::shape_size() const { + return shape_.size(); +} +inline void GraphTransferGraphInputNodeInfo::clear_shape() { + shape_.Clear(); +} +inline ::google::protobuf::int64 GraphTransferGraphInputNodeInfo::shape(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.GraphTransferGraphInputNodeInfo.shape) + return shape_.Get(index); +} +inline void GraphTransferGraphInputNodeInfo::set_shape(int index, ::google::protobuf::int64 value) { + shape_.Set(index, value); + // @@protoc_insertion_point(field_set:tensorflow.GraphTransferGraphInputNodeInfo.shape) +} +inline void GraphTransferGraphInputNodeInfo::add_shape(::google::protobuf::int64 value) { + shape_.Add(value); + // @@protoc_insertion_point(field_add:tensorflow.GraphTransferGraphInputNodeInfo.shape) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& +GraphTransferGraphInputNodeInfo::shape() const { + // @@protoc_insertion_point(field_list:tensorflow.GraphTransferGraphInputNodeInfo.shape) + return shape_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* +GraphTransferGraphInputNodeInfo::mutable_shape() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.GraphTransferGraphInputNodeInfo.shape) + return &shape_; +} + +// .tensorflow.DataType dtype = 3; +inline void GraphTransferGraphInputNodeInfo::clear_dtype() { + dtype_ = 0; +} +inline ::tensorflow::DataType GraphTransferGraphInputNodeInfo::dtype() const { + // @@protoc_insertion_point(field_get:tensorflow.GraphTransferGraphInputNodeInfo.dtype) + return static_cast< ::tensorflow::DataType >(dtype_); +} +inline void GraphTransferGraphInputNodeInfo::set_dtype(::tensorflow::DataType value) { + + dtype_ = value; + // @@protoc_insertion_point(field_set:tensorflow.GraphTransferGraphInputNodeInfo.dtype) +} + +// ------------------------------------------------------------------- + +// GraphTransferGraphOutputNodeInfo + +// string name = 1; +inline void GraphTransferGraphOutputNodeInfo::clear_name() { + name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& GraphTransferGraphOutputNodeInfo::name() const { + // @@protoc_insertion_point(field_get:tensorflow.GraphTransferGraphOutputNodeInfo.name) + return name_.Get(); +} +inline void GraphTransferGraphOutputNodeInfo::set_name(const ::std::string& value) { + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.GraphTransferGraphOutputNodeInfo.name) +} +#if LANG_CXX11 +inline void GraphTransferGraphOutputNodeInfo::set_name(::std::string&& value) { + + name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.GraphTransferGraphOutputNodeInfo.name) +} +#endif +inline void GraphTransferGraphOutputNodeInfo::set_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.GraphTransferGraphOutputNodeInfo.name) +} +inline void GraphTransferGraphOutputNodeInfo::set_name(const char* value, + size_t size) { + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.GraphTransferGraphOutputNodeInfo.name) +} +inline ::std::string* GraphTransferGraphOutputNodeInfo::mutable_name() { + + // @@protoc_insertion_point(field_mutable:tensorflow.GraphTransferGraphOutputNodeInfo.name) + return name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* GraphTransferGraphOutputNodeInfo::release_name() { + // @@protoc_insertion_point(field_release:tensorflow.GraphTransferGraphOutputNodeInfo.name) + + return name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void GraphTransferGraphOutputNodeInfo::set_allocated_name(::std::string* name) { + if (name != NULL) { + + } else { + + } + name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.GraphTransferGraphOutputNodeInfo.name) +} +inline ::std::string* GraphTransferGraphOutputNodeInfo::unsafe_arena_release_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.GraphTransferGraphOutputNodeInfo.name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void GraphTransferGraphOutputNodeInfo::unsafe_arena_set_allocated_name( + ::std::string* name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (name != NULL) { + + } else { + + } + name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.GraphTransferGraphOutputNodeInfo.name) +} + +// repeated int64 shape = 2; +inline int GraphTransferGraphOutputNodeInfo::shape_size() const { + return shape_.size(); +} +inline void GraphTransferGraphOutputNodeInfo::clear_shape() { + shape_.Clear(); +} +inline ::google::protobuf::int64 GraphTransferGraphOutputNodeInfo::shape(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.GraphTransferGraphOutputNodeInfo.shape) + return shape_.Get(index); +} +inline void GraphTransferGraphOutputNodeInfo::set_shape(int index, ::google::protobuf::int64 value) { + shape_.Set(index, value); + // @@protoc_insertion_point(field_set:tensorflow.GraphTransferGraphOutputNodeInfo.shape) +} +inline void GraphTransferGraphOutputNodeInfo::add_shape(::google::protobuf::int64 value) { + shape_.Add(value); + // @@protoc_insertion_point(field_add:tensorflow.GraphTransferGraphOutputNodeInfo.shape) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& +GraphTransferGraphOutputNodeInfo::shape() const { + // @@protoc_insertion_point(field_list:tensorflow.GraphTransferGraphOutputNodeInfo.shape) + return shape_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* +GraphTransferGraphOutputNodeInfo::mutable_shape() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.GraphTransferGraphOutputNodeInfo.shape) + return &shape_; +} + +// .tensorflow.DataType dtype = 3; +inline void GraphTransferGraphOutputNodeInfo::clear_dtype() { + dtype_ = 0; +} +inline ::tensorflow::DataType GraphTransferGraphOutputNodeInfo::dtype() const { + // @@protoc_insertion_point(field_get:tensorflow.GraphTransferGraphOutputNodeInfo.dtype) + return static_cast< ::tensorflow::DataType >(dtype_); +} +inline void GraphTransferGraphOutputNodeInfo::set_dtype(::tensorflow::DataType value) { + + dtype_ = value; + // @@protoc_insertion_point(field_set:tensorflow.GraphTransferGraphOutputNodeInfo.dtype) +} + +// ------------------------------------------------------------------- + +// GraphTransferInfo + +// repeated .tensorflow.GraphTransferNodeInfo node_info = 1; +inline int GraphTransferInfo::node_info_size() const { + return node_info_.size(); +} +inline void GraphTransferInfo::clear_node_info() { + node_info_.Clear(); +} +inline ::tensorflow::GraphTransferNodeInfo* GraphTransferInfo::mutable_node_info(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.GraphTransferInfo.node_info) + return node_info_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::GraphTransferNodeInfo >* +GraphTransferInfo::mutable_node_info() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.GraphTransferInfo.node_info) + return &node_info_; +} +inline const ::tensorflow::GraphTransferNodeInfo& GraphTransferInfo::node_info(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.GraphTransferInfo.node_info) + return node_info_.Get(index); +} +inline ::tensorflow::GraphTransferNodeInfo* GraphTransferInfo::add_node_info() { + // @@protoc_insertion_point(field_add:tensorflow.GraphTransferInfo.node_info) + return node_info_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::GraphTransferNodeInfo >& +GraphTransferInfo::node_info() const { + // @@protoc_insertion_point(field_list:tensorflow.GraphTransferInfo.node_info) + return node_info_; +} + +// repeated .tensorflow.GraphTransferConstNodeInfo const_node_info = 2; +inline int GraphTransferInfo::const_node_info_size() const { + return const_node_info_.size(); +} +inline void GraphTransferInfo::clear_const_node_info() { + const_node_info_.Clear(); +} +inline ::tensorflow::GraphTransferConstNodeInfo* GraphTransferInfo::mutable_const_node_info(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.GraphTransferInfo.const_node_info) + return const_node_info_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::GraphTransferConstNodeInfo >* +GraphTransferInfo::mutable_const_node_info() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.GraphTransferInfo.const_node_info) + return &const_node_info_; +} +inline const ::tensorflow::GraphTransferConstNodeInfo& GraphTransferInfo::const_node_info(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.GraphTransferInfo.const_node_info) + return const_node_info_.Get(index); +} +inline ::tensorflow::GraphTransferConstNodeInfo* GraphTransferInfo::add_const_node_info() { + // @@protoc_insertion_point(field_add:tensorflow.GraphTransferInfo.const_node_info) + return const_node_info_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::GraphTransferConstNodeInfo >& +GraphTransferInfo::const_node_info() const { + // @@protoc_insertion_point(field_list:tensorflow.GraphTransferInfo.const_node_info) + return const_node_info_; +} + +// repeated .tensorflow.GraphTransferNodeInputInfo node_input_info = 3; +inline int GraphTransferInfo::node_input_info_size() const { + return node_input_info_.size(); +} +inline void GraphTransferInfo::clear_node_input_info() { + node_input_info_.Clear(); +} +inline ::tensorflow::GraphTransferNodeInputInfo* GraphTransferInfo::mutable_node_input_info(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.GraphTransferInfo.node_input_info) + return node_input_info_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::GraphTransferNodeInputInfo >* +GraphTransferInfo::mutable_node_input_info() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.GraphTransferInfo.node_input_info) + return &node_input_info_; +} +inline const ::tensorflow::GraphTransferNodeInputInfo& GraphTransferInfo::node_input_info(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.GraphTransferInfo.node_input_info) + return node_input_info_.Get(index); +} +inline ::tensorflow::GraphTransferNodeInputInfo* GraphTransferInfo::add_node_input_info() { + // @@protoc_insertion_point(field_add:tensorflow.GraphTransferInfo.node_input_info) + return node_input_info_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::GraphTransferNodeInputInfo >& +GraphTransferInfo::node_input_info() const { + // @@protoc_insertion_point(field_list:tensorflow.GraphTransferInfo.node_input_info) + return node_input_info_; +} + +// repeated .tensorflow.GraphTransferNodeOutputInfo node_output_info = 4; +inline int GraphTransferInfo::node_output_info_size() const { + return node_output_info_.size(); +} +inline void GraphTransferInfo::clear_node_output_info() { + node_output_info_.Clear(); +} +inline ::tensorflow::GraphTransferNodeOutputInfo* GraphTransferInfo::mutable_node_output_info(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.GraphTransferInfo.node_output_info) + return node_output_info_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::GraphTransferNodeOutputInfo >* +GraphTransferInfo::mutable_node_output_info() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.GraphTransferInfo.node_output_info) + return &node_output_info_; +} +inline const ::tensorflow::GraphTransferNodeOutputInfo& GraphTransferInfo::node_output_info(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.GraphTransferInfo.node_output_info) + return node_output_info_.Get(index); +} +inline ::tensorflow::GraphTransferNodeOutputInfo* GraphTransferInfo::add_node_output_info() { + // @@protoc_insertion_point(field_add:tensorflow.GraphTransferInfo.node_output_info) + return node_output_info_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::GraphTransferNodeOutputInfo >& +GraphTransferInfo::node_output_info() const { + // @@protoc_insertion_point(field_list:tensorflow.GraphTransferInfo.node_output_info) + return node_output_info_; +} + +// repeated .tensorflow.GraphTransferGraphInputNodeInfo graph_input_node_info = 5; +inline int GraphTransferInfo::graph_input_node_info_size() const { + return graph_input_node_info_.size(); +} +inline void GraphTransferInfo::clear_graph_input_node_info() { + graph_input_node_info_.Clear(); +} +inline ::tensorflow::GraphTransferGraphInputNodeInfo* GraphTransferInfo::mutable_graph_input_node_info(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.GraphTransferInfo.graph_input_node_info) + return graph_input_node_info_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::GraphTransferGraphInputNodeInfo >* +GraphTransferInfo::mutable_graph_input_node_info() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.GraphTransferInfo.graph_input_node_info) + return &graph_input_node_info_; +} +inline const ::tensorflow::GraphTransferGraphInputNodeInfo& GraphTransferInfo::graph_input_node_info(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.GraphTransferInfo.graph_input_node_info) + return graph_input_node_info_.Get(index); +} +inline ::tensorflow::GraphTransferGraphInputNodeInfo* GraphTransferInfo::add_graph_input_node_info() { + // @@protoc_insertion_point(field_add:tensorflow.GraphTransferInfo.graph_input_node_info) + return graph_input_node_info_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::GraphTransferGraphInputNodeInfo >& +GraphTransferInfo::graph_input_node_info() const { + // @@protoc_insertion_point(field_list:tensorflow.GraphTransferInfo.graph_input_node_info) + return graph_input_node_info_; +} + +// repeated .tensorflow.GraphTransferGraphOutputNodeInfo graph_output_node_info = 6; +inline int GraphTransferInfo::graph_output_node_info_size() const { + return graph_output_node_info_.size(); +} +inline void GraphTransferInfo::clear_graph_output_node_info() { + graph_output_node_info_.Clear(); +} +inline ::tensorflow::GraphTransferGraphOutputNodeInfo* GraphTransferInfo::mutable_graph_output_node_info(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.GraphTransferInfo.graph_output_node_info) + return graph_output_node_info_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::GraphTransferGraphOutputNodeInfo >* +GraphTransferInfo::mutable_graph_output_node_info() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.GraphTransferInfo.graph_output_node_info) + return &graph_output_node_info_; +} +inline const ::tensorflow::GraphTransferGraphOutputNodeInfo& GraphTransferInfo::graph_output_node_info(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.GraphTransferInfo.graph_output_node_info) + return graph_output_node_info_.Get(index); +} +inline ::tensorflow::GraphTransferGraphOutputNodeInfo* GraphTransferInfo::add_graph_output_node_info() { + // @@protoc_insertion_point(field_add:tensorflow.GraphTransferInfo.graph_output_node_info) + return graph_output_node_info_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::GraphTransferGraphOutputNodeInfo >& +GraphTransferInfo::graph_output_node_info() const { + // @@protoc_insertion_point(field_list:tensorflow.GraphTransferInfo.graph_output_node_info) + return graph_output_node_info_; +} + +// .tensorflow.GraphTransferInfo.Destination destination = 7; +inline void GraphTransferInfo::clear_destination() { + destination_ = 0; +} +inline ::tensorflow::GraphTransferInfo_Destination GraphTransferInfo::destination() const { + // @@protoc_insertion_point(field_get:tensorflow.GraphTransferInfo.destination) + return static_cast< ::tensorflow::GraphTransferInfo_Destination >(destination_); +} +inline void GraphTransferInfo::set_destination(::tensorflow::GraphTransferInfo_Destination value) { + + destination_ = value; + // @@protoc_insertion_point(field_set:tensorflow.GraphTransferInfo.destination) +} + +#ifdef __GNUC__ + #pragma GCC diagnostic pop +#endif // __GNUC__ +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + + +// @@protoc_insertion_point(namespace_scope) + +} // namespace tensorflow + +namespace google { +namespace protobuf { + +template <> struct is_proto_enum< ::tensorflow::GraphTransferInfo_Destination> : ::std::true_type {}; +template <> +inline const EnumDescriptor* GetEnumDescriptor< ::tensorflow::GraphTransferInfo_Destination>() { + return ::tensorflow::GraphTransferInfo_Destination_descriptor(); +} + +} // namespace protobuf +} // namespace google + +// @@protoc_insertion_point(global_scope) + +#endif // PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2fgraph_5ftransfer_5finfo_2eproto diff --git a/linux/include/tensorflow/core/framework/graph_transfer_info.pb_text-impl.h b/linux/include/tensorflow/core/framework/graph_transfer_info.pb_text-impl.h new file mode 100644 index 000000000..426ff6eb2 --- /dev/null +++ b/linux/include/tensorflow/core/framework/graph_transfer_info.pb_text-impl.h @@ -0,0 +1,76 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_framework_graph_transfer_info_proto_IMPL_H_ +#define tensorflow_core_framework_graph_transfer_info_proto_IMPL_H_ + +#include "tensorflow/core/framework/graph_transfer_info.pb.h" +#include "tensorflow/core/framework/graph_transfer_info.pb_text.h" +#include "tensorflow/core/framework/types.pb.h" +#include "tensorflow/core/framework/types.pb_text-impl.h" +#include "tensorflow/core/lib/strings/proto_text_util.h" +#include "tensorflow/core/lib/strings/scanner.h" + +namespace tensorflow { + +namespace internal { + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::GraphTransferNodeInput& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::GraphTransferNodeInput* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::GraphTransferNodeInfo& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::GraphTransferNodeInfo* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::GraphTransferConstNodeInfo& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::GraphTransferConstNodeInfo* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::GraphTransferNodeInputInfo& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::GraphTransferNodeInputInfo* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::GraphTransferNodeOutputInfo& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::GraphTransferNodeOutputInfo* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::GraphTransferGraphInputNodeInfo& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::GraphTransferGraphInputNodeInfo* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::GraphTransferGraphOutputNodeInfo& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::GraphTransferGraphOutputNodeInfo* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::GraphTransferInfo& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::GraphTransferInfo* msg); + +} // namespace internal + +} // namespace tensorflow + +#endif // tensorflow_core_framework_graph_transfer_info_proto_IMPL_H_ diff --git a/linux/include/tensorflow/core/framework/graph_transfer_info.pb_text.h b/linux/include/tensorflow/core/framework/graph_transfer_info.pb_text.h new file mode 100644 index 000000000..ae664e299 --- /dev/null +++ b/linux/include/tensorflow/core/framework/graph_transfer_info.pb_text.h @@ -0,0 +1,98 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_framework_graph_transfer_info_proto_H_ +#define tensorflow_core_framework_graph_transfer_info_proto_H_ + +#include "tensorflow/core/framework/graph_transfer_info.pb.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/protobuf.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +// Message-text conversion for tensorflow.GraphTransferNodeInput +string ProtoDebugString( + const ::tensorflow::GraphTransferNodeInput& msg); +string ProtoShortDebugString( + const ::tensorflow::GraphTransferNodeInput& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::GraphTransferNodeInput* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.GraphTransferNodeInfo +string ProtoDebugString( + const ::tensorflow::GraphTransferNodeInfo& msg); +string ProtoShortDebugString( + const ::tensorflow::GraphTransferNodeInfo& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::GraphTransferNodeInfo* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.GraphTransferConstNodeInfo +string ProtoDebugString( + const ::tensorflow::GraphTransferConstNodeInfo& msg); +string ProtoShortDebugString( + const ::tensorflow::GraphTransferConstNodeInfo& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::GraphTransferConstNodeInfo* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.GraphTransferNodeInputInfo +string ProtoDebugString( + const ::tensorflow::GraphTransferNodeInputInfo& msg); +string ProtoShortDebugString( + const ::tensorflow::GraphTransferNodeInputInfo& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::GraphTransferNodeInputInfo* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.GraphTransferNodeOutputInfo +string ProtoDebugString( + const ::tensorflow::GraphTransferNodeOutputInfo& msg); +string ProtoShortDebugString( + const ::tensorflow::GraphTransferNodeOutputInfo& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::GraphTransferNodeOutputInfo* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.GraphTransferGraphInputNodeInfo +string ProtoDebugString( + const ::tensorflow::GraphTransferGraphInputNodeInfo& msg); +string ProtoShortDebugString( + const ::tensorflow::GraphTransferGraphInputNodeInfo& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::GraphTransferGraphInputNodeInfo* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.GraphTransferGraphOutputNodeInfo +string ProtoDebugString( + const ::tensorflow::GraphTransferGraphOutputNodeInfo& msg); +string ProtoShortDebugString( + const ::tensorflow::GraphTransferGraphOutputNodeInfo& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::GraphTransferGraphOutputNodeInfo* msg) + TF_MUST_USE_RESULT; + +// Enum text output for tensorflow.GraphTransferInfo.Destination +const char* EnumName_GraphTransferInfo_Destination( + ::tensorflow::GraphTransferInfo_Destination value); + +// Message-text conversion for tensorflow.GraphTransferInfo +string ProtoDebugString( + const ::tensorflow::GraphTransferInfo& msg); +string ProtoShortDebugString( + const ::tensorflow::GraphTransferInfo& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::GraphTransferInfo* msg) + TF_MUST_USE_RESULT; + +} // namespace tensorflow + +#endif // tensorflow_core_framework_graph_transfer_info_proto_H_ diff --git a/linux/include/tensorflow/core/framework/iterator.pb.h b/linux/include/tensorflow/core/framework/iterator.pb.h new file mode 100644 index 000000000..2defd8ed2 --- /dev/null +++ b/linux/include/tensorflow/core/framework/iterator.pb.h @@ -0,0 +1,383 @@ +// Generated by the protocol buffer compiler. DO NOT EDIT! +// source: tensorflow/core/framework/iterator.proto + +#ifndef PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2fiterator_2eproto +#define PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2fiterator_2eproto + +#include + +#include + +#if GOOGLE_PROTOBUF_VERSION < 3006000 +#error This file was generated by a newer version of protoc which is +#error incompatible with your Protocol Buffer headers. Please update +#error your headers. +#endif +#if 3006000 < GOOGLE_PROTOBUF_MIN_PROTOC_VERSION +#error This file was generated by an older version of protoc which is +#error incompatible with your Protocol Buffer headers. Please +#error regenerate this file with a newer version of protoc. +#endif + +#include +#include +#include +#include +#include +#include +#include +#include +#include // IWYU pragma: export +#include // IWYU pragma: export +#include +// @@protoc_insertion_point(includes) +#define PROTOBUF_INTERNAL_EXPORT_protobuf_tensorflow_2fcore_2fframework_2fiterator_2eproto + +namespace protobuf_tensorflow_2fcore_2fframework_2fiterator_2eproto { +// Internal implementation detail -- do not use these members. +struct TableStruct { + static const ::google::protobuf::internal::ParseTableField entries[]; + static const ::google::protobuf::internal::AuxillaryParseTableField aux[]; + static const ::google::protobuf::internal::ParseTable schema[1]; + static const ::google::protobuf::internal::FieldMetadata field_metadata[]; + static const ::google::protobuf::internal::SerializationTable serialization_table[]; + static const ::google::protobuf::uint32 offsets[]; +}; +void AddDescriptors(); +} // namespace protobuf_tensorflow_2fcore_2fframework_2fiterator_2eproto +namespace tensorflow { +class IteratorStateMetadata; +class IteratorStateMetadataDefaultTypeInternal; +extern IteratorStateMetadataDefaultTypeInternal _IteratorStateMetadata_default_instance_; +} // namespace tensorflow +namespace google { +namespace protobuf { +template<> ::tensorflow::IteratorStateMetadata* Arena::CreateMaybeMessage<::tensorflow::IteratorStateMetadata>(Arena*); +} // namespace protobuf +} // namespace google +namespace tensorflow { + +// =================================================================== + +class IteratorStateMetadata : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.IteratorStateMetadata) */ { + public: + IteratorStateMetadata(); + virtual ~IteratorStateMetadata(); + + IteratorStateMetadata(const IteratorStateMetadata& from); + + inline IteratorStateMetadata& operator=(const IteratorStateMetadata& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + IteratorStateMetadata(IteratorStateMetadata&& from) noexcept + : IteratorStateMetadata() { + *this = ::std::move(from); + } + + inline IteratorStateMetadata& operator=(IteratorStateMetadata&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const IteratorStateMetadata& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const IteratorStateMetadata* internal_default_instance() { + return reinterpret_cast( + &_IteratorStateMetadata_default_instance_); + } + static constexpr int kIndexInFileMessages = + 0; + + void UnsafeArenaSwap(IteratorStateMetadata* other); + void Swap(IteratorStateMetadata* other); + friend void swap(IteratorStateMetadata& a, IteratorStateMetadata& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline IteratorStateMetadata* New() const final { + return CreateMaybeMessage(NULL); + } + + IteratorStateMetadata* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const IteratorStateMetadata& from); + void MergeFrom(const IteratorStateMetadata& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(IteratorStateMetadata* other); + protected: + explicit IteratorStateMetadata(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated string keys = 2; + int keys_size() const; + void clear_keys(); + static const int kKeysFieldNumber = 2; + const ::std::string& keys(int index) const; + ::std::string* mutable_keys(int index); + void set_keys(int index, const ::std::string& value); + #if LANG_CXX11 + void set_keys(int index, ::std::string&& value); + #endif + void set_keys(int index, const char* value); + void set_keys(int index, const char* value, size_t size); + ::std::string* add_keys(); + void add_keys(const ::std::string& value); + #if LANG_CXX11 + void add_keys(::std::string&& value); + #endif + void add_keys(const char* value); + void add_keys(const char* value, size_t size); + const ::google::protobuf::RepeatedPtrField< ::std::string>& keys() const; + ::google::protobuf::RepeatedPtrField< ::std::string>* mutable_keys(); + + // string version = 1; + void clear_version(); + static const int kVersionFieldNumber = 1; + const ::std::string& version() const; + void set_version(const ::std::string& value); + #if LANG_CXX11 + void set_version(::std::string&& value); + #endif + void set_version(const char* value); + void set_version(const char* value, size_t size); + ::std::string* mutable_version(); + ::std::string* release_version(); + void set_allocated_version(::std::string* version); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_version(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_version( + ::std::string* version); + + // @@protoc_insertion_point(class_scope:tensorflow.IteratorStateMetadata) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedPtrField< ::std::string> keys_; + ::google::protobuf::internal::ArenaStringPtr version_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fiterator_2eproto::TableStruct; +}; +// =================================================================== + + +// =================================================================== + +#ifdef __GNUC__ + #pragma GCC diagnostic push + #pragma GCC diagnostic ignored "-Wstrict-aliasing" +#endif // __GNUC__ +// IteratorStateMetadata + +// string version = 1; +inline void IteratorStateMetadata::clear_version() { + version_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& IteratorStateMetadata::version() const { + // @@protoc_insertion_point(field_get:tensorflow.IteratorStateMetadata.version) + return version_.Get(); +} +inline void IteratorStateMetadata::set_version(const ::std::string& value) { + + version_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.IteratorStateMetadata.version) +} +#if LANG_CXX11 +inline void IteratorStateMetadata::set_version(::std::string&& value) { + + version_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.IteratorStateMetadata.version) +} +#endif +inline void IteratorStateMetadata::set_version(const char* value) { + GOOGLE_DCHECK(value != NULL); + + version_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.IteratorStateMetadata.version) +} +inline void IteratorStateMetadata::set_version(const char* value, + size_t size) { + + version_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.IteratorStateMetadata.version) +} +inline ::std::string* IteratorStateMetadata::mutable_version() { + + // @@protoc_insertion_point(field_mutable:tensorflow.IteratorStateMetadata.version) + return version_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* IteratorStateMetadata::release_version() { + // @@protoc_insertion_point(field_release:tensorflow.IteratorStateMetadata.version) + + return version_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void IteratorStateMetadata::set_allocated_version(::std::string* version) { + if (version != NULL) { + + } else { + + } + version_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), version, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.IteratorStateMetadata.version) +} +inline ::std::string* IteratorStateMetadata::unsafe_arena_release_version() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.IteratorStateMetadata.version) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return version_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void IteratorStateMetadata::unsafe_arena_set_allocated_version( + ::std::string* version) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (version != NULL) { + + } else { + + } + version_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + version, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.IteratorStateMetadata.version) +} + +// repeated string keys = 2; +inline int IteratorStateMetadata::keys_size() const { + return keys_.size(); +} +inline void IteratorStateMetadata::clear_keys() { + keys_.Clear(); +} +inline const ::std::string& IteratorStateMetadata::keys(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.IteratorStateMetadata.keys) + return keys_.Get(index); +} +inline ::std::string* IteratorStateMetadata::mutable_keys(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.IteratorStateMetadata.keys) + return keys_.Mutable(index); +} +inline void IteratorStateMetadata::set_keys(int index, const ::std::string& value) { + // @@protoc_insertion_point(field_set:tensorflow.IteratorStateMetadata.keys) + keys_.Mutable(index)->assign(value); +} +#if LANG_CXX11 +inline void IteratorStateMetadata::set_keys(int index, ::std::string&& value) { + // @@protoc_insertion_point(field_set:tensorflow.IteratorStateMetadata.keys) + keys_.Mutable(index)->assign(std::move(value)); +} +#endif +inline void IteratorStateMetadata::set_keys(int index, const char* value) { + GOOGLE_DCHECK(value != NULL); + keys_.Mutable(index)->assign(value); + // @@protoc_insertion_point(field_set_char:tensorflow.IteratorStateMetadata.keys) +} +inline void IteratorStateMetadata::set_keys(int index, const char* value, size_t size) { + keys_.Mutable(index)->assign( + reinterpret_cast(value), size); + // @@protoc_insertion_point(field_set_pointer:tensorflow.IteratorStateMetadata.keys) +} +inline ::std::string* IteratorStateMetadata::add_keys() { + // @@protoc_insertion_point(field_add_mutable:tensorflow.IteratorStateMetadata.keys) + return keys_.Add(); +} +inline void IteratorStateMetadata::add_keys(const ::std::string& value) { + keys_.Add()->assign(value); + // @@protoc_insertion_point(field_add:tensorflow.IteratorStateMetadata.keys) +} +#if LANG_CXX11 +inline void IteratorStateMetadata::add_keys(::std::string&& value) { + keys_.Add(std::move(value)); + // @@protoc_insertion_point(field_add:tensorflow.IteratorStateMetadata.keys) +} +#endif +inline void IteratorStateMetadata::add_keys(const char* value) { + GOOGLE_DCHECK(value != NULL); + keys_.Add()->assign(value); + // @@protoc_insertion_point(field_add_char:tensorflow.IteratorStateMetadata.keys) +} +inline void IteratorStateMetadata::add_keys(const char* value, size_t size) { + keys_.Add()->assign(reinterpret_cast(value), size); + // @@protoc_insertion_point(field_add_pointer:tensorflow.IteratorStateMetadata.keys) +} +inline const ::google::protobuf::RepeatedPtrField< ::std::string>& +IteratorStateMetadata::keys() const { + // @@protoc_insertion_point(field_list:tensorflow.IteratorStateMetadata.keys) + return keys_; +} +inline ::google::protobuf::RepeatedPtrField< ::std::string>* +IteratorStateMetadata::mutable_keys() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.IteratorStateMetadata.keys) + return &keys_; +} + +#ifdef __GNUC__ + #pragma GCC diagnostic pop +#endif // __GNUC__ + +// @@protoc_insertion_point(namespace_scope) + +} // namespace tensorflow + +// @@protoc_insertion_point(global_scope) + +#endif // PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2fiterator_2eproto diff --git a/linux/include/tensorflow/core/framework/iterator.pb_text-impl.h b/linux/include/tensorflow/core/framework/iterator.pb_text-impl.h new file mode 100644 index 000000000..723f118c6 --- /dev/null +++ b/linux/include/tensorflow/core/framework/iterator.pb_text-impl.h @@ -0,0 +1,25 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_framework_iterator_proto_IMPL_H_ +#define tensorflow_core_framework_iterator_proto_IMPL_H_ + +#include "tensorflow/core/framework/iterator.pb.h" +#include "tensorflow/core/framework/iterator.pb_text.h" +#include "tensorflow/core/lib/strings/proto_text_util.h" +#include "tensorflow/core/lib/strings/scanner.h" + +namespace tensorflow { + +namespace internal { + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::IteratorStateMetadata& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::IteratorStateMetadata* msg); + +} // namespace internal + +} // namespace tensorflow + +#endif // tensorflow_core_framework_iterator_proto_IMPL_H_ diff --git a/linux/include/tensorflow/core/framework/iterator.pb_text.h b/linux/include/tensorflow/core/framework/iterator.pb_text.h new file mode 100644 index 000000000..56304d627 --- /dev/null +++ b/linux/include/tensorflow/core/framework/iterator.pb_text.h @@ -0,0 +1,24 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_framework_iterator_proto_H_ +#define tensorflow_core_framework_iterator_proto_H_ + +#include "tensorflow/core/framework/iterator.pb.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/protobuf.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +// Message-text conversion for tensorflow.IteratorStateMetadata +string ProtoDebugString( + const ::tensorflow::IteratorStateMetadata& msg); +string ProtoShortDebugString( + const ::tensorflow::IteratorStateMetadata& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::IteratorStateMetadata* msg) + TF_MUST_USE_RESULT; + +} // namespace tensorflow + +#endif // tensorflow_core_framework_iterator_proto_H_ diff --git a/linux/include/tensorflow/core/framework/kernel_def.pb.h b/linux/include/tensorflow/core/framework/kernel_def.pb.h new file mode 100644 index 000000000..505f7a5cb --- /dev/null +++ b/linux/include/tensorflow/core/framework/kernel_def.pb.h @@ -0,0 +1,1086 @@ +// Generated by the protocol buffer compiler. DO NOT EDIT! +// source: tensorflow/core/framework/kernel_def.proto + +#ifndef PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2fkernel_5fdef_2eproto +#define PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2fkernel_5fdef_2eproto + +#include + +#include + +#if GOOGLE_PROTOBUF_VERSION < 3006000 +#error This file was generated by a newer version of protoc which is +#error incompatible with your Protocol Buffer headers. Please update +#error your headers. +#endif +#if 3006000 < GOOGLE_PROTOBUF_MIN_PROTOC_VERSION +#error This file was generated by an older version of protoc which is +#error incompatible with your Protocol Buffer headers. Please +#error regenerate this file with a newer version of protoc. +#endif + +#include +#include +#include +#include +#include +#include +#include +#include +#include // IWYU pragma: export +#include // IWYU pragma: export +#include +#include "tensorflow/core/framework/attr_value.pb.h" +// @@protoc_insertion_point(includes) +#define PROTOBUF_INTERNAL_EXPORT_protobuf_tensorflow_2fcore_2fframework_2fkernel_5fdef_2eproto + +namespace protobuf_tensorflow_2fcore_2fframework_2fkernel_5fdef_2eproto { +// Internal implementation detail -- do not use these members. +struct TableStruct { + static const ::google::protobuf::internal::ParseTableField entries[]; + static const ::google::protobuf::internal::AuxillaryParseTableField aux[]; + static const ::google::protobuf::internal::ParseTable schema[3]; + static const ::google::protobuf::internal::FieldMetadata field_metadata[]; + static const ::google::protobuf::internal::SerializationTable serialization_table[]; + static const ::google::protobuf::uint32 offsets[]; +}; +void AddDescriptors(); +} // namespace protobuf_tensorflow_2fcore_2fframework_2fkernel_5fdef_2eproto +namespace tensorflow { +class KernelDef; +class KernelDefDefaultTypeInternal; +extern KernelDefDefaultTypeInternal _KernelDef_default_instance_; +class KernelDef_AttrConstraint; +class KernelDef_AttrConstraintDefaultTypeInternal; +extern KernelDef_AttrConstraintDefaultTypeInternal _KernelDef_AttrConstraint_default_instance_; +class KernelList; +class KernelListDefaultTypeInternal; +extern KernelListDefaultTypeInternal _KernelList_default_instance_; +} // namespace tensorflow +namespace google { +namespace protobuf { +template<> ::tensorflow::KernelDef* Arena::CreateMaybeMessage<::tensorflow::KernelDef>(Arena*); +template<> ::tensorflow::KernelDef_AttrConstraint* Arena::CreateMaybeMessage<::tensorflow::KernelDef_AttrConstraint>(Arena*); +template<> ::tensorflow::KernelList* Arena::CreateMaybeMessage<::tensorflow::KernelList>(Arena*); +} // namespace protobuf +} // namespace google +namespace tensorflow { + +// =================================================================== + +class KernelDef_AttrConstraint : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.KernelDef.AttrConstraint) */ { + public: + KernelDef_AttrConstraint(); + virtual ~KernelDef_AttrConstraint(); + + KernelDef_AttrConstraint(const KernelDef_AttrConstraint& from); + + inline KernelDef_AttrConstraint& operator=(const KernelDef_AttrConstraint& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + KernelDef_AttrConstraint(KernelDef_AttrConstraint&& from) noexcept + : KernelDef_AttrConstraint() { + *this = ::std::move(from); + } + + inline KernelDef_AttrConstraint& operator=(KernelDef_AttrConstraint&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const KernelDef_AttrConstraint& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const KernelDef_AttrConstraint* internal_default_instance() { + return reinterpret_cast( + &_KernelDef_AttrConstraint_default_instance_); + } + static constexpr int kIndexInFileMessages = + 0; + + void UnsafeArenaSwap(KernelDef_AttrConstraint* other); + void Swap(KernelDef_AttrConstraint* other); + friend void swap(KernelDef_AttrConstraint& a, KernelDef_AttrConstraint& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline KernelDef_AttrConstraint* New() const final { + return CreateMaybeMessage(NULL); + } + + KernelDef_AttrConstraint* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const KernelDef_AttrConstraint& from); + void MergeFrom(const KernelDef_AttrConstraint& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(KernelDef_AttrConstraint* other); + protected: + explicit KernelDef_AttrConstraint(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // string name = 1; + void clear_name(); + static const int kNameFieldNumber = 1; + const ::std::string& name() const; + void set_name(const ::std::string& value); + #if LANG_CXX11 + void set_name(::std::string&& value); + #endif + void set_name(const char* value); + void set_name(const char* value, size_t size); + ::std::string* mutable_name(); + ::std::string* release_name(); + void set_allocated_name(::std::string* name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_name( + ::std::string* name); + + // .tensorflow.AttrValue allowed_values = 2; + bool has_allowed_values() const; + void clear_allowed_values(); + static const int kAllowedValuesFieldNumber = 2; + private: + const ::tensorflow::AttrValue& _internal_allowed_values() const; + public: + const ::tensorflow::AttrValue& allowed_values() const; + ::tensorflow::AttrValue* release_allowed_values(); + ::tensorflow::AttrValue* mutable_allowed_values(); + void set_allocated_allowed_values(::tensorflow::AttrValue* allowed_values); + void unsafe_arena_set_allocated_allowed_values( + ::tensorflow::AttrValue* allowed_values); + ::tensorflow::AttrValue* unsafe_arena_release_allowed_values(); + + // @@protoc_insertion_point(class_scope:tensorflow.KernelDef.AttrConstraint) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::internal::ArenaStringPtr name_; + ::tensorflow::AttrValue* allowed_values_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fkernel_5fdef_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class KernelDef : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.KernelDef) */ { + public: + KernelDef(); + virtual ~KernelDef(); + + KernelDef(const KernelDef& from); + + inline KernelDef& operator=(const KernelDef& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + KernelDef(KernelDef&& from) noexcept + : KernelDef() { + *this = ::std::move(from); + } + + inline KernelDef& operator=(KernelDef&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const KernelDef& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const KernelDef* internal_default_instance() { + return reinterpret_cast( + &_KernelDef_default_instance_); + } + static constexpr int kIndexInFileMessages = + 1; + + void UnsafeArenaSwap(KernelDef* other); + void Swap(KernelDef* other); + friend void swap(KernelDef& a, KernelDef& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline KernelDef* New() const final { + return CreateMaybeMessage(NULL); + } + + KernelDef* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const KernelDef& from); + void MergeFrom(const KernelDef& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(KernelDef* other); + protected: + explicit KernelDef(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + typedef KernelDef_AttrConstraint AttrConstraint; + + // accessors ------------------------------------------------------- + + // repeated .tensorflow.KernelDef.AttrConstraint constraint = 3; + int constraint_size() const; + void clear_constraint(); + static const int kConstraintFieldNumber = 3; + ::tensorflow::KernelDef_AttrConstraint* mutable_constraint(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::KernelDef_AttrConstraint >* + mutable_constraint(); + const ::tensorflow::KernelDef_AttrConstraint& constraint(int index) const; + ::tensorflow::KernelDef_AttrConstraint* add_constraint(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::KernelDef_AttrConstraint >& + constraint() const; + + // repeated string host_memory_arg = 4; + int host_memory_arg_size() const; + void clear_host_memory_arg(); + static const int kHostMemoryArgFieldNumber = 4; + const ::std::string& host_memory_arg(int index) const; + ::std::string* mutable_host_memory_arg(int index); + void set_host_memory_arg(int index, const ::std::string& value); + #if LANG_CXX11 + void set_host_memory_arg(int index, ::std::string&& value); + #endif + void set_host_memory_arg(int index, const char* value); + void set_host_memory_arg(int index, const char* value, size_t size); + ::std::string* add_host_memory_arg(); + void add_host_memory_arg(const ::std::string& value); + #if LANG_CXX11 + void add_host_memory_arg(::std::string&& value); + #endif + void add_host_memory_arg(const char* value); + void add_host_memory_arg(const char* value, size_t size); + const ::google::protobuf::RepeatedPtrField< ::std::string>& host_memory_arg() const; + ::google::protobuf::RepeatedPtrField< ::std::string>* mutable_host_memory_arg(); + + // string op = 1; + void clear_op(); + static const int kOpFieldNumber = 1; + const ::std::string& op() const; + void set_op(const ::std::string& value); + #if LANG_CXX11 + void set_op(::std::string&& value); + #endif + void set_op(const char* value); + void set_op(const char* value, size_t size); + ::std::string* mutable_op(); + ::std::string* release_op(); + void set_allocated_op(::std::string* op); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_op(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_op( + ::std::string* op); + + // string device_type = 2; + void clear_device_type(); + static const int kDeviceTypeFieldNumber = 2; + const ::std::string& device_type() const; + void set_device_type(const ::std::string& value); + #if LANG_CXX11 + void set_device_type(::std::string&& value); + #endif + void set_device_type(const char* value); + void set_device_type(const char* value, size_t size); + ::std::string* mutable_device_type(); + ::std::string* release_device_type(); + void set_allocated_device_type(::std::string* device_type); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_device_type(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_device_type( + ::std::string* device_type); + + // string label = 5; + void clear_label(); + static const int kLabelFieldNumber = 5; + const ::std::string& label() const; + void set_label(const ::std::string& value); + #if LANG_CXX11 + void set_label(::std::string&& value); + #endif + void set_label(const char* value); + void set_label(const char* value, size_t size); + ::std::string* mutable_label(); + ::std::string* release_label(); + void set_allocated_label(::std::string* label); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_label(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_label( + ::std::string* label); + + // @@protoc_insertion_point(class_scope:tensorflow.KernelDef) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::KernelDef_AttrConstraint > constraint_; + ::google::protobuf::RepeatedPtrField< ::std::string> host_memory_arg_; + ::google::protobuf::internal::ArenaStringPtr op_; + ::google::protobuf::internal::ArenaStringPtr device_type_; + ::google::protobuf::internal::ArenaStringPtr label_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fkernel_5fdef_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class KernelList : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.KernelList) */ { + public: + KernelList(); + virtual ~KernelList(); + + KernelList(const KernelList& from); + + inline KernelList& operator=(const KernelList& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + KernelList(KernelList&& from) noexcept + : KernelList() { + *this = ::std::move(from); + } + + inline KernelList& operator=(KernelList&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const KernelList& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const KernelList* internal_default_instance() { + return reinterpret_cast( + &_KernelList_default_instance_); + } + static constexpr int kIndexInFileMessages = + 2; + + void UnsafeArenaSwap(KernelList* other); + void Swap(KernelList* other); + friend void swap(KernelList& a, KernelList& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline KernelList* New() const final { + return CreateMaybeMessage(NULL); + } + + KernelList* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const KernelList& from); + void MergeFrom(const KernelList& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(KernelList* other); + protected: + explicit KernelList(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated .tensorflow.KernelDef kernel = 1; + int kernel_size() const; + void clear_kernel(); + static const int kKernelFieldNumber = 1; + ::tensorflow::KernelDef* mutable_kernel(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::KernelDef >* + mutable_kernel(); + const ::tensorflow::KernelDef& kernel(int index) const; + ::tensorflow::KernelDef* add_kernel(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::KernelDef >& + kernel() const; + + // @@protoc_insertion_point(class_scope:tensorflow.KernelList) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::KernelDef > kernel_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fkernel_5fdef_2eproto::TableStruct; +}; +// =================================================================== + + +// =================================================================== + +#ifdef __GNUC__ + #pragma GCC diagnostic push + #pragma GCC diagnostic ignored "-Wstrict-aliasing" +#endif // __GNUC__ +// KernelDef_AttrConstraint + +// string name = 1; +inline void KernelDef_AttrConstraint::clear_name() { + name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& KernelDef_AttrConstraint::name() const { + // @@protoc_insertion_point(field_get:tensorflow.KernelDef.AttrConstraint.name) + return name_.Get(); +} +inline void KernelDef_AttrConstraint::set_name(const ::std::string& value) { + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.KernelDef.AttrConstraint.name) +} +#if LANG_CXX11 +inline void KernelDef_AttrConstraint::set_name(::std::string&& value) { + + name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.KernelDef.AttrConstraint.name) +} +#endif +inline void KernelDef_AttrConstraint::set_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.KernelDef.AttrConstraint.name) +} +inline void KernelDef_AttrConstraint::set_name(const char* value, + size_t size) { + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.KernelDef.AttrConstraint.name) +} +inline ::std::string* KernelDef_AttrConstraint::mutable_name() { + + // @@protoc_insertion_point(field_mutable:tensorflow.KernelDef.AttrConstraint.name) + return name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* KernelDef_AttrConstraint::release_name() { + // @@protoc_insertion_point(field_release:tensorflow.KernelDef.AttrConstraint.name) + + return name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void KernelDef_AttrConstraint::set_allocated_name(::std::string* name) { + if (name != NULL) { + + } else { + + } + name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.KernelDef.AttrConstraint.name) +} +inline ::std::string* KernelDef_AttrConstraint::unsafe_arena_release_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.KernelDef.AttrConstraint.name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void KernelDef_AttrConstraint::unsafe_arena_set_allocated_name( + ::std::string* name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (name != NULL) { + + } else { + + } + name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.KernelDef.AttrConstraint.name) +} + +// .tensorflow.AttrValue allowed_values = 2; +inline bool KernelDef_AttrConstraint::has_allowed_values() const { + return this != internal_default_instance() && allowed_values_ != NULL; +} +inline const ::tensorflow::AttrValue& KernelDef_AttrConstraint::_internal_allowed_values() const { + return *allowed_values_; +} +inline const ::tensorflow::AttrValue& KernelDef_AttrConstraint::allowed_values() const { + const ::tensorflow::AttrValue* p = allowed_values_; + // @@protoc_insertion_point(field_get:tensorflow.KernelDef.AttrConstraint.allowed_values) + return p != NULL ? *p : *reinterpret_cast( + &::tensorflow::_AttrValue_default_instance_); +} +inline ::tensorflow::AttrValue* KernelDef_AttrConstraint::release_allowed_values() { + // @@protoc_insertion_point(field_release:tensorflow.KernelDef.AttrConstraint.allowed_values) + + ::tensorflow::AttrValue* temp = allowed_values_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + allowed_values_ = NULL; + return temp; +} +inline ::tensorflow::AttrValue* KernelDef_AttrConstraint::unsafe_arena_release_allowed_values() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.KernelDef.AttrConstraint.allowed_values) + + ::tensorflow::AttrValue* temp = allowed_values_; + allowed_values_ = NULL; + return temp; +} +inline ::tensorflow::AttrValue* KernelDef_AttrConstraint::mutable_allowed_values() { + + if (allowed_values_ == NULL) { + auto* p = CreateMaybeMessage<::tensorflow::AttrValue>(GetArenaNoVirtual()); + allowed_values_ = p; + } + // @@protoc_insertion_point(field_mutable:tensorflow.KernelDef.AttrConstraint.allowed_values) + return allowed_values_; +} +inline void KernelDef_AttrConstraint::set_allocated_allowed_values(::tensorflow::AttrValue* allowed_values) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(allowed_values_); + } + if (allowed_values) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(allowed_values)->GetArena(); + if (message_arena != submessage_arena) { + allowed_values = ::google::protobuf::internal::GetOwnedMessage( + message_arena, allowed_values, submessage_arena); + } + + } else { + + } + allowed_values_ = allowed_values; + // @@protoc_insertion_point(field_set_allocated:tensorflow.KernelDef.AttrConstraint.allowed_values) +} + +// ------------------------------------------------------------------- + +// KernelDef + +// string op = 1; +inline void KernelDef::clear_op() { + op_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& KernelDef::op() const { + // @@protoc_insertion_point(field_get:tensorflow.KernelDef.op) + return op_.Get(); +} +inline void KernelDef::set_op(const ::std::string& value) { + + op_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.KernelDef.op) +} +#if LANG_CXX11 +inline void KernelDef::set_op(::std::string&& value) { + + op_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.KernelDef.op) +} +#endif +inline void KernelDef::set_op(const char* value) { + GOOGLE_DCHECK(value != NULL); + + op_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.KernelDef.op) +} +inline void KernelDef::set_op(const char* value, + size_t size) { + + op_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.KernelDef.op) +} +inline ::std::string* KernelDef::mutable_op() { + + // @@protoc_insertion_point(field_mutable:tensorflow.KernelDef.op) + return op_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* KernelDef::release_op() { + // @@protoc_insertion_point(field_release:tensorflow.KernelDef.op) + + return op_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void KernelDef::set_allocated_op(::std::string* op) { + if (op != NULL) { + + } else { + + } + op_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), op, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.KernelDef.op) +} +inline ::std::string* KernelDef::unsafe_arena_release_op() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.KernelDef.op) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return op_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void KernelDef::unsafe_arena_set_allocated_op( + ::std::string* op) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (op != NULL) { + + } else { + + } + op_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + op, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.KernelDef.op) +} + +// string device_type = 2; +inline void KernelDef::clear_device_type() { + device_type_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& KernelDef::device_type() const { + // @@protoc_insertion_point(field_get:tensorflow.KernelDef.device_type) + return device_type_.Get(); +} +inline void KernelDef::set_device_type(const ::std::string& value) { + + device_type_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.KernelDef.device_type) +} +#if LANG_CXX11 +inline void KernelDef::set_device_type(::std::string&& value) { + + device_type_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.KernelDef.device_type) +} +#endif +inline void KernelDef::set_device_type(const char* value) { + GOOGLE_DCHECK(value != NULL); + + device_type_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.KernelDef.device_type) +} +inline void KernelDef::set_device_type(const char* value, + size_t size) { + + device_type_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.KernelDef.device_type) +} +inline ::std::string* KernelDef::mutable_device_type() { + + // @@protoc_insertion_point(field_mutable:tensorflow.KernelDef.device_type) + return device_type_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* KernelDef::release_device_type() { + // @@protoc_insertion_point(field_release:tensorflow.KernelDef.device_type) + + return device_type_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void KernelDef::set_allocated_device_type(::std::string* device_type) { + if (device_type != NULL) { + + } else { + + } + device_type_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), device_type, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.KernelDef.device_type) +} +inline ::std::string* KernelDef::unsafe_arena_release_device_type() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.KernelDef.device_type) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return device_type_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void KernelDef::unsafe_arena_set_allocated_device_type( + ::std::string* device_type) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (device_type != NULL) { + + } else { + + } + device_type_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + device_type, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.KernelDef.device_type) +} + +// repeated .tensorflow.KernelDef.AttrConstraint constraint = 3; +inline int KernelDef::constraint_size() const { + return constraint_.size(); +} +inline void KernelDef::clear_constraint() { + constraint_.Clear(); +} +inline ::tensorflow::KernelDef_AttrConstraint* KernelDef::mutable_constraint(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.KernelDef.constraint) + return constraint_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::KernelDef_AttrConstraint >* +KernelDef::mutable_constraint() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.KernelDef.constraint) + return &constraint_; +} +inline const ::tensorflow::KernelDef_AttrConstraint& KernelDef::constraint(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.KernelDef.constraint) + return constraint_.Get(index); +} +inline ::tensorflow::KernelDef_AttrConstraint* KernelDef::add_constraint() { + // @@protoc_insertion_point(field_add:tensorflow.KernelDef.constraint) + return constraint_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::KernelDef_AttrConstraint >& +KernelDef::constraint() const { + // @@protoc_insertion_point(field_list:tensorflow.KernelDef.constraint) + return constraint_; +} + +// repeated string host_memory_arg = 4; +inline int KernelDef::host_memory_arg_size() const { + return host_memory_arg_.size(); +} +inline void KernelDef::clear_host_memory_arg() { + host_memory_arg_.Clear(); +} +inline const ::std::string& KernelDef::host_memory_arg(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.KernelDef.host_memory_arg) + return host_memory_arg_.Get(index); +} +inline ::std::string* KernelDef::mutable_host_memory_arg(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.KernelDef.host_memory_arg) + return host_memory_arg_.Mutable(index); +} +inline void KernelDef::set_host_memory_arg(int index, const ::std::string& value) { + // @@protoc_insertion_point(field_set:tensorflow.KernelDef.host_memory_arg) + host_memory_arg_.Mutable(index)->assign(value); +} +#if LANG_CXX11 +inline void KernelDef::set_host_memory_arg(int index, ::std::string&& value) { + // @@protoc_insertion_point(field_set:tensorflow.KernelDef.host_memory_arg) + host_memory_arg_.Mutable(index)->assign(std::move(value)); +} +#endif +inline void KernelDef::set_host_memory_arg(int index, const char* value) { + GOOGLE_DCHECK(value != NULL); + host_memory_arg_.Mutable(index)->assign(value); + // @@protoc_insertion_point(field_set_char:tensorflow.KernelDef.host_memory_arg) +} +inline void KernelDef::set_host_memory_arg(int index, const char* value, size_t size) { + host_memory_arg_.Mutable(index)->assign( + reinterpret_cast(value), size); + // @@protoc_insertion_point(field_set_pointer:tensorflow.KernelDef.host_memory_arg) +} +inline ::std::string* KernelDef::add_host_memory_arg() { + // @@protoc_insertion_point(field_add_mutable:tensorflow.KernelDef.host_memory_arg) + return host_memory_arg_.Add(); +} +inline void KernelDef::add_host_memory_arg(const ::std::string& value) { + host_memory_arg_.Add()->assign(value); + // @@protoc_insertion_point(field_add:tensorflow.KernelDef.host_memory_arg) +} +#if LANG_CXX11 +inline void KernelDef::add_host_memory_arg(::std::string&& value) { + host_memory_arg_.Add(std::move(value)); + // @@protoc_insertion_point(field_add:tensorflow.KernelDef.host_memory_arg) +} +#endif +inline void KernelDef::add_host_memory_arg(const char* value) { + GOOGLE_DCHECK(value != NULL); + host_memory_arg_.Add()->assign(value); + // @@protoc_insertion_point(field_add_char:tensorflow.KernelDef.host_memory_arg) +} +inline void KernelDef::add_host_memory_arg(const char* value, size_t size) { + host_memory_arg_.Add()->assign(reinterpret_cast(value), size); + // @@protoc_insertion_point(field_add_pointer:tensorflow.KernelDef.host_memory_arg) +} +inline const ::google::protobuf::RepeatedPtrField< ::std::string>& +KernelDef::host_memory_arg() const { + // @@protoc_insertion_point(field_list:tensorflow.KernelDef.host_memory_arg) + return host_memory_arg_; +} +inline ::google::protobuf::RepeatedPtrField< ::std::string>* +KernelDef::mutable_host_memory_arg() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.KernelDef.host_memory_arg) + return &host_memory_arg_; +} + +// string label = 5; +inline void KernelDef::clear_label() { + label_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& KernelDef::label() const { + // @@protoc_insertion_point(field_get:tensorflow.KernelDef.label) + return label_.Get(); +} +inline void KernelDef::set_label(const ::std::string& value) { + + label_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.KernelDef.label) +} +#if LANG_CXX11 +inline void KernelDef::set_label(::std::string&& value) { + + label_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.KernelDef.label) +} +#endif +inline void KernelDef::set_label(const char* value) { + GOOGLE_DCHECK(value != NULL); + + label_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.KernelDef.label) +} +inline void KernelDef::set_label(const char* value, + size_t size) { + + label_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.KernelDef.label) +} +inline ::std::string* KernelDef::mutable_label() { + + // @@protoc_insertion_point(field_mutable:tensorflow.KernelDef.label) + return label_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* KernelDef::release_label() { + // @@protoc_insertion_point(field_release:tensorflow.KernelDef.label) + + return label_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void KernelDef::set_allocated_label(::std::string* label) { + if (label != NULL) { + + } else { + + } + label_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), label, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.KernelDef.label) +} +inline ::std::string* KernelDef::unsafe_arena_release_label() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.KernelDef.label) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return label_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void KernelDef::unsafe_arena_set_allocated_label( + ::std::string* label) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (label != NULL) { + + } else { + + } + label_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + label, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.KernelDef.label) +} + +// ------------------------------------------------------------------- + +// KernelList + +// repeated .tensorflow.KernelDef kernel = 1; +inline int KernelList::kernel_size() const { + return kernel_.size(); +} +inline void KernelList::clear_kernel() { + kernel_.Clear(); +} +inline ::tensorflow::KernelDef* KernelList::mutable_kernel(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.KernelList.kernel) + return kernel_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::KernelDef >* +KernelList::mutable_kernel() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.KernelList.kernel) + return &kernel_; +} +inline const ::tensorflow::KernelDef& KernelList::kernel(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.KernelList.kernel) + return kernel_.Get(index); +} +inline ::tensorflow::KernelDef* KernelList::add_kernel() { + // @@protoc_insertion_point(field_add:tensorflow.KernelList.kernel) + return kernel_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::KernelDef >& +KernelList::kernel() const { + // @@protoc_insertion_point(field_list:tensorflow.KernelList.kernel) + return kernel_; +} + +#ifdef __GNUC__ + #pragma GCC diagnostic pop +#endif // __GNUC__ +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + + +// @@protoc_insertion_point(namespace_scope) + +} // namespace tensorflow + +// @@protoc_insertion_point(global_scope) + +#endif // PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2fkernel_5fdef_2eproto diff --git a/linux/include/tensorflow/core/framework/kernel_def.pb_text-impl.h b/linux/include/tensorflow/core/framework/kernel_def.pb_text-impl.h new file mode 100644 index 000000000..867907ee3 --- /dev/null +++ b/linux/include/tensorflow/core/framework/kernel_def.pb_text-impl.h @@ -0,0 +1,49 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_framework_kernel_def_proto_IMPL_H_ +#define tensorflow_core_framework_kernel_def_proto_IMPL_H_ + +#include "tensorflow/core/framework/attr_value.pb.h" +#include "tensorflow/core/framework/attr_value.pb_text-impl.h" +#include "tensorflow/core/framework/kernel_def.pb.h" +#include "tensorflow/core/framework/kernel_def.pb_text.h" +#include "tensorflow/core/framework/resource_handle.pb.h" +#include "tensorflow/core/framework/resource_handle.pb_text-impl.h" +#include "tensorflow/core/framework/tensor.pb.h" +#include "tensorflow/core/framework/tensor.pb_text-impl.h" +#include "tensorflow/core/framework/tensor_shape.pb.h" +#include "tensorflow/core/framework/tensor_shape.pb_text-impl.h" +#include "tensorflow/core/framework/types.pb.h" +#include "tensorflow/core/framework/types.pb_text-impl.h" +#include "tensorflow/core/lib/strings/proto_text_util.h" +#include "tensorflow/core/lib/strings/scanner.h" + +namespace tensorflow { + +namespace internal { + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::KernelDef_AttrConstraint& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::KernelDef_AttrConstraint* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::KernelDef& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::KernelDef* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::KernelList& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::KernelList* msg); + +} // namespace internal + +} // namespace tensorflow + +#endif // tensorflow_core_framework_kernel_def_proto_IMPL_H_ diff --git a/linux/include/tensorflow/core/framework/kernel_def.pb_text.h b/linux/include/tensorflow/core/framework/kernel_def.pb_text.h new file mode 100644 index 000000000..ae51ddc9e --- /dev/null +++ b/linux/include/tensorflow/core/framework/kernel_def.pb_text.h @@ -0,0 +1,44 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_framework_kernel_def_proto_H_ +#define tensorflow_core_framework_kernel_def_proto_H_ + +#include "tensorflow/core/framework/kernel_def.pb.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/protobuf.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +// Message-text conversion for tensorflow.KernelDef.AttrConstraint +string ProtoDebugString( + const ::tensorflow::KernelDef_AttrConstraint& msg); +string ProtoShortDebugString( + const ::tensorflow::KernelDef_AttrConstraint& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::KernelDef_AttrConstraint* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.KernelDef +string ProtoDebugString( + const ::tensorflow::KernelDef& msg); +string ProtoShortDebugString( + const ::tensorflow::KernelDef& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::KernelDef* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.KernelList +string ProtoDebugString( + const ::tensorflow::KernelList& msg); +string ProtoShortDebugString( + const ::tensorflow::KernelList& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::KernelList* msg) + TF_MUST_USE_RESULT; + +} // namespace tensorflow + +#endif // tensorflow_core_framework_kernel_def_proto_H_ diff --git a/linux/include/tensorflow/core/framework/kernel_def_builder.h b/linux/include/tensorflow/core/framework/kernel_def_builder.h new file mode 100644 index 000000000..32dd21f94 --- /dev/null +++ b/linux/include/tensorflow/core/framework/kernel_def_builder.h @@ -0,0 +1,87 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_FRAMEWORK_KERNEL_DEF_BUILDER_H_ +#define TENSORFLOW_CORE_FRAMEWORK_KERNEL_DEF_BUILDER_H_ + +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/gtl/array_slice.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +// Forward declare proto so that kernels don't need to depend on it +class KernelDef; + +// Builder class passed to the REGISTER_KERNEL_BUILDER() macro. +class KernelDefBuilder { + public: + // Starts with just the name field set. + // Caller MUST call Build() and take ownership of the result. + explicit KernelDefBuilder(const char* op_name); + ~KernelDefBuilder(); + + // Required: specify the type of device this kernel supports. + // Returns *this. + KernelDefBuilder& Device(const char* device_type); + // KernelDefBuilder& Device(DeviceType device_type); + + // Specify that this kernel supports a limited set of values for a + // particular type or list(type) attr (a further restriction than + // what the Op allows). + // Returns *this. + KernelDefBuilder& TypeConstraint(const char* attr_name, + gtl::ArraySlice allowed); + + // Like TypeConstraint but supports just a single type. + KernelDefBuilder& TypeConstraint(const char* attr_name, DataType allowed); + + // Like TypeConstraint, but (a) gets the type from a template parameter + // and (b) only supports a constraint to a single type. + template + KernelDefBuilder& TypeConstraint(const char* attr_name); + // TODO(josh11b): Support other types of attr constraints as needed. + + // Specify that this kernel requires/provides an input/output arg + // in host memory (instead of the default, device memory). + // Returns *this. + KernelDefBuilder& HostMemory(const char* arg_name); + + // Specify that this kernel requires a particular value for the + // "_kernel" attr. May only be specified once. Returns *this. + KernelDefBuilder& Label(const char* label); + + // Returns a pointer to a KernelDef with fields set based on the + // above calls to this instance. + // Caller takes ownership of the result. + const KernelDef* Build(); + + private: + KernelDef* kernel_def_; + + TF_DISALLOW_COPY_AND_ASSIGN(KernelDefBuilder); +}; + +// IMPLEMENTATION + +template +KernelDefBuilder& KernelDefBuilder::TypeConstraint(const char* attr_name) { + return this->TypeConstraint(attr_name, DataTypeToEnum::v()); +} + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_FRAMEWORK_KERNEL_DEF_BUILDER_H_ diff --git a/linux/include/tensorflow/core/framework/kernel_def_util.h b/linux/include/tensorflow/core/framework/kernel_def_util.h new file mode 100644 index 000000000..b973cefc4 --- /dev/null +++ b/linux/include/tensorflow/core/framework/kernel_def_util.h @@ -0,0 +1,31 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_FRAMEWORK_KERNEL_DEF_UTIL_H_ +#define TENSORFLOW_CORE_FRAMEWORK_KERNEL_DEF_UTIL_H_ + +#include "tensorflow/core/framework/kernel_def.pb.h" +#include "tensorflow/core/framework/node_def_util.h" + +namespace tensorflow { + +// Returns whether the attrs satisfy the constraints in the kernel_def. Returns +// an error if attrs in kernel_def are not found, or have a mismatching type. +Status KernelAttrsMatch(const KernelDef& kernel_def, AttrSlice attrs, + bool* match); + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_FRAMEWORK_KERNEL_DEF_UTIL_H_ diff --git a/linux/include/tensorflow/core/framework/log_memory.h b/linux/include/tensorflow/core/framework/log_memory.h new file mode 100644 index 000000000..1b926ddaa --- /dev/null +++ b/linux/include/tensorflow/core/framework/log_memory.h @@ -0,0 +1,111 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_FRAMEWORK_LOG_MEMORY_H_ +#define TENSORFLOW_CORE_FRAMEWORK_LOG_MEMORY_H_ + +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/platform/protobuf.h" + +namespace tensorflow { + +// LogMemory contains methods for recording memory allocations and +// frees, associating each allocation with a step identified by a +// process-wide id. For now, logging is enabled whenever VLOG_IS_ON(1) +// for the log_memory module. +// +// Limitations: We don't log memory allocations by Eigen on the CPU +// since that would require major changes to plumb through to the +// Eigen::{DefaultDevice,ThreadPoolDevice} allocate and deallocate +// methods. We do log Eigen allocations on GPU since the plumbing was +// already in place. +class LogMemory { + public: + // Allocations sometimes happen outside any computation step, and + // SpecialStepIds lists the ids used for those steps. + enum SpecialStepIds { + // Used when performing a just-in-time constant folding optimization. + CONSTANT_FOLDING_STEP_ID = -1, + // Used when constructing an Op kernel before executing a step. + OP_KERNEL_CONSTRUCTION_STEP_ID = -2, + // Used when allocating a tensor buffer from external code, e.g., + // the C API. + EXTERNAL_TENSOR_ALLOCATION_STEP_ID = -3, + // Used when allocating a buffer for network transfer. + NETWORK_BUFFER_STEP_ID = -4, + // Used when allocating a buffer to fill a Proto from the GPU. + PROTO_BUFFER_STEP_ID = -5, + // Used when allocating a Tensor where the caller has not indicated + // the step. + UNKNOWN_STEP_ID = -6, + }; + + static const string kLogMemoryLabel; + + // Test to see if memory logging is enabled. For now, logging is + // enabled whenever VLOG_IS_ON(1) for the log_memory module. + static bool IsEnabled(); + + // Log the beginning of a step. + static void RecordStep(int64 step_id, const string& handle); + + // Log a tensor buffer allocation. The name indicates which kernel + // made the allocation. If the allocation is made through an + // OpKernelContext the step_id indicates which step is executing, + // otherwise step_id is one of the SpecialStepIds defined in + // op_kernel.h, e.g. Op Kernel construction or an optimization pass + // such as constant folding. + static void RecordTensorAllocation(const string& kernel_name, int64 step_id, + const Tensor& tensor); + + // Log a tensor buffer deallocation. The deallocation is triggered + // when the buffer's refcount falls to zero, and the tracking + // mechanism does not associate it with a particular step or + // kernel. The allocation_id/allocator_name should match a + // corresponding tensor previously passed in to + // RecordTensorAllocation. + static void RecordTensorDeallocation(int64 allocation_id, + const string& allocator_name); + + // Log the use of a tensor as an output from a kernel. + static void RecordTensorOutput(const string& kernel_name, int64 step_id, + int index, const Tensor& tensor); + + // Log a "raw" allocation, which is just a buffer sized in + // bytes. The Eigen allocator, and memory copies, record their + // allocations this way, since they do not allocate TensorFlow + // tensors. The operation is set to the OpKernel name if this is + // called from within an Op execution, otherwise it indicates an + // operation such as memcpy. The step_id if >=0 indicates which step + // is executing, otherwise step_id is one of the SpecialStepIds + // defined in op_kernel.h, e.g. Op Kernel construction or an + // optimization pass such as constant folding. + static void RecordRawAllocation(const string& operation, int64 step_id, + size_t num_bytes, void* ptr, + Allocator* allocator); + + // Log a "raw" deallocation of a buffer. When deferred is true, the + // buffer won't be used again, but a GPU kernel may still be + // enqueued using the buffer. A deferred deallocation should always + // be followed by a matching non-deferred deallocation when the + // buffer is actually returned and can be reused. + static void RecordRawDeallocation(const string& operation, int64 step_id, + void* ptr, Allocator* allocator, + bool deferred); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_FRAMEWORK_LOG_MEMORY_H_ diff --git a/linux/include/tensorflow/core/framework/log_memory.pb.h b/linux/include/tensorflow/core/framework/log_memory.pb.h new file mode 100644 index 000000000..6975b19a1 --- /dev/null +++ b/linux/include/tensorflow/core/framework/log_memory.pb.h @@ -0,0 +1,1992 @@ +// Generated by the protocol buffer compiler. DO NOT EDIT! +// source: tensorflow/core/framework/log_memory.proto + +#ifndef PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2flog_5fmemory_2eproto +#define PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2flog_5fmemory_2eproto + +#include + +#include + +#if GOOGLE_PROTOBUF_VERSION < 3006000 +#error This file was generated by a newer version of protoc which is +#error incompatible with your Protocol Buffer headers. Please update +#error your headers. +#endif +#if 3006000 < GOOGLE_PROTOBUF_MIN_PROTOC_VERSION +#error This file was generated by an older version of protoc which is +#error incompatible with your Protocol Buffer headers. Please +#error regenerate this file with a newer version of protoc. +#endif + +#include +#include +#include +#include +#include +#include +#include +#include +#include // IWYU pragma: export +#include // IWYU pragma: export +#include +#include "tensorflow/core/framework/tensor_description.pb.h" +// @@protoc_insertion_point(includes) +#define PROTOBUF_INTERNAL_EXPORT_protobuf_tensorflow_2fcore_2fframework_2flog_5fmemory_2eproto + +namespace protobuf_tensorflow_2fcore_2fframework_2flog_5fmemory_2eproto { +// Internal implementation detail -- do not use these members. +struct TableStruct { + static const ::google::protobuf::internal::ParseTableField entries[]; + static const ::google::protobuf::internal::AuxillaryParseTableField aux[]; + static const ::google::protobuf::internal::ParseTable schema[6]; + static const ::google::protobuf::internal::FieldMetadata field_metadata[]; + static const ::google::protobuf::internal::SerializationTable serialization_table[]; + static const ::google::protobuf::uint32 offsets[]; +}; +void AddDescriptors(); +} // namespace protobuf_tensorflow_2fcore_2fframework_2flog_5fmemory_2eproto +namespace tensorflow { +class MemoryLogRawAllocation; +class MemoryLogRawAllocationDefaultTypeInternal; +extern MemoryLogRawAllocationDefaultTypeInternal _MemoryLogRawAllocation_default_instance_; +class MemoryLogRawDeallocation; +class MemoryLogRawDeallocationDefaultTypeInternal; +extern MemoryLogRawDeallocationDefaultTypeInternal _MemoryLogRawDeallocation_default_instance_; +class MemoryLogStep; +class MemoryLogStepDefaultTypeInternal; +extern MemoryLogStepDefaultTypeInternal _MemoryLogStep_default_instance_; +class MemoryLogTensorAllocation; +class MemoryLogTensorAllocationDefaultTypeInternal; +extern MemoryLogTensorAllocationDefaultTypeInternal _MemoryLogTensorAllocation_default_instance_; +class MemoryLogTensorDeallocation; +class MemoryLogTensorDeallocationDefaultTypeInternal; +extern MemoryLogTensorDeallocationDefaultTypeInternal _MemoryLogTensorDeallocation_default_instance_; +class MemoryLogTensorOutput; +class MemoryLogTensorOutputDefaultTypeInternal; +extern MemoryLogTensorOutputDefaultTypeInternal _MemoryLogTensorOutput_default_instance_; +} // namespace tensorflow +namespace google { +namespace protobuf { +template<> ::tensorflow::MemoryLogRawAllocation* Arena::CreateMaybeMessage<::tensorflow::MemoryLogRawAllocation>(Arena*); +template<> ::tensorflow::MemoryLogRawDeallocation* Arena::CreateMaybeMessage<::tensorflow::MemoryLogRawDeallocation>(Arena*); +template<> ::tensorflow::MemoryLogStep* Arena::CreateMaybeMessage<::tensorflow::MemoryLogStep>(Arena*); +template<> ::tensorflow::MemoryLogTensorAllocation* Arena::CreateMaybeMessage<::tensorflow::MemoryLogTensorAllocation>(Arena*); +template<> ::tensorflow::MemoryLogTensorDeallocation* Arena::CreateMaybeMessage<::tensorflow::MemoryLogTensorDeallocation>(Arena*); +template<> ::tensorflow::MemoryLogTensorOutput* Arena::CreateMaybeMessage<::tensorflow::MemoryLogTensorOutput>(Arena*); +} // namespace protobuf +} // namespace google +namespace tensorflow { + +// =================================================================== + +class MemoryLogStep : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.MemoryLogStep) */ { + public: + MemoryLogStep(); + virtual ~MemoryLogStep(); + + MemoryLogStep(const MemoryLogStep& from); + + inline MemoryLogStep& operator=(const MemoryLogStep& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + MemoryLogStep(MemoryLogStep&& from) noexcept + : MemoryLogStep() { + *this = ::std::move(from); + } + + inline MemoryLogStep& operator=(MemoryLogStep&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const MemoryLogStep& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const MemoryLogStep* internal_default_instance() { + return reinterpret_cast( + &_MemoryLogStep_default_instance_); + } + static constexpr int kIndexInFileMessages = + 0; + + void UnsafeArenaSwap(MemoryLogStep* other); + void Swap(MemoryLogStep* other); + friend void swap(MemoryLogStep& a, MemoryLogStep& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline MemoryLogStep* New() const final { + return CreateMaybeMessage(NULL); + } + + MemoryLogStep* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const MemoryLogStep& from); + void MergeFrom(const MemoryLogStep& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(MemoryLogStep* other); + protected: + explicit MemoryLogStep(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // string handle = 2; + void clear_handle(); + static const int kHandleFieldNumber = 2; + const ::std::string& handle() const; + void set_handle(const ::std::string& value); + #if LANG_CXX11 + void set_handle(::std::string&& value); + #endif + void set_handle(const char* value); + void set_handle(const char* value, size_t size); + ::std::string* mutable_handle(); + ::std::string* release_handle(); + void set_allocated_handle(::std::string* handle); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_handle(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_handle( + ::std::string* handle); + + // int64 step_id = 1; + void clear_step_id(); + static const int kStepIdFieldNumber = 1; + ::google::protobuf::int64 step_id() const; + void set_step_id(::google::protobuf::int64 value); + + // @@protoc_insertion_point(class_scope:tensorflow.MemoryLogStep) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::internal::ArenaStringPtr handle_; + ::google::protobuf::int64 step_id_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2flog_5fmemory_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class MemoryLogTensorAllocation : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.MemoryLogTensorAllocation) */ { + public: + MemoryLogTensorAllocation(); + virtual ~MemoryLogTensorAllocation(); + + MemoryLogTensorAllocation(const MemoryLogTensorAllocation& from); + + inline MemoryLogTensorAllocation& operator=(const MemoryLogTensorAllocation& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + MemoryLogTensorAllocation(MemoryLogTensorAllocation&& from) noexcept + : MemoryLogTensorAllocation() { + *this = ::std::move(from); + } + + inline MemoryLogTensorAllocation& operator=(MemoryLogTensorAllocation&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const MemoryLogTensorAllocation& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const MemoryLogTensorAllocation* internal_default_instance() { + return reinterpret_cast( + &_MemoryLogTensorAllocation_default_instance_); + } + static constexpr int kIndexInFileMessages = + 1; + + void UnsafeArenaSwap(MemoryLogTensorAllocation* other); + void Swap(MemoryLogTensorAllocation* other); + friend void swap(MemoryLogTensorAllocation& a, MemoryLogTensorAllocation& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline MemoryLogTensorAllocation* New() const final { + return CreateMaybeMessage(NULL); + } + + MemoryLogTensorAllocation* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const MemoryLogTensorAllocation& from); + void MergeFrom(const MemoryLogTensorAllocation& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(MemoryLogTensorAllocation* other); + protected: + explicit MemoryLogTensorAllocation(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // string kernel_name = 2; + void clear_kernel_name(); + static const int kKernelNameFieldNumber = 2; + const ::std::string& kernel_name() const; + void set_kernel_name(const ::std::string& value); + #if LANG_CXX11 + void set_kernel_name(::std::string&& value); + #endif + void set_kernel_name(const char* value); + void set_kernel_name(const char* value, size_t size); + ::std::string* mutable_kernel_name(); + ::std::string* release_kernel_name(); + void set_allocated_kernel_name(::std::string* kernel_name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_kernel_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_kernel_name( + ::std::string* kernel_name); + + // .tensorflow.TensorDescription tensor = 3; + bool has_tensor() const; + void clear_tensor(); + static const int kTensorFieldNumber = 3; + private: + const ::tensorflow::TensorDescription& _internal_tensor() const; + public: + const ::tensorflow::TensorDescription& tensor() const; + ::tensorflow::TensorDescription* release_tensor(); + ::tensorflow::TensorDescription* mutable_tensor(); + void set_allocated_tensor(::tensorflow::TensorDescription* tensor); + void unsafe_arena_set_allocated_tensor( + ::tensorflow::TensorDescription* tensor); + ::tensorflow::TensorDescription* unsafe_arena_release_tensor(); + + // int64 step_id = 1; + void clear_step_id(); + static const int kStepIdFieldNumber = 1; + ::google::protobuf::int64 step_id() const; + void set_step_id(::google::protobuf::int64 value); + + // @@protoc_insertion_point(class_scope:tensorflow.MemoryLogTensorAllocation) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::internal::ArenaStringPtr kernel_name_; + ::tensorflow::TensorDescription* tensor_; + ::google::protobuf::int64 step_id_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2flog_5fmemory_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class MemoryLogTensorDeallocation : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.MemoryLogTensorDeallocation) */ { + public: + MemoryLogTensorDeallocation(); + virtual ~MemoryLogTensorDeallocation(); + + MemoryLogTensorDeallocation(const MemoryLogTensorDeallocation& from); + + inline MemoryLogTensorDeallocation& operator=(const MemoryLogTensorDeallocation& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + MemoryLogTensorDeallocation(MemoryLogTensorDeallocation&& from) noexcept + : MemoryLogTensorDeallocation() { + *this = ::std::move(from); + } + + inline MemoryLogTensorDeallocation& operator=(MemoryLogTensorDeallocation&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const MemoryLogTensorDeallocation& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const MemoryLogTensorDeallocation* internal_default_instance() { + return reinterpret_cast( + &_MemoryLogTensorDeallocation_default_instance_); + } + static constexpr int kIndexInFileMessages = + 2; + + void UnsafeArenaSwap(MemoryLogTensorDeallocation* other); + void Swap(MemoryLogTensorDeallocation* other); + friend void swap(MemoryLogTensorDeallocation& a, MemoryLogTensorDeallocation& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline MemoryLogTensorDeallocation* New() const final { + return CreateMaybeMessage(NULL); + } + + MemoryLogTensorDeallocation* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const MemoryLogTensorDeallocation& from); + void MergeFrom(const MemoryLogTensorDeallocation& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(MemoryLogTensorDeallocation* other); + protected: + explicit MemoryLogTensorDeallocation(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // string allocator_name = 2; + void clear_allocator_name(); + static const int kAllocatorNameFieldNumber = 2; + const ::std::string& allocator_name() const; + void set_allocator_name(const ::std::string& value); + #if LANG_CXX11 + void set_allocator_name(::std::string&& value); + #endif + void set_allocator_name(const char* value); + void set_allocator_name(const char* value, size_t size); + ::std::string* mutable_allocator_name(); + ::std::string* release_allocator_name(); + void set_allocated_allocator_name(::std::string* allocator_name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_allocator_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_allocator_name( + ::std::string* allocator_name); + + // int64 allocation_id = 1; + void clear_allocation_id(); + static const int kAllocationIdFieldNumber = 1; + ::google::protobuf::int64 allocation_id() const; + void set_allocation_id(::google::protobuf::int64 value); + + // @@protoc_insertion_point(class_scope:tensorflow.MemoryLogTensorDeallocation) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::internal::ArenaStringPtr allocator_name_; + ::google::protobuf::int64 allocation_id_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2flog_5fmemory_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class MemoryLogTensorOutput : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.MemoryLogTensorOutput) */ { + public: + MemoryLogTensorOutput(); + virtual ~MemoryLogTensorOutput(); + + MemoryLogTensorOutput(const MemoryLogTensorOutput& from); + + inline MemoryLogTensorOutput& operator=(const MemoryLogTensorOutput& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + MemoryLogTensorOutput(MemoryLogTensorOutput&& from) noexcept + : MemoryLogTensorOutput() { + *this = ::std::move(from); + } + + inline MemoryLogTensorOutput& operator=(MemoryLogTensorOutput&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const MemoryLogTensorOutput& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const MemoryLogTensorOutput* internal_default_instance() { + return reinterpret_cast( + &_MemoryLogTensorOutput_default_instance_); + } + static constexpr int kIndexInFileMessages = + 3; + + void UnsafeArenaSwap(MemoryLogTensorOutput* other); + void Swap(MemoryLogTensorOutput* other); + friend void swap(MemoryLogTensorOutput& a, MemoryLogTensorOutput& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline MemoryLogTensorOutput* New() const final { + return CreateMaybeMessage(NULL); + } + + MemoryLogTensorOutput* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const MemoryLogTensorOutput& from); + void MergeFrom(const MemoryLogTensorOutput& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(MemoryLogTensorOutput* other); + protected: + explicit MemoryLogTensorOutput(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // string kernel_name = 2; + void clear_kernel_name(); + static const int kKernelNameFieldNumber = 2; + const ::std::string& kernel_name() const; + void set_kernel_name(const ::std::string& value); + #if LANG_CXX11 + void set_kernel_name(::std::string&& value); + #endif + void set_kernel_name(const char* value); + void set_kernel_name(const char* value, size_t size); + ::std::string* mutable_kernel_name(); + ::std::string* release_kernel_name(); + void set_allocated_kernel_name(::std::string* kernel_name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_kernel_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_kernel_name( + ::std::string* kernel_name); + + // .tensorflow.TensorDescription tensor = 4; + bool has_tensor() const; + void clear_tensor(); + static const int kTensorFieldNumber = 4; + private: + const ::tensorflow::TensorDescription& _internal_tensor() const; + public: + const ::tensorflow::TensorDescription& tensor() const; + ::tensorflow::TensorDescription* release_tensor(); + ::tensorflow::TensorDescription* mutable_tensor(); + void set_allocated_tensor(::tensorflow::TensorDescription* tensor); + void unsafe_arena_set_allocated_tensor( + ::tensorflow::TensorDescription* tensor); + ::tensorflow::TensorDescription* unsafe_arena_release_tensor(); + + // int64 step_id = 1; + void clear_step_id(); + static const int kStepIdFieldNumber = 1; + ::google::protobuf::int64 step_id() const; + void set_step_id(::google::protobuf::int64 value); + + // int32 index = 3; + void clear_index(); + static const int kIndexFieldNumber = 3; + ::google::protobuf::int32 index() const; + void set_index(::google::protobuf::int32 value); + + // @@protoc_insertion_point(class_scope:tensorflow.MemoryLogTensorOutput) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::internal::ArenaStringPtr kernel_name_; + ::tensorflow::TensorDescription* tensor_; + ::google::protobuf::int64 step_id_; + ::google::protobuf::int32 index_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2flog_5fmemory_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class MemoryLogRawAllocation : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.MemoryLogRawAllocation) */ { + public: + MemoryLogRawAllocation(); + virtual ~MemoryLogRawAllocation(); + + MemoryLogRawAllocation(const MemoryLogRawAllocation& from); + + inline MemoryLogRawAllocation& operator=(const MemoryLogRawAllocation& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + MemoryLogRawAllocation(MemoryLogRawAllocation&& from) noexcept + : MemoryLogRawAllocation() { + *this = ::std::move(from); + } + + inline MemoryLogRawAllocation& operator=(MemoryLogRawAllocation&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const MemoryLogRawAllocation& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const MemoryLogRawAllocation* internal_default_instance() { + return reinterpret_cast( + &_MemoryLogRawAllocation_default_instance_); + } + static constexpr int kIndexInFileMessages = + 4; + + void UnsafeArenaSwap(MemoryLogRawAllocation* other); + void Swap(MemoryLogRawAllocation* other); + friend void swap(MemoryLogRawAllocation& a, MemoryLogRawAllocation& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline MemoryLogRawAllocation* New() const final { + return CreateMaybeMessage(NULL); + } + + MemoryLogRawAllocation* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const MemoryLogRawAllocation& from); + void MergeFrom(const MemoryLogRawAllocation& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(MemoryLogRawAllocation* other); + protected: + explicit MemoryLogRawAllocation(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // string operation = 2; + void clear_operation(); + static const int kOperationFieldNumber = 2; + const ::std::string& operation() const; + void set_operation(const ::std::string& value); + #if LANG_CXX11 + void set_operation(::std::string&& value); + #endif + void set_operation(const char* value); + void set_operation(const char* value, size_t size); + ::std::string* mutable_operation(); + ::std::string* release_operation(); + void set_allocated_operation(::std::string* operation); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_operation(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_operation( + ::std::string* operation); + + // string allocator_name = 6; + void clear_allocator_name(); + static const int kAllocatorNameFieldNumber = 6; + const ::std::string& allocator_name() const; + void set_allocator_name(const ::std::string& value); + #if LANG_CXX11 + void set_allocator_name(::std::string&& value); + #endif + void set_allocator_name(const char* value); + void set_allocator_name(const char* value, size_t size); + ::std::string* mutable_allocator_name(); + ::std::string* release_allocator_name(); + void set_allocated_allocator_name(::std::string* allocator_name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_allocator_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_allocator_name( + ::std::string* allocator_name); + + // int64 step_id = 1; + void clear_step_id(); + static const int kStepIdFieldNumber = 1; + ::google::protobuf::int64 step_id() const; + void set_step_id(::google::protobuf::int64 value); + + // int64 num_bytes = 3; + void clear_num_bytes(); + static const int kNumBytesFieldNumber = 3; + ::google::protobuf::int64 num_bytes() const; + void set_num_bytes(::google::protobuf::int64 value); + + // uint64 ptr = 4; + void clear_ptr(); + static const int kPtrFieldNumber = 4; + ::google::protobuf::uint64 ptr() const; + void set_ptr(::google::protobuf::uint64 value); + + // int64 allocation_id = 5; + void clear_allocation_id(); + static const int kAllocationIdFieldNumber = 5; + ::google::protobuf::int64 allocation_id() const; + void set_allocation_id(::google::protobuf::int64 value); + + // @@protoc_insertion_point(class_scope:tensorflow.MemoryLogRawAllocation) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::internal::ArenaStringPtr operation_; + ::google::protobuf::internal::ArenaStringPtr allocator_name_; + ::google::protobuf::int64 step_id_; + ::google::protobuf::int64 num_bytes_; + ::google::protobuf::uint64 ptr_; + ::google::protobuf::int64 allocation_id_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2flog_5fmemory_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class MemoryLogRawDeallocation : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.MemoryLogRawDeallocation) */ { + public: + MemoryLogRawDeallocation(); + virtual ~MemoryLogRawDeallocation(); + + MemoryLogRawDeallocation(const MemoryLogRawDeallocation& from); + + inline MemoryLogRawDeallocation& operator=(const MemoryLogRawDeallocation& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + MemoryLogRawDeallocation(MemoryLogRawDeallocation&& from) noexcept + : MemoryLogRawDeallocation() { + *this = ::std::move(from); + } + + inline MemoryLogRawDeallocation& operator=(MemoryLogRawDeallocation&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const MemoryLogRawDeallocation& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const MemoryLogRawDeallocation* internal_default_instance() { + return reinterpret_cast( + &_MemoryLogRawDeallocation_default_instance_); + } + static constexpr int kIndexInFileMessages = + 5; + + void UnsafeArenaSwap(MemoryLogRawDeallocation* other); + void Swap(MemoryLogRawDeallocation* other); + friend void swap(MemoryLogRawDeallocation& a, MemoryLogRawDeallocation& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline MemoryLogRawDeallocation* New() const final { + return CreateMaybeMessage(NULL); + } + + MemoryLogRawDeallocation* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const MemoryLogRawDeallocation& from); + void MergeFrom(const MemoryLogRawDeallocation& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(MemoryLogRawDeallocation* other); + protected: + explicit MemoryLogRawDeallocation(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // string operation = 2; + void clear_operation(); + static const int kOperationFieldNumber = 2; + const ::std::string& operation() const; + void set_operation(const ::std::string& value); + #if LANG_CXX11 + void set_operation(::std::string&& value); + #endif + void set_operation(const char* value); + void set_operation(const char* value, size_t size); + ::std::string* mutable_operation(); + ::std::string* release_operation(); + void set_allocated_operation(::std::string* operation); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_operation(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_operation( + ::std::string* operation); + + // string allocator_name = 4; + void clear_allocator_name(); + static const int kAllocatorNameFieldNumber = 4; + const ::std::string& allocator_name() const; + void set_allocator_name(const ::std::string& value); + #if LANG_CXX11 + void set_allocator_name(::std::string&& value); + #endif + void set_allocator_name(const char* value); + void set_allocator_name(const char* value, size_t size); + ::std::string* mutable_allocator_name(); + ::std::string* release_allocator_name(); + void set_allocated_allocator_name(::std::string* allocator_name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_allocator_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_allocator_name( + ::std::string* allocator_name); + + // int64 step_id = 1; + void clear_step_id(); + static const int kStepIdFieldNumber = 1; + ::google::protobuf::int64 step_id() const; + void set_step_id(::google::protobuf::int64 value); + + // int64 allocation_id = 3; + void clear_allocation_id(); + static const int kAllocationIdFieldNumber = 3; + ::google::protobuf::int64 allocation_id() const; + void set_allocation_id(::google::protobuf::int64 value); + + // bool deferred = 5; + void clear_deferred(); + static const int kDeferredFieldNumber = 5; + bool deferred() const; + void set_deferred(bool value); + + // @@protoc_insertion_point(class_scope:tensorflow.MemoryLogRawDeallocation) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::internal::ArenaStringPtr operation_; + ::google::protobuf::internal::ArenaStringPtr allocator_name_; + ::google::protobuf::int64 step_id_; + ::google::protobuf::int64 allocation_id_; + bool deferred_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2flog_5fmemory_2eproto::TableStruct; +}; +// =================================================================== + + +// =================================================================== + +#ifdef __GNUC__ + #pragma GCC diagnostic push + #pragma GCC diagnostic ignored "-Wstrict-aliasing" +#endif // __GNUC__ +// MemoryLogStep + +// int64 step_id = 1; +inline void MemoryLogStep::clear_step_id() { + step_id_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 MemoryLogStep::step_id() const { + // @@protoc_insertion_point(field_get:tensorflow.MemoryLogStep.step_id) + return step_id_; +} +inline void MemoryLogStep::set_step_id(::google::protobuf::int64 value) { + + step_id_ = value; + // @@protoc_insertion_point(field_set:tensorflow.MemoryLogStep.step_id) +} + +// string handle = 2; +inline void MemoryLogStep::clear_handle() { + handle_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& MemoryLogStep::handle() const { + // @@protoc_insertion_point(field_get:tensorflow.MemoryLogStep.handle) + return handle_.Get(); +} +inline void MemoryLogStep::set_handle(const ::std::string& value) { + + handle_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.MemoryLogStep.handle) +} +#if LANG_CXX11 +inline void MemoryLogStep::set_handle(::std::string&& value) { + + handle_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.MemoryLogStep.handle) +} +#endif +inline void MemoryLogStep::set_handle(const char* value) { + GOOGLE_DCHECK(value != NULL); + + handle_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.MemoryLogStep.handle) +} +inline void MemoryLogStep::set_handle(const char* value, + size_t size) { + + handle_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.MemoryLogStep.handle) +} +inline ::std::string* MemoryLogStep::mutable_handle() { + + // @@protoc_insertion_point(field_mutable:tensorflow.MemoryLogStep.handle) + return handle_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* MemoryLogStep::release_handle() { + // @@protoc_insertion_point(field_release:tensorflow.MemoryLogStep.handle) + + return handle_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void MemoryLogStep::set_allocated_handle(::std::string* handle) { + if (handle != NULL) { + + } else { + + } + handle_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), handle, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.MemoryLogStep.handle) +} +inline ::std::string* MemoryLogStep::unsafe_arena_release_handle() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.MemoryLogStep.handle) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return handle_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void MemoryLogStep::unsafe_arena_set_allocated_handle( + ::std::string* handle) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (handle != NULL) { + + } else { + + } + handle_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + handle, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.MemoryLogStep.handle) +} + +// ------------------------------------------------------------------- + +// MemoryLogTensorAllocation + +// int64 step_id = 1; +inline void MemoryLogTensorAllocation::clear_step_id() { + step_id_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 MemoryLogTensorAllocation::step_id() const { + // @@protoc_insertion_point(field_get:tensorflow.MemoryLogTensorAllocation.step_id) + return step_id_; +} +inline void MemoryLogTensorAllocation::set_step_id(::google::protobuf::int64 value) { + + step_id_ = value; + // @@protoc_insertion_point(field_set:tensorflow.MemoryLogTensorAllocation.step_id) +} + +// string kernel_name = 2; +inline void MemoryLogTensorAllocation::clear_kernel_name() { + kernel_name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& MemoryLogTensorAllocation::kernel_name() const { + // @@protoc_insertion_point(field_get:tensorflow.MemoryLogTensorAllocation.kernel_name) + return kernel_name_.Get(); +} +inline void MemoryLogTensorAllocation::set_kernel_name(const ::std::string& value) { + + kernel_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.MemoryLogTensorAllocation.kernel_name) +} +#if LANG_CXX11 +inline void MemoryLogTensorAllocation::set_kernel_name(::std::string&& value) { + + kernel_name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.MemoryLogTensorAllocation.kernel_name) +} +#endif +inline void MemoryLogTensorAllocation::set_kernel_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + kernel_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.MemoryLogTensorAllocation.kernel_name) +} +inline void MemoryLogTensorAllocation::set_kernel_name(const char* value, + size_t size) { + + kernel_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.MemoryLogTensorAllocation.kernel_name) +} +inline ::std::string* MemoryLogTensorAllocation::mutable_kernel_name() { + + // @@protoc_insertion_point(field_mutable:tensorflow.MemoryLogTensorAllocation.kernel_name) + return kernel_name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* MemoryLogTensorAllocation::release_kernel_name() { + // @@protoc_insertion_point(field_release:tensorflow.MemoryLogTensorAllocation.kernel_name) + + return kernel_name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void MemoryLogTensorAllocation::set_allocated_kernel_name(::std::string* kernel_name) { + if (kernel_name != NULL) { + + } else { + + } + kernel_name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), kernel_name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.MemoryLogTensorAllocation.kernel_name) +} +inline ::std::string* MemoryLogTensorAllocation::unsafe_arena_release_kernel_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.MemoryLogTensorAllocation.kernel_name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return kernel_name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void MemoryLogTensorAllocation::unsafe_arena_set_allocated_kernel_name( + ::std::string* kernel_name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (kernel_name != NULL) { + + } else { + + } + kernel_name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + kernel_name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.MemoryLogTensorAllocation.kernel_name) +} + +// .tensorflow.TensorDescription tensor = 3; +inline bool MemoryLogTensorAllocation::has_tensor() const { + return this != internal_default_instance() && tensor_ != NULL; +} +inline const ::tensorflow::TensorDescription& MemoryLogTensorAllocation::_internal_tensor() const { + return *tensor_; +} +inline const ::tensorflow::TensorDescription& MemoryLogTensorAllocation::tensor() const { + const ::tensorflow::TensorDescription* p = tensor_; + // @@protoc_insertion_point(field_get:tensorflow.MemoryLogTensorAllocation.tensor) + return p != NULL ? *p : *reinterpret_cast( + &::tensorflow::_TensorDescription_default_instance_); +} +inline ::tensorflow::TensorDescription* MemoryLogTensorAllocation::release_tensor() { + // @@protoc_insertion_point(field_release:tensorflow.MemoryLogTensorAllocation.tensor) + + ::tensorflow::TensorDescription* temp = tensor_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + tensor_ = NULL; + return temp; +} +inline ::tensorflow::TensorDescription* MemoryLogTensorAllocation::unsafe_arena_release_tensor() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.MemoryLogTensorAllocation.tensor) + + ::tensorflow::TensorDescription* temp = tensor_; + tensor_ = NULL; + return temp; +} +inline ::tensorflow::TensorDescription* MemoryLogTensorAllocation::mutable_tensor() { + + if (tensor_ == NULL) { + auto* p = CreateMaybeMessage<::tensorflow::TensorDescription>(GetArenaNoVirtual()); + tensor_ = p; + } + // @@protoc_insertion_point(field_mutable:tensorflow.MemoryLogTensorAllocation.tensor) + return tensor_; +} +inline void MemoryLogTensorAllocation::set_allocated_tensor(::tensorflow::TensorDescription* tensor) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(tensor_); + } + if (tensor) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(tensor)->GetArena(); + if (message_arena != submessage_arena) { + tensor = ::google::protobuf::internal::GetOwnedMessage( + message_arena, tensor, submessage_arena); + } + + } else { + + } + tensor_ = tensor; + // @@protoc_insertion_point(field_set_allocated:tensorflow.MemoryLogTensorAllocation.tensor) +} + +// ------------------------------------------------------------------- + +// MemoryLogTensorDeallocation + +// int64 allocation_id = 1; +inline void MemoryLogTensorDeallocation::clear_allocation_id() { + allocation_id_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 MemoryLogTensorDeallocation::allocation_id() const { + // @@protoc_insertion_point(field_get:tensorflow.MemoryLogTensorDeallocation.allocation_id) + return allocation_id_; +} +inline void MemoryLogTensorDeallocation::set_allocation_id(::google::protobuf::int64 value) { + + allocation_id_ = value; + // @@protoc_insertion_point(field_set:tensorflow.MemoryLogTensorDeallocation.allocation_id) +} + +// string allocator_name = 2; +inline void MemoryLogTensorDeallocation::clear_allocator_name() { + allocator_name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& MemoryLogTensorDeallocation::allocator_name() const { + // @@protoc_insertion_point(field_get:tensorflow.MemoryLogTensorDeallocation.allocator_name) + return allocator_name_.Get(); +} +inline void MemoryLogTensorDeallocation::set_allocator_name(const ::std::string& value) { + + allocator_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.MemoryLogTensorDeallocation.allocator_name) +} +#if LANG_CXX11 +inline void MemoryLogTensorDeallocation::set_allocator_name(::std::string&& value) { + + allocator_name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.MemoryLogTensorDeallocation.allocator_name) +} +#endif +inline void MemoryLogTensorDeallocation::set_allocator_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + allocator_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.MemoryLogTensorDeallocation.allocator_name) +} +inline void MemoryLogTensorDeallocation::set_allocator_name(const char* value, + size_t size) { + + allocator_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.MemoryLogTensorDeallocation.allocator_name) +} +inline ::std::string* MemoryLogTensorDeallocation::mutable_allocator_name() { + + // @@protoc_insertion_point(field_mutable:tensorflow.MemoryLogTensorDeallocation.allocator_name) + return allocator_name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* MemoryLogTensorDeallocation::release_allocator_name() { + // @@protoc_insertion_point(field_release:tensorflow.MemoryLogTensorDeallocation.allocator_name) + + return allocator_name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void MemoryLogTensorDeallocation::set_allocated_allocator_name(::std::string* allocator_name) { + if (allocator_name != NULL) { + + } else { + + } + allocator_name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), allocator_name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.MemoryLogTensorDeallocation.allocator_name) +} +inline ::std::string* MemoryLogTensorDeallocation::unsafe_arena_release_allocator_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.MemoryLogTensorDeallocation.allocator_name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return allocator_name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void MemoryLogTensorDeallocation::unsafe_arena_set_allocated_allocator_name( + ::std::string* allocator_name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (allocator_name != NULL) { + + } else { + + } + allocator_name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + allocator_name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.MemoryLogTensorDeallocation.allocator_name) +} + +// ------------------------------------------------------------------- + +// MemoryLogTensorOutput + +// int64 step_id = 1; +inline void MemoryLogTensorOutput::clear_step_id() { + step_id_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 MemoryLogTensorOutput::step_id() const { + // @@protoc_insertion_point(field_get:tensorflow.MemoryLogTensorOutput.step_id) + return step_id_; +} +inline void MemoryLogTensorOutput::set_step_id(::google::protobuf::int64 value) { + + step_id_ = value; + // @@protoc_insertion_point(field_set:tensorflow.MemoryLogTensorOutput.step_id) +} + +// string kernel_name = 2; +inline void MemoryLogTensorOutput::clear_kernel_name() { + kernel_name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& MemoryLogTensorOutput::kernel_name() const { + // @@protoc_insertion_point(field_get:tensorflow.MemoryLogTensorOutput.kernel_name) + return kernel_name_.Get(); +} +inline void MemoryLogTensorOutput::set_kernel_name(const ::std::string& value) { + + kernel_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.MemoryLogTensorOutput.kernel_name) +} +#if LANG_CXX11 +inline void MemoryLogTensorOutput::set_kernel_name(::std::string&& value) { + + kernel_name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.MemoryLogTensorOutput.kernel_name) +} +#endif +inline void MemoryLogTensorOutput::set_kernel_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + kernel_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.MemoryLogTensorOutput.kernel_name) +} +inline void MemoryLogTensorOutput::set_kernel_name(const char* value, + size_t size) { + + kernel_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.MemoryLogTensorOutput.kernel_name) +} +inline ::std::string* MemoryLogTensorOutput::mutable_kernel_name() { + + // @@protoc_insertion_point(field_mutable:tensorflow.MemoryLogTensorOutput.kernel_name) + return kernel_name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* MemoryLogTensorOutput::release_kernel_name() { + // @@protoc_insertion_point(field_release:tensorflow.MemoryLogTensorOutput.kernel_name) + + return kernel_name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void MemoryLogTensorOutput::set_allocated_kernel_name(::std::string* kernel_name) { + if (kernel_name != NULL) { + + } else { + + } + kernel_name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), kernel_name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.MemoryLogTensorOutput.kernel_name) +} +inline ::std::string* MemoryLogTensorOutput::unsafe_arena_release_kernel_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.MemoryLogTensorOutput.kernel_name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return kernel_name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void MemoryLogTensorOutput::unsafe_arena_set_allocated_kernel_name( + ::std::string* kernel_name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (kernel_name != NULL) { + + } else { + + } + kernel_name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + kernel_name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.MemoryLogTensorOutput.kernel_name) +} + +// int32 index = 3; +inline void MemoryLogTensorOutput::clear_index() { + index_ = 0; +} +inline ::google::protobuf::int32 MemoryLogTensorOutput::index() const { + // @@protoc_insertion_point(field_get:tensorflow.MemoryLogTensorOutput.index) + return index_; +} +inline void MemoryLogTensorOutput::set_index(::google::protobuf::int32 value) { + + index_ = value; + // @@protoc_insertion_point(field_set:tensorflow.MemoryLogTensorOutput.index) +} + +// .tensorflow.TensorDescription tensor = 4; +inline bool MemoryLogTensorOutput::has_tensor() const { + return this != internal_default_instance() && tensor_ != NULL; +} +inline const ::tensorflow::TensorDescription& MemoryLogTensorOutput::_internal_tensor() const { + return *tensor_; +} +inline const ::tensorflow::TensorDescription& MemoryLogTensorOutput::tensor() const { + const ::tensorflow::TensorDescription* p = tensor_; + // @@protoc_insertion_point(field_get:tensorflow.MemoryLogTensorOutput.tensor) + return p != NULL ? *p : *reinterpret_cast( + &::tensorflow::_TensorDescription_default_instance_); +} +inline ::tensorflow::TensorDescription* MemoryLogTensorOutput::release_tensor() { + // @@protoc_insertion_point(field_release:tensorflow.MemoryLogTensorOutput.tensor) + + ::tensorflow::TensorDescription* temp = tensor_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + tensor_ = NULL; + return temp; +} +inline ::tensorflow::TensorDescription* MemoryLogTensorOutput::unsafe_arena_release_tensor() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.MemoryLogTensorOutput.tensor) + + ::tensorflow::TensorDescription* temp = tensor_; + tensor_ = NULL; + return temp; +} +inline ::tensorflow::TensorDescription* MemoryLogTensorOutput::mutable_tensor() { + + if (tensor_ == NULL) { + auto* p = CreateMaybeMessage<::tensorflow::TensorDescription>(GetArenaNoVirtual()); + tensor_ = p; + } + // @@protoc_insertion_point(field_mutable:tensorflow.MemoryLogTensorOutput.tensor) + return tensor_; +} +inline void MemoryLogTensorOutput::set_allocated_tensor(::tensorflow::TensorDescription* tensor) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(tensor_); + } + if (tensor) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(tensor)->GetArena(); + if (message_arena != submessage_arena) { + tensor = ::google::protobuf::internal::GetOwnedMessage( + message_arena, tensor, submessage_arena); + } + + } else { + + } + tensor_ = tensor; + // @@protoc_insertion_point(field_set_allocated:tensorflow.MemoryLogTensorOutput.tensor) +} + +// ------------------------------------------------------------------- + +// MemoryLogRawAllocation + +// int64 step_id = 1; +inline void MemoryLogRawAllocation::clear_step_id() { + step_id_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 MemoryLogRawAllocation::step_id() const { + // @@protoc_insertion_point(field_get:tensorflow.MemoryLogRawAllocation.step_id) + return step_id_; +} +inline void MemoryLogRawAllocation::set_step_id(::google::protobuf::int64 value) { + + step_id_ = value; + // @@protoc_insertion_point(field_set:tensorflow.MemoryLogRawAllocation.step_id) +} + +// string operation = 2; +inline void MemoryLogRawAllocation::clear_operation() { + operation_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& MemoryLogRawAllocation::operation() const { + // @@protoc_insertion_point(field_get:tensorflow.MemoryLogRawAllocation.operation) + return operation_.Get(); +} +inline void MemoryLogRawAllocation::set_operation(const ::std::string& value) { + + operation_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.MemoryLogRawAllocation.operation) +} +#if LANG_CXX11 +inline void MemoryLogRawAllocation::set_operation(::std::string&& value) { + + operation_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.MemoryLogRawAllocation.operation) +} +#endif +inline void MemoryLogRawAllocation::set_operation(const char* value) { + GOOGLE_DCHECK(value != NULL); + + operation_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.MemoryLogRawAllocation.operation) +} +inline void MemoryLogRawAllocation::set_operation(const char* value, + size_t size) { + + operation_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.MemoryLogRawAllocation.operation) +} +inline ::std::string* MemoryLogRawAllocation::mutable_operation() { + + // @@protoc_insertion_point(field_mutable:tensorflow.MemoryLogRawAllocation.operation) + return operation_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* MemoryLogRawAllocation::release_operation() { + // @@protoc_insertion_point(field_release:tensorflow.MemoryLogRawAllocation.operation) + + return operation_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void MemoryLogRawAllocation::set_allocated_operation(::std::string* operation) { + if (operation != NULL) { + + } else { + + } + operation_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), operation, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.MemoryLogRawAllocation.operation) +} +inline ::std::string* MemoryLogRawAllocation::unsafe_arena_release_operation() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.MemoryLogRawAllocation.operation) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return operation_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void MemoryLogRawAllocation::unsafe_arena_set_allocated_operation( + ::std::string* operation) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (operation != NULL) { + + } else { + + } + operation_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + operation, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.MemoryLogRawAllocation.operation) +} + +// int64 num_bytes = 3; +inline void MemoryLogRawAllocation::clear_num_bytes() { + num_bytes_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 MemoryLogRawAllocation::num_bytes() const { + // @@protoc_insertion_point(field_get:tensorflow.MemoryLogRawAllocation.num_bytes) + return num_bytes_; +} +inline void MemoryLogRawAllocation::set_num_bytes(::google::protobuf::int64 value) { + + num_bytes_ = value; + // @@protoc_insertion_point(field_set:tensorflow.MemoryLogRawAllocation.num_bytes) +} + +// uint64 ptr = 4; +inline void MemoryLogRawAllocation::clear_ptr() { + ptr_ = GOOGLE_ULONGLONG(0); +} +inline ::google::protobuf::uint64 MemoryLogRawAllocation::ptr() const { + // @@protoc_insertion_point(field_get:tensorflow.MemoryLogRawAllocation.ptr) + return ptr_; +} +inline void MemoryLogRawAllocation::set_ptr(::google::protobuf::uint64 value) { + + ptr_ = value; + // @@protoc_insertion_point(field_set:tensorflow.MemoryLogRawAllocation.ptr) +} + +// int64 allocation_id = 5; +inline void MemoryLogRawAllocation::clear_allocation_id() { + allocation_id_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 MemoryLogRawAllocation::allocation_id() const { + // @@protoc_insertion_point(field_get:tensorflow.MemoryLogRawAllocation.allocation_id) + return allocation_id_; +} +inline void MemoryLogRawAllocation::set_allocation_id(::google::protobuf::int64 value) { + + allocation_id_ = value; + // @@protoc_insertion_point(field_set:tensorflow.MemoryLogRawAllocation.allocation_id) +} + +// string allocator_name = 6; +inline void MemoryLogRawAllocation::clear_allocator_name() { + allocator_name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& MemoryLogRawAllocation::allocator_name() const { + // @@protoc_insertion_point(field_get:tensorflow.MemoryLogRawAllocation.allocator_name) + return allocator_name_.Get(); +} +inline void MemoryLogRawAllocation::set_allocator_name(const ::std::string& value) { + + allocator_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.MemoryLogRawAllocation.allocator_name) +} +#if LANG_CXX11 +inline void MemoryLogRawAllocation::set_allocator_name(::std::string&& value) { + + allocator_name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.MemoryLogRawAllocation.allocator_name) +} +#endif +inline void MemoryLogRawAllocation::set_allocator_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + allocator_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.MemoryLogRawAllocation.allocator_name) +} +inline void MemoryLogRawAllocation::set_allocator_name(const char* value, + size_t size) { + + allocator_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.MemoryLogRawAllocation.allocator_name) +} +inline ::std::string* MemoryLogRawAllocation::mutable_allocator_name() { + + // @@protoc_insertion_point(field_mutable:tensorflow.MemoryLogRawAllocation.allocator_name) + return allocator_name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* MemoryLogRawAllocation::release_allocator_name() { + // @@protoc_insertion_point(field_release:tensorflow.MemoryLogRawAllocation.allocator_name) + + return allocator_name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void MemoryLogRawAllocation::set_allocated_allocator_name(::std::string* allocator_name) { + if (allocator_name != NULL) { + + } else { + + } + allocator_name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), allocator_name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.MemoryLogRawAllocation.allocator_name) +} +inline ::std::string* MemoryLogRawAllocation::unsafe_arena_release_allocator_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.MemoryLogRawAllocation.allocator_name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return allocator_name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void MemoryLogRawAllocation::unsafe_arena_set_allocated_allocator_name( + ::std::string* allocator_name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (allocator_name != NULL) { + + } else { + + } + allocator_name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + allocator_name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.MemoryLogRawAllocation.allocator_name) +} + +// ------------------------------------------------------------------- + +// MemoryLogRawDeallocation + +// int64 step_id = 1; +inline void MemoryLogRawDeallocation::clear_step_id() { + step_id_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 MemoryLogRawDeallocation::step_id() const { + // @@protoc_insertion_point(field_get:tensorflow.MemoryLogRawDeallocation.step_id) + return step_id_; +} +inline void MemoryLogRawDeallocation::set_step_id(::google::protobuf::int64 value) { + + step_id_ = value; + // @@protoc_insertion_point(field_set:tensorflow.MemoryLogRawDeallocation.step_id) +} + +// string operation = 2; +inline void MemoryLogRawDeallocation::clear_operation() { + operation_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& MemoryLogRawDeallocation::operation() const { + // @@protoc_insertion_point(field_get:tensorflow.MemoryLogRawDeallocation.operation) + return operation_.Get(); +} +inline void MemoryLogRawDeallocation::set_operation(const ::std::string& value) { + + operation_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.MemoryLogRawDeallocation.operation) +} +#if LANG_CXX11 +inline void MemoryLogRawDeallocation::set_operation(::std::string&& value) { + + operation_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.MemoryLogRawDeallocation.operation) +} +#endif +inline void MemoryLogRawDeallocation::set_operation(const char* value) { + GOOGLE_DCHECK(value != NULL); + + operation_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.MemoryLogRawDeallocation.operation) +} +inline void MemoryLogRawDeallocation::set_operation(const char* value, + size_t size) { + + operation_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.MemoryLogRawDeallocation.operation) +} +inline ::std::string* MemoryLogRawDeallocation::mutable_operation() { + + // @@protoc_insertion_point(field_mutable:tensorflow.MemoryLogRawDeallocation.operation) + return operation_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* MemoryLogRawDeallocation::release_operation() { + // @@protoc_insertion_point(field_release:tensorflow.MemoryLogRawDeallocation.operation) + + return operation_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void MemoryLogRawDeallocation::set_allocated_operation(::std::string* operation) { + if (operation != NULL) { + + } else { + + } + operation_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), operation, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.MemoryLogRawDeallocation.operation) +} +inline ::std::string* MemoryLogRawDeallocation::unsafe_arena_release_operation() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.MemoryLogRawDeallocation.operation) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return operation_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void MemoryLogRawDeallocation::unsafe_arena_set_allocated_operation( + ::std::string* operation) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (operation != NULL) { + + } else { + + } + operation_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + operation, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.MemoryLogRawDeallocation.operation) +} + +// int64 allocation_id = 3; +inline void MemoryLogRawDeallocation::clear_allocation_id() { + allocation_id_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 MemoryLogRawDeallocation::allocation_id() const { + // @@protoc_insertion_point(field_get:tensorflow.MemoryLogRawDeallocation.allocation_id) + return allocation_id_; +} +inline void MemoryLogRawDeallocation::set_allocation_id(::google::protobuf::int64 value) { + + allocation_id_ = value; + // @@protoc_insertion_point(field_set:tensorflow.MemoryLogRawDeallocation.allocation_id) +} + +// string allocator_name = 4; +inline void MemoryLogRawDeallocation::clear_allocator_name() { + allocator_name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& MemoryLogRawDeallocation::allocator_name() const { + // @@protoc_insertion_point(field_get:tensorflow.MemoryLogRawDeallocation.allocator_name) + return allocator_name_.Get(); +} +inline void MemoryLogRawDeallocation::set_allocator_name(const ::std::string& value) { + + allocator_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.MemoryLogRawDeallocation.allocator_name) +} +#if LANG_CXX11 +inline void MemoryLogRawDeallocation::set_allocator_name(::std::string&& value) { + + allocator_name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.MemoryLogRawDeallocation.allocator_name) +} +#endif +inline void MemoryLogRawDeallocation::set_allocator_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + allocator_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.MemoryLogRawDeallocation.allocator_name) +} +inline void MemoryLogRawDeallocation::set_allocator_name(const char* value, + size_t size) { + + allocator_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.MemoryLogRawDeallocation.allocator_name) +} +inline ::std::string* MemoryLogRawDeallocation::mutable_allocator_name() { + + // @@protoc_insertion_point(field_mutable:tensorflow.MemoryLogRawDeallocation.allocator_name) + return allocator_name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* MemoryLogRawDeallocation::release_allocator_name() { + // @@protoc_insertion_point(field_release:tensorflow.MemoryLogRawDeallocation.allocator_name) + + return allocator_name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void MemoryLogRawDeallocation::set_allocated_allocator_name(::std::string* allocator_name) { + if (allocator_name != NULL) { + + } else { + + } + allocator_name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), allocator_name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.MemoryLogRawDeallocation.allocator_name) +} +inline ::std::string* MemoryLogRawDeallocation::unsafe_arena_release_allocator_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.MemoryLogRawDeallocation.allocator_name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return allocator_name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void MemoryLogRawDeallocation::unsafe_arena_set_allocated_allocator_name( + ::std::string* allocator_name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (allocator_name != NULL) { + + } else { + + } + allocator_name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + allocator_name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.MemoryLogRawDeallocation.allocator_name) +} + +// bool deferred = 5; +inline void MemoryLogRawDeallocation::clear_deferred() { + deferred_ = false; +} +inline bool MemoryLogRawDeallocation::deferred() const { + // @@protoc_insertion_point(field_get:tensorflow.MemoryLogRawDeallocation.deferred) + return deferred_; +} +inline void MemoryLogRawDeallocation::set_deferred(bool value) { + + deferred_ = value; + // @@protoc_insertion_point(field_set:tensorflow.MemoryLogRawDeallocation.deferred) +} + +#ifdef __GNUC__ + #pragma GCC diagnostic pop +#endif // __GNUC__ +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + + +// @@protoc_insertion_point(namespace_scope) + +} // namespace tensorflow + +// @@protoc_insertion_point(global_scope) + +#endif // PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2flog_5fmemory_2eproto diff --git a/linux/include/tensorflow/core/framework/log_memory.pb_text-impl.h b/linux/include/tensorflow/core/framework/log_memory.pb_text-impl.h new file mode 100644 index 000000000..0df28aae1 --- /dev/null +++ b/linux/include/tensorflow/core/framework/log_memory.pb_text-impl.h @@ -0,0 +1,68 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_framework_log_memory_proto_IMPL_H_ +#define tensorflow_core_framework_log_memory_proto_IMPL_H_ + +#include "tensorflow/core/framework/allocation_description.pb.h" +#include "tensorflow/core/framework/allocation_description.pb_text-impl.h" +#include "tensorflow/core/framework/log_memory.pb.h" +#include "tensorflow/core/framework/log_memory.pb_text.h" +#include "tensorflow/core/framework/tensor_description.pb.h" +#include "tensorflow/core/framework/tensor_description.pb_text-impl.h" +#include "tensorflow/core/framework/tensor_shape.pb.h" +#include "tensorflow/core/framework/tensor_shape.pb_text-impl.h" +#include "tensorflow/core/framework/types.pb.h" +#include "tensorflow/core/framework/types.pb_text-impl.h" +#include "tensorflow/core/lib/strings/proto_text_util.h" +#include "tensorflow/core/lib/strings/scanner.h" + +namespace tensorflow { + +namespace internal { + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::MemoryLogStep& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::MemoryLogStep* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::MemoryLogTensorAllocation& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::MemoryLogTensorAllocation* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::MemoryLogTensorDeallocation& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::MemoryLogTensorDeallocation* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::MemoryLogTensorOutput& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::MemoryLogTensorOutput* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::MemoryLogRawAllocation& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::MemoryLogRawAllocation* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::MemoryLogRawDeallocation& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::MemoryLogRawDeallocation* msg); + +} // namespace internal + +} // namespace tensorflow + +#endif // tensorflow_core_framework_log_memory_proto_IMPL_H_ diff --git a/linux/include/tensorflow/core/framework/log_memory.pb_text.h b/linux/include/tensorflow/core/framework/log_memory.pb_text.h new file mode 100644 index 000000000..b5c2f47e5 --- /dev/null +++ b/linux/include/tensorflow/core/framework/log_memory.pb_text.h @@ -0,0 +1,74 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_framework_log_memory_proto_H_ +#define tensorflow_core_framework_log_memory_proto_H_ + +#include "tensorflow/core/framework/log_memory.pb.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/protobuf.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +// Message-text conversion for tensorflow.MemoryLogStep +string ProtoDebugString( + const ::tensorflow::MemoryLogStep& msg); +string ProtoShortDebugString( + const ::tensorflow::MemoryLogStep& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::MemoryLogStep* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.MemoryLogTensorAllocation +string ProtoDebugString( + const ::tensorflow::MemoryLogTensorAllocation& msg); +string ProtoShortDebugString( + const ::tensorflow::MemoryLogTensorAllocation& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::MemoryLogTensorAllocation* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.MemoryLogTensorDeallocation +string ProtoDebugString( + const ::tensorflow::MemoryLogTensorDeallocation& msg); +string ProtoShortDebugString( + const ::tensorflow::MemoryLogTensorDeallocation& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::MemoryLogTensorDeallocation* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.MemoryLogTensorOutput +string ProtoDebugString( + const ::tensorflow::MemoryLogTensorOutput& msg); +string ProtoShortDebugString( + const ::tensorflow::MemoryLogTensorOutput& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::MemoryLogTensorOutput* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.MemoryLogRawAllocation +string ProtoDebugString( + const ::tensorflow::MemoryLogRawAllocation& msg); +string ProtoShortDebugString( + const ::tensorflow::MemoryLogRawAllocation& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::MemoryLogRawAllocation* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.MemoryLogRawDeallocation +string ProtoDebugString( + const ::tensorflow::MemoryLogRawDeallocation& msg); +string ProtoShortDebugString( + const ::tensorflow::MemoryLogRawDeallocation& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::MemoryLogRawDeallocation* msg) + TF_MUST_USE_RESULT; + +} // namespace tensorflow + +#endif // tensorflow_core_framework_log_memory_proto_H_ diff --git a/linux/include/tensorflow/core/framework/lookup_interface.h b/linux/include/tensorflow/core/framework/lookup_interface.h new file mode 100644 index 000000000..0622dd06c --- /dev/null +++ b/linux/include/tensorflow/core/framework/lookup_interface.h @@ -0,0 +1,145 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_FRAMEWORK_LOOKUP_INTERFACE_H_ +#define TENSORFLOW_CORE_FRAMEWORK_LOOKUP_INTERFACE_H_ + +#include "tensorflow/core/framework/resource_mgr.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/lib/core/status.h" + +namespace tensorflow { + +class OpKernelContext; + +namespace lookup { + +// Forward declaration so we can define GetInitializableLookupTable() in +// LookupInterface. +class InitializableLookupTable; + +// Lookup interface for batch lookups used by table lookup ops. +class LookupInterface : public ResourceBase { + public: + // Performs batch lookups, for every element in the key tensor, Find returns + // the corresponding value into the values tensor. + // If an element is not present in the table, the given default value is used. + + // For tables that require initialization, Find is available once the table + // is marked as initialized. + + // Returns the following statuses: + // - OK: when the find finishes successfully. + // - FailedPrecondition: if the table is not initialized. + // - InvalidArgument: if any of the preconditions on the lookup key or value + // fails. + // - In addition, other implementations may provide another non-OK status + // specific to their failure modes. + virtual Status Find(OpKernelContext* ctx, const Tensor& keys, Tensor* values, + const Tensor& default_value) = 0; + + // Inserts elements into the table. Each element of the key tensor is + // associated with the corresponding element in the value tensor. + // This method is only implemented in mutable tables that can be updated over + // the execution of the graph. It returns Status::NotImplemented for read-only + // tables that are initialized once before they can be looked up. + + // Returns the following statuses: + // - OK: when the insert finishes successfully. + // - InvalidArgument: if any of the preconditions on the lookup key or value + // fails. + // - Unimplemented: if the table does not support insertions. + virtual Status Insert(OpKernelContext* ctx, const Tensor& keys, + const Tensor& values) = 0; + + // Returns the number of elements in the table. + virtual size_t size() const = 0; + + // Exports the values of the table to two tensors named keys and values. + // Note that the shape of the tensors is completely up to the implementation + // of the table and can be different than the tensors used for the Insert + // function above. + virtual Status ExportValues(OpKernelContext* ctx) = 0; + + // Imports previously exported keys and values. + // As mentioned above, the shape of the keys and values tensors are determined + // by the ExportValues function above and can be different than for the + // Insert function. + virtual Status ImportValues(OpKernelContext* ctx, const Tensor& keys, + const Tensor& values) = 0; + + // Returns the data type of the key. + virtual DataType key_dtype() const = 0; + + // Returns the data type of the value. + virtual DataType value_dtype() const = 0; + + // Returns the shape of a key in the table. + virtual TensorShape key_shape() const = 0; + + // Returns the shape of a value in the table. + virtual TensorShape value_shape() const = 0; + + // Check format of the key and value tensors for the Insert function. + // Returns OK if all the following requirements are satisfied, otherwise it + // returns InvalidArgument: + // - DataType of the tensor keys equals to the table key_dtype + // - DataType of the tensor values equals to the table value_dtype + // - the values tensor has the required shape given keys and the tables's + // value shape. + virtual Status CheckKeyAndValueTensorsForInsert(const Tensor& keys, + const Tensor& values); + + // Similar to the function above but instead checks eligibility for the Import + // function. + virtual Status CheckKeyAndValueTensorsForImport(const Tensor& keys, + const Tensor& values); + + // Check the arguments of a find operation. Returns OK if all the following + // requirements are satisfied, otherwise it returns InvalidArgument: + // - DataType of the tensor keys equals to the table key_dtype + // - DataType of the tensor default_value equals to the table value_dtype + // - the default_value tensor shape matches the table's value shape. + Status CheckFindArguments(const Tensor& keys, const Tensor& default_value); + + string DebugString() override { + return strings::StrCat("A lookup table of size: ", size()); + } + + // Returns an InitializableLookupTable, a subclass of LookupInterface, if the + // current object is an InitializableLookupTable. Otherwise, returns nullptr. + virtual InitializableLookupTable* GetInitializableLookupTable() { + return nullptr; + } + + protected: + virtual ~LookupInterface() = default; + + // Makes sure that the key and value tensor DataType's match the table + // key_dtype and value_dtype. + Status CheckKeyAndValueTypes(const Tensor& keys, const Tensor& values); + + // Makes sure that the provided shape is consistent with the table keys shape. + Status CheckKeyShape(const TensorShape& shape); + + private: + Status CheckKeyAndValueTensorsHelper(const Tensor& keys, + const Tensor& values); +}; + +} // namespace lookup +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_FRAMEWORK_LOOKUP_INTERFACE_H_ diff --git a/linux/include/tensorflow/core/framework/memory_types.h b/linux/include/tensorflow/core/framework/memory_types.h new file mode 100644 index 000000000..f719131bc --- /dev/null +++ b/linux/include/tensorflow/core/framework/memory_types.h @@ -0,0 +1,38 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_FRAMEWORK_MEMORY_TYPES_H_ +#define TENSORFLOW_CORE_FRAMEWORK_MEMORY_TYPES_H_ + +#include "tensorflow/core/framework/op.h" +#include "tensorflow/core/framework/types.h" + +namespace tensorflow { + +class NodeDef; + +// Returns into *{input,output}_memory_types the memory type of each +// {input,output} tensor. +// +// REQUIRES: * '*_memory_types' is not nullptr. +// * def has all attrs specified (e.g. using AddDefaultsToNodeDef()). +Status MemoryTypesForNode(const OpRegistryInterface* op_registry, + const DeviceType& device_type, const NodeDef& ndef, + MemoryTypeVector* input_memory_types, + MemoryTypeVector* output_memory_types); + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_FRAMEWORK_MEMORY_TYPES_H_ diff --git a/linux/include/tensorflow/core/framework/model.h b/linux/include/tensorflow/core/framework/model.h new file mode 100644 index 000000000..eae0fa70e --- /dev/null +++ b/linux/include/tensorflow/core/framework/model.h @@ -0,0 +1,408 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CORE_FRAMEWORK_MODEL_H_ +#define TENSORFLOW_CORE_FRAMEWORK_MODEL_H_ + +#include +#include +#include +#include // (b/114492873): move this include into core/platform +#include +#include + +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/gtl/cleanup.h" +#include "tensorflow/core/lib/gtl/map_util.h" +#include "tensorflow/core/lib/random/random.h" +#include "tensorflow/core/platform/cpu_info.h" +#include "tensorflow/core/platform/env.h" + +namespace tensorflow { +namespace data { +namespace model { + +// Represents thread-safe state that can be shared between an input pipeline and +// the performance model. +struct SharedState { + public: + explicit SharedState(int64 value, std::shared_ptr mu, + std::shared_ptr cond_var) + : value(value), mu(std::move(mu)), cond_var(std::move(cond_var)) {} + + std::shared_ptr mu; + std::shared_ptr cond_var; + int64 value; +}; + +// Abstract representation of a TensorFlow input pipeline that can be used +// for collecting runtime information and optimizing performance. It collects +// runtime information about execution of the input pipeline that is used to +// create a performance model, which is in turn used to identify optimal values +// of tunable parameters. +// +// Developers of tf.data transformations are not expected to interact with this +// class directly. Boiler plate code for creating the abstract representation of +// the input pipeline and collecting runtime information has been added to the +// implementation of `DatasetBase` and `DatasetBaseIterator` respectively. +class Model { + public: + Model() = default; + + // Adds a constant parameter for the given node. + void AddConstantParameter(const string& node_name, + const string& parameter_name, int64 value) + LOCKS_EXCLUDED(mu_); + + // Adds a node with the given name and given output (identified by name). + void AddNode(const string& name, const string& output_name) + LOCKS_EXCLUDED(mu_); + + // Increments the processing time for the given node.. + void AddProcessingTime(const string& name, int64 delta) LOCKS_EXCLUDED(mu_); + + // Adds a tunable parameter for the given node. + void AddTunableParameter(const string& node_name, + const string& parameter_name, + std::shared_ptr value, int64 min, + int64 max) LOCKS_EXCLUDED(mu_); + + // Runs optimization. + void Optimize(int64 cpu_budget) LOCKS_EXCLUDED(mu_); + + // Records that a node has produced an element. + void RecordElement(const string& name) LOCKS_EXCLUDED(mu_); + + // Records that the given node has started work. If `stop_output` is set, it + // also records that the output of the given node has stopped work. + void RecordStart(const string& name, bool stop_output) LOCKS_EXCLUDED(mu_); + + // Records that the given node has stopped work. If `stop_output` is set, it + // also records that the output of the given node has started work. + void RecordStop(const string& name, bool start_output) LOCKS_EXCLUDED(mu_); + + // Removes the given node. + void RemoveNode(const string& name) LOCKS_EXCLUDED(mu_); + + private: + // Abstract representation of a TensorFlow input pipeline node. It collects + // information about inputs to this node, processing time spent executing the + // node logic, number of elements produced by the node, various other + // information (e.g. batch size or execution parallelism). + // + // Developers of tf.data transformations are not expected to interact with + // this class directly. Boiler plate code for creating the abstract + // representation of the input pipeline and collecting common information has + // been added to the implementation of `DatasetBase` and `DatasetBaseIterator` + // respectively. + // + // In addition, `DatasetBaseIterator` provides wrappers that can be used for + // transformation-specific information collection. The `SetMetadata` wrapper + // can be used to pass arbitrary metadata to the modeling framework, while the + // `StartWork` and `StopWork` wrappers should be used to correctly account for + // processing time of multi-threaded transformation that yield the CPU; such + // transformations should invoke `StartWork()` when a transformation thread + // starts executing (e.g. when created or woken up) and `StopWork()` when a + // transformation thread stops executing (e.g. when returning or waiting). + // + // TODO(jsimsa): Create an API to capture the abstract semantics of each + // tf.data transformation and replace switch-case blocks with inheritance. + class Node { + public: + // Represents a tunable parameter. + struct Tunable { + Tunable(std::shared_ptr state, int64 min, int64 max) + : value(state->value), min(min), max(max), state(std::move(state)) {} + + // Identifies the model value of the parameter. This can be different from + // the actual value (e.g. during optimization search). + int64 value; + + // Identifies the minimum value of the parameter. + int64 min; + + // Identifies the maximum value of the parameter. + int64 max; + + // Shared state of the parameter. + std::shared_ptr state; + }; + + Node(int64 id, const string& name, std::shared_ptr output) + : id_(id), name_(name), type_(TypeFromName(name)), output_(output) {} + + // Adds a constant parameter. + void add_constant_param(const string& name, int64 value) + LOCKS_EXCLUDED(mu_) { + mutex_lock l(mu_); + constant_params_[name] = value; + } + + // Adds an input. + void add_input(std::shared_ptr node) LOCKS_EXCLUDED(mu_) { + mutex_lock l(mu_); + inputs_.push_back(node); + } + + // Increments the aggregate processing time by the given delta. + void add_processing_time(int64 delta) LOCKS_EXCLUDED(mu_) { + mutex_lock l(mu_); + processing_time_ += delta; + } + + // Adds a tunable parameter. + void add_tunable_param(const string& name, + std::shared_ptr state, int64 min, + int64 max) LOCKS_EXCLUDED(mu_) { + mutex_lock l(mu_); + tunable_params_[name] = + std::make_shared(std::move(state), min, max); + } + + // Returns the unique node ID. + int64 id() LOCKS_EXCLUDED(mu_) { return id_; } + + // Returns the node inputs. + std::list> inputs() LOCKS_EXCLUDED(mu_) { + tf_shared_lock l(mu_); + return inputs_; + } + + // Returns the node name. + const string& name() LOCKS_EXCLUDED(mu_) { + tf_shared_lock l(mu_); + return name_; + } + + // Returns the number of elements produced by the node. + int64 num_elements() LOCKS_EXCLUDED(mu_) { + tf_shared_lock l(mu_); + return num_elements_; + } + + // Returns the node output. + std::shared_ptr output() LOCKS_EXCLUDED(mu_) { + tf_shared_lock l(mu_); + return output_; + } + + // Records that the node produced an element. + void record_element() LOCKS_EXCLUDED(mu_) { + mutex_lock l(mu_); + num_elements_++; + } + + // Records that a node thread has started executing. + void record_start() LOCKS_EXCLUDED(mu_) { + mutex_lock l(mu_); + work_start_[std::this_thread::get_id()] = Env::Default()->NowNanos(); + } + + // Records that a node thread has stopped executing. + void record_stop() LOCKS_EXCLUDED(mu_) { + mutex_lock l(mu_); + std::thread::id tid = std::this_thread::get_id(); + auto start_time = gtl::FindOrNull(work_start_, tid); + DCHECK(start_time) + << "Encountered a stop event that was not preceded by a start event."; + if (start_time) { + processing_time_ += Env::Default()->NowNanos() - *start_time; + work_start_.erase(tid); + } + } + + // Removes an input. + void remove_input(std::shared_ptr input) LOCKS_EXCLUDED(mu_) { + mutex_lock l(mu_); + inputs_.remove(input); + } + + // Set the node output. + void set_output(std::shared_ptr output) LOCKS_EXCLUDED(mu_) { + mutex_lock l(mu_); + output_ = output; + } + + // Collects tunable parameters in the subtree rooted in this node. + void CollectTunables(std::vector>* tunables) + LOCKS_EXCLUDED(mu_); + + // Returns the per-element output time for this node. + int64 OutputTime(std::vector* input_times) LOCKS_EXCLUDED(mu_) { + tf_shared_lock l(mu_); + return OutputTimeLocked(input_times); + } + + // Returns the per-element processing time spent in the subtree rooted in + // this node. + int64 ProcessingTime() LOCKS_EXCLUDED(mu_) { + tf_shared_lock l(mu_); + return ProcessingTimeLocked(); + } + + private: + enum class Type { + BATCH = 0, + CACHE, + CONCATENATE, + FILTER, + FLAT_MAP, + INTERLEAVE, + MAP, + MAP_AND_BATCH, + PADDED_BATCH, + PARALLEL_INTERLEAVE, + PARALLEL_INTERLEAVE_V2, + PARALLEL_MAP, + PREFETCH, + REPEAT, + SHUFFLE, + SKIP, + TAKE, + ZIP, + UNKNOWN, + }; + + // Gets a value of the given parameter (tunable or constant). + int64 GetParameterValue(const string& name) SHARED_LOCKS_REQUIRED(mu_); + + // Returns the per-element processing time spent in this node. + int64 NanosPerElement() LOCKS_EXCLUDED(mu_) { + tf_shared_lock l(mu_); + return NanosPerElementLocked(); + } + + int64 NanosPerElementLocked() SHARED_LOCKS_REQUIRED(mu_) { + if (num_elements_ == 0) { + return 0; + } + return (int64)((double)processing_time_ / (double)num_elements_); + } + + int64 OutputTimeLocked(std::vector* input_times) + SHARED_LOCKS_REQUIRED(mu_); + + int64 OutputTimeForInputs(std::vector* input_times) + SHARED_LOCKS_REQUIRED(mu_) { + int64 sum = 0; + for (auto input : inputs_) { + sum += input->OutputTime(input_times); + } + return sum; + } + + int64 ProcessingTimeLocked() SHARED_LOCKS_REQUIRED(mu_); + + // Returns the per-element processing time spent in the inputs of this node. + int64 ProcessingTimeForInputs() SHARED_LOCKS_REQUIRED(mu_) { + int64 sum = 0; + for (auto input : inputs_) { + sum += input->ProcessingTime(); + } + return sum; + } + + Type TypeFromName(const string& name) SHARED_LOCKS_REQUIRED(mu_) { + if (name_ == "Batch") { + return Type::BATCH; + } + if (str_util::EndsWith(name_, "Cache")) { + return Type::CACHE; + } + if (name_ == "Concatenate") { + return Type::CONCATENATE; + } + if (name_ == "Filter") { + return Type::FILTER; + } + if (name_ == "FlatMap") { + return Type::FLAT_MAP; + } + if (name_ == "Interleave") { + return Type::INTERLEAVE; + } + if (name_ == "Map") { + return Type::MAP; + } + if (name_ == "MapAndBatch") { + return Type::MAP_AND_BATCH; + } + if (name_ == "PaddedBatch") { + return Type::PADDED_BATCH; + } + if (name_ == "ParallelInterleave") { + return Type::PARALLEL_INTERLEAVE; + } + if (name_ == "ParallelInterleaveV2") { + return Type::PARALLEL_INTERLEAVE_V2; + } + if (name_ == "ParallelMap") { + return Type::PARALLEL_MAP; + } + if (name_ == "Prefetch") { + return Type::PREFETCH; + } + if (str_util::EndsWith(name_, "Repeat")) { + return Type::REPEAT; + } + if (name_ == "Shuffle") { + return Type::SHUFFLE; + } + if (str_util::EndsWith(name_, "Skip")) { + return Type::SKIP; + } + if (str_util::EndsWith(name_, "Take")) { + return Type::TAKE; + } + if (name_ == "Zip") { + return Type::ZIP; + } + return Type::UNKNOWN; + } + + mutex mu_; + const int64 id_; + const string name_; + const Type type_; + int64 processing_time_ GUARDED_BY(mu_) = 0; + int64 num_elements_ GUARDED_BY(mu_) = 0; + std::map work_start_ GUARDED_BY(mu_); + std::map constant_params_ GUARDED_BY(mu_); + // Tunables are shared with the model during optimization. + std::map> tunable_params_ GUARDED_BY(mu_); + std::list> inputs_ GUARDED_BY(mu_); + std::shared_ptr output_ GUARDED_BY(mu_); + }; + + std::vector> CollectTunables() + SHARED_LOCKS_REQUIRED(mu_); + + int64 OutputTime() SHARED_LOCKS_REQUIRED(mu_); + + int64 ProcessingTime() SHARED_LOCKS_REQUIRED(mu_); + + // Used for coordination between different input pipeline threads. Exclusive + // access is required only when adding or removing nodes. Concurrent access to + // existing nodes is protected by a node mutex. + mutex mu_; + int64 id_counter_ GUARDED_BY(mu_) = 1; + std::shared_ptr output_ GUARDED_BY(mu_); + std::map> lookup_table_ GUARDED_BY(mu_); +}; + +} // namespace model +} // namespace data +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_FRAMEWORK_MODEL_H_ diff --git a/linux/include/tensorflow/core/framework/node_def.pb.h b/linux/include/tensorflow/core/framework/node_def.pb.h new file mode 100644 index 000000000..f17231e6b --- /dev/null +++ b/linux/include/tensorflow/core/framework/node_def.pb.h @@ -0,0 +1,645 @@ +// Generated by the protocol buffer compiler. DO NOT EDIT! +// source: tensorflow/core/framework/node_def.proto + +#ifndef PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2fnode_5fdef_2eproto +#define PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2fnode_5fdef_2eproto + +#include + +#include + +#if GOOGLE_PROTOBUF_VERSION < 3006000 +#error This file was generated by a newer version of protoc which is +#error incompatible with your Protocol Buffer headers. Please update +#error your headers. +#endif +#if 3006000 < GOOGLE_PROTOBUF_MIN_PROTOC_VERSION +#error This file was generated by an older version of protoc which is +#error incompatible with your Protocol Buffer headers. Please +#error regenerate this file with a newer version of protoc. +#endif + +#include +#include +#include +#include +#include +#include +#include +#include +#include // IWYU pragma: export +#include // IWYU pragma: export +#include // IWYU pragma: export +#include +#include +#include +#include "tensorflow/core/framework/attr_value.pb.h" +// @@protoc_insertion_point(includes) +#define PROTOBUF_INTERNAL_EXPORT_protobuf_tensorflow_2fcore_2fframework_2fnode_5fdef_2eproto + +namespace protobuf_tensorflow_2fcore_2fframework_2fnode_5fdef_2eproto { +// Internal implementation detail -- do not use these members. +struct TableStruct { + static const ::google::protobuf::internal::ParseTableField entries[]; + static const ::google::protobuf::internal::AuxillaryParseTableField aux[]; + static const ::google::protobuf::internal::ParseTable schema[2]; + static const ::google::protobuf::internal::FieldMetadata field_metadata[]; + static const ::google::protobuf::internal::SerializationTable serialization_table[]; + static const ::google::protobuf::uint32 offsets[]; +}; +void AddDescriptors(); +} // namespace protobuf_tensorflow_2fcore_2fframework_2fnode_5fdef_2eproto +namespace tensorflow { +class NodeDef; +class NodeDefDefaultTypeInternal; +extern NodeDefDefaultTypeInternal _NodeDef_default_instance_; +class NodeDef_AttrEntry_DoNotUse; +class NodeDef_AttrEntry_DoNotUseDefaultTypeInternal; +extern NodeDef_AttrEntry_DoNotUseDefaultTypeInternal _NodeDef_AttrEntry_DoNotUse_default_instance_; +} // namespace tensorflow +namespace google { +namespace protobuf { +template<> ::tensorflow::NodeDef* Arena::CreateMaybeMessage<::tensorflow::NodeDef>(Arena*); +template<> ::tensorflow::NodeDef_AttrEntry_DoNotUse* Arena::CreateMaybeMessage<::tensorflow::NodeDef_AttrEntry_DoNotUse>(Arena*); +} // namespace protobuf +} // namespace google +namespace tensorflow { + +// =================================================================== + +class NodeDef_AttrEntry_DoNotUse : public ::google::protobuf::internal::MapEntry { +public: + typedef ::google::protobuf::internal::MapEntry SuperType; + NodeDef_AttrEntry_DoNotUse(); + NodeDef_AttrEntry_DoNotUse(::google::protobuf::Arena* arena); + void MergeFrom(const NodeDef_AttrEntry_DoNotUse& other); + static const NodeDef_AttrEntry_DoNotUse* internal_default_instance() { return reinterpret_cast(&_NodeDef_AttrEntry_DoNotUse_default_instance_); } + void MergeFrom(const ::google::protobuf::Message& other) final; + ::google::protobuf::Metadata GetMetadata() const; +}; + +// ------------------------------------------------------------------- + +class NodeDef : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.NodeDef) */ { + public: + NodeDef(); + virtual ~NodeDef(); + + NodeDef(const NodeDef& from); + + inline NodeDef& operator=(const NodeDef& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + NodeDef(NodeDef&& from) noexcept + : NodeDef() { + *this = ::std::move(from); + } + + inline NodeDef& operator=(NodeDef&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const NodeDef& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const NodeDef* internal_default_instance() { + return reinterpret_cast( + &_NodeDef_default_instance_); + } + static constexpr int kIndexInFileMessages = + 1; + + void UnsafeArenaSwap(NodeDef* other); + void Swap(NodeDef* other); + friend void swap(NodeDef& a, NodeDef& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline NodeDef* New() const final { + return CreateMaybeMessage(NULL); + } + + NodeDef* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const NodeDef& from); + void MergeFrom(const NodeDef& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(NodeDef* other); + protected: + explicit NodeDef(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + + // accessors ------------------------------------------------------- + + // repeated string input = 3; + int input_size() const; + void clear_input(); + static const int kInputFieldNumber = 3; + const ::std::string& input(int index) const; + ::std::string* mutable_input(int index); + void set_input(int index, const ::std::string& value); + #if LANG_CXX11 + void set_input(int index, ::std::string&& value); + #endif + void set_input(int index, const char* value); + void set_input(int index, const char* value, size_t size); + ::std::string* add_input(); + void add_input(const ::std::string& value); + #if LANG_CXX11 + void add_input(::std::string&& value); + #endif + void add_input(const char* value); + void add_input(const char* value, size_t size); + const ::google::protobuf::RepeatedPtrField< ::std::string>& input() const; + ::google::protobuf::RepeatedPtrField< ::std::string>* mutable_input(); + + // map attr = 5; + int attr_size() const; + void clear_attr(); + static const int kAttrFieldNumber = 5; + const ::google::protobuf::Map< ::std::string, ::tensorflow::AttrValue >& + attr() const; + ::google::protobuf::Map< ::std::string, ::tensorflow::AttrValue >* + mutable_attr(); + + // string name = 1; + void clear_name(); + static const int kNameFieldNumber = 1; + const ::std::string& name() const; + void set_name(const ::std::string& value); + #if LANG_CXX11 + void set_name(::std::string&& value); + #endif + void set_name(const char* value); + void set_name(const char* value, size_t size); + ::std::string* mutable_name(); + ::std::string* release_name(); + void set_allocated_name(::std::string* name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_name( + ::std::string* name); + + // string op = 2; + void clear_op(); + static const int kOpFieldNumber = 2; + const ::std::string& op() const; + void set_op(const ::std::string& value); + #if LANG_CXX11 + void set_op(::std::string&& value); + #endif + void set_op(const char* value); + void set_op(const char* value, size_t size); + ::std::string* mutable_op(); + ::std::string* release_op(); + void set_allocated_op(::std::string* op); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_op(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_op( + ::std::string* op); + + // string device = 4; + void clear_device(); + static const int kDeviceFieldNumber = 4; + const ::std::string& device() const; + void set_device(const ::std::string& value); + #if LANG_CXX11 + void set_device(::std::string&& value); + #endif + void set_device(const char* value); + void set_device(const char* value, size_t size); + ::std::string* mutable_device(); + ::std::string* release_device(); + void set_allocated_device(::std::string* device); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_device(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_device( + ::std::string* device); + + // @@protoc_insertion_point(class_scope:tensorflow.NodeDef) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedPtrField< ::std::string> input_; + ::google::protobuf::internal::MapField< + NodeDef_AttrEntry_DoNotUse, + ::std::string, ::tensorflow::AttrValue, + ::google::protobuf::internal::WireFormatLite::TYPE_STRING, + ::google::protobuf::internal::WireFormatLite::TYPE_MESSAGE, + 0 > attr_; + ::google::protobuf::internal::ArenaStringPtr name_; + ::google::protobuf::internal::ArenaStringPtr op_; + ::google::protobuf::internal::ArenaStringPtr device_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fnode_5fdef_2eproto::TableStruct; +}; +// =================================================================== + + +// =================================================================== + +#ifdef __GNUC__ + #pragma GCC diagnostic push + #pragma GCC diagnostic ignored "-Wstrict-aliasing" +#endif // __GNUC__ +// ------------------------------------------------------------------- + +// NodeDef + +// string name = 1; +inline void NodeDef::clear_name() { + name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& NodeDef::name() const { + // @@protoc_insertion_point(field_get:tensorflow.NodeDef.name) + return name_.Get(); +} +inline void NodeDef::set_name(const ::std::string& value) { + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.NodeDef.name) +} +#if LANG_CXX11 +inline void NodeDef::set_name(::std::string&& value) { + + name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.NodeDef.name) +} +#endif +inline void NodeDef::set_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.NodeDef.name) +} +inline void NodeDef::set_name(const char* value, + size_t size) { + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.NodeDef.name) +} +inline ::std::string* NodeDef::mutable_name() { + + // @@protoc_insertion_point(field_mutable:tensorflow.NodeDef.name) + return name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* NodeDef::release_name() { + // @@protoc_insertion_point(field_release:tensorflow.NodeDef.name) + + return name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void NodeDef::set_allocated_name(::std::string* name) { + if (name != NULL) { + + } else { + + } + name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.NodeDef.name) +} +inline ::std::string* NodeDef::unsafe_arena_release_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.NodeDef.name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void NodeDef::unsafe_arena_set_allocated_name( + ::std::string* name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (name != NULL) { + + } else { + + } + name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.NodeDef.name) +} + +// string op = 2; +inline void NodeDef::clear_op() { + op_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& NodeDef::op() const { + // @@protoc_insertion_point(field_get:tensorflow.NodeDef.op) + return op_.Get(); +} +inline void NodeDef::set_op(const ::std::string& value) { + + op_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.NodeDef.op) +} +#if LANG_CXX11 +inline void NodeDef::set_op(::std::string&& value) { + + op_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.NodeDef.op) +} +#endif +inline void NodeDef::set_op(const char* value) { + GOOGLE_DCHECK(value != NULL); + + op_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.NodeDef.op) +} +inline void NodeDef::set_op(const char* value, + size_t size) { + + op_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.NodeDef.op) +} +inline ::std::string* NodeDef::mutable_op() { + + // @@protoc_insertion_point(field_mutable:tensorflow.NodeDef.op) + return op_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* NodeDef::release_op() { + // @@protoc_insertion_point(field_release:tensorflow.NodeDef.op) + + return op_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void NodeDef::set_allocated_op(::std::string* op) { + if (op != NULL) { + + } else { + + } + op_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), op, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.NodeDef.op) +} +inline ::std::string* NodeDef::unsafe_arena_release_op() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.NodeDef.op) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return op_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void NodeDef::unsafe_arena_set_allocated_op( + ::std::string* op) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (op != NULL) { + + } else { + + } + op_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + op, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.NodeDef.op) +} + +// repeated string input = 3; +inline int NodeDef::input_size() const { + return input_.size(); +} +inline void NodeDef::clear_input() { + input_.Clear(); +} +inline const ::std::string& NodeDef::input(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.NodeDef.input) + return input_.Get(index); +} +inline ::std::string* NodeDef::mutable_input(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.NodeDef.input) + return input_.Mutable(index); +} +inline void NodeDef::set_input(int index, const ::std::string& value) { + // @@protoc_insertion_point(field_set:tensorflow.NodeDef.input) + input_.Mutable(index)->assign(value); +} +#if LANG_CXX11 +inline void NodeDef::set_input(int index, ::std::string&& value) { + // @@protoc_insertion_point(field_set:tensorflow.NodeDef.input) + input_.Mutable(index)->assign(std::move(value)); +} +#endif +inline void NodeDef::set_input(int index, const char* value) { + GOOGLE_DCHECK(value != NULL); + input_.Mutable(index)->assign(value); + // @@protoc_insertion_point(field_set_char:tensorflow.NodeDef.input) +} +inline void NodeDef::set_input(int index, const char* value, size_t size) { + input_.Mutable(index)->assign( + reinterpret_cast(value), size); + // @@protoc_insertion_point(field_set_pointer:tensorflow.NodeDef.input) +} +inline ::std::string* NodeDef::add_input() { + // @@protoc_insertion_point(field_add_mutable:tensorflow.NodeDef.input) + return input_.Add(); +} +inline void NodeDef::add_input(const ::std::string& value) { + input_.Add()->assign(value); + // @@protoc_insertion_point(field_add:tensorflow.NodeDef.input) +} +#if LANG_CXX11 +inline void NodeDef::add_input(::std::string&& value) { + input_.Add(std::move(value)); + // @@protoc_insertion_point(field_add:tensorflow.NodeDef.input) +} +#endif +inline void NodeDef::add_input(const char* value) { + GOOGLE_DCHECK(value != NULL); + input_.Add()->assign(value); + // @@protoc_insertion_point(field_add_char:tensorflow.NodeDef.input) +} +inline void NodeDef::add_input(const char* value, size_t size) { + input_.Add()->assign(reinterpret_cast(value), size); + // @@protoc_insertion_point(field_add_pointer:tensorflow.NodeDef.input) +} +inline const ::google::protobuf::RepeatedPtrField< ::std::string>& +NodeDef::input() const { + // @@protoc_insertion_point(field_list:tensorflow.NodeDef.input) + return input_; +} +inline ::google::protobuf::RepeatedPtrField< ::std::string>* +NodeDef::mutable_input() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.NodeDef.input) + return &input_; +} + +// string device = 4; +inline void NodeDef::clear_device() { + device_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& NodeDef::device() const { + // @@protoc_insertion_point(field_get:tensorflow.NodeDef.device) + return device_.Get(); +} +inline void NodeDef::set_device(const ::std::string& value) { + + device_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.NodeDef.device) +} +#if LANG_CXX11 +inline void NodeDef::set_device(::std::string&& value) { + + device_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.NodeDef.device) +} +#endif +inline void NodeDef::set_device(const char* value) { + GOOGLE_DCHECK(value != NULL); + + device_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.NodeDef.device) +} +inline void NodeDef::set_device(const char* value, + size_t size) { + + device_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.NodeDef.device) +} +inline ::std::string* NodeDef::mutable_device() { + + // @@protoc_insertion_point(field_mutable:tensorflow.NodeDef.device) + return device_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* NodeDef::release_device() { + // @@protoc_insertion_point(field_release:tensorflow.NodeDef.device) + + return device_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void NodeDef::set_allocated_device(::std::string* device) { + if (device != NULL) { + + } else { + + } + device_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), device, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.NodeDef.device) +} +inline ::std::string* NodeDef::unsafe_arena_release_device() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.NodeDef.device) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return device_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void NodeDef::unsafe_arena_set_allocated_device( + ::std::string* device) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (device != NULL) { + + } else { + + } + device_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + device, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.NodeDef.device) +} + +// map attr = 5; +inline int NodeDef::attr_size() const { + return attr_.size(); +} +inline const ::google::protobuf::Map< ::std::string, ::tensorflow::AttrValue >& +NodeDef::attr() const { + // @@protoc_insertion_point(field_map:tensorflow.NodeDef.attr) + return attr_.GetMap(); +} +inline ::google::protobuf::Map< ::std::string, ::tensorflow::AttrValue >* +NodeDef::mutable_attr() { + // @@protoc_insertion_point(field_mutable_map:tensorflow.NodeDef.attr) + return attr_.MutableMap(); +} + +#ifdef __GNUC__ + #pragma GCC diagnostic pop +#endif // __GNUC__ +// ------------------------------------------------------------------- + + +// @@protoc_insertion_point(namespace_scope) + +} // namespace tensorflow + +// @@protoc_insertion_point(global_scope) + +#endif // PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2fnode_5fdef_2eproto diff --git a/linux/include/tensorflow/core/framework/node_def.pb_text-impl.h b/linux/include/tensorflow/core/framework/node_def.pb_text-impl.h new file mode 100644 index 000000000..a4c5f6f33 --- /dev/null +++ b/linux/include/tensorflow/core/framework/node_def.pb_text-impl.h @@ -0,0 +1,35 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_framework_node_def_proto_IMPL_H_ +#define tensorflow_core_framework_node_def_proto_IMPL_H_ + +#include "tensorflow/core/framework/attr_value.pb.h" +#include "tensorflow/core/framework/attr_value.pb_text-impl.h" +#include "tensorflow/core/framework/node_def.pb.h" +#include "tensorflow/core/framework/node_def.pb_text.h" +#include "tensorflow/core/framework/resource_handle.pb.h" +#include "tensorflow/core/framework/resource_handle.pb_text-impl.h" +#include "tensorflow/core/framework/tensor.pb.h" +#include "tensorflow/core/framework/tensor.pb_text-impl.h" +#include "tensorflow/core/framework/tensor_shape.pb.h" +#include "tensorflow/core/framework/tensor_shape.pb_text-impl.h" +#include "tensorflow/core/framework/types.pb.h" +#include "tensorflow/core/framework/types.pb_text-impl.h" +#include "tensorflow/core/lib/strings/proto_text_util.h" +#include "tensorflow/core/lib/strings/scanner.h" + +namespace tensorflow { + +namespace internal { + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::NodeDef& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::NodeDef* msg); + +} // namespace internal + +} // namespace tensorflow + +#endif // tensorflow_core_framework_node_def_proto_IMPL_H_ diff --git a/linux/include/tensorflow/core/framework/node_def.pb_text.h b/linux/include/tensorflow/core/framework/node_def.pb_text.h new file mode 100644 index 000000000..c1426f834 --- /dev/null +++ b/linux/include/tensorflow/core/framework/node_def.pb_text.h @@ -0,0 +1,24 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_framework_node_def_proto_H_ +#define tensorflow_core_framework_node_def_proto_H_ + +#include "tensorflow/core/framework/node_def.pb.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/protobuf.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +// Message-text conversion for tensorflow.NodeDef +string ProtoDebugString( + const ::tensorflow::NodeDef& msg); +string ProtoShortDebugString( + const ::tensorflow::NodeDef& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::NodeDef* msg) + TF_MUST_USE_RESULT; + +} // namespace tensorflow + +#endif // tensorflow_core_framework_node_def_proto_H_ diff --git a/linux/include/tensorflow/core/framework/node_def_builder.h b/linux/include/tensorflow/core/framework/node_def_builder.h new file mode 100644 index 000000000..ad07ec548 --- /dev/null +++ b/linux/include/tensorflow/core/framework/node_def_builder.h @@ -0,0 +1,178 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_FRAMEWORK_NODE_DEF_BUILDER_H_ +#define TENSORFLOW_CORE_FRAMEWORK_NODE_DEF_BUILDER_H_ + +#include +#include +#include "tensorflow/core/framework/attr_value_util.h" +#include "tensorflow/core/framework/node_def.pb.h" +#include "tensorflow/core/framework/node_def_util.h" +#include "tensorflow/core/framework/op.h" +#include "tensorflow/core/framework/op_def.pb.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/lib/gtl/array_slice.h" +#include "tensorflow/core/lib/strings/strcat.h" + +namespace tensorflow { + +class NodeDefBuilder; +typedef std::function + FakeInputFunctor; + +// This is a helper for creating a NodeDef. Automatically sets attrs +// that can be inferred from the inputs, and uses default values +// (where they exist) for unspecified attrs. Example usage: +// +// NodeDef node_def; +// Status status = NodeDefBuilder(node_name, op_name) +// .Input(...) +// .Attr(...) +// .Finalize(&node_def); +// if (!status.ok()) return status; +// // Use node_def here. +class NodeDefBuilder { + public: + // To specify an output to be consumed by one of the Input() methods below. + struct NodeOut { + NodeOut(StringPiece n, int i, DataType dt); + NodeOut(); // uninitialized, call Reset() before use. + void Reset(StringPiece n, int i, DataType dt); + string node; + int index; + DataType data_type; + }; + + // Specify the name and the Op (either via an OpDef or the name of + // the Op plus a registry) for the NodeDef. Other fields are + // specified by calling the methods below. + // REQUIRES: The OpDef must satisfy ValidateOpDef(). + NodeDefBuilder(StringPiece name, StringPiece op_name, + const OpRegistryInterface* op_registry = OpRegistry::Global()); + // REQUIRES: in addition, *op_def must outlive *this. + NodeDefBuilder(StringPiece name, const OpDef* op_def); + + // You must call one Input() function per input_arg in the Op, + // *and in the same order as the input_args appear in the OpDef.* + + // For inputs that take a single tensor. + NodeDefBuilder& Input(StringPiece src_node, int src_index, DataType dt); + NodeDefBuilder& Input(const NodeOut& src); + + // For inputs that take a list of tensors. + NodeDefBuilder& Input(gtl::ArraySlice src_list); + + // To create inputs in tests, see fake_input.h. + NodeDefBuilder& Input(FakeInputFunctor fake_input); + + // Specify that this node must only run after src_node. + NodeDefBuilder& ControlInput(StringPiece src_node); + + // Constrains what devices this node may be scheduled on. + NodeDefBuilder& Device(StringPiece device_spec); + + // Sets the attr, if not already set. If already set with a different + // value, an error will be returned from Finalize(). + NodeDefBuilder& Attr(StringPiece name, const AttrValue& value); + NodeDefBuilder& Attr(StringPiece name, StringPiece value); + NodeDefBuilder& Attr(StringPiece name, const char* value); + NodeDefBuilder& Attr(StringPiece name, int32 value); + NodeDefBuilder& Attr(StringPiece name, int64 value); + NodeDefBuilder& Attr(StringPiece name, float value); + NodeDefBuilder& Attr(StringPiece name, double value); + NodeDefBuilder& Attr(StringPiece name, bool value); + NodeDefBuilder& Attr(StringPiece name, DataType value); + NodeDefBuilder& Attr(StringPiece name, const PartialTensorShape& value); + NodeDefBuilder& Attr(StringPiece name, const Tensor& value); + NodeDefBuilder& Attr(StringPiece name, const TensorProto& value); + NodeDefBuilder& Attr(StringPiece name, const NameAttrList& value); + NodeDefBuilder& Attr(StringPiece name, gtl::ArraySlice value); + NodeDefBuilder& Attr(StringPiece name, gtl::ArraySlice value); + NodeDefBuilder& Attr(StringPiece name, gtl::ArraySlice value); + NodeDefBuilder& Attr(StringPiece name, gtl::ArraySlice value); + NodeDefBuilder& Attr(StringPiece name, gtl::ArraySlice value); + NodeDefBuilder& Attr(StringPiece name, gtl::ArraySlice value); + NodeDefBuilder& Attr(StringPiece name, gtl::ArraySlice value); + NodeDefBuilder& Attr(StringPiece name, const std::vector& value); + NodeDefBuilder& Attr(StringPiece name, gtl::ArraySlice value); + NodeDefBuilder& Attr(StringPiece name, gtl::ArraySlice value); + NodeDefBuilder& Attr(StringPiece name, + gtl::ArraySlice value); + NodeDefBuilder& Attr(StringPiece name, + gtl::ArraySlice value); + NodeDefBuilder& Attr(StringPiece name, gtl::ArraySlice value); + NodeDefBuilder& Attr(StringPiece name, gtl::ArraySlice value); + + template + NodeDefBuilder& Attr(StringPiece name, std::initializer_list value) { + return Attr(name, gtl::ArraySlice(value)); + } + + // Finish building the NodeDef, returning any errors or setting + // *node_def if none. + // WARNING: Not all problems are detected! The resulting NodeDef may + // not be valid! Call ValidateNodeDef() from node_def_utils to be sure. + Status Finalize(NodeDef* node_def) const; + + // Accessors for the values set in the constructor. + const string& node_name() const { return node_def_.name(); } + const OpDef& op_def() const { return *op_def_; } + + private: + // Called in the constructors. + void Initialize(); + + // Get the current ArgDef and advance to the next one. Returns nullptr + // if no more inputs are available. + const OpDef::ArgDef* NextArgDef(); + + // Returns true if there is still an input_arg available in *op_def_, + // otherwise adds to error_ and returns false. + bool NextArgAvailable(); + + // These do the main work of the Input() methods. + void SingleInput(const OpDef::ArgDef* input_arg, StringPiece src_node, + int src_index, DataType dt); + void ListInput(const OpDef::ArgDef* input_arg, + gtl::ArraySlice src_list); + + // Add "src_node:src_index" to the list of inputs in the node_def_. + void AddInput(StringPiece src_node, int src_index); + + // Generate an error if you can't pass dt when expected is expected. + void VerifyInputType(const OpDef::ArgDef* input_arg, DataType expected, + DataType dt); + + // If input_arg->is_ref() is true, generate an error if dt is not a ref. + void VerifyInputRef(const OpDef::ArgDef* input_arg, DataType dt); + + // Makes dt a ref type if that is what the input_arg specifies. + DataType MaybeAddRef(const OpDef::ArgDef* input_arg, DataType dt) { + return input_arg->is_ref() ? MakeRefType(dt) : dt; + } + + const OpDef* op_def_; + NodeDef node_def_; + int inputs_specified_; + std::vector control_inputs_; + std::vector errors_; +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_FRAMEWORK_NODE_DEF_BUILDER_H_ diff --git a/linux/include/tensorflow/core/framework/node_def_util.h b/linux/include/tensorflow/core/framework/node_def_util.h new file mode 100644 index 000000000..0ff67554e --- /dev/null +++ b/linux/include/tensorflow/core/framework/node_def_util.h @@ -0,0 +1,322 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_FRAMEWORK_NODE_DEF_UTIL_H_ +#define TENSORFLOW_CORE_FRAMEWORK_NODE_DEF_UTIL_H_ + +#include +#include + +#include "tensorflow/core/framework/attr_value_util.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/core/stringpiece.h" +#include "tensorflow/core/lib/gtl/flatmap.h" +#include "tensorflow/core/lib/hash/hash.h" +#include "tensorflow/core/platform/protobuf.h" + +namespace tensorflow { + +class Node; + +// We forward declare protos so that kernels don't need to depend on them +class NodeDef; +class OpDef; + +// Name of the attribute used to encode node colocation constraints. +// +// Nodes can be co-located on the same device. Desire for explicit co-location +// is described by list(string) attribute containing the name of colocation +// groups. +extern const char* const kColocationAttrName; + +// String prefix applied to the operation name for colocation constraints. +extern const char* const kColocationGroupPrefix; + +// Produce a human-readable version of a Node or NodeDef that is more concise +// than a text-format proto. +string SummarizeNode(const Node& node); +string SummarizeNodeDef(const NodeDef& node_def); + +// Produces a formatted string pattern from the node which can uniquely identify +// this node upstream to produce an informative error message. The pattern +// followed is: {{node }} +string FormatNodeForError(const Node& node); +string FormatNodeDefForError(const NodeDef& node_def); + +typedef protobuf::Map AttrValueMap; + +// Adds an attr with name and value to *node_def. +// The type of the attr is based on the type of value. +void AddNodeAttr(StringPiece name, const AttrValue& value, NodeDef* node_def); +void AddNodeAttr(StringPiece name, StringPiece value, NodeDef* node_def); +void AddNodeAttr(StringPiece name, const char* value, NodeDef* node_def); +void AddNodeAttr(StringPiece name, int32 value, NodeDef* node_def); +void AddNodeAttr(StringPiece name, int64 value, NodeDef* node_def); +void AddNodeAttr(StringPiece name, float value, NodeDef* node_def); +void AddNodeAttr(StringPiece name, double value, NodeDef* node_def); +void AddNodeAttr(StringPiece name, bool value, NodeDef* node_def); +void AddNodeAttr(StringPiece name, DataType value, NodeDef* node_def); +void AddNodeAttr(StringPiece name, const PartialTensorShape& value, + NodeDef* node_def); +void AddNodeAttr(StringPiece name, const Tensor& value, NodeDef* node_def); +void AddNodeAttr(StringPiece name, const TensorProto& value, NodeDef* node_def); +void AddNodeAttr(StringPiece name, const NameAttrList& value, + NodeDef* node_def); +void AddNodeAttr(StringPiece name, gtl::ArraySlice value, + NodeDef* node_def); +void AddNodeAttr(StringPiece name, gtl::ArraySlice value, + NodeDef* node_def); +void AddNodeAttr(StringPiece name, gtl::ArraySlice value, + NodeDef* node_def); +void AddNodeAttr(StringPiece name, gtl::ArraySlice value, + NodeDef* node_def); +void AddNodeAttr(StringPiece name, gtl::ArraySlice value, + NodeDef* node_def); +void AddNodeAttr(StringPiece name, gtl::ArraySlice value, + NodeDef* node_def); +void AddNodeAttr(StringPiece name, gtl::ArraySlice value, + NodeDef* node_def); +void AddNodeAttr(StringPiece name, const std::vector& value, + NodeDef* node_def); +void AddNodeAttr(StringPiece name, gtl::ArraySlice value, + NodeDef* node_def); +void AddNodeAttr(StringPiece name, gtl::ArraySlice value, + NodeDef* node_def); +void AddNodeAttr(StringPiece name, gtl::ArraySlice value, + NodeDef* node_def); +void AddNodeAttr(StringPiece name, gtl::ArraySlice value, + NodeDef* node_def); +void AddNodeAttr(StringPiece name, gtl::ArraySlice value, + NodeDef* node_def); +void AddNodeAttr(StringPiece name, gtl::ArraySlice value, + NodeDef* node_def); + +// Version to workaround C++'s "perfect" forwarding not being able to +// forward {...} initialization. +template +void AddNodeAttr(StringPiece name, std::initializer_list value, + NodeDef* node_def) { + AddNodeAttr(name, gtl::ArraySlice(value), node_def); +} + +// Adds an attr to an attr value map. +void AddAttr(StringPiece name, const AttrValue& value, AttrValueMap* map); +void AddAttr(StringPiece name, bool value, AttrValueMap* map); + +class AttrSlice { + public: + AttrSlice(const NodeDef& node_def); // NOLINT(runtime/explicit) + + AttrSlice(); // Empty + explicit AttrSlice(const AttrValueMap* a); + + int size() const { return attrs_->size(); } + + // Returns the attr with attr_name if found. Otherwise, returns + // nullptr. + const AttrValue* Find(StringPiece attr_name) const; + + // Returns the attr_value for attr_name if found. Otherwise, returns a + // NotFound status. + Status Find(StringPiece attr_name, const AttrValue** attr_value) const; + + // Helper class to avoid allocations in EqualAttrs. + // TODO(irving): Will go away once NodeInfo is used. + struct Scratch { + string a; + string b; + }; + + // Check if all attrs and attr values match. Does not take defaults into + // account. + // + // TODO(irving): There is a bug in this routine inherited from its + // OptimizerCSE::EqualAttrs precedecessor. The same tensor attr can be + // represented in more than one way as an AttrValue, since TensorProto is + // not 1-1. This bug will go away once I replace everything with NodeInfo, + // which stores a Tensor object directly. The Scratch object will also go + // away. + bool EqualAttrs(AttrSlice other, Scratch* scratch) const; + + // If this AttrSlice has an attached NodeDef, summarize it. This is for + // error messages only: we intentionally do not provide direct access to the + // NodeDef, since it is not always there. + string SummarizeNode() const; + + // Iteration over all attrs + AttrValueMap::const_iterator begin() const { return attrs_->begin(); } + AttrValueMap::const_iterator end() const { return attrs_->end(); } + + private: + const NodeDef* ndef_; + const AttrValueMap* attrs_; +}; + +// Return true if the attr with the name attr_name is defined in node_def. +bool HasNodeAttr(const NodeDef& node_def, StringPiece attr_name); + +// Look up the attr with name attr_name and set *value to its value. If no +// attr with attr_name is found in node_def, or the attr does not have +// a matching type, a non-ok status will be returned. +Status GetNodeAttr(const AttrSlice& attrs, StringPiece attr_name, + string* value); // type: "string" +Status GetNodeAttr(const AttrSlice& attrs, StringPiece attr_name, + int64* value); // type: "int" +Status GetNodeAttr(const AttrSlice& attrs, StringPiece attr_name, + int32* value); // type: "int" +Status GetNodeAttr(const AttrSlice& attrs, StringPiece attr_name, + float* value); // type: "float" +Status GetNodeAttr(const AttrSlice& attrs, StringPiece attr_name, + bool* value); // type: "bool" +Status GetNodeAttr(const AttrSlice& attrs, StringPiece attr_name, + DataType* value); // type: "type" +Status GetNodeAttr(const AttrSlice& attrs, StringPiece attr_name, + TensorShapeProto* value); // type: "shape" +Status GetNodeAttr(const AttrSlice& attrs, StringPiece attr_name, + TensorShape* value); // type: "shape" +Status GetNodeAttr(const AttrSlice& attrs, StringPiece attr_name, + PartialTensorShape* value); // type: "shape" +Status GetNodeAttr(const AttrSlice& attrs, StringPiece attr_name, + Tensor* value); // type: "tensor" +Status GetNodeAttr(const AttrSlice& attrs, StringPiece attr_name, + std::vector* value); // type "list(string)" +Status GetNodeAttr(const AttrSlice& attrs, StringPiece attr_name, + std::vector* value); // type "list(int)" +Status GetNodeAttr(const AttrSlice& attrs, StringPiece attr_name, + std::vector* value); // type "list(int)" +Status GetNodeAttr(const AttrSlice& attrs, StringPiece attr_name, + std::vector* value); // type "list(float)" +Status GetNodeAttr(const AttrSlice& attrs, StringPiece attr_name, + std::vector* value); // type "list(bool)" +Status GetNodeAttr(const AttrSlice& attrs, StringPiece attr_name, + std::vector* value); // type "list(type)" +Status GetNodeAttr(const AttrSlice& attrs, StringPiece attr_name, + DataTypeVector* value); // type "list(type)" +Status GetNodeAttr(const AttrSlice& attrs, StringPiece attr_name, + std::vector* value); // type "list(shape)" +Status GetNodeAttr(const AttrSlice& attrs, StringPiece attr_name, + std::vector* value); // type "list(shape)" +Status GetNodeAttr( + const AttrSlice& attrs, StringPiece attr_name, + std::vector* value); // type "list(shape)" +Status GetNodeAttr(const AttrSlice& attrs, StringPiece attr_name, + std::vector* value); // type: "list(tensor)" + +// This version avoids copying the TensorProto. +// REQUIRES: Must not use *value beyond the lifetime of node_def. +Status GetNodeAttr(const AttrSlice& attrs, StringPiece attr_name, + const TensorProto** value); // type: "tensor" + +// This version avoids copying the NameAttrList. +// REQUIRES: Must not use *value beyond the lifetime of node_def. +Status GetNodeAttr(const AttrSlice& attrs, StringPiece attr_name, + const NameAttrList** value); // type: "func" + +// These versions copies the NameAttrList(s). +Status GetNodeAttr(const AttrSlice& attrs, StringPiece attr_name, + NameAttrList* value); // type: "func" +Status GetNodeAttr(const AttrSlice& attrs, StringPiece attr_name, + std::vector* value); // type: "list(func)" + +// Look up the attr with name attr_name and set *value to its value. If no +// attr with attr_name is found in node_def, or the attr does not have +// a matching type, false is returned. +bool GetNodeAttrSimple(const AttrSlice& attrs, StringPiece attr_name, + string* value); // type: "string" +bool GetNodeAttrSimple(const AttrSlice& attrs, StringPiece attr_name, + std::vector* value); // type: "string" + +// Look up the attr with name attr_name and return a reference to its value. +// If no attr with attr_name is found in node_def, or the attr does not have +// a matching type, a reference to an empty string is returned. +// REQUIRES: Must not use the returned value beyond the lifetime of node_def. +const string& GetNodeAttrString(const AttrSlice& attrs, StringPiece attr_name); + +// Computes the input type for a specific node input. +// REQUIRES: ValidateOpDef(op_def).ok() +Status InputTypeForNode(const NodeDef& node_def, const OpDef& op_def, + int input_port, DataType* input_type); +// Computes the input types for a specific node. +// REQUIRES: ValidateOpDef(op_def).ok() +Status InputTypesForNode(const NodeDef& node_def, const OpDef& op_def, + DataTypeVector* inputs); +// Computes the output type for a specific node output. +// REQUIRES: ValidateOpDef(op_def).ok() +Status OutputTypeForNode(const NodeDef& node_def, const OpDef& op_def, + int output_port, DataType* output_type); +// Computes the output types for a specific node. +// REQUIRES: ValidateOpDef(op_def).ok() +Status OutputTypesForNode(const NodeDef& node_def, const OpDef& op_def, + DataTypeVector* outputs); +// Computes the input and output types for a specific node. +// REQUIRES: ValidateOpDef(op_def).ok() +Status InOutTypesForNode(const NodeDef& node_def, const OpDef& op_def, + DataTypeVector* inputs, DataTypeVector* outputs); +// Computes the number of outputs for a specific node. +// REQUIRES: ValidateOpDef(op_def).ok() +Status NumOutputsForNode(const NodeDef& node_def, const OpDef& op_def, + int* num_outputs); + +// Validates that the NodeDef: +// * Defines all expected attrs from the OpDef. +// * All attrs satisfies constraints from the OpDef. +// * Has a signature matching SignatureForNode(). +// etc. +Status ValidateNodeDef(const NodeDef& node_def, const OpDef& op_def); + +// Computes the mapping from input/output argument name to the +// corresponding input/output index range. For example, +// input "foo" corresponds to input indices +// [ (*inputs)["foo"].first, (*inputs)["foo"].second ). +// NOTE(mrry): To reduce allocations when the map is used and save +// space, the returned `NameRangeMap` objects borrow the input/output +// argument names from `op_def`. The `op_def` must outlive the +// returned `NameRangeMap` objects. +typedef gtl::FlatMap, hash> + NameRangeMap; +Status NameRangesForNode(const NodeDef& node_def, const OpDef& op_def, + NameRangeMap* inputs, NameRangeMap* outputs); +Status NameRangesForNode(const Node& node, const OpDef& op_def, + NameRangeMap* inputs, NameRangeMap* outputs); + +// Adds default values to *node_def for unspecified attrs from op_def. +void AddDefaultsToNodeDef(const OpDef& op_def, NodeDef* node_def); + +// Validates the syntax of a NodeDef provided externally. +// +// The following is an EBNF-style syntax for NodeDef objects. Note that +// Node objects are actually specified as tensorflow::NodeDef protocol buffers, +// which contain many other fields that are not (currently) validated. +// +// Node = NodeName, Inputs +// Inputs = ( DataInput * ), ( ControlInput * ) +// DataInput = NodeName, ( ":", [1-9], [0-9] * ) ? +// ControlInput = "^", NodeName +// NodeName = [A-Za-z0-9.], [A-Za-z0-9_./] * +Status ValidateExternalNodeDefSyntax(const NodeDef& node_def); + +// Returns "status" with kernel's NodeDef attached as additional text +// in the error message. +Status AttachDef(const Status& status, const NodeDef& node_def); +Status AttachDef(const Status& status, const Node& node); + +// Appends the given prefix and suffix to the original node name in order to +// make the name unique. If it's an "Enter" node, use the same way to reset +// attribute "frame_name". +Status AddPrefixAndSuffixToNode(StringPiece prefix, StringPiece suffix, + NodeDef* node_def); +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_FRAMEWORK_NODE_DEF_UTIL_H_ diff --git a/linux/include/tensorflow/core/framework/numeric_op.h b/linux/include/tensorflow/core/framework/numeric_op.h new file mode 100644 index 000000000..0167e21f1 --- /dev/null +++ b/linux/include/tensorflow/core/framework/numeric_op.h @@ -0,0 +1,113 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_FRAMEWORK_NUMERIC_OP_H_ +#define TENSORFLOW_CORE_FRAMEWORK_NUMERIC_OP_H_ + +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/framework/types.pb.h" +#include "tensorflow/core/lib/core/errors.h" +#include "tensorflow/core/lib/core/status.h" + +namespace tensorflow { + +// One input and one output, both the same type. +template +class UnaryOp : public OpKernel { + public: + explicit UnaryOp(OpKernelConstruction* context) : OpKernel(context) { + const DataType dt = DataTypeToEnum::v(); + OP_REQUIRES_OK(context, context->MatchSignature({dt}, {dt})); + } +}; + +// Two inputs and one output, all the same type. +template +class BinaryOp : public OpKernel { + public: + explicit BinaryOp(OpKernelConstruction* context) : OpKernel(context) { + const DataType dt = DataTypeToEnum::v(); + OP_REQUIRES_OK(context, context->MatchSignature({dt, dt}, {dt})); + } +}; + +// For operations where the input and output are the same shape. +// +// For usage, see ../framework/elementwise_ops.cc. +template +class UnaryElementWiseOp : public UnaryOp { + public: + using UnaryOp::UnaryOp; + + void Compute(OpKernelContext* context) override { + // Output shape is the same as input shape. + const Tensor& input = context->input(0); + Tensor* output = nullptr; + OP_REQUIRES_OK(context, context->forward_input_or_allocate_output( + {0}, 0, input.shape(), &output)); + static_cast(this)->Operate(context, input, output); + } +}; + +// For binary elementwise operations. +template +class BinaryElementWiseOp : public BinaryOp { + public: + using BinaryOp::BinaryOp; + + void Compute(OpKernelContext* context) override { + const Tensor& a = context->input(0); + const Tensor& b = context->input(1); + + if (!context->ValidateInputsAreSameShape(this)) { + return; + } + + Tensor* output = nullptr; + OP_REQUIRES_OK(context, context->forward_input_or_allocate_output( + {0, 1}, 0, a.shape(), &output)); + + // Dispatch to the descendant's Operate() function. + switch (a.dims()) { +#define NDIM_CASE(NDIMS) \ + case NDIMS: { \ + static_cast(this)->template Operate(context, a, b, output); \ + break; \ + } + + NDIM_CASE(0); + NDIM_CASE(1); + NDIM_CASE(2); + NDIM_CASE(3); + NDIM_CASE(4); + NDIM_CASE(5); + NDIM_CASE(6); + NDIM_CASE(7); + NDIM_CASE(8); +#undef NDIM_CASE + + default: + context->SetStatus(errors::InvalidArgument( + "We only handle up to Tensor::dims() up to 8, not ", a.dims())); + break; + } + } +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_FRAMEWORK_NUMERIC_OP_H_ diff --git a/linux/include/tensorflow/core/framework/numeric_types.h b/linux/include/tensorflow/core/framework/numeric_types.h new file mode 100644 index 000000000..3236d1897 --- /dev/null +++ b/linux/include/tensorflow/core/framework/numeric_types.h @@ -0,0 +1,125 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_FRAMEWORK_NUMERIC_TYPES_H_ +#define TENSORFLOW_CORE_FRAMEWORK_NUMERIC_TYPES_H_ + +#include +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +// Disable clang-format to prevent 'FixedPoint' header from being included +// before 'Tensor' header on which it depends. +// clang-format off +#include "third_party/eigen3/unsupported/Eigen/CXX11/FixedPoint" +// clang-format on + +#include "tensorflow/core/lib/bfloat16/bfloat16.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +// Single precision complex. +typedef std::complex complex64; +// Double precision complex. +typedef std::complex complex128; + +// We use Eigen's QInt implementations for our quantized int types. +typedef Eigen::QInt8 qint8; +typedef Eigen::QUInt8 quint8; +typedef Eigen::QInt32 qint32; +typedef Eigen::QInt16 qint16; +typedef Eigen::QUInt16 quint16; + +} // namespace tensorflow + + + + +static inline tensorflow::bfloat16 FloatToBFloat16(float float_val) { +#if __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__ + return *reinterpret_cast( + reinterpret_cast(&float_val)); +#else + return *reinterpret_cast( + &(reinterpret_cast(&float_val)[1])); +#endif +} + +namespace Eigen { +// TODO(xpan): We probably need to overwrite more methods to have correct eigen +// behavior. E.g. epsilon(), dummy_precision, etc. See NumTraits.h in eigen. +template <> +struct NumTraits + : GenericNumTraits { + enum { + IsInteger = 0, + IsSigned = 1, + RequireInitialization = 0 + }; + static EIGEN_STRONG_INLINE tensorflow::bfloat16 highest() { + return FloatToBFloat16(NumTraits::highest()); + } + + static EIGEN_STRONG_INLINE tensorflow::bfloat16 lowest() { + return FloatToBFloat16(NumTraits::lowest()); + } + + static EIGEN_STRONG_INLINE tensorflow::bfloat16 infinity() { + return FloatToBFloat16(NumTraits::infinity()); + } + + static EIGEN_STRONG_INLINE tensorflow::bfloat16 quiet_NaN() { + return FloatToBFloat16(NumTraits::quiet_NaN()); + } +}; + + +using ::tensorflow::operator==; +using ::tensorflow::operator!=; + +namespace numext { + +template <> +EIGEN_DEVICE_FUNC EIGEN_ALWAYS_INLINE tensorflow::bfloat16 log( + const tensorflow::bfloat16& x) { + return static_cast(::logf(static_cast(x))); +} + +template <> +EIGEN_DEVICE_FUNC EIGEN_ALWAYS_INLINE tensorflow::bfloat16 exp( + const tensorflow::bfloat16& x) { + return static_cast(::expf(static_cast(x))); +} + +template <> +EIGEN_DEVICE_FUNC EIGEN_ALWAYS_INLINE tensorflow::bfloat16 abs( + const tensorflow::bfloat16& x) { + return static_cast(::fabsf(static_cast(x))); +} + +} // namespace numext +} // namespace Eigen + +#if defined(_MSC_VER) && !defined(__clang__) +namespace std { +template <> +struct hash { + std::size_t operator()(const Eigen::half& a) const { + return static_cast(a.x); + } +}; +} // namespace std +#endif // _MSC_VER + +#endif // TENSORFLOW_CORE_FRAMEWORK_NUMERIC_TYPES_H_ diff --git a/linux/include/tensorflow/core/framework/op.h b/linux/include/tensorflow/core/framework/op.h new file mode 100644 index 000000000..81ed5f95f --- /dev/null +++ b/linux/include/tensorflow/core/framework/op.h @@ -0,0 +1,312 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_FRAMEWORK_OP_H_ +#define TENSORFLOW_CORE_FRAMEWORK_OP_H_ + +#include +#include + +#include +#include "tensorflow/core/framework/op_def_builder.h" +#include "tensorflow/core/framework/op_def_util.h" +#include "tensorflow/core/framework/selective_registration.h" +#include "tensorflow/core/lib/core/errors.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/lib/strings/str_util.h" +#include "tensorflow/core/lib/strings/strcat.h" +#include "tensorflow/core/platform/logging.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/mutex.h" +#include "tensorflow/core/platform/thread_annotations.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +// Users that want to look up an OpDef by type name should take an +// OpRegistryInterface. Functions accepting a +// (const) OpRegistryInterface* may call LookUp() from multiple threads. +class OpRegistryInterface { + public: + virtual ~OpRegistryInterface(); + + // Returns an error status and sets *op_reg_data to nullptr if no OpDef is + // registered under that name, otherwise returns the registered OpDef. + // Caller must not delete the returned pointer. + virtual Status LookUp(const string& op_type_name, + const OpRegistrationData** op_reg_data) const = 0; + + // Shorthand for calling LookUp to get the OpDef. + Status LookUpOpDef(const string& op_type_name, const OpDef** op_def) const; +}; + +// The standard implementation of OpRegistryInterface, along with a +// global singleton used for registering ops via the REGISTER +// macros below. Thread-safe. +// +// Example registration: +// OpRegistry::Global()->Register( +// [](OpRegistrationData* op_reg_data)->Status { +// // Populate *op_reg_data here. +// return Status::OK(); +// }); +class OpRegistry : public OpRegistryInterface { + public: + typedef std::function OpRegistrationDataFactory; + + OpRegistry(); + ~OpRegistry() override; + + void Register(const OpRegistrationDataFactory& op_data_factory); + + Status LookUp(const string& op_type_name, + const OpRegistrationData** op_reg_data) const override; + + // Fills *ops with all registered OpDefs (except those with names + // starting with '_' if include_internal == false) sorted in + // ascending alphabetical order. + void Export(bool include_internal, OpList* ops) const; + + // Returns ASCII-format OpList for all registered OpDefs (except + // those with names starting with '_' if include_internal == false). + string DebugString(bool include_internal) const; + + // A singleton available at startup. + static OpRegistry* Global(); + + // Get all registered ops. + void GetRegisteredOps(std::vector* op_defs); + + // Get all `OpRegistrationData`s. + void GetOpRegistrationData(std::vector* op_data); + + // Watcher, a function object. + // The watcher, if set by SetWatcher(), is called every time an op is + // registered via the Register function. The watcher is passed the Status + // obtained from building and adding the OpDef to the registry, and the OpDef + // itself if it was successfully built. A watcher returns a Status which is in + // turn returned as the final registration status. + typedef std::function Watcher; + + // An OpRegistry object has only one watcher. This interface is not thread + // safe, as different clients are free to set the watcher any time. + // Clients are expected to atomically perform the following sequence of + // operations : + // SetWatcher(a_watcher); + // Register some ops; + // op_registry->ProcessRegistrations(); + // SetWatcher(nullptr); + // Returns a non-OK status if a non-null watcher is over-written by another + // non-null watcher. + Status SetWatcher(const Watcher& watcher); + + // Process the current list of deferred registrations. Note that calls to + // Export, LookUp and DebugString would also implicitly process the deferred + // registrations. Returns the status of the first failed op registration or + // Status::OK() otherwise. + Status ProcessRegistrations() const; + + // Defer the registrations until a later call to a function that processes + // deferred registrations are made. Normally, registrations that happen after + // calls to Export, LookUp, ProcessRegistrations and DebugString are processed + // immediately. Call this to defer future registrations. + void DeferRegistrations(); + + // Clear the registrations that have been deferred. + void ClearDeferredRegistrations(); + + private: + // Ensures that all the functions in deferred_ get called, their OpDef's + // registered, and returns with deferred_ empty. Returns true the first + // time it is called. Prints a fatal log if any op registration fails. + bool MustCallDeferred() const EXCLUSIVE_LOCKS_REQUIRED(mu_); + + // Calls the functions in deferred_ and registers their OpDef's + // It returns the Status of the first failed op registration or Status::OK() + // otherwise. + Status CallDeferred() const EXCLUSIVE_LOCKS_REQUIRED(mu_); + + // Add 'def' to the registry with additional data 'data'. On failure, or if + // there is already an OpDef with that name registered, returns a non-okay + // status. + Status RegisterAlreadyLocked(const OpRegistrationDataFactory& op_data_factory) + const EXCLUSIVE_LOCKS_REQUIRED(mu_); + + mutable mutex mu_; + // Functions in deferred_ may only be called with mu_ held. + mutable std::vector deferred_ GUARDED_BY(mu_); + // Values are owned. + mutable std::unordered_map registry_ + GUARDED_BY(mu_); + mutable bool initialized_ GUARDED_BY(mu_); + + // Registry watcher. + mutable Watcher watcher_ GUARDED_BY(mu_); +}; + +// An adapter to allow an OpList to be used as an OpRegistryInterface. +// +// Note that shape inference functions are not passed in to OpListOpRegistry, so +// it will return an unusable shape inference function for every op it supports; +// therefore, it should only be used in contexts where this is okay. +class OpListOpRegistry : public OpRegistryInterface { + public: + // Does not take ownership of op_list, *op_list must outlive *this. + OpListOpRegistry(const OpList* op_list); + ~OpListOpRegistry() override; + Status LookUp(const string& op_type_name, + const OpRegistrationData** op_reg_data) const override; + + private: + // Values are owned. + std::unordered_map index_; +}; + +// Support for defining the OpDef (specifying the semantics of the Op and how +// it should be created) and registering it in the OpRegistry::Global() +// registry. Usage: +// +// REGISTER_OP("my_op_name") +// .Attr(":") +// .Attr(":=") +// .Input(":") +// .Input(":Ref()") +// .Output(":") +// .Doc(R"( +// <1-line summary> +// +// : +// : +// )"); +// +// Note: .Doc() should be last. +// For details, see the OpDefBuilder class in op_def_builder.h. + +namespace register_op { + +// OpDefBuilderWrapper is a templated class that is used in the REGISTER_OP +// calls. This allows the result of REGISTER_OP to be used in chaining, as in +// REGISTER_OP(a).Attr("...").Input("...");, while still allowing selective +// registration to turn the entire call-chain into a no-op. +template +class OpDefBuilderWrapper; + +// Template specialization that forwards all calls to the contained builder. +template <> +class OpDefBuilderWrapper { + public: + OpDefBuilderWrapper(const char name[]) : builder_(name) {} + OpDefBuilderWrapper& Attr(string spec) { + builder_.Attr(std::move(spec)); + return *this; + } + OpDefBuilderWrapper& Input(string spec) { + builder_.Input(std::move(spec)); + return *this; + } + OpDefBuilderWrapper& Output(string spec) { + builder_.Output(std::move(spec)); + return *this; + } + OpDefBuilderWrapper& SetIsCommutative() { + builder_.SetIsCommutative(); + return *this; + } + OpDefBuilderWrapper& SetIsAggregate() { + builder_.SetIsAggregate(); + return *this; + } + OpDefBuilderWrapper& SetIsStateful() { + builder_.SetIsStateful(); + return *this; + } + OpDefBuilderWrapper& SetAllowsUninitializedInput() { + builder_.SetAllowsUninitializedInput(); + return *this; + } + OpDefBuilderWrapper& Deprecated(int version, string explanation) { + builder_.Deprecated(version, std::move(explanation)); + return *this; + } + OpDefBuilderWrapper& Doc(string text) { + builder_.Doc(std::move(text)); + return *this; + } + OpDefBuilderWrapper& SetShapeFn( + Status (*fn)(shape_inference::InferenceContext*)) { + builder_.SetShapeFn(fn); + return *this; + } + const ::tensorflow::OpDefBuilder& builder() const { return builder_; } + + private: + mutable ::tensorflow::OpDefBuilder builder_; +}; + +// Template specialization that turns all calls into no-ops. +template <> +class OpDefBuilderWrapper { + public: + constexpr OpDefBuilderWrapper(const char name[]) {} + OpDefBuilderWrapper& Attr(StringPiece spec) { return *this; } + OpDefBuilderWrapper& Input(StringPiece spec) { return *this; } + OpDefBuilderWrapper& Output(StringPiece spec) { return *this; } + OpDefBuilderWrapper& SetIsCommutative() { return *this; } + OpDefBuilderWrapper& SetIsAggregate() { return *this; } + OpDefBuilderWrapper& SetIsStateful() { return *this; } + OpDefBuilderWrapper& SetAllowsUninitializedInput() { return *this; } + OpDefBuilderWrapper& Deprecated(int, StringPiece) { return *this; } + OpDefBuilderWrapper& Doc(StringPiece text) { return *this; } + OpDefBuilderWrapper& SetShapeFn( + Status (*fn)(shape_inference::InferenceContext*)) { + return *this; + } +}; + +struct OpDefBuilderReceiver { + // To call OpRegistry::Global()->Register(...), used by the + // REGISTER_OP macro below. + // Note: These are implicitly converting constructors. + OpDefBuilderReceiver( + const OpDefBuilderWrapper& wrapper); // NOLINT(runtime/explicit) + constexpr OpDefBuilderReceiver(const OpDefBuilderWrapper&) { + } // NOLINT(runtime/explicit) +}; +} // namespace register_op + +#define REGISTER_OP(name) REGISTER_OP_UNIQ_HELPER(__COUNTER__, name) +#define REGISTER_OP_UNIQ_HELPER(ctr, name) REGISTER_OP_UNIQ(ctr, name) +#define REGISTER_OP_UNIQ(ctr, name) \ + static ::tensorflow::register_op::OpDefBuilderReceiver register_op##ctr \ + TF_ATTRIBUTE_UNUSED = \ + ::tensorflow::register_op::OpDefBuilderWrapper(name) + +// The `REGISTER_SYSTEM_OP()` macro acts as `REGISTER_OP()` except +// that the op is registered unconditionally even when selective +// registration is used. +#define REGISTER_SYSTEM_OP(name) \ + REGISTER_SYSTEM_OP_UNIQ_HELPER(__COUNTER__, name) +#define REGISTER_SYSTEM_OP_UNIQ_HELPER(ctr, name) \ + REGISTER_SYSTEM_OP_UNIQ(ctr, name) +#define REGISTER_SYSTEM_OP_UNIQ(ctr, name) \ + static ::tensorflow::register_op::OpDefBuilderReceiver register_op##ctr \ + TF_ATTRIBUTE_UNUSED = \ + ::tensorflow::register_op::OpDefBuilderWrapper(name) + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_FRAMEWORK_OP_H_ diff --git a/linux/include/tensorflow/core/framework/op_def.pb.h b/linux/include/tensorflow/core/framework/op_def.pb.h new file mode 100644 index 000000000..0bf25d5e7 --- /dev/null +++ b/linux/include/tensorflow/core/framework/op_def.pb.h @@ -0,0 +1,2461 @@ +// Generated by the protocol buffer compiler. DO NOT EDIT! +// source: tensorflow/core/framework/op_def.proto + +#ifndef PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2fop_5fdef_2eproto +#define PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2fop_5fdef_2eproto + +#include + +#include + +#if GOOGLE_PROTOBUF_VERSION < 3006000 +#error This file was generated by a newer version of protoc which is +#error incompatible with your Protocol Buffer headers. Please update +#error your headers. +#endif +#if 3006000 < GOOGLE_PROTOBUF_MIN_PROTOC_VERSION +#error This file was generated by an older version of protoc which is +#error incompatible with your Protocol Buffer headers. Please +#error regenerate this file with a newer version of protoc. +#endif + +#include +#include +#include +#include +#include +#include +#include +#include +#include // IWYU pragma: export +#include // IWYU pragma: export +#include +#include "tensorflow/core/framework/attr_value.pb.h" +#include "tensorflow/core/framework/types.pb.h" +// @@protoc_insertion_point(includes) +#define PROTOBUF_INTERNAL_EXPORT_protobuf_tensorflow_2fcore_2fframework_2fop_5fdef_2eproto + +namespace protobuf_tensorflow_2fcore_2fframework_2fop_5fdef_2eproto { +// Internal implementation detail -- do not use these members. +struct TableStruct { + static const ::google::protobuf::internal::ParseTableField entries[]; + static const ::google::protobuf::internal::AuxillaryParseTableField aux[]; + static const ::google::protobuf::internal::ParseTable schema[5]; + static const ::google::protobuf::internal::FieldMetadata field_metadata[]; + static const ::google::protobuf::internal::SerializationTable serialization_table[]; + static const ::google::protobuf::uint32 offsets[]; +}; +void AddDescriptors(); +} // namespace protobuf_tensorflow_2fcore_2fframework_2fop_5fdef_2eproto +namespace tensorflow { +class OpDef; +class OpDefDefaultTypeInternal; +extern OpDefDefaultTypeInternal _OpDef_default_instance_; +class OpDef_ArgDef; +class OpDef_ArgDefDefaultTypeInternal; +extern OpDef_ArgDefDefaultTypeInternal _OpDef_ArgDef_default_instance_; +class OpDef_AttrDef; +class OpDef_AttrDefDefaultTypeInternal; +extern OpDef_AttrDefDefaultTypeInternal _OpDef_AttrDef_default_instance_; +class OpDeprecation; +class OpDeprecationDefaultTypeInternal; +extern OpDeprecationDefaultTypeInternal _OpDeprecation_default_instance_; +class OpList; +class OpListDefaultTypeInternal; +extern OpListDefaultTypeInternal _OpList_default_instance_; +} // namespace tensorflow +namespace google { +namespace protobuf { +template<> ::tensorflow::OpDef* Arena::CreateMaybeMessage<::tensorflow::OpDef>(Arena*); +template<> ::tensorflow::OpDef_ArgDef* Arena::CreateMaybeMessage<::tensorflow::OpDef_ArgDef>(Arena*); +template<> ::tensorflow::OpDef_AttrDef* Arena::CreateMaybeMessage<::tensorflow::OpDef_AttrDef>(Arena*); +template<> ::tensorflow::OpDeprecation* Arena::CreateMaybeMessage<::tensorflow::OpDeprecation>(Arena*); +template<> ::tensorflow::OpList* Arena::CreateMaybeMessage<::tensorflow::OpList>(Arena*); +} // namespace protobuf +} // namespace google +namespace tensorflow { + +// =================================================================== + +class OpDef_ArgDef : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.OpDef.ArgDef) */ { + public: + OpDef_ArgDef(); + virtual ~OpDef_ArgDef(); + + OpDef_ArgDef(const OpDef_ArgDef& from); + + inline OpDef_ArgDef& operator=(const OpDef_ArgDef& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + OpDef_ArgDef(OpDef_ArgDef&& from) noexcept + : OpDef_ArgDef() { + *this = ::std::move(from); + } + + inline OpDef_ArgDef& operator=(OpDef_ArgDef&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const OpDef_ArgDef& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const OpDef_ArgDef* internal_default_instance() { + return reinterpret_cast( + &_OpDef_ArgDef_default_instance_); + } + static constexpr int kIndexInFileMessages = + 0; + + void UnsafeArenaSwap(OpDef_ArgDef* other); + void Swap(OpDef_ArgDef* other); + friend void swap(OpDef_ArgDef& a, OpDef_ArgDef& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline OpDef_ArgDef* New() const final { + return CreateMaybeMessage(NULL); + } + + OpDef_ArgDef* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const OpDef_ArgDef& from); + void MergeFrom(const OpDef_ArgDef& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(OpDef_ArgDef* other); + protected: + explicit OpDef_ArgDef(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // string name = 1; + void clear_name(); + static const int kNameFieldNumber = 1; + const ::std::string& name() const; + void set_name(const ::std::string& value); + #if LANG_CXX11 + void set_name(::std::string&& value); + #endif + void set_name(const char* value); + void set_name(const char* value, size_t size); + ::std::string* mutable_name(); + ::std::string* release_name(); + void set_allocated_name(::std::string* name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_name( + ::std::string* name); + + // string description = 2; + void clear_description(); + static const int kDescriptionFieldNumber = 2; + const ::std::string& description() const; + void set_description(const ::std::string& value); + #if LANG_CXX11 + void set_description(::std::string&& value); + #endif + void set_description(const char* value); + void set_description(const char* value, size_t size); + ::std::string* mutable_description(); + ::std::string* release_description(); + void set_allocated_description(::std::string* description); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_description(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_description( + ::std::string* description); + + // string type_attr = 4; + void clear_type_attr(); + static const int kTypeAttrFieldNumber = 4; + const ::std::string& type_attr() const; + void set_type_attr(const ::std::string& value); + #if LANG_CXX11 + void set_type_attr(::std::string&& value); + #endif + void set_type_attr(const char* value); + void set_type_attr(const char* value, size_t size); + ::std::string* mutable_type_attr(); + ::std::string* release_type_attr(); + void set_allocated_type_attr(::std::string* type_attr); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_type_attr(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_type_attr( + ::std::string* type_attr); + + // string number_attr = 5; + void clear_number_attr(); + static const int kNumberAttrFieldNumber = 5; + const ::std::string& number_attr() const; + void set_number_attr(const ::std::string& value); + #if LANG_CXX11 + void set_number_attr(::std::string&& value); + #endif + void set_number_attr(const char* value); + void set_number_attr(const char* value, size_t size); + ::std::string* mutable_number_attr(); + ::std::string* release_number_attr(); + void set_allocated_number_attr(::std::string* number_attr); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_number_attr(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_number_attr( + ::std::string* number_attr); + + // string type_list_attr = 6; + void clear_type_list_attr(); + static const int kTypeListAttrFieldNumber = 6; + const ::std::string& type_list_attr() const; + void set_type_list_attr(const ::std::string& value); + #if LANG_CXX11 + void set_type_list_attr(::std::string&& value); + #endif + void set_type_list_attr(const char* value); + void set_type_list_attr(const char* value, size_t size); + ::std::string* mutable_type_list_attr(); + ::std::string* release_type_list_attr(); + void set_allocated_type_list_attr(::std::string* type_list_attr); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_type_list_attr(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_type_list_attr( + ::std::string* type_list_attr); + + // .tensorflow.DataType type = 3; + void clear_type(); + static const int kTypeFieldNumber = 3; + ::tensorflow::DataType type() const; + void set_type(::tensorflow::DataType value); + + // bool is_ref = 16; + void clear_is_ref(); + static const int kIsRefFieldNumber = 16; + bool is_ref() const; + void set_is_ref(bool value); + + // @@protoc_insertion_point(class_scope:tensorflow.OpDef.ArgDef) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::internal::ArenaStringPtr name_; + ::google::protobuf::internal::ArenaStringPtr description_; + ::google::protobuf::internal::ArenaStringPtr type_attr_; + ::google::protobuf::internal::ArenaStringPtr number_attr_; + ::google::protobuf::internal::ArenaStringPtr type_list_attr_; + int type_; + bool is_ref_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fop_5fdef_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class OpDef_AttrDef : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.OpDef.AttrDef) */ { + public: + OpDef_AttrDef(); + virtual ~OpDef_AttrDef(); + + OpDef_AttrDef(const OpDef_AttrDef& from); + + inline OpDef_AttrDef& operator=(const OpDef_AttrDef& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + OpDef_AttrDef(OpDef_AttrDef&& from) noexcept + : OpDef_AttrDef() { + *this = ::std::move(from); + } + + inline OpDef_AttrDef& operator=(OpDef_AttrDef&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const OpDef_AttrDef& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const OpDef_AttrDef* internal_default_instance() { + return reinterpret_cast( + &_OpDef_AttrDef_default_instance_); + } + static constexpr int kIndexInFileMessages = + 1; + + void UnsafeArenaSwap(OpDef_AttrDef* other); + void Swap(OpDef_AttrDef* other); + friend void swap(OpDef_AttrDef& a, OpDef_AttrDef& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline OpDef_AttrDef* New() const final { + return CreateMaybeMessage(NULL); + } + + OpDef_AttrDef* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const OpDef_AttrDef& from); + void MergeFrom(const OpDef_AttrDef& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(OpDef_AttrDef* other); + protected: + explicit OpDef_AttrDef(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // string name = 1; + void clear_name(); + static const int kNameFieldNumber = 1; + const ::std::string& name() const; + void set_name(const ::std::string& value); + #if LANG_CXX11 + void set_name(::std::string&& value); + #endif + void set_name(const char* value); + void set_name(const char* value, size_t size); + ::std::string* mutable_name(); + ::std::string* release_name(); + void set_allocated_name(::std::string* name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_name( + ::std::string* name); + + // string type = 2; + void clear_type(); + static const int kTypeFieldNumber = 2; + const ::std::string& type() const; + void set_type(const ::std::string& value); + #if LANG_CXX11 + void set_type(::std::string&& value); + #endif + void set_type(const char* value); + void set_type(const char* value, size_t size); + ::std::string* mutable_type(); + ::std::string* release_type(); + void set_allocated_type(::std::string* type); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_type(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_type( + ::std::string* type); + + // string description = 4; + void clear_description(); + static const int kDescriptionFieldNumber = 4; + const ::std::string& description() const; + void set_description(const ::std::string& value); + #if LANG_CXX11 + void set_description(::std::string&& value); + #endif + void set_description(const char* value); + void set_description(const char* value, size_t size); + ::std::string* mutable_description(); + ::std::string* release_description(); + void set_allocated_description(::std::string* description); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_description(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_description( + ::std::string* description); + + // .tensorflow.AttrValue default_value = 3; + bool has_default_value() const; + void clear_default_value(); + static const int kDefaultValueFieldNumber = 3; + private: + const ::tensorflow::AttrValue& _internal_default_value() const; + public: + const ::tensorflow::AttrValue& default_value() const; + ::tensorflow::AttrValue* release_default_value(); + ::tensorflow::AttrValue* mutable_default_value(); + void set_allocated_default_value(::tensorflow::AttrValue* default_value); + void unsafe_arena_set_allocated_default_value( + ::tensorflow::AttrValue* default_value); + ::tensorflow::AttrValue* unsafe_arena_release_default_value(); + + // .tensorflow.AttrValue allowed_values = 7; + bool has_allowed_values() const; + void clear_allowed_values(); + static const int kAllowedValuesFieldNumber = 7; + private: + const ::tensorflow::AttrValue& _internal_allowed_values() const; + public: + const ::tensorflow::AttrValue& allowed_values() const; + ::tensorflow::AttrValue* release_allowed_values(); + ::tensorflow::AttrValue* mutable_allowed_values(); + void set_allocated_allowed_values(::tensorflow::AttrValue* allowed_values); + void unsafe_arena_set_allocated_allowed_values( + ::tensorflow::AttrValue* allowed_values); + ::tensorflow::AttrValue* unsafe_arena_release_allowed_values(); + + // int64 minimum = 6; + void clear_minimum(); + static const int kMinimumFieldNumber = 6; + ::google::protobuf::int64 minimum() const; + void set_minimum(::google::protobuf::int64 value); + + // bool has_minimum = 5; + void clear_has_minimum(); + static const int kHasMinimumFieldNumber = 5; + bool has_minimum() const; + void set_has_minimum(bool value); + + // @@protoc_insertion_point(class_scope:tensorflow.OpDef.AttrDef) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::internal::ArenaStringPtr name_; + ::google::protobuf::internal::ArenaStringPtr type_; + ::google::protobuf::internal::ArenaStringPtr description_; + ::tensorflow::AttrValue* default_value_; + ::tensorflow::AttrValue* allowed_values_; + ::google::protobuf::int64 minimum_; + bool has_minimum_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fop_5fdef_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class OpDef : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.OpDef) */ { + public: + OpDef(); + virtual ~OpDef(); + + OpDef(const OpDef& from); + + inline OpDef& operator=(const OpDef& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + OpDef(OpDef&& from) noexcept + : OpDef() { + *this = ::std::move(from); + } + + inline OpDef& operator=(OpDef&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const OpDef& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const OpDef* internal_default_instance() { + return reinterpret_cast( + &_OpDef_default_instance_); + } + static constexpr int kIndexInFileMessages = + 2; + + void UnsafeArenaSwap(OpDef* other); + void Swap(OpDef* other); + friend void swap(OpDef& a, OpDef& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline OpDef* New() const final { + return CreateMaybeMessage(NULL); + } + + OpDef* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const OpDef& from); + void MergeFrom(const OpDef& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(OpDef* other); + protected: + explicit OpDef(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + typedef OpDef_ArgDef ArgDef; + typedef OpDef_AttrDef AttrDef; + + // accessors ------------------------------------------------------- + + // repeated .tensorflow.OpDef.ArgDef input_arg = 2; + int input_arg_size() const; + void clear_input_arg(); + static const int kInputArgFieldNumber = 2; + ::tensorflow::OpDef_ArgDef* mutable_input_arg(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::OpDef_ArgDef >* + mutable_input_arg(); + const ::tensorflow::OpDef_ArgDef& input_arg(int index) const; + ::tensorflow::OpDef_ArgDef* add_input_arg(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::OpDef_ArgDef >& + input_arg() const; + + // repeated .tensorflow.OpDef.ArgDef output_arg = 3; + int output_arg_size() const; + void clear_output_arg(); + static const int kOutputArgFieldNumber = 3; + ::tensorflow::OpDef_ArgDef* mutable_output_arg(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::OpDef_ArgDef >* + mutable_output_arg(); + const ::tensorflow::OpDef_ArgDef& output_arg(int index) const; + ::tensorflow::OpDef_ArgDef* add_output_arg(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::OpDef_ArgDef >& + output_arg() const; + + // repeated .tensorflow.OpDef.AttrDef attr = 4; + int attr_size() const; + void clear_attr(); + static const int kAttrFieldNumber = 4; + ::tensorflow::OpDef_AttrDef* mutable_attr(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::OpDef_AttrDef >* + mutable_attr(); + const ::tensorflow::OpDef_AttrDef& attr(int index) const; + ::tensorflow::OpDef_AttrDef* add_attr(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::OpDef_AttrDef >& + attr() const; + + // string name = 1; + void clear_name(); + static const int kNameFieldNumber = 1; + const ::std::string& name() const; + void set_name(const ::std::string& value); + #if LANG_CXX11 + void set_name(::std::string&& value); + #endif + void set_name(const char* value); + void set_name(const char* value, size_t size); + ::std::string* mutable_name(); + ::std::string* release_name(); + void set_allocated_name(::std::string* name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_name( + ::std::string* name); + + // string summary = 5; + void clear_summary(); + static const int kSummaryFieldNumber = 5; + const ::std::string& summary() const; + void set_summary(const ::std::string& value); + #if LANG_CXX11 + void set_summary(::std::string&& value); + #endif + void set_summary(const char* value); + void set_summary(const char* value, size_t size); + ::std::string* mutable_summary(); + ::std::string* release_summary(); + void set_allocated_summary(::std::string* summary); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_summary(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_summary( + ::std::string* summary); + + // string description = 6; + void clear_description(); + static const int kDescriptionFieldNumber = 6; + const ::std::string& description() const; + void set_description(const ::std::string& value); + #if LANG_CXX11 + void set_description(::std::string&& value); + #endif + void set_description(const char* value); + void set_description(const char* value, size_t size); + ::std::string* mutable_description(); + ::std::string* release_description(); + void set_allocated_description(::std::string* description); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_description(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_description( + ::std::string* description); + + // .tensorflow.OpDeprecation deprecation = 8; + bool has_deprecation() const; + void clear_deprecation(); + static const int kDeprecationFieldNumber = 8; + private: + const ::tensorflow::OpDeprecation& _internal_deprecation() const; + public: + const ::tensorflow::OpDeprecation& deprecation() const; + ::tensorflow::OpDeprecation* release_deprecation(); + ::tensorflow::OpDeprecation* mutable_deprecation(); + void set_allocated_deprecation(::tensorflow::OpDeprecation* deprecation); + void unsafe_arena_set_allocated_deprecation( + ::tensorflow::OpDeprecation* deprecation); + ::tensorflow::OpDeprecation* unsafe_arena_release_deprecation(); + + // bool is_commutative = 18; + void clear_is_commutative(); + static const int kIsCommutativeFieldNumber = 18; + bool is_commutative() const; + void set_is_commutative(bool value); + + // bool is_aggregate = 16; + void clear_is_aggregate(); + static const int kIsAggregateFieldNumber = 16; + bool is_aggregate() const; + void set_is_aggregate(bool value); + + // bool is_stateful = 17; + void clear_is_stateful(); + static const int kIsStatefulFieldNumber = 17; + bool is_stateful() const; + void set_is_stateful(bool value); + + // bool allows_uninitialized_input = 19; + void clear_allows_uninitialized_input(); + static const int kAllowsUninitializedInputFieldNumber = 19; + bool allows_uninitialized_input() const; + void set_allows_uninitialized_input(bool value); + + // @@protoc_insertion_point(class_scope:tensorflow.OpDef) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::OpDef_ArgDef > input_arg_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::OpDef_ArgDef > output_arg_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::OpDef_AttrDef > attr_; + ::google::protobuf::internal::ArenaStringPtr name_; + ::google::protobuf::internal::ArenaStringPtr summary_; + ::google::protobuf::internal::ArenaStringPtr description_; + ::tensorflow::OpDeprecation* deprecation_; + bool is_commutative_; + bool is_aggregate_; + bool is_stateful_; + bool allows_uninitialized_input_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fop_5fdef_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class OpDeprecation : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.OpDeprecation) */ { + public: + OpDeprecation(); + virtual ~OpDeprecation(); + + OpDeprecation(const OpDeprecation& from); + + inline OpDeprecation& operator=(const OpDeprecation& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + OpDeprecation(OpDeprecation&& from) noexcept + : OpDeprecation() { + *this = ::std::move(from); + } + + inline OpDeprecation& operator=(OpDeprecation&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const OpDeprecation& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const OpDeprecation* internal_default_instance() { + return reinterpret_cast( + &_OpDeprecation_default_instance_); + } + static constexpr int kIndexInFileMessages = + 3; + + void UnsafeArenaSwap(OpDeprecation* other); + void Swap(OpDeprecation* other); + friend void swap(OpDeprecation& a, OpDeprecation& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline OpDeprecation* New() const final { + return CreateMaybeMessage(NULL); + } + + OpDeprecation* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const OpDeprecation& from); + void MergeFrom(const OpDeprecation& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(OpDeprecation* other); + protected: + explicit OpDeprecation(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // string explanation = 2; + void clear_explanation(); + static const int kExplanationFieldNumber = 2; + const ::std::string& explanation() const; + void set_explanation(const ::std::string& value); + #if LANG_CXX11 + void set_explanation(::std::string&& value); + #endif + void set_explanation(const char* value); + void set_explanation(const char* value, size_t size); + ::std::string* mutable_explanation(); + ::std::string* release_explanation(); + void set_allocated_explanation(::std::string* explanation); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_explanation(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_explanation( + ::std::string* explanation); + + // int32 version = 1; + void clear_version(); + static const int kVersionFieldNumber = 1; + ::google::protobuf::int32 version() const; + void set_version(::google::protobuf::int32 value); + + // @@protoc_insertion_point(class_scope:tensorflow.OpDeprecation) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::internal::ArenaStringPtr explanation_; + ::google::protobuf::int32 version_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fop_5fdef_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class OpList : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.OpList) */ { + public: + OpList(); + virtual ~OpList(); + + OpList(const OpList& from); + + inline OpList& operator=(const OpList& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + OpList(OpList&& from) noexcept + : OpList() { + *this = ::std::move(from); + } + + inline OpList& operator=(OpList&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const OpList& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const OpList* internal_default_instance() { + return reinterpret_cast( + &_OpList_default_instance_); + } + static constexpr int kIndexInFileMessages = + 4; + + void UnsafeArenaSwap(OpList* other); + void Swap(OpList* other); + friend void swap(OpList& a, OpList& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline OpList* New() const final { + return CreateMaybeMessage(NULL); + } + + OpList* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const OpList& from); + void MergeFrom(const OpList& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(OpList* other); + protected: + explicit OpList(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated .tensorflow.OpDef op = 1; + int op_size() const; + void clear_op(); + static const int kOpFieldNumber = 1; + ::tensorflow::OpDef* mutable_op(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::OpDef >* + mutable_op(); + const ::tensorflow::OpDef& op(int index) const; + ::tensorflow::OpDef* add_op(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::OpDef >& + op() const; + + // @@protoc_insertion_point(class_scope:tensorflow.OpList) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::OpDef > op_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fop_5fdef_2eproto::TableStruct; +}; +// =================================================================== + + +// =================================================================== + +#ifdef __GNUC__ + #pragma GCC diagnostic push + #pragma GCC diagnostic ignored "-Wstrict-aliasing" +#endif // __GNUC__ +// OpDef_ArgDef + +// string name = 1; +inline void OpDef_ArgDef::clear_name() { + name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& OpDef_ArgDef::name() const { + // @@protoc_insertion_point(field_get:tensorflow.OpDef.ArgDef.name) + return name_.Get(); +} +inline void OpDef_ArgDef::set_name(const ::std::string& value) { + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.OpDef.ArgDef.name) +} +#if LANG_CXX11 +inline void OpDef_ArgDef::set_name(::std::string&& value) { + + name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.OpDef.ArgDef.name) +} +#endif +inline void OpDef_ArgDef::set_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.OpDef.ArgDef.name) +} +inline void OpDef_ArgDef::set_name(const char* value, + size_t size) { + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.OpDef.ArgDef.name) +} +inline ::std::string* OpDef_ArgDef::mutable_name() { + + // @@protoc_insertion_point(field_mutable:tensorflow.OpDef.ArgDef.name) + return name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* OpDef_ArgDef::release_name() { + // @@protoc_insertion_point(field_release:tensorflow.OpDef.ArgDef.name) + + return name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void OpDef_ArgDef::set_allocated_name(::std::string* name) { + if (name != NULL) { + + } else { + + } + name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.OpDef.ArgDef.name) +} +inline ::std::string* OpDef_ArgDef::unsafe_arena_release_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.OpDef.ArgDef.name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void OpDef_ArgDef::unsafe_arena_set_allocated_name( + ::std::string* name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (name != NULL) { + + } else { + + } + name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.OpDef.ArgDef.name) +} + +// string description = 2; +inline void OpDef_ArgDef::clear_description() { + description_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& OpDef_ArgDef::description() const { + // @@protoc_insertion_point(field_get:tensorflow.OpDef.ArgDef.description) + return description_.Get(); +} +inline void OpDef_ArgDef::set_description(const ::std::string& value) { + + description_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.OpDef.ArgDef.description) +} +#if LANG_CXX11 +inline void OpDef_ArgDef::set_description(::std::string&& value) { + + description_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.OpDef.ArgDef.description) +} +#endif +inline void OpDef_ArgDef::set_description(const char* value) { + GOOGLE_DCHECK(value != NULL); + + description_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.OpDef.ArgDef.description) +} +inline void OpDef_ArgDef::set_description(const char* value, + size_t size) { + + description_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.OpDef.ArgDef.description) +} +inline ::std::string* OpDef_ArgDef::mutable_description() { + + // @@protoc_insertion_point(field_mutable:tensorflow.OpDef.ArgDef.description) + return description_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* OpDef_ArgDef::release_description() { + // @@protoc_insertion_point(field_release:tensorflow.OpDef.ArgDef.description) + + return description_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void OpDef_ArgDef::set_allocated_description(::std::string* description) { + if (description != NULL) { + + } else { + + } + description_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), description, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.OpDef.ArgDef.description) +} +inline ::std::string* OpDef_ArgDef::unsafe_arena_release_description() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.OpDef.ArgDef.description) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return description_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void OpDef_ArgDef::unsafe_arena_set_allocated_description( + ::std::string* description) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (description != NULL) { + + } else { + + } + description_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + description, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.OpDef.ArgDef.description) +} + +// .tensorflow.DataType type = 3; +inline void OpDef_ArgDef::clear_type() { + type_ = 0; +} +inline ::tensorflow::DataType OpDef_ArgDef::type() const { + // @@protoc_insertion_point(field_get:tensorflow.OpDef.ArgDef.type) + return static_cast< ::tensorflow::DataType >(type_); +} +inline void OpDef_ArgDef::set_type(::tensorflow::DataType value) { + + type_ = value; + // @@protoc_insertion_point(field_set:tensorflow.OpDef.ArgDef.type) +} + +// string type_attr = 4; +inline void OpDef_ArgDef::clear_type_attr() { + type_attr_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& OpDef_ArgDef::type_attr() const { + // @@protoc_insertion_point(field_get:tensorflow.OpDef.ArgDef.type_attr) + return type_attr_.Get(); +} +inline void OpDef_ArgDef::set_type_attr(const ::std::string& value) { + + type_attr_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.OpDef.ArgDef.type_attr) +} +#if LANG_CXX11 +inline void OpDef_ArgDef::set_type_attr(::std::string&& value) { + + type_attr_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.OpDef.ArgDef.type_attr) +} +#endif +inline void OpDef_ArgDef::set_type_attr(const char* value) { + GOOGLE_DCHECK(value != NULL); + + type_attr_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.OpDef.ArgDef.type_attr) +} +inline void OpDef_ArgDef::set_type_attr(const char* value, + size_t size) { + + type_attr_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.OpDef.ArgDef.type_attr) +} +inline ::std::string* OpDef_ArgDef::mutable_type_attr() { + + // @@protoc_insertion_point(field_mutable:tensorflow.OpDef.ArgDef.type_attr) + return type_attr_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* OpDef_ArgDef::release_type_attr() { + // @@protoc_insertion_point(field_release:tensorflow.OpDef.ArgDef.type_attr) + + return type_attr_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void OpDef_ArgDef::set_allocated_type_attr(::std::string* type_attr) { + if (type_attr != NULL) { + + } else { + + } + type_attr_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), type_attr, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.OpDef.ArgDef.type_attr) +} +inline ::std::string* OpDef_ArgDef::unsafe_arena_release_type_attr() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.OpDef.ArgDef.type_attr) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return type_attr_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void OpDef_ArgDef::unsafe_arena_set_allocated_type_attr( + ::std::string* type_attr) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (type_attr != NULL) { + + } else { + + } + type_attr_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + type_attr, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.OpDef.ArgDef.type_attr) +} + +// string number_attr = 5; +inline void OpDef_ArgDef::clear_number_attr() { + number_attr_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& OpDef_ArgDef::number_attr() const { + // @@protoc_insertion_point(field_get:tensorflow.OpDef.ArgDef.number_attr) + return number_attr_.Get(); +} +inline void OpDef_ArgDef::set_number_attr(const ::std::string& value) { + + number_attr_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.OpDef.ArgDef.number_attr) +} +#if LANG_CXX11 +inline void OpDef_ArgDef::set_number_attr(::std::string&& value) { + + number_attr_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.OpDef.ArgDef.number_attr) +} +#endif +inline void OpDef_ArgDef::set_number_attr(const char* value) { + GOOGLE_DCHECK(value != NULL); + + number_attr_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.OpDef.ArgDef.number_attr) +} +inline void OpDef_ArgDef::set_number_attr(const char* value, + size_t size) { + + number_attr_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.OpDef.ArgDef.number_attr) +} +inline ::std::string* OpDef_ArgDef::mutable_number_attr() { + + // @@protoc_insertion_point(field_mutable:tensorflow.OpDef.ArgDef.number_attr) + return number_attr_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* OpDef_ArgDef::release_number_attr() { + // @@protoc_insertion_point(field_release:tensorflow.OpDef.ArgDef.number_attr) + + return number_attr_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void OpDef_ArgDef::set_allocated_number_attr(::std::string* number_attr) { + if (number_attr != NULL) { + + } else { + + } + number_attr_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), number_attr, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.OpDef.ArgDef.number_attr) +} +inline ::std::string* OpDef_ArgDef::unsafe_arena_release_number_attr() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.OpDef.ArgDef.number_attr) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return number_attr_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void OpDef_ArgDef::unsafe_arena_set_allocated_number_attr( + ::std::string* number_attr) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (number_attr != NULL) { + + } else { + + } + number_attr_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + number_attr, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.OpDef.ArgDef.number_attr) +} + +// string type_list_attr = 6; +inline void OpDef_ArgDef::clear_type_list_attr() { + type_list_attr_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& OpDef_ArgDef::type_list_attr() const { + // @@protoc_insertion_point(field_get:tensorflow.OpDef.ArgDef.type_list_attr) + return type_list_attr_.Get(); +} +inline void OpDef_ArgDef::set_type_list_attr(const ::std::string& value) { + + type_list_attr_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.OpDef.ArgDef.type_list_attr) +} +#if LANG_CXX11 +inline void OpDef_ArgDef::set_type_list_attr(::std::string&& value) { + + type_list_attr_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.OpDef.ArgDef.type_list_attr) +} +#endif +inline void OpDef_ArgDef::set_type_list_attr(const char* value) { + GOOGLE_DCHECK(value != NULL); + + type_list_attr_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.OpDef.ArgDef.type_list_attr) +} +inline void OpDef_ArgDef::set_type_list_attr(const char* value, + size_t size) { + + type_list_attr_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.OpDef.ArgDef.type_list_attr) +} +inline ::std::string* OpDef_ArgDef::mutable_type_list_attr() { + + // @@protoc_insertion_point(field_mutable:tensorflow.OpDef.ArgDef.type_list_attr) + return type_list_attr_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* OpDef_ArgDef::release_type_list_attr() { + // @@protoc_insertion_point(field_release:tensorflow.OpDef.ArgDef.type_list_attr) + + return type_list_attr_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void OpDef_ArgDef::set_allocated_type_list_attr(::std::string* type_list_attr) { + if (type_list_attr != NULL) { + + } else { + + } + type_list_attr_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), type_list_attr, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.OpDef.ArgDef.type_list_attr) +} +inline ::std::string* OpDef_ArgDef::unsafe_arena_release_type_list_attr() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.OpDef.ArgDef.type_list_attr) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return type_list_attr_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void OpDef_ArgDef::unsafe_arena_set_allocated_type_list_attr( + ::std::string* type_list_attr) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (type_list_attr != NULL) { + + } else { + + } + type_list_attr_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + type_list_attr, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.OpDef.ArgDef.type_list_attr) +} + +// bool is_ref = 16; +inline void OpDef_ArgDef::clear_is_ref() { + is_ref_ = false; +} +inline bool OpDef_ArgDef::is_ref() const { + // @@protoc_insertion_point(field_get:tensorflow.OpDef.ArgDef.is_ref) + return is_ref_; +} +inline void OpDef_ArgDef::set_is_ref(bool value) { + + is_ref_ = value; + // @@protoc_insertion_point(field_set:tensorflow.OpDef.ArgDef.is_ref) +} + +// ------------------------------------------------------------------- + +// OpDef_AttrDef + +// string name = 1; +inline void OpDef_AttrDef::clear_name() { + name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& OpDef_AttrDef::name() const { + // @@protoc_insertion_point(field_get:tensorflow.OpDef.AttrDef.name) + return name_.Get(); +} +inline void OpDef_AttrDef::set_name(const ::std::string& value) { + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.OpDef.AttrDef.name) +} +#if LANG_CXX11 +inline void OpDef_AttrDef::set_name(::std::string&& value) { + + name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.OpDef.AttrDef.name) +} +#endif +inline void OpDef_AttrDef::set_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.OpDef.AttrDef.name) +} +inline void OpDef_AttrDef::set_name(const char* value, + size_t size) { + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.OpDef.AttrDef.name) +} +inline ::std::string* OpDef_AttrDef::mutable_name() { + + // @@protoc_insertion_point(field_mutable:tensorflow.OpDef.AttrDef.name) + return name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* OpDef_AttrDef::release_name() { + // @@protoc_insertion_point(field_release:tensorflow.OpDef.AttrDef.name) + + return name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void OpDef_AttrDef::set_allocated_name(::std::string* name) { + if (name != NULL) { + + } else { + + } + name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.OpDef.AttrDef.name) +} +inline ::std::string* OpDef_AttrDef::unsafe_arena_release_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.OpDef.AttrDef.name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void OpDef_AttrDef::unsafe_arena_set_allocated_name( + ::std::string* name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (name != NULL) { + + } else { + + } + name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.OpDef.AttrDef.name) +} + +// string type = 2; +inline void OpDef_AttrDef::clear_type() { + type_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& OpDef_AttrDef::type() const { + // @@protoc_insertion_point(field_get:tensorflow.OpDef.AttrDef.type) + return type_.Get(); +} +inline void OpDef_AttrDef::set_type(const ::std::string& value) { + + type_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.OpDef.AttrDef.type) +} +#if LANG_CXX11 +inline void OpDef_AttrDef::set_type(::std::string&& value) { + + type_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.OpDef.AttrDef.type) +} +#endif +inline void OpDef_AttrDef::set_type(const char* value) { + GOOGLE_DCHECK(value != NULL); + + type_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.OpDef.AttrDef.type) +} +inline void OpDef_AttrDef::set_type(const char* value, + size_t size) { + + type_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.OpDef.AttrDef.type) +} +inline ::std::string* OpDef_AttrDef::mutable_type() { + + // @@protoc_insertion_point(field_mutable:tensorflow.OpDef.AttrDef.type) + return type_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* OpDef_AttrDef::release_type() { + // @@protoc_insertion_point(field_release:tensorflow.OpDef.AttrDef.type) + + return type_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void OpDef_AttrDef::set_allocated_type(::std::string* type) { + if (type != NULL) { + + } else { + + } + type_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), type, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.OpDef.AttrDef.type) +} +inline ::std::string* OpDef_AttrDef::unsafe_arena_release_type() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.OpDef.AttrDef.type) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return type_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void OpDef_AttrDef::unsafe_arena_set_allocated_type( + ::std::string* type) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (type != NULL) { + + } else { + + } + type_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + type, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.OpDef.AttrDef.type) +} + +// .tensorflow.AttrValue default_value = 3; +inline bool OpDef_AttrDef::has_default_value() const { + return this != internal_default_instance() && default_value_ != NULL; +} +inline const ::tensorflow::AttrValue& OpDef_AttrDef::_internal_default_value() const { + return *default_value_; +} +inline const ::tensorflow::AttrValue& OpDef_AttrDef::default_value() const { + const ::tensorflow::AttrValue* p = default_value_; + // @@protoc_insertion_point(field_get:tensorflow.OpDef.AttrDef.default_value) + return p != NULL ? *p : *reinterpret_cast( + &::tensorflow::_AttrValue_default_instance_); +} +inline ::tensorflow::AttrValue* OpDef_AttrDef::release_default_value() { + // @@protoc_insertion_point(field_release:tensorflow.OpDef.AttrDef.default_value) + + ::tensorflow::AttrValue* temp = default_value_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + default_value_ = NULL; + return temp; +} +inline ::tensorflow::AttrValue* OpDef_AttrDef::unsafe_arena_release_default_value() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.OpDef.AttrDef.default_value) + + ::tensorflow::AttrValue* temp = default_value_; + default_value_ = NULL; + return temp; +} +inline ::tensorflow::AttrValue* OpDef_AttrDef::mutable_default_value() { + + if (default_value_ == NULL) { + auto* p = CreateMaybeMessage<::tensorflow::AttrValue>(GetArenaNoVirtual()); + default_value_ = p; + } + // @@protoc_insertion_point(field_mutable:tensorflow.OpDef.AttrDef.default_value) + return default_value_; +} +inline void OpDef_AttrDef::set_allocated_default_value(::tensorflow::AttrValue* default_value) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(default_value_); + } + if (default_value) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(default_value)->GetArena(); + if (message_arena != submessage_arena) { + default_value = ::google::protobuf::internal::GetOwnedMessage( + message_arena, default_value, submessage_arena); + } + + } else { + + } + default_value_ = default_value; + // @@protoc_insertion_point(field_set_allocated:tensorflow.OpDef.AttrDef.default_value) +} + +// string description = 4; +inline void OpDef_AttrDef::clear_description() { + description_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& OpDef_AttrDef::description() const { + // @@protoc_insertion_point(field_get:tensorflow.OpDef.AttrDef.description) + return description_.Get(); +} +inline void OpDef_AttrDef::set_description(const ::std::string& value) { + + description_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.OpDef.AttrDef.description) +} +#if LANG_CXX11 +inline void OpDef_AttrDef::set_description(::std::string&& value) { + + description_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.OpDef.AttrDef.description) +} +#endif +inline void OpDef_AttrDef::set_description(const char* value) { + GOOGLE_DCHECK(value != NULL); + + description_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.OpDef.AttrDef.description) +} +inline void OpDef_AttrDef::set_description(const char* value, + size_t size) { + + description_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.OpDef.AttrDef.description) +} +inline ::std::string* OpDef_AttrDef::mutable_description() { + + // @@protoc_insertion_point(field_mutable:tensorflow.OpDef.AttrDef.description) + return description_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* OpDef_AttrDef::release_description() { + // @@protoc_insertion_point(field_release:tensorflow.OpDef.AttrDef.description) + + return description_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void OpDef_AttrDef::set_allocated_description(::std::string* description) { + if (description != NULL) { + + } else { + + } + description_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), description, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.OpDef.AttrDef.description) +} +inline ::std::string* OpDef_AttrDef::unsafe_arena_release_description() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.OpDef.AttrDef.description) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return description_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void OpDef_AttrDef::unsafe_arena_set_allocated_description( + ::std::string* description) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (description != NULL) { + + } else { + + } + description_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + description, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.OpDef.AttrDef.description) +} + +// bool has_minimum = 5; +inline void OpDef_AttrDef::clear_has_minimum() { + has_minimum_ = false; +} +inline bool OpDef_AttrDef::has_minimum() const { + // @@protoc_insertion_point(field_get:tensorflow.OpDef.AttrDef.has_minimum) + return has_minimum_; +} +inline void OpDef_AttrDef::set_has_minimum(bool value) { + + has_minimum_ = value; + // @@protoc_insertion_point(field_set:tensorflow.OpDef.AttrDef.has_minimum) +} + +// int64 minimum = 6; +inline void OpDef_AttrDef::clear_minimum() { + minimum_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 OpDef_AttrDef::minimum() const { + // @@protoc_insertion_point(field_get:tensorflow.OpDef.AttrDef.minimum) + return minimum_; +} +inline void OpDef_AttrDef::set_minimum(::google::protobuf::int64 value) { + + minimum_ = value; + // @@protoc_insertion_point(field_set:tensorflow.OpDef.AttrDef.minimum) +} + +// .tensorflow.AttrValue allowed_values = 7; +inline bool OpDef_AttrDef::has_allowed_values() const { + return this != internal_default_instance() && allowed_values_ != NULL; +} +inline const ::tensorflow::AttrValue& OpDef_AttrDef::_internal_allowed_values() const { + return *allowed_values_; +} +inline const ::tensorflow::AttrValue& OpDef_AttrDef::allowed_values() const { + const ::tensorflow::AttrValue* p = allowed_values_; + // @@protoc_insertion_point(field_get:tensorflow.OpDef.AttrDef.allowed_values) + return p != NULL ? *p : *reinterpret_cast( + &::tensorflow::_AttrValue_default_instance_); +} +inline ::tensorflow::AttrValue* OpDef_AttrDef::release_allowed_values() { + // @@protoc_insertion_point(field_release:tensorflow.OpDef.AttrDef.allowed_values) + + ::tensorflow::AttrValue* temp = allowed_values_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + allowed_values_ = NULL; + return temp; +} +inline ::tensorflow::AttrValue* OpDef_AttrDef::unsafe_arena_release_allowed_values() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.OpDef.AttrDef.allowed_values) + + ::tensorflow::AttrValue* temp = allowed_values_; + allowed_values_ = NULL; + return temp; +} +inline ::tensorflow::AttrValue* OpDef_AttrDef::mutable_allowed_values() { + + if (allowed_values_ == NULL) { + auto* p = CreateMaybeMessage<::tensorflow::AttrValue>(GetArenaNoVirtual()); + allowed_values_ = p; + } + // @@protoc_insertion_point(field_mutable:tensorflow.OpDef.AttrDef.allowed_values) + return allowed_values_; +} +inline void OpDef_AttrDef::set_allocated_allowed_values(::tensorflow::AttrValue* allowed_values) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(allowed_values_); + } + if (allowed_values) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(allowed_values)->GetArena(); + if (message_arena != submessage_arena) { + allowed_values = ::google::protobuf::internal::GetOwnedMessage( + message_arena, allowed_values, submessage_arena); + } + + } else { + + } + allowed_values_ = allowed_values; + // @@protoc_insertion_point(field_set_allocated:tensorflow.OpDef.AttrDef.allowed_values) +} + +// ------------------------------------------------------------------- + +// OpDef + +// string name = 1; +inline void OpDef::clear_name() { + name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& OpDef::name() const { + // @@protoc_insertion_point(field_get:tensorflow.OpDef.name) + return name_.Get(); +} +inline void OpDef::set_name(const ::std::string& value) { + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.OpDef.name) +} +#if LANG_CXX11 +inline void OpDef::set_name(::std::string&& value) { + + name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.OpDef.name) +} +#endif +inline void OpDef::set_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.OpDef.name) +} +inline void OpDef::set_name(const char* value, + size_t size) { + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.OpDef.name) +} +inline ::std::string* OpDef::mutable_name() { + + // @@protoc_insertion_point(field_mutable:tensorflow.OpDef.name) + return name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* OpDef::release_name() { + // @@protoc_insertion_point(field_release:tensorflow.OpDef.name) + + return name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void OpDef::set_allocated_name(::std::string* name) { + if (name != NULL) { + + } else { + + } + name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.OpDef.name) +} +inline ::std::string* OpDef::unsafe_arena_release_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.OpDef.name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void OpDef::unsafe_arena_set_allocated_name( + ::std::string* name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (name != NULL) { + + } else { + + } + name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.OpDef.name) +} + +// repeated .tensorflow.OpDef.ArgDef input_arg = 2; +inline int OpDef::input_arg_size() const { + return input_arg_.size(); +} +inline void OpDef::clear_input_arg() { + input_arg_.Clear(); +} +inline ::tensorflow::OpDef_ArgDef* OpDef::mutable_input_arg(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.OpDef.input_arg) + return input_arg_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::OpDef_ArgDef >* +OpDef::mutable_input_arg() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.OpDef.input_arg) + return &input_arg_; +} +inline const ::tensorflow::OpDef_ArgDef& OpDef::input_arg(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.OpDef.input_arg) + return input_arg_.Get(index); +} +inline ::tensorflow::OpDef_ArgDef* OpDef::add_input_arg() { + // @@protoc_insertion_point(field_add:tensorflow.OpDef.input_arg) + return input_arg_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::OpDef_ArgDef >& +OpDef::input_arg() const { + // @@protoc_insertion_point(field_list:tensorflow.OpDef.input_arg) + return input_arg_; +} + +// repeated .tensorflow.OpDef.ArgDef output_arg = 3; +inline int OpDef::output_arg_size() const { + return output_arg_.size(); +} +inline void OpDef::clear_output_arg() { + output_arg_.Clear(); +} +inline ::tensorflow::OpDef_ArgDef* OpDef::mutable_output_arg(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.OpDef.output_arg) + return output_arg_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::OpDef_ArgDef >* +OpDef::mutable_output_arg() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.OpDef.output_arg) + return &output_arg_; +} +inline const ::tensorflow::OpDef_ArgDef& OpDef::output_arg(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.OpDef.output_arg) + return output_arg_.Get(index); +} +inline ::tensorflow::OpDef_ArgDef* OpDef::add_output_arg() { + // @@protoc_insertion_point(field_add:tensorflow.OpDef.output_arg) + return output_arg_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::OpDef_ArgDef >& +OpDef::output_arg() const { + // @@protoc_insertion_point(field_list:tensorflow.OpDef.output_arg) + return output_arg_; +} + +// repeated .tensorflow.OpDef.AttrDef attr = 4; +inline int OpDef::attr_size() const { + return attr_.size(); +} +inline void OpDef::clear_attr() { + attr_.Clear(); +} +inline ::tensorflow::OpDef_AttrDef* OpDef::mutable_attr(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.OpDef.attr) + return attr_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::OpDef_AttrDef >* +OpDef::mutable_attr() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.OpDef.attr) + return &attr_; +} +inline const ::tensorflow::OpDef_AttrDef& OpDef::attr(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.OpDef.attr) + return attr_.Get(index); +} +inline ::tensorflow::OpDef_AttrDef* OpDef::add_attr() { + // @@protoc_insertion_point(field_add:tensorflow.OpDef.attr) + return attr_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::OpDef_AttrDef >& +OpDef::attr() const { + // @@protoc_insertion_point(field_list:tensorflow.OpDef.attr) + return attr_; +} + +// .tensorflow.OpDeprecation deprecation = 8; +inline bool OpDef::has_deprecation() const { + return this != internal_default_instance() && deprecation_ != NULL; +} +inline void OpDef::clear_deprecation() { + if (GetArenaNoVirtual() == NULL && deprecation_ != NULL) { + delete deprecation_; + } + deprecation_ = NULL; +} +inline const ::tensorflow::OpDeprecation& OpDef::_internal_deprecation() const { + return *deprecation_; +} +inline const ::tensorflow::OpDeprecation& OpDef::deprecation() const { + const ::tensorflow::OpDeprecation* p = deprecation_; + // @@protoc_insertion_point(field_get:tensorflow.OpDef.deprecation) + return p != NULL ? *p : *reinterpret_cast( + &::tensorflow::_OpDeprecation_default_instance_); +} +inline ::tensorflow::OpDeprecation* OpDef::release_deprecation() { + // @@protoc_insertion_point(field_release:tensorflow.OpDef.deprecation) + + ::tensorflow::OpDeprecation* temp = deprecation_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + deprecation_ = NULL; + return temp; +} +inline ::tensorflow::OpDeprecation* OpDef::unsafe_arena_release_deprecation() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.OpDef.deprecation) + + ::tensorflow::OpDeprecation* temp = deprecation_; + deprecation_ = NULL; + return temp; +} +inline ::tensorflow::OpDeprecation* OpDef::mutable_deprecation() { + + if (deprecation_ == NULL) { + auto* p = CreateMaybeMessage<::tensorflow::OpDeprecation>(GetArenaNoVirtual()); + deprecation_ = p; + } + // @@protoc_insertion_point(field_mutable:tensorflow.OpDef.deprecation) + return deprecation_; +} +inline void OpDef::set_allocated_deprecation(::tensorflow::OpDeprecation* deprecation) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete deprecation_; + } + if (deprecation) { + ::google::protobuf::Arena* submessage_arena = + ::google::protobuf::Arena::GetArena(deprecation); + if (message_arena != submessage_arena) { + deprecation = ::google::protobuf::internal::GetOwnedMessage( + message_arena, deprecation, submessage_arena); + } + + } else { + + } + deprecation_ = deprecation; + // @@protoc_insertion_point(field_set_allocated:tensorflow.OpDef.deprecation) +} + +// string summary = 5; +inline void OpDef::clear_summary() { + summary_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& OpDef::summary() const { + // @@protoc_insertion_point(field_get:tensorflow.OpDef.summary) + return summary_.Get(); +} +inline void OpDef::set_summary(const ::std::string& value) { + + summary_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.OpDef.summary) +} +#if LANG_CXX11 +inline void OpDef::set_summary(::std::string&& value) { + + summary_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.OpDef.summary) +} +#endif +inline void OpDef::set_summary(const char* value) { + GOOGLE_DCHECK(value != NULL); + + summary_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.OpDef.summary) +} +inline void OpDef::set_summary(const char* value, + size_t size) { + + summary_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.OpDef.summary) +} +inline ::std::string* OpDef::mutable_summary() { + + // @@protoc_insertion_point(field_mutable:tensorflow.OpDef.summary) + return summary_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* OpDef::release_summary() { + // @@protoc_insertion_point(field_release:tensorflow.OpDef.summary) + + return summary_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void OpDef::set_allocated_summary(::std::string* summary) { + if (summary != NULL) { + + } else { + + } + summary_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), summary, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.OpDef.summary) +} +inline ::std::string* OpDef::unsafe_arena_release_summary() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.OpDef.summary) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return summary_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void OpDef::unsafe_arena_set_allocated_summary( + ::std::string* summary) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (summary != NULL) { + + } else { + + } + summary_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + summary, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.OpDef.summary) +} + +// string description = 6; +inline void OpDef::clear_description() { + description_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& OpDef::description() const { + // @@protoc_insertion_point(field_get:tensorflow.OpDef.description) + return description_.Get(); +} +inline void OpDef::set_description(const ::std::string& value) { + + description_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.OpDef.description) +} +#if LANG_CXX11 +inline void OpDef::set_description(::std::string&& value) { + + description_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.OpDef.description) +} +#endif +inline void OpDef::set_description(const char* value) { + GOOGLE_DCHECK(value != NULL); + + description_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.OpDef.description) +} +inline void OpDef::set_description(const char* value, + size_t size) { + + description_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.OpDef.description) +} +inline ::std::string* OpDef::mutable_description() { + + // @@protoc_insertion_point(field_mutable:tensorflow.OpDef.description) + return description_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* OpDef::release_description() { + // @@protoc_insertion_point(field_release:tensorflow.OpDef.description) + + return description_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void OpDef::set_allocated_description(::std::string* description) { + if (description != NULL) { + + } else { + + } + description_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), description, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.OpDef.description) +} +inline ::std::string* OpDef::unsafe_arena_release_description() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.OpDef.description) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return description_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void OpDef::unsafe_arena_set_allocated_description( + ::std::string* description) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (description != NULL) { + + } else { + + } + description_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + description, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.OpDef.description) +} + +// bool is_commutative = 18; +inline void OpDef::clear_is_commutative() { + is_commutative_ = false; +} +inline bool OpDef::is_commutative() const { + // @@protoc_insertion_point(field_get:tensorflow.OpDef.is_commutative) + return is_commutative_; +} +inline void OpDef::set_is_commutative(bool value) { + + is_commutative_ = value; + // @@protoc_insertion_point(field_set:tensorflow.OpDef.is_commutative) +} + +// bool is_aggregate = 16; +inline void OpDef::clear_is_aggregate() { + is_aggregate_ = false; +} +inline bool OpDef::is_aggregate() const { + // @@protoc_insertion_point(field_get:tensorflow.OpDef.is_aggregate) + return is_aggregate_; +} +inline void OpDef::set_is_aggregate(bool value) { + + is_aggregate_ = value; + // @@protoc_insertion_point(field_set:tensorflow.OpDef.is_aggregate) +} + +// bool is_stateful = 17; +inline void OpDef::clear_is_stateful() { + is_stateful_ = false; +} +inline bool OpDef::is_stateful() const { + // @@protoc_insertion_point(field_get:tensorflow.OpDef.is_stateful) + return is_stateful_; +} +inline void OpDef::set_is_stateful(bool value) { + + is_stateful_ = value; + // @@protoc_insertion_point(field_set:tensorflow.OpDef.is_stateful) +} + +// bool allows_uninitialized_input = 19; +inline void OpDef::clear_allows_uninitialized_input() { + allows_uninitialized_input_ = false; +} +inline bool OpDef::allows_uninitialized_input() const { + // @@protoc_insertion_point(field_get:tensorflow.OpDef.allows_uninitialized_input) + return allows_uninitialized_input_; +} +inline void OpDef::set_allows_uninitialized_input(bool value) { + + allows_uninitialized_input_ = value; + // @@protoc_insertion_point(field_set:tensorflow.OpDef.allows_uninitialized_input) +} + +// ------------------------------------------------------------------- + +// OpDeprecation + +// int32 version = 1; +inline void OpDeprecation::clear_version() { + version_ = 0; +} +inline ::google::protobuf::int32 OpDeprecation::version() const { + // @@protoc_insertion_point(field_get:tensorflow.OpDeprecation.version) + return version_; +} +inline void OpDeprecation::set_version(::google::protobuf::int32 value) { + + version_ = value; + // @@protoc_insertion_point(field_set:tensorflow.OpDeprecation.version) +} + +// string explanation = 2; +inline void OpDeprecation::clear_explanation() { + explanation_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& OpDeprecation::explanation() const { + // @@protoc_insertion_point(field_get:tensorflow.OpDeprecation.explanation) + return explanation_.Get(); +} +inline void OpDeprecation::set_explanation(const ::std::string& value) { + + explanation_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.OpDeprecation.explanation) +} +#if LANG_CXX11 +inline void OpDeprecation::set_explanation(::std::string&& value) { + + explanation_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.OpDeprecation.explanation) +} +#endif +inline void OpDeprecation::set_explanation(const char* value) { + GOOGLE_DCHECK(value != NULL); + + explanation_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.OpDeprecation.explanation) +} +inline void OpDeprecation::set_explanation(const char* value, + size_t size) { + + explanation_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.OpDeprecation.explanation) +} +inline ::std::string* OpDeprecation::mutable_explanation() { + + // @@protoc_insertion_point(field_mutable:tensorflow.OpDeprecation.explanation) + return explanation_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* OpDeprecation::release_explanation() { + // @@protoc_insertion_point(field_release:tensorflow.OpDeprecation.explanation) + + return explanation_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void OpDeprecation::set_allocated_explanation(::std::string* explanation) { + if (explanation != NULL) { + + } else { + + } + explanation_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), explanation, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.OpDeprecation.explanation) +} +inline ::std::string* OpDeprecation::unsafe_arena_release_explanation() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.OpDeprecation.explanation) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return explanation_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void OpDeprecation::unsafe_arena_set_allocated_explanation( + ::std::string* explanation) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (explanation != NULL) { + + } else { + + } + explanation_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + explanation, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.OpDeprecation.explanation) +} + +// ------------------------------------------------------------------- + +// OpList + +// repeated .tensorflow.OpDef op = 1; +inline int OpList::op_size() const { + return op_.size(); +} +inline void OpList::clear_op() { + op_.Clear(); +} +inline ::tensorflow::OpDef* OpList::mutable_op(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.OpList.op) + return op_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::OpDef >* +OpList::mutable_op() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.OpList.op) + return &op_; +} +inline const ::tensorflow::OpDef& OpList::op(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.OpList.op) + return op_.Get(index); +} +inline ::tensorflow::OpDef* OpList::add_op() { + // @@protoc_insertion_point(field_add:tensorflow.OpList.op) + return op_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::OpDef >& +OpList::op() const { + // @@protoc_insertion_point(field_list:tensorflow.OpList.op) + return op_; +} + +#ifdef __GNUC__ + #pragma GCC diagnostic pop +#endif // __GNUC__ +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + + +// @@protoc_insertion_point(namespace_scope) + +} // namespace tensorflow + +// @@protoc_insertion_point(global_scope) + +#endif // PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2fop_5fdef_2eproto diff --git a/linux/include/tensorflow/core/framework/op_def.pb_text-impl.h b/linux/include/tensorflow/core/framework/op_def.pb_text-impl.h new file mode 100644 index 000000000..561c6b683 --- /dev/null +++ b/linux/include/tensorflow/core/framework/op_def.pb_text-impl.h @@ -0,0 +1,63 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_framework_op_def_proto_IMPL_H_ +#define tensorflow_core_framework_op_def_proto_IMPL_H_ + +#include "tensorflow/core/framework/attr_value.pb.h" +#include "tensorflow/core/framework/attr_value.pb_text-impl.h" +#include "tensorflow/core/framework/op_def.pb.h" +#include "tensorflow/core/framework/op_def.pb_text.h" +#include "tensorflow/core/framework/resource_handle.pb.h" +#include "tensorflow/core/framework/resource_handle.pb_text-impl.h" +#include "tensorflow/core/framework/tensor.pb.h" +#include "tensorflow/core/framework/tensor.pb_text-impl.h" +#include "tensorflow/core/framework/tensor_shape.pb.h" +#include "tensorflow/core/framework/tensor_shape.pb_text-impl.h" +#include "tensorflow/core/framework/types.pb.h" +#include "tensorflow/core/framework/types.pb_text-impl.h" +#include "tensorflow/core/lib/strings/proto_text_util.h" +#include "tensorflow/core/lib/strings/scanner.h" + +namespace tensorflow { + +namespace internal { + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::OpDef_ArgDef& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::OpDef_ArgDef* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::OpDef_AttrDef& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::OpDef_AttrDef* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::OpDef& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::OpDef* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::OpDeprecation& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::OpDeprecation* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::OpList& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::OpList* msg); + +} // namespace internal + +} // namespace tensorflow + +#endif // tensorflow_core_framework_op_def_proto_IMPL_H_ diff --git a/linux/include/tensorflow/core/framework/op_def.pb_text.h b/linux/include/tensorflow/core/framework/op_def.pb_text.h new file mode 100644 index 000000000..8d3a75657 --- /dev/null +++ b/linux/include/tensorflow/core/framework/op_def.pb_text.h @@ -0,0 +1,64 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_framework_op_def_proto_H_ +#define tensorflow_core_framework_op_def_proto_H_ + +#include "tensorflow/core/framework/op_def.pb.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/protobuf.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +// Message-text conversion for tensorflow.OpDef.ArgDef +string ProtoDebugString( + const ::tensorflow::OpDef_ArgDef& msg); +string ProtoShortDebugString( + const ::tensorflow::OpDef_ArgDef& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::OpDef_ArgDef* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.OpDef.AttrDef +string ProtoDebugString( + const ::tensorflow::OpDef_AttrDef& msg); +string ProtoShortDebugString( + const ::tensorflow::OpDef_AttrDef& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::OpDef_AttrDef* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.OpDef +string ProtoDebugString( + const ::tensorflow::OpDef& msg); +string ProtoShortDebugString( + const ::tensorflow::OpDef& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::OpDef* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.OpDeprecation +string ProtoDebugString( + const ::tensorflow::OpDeprecation& msg); +string ProtoShortDebugString( + const ::tensorflow::OpDeprecation& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::OpDeprecation* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.OpList +string ProtoDebugString( + const ::tensorflow::OpList& msg); +string ProtoShortDebugString( + const ::tensorflow::OpList& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::OpList* msg) + TF_MUST_USE_RESULT; + +} // namespace tensorflow + +#endif // tensorflow_core_framework_op_def_proto_H_ diff --git a/linux/include/tensorflow/core/framework/op_def_builder.h b/linux/include/tensorflow/core/framework/op_def_builder.h new file mode 100644 index 000000000..8077b2059 --- /dev/null +++ b/linux/include/tensorflow/core/framework/op_def_builder.h @@ -0,0 +1,165 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// Class and associated machinery for specifying an Op's OpDef and shape +// inference function for Op registration. + +#ifndef TENSORFLOW_CORE_FRAMEWORK_OP_DEF_BUILDER_H_ +#define TENSORFLOW_CORE_FRAMEWORK_OP_DEF_BUILDER_H_ + +#include +#include +#include "tensorflow/core/framework/op_def.pb.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/lib/core/stringpiece.h" +#include "tensorflow/core/platform/macros.h" + +namespace tensorflow { + +namespace shape_inference { +class InferenceContext; +} +typedef std::function + OpShapeInferenceFn; + +struct OpRegistrationData { + public: + OpRegistrationData() {} + OpRegistrationData(const OpDef& def) : op_def(def) {} + OpRegistrationData(const OpDef& def, const OpShapeInferenceFn& fn, + bool is_function = false) + : op_def(def), shape_inference_fn(fn), is_function_op(is_function) {} + + OpDef op_def; + OpShapeInferenceFn shape_inference_fn; + bool is_function_op = false; +}; + +// Builder class passed to the REGISTER_OP() macro. +class OpDefBuilder { + public: + // Constructs an OpDef with just the name field set. + explicit OpDefBuilder(string op_name); + + // Adds an attr to this OpDefBuilder (and returns *this). The spec has + // format ":" or ":=" + // where matches regexp [a-zA-Z][a-zA-Z0-9_]* + // (by convention only using capital letters for attrs that can be inferred) + // can be: + // "string", "int", "float", "bool", "type", "shape", or "tensor" + // "numbertype", "realnumbertype", "quantizedtype" + // (meaning "type" with a restriction on valid values) + // "{int32,int64}" or {realnumbertype,quantizedtype,string}" + // (meaning "type" with a restriction containing unions of value types) + // "{\"foo\", \"bar\n baz\"}", or "{'foo', 'bar\n baz'}" + // (meaning "string" with a restriction on valid values) + // "list(string)", ..., "list(tensor)", "list(numbertype)", ... + // (meaning lists of the above types) + // "int >= 2" (meaning "int" with a restriction on valid values) + // "list(string) >= 2", "list(int) >= 2" + // (meaning "list(string)" / "list(int)" with length at least 2) + // , if included, should use the Proto text format + // of . For lists use [a, b, c] format. + // + // Note that any attr specifying the length of an input or output will + // get a default minimum of 1 unless the >= # syntax is used. + // + // TODO(josh11b): Perhaps support restrictions and defaults as optional + // extra arguments to Attr() instead of encoding them in the spec string. + // TODO(josh11b): Would like to have better dtype handling for tensor attrs: + // * Ability to say the type of an input/output matches the type of + // the tensor. + // * Ability to restrict the type of the tensor like the existing + // restrictions for type attrs. + // Perhaps by linking the type of the tensor to a type attr? + OpDefBuilder& Attr(string spec); + + // Adds an input or output to this OpDefBuilder (and returns *this). + // The spec has form ":" or ":Ref()" + // where matches regexp [a-z][a-z0-9_]* and can be: + // * For a single tensor: + // * For a sequence of tensors with the same type: * + // * For a sequence of tensors with different types: + // Where: + // is either one of "float", "int32", "string", ... + // or the name of an attr (see above) with type "type". + // is the name of an attr with type "int". + // is the name of an attr with type "list(type)". + // TODO(josh11b): Indicate Ref() via an optional argument instead of + // in the spec? + // TODO(josh11b): SparseInput() and SparseOutput() matching the Python + // handling? + OpDefBuilder& Input(string spec); + OpDefBuilder& Output(string spec); + + // Turns on the indicated boolean flag in this OpDefBuilder (and + // returns *this). + OpDefBuilder& SetIsCommutative(); + OpDefBuilder& SetIsAggregate(); + OpDefBuilder& SetIsStateful(); + OpDefBuilder& SetAllowsUninitializedInput(); + + // Deprecate the op at a certain GraphDef version. + OpDefBuilder& Deprecated(int version, string explanation); + + // Adds docs to this OpDefBuilder (and returns *this). + // Docs have the format: + // <1-line summary> + // + // : + // : + // + // Where is the name of an attr, input, or output. Please + // wrap docs at 72 columns so that it may be indented in the + // generated output. For tensor inputs or outputs (not attrs), you + // may start the description with an "=" (like name:= ) + // to suppress the automatically-generated type documentation in + // generated output. +#ifndef TF_LEAN_BINARY + OpDefBuilder& Doc(string text); +#else + OpDefBuilder& Doc(string text) { return *this; } +#endif + + // Sets the shape function to be used for shape inference. + // + // Note that currently (October 2016), python code still requires a + // RegisterShape call to invoke this; see call_cpp_shape_fn in + // python/framework/common_shapes.py + OpDefBuilder& SetShapeFn(Status (*fn)(shape_inference::InferenceContext*)); + + // Sets op_reg_data->op_def to the requested OpDef and + // op_reg_data->shape_inference_fn to the requested shape inference function, + // or returns an error. + // Must be called after all of the above methods. + // + // Note that OpDefBuilder only reports parsing errors. You should also + // call ValidateOpDef() to detect other problems. + Status Finalize(OpRegistrationData* op_reg_data) const; + + private: + OpDef* op_def() { return &op_reg_data_.op_def; } + + OpRegistrationData op_reg_data_; + std::vector attrs_; + std::vector inputs_; + std::vector outputs_; + string doc_; + std::vector errors_; +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_FRAMEWORK_OP_DEF_BUILDER_H_ diff --git a/linux/include/tensorflow/core/framework/op_def_util.h b/linux/include/tensorflow/core/framework/op_def_util.h new file mode 100644 index 000000000..85afe2bde --- /dev/null +++ b/linux/include/tensorflow/core/framework/op_def_util.h @@ -0,0 +1,106 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// TODO(josh11b): Probably not needed for OpKernel authors, so doesn't +// need to be as publicly accessible as other files in framework/. + +#ifndef TENSORFLOW_CORE_FRAMEWORK_OP_DEF_UTIL_H_ +#define TENSORFLOW_CORE_FRAMEWORK_OP_DEF_UTIL_H_ + +#include +#include "tensorflow/core/framework/api_def.pb.h" +#include "tensorflow/core/framework/op_def.pb.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/platform/protobuf.h" + +namespace tensorflow { + +// Performs a consistency check across the fields of the op_def. +Status ValidateOpDef(const OpDef& op_def); + +// Check if an op is deprecated at the given GraphDef version. If the op is +// deprecated at a future version, a warning will be logged. +Status CheckOpDeprecation(const OpDef& op_def, int graph_def_version); + +// Validates that attr_value satisfies the type and constraints from attr. +// REQUIRES: attr has already been validated. +Status ValidateAttrValue(const AttrValue& attr_value, + const OpDef::AttrDef& attr); + +// The following search through op_def for an attr with the indicated name. +// Returns nullptr if no such attr is found. +const OpDef::AttrDef* FindAttr(StringPiece name, const OpDef& op_def); +OpDef::AttrDef* FindAttrMutable(StringPiece name, OpDef* op_def); + +// Searches op_def for input argument with the indicated name. +// Returns nullptr if no such attr is found. +const OpDef::ArgDef* FindInputArg(StringPiece name, const OpDef& op_def); + +// Searches api_def for input argument with the indicated name. +// Returns nullptr if no such attr is found. +const ApiDef::Arg* FindInputArg(StringPiece name, const ApiDef& api_def); + +// Produce a human-readable version of an op_def that is more concise +// than a text-format proto. Excludes descriptions. +string SummarizeOpDef(const OpDef& op_def); + +// Returns an error if new_op is not backwards-compatible with (more +// accepting than) old_op. +// REQUIRES: old_op and new_op must pass validation. +Status OpDefCompatible(const OpDef& old_op, const OpDef& new_op); + +// Returns an error if any attr in penultimate_op that is not in old_op +// has a different default value in new_op. In general it is not safe +// to change the default for an attr that has been added to an op. +Status OpDefAddedDefaultsUnchanged(const OpDef& old_op, + const OpDef& penultimate_op, + const OpDef& new_op); + +// Returns an error if the default value for any attr is added/removed/modified +// in new_op compared to old_op. +Status OpDefAttrDefaultsUnchanged(const OpDef& old_op, const OpDef& new_op); + +// Remove all docs from *op_def / *op_list. +void RemoveDescriptionsFromOpDef(OpDef* op_def); +void RemoveDescriptionsFromOpList(OpList* op_list); + +// Remove docs from *op_def but leave explanations of deprecations. +void RemoveNonDeprecationDescriptionsFromOpDef(OpDef* op_def); + +// Returns true if `a1` is equal to `a2`. +// Equality includes all the fields. +bool AttrDefEqual(const OpDef::AttrDef& a1, const OpDef::AttrDef& a2); + +// Returns hash of `a` that is consistent with AttrDefEqual. +uint64 AttrDefHash(const OpDef::AttrDef& a); + +// Returns true if all AttrDefs in `a1` equal corresponding AttrDefs in +// `a2`. Correspondence is established by name. +bool RepeatedAttrDefEqual(const protobuf::RepeatedPtrField& a1, + const protobuf::RepeatedPtrField& a2); + +// Returns hash of `a` that is consistent with RepeatedAttrDefEqual +uint64 RepeatedAttrDefHash(const protobuf::RepeatedPtrField& a); + +// Returns true if `o1` is equal to `o2`. +// Equality includes all the fields. OpDef.attr field is treated as a set. +bool OpDefEqual(const OpDef& o1, const OpDef& o2); + +// Returns hash of `o` that is consistent with AttrDefEqual. +uint64 OpDefHash(const OpDef& o); + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_FRAMEWORK_OP_DEF_UTIL_H_ diff --git a/linux/include/tensorflow/core/framework/op_gen_lib.h b/linux/include/tensorflow/core/framework/op_gen_lib.h new file mode 100644 index 000000000..c269e2df0 --- /dev/null +++ b/linux/include/tensorflow/core/framework/op_gen_lib.h @@ -0,0 +1,100 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_FRAMEWORK_OP_GEN_LIB_H_ +#define TENSORFLOW_CORE_FRAMEWORK_OP_GEN_LIB_H_ + +#include +#include +#include "tensorflow/core/framework/api_def.pb.h" +#include "tensorflow/core/framework/op_def.pb.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/lib/core/stringpiece.h" +#include "tensorflow/core/platform/env.h" + +namespace tensorflow { + +// Forward declare protos so their symbols can be removed from .so exports +class OpDef; + +inline string Spaces(int n) { return string(n, ' '); } + +// Wrap prefix + str to be at most width characters, indenting every line +// after the first by prefix.size() spaces. Intended use case is something +// like prefix = " Foo(" and str is a list of arguments (terminated by a ")"). +// TODO(josh11b): Option to wrap on ", " instead of " " when possible. +string WordWrap(StringPiece prefix, StringPiece str, int width); + +// Looks for an "=" at the beginning of *description. If found, strips it off +// (and any following spaces) from *description and return true. Otherwise +// returns false. +bool ConsumeEquals(StringPiece* description); + +// Convert text-serialized protobufs to/from multiline format. +string PBTxtToMultiline(StringPiece pbtxt, + const std::vector& multi_line_fields); +string PBTxtFromMultiline(StringPiece multiline_pbtxt); + +// Takes a list of files with ApiDefs text protos, and allows you to +// look up the specific ApiDef for any given op. +class ApiDefMap { + public: + // OpList must be a superset of ops of any subsequently loaded + // ApiDef. + explicit ApiDefMap(const OpList& op_list); + ~ApiDefMap(); + + // You can call this method multiple times to load multiple + // sets of files. Api definitions are merged if the same + // op definition is loaded multiple times. Later-loaded + // definitions take precedence. + // ApiDefs loaded from files must contain a subset of ops defined + // in the OpList passed to the constructor. + Status LoadFileList(Env* env, const std::vector& filenames); + + // Load a single file. Api definitions are merged if the same + // op definition is loaded multiple times. Later-loaded + // definitions take precedence. + // ApiDefs loaded from file must contain a subset of ops defined + // in the OpList passed to the constructor. + Status LoadFile(Env* env, const string& filename); + + // Load ApiDefs from string containing ApiDefs text proto. + // api_def_file_contents is expected to be in "multiline format". + // ApiDefs must contain a subset of ops defined in OpsList + // passed to the constructor. + Status LoadApiDef(const string& api_def_file_contents); + + // Updates ApiDef docs. For example, if ApiDef renames an argument + // or attribute, applies these renames to descriptions as well. + // UpdateDocs should only be called once after all ApiDefs are loaded + // since it replaces original op names. + void UpdateDocs(); + + // Look up ApiDef proto based on the given graph op name. + // If graph op name is not in this ApiDefMap, returns nullptr. + // + // Note: Returned ApiDef pointer should stay valid even after calling + // Load* functions defined above. Subsequent calls to Load* might modify + // returned ApiDef contents, but should never remove the ApiDef itself. + const ApiDef* GetApiDef(const string& name) const; + + private: + std::unordered_map map_; +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_FRAMEWORK_OP_GEN_LIB_H_ diff --git a/linux/include/tensorflow/core/framework/op_kernel.h b/linux/include/tensorflow/core/framework/op_kernel.h new file mode 100644 index 000000000..4bbd6c3d7 --- /dev/null +++ b/linux/include/tensorflow/core/framework/op_kernel.h @@ -0,0 +1,1613 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_FRAMEWORK_OP_KERNEL_H_ +#define TENSORFLOW_CORE_FRAMEWORK_OP_KERNEL_H_ + +#include + +#include +#include +#include "tensorflow/core/framework/allocator.h" +#include "tensorflow/core/framework/cancellation.h" +#include "tensorflow/core/framework/control_flow.h" +#include "tensorflow/core/framework/device_base.h" +#include "tensorflow/core/framework/kernel_def.pb.h" +#include "tensorflow/core/framework/kernel_def_builder.h" +#include "tensorflow/core/framework/node_def_util.h" +#include "tensorflow/core/framework/op.h" // TODO(b/62899350): Remove +#include "tensorflow/core/framework/rendezvous.h" +#include "tensorflow/core/framework/selective_registration.h" +#include "tensorflow/core/framework/session_state.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/tensor_shape.pb.h" // TODO(b/62899350): Remove +#include "tensorflow/core/framework/tracking_allocator.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/framework/types.pb.h" +#include "tensorflow/core/framework/unique_tensor_references.h" +#include "tensorflow/core/lib/core/errors.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/lib/gtl/array_slice.h" +#include "tensorflow/core/lib/gtl/manual_constructor.h" +#include "tensorflow/core/platform/env.h" +#include "tensorflow/core/platform/logging.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/mutex.h" +#include "tensorflow/core/platform/thread_annotations.h" +#include "tensorflow/core/platform/types.h" + +namespace Eigen { +struct ThreadPoolDevice; +struct GpuDevice; +struct SyclDevice; +} // end namespace Eigen + +namespace tensorflow { + +namespace checkpoint { +class TensorSliceReaderCacheWrapper; +} // namespace checkpoint + +class AsyncOpKernel; +class CallFrameInterface; +class FunctionLibraryRuntime; +class OpKernelConstruction; // declared below +class OpKernelContext; // declared below, +class OpRegistryInterface; +class ResourceMgr; +class ScopedStepContainer; +class CollectiveExecutor; +class StepStatsCollectorInterface; + +class OpKernel { + public: + // OpKernel won't be instantiated by the scheduler, so you may perform + // expensive initialization in the descendant's constructor. + explicit OpKernel(OpKernelConstruction* context); + + // Specialized constructor that enables the descendant to provide a different + // `NodeDef` value. For example, this constructor can be used to provide a + // stripped-down `NodeDef` that does not contain the full set of attrs (such + // as tensor values) if the descendant stores them in a different form. + explicit OpKernel(OpKernelConstruction* context, + std::unique_ptr node_def); + + virtual ~OpKernel(); + + // An OpKernel's computation can be either synchronous or + // asynchronous. All OpKernel Compute() methods must be thread-safe as they + // may be called concurrently (e.g. by multiple executions of the same graph + // concurrently). + // + // Most OpKernels should compute synchronously. They should + // subclass OpKernel and override the Compute() method and have it + // return after completing the supplied work. + // + // A few special kernels might need to be asynchronous to bound the + // number of threads (e.g., network receive operations). These + // kernels must subclass AsyncOpKernel and override + // AsyncOpKernel::ComputeAsync(). + // + // In both cases, implementations of Compute() and ComputeAsync() + // get inputs and write outputs through the given OpKernelContext + // and returns a status via context->SetStatus(). They must be + // thread-safe. + + // Synchronous compute. + // + // "context" is guaranteed to be alive until Compute() returns. + virtual void Compute(OpKernelContext* context) = 0; + + // Returns nullptr iff this op kernel is synchronous. + virtual AsyncOpKernel* AsAsync() { return nullptr; } + virtual const AsyncOpKernel* AsAsync() const { return nullptr; } + + // Returns true iff this op kernel is considered "expensive". The + // runtime may use this flag to optimize graph execution for example + // to "inline" inexpensive kernels. + virtual bool IsExpensive() { return expensive_; } + + // Accessors. + const NodeDef& def() const { return *def_; } + const string& name() const; // Same as def().name() + const string& type_string() const; // Same as def().op() + const string& requested_device() const; // Same as def().device() + bool is_internal() const { return is_internal_; } + + int num_inputs() const { return input_types_.size(); } + DataType input_type(int i) const { return input_types_[i]; } + const DataTypeVector& input_types() const { return input_types_; } + const MemoryTypeVector& input_memory_types() const { + return input_memory_types_; + } + const string& requested_input(int i) const; // Same as def().input(i) + + int num_outputs() const { return output_types_.size(); } + DataType output_type(int o) const { return output_types_[o]; } + const DataTypeVector& output_types() const { return output_types_; } + const MemoryTypeVector& output_memory_types() const { + return output_memory_types_; + } + + Status InputRange(StringPiece input_name, int* start, int* stop) const; + Status OutputRange(StringPiece output_name, int* start, int* stop) const; + + // We allow legacy scalars within Google up until GraphDef version 6. + // TODO(irving): Remove when we can drop support for GraphDef version 5. + bool allow_legacy_scalars() const { +#if defined(PLATFORM_GOOGLE) || defined(PLATFORM_GOOGLE_ANDROID) + return graph_def_version_ < 6; +#else + return false; +#endif + } + + // Allow either scalars or (if allowing legacy scalars) shape (1,). + bool IsLegacyScalar(const TensorShape& shape) const { + return shape.dims() == 0 || (allow_legacy_scalars() && shape.dims() == 1 && + shape.dim_size(0) == 1); + } + + // Allow rank 1 or (if allowing legacy scalars) rank 0. + bool IsLegacyVector(const TensorShape& shape) const { + return shape.dims() == 1 || (allow_legacy_scalars() && shape.dims() == 0); + } + + // Turn a shape Tensor into a TensorShape + // TODO(irving): Move to TensorShapeUtils once !allow_legacy_scalars + Status MakeShape(const Tensor& shape, TensorShape* out) const; + + private: + const std::unique_ptr def_; + const DataTypeVector input_types_; + const MemoryTypeVector input_memory_types_; + const DataTypeVector output_types_; + const MemoryTypeVector output_memory_types_; + const int graph_def_version_; + const bool is_internal_; // True if this is an internal operation + NameRangeMap input_name_map_; + NameRangeMap output_name_map_; + bool expensive_; + + TF_DISALLOW_COPY_AND_ASSIGN(OpKernel); +}; + +class AsyncOpKernel : public OpKernel { + public: + using OpKernel::OpKernel; // Lift OpKernel constructors. + + // Asynchronous compute. + // + // Implementations of ComputeAsync() must run "done" to signal the + // completion of the computation. "context" is guaranteed to be + // alive until the "done" callback starts. + typedef std::function DoneCallback; + virtual void ComputeAsync(OpKernelContext* context, DoneCallback done) = 0; + + AsyncOpKernel* AsAsync() final { return this; } + const AsyncOpKernel* AsAsync() const final { return this; } + + void Compute(OpKernelContext* context) final; + + bool IsExpensive() override { return true; } +}; + +// Wraps a tensor that is held by an Op across calls to Compute(). For +// memory safety when using asynchronous devices like GPUs, the system +// must be notified when a Tensor is used inside an Op execution. The +// wrapper ensures that all uses of the Tensor are tracked, because in +// order to retrieve the Tensor the caller must use AccessTensor which +// notifies the context. +class PersistentTensor { + public: + PersistentTensor() {} + explicit PersistentTensor(const Tensor& tensor) : tensor_(tensor) {} + + // Caller does not own the returned Tensor*. + Tensor* AccessTensor(OpKernelConstruction* context); + // Caller does not own the returned Tensor*. + Tensor* AccessTensor(OpKernelContext* context); + + // The check for initialization does not need to access the + // underlying tensor buffer. + bool IsInitialized() const { return tensor_.IsInitialized(); } + + int64 NumElements() const { return tensor_.NumElements(); } + + int64 AllocatedBytes() const { return tensor_.AllocatedBytes(); } + + private: + Tensor tensor_; +}; + +class OpKernelConstruction { + public: + OpKernelConstruction(DeviceType device_type, DeviceBase* device, + Allocator* allocator, const NodeDef* node_def, + const OpDef* op_def, FunctionLibraryRuntime* flib, + const DataTypeSlice& input_types, + const MemoryTypeSlice& input_memory_types, + const DataTypeSlice& output_types, + const MemoryTypeSlice& output_memory_types, + int graph_def_version, Status* status); + + Env* env() const { return device_->env(); } + + // Allocation of tensors during kernel construction: + // + // It is legal to temporarily allocate scratch tensor storage during + // Op kernel construction. Scratch tensors should be allocated using + // allocate_temp below. Some kernels need to keep tensors in between + // invocations. If such a Tensor is allocated during kernel + // construction this must be done using allocate_persistent, and the + // Op may only store the returned PersistentTensor object. When the + // Tensor is needed in a subsequent invocation, it can be retrieved + // from the PersistentTensor using the AccessTensor method. This + // ensures that the system is made aware of any use of the tensor's + // allocated memory, which is needed for correctness on asynchronous + // devices such as GPUs. + + // Allocates a temporary Tensor of the specified type and shape. The + // Tensor must not be used after kernel construction is + // complete. See comment above. + Status allocate_temp(DataType type, const TensorShape& shape, + Tensor* out_temp); + + // Allocates a Tensor of the specified type and shape which the Op + // plans to maintain as persistent state. out_persistent holds the + // PersistentTensor which is the object the caller should store. For + // convenience, if out_tensor is non-null then it will be filled in + // with a Tensor* pointing to the newly-allocated tensor which the + // caller can use instead of calling + // out_persistent->AccessTensor. The caller does not own out_tensor + // and should not keep a copy of it. See comment above. + Status allocate_persistent(DataType type, const TensorShape& shape, + PersistentTensor* out_persistent, + Tensor** out_tensor); + + // User-supplied configuration of this operation. + const NodeDef& def() const { return *def_; } + + // For inspecting the inputs to this operation. + int num_inputs() const { return input_types_.size(); } + DataType input_type(int i) const { return input_types_[i]; } + const DataTypeSlice& input_types() const { return input_types_; } + const MemoryTypeSlice& input_memory_types() const { + return input_memory_types_; + } + + // For inspecting the outputs expected from this operation. + int num_outputs() const { return output_types_.size(); } + DataType output_type(int i) const { return output_types_[i]; } + const DataTypeSlice& output_types() const { return output_types_; } + const MemoryTypeSlice& output_memory_types() const { + return output_memory_types_; + } + + // If expected_inputs == inputs() and expected_outputs == output_types(), + // returns OK, else returns INVALID_ARGUMENT with an error message. + // Recommended for Ops with dynamic signatures. + Status MatchSignature(const DataTypeSlice expected_inputs, + const DataTypeSlice expected_outputs); + + // For recording configuration errors during construction. + void SetStatus(const Status& status); + const Status& status() const { return *status_; } + + // Look up the attr with name attr_name and set *value to its value. If no + // attr with attr_name is found in def(), or the attr does not have + // a matching type, a non-ok status will be returned. + template + Status GetAttr(StringPiece attr_name, T* value) const; + + // Return true if the attr_name is defined in def(). + bool HasAttr(StringPiece attr_name) const; + + // Return the device type. + const DeviceType& device_type() const { return device_type_; } + + // If not nullptr, the kernel can instantiate functions defined in + // the library. E.g., + // CHECK_NOTNULL(function_library())->Instantiate("Foo", ...). + FunctionLibraryRuntime* function_library() const { return flib_; } + + // The GraphDef version whose behavior we should follow. + int graph_def_version() const { return graph_def_version_; } + + // Helper routines for the OP_REQUIRES macros + void CtxFailure(const Status& s); + void CtxFailureWithWarning(const Status& s); + void CtxFailure(const char* file, int line, const Status& s); + void CtxFailureWithWarning(const char* file, int line, const Status& s); + + // Unrecommended functions: these are functions that have some + // current uses but are not recommended for use, and may go away at + // some future major version release. + + // May be used, e.g., to get GPU handles, etc. + // + // Currently only used to call MakeTensorFromProto() for + // implementing ConstantOp for every device. See comments + // on Device::MakeTensorFromProto for longer-term replacement + // ideas. + DeviceBase* device() const { return device_; } + + private: + const DeviceType device_type_; + DeviceBase* const device_; + Allocator* allocator_; + const NodeDef* def_; + const OpDef* op_def_; + FunctionLibraryRuntime* flib_; + DataTypeSlice input_types_; + MemoryTypeSlice input_memory_types_; + DataTypeSlice output_types_; + MemoryTypeSlice output_memory_types_; + const int graph_def_version_; + Status* status_; + + // Allow op_def_ across from OpKernel, but not from subclasses. + // TODO(irving): Remove protos from this header entirely. + friend class OpKernel; + + TF_DISALLOW_COPY_AND_ASSIGN(OpKernelConstruction); +}; + +// TODO(mrry): Consider converting to a random_access_iterator, and upgrading +// tensorflow::gtl::iterator_range to make the below container classes +// unnecessary. +template +class OpArgIterator { + public: + using iterator_category = std::forward_iterator_tag; + using value_type = ElementType; + using pointer = ElementType*; + using reference = ElementType&; + using difference_type = ptrdiff_t; + + OpArgIterator(const ListType* list, int i) : list_(list), i_(i) {} + + bool operator==(const OpArgIterator& rhs) { + DCHECK(list_ == rhs.list_); + return i_ == rhs.i_; + } + + bool operator!=(const OpArgIterator& rhs) { + DCHECK(list_ == rhs.list_); + return i_ != rhs.i_; + } + + OpArgIterator operator++() { // prefix ++it + ++i_; + return *this; + } + + OpArgIterator operator++(int) { // postfix it++ + OpArgIterator old_value = *this; + ++i_; + return old_value; + } + + reference operator*() { return (*list_)[i_]; } + pointer operator->() { return &(*list_)[i_]; } + + private: + const ListType* const list_; + int i_; +}; + +// Utility class for representing a list of immutable input tensors +// that are passed to the op as a single named argument. +class OpInputList { + public: + typedef OpArgIterator Iterator; + OpInputList() : ctx_(nullptr), start_(0), stop_(0) {} + OpInputList(OpKernelContext* ctx, int start, int stop) + : ctx_(ctx), start_(start), stop_(stop) {} + OpInputList& operator=(const OpInputList& other) = default; + const Tensor& operator[](int i) const; + int size() const { return stop_ - start_; } + Iterator begin() const { return Iterator(this, 0); } + Iterator end() const { return Iterator(this, size()); } + + private: + OpKernelContext* ctx_; // not owned + int start_; + int stop_; +}; + +// Utility class for representing a list of mutable ("ref") input tensors +// that are passed to the op as a single named argument. +class OpMutableInputList { + public: + typedef OpArgIterator Iterator; + OpMutableInputList(OpKernelContext* ctx, int start, int stop) + : ctx_(ctx), start_(start), stop_(stop) {} + OpMutableInputList() : ctx_(nullptr), start_(0), stop_(0) {} + OpMutableInputList& operator=(const OpMutableInputList& other) = default; + Tensor at(int i, bool lock_held); + mutex* ref_mutex(int i); + int size() const { return stop_ - start_; } + Iterator begin() const { return Iterator(this, 0); } + Iterator end() const { return Iterator(this, size()); } + + private: + OpKernelContext* ctx_; // not owned + int start_; + int stop_; +}; + +// Utility class for representing a list of output tensors that are +// grouped as a single named output. +class OpOutputList { + public: + typedef OpArgIterator Iterator; + OpOutputList() : ctx_(nullptr), start_(0), stop_(0) {} + OpOutputList(OpKernelContext* ctx, int start, int stop) + : ctx_(ctx), start_(start), stop_(stop) {} + OpOutputList& operator=(const OpOutputList& other) = default; + Tensor* operator[](int i); + bool required(int i) const; + DataType expected_output_dtype(int i) const; + Status allocate(int i, const TensorShape& shape, Tensor** output); + void set(int i, const Tensor& tensor); + void set_ref(int i, mutex* mu, Tensor* tensor_for_ref); + int size() const { return stop_ - start_; } + Iterator begin() const { return Iterator(this, 0); } + Iterator end() const { return Iterator(this, size()); } + + private: + OpKernelContext* ctx_; // not owned + int start_; + int stop_; +}; + +// Holds a tensor or tensor reference. For tensor references, we need +// a mutex to prevent concurrent access to the tensor. +struct TensorValue { + TensorValue() : mutex_if_ref(nullptr), tensor(nullptr) {} + TensorValue(Tensor* t) // NOLINT(runtime/explicit) + : mutex_if_ref(nullptr), tensor(t) {} + TensorValue(mutex* mu, Tensor* t) : mutex_if_ref(mu), tensor(t) {} + Tensor* operator->() const { return tensor; } + bool is_ref() const { return mutex_if_ref != nullptr; } + + mutex* mutex_if_ref; // nullptr if not a ref, != nullptr if a ref + Tensor* tensor; +}; + +class OpKernelContext { + public: + // The first element of a WrappedAllocator is a "base" Allocator and + // the second element is that Allocator wrapped by a + // TrackingAllocator + typedef std::pair WrappedAllocator; + + // TODO(zhifengc): Do some cleanup of Params. + // The Params struct is passed in to initialize an OpKernelContext, + // and must outlive the OpKernelContext. + struct Params { + ~Params() { delete eigen_gpu_device; } + + // The step being executed. + int64 step_id = 0; + + // The op kernel being computed. + OpKernel* op_kernel = nullptr; + + // The device on which the kernel is running. + DeviceBase* device = nullptr; + + // The Eigen GPU device wrapper, which may include a per-op + // wrapped allocator. The concrete type of this object depends on + // the type of this->device, so eigen_gpu_device can't be an + // inline member and must be heap allocated. However, we don't + // want to allocate a new eigen_gpu_device for every Op that is + // executed. Instead this member is allocated on first use using + // ensure_eigen_gpu_device, and then if the Params structure is + // re-used for subsequent Ops, the eigen_gpu_device is + // ReInitialized in the OpKernelContext constructor. Unlike the + // other pointers in Params, this one is owned by Params. + PerOpGpuDevice* eigen_gpu_device = nullptr; + + inline void ensure_eigen_gpu_device() { + DCHECK(device); + if (nullptr == eigen_gpu_device) { + // Surprisingly, MakeGpuDevice will return nullptr if the + // device is not a GPU device. This is ok, since those devices + // will never use eigen_gpu_device. It seems better to have + // ensure_eigen_gpu_device fall through and regenerate the + // nullptr every time an OpKernelContext is instantiated, than + // to do an unnecessary allocation of a dummy eigen GPU + // device for CPU device Ops. + eigen_gpu_device = device->MakeGpuDevice(); + } + } + + bool track_allocations = false; + bool log_memory = false; + bool record_tensor_accesses = false; + + // Array indexed by output number for this node + const AllocatorAttributes* output_attr_array = nullptr; + + // Shared resources accessible by this op kernel invocation. + ResourceMgr* resource_manager = nullptr; + + // Per-step resources accessible by this op kernel invocation should be + // stored in this container.. + ScopedStepContainer* step_container = nullptr; + + // Mechanism used by this op kernel invocation to communicate with + // computations running on other devices. + Rendezvous* rendezvous = nullptr; + + // Mechanism for executing a collective op that needs to coordinate + // with parallel instances running on other devices. + CollectiveExecutor* collective_executor = nullptr; + + // The session state for this op. + SessionState* session_state = nullptr; + + // The tensor store for this op. + TensorStore* tensor_store = nullptr; + + // Mechanism used by this op kernel invocation to register a callback + // for its cancellation. + CancellationManager* cancellation_manager = nullptr; + + // Inputs to this op kernel. + const gtl::InlinedVector* inputs = nullptr; + bool is_input_dead = false; + + const gtl::InlinedVector* input_alloc_attrs = + nullptr; + + // Device contexts. + const gtl::InlinedVector* input_device_contexts = + nullptr; + DeviceContext* op_device_context = nullptr; + + // Control-flow op supports. + FrameAndIter frame_iter; + + // Function call supports. + CallFrameInterface* call_frame = nullptr; + FunctionLibraryRuntime* function_library = nullptr; + std::function)>* runner = nullptr; + StepStatsCollectorInterface* stats_collector = nullptr; + + // TensorSliceReaderCache support. + checkpoint::TensorSliceReaderCacheWrapper* slice_reader_cache = nullptr; + + // Support for forwarding reservations (used by ScopedAllocator). + static const int kNeverForward = -2; + static const int kNoReservation = -1; + // Values in [0,...) represent reservations for the indexed output. + const int* forward_from_array = nullptr; + }; + + // params must outlive the OpKernelContext. + explicit OpKernelContext(Params* params); + OpKernelContext(Params* params, int noutputs); + ~OpKernelContext(); + + Env* env() const { return params_->device->env(); } + + int64 step_id() const { return params_->step_id; } + + const OpKernel& op_kernel() const { return *params_->op_kernel; } + + // Input/output signature. + + int num_inputs() const { return params_->inputs->size(); } + DataType input_dtype(int index) const; + Status input_dtype(StringPiece name, DataType* dtype) const; + MemoryType input_memory_type(int index) const; + + int num_outputs() const { return outputs_.size(); } + DataType expected_output_dtype(int index) const; + MemoryType output_memory_type(int index) const; + + // Input + + // Returns an immutable input tensor. May only be used for non-Ref + // inputs. For Ref inputs use mutable_input below. + // REQUIRES: !IsRefType(input_dtype(index)) + // TODO(mrry): Convert this to return Status. + const Tensor& input(int index); + + // Returns the named immutable input tensor in "tensor", as defined + // in the OpDef. May only be used for non-Ref inputs. For Ref inputs + // use mutable_input below. + // REQUIRES: !IsRefType(input_dtype(index)) + // REQUIRES: the named input must not be a list. + Status input(StringPiece name, const Tensor** tensor); + + // Returns the named list-valued immutable input in "list", as + // defined in the OpDef. If the named output is not list-valued, + // returns a one-element list. May only be used for non-Ref + // inputs. For Ref inputs use mutable_input below. + // REQUIRES: !IsRefType(input_dtype(index)) + Status input_list(StringPiece name, OpInputList* list); + + // For mutable inputs, use the following together to make sure there + // is no concurrent access to mutable_input(), e.g.: + // { + // Tensor& t = context->mutable_input(index); + // mutex_lock lock(*context->input_ref_mutex(index)); + // // modify the values in t + // } + // REQUIRES: IsRefType(input_dtype(index)) + Status input_ref_mutex(StringPiece name, mutex** out_mutex); + + // Returns a mutable input tensor. Must be used to access Ref + // inputs. REQUIRES: IsRefType(input_dtype(index)). The caller may + // modify the values stored in the Tensor buffer, and modifications + // will be visible to other Ops reading the same ref tensor. If + // !lock_held the input mutex will be acquired before returning the + // Tensor. + // TODO(mrry): Convert this to return Status. + Tensor mutable_input(int index, bool lock_held); + + // Returns the named mutable input tensor in "tensor", as defined in + // the OpDef. Must be used to access Ref inputs. The values stored + // in the Tensor buffer may be modified, and modifications will be + // visible to other Ops reading the same ref tensor. If !lock_held + // the input mutex will be acquired before returning the Tensor. + // REQUIRES: the named input must not be a list. + // REQUIRES: the named input must be a ref tensor. + Status mutable_input(StringPiece name, Tensor* tensor, bool lock_held); + + // Returns the named list-valued mutable input in "list", as defined + // in the OpDef. If the named input is not list-valued, returns a + // one-element list. Must be used to access Ref inputs. The values + // stored in the Tensor buffer may be modified, and modifications + // will be visible to other Ops reading the same ref tensor. + // REQUIRES: the named input must be a ref tensor. + Status mutable_input_list(StringPiece name, OpMutableInputList* list); + + // Replace the corresponding Ref Input to use the storage buffer + // used by tensor. If !lock_held the input mutex will be acquired + // before returning the Tensor. + // REQUIRES: IsRefType(input_dtype(index)). + void replace_ref_input(int index, const Tensor& tensor, bool lock_held); + + // Replace the corresponding named Ref Input to use the storage + // buffer used by tensor. If !lock_held the input mutex will be + // acquired before returning the Tensor. + // REQUIRES: IsRefType(input_dtype(index)). + Status replace_ref_input(StringPiece name, const Tensor& tensor, + bool lock_held); + + // Deletes the Tensor object used as the Ref Input at + // input_index. This is not usually necessary and should be used + // with caution. If !lock_held the input mutex will be acquired + // before returning the Tensor. + // REQUIRES: IsRefType(input_dtype(input_index)). + void delete_ref_input(int input_index, bool lock_held); + + // Return true if there is input at the given index. An operator has no + // input at index if its tensor is null. This is primarily used by the + // merge operator. + // TODO(mrry): Convert this to return Status. + bool has_input(int index) const; + + // Returns true if all inputs are the same shape, otherwise sets the + // status to a non-OK value and returns false. + // Usage: if (!context->ValidateInputsAreSameShape(this)) return; + bool ValidateInputsAreSameShape(OpKernel* op); + + // Input to output forwarding. + + // Set the output Ref Tensor at output_index to be an alias of the + // input Ref Tensor at input_index. + // REQUIRES: IsRefType(input_dtype(input_index)). + // REQUIRES: IsRefType(output_dtype(output_index)). + void forward_ref_input_to_ref_output(int input_index, int output_index); + + // Returns true when an alias to input[input_index], reshaped to output_shape, + // which is safe to use for in-place computation was written to *output. + // Returns false if input[input_index] has a refcount greater than one, or if + // its type does not match the expected output type of output[output_index], + // or the number of elements in input[input_index] does not equal the number + // of elements in output_shape. + bool forward_input_to_output_with_shape(int input_index, int output_index, + const TensorShape& output_shape, + Tensor** output) TF_MUST_USE_RESULT; + Status forward_input_to_output_with_shape(StringPiece input_name, + StringPiece output_name, + const TensorShape& output_shape, + Tensor** output) TF_MUST_USE_RESULT; + + // Returns a pointer to a Tensor aliasing the underlying buffer backing + // input[input_index] iff + // * input[input_index] is not a ref, + // * the data type, shape, memory type, and allocator attributes of + // input[input_index] are compatible with those given in dtype, shape, + // memory_type, and attr, + // * refcount on the underlying buffer is one. + // * Either there is no forwarding reservation for either input_index + // or output_index or the specified input is reserved for the specified + // output. More precisely: + // + // These cases mean neither input nor output has a reservation: + // forward_from_array = nullptr + // OR (input_index is not in forward_from_array AND + // (output_index == kNoReservation OR + // forward_from_array[output_index] == kNoReservation)) + // + // This case means that input_index is reserved for output_index: + // forward_from_array[output_index] == input_index + // + // This case means the output is reserved to always be allocated, + // never assigned a forwarded input: + // forward_from_array[output_index] == kNeverForward + // + // Otherwise returns nullptr. + // NOTE: For Cuda kernels that read inputs using the __ldg() intrinsic, + // forwarding is only safe if there are no reads via __ldg() after writes + // to the same address. + std::unique_ptr forward_input( + int input_index, int output_index, DataType output_dtype, + const TensorShape& output_shape, MemoryType output_memory_type, + const AllocatorAttributes& output_attr) TF_MUST_USE_RESULT; + + // Tries to forward one of the inputs given in input_indices to + // output[output_index]. If none of the given inputs can be forwarded, calls + // allocate_output() to allocate a new output buffer. + Status forward_input_or_allocate_output( + gtl::ArraySlice candidate_input_indices, int output_index, + const TensorShape& output_shape, Tensor** output) TF_MUST_USE_RESULT; + Status forward_input_or_allocate_output( + gtl::ArraySlice candidate_input_names, + StringPiece output_name, const TensorShape& output_shape, + Tensor** output) TF_MUST_USE_RESULT; + + // Tries to reuse one of the inputs given in input_indices as a temporary. + // If none of the given inputs can be forwarded, calls + // allocate_temp() to allocate a new temporary buffer. + Status forward_input_or_allocate_temp( + gtl::ArraySlice candidate_input_indices, DataType type, + const TensorShape& shape, const AllocatorAttributes& allocator_attr, + Tensor* out_temp) TF_MUST_USE_RESULT; + + Status forward_input_or_allocate_temp( + gtl::ArraySlice candidate_input_indices, DataType type, + const TensorShape& shape, Tensor* out_temp) TF_MUST_USE_RESULT { + return forward_input_or_allocate_temp(candidate_input_indices, type, shape, + AllocatorAttributes(), out_temp); + } + + // Output + + // Returns the named list-valued output in "list", as defined in the OpDef. + // If the named output is not list-valued, returns a one-element list. + Status output_list(StringPiece name, OpOutputList* list); + + // If output_required(index) returns true, the OpKernel's Compute() method + // should call allocate_output(index, ...), set_output(index, ...), + // set_output_ref(index, ...), or set the status to a non-ok value. + // If it returns false, it may output, but is not required to do so. + // TODO(mrry): Convert this to return Status, and implement a string + // name version. + bool output_required(int index) const { + return true; // TODO(josh11b): implement + } + + // Allocation of tensors during kernel execution inside the Compute + // method: + // + // There are three methods to allocate Tensors when an Op kernel + // executes. + // + // 1) allocate_persistent. This is only needed for Tensors that will + // be stored by the Op between invocations, and it *must* be used + // for those Tensors. The call returns a PersistentTensor, and that + // is the only object the Op is allowed to hold on to between + // invocations. When the Tensor is needed in a subsequent + // invocation, it can be retrieved from the PersistentTensor using + // the AccessTensor method. This ensures that the system is made + // aware of any use of the tensor's allocated memory, which is + // needed for correctness on asynchronous devices such as GPUs. + // + // 2) allocate_output. This should be used to allocate any tensor + // that is going to be used as an output from the Op at the end of + // the current execution. The caller indicates which output the + // Tensor will be assigned to, and the call returns the + // newly-allocated Tensor. The Tensor can subsequently be assigned + // to during kernel execution, and will be used as the designated + // output when the kernel execution completes. + // + // 3) allocate_temp. This should be used to allocate any scratch + // storage that is needed while the kernel is executing, and will + // not be retained by the Op. + // + // In some cases a Tensor needs to be used as an output even though + // it was previously allocated elsewhere. The Tensor may have been + // passed as an input, or stored in a PersistentTensor during a + // previous kernel execution, or allocated earlier in the kernel + // execution at a time when it was not known which output it would + // be assigned to. In this case the kernel can use set_output or + // set_output_ref to indicate that the tensor should be used as the + // designated output. It is legal to use any previously-allocated + // Tensor as an argument to set_output or set_output_ref, including + // Tensors allocated via allocate_temp. There may be a performance + // penalty to using a Tensor that was not allocated using + // allocate_output. This is because allocate_output uses the + // AllocatorAttributes stored in output_attr_array for the + // designated output. In some cases, using the wrong attributes may + // cause an extra copy of the Tensor's buffer. + + // Allocates output for the specified output index with shape. + // OpKernelContext retains ownership of the returned pointer. See + // comment above. + // + // If memory allocation fails, returns an error status. + // + // REQUIRES: !IsRefType(expected_output_dtype(index)) + Status allocate_output(int index, const TensorShape& shape, + Tensor** tensor) TF_MUST_USE_RESULT; + Status allocate_output(StringPiece name, const TensorShape& shape, + Tensor** tensor) TF_MUST_USE_RESULT; + // The following methods use the supplied attributes instead of + // those in output_attr_array. The caller is responsible for + // ensuring that the attributes are "compatible" with the + // output_attr_array, e.g. the tensor is allocated on the correct + // device. See comment above. + Status allocate_output(int index, const TensorShape& shape, Tensor** tensor, + AllocatorAttributes attr) TF_MUST_USE_RESULT; + Status allocate_output(StringPiece name, const TensorShape& shape, + Tensor** tensor, + AllocatorAttributes attr) TF_MUST_USE_RESULT; + + // Allocates a temporary Tensor of the specified type and + // shape. Devices such as GPUs that enqueue Ops for lazy execution + // may retain references to the temporary tensors after the Op's + // Compute method has run. See comment above. + Status allocate_temp(DataType type, const TensorShape& shape, + Tensor* out_temp, AllocatorAttributes allocator_attr, + const AllocationAttributes& allocation_attr); + Status allocate_temp(DataType type, const TensorShape& shape, + Tensor* out_temp, AllocatorAttributes allocator_attr) { + return allocate_temp(type, shape, out_temp, allocator_attr, + AllocationAttributes()); + } + Status allocate_temp(DataType type, const TensorShape& shape, + Tensor* out_temp) { + return allocate_temp(type, shape, out_temp, AllocatorAttributes()); + } + + // Allocates a Tensor of the specified type and shape which the Op + // plans to maintain as persistent state. out_persistent holds the + // PersistentTensor which is the object the caller should store. For + // convenience, if out_tensor is non-null then it will be filled in + // with a Tensor* pointing to the newly-allocated tensor which the + // caller can use instead of calling + // out_persistent->AccessTensor. The caller does not own out_tensor + // and should not keep a copy of it. See comment above. + Status allocate_persistent(DataType type, const TensorShape& shape, + PersistentTensor* out_persistent, + Tensor** out_tensor, AllocatorAttributes attr); + Status allocate_persistent(DataType type, const TensorShape& shape, + PersistentTensor* out_persistent, + Tensor** out_tensor) { + return allocate_persistent(type, shape, out_persistent, out_tensor, + AllocatorAttributes()); + } + + // Copies a tensor (allocated by the caller) to the specified output + // index. REQUIRES: !IsRefType(expected_output_dtype(index)) + // REQUIRES: 'tensor' must have the same MemoryType as + // output_memory_types[index]. See comment above. + Status set_output(StringPiece name, const Tensor& tensor); + + // To output a reference. Caller retains ownership of mu and tensor_for_ref, + // and they must outlive all uses within the step. See comment above. + // REQUIRES: IsRefType(expected_output_dtype(index)) + Status set_output_ref(StringPiece name, mutex* mu, Tensor* tensor_for_ref); + + // Returns nullptr if allocate_output() or set_output() have not been called. + Status mutable_output(StringPiece name, Tensor** tensor); + + // Records device specific state about how the input tensors were + // computed. + // + // If using the templated function, the type must be a subclass + // of DeviceContext. + // + // Get the DeviceContext used for the index input. Returns nullptr + // if no DeviceContext was provided. + template + T* input_device_context(int index); + DeviceContext* input_device_context(int index); + + // Return the DeviceContext that should be used for this Op. + // + // If using the templated function, the type must be a subclass + // of DeviceContext. + // + // Returns nullptr if the device did not provide one. + template + T* op_device_context(); + DeviceContext* op_device_context() { + DeviceContext* ret = params_->op_device_context; + if (ret == nullptr) { + auto* dev_info = device()->tensorflow_gpu_device_info(); + if (dev_info) ret = dev_info->default_context; + } + return ret; + } + + AllocatorAttributes input_alloc_attr(int index) const { + if (params_->input_alloc_attrs == nullptr) { + return AllocatorAttributes(); + } else { + DCHECK_GE(index, 0); + DCHECK_LT(index, params_->input_alloc_attrs->size()); + return (*params_->input_alloc_attrs)[index]; + } + } + + AllocatorAttributes output_alloc_attr(int index) const { + return params_->output_attr_array[index]; + } + + gtl::InlinedVector wrapped_allocators() const { + mutex_lock lock(mu_); + gtl::InlinedVector retrieved = wrapped_allocators_; + return retrieved; + } + + // Communication. + // + // An op kernel communicates with outside environment through + // Rendezvous Send() and Recv(). + Rendezvous* rendezvous() const { return params_->rendezvous; } + + CollectiveExecutor* collective_executor() const { + return params_->collective_executor; + } + + // An op kernel can access the session state it belongs to. + SessionState* session_state() const { return params_->session_state; } + + // An op kernel can access the tensor store of the run it belongs to. + TensorStore* tensor_store() const { return params_->tensor_store; } + + // Function call support. + // + // If this kernel invocation is within a function execution, + // call_frame() returns the call frame for the function call. + CallFrameInterface* call_frame() const { return params_->call_frame; } + + // If not nullptr, the kernel invoke functions defined in the + // library. E.g., CHECK_NOTNULL(function_library())->Run("Foo", ...). + FunctionLibraryRuntime* function_library() const { + return params_->function_library; + } + + std::function)>* runner() const { + return params_->runner; + } + StepStatsCollectorInterface* stats_collector() const { + return params_->stats_collector; + } + + // Shared resources accessible to this kernel. + ResourceMgr* resource_manager() const { return params_->resource_manager; } + + checkpoint::TensorSliceReaderCacheWrapper* slice_reader_cache() const { + return params_->slice_reader_cache; + } + + // Execution. + // + // OpKernels can use these eigen devices to carry out their + // numerical computation. + const Eigen::ThreadPoolDevice& eigen_cpu_device() const { + return *device()->eigen_cpu_device(); + } + const Eigen::GpuDevice& eigen_gpu_device() const { + return params_->eigen_gpu_device->device(); + } +#ifdef TENSORFLOW_USE_SYCL + const Eigen::SyclDevice& eigen_sycl_device() const { + return *device()->eigen_sycl_device(); + } +#endif + template + const EigenDeviceType& eigen_device() const; + + // Error handling. + + // If expected_inputs == inputs() and expected_outputs == output_types(), + // returns OK, else returns INVALID_ARGUMENT with an error message. + // Recommended for Ops with dynamic signatures, where validation can only + // be performed at runtime. + Status MatchSignature(const DataTypeSlice expected_inputs, + const DataTypeSlice expected_outputs); + + // An OpKernel should call SetStatus() if Compute() encounters an + // error. + void SetStatus(const Status& status); + const Status& status() const { return status_; } + + // Cancellation. + // + // EXPERIMENTAL. See the implementation in tensorflow::TensorQueue for an + // example of how to use this API. + CancellationManager* cancellation_manager() const { + return params_->cancellation_manager; + } + + // Other accessors. + + // For control flow. + FrameAndIter frame_iter() const { return params_->frame_iter; } + bool is_input_dead() const { return params_->is_input_dead; } + + // May be used, e.g., to get GPU handles, etc. + // TODO(tucker): Add example usage. + DeviceBase* device() const { return params_->device; } + + // Retrieve list of referenced tensors in out_vector. Once this is + // called, it is not legal to reference any more tensors. Should + // not be called from Op kernels. + void retrieve_accessed_tensors(TensorReferenceVector* out_vector); + + // Per-step container for use by white-listed internal ops. + ScopedStepContainer* step_container() const { + return params_->step_container; + } + + // Helper routines for the OP_REQUIRES macros + void CtxFailure(const Status& s); + void CtxFailureWithWarning(const Status& s); + void CtxFailure(const char* file, int line, const Status& s); + void CtxFailureWithWarning(const char* file, int line, const Status& s); + + // Unrecommended functions: these are functions that have some + // current uses but are not recommended for use, and may go away at + // some future major version release. + // + // The following functions all have versions that return Status + // to capture error conditions, and are strongly preferred. + Tensor* mutable_output(int index); + void set_output(int index, const Tensor& tensor); + mutex* input_ref_mutex(int index); + void set_output_ref(int index, mutex* mu, Tensor* tensor_for_ref); + TensorValue release_output(int index); + + bool track_allocations() const { return params_->track_allocations; } + + // Records temp memory allocation. Tensor object is recorded to identify the + // case where temp memory is used as output memory. + void record_temp_memory_allocation(int64 size, const Tensor& t) + LOCKS_EXCLUDED(stats_mu_); + + // Returns recorded size of temporary memory; + int64 temp_memory_allocated() const LOCKS_EXCLUDED(stats_mu_); + + // Records persistent memory allocation, size can be negative indicating + // deallocation. + void record_persistent_memory_allocation(int64 size, int64 alloc_id = -1) + LOCKS_EXCLUDED(stats_mu_); + + // Returns recorded size and ids of persistent memory. + int64 persistent_memory_allocated() const LOCKS_EXCLUDED(stats_mu_); + + std::vector persistent_alloc_ids() const LOCKS_EXCLUDED(stats_mu_); + + // Resets counters for temp and persistent memory and recorded ids. + void clear_recorded_memory() LOCKS_EXCLUDED(stats_mu_); + + bool input_is_ref(int index) const; + + private: + Allocator* get_allocator(AllocatorAttributes attr); + + // Internal method to add a tensor's buffer to the list of buffers + // referenced during the execution of the Op, so that GPUs may + // accurately track the memory that may not be reused until the Op + // execution completes. + void record_tensor_reference(const Tensor& tensor); + void really_record_tensor_reference(const Tensor& tensor); + + // Internal common method used when allocating tensor memory + Status allocate_tensor(DataType type, const TensorShape& shape, + Tensor* out_tensor, + AllocatorAttributes allocator_attr) { + return allocate_tensor(type, shape, out_tensor, allocator_attr, + AllocationAttributes()); + } + + Status allocate_tensor(DataType type, const TensorShape& shape, + Tensor* out_tensor, AllocatorAttributes allocator_attr, + const AllocationAttributes& allocation_attr); + + // This is called by PersistentTensor::AccessTensor whenever the + // wrapped tensor is retrieved, to ensure the runtime knows that the + // Tensor is being accessed within an Op. This is necessary for + // memory safety of devices like GPUs that queue Ops for + // asynchronous execution after the Compute() method completes. + friend class PersistentTensor; + void NotifyUseOfPersistentTensor(const Tensor& tensor); + + Status status_; + friend class CollectiveExecutor; // for access to params_ + Params* params_; // not owned + mutable mutex mu_; // mutable so const accessors can acquire the lock + gtl::InlinedVector wrapped_allocators_ GUARDED_BY(mu_); + gtl::InlinedVector outputs_; + + // Constructed only if record_tensor_accesses>. + ManualConstructor referenced_tensors_ GUARDED_BY(mu_); + + // The following data members are only used when allocation tracking is + // enabled. + mutable mutex stats_mu_; + int64 temp_memory_allocated_ GUARDED_BY(stats_mu_); + int64 persistent_memory_allocated_ GUARDED_BY(stats_mu_); + std::unique_ptr, 2>> + temp_tensor_buffer_and_size_ GUARDED_BY(stats_mu_); + std::unique_ptr> persistent_alloc_ids_ + GUARDED_BY(stats_mu_); + + TF_DISALLOW_COPY_AND_ASSIGN(OpKernelContext); +}; + +// Register your OpKernel by specifying the Op's name, the device the +// kernel runs on, any type attr constraints for this kernel, any +// host-memory args, and the class to instantiate. Examples: +// +// // A kernel that supports all types. +// REGISTER_KERNEL_BUILDER(Name("Save").Device(DEVICE_CPU), SaveOp); +// +// // The following are equivalent ways of specifying that the kernel only +// // works if the "T" type attr is set to DT_FLOAT. +// REGISTER_KERNEL_BUILDER( +// Name("Sub").Device(DEVICE_CPU).TypeConstraint("T"), +// SubOp); +// // (You would then repeat this for every type supported by "Sub".) +// +// // This form allows you to specify a list of types as the constraint. +// REGISTER_KERNEL_BUILDER(Name("Sub") +// .Device(DEVICE_CPU) +// .TypeConstraint("T", {DT_FLOAT}), +// SubOp); +// +// // A kernel that expects one of the input tensors in host memory. +// REGISTER_KERNEL_BUILDER( +// Name("Reshape").Device(DEVICE_GPU).HostMemory("shape"), ReshapeOp); +// +// See kernel_def_builder for details. + +// Instantiate an OpKernel that has been registered. Returns nullptr +// if no operation for that type of device / input signature combination +// (and a NOT_FOUND *status), or there is an error in construction (and +// an INVALID_ARGUMENT *status). Otherwise, the caller takes ownership +// of the returned pointer. +// EXPECTED USAGE: unique_ptr op = CreateOpKernel(...); +// REQUIRES: def has all attrs specified (e.g. using AddDefaultsToNodeDef()). +std::unique_ptr CreateOpKernel(DeviceType device_type, + DeviceBase* device, + Allocator* allocator, + const NodeDef& def, + int graph_def_version, Status* status); +Status CreateOpKernel(DeviceType device_type, DeviceBase* device, + Allocator* allocator, FunctionLibraryRuntime* flib, + const NodeDef& def, int graph_def_version, + OpKernel** kernel); + +// Returns into 'device_types' the subset of prioritized_types that this +// binary has registered for the given NodeDef. +// +// REQUIRES: * 'device_types' is not nullptr. +// * def has all attrs specified (e.g. using AddDefaultsToNodeDef()). +Status SupportedDeviceTypesForNode( + const std::vector& prioritized_types, const NodeDef& def, + DeviceTypeVector* device_types); + +// Returns a message with a description of the kernels registered for op +// `op_name`. +string KernelsRegisteredForOp(StringPiece op_name); + +// Call once after Op registration has completed. +Status ValidateKernelRegistrations(const OpRegistryInterface& op_registry); + +// ----------------------------------------------------------------------------- +// OpKernel registration implementation follows, please ignore. + +// Allow the REGISTER_KERNEL_BUILDER(Name("op_name").Device(...)...) syntax. +namespace register_kernel { + +class Name : public KernelDefBuilder { + public: + // With selective registration, kernels whose implementation class is not used + // by any kernel are disabled with the SHOULD_REGISTER_OP_KERNEL call in + // REGISTER_KERNEL_BUILDER_UNIQ. However, an unused kernel that shares an + // implementation class with a used kernel would get through that mechanism. + // + // This mechanism stops that registration by changing the name of the kernel + // for the unused op to one that is ignored by + // OpKernelRegistrar::InitInternal. Note that this method alone is + // not sufficient - the compiler can't evaluate the entire KernelDefBuilder at + // compilation time, so this method doesn't actually reduce code size. + explicit Name(const char* op) + : KernelDefBuilder(SHOULD_REGISTER_OP(op) ? op : "_no_register") {} +}; + +namespace system { + +class Name : public KernelDefBuilder { + public: + // For system kernels, we ignore selective registration and + // unconditionally register the kernel. + explicit Name(const char* op) : KernelDefBuilder(op) {} +}; + +} // namespace system + +} // namespace register_kernel + +#define REGISTER_KERNEL_BUILDER(kernel_builder, ...) \ + REGISTER_KERNEL_BUILDER_UNIQ_HELPER(__COUNTER__, kernel_builder, __VA_ARGS__) + +#define REGISTER_KERNEL_BUILDER_UNIQ_HELPER(ctr, kernel_builder, ...) \ + REGISTER_KERNEL_BUILDER_UNIQ(ctr, kernel_builder, __VA_ARGS__) + +#define REGISTER_KERNEL_BUILDER_UNIQ(ctr, kernel_builder, ...) \ + constexpr bool should_register_##ctr##__flag = \ + SHOULD_REGISTER_OP_KERNEL(#__VA_ARGS__); \ + static ::tensorflow::kernel_factory::OpKernelRegistrar \ + registrar__body__##ctr##__object( \ + should_register_##ctr##__flag \ + ? ::tensorflow::register_kernel::kernel_builder.Build() \ + : nullptr, \ + #__VA_ARGS__, \ + [](::tensorflow::OpKernelConstruction* context) \ + -> ::tensorflow::OpKernel* { \ + return new __VA_ARGS__(context); \ + }); + +// The `REGISTER_SYSTEM_KERNEL_BUILDER()` macro acts as +// `REGISTER_KERNEL_BUILDER()` except that the kernel is registered +// unconditionally even when selective registration is used. +#define REGISTER_SYSTEM_KERNEL_BUILDER(kernel_builder, ...) \ + REGISTER_SYSTEM_KERNEL_BUILDER_UNIQ_HELPER(__COUNTER__, kernel_builder, \ + __VA_ARGS__) + +#define REGISTER_SYSTEM_KERNEL_BUILDER_UNIQ_HELPER(ctr, kernel_builder, ...) \ + REGISTER_SYSTEM_KERNEL_BUILDER_UNIQ(ctr, kernel_builder, __VA_ARGS__) + +#define REGISTER_SYSTEM_KERNEL_BUILDER_UNIQ(ctr, kernel_builder, ...) \ + static ::tensorflow::kernel_factory::OpKernelRegistrar \ + registrar__body__##ctr##__object( \ + ::tensorflow::register_kernel::system::kernel_builder.Build(), \ + #__VA_ARGS__, \ + [](::tensorflow::OpKernelConstruction* context) \ + -> ::tensorflow::OpKernel* { \ + return new __VA_ARGS__(context); \ + }); + +void* GlobalKernelRegistry(); + +// If node_def has a corresponding kernel registered on device_type, +// returns OK and fill in the kernel def and kernel_class_name. and +// may be null. +Status FindKernelDef(const DeviceType& device_type, const NodeDef& node_def, + const KernelDef** def, string* kernel_class_name); + +// Writes a list of all registered kernels to LOG(INFO), to help users debug +// missing kernel errors. +void LogAllRegisteredKernels(); + +// Gets a list of all registered kernels. +KernelList GetAllRegisteredKernels(); + +// Gets a list of all registered kernels for which predicate returns true +KernelList GetFilteredRegisteredKernels( + const std::function& predicate); + +// Gets a list of all registered kernels for a given op +KernelList GetRegisteredKernelsForOp(StringPiece op_name); + +namespace kernel_factory { + +class OpKernelRegistrar { + public: + typedef OpKernel* (*Factory)(OpKernelConstruction*); + + OpKernelRegistrar(const KernelDef* kernel_def, StringPiece kernel_class_name, + Factory factory) { + // Perform the check in the header to allow compile-time optimization + // to a no-op, allowing the linker to remove the kernel symbols. + if (kernel_def != nullptr) { + InitInternal(kernel_def, kernel_class_name, factory); + } + } + + private: + void InitInternal(const KernelDef* kernel_def, StringPiece kernel_class_name, + Factory factory); +}; + +} // namespace kernel_factory + +// ----------------------------------------------------------------------------- +// Template and inline method implementations, please ignore + +template +Status OpKernelConstruction::GetAttr(StringPiece attr_name, T* value) const { + return GetNodeAttr(def(), attr_name, value); +} + +inline DataType OpKernelContext::input_dtype(int index) const { + DCHECK_GE(index, 0); + DCHECK_LT(index, num_inputs()); + const TensorValue& value((*params_->inputs)[index]); + if (value.is_ref()) { + return MakeRefType(value->dtype()); + } else { + return value->dtype(); + } +} + +inline MemoryType OpKernelContext::input_memory_type(int index) const { + DCHECK_GE(index, 0); + DCHECK_LT(index, num_inputs()); + return op_kernel().input_memory_types()[index]; +} + +inline DataType OpKernelContext::expected_output_dtype(int index) const { + DCHECK_GE(index, 0); + DCHECK_LT(index, num_outputs()); + return params_->op_kernel->output_type(index); +} + +inline MemoryType OpKernelContext::output_memory_type(int index) const { + DCHECK_GE(index, 0); + DCHECK_LT(index, num_outputs()); + return op_kernel().output_memory_types()[index]; +} + +inline bool OpKernelContext::input_is_ref(int index) const { + const TensorValue& value((*params_->inputs)[index]); + return value.is_ref(); +} + +inline void OpKernelContext::record_tensor_reference(const Tensor& tensor) { + DCHECK_EQ(params_->device->RequiresRecordingAccessedTensors(), + params_->record_tensor_accesses); + if (params_->record_tensor_accesses) { + really_record_tensor_reference(tensor); + } +} + +inline void OpKernelContext::retrieve_accessed_tensors( + TensorReferenceVector* out_vector) { + if (params_->record_tensor_accesses) { + mutex_lock l(mu_); + referenced_tensors_->FreezeAndReturnReferences(out_vector); + } +} + +// no input if tensor == nullptr. +inline bool OpKernelContext::has_input(int index) const { + DCHECK_GE(index, 0); + DCHECK_LT(index, num_inputs()); + return (*params_->inputs)[index].tensor != nullptr; +} + +inline mutex* OpKernelContext::input_ref_mutex(int index) { + DCHECK_GE(index, 0); + DCHECK_LT(index, num_inputs()); + DCHECK(input_is_ref(index)); + return (*params_->inputs)[index].mutex_if_ref; +} + +inline void OpKernelContext::NotifyUseOfPersistentTensor(const Tensor& t) { + if (t.IsInitialized()) { + record_tensor_reference(t); + } +} + +inline Tensor* OpKernelContext::mutable_output(int index) { + DCHECK_GE(index, 0); + DCHECK_LT(index, num_outputs()); + // No need to record_tensor_reference since the output must already + // have been set by a call that did so. + return outputs_[index].tensor; +} + +inline TensorValue OpKernelContext::release_output(int index) { + DCHECK_GE(index, 0); + DCHECK_LT(index, num_outputs()); + TensorValue value = outputs_[index]; + outputs_[index] = TensorValue(); + return value; +} + +inline Status OpKernelContext::forward_input_or_allocate_output( + gtl::ArraySlice candidate_input_indices, int output_index, + const TensorShape& output_shape, Tensor** output) { + for (int input_index : candidate_input_indices) { + if (forward_input_to_output_with_shape(input_index, output_index, + output_shape, output)) { + return Status::OK(); + } + } + return allocate_output(output_index, output_shape, output); +} + +inline Status OpKernelContext::forward_input_or_allocate_output( + gtl::ArraySlice candidate_input_names, StringPiece output_name, + const TensorShape& output_shape, Tensor** output) { + for (const StringPiece& input_name : candidate_input_names) { + if (forward_input_to_output_with_shape(input_name, output_name, + output_shape, output) + .ok()) { + return Status::OK(); + } + } + return allocate_output(output_name, output_shape, output); +} + +template +T* OpKernelContext::op_device_context() { + static_assert(std::is_base_of::value, + "T is not a subclass of DeviceContext"); + return static_cast(op_device_context()); +} + +template +T* OpKernelContext::input_device_context(int index) { + DCHECK_GE(index, 0); + DCHECK_LT(index, params_->input_device_contexts->size()); + static_assert(std::is_base_of::value, + "T is not a subclass of DeviceContext"); + return static_cast((*params_->input_device_contexts)[index]); +} + +inline DeviceContext* OpKernelContext::input_device_context(int index) { + DCHECK_GE(index, 0); + DCHECK_LT(index, params_->input_device_contexts->size()); + return (*params_->input_device_contexts)[index]; +} + +inline const Tensor& OpInputList::operator[](int i) const { + DCHECK_GE(i, 0); + DCHECK_LT(i, stop_ - start_); + return ctx_->input(start_ + i); +} + +inline mutex* OpMutableInputList::ref_mutex(int i) { + DCHECK_GE(i, 0); + DCHECK_LT(i, stop_ - start_); + return ctx_->input_ref_mutex(start_ + i); +} + +inline Tensor OpMutableInputList::at(int i, bool lock_held) { + DCHECK_GE(i, 0); + DCHECK_LT(i, stop_ - start_); + return ctx_->mutable_input(start_ + i, lock_held); +} + +inline Tensor* OpOutputList::operator[](int i) { + DCHECK_GE(i, 0); + DCHECK_LT(i, stop_ - start_); + return ctx_->mutable_output(start_ + i); +} + +inline bool OpOutputList::required(int i) const { + DCHECK_GE(i, 0); + DCHECK_LT(i, stop_ - start_); + return ctx_->output_required(start_ + i); +} + +inline DataType OpOutputList::expected_output_dtype(int i) const { + DCHECK_GE(i, 0); + DCHECK_LT(i, stop_ - start_); + return ctx_->expected_output_dtype(start_ + i); +} + +inline Status OpOutputList::allocate(int i, const TensorShape& shape, + Tensor** output) { + DCHECK_GE(i, 0); + DCHECK_LT(i, stop_ - start_); + return ctx_->allocate_output(start_ + i, shape, output); +} + +inline void OpOutputList::set(int i, const Tensor& tensor) { + DCHECK_GE(i, 0); + DCHECK_LT(i, stop_ - start_); + ctx_->set_output(start_ + i, tensor); +} + +inline void OpOutputList::set_ref(int i, mutex* mu, Tensor* tensor_for_ref) { + DCHECK_GE(i, 0); + DCHECK_LT(i, stop_ - start_); + ctx_->set_output_ref(i, mu, tensor_for_ref); +} + +// Convenience macros for asserting and handling exceptional conditions. +// Analogous to the CHECK* macros provided by logging.h. +// +// Example use: +// void Compute(OperationContext* context) { +// OP_REQUIRES(context, context->num_inputs() == 2, +// errors::InvalidArgument("FooOp requires 2 arguments")); +// ... +// Status status = SomeUncertainMethod(); +// OP_REQUIRES_OK(context, status); +// ... +// } + +// Generate a fatal error if OP_REQUIRES or OP_REQUIRES_OK are used in +// AsyncOpKernel implementations. If these macros are used and the condition +// does not hold, the `done` callback will never be called and the system will +// deadlock, so a crash failure is preferable. Since the OP_REQUIRES[_OK] macros +// are legal to use in AsyncOpKernel constructors, we use overload resolution +// to distinguish between OpKernelConstruction* and OpKernelContext* context +// types. +class XlaOpKernelContext; +inline void CheckNotInComputeAsync(XlaOpKernelContext*, const char*) {} +inline void CheckNotInComputeAsync(OpKernelConstruction*, const char*) {} +void CheckNotInComputeAsync(OpKernelContext* ctx, + const char* correct_macro_name); + +#define OP_REQUIRES(CTX, EXP, STATUS) \ + do { \ + if (!TF_PREDICT_TRUE(EXP)) { \ + CheckNotInComputeAsync((CTX), "OP_REQUIRES_ASYNC"); \ + (CTX)->CtxFailure(__FILE__, __LINE__, (STATUS)); \ + return; \ + } \ + } while (0) + +#define OP_REQUIRES_OK(CTX, ...) \ + do { \ + ::tensorflow::Status _s(__VA_ARGS__); \ + if (!TF_PREDICT_TRUE(_s.ok())) { \ + CheckNotInComputeAsync((CTX), "OP_REQUIRES_OK_ASYNC"); \ + (CTX)->CtxFailureWithWarning(__FILE__, __LINE__, _s); \ + return; \ + } \ + } while (0) + +#define OP_REQUIRES_ASYNC(CTX, EXP, STATUS, CALLBACK) \ + do { \ + if (!TF_PREDICT_TRUE(EXP)) { \ + (CTX)->CtxFailure(__FILE__, __LINE__, (STATUS)); \ + (CALLBACK)(); \ + return; \ + } \ + } while (0) + +#define OP_REQUIRES_OK_ASYNC(CTX, STATUS, CALLBACK) \ + do { \ + ::tensorflow::Status _s(STATUS); \ + if (!TF_PREDICT_TRUE(_s.ok())) { \ + (CTX)->CtxFailureWithWarning(__FILE__, __LINE__, _s); \ + (CALLBACK)(); \ + return; \ + } \ + } while (0) + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_FRAMEWORK_OP_KERNEL_H_ diff --git a/linux/include/tensorflow/core/framework/op_segment.h b/linux/include/tensorflow/core/framework/op_segment.h new file mode 100644 index 000000000..37d939ea2 --- /dev/null +++ b/linux/include/tensorflow/core/framework/op_segment.h @@ -0,0 +1,88 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_FRAMEWORK_OP_SEGMENT_H_ +#define TENSORFLOW_FRAMEWORK_OP_SEGMENT_H_ + +#include +#include + +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/mutex.h" +#include "tensorflow/core/platform/thread_annotations.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +// OpSegment keeps track of OpKernels registered for sessions running +// on a device. +// +// The implementation maintains a two-level map. The 1st level maps +// session handle to the map of registered OpKernels. The 2nd level +// map maps node names to instantiated OpKernel objects. +// +// Each 2-nd level map is reference-counted and the caller can call +// AddHold to obtain a reference on all kernels of a session and +// ensure these kernels are alive until a corresponding RemoveHold is +// called on the same session. +class OpSegment { + public: + OpSegment(); + ~OpSegment(); + + // A hold can be placed on a session, preventing all its kernels + // from being deleted. + void AddHold(const string& session_handle); + void RemoveHold(const string& session_handle); + + // If the kernel for "node_name" has been created in the + // "session_handle", returns the existing op kernel in "*kernel". + // Otherwise, creates the kernel by calling create_fn(), cache it, + // and returns it in "*kernel". If create_fn() fails, returns the + // error. + // + // OpSegment keeps the ownership of the returned "*kernel". + typedef std::function CreateKernelFn; + Status FindOrCreate(const string& session_handle, const string& node_name, + OpKernel** kernel, CreateKernelFn create_fn); + + // Returns true if OpSegment should own the kernel. + static bool ShouldOwnKernel(FunctionLibraryRuntime* lib, + const string& node_op); + + private: + // op name -> OpKernel + typedef std::unordered_map KernelMap; + struct Item { + int num_holds = 1; // Num of holds put on the session. + KernelMap name_kernel; // op name -> kernel. + ~Item(); + }; + + // session handle -> item. + // Session handles are produced by strings::FpToString() + typedef std::unordered_map SessionMap; + + mutable mutex mu_; + SessionMap sessions_ GUARDED_BY(mu_); + + TF_DISALLOW_COPY_AND_ASSIGN(OpSegment); +}; + +} // end namespace tensorflow + +#endif // TENSORFLOW_FRAMEWORK_OP_SEGMENT_H_ diff --git a/linux/include/tensorflow/core/framework/partial_tensor_shape.h b/linux/include/tensorflow/core/framework/partial_tensor_shape.h new file mode 100644 index 000000000..fa1ce07dd --- /dev/null +++ b/linux/include/tensorflow/core/framework/partial_tensor_shape.h @@ -0,0 +1,22 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_FRAMEWORK_PARTIAL_TENSOR_SHAPE_H_ +#define TENSORFLOW_CORE_FRAMEWORK_PARTIAL_TENSOR_SHAPE_H_ + +// TODO(irving): Remove this forwarding header +#include "tensorflow/core/framework/tensor_shape.h" + +#endif // TENSORFLOW_CORE_FRAMEWORK_PARTIAL_TENSOR_SHAPE_H_ diff --git a/linux/include/tensorflow/core/framework/queue_interface.h b/linux/include/tensorflow/core/framework/queue_interface.h new file mode 100644 index 000000000..4ca4416c5 --- /dev/null +++ b/linux/include/tensorflow/core/framework/queue_interface.h @@ -0,0 +1,102 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_FRAMEWORK_QUEUE_INTERFACE_H_ +#define TENSORFLOW_CORE_FRAMEWORK_QUEUE_INTERFACE_H_ + +#include +#include + +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/resource_mgr.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +// All implementations must be thread-safe. +class QueueInterface : public ResourceBase { + public: + typedef std::vector Tuple; + typedef AsyncOpKernel::DoneCallback DoneCallback; + typedef std::function CallbackWithTuple; + + virtual Status ValidateTuple(const Tuple& tuple) = 0; + virtual Status ValidateManyTuple(const Tuple& tuple) = 0; + + // Stashes a function object for future execution, that will eventually + // enqueue the tuple of tensors into the queue, and returns immediately. The + // function object is guaranteed to call 'callback'. + virtual void TryEnqueue(const Tuple& tuple, OpKernelContext* ctx, + DoneCallback callback) = 0; + + // Same as above, but the component tensors are sliced along the 0th dimension + // to make multiple queue-element components. + virtual void TryEnqueueMany(const Tuple& tuple, OpKernelContext* ctx, + DoneCallback callback) = 0; + + // Stashes a function object for future execution, that will eventually + // dequeue an element from the queue and call 'callback' with that tuple + // element as argument. + virtual void TryDequeue(OpKernelContext* ctx, CallbackWithTuple callback) = 0; + + // Same as above, but the stashed function object will attempt to dequeue + // num_elements items. If allow_small_batch is true, and the Queue is + // closed but at least 1 element is available, there is no blocking + // and between 1 and num_elements items are immediately returned. + // If the queue does not support the allow_small_batch flag will + // return an Unimplemented error. + virtual void TryDequeueMany(int num_elements, OpKernelContext* ctx, + bool allow_small_batch, + CallbackWithTuple callback) = 0; + + // Signals that no more elements will be enqueued, and optionally + // cancels pending Enqueue(Many) operations. + // + // After calling this function, subsequent calls to Enqueue(Many) + // will fail. If `cancel_pending_enqueues` is true, all pending + // calls to Enqueue(Many) will fail as well. + // + // After calling this function, all current and subsequent calls to + // Dequeue(Many) will fail instead of blocking (though they may + // succeed if they can be satisfied by the elements in the queue at + // the time it was closed). + virtual void Close(OpKernelContext* ctx, bool cancel_pending_enqueues, + DoneCallback callback) = 0; + + // Returns true if a given queue is closed and false if it is open. + virtual bool is_closed() const = 0; + + // Assuming *this represents a shared queue, verify that it matches + // another instantiation indicated by node_def. + virtual Status MatchesNodeDef(const NodeDef& node_def) = 0; + + // Returns the number of elements in the queue. + virtual int32 size() = 0; + + virtual const DataTypeVector& component_dtypes() const = 0; + + string DebugString() override { + return strings::StrCat("A Queue of size: ", size()); + } + + protected: + virtual ~QueueInterface() {} +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_FRAMEWORK_QUEUE_INTERFACE_H_ diff --git a/linux/include/tensorflow/core/framework/reader_base.h b/linux/include/tensorflow/core/framework/reader_base.h new file mode 100644 index 000000000..5b82e9181 --- /dev/null +++ b/linux/include/tensorflow/core/framework/reader_base.h @@ -0,0 +1,138 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_FRAMEWORK_READER_BASE_H_ +#define TENSORFLOW_CORE_FRAMEWORK_READER_BASE_H_ + +#include +#include +#include "tensorflow/core/framework/queue_interface.h" +#include "tensorflow/core/framework/reader_interface.h" +#include "tensorflow/core/lib/core/stringpiece.h" + +namespace tensorflow { + +class ReaderBaseState; + +// Default implementation of ReaderInterface. +class ReaderBase : public ReaderInterface { + public: + // name: For use in error messages, should mention both the name of + // the op and the node. + explicit ReaderBase(const string& name); + + // Note that methods with names ending in "Locked" are called while + // the ReaderBase's mutex is held. + + // Implement this function in descendants ----------------------------------- + + // Produce the next key/value pair from the current work item. + // This is called "Locked" since it is executed under a mutex + // that serializes all Reader calls. + // Usage: + // a) If a record was successfully produced, set *produced = true, + // and fill in *key and *value. + // b) If no more records will be produced for this work item, set + // *at_end = true. + // c) If a record was produced, but no more will be produced, you + // may either do both (a) and (b), or do (a) in this call and do (b) in + // the next call to ReadLocked(). + // d) If there was an error producing (e.g. an error reading the file, + // data corruption), return a non-OK() status. ReadLocked may be + // called again if the user reruns this part of the graph. + virtual Status ReadLocked(string* key, string* value, bool* produced, + bool* at_end) = 0; + + // Descendants may optionally implement these ------------------------------- + + // Produce up to num_records next key/value pairs from the current + // work item, in the same manner of ReadLocked. + virtual Status ReadUpToLocked(int64 num_records, std::vector* keys, + std::vector* values, int64* num_read, + bool* at_end); + + // Called when work starts / finishes. + virtual Status OnWorkStartedLocked() { return Status::OK(); } + virtual Status OnWorkFinishedLocked() { return Status::OK(); } + + // Called to reset the Reader to a newly constructed state. + virtual Status ResetLocked(); + + // Default implementation generates an Unimplemented error. + // See the protected helper methods below. + virtual Status SerializeStateLocked(string* state); + virtual Status RestoreStateLocked(const string& state); + + // Accessors ---------------------------------------------------------------- + + // Always true during a call to ReadLocked(). + bool work_in_progress() const { return work_finished_ < work_started_; } + + // Returns the name of the current work item (valid if + // work_in_progress() returns true). May change between calls to + // ReadLocked(). + const string& current_work() const { return work_; } + + // What was passed to the constructor. + const string& name() const { return name_; } + + // Produce the key name (from current_work and the actual key). + string KeyName(const string& key) const; + + protected: + // For descendants wishing to implement serialize & restore state. + + // Writes ReaderBase state to *state. + void SaveBaseState(ReaderBaseState* state) const; + + // Restores ReaderBase state from state. Assumes state was filled + // using SaveBaseState() above. + Status RestoreBaseState(const ReaderBaseState& state); + + private: + // For descendants that wish to obtain the next work item in a different way. + // For implementing Read(). Dequeues the next work item from + // *queue, and if successful returns "work" (a string). May block. + virtual string GetNextWorkLocked(QueueInterface* queue, + OpKernelContext* context) const; + + // Implementations of ReaderInterface methods. These ensure thread-safety + // and call the methods above to do the work. + void Read(QueueInterface* queue, string* key, string* value, + OpKernelContext* context) override; + + // Produces up to num_records. + // In this implementation all the records come from the same work unit. + int64 ReadUpTo(const int64 num_records, QueueInterface* queue, + std::vector* keys, std::vector* value, + OpKernelContext* context) override; + + Status Reset() override; + int64 NumRecordsProduced() override; + int64 NumWorkUnitsCompleted() override; + Status SerializeState(string* state) override; + Status RestoreState(const string& state) override; + + mutable mutex mu_; + const string name_; + int64 work_started_ = 0; + int64 work_finished_ = 0; + int64 num_records_produced_ = 0; + string work_; +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_FRAMEWORK_READER_BASE_H_ diff --git a/linux/include/tensorflow/core/framework/reader_base.pb.h b/linux/include/tensorflow/core/framework/reader_base.pb.h new file mode 100644 index 000000000..5d5f771df --- /dev/null +++ b/linux/include/tensorflow/core/framework/reader_base.pb.h @@ -0,0 +1,354 @@ +// Generated by the protocol buffer compiler. DO NOT EDIT! +// source: tensorflow/core/framework/reader_base.proto + +#ifndef PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2freader_5fbase_2eproto +#define PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2freader_5fbase_2eproto + +#include + +#include + +#if GOOGLE_PROTOBUF_VERSION < 3006000 +#error This file was generated by a newer version of protoc which is +#error incompatible with your Protocol Buffer headers. Please update +#error your headers. +#endif +#if 3006000 < GOOGLE_PROTOBUF_MIN_PROTOC_VERSION +#error This file was generated by an older version of protoc which is +#error incompatible with your Protocol Buffer headers. Please +#error regenerate this file with a newer version of protoc. +#endif + +#include +#include +#include +#include +#include +#include +#include +#include +#include // IWYU pragma: export +#include // IWYU pragma: export +#include +// @@protoc_insertion_point(includes) +#define PROTOBUF_INTERNAL_EXPORT_protobuf_tensorflow_2fcore_2fframework_2freader_5fbase_2eproto + +namespace protobuf_tensorflow_2fcore_2fframework_2freader_5fbase_2eproto { +// Internal implementation detail -- do not use these members. +struct TableStruct { + static const ::google::protobuf::internal::ParseTableField entries[]; + static const ::google::protobuf::internal::AuxillaryParseTableField aux[]; + static const ::google::protobuf::internal::ParseTable schema[1]; + static const ::google::protobuf::internal::FieldMetadata field_metadata[]; + static const ::google::protobuf::internal::SerializationTable serialization_table[]; + static const ::google::protobuf::uint32 offsets[]; +}; +void AddDescriptors(); +} // namespace protobuf_tensorflow_2fcore_2fframework_2freader_5fbase_2eproto +namespace tensorflow { +class ReaderBaseState; +class ReaderBaseStateDefaultTypeInternal; +extern ReaderBaseStateDefaultTypeInternal _ReaderBaseState_default_instance_; +} // namespace tensorflow +namespace google { +namespace protobuf { +template<> ::tensorflow::ReaderBaseState* Arena::CreateMaybeMessage<::tensorflow::ReaderBaseState>(Arena*); +} // namespace protobuf +} // namespace google +namespace tensorflow { + +// =================================================================== + +class ReaderBaseState : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.ReaderBaseState) */ { + public: + ReaderBaseState(); + virtual ~ReaderBaseState(); + + ReaderBaseState(const ReaderBaseState& from); + + inline ReaderBaseState& operator=(const ReaderBaseState& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + ReaderBaseState(ReaderBaseState&& from) noexcept + : ReaderBaseState() { + *this = ::std::move(from); + } + + inline ReaderBaseState& operator=(ReaderBaseState&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const ReaderBaseState& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const ReaderBaseState* internal_default_instance() { + return reinterpret_cast( + &_ReaderBaseState_default_instance_); + } + static constexpr int kIndexInFileMessages = + 0; + + void UnsafeArenaSwap(ReaderBaseState* other); + void Swap(ReaderBaseState* other); + friend void swap(ReaderBaseState& a, ReaderBaseState& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline ReaderBaseState* New() const final { + return CreateMaybeMessage(NULL); + } + + ReaderBaseState* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const ReaderBaseState& from); + void MergeFrom(const ReaderBaseState& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(ReaderBaseState* other); + protected: + explicit ReaderBaseState(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // bytes current_work = 4; + void clear_current_work(); + static const int kCurrentWorkFieldNumber = 4; + const ::std::string& current_work() const; + void set_current_work(const ::std::string& value); + #if LANG_CXX11 + void set_current_work(::std::string&& value); + #endif + void set_current_work(const char* value); + void set_current_work(const void* value, size_t size); + ::std::string* mutable_current_work(); + ::std::string* release_current_work(); + void set_allocated_current_work(::std::string* current_work); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_current_work(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_current_work( + ::std::string* current_work); + + // int64 work_started = 1; + void clear_work_started(); + static const int kWorkStartedFieldNumber = 1; + ::google::protobuf::int64 work_started() const; + void set_work_started(::google::protobuf::int64 value); + + // int64 work_finished = 2; + void clear_work_finished(); + static const int kWorkFinishedFieldNumber = 2; + ::google::protobuf::int64 work_finished() const; + void set_work_finished(::google::protobuf::int64 value); + + // int64 num_records_produced = 3; + void clear_num_records_produced(); + static const int kNumRecordsProducedFieldNumber = 3; + ::google::protobuf::int64 num_records_produced() const; + void set_num_records_produced(::google::protobuf::int64 value); + + // @@protoc_insertion_point(class_scope:tensorflow.ReaderBaseState) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::internal::ArenaStringPtr current_work_; + ::google::protobuf::int64 work_started_; + ::google::protobuf::int64 work_finished_; + ::google::protobuf::int64 num_records_produced_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2freader_5fbase_2eproto::TableStruct; +}; +// =================================================================== + + +// =================================================================== + +#ifdef __GNUC__ + #pragma GCC diagnostic push + #pragma GCC diagnostic ignored "-Wstrict-aliasing" +#endif // __GNUC__ +// ReaderBaseState + +// int64 work_started = 1; +inline void ReaderBaseState::clear_work_started() { + work_started_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 ReaderBaseState::work_started() const { + // @@protoc_insertion_point(field_get:tensorflow.ReaderBaseState.work_started) + return work_started_; +} +inline void ReaderBaseState::set_work_started(::google::protobuf::int64 value) { + + work_started_ = value; + // @@protoc_insertion_point(field_set:tensorflow.ReaderBaseState.work_started) +} + +// int64 work_finished = 2; +inline void ReaderBaseState::clear_work_finished() { + work_finished_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 ReaderBaseState::work_finished() const { + // @@protoc_insertion_point(field_get:tensorflow.ReaderBaseState.work_finished) + return work_finished_; +} +inline void ReaderBaseState::set_work_finished(::google::protobuf::int64 value) { + + work_finished_ = value; + // @@protoc_insertion_point(field_set:tensorflow.ReaderBaseState.work_finished) +} + +// int64 num_records_produced = 3; +inline void ReaderBaseState::clear_num_records_produced() { + num_records_produced_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 ReaderBaseState::num_records_produced() const { + // @@protoc_insertion_point(field_get:tensorflow.ReaderBaseState.num_records_produced) + return num_records_produced_; +} +inline void ReaderBaseState::set_num_records_produced(::google::protobuf::int64 value) { + + num_records_produced_ = value; + // @@protoc_insertion_point(field_set:tensorflow.ReaderBaseState.num_records_produced) +} + +// bytes current_work = 4; +inline void ReaderBaseState::clear_current_work() { + current_work_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& ReaderBaseState::current_work() const { + // @@protoc_insertion_point(field_get:tensorflow.ReaderBaseState.current_work) + return current_work_.Get(); +} +inline void ReaderBaseState::set_current_work(const ::std::string& value) { + + current_work_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.ReaderBaseState.current_work) +} +#if LANG_CXX11 +inline void ReaderBaseState::set_current_work(::std::string&& value) { + + current_work_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.ReaderBaseState.current_work) +} +#endif +inline void ReaderBaseState::set_current_work(const char* value) { + GOOGLE_DCHECK(value != NULL); + + current_work_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.ReaderBaseState.current_work) +} +inline void ReaderBaseState::set_current_work(const void* value, + size_t size) { + + current_work_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.ReaderBaseState.current_work) +} +inline ::std::string* ReaderBaseState::mutable_current_work() { + + // @@protoc_insertion_point(field_mutable:tensorflow.ReaderBaseState.current_work) + return current_work_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* ReaderBaseState::release_current_work() { + // @@protoc_insertion_point(field_release:tensorflow.ReaderBaseState.current_work) + + return current_work_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void ReaderBaseState::set_allocated_current_work(::std::string* current_work) { + if (current_work != NULL) { + + } else { + + } + current_work_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), current_work, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.ReaderBaseState.current_work) +} +inline ::std::string* ReaderBaseState::unsafe_arena_release_current_work() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.ReaderBaseState.current_work) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return current_work_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void ReaderBaseState::unsafe_arena_set_allocated_current_work( + ::std::string* current_work) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (current_work != NULL) { + + } else { + + } + current_work_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + current_work, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.ReaderBaseState.current_work) +} + +#ifdef __GNUC__ + #pragma GCC diagnostic pop +#endif // __GNUC__ + +// @@protoc_insertion_point(namespace_scope) + +} // namespace tensorflow + +// @@protoc_insertion_point(global_scope) + +#endif // PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2freader_5fbase_2eproto diff --git a/linux/include/tensorflow/core/framework/reader_base.pb_text-impl.h b/linux/include/tensorflow/core/framework/reader_base.pb_text-impl.h new file mode 100644 index 000000000..f360decda --- /dev/null +++ b/linux/include/tensorflow/core/framework/reader_base.pb_text-impl.h @@ -0,0 +1,25 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_framework_reader_base_proto_IMPL_H_ +#define tensorflow_core_framework_reader_base_proto_IMPL_H_ + +#include "tensorflow/core/framework/reader_base.pb.h" +#include "tensorflow/core/framework/reader_base.pb_text.h" +#include "tensorflow/core/lib/strings/proto_text_util.h" +#include "tensorflow/core/lib/strings/scanner.h" + +namespace tensorflow { + +namespace internal { + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::ReaderBaseState& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::ReaderBaseState* msg); + +} // namespace internal + +} // namespace tensorflow + +#endif // tensorflow_core_framework_reader_base_proto_IMPL_H_ diff --git a/linux/include/tensorflow/core/framework/reader_base.pb_text.h b/linux/include/tensorflow/core/framework/reader_base.pb_text.h new file mode 100644 index 000000000..7af57d46a --- /dev/null +++ b/linux/include/tensorflow/core/framework/reader_base.pb_text.h @@ -0,0 +1,24 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_framework_reader_base_proto_H_ +#define tensorflow_core_framework_reader_base_proto_H_ + +#include "tensorflow/core/framework/reader_base.pb.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/protobuf.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +// Message-text conversion for tensorflow.ReaderBaseState +string ProtoDebugString( + const ::tensorflow::ReaderBaseState& msg); +string ProtoShortDebugString( + const ::tensorflow::ReaderBaseState& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::ReaderBaseState* msg) + TF_MUST_USE_RESULT; + +} // namespace tensorflow + +#endif // tensorflow_core_framework_reader_base_proto_H_ diff --git a/linux/include/tensorflow/core/framework/reader_interface.h b/linux/include/tensorflow/core/framework/reader_interface.h new file mode 100644 index 000000000..f894acbe1 --- /dev/null +++ b/linux/include/tensorflow/core/framework/reader_interface.h @@ -0,0 +1,87 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_FRAMEWORK_READER_INTERFACE_H_ +#define TENSORFLOW_CORE_FRAMEWORK_READER_INTERFACE_H_ + +#include +#include +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/resource_mgr.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +class QueueInterface; +class ReaderInterface; + +// Readers are the mechanism for reading records from files in +// TensorFlow graphs. Each supported file format has a corresponding +// ReaderInterface descendant and a corresponding Op & OpKernel +// (implemented using ReaderOpKernel from reader_op_kernel.h). +// +// To use a Reader, you first encode "work" (some string, typically a +// filename) in the Reader's "work queue". It then processes the +// "work" (reading records from the file), to produce key/value +// strings. The methods of this class are called by ReaderFoo ops, +// so see ../ops/io_ops.cc for detailed descriptions. +// +// All descendants of this class must be thread-safe. +class ReaderInterface : public ResourceBase { + public: + // Read a single record into *key / *value. May get more work from + // *queue if the current work is complete. Sets the status on + // *context with an OutOfRange Status if the current work is + // complete and the queue is done (closed and empty). + // This method may block. + virtual void Read(QueueInterface* queue, string* key, string* value, + OpKernelContext* context) = 0; + + // Read up to num_records records into keys / values. May get more work from + // *queue if the current work is complete. Sets the status on + // *context with an OutOfRange Status if the current work is + // complete and the queue is done (closed and empty). + // This method may block. + // The std::vector keys/value pointers are assumed to point to empty + // structures (that have most likely been reserve(num_records)). + // Returns how many records were actually read. + virtual int64 ReadUpTo(const int64 num_records, QueueInterface* queue, + std::vector* keys, std::vector* value, + OpKernelContext* context) = 0; + + // Restore this reader to its newly-constructed state. + virtual Status Reset() = 0; + + // Accessors + virtual int64 NumRecordsProduced() = 0; + virtual int64 NumWorkUnitsCompleted() = 0; + + // -- Serialization/Restoration support -- + // Not all readers will support saving and restoring state. + virtual Status SerializeState(string* state) = 0; + // Note: Must Reset on error. + virtual Status RestoreState(const string& state) = 0; + + string DebugString() override { return "a reader"; } + + protected: + virtual ~ReaderInterface() {} +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_FRAMEWORK_READER_INTERFACE_H_ diff --git a/linux/include/tensorflow/core/framework/reader_op_kernel.h b/linux/include/tensorflow/core/framework/reader_op_kernel.h new file mode 100644 index 000000000..e65a8695b --- /dev/null +++ b/linux/include/tensorflow/core/framework/reader_op_kernel.h @@ -0,0 +1,88 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_FRAMEWORK_READER_OP_KERNEL_H_ +#define TENSORFLOW_CORE_FRAMEWORK_READER_OP_KERNEL_H_ + +#include +#include + +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/reader_interface.h" +#include "tensorflow/core/framework/resource_mgr.h" +#include "tensorflow/core/framework/resource_op_kernel.h" +#include "tensorflow/core/platform/mutex.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +// NOTE: This is now a very thin layer over ResourceOpKernel. +// TODO(sjhwang): Remove dependencies to this class, then delete this. + +// Implementation for ops providing a Reader. +class ReaderOpKernel : public ResourceOpKernel { + public: + using ResourceOpKernel::ResourceOpKernel; + + // Must be called by descendants before the first call to Compute() + // (typically called during construction). factory must return a + // ReaderInterface descendant allocated with new that ReaderOpKernel + // will take ownership of. + void SetReaderFactory(std::function factory) + LOCKS_EXCLUDED(mu_) { + mutex_lock l(mu_); + DCHECK(resource_ == nullptr); + factory_ = factory; + } + + void Compute(OpKernelContext* context) override { + if (!IsCancellable()) { + ResourceOpKernel::Compute(context); + } else { + // Install cancellation + CancellationManager* cm = context->cancellation_manager(); + CancellationToken token = cm->get_cancellation_token(); + bool already_cancelled = + !cm->RegisterCallback(token, [this]() { this->Cancel(); }); + + if (!already_cancelled) { + ResourceOpKernel::Compute(context); + } else { + context->SetStatus(errors::Cancelled("read operation was cancelled")); + } + } + } + + private: + virtual bool IsCancellable() const { return false; } + virtual void Cancel() {} + + Status CreateResource(ReaderInterface** reader) + EXCLUSIVE_LOCKS_REQUIRED(mu_) override { + *reader = factory_(); + if (*reader == nullptr) { + return errors::ResourceExhausted("Failed to allocate reader"); + } + std::function temp = nullptr; + factory_.swap(temp); + return Status::OK(); + } + + std::function factory_ GUARDED_BY(mu_); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_FRAMEWORK_READER_OP_KERNEL_H_ diff --git a/linux/include/tensorflow/core/framework/register_types.h b/linux/include/tensorflow/core/framework/register_types.h new file mode 100644 index 000000000..ddb5b10c1 --- /dev/null +++ b/linux/include/tensorflow/core/framework/register_types.h @@ -0,0 +1,231 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_FRAMEWORK_REGISTER_TYPES_H_ +#define TENSORFLOW_CORE_FRAMEWORK_REGISTER_TYPES_H_ +// This file is used by cuda code and must remain compilable by nvcc. + +#include "tensorflow/core/framework/numeric_types.h" +#include "tensorflow/core/framework/resource_handle.h" +#include "tensorflow/core/framework/variant.h" +#include "tensorflow/core/platform/types.h" + +// Two sets of macros: +// - TF_CALL_float, TF_CALL_double, etc. which call the given macro with +// the type name as the only parameter - except on platforms for which +// the type should not be included. +// - Macros to apply another macro to lists of supported types. These also call +// into TF_CALL_float, TF_CALL_double, etc. so they filter by target platform +// as well. +// If you change the lists of types, please also update the list in types.cc. +// +// See example uses of these macros in core/ops. +// +// +// Each of these TF_CALL_XXX_TYPES(m) macros invokes the macro "m" multiple +// times by passing each invocation a data type supported by TensorFlow. +// +// The different variations pass different subsets of the types. +// TF_CALL_ALL_TYPES(m) applied "m" to all types supported by TensorFlow. +// The set of types depends on the compilation platform. +//. +// This can be used to register a different template instantiation of +// an OpKernel for different signatures, e.g.: +/* + #define REGISTER_PARTITION(type) \ + REGISTER_KERNEL_BUILDER( \ + Name("Partition").Device(DEVICE_CPU).TypeConstraint("T"), \ + PartitionOp); + TF_CALL_ALL_TYPES(REGISTER_PARTITION) + #undef REGISTER_PARTITION +*/ + +#if !defined(IS_MOBILE_PLATFORM) || defined(SUPPORT_SELECTIVE_REGISTRATION) || \ + defined(ANDROID_TEGRA) + +// All types are supported, so all macros are invoked. +// +// Note: macros are defined in same order as types in types.proto, for +// readability. +#define TF_CALL_float(m) m(float) +#define TF_CALL_double(m) m(double) +#define TF_CALL_int32(m) m(::tensorflow::int32) +#define TF_CALL_uint32(m) m(::tensorflow::uint32) +#define TF_CALL_uint8(m) m(::tensorflow::uint8) +#define TF_CALL_int16(m) m(::tensorflow::int16) + +#define TF_CALL_int8(m) m(::tensorflow::int8) +#define TF_CALL_string(m) m(string) +#define TF_CALL_resource(m) m(::tensorflow::ResourceHandle) +#define TF_CALL_variant(m) m(::tensorflow::Variant) +#define TF_CALL_complex64(m) m(::tensorflow::complex64) +#define TF_CALL_int64(m) m(::tensorflow::int64) +#define TF_CALL_uint64(m) m(::tensorflow::uint64) +#define TF_CALL_bool(m) m(bool) + +#define TF_CALL_qint8(m) m(::tensorflow::qint8) +#define TF_CALL_quint8(m) m(::tensorflow::quint8) +#define TF_CALL_qint32(m) m(::tensorflow::qint32) +#define TF_CALL_bfloat16(m) m(::tensorflow::bfloat16) +#define TF_CALL_qint16(m) m(::tensorflow::qint16) + +#define TF_CALL_quint16(m) m(::tensorflow::quint16) +#define TF_CALL_uint16(m) m(::tensorflow::uint16) +#define TF_CALL_complex128(m) m(::tensorflow::complex128) +#define TF_CALL_half(m) m(Eigen::half) + +#elif defined(__ANDROID_TYPES_FULL__) + +// Only string, half, float, int32, int64, bool, and quantized types +// supported. +#define TF_CALL_float(m) m(float) +#define TF_CALL_double(m) +#define TF_CALL_int32(m) m(::tensorflow::int32) +#define TF_CALL_uint32(m) +#define TF_CALL_uint8(m) +#define TF_CALL_int16(m) + +#define TF_CALL_int8(m) +#define TF_CALL_string(m) m(string) +#define TF_CALL_resource(m) +#define TF_CALL_variant(m) +#define TF_CALL_complex64(m) +#define TF_CALL_int64(m) m(::tensorflow::int64) +#define TF_CALL_uint64(m) +#define TF_CALL_bool(m) m(bool) + +#define TF_CALL_qint8(m) m(::tensorflow::qint8) +#define TF_CALL_quint8(m) m(::tensorflow::quint8) +#define TF_CALL_qint32(m) m(::tensorflow::qint32) +#define TF_CALL_bfloat16(m) +#define TF_CALL_qint16(m) m(::tensorflow::qint16) + +#define TF_CALL_quint16(m) m(::tensorflow::quint16) +#define TF_CALL_uint16(m) +#define TF_CALL_complex128(m) +#define TF_CALL_half(m) m(Eigen::half) + +#else // defined(IS_MOBILE_PLATFORM) && !defined(__ANDROID_TYPES_FULL__) + +// Only float, int32, and bool are supported. +#define TF_CALL_float(m) m(float) +#define TF_CALL_double(m) +#define TF_CALL_int32(m) m(::tensorflow::int32) +#define TF_CALL_uint32(m) +#define TF_CALL_uint8(m) +#define TF_CALL_int16(m) + +#define TF_CALL_int8(m) +#define TF_CALL_string(m) +#define TF_CALL_resource(m) +#define TF_CALL_variant(m) +#define TF_CALL_complex64(m) +#define TF_CALL_int64(m) +#define TF_CALL_uint64(m) +#define TF_CALL_bool(m) m(bool) + +#define TF_CALL_qint8(m) +#define TF_CALL_quint8(m) +#define TF_CALL_qint32(m) +#define TF_CALL_bfloat16(m) +#define TF_CALL_qint16(m) + +#define TF_CALL_quint16(m) +#define TF_CALL_uint16(m) +#define TF_CALL_complex128(m) +#define TF_CALL_half(m) + +#endif // defined(IS_MOBILE_PLATFORM) - end of TF_CALL_type defines + +// Defines for sets of types. + +// TODO(b/111604096): Add uint32 and uint64 to TF_CALL_INTEGRAL_TYPES. +// +// The uint32 and uint64 types were introduced in 10/2017 to be used via XLA and +// thus were not included in TF_CALL_INTEGRAL_TYPES. Including them in +// TF_CALL_INTEGRAL_TYPES should only happen after evaluating the effect on the +// TF binary size and performance. +#define TF_CALL_INTEGRAL_TYPES(m) \ + TF_CALL_int64(m) TF_CALL_int32(m) TF_CALL_uint16(m) TF_CALL_int16(m) \ + TF_CALL_uint8(m) TF_CALL_int8(m) + +#define TF_CALL_FLOAT_TYPES(m) \ + TF_CALL_half(m) TF_CALL_bfloat16(m) TF_CALL_float(m) TF_CALL_double(m) + +#define TF_CALL_REAL_NUMBER_TYPES(m) \ + TF_CALL_INTEGRAL_TYPES(m) \ + TF_CALL_FLOAT_TYPES(m) + +#define TF_CALL_REAL_NUMBER_TYPES_NO_BFLOAT16(m) \ + TF_CALL_INTEGRAL_TYPES(m) TF_CALL_half(m) TF_CALL_float(m) TF_CALL_double(m) + +#define TF_CALL_REAL_NUMBER_TYPES_NO_INT32(m) \ + TF_CALL_half(m) TF_CALL_bfloat16(m) TF_CALL_float(m) TF_CALL_double(m) \ + TF_CALL_int64(m) TF_CALL_uint16(m) TF_CALL_int16(m) TF_CALL_uint8(m) \ + TF_CALL_int8(m) + +// Call "m" for all number types, including complex64 and complex128. +#define TF_CALL_NUMBER_TYPES(m) \ + TF_CALL_REAL_NUMBER_TYPES(m) TF_CALL_complex64(m) TF_CALL_complex128(m) + +#define TF_CALL_NUMBER_TYPES_NO_INT32(m) \ + TF_CALL_REAL_NUMBER_TYPES_NO_INT32(m) \ + TF_CALL_complex64(m) TF_CALL_complex128(m) + +#define TF_CALL_POD_TYPES(m) TF_CALL_NUMBER_TYPES(m) TF_CALL_bool(m) + +// Call "m" on all types. +#define TF_CALL_ALL_TYPES(m) \ + TF_CALL_POD_TYPES(m) TF_CALL_string(m) TF_CALL_resource(m) TF_CALL_variant(m) + +// Call "m" on POD and string types. +#define TF_CALL_POD_STRING_TYPES(m) TF_CALL_POD_TYPES(m) TF_CALL_string(m) + +// Call "m" on all number types supported on GPU. +#define TF_CALL_GPU_NUMBER_TYPES(m) \ + TF_CALL_half(m) TF_CALL_float(m) TF_CALL_double(m) + +// Call "m" on all types supported on GPU. +#define TF_CALL_GPU_ALL_TYPES(m) \ + TF_CALL_GPU_NUMBER_TYPES(m) \ + TF_CALL_bool(m) TF_CALL_complex64(m) TF_CALL_complex128(m) + +#define TF_CALL_GPU_NUMBER_TYPES_NO_HALF(m) TF_CALL_float(m) TF_CALL_double(m) + +// Call "m" on all quantized types. +// TODO(cwhipkey): include TF_CALL_qint16(m) TF_CALL_quint16(m) +#define TF_CALL_QUANTIZED_TYPES(m) \ + TF_CALL_qint8(m) TF_CALL_quint8(m) TF_CALL_qint32(m) + +// Types used for save and restore ops. +#define TF_CALL_SAVE_RESTORE_TYPES(m) \ + TF_CALL_INTEGRAL_TYPES(m) \ + TF_CALL_half(m) TF_CALL_float(m) TF_CALL_double(m) TF_CALL_complex64(m) \ + TF_CALL_complex128(m) TF_CALL_bool(m) TF_CALL_string(m) \ + TF_CALL_QUANTIZED_TYPES(m) + +#ifdef TENSORFLOW_SYCL_NO_DOUBLE +#define TF_CALL_SYCL_double(m) +#else // TENSORFLOW_SYCL_NO_DOUBLE +#define TF_CALL_SYCL_double(m) TF_CALL_double(m) +#endif // TENSORFLOW_SYCL_NO_DOUBLE + +#ifdef __ANDROID_TYPES_SLIM__ +#define TF_CALL_SYCL_NUMBER_TYPES(m) TF_CALL_float(m) +#else // __ANDROID_TYPES_SLIM__ +#define TF_CALL_SYCL_NUMBER_TYPES(m) TF_CALL_float(m) TF_CALL_SYCL_double(m) +#endif // __ANDROID_TYPES_SLIM__ + +#endif // TENSORFLOW_CORE_FRAMEWORK_REGISTER_TYPES_H_ diff --git a/linux/include/tensorflow/core/framework/register_types_traits.h b/linux/include/tensorflow/core/framework/register_types_traits.h new file mode 100644 index 000000000..d475a1972 --- /dev/null +++ b/linux/include/tensorflow/core/framework/register_types_traits.h @@ -0,0 +1,105 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_FRAMEWORK_REGISTER_TYPES_TRAITS_H_ +#define TENSORFLOW_CORE_FRAMEWORK_REGISTER_TYPES_TRAITS_H_ +// This file is used by cuda code and must remain compilable by nvcc. + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +typedef Eigen::ThreadPoolDevice CPUDevice; +typedef Eigen::GpuDevice GPUDevice; + +#ifdef TENSORFLOW_USE_SYCL +typedef Eigen::SyclDevice SYCLDevice; +#endif // TENSORFLOW_USE_SYCL + +#include "tensorflow/core/framework/numeric_types.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +// Remap POD types by size to equivalent proxy types. This works +// since all we are doing is copying data around. +struct UnusableProxyType; +template +struct proxy_type_pod { + typedef UnusableProxyType type; +}; +template <> +struct proxy_type_pod { + typedef ::tensorflow::complex128 type; +}; +template <> +struct proxy_type_pod { + typedef ::tensorflow::int64 type; +}; +template <> +struct proxy_type_pod { + typedef ::tensorflow::int32 type; +}; +template <> +struct proxy_type_pod { + typedef ::tensorflow::int16 type; +}; +template <> +struct proxy_type_pod { + typedef ::tensorflow::int8 type; +}; +template <> +struct proxy_type_pod { + typedef double type; +}; +template <> +struct proxy_type_pod { + typedef float type; +}; +template <> +struct proxy_type_pod { + typedef Eigen::half type; +}; + +#ifdef TENSORFLOW_USE_SYCL +template <> +struct proxy_type_pod { + typedef double type; +}; +template <> +struct proxy_type_pod { + typedef float type; +}; +#endif // TENSORFLOW_USE_SYCL + +/// If POD we use proxy_type_pod, otherwise this maps to identiy. +template +struct proxy_type { + typedef typename std::conditional< + std::is_arithmetic::value, + typename proxy_type_pod::type, T>::type type; + static_assert(sizeof(type) == sizeof(T), "proxy_type_pod is not valid"); +}; + +/// The active proxy types +#define TF_CALL_CPU_PROXY_TYPES(m) \ + TF_CALL_int64(m) TF_CALL_int32(m) TF_CALL_uint16(m) TF_CALL_int16(m) \ + TF_CALL_int8(m) TF_CALL_complex128(m) +#define TF_CALL_GPU_PROXY_TYPES(m) \ + TF_CALL_double(m) TF_CALL_float(m) TF_CALL_half(m) TF_CALL_int32(m) +#ifdef TENSORFLOW_USE_SYCL +#define TF_CALL_SYCL_PROXY_TYPES(m) \ + TF_CALL_double(m) TF_CALL_float(m) TF_CALL_int32(m) +#endif // TENSORFLOW_USE_SYCL +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_FRAMEWORK_REGISTER_TYPES_TRAITS_H_ diff --git a/linux/include/tensorflow/core/framework/remote_fused_graph_execute_info.pb.h b/linux/include/tensorflow/core/framework/remote_fused_graph_execute_info.pb.h new file mode 100644 index 000000000..f52c3c02c --- /dev/null +++ b/linux/include/tensorflow/core/framework/remote_fused_graph_execute_info.pb.h @@ -0,0 +1,957 @@ +// Generated by the protocol buffer compiler. DO NOT EDIT! +// source: tensorflow/core/framework/remote_fused_graph_execute_info.proto + +#ifndef PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2fremote_5ffused_5fgraph_5fexecute_5finfo_2eproto +#define PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2fremote_5ffused_5fgraph_5fexecute_5finfo_2eproto + +#include + +#include + +#if GOOGLE_PROTOBUF_VERSION < 3006000 +#error This file was generated by a newer version of protoc which is +#error incompatible with your Protocol Buffer headers. Please update +#error your headers. +#endif +#if 3006000 < GOOGLE_PROTOBUF_MIN_PROTOC_VERSION +#error This file was generated by an older version of protoc which is +#error incompatible with your Protocol Buffer headers. Please +#error regenerate this file with a newer version of protoc. +#endif + +#include +#include +#include +#include +#include +#include +#include +#include +#include // IWYU pragma: export +#include // IWYU pragma: export +#include +#include "tensorflow/core/framework/graph.pb.h" +#include "tensorflow/core/framework/tensor_shape.pb.h" +#include "tensorflow/core/framework/types.pb.h" +// @@protoc_insertion_point(includes) +#define PROTOBUF_INTERNAL_EXPORT_protobuf_tensorflow_2fcore_2fframework_2fremote_5ffused_5fgraph_5fexecute_5finfo_2eproto + +namespace protobuf_tensorflow_2fcore_2fframework_2fremote_5ffused_5fgraph_5fexecute_5finfo_2eproto { +// Internal implementation detail -- do not use these members. +struct TableStruct { + static const ::google::protobuf::internal::ParseTableField entries[]; + static const ::google::protobuf::internal::AuxillaryParseTableField aux[]; + static const ::google::protobuf::internal::ParseTable schema[2]; + static const ::google::protobuf::internal::FieldMetadata field_metadata[]; + static const ::google::protobuf::internal::SerializationTable serialization_table[]; + static const ::google::protobuf::uint32 offsets[]; +}; +void AddDescriptors(); +} // namespace protobuf_tensorflow_2fcore_2fframework_2fremote_5ffused_5fgraph_5fexecute_5finfo_2eproto +namespace tensorflow { +class RemoteFusedGraphExecuteInfo; +class RemoteFusedGraphExecuteInfoDefaultTypeInternal; +extern RemoteFusedGraphExecuteInfoDefaultTypeInternal _RemoteFusedGraphExecuteInfo_default_instance_; +class RemoteFusedGraphExecuteInfo_TensorShapeTypeProto; +class RemoteFusedGraphExecuteInfo_TensorShapeTypeProtoDefaultTypeInternal; +extern RemoteFusedGraphExecuteInfo_TensorShapeTypeProtoDefaultTypeInternal _RemoteFusedGraphExecuteInfo_TensorShapeTypeProto_default_instance_; +} // namespace tensorflow +namespace google { +namespace protobuf { +template<> ::tensorflow::RemoteFusedGraphExecuteInfo* Arena::CreateMaybeMessage<::tensorflow::RemoteFusedGraphExecuteInfo>(Arena*); +template<> ::tensorflow::RemoteFusedGraphExecuteInfo_TensorShapeTypeProto* Arena::CreateMaybeMessage<::tensorflow::RemoteFusedGraphExecuteInfo_TensorShapeTypeProto>(Arena*); +} // namespace protobuf +} // namespace google +namespace tensorflow { + +// =================================================================== + +class RemoteFusedGraphExecuteInfo_TensorShapeTypeProto : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.RemoteFusedGraphExecuteInfo.TensorShapeTypeProto) */ { + public: + RemoteFusedGraphExecuteInfo_TensorShapeTypeProto(); + virtual ~RemoteFusedGraphExecuteInfo_TensorShapeTypeProto(); + + RemoteFusedGraphExecuteInfo_TensorShapeTypeProto(const RemoteFusedGraphExecuteInfo_TensorShapeTypeProto& from); + + inline RemoteFusedGraphExecuteInfo_TensorShapeTypeProto& operator=(const RemoteFusedGraphExecuteInfo_TensorShapeTypeProto& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + RemoteFusedGraphExecuteInfo_TensorShapeTypeProto(RemoteFusedGraphExecuteInfo_TensorShapeTypeProto&& from) noexcept + : RemoteFusedGraphExecuteInfo_TensorShapeTypeProto() { + *this = ::std::move(from); + } + + inline RemoteFusedGraphExecuteInfo_TensorShapeTypeProto& operator=(RemoteFusedGraphExecuteInfo_TensorShapeTypeProto&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const RemoteFusedGraphExecuteInfo_TensorShapeTypeProto& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const RemoteFusedGraphExecuteInfo_TensorShapeTypeProto* internal_default_instance() { + return reinterpret_cast( + &_RemoteFusedGraphExecuteInfo_TensorShapeTypeProto_default_instance_); + } + static constexpr int kIndexInFileMessages = + 0; + + void UnsafeArenaSwap(RemoteFusedGraphExecuteInfo_TensorShapeTypeProto* other); + void Swap(RemoteFusedGraphExecuteInfo_TensorShapeTypeProto* other); + friend void swap(RemoteFusedGraphExecuteInfo_TensorShapeTypeProto& a, RemoteFusedGraphExecuteInfo_TensorShapeTypeProto& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline RemoteFusedGraphExecuteInfo_TensorShapeTypeProto* New() const final { + return CreateMaybeMessage(NULL); + } + + RemoteFusedGraphExecuteInfo_TensorShapeTypeProto* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const RemoteFusedGraphExecuteInfo_TensorShapeTypeProto& from); + void MergeFrom(const RemoteFusedGraphExecuteInfo_TensorShapeTypeProto& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(RemoteFusedGraphExecuteInfo_TensorShapeTypeProto* other); + protected: + explicit RemoteFusedGraphExecuteInfo_TensorShapeTypeProto(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // .tensorflow.TensorShapeProto shape = 2; + bool has_shape() const; + void clear_shape(); + static const int kShapeFieldNumber = 2; + private: + const ::tensorflow::TensorShapeProto& _internal_shape() const; + public: + const ::tensorflow::TensorShapeProto& shape() const; + ::tensorflow::TensorShapeProto* release_shape(); + ::tensorflow::TensorShapeProto* mutable_shape(); + void set_allocated_shape(::tensorflow::TensorShapeProto* shape); + void unsafe_arena_set_allocated_shape( + ::tensorflow::TensorShapeProto* shape); + ::tensorflow::TensorShapeProto* unsafe_arena_release_shape(); + + // .tensorflow.DataType dtype = 1; + void clear_dtype(); + static const int kDtypeFieldNumber = 1; + ::tensorflow::DataType dtype() const; + void set_dtype(::tensorflow::DataType value); + + // @@protoc_insertion_point(class_scope:tensorflow.RemoteFusedGraphExecuteInfo.TensorShapeTypeProto) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::tensorflow::TensorShapeProto* shape_; + int dtype_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fremote_5ffused_5fgraph_5fexecute_5finfo_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class RemoteFusedGraphExecuteInfo : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.RemoteFusedGraphExecuteInfo) */ { + public: + RemoteFusedGraphExecuteInfo(); + virtual ~RemoteFusedGraphExecuteInfo(); + + RemoteFusedGraphExecuteInfo(const RemoteFusedGraphExecuteInfo& from); + + inline RemoteFusedGraphExecuteInfo& operator=(const RemoteFusedGraphExecuteInfo& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + RemoteFusedGraphExecuteInfo(RemoteFusedGraphExecuteInfo&& from) noexcept + : RemoteFusedGraphExecuteInfo() { + *this = ::std::move(from); + } + + inline RemoteFusedGraphExecuteInfo& operator=(RemoteFusedGraphExecuteInfo&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const RemoteFusedGraphExecuteInfo& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const RemoteFusedGraphExecuteInfo* internal_default_instance() { + return reinterpret_cast( + &_RemoteFusedGraphExecuteInfo_default_instance_); + } + static constexpr int kIndexInFileMessages = + 1; + + void UnsafeArenaSwap(RemoteFusedGraphExecuteInfo* other); + void Swap(RemoteFusedGraphExecuteInfo* other); + friend void swap(RemoteFusedGraphExecuteInfo& a, RemoteFusedGraphExecuteInfo& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline RemoteFusedGraphExecuteInfo* New() const final { + return CreateMaybeMessage(NULL); + } + + RemoteFusedGraphExecuteInfo* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const RemoteFusedGraphExecuteInfo& from); + void MergeFrom(const RemoteFusedGraphExecuteInfo& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(RemoteFusedGraphExecuteInfo* other); + protected: + explicit RemoteFusedGraphExecuteInfo(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + typedef RemoteFusedGraphExecuteInfo_TensorShapeTypeProto TensorShapeTypeProto; + + // accessors ------------------------------------------------------- + + // repeated string graph_input_node_name = 2; + int graph_input_node_name_size() const; + void clear_graph_input_node_name(); + static const int kGraphInputNodeNameFieldNumber = 2; + const ::std::string& graph_input_node_name(int index) const; + ::std::string* mutable_graph_input_node_name(int index); + void set_graph_input_node_name(int index, const ::std::string& value); + #if LANG_CXX11 + void set_graph_input_node_name(int index, ::std::string&& value); + #endif + void set_graph_input_node_name(int index, const char* value); + void set_graph_input_node_name(int index, const char* value, size_t size); + ::std::string* add_graph_input_node_name(); + void add_graph_input_node_name(const ::std::string& value); + #if LANG_CXX11 + void add_graph_input_node_name(::std::string&& value); + #endif + void add_graph_input_node_name(const char* value); + void add_graph_input_node_name(const char* value, size_t size); + const ::google::protobuf::RepeatedPtrField< ::std::string>& graph_input_node_name() const; + ::google::protobuf::RepeatedPtrField< ::std::string>* mutable_graph_input_node_name(); + + // repeated string graph_output_node_name = 3; + int graph_output_node_name_size() const; + void clear_graph_output_node_name(); + static const int kGraphOutputNodeNameFieldNumber = 3; + const ::std::string& graph_output_node_name(int index) const; + ::std::string* mutable_graph_output_node_name(int index); + void set_graph_output_node_name(int index, const ::std::string& value); + #if LANG_CXX11 + void set_graph_output_node_name(int index, ::std::string&& value); + #endif + void set_graph_output_node_name(int index, const char* value); + void set_graph_output_node_name(int index, const char* value, size_t size); + ::std::string* add_graph_output_node_name(); + void add_graph_output_node_name(const ::std::string& value); + #if LANG_CXX11 + void add_graph_output_node_name(::std::string&& value); + #endif + void add_graph_output_node_name(const char* value); + void add_graph_output_node_name(const char* value, size_t size); + const ::google::protobuf::RepeatedPtrField< ::std::string>& graph_output_node_name() const; + ::google::protobuf::RepeatedPtrField< ::std::string>* mutable_graph_output_node_name(); + + // repeated .tensorflow.RemoteFusedGraphExecuteInfo.TensorShapeTypeProto default_graph_input_tensor_shape = 6; + int default_graph_input_tensor_shape_size() const; + void clear_default_graph_input_tensor_shape(); + static const int kDefaultGraphInputTensorShapeFieldNumber = 6; + ::tensorflow::RemoteFusedGraphExecuteInfo_TensorShapeTypeProto* mutable_default_graph_input_tensor_shape(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::RemoteFusedGraphExecuteInfo_TensorShapeTypeProto >* + mutable_default_graph_input_tensor_shape(); + const ::tensorflow::RemoteFusedGraphExecuteInfo_TensorShapeTypeProto& default_graph_input_tensor_shape(int index) const; + ::tensorflow::RemoteFusedGraphExecuteInfo_TensorShapeTypeProto* add_default_graph_input_tensor_shape(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::RemoteFusedGraphExecuteInfo_TensorShapeTypeProto >& + default_graph_input_tensor_shape() const; + + // repeated .tensorflow.RemoteFusedGraphExecuteInfo.TensorShapeTypeProto default_graph_output_tensor_shape = 7; + int default_graph_output_tensor_shape_size() const; + void clear_default_graph_output_tensor_shape(); + static const int kDefaultGraphOutputTensorShapeFieldNumber = 7; + ::tensorflow::RemoteFusedGraphExecuteInfo_TensorShapeTypeProto* mutable_default_graph_output_tensor_shape(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::RemoteFusedGraphExecuteInfo_TensorShapeTypeProto >* + mutable_default_graph_output_tensor_shape(); + const ::tensorflow::RemoteFusedGraphExecuteInfo_TensorShapeTypeProto& default_graph_output_tensor_shape(int index) const; + ::tensorflow::RemoteFusedGraphExecuteInfo_TensorShapeTypeProto* add_default_graph_output_tensor_shape(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::RemoteFusedGraphExecuteInfo_TensorShapeTypeProto >& + default_graph_output_tensor_shape() const; + + // string executor_name = 4; + void clear_executor_name(); + static const int kExecutorNameFieldNumber = 4; + const ::std::string& executor_name() const; + void set_executor_name(const ::std::string& value); + #if LANG_CXX11 + void set_executor_name(::std::string&& value); + #endif + void set_executor_name(const char* value); + void set_executor_name(const char* value, size_t size); + ::std::string* mutable_executor_name(); + ::std::string* release_executor_name(); + void set_allocated_executor_name(::std::string* executor_name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_executor_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_executor_name( + ::std::string* executor_name); + + // bytes serialized_executor_parameters = 5; + void clear_serialized_executor_parameters(); + static const int kSerializedExecutorParametersFieldNumber = 5; + const ::std::string& serialized_executor_parameters() const; + void set_serialized_executor_parameters(const ::std::string& value); + #if LANG_CXX11 + void set_serialized_executor_parameters(::std::string&& value); + #endif + void set_serialized_executor_parameters(const char* value); + void set_serialized_executor_parameters(const void* value, size_t size); + ::std::string* mutable_serialized_executor_parameters(); + ::std::string* release_serialized_executor_parameters(); + void set_allocated_serialized_executor_parameters(::std::string* serialized_executor_parameters); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_serialized_executor_parameters(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_serialized_executor_parameters( + ::std::string* serialized_executor_parameters); + + // .tensorflow.GraphDef remote_graph = 1; + bool has_remote_graph() const; + void clear_remote_graph(); + static const int kRemoteGraphFieldNumber = 1; + private: + const ::tensorflow::GraphDef& _internal_remote_graph() const; + public: + const ::tensorflow::GraphDef& remote_graph() const; + ::tensorflow::GraphDef* release_remote_graph(); + ::tensorflow::GraphDef* mutable_remote_graph(); + void set_allocated_remote_graph(::tensorflow::GraphDef* remote_graph); + void unsafe_arena_set_allocated_remote_graph( + ::tensorflow::GraphDef* remote_graph); + ::tensorflow::GraphDef* unsafe_arena_release_remote_graph(); + + // @@protoc_insertion_point(class_scope:tensorflow.RemoteFusedGraphExecuteInfo) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedPtrField< ::std::string> graph_input_node_name_; + ::google::protobuf::RepeatedPtrField< ::std::string> graph_output_node_name_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::RemoteFusedGraphExecuteInfo_TensorShapeTypeProto > default_graph_input_tensor_shape_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::RemoteFusedGraphExecuteInfo_TensorShapeTypeProto > default_graph_output_tensor_shape_; + ::google::protobuf::internal::ArenaStringPtr executor_name_; + ::google::protobuf::internal::ArenaStringPtr serialized_executor_parameters_; + ::tensorflow::GraphDef* remote_graph_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fremote_5ffused_5fgraph_5fexecute_5finfo_2eproto::TableStruct; +}; +// =================================================================== + + +// =================================================================== + +#ifdef __GNUC__ + #pragma GCC diagnostic push + #pragma GCC diagnostic ignored "-Wstrict-aliasing" +#endif // __GNUC__ +// RemoteFusedGraphExecuteInfo_TensorShapeTypeProto + +// .tensorflow.DataType dtype = 1; +inline void RemoteFusedGraphExecuteInfo_TensorShapeTypeProto::clear_dtype() { + dtype_ = 0; +} +inline ::tensorflow::DataType RemoteFusedGraphExecuteInfo_TensorShapeTypeProto::dtype() const { + // @@protoc_insertion_point(field_get:tensorflow.RemoteFusedGraphExecuteInfo.TensorShapeTypeProto.dtype) + return static_cast< ::tensorflow::DataType >(dtype_); +} +inline void RemoteFusedGraphExecuteInfo_TensorShapeTypeProto::set_dtype(::tensorflow::DataType value) { + + dtype_ = value; + // @@protoc_insertion_point(field_set:tensorflow.RemoteFusedGraphExecuteInfo.TensorShapeTypeProto.dtype) +} + +// .tensorflow.TensorShapeProto shape = 2; +inline bool RemoteFusedGraphExecuteInfo_TensorShapeTypeProto::has_shape() const { + return this != internal_default_instance() && shape_ != NULL; +} +inline const ::tensorflow::TensorShapeProto& RemoteFusedGraphExecuteInfo_TensorShapeTypeProto::_internal_shape() const { + return *shape_; +} +inline const ::tensorflow::TensorShapeProto& RemoteFusedGraphExecuteInfo_TensorShapeTypeProto::shape() const { + const ::tensorflow::TensorShapeProto* p = shape_; + // @@protoc_insertion_point(field_get:tensorflow.RemoteFusedGraphExecuteInfo.TensorShapeTypeProto.shape) + return p != NULL ? *p : *reinterpret_cast( + &::tensorflow::_TensorShapeProto_default_instance_); +} +inline ::tensorflow::TensorShapeProto* RemoteFusedGraphExecuteInfo_TensorShapeTypeProto::release_shape() { + // @@protoc_insertion_point(field_release:tensorflow.RemoteFusedGraphExecuteInfo.TensorShapeTypeProto.shape) + + ::tensorflow::TensorShapeProto* temp = shape_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + shape_ = NULL; + return temp; +} +inline ::tensorflow::TensorShapeProto* RemoteFusedGraphExecuteInfo_TensorShapeTypeProto::unsafe_arena_release_shape() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.RemoteFusedGraphExecuteInfo.TensorShapeTypeProto.shape) + + ::tensorflow::TensorShapeProto* temp = shape_; + shape_ = NULL; + return temp; +} +inline ::tensorflow::TensorShapeProto* RemoteFusedGraphExecuteInfo_TensorShapeTypeProto::mutable_shape() { + + if (shape_ == NULL) { + auto* p = CreateMaybeMessage<::tensorflow::TensorShapeProto>(GetArenaNoVirtual()); + shape_ = p; + } + // @@protoc_insertion_point(field_mutable:tensorflow.RemoteFusedGraphExecuteInfo.TensorShapeTypeProto.shape) + return shape_; +} +inline void RemoteFusedGraphExecuteInfo_TensorShapeTypeProto::set_allocated_shape(::tensorflow::TensorShapeProto* shape) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(shape_); + } + if (shape) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(shape)->GetArena(); + if (message_arena != submessage_arena) { + shape = ::google::protobuf::internal::GetOwnedMessage( + message_arena, shape, submessage_arena); + } + + } else { + + } + shape_ = shape; + // @@protoc_insertion_point(field_set_allocated:tensorflow.RemoteFusedGraphExecuteInfo.TensorShapeTypeProto.shape) +} + +// ------------------------------------------------------------------- + +// RemoteFusedGraphExecuteInfo + +// .tensorflow.GraphDef remote_graph = 1; +inline bool RemoteFusedGraphExecuteInfo::has_remote_graph() const { + return this != internal_default_instance() && remote_graph_ != NULL; +} +inline const ::tensorflow::GraphDef& RemoteFusedGraphExecuteInfo::_internal_remote_graph() const { + return *remote_graph_; +} +inline const ::tensorflow::GraphDef& RemoteFusedGraphExecuteInfo::remote_graph() const { + const ::tensorflow::GraphDef* p = remote_graph_; + // @@protoc_insertion_point(field_get:tensorflow.RemoteFusedGraphExecuteInfo.remote_graph) + return p != NULL ? *p : *reinterpret_cast( + &::tensorflow::_GraphDef_default_instance_); +} +inline ::tensorflow::GraphDef* RemoteFusedGraphExecuteInfo::release_remote_graph() { + // @@protoc_insertion_point(field_release:tensorflow.RemoteFusedGraphExecuteInfo.remote_graph) + + ::tensorflow::GraphDef* temp = remote_graph_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + remote_graph_ = NULL; + return temp; +} +inline ::tensorflow::GraphDef* RemoteFusedGraphExecuteInfo::unsafe_arena_release_remote_graph() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.RemoteFusedGraphExecuteInfo.remote_graph) + + ::tensorflow::GraphDef* temp = remote_graph_; + remote_graph_ = NULL; + return temp; +} +inline ::tensorflow::GraphDef* RemoteFusedGraphExecuteInfo::mutable_remote_graph() { + + if (remote_graph_ == NULL) { + auto* p = CreateMaybeMessage<::tensorflow::GraphDef>(GetArenaNoVirtual()); + remote_graph_ = p; + } + // @@protoc_insertion_point(field_mutable:tensorflow.RemoteFusedGraphExecuteInfo.remote_graph) + return remote_graph_; +} +inline void RemoteFusedGraphExecuteInfo::set_allocated_remote_graph(::tensorflow::GraphDef* remote_graph) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(remote_graph_); + } + if (remote_graph) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(remote_graph)->GetArena(); + if (message_arena != submessage_arena) { + remote_graph = ::google::protobuf::internal::GetOwnedMessage( + message_arena, remote_graph, submessage_arena); + } + + } else { + + } + remote_graph_ = remote_graph; + // @@protoc_insertion_point(field_set_allocated:tensorflow.RemoteFusedGraphExecuteInfo.remote_graph) +} + +// repeated string graph_input_node_name = 2; +inline int RemoteFusedGraphExecuteInfo::graph_input_node_name_size() const { + return graph_input_node_name_.size(); +} +inline void RemoteFusedGraphExecuteInfo::clear_graph_input_node_name() { + graph_input_node_name_.Clear(); +} +inline const ::std::string& RemoteFusedGraphExecuteInfo::graph_input_node_name(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.RemoteFusedGraphExecuteInfo.graph_input_node_name) + return graph_input_node_name_.Get(index); +} +inline ::std::string* RemoteFusedGraphExecuteInfo::mutable_graph_input_node_name(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.RemoteFusedGraphExecuteInfo.graph_input_node_name) + return graph_input_node_name_.Mutable(index); +} +inline void RemoteFusedGraphExecuteInfo::set_graph_input_node_name(int index, const ::std::string& value) { + // @@protoc_insertion_point(field_set:tensorflow.RemoteFusedGraphExecuteInfo.graph_input_node_name) + graph_input_node_name_.Mutable(index)->assign(value); +} +#if LANG_CXX11 +inline void RemoteFusedGraphExecuteInfo::set_graph_input_node_name(int index, ::std::string&& value) { + // @@protoc_insertion_point(field_set:tensorflow.RemoteFusedGraphExecuteInfo.graph_input_node_name) + graph_input_node_name_.Mutable(index)->assign(std::move(value)); +} +#endif +inline void RemoteFusedGraphExecuteInfo::set_graph_input_node_name(int index, const char* value) { + GOOGLE_DCHECK(value != NULL); + graph_input_node_name_.Mutable(index)->assign(value); + // @@protoc_insertion_point(field_set_char:tensorflow.RemoteFusedGraphExecuteInfo.graph_input_node_name) +} +inline void RemoteFusedGraphExecuteInfo::set_graph_input_node_name(int index, const char* value, size_t size) { + graph_input_node_name_.Mutable(index)->assign( + reinterpret_cast(value), size); + // @@protoc_insertion_point(field_set_pointer:tensorflow.RemoteFusedGraphExecuteInfo.graph_input_node_name) +} +inline ::std::string* RemoteFusedGraphExecuteInfo::add_graph_input_node_name() { + // @@protoc_insertion_point(field_add_mutable:tensorflow.RemoteFusedGraphExecuteInfo.graph_input_node_name) + return graph_input_node_name_.Add(); +} +inline void RemoteFusedGraphExecuteInfo::add_graph_input_node_name(const ::std::string& value) { + graph_input_node_name_.Add()->assign(value); + // @@protoc_insertion_point(field_add:tensorflow.RemoteFusedGraphExecuteInfo.graph_input_node_name) +} +#if LANG_CXX11 +inline void RemoteFusedGraphExecuteInfo::add_graph_input_node_name(::std::string&& value) { + graph_input_node_name_.Add(std::move(value)); + // @@protoc_insertion_point(field_add:tensorflow.RemoteFusedGraphExecuteInfo.graph_input_node_name) +} +#endif +inline void RemoteFusedGraphExecuteInfo::add_graph_input_node_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + graph_input_node_name_.Add()->assign(value); + // @@protoc_insertion_point(field_add_char:tensorflow.RemoteFusedGraphExecuteInfo.graph_input_node_name) +} +inline void RemoteFusedGraphExecuteInfo::add_graph_input_node_name(const char* value, size_t size) { + graph_input_node_name_.Add()->assign(reinterpret_cast(value), size); + // @@protoc_insertion_point(field_add_pointer:tensorflow.RemoteFusedGraphExecuteInfo.graph_input_node_name) +} +inline const ::google::protobuf::RepeatedPtrField< ::std::string>& +RemoteFusedGraphExecuteInfo::graph_input_node_name() const { + // @@protoc_insertion_point(field_list:tensorflow.RemoteFusedGraphExecuteInfo.graph_input_node_name) + return graph_input_node_name_; +} +inline ::google::protobuf::RepeatedPtrField< ::std::string>* +RemoteFusedGraphExecuteInfo::mutable_graph_input_node_name() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.RemoteFusedGraphExecuteInfo.graph_input_node_name) + return &graph_input_node_name_; +} + +// repeated string graph_output_node_name = 3; +inline int RemoteFusedGraphExecuteInfo::graph_output_node_name_size() const { + return graph_output_node_name_.size(); +} +inline void RemoteFusedGraphExecuteInfo::clear_graph_output_node_name() { + graph_output_node_name_.Clear(); +} +inline const ::std::string& RemoteFusedGraphExecuteInfo::graph_output_node_name(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.RemoteFusedGraphExecuteInfo.graph_output_node_name) + return graph_output_node_name_.Get(index); +} +inline ::std::string* RemoteFusedGraphExecuteInfo::mutable_graph_output_node_name(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.RemoteFusedGraphExecuteInfo.graph_output_node_name) + return graph_output_node_name_.Mutable(index); +} +inline void RemoteFusedGraphExecuteInfo::set_graph_output_node_name(int index, const ::std::string& value) { + // @@protoc_insertion_point(field_set:tensorflow.RemoteFusedGraphExecuteInfo.graph_output_node_name) + graph_output_node_name_.Mutable(index)->assign(value); +} +#if LANG_CXX11 +inline void RemoteFusedGraphExecuteInfo::set_graph_output_node_name(int index, ::std::string&& value) { + // @@protoc_insertion_point(field_set:tensorflow.RemoteFusedGraphExecuteInfo.graph_output_node_name) + graph_output_node_name_.Mutable(index)->assign(std::move(value)); +} +#endif +inline void RemoteFusedGraphExecuteInfo::set_graph_output_node_name(int index, const char* value) { + GOOGLE_DCHECK(value != NULL); + graph_output_node_name_.Mutable(index)->assign(value); + // @@protoc_insertion_point(field_set_char:tensorflow.RemoteFusedGraphExecuteInfo.graph_output_node_name) +} +inline void RemoteFusedGraphExecuteInfo::set_graph_output_node_name(int index, const char* value, size_t size) { + graph_output_node_name_.Mutable(index)->assign( + reinterpret_cast(value), size); + // @@protoc_insertion_point(field_set_pointer:tensorflow.RemoteFusedGraphExecuteInfo.graph_output_node_name) +} +inline ::std::string* RemoteFusedGraphExecuteInfo::add_graph_output_node_name() { + // @@protoc_insertion_point(field_add_mutable:tensorflow.RemoteFusedGraphExecuteInfo.graph_output_node_name) + return graph_output_node_name_.Add(); +} +inline void RemoteFusedGraphExecuteInfo::add_graph_output_node_name(const ::std::string& value) { + graph_output_node_name_.Add()->assign(value); + // @@protoc_insertion_point(field_add:tensorflow.RemoteFusedGraphExecuteInfo.graph_output_node_name) +} +#if LANG_CXX11 +inline void RemoteFusedGraphExecuteInfo::add_graph_output_node_name(::std::string&& value) { + graph_output_node_name_.Add(std::move(value)); + // @@protoc_insertion_point(field_add:tensorflow.RemoteFusedGraphExecuteInfo.graph_output_node_name) +} +#endif +inline void RemoteFusedGraphExecuteInfo::add_graph_output_node_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + graph_output_node_name_.Add()->assign(value); + // @@protoc_insertion_point(field_add_char:tensorflow.RemoteFusedGraphExecuteInfo.graph_output_node_name) +} +inline void RemoteFusedGraphExecuteInfo::add_graph_output_node_name(const char* value, size_t size) { + graph_output_node_name_.Add()->assign(reinterpret_cast(value), size); + // @@protoc_insertion_point(field_add_pointer:tensorflow.RemoteFusedGraphExecuteInfo.graph_output_node_name) +} +inline const ::google::protobuf::RepeatedPtrField< ::std::string>& +RemoteFusedGraphExecuteInfo::graph_output_node_name() const { + // @@protoc_insertion_point(field_list:tensorflow.RemoteFusedGraphExecuteInfo.graph_output_node_name) + return graph_output_node_name_; +} +inline ::google::protobuf::RepeatedPtrField< ::std::string>* +RemoteFusedGraphExecuteInfo::mutable_graph_output_node_name() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.RemoteFusedGraphExecuteInfo.graph_output_node_name) + return &graph_output_node_name_; +} + +// string executor_name = 4; +inline void RemoteFusedGraphExecuteInfo::clear_executor_name() { + executor_name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& RemoteFusedGraphExecuteInfo::executor_name() const { + // @@protoc_insertion_point(field_get:tensorflow.RemoteFusedGraphExecuteInfo.executor_name) + return executor_name_.Get(); +} +inline void RemoteFusedGraphExecuteInfo::set_executor_name(const ::std::string& value) { + + executor_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.RemoteFusedGraphExecuteInfo.executor_name) +} +#if LANG_CXX11 +inline void RemoteFusedGraphExecuteInfo::set_executor_name(::std::string&& value) { + + executor_name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.RemoteFusedGraphExecuteInfo.executor_name) +} +#endif +inline void RemoteFusedGraphExecuteInfo::set_executor_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + executor_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.RemoteFusedGraphExecuteInfo.executor_name) +} +inline void RemoteFusedGraphExecuteInfo::set_executor_name(const char* value, + size_t size) { + + executor_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.RemoteFusedGraphExecuteInfo.executor_name) +} +inline ::std::string* RemoteFusedGraphExecuteInfo::mutable_executor_name() { + + // @@protoc_insertion_point(field_mutable:tensorflow.RemoteFusedGraphExecuteInfo.executor_name) + return executor_name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* RemoteFusedGraphExecuteInfo::release_executor_name() { + // @@protoc_insertion_point(field_release:tensorflow.RemoteFusedGraphExecuteInfo.executor_name) + + return executor_name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void RemoteFusedGraphExecuteInfo::set_allocated_executor_name(::std::string* executor_name) { + if (executor_name != NULL) { + + } else { + + } + executor_name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), executor_name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.RemoteFusedGraphExecuteInfo.executor_name) +} +inline ::std::string* RemoteFusedGraphExecuteInfo::unsafe_arena_release_executor_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.RemoteFusedGraphExecuteInfo.executor_name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return executor_name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void RemoteFusedGraphExecuteInfo::unsafe_arena_set_allocated_executor_name( + ::std::string* executor_name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (executor_name != NULL) { + + } else { + + } + executor_name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + executor_name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.RemoteFusedGraphExecuteInfo.executor_name) +} + +// bytes serialized_executor_parameters = 5; +inline void RemoteFusedGraphExecuteInfo::clear_serialized_executor_parameters() { + serialized_executor_parameters_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& RemoteFusedGraphExecuteInfo::serialized_executor_parameters() const { + // @@protoc_insertion_point(field_get:tensorflow.RemoteFusedGraphExecuteInfo.serialized_executor_parameters) + return serialized_executor_parameters_.Get(); +} +inline void RemoteFusedGraphExecuteInfo::set_serialized_executor_parameters(const ::std::string& value) { + + serialized_executor_parameters_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.RemoteFusedGraphExecuteInfo.serialized_executor_parameters) +} +#if LANG_CXX11 +inline void RemoteFusedGraphExecuteInfo::set_serialized_executor_parameters(::std::string&& value) { + + serialized_executor_parameters_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.RemoteFusedGraphExecuteInfo.serialized_executor_parameters) +} +#endif +inline void RemoteFusedGraphExecuteInfo::set_serialized_executor_parameters(const char* value) { + GOOGLE_DCHECK(value != NULL); + + serialized_executor_parameters_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.RemoteFusedGraphExecuteInfo.serialized_executor_parameters) +} +inline void RemoteFusedGraphExecuteInfo::set_serialized_executor_parameters(const void* value, + size_t size) { + + serialized_executor_parameters_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.RemoteFusedGraphExecuteInfo.serialized_executor_parameters) +} +inline ::std::string* RemoteFusedGraphExecuteInfo::mutable_serialized_executor_parameters() { + + // @@protoc_insertion_point(field_mutable:tensorflow.RemoteFusedGraphExecuteInfo.serialized_executor_parameters) + return serialized_executor_parameters_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* RemoteFusedGraphExecuteInfo::release_serialized_executor_parameters() { + // @@protoc_insertion_point(field_release:tensorflow.RemoteFusedGraphExecuteInfo.serialized_executor_parameters) + + return serialized_executor_parameters_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void RemoteFusedGraphExecuteInfo::set_allocated_serialized_executor_parameters(::std::string* serialized_executor_parameters) { + if (serialized_executor_parameters != NULL) { + + } else { + + } + serialized_executor_parameters_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), serialized_executor_parameters, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.RemoteFusedGraphExecuteInfo.serialized_executor_parameters) +} +inline ::std::string* RemoteFusedGraphExecuteInfo::unsafe_arena_release_serialized_executor_parameters() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.RemoteFusedGraphExecuteInfo.serialized_executor_parameters) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return serialized_executor_parameters_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void RemoteFusedGraphExecuteInfo::unsafe_arena_set_allocated_serialized_executor_parameters( + ::std::string* serialized_executor_parameters) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (serialized_executor_parameters != NULL) { + + } else { + + } + serialized_executor_parameters_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + serialized_executor_parameters, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.RemoteFusedGraphExecuteInfo.serialized_executor_parameters) +} + +// repeated .tensorflow.RemoteFusedGraphExecuteInfo.TensorShapeTypeProto default_graph_input_tensor_shape = 6; +inline int RemoteFusedGraphExecuteInfo::default_graph_input_tensor_shape_size() const { + return default_graph_input_tensor_shape_.size(); +} +inline void RemoteFusedGraphExecuteInfo::clear_default_graph_input_tensor_shape() { + default_graph_input_tensor_shape_.Clear(); +} +inline ::tensorflow::RemoteFusedGraphExecuteInfo_TensorShapeTypeProto* RemoteFusedGraphExecuteInfo::mutable_default_graph_input_tensor_shape(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.RemoteFusedGraphExecuteInfo.default_graph_input_tensor_shape) + return default_graph_input_tensor_shape_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::RemoteFusedGraphExecuteInfo_TensorShapeTypeProto >* +RemoteFusedGraphExecuteInfo::mutable_default_graph_input_tensor_shape() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.RemoteFusedGraphExecuteInfo.default_graph_input_tensor_shape) + return &default_graph_input_tensor_shape_; +} +inline const ::tensorflow::RemoteFusedGraphExecuteInfo_TensorShapeTypeProto& RemoteFusedGraphExecuteInfo::default_graph_input_tensor_shape(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.RemoteFusedGraphExecuteInfo.default_graph_input_tensor_shape) + return default_graph_input_tensor_shape_.Get(index); +} +inline ::tensorflow::RemoteFusedGraphExecuteInfo_TensorShapeTypeProto* RemoteFusedGraphExecuteInfo::add_default_graph_input_tensor_shape() { + // @@protoc_insertion_point(field_add:tensorflow.RemoteFusedGraphExecuteInfo.default_graph_input_tensor_shape) + return default_graph_input_tensor_shape_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::RemoteFusedGraphExecuteInfo_TensorShapeTypeProto >& +RemoteFusedGraphExecuteInfo::default_graph_input_tensor_shape() const { + // @@protoc_insertion_point(field_list:tensorflow.RemoteFusedGraphExecuteInfo.default_graph_input_tensor_shape) + return default_graph_input_tensor_shape_; +} + +// repeated .tensorflow.RemoteFusedGraphExecuteInfo.TensorShapeTypeProto default_graph_output_tensor_shape = 7; +inline int RemoteFusedGraphExecuteInfo::default_graph_output_tensor_shape_size() const { + return default_graph_output_tensor_shape_.size(); +} +inline void RemoteFusedGraphExecuteInfo::clear_default_graph_output_tensor_shape() { + default_graph_output_tensor_shape_.Clear(); +} +inline ::tensorflow::RemoteFusedGraphExecuteInfo_TensorShapeTypeProto* RemoteFusedGraphExecuteInfo::mutable_default_graph_output_tensor_shape(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.RemoteFusedGraphExecuteInfo.default_graph_output_tensor_shape) + return default_graph_output_tensor_shape_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::RemoteFusedGraphExecuteInfo_TensorShapeTypeProto >* +RemoteFusedGraphExecuteInfo::mutable_default_graph_output_tensor_shape() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.RemoteFusedGraphExecuteInfo.default_graph_output_tensor_shape) + return &default_graph_output_tensor_shape_; +} +inline const ::tensorflow::RemoteFusedGraphExecuteInfo_TensorShapeTypeProto& RemoteFusedGraphExecuteInfo::default_graph_output_tensor_shape(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.RemoteFusedGraphExecuteInfo.default_graph_output_tensor_shape) + return default_graph_output_tensor_shape_.Get(index); +} +inline ::tensorflow::RemoteFusedGraphExecuteInfo_TensorShapeTypeProto* RemoteFusedGraphExecuteInfo::add_default_graph_output_tensor_shape() { + // @@protoc_insertion_point(field_add:tensorflow.RemoteFusedGraphExecuteInfo.default_graph_output_tensor_shape) + return default_graph_output_tensor_shape_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::RemoteFusedGraphExecuteInfo_TensorShapeTypeProto >& +RemoteFusedGraphExecuteInfo::default_graph_output_tensor_shape() const { + // @@protoc_insertion_point(field_list:tensorflow.RemoteFusedGraphExecuteInfo.default_graph_output_tensor_shape) + return default_graph_output_tensor_shape_; +} + +#ifdef __GNUC__ + #pragma GCC diagnostic pop +#endif // __GNUC__ +// ------------------------------------------------------------------- + + +// @@protoc_insertion_point(namespace_scope) + +} // namespace tensorflow + +// @@protoc_insertion_point(global_scope) + +#endif // PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2fremote_5ffused_5fgraph_5fexecute_5finfo_2eproto diff --git a/linux/include/tensorflow/core/framework/remote_fused_graph_execute_info.pb_text-impl.h b/linux/include/tensorflow/core/framework/remote_fused_graph_execute_info.pb_text-impl.h new file mode 100644 index 000000000..4afbd0be2 --- /dev/null +++ b/linux/include/tensorflow/core/framework/remote_fused_graph_execute_info.pb_text-impl.h @@ -0,0 +1,52 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_framework_remote_fused_graph_execute_info_proto_IMPL_H_ +#define tensorflow_core_framework_remote_fused_graph_execute_info_proto_IMPL_H_ + +#include "tensorflow/core/framework/attr_value.pb.h" +#include "tensorflow/core/framework/attr_value.pb_text-impl.h" +#include "tensorflow/core/framework/function.pb.h" +#include "tensorflow/core/framework/function.pb_text-impl.h" +#include "tensorflow/core/framework/graph.pb.h" +#include "tensorflow/core/framework/graph.pb_text-impl.h" +#include "tensorflow/core/framework/node_def.pb.h" +#include "tensorflow/core/framework/node_def.pb_text-impl.h" +#include "tensorflow/core/framework/op_def.pb.h" +#include "tensorflow/core/framework/op_def.pb_text-impl.h" +#include "tensorflow/core/framework/remote_fused_graph_execute_info.pb.h" +#include "tensorflow/core/framework/remote_fused_graph_execute_info.pb_text.h" +#include "tensorflow/core/framework/resource_handle.pb.h" +#include "tensorflow/core/framework/resource_handle.pb_text-impl.h" +#include "tensorflow/core/framework/tensor.pb.h" +#include "tensorflow/core/framework/tensor.pb_text-impl.h" +#include "tensorflow/core/framework/tensor_shape.pb.h" +#include "tensorflow/core/framework/tensor_shape.pb_text-impl.h" +#include "tensorflow/core/framework/types.pb.h" +#include "tensorflow/core/framework/types.pb_text-impl.h" +#include "tensorflow/core/framework/versions.pb.h" +#include "tensorflow/core/framework/versions.pb_text-impl.h" +#include "tensorflow/core/lib/strings/proto_text_util.h" +#include "tensorflow/core/lib/strings/scanner.h" + +namespace tensorflow { + +namespace internal { + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::RemoteFusedGraphExecuteInfo_TensorShapeTypeProto& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::RemoteFusedGraphExecuteInfo_TensorShapeTypeProto* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::RemoteFusedGraphExecuteInfo& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::RemoteFusedGraphExecuteInfo* msg); + +} // namespace internal + +} // namespace tensorflow + +#endif // tensorflow_core_framework_remote_fused_graph_execute_info_proto_IMPL_H_ diff --git a/linux/include/tensorflow/core/framework/remote_fused_graph_execute_info.pb_text.h b/linux/include/tensorflow/core/framework/remote_fused_graph_execute_info.pb_text.h new file mode 100644 index 000000000..9e0f3985f --- /dev/null +++ b/linux/include/tensorflow/core/framework/remote_fused_graph_execute_info.pb_text.h @@ -0,0 +1,34 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_framework_remote_fused_graph_execute_info_proto_H_ +#define tensorflow_core_framework_remote_fused_graph_execute_info_proto_H_ + +#include "tensorflow/core/framework/remote_fused_graph_execute_info.pb.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/protobuf.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +// Message-text conversion for tensorflow.RemoteFusedGraphExecuteInfo.TensorShapeTypeProto +string ProtoDebugString( + const ::tensorflow::RemoteFusedGraphExecuteInfo_TensorShapeTypeProto& msg); +string ProtoShortDebugString( + const ::tensorflow::RemoteFusedGraphExecuteInfo_TensorShapeTypeProto& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::RemoteFusedGraphExecuteInfo_TensorShapeTypeProto* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.RemoteFusedGraphExecuteInfo +string ProtoDebugString( + const ::tensorflow::RemoteFusedGraphExecuteInfo& msg); +string ProtoShortDebugString( + const ::tensorflow::RemoteFusedGraphExecuteInfo& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::RemoteFusedGraphExecuteInfo* msg) + TF_MUST_USE_RESULT; + +} // namespace tensorflow + +#endif // tensorflow_core_framework_remote_fused_graph_execute_info_proto_H_ diff --git a/linux/include/tensorflow/core/framework/rendezvous.h b/linux/include/tensorflow/core/framework/rendezvous.h new file mode 100644 index 000000000..01e43e44e --- /dev/null +++ b/linux/include/tensorflow/core/framework/rendezvous.h @@ -0,0 +1,134 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_FRAMEWORK_RENDEZVOUS_H_ +#define TENSORFLOW_FRAMEWORK_RENDEZVOUS_H_ + +#include + +#include "tensorflow/core/framework/control_flow.h" +#include "tensorflow/core/framework/device_base.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/lib/core/refcount.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/util/device_name_utils.h" + +namespace tensorflow { + +// A Rendezvous is an abstraction for passing tensors from producers +// to consumers. A rendezvous is a table of channels. Each channel is +// keyed by a rendezvous key. The key encodes a pair of , where the producer and the consumer are tensorflow +// devices. +// +// The producer calls the Send() method to send one tensor over one +// named channel. The consumer calls the Recv() method to receive one +// tensor from a named channel. A sequence of tensors can be passed +// from the producer to the consumer. The consumer receives them in +// the order as the producer sends them. +// +// A consumer may safely request the tensor before or after it has +// been produced. A consumer has the choice of making a blocking call +// or providing a callback: in either case, the consumer receives the +// Tensor as soon as it is available. A producer never blocks. +class Rendezvous : public core::RefCounted { + public: + struct Args { + DeviceContext* device_context = nullptr; + AllocatorAttributes alloc_attrs; + }; + + // Constructs a rendezvous key for the tensor of "name" sent from + // "src_device" to "dst_device". The tensor is generated in the frame + // and iteration specified by "frame_iter". + static string CreateKey(const string& src_device, uint64 src_incarnation, + const string& dst_device, const string& name, + const FrameAndIter& frame_iter); + + // Parses the key constructed by CreateKey and parse src/dst device + // names into structures respectively. + struct ParsedKey { + StringPiece src_device; + DeviceNameUtils::ParsedName src; + uint64 src_incarnation = 0; + StringPiece dst_device; + DeviceNameUtils::ParsedName dst; + StringPiece edge_name; + + ParsedKey() {} + ParsedKey(const ParsedKey& b) { *this = b; } + + ParsedKey& operator=(const ParsedKey& b); + StringPiece FullKey() const { return buf_; } + + private: + friend class Rendezvous; + friend class SendOp; + friend class RecvOp; + string buf_; + }; + static Status ParseKey(StringPiece key, ParsedKey* out); + + // The caller is a tensor producer and it sends a message (a tensor + // "val" and a bool "is_dead") under the given "key". + // + // {val, is_dead} is bundled as a message sent and received. + // Typically, is_dead is set by some control flow nodes + // (e.g., a not-taken branch). args is passed by Send to the + // Recv function to communicate any information that the Recv + // function might need. This is typically only necessary for + // Send/Recv on the same worker. + // + // Send() never blocks. + virtual Status Send(const ParsedKey& key, const Args& args, const Tensor& val, + const bool is_dead) = 0; + + // Callback provided by a tensor consumer waiting on the rendezvous. + // It will be invoked when the tensor is available, or when a non-OK + // status arises in the production of that tensor. It also gets + // two Rendezvous::Args, one provided by the sender, the other by the + // receiver, which may be needed when a non-CPU device is in use + // by either side. + typedef std::function + DoneCallback; + + virtual void RecvAsync(const ParsedKey& key, const Args& args, + DoneCallback done) = 0; + + // Synchronous wrapper for RecvAsync. + Status Recv(const ParsedKey& key, const Args& args, Tensor* val, + bool* is_dead, int64 timeout_ms); + Status Recv(const ParsedKey& key, const Args& args, Tensor* val, + bool* is_dead); + + // Aborts all pending and future Send/Recv with the given "status". + // + // StartAbort() does not wait for ongoing calls to finish. + // REQUIRES: !status.ok() + virtual void StartAbort(const Status& status) = 0; + + protected: + ~Rendezvous() override; +}; + +// Returns a Rendezvous instance that is limited to use only by +// producers and consumers in the local process. The caller assumes +// ownership of one Ref() on the returned object. +Rendezvous* NewLocalRendezvous(); + +} // end namespace tensorflow + +#endif // TENSORFLOW_FRAMEWORK_RENDEZVOUS_H_ diff --git a/linux/include/tensorflow/core/framework/resource_handle.h b/linux/include/tensorflow/core/framework/resource_handle.h new file mode 100644 index 000000000..db213669a --- /dev/null +++ b/linux/include/tensorflow/core/framework/resource_handle.h @@ -0,0 +1,91 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_FRAMEWORK_RESOURCE_HANDLE_H_ +#define TENSORFLOW_FRAMEWORK_RESOURCE_HANDLE_H_ + +#include "tensorflow/core/platform/tensor_coding.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +class ResourceHandleProto; + +// Class representing a handle to a tensorflow resource. Handles are +// not valid across executions, but can be serialized back and forth from within +// a single run. +// +// This is the native C++ class equivalent of ResourceHandleProto. They are +// separate so that kernels do not need to depend on protos. +class ResourceHandle { + public: + ResourceHandle(); + ResourceHandle(const ResourceHandleProto& proto); + ~ResourceHandle(); + + // Unique name for the device containing the resource. + const string& device() const { return device_; } + void set_device(const string& device) { device_ = device; } + + // Container in which this resource is placed. + const string& container() const { return container_; } + void set_container(const string& container) { container_ = container; } + + // Unique name of this resource. + const string& name() const { return name_; } + void set_name(const string& name) { name_ = name; } + + // Hash code for the type of the resource. Is only valid in the same device + // and in the same execution. + uint64 hash_code() const { return hash_code_; } + void set_hash_code(uint64 hash_code) { hash_code_ = hash_code; } + + // For debug-only, the name of the type pointed to by this handle, if + // available. + const string& maybe_type_name() const { return maybe_type_name_; } + void set_maybe_type_name(const string& value) { maybe_type_name_ = value; } + + // Conversion to and from ResourceHandleProto + void AsProto(ResourceHandleProto* proto) const; + void FromProto(const ResourceHandleProto& proto); + + // Serialization via ResourceHandleProto + string SerializeAsString() const; + bool ParseFromString(const string& s); + + string DebugString() const; + + public: + string device_; + string container_; + string name_; + uint64 hash_code_ = 0; + string maybe_type_name_; +}; + +// For backwards compatibility for when this was a proto +string ProtoDebugString(const ResourceHandle& handle); + +// Encodes a list of ResourceHandle protos in the given StringListEncoder. +void EncodeResourceHandleList(const ResourceHandle* p, int64 n, + std::unique_ptr e); + +// Decodes a list of ResourceHandle protos from the given StringListDecoder. +bool DecodeResourceHandleList(std::unique_ptr d, + ResourceHandle* ps, int64 n); + +} // namespace tensorflow + +#endif // TENSORFLOW_FRAMEWORK_RESOURCE_HANDLE_H_ diff --git a/linux/include/tensorflow/core/framework/resource_handle.pb.h b/linux/include/tensorflow/core/framework/resource_handle.pb.h new file mode 100644 index 000000000..9f748d668 --- /dev/null +++ b/linux/include/tensorflow/core/framework/resource_handle.pb.h @@ -0,0 +1,609 @@ +// Generated by the protocol buffer compiler. DO NOT EDIT! +// source: tensorflow/core/framework/resource_handle.proto + +#ifndef PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2fresource_5fhandle_2eproto +#define PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2fresource_5fhandle_2eproto + +#include + +#include + +#if GOOGLE_PROTOBUF_VERSION < 3006000 +#error This file was generated by a newer version of protoc which is +#error incompatible with your Protocol Buffer headers. Please update +#error your headers. +#endif +#if 3006000 < GOOGLE_PROTOBUF_MIN_PROTOC_VERSION +#error This file was generated by an older version of protoc which is +#error incompatible with your Protocol Buffer headers. Please +#error regenerate this file with a newer version of protoc. +#endif + +#include +#include +#include +#include +#include +#include +#include +#include +#include // IWYU pragma: export +#include // IWYU pragma: export +#include +// @@protoc_insertion_point(includes) +#define PROTOBUF_INTERNAL_EXPORT_protobuf_tensorflow_2fcore_2fframework_2fresource_5fhandle_2eproto + +namespace protobuf_tensorflow_2fcore_2fframework_2fresource_5fhandle_2eproto { +// Internal implementation detail -- do not use these members. +struct TableStruct { + static const ::google::protobuf::internal::ParseTableField entries[]; + static const ::google::protobuf::internal::AuxillaryParseTableField aux[]; + static const ::google::protobuf::internal::ParseTable schema[1]; + static const ::google::protobuf::internal::FieldMetadata field_metadata[]; + static const ::google::protobuf::internal::SerializationTable serialization_table[]; + static const ::google::protobuf::uint32 offsets[]; +}; +void AddDescriptors(); +} // namespace protobuf_tensorflow_2fcore_2fframework_2fresource_5fhandle_2eproto +namespace tensorflow { +class ResourceHandleProto; +class ResourceHandleProtoDefaultTypeInternal; +extern ResourceHandleProtoDefaultTypeInternal _ResourceHandleProto_default_instance_; +} // namespace tensorflow +namespace google { +namespace protobuf { +template<> ::tensorflow::ResourceHandleProto* Arena::CreateMaybeMessage<::tensorflow::ResourceHandleProto>(Arena*); +} // namespace protobuf +} // namespace google +namespace tensorflow { + +// =================================================================== + +class ResourceHandleProto : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.ResourceHandleProto) */ { + public: + ResourceHandleProto(); + virtual ~ResourceHandleProto(); + + ResourceHandleProto(const ResourceHandleProto& from); + + inline ResourceHandleProto& operator=(const ResourceHandleProto& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + ResourceHandleProto(ResourceHandleProto&& from) noexcept + : ResourceHandleProto() { + *this = ::std::move(from); + } + + inline ResourceHandleProto& operator=(ResourceHandleProto&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const ResourceHandleProto& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const ResourceHandleProto* internal_default_instance() { + return reinterpret_cast( + &_ResourceHandleProto_default_instance_); + } + static constexpr int kIndexInFileMessages = + 0; + + void UnsafeArenaSwap(ResourceHandleProto* other); + void Swap(ResourceHandleProto* other); + friend void swap(ResourceHandleProto& a, ResourceHandleProto& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline ResourceHandleProto* New() const final { + return CreateMaybeMessage(NULL); + } + + ResourceHandleProto* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const ResourceHandleProto& from); + void MergeFrom(const ResourceHandleProto& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(ResourceHandleProto* other); + protected: + explicit ResourceHandleProto(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // string device = 1; + void clear_device(); + static const int kDeviceFieldNumber = 1; + const ::std::string& device() const; + void set_device(const ::std::string& value); + #if LANG_CXX11 + void set_device(::std::string&& value); + #endif + void set_device(const char* value); + void set_device(const char* value, size_t size); + ::std::string* mutable_device(); + ::std::string* release_device(); + void set_allocated_device(::std::string* device); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_device(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_device( + ::std::string* device); + + // string container = 2; + void clear_container(); + static const int kContainerFieldNumber = 2; + const ::std::string& container() const; + void set_container(const ::std::string& value); + #if LANG_CXX11 + void set_container(::std::string&& value); + #endif + void set_container(const char* value); + void set_container(const char* value, size_t size); + ::std::string* mutable_container(); + ::std::string* release_container(); + void set_allocated_container(::std::string* container); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_container(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_container( + ::std::string* container); + + // string name = 3; + void clear_name(); + static const int kNameFieldNumber = 3; + const ::std::string& name() const; + void set_name(const ::std::string& value); + #if LANG_CXX11 + void set_name(::std::string&& value); + #endif + void set_name(const char* value); + void set_name(const char* value, size_t size); + ::std::string* mutable_name(); + ::std::string* release_name(); + void set_allocated_name(::std::string* name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_name( + ::std::string* name); + + // string maybe_type_name = 5; + void clear_maybe_type_name(); + static const int kMaybeTypeNameFieldNumber = 5; + const ::std::string& maybe_type_name() const; + void set_maybe_type_name(const ::std::string& value); + #if LANG_CXX11 + void set_maybe_type_name(::std::string&& value); + #endif + void set_maybe_type_name(const char* value); + void set_maybe_type_name(const char* value, size_t size); + ::std::string* mutable_maybe_type_name(); + ::std::string* release_maybe_type_name(); + void set_allocated_maybe_type_name(::std::string* maybe_type_name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_maybe_type_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_maybe_type_name( + ::std::string* maybe_type_name); + + // uint64 hash_code = 4; + void clear_hash_code(); + static const int kHashCodeFieldNumber = 4; + ::google::protobuf::uint64 hash_code() const; + void set_hash_code(::google::protobuf::uint64 value); + + // @@protoc_insertion_point(class_scope:tensorflow.ResourceHandleProto) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::internal::ArenaStringPtr device_; + ::google::protobuf::internal::ArenaStringPtr container_; + ::google::protobuf::internal::ArenaStringPtr name_; + ::google::protobuf::internal::ArenaStringPtr maybe_type_name_; + ::google::protobuf::uint64 hash_code_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fresource_5fhandle_2eproto::TableStruct; +}; +// =================================================================== + + +// =================================================================== + +#ifdef __GNUC__ + #pragma GCC diagnostic push + #pragma GCC diagnostic ignored "-Wstrict-aliasing" +#endif // __GNUC__ +// ResourceHandleProto + +// string device = 1; +inline void ResourceHandleProto::clear_device() { + device_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& ResourceHandleProto::device() const { + // @@protoc_insertion_point(field_get:tensorflow.ResourceHandleProto.device) + return device_.Get(); +} +inline void ResourceHandleProto::set_device(const ::std::string& value) { + + device_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.ResourceHandleProto.device) +} +#if LANG_CXX11 +inline void ResourceHandleProto::set_device(::std::string&& value) { + + device_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.ResourceHandleProto.device) +} +#endif +inline void ResourceHandleProto::set_device(const char* value) { + GOOGLE_DCHECK(value != NULL); + + device_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.ResourceHandleProto.device) +} +inline void ResourceHandleProto::set_device(const char* value, + size_t size) { + + device_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.ResourceHandleProto.device) +} +inline ::std::string* ResourceHandleProto::mutable_device() { + + // @@protoc_insertion_point(field_mutable:tensorflow.ResourceHandleProto.device) + return device_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* ResourceHandleProto::release_device() { + // @@protoc_insertion_point(field_release:tensorflow.ResourceHandleProto.device) + + return device_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void ResourceHandleProto::set_allocated_device(::std::string* device) { + if (device != NULL) { + + } else { + + } + device_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), device, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.ResourceHandleProto.device) +} +inline ::std::string* ResourceHandleProto::unsafe_arena_release_device() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.ResourceHandleProto.device) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return device_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void ResourceHandleProto::unsafe_arena_set_allocated_device( + ::std::string* device) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (device != NULL) { + + } else { + + } + device_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + device, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.ResourceHandleProto.device) +} + +// string container = 2; +inline void ResourceHandleProto::clear_container() { + container_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& ResourceHandleProto::container() const { + // @@protoc_insertion_point(field_get:tensorflow.ResourceHandleProto.container) + return container_.Get(); +} +inline void ResourceHandleProto::set_container(const ::std::string& value) { + + container_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.ResourceHandleProto.container) +} +#if LANG_CXX11 +inline void ResourceHandleProto::set_container(::std::string&& value) { + + container_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.ResourceHandleProto.container) +} +#endif +inline void ResourceHandleProto::set_container(const char* value) { + GOOGLE_DCHECK(value != NULL); + + container_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.ResourceHandleProto.container) +} +inline void ResourceHandleProto::set_container(const char* value, + size_t size) { + + container_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.ResourceHandleProto.container) +} +inline ::std::string* ResourceHandleProto::mutable_container() { + + // @@protoc_insertion_point(field_mutable:tensorflow.ResourceHandleProto.container) + return container_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* ResourceHandleProto::release_container() { + // @@protoc_insertion_point(field_release:tensorflow.ResourceHandleProto.container) + + return container_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void ResourceHandleProto::set_allocated_container(::std::string* container) { + if (container != NULL) { + + } else { + + } + container_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), container, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.ResourceHandleProto.container) +} +inline ::std::string* ResourceHandleProto::unsafe_arena_release_container() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.ResourceHandleProto.container) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return container_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void ResourceHandleProto::unsafe_arena_set_allocated_container( + ::std::string* container) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (container != NULL) { + + } else { + + } + container_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + container, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.ResourceHandleProto.container) +} + +// string name = 3; +inline void ResourceHandleProto::clear_name() { + name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& ResourceHandleProto::name() const { + // @@protoc_insertion_point(field_get:tensorflow.ResourceHandleProto.name) + return name_.Get(); +} +inline void ResourceHandleProto::set_name(const ::std::string& value) { + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.ResourceHandleProto.name) +} +#if LANG_CXX11 +inline void ResourceHandleProto::set_name(::std::string&& value) { + + name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.ResourceHandleProto.name) +} +#endif +inline void ResourceHandleProto::set_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.ResourceHandleProto.name) +} +inline void ResourceHandleProto::set_name(const char* value, + size_t size) { + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.ResourceHandleProto.name) +} +inline ::std::string* ResourceHandleProto::mutable_name() { + + // @@protoc_insertion_point(field_mutable:tensorflow.ResourceHandleProto.name) + return name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* ResourceHandleProto::release_name() { + // @@protoc_insertion_point(field_release:tensorflow.ResourceHandleProto.name) + + return name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void ResourceHandleProto::set_allocated_name(::std::string* name) { + if (name != NULL) { + + } else { + + } + name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.ResourceHandleProto.name) +} +inline ::std::string* ResourceHandleProto::unsafe_arena_release_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.ResourceHandleProto.name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void ResourceHandleProto::unsafe_arena_set_allocated_name( + ::std::string* name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (name != NULL) { + + } else { + + } + name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.ResourceHandleProto.name) +} + +// uint64 hash_code = 4; +inline void ResourceHandleProto::clear_hash_code() { + hash_code_ = GOOGLE_ULONGLONG(0); +} +inline ::google::protobuf::uint64 ResourceHandleProto::hash_code() const { + // @@protoc_insertion_point(field_get:tensorflow.ResourceHandleProto.hash_code) + return hash_code_; +} +inline void ResourceHandleProto::set_hash_code(::google::protobuf::uint64 value) { + + hash_code_ = value; + // @@protoc_insertion_point(field_set:tensorflow.ResourceHandleProto.hash_code) +} + +// string maybe_type_name = 5; +inline void ResourceHandleProto::clear_maybe_type_name() { + maybe_type_name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& ResourceHandleProto::maybe_type_name() const { + // @@protoc_insertion_point(field_get:tensorflow.ResourceHandleProto.maybe_type_name) + return maybe_type_name_.Get(); +} +inline void ResourceHandleProto::set_maybe_type_name(const ::std::string& value) { + + maybe_type_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.ResourceHandleProto.maybe_type_name) +} +#if LANG_CXX11 +inline void ResourceHandleProto::set_maybe_type_name(::std::string&& value) { + + maybe_type_name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.ResourceHandleProto.maybe_type_name) +} +#endif +inline void ResourceHandleProto::set_maybe_type_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + maybe_type_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.ResourceHandleProto.maybe_type_name) +} +inline void ResourceHandleProto::set_maybe_type_name(const char* value, + size_t size) { + + maybe_type_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.ResourceHandleProto.maybe_type_name) +} +inline ::std::string* ResourceHandleProto::mutable_maybe_type_name() { + + // @@protoc_insertion_point(field_mutable:tensorflow.ResourceHandleProto.maybe_type_name) + return maybe_type_name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* ResourceHandleProto::release_maybe_type_name() { + // @@protoc_insertion_point(field_release:tensorflow.ResourceHandleProto.maybe_type_name) + + return maybe_type_name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void ResourceHandleProto::set_allocated_maybe_type_name(::std::string* maybe_type_name) { + if (maybe_type_name != NULL) { + + } else { + + } + maybe_type_name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), maybe_type_name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.ResourceHandleProto.maybe_type_name) +} +inline ::std::string* ResourceHandleProto::unsafe_arena_release_maybe_type_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.ResourceHandleProto.maybe_type_name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return maybe_type_name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void ResourceHandleProto::unsafe_arena_set_allocated_maybe_type_name( + ::std::string* maybe_type_name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (maybe_type_name != NULL) { + + } else { + + } + maybe_type_name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + maybe_type_name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.ResourceHandleProto.maybe_type_name) +} + +#ifdef __GNUC__ + #pragma GCC diagnostic pop +#endif // __GNUC__ + +// @@protoc_insertion_point(namespace_scope) + +} // namespace tensorflow + +// @@protoc_insertion_point(global_scope) + +#endif // PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2fresource_5fhandle_2eproto diff --git a/linux/include/tensorflow/core/framework/resource_handle.pb_text-impl.h b/linux/include/tensorflow/core/framework/resource_handle.pb_text-impl.h new file mode 100644 index 000000000..824df1675 --- /dev/null +++ b/linux/include/tensorflow/core/framework/resource_handle.pb_text-impl.h @@ -0,0 +1,25 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_framework_resource_handle_proto_IMPL_H_ +#define tensorflow_core_framework_resource_handle_proto_IMPL_H_ + +#include "tensorflow/core/framework/resource_handle.pb.h" +#include "tensorflow/core/framework/resource_handle.pb_text.h" +#include "tensorflow/core/lib/strings/proto_text_util.h" +#include "tensorflow/core/lib/strings/scanner.h" + +namespace tensorflow { + +namespace internal { + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::ResourceHandleProto& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::ResourceHandleProto* msg); + +} // namespace internal + +} // namespace tensorflow + +#endif // tensorflow_core_framework_resource_handle_proto_IMPL_H_ diff --git a/linux/include/tensorflow/core/framework/resource_handle.pb_text.h b/linux/include/tensorflow/core/framework/resource_handle.pb_text.h new file mode 100644 index 000000000..2535f4f10 --- /dev/null +++ b/linux/include/tensorflow/core/framework/resource_handle.pb_text.h @@ -0,0 +1,24 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_framework_resource_handle_proto_H_ +#define tensorflow_core_framework_resource_handle_proto_H_ + +#include "tensorflow/core/framework/resource_handle.pb.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/protobuf.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +// Message-text conversion for tensorflow.ResourceHandleProto +string ProtoDebugString( + const ::tensorflow::ResourceHandleProto& msg); +string ProtoShortDebugString( + const ::tensorflow::ResourceHandleProto& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::ResourceHandleProto* msg) + TF_MUST_USE_RESULT; + +} // namespace tensorflow + +#endif // tensorflow_core_framework_resource_handle_proto_H_ diff --git a/linux/include/tensorflow/core/framework/resource_mgr.h b/linux/include/tensorflow/core/framework/resource_mgr.h new file mode 100644 index 000000000..4a531648d --- /dev/null +++ b/linux/include/tensorflow/core/framework/resource_mgr.h @@ -0,0 +1,677 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_FRAMEWORK_RESOURCE_MGR_H_ +#define TENSORFLOW_CORE_FRAMEWORK_RESOURCE_MGR_H_ + +#include +#include +#include +#include +#include + +#include "tensorflow/core/framework/common_shape_fns.h" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/resource_handle.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/framework/type_index.h" +#include "tensorflow/core/lib/core/errors.h" +#include "tensorflow/core/lib/core/refcount.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/lib/hash/hash.h" +#include "tensorflow/core/platform/logging.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/mutex.h" +#include "tensorflow/core/platform/thread_annotations.h" + +namespace tensorflow { + +// A ResourceMgr instance keeps track of named and typed resources +// grouped into containers. +// +// Each resource must be represented as a sub-class of ResourceBase, +// which is reference counted explicitly. Each named resource is +// registered with ResourceMgr under a named "container" name. At any +// time, there is at most one instance of a resource given the container +// name, the resource type and the resource name. +// +// All resources for a given container can be dropped by one call of +// Cleanup(). +// +// E.g., +// struct MyVar : public ResourceBase { +// mutex mu; +// Tensor val; +// } +// +// ResourceMgr rm; +// +// // Create a var. +// MyVar* my_var = new MyVar; +// my_var->val = Tensor(DT_FLOAT, my_shape); +// my_var->val.flat().setZeros(); // 0 initialized. +// ctx->SetStatus(rm.Create("my_container", "my_name", my_var)); +// +// // += a variable. +// MyVar* my_var = nullptr; +// Status s = rm.Lookup("my_container", "my_name", &my_var); +// if (s.ok()) { +// my_var->val.flat() += grad; +// } +// my_var->Unref(); // Or use ScopedUnref(). +// ctx->SetStatus(s); +class ResourceBase : public core::RefCounted { + public: + // Returns a debug string for *this. + virtual string DebugString() = 0; + + // Returns memory used by this resource. + virtual int64 MemoryUsed() const { return 0; } +}; + +// Container used for per-step resources. +class ScopedStepContainer { + public: + // step_id: the unique ID of this step. Doesn't have to be sequential, just + // has to be unique. + // cleanup: callback to delete a container of this name. + ScopedStepContainer(const int64 step_id, + std::function cleanup) + : name_(strings::StrCat("__per_step_", step_id)), cleanup_(cleanup) {} + ~ScopedStepContainer() { cleanup_(name_); } + + const string& name() const { return name_; } + + private: + const string name_; + const std::function cleanup_; +}; + +class ResourceMgr { + public: + ResourceMgr(); + explicit ResourceMgr(const string& default_container); + ~ResourceMgr(); + + // Returns the default container name for *this. + const string& default_container() const { return default_container_; } + + // Creates a resource "name" in the "container". The caller transfers + // the ownership of one ref on "resource" to *this + // + // REQUIRES: std::is_base_of + // REQUIRES: resource != nullptr. + template + Status Create(const string& container, const string& name, + T* resource) TF_MUST_USE_RESULT; + + // If "container" has a resource "name", returns it in "*resource" and + // the caller takes the ownership of one ref on "*resource". + // + // REQUIRES: std::is_base_of + // REQUIRES: resource != nullptr + template + Status Lookup(const string& container, const string& name, + T** resource) const TF_MUST_USE_RESULT; + + // Similar to Lookup, but looks up multiple resources at once, with only a + // single lock acquisition. + template + Status LookupMany(absl::Span const> + containers_and_names, + std::vector>* + resource) const TF_MUST_USE_RESULT; + + // If "container" has a resource "name", returns it in + // "*resource". Otherwise, invokes creator() to create the resource. + // The caller takes the ownership of one ref on "*resource". + // + // WARNING: creator() must not call any methods on ResourceMgr during its + // execution, because a non-reentrant lock is held during the creator() call + // in order to guarantee atomicity of LookupOrCreate(). + // + // REQUIRES: std::is_base_of + // REQUIRES: resource != nullptr + template + Status LookupOrCreate(const string& container, const string& name, + T** resource, + std::function creator) TF_MUST_USE_RESULT; + + // Deletes the resource "name" from the "container". + // + // REQUIRES: std::is_base_of + template + Status Delete(const string& container, const string& name) TF_MUST_USE_RESULT; + + // Deletes the resource pointed by "handle". + Status Delete(const ResourceHandle& handle) TF_MUST_USE_RESULT; + + // Deletes all resources from the "container" and removes the container. + Status Cleanup(const string& container) TF_MUST_USE_RESULT; + + // Deletes all resources in all containers. + void Clear(); + + // Returns a text description for all resources. + string DebugString() const; + + private: + typedef std::pair Key; + struct KeyHash { + std::size_t operator()(const Key& k) const { + return Hash64(k.second.data(), k.second.size(), k.first); + } + }; + struct KeyEqual { + bool operator()(const Key& x, const Key& y) const { + return (x.second == y.second) && (x.first == y.first); + } + }; + typedef std::unordered_map Container; + + const string default_container_; + mutable mutex mu_; + std::unordered_map containers_ GUARDED_BY(mu_); + + template + Status LookupInternal(const string& container, const string& name, + T** resource) const + SHARED_LOCKS_REQUIRED(mu_) TF_MUST_USE_RESULT; + + Status DoCreate(const string& container, TypeIndex type, const string& name, + ResourceBase* resource) + EXCLUSIVE_LOCKS_REQUIRED(mu_) TF_MUST_USE_RESULT; + + Status DoLookup(const string& container, TypeIndex type, const string& name, + ResourceBase** resource) const + SHARED_LOCKS_REQUIRED(mu_) TF_MUST_USE_RESULT; + + Status DoDelete(const string& container, uint64 type_hash_code, + const string& resource_name, + const string& type_name) TF_MUST_USE_RESULT; + Status DoDelete(const string& container, TypeIndex type, + const string& resource_name) TF_MUST_USE_RESULT; + + // Inserts the type name for 'hash_code' into the hash_code to type name map. + Status InsertDebugTypeName(uint64 hash_code, const string& type_name) + EXCLUSIVE_LOCKS_REQUIRED(mu_) TF_MUST_USE_RESULT; + + // Returns the type name for the 'hash_code'. + // Returns "" if a resource with such a type was never inserted into + // the container. + const char* DebugTypeName(uint64 hash_code) const + EXCLUSIVE_LOCKS_REQUIRED(mu_); + + // Map from type hash_code to type name. + std::unordered_map debug_type_names_ GUARDED_BY(mu_); + + TF_DISALLOW_COPY_AND_ASSIGN(ResourceMgr); +}; + +// Makes a resource handle with the specified type for a given container / +// name. +ResourceHandle MakeResourceHandle(OpKernelContext* ctx, const string& container, + const string& name, + const TypeIndex& type_index); + +template +ResourceHandle MakeResourceHandle(OpKernelContext* ctx, const string& container, + const string& name) { + return MakeResourceHandle(ctx, container, name, MakeTypeIndex()); +} + +Status MakeResourceHandleToOutput(OpKernelContext* context, int output_index, + const string& container, const string& name, + const TypeIndex& type_index); + +template +ResourceHandle MakePerStepResourceHandle(OpKernelContext* ctx, + const string& name); + +// Returns a resource handle from a numbered op input. +const ResourceHandle& HandleFromInput(OpKernelContext* ctx, int input); +Status HandleFromInput(OpKernelContext* ctx, StringPiece input, + ResourceHandle* handle); + +// Create a resource pointed by a given resource handle. +// +// If successful, the caller transfers the ownership of one ref on `resource` to +// `ctx->resource_mgr()`. +template +Status CreateResource(OpKernelContext* ctx, const ResourceHandle& p, T* value); + +// Looks up a resource pointed by a given resource handle. +// +// If the lookup is successful, the caller takes the ownership of one ref on +// `*value`, and must call its `Unref()` method when it has finished using it. +template +Status LookupResource(OpKernelContext* ctx, const ResourceHandle& p, T** value); + +// Looks up multiple resources pointed by a sequence of resource handles. +template +Status LookupResources( + OpKernelContext* ctx, absl::Span p, + std::vector>* values); + +// Looks up or creates a resource. +// +// If successful, the caller takes the ownership of one ref on `*value`, and +// must call its `Unref()` method when it has finished using it. If the +// `creator` is invoked, its reference on the created resource is transferred +// to `ctx->resource_mgr()`. +template +Status LookupOrCreateResource(OpKernelContext* ctx, const ResourceHandle& p, + T** value, std::function creator); + +// Destroys a resource pointed by a given resource handle. +template +Status DeleteResource(OpKernelContext* ctx, const ResourceHandle& p); + +// Same as above, but uses the hash code of the type directly. +// The type name information will be missing in the debug output when the +// resource is not present in the container. +Status DeleteResource(OpKernelContext* ctx, const ResourceHandle& p); + +// Policy helper to decide which container/shared_name to use for a +// stateful kernel that accesses shared resource. +class ContainerInfo { + public: + // Analyze the node attribute of 'ndef' and decides the container and + // resource name the kernel should use for accessing the shared + // resource. + // + // 'ndef' is expected to have node attribute "container" and + // "shared_name". Returns non-OK if they are not provided or they are + // invalid. + // + // The policy is as following: + // * If the attribute "container" is non-empty, it is used as is. + // Otherwise, uses the resource manager's default container. + // * If the attribute "shared_name" is non-empty, it is used as is. + // Otherwise, if "use_node_name_as_default" is true, the kernel's + // node name is used as the resource name. Otherwise, a string + // unique to this process is used. + Status Init(ResourceMgr* rmgr, const NodeDef& ndef, + bool use_node_name_as_default); + Status Init(ResourceMgr* rmgr, const NodeDef& ndef) { + return Init(rmgr, ndef, false); + } + + // The policy decides that the kernel should access the resource in + // resource_manager(), the resource is in the container() and its + // name is name(). If resource_is_private_to_kernel() is true, the + // kernel should delete the resource when the kernel is deleted. + ResourceMgr* resource_manager() const { return rmgr_; } + const string& container() const { return container_; } + const string& name() const { return name_; } + bool resource_is_private_to_kernel() const { + return resource_is_private_to_kernel_; + } + + // Returns a readable string for *this. + string DebugString() const; + + private: + ResourceMgr* rmgr_ = nullptr; + string container_; + string name_; + bool resource_is_private_to_kernel_ = false; +}; + +// Helper for kernels to obtain 'resource' from the +// ctx->resource_manager(). +// +// "input_name" specifies the kernel's ref input which gives a string +// tensor with two elements, which specifies the container and +// resource name. +// +// Returns OK if the resource is found and transfers one ref of +// *resource to the caller. Otherwise, returns an error. +template +Status GetResourceFromContext(OpKernelContext* ctx, const string& input_name, + T** resource); + +// Utility op kernel to check if a handle to resource type T is initialized. +template +class IsResourceInitialized : public OpKernel { + public: + explicit IsResourceInitialized(OpKernelConstruction* c) : OpKernel(c) {} + + void Compute(OpKernelContext* ctx) override; +}; + +// Registers an op which produces just a resource handle to a resource of the +// specified type. The type will be a part of the generated op name. +// TODO(apassos): figure out how to get non-cpu-allocated tensors to work +// through constant folding so this doesn't have to be marked as stateful. +#define REGISTER_RESOURCE_HANDLE_OP(Type) \ + REGISTER_OP(#Type "HandleOp") \ + .Attr("container: string = ''") \ + .Attr("shared_name: string = ''") \ + .Output("resource: resource") \ + .SetIsStateful() \ + .SetShapeFn(tensorflow::shape_inference::ScalarShape) + +// Utility op kernel to produce a handle to a resource of type T. +template +class ResourceHandleOp : public OpKernel { + public: + explicit ResourceHandleOp(OpKernelConstruction* context); + + void Compute(OpKernelContext* ctx) override; + + bool IsExpensive() override { return false; } + + private: + string container_; + string name_; + mutex mutex_; + Tensor resource_; + std::atomic initialized_{false}; +}; + +// Utility op kernel to produce a handle to a resource of type T. +template +class ResourceHandlesOp : public OpKernel { + public: + explicit ResourceHandlesOp(OpKernelConstruction* context); + + void Compute(OpKernelContext* ctx) override; + + bool IsExpensive() override { return false; } + + private: + std::vector containers_; + std::vector names_; + mutex mutex_; + std::vector resources_; + std::atomic initialized_{false}; +}; + +Status ResourceHandlesShape(shape_inference::InferenceContext* c); + +// Registers a kernel for an op which produces a handle to a resource of the +// specified type. +#define REGISTER_RESOURCE_HANDLE_KERNEL(Type) \ + REGISTER_KERNEL_BUILDER(Name(#Type "HandleOp").Device(DEVICE_CPU), \ + ResourceHandleOp) + +// Implementation details below. + +template +void CheckDeriveFromResourceBase() { + static_assert(std::is_base_of::value, + "T must derive from ResourceBase"); +} + +template +Status ResourceMgr::Create(const string& container, const string& name, + T* resource) { + CheckDeriveFromResourceBase(); + CHECK(resource != nullptr); + mutex_lock l(mu_); + return DoCreate(container, MakeTypeIndex(), name, resource); +} + +template +Status ResourceMgr::Lookup(const string& container, const string& name, + T** resource) const { + CheckDeriveFromResourceBase(); + tf_shared_lock l(mu_); + return LookupInternal(container, name, resource); +} + +template +Status ResourceMgr::LookupMany( + absl::Span const> + containers_and_names, + std::vector>* resources) const { + CheckDeriveFromResourceBase(); + tf_shared_lock l(mu_); + resources->resize(containers_and_names.size()); + for (size_t i = 0; i < containers_and_names.size(); ++i) { + T* resource; + TF_RETURN_IF_ERROR(LookupInternal(*containers_and_names[i].first, + *containers_and_names[i].second, + &resource)); + (*resources)[i].reset(resource); + } + return Status::OK(); +} + +template +Status ResourceMgr::LookupInternal(const string& container, const string& name, + T** resource) const { + ResourceBase* found = nullptr; + Status s = DoLookup(container, MakeTypeIndex(), name, &found); + if (s.ok()) { + // It's safe to down cast 'found' to T* since + // typeid(T).hash_code() is part of the map key. + *resource = static_cast(found); + } + return s; +} + +template +Status ResourceMgr::LookupOrCreate(const string& container, const string& name, + T** resource, + std::function creator) { + CheckDeriveFromResourceBase(); + *resource = nullptr; + Status s; + { + tf_shared_lock l(mu_); + s = LookupInternal(container, name, resource); + if (s.ok()) return s; + } + mutex_lock l(mu_); + s = LookupInternal(container, name, resource); + if (s.ok()) return s; + TF_RETURN_IF_ERROR(creator(resource)); + s = DoCreate(container, MakeTypeIndex(), name, *resource); + if (!s.ok()) { + return errors::Internal("LookupOrCreate failed unexpectedly"); + } + (*resource)->Ref(); + return s; +} + +template +Status ResourceMgr::Delete(const string& container, const string& name) { + CheckDeriveFromResourceBase(); + return DoDelete(container, MakeTypeIndex(), name); +} + +template +Status GetResourceFromContext(OpKernelContext* ctx, const string& input_name, + T** resource) { + DataType dtype; + TF_RETURN_IF_ERROR(ctx->input_dtype(input_name, &dtype)); + if (dtype == DT_RESOURCE) { + const Tensor* handle; + TF_RETURN_IF_ERROR(ctx->input(input_name, &handle)); + return LookupResource(ctx, handle->scalar()(), resource); + } + string container; + string shared_name; + { + mutex* mu; + TF_RETURN_IF_ERROR(ctx->input_ref_mutex(input_name, &mu)); + mutex_lock l(*mu); + Tensor tensor; + TF_RETURN_IF_ERROR(ctx->mutable_input(input_name, &tensor, true)); + if (tensor.NumElements() != 2) { + return errors::InvalidArgument( + "Resource handle must have 2 elements, but had shape: ", + tensor.shape().DebugString()); + } + container = tensor.flat()(0); + shared_name = tensor.flat()(1); + } + return ctx->resource_manager()->Lookup(container, shared_name, resource); +} + +template +ResourceHandle MakePerStepResourceHandle(OpKernelContext* ctx, + const string& name) { + return MakeResourceHandle(ctx, ctx->step_container()->name(), name); +} + +namespace internal { + +Status ValidateDevice(OpKernelContext* ctx, const ResourceHandle& p); + +template +Status ValidateDeviceAndType(OpKernelContext* ctx, const ResourceHandle& p) { + TF_RETURN_IF_ERROR(internal::ValidateDevice(ctx, p)); + auto type_index = MakeTypeIndex(); + if (type_index.hash_code() != p.hash_code()) { + return errors::InvalidArgument( + "Trying to access resource using the wrong type. Expected ", + p.maybe_type_name(), " got ", type_index.name()); + } + return Status::OK(); +} + +} // namespace internal + +template +Status CreateResource(OpKernelContext* ctx, const ResourceHandle& p, T* value) { + TF_RETURN_IF_ERROR(internal::ValidateDeviceAndType(ctx, p)); + return ctx->resource_manager()->Create(p.container(), p.name(), value); +} + +template +Status LookupResource(OpKernelContext* ctx, const ResourceHandle& p, + T** value) { + TF_RETURN_IF_ERROR(internal::ValidateDeviceAndType(ctx, p)); + return ctx->resource_manager()->Lookup(p.container(), p.name(), value); +} + +template +Status LookupResources( + OpKernelContext* ctx, absl::Span p, + std::vector>* values) { + std::vector> containers_and_names( + p.size()); + for (size_t i = 0; i < p.size(); ++i) { + TF_RETURN_IF_ERROR(internal::ValidateDeviceAndType(ctx, *p[i])); + containers_and_names[i] = {&p[i]->container(), &p[i]->name()}; + } + return ctx->resource_manager()->LookupMany(containers_and_names, values); +} + +template +Status LookupOrCreateResource(OpKernelContext* ctx, const ResourceHandle& p, + T** value, std::function creator) { + TF_RETURN_IF_ERROR(internal::ValidateDeviceAndType(ctx, p)); + return ctx->resource_manager()->LookupOrCreate(p.container(), p.name(), value, + creator); +} + +template +Status DeleteResource(OpKernelContext* ctx, const ResourceHandle& p) { + TF_RETURN_IF_ERROR(internal::ValidateDeviceAndType(ctx, p)); + return ctx->resource_manager()->Delete(p.container(), p.name()); +} + +Status DeleteResource(OpKernelContext* ctx, const ResourceHandle& p); + +template +void IsResourceInitialized::Compute(OpKernelContext* ctx) { + Tensor* output; + OP_REQUIRES_OK(ctx, ctx->allocate_output(0, {}, &output)); + T* object; + bool found; + if (LookupResource(ctx, HandleFromInput(ctx, 0), &object).ok()) { + found = true; + object->Unref(); + } else { + found = false; + } + + output->flat()(0) = found; +} + +template +ResourceHandleOp::ResourceHandleOp(OpKernelConstruction* context) + : OpKernel(context) { + OP_REQUIRES_OK(context, context->GetAttr("container", &container_)); + OP_REQUIRES_OK(context, context->GetAttr("shared_name", &name_)); +} + +template +void ResourceHandleOp::Compute(OpKernelContext* ctx) { + if (!initialized_.load()) { + mutex_lock ml(mutex_); + // Checking again to see if another thread has initialized the resource. + if (!initialized_.load()) { + AllocatorAttributes attr; + attr.set_on_host(true); + OP_REQUIRES_OK(ctx, ctx->allocate_temp(DT_RESOURCE, TensorShape({}), + &resource_, attr)); + resource_.scalar()() = + MakeResourceHandle(ctx, container_, name_); + initialized_.store(true); + } + } + ctx->set_output(0, resource_); +} + +template +ResourceHandlesOp::ResourceHandlesOp(OpKernelConstruction* context) + : OpKernel(context) { + int n; + OP_REQUIRES_OK(context, context->GetAttr("N", &n)); + OP_REQUIRES_OK(context, context->GetAttr("containers", &containers_)); + OP_REQUIRES_OK(context, context->GetAttr("shared_names", &names_)); + OP_REQUIRES( + context, containers_.size() == n, + errors::InvalidArgument("Number of containers (", containers_.size(), + ") must be equal to N (", n, ")")); + OP_REQUIRES(context, names_.size() == n, + errors::InvalidArgument("Number of names (", containers_.size(), + ") must be equal to N (", n, ")")); + resources_.resize(n); +} + +template +void ResourceHandlesOp::Compute(OpKernelContext* ctx) { + if (!initialized_.load()) { + mutex_lock ml(mutex_); + // Checking again to see if another thread has initialized the resource. + if (!initialized_.load()) { + AllocatorAttributes attr; + attr.set_on_host(true); + for (size_t i = 0; i < resources_.size(); ++i) { + OP_REQUIRES_OK(ctx, ctx->allocate_temp(DT_RESOURCE, TensorShape({}), + &resources_[i], attr)); + ResourceHandle h = + MakeResourceHandle(ctx, containers_[i], names_[i]); + resources_[i].template scalar()() = h; + } + initialized_.store(true); + } + } + for (size_t i = 0; i < resources_.size(); ++i) { + ctx->set_output(i, resources_[i]); + } +} + +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_FRAMEWORK_RESOURCE_MGR_H_ diff --git a/linux/include/tensorflow/core/framework/resource_op_kernel.h b/linux/include/tensorflow/core/framework/resource_op_kernel.h new file mode 100644 index 000000000..fbcd439de --- /dev/null +++ b/linux/include/tensorflow/core/framework/resource_op_kernel.h @@ -0,0 +1,139 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_FRAMEWORK_RESOURCE_OP_KERNEL_H_ +#define TENSORFLOW_CORE_FRAMEWORK_RESOURCE_OP_KERNEL_H_ + +#include + +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/resource_mgr.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/platform/logging.h" +#include "tensorflow/core/platform/mutex.h" +#include "tensorflow/core/platform/thread_annotations.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +// ResourceOpKernel is a virtual base class for resource op implementing +// interface type T. The inherited op looks up the resource name (determined by +// ContainerInfo), and creates a new resource if necessary. +// +// Requirements: +// - Op must be marked as stateful. +// - Op must have `container` and `shared_name` attributes. Empty `container` +// means using the default container. Empty `shared_name` means private +// resource. +// - Subclass must override CreateResource(). +// - Subclass is encouraged to override VerifyResource(). +template +class ResourceOpKernel : public OpKernel { + public: + explicit ResourceOpKernel(OpKernelConstruction* context) : OpKernel(context) { + has_resource_type_ = (context->output_type(0) == DT_RESOURCE); + if (!has_resource_type_) { + // The resource variant of the op may be placed on non-CPU devices, but + // this allocation is always on the host. Fortunately we don't need it in + // the resource case. + OP_REQUIRES_OK(context, + context->allocate_persistent(DT_STRING, TensorShape({2}), + &handle_, nullptr)); + } + } + + // The resource is deleted from the resource manager only when it is private + // to kernel. Ideally the resource should be deleted when it is no longer held + // by anyone, but it would break backward compatibility. + ~ResourceOpKernel() override { + if (resource_ != nullptr) { + resource_->Unref(); + if (cinfo_.resource_is_private_to_kernel()) { + if (!cinfo_.resource_manager() + ->template Delete(cinfo_.container(), cinfo_.name()) + .ok()) { + // Do nothing; the resource can have been deleted by session resets. + } + } + } + } + + void Compute(OpKernelContext* context) override LOCKS_EXCLUDED(mu_) { + mutex_lock l(mu_); + if (resource_ == nullptr) { + ResourceMgr* mgr = context->resource_manager(); + OP_REQUIRES_OK(context, cinfo_.Init(mgr, def())); + + T* resource; + OP_REQUIRES_OK( + context, + mgr->LookupOrCreate(cinfo_.container(), cinfo_.name(), &resource, + [this](T** ret) EXCLUSIVE_LOCKS_REQUIRED(mu_) { + Status s = CreateResource(ret); + if (!s.ok() && *ret != nullptr) { + CHECK((*ret)->Unref()); + } + return s; + })); + + Status s = VerifyResource(resource); + if (TF_PREDICT_FALSE(!s.ok())) { + resource->Unref(); + context->SetStatus(s); + return; + } + + if (!has_resource_type_) { + auto h = handle_.AccessTensor(context)->template flat(); + h(0) = cinfo_.container(); + h(1) = cinfo_.name(); + } + resource_ = resource; + } + if (has_resource_type_) { + OP_REQUIRES_OK(context, MakeResourceHandleToOutput( + context, 0, cinfo_.container(), cinfo_.name(), + MakeTypeIndex())); + } else { + context->set_output_ref(0, &mu_, handle_.AccessTensor(context)); + } + } + + protected: + // Variables accessible from subclasses. + mutex mu_; + ContainerInfo cinfo_ GUARDED_BY(mu_); + T* resource_ GUARDED_BY(mu_) = nullptr; + + private: + // Must return a T descendant allocated with new that ResourceOpKernel will + // take ownership of. + virtual Status CreateResource(T** resource) EXCLUSIVE_LOCKS_REQUIRED(mu_) = 0; + + // During the first Compute(), resource is either created or looked up using + // shared_name. In the latter case, the resource found should be verified if + // it is compatible with this op's configuration. The verification may fail in + // cases such as two graphs asking queues of the same shared name to have + // inconsistent capacities. + virtual Status VerifyResource(T* resource) { return Status::OK(); } + + PersistentTensor handle_ GUARDED_BY(mu_); + + // Is the output of the operator of type DT_RESOURCE? + bool has_resource_type_; +}; +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_FRAMEWORK_RESOURCE_OP_KERNEL_H_ diff --git a/linux/include/tensorflow/core/framework/resource_var.h b/linux/include/tensorflow/core/framework/resource_var.h new file mode 100644 index 000000000..ff7b3e78a --- /dev/null +++ b/linux/include/tensorflow/core/framework/resource_var.h @@ -0,0 +1,60 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_FRAMEWORK_RESOURCE_VAR_H_ +#define TENSORFLOW_CORE_FRAMEWORK_RESOURCE_VAR_H_ + +#include "tensorflow/core/framework/resource_mgr.h" + +namespace tensorflow { + +// Resource stored by variables in the resource manager +// (new, resource-style version). +class Var : public ResourceBase { + public: + explicit Var(DataType dtype) : tensor_(dtype) {} + // Not copyable or movable. + Var(const Var&) = delete; + Var& operator=(const Var&) = delete; + + // When locking multiple variables, the locks must be acquired in order of + // increasing mu() address. + // TODO(ebrevdo): Use LockSet instead of exposing mu. + mutex* mu() { return &mu_; } + Tensor* tensor() { return &tensor_; } + + string DebugString() override { + return strings::StrCat(DataTypeString(tensor_.dtype()), "/", + tensor_.shape().DebugString()); + } + + // Only used in the resource variable path. In resource variables, + // tensor.IsInitialized() can be true (i.e. have memory allocated to it) while + // there is not a good value there due to a race condition, and it's possible + // to stumble upon this during variable.initialized_value(). So it's best to + // just store directly whether the variable is initialized. + bool is_initialized = false; // GUARDED_BY(mu_) but annotalysis doesn't like + // it. + + private: + mutex mu_; + Tensor tensor_; + + ~Var() override {} +}; + +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_FRAMEWORK_RESOURCE_VAR_H_ diff --git a/linux/include/tensorflow/core/framework/run_handler.h b/linux/include/tensorflow/core/framework/run_handler.h new file mode 100644 index 000000000..72fa6301b --- /dev/null +++ b/linux/include/tensorflow/core/framework/run_handler.h @@ -0,0 +1,95 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_FRAMEWORK_RUN_HANDLER_H_ +#define TENSORFLOW_CORE_FRAMEWORK_RUN_HANDLER_H_ + +#include "tensorflow/core/lib/core/threadpool.h" +#include "tensorflow/core/lib/histogram/histogram.h" +#include "tensorflow/core/platform/mutex.h" +#include "tensorflow/core/platform/thread_annotations.h" +#include "tensorflow/core/protobuf/config.pb.h" + +namespace tensorflow { + +class RunHandler; + +// RunHandlerPool is a fixed size pool of pre-allocated RunHandlers +// that can be used for tracking inter-op work for a given Session::Run(). +// RunHandler(s) in the pool are initially 'inactive'. A RunHandler becomes +// 'active' when its unique_ptr is returned by Get() and is being used by a +// client. It becomes 'inactive' once more when its unique_ptr gets destroyed. +// +// Expected usage: +// +// * Create a single RunHandlerPool (say run_handler_pool_). +// +// * When a Session::Run() is invoked, obtain a handler by: +// auto handler = run_handler_pool_->Get(); +// +// * Use handler for scheduling all inter-op work by: +// handler->ScheduleInterOpClosure(closure); +// +// This class is thread safe. +class RunHandlerPool { + public: + explicit RunHandlerPool(int num_inter_op_threads); + ~RunHandlerPool(); + + // Returns an inactive RunHandler from the pool. + // + // RunHandlers in RunHandlerPool are initially 'inactive'. + // A RunHandler becomes 'active' when its unique_ptr its returned by Get() + // and is being used by a client. It becomes 'inactive' once more when the + // unique_ptr is destroyed. + // + // Will block unless there is an inactive handler. + std::unique_ptr Get(); + + private: + class Impl; + friend class RunHandler; + + std::unique_ptr impl_; +}; + +// RunHandler can be used to schedule inter-op closures to run on a global pool +// shared across all Session::Run(s). +// +// It can only be created via RunHandlerPool::Get(). +// +// This class can be used instead of directly scheduling closures on a global +// pool since it maintains a global view across all sessions and optimizes pool +// scheduling to improve (median and tail) latency. +// +// This class is thread safe. +class RunHandler { + public: + void ScheduleInterOpClosure(std::function fn); + + ~RunHandler(); + + private: + class Impl; + friend class RunHandlerPool::Impl; + + explicit RunHandler(Impl* impl); + + Impl* impl_; // NOT OWNED. +}; + +} // end namespace tensorflow. + +#endif // TENSORFLOW_CORE_FRAMEWORK_RUN_HANDLER_H_ diff --git a/linux/include/tensorflow/core/framework/run_handler_util.h b/linux/include/tensorflow/core/framework/run_handler_util.h new file mode 100644 index 000000000..c0c36aecc --- /dev/null +++ b/linux/include/tensorflow/core/framework/run_handler_util.h @@ -0,0 +1,43 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_FRAMEWORK_RUN_HANDLER_UTIL_H_ +#define TENSORFLOW_CORE_FRAMEWORK_RUN_HANDLER_UTIL_H_ + +#include +#include + +namespace tensorflow { + +// Assign thread ranges to requests. +// Requests are numbered 0...num_active_requests-1, and +// threads are numbered 0...num_threads-1. +// On return, the range start_vec->at(i)...end_vec->at(i)-1 +// indicates the subrange of the threads available to request i. +// The ranges given to different requests may overlap. +// Lower numbered requests will tend to be assigned more threads. +// Thus, a client might associate older requests with lower +// array indices so they receive access to more threads. +// However, the routine ensures that each request is given access +// to at least min(min_threads_per_request, num_threads) threads. +// Every thread will be assigned to at least one request range, +// assuming there is at least one request. +void ComputeInterOpSchedulingRanges(int num_active_requests, int num_threads, + int min_threads_per_request, + std::vector* start_vec, + std::vector* end_vec); + +} // end namespace tensorflow +#endif // TENSORFLOW_CORE_FRAMEWORK_RUN_HANDLER_UTIL_H_ diff --git a/linux/include/tensorflow/core/framework/selective_registration.h b/linux/include/tensorflow/core/framework/selective_registration.h new file mode 100644 index 000000000..4b281a04b --- /dev/null +++ b/linux/include/tensorflow/core/framework/selective_registration.h @@ -0,0 +1,58 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_FRAMEWORK_SELECTIVE_REGISTRATION_H_ +#define TENSORFLOW_CORE_FRAMEWORK_SELECTIVE_REGISTRATION_H_ + +#include + +#ifdef SELECTIVE_REGISTRATION + +// Experimental selective registration support to reduce binary size. +// +// To use selective registration, when building: +// 1. define SELECTIVE_REGISTRATION, e.g. in gcc by passing +// -DSELECTIVE_REGISTRATION to compilation. +// 2. Provide ops_to_register.h. This file is not included in the repo and must +// be placed by the user or a tool where the compiler can find it. It must +// define the constants and functions used in the macros below. The +// functions should be defined as valid constexpr functions, so that they are +// evaluated at compile time: this is needed to make symbols referenced by +// un-registered objects unused, and therefore allow the linker to strip them +// out. See python/tools/print_selective_registration_header.py for a tool +// that can be used to generate ops_to_register.h. +// +// ops_to_register.h should define macros for: +// // Ops for which this is false will not be registered. +// SHOULD_REGISTER_OP(op) +// // If this is false, then no gradient ops are registered. +// SHOULD_REGISTER_OP_GRADIENT +// // Op kernel classes where this is false won't be registered. +// SHOULD_REGISTER_OP_KERNEL(clz) +// The macros should be defined using constexprs. + +#include "ops_to_register.h" + +#if (!defined(SHOULD_REGISTER_OP) || !defined(SHOULD_REGISTER_OP_GRADIENT) || \ + !defined(SHOULD_REGISTER_OP_KERNEL)) +static_assert(false, "ops_to_register.h must define SHOULD_REGISTER macros"); +#endif +#else +#define SHOULD_REGISTER_OP(op) true +#define SHOULD_REGISTER_OP_GRADIENT true +#define SHOULD_REGISTER_OP_KERNEL(clz) true +#endif + +#endif // TENSORFLOW_CORE_FRAMEWORK_SELECTIVE_REGISTRATION_H_ diff --git a/linux/include/tensorflow/core/framework/session_state.h b/linux/include/tensorflow/core/framework/session_state.h new file mode 100644 index 000000000..63568685f --- /dev/null +++ b/linux/include/tensorflow/core/framework/session_state.h @@ -0,0 +1,93 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_FRAMEWORK_SESSION_STATE_H_ +#define TENSORFLOW_CORE_FRAMEWORK_SESSION_STATE_H_ + +#include +#include +#include + +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/lib/core/errors.h" +#include "tensorflow/core/platform/mutex.h" + +namespace tensorflow { + +// The session state remembers the tensors we choose to keep across +// multiple run calls. +class SessionState { + public: + // Get a tensor from the session state. + Status GetTensor(const string& handle, Tensor* tensor); + + // Store a tensor in the session state. + Status AddTensor(const string& handle, const Tensor& tensor); + + // Delete a tensdor from the session state. + Status DeleteTensor(const string& handle); + + int64 GetNewId(); + + static const char* kTensorHandleResourceTypeName; + + private: + mutex state_lock_; + + // For generating unique ids for tensors stored in the session. + int64 tensor_id_ = 0; + + // The live tensors in the session. A map from tensor handle to tensor. + std::unordered_map tensors_; +}; + +// The tensor store remembers the tensors we choose to keep for the +// current run call. It is available to every op kernel. +class TensorStore { + public: + struct TensorAndKey { + Tensor tensor; + int64 id; + string device_name; + + string GetHandle(const string& tensor_name) { + return strings::StrCat(tensor_name, ";", id, ";", device_name); + } + }; + + // Add the named tensor to the tensor store for this run. + Status AddTensor(const string& name, const TensorAndKey& tk); + + // Save the tensors in the tensor store of this run to the session. + Status SaveTensors(const std::vector& output_names, + SessionState* session_state); + + // Returns true if no tensors have been added to this store. + bool empty() { + mutex_lock l(lock_); + return tensors_.empty(); + } + + private: + mutex lock_; + + // The tensors that will be saved to session state when this run completes. + // A map from tensor string name to tensor. + std::unordered_map tensors_ GUARDED_BY(lock_); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_FRAMEWORK_SESSION_STATE_H_ diff --git a/linux/include/tensorflow/core/framework/shape_inference.h b/linux/include/tensorflow/core/framework/shape_inference.h new file mode 100644 index 000000000..81258b55b --- /dev/null +++ b/linux/include/tensorflow/core/framework/shape_inference.h @@ -0,0 +1,823 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CORE_FRAMEWORK_SHAPE_INFERENCE_H_ +#define TENSORFLOW_CORE_FRAMEWORK_SHAPE_INFERENCE_H_ + +#include + +#include "tensorflow/core/framework/node_def_util.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/lib/core/errors.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/lib/gtl/inlined_vector.h" +#include "tensorflow/core/platform/macros.h" + +namespace tensorflow { + +class ShapeRefiner; +class ShapeRefinerTest; + +namespace grappler { +class GraphProperties; +class SymbolicShapeManager; +} // namespace grappler + +namespace shape_inference { + +struct DimensionOrConstant; +class InferenceContext; + +// Dimension values are accessed through InferenceContext. +class Dimension { + private: + Dimension(); + Dimension(int64 value); + ~Dimension() {} + + const int64 value_; + + friend class InferenceContext; + friend class ShapeManager; + TF_DISALLOW_COPY_AND_ASSIGN(Dimension); +}; + +class DimensionHandle { + public: + DimensionHandle() {} + bool SameHandle(DimensionHandle d) const { return ptr_ == d.ptr_; } + std::size_t Handle() const { return reinterpret_cast(ptr_); } + + private: + DimensionHandle(const Dimension* dim) { ptr_ = dim; } + + const Dimension* operator->() const { return ptr_; } + bool IsSet() const { return ptr_ != nullptr; } + + const Dimension* ptr_ = nullptr; + + friend struct DimensionOrConstant; + friend class InferenceContext; + friend class ShapeInferenceTest; + friend class ShapeInferenceTestutil; + friend class ::tensorflow::ShapeRefinerTest; + friend class ShapeManager; + friend class ::tensorflow::grappler::GraphProperties; + friend class ::tensorflow::grappler::SymbolicShapeManager; + + // Intentionally copyable. +}; + +// Shape rank and dimensions are accessed through InferenceContext. +class Shape { + private: + Shape(); + Shape(const std::vector& dims); + ~Shape() {} + + const int32 rank_; + const std::vector dims_; + + friend class InferenceContext; + friend class ShapeManager; + friend class ::tensorflow::grappler::SymbolicShapeManager; + + TF_DISALLOW_COPY_AND_ASSIGN(Shape); +}; + +class ShapeHandle { + public: + ShapeHandle() {} + bool SameHandle(ShapeHandle s) const { return ptr_ == s.ptr_; } + std::size_t Handle() const { return reinterpret_cast(ptr_); } + + private: + ShapeHandle(const Shape* shape) { ptr_ = shape; } + const Shape* operator->() const { return ptr_; } + bool IsSet() const { return ptr_ != nullptr; } + + const Shape* ptr_ = nullptr; + + friend class InferenceContext; + friend class ShapeInferenceTest; + friend class ShapeInferenceTestutil; + friend class ::tensorflow::ShapeRefinerTest; + friend class ShapeManager; + friend class ::tensorflow::grappler::SymbolicShapeManager; + + // Intentionally copyable. +}; + +// Struct used to allow functions to take DimensionHandle or a dimension value. +// Not meant to be constructed directly. +struct DimensionOrConstant { + public: + // Intentionally not explicit. + DimensionOrConstant(DimensionHandle dim); + + // val must be non-negative or InferenceContext::kUnknownDim. + DimensionOrConstant(int64 val); + + // dim takes precedence. If dim != nullptr, val is ignored. + DimensionHandle dim; + int64 val; + + private: + DimensionOrConstant(); +}; + +struct ShapeAndType { + ShapeAndType() {} + ShapeAndType(ShapeHandle s, DataType t) : shape(s), dtype(t) {} + + ShapeHandle shape; + DataType dtype = DT_INVALID; +}; + +// Shape inference functions registered on ops in REGISTER_OP implement +// their shape functions in terms of this InferenceContext. An InferenceContext +// is created by the framework and passed to a shape inference function. The +// shape inference function calls functions on the context, and should call +// set_output() to set the shape on all outputs. +// +// To infer shapes for user-defined functions see ShapeRefiner. +// +// All Shape* and Dimension* returned by functions of InferenceContext are owned +// by the InferenceContext. +class InferenceContext { + public: + static constexpr int64 kUnknownDim = -1; + static constexpr int32 kUnknownRank = -1; + + // is NULL-padded to be the same size as . + // + // Elements of are used for when a shape function + // makes a call to MakeShapeFromShapeTensor; in particular, when the + // input_tensors[i] is nullptr but the shape represented by it is partially + // known from analysis of the graph. + // can have fewer elements than . + // Values of do not need to outlive the context. + // + // REQUIRES: is not NULL, and must outlive the InferenceContext. + InferenceContext(int graph_def_version, const NodeDef* node_def, + const OpDef& op_def, + const std::vector& input_shapes, + const std::vector& input_tensors, + const std::vector& input_tensors_as_shapes, + std::vector>> + input_handle_shapes_and_types); + + // is NULL-padded to be the same size as . + // + // Elements of are used for when a shape + // function makes a call to MakeShapeFromShapeTensor; in particular, when + // the input_tensors[i] is nullptr but the shape represented by it is + // partially known from analysis of the graph. + // can have fewer elements than . Values of + // do not need to outlive the context. + // + // REQUIRES: is not NULL, and must outlive the + // InferenceContext. + InferenceContext( + int graph_def_version, const NodeDef* node_def, const OpDef& op_def, + const std::vector& input_shapes, + const std::vector& input_tensors, + const std::vector& input_tensors_as_shapes, + const std::vector< + std::unique_ptr>>>& + input_handle_shapes_and_types); + + // is NULL-padded to be the same size as . + // + // Elements of are used for when a shape + // function makes a call to MakeShapeFromShapeTensor; in particular, when + // the input_tensors[i] is nullptr but the shape represented by it is + // partially known from analysis of the graph. + // can have fewer elements than . Values of + // do not need to outlive the context. + // + // REQUIRES: is not NULL, and must outlive the + // InferenceContext. + InferenceContext( + int graph_def_version, const NodeDef* node_def, const OpDef& op_def, + const std::vector& input_shapes, + const std::vector& input_tensors, + const std::vector& input_tensors_as_shapes, + const std::vector>>>& + input_handle_shapes_and_types); + + ~InferenceContext(); + + // Runs the shape inference function 'fn' with 'this' as the + // argument, returns the status of the inference. + // + // On error, additional context is provided in the error message. + Status Run( + const std::function& fn); + + // Merge the stored shape of the input in position idx with according + // to the following rules: + // + // - If the ShapeHandles are the same or is unknown, there will be no + // change. Otherwise if the stored shape is unknown, the new shape will be + // . + // - If both shapes are known, then they must have the same rank. + // - For any one dimension, if the values for that dimension in both shapes + // are known, then the values must match. + // - If one shape has equal or more information than the other shape in every + // dimension, the new shape will become the shape with more information. + // - Example: merging [2,?] and [?,2] results in [2,2] + // - Example: [2,2] cannot be merged with [1,2] + // + // This requires idx to be in the [0, num_inputs) range. If the merge is + // successful, return true. Return false otherwise. + bool MergeInput(int idx, ShapeHandle shape) { + ShapeHandle new_shape; + if (!Merge(inputs_[idx], shape, &new_shape).ok()) return false; + inputs_[idx] = new_shape; + return true; + } + + // Relax the stored shape of the input in position idx with according + // to the following rules: + // + // - If the ShapeHandles are the same then the stored shape will be returned. + // - If either of the ShapeHandles are unknown, then a new UnknownShape will + // be returned. A new shape must be returned because we cannot claim that + // the resulting shape is necessarily the same as either of the input + // shapes. + // - If the shapes both have known ranks but their ranks are different, a new + // UnknownShape will be returned. + // - For any one dimension, if the value for that dimension in either of the + // shapes is unknown, a new shape will be returned with a new UnknownDim in + // that dimension. + // - For any one dimension, if the values for that dimension in both shapes + // are known but do not match, a new shape will be returned with a new + // UnknownDim in that dimension. + // - If both shapes have the same known rank and match in every dimension, + // the stored shape will be returned. + // - Example: relaxing [2,?] and [?,2] results in [?,?] + // - Example: relaxing [2,2] and [3,2] results in [?,2] + // - Example: relaxing [2,2] with [1,2,3] results in ? + // + // This requires idx to be in the [0, num_inputs) range. If the relax is + // successful and the new shape differs from the old one, store the new + // shape and return true. Return false otherwise. + bool RelaxInput(int idx, ShapeHandle shape) { + ShapeHandle new_shape; + Relax(inputs_[idx], shape, &new_shape); + if (inputs_[idx].SameHandle(new_shape)) { + return false; + } + inputs_[idx] = new_shape; + return true; + } + + void SetInput(int idx, ShapeHandle shape) { inputs_[idx] = shape; } + + ShapeHandle input(int64 idx) const { return inputs_[idx]; } + Status input(StringPiece input_name, std::vector* output) const; + int num_inputs() const { return inputs_.size(); } + + // Returns the input tensor at index , or nullptr if the input tensor is + // not available at the time of shape inference. + const Tensor* input_tensor(int idx) { + // Mark that this idx was requested. + requested_input_tensor_[idx] = true; + return input_tensors_[idx]; + } + + // Returns true iff input_tensor(idx) was called by the shape function. + bool requested_input_tensor(int idx) const { + return requested_input_tensor_[idx]; + } + + // Returns true if MakeShapeFromInputTensor was called but the constant + // input_tensor was not present. + bool requested_input_tensor_as_partial_shape(int idx) const { + return requested_input_tensor_as_partial_shape_[idx]; + } + + void set_input_tensors(const std::vector& input_tensors) { + input_tensors_ = input_tensors; + } + + void set_input_tensors_as_shapes( + const std::vector& input_tensors_as_shapes) { + input_tensors_as_shapes_ = input_tensors_as_shapes; + } + + const std::vector& input_tensors_as_shapes() const { + return input_tensors_as_shapes_; + } + + ShapeHandle output(int64 idx) const { return outputs_[idx]; } + void set_output(int idx, ShapeHandle shape) { outputs_[idx] = shape; } + Status set_output(StringPiece output_name, + const std::vector& shapes); + + int num_outputs() const { return outputs_.size(); } + ShapeHandle output(int idx) const { return outputs_[idx]; } + Status output(StringPiece output_name, + std::vector* output) const; + + AttrSlice attrs() const { return AttrSlice(*node_def_); } + + string op() const; + + // idx can be negative for an offset from end of dimensions. + // idx must be in the range [-1 * s.rank, s.rank). + DimensionHandle Dim(ShapeHandle s, int64 idx) { + if (s->rank_ == kUnknownRank) { + return UnknownDim(); + } + return DimKnownRank(s, idx); + } + // As above, but asserts that the rank of the shape is known. + static DimensionHandle DimKnownRank(ShapeHandle s, int64 idx) { + CHECK_NE(s->rank_, kUnknownRank); + if (idx < 0) { + return s->dims_[s->dims_.size() + idx]; + } + return s->dims_[idx]; + } + + static int32 Rank(ShapeHandle s) { + DCHECK(s.IsSet()); + return s.IsSet() ? s->rank_ : kUnknownRank; + } + static bool RankKnown(ShapeHandle s) { + return (s.IsSet() && (Rank(s) != kUnknownRank)); + } + static inline int64 Value(DimensionOrConstant d) { + return d.dim.IsSet() ? d.dim->value_ : d.val; + } + static inline bool ValueKnown(DimensionOrConstant d) { + return Value(d) != kUnknownDim; + } + + // Fills the output proto with the shape defined by the handle. + // "proto" is expected to be empty prior to the call. + void ShapeHandleToProto(ShapeHandle handle, TensorShapeProto* proto); + + // Returns true if the rank and all dimensions of the Shape are known. + bool FullyDefined(ShapeHandle s); + + // Returns the total number of elements, or an unknown dimension for an + // incomplete shape. + DimensionHandle NumElements(ShapeHandle s); + + string DebugString(ShapeHandle s); + string DebugString(DimensionHandle d); + string DebugString(const ShapeAndType& shape_and_type); + string DebugString(gtl::ArraySlice shape_and_types); + + // Describes the whole context, for debugging purposes. + string DebugString() const; + + // If has rank , or its rank is unknown, return OK and return + // the shape with asserted rank in <*out>. Otherwise return an error. + // + // Note that <*out> may be set to . + Status WithRank(ShapeHandle shape, int64 rank, + ShapeHandle* out) TF_MUST_USE_RESULT; + Status WithRankAtLeast(ShapeHandle shape, int64 rank, + ShapeHandle* out) TF_MUST_USE_RESULT; + Status WithRankAtMost(ShapeHandle shape, int64 rank, + ShapeHandle* out) TF_MUST_USE_RESULT; + + // If has value , or its value is unknown, returns OK and returns + // the dimension with asserted value in <*out>. Otherwise returns an error. + // + // Note that <*out> may be set to . + Status WithValue(DimensionHandle dim, int64 value, + DimensionHandle* out) TF_MUST_USE_RESULT; + + // Merges and and returns the merged shape in <*out>. See + // 'MergeInput' function for full details and examples. + Status Merge(ShapeHandle s0, ShapeHandle s1, + ShapeHandle* out) TF_MUST_USE_RESULT; + + // Asserts that 's rank >= 's rank, and the first + // dimensions of are compatible with the dimensions of + // . + // Returns the merged results in <*s_out> and <*prefix_out>. + Status MergePrefix(ShapeHandle s, ShapeHandle prefix, ShapeHandle* s_out, + ShapeHandle* prefix_out) TF_MUST_USE_RESULT; + + // Merges and and returns the merged dimension in <*out>. If + // and have incompatible values, returns an error. + // + // Note that <*out> may be set to or . + Status Merge(DimensionHandle d0, DimensionHandle d1, + DimensionHandle* out) TF_MUST_USE_RESULT; + + // Returns in <*out> a sub-shape of with dimensions [start:]. + // can be negative to index from the end of the shape. If > + // rank of , then an empty subshape is returned. + Status Subshape(ShapeHandle s, int64 start, + ShapeHandle* out) TF_MUST_USE_RESULT; + + // Returns in <*out> a sub-shape of , with dimensions [start:end]. + // and can be negative, to index from the end of the shape. + // and are set to the rank of if > rank of . + Status Subshape(ShapeHandle s, int64 start, int64 end, + ShapeHandle* out) TF_MUST_USE_RESULT; + + // Returns in <*out> a sub-shape of , with dimensions [start:end:stride]. + // and can be negative, to index from the end of the shape. + // and are set to the rank of if > rank of . + // can be negative, to reverse the . + Status Subshape(ShapeHandle s, int64 start, int64 end, int64 stride, + ShapeHandle* out) TF_MUST_USE_RESULT; + + // Returns in <*out> the result of appending the dimensions of to those + // of . + Status Concatenate(ShapeHandle s1, ShapeHandle s2, + ShapeHandle* out) TF_MUST_USE_RESULT; + + // Returns in the shape from replacing with + // . + Status ReplaceDim(ShapeHandle s, int64 dim_index, DimensionHandle new_dim, + ShapeHandle* out) TF_MUST_USE_RESULT; + + // Returns a new shape with the given dims. The returned value is owned by + // this context. + ShapeHandle MakeShape(const std::vector& dims); + ShapeHandle MakeShape(std::initializer_list dims); + + // Returns a new unknown shape. + ShapeHandle UnknownShape(); + + // Returns a shape with specified rank but unknown dims. + ShapeHandle UnknownShapeOfRank(int64 rank); + + // Returns a new shape of zero dimensions. + ShapeHandle Scalar(); + + // Returns a new shape of one dimension. + ShapeHandle Vector(DimensionOrConstant dim); + + // Returns a new shape of two dimensions. + ShapeHandle Matrix(DimensionOrConstant dim1, DimensionOrConstant dim2); + + // Returns in a new shape whose dimension sizes come from input tensor + // . The tensor must be a 1-dimensional int32 or int64 tensor. If + // the input tensor is NULL, then an unknown shape is returned. + Status MakeShapeFromShapeTensor(int input_idx, ShapeHandle* out); + + // Like the function above, but treats scalar values as unknown + // shapes. **NOTE** If the scalar is statically known, its value + // must be -1 or an error is returned. + Status MakeShapeFromShapeTensorTreatScalarAsUnknownShape(int input_idx, + ShapeHandle* out); + + // Returns in a new shape corresponding to . + Status MakeShapeFromShapeProto(const TensorShapeProto& proto, + ShapeHandle* out); + + // Returns in a new shape corresponding to . + Status MakeShapeFromPartialTensorShape( + const PartialTensorShape& partial_shape, ShapeHandle* out); + + // Returns in a new shape corresponding to . + Status MakeShapeFromTensorShape(const TensorShape& shape, ShapeHandle* out); + + // Returns a new dimension of the given size. The returned value is owned by + // this context. + inline DimensionHandle MakeDim(DimensionOrConstant d) { + return shape_manager_.MakeDim(d); + } + + inline DimensionHandle UnknownDim() { return MakeDim(kUnknownDim); } + + // Returns in a scalar value from an input tensor . The input tensor + // must be a 1-dimensional int32 or int64 tensor. Caller must ensure that the + // input tensor is not NULL. + Status GetScalarFromTensor(const Tensor* t, int64* val); + + // Returns a new dimension whose value is given by a scalar input tensor. + // The input tensor must be in host memory, since it is dereferenced to get + // the value. + Status MakeDimForScalarInput(int idx, DimensionHandle* out); + + // Returns a new dimension whose value is given by a scalar input tensor. + // This allows for a negative input dimension given the rank of a separate + // tensor. This rank can be negative if unknown. + // The input tensor must be in host memory, since it is dereferenced to get + // the value. + Status MakeDimForScalarInputWithNegativeIndexing(int idx, int input_rank, + DimensionHandle* out); + + // Look up the attr for the NodeDef being evaluated with name attr_name and + // set *value to its value. If no attr with attr_name is found in def(), or + // the attr does not have a matching type, a non-ok status will be returned. + template + Status GetAttr(StringPiece attr_name, T* value) const; + + // Returns in the result of dividing by . + // Returns an error if is not positive or if + // and does not evenly divide . + Status Divide(DimensionHandle dividend, DimensionOrConstant divisor, + bool evenly_divisible, DimensionHandle* out); + + // Returns in the sum of and . + Status Add(DimensionHandle first, DimensionOrConstant second, + DimensionHandle* out); + + // Returns in the dimension that is minus . + Status Subtract(DimensionHandle first, DimensionOrConstant second, + DimensionHandle* out); + + // Returns in the product of and . + Status Multiply(DimensionHandle first, DimensionOrConstant second, + DimensionHandle* out); + + // Returns in the minimum of and . If either or + // is zero the results is zero. Otherwise, if either or + // is unknown the results is unknown. + Status Min(DimensionHandle first, DimensionOrConstant second, + DimensionHandle* out); + + // Returns in the maximum of and . If either or + // is unknown the results is unknown. + Status Max(DimensionHandle first, DimensionOrConstant second, + DimensionHandle* out); + + Status construction_status() const { return construction_status_; } + + // Methods to propagate shape and dtype on edges of handles. Handles are the + // dtype DT_RESOURCE which can be used to access state stored in a + // ResourceManager. When ops (such as variables) consume these handles to + // produce tensors they might need to know side-information about the shapes + // and dtypes of tensors which can be accessed via the handle. These methods + // propagate that information. Output handle dtypes and shapes are ignored if + // the output tensor is not of type DT_RESOURCE. + + // Merge the stored shapes and types corresponding to the input handle in + // position idx with the specified shapes and types. This requires idx to be + // in the [0, num_inputs) range. + // + // If the merge is successful and any of the new shapes differs from the old + // one, or any of the old dtypes was DT_INVALID, store the new shapes and + // return true. Return false otherwise. + // + // See 'MergeInput' function for full details and examples. + bool MergeInputHandleShapesAndTypes( + int idx, + const std::vector& shapes_and_types) TF_MUST_USE_RESULT; + + // As MergeInputHandleShapesAndTypes, but for an output. + bool MergeOutputHandleShapesAndTypes( + int idx, + const std::vector& shapes_and_types) TF_MUST_USE_RESULT; + + // Relaxes the stored shapes and types corresponding to the input handle in + // position idx with the specified shapes and types. This requires idx to be + // in the [0, num_inputs) range. + // + // If the relax is successful and any of the new shapes differs from the old + // one, or any of the old dtypes was DT_INVALID, store the new shapes and + // return true. Return false otherwise. + // + // See 'RelaxInput' function for full details and examples. + bool RelaxInputHandleShapesAndMergeTypes( + int idx, + const std::vector& shapes_and_types) TF_MUST_USE_RESULT; + + // As RelaxInputHandleShapesAndTypes, but for an output. + bool RelaxOutputHandleShapesAndMergeTypes( + int idx, + const std::vector& shapes_and_types) TF_MUST_USE_RESULT; + + void set_input_handle_shapes_and_types( + int idx, const std::vector& shapes_and_types) { + input_handle_shapes_and_types_[idx].reset( + new std::vector(shapes_and_types)); + } + + // Returns the output handle shapes and types, for the resource tensor output + // at index . Returns NULL if the shape and types were never set. + const std::vector* output_handle_shapes_and_types(int idx) { + return output_handle_shapes_and_types_[idx].get(); + } + + // Returns the inputs handle shapes and types, for the resource tensor output + // at index . Returns NULL if the shape and types were not available. + const std::vector* input_handle_shapes_and_types(int idx) { + return input_handle_shapes_and_types_[idx].get(); + } + + void set_output_handle_shapes_and_types( + int idx, const std::vector& shapes_and_types) { + output_handle_shapes_and_types_[idx].reset( + new std::vector(shapes_and_types)); + } + + // Note that shape functions should usually call MakeShapeFromShapeTensor, + // as it does more analysis to provide partial shapes. + // + // Returns in a new shape whose dimension sizes come from tensor . + // The tensor must be a 1-dimensional int32 or int64 tensor. If is NULL, + // then an unknown shape is returned. + Status MakeShapeFromTensor(const Tensor* t, ShapeHandle tensor_shape, + ShapeHandle* out); + + int graph_def_version() const { return graph_def_version_; } + + const std::vector>& MergedShapes() const { + return merged_shapes_; + } + const std::vector>& MergedDims() + const { + return merged_dims_; + } + + private: + // Creates and stores shapes for use in InferenceContext. + class ShapeManager { + public: + ShapeManager(); + ~ShapeManager(); + + // Returns a new shape with the given dims. The returned value is owned by + // this class. + ShapeHandle MakeShape(const std::vector& dims); + + // Returns a new unknown shape. + ShapeHandle UnknownShape(); + + // Returns a new dimension of the given size. The returned value + // is owned by this class. + inline DimensionHandle MakeDim(DimensionOrConstant d) { + if (d.dim.IsSet()) { + return d.dim; + } else { + all_dims_.push_back(new Dimension(d.val)); + return all_dims_.back(); + } + } + + private: + std::vector all_shapes_; // values are owned. + std::vector all_dims_; // values are owned. + }; + + friend class ::tensorflow::grappler::GraphProperties; + + // Friend for user-defined function shape inference purposes. + friend class ::tensorflow::ShapeRefiner; + + friend class ShapeInferenceTest; // For testing Relax functions. + friend class ShapeInferenceTestutil; // For testing shapes. + + // Shared initialization across the two constructors. Remove + // once we get rid of one of them. + void PreInputInit(const OpDef& op_def, + const std::vector& input_tensors, + const std::vector& input_tensors_as_shapes); + void PostInputInit(std::vector>> + input_handle_data); + + DimensionHandle GetDimension(const DimensionOrConstant& d); + + Status ReturnUnknownShape(ShapeHandle* out) { + *out = UnknownShape(); + return Status::OK(); + } + Status ReturnCreatedShape(const std::vector& dims, + ShapeHandle* out) { + *out = MakeShape(dims); + return Status::OK(); + } + + // Adds additional context to the given status. + Status AttachContext(const Status& status); + + // Relaxes an existing value with a new value and returns the + // relaxed dimension in <*out>. If and have incompatible + // values, returns an error. + // + // Note that <*out> may be set to or . + void Relax(DimensionHandle d_old, DimensionHandle d_new, + DimensionHandle* out); + // Relaxes an existing shape with a new shape and returns the + // relaxed shape in <*out>. See 'RelaxInput' function for full details and + // examples. + void Relax(ShapeHandle s_old, ShapeHandle s_new, ShapeHandle* out); + + // Used to implement MergeInputHandleShapesAndTypes and + // MergeOutputHandleShapesAndTypes. + bool MergeHandleShapesAndTypes( + const std::vector& shapes_and_types, + std::vector* to_update) TF_MUST_USE_RESULT; + // Used to implement RelaxInputHandleShapesAndMergeTypes and + // RelaxOutputHandleShapesAndMergeTypes. + bool RelaxHandleShapesAndMergeTypes( + const std::vector& shapes_and_types, + std::vector* to_update) TF_MUST_USE_RESULT; + + // Forget all the previous merged shapes and dims. + void ForgetMerges() { + merged_shapes_.clear(); + merged_dims_.clear(); + } + + // Helper method for MakeShapeFromTensor and MakeShapeFromShapeTensor. + Status InternalMakeShapeFromTensor( + bool treat_unknown_scalar_tensor_as_unknown_shape, const Tensor* t, + ShapeHandle tensor_shape, ShapeHandle* out); + + ShapeManager shape_manager_; + + // inputs_, outputs_, and input_tensors_as_shapes_ refer to values from + // `shape_manager_`. + std::vector inputs_; + std::vector input_tensors_; + std::vector requested_input_tensor_; + std::vector outputs_; + // Can have fewer elements than inputs_. + std::vector input_tensors_as_shapes_; + std::vector requested_input_tensor_as_partial_shape_; + + // input_handle_shapes_and_types_[i] is the list of shape/type pairs available + // through the resource handle passed along input i of the node. + // + // Values may be NULL. + std::vector>> + input_handle_shapes_and_types_; + + // output_handle_shapes_and_types_[i] is the list of shape/type pairs + // available through the resource handle passed along output i of the node. + // + // Values may be NULL. + std::vector>> + output_handle_shapes_and_types_; + + const int graph_def_version_; + const NodeDef* node_def_; + NameRangeMap input_name_map_; + NameRangeMap output_name_map_; + + // An error set during construction. TODO(cwhipkey): remove when test + // constructor is removed. + Status construction_status_; + + // Pair of shape or dim handles that are equivalent, ie that represent the + // same underlying shape of dimension. Note that for each pair at least one of + // the handles must contain an unknown shape, since we don't keep track of + // known shapes or dims here. + std::vector> merged_shapes_; + std::vector> merged_dims_; + + TF_DISALLOW_COPY_AND_ASSIGN(InferenceContext); +}; + +// ----------------------------------------------------------------------------- +// Template and inline method implementations, please ignore + +inline Dimension::Dimension() : value_(InferenceContext::kUnknownDim) {} +inline Dimension::Dimension(int64 value) : value_(value) { + DCHECK(value >= 0 || value == InferenceContext::kUnknownDim) + << "Dimension must be non-negative or equal to " + "InferenceContext::kUnknownDim but got " + << value; +} + +inline Shape::Shape() : rank_(InferenceContext::kUnknownRank) {} +inline Shape::Shape(const std::vector& dims) + : rank_(dims.size()), dims_(dims) {} + +inline DimensionOrConstant::DimensionOrConstant(DimensionHandle dim) + : dim(dim) { + DCHECK(dim.IsSet()) << "Internal error: Got nullptr for Dimension."; +} + +inline DimensionOrConstant::DimensionOrConstant(int64 val) : val(val) { + DCHECK(val >= 0 || val == InferenceContext::kUnknownDim) + << "Dimension must be non-negative or equal to " + "InferenceContext::kUnknownDim but got " + << val; +} + +template +Status InferenceContext::GetAttr(StringPiece attr_name, T* value) const { + return GetNodeAttr(*node_def_, attr_name, value); +} + +} // namespace shape_inference +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_FRAMEWORK_SHAPE_INFERENCE_H_ diff --git a/linux/include/tensorflow/core/framework/stats_aggregator.h b/linux/include/tensorflow/core/framework/stats_aggregator.h new file mode 100644 index 000000000..af53ed0a3 --- /dev/null +++ b/linux/include/tensorflow/core/framework/stats_aggregator.h @@ -0,0 +1,95 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CORE_FRAMEWORK_STATS_AGGREGATOR_H_ +#define TENSORFLOW_CORE_FRAMEWORK_STATS_AGGREGATOR_H_ + +#include +#include + +#include "tensorflow/core/framework/resource_mgr.h" +#include "tensorflow/core/lib/gtl/array_slice.h" + +namespace tensorflow { + +class Summary; + +namespace data { + +// A `StatsAggregator` accumulates statistics incrementally. A +// `StatsAggregator` can accumulate multiple different statistics, distinguished +// by a string name. +// +// The class currently supports accumulating `Histogram` objects, and we expect +// to add other methods in future. +// +// NOTE(mrry): `StatsAggregator` is a virtual interface because we anticipate +// that many different implementations will the same interface. For example, the +// current implementation in "stats_aggregator_ops.cc" is a simple in-memory +// implementation that integrates with the pull-based summary API, and we may +// add implementations that work with the push-based `SummaryWriterInterface`, +// as well as custom monitoring services. +class StatsAggregator { + public: + virtual ~StatsAggregator() {} + + // Add the given `values` to the histogram with the given `name`. Each + // element of `values` will be treated as a separate sample in the histogram. + virtual void AddToHistogram(const string& name, + gtl::ArraySlice values) = 0; + + // TODO(shivaniagarawal): consistency in double and float usage. + // Add the given `value` as Scalar with the given `name`. + virtual void AddScalar(const string& name, float value) = 0; + + // Stores a protocol buffer representation of the aggregator state in the + // given `out_summary`. + // TODO(mrry): Consider separating this method from the `StatsAggregator` + // interface. It is possible that not all implementations will support + // encoding their state as a protocol buffer. + virtual void EncodeToProto(Summary* out_summary) = 0; + + // Increment the `label` cell of metrics mapped with `name` by given `value`. + virtual void IncrementCounter(const string& name, const string& label, + int64 val) = 0; +}; + +// A `StatsAggregatorResource` wraps a shareable `StatsAggregator` as a resource +// in the TensorFlow resource manager. +// +// NOTE(mrry): This class is separate from `StatsAggregator` in order to +// simplify the memory management of the shared object. Most users of +// `StatsAggregator` interact with a `std::shared_ptr` whereas +// the `ResourceBase` API requires explicit reference counting. +class StatsAggregatorResource : public ResourceBase { + public: + // Creates a new resource from the given `stats_aggregator`. + StatsAggregatorResource(std::unique_ptr stats_aggregator) + : stats_aggregator_(stats_aggregator.release()) {} + + // Returns the wrapped `StatsAggregator`. + std::shared_ptr stats_aggregator() const { + return stats_aggregator_; + } + + string DebugString() { return "StatsAggregatorResource"; } + + private: + const std::shared_ptr stats_aggregator_; +}; + +} // namespace data +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_FRAMEWORK_STATS_AGGREGATOR_H_ diff --git a/linux/include/tensorflow/core/framework/step_stats.pb.h b/linux/include/tensorflow/core/framework/step_stats.pb.h new file mode 100644 index 000000000..974582672 --- /dev/null +++ b/linux/include/tensorflow/core/framework/step_stats.pb.h @@ -0,0 +1,2277 @@ +// Generated by the protocol buffer compiler. DO NOT EDIT! +// source: tensorflow/core/framework/step_stats.proto + +#ifndef PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2fstep_5fstats_2eproto +#define PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2fstep_5fstats_2eproto + +#include + +#include + +#if GOOGLE_PROTOBUF_VERSION < 3006000 +#error This file was generated by a newer version of protoc which is +#error incompatible with your Protocol Buffer headers. Please update +#error your headers. +#endif +#if 3006000 < GOOGLE_PROTOBUF_MIN_PROTOC_VERSION +#error This file was generated by an older version of protoc which is +#error incompatible with your Protocol Buffer headers. Please +#error regenerate this file with a newer version of protoc. +#endif + +#include +#include +#include +#include +#include +#include +#include +#include +#include // IWYU pragma: export +#include // IWYU pragma: export +#include +#include "tensorflow/core/framework/allocation_description.pb.h" +#include "tensorflow/core/framework/tensor_description.pb.h" +// @@protoc_insertion_point(includes) +#define PROTOBUF_INTERNAL_EXPORT_protobuf_tensorflow_2fcore_2fframework_2fstep_5fstats_2eproto + +namespace protobuf_tensorflow_2fcore_2fframework_2fstep_5fstats_2eproto { +// Internal implementation detail -- do not use these members. +struct TableStruct { + static const ::google::protobuf::internal::ParseTableField entries[]; + static const ::google::protobuf::internal::AuxillaryParseTableField aux[]; + static const ::google::protobuf::internal::ParseTable schema[7]; + static const ::google::protobuf::internal::FieldMetadata field_metadata[]; + static const ::google::protobuf::internal::SerializationTable serialization_table[]; + static const ::google::protobuf::uint32 offsets[]; +}; +void AddDescriptors(); +} // namespace protobuf_tensorflow_2fcore_2fframework_2fstep_5fstats_2eproto +namespace tensorflow { +class AllocationRecord; +class AllocationRecordDefaultTypeInternal; +extern AllocationRecordDefaultTypeInternal _AllocationRecord_default_instance_; +class AllocatorMemoryUsed; +class AllocatorMemoryUsedDefaultTypeInternal; +extern AllocatorMemoryUsedDefaultTypeInternal _AllocatorMemoryUsed_default_instance_; +class DeviceStepStats; +class DeviceStepStatsDefaultTypeInternal; +extern DeviceStepStatsDefaultTypeInternal _DeviceStepStats_default_instance_; +class MemoryStats; +class MemoryStatsDefaultTypeInternal; +extern MemoryStatsDefaultTypeInternal _MemoryStats_default_instance_; +class NodeExecStats; +class NodeExecStatsDefaultTypeInternal; +extern NodeExecStatsDefaultTypeInternal _NodeExecStats_default_instance_; +class NodeOutput; +class NodeOutputDefaultTypeInternal; +extern NodeOutputDefaultTypeInternal _NodeOutput_default_instance_; +class StepStats; +class StepStatsDefaultTypeInternal; +extern StepStatsDefaultTypeInternal _StepStats_default_instance_; +} // namespace tensorflow +namespace google { +namespace protobuf { +template<> ::tensorflow::AllocationRecord* Arena::CreateMaybeMessage<::tensorflow::AllocationRecord>(Arena*); +template<> ::tensorflow::AllocatorMemoryUsed* Arena::CreateMaybeMessage<::tensorflow::AllocatorMemoryUsed>(Arena*); +template<> ::tensorflow::DeviceStepStats* Arena::CreateMaybeMessage<::tensorflow::DeviceStepStats>(Arena*); +template<> ::tensorflow::MemoryStats* Arena::CreateMaybeMessage<::tensorflow::MemoryStats>(Arena*); +template<> ::tensorflow::NodeExecStats* Arena::CreateMaybeMessage<::tensorflow::NodeExecStats>(Arena*); +template<> ::tensorflow::NodeOutput* Arena::CreateMaybeMessage<::tensorflow::NodeOutput>(Arena*); +template<> ::tensorflow::StepStats* Arena::CreateMaybeMessage<::tensorflow::StepStats>(Arena*); +} // namespace protobuf +} // namespace google +namespace tensorflow { + +// =================================================================== + +class AllocationRecord : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.AllocationRecord) */ { + public: + AllocationRecord(); + virtual ~AllocationRecord(); + + AllocationRecord(const AllocationRecord& from); + + inline AllocationRecord& operator=(const AllocationRecord& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + AllocationRecord(AllocationRecord&& from) noexcept + : AllocationRecord() { + *this = ::std::move(from); + } + + inline AllocationRecord& operator=(AllocationRecord&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const AllocationRecord& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const AllocationRecord* internal_default_instance() { + return reinterpret_cast( + &_AllocationRecord_default_instance_); + } + static constexpr int kIndexInFileMessages = + 0; + + void UnsafeArenaSwap(AllocationRecord* other); + void Swap(AllocationRecord* other); + friend void swap(AllocationRecord& a, AllocationRecord& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline AllocationRecord* New() const final { + return CreateMaybeMessage(NULL); + } + + AllocationRecord* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const AllocationRecord& from); + void MergeFrom(const AllocationRecord& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(AllocationRecord* other); + protected: + explicit AllocationRecord(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // int64 alloc_micros = 1; + void clear_alloc_micros(); + static const int kAllocMicrosFieldNumber = 1; + ::google::protobuf::int64 alloc_micros() const; + void set_alloc_micros(::google::protobuf::int64 value); + + // int64 alloc_bytes = 2; + void clear_alloc_bytes(); + static const int kAllocBytesFieldNumber = 2; + ::google::protobuf::int64 alloc_bytes() const; + void set_alloc_bytes(::google::protobuf::int64 value); + + // @@protoc_insertion_point(class_scope:tensorflow.AllocationRecord) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::int64 alloc_micros_; + ::google::protobuf::int64 alloc_bytes_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fstep_5fstats_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class AllocatorMemoryUsed : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.AllocatorMemoryUsed) */ { + public: + AllocatorMemoryUsed(); + virtual ~AllocatorMemoryUsed(); + + AllocatorMemoryUsed(const AllocatorMemoryUsed& from); + + inline AllocatorMemoryUsed& operator=(const AllocatorMemoryUsed& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + AllocatorMemoryUsed(AllocatorMemoryUsed&& from) noexcept + : AllocatorMemoryUsed() { + *this = ::std::move(from); + } + + inline AllocatorMemoryUsed& operator=(AllocatorMemoryUsed&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const AllocatorMemoryUsed& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const AllocatorMemoryUsed* internal_default_instance() { + return reinterpret_cast( + &_AllocatorMemoryUsed_default_instance_); + } + static constexpr int kIndexInFileMessages = + 1; + + void UnsafeArenaSwap(AllocatorMemoryUsed* other); + void Swap(AllocatorMemoryUsed* other); + friend void swap(AllocatorMemoryUsed& a, AllocatorMemoryUsed& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline AllocatorMemoryUsed* New() const final { + return CreateMaybeMessage(NULL); + } + + AllocatorMemoryUsed* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const AllocatorMemoryUsed& from); + void MergeFrom(const AllocatorMemoryUsed& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(AllocatorMemoryUsed* other); + protected: + explicit AllocatorMemoryUsed(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated .tensorflow.AllocationRecord allocation_records = 6; + int allocation_records_size() const; + void clear_allocation_records(); + static const int kAllocationRecordsFieldNumber = 6; + ::tensorflow::AllocationRecord* mutable_allocation_records(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::AllocationRecord >* + mutable_allocation_records(); + const ::tensorflow::AllocationRecord& allocation_records(int index) const; + ::tensorflow::AllocationRecord* add_allocation_records(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::AllocationRecord >& + allocation_records() const; + + // string allocator_name = 1; + void clear_allocator_name(); + static const int kAllocatorNameFieldNumber = 1; + const ::std::string& allocator_name() const; + void set_allocator_name(const ::std::string& value); + #if LANG_CXX11 + void set_allocator_name(::std::string&& value); + #endif + void set_allocator_name(const char* value); + void set_allocator_name(const char* value, size_t size); + ::std::string* mutable_allocator_name(); + ::std::string* release_allocator_name(); + void set_allocated_allocator_name(::std::string* allocator_name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_allocator_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_allocator_name( + ::std::string* allocator_name); + + // int64 total_bytes = 2; + void clear_total_bytes(); + static const int kTotalBytesFieldNumber = 2; + ::google::protobuf::int64 total_bytes() const; + void set_total_bytes(::google::protobuf::int64 value); + + // int64 peak_bytes = 3; + void clear_peak_bytes(); + static const int kPeakBytesFieldNumber = 3; + ::google::protobuf::int64 peak_bytes() const; + void set_peak_bytes(::google::protobuf::int64 value); + + // int64 live_bytes = 4; + void clear_live_bytes(); + static const int kLiveBytesFieldNumber = 4; + ::google::protobuf::int64 live_bytes() const; + void set_live_bytes(::google::protobuf::int64 value); + + // int64 allocator_bytes_in_use = 5; + void clear_allocator_bytes_in_use(); + static const int kAllocatorBytesInUseFieldNumber = 5; + ::google::protobuf::int64 allocator_bytes_in_use() const; + void set_allocator_bytes_in_use(::google::protobuf::int64 value); + + // @@protoc_insertion_point(class_scope:tensorflow.AllocatorMemoryUsed) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::AllocationRecord > allocation_records_; + ::google::protobuf::internal::ArenaStringPtr allocator_name_; + ::google::protobuf::int64 total_bytes_; + ::google::protobuf::int64 peak_bytes_; + ::google::protobuf::int64 live_bytes_; + ::google::protobuf::int64 allocator_bytes_in_use_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fstep_5fstats_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class NodeOutput : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.NodeOutput) */ { + public: + NodeOutput(); + virtual ~NodeOutput(); + + NodeOutput(const NodeOutput& from); + + inline NodeOutput& operator=(const NodeOutput& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + NodeOutput(NodeOutput&& from) noexcept + : NodeOutput() { + *this = ::std::move(from); + } + + inline NodeOutput& operator=(NodeOutput&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const NodeOutput& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const NodeOutput* internal_default_instance() { + return reinterpret_cast( + &_NodeOutput_default_instance_); + } + static constexpr int kIndexInFileMessages = + 2; + + void UnsafeArenaSwap(NodeOutput* other); + void Swap(NodeOutput* other); + friend void swap(NodeOutput& a, NodeOutput& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline NodeOutput* New() const final { + return CreateMaybeMessage(NULL); + } + + NodeOutput* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const NodeOutput& from); + void MergeFrom(const NodeOutput& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(NodeOutput* other); + protected: + explicit NodeOutput(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // .tensorflow.TensorDescription tensor_description = 3; + bool has_tensor_description() const; + void clear_tensor_description(); + static const int kTensorDescriptionFieldNumber = 3; + private: + const ::tensorflow::TensorDescription& _internal_tensor_description() const; + public: + const ::tensorflow::TensorDescription& tensor_description() const; + ::tensorflow::TensorDescription* release_tensor_description(); + ::tensorflow::TensorDescription* mutable_tensor_description(); + void set_allocated_tensor_description(::tensorflow::TensorDescription* tensor_description); + void unsafe_arena_set_allocated_tensor_description( + ::tensorflow::TensorDescription* tensor_description); + ::tensorflow::TensorDescription* unsafe_arena_release_tensor_description(); + + // int32 slot = 1; + void clear_slot(); + static const int kSlotFieldNumber = 1; + ::google::protobuf::int32 slot() const; + void set_slot(::google::protobuf::int32 value); + + // @@protoc_insertion_point(class_scope:tensorflow.NodeOutput) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::tensorflow::TensorDescription* tensor_description_; + ::google::protobuf::int32 slot_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fstep_5fstats_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class MemoryStats : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.MemoryStats) */ { + public: + MemoryStats(); + virtual ~MemoryStats(); + + MemoryStats(const MemoryStats& from); + + inline MemoryStats& operator=(const MemoryStats& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + MemoryStats(MemoryStats&& from) noexcept + : MemoryStats() { + *this = ::std::move(from); + } + + inline MemoryStats& operator=(MemoryStats&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const MemoryStats& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const MemoryStats* internal_default_instance() { + return reinterpret_cast( + &_MemoryStats_default_instance_); + } + static constexpr int kIndexInFileMessages = + 3; + + void UnsafeArenaSwap(MemoryStats* other); + void Swap(MemoryStats* other); + friend void swap(MemoryStats& a, MemoryStats& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline MemoryStats* New() const final { + return CreateMaybeMessage(NULL); + } + + MemoryStats* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const MemoryStats& from); + void MergeFrom(const MemoryStats& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(MemoryStats* other); + protected: + explicit MemoryStats(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated int64 persistent_tensor_alloc_ids = 5; + int persistent_tensor_alloc_ids_size() const; + void clear_persistent_tensor_alloc_ids(); + static const int kPersistentTensorAllocIdsFieldNumber = 5; + ::google::protobuf::int64 persistent_tensor_alloc_ids(int index) const; + void set_persistent_tensor_alloc_ids(int index, ::google::protobuf::int64 value); + void add_persistent_tensor_alloc_ids(::google::protobuf::int64 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& + persistent_tensor_alloc_ids() const; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* + mutable_persistent_tensor_alloc_ids(); + + // repeated int64 device_persistent_tensor_alloc_ids = 6 [deprecated = true]; + GOOGLE_PROTOBUF_DEPRECATED_ATTR int device_persistent_tensor_alloc_ids_size() const; + GOOGLE_PROTOBUF_DEPRECATED_ATTR void clear_device_persistent_tensor_alloc_ids(); + GOOGLE_PROTOBUF_DEPRECATED_ATTR static const int kDevicePersistentTensorAllocIdsFieldNumber = 6; + GOOGLE_PROTOBUF_DEPRECATED_ATTR ::google::protobuf::int64 device_persistent_tensor_alloc_ids(int index) const; + GOOGLE_PROTOBUF_DEPRECATED_ATTR void set_device_persistent_tensor_alloc_ids(int index, ::google::protobuf::int64 value); + GOOGLE_PROTOBUF_DEPRECATED_ATTR void add_device_persistent_tensor_alloc_ids(::google::protobuf::int64 value); + GOOGLE_PROTOBUF_DEPRECATED_ATTR const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& + device_persistent_tensor_alloc_ids() const; + GOOGLE_PROTOBUF_DEPRECATED_ATTR ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* + mutable_device_persistent_tensor_alloc_ids(); + + // int64 temp_memory_size = 1; + void clear_temp_memory_size(); + static const int kTempMemorySizeFieldNumber = 1; + ::google::protobuf::int64 temp_memory_size() const; + void set_temp_memory_size(::google::protobuf::int64 value); + + // int64 device_temp_memory_size = 2 [deprecated = true]; + GOOGLE_PROTOBUF_DEPRECATED_ATTR void clear_device_temp_memory_size(); + GOOGLE_PROTOBUF_DEPRECATED_ATTR static const int kDeviceTempMemorySizeFieldNumber = 2; + GOOGLE_PROTOBUF_DEPRECATED_ATTR ::google::protobuf::int64 device_temp_memory_size() const; + GOOGLE_PROTOBUF_DEPRECATED_ATTR void set_device_temp_memory_size(::google::protobuf::int64 value); + + // int64 persistent_memory_size = 3; + void clear_persistent_memory_size(); + static const int kPersistentMemorySizeFieldNumber = 3; + ::google::protobuf::int64 persistent_memory_size() const; + void set_persistent_memory_size(::google::protobuf::int64 value); + + // int64 device_persistent_memory_size = 4 [deprecated = true]; + GOOGLE_PROTOBUF_DEPRECATED_ATTR void clear_device_persistent_memory_size(); + GOOGLE_PROTOBUF_DEPRECATED_ATTR static const int kDevicePersistentMemorySizeFieldNumber = 4; + GOOGLE_PROTOBUF_DEPRECATED_ATTR ::google::protobuf::int64 device_persistent_memory_size() const; + GOOGLE_PROTOBUF_DEPRECATED_ATTR void set_device_persistent_memory_size(::google::protobuf::int64 value); + + // @@protoc_insertion_point(class_scope:tensorflow.MemoryStats) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 > persistent_tensor_alloc_ids_; + mutable int _persistent_tensor_alloc_ids_cached_byte_size_; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 > device_persistent_tensor_alloc_ids_; + mutable int _device_persistent_tensor_alloc_ids_cached_byte_size_; + ::google::protobuf::int64 temp_memory_size_; + ::google::protobuf::int64 device_temp_memory_size_; + ::google::protobuf::int64 persistent_memory_size_; + ::google::protobuf::int64 device_persistent_memory_size_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fstep_5fstats_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class NodeExecStats : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.NodeExecStats) */ { + public: + NodeExecStats(); + virtual ~NodeExecStats(); + + NodeExecStats(const NodeExecStats& from); + + inline NodeExecStats& operator=(const NodeExecStats& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + NodeExecStats(NodeExecStats&& from) noexcept + : NodeExecStats() { + *this = ::std::move(from); + } + + inline NodeExecStats& operator=(NodeExecStats&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const NodeExecStats& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const NodeExecStats* internal_default_instance() { + return reinterpret_cast( + &_NodeExecStats_default_instance_); + } + static constexpr int kIndexInFileMessages = + 4; + + void UnsafeArenaSwap(NodeExecStats* other); + void Swap(NodeExecStats* other); + friend void swap(NodeExecStats& a, NodeExecStats& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline NodeExecStats* New() const final { + return CreateMaybeMessage(NULL); + } + + NodeExecStats* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const NodeExecStats& from); + void MergeFrom(const NodeExecStats& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(NodeExecStats* other); + protected: + explicit NodeExecStats(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated .tensorflow.AllocatorMemoryUsed memory = 6; + int memory_size() const; + void clear_memory(); + static const int kMemoryFieldNumber = 6; + ::tensorflow::AllocatorMemoryUsed* mutable_memory(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::AllocatorMemoryUsed >* + mutable_memory(); + const ::tensorflow::AllocatorMemoryUsed& memory(int index) const; + ::tensorflow::AllocatorMemoryUsed* add_memory(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::AllocatorMemoryUsed >& + memory() const; + + // repeated .tensorflow.NodeOutput output = 7; + int output_size() const; + void clear_output(); + static const int kOutputFieldNumber = 7; + ::tensorflow::NodeOutput* mutable_output(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::NodeOutput >* + mutable_output(); + const ::tensorflow::NodeOutput& output(int index) const; + ::tensorflow::NodeOutput* add_output(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::NodeOutput >& + output() const; + + // repeated .tensorflow.AllocationDescription referenced_tensor = 11; + int referenced_tensor_size() const; + void clear_referenced_tensor(); + static const int kReferencedTensorFieldNumber = 11; + ::tensorflow::AllocationDescription* mutable_referenced_tensor(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::AllocationDescription >* + mutable_referenced_tensor(); + const ::tensorflow::AllocationDescription& referenced_tensor(int index) const; + ::tensorflow::AllocationDescription* add_referenced_tensor(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::AllocationDescription >& + referenced_tensor() const; + + // string node_name = 1; + void clear_node_name(); + static const int kNodeNameFieldNumber = 1; + const ::std::string& node_name() const; + void set_node_name(const ::std::string& value); + #if LANG_CXX11 + void set_node_name(::std::string&& value); + #endif + void set_node_name(const char* value); + void set_node_name(const char* value, size_t size); + ::std::string* mutable_node_name(); + ::std::string* release_node_name(); + void set_allocated_node_name(::std::string* node_name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_node_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_node_name( + ::std::string* node_name); + + // string timeline_label = 8; + void clear_timeline_label(); + static const int kTimelineLabelFieldNumber = 8; + const ::std::string& timeline_label() const; + void set_timeline_label(const ::std::string& value); + #if LANG_CXX11 + void set_timeline_label(::std::string&& value); + #endif + void set_timeline_label(const char* value); + void set_timeline_label(const char* value, size_t size); + ::std::string* mutable_timeline_label(); + ::std::string* release_timeline_label(); + void set_allocated_timeline_label(::std::string* timeline_label); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_timeline_label(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_timeline_label( + ::std::string* timeline_label); + + // .tensorflow.MemoryStats memory_stats = 12; + bool has_memory_stats() const; + void clear_memory_stats(); + static const int kMemoryStatsFieldNumber = 12; + private: + const ::tensorflow::MemoryStats& _internal_memory_stats() const; + public: + const ::tensorflow::MemoryStats& memory_stats() const; + ::tensorflow::MemoryStats* release_memory_stats(); + ::tensorflow::MemoryStats* mutable_memory_stats(); + void set_allocated_memory_stats(::tensorflow::MemoryStats* memory_stats); + void unsafe_arena_set_allocated_memory_stats( + ::tensorflow::MemoryStats* memory_stats); + ::tensorflow::MemoryStats* unsafe_arena_release_memory_stats(); + + // int64 all_start_micros = 2; + void clear_all_start_micros(); + static const int kAllStartMicrosFieldNumber = 2; + ::google::protobuf::int64 all_start_micros() const; + void set_all_start_micros(::google::protobuf::int64 value); + + // int64 op_start_rel_micros = 3; + void clear_op_start_rel_micros(); + static const int kOpStartRelMicrosFieldNumber = 3; + ::google::protobuf::int64 op_start_rel_micros() const; + void set_op_start_rel_micros(::google::protobuf::int64 value); + + // int64 op_end_rel_micros = 4; + void clear_op_end_rel_micros(); + static const int kOpEndRelMicrosFieldNumber = 4; + ::google::protobuf::int64 op_end_rel_micros() const; + void set_op_end_rel_micros(::google::protobuf::int64 value); + + // int64 all_end_rel_micros = 5; + void clear_all_end_rel_micros(); + static const int kAllEndRelMicrosFieldNumber = 5; + ::google::protobuf::int64 all_end_rel_micros() const; + void set_all_end_rel_micros(::google::protobuf::int64 value); + + // int64 scheduled_micros = 9; + void clear_scheduled_micros(); + static const int kScheduledMicrosFieldNumber = 9; + ::google::protobuf::int64 scheduled_micros() const; + void set_scheduled_micros(::google::protobuf::int64 value); + + // int64 all_start_nanos = 13; + void clear_all_start_nanos(); + static const int kAllStartNanosFieldNumber = 13; + ::google::protobuf::int64 all_start_nanos() const; + void set_all_start_nanos(::google::protobuf::int64 value); + + // int64 op_start_rel_nanos = 14; + void clear_op_start_rel_nanos(); + static const int kOpStartRelNanosFieldNumber = 14; + ::google::protobuf::int64 op_start_rel_nanos() const; + void set_op_start_rel_nanos(::google::protobuf::int64 value); + + // int64 op_end_rel_nanos = 15; + void clear_op_end_rel_nanos(); + static const int kOpEndRelNanosFieldNumber = 15; + ::google::protobuf::int64 op_end_rel_nanos() const; + void set_op_end_rel_nanos(::google::protobuf::int64 value); + + // int64 all_end_rel_nanos = 16; + void clear_all_end_rel_nanos(); + static const int kAllEndRelNanosFieldNumber = 16; + ::google::protobuf::int64 all_end_rel_nanos() const; + void set_all_end_rel_nanos(::google::protobuf::int64 value); + + // int64 scheduled_nanos = 17; + void clear_scheduled_nanos(); + static const int kScheduledNanosFieldNumber = 17; + ::google::protobuf::int64 scheduled_nanos() const; + void set_scheduled_nanos(::google::protobuf::int64 value); + + // uint32 thread_id = 10; + void clear_thread_id(); + static const int kThreadIdFieldNumber = 10; + ::google::protobuf::uint32 thread_id() const; + void set_thread_id(::google::protobuf::uint32 value); + + // @@protoc_insertion_point(class_scope:tensorflow.NodeExecStats) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::AllocatorMemoryUsed > memory_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::NodeOutput > output_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::AllocationDescription > referenced_tensor_; + ::google::protobuf::internal::ArenaStringPtr node_name_; + ::google::protobuf::internal::ArenaStringPtr timeline_label_; + ::tensorflow::MemoryStats* memory_stats_; + ::google::protobuf::int64 all_start_micros_; + ::google::protobuf::int64 op_start_rel_micros_; + ::google::protobuf::int64 op_end_rel_micros_; + ::google::protobuf::int64 all_end_rel_micros_; + ::google::protobuf::int64 scheduled_micros_; + ::google::protobuf::int64 all_start_nanos_; + ::google::protobuf::int64 op_start_rel_nanos_; + ::google::protobuf::int64 op_end_rel_nanos_; + ::google::protobuf::int64 all_end_rel_nanos_; + ::google::protobuf::int64 scheduled_nanos_; + ::google::protobuf::uint32 thread_id_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fstep_5fstats_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class DeviceStepStats : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.DeviceStepStats) */ { + public: + DeviceStepStats(); + virtual ~DeviceStepStats(); + + DeviceStepStats(const DeviceStepStats& from); + + inline DeviceStepStats& operator=(const DeviceStepStats& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + DeviceStepStats(DeviceStepStats&& from) noexcept + : DeviceStepStats() { + *this = ::std::move(from); + } + + inline DeviceStepStats& operator=(DeviceStepStats&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const DeviceStepStats& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const DeviceStepStats* internal_default_instance() { + return reinterpret_cast( + &_DeviceStepStats_default_instance_); + } + static constexpr int kIndexInFileMessages = + 5; + + void UnsafeArenaSwap(DeviceStepStats* other); + void Swap(DeviceStepStats* other); + friend void swap(DeviceStepStats& a, DeviceStepStats& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline DeviceStepStats* New() const final { + return CreateMaybeMessage(NULL); + } + + DeviceStepStats* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const DeviceStepStats& from); + void MergeFrom(const DeviceStepStats& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(DeviceStepStats* other); + protected: + explicit DeviceStepStats(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated .tensorflow.NodeExecStats node_stats = 2; + int node_stats_size() const; + void clear_node_stats(); + static const int kNodeStatsFieldNumber = 2; + ::tensorflow::NodeExecStats* mutable_node_stats(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::NodeExecStats >* + mutable_node_stats(); + const ::tensorflow::NodeExecStats& node_stats(int index) const; + ::tensorflow::NodeExecStats* add_node_stats(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::NodeExecStats >& + node_stats() const; + + // string device = 1; + void clear_device(); + static const int kDeviceFieldNumber = 1; + const ::std::string& device() const; + void set_device(const ::std::string& value); + #if LANG_CXX11 + void set_device(::std::string&& value); + #endif + void set_device(const char* value); + void set_device(const char* value, size_t size); + ::std::string* mutable_device(); + ::std::string* release_device(); + void set_allocated_device(::std::string* device); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_device(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_device( + ::std::string* device); + + // @@protoc_insertion_point(class_scope:tensorflow.DeviceStepStats) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::NodeExecStats > node_stats_; + ::google::protobuf::internal::ArenaStringPtr device_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fstep_5fstats_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class StepStats : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.StepStats) */ { + public: + StepStats(); + virtual ~StepStats(); + + StepStats(const StepStats& from); + + inline StepStats& operator=(const StepStats& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + StepStats(StepStats&& from) noexcept + : StepStats() { + *this = ::std::move(from); + } + + inline StepStats& operator=(StepStats&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const StepStats& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const StepStats* internal_default_instance() { + return reinterpret_cast( + &_StepStats_default_instance_); + } + static constexpr int kIndexInFileMessages = + 6; + + void UnsafeArenaSwap(StepStats* other); + void Swap(StepStats* other); + friend void swap(StepStats& a, StepStats& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline StepStats* New() const final { + return CreateMaybeMessage(NULL); + } + + StepStats* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const StepStats& from); + void MergeFrom(const StepStats& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(StepStats* other); + protected: + explicit StepStats(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated .tensorflow.DeviceStepStats dev_stats = 1; + int dev_stats_size() const; + void clear_dev_stats(); + static const int kDevStatsFieldNumber = 1; + ::tensorflow::DeviceStepStats* mutable_dev_stats(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::DeviceStepStats >* + mutable_dev_stats(); + const ::tensorflow::DeviceStepStats& dev_stats(int index) const; + ::tensorflow::DeviceStepStats* add_dev_stats(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::DeviceStepStats >& + dev_stats() const; + + // @@protoc_insertion_point(class_scope:tensorflow.StepStats) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::DeviceStepStats > dev_stats_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fstep_5fstats_2eproto::TableStruct; +}; +// =================================================================== + + +// =================================================================== + +#ifdef __GNUC__ + #pragma GCC diagnostic push + #pragma GCC diagnostic ignored "-Wstrict-aliasing" +#endif // __GNUC__ +// AllocationRecord + +// int64 alloc_micros = 1; +inline void AllocationRecord::clear_alloc_micros() { + alloc_micros_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 AllocationRecord::alloc_micros() const { + // @@protoc_insertion_point(field_get:tensorflow.AllocationRecord.alloc_micros) + return alloc_micros_; +} +inline void AllocationRecord::set_alloc_micros(::google::protobuf::int64 value) { + + alloc_micros_ = value; + // @@protoc_insertion_point(field_set:tensorflow.AllocationRecord.alloc_micros) +} + +// int64 alloc_bytes = 2; +inline void AllocationRecord::clear_alloc_bytes() { + alloc_bytes_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 AllocationRecord::alloc_bytes() const { + // @@protoc_insertion_point(field_get:tensorflow.AllocationRecord.alloc_bytes) + return alloc_bytes_; +} +inline void AllocationRecord::set_alloc_bytes(::google::protobuf::int64 value) { + + alloc_bytes_ = value; + // @@protoc_insertion_point(field_set:tensorflow.AllocationRecord.alloc_bytes) +} + +// ------------------------------------------------------------------- + +// AllocatorMemoryUsed + +// string allocator_name = 1; +inline void AllocatorMemoryUsed::clear_allocator_name() { + allocator_name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& AllocatorMemoryUsed::allocator_name() const { + // @@protoc_insertion_point(field_get:tensorflow.AllocatorMemoryUsed.allocator_name) + return allocator_name_.Get(); +} +inline void AllocatorMemoryUsed::set_allocator_name(const ::std::string& value) { + + allocator_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.AllocatorMemoryUsed.allocator_name) +} +#if LANG_CXX11 +inline void AllocatorMemoryUsed::set_allocator_name(::std::string&& value) { + + allocator_name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.AllocatorMemoryUsed.allocator_name) +} +#endif +inline void AllocatorMemoryUsed::set_allocator_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + allocator_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.AllocatorMemoryUsed.allocator_name) +} +inline void AllocatorMemoryUsed::set_allocator_name(const char* value, + size_t size) { + + allocator_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.AllocatorMemoryUsed.allocator_name) +} +inline ::std::string* AllocatorMemoryUsed::mutable_allocator_name() { + + // @@protoc_insertion_point(field_mutable:tensorflow.AllocatorMemoryUsed.allocator_name) + return allocator_name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* AllocatorMemoryUsed::release_allocator_name() { + // @@protoc_insertion_point(field_release:tensorflow.AllocatorMemoryUsed.allocator_name) + + return allocator_name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void AllocatorMemoryUsed::set_allocated_allocator_name(::std::string* allocator_name) { + if (allocator_name != NULL) { + + } else { + + } + allocator_name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), allocator_name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.AllocatorMemoryUsed.allocator_name) +} +inline ::std::string* AllocatorMemoryUsed::unsafe_arena_release_allocator_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.AllocatorMemoryUsed.allocator_name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return allocator_name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void AllocatorMemoryUsed::unsafe_arena_set_allocated_allocator_name( + ::std::string* allocator_name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (allocator_name != NULL) { + + } else { + + } + allocator_name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + allocator_name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.AllocatorMemoryUsed.allocator_name) +} + +// int64 total_bytes = 2; +inline void AllocatorMemoryUsed::clear_total_bytes() { + total_bytes_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 AllocatorMemoryUsed::total_bytes() const { + // @@protoc_insertion_point(field_get:tensorflow.AllocatorMemoryUsed.total_bytes) + return total_bytes_; +} +inline void AllocatorMemoryUsed::set_total_bytes(::google::protobuf::int64 value) { + + total_bytes_ = value; + // @@protoc_insertion_point(field_set:tensorflow.AllocatorMemoryUsed.total_bytes) +} + +// int64 peak_bytes = 3; +inline void AllocatorMemoryUsed::clear_peak_bytes() { + peak_bytes_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 AllocatorMemoryUsed::peak_bytes() const { + // @@protoc_insertion_point(field_get:tensorflow.AllocatorMemoryUsed.peak_bytes) + return peak_bytes_; +} +inline void AllocatorMemoryUsed::set_peak_bytes(::google::protobuf::int64 value) { + + peak_bytes_ = value; + // @@protoc_insertion_point(field_set:tensorflow.AllocatorMemoryUsed.peak_bytes) +} + +// int64 live_bytes = 4; +inline void AllocatorMemoryUsed::clear_live_bytes() { + live_bytes_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 AllocatorMemoryUsed::live_bytes() const { + // @@protoc_insertion_point(field_get:tensorflow.AllocatorMemoryUsed.live_bytes) + return live_bytes_; +} +inline void AllocatorMemoryUsed::set_live_bytes(::google::protobuf::int64 value) { + + live_bytes_ = value; + // @@protoc_insertion_point(field_set:tensorflow.AllocatorMemoryUsed.live_bytes) +} + +// repeated .tensorflow.AllocationRecord allocation_records = 6; +inline int AllocatorMemoryUsed::allocation_records_size() const { + return allocation_records_.size(); +} +inline void AllocatorMemoryUsed::clear_allocation_records() { + allocation_records_.Clear(); +} +inline ::tensorflow::AllocationRecord* AllocatorMemoryUsed::mutable_allocation_records(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.AllocatorMemoryUsed.allocation_records) + return allocation_records_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::AllocationRecord >* +AllocatorMemoryUsed::mutable_allocation_records() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.AllocatorMemoryUsed.allocation_records) + return &allocation_records_; +} +inline const ::tensorflow::AllocationRecord& AllocatorMemoryUsed::allocation_records(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.AllocatorMemoryUsed.allocation_records) + return allocation_records_.Get(index); +} +inline ::tensorflow::AllocationRecord* AllocatorMemoryUsed::add_allocation_records() { + // @@protoc_insertion_point(field_add:tensorflow.AllocatorMemoryUsed.allocation_records) + return allocation_records_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::AllocationRecord >& +AllocatorMemoryUsed::allocation_records() const { + // @@protoc_insertion_point(field_list:tensorflow.AllocatorMemoryUsed.allocation_records) + return allocation_records_; +} + +// int64 allocator_bytes_in_use = 5; +inline void AllocatorMemoryUsed::clear_allocator_bytes_in_use() { + allocator_bytes_in_use_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 AllocatorMemoryUsed::allocator_bytes_in_use() const { + // @@protoc_insertion_point(field_get:tensorflow.AllocatorMemoryUsed.allocator_bytes_in_use) + return allocator_bytes_in_use_; +} +inline void AllocatorMemoryUsed::set_allocator_bytes_in_use(::google::protobuf::int64 value) { + + allocator_bytes_in_use_ = value; + // @@protoc_insertion_point(field_set:tensorflow.AllocatorMemoryUsed.allocator_bytes_in_use) +} + +// ------------------------------------------------------------------- + +// NodeOutput + +// int32 slot = 1; +inline void NodeOutput::clear_slot() { + slot_ = 0; +} +inline ::google::protobuf::int32 NodeOutput::slot() const { + // @@protoc_insertion_point(field_get:tensorflow.NodeOutput.slot) + return slot_; +} +inline void NodeOutput::set_slot(::google::protobuf::int32 value) { + + slot_ = value; + // @@protoc_insertion_point(field_set:tensorflow.NodeOutput.slot) +} + +// .tensorflow.TensorDescription tensor_description = 3; +inline bool NodeOutput::has_tensor_description() const { + return this != internal_default_instance() && tensor_description_ != NULL; +} +inline const ::tensorflow::TensorDescription& NodeOutput::_internal_tensor_description() const { + return *tensor_description_; +} +inline const ::tensorflow::TensorDescription& NodeOutput::tensor_description() const { + const ::tensorflow::TensorDescription* p = tensor_description_; + // @@protoc_insertion_point(field_get:tensorflow.NodeOutput.tensor_description) + return p != NULL ? *p : *reinterpret_cast( + &::tensorflow::_TensorDescription_default_instance_); +} +inline ::tensorflow::TensorDescription* NodeOutput::release_tensor_description() { + // @@protoc_insertion_point(field_release:tensorflow.NodeOutput.tensor_description) + + ::tensorflow::TensorDescription* temp = tensor_description_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + tensor_description_ = NULL; + return temp; +} +inline ::tensorflow::TensorDescription* NodeOutput::unsafe_arena_release_tensor_description() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.NodeOutput.tensor_description) + + ::tensorflow::TensorDescription* temp = tensor_description_; + tensor_description_ = NULL; + return temp; +} +inline ::tensorflow::TensorDescription* NodeOutput::mutable_tensor_description() { + + if (tensor_description_ == NULL) { + auto* p = CreateMaybeMessage<::tensorflow::TensorDescription>(GetArenaNoVirtual()); + tensor_description_ = p; + } + // @@protoc_insertion_point(field_mutable:tensorflow.NodeOutput.tensor_description) + return tensor_description_; +} +inline void NodeOutput::set_allocated_tensor_description(::tensorflow::TensorDescription* tensor_description) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(tensor_description_); + } + if (tensor_description) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(tensor_description)->GetArena(); + if (message_arena != submessage_arena) { + tensor_description = ::google::protobuf::internal::GetOwnedMessage( + message_arena, tensor_description, submessage_arena); + } + + } else { + + } + tensor_description_ = tensor_description; + // @@protoc_insertion_point(field_set_allocated:tensorflow.NodeOutput.tensor_description) +} + +// ------------------------------------------------------------------- + +// MemoryStats + +// int64 temp_memory_size = 1; +inline void MemoryStats::clear_temp_memory_size() { + temp_memory_size_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 MemoryStats::temp_memory_size() const { + // @@protoc_insertion_point(field_get:tensorflow.MemoryStats.temp_memory_size) + return temp_memory_size_; +} +inline void MemoryStats::set_temp_memory_size(::google::protobuf::int64 value) { + + temp_memory_size_ = value; + // @@protoc_insertion_point(field_set:tensorflow.MemoryStats.temp_memory_size) +} + +// int64 persistent_memory_size = 3; +inline void MemoryStats::clear_persistent_memory_size() { + persistent_memory_size_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 MemoryStats::persistent_memory_size() const { + // @@protoc_insertion_point(field_get:tensorflow.MemoryStats.persistent_memory_size) + return persistent_memory_size_; +} +inline void MemoryStats::set_persistent_memory_size(::google::protobuf::int64 value) { + + persistent_memory_size_ = value; + // @@protoc_insertion_point(field_set:tensorflow.MemoryStats.persistent_memory_size) +} + +// repeated int64 persistent_tensor_alloc_ids = 5; +inline int MemoryStats::persistent_tensor_alloc_ids_size() const { + return persistent_tensor_alloc_ids_.size(); +} +inline void MemoryStats::clear_persistent_tensor_alloc_ids() { + persistent_tensor_alloc_ids_.Clear(); +} +inline ::google::protobuf::int64 MemoryStats::persistent_tensor_alloc_ids(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.MemoryStats.persistent_tensor_alloc_ids) + return persistent_tensor_alloc_ids_.Get(index); +} +inline void MemoryStats::set_persistent_tensor_alloc_ids(int index, ::google::protobuf::int64 value) { + persistent_tensor_alloc_ids_.Set(index, value); + // @@protoc_insertion_point(field_set:tensorflow.MemoryStats.persistent_tensor_alloc_ids) +} +inline void MemoryStats::add_persistent_tensor_alloc_ids(::google::protobuf::int64 value) { + persistent_tensor_alloc_ids_.Add(value); + // @@protoc_insertion_point(field_add:tensorflow.MemoryStats.persistent_tensor_alloc_ids) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& +MemoryStats::persistent_tensor_alloc_ids() const { + // @@protoc_insertion_point(field_list:tensorflow.MemoryStats.persistent_tensor_alloc_ids) + return persistent_tensor_alloc_ids_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* +MemoryStats::mutable_persistent_tensor_alloc_ids() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.MemoryStats.persistent_tensor_alloc_ids) + return &persistent_tensor_alloc_ids_; +} + +// int64 device_temp_memory_size = 2 [deprecated = true]; +inline void MemoryStats::clear_device_temp_memory_size() { + device_temp_memory_size_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 MemoryStats::device_temp_memory_size() const { + // @@protoc_insertion_point(field_get:tensorflow.MemoryStats.device_temp_memory_size) + return device_temp_memory_size_; +} +inline void MemoryStats::set_device_temp_memory_size(::google::protobuf::int64 value) { + + device_temp_memory_size_ = value; + // @@protoc_insertion_point(field_set:tensorflow.MemoryStats.device_temp_memory_size) +} + +// int64 device_persistent_memory_size = 4 [deprecated = true]; +inline void MemoryStats::clear_device_persistent_memory_size() { + device_persistent_memory_size_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 MemoryStats::device_persistent_memory_size() const { + // @@protoc_insertion_point(field_get:tensorflow.MemoryStats.device_persistent_memory_size) + return device_persistent_memory_size_; +} +inline void MemoryStats::set_device_persistent_memory_size(::google::protobuf::int64 value) { + + device_persistent_memory_size_ = value; + // @@protoc_insertion_point(field_set:tensorflow.MemoryStats.device_persistent_memory_size) +} + +// repeated int64 device_persistent_tensor_alloc_ids = 6 [deprecated = true]; +inline int MemoryStats::device_persistent_tensor_alloc_ids_size() const { + return device_persistent_tensor_alloc_ids_.size(); +} +inline void MemoryStats::clear_device_persistent_tensor_alloc_ids() { + device_persistent_tensor_alloc_ids_.Clear(); +} +inline ::google::protobuf::int64 MemoryStats::device_persistent_tensor_alloc_ids(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.MemoryStats.device_persistent_tensor_alloc_ids) + return device_persistent_tensor_alloc_ids_.Get(index); +} +inline void MemoryStats::set_device_persistent_tensor_alloc_ids(int index, ::google::protobuf::int64 value) { + device_persistent_tensor_alloc_ids_.Set(index, value); + // @@protoc_insertion_point(field_set:tensorflow.MemoryStats.device_persistent_tensor_alloc_ids) +} +inline void MemoryStats::add_device_persistent_tensor_alloc_ids(::google::protobuf::int64 value) { + device_persistent_tensor_alloc_ids_.Add(value); + // @@protoc_insertion_point(field_add:tensorflow.MemoryStats.device_persistent_tensor_alloc_ids) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& +MemoryStats::device_persistent_tensor_alloc_ids() const { + // @@protoc_insertion_point(field_list:tensorflow.MemoryStats.device_persistent_tensor_alloc_ids) + return device_persistent_tensor_alloc_ids_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* +MemoryStats::mutable_device_persistent_tensor_alloc_ids() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.MemoryStats.device_persistent_tensor_alloc_ids) + return &device_persistent_tensor_alloc_ids_; +} + +// ------------------------------------------------------------------- + +// NodeExecStats + +// string node_name = 1; +inline void NodeExecStats::clear_node_name() { + node_name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& NodeExecStats::node_name() const { + // @@protoc_insertion_point(field_get:tensorflow.NodeExecStats.node_name) + return node_name_.Get(); +} +inline void NodeExecStats::set_node_name(const ::std::string& value) { + + node_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.NodeExecStats.node_name) +} +#if LANG_CXX11 +inline void NodeExecStats::set_node_name(::std::string&& value) { + + node_name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.NodeExecStats.node_name) +} +#endif +inline void NodeExecStats::set_node_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + node_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.NodeExecStats.node_name) +} +inline void NodeExecStats::set_node_name(const char* value, + size_t size) { + + node_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.NodeExecStats.node_name) +} +inline ::std::string* NodeExecStats::mutable_node_name() { + + // @@protoc_insertion_point(field_mutable:tensorflow.NodeExecStats.node_name) + return node_name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* NodeExecStats::release_node_name() { + // @@protoc_insertion_point(field_release:tensorflow.NodeExecStats.node_name) + + return node_name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void NodeExecStats::set_allocated_node_name(::std::string* node_name) { + if (node_name != NULL) { + + } else { + + } + node_name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), node_name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.NodeExecStats.node_name) +} +inline ::std::string* NodeExecStats::unsafe_arena_release_node_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.NodeExecStats.node_name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return node_name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void NodeExecStats::unsafe_arena_set_allocated_node_name( + ::std::string* node_name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (node_name != NULL) { + + } else { + + } + node_name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + node_name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.NodeExecStats.node_name) +} + +// int64 all_start_micros = 2; +inline void NodeExecStats::clear_all_start_micros() { + all_start_micros_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 NodeExecStats::all_start_micros() const { + // @@protoc_insertion_point(field_get:tensorflow.NodeExecStats.all_start_micros) + return all_start_micros_; +} +inline void NodeExecStats::set_all_start_micros(::google::protobuf::int64 value) { + + all_start_micros_ = value; + // @@protoc_insertion_point(field_set:tensorflow.NodeExecStats.all_start_micros) +} + +// int64 op_start_rel_micros = 3; +inline void NodeExecStats::clear_op_start_rel_micros() { + op_start_rel_micros_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 NodeExecStats::op_start_rel_micros() const { + // @@protoc_insertion_point(field_get:tensorflow.NodeExecStats.op_start_rel_micros) + return op_start_rel_micros_; +} +inline void NodeExecStats::set_op_start_rel_micros(::google::protobuf::int64 value) { + + op_start_rel_micros_ = value; + // @@protoc_insertion_point(field_set:tensorflow.NodeExecStats.op_start_rel_micros) +} + +// int64 op_end_rel_micros = 4; +inline void NodeExecStats::clear_op_end_rel_micros() { + op_end_rel_micros_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 NodeExecStats::op_end_rel_micros() const { + // @@protoc_insertion_point(field_get:tensorflow.NodeExecStats.op_end_rel_micros) + return op_end_rel_micros_; +} +inline void NodeExecStats::set_op_end_rel_micros(::google::protobuf::int64 value) { + + op_end_rel_micros_ = value; + // @@protoc_insertion_point(field_set:tensorflow.NodeExecStats.op_end_rel_micros) +} + +// int64 all_end_rel_micros = 5; +inline void NodeExecStats::clear_all_end_rel_micros() { + all_end_rel_micros_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 NodeExecStats::all_end_rel_micros() const { + // @@protoc_insertion_point(field_get:tensorflow.NodeExecStats.all_end_rel_micros) + return all_end_rel_micros_; +} +inline void NodeExecStats::set_all_end_rel_micros(::google::protobuf::int64 value) { + + all_end_rel_micros_ = value; + // @@protoc_insertion_point(field_set:tensorflow.NodeExecStats.all_end_rel_micros) +} + +// repeated .tensorflow.AllocatorMemoryUsed memory = 6; +inline int NodeExecStats::memory_size() const { + return memory_.size(); +} +inline void NodeExecStats::clear_memory() { + memory_.Clear(); +} +inline ::tensorflow::AllocatorMemoryUsed* NodeExecStats::mutable_memory(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.NodeExecStats.memory) + return memory_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::AllocatorMemoryUsed >* +NodeExecStats::mutable_memory() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.NodeExecStats.memory) + return &memory_; +} +inline const ::tensorflow::AllocatorMemoryUsed& NodeExecStats::memory(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.NodeExecStats.memory) + return memory_.Get(index); +} +inline ::tensorflow::AllocatorMemoryUsed* NodeExecStats::add_memory() { + // @@protoc_insertion_point(field_add:tensorflow.NodeExecStats.memory) + return memory_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::AllocatorMemoryUsed >& +NodeExecStats::memory() const { + // @@protoc_insertion_point(field_list:tensorflow.NodeExecStats.memory) + return memory_; +} + +// repeated .tensorflow.NodeOutput output = 7; +inline int NodeExecStats::output_size() const { + return output_.size(); +} +inline void NodeExecStats::clear_output() { + output_.Clear(); +} +inline ::tensorflow::NodeOutput* NodeExecStats::mutable_output(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.NodeExecStats.output) + return output_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::NodeOutput >* +NodeExecStats::mutable_output() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.NodeExecStats.output) + return &output_; +} +inline const ::tensorflow::NodeOutput& NodeExecStats::output(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.NodeExecStats.output) + return output_.Get(index); +} +inline ::tensorflow::NodeOutput* NodeExecStats::add_output() { + // @@protoc_insertion_point(field_add:tensorflow.NodeExecStats.output) + return output_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::NodeOutput >& +NodeExecStats::output() const { + // @@protoc_insertion_point(field_list:tensorflow.NodeExecStats.output) + return output_; +} + +// string timeline_label = 8; +inline void NodeExecStats::clear_timeline_label() { + timeline_label_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& NodeExecStats::timeline_label() const { + // @@protoc_insertion_point(field_get:tensorflow.NodeExecStats.timeline_label) + return timeline_label_.Get(); +} +inline void NodeExecStats::set_timeline_label(const ::std::string& value) { + + timeline_label_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.NodeExecStats.timeline_label) +} +#if LANG_CXX11 +inline void NodeExecStats::set_timeline_label(::std::string&& value) { + + timeline_label_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.NodeExecStats.timeline_label) +} +#endif +inline void NodeExecStats::set_timeline_label(const char* value) { + GOOGLE_DCHECK(value != NULL); + + timeline_label_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.NodeExecStats.timeline_label) +} +inline void NodeExecStats::set_timeline_label(const char* value, + size_t size) { + + timeline_label_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.NodeExecStats.timeline_label) +} +inline ::std::string* NodeExecStats::mutable_timeline_label() { + + // @@protoc_insertion_point(field_mutable:tensorflow.NodeExecStats.timeline_label) + return timeline_label_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* NodeExecStats::release_timeline_label() { + // @@protoc_insertion_point(field_release:tensorflow.NodeExecStats.timeline_label) + + return timeline_label_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void NodeExecStats::set_allocated_timeline_label(::std::string* timeline_label) { + if (timeline_label != NULL) { + + } else { + + } + timeline_label_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), timeline_label, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.NodeExecStats.timeline_label) +} +inline ::std::string* NodeExecStats::unsafe_arena_release_timeline_label() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.NodeExecStats.timeline_label) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return timeline_label_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void NodeExecStats::unsafe_arena_set_allocated_timeline_label( + ::std::string* timeline_label) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (timeline_label != NULL) { + + } else { + + } + timeline_label_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + timeline_label, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.NodeExecStats.timeline_label) +} + +// int64 scheduled_micros = 9; +inline void NodeExecStats::clear_scheduled_micros() { + scheduled_micros_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 NodeExecStats::scheduled_micros() const { + // @@protoc_insertion_point(field_get:tensorflow.NodeExecStats.scheduled_micros) + return scheduled_micros_; +} +inline void NodeExecStats::set_scheduled_micros(::google::protobuf::int64 value) { + + scheduled_micros_ = value; + // @@protoc_insertion_point(field_set:tensorflow.NodeExecStats.scheduled_micros) +} + +// uint32 thread_id = 10; +inline void NodeExecStats::clear_thread_id() { + thread_id_ = 0u; +} +inline ::google::protobuf::uint32 NodeExecStats::thread_id() const { + // @@protoc_insertion_point(field_get:tensorflow.NodeExecStats.thread_id) + return thread_id_; +} +inline void NodeExecStats::set_thread_id(::google::protobuf::uint32 value) { + + thread_id_ = value; + // @@protoc_insertion_point(field_set:tensorflow.NodeExecStats.thread_id) +} + +// repeated .tensorflow.AllocationDescription referenced_tensor = 11; +inline int NodeExecStats::referenced_tensor_size() const { + return referenced_tensor_.size(); +} +inline ::tensorflow::AllocationDescription* NodeExecStats::mutable_referenced_tensor(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.NodeExecStats.referenced_tensor) + return referenced_tensor_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::AllocationDescription >* +NodeExecStats::mutable_referenced_tensor() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.NodeExecStats.referenced_tensor) + return &referenced_tensor_; +} +inline const ::tensorflow::AllocationDescription& NodeExecStats::referenced_tensor(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.NodeExecStats.referenced_tensor) + return referenced_tensor_.Get(index); +} +inline ::tensorflow::AllocationDescription* NodeExecStats::add_referenced_tensor() { + // @@protoc_insertion_point(field_add:tensorflow.NodeExecStats.referenced_tensor) + return referenced_tensor_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::AllocationDescription >& +NodeExecStats::referenced_tensor() const { + // @@protoc_insertion_point(field_list:tensorflow.NodeExecStats.referenced_tensor) + return referenced_tensor_; +} + +// .tensorflow.MemoryStats memory_stats = 12; +inline bool NodeExecStats::has_memory_stats() const { + return this != internal_default_instance() && memory_stats_ != NULL; +} +inline void NodeExecStats::clear_memory_stats() { + if (GetArenaNoVirtual() == NULL && memory_stats_ != NULL) { + delete memory_stats_; + } + memory_stats_ = NULL; +} +inline const ::tensorflow::MemoryStats& NodeExecStats::_internal_memory_stats() const { + return *memory_stats_; +} +inline const ::tensorflow::MemoryStats& NodeExecStats::memory_stats() const { + const ::tensorflow::MemoryStats* p = memory_stats_; + // @@protoc_insertion_point(field_get:tensorflow.NodeExecStats.memory_stats) + return p != NULL ? *p : *reinterpret_cast( + &::tensorflow::_MemoryStats_default_instance_); +} +inline ::tensorflow::MemoryStats* NodeExecStats::release_memory_stats() { + // @@protoc_insertion_point(field_release:tensorflow.NodeExecStats.memory_stats) + + ::tensorflow::MemoryStats* temp = memory_stats_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + memory_stats_ = NULL; + return temp; +} +inline ::tensorflow::MemoryStats* NodeExecStats::unsafe_arena_release_memory_stats() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.NodeExecStats.memory_stats) + + ::tensorflow::MemoryStats* temp = memory_stats_; + memory_stats_ = NULL; + return temp; +} +inline ::tensorflow::MemoryStats* NodeExecStats::mutable_memory_stats() { + + if (memory_stats_ == NULL) { + auto* p = CreateMaybeMessage<::tensorflow::MemoryStats>(GetArenaNoVirtual()); + memory_stats_ = p; + } + // @@protoc_insertion_point(field_mutable:tensorflow.NodeExecStats.memory_stats) + return memory_stats_; +} +inline void NodeExecStats::set_allocated_memory_stats(::tensorflow::MemoryStats* memory_stats) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete memory_stats_; + } + if (memory_stats) { + ::google::protobuf::Arena* submessage_arena = + ::google::protobuf::Arena::GetArena(memory_stats); + if (message_arena != submessage_arena) { + memory_stats = ::google::protobuf::internal::GetOwnedMessage( + message_arena, memory_stats, submessage_arena); + } + + } else { + + } + memory_stats_ = memory_stats; + // @@protoc_insertion_point(field_set_allocated:tensorflow.NodeExecStats.memory_stats) +} + +// int64 all_start_nanos = 13; +inline void NodeExecStats::clear_all_start_nanos() { + all_start_nanos_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 NodeExecStats::all_start_nanos() const { + // @@protoc_insertion_point(field_get:tensorflow.NodeExecStats.all_start_nanos) + return all_start_nanos_; +} +inline void NodeExecStats::set_all_start_nanos(::google::protobuf::int64 value) { + + all_start_nanos_ = value; + // @@protoc_insertion_point(field_set:tensorflow.NodeExecStats.all_start_nanos) +} + +// int64 op_start_rel_nanos = 14; +inline void NodeExecStats::clear_op_start_rel_nanos() { + op_start_rel_nanos_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 NodeExecStats::op_start_rel_nanos() const { + // @@protoc_insertion_point(field_get:tensorflow.NodeExecStats.op_start_rel_nanos) + return op_start_rel_nanos_; +} +inline void NodeExecStats::set_op_start_rel_nanos(::google::protobuf::int64 value) { + + op_start_rel_nanos_ = value; + // @@protoc_insertion_point(field_set:tensorflow.NodeExecStats.op_start_rel_nanos) +} + +// int64 op_end_rel_nanos = 15; +inline void NodeExecStats::clear_op_end_rel_nanos() { + op_end_rel_nanos_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 NodeExecStats::op_end_rel_nanos() const { + // @@protoc_insertion_point(field_get:tensorflow.NodeExecStats.op_end_rel_nanos) + return op_end_rel_nanos_; +} +inline void NodeExecStats::set_op_end_rel_nanos(::google::protobuf::int64 value) { + + op_end_rel_nanos_ = value; + // @@protoc_insertion_point(field_set:tensorflow.NodeExecStats.op_end_rel_nanos) +} + +// int64 all_end_rel_nanos = 16; +inline void NodeExecStats::clear_all_end_rel_nanos() { + all_end_rel_nanos_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 NodeExecStats::all_end_rel_nanos() const { + // @@protoc_insertion_point(field_get:tensorflow.NodeExecStats.all_end_rel_nanos) + return all_end_rel_nanos_; +} +inline void NodeExecStats::set_all_end_rel_nanos(::google::protobuf::int64 value) { + + all_end_rel_nanos_ = value; + // @@protoc_insertion_point(field_set:tensorflow.NodeExecStats.all_end_rel_nanos) +} + +// int64 scheduled_nanos = 17; +inline void NodeExecStats::clear_scheduled_nanos() { + scheduled_nanos_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 NodeExecStats::scheduled_nanos() const { + // @@protoc_insertion_point(field_get:tensorflow.NodeExecStats.scheduled_nanos) + return scheduled_nanos_; +} +inline void NodeExecStats::set_scheduled_nanos(::google::protobuf::int64 value) { + + scheduled_nanos_ = value; + // @@protoc_insertion_point(field_set:tensorflow.NodeExecStats.scheduled_nanos) +} + +// ------------------------------------------------------------------- + +// DeviceStepStats + +// string device = 1; +inline void DeviceStepStats::clear_device() { + device_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& DeviceStepStats::device() const { + // @@protoc_insertion_point(field_get:tensorflow.DeviceStepStats.device) + return device_.Get(); +} +inline void DeviceStepStats::set_device(const ::std::string& value) { + + device_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.DeviceStepStats.device) +} +#if LANG_CXX11 +inline void DeviceStepStats::set_device(::std::string&& value) { + + device_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.DeviceStepStats.device) +} +#endif +inline void DeviceStepStats::set_device(const char* value) { + GOOGLE_DCHECK(value != NULL); + + device_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.DeviceStepStats.device) +} +inline void DeviceStepStats::set_device(const char* value, + size_t size) { + + device_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.DeviceStepStats.device) +} +inline ::std::string* DeviceStepStats::mutable_device() { + + // @@protoc_insertion_point(field_mutable:tensorflow.DeviceStepStats.device) + return device_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* DeviceStepStats::release_device() { + // @@protoc_insertion_point(field_release:tensorflow.DeviceStepStats.device) + + return device_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void DeviceStepStats::set_allocated_device(::std::string* device) { + if (device != NULL) { + + } else { + + } + device_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), device, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.DeviceStepStats.device) +} +inline ::std::string* DeviceStepStats::unsafe_arena_release_device() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.DeviceStepStats.device) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return device_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void DeviceStepStats::unsafe_arena_set_allocated_device( + ::std::string* device) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (device != NULL) { + + } else { + + } + device_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + device, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.DeviceStepStats.device) +} + +// repeated .tensorflow.NodeExecStats node_stats = 2; +inline int DeviceStepStats::node_stats_size() const { + return node_stats_.size(); +} +inline void DeviceStepStats::clear_node_stats() { + node_stats_.Clear(); +} +inline ::tensorflow::NodeExecStats* DeviceStepStats::mutable_node_stats(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.DeviceStepStats.node_stats) + return node_stats_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::NodeExecStats >* +DeviceStepStats::mutable_node_stats() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.DeviceStepStats.node_stats) + return &node_stats_; +} +inline const ::tensorflow::NodeExecStats& DeviceStepStats::node_stats(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.DeviceStepStats.node_stats) + return node_stats_.Get(index); +} +inline ::tensorflow::NodeExecStats* DeviceStepStats::add_node_stats() { + // @@protoc_insertion_point(field_add:tensorflow.DeviceStepStats.node_stats) + return node_stats_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::NodeExecStats >& +DeviceStepStats::node_stats() const { + // @@protoc_insertion_point(field_list:tensorflow.DeviceStepStats.node_stats) + return node_stats_; +} + +// ------------------------------------------------------------------- + +// StepStats + +// repeated .tensorflow.DeviceStepStats dev_stats = 1; +inline int StepStats::dev_stats_size() const { + return dev_stats_.size(); +} +inline void StepStats::clear_dev_stats() { + dev_stats_.Clear(); +} +inline ::tensorflow::DeviceStepStats* StepStats::mutable_dev_stats(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.StepStats.dev_stats) + return dev_stats_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::DeviceStepStats >* +StepStats::mutable_dev_stats() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.StepStats.dev_stats) + return &dev_stats_; +} +inline const ::tensorflow::DeviceStepStats& StepStats::dev_stats(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.StepStats.dev_stats) + return dev_stats_.Get(index); +} +inline ::tensorflow::DeviceStepStats* StepStats::add_dev_stats() { + // @@protoc_insertion_point(field_add:tensorflow.StepStats.dev_stats) + return dev_stats_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::DeviceStepStats >& +StepStats::dev_stats() const { + // @@protoc_insertion_point(field_list:tensorflow.StepStats.dev_stats) + return dev_stats_; +} + +#ifdef __GNUC__ + #pragma GCC diagnostic pop +#endif // __GNUC__ +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + + +// @@protoc_insertion_point(namespace_scope) + +} // namespace tensorflow + +// @@protoc_insertion_point(global_scope) + +#endif // PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2fstep_5fstats_2eproto diff --git a/linux/include/tensorflow/core/framework/step_stats.pb_text-impl.h b/linux/include/tensorflow/core/framework/step_stats.pb_text-impl.h new file mode 100644 index 000000000..efec6cfd6 --- /dev/null +++ b/linux/include/tensorflow/core/framework/step_stats.pb_text-impl.h @@ -0,0 +1,75 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_framework_step_stats_proto_IMPL_H_ +#define tensorflow_core_framework_step_stats_proto_IMPL_H_ + +#include "tensorflow/core/framework/allocation_description.pb.h" +#include "tensorflow/core/framework/allocation_description.pb_text-impl.h" +#include "tensorflow/core/framework/step_stats.pb.h" +#include "tensorflow/core/framework/step_stats.pb_text.h" +#include "tensorflow/core/framework/tensor_description.pb.h" +#include "tensorflow/core/framework/tensor_description.pb_text-impl.h" +#include "tensorflow/core/framework/tensor_shape.pb.h" +#include "tensorflow/core/framework/tensor_shape.pb_text-impl.h" +#include "tensorflow/core/framework/types.pb.h" +#include "tensorflow/core/framework/types.pb_text-impl.h" +#include "tensorflow/core/lib/strings/proto_text_util.h" +#include "tensorflow/core/lib/strings/scanner.h" + +namespace tensorflow { + +namespace internal { + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::AllocationRecord& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::AllocationRecord* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::AllocatorMemoryUsed& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::AllocatorMemoryUsed* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::NodeOutput& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::NodeOutput* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::MemoryStats& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::MemoryStats* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::NodeExecStats& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::NodeExecStats* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::DeviceStepStats& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::DeviceStepStats* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::StepStats& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::StepStats* msg); + +} // namespace internal + +} // namespace tensorflow + +#endif // tensorflow_core_framework_step_stats_proto_IMPL_H_ diff --git a/linux/include/tensorflow/core/framework/step_stats.pb_text.h b/linux/include/tensorflow/core/framework/step_stats.pb_text.h new file mode 100644 index 000000000..14ea21489 --- /dev/null +++ b/linux/include/tensorflow/core/framework/step_stats.pb_text.h @@ -0,0 +1,84 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_framework_step_stats_proto_H_ +#define tensorflow_core_framework_step_stats_proto_H_ + +#include "tensorflow/core/framework/step_stats.pb.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/protobuf.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +// Message-text conversion for tensorflow.AllocationRecord +string ProtoDebugString( + const ::tensorflow::AllocationRecord& msg); +string ProtoShortDebugString( + const ::tensorflow::AllocationRecord& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::AllocationRecord* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.AllocatorMemoryUsed +string ProtoDebugString( + const ::tensorflow::AllocatorMemoryUsed& msg); +string ProtoShortDebugString( + const ::tensorflow::AllocatorMemoryUsed& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::AllocatorMemoryUsed* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.NodeOutput +string ProtoDebugString( + const ::tensorflow::NodeOutput& msg); +string ProtoShortDebugString( + const ::tensorflow::NodeOutput& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::NodeOutput* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.MemoryStats +string ProtoDebugString( + const ::tensorflow::MemoryStats& msg); +string ProtoShortDebugString( + const ::tensorflow::MemoryStats& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::MemoryStats* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.NodeExecStats +string ProtoDebugString( + const ::tensorflow::NodeExecStats& msg); +string ProtoShortDebugString( + const ::tensorflow::NodeExecStats& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::NodeExecStats* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.DeviceStepStats +string ProtoDebugString( + const ::tensorflow::DeviceStepStats& msg); +string ProtoShortDebugString( + const ::tensorflow::DeviceStepStats& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::DeviceStepStats* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.StepStats +string ProtoDebugString( + const ::tensorflow::StepStats& msg); +string ProtoShortDebugString( + const ::tensorflow::StepStats& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::StepStats* msg) + TF_MUST_USE_RESULT; + +} // namespace tensorflow + +#endif // tensorflow_core_framework_step_stats_proto_H_ diff --git a/linux/include/tensorflow/core/framework/summary.pb.h b/linux/include/tensorflow/core/framework/summary.pb.h new file mode 100644 index 000000000..9e866f34c --- /dev/null +++ b/linux/include/tensorflow/core/framework/summary.pb.h @@ -0,0 +1,3113 @@ +// Generated by the protocol buffer compiler. DO NOT EDIT! +// source: tensorflow/core/framework/summary.proto + +#ifndef PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2fsummary_2eproto +#define PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2fsummary_2eproto + +#include + +#include + +#if GOOGLE_PROTOBUF_VERSION < 3006000 +#error This file was generated by a newer version of protoc which is +#error incompatible with your Protocol Buffer headers. Please update +#error your headers. +#endif +#if 3006000 < GOOGLE_PROTOBUF_MIN_PROTOC_VERSION +#error This file was generated by an older version of protoc which is +#error incompatible with your Protocol Buffer headers. Please +#error regenerate this file with a newer version of protoc. +#endif + +#include +#include +#include +#include +#include +#include +#include +#include +#include // IWYU pragma: export +#include // IWYU pragma: export +#include +#include "tensorflow/core/framework/tensor.pb.h" +// @@protoc_insertion_point(includes) +#define PROTOBUF_INTERNAL_EXPORT_protobuf_tensorflow_2fcore_2fframework_2fsummary_2eproto + +namespace protobuf_tensorflow_2fcore_2fframework_2fsummary_2eproto { +// Internal implementation detail -- do not use these members. +struct TableStruct { + static const ::google::protobuf::internal::ParseTableField entries[]; + static const ::google::protobuf::internal::AuxillaryParseTableField aux[]; + static const ::google::protobuf::internal::ParseTable schema[8]; + static const ::google::protobuf::internal::FieldMetadata field_metadata[]; + static const ::google::protobuf::internal::SerializationTable serialization_table[]; + static const ::google::protobuf::uint32 offsets[]; +}; +void AddDescriptors(); +} // namespace protobuf_tensorflow_2fcore_2fframework_2fsummary_2eproto +namespace tensorflow { +class HistogramProto; +class HistogramProtoDefaultTypeInternal; +extern HistogramProtoDefaultTypeInternal _HistogramProto_default_instance_; +class Summary; +class SummaryDefaultTypeInternal; +extern SummaryDefaultTypeInternal _Summary_default_instance_; +class SummaryDescription; +class SummaryDescriptionDefaultTypeInternal; +extern SummaryDescriptionDefaultTypeInternal _SummaryDescription_default_instance_; +class SummaryMetadata; +class SummaryMetadataDefaultTypeInternal; +extern SummaryMetadataDefaultTypeInternal _SummaryMetadata_default_instance_; +class SummaryMetadata_PluginData; +class SummaryMetadata_PluginDataDefaultTypeInternal; +extern SummaryMetadata_PluginDataDefaultTypeInternal _SummaryMetadata_PluginData_default_instance_; +class Summary_Audio; +class Summary_AudioDefaultTypeInternal; +extern Summary_AudioDefaultTypeInternal _Summary_Audio_default_instance_; +class Summary_Image; +class Summary_ImageDefaultTypeInternal; +extern Summary_ImageDefaultTypeInternal _Summary_Image_default_instance_; +class Summary_Value; +class Summary_ValueDefaultTypeInternal; +extern Summary_ValueDefaultTypeInternal _Summary_Value_default_instance_; +} // namespace tensorflow +namespace google { +namespace protobuf { +template<> ::tensorflow::HistogramProto* Arena::CreateMaybeMessage<::tensorflow::HistogramProto>(Arena*); +template<> ::tensorflow::Summary* Arena::CreateMaybeMessage<::tensorflow::Summary>(Arena*); +template<> ::tensorflow::SummaryDescription* Arena::CreateMaybeMessage<::tensorflow::SummaryDescription>(Arena*); +template<> ::tensorflow::SummaryMetadata* Arena::CreateMaybeMessage<::tensorflow::SummaryMetadata>(Arena*); +template<> ::tensorflow::SummaryMetadata_PluginData* Arena::CreateMaybeMessage<::tensorflow::SummaryMetadata_PluginData>(Arena*); +template<> ::tensorflow::Summary_Audio* Arena::CreateMaybeMessage<::tensorflow::Summary_Audio>(Arena*); +template<> ::tensorflow::Summary_Image* Arena::CreateMaybeMessage<::tensorflow::Summary_Image>(Arena*); +template<> ::tensorflow::Summary_Value* Arena::CreateMaybeMessage<::tensorflow::Summary_Value>(Arena*); +} // namespace protobuf +} // namespace google +namespace tensorflow { + +// =================================================================== + +class SummaryDescription : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.SummaryDescription) */ { + public: + SummaryDescription(); + virtual ~SummaryDescription(); + + SummaryDescription(const SummaryDescription& from); + + inline SummaryDescription& operator=(const SummaryDescription& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + SummaryDescription(SummaryDescription&& from) noexcept + : SummaryDescription() { + *this = ::std::move(from); + } + + inline SummaryDescription& operator=(SummaryDescription&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const SummaryDescription& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const SummaryDescription* internal_default_instance() { + return reinterpret_cast( + &_SummaryDescription_default_instance_); + } + static constexpr int kIndexInFileMessages = + 0; + + void UnsafeArenaSwap(SummaryDescription* other); + void Swap(SummaryDescription* other); + friend void swap(SummaryDescription& a, SummaryDescription& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline SummaryDescription* New() const final { + return CreateMaybeMessage(NULL); + } + + SummaryDescription* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const SummaryDescription& from); + void MergeFrom(const SummaryDescription& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(SummaryDescription* other); + protected: + explicit SummaryDescription(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // string type_hint = 1; + void clear_type_hint(); + static const int kTypeHintFieldNumber = 1; + const ::std::string& type_hint() const; + void set_type_hint(const ::std::string& value); + #if LANG_CXX11 + void set_type_hint(::std::string&& value); + #endif + void set_type_hint(const char* value); + void set_type_hint(const char* value, size_t size); + ::std::string* mutable_type_hint(); + ::std::string* release_type_hint(); + void set_allocated_type_hint(::std::string* type_hint); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_type_hint(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_type_hint( + ::std::string* type_hint); + + // @@protoc_insertion_point(class_scope:tensorflow.SummaryDescription) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::internal::ArenaStringPtr type_hint_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fsummary_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class HistogramProto : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.HistogramProto) */ { + public: + HistogramProto(); + virtual ~HistogramProto(); + + HistogramProto(const HistogramProto& from); + + inline HistogramProto& operator=(const HistogramProto& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + HistogramProto(HistogramProto&& from) noexcept + : HistogramProto() { + *this = ::std::move(from); + } + + inline HistogramProto& operator=(HistogramProto&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const HistogramProto& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const HistogramProto* internal_default_instance() { + return reinterpret_cast( + &_HistogramProto_default_instance_); + } + static constexpr int kIndexInFileMessages = + 1; + + void UnsafeArenaSwap(HistogramProto* other); + void Swap(HistogramProto* other); + friend void swap(HistogramProto& a, HistogramProto& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline HistogramProto* New() const final { + return CreateMaybeMessage(NULL); + } + + HistogramProto* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const HistogramProto& from); + void MergeFrom(const HistogramProto& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(HistogramProto* other); + protected: + explicit HistogramProto(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated double bucket_limit = 6 [packed = true]; + int bucket_limit_size() const; + void clear_bucket_limit(); + static const int kBucketLimitFieldNumber = 6; + double bucket_limit(int index) const; + void set_bucket_limit(int index, double value); + void add_bucket_limit(double value); + const ::google::protobuf::RepeatedField< double >& + bucket_limit() const; + ::google::protobuf::RepeatedField< double >* + mutable_bucket_limit(); + + // repeated double bucket = 7 [packed = true]; + int bucket_size() const; + void clear_bucket(); + static const int kBucketFieldNumber = 7; + double bucket(int index) const; + void set_bucket(int index, double value); + void add_bucket(double value); + const ::google::protobuf::RepeatedField< double >& + bucket() const; + ::google::protobuf::RepeatedField< double >* + mutable_bucket(); + + // double min = 1; + void clear_min(); + static const int kMinFieldNumber = 1; + double min() const; + void set_min(double value); + + // double max = 2; + void clear_max(); + static const int kMaxFieldNumber = 2; + double max() const; + void set_max(double value); + + // double num = 3; + void clear_num(); + static const int kNumFieldNumber = 3; + double num() const; + void set_num(double value); + + // double sum = 4; + void clear_sum(); + static const int kSumFieldNumber = 4; + double sum() const; + void set_sum(double value); + + // double sum_squares = 5; + void clear_sum_squares(); + static const int kSumSquaresFieldNumber = 5; + double sum_squares() const; + void set_sum_squares(double value); + + // @@protoc_insertion_point(class_scope:tensorflow.HistogramProto) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedField< double > bucket_limit_; + mutable int _bucket_limit_cached_byte_size_; + ::google::protobuf::RepeatedField< double > bucket_; + mutable int _bucket_cached_byte_size_; + double min_; + double max_; + double num_; + double sum_; + double sum_squares_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fsummary_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class SummaryMetadata_PluginData : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.SummaryMetadata.PluginData) */ { + public: + SummaryMetadata_PluginData(); + virtual ~SummaryMetadata_PluginData(); + + SummaryMetadata_PluginData(const SummaryMetadata_PluginData& from); + + inline SummaryMetadata_PluginData& operator=(const SummaryMetadata_PluginData& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + SummaryMetadata_PluginData(SummaryMetadata_PluginData&& from) noexcept + : SummaryMetadata_PluginData() { + *this = ::std::move(from); + } + + inline SummaryMetadata_PluginData& operator=(SummaryMetadata_PluginData&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const SummaryMetadata_PluginData& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const SummaryMetadata_PluginData* internal_default_instance() { + return reinterpret_cast( + &_SummaryMetadata_PluginData_default_instance_); + } + static constexpr int kIndexInFileMessages = + 2; + + void UnsafeArenaSwap(SummaryMetadata_PluginData* other); + void Swap(SummaryMetadata_PluginData* other); + friend void swap(SummaryMetadata_PluginData& a, SummaryMetadata_PluginData& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline SummaryMetadata_PluginData* New() const final { + return CreateMaybeMessage(NULL); + } + + SummaryMetadata_PluginData* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const SummaryMetadata_PluginData& from); + void MergeFrom(const SummaryMetadata_PluginData& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(SummaryMetadata_PluginData* other); + protected: + explicit SummaryMetadata_PluginData(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // string plugin_name = 1; + void clear_plugin_name(); + static const int kPluginNameFieldNumber = 1; + const ::std::string& plugin_name() const; + void set_plugin_name(const ::std::string& value); + #if LANG_CXX11 + void set_plugin_name(::std::string&& value); + #endif + void set_plugin_name(const char* value); + void set_plugin_name(const char* value, size_t size); + ::std::string* mutable_plugin_name(); + ::std::string* release_plugin_name(); + void set_allocated_plugin_name(::std::string* plugin_name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_plugin_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_plugin_name( + ::std::string* plugin_name); + + // bytes content = 2; + void clear_content(); + static const int kContentFieldNumber = 2; + const ::std::string& content() const; + void set_content(const ::std::string& value); + #if LANG_CXX11 + void set_content(::std::string&& value); + #endif + void set_content(const char* value); + void set_content(const void* value, size_t size); + ::std::string* mutable_content(); + ::std::string* release_content(); + void set_allocated_content(::std::string* content); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_content(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_content( + ::std::string* content); + + // @@protoc_insertion_point(class_scope:tensorflow.SummaryMetadata.PluginData) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::internal::ArenaStringPtr plugin_name_; + ::google::protobuf::internal::ArenaStringPtr content_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fsummary_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class SummaryMetadata : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.SummaryMetadata) */ { + public: + SummaryMetadata(); + virtual ~SummaryMetadata(); + + SummaryMetadata(const SummaryMetadata& from); + + inline SummaryMetadata& operator=(const SummaryMetadata& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + SummaryMetadata(SummaryMetadata&& from) noexcept + : SummaryMetadata() { + *this = ::std::move(from); + } + + inline SummaryMetadata& operator=(SummaryMetadata&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const SummaryMetadata& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const SummaryMetadata* internal_default_instance() { + return reinterpret_cast( + &_SummaryMetadata_default_instance_); + } + static constexpr int kIndexInFileMessages = + 3; + + void UnsafeArenaSwap(SummaryMetadata* other); + void Swap(SummaryMetadata* other); + friend void swap(SummaryMetadata& a, SummaryMetadata& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline SummaryMetadata* New() const final { + return CreateMaybeMessage(NULL); + } + + SummaryMetadata* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const SummaryMetadata& from); + void MergeFrom(const SummaryMetadata& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(SummaryMetadata* other); + protected: + explicit SummaryMetadata(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + typedef SummaryMetadata_PluginData PluginData; + + // accessors ------------------------------------------------------- + + // string display_name = 2; + void clear_display_name(); + static const int kDisplayNameFieldNumber = 2; + const ::std::string& display_name() const; + void set_display_name(const ::std::string& value); + #if LANG_CXX11 + void set_display_name(::std::string&& value); + #endif + void set_display_name(const char* value); + void set_display_name(const char* value, size_t size); + ::std::string* mutable_display_name(); + ::std::string* release_display_name(); + void set_allocated_display_name(::std::string* display_name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_display_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_display_name( + ::std::string* display_name); + + // string summary_description = 3; + void clear_summary_description(); + static const int kSummaryDescriptionFieldNumber = 3; + const ::std::string& summary_description() const; + void set_summary_description(const ::std::string& value); + #if LANG_CXX11 + void set_summary_description(::std::string&& value); + #endif + void set_summary_description(const char* value); + void set_summary_description(const char* value, size_t size); + ::std::string* mutable_summary_description(); + ::std::string* release_summary_description(); + void set_allocated_summary_description(::std::string* summary_description); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_summary_description(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_summary_description( + ::std::string* summary_description); + + // .tensorflow.SummaryMetadata.PluginData plugin_data = 1; + bool has_plugin_data() const; + void clear_plugin_data(); + static const int kPluginDataFieldNumber = 1; + private: + const ::tensorflow::SummaryMetadata_PluginData& _internal_plugin_data() const; + public: + const ::tensorflow::SummaryMetadata_PluginData& plugin_data() const; + ::tensorflow::SummaryMetadata_PluginData* release_plugin_data(); + ::tensorflow::SummaryMetadata_PluginData* mutable_plugin_data(); + void set_allocated_plugin_data(::tensorflow::SummaryMetadata_PluginData* plugin_data); + void unsafe_arena_set_allocated_plugin_data( + ::tensorflow::SummaryMetadata_PluginData* plugin_data); + ::tensorflow::SummaryMetadata_PluginData* unsafe_arena_release_plugin_data(); + + // @@protoc_insertion_point(class_scope:tensorflow.SummaryMetadata) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::internal::ArenaStringPtr display_name_; + ::google::protobuf::internal::ArenaStringPtr summary_description_; + ::tensorflow::SummaryMetadata_PluginData* plugin_data_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fsummary_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class Summary_Image : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.Summary.Image) */ { + public: + Summary_Image(); + virtual ~Summary_Image(); + + Summary_Image(const Summary_Image& from); + + inline Summary_Image& operator=(const Summary_Image& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + Summary_Image(Summary_Image&& from) noexcept + : Summary_Image() { + *this = ::std::move(from); + } + + inline Summary_Image& operator=(Summary_Image&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const Summary_Image& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const Summary_Image* internal_default_instance() { + return reinterpret_cast( + &_Summary_Image_default_instance_); + } + static constexpr int kIndexInFileMessages = + 4; + + void UnsafeArenaSwap(Summary_Image* other); + void Swap(Summary_Image* other); + friend void swap(Summary_Image& a, Summary_Image& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline Summary_Image* New() const final { + return CreateMaybeMessage(NULL); + } + + Summary_Image* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const Summary_Image& from); + void MergeFrom(const Summary_Image& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(Summary_Image* other); + protected: + explicit Summary_Image(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // bytes encoded_image_string = 4; + void clear_encoded_image_string(); + static const int kEncodedImageStringFieldNumber = 4; + const ::std::string& encoded_image_string() const; + void set_encoded_image_string(const ::std::string& value); + #if LANG_CXX11 + void set_encoded_image_string(::std::string&& value); + #endif + void set_encoded_image_string(const char* value); + void set_encoded_image_string(const void* value, size_t size); + ::std::string* mutable_encoded_image_string(); + ::std::string* release_encoded_image_string(); + void set_allocated_encoded_image_string(::std::string* encoded_image_string); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_encoded_image_string(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_encoded_image_string( + ::std::string* encoded_image_string); + + // int32 height = 1; + void clear_height(); + static const int kHeightFieldNumber = 1; + ::google::protobuf::int32 height() const; + void set_height(::google::protobuf::int32 value); + + // int32 width = 2; + void clear_width(); + static const int kWidthFieldNumber = 2; + ::google::protobuf::int32 width() const; + void set_width(::google::protobuf::int32 value); + + // int32 colorspace = 3; + void clear_colorspace(); + static const int kColorspaceFieldNumber = 3; + ::google::protobuf::int32 colorspace() const; + void set_colorspace(::google::protobuf::int32 value); + + // @@protoc_insertion_point(class_scope:tensorflow.Summary.Image) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::internal::ArenaStringPtr encoded_image_string_; + ::google::protobuf::int32 height_; + ::google::protobuf::int32 width_; + ::google::protobuf::int32 colorspace_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fsummary_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class Summary_Audio : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.Summary.Audio) */ { + public: + Summary_Audio(); + virtual ~Summary_Audio(); + + Summary_Audio(const Summary_Audio& from); + + inline Summary_Audio& operator=(const Summary_Audio& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + Summary_Audio(Summary_Audio&& from) noexcept + : Summary_Audio() { + *this = ::std::move(from); + } + + inline Summary_Audio& operator=(Summary_Audio&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const Summary_Audio& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const Summary_Audio* internal_default_instance() { + return reinterpret_cast( + &_Summary_Audio_default_instance_); + } + static constexpr int kIndexInFileMessages = + 5; + + void UnsafeArenaSwap(Summary_Audio* other); + void Swap(Summary_Audio* other); + friend void swap(Summary_Audio& a, Summary_Audio& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline Summary_Audio* New() const final { + return CreateMaybeMessage(NULL); + } + + Summary_Audio* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const Summary_Audio& from); + void MergeFrom(const Summary_Audio& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(Summary_Audio* other); + protected: + explicit Summary_Audio(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // bytes encoded_audio_string = 4; + void clear_encoded_audio_string(); + static const int kEncodedAudioStringFieldNumber = 4; + const ::std::string& encoded_audio_string() const; + void set_encoded_audio_string(const ::std::string& value); + #if LANG_CXX11 + void set_encoded_audio_string(::std::string&& value); + #endif + void set_encoded_audio_string(const char* value); + void set_encoded_audio_string(const void* value, size_t size); + ::std::string* mutable_encoded_audio_string(); + ::std::string* release_encoded_audio_string(); + void set_allocated_encoded_audio_string(::std::string* encoded_audio_string); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_encoded_audio_string(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_encoded_audio_string( + ::std::string* encoded_audio_string); + + // string content_type = 5; + void clear_content_type(); + static const int kContentTypeFieldNumber = 5; + const ::std::string& content_type() const; + void set_content_type(const ::std::string& value); + #if LANG_CXX11 + void set_content_type(::std::string&& value); + #endif + void set_content_type(const char* value); + void set_content_type(const char* value, size_t size); + ::std::string* mutable_content_type(); + ::std::string* release_content_type(); + void set_allocated_content_type(::std::string* content_type); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_content_type(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_content_type( + ::std::string* content_type); + + // int64 num_channels = 2; + void clear_num_channels(); + static const int kNumChannelsFieldNumber = 2; + ::google::protobuf::int64 num_channels() const; + void set_num_channels(::google::protobuf::int64 value); + + // int64 length_frames = 3; + void clear_length_frames(); + static const int kLengthFramesFieldNumber = 3; + ::google::protobuf::int64 length_frames() const; + void set_length_frames(::google::protobuf::int64 value); + + // float sample_rate = 1; + void clear_sample_rate(); + static const int kSampleRateFieldNumber = 1; + float sample_rate() const; + void set_sample_rate(float value); + + // @@protoc_insertion_point(class_scope:tensorflow.Summary.Audio) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::internal::ArenaStringPtr encoded_audio_string_; + ::google::protobuf::internal::ArenaStringPtr content_type_; + ::google::protobuf::int64 num_channels_; + ::google::protobuf::int64 length_frames_; + float sample_rate_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fsummary_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class Summary_Value : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.Summary.Value) */ { + public: + Summary_Value(); + virtual ~Summary_Value(); + + Summary_Value(const Summary_Value& from); + + inline Summary_Value& operator=(const Summary_Value& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + Summary_Value(Summary_Value&& from) noexcept + : Summary_Value() { + *this = ::std::move(from); + } + + inline Summary_Value& operator=(Summary_Value&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const Summary_Value& default_instance(); + + enum ValueCase { + kSimpleValue = 2, + kObsoleteOldStyleHistogram = 3, + kImage = 4, + kHisto = 5, + kAudio = 6, + kTensor = 8, + VALUE_NOT_SET = 0, + }; + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const Summary_Value* internal_default_instance() { + return reinterpret_cast( + &_Summary_Value_default_instance_); + } + static constexpr int kIndexInFileMessages = + 6; + + void UnsafeArenaSwap(Summary_Value* other); + void Swap(Summary_Value* other); + friend void swap(Summary_Value& a, Summary_Value& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline Summary_Value* New() const final { + return CreateMaybeMessage(NULL); + } + + Summary_Value* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const Summary_Value& from); + void MergeFrom(const Summary_Value& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(Summary_Value* other); + protected: + explicit Summary_Value(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // string tag = 1; + void clear_tag(); + static const int kTagFieldNumber = 1; + const ::std::string& tag() const; + void set_tag(const ::std::string& value); + #if LANG_CXX11 + void set_tag(::std::string&& value); + #endif + void set_tag(const char* value); + void set_tag(const char* value, size_t size); + ::std::string* mutable_tag(); + ::std::string* release_tag(); + void set_allocated_tag(::std::string* tag); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_tag(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_tag( + ::std::string* tag); + + // string node_name = 7; + void clear_node_name(); + static const int kNodeNameFieldNumber = 7; + const ::std::string& node_name() const; + void set_node_name(const ::std::string& value); + #if LANG_CXX11 + void set_node_name(::std::string&& value); + #endif + void set_node_name(const char* value); + void set_node_name(const char* value, size_t size); + ::std::string* mutable_node_name(); + ::std::string* release_node_name(); + void set_allocated_node_name(::std::string* node_name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_node_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_node_name( + ::std::string* node_name); + + // .tensorflow.SummaryMetadata metadata = 9; + bool has_metadata() const; + void clear_metadata(); + static const int kMetadataFieldNumber = 9; + private: + const ::tensorflow::SummaryMetadata& _internal_metadata() const; + public: + const ::tensorflow::SummaryMetadata& metadata() const; + ::tensorflow::SummaryMetadata* release_metadata(); + ::tensorflow::SummaryMetadata* mutable_metadata(); + void set_allocated_metadata(::tensorflow::SummaryMetadata* metadata); + void unsafe_arena_set_allocated_metadata( + ::tensorflow::SummaryMetadata* metadata); + ::tensorflow::SummaryMetadata* unsafe_arena_release_metadata(); + + // float simple_value = 2; + private: + bool has_simple_value() const; + public: + void clear_simple_value(); + static const int kSimpleValueFieldNumber = 2; + float simple_value() const; + void set_simple_value(float value); + + // bytes obsolete_old_style_histogram = 3; + private: + bool has_obsolete_old_style_histogram() const; + public: + void clear_obsolete_old_style_histogram(); + static const int kObsoleteOldStyleHistogramFieldNumber = 3; + const ::std::string& obsolete_old_style_histogram() const; + void set_obsolete_old_style_histogram(const ::std::string& value); + #if LANG_CXX11 + void set_obsolete_old_style_histogram(::std::string&& value); + #endif + void set_obsolete_old_style_histogram(const char* value); + void set_obsolete_old_style_histogram(const void* value, size_t size); + ::std::string* mutable_obsolete_old_style_histogram(); + ::std::string* release_obsolete_old_style_histogram(); + void set_allocated_obsolete_old_style_histogram(::std::string* obsolete_old_style_histogram); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_obsolete_old_style_histogram(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_obsolete_old_style_histogram( + ::std::string* obsolete_old_style_histogram); + + // .tensorflow.Summary.Image image = 4; + bool has_image() const; + void clear_image(); + static const int kImageFieldNumber = 4; + private: + const ::tensorflow::Summary_Image& _internal_image() const; + public: + const ::tensorflow::Summary_Image& image() const; + ::tensorflow::Summary_Image* release_image(); + ::tensorflow::Summary_Image* mutable_image(); + void set_allocated_image(::tensorflow::Summary_Image* image); + void unsafe_arena_set_allocated_image( + ::tensorflow::Summary_Image* image); + ::tensorflow::Summary_Image* unsafe_arena_release_image(); + + // .tensorflow.HistogramProto histo = 5; + bool has_histo() const; + void clear_histo(); + static const int kHistoFieldNumber = 5; + private: + const ::tensorflow::HistogramProto& _internal_histo() const; + public: + const ::tensorflow::HistogramProto& histo() const; + ::tensorflow::HistogramProto* release_histo(); + ::tensorflow::HistogramProto* mutable_histo(); + void set_allocated_histo(::tensorflow::HistogramProto* histo); + void unsafe_arena_set_allocated_histo( + ::tensorflow::HistogramProto* histo); + ::tensorflow::HistogramProto* unsafe_arena_release_histo(); + + // .tensorflow.Summary.Audio audio = 6; + bool has_audio() const; + void clear_audio(); + static const int kAudioFieldNumber = 6; + private: + const ::tensorflow::Summary_Audio& _internal_audio() const; + public: + const ::tensorflow::Summary_Audio& audio() const; + ::tensorflow::Summary_Audio* release_audio(); + ::tensorflow::Summary_Audio* mutable_audio(); + void set_allocated_audio(::tensorflow::Summary_Audio* audio); + void unsafe_arena_set_allocated_audio( + ::tensorflow::Summary_Audio* audio); + ::tensorflow::Summary_Audio* unsafe_arena_release_audio(); + + // .tensorflow.TensorProto tensor = 8; + bool has_tensor() const; + void clear_tensor(); + static const int kTensorFieldNumber = 8; + private: + const ::tensorflow::TensorProto& _internal_tensor() const; + public: + const ::tensorflow::TensorProto& tensor() const; + ::tensorflow::TensorProto* release_tensor(); + ::tensorflow::TensorProto* mutable_tensor(); + void set_allocated_tensor(::tensorflow::TensorProto* tensor); + void unsafe_arena_set_allocated_tensor( + ::tensorflow::TensorProto* tensor); + ::tensorflow::TensorProto* unsafe_arena_release_tensor(); + + void clear_value(); + ValueCase value_case() const; + // @@protoc_insertion_point(class_scope:tensorflow.Summary.Value) + private: + void set_has_simple_value(); + void set_has_obsolete_old_style_histogram(); + void set_has_image(); + void set_has_histo(); + void set_has_audio(); + void set_has_tensor(); + + inline bool has_value() const; + inline void clear_has_value(); + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::internal::ArenaStringPtr tag_; + ::google::protobuf::internal::ArenaStringPtr node_name_; + ::tensorflow::SummaryMetadata* metadata_; + union ValueUnion { + ValueUnion() {} + float simple_value_; + ::google::protobuf::internal::ArenaStringPtr obsolete_old_style_histogram_; + ::tensorflow::Summary_Image* image_; + ::tensorflow::HistogramProto* histo_; + ::tensorflow::Summary_Audio* audio_; + ::tensorflow::TensorProto* tensor_; + } value_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + ::google::protobuf::uint32 _oneof_case_[1]; + + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fsummary_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class Summary : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.Summary) */ { + public: + Summary(); + virtual ~Summary(); + + Summary(const Summary& from); + + inline Summary& operator=(const Summary& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + Summary(Summary&& from) noexcept + : Summary() { + *this = ::std::move(from); + } + + inline Summary& operator=(Summary&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const Summary& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const Summary* internal_default_instance() { + return reinterpret_cast( + &_Summary_default_instance_); + } + static constexpr int kIndexInFileMessages = + 7; + + void UnsafeArenaSwap(Summary* other); + void Swap(Summary* other); + friend void swap(Summary& a, Summary& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline Summary* New() const final { + return CreateMaybeMessage(NULL); + } + + Summary* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const Summary& from); + void MergeFrom(const Summary& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(Summary* other); + protected: + explicit Summary(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + typedef Summary_Image Image; + typedef Summary_Audio Audio; + typedef Summary_Value Value; + + // accessors ------------------------------------------------------- + + // repeated .tensorflow.Summary.Value value = 1; + int value_size() const; + void clear_value(); + static const int kValueFieldNumber = 1; + ::tensorflow::Summary_Value* mutable_value(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::Summary_Value >* + mutable_value(); + const ::tensorflow::Summary_Value& value(int index) const; + ::tensorflow::Summary_Value* add_value(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::Summary_Value >& + value() const; + + // @@protoc_insertion_point(class_scope:tensorflow.Summary) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::Summary_Value > value_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fsummary_2eproto::TableStruct; +}; +// =================================================================== + + +// =================================================================== + +#ifdef __GNUC__ + #pragma GCC diagnostic push + #pragma GCC diagnostic ignored "-Wstrict-aliasing" +#endif // __GNUC__ +// SummaryDescription + +// string type_hint = 1; +inline void SummaryDescription::clear_type_hint() { + type_hint_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& SummaryDescription::type_hint() const { + // @@protoc_insertion_point(field_get:tensorflow.SummaryDescription.type_hint) + return type_hint_.Get(); +} +inline void SummaryDescription::set_type_hint(const ::std::string& value) { + + type_hint_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.SummaryDescription.type_hint) +} +#if LANG_CXX11 +inline void SummaryDescription::set_type_hint(::std::string&& value) { + + type_hint_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.SummaryDescription.type_hint) +} +#endif +inline void SummaryDescription::set_type_hint(const char* value) { + GOOGLE_DCHECK(value != NULL); + + type_hint_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.SummaryDescription.type_hint) +} +inline void SummaryDescription::set_type_hint(const char* value, + size_t size) { + + type_hint_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.SummaryDescription.type_hint) +} +inline ::std::string* SummaryDescription::mutable_type_hint() { + + // @@protoc_insertion_point(field_mutable:tensorflow.SummaryDescription.type_hint) + return type_hint_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* SummaryDescription::release_type_hint() { + // @@protoc_insertion_point(field_release:tensorflow.SummaryDescription.type_hint) + + return type_hint_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void SummaryDescription::set_allocated_type_hint(::std::string* type_hint) { + if (type_hint != NULL) { + + } else { + + } + type_hint_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), type_hint, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.SummaryDescription.type_hint) +} +inline ::std::string* SummaryDescription::unsafe_arena_release_type_hint() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.SummaryDescription.type_hint) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return type_hint_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void SummaryDescription::unsafe_arena_set_allocated_type_hint( + ::std::string* type_hint) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (type_hint != NULL) { + + } else { + + } + type_hint_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + type_hint, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.SummaryDescription.type_hint) +} + +// ------------------------------------------------------------------- + +// HistogramProto + +// double min = 1; +inline void HistogramProto::clear_min() { + min_ = 0; +} +inline double HistogramProto::min() const { + // @@protoc_insertion_point(field_get:tensorflow.HistogramProto.min) + return min_; +} +inline void HistogramProto::set_min(double value) { + + min_ = value; + // @@protoc_insertion_point(field_set:tensorflow.HistogramProto.min) +} + +// double max = 2; +inline void HistogramProto::clear_max() { + max_ = 0; +} +inline double HistogramProto::max() const { + // @@protoc_insertion_point(field_get:tensorflow.HistogramProto.max) + return max_; +} +inline void HistogramProto::set_max(double value) { + + max_ = value; + // @@protoc_insertion_point(field_set:tensorflow.HistogramProto.max) +} + +// double num = 3; +inline void HistogramProto::clear_num() { + num_ = 0; +} +inline double HistogramProto::num() const { + // @@protoc_insertion_point(field_get:tensorflow.HistogramProto.num) + return num_; +} +inline void HistogramProto::set_num(double value) { + + num_ = value; + // @@protoc_insertion_point(field_set:tensorflow.HistogramProto.num) +} + +// double sum = 4; +inline void HistogramProto::clear_sum() { + sum_ = 0; +} +inline double HistogramProto::sum() const { + // @@protoc_insertion_point(field_get:tensorflow.HistogramProto.sum) + return sum_; +} +inline void HistogramProto::set_sum(double value) { + + sum_ = value; + // @@protoc_insertion_point(field_set:tensorflow.HistogramProto.sum) +} + +// double sum_squares = 5; +inline void HistogramProto::clear_sum_squares() { + sum_squares_ = 0; +} +inline double HistogramProto::sum_squares() const { + // @@protoc_insertion_point(field_get:tensorflow.HistogramProto.sum_squares) + return sum_squares_; +} +inline void HistogramProto::set_sum_squares(double value) { + + sum_squares_ = value; + // @@protoc_insertion_point(field_set:tensorflow.HistogramProto.sum_squares) +} + +// repeated double bucket_limit = 6 [packed = true]; +inline int HistogramProto::bucket_limit_size() const { + return bucket_limit_.size(); +} +inline void HistogramProto::clear_bucket_limit() { + bucket_limit_.Clear(); +} +inline double HistogramProto::bucket_limit(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.HistogramProto.bucket_limit) + return bucket_limit_.Get(index); +} +inline void HistogramProto::set_bucket_limit(int index, double value) { + bucket_limit_.Set(index, value); + // @@protoc_insertion_point(field_set:tensorflow.HistogramProto.bucket_limit) +} +inline void HistogramProto::add_bucket_limit(double value) { + bucket_limit_.Add(value); + // @@protoc_insertion_point(field_add:tensorflow.HistogramProto.bucket_limit) +} +inline const ::google::protobuf::RepeatedField< double >& +HistogramProto::bucket_limit() const { + // @@protoc_insertion_point(field_list:tensorflow.HistogramProto.bucket_limit) + return bucket_limit_; +} +inline ::google::protobuf::RepeatedField< double >* +HistogramProto::mutable_bucket_limit() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.HistogramProto.bucket_limit) + return &bucket_limit_; +} + +// repeated double bucket = 7 [packed = true]; +inline int HistogramProto::bucket_size() const { + return bucket_.size(); +} +inline void HistogramProto::clear_bucket() { + bucket_.Clear(); +} +inline double HistogramProto::bucket(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.HistogramProto.bucket) + return bucket_.Get(index); +} +inline void HistogramProto::set_bucket(int index, double value) { + bucket_.Set(index, value); + // @@protoc_insertion_point(field_set:tensorflow.HistogramProto.bucket) +} +inline void HistogramProto::add_bucket(double value) { + bucket_.Add(value); + // @@protoc_insertion_point(field_add:tensorflow.HistogramProto.bucket) +} +inline const ::google::protobuf::RepeatedField< double >& +HistogramProto::bucket() const { + // @@protoc_insertion_point(field_list:tensorflow.HistogramProto.bucket) + return bucket_; +} +inline ::google::protobuf::RepeatedField< double >* +HistogramProto::mutable_bucket() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.HistogramProto.bucket) + return &bucket_; +} + +// ------------------------------------------------------------------- + +// SummaryMetadata_PluginData + +// string plugin_name = 1; +inline void SummaryMetadata_PluginData::clear_plugin_name() { + plugin_name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& SummaryMetadata_PluginData::plugin_name() const { + // @@protoc_insertion_point(field_get:tensorflow.SummaryMetadata.PluginData.plugin_name) + return plugin_name_.Get(); +} +inline void SummaryMetadata_PluginData::set_plugin_name(const ::std::string& value) { + + plugin_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.SummaryMetadata.PluginData.plugin_name) +} +#if LANG_CXX11 +inline void SummaryMetadata_PluginData::set_plugin_name(::std::string&& value) { + + plugin_name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.SummaryMetadata.PluginData.plugin_name) +} +#endif +inline void SummaryMetadata_PluginData::set_plugin_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + plugin_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.SummaryMetadata.PluginData.plugin_name) +} +inline void SummaryMetadata_PluginData::set_plugin_name(const char* value, + size_t size) { + + plugin_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.SummaryMetadata.PluginData.plugin_name) +} +inline ::std::string* SummaryMetadata_PluginData::mutable_plugin_name() { + + // @@protoc_insertion_point(field_mutable:tensorflow.SummaryMetadata.PluginData.plugin_name) + return plugin_name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* SummaryMetadata_PluginData::release_plugin_name() { + // @@protoc_insertion_point(field_release:tensorflow.SummaryMetadata.PluginData.plugin_name) + + return plugin_name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void SummaryMetadata_PluginData::set_allocated_plugin_name(::std::string* plugin_name) { + if (plugin_name != NULL) { + + } else { + + } + plugin_name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), plugin_name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.SummaryMetadata.PluginData.plugin_name) +} +inline ::std::string* SummaryMetadata_PluginData::unsafe_arena_release_plugin_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.SummaryMetadata.PluginData.plugin_name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return plugin_name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void SummaryMetadata_PluginData::unsafe_arena_set_allocated_plugin_name( + ::std::string* plugin_name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (plugin_name != NULL) { + + } else { + + } + plugin_name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + plugin_name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.SummaryMetadata.PluginData.plugin_name) +} + +// bytes content = 2; +inline void SummaryMetadata_PluginData::clear_content() { + content_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& SummaryMetadata_PluginData::content() const { + // @@protoc_insertion_point(field_get:tensorflow.SummaryMetadata.PluginData.content) + return content_.Get(); +} +inline void SummaryMetadata_PluginData::set_content(const ::std::string& value) { + + content_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.SummaryMetadata.PluginData.content) +} +#if LANG_CXX11 +inline void SummaryMetadata_PluginData::set_content(::std::string&& value) { + + content_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.SummaryMetadata.PluginData.content) +} +#endif +inline void SummaryMetadata_PluginData::set_content(const char* value) { + GOOGLE_DCHECK(value != NULL); + + content_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.SummaryMetadata.PluginData.content) +} +inline void SummaryMetadata_PluginData::set_content(const void* value, + size_t size) { + + content_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.SummaryMetadata.PluginData.content) +} +inline ::std::string* SummaryMetadata_PluginData::mutable_content() { + + // @@protoc_insertion_point(field_mutable:tensorflow.SummaryMetadata.PluginData.content) + return content_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* SummaryMetadata_PluginData::release_content() { + // @@protoc_insertion_point(field_release:tensorflow.SummaryMetadata.PluginData.content) + + return content_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void SummaryMetadata_PluginData::set_allocated_content(::std::string* content) { + if (content != NULL) { + + } else { + + } + content_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), content, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.SummaryMetadata.PluginData.content) +} +inline ::std::string* SummaryMetadata_PluginData::unsafe_arena_release_content() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.SummaryMetadata.PluginData.content) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return content_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void SummaryMetadata_PluginData::unsafe_arena_set_allocated_content( + ::std::string* content) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (content != NULL) { + + } else { + + } + content_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + content, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.SummaryMetadata.PluginData.content) +} + +// ------------------------------------------------------------------- + +// SummaryMetadata + +// .tensorflow.SummaryMetadata.PluginData plugin_data = 1; +inline bool SummaryMetadata::has_plugin_data() const { + return this != internal_default_instance() && plugin_data_ != NULL; +} +inline void SummaryMetadata::clear_plugin_data() { + if (GetArenaNoVirtual() == NULL && plugin_data_ != NULL) { + delete plugin_data_; + } + plugin_data_ = NULL; +} +inline const ::tensorflow::SummaryMetadata_PluginData& SummaryMetadata::_internal_plugin_data() const { + return *plugin_data_; +} +inline const ::tensorflow::SummaryMetadata_PluginData& SummaryMetadata::plugin_data() const { + const ::tensorflow::SummaryMetadata_PluginData* p = plugin_data_; + // @@protoc_insertion_point(field_get:tensorflow.SummaryMetadata.plugin_data) + return p != NULL ? *p : *reinterpret_cast( + &::tensorflow::_SummaryMetadata_PluginData_default_instance_); +} +inline ::tensorflow::SummaryMetadata_PluginData* SummaryMetadata::release_plugin_data() { + // @@protoc_insertion_point(field_release:tensorflow.SummaryMetadata.plugin_data) + + ::tensorflow::SummaryMetadata_PluginData* temp = plugin_data_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + plugin_data_ = NULL; + return temp; +} +inline ::tensorflow::SummaryMetadata_PluginData* SummaryMetadata::unsafe_arena_release_plugin_data() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.SummaryMetadata.plugin_data) + + ::tensorflow::SummaryMetadata_PluginData* temp = plugin_data_; + plugin_data_ = NULL; + return temp; +} +inline ::tensorflow::SummaryMetadata_PluginData* SummaryMetadata::mutable_plugin_data() { + + if (plugin_data_ == NULL) { + auto* p = CreateMaybeMessage<::tensorflow::SummaryMetadata_PluginData>(GetArenaNoVirtual()); + plugin_data_ = p; + } + // @@protoc_insertion_point(field_mutable:tensorflow.SummaryMetadata.plugin_data) + return plugin_data_; +} +inline void SummaryMetadata::set_allocated_plugin_data(::tensorflow::SummaryMetadata_PluginData* plugin_data) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete plugin_data_; + } + if (plugin_data) { + ::google::protobuf::Arena* submessage_arena = + ::google::protobuf::Arena::GetArena(plugin_data); + if (message_arena != submessage_arena) { + plugin_data = ::google::protobuf::internal::GetOwnedMessage( + message_arena, plugin_data, submessage_arena); + } + + } else { + + } + plugin_data_ = plugin_data; + // @@protoc_insertion_point(field_set_allocated:tensorflow.SummaryMetadata.plugin_data) +} + +// string display_name = 2; +inline void SummaryMetadata::clear_display_name() { + display_name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& SummaryMetadata::display_name() const { + // @@protoc_insertion_point(field_get:tensorflow.SummaryMetadata.display_name) + return display_name_.Get(); +} +inline void SummaryMetadata::set_display_name(const ::std::string& value) { + + display_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.SummaryMetadata.display_name) +} +#if LANG_CXX11 +inline void SummaryMetadata::set_display_name(::std::string&& value) { + + display_name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.SummaryMetadata.display_name) +} +#endif +inline void SummaryMetadata::set_display_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + display_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.SummaryMetadata.display_name) +} +inline void SummaryMetadata::set_display_name(const char* value, + size_t size) { + + display_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.SummaryMetadata.display_name) +} +inline ::std::string* SummaryMetadata::mutable_display_name() { + + // @@protoc_insertion_point(field_mutable:tensorflow.SummaryMetadata.display_name) + return display_name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* SummaryMetadata::release_display_name() { + // @@protoc_insertion_point(field_release:tensorflow.SummaryMetadata.display_name) + + return display_name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void SummaryMetadata::set_allocated_display_name(::std::string* display_name) { + if (display_name != NULL) { + + } else { + + } + display_name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), display_name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.SummaryMetadata.display_name) +} +inline ::std::string* SummaryMetadata::unsafe_arena_release_display_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.SummaryMetadata.display_name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return display_name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void SummaryMetadata::unsafe_arena_set_allocated_display_name( + ::std::string* display_name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (display_name != NULL) { + + } else { + + } + display_name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + display_name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.SummaryMetadata.display_name) +} + +// string summary_description = 3; +inline void SummaryMetadata::clear_summary_description() { + summary_description_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& SummaryMetadata::summary_description() const { + // @@protoc_insertion_point(field_get:tensorflow.SummaryMetadata.summary_description) + return summary_description_.Get(); +} +inline void SummaryMetadata::set_summary_description(const ::std::string& value) { + + summary_description_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.SummaryMetadata.summary_description) +} +#if LANG_CXX11 +inline void SummaryMetadata::set_summary_description(::std::string&& value) { + + summary_description_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.SummaryMetadata.summary_description) +} +#endif +inline void SummaryMetadata::set_summary_description(const char* value) { + GOOGLE_DCHECK(value != NULL); + + summary_description_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.SummaryMetadata.summary_description) +} +inline void SummaryMetadata::set_summary_description(const char* value, + size_t size) { + + summary_description_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.SummaryMetadata.summary_description) +} +inline ::std::string* SummaryMetadata::mutable_summary_description() { + + // @@protoc_insertion_point(field_mutable:tensorflow.SummaryMetadata.summary_description) + return summary_description_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* SummaryMetadata::release_summary_description() { + // @@protoc_insertion_point(field_release:tensorflow.SummaryMetadata.summary_description) + + return summary_description_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void SummaryMetadata::set_allocated_summary_description(::std::string* summary_description) { + if (summary_description != NULL) { + + } else { + + } + summary_description_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), summary_description, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.SummaryMetadata.summary_description) +} +inline ::std::string* SummaryMetadata::unsafe_arena_release_summary_description() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.SummaryMetadata.summary_description) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return summary_description_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void SummaryMetadata::unsafe_arena_set_allocated_summary_description( + ::std::string* summary_description) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (summary_description != NULL) { + + } else { + + } + summary_description_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + summary_description, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.SummaryMetadata.summary_description) +} + +// ------------------------------------------------------------------- + +// Summary_Image + +// int32 height = 1; +inline void Summary_Image::clear_height() { + height_ = 0; +} +inline ::google::protobuf::int32 Summary_Image::height() const { + // @@protoc_insertion_point(field_get:tensorflow.Summary.Image.height) + return height_; +} +inline void Summary_Image::set_height(::google::protobuf::int32 value) { + + height_ = value; + // @@protoc_insertion_point(field_set:tensorflow.Summary.Image.height) +} + +// int32 width = 2; +inline void Summary_Image::clear_width() { + width_ = 0; +} +inline ::google::protobuf::int32 Summary_Image::width() const { + // @@protoc_insertion_point(field_get:tensorflow.Summary.Image.width) + return width_; +} +inline void Summary_Image::set_width(::google::protobuf::int32 value) { + + width_ = value; + // @@protoc_insertion_point(field_set:tensorflow.Summary.Image.width) +} + +// int32 colorspace = 3; +inline void Summary_Image::clear_colorspace() { + colorspace_ = 0; +} +inline ::google::protobuf::int32 Summary_Image::colorspace() const { + // @@protoc_insertion_point(field_get:tensorflow.Summary.Image.colorspace) + return colorspace_; +} +inline void Summary_Image::set_colorspace(::google::protobuf::int32 value) { + + colorspace_ = value; + // @@protoc_insertion_point(field_set:tensorflow.Summary.Image.colorspace) +} + +// bytes encoded_image_string = 4; +inline void Summary_Image::clear_encoded_image_string() { + encoded_image_string_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& Summary_Image::encoded_image_string() const { + // @@protoc_insertion_point(field_get:tensorflow.Summary.Image.encoded_image_string) + return encoded_image_string_.Get(); +} +inline void Summary_Image::set_encoded_image_string(const ::std::string& value) { + + encoded_image_string_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.Summary.Image.encoded_image_string) +} +#if LANG_CXX11 +inline void Summary_Image::set_encoded_image_string(::std::string&& value) { + + encoded_image_string_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.Summary.Image.encoded_image_string) +} +#endif +inline void Summary_Image::set_encoded_image_string(const char* value) { + GOOGLE_DCHECK(value != NULL); + + encoded_image_string_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.Summary.Image.encoded_image_string) +} +inline void Summary_Image::set_encoded_image_string(const void* value, + size_t size) { + + encoded_image_string_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.Summary.Image.encoded_image_string) +} +inline ::std::string* Summary_Image::mutable_encoded_image_string() { + + // @@protoc_insertion_point(field_mutable:tensorflow.Summary.Image.encoded_image_string) + return encoded_image_string_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* Summary_Image::release_encoded_image_string() { + // @@protoc_insertion_point(field_release:tensorflow.Summary.Image.encoded_image_string) + + return encoded_image_string_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void Summary_Image::set_allocated_encoded_image_string(::std::string* encoded_image_string) { + if (encoded_image_string != NULL) { + + } else { + + } + encoded_image_string_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), encoded_image_string, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.Summary.Image.encoded_image_string) +} +inline ::std::string* Summary_Image::unsafe_arena_release_encoded_image_string() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.Summary.Image.encoded_image_string) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return encoded_image_string_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void Summary_Image::unsafe_arena_set_allocated_encoded_image_string( + ::std::string* encoded_image_string) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (encoded_image_string != NULL) { + + } else { + + } + encoded_image_string_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + encoded_image_string, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.Summary.Image.encoded_image_string) +} + +// ------------------------------------------------------------------- + +// Summary_Audio + +// float sample_rate = 1; +inline void Summary_Audio::clear_sample_rate() { + sample_rate_ = 0; +} +inline float Summary_Audio::sample_rate() const { + // @@protoc_insertion_point(field_get:tensorflow.Summary.Audio.sample_rate) + return sample_rate_; +} +inline void Summary_Audio::set_sample_rate(float value) { + + sample_rate_ = value; + // @@protoc_insertion_point(field_set:tensorflow.Summary.Audio.sample_rate) +} + +// int64 num_channels = 2; +inline void Summary_Audio::clear_num_channels() { + num_channels_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 Summary_Audio::num_channels() const { + // @@protoc_insertion_point(field_get:tensorflow.Summary.Audio.num_channels) + return num_channels_; +} +inline void Summary_Audio::set_num_channels(::google::protobuf::int64 value) { + + num_channels_ = value; + // @@protoc_insertion_point(field_set:tensorflow.Summary.Audio.num_channels) +} + +// int64 length_frames = 3; +inline void Summary_Audio::clear_length_frames() { + length_frames_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 Summary_Audio::length_frames() const { + // @@protoc_insertion_point(field_get:tensorflow.Summary.Audio.length_frames) + return length_frames_; +} +inline void Summary_Audio::set_length_frames(::google::protobuf::int64 value) { + + length_frames_ = value; + // @@protoc_insertion_point(field_set:tensorflow.Summary.Audio.length_frames) +} + +// bytes encoded_audio_string = 4; +inline void Summary_Audio::clear_encoded_audio_string() { + encoded_audio_string_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& Summary_Audio::encoded_audio_string() const { + // @@protoc_insertion_point(field_get:tensorflow.Summary.Audio.encoded_audio_string) + return encoded_audio_string_.Get(); +} +inline void Summary_Audio::set_encoded_audio_string(const ::std::string& value) { + + encoded_audio_string_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.Summary.Audio.encoded_audio_string) +} +#if LANG_CXX11 +inline void Summary_Audio::set_encoded_audio_string(::std::string&& value) { + + encoded_audio_string_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.Summary.Audio.encoded_audio_string) +} +#endif +inline void Summary_Audio::set_encoded_audio_string(const char* value) { + GOOGLE_DCHECK(value != NULL); + + encoded_audio_string_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.Summary.Audio.encoded_audio_string) +} +inline void Summary_Audio::set_encoded_audio_string(const void* value, + size_t size) { + + encoded_audio_string_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.Summary.Audio.encoded_audio_string) +} +inline ::std::string* Summary_Audio::mutable_encoded_audio_string() { + + // @@protoc_insertion_point(field_mutable:tensorflow.Summary.Audio.encoded_audio_string) + return encoded_audio_string_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* Summary_Audio::release_encoded_audio_string() { + // @@protoc_insertion_point(field_release:tensorflow.Summary.Audio.encoded_audio_string) + + return encoded_audio_string_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void Summary_Audio::set_allocated_encoded_audio_string(::std::string* encoded_audio_string) { + if (encoded_audio_string != NULL) { + + } else { + + } + encoded_audio_string_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), encoded_audio_string, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.Summary.Audio.encoded_audio_string) +} +inline ::std::string* Summary_Audio::unsafe_arena_release_encoded_audio_string() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.Summary.Audio.encoded_audio_string) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return encoded_audio_string_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void Summary_Audio::unsafe_arena_set_allocated_encoded_audio_string( + ::std::string* encoded_audio_string) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (encoded_audio_string != NULL) { + + } else { + + } + encoded_audio_string_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + encoded_audio_string, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.Summary.Audio.encoded_audio_string) +} + +// string content_type = 5; +inline void Summary_Audio::clear_content_type() { + content_type_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& Summary_Audio::content_type() const { + // @@protoc_insertion_point(field_get:tensorflow.Summary.Audio.content_type) + return content_type_.Get(); +} +inline void Summary_Audio::set_content_type(const ::std::string& value) { + + content_type_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.Summary.Audio.content_type) +} +#if LANG_CXX11 +inline void Summary_Audio::set_content_type(::std::string&& value) { + + content_type_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.Summary.Audio.content_type) +} +#endif +inline void Summary_Audio::set_content_type(const char* value) { + GOOGLE_DCHECK(value != NULL); + + content_type_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.Summary.Audio.content_type) +} +inline void Summary_Audio::set_content_type(const char* value, + size_t size) { + + content_type_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.Summary.Audio.content_type) +} +inline ::std::string* Summary_Audio::mutable_content_type() { + + // @@protoc_insertion_point(field_mutable:tensorflow.Summary.Audio.content_type) + return content_type_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* Summary_Audio::release_content_type() { + // @@protoc_insertion_point(field_release:tensorflow.Summary.Audio.content_type) + + return content_type_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void Summary_Audio::set_allocated_content_type(::std::string* content_type) { + if (content_type != NULL) { + + } else { + + } + content_type_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), content_type, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.Summary.Audio.content_type) +} +inline ::std::string* Summary_Audio::unsafe_arena_release_content_type() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.Summary.Audio.content_type) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return content_type_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void Summary_Audio::unsafe_arena_set_allocated_content_type( + ::std::string* content_type) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (content_type != NULL) { + + } else { + + } + content_type_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + content_type, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.Summary.Audio.content_type) +} + +// ------------------------------------------------------------------- + +// Summary_Value + +// string node_name = 7; +inline void Summary_Value::clear_node_name() { + node_name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& Summary_Value::node_name() const { + // @@protoc_insertion_point(field_get:tensorflow.Summary.Value.node_name) + return node_name_.Get(); +} +inline void Summary_Value::set_node_name(const ::std::string& value) { + + node_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.Summary.Value.node_name) +} +#if LANG_CXX11 +inline void Summary_Value::set_node_name(::std::string&& value) { + + node_name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.Summary.Value.node_name) +} +#endif +inline void Summary_Value::set_node_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + node_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.Summary.Value.node_name) +} +inline void Summary_Value::set_node_name(const char* value, + size_t size) { + + node_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.Summary.Value.node_name) +} +inline ::std::string* Summary_Value::mutable_node_name() { + + // @@protoc_insertion_point(field_mutable:tensorflow.Summary.Value.node_name) + return node_name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* Summary_Value::release_node_name() { + // @@protoc_insertion_point(field_release:tensorflow.Summary.Value.node_name) + + return node_name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void Summary_Value::set_allocated_node_name(::std::string* node_name) { + if (node_name != NULL) { + + } else { + + } + node_name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), node_name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.Summary.Value.node_name) +} +inline ::std::string* Summary_Value::unsafe_arena_release_node_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.Summary.Value.node_name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return node_name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void Summary_Value::unsafe_arena_set_allocated_node_name( + ::std::string* node_name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (node_name != NULL) { + + } else { + + } + node_name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + node_name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.Summary.Value.node_name) +} + +// string tag = 1; +inline void Summary_Value::clear_tag() { + tag_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& Summary_Value::tag() const { + // @@protoc_insertion_point(field_get:tensorflow.Summary.Value.tag) + return tag_.Get(); +} +inline void Summary_Value::set_tag(const ::std::string& value) { + + tag_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.Summary.Value.tag) +} +#if LANG_CXX11 +inline void Summary_Value::set_tag(::std::string&& value) { + + tag_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.Summary.Value.tag) +} +#endif +inline void Summary_Value::set_tag(const char* value) { + GOOGLE_DCHECK(value != NULL); + + tag_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.Summary.Value.tag) +} +inline void Summary_Value::set_tag(const char* value, + size_t size) { + + tag_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.Summary.Value.tag) +} +inline ::std::string* Summary_Value::mutable_tag() { + + // @@protoc_insertion_point(field_mutable:tensorflow.Summary.Value.tag) + return tag_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* Summary_Value::release_tag() { + // @@protoc_insertion_point(field_release:tensorflow.Summary.Value.tag) + + return tag_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void Summary_Value::set_allocated_tag(::std::string* tag) { + if (tag != NULL) { + + } else { + + } + tag_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), tag, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.Summary.Value.tag) +} +inline ::std::string* Summary_Value::unsafe_arena_release_tag() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.Summary.Value.tag) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return tag_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void Summary_Value::unsafe_arena_set_allocated_tag( + ::std::string* tag) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (tag != NULL) { + + } else { + + } + tag_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + tag, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.Summary.Value.tag) +} + +// .tensorflow.SummaryMetadata metadata = 9; +inline bool Summary_Value::has_metadata() const { + return this != internal_default_instance() && metadata_ != NULL; +} +inline void Summary_Value::clear_metadata() { + if (GetArenaNoVirtual() == NULL && metadata_ != NULL) { + delete metadata_; + } + metadata_ = NULL; +} +inline const ::tensorflow::SummaryMetadata& Summary_Value::_internal_metadata() const { + return *metadata_; +} +inline const ::tensorflow::SummaryMetadata& Summary_Value::metadata() const { + const ::tensorflow::SummaryMetadata* p = metadata_; + // @@protoc_insertion_point(field_get:tensorflow.Summary.Value.metadata) + return p != NULL ? *p : *reinterpret_cast( + &::tensorflow::_SummaryMetadata_default_instance_); +} +inline ::tensorflow::SummaryMetadata* Summary_Value::release_metadata() { + // @@protoc_insertion_point(field_release:tensorflow.Summary.Value.metadata) + + ::tensorflow::SummaryMetadata* temp = metadata_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + metadata_ = NULL; + return temp; +} +inline ::tensorflow::SummaryMetadata* Summary_Value::unsafe_arena_release_metadata() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.Summary.Value.metadata) + + ::tensorflow::SummaryMetadata* temp = metadata_; + metadata_ = NULL; + return temp; +} +inline ::tensorflow::SummaryMetadata* Summary_Value::mutable_metadata() { + + if (metadata_ == NULL) { + auto* p = CreateMaybeMessage<::tensorflow::SummaryMetadata>(GetArenaNoVirtual()); + metadata_ = p; + } + // @@protoc_insertion_point(field_mutable:tensorflow.Summary.Value.metadata) + return metadata_; +} +inline void Summary_Value::set_allocated_metadata(::tensorflow::SummaryMetadata* metadata) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete metadata_; + } + if (metadata) { + ::google::protobuf::Arena* submessage_arena = + ::google::protobuf::Arena::GetArena(metadata); + if (message_arena != submessage_arena) { + metadata = ::google::protobuf::internal::GetOwnedMessage( + message_arena, metadata, submessage_arena); + } + + } else { + + } + metadata_ = metadata; + // @@protoc_insertion_point(field_set_allocated:tensorflow.Summary.Value.metadata) +} + +// float simple_value = 2; +inline bool Summary_Value::has_simple_value() const { + return value_case() == kSimpleValue; +} +inline void Summary_Value::set_has_simple_value() { + _oneof_case_[0] = kSimpleValue; +} +inline void Summary_Value::clear_simple_value() { + if (has_simple_value()) { + value_.simple_value_ = 0; + clear_has_value(); + } +} +inline float Summary_Value::simple_value() const { + // @@protoc_insertion_point(field_get:tensorflow.Summary.Value.simple_value) + if (has_simple_value()) { + return value_.simple_value_; + } + return 0; +} +inline void Summary_Value::set_simple_value(float value) { + if (!has_simple_value()) { + clear_value(); + set_has_simple_value(); + } + value_.simple_value_ = value; + // @@protoc_insertion_point(field_set:tensorflow.Summary.Value.simple_value) +} + +// bytes obsolete_old_style_histogram = 3; +inline bool Summary_Value::has_obsolete_old_style_histogram() const { + return value_case() == kObsoleteOldStyleHistogram; +} +inline void Summary_Value::set_has_obsolete_old_style_histogram() { + _oneof_case_[0] = kObsoleteOldStyleHistogram; +} +inline void Summary_Value::clear_obsolete_old_style_histogram() { + if (has_obsolete_old_style_histogram()) { + value_.obsolete_old_style_histogram_.Destroy(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); + clear_has_value(); + } +} +inline const ::std::string& Summary_Value::obsolete_old_style_histogram() const { + // @@protoc_insertion_point(field_get:tensorflow.Summary.Value.obsolete_old_style_histogram) + if (has_obsolete_old_style_histogram()) { + return value_.obsolete_old_style_histogram_.Get(); + } + return *&::google::protobuf::internal::GetEmptyStringAlreadyInited(); +} +inline void Summary_Value::set_obsolete_old_style_histogram(const ::std::string& value) { + if (!has_obsolete_old_style_histogram()) { + clear_value(); + set_has_obsolete_old_style_histogram(); + value_.obsolete_old_style_histogram_.UnsafeSetDefault(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); + } + value_.obsolete_old_style_histogram_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.Summary.Value.obsolete_old_style_histogram) +} +#if LANG_CXX11 +inline void Summary_Value::set_obsolete_old_style_histogram(::std::string&& value) { + // @@protoc_insertion_point(field_set:tensorflow.Summary.Value.obsolete_old_style_histogram) + if (!has_obsolete_old_style_histogram()) { + clear_value(); + set_has_obsolete_old_style_histogram(); + value_.obsolete_old_style_histogram_.UnsafeSetDefault(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); + } + value_.obsolete_old_style_histogram_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.Summary.Value.obsolete_old_style_histogram) +} +#endif +inline void Summary_Value::set_obsolete_old_style_histogram(const char* value) { + GOOGLE_DCHECK(value != NULL); + if (!has_obsolete_old_style_histogram()) { + clear_value(); + set_has_obsolete_old_style_histogram(); + value_.obsolete_old_style_histogram_.UnsafeSetDefault(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); + } + value_.obsolete_old_style_histogram_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + ::std::string(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.Summary.Value.obsolete_old_style_histogram) +} +inline void Summary_Value::set_obsolete_old_style_histogram(const void* value, + size_t size) { + if (!has_obsolete_old_style_histogram()) { + clear_value(); + set_has_obsolete_old_style_histogram(); + value_.obsolete_old_style_histogram_.UnsafeSetDefault(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); + } + value_.obsolete_old_style_histogram_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.Summary.Value.obsolete_old_style_histogram) +} +inline ::std::string* Summary_Value::mutable_obsolete_old_style_histogram() { + if (!has_obsolete_old_style_histogram()) { + clear_value(); + set_has_obsolete_old_style_histogram(); + value_.obsolete_old_style_histogram_.UnsafeSetDefault(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); + } + return value_.obsolete_old_style_histogram_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_mutable:tensorflow.Summary.Value.obsolete_old_style_histogram) +} +inline ::std::string* Summary_Value::release_obsolete_old_style_histogram() { + // @@protoc_insertion_point(field_release:tensorflow.Summary.Value.obsolete_old_style_histogram) + if (has_obsolete_old_style_histogram()) { + clear_has_value(); + return value_.obsolete_old_style_histogram_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); + } else { + return NULL; + } +} +inline void Summary_Value::set_allocated_obsolete_old_style_histogram(::std::string* obsolete_old_style_histogram) { + if (!has_obsolete_old_style_histogram()) { + value_.obsolete_old_style_histogram_.UnsafeSetDefault(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); + } + clear_value(); + if (obsolete_old_style_histogram != NULL) { + set_has_obsolete_old_style_histogram(); + value_.obsolete_old_style_histogram_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), obsolete_old_style_histogram, + GetArenaNoVirtual()); + } + // @@protoc_insertion_point(field_set_allocated:tensorflow.Summary.Value.obsolete_old_style_histogram) +} +inline ::std::string* Summary_Value::unsafe_arena_release_obsolete_old_style_histogram() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.Summary.Value.obsolete_old_style_histogram) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (has_obsolete_old_style_histogram()) { + clear_has_value(); + return value_.obsolete_old_style_histogram_.UnsafeArenaRelease( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); + } else { + return NULL; + } +} +inline void Summary_Value::unsafe_arena_set_allocated_obsolete_old_style_histogram(::std::string* obsolete_old_style_histogram) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (!has_obsolete_old_style_histogram()) { + value_.obsolete_old_style_histogram_.UnsafeSetDefault(&::google::protobuf::internal::GetEmptyStringAlreadyInited()); + } + clear_value(); + if (obsolete_old_style_histogram) { + set_has_obsolete_old_style_histogram(); + value_.obsolete_old_style_histogram_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), obsolete_old_style_histogram, GetArenaNoVirtual()); + } + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.Summary.Value.obsolete_old_style_histogram) +} + +// .tensorflow.Summary.Image image = 4; +inline bool Summary_Value::has_image() const { + return value_case() == kImage; +} +inline void Summary_Value::set_has_image() { + _oneof_case_[0] = kImage; +} +inline void Summary_Value::clear_image() { + if (has_image()) { + if (GetArenaNoVirtual() == NULL) { + delete value_.image_; + } + clear_has_value(); + } +} +inline const ::tensorflow::Summary_Image& Summary_Value::_internal_image() const { + return *value_.image_; +} +inline ::tensorflow::Summary_Image* Summary_Value::release_image() { + // @@protoc_insertion_point(field_release:tensorflow.Summary.Value.image) + if (has_image()) { + clear_has_value(); + ::tensorflow::Summary_Image* temp = value_.image_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + value_.image_ = NULL; + return temp; + } else { + return NULL; + } +} +inline const ::tensorflow::Summary_Image& Summary_Value::image() const { + // @@protoc_insertion_point(field_get:tensorflow.Summary.Value.image) + return has_image() + ? *value_.image_ + : *reinterpret_cast< ::tensorflow::Summary_Image*>(&::tensorflow::_Summary_Image_default_instance_); +} +inline ::tensorflow::Summary_Image* Summary_Value::unsafe_arena_release_image() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.Summary.Value.image) + if (has_image()) { + clear_has_value(); + ::tensorflow::Summary_Image* temp = value_.image_; + value_.image_ = NULL; + return temp; + } else { + return NULL; + } +} +inline void Summary_Value::unsafe_arena_set_allocated_image(::tensorflow::Summary_Image* image) { + clear_value(); + if (image) { + set_has_image(); + value_.image_ = image; + } + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.Summary.Value.image) +} +inline ::tensorflow::Summary_Image* Summary_Value::mutable_image() { + if (!has_image()) { + clear_value(); + set_has_image(); + value_.image_ = CreateMaybeMessage< ::tensorflow::Summary_Image >( + GetArenaNoVirtual()); + } + // @@protoc_insertion_point(field_mutable:tensorflow.Summary.Value.image) + return value_.image_; +} + +// .tensorflow.HistogramProto histo = 5; +inline bool Summary_Value::has_histo() const { + return value_case() == kHisto; +} +inline void Summary_Value::set_has_histo() { + _oneof_case_[0] = kHisto; +} +inline void Summary_Value::clear_histo() { + if (has_histo()) { + if (GetArenaNoVirtual() == NULL) { + delete value_.histo_; + } + clear_has_value(); + } +} +inline const ::tensorflow::HistogramProto& Summary_Value::_internal_histo() const { + return *value_.histo_; +} +inline ::tensorflow::HistogramProto* Summary_Value::release_histo() { + // @@protoc_insertion_point(field_release:tensorflow.Summary.Value.histo) + if (has_histo()) { + clear_has_value(); + ::tensorflow::HistogramProto* temp = value_.histo_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + value_.histo_ = NULL; + return temp; + } else { + return NULL; + } +} +inline const ::tensorflow::HistogramProto& Summary_Value::histo() const { + // @@protoc_insertion_point(field_get:tensorflow.Summary.Value.histo) + return has_histo() + ? *value_.histo_ + : *reinterpret_cast< ::tensorflow::HistogramProto*>(&::tensorflow::_HistogramProto_default_instance_); +} +inline ::tensorflow::HistogramProto* Summary_Value::unsafe_arena_release_histo() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.Summary.Value.histo) + if (has_histo()) { + clear_has_value(); + ::tensorflow::HistogramProto* temp = value_.histo_; + value_.histo_ = NULL; + return temp; + } else { + return NULL; + } +} +inline void Summary_Value::unsafe_arena_set_allocated_histo(::tensorflow::HistogramProto* histo) { + clear_value(); + if (histo) { + set_has_histo(); + value_.histo_ = histo; + } + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.Summary.Value.histo) +} +inline ::tensorflow::HistogramProto* Summary_Value::mutable_histo() { + if (!has_histo()) { + clear_value(); + set_has_histo(); + value_.histo_ = CreateMaybeMessage< ::tensorflow::HistogramProto >( + GetArenaNoVirtual()); + } + // @@protoc_insertion_point(field_mutable:tensorflow.Summary.Value.histo) + return value_.histo_; +} + +// .tensorflow.Summary.Audio audio = 6; +inline bool Summary_Value::has_audio() const { + return value_case() == kAudio; +} +inline void Summary_Value::set_has_audio() { + _oneof_case_[0] = kAudio; +} +inline void Summary_Value::clear_audio() { + if (has_audio()) { + if (GetArenaNoVirtual() == NULL) { + delete value_.audio_; + } + clear_has_value(); + } +} +inline const ::tensorflow::Summary_Audio& Summary_Value::_internal_audio() const { + return *value_.audio_; +} +inline ::tensorflow::Summary_Audio* Summary_Value::release_audio() { + // @@protoc_insertion_point(field_release:tensorflow.Summary.Value.audio) + if (has_audio()) { + clear_has_value(); + ::tensorflow::Summary_Audio* temp = value_.audio_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + value_.audio_ = NULL; + return temp; + } else { + return NULL; + } +} +inline const ::tensorflow::Summary_Audio& Summary_Value::audio() const { + // @@protoc_insertion_point(field_get:tensorflow.Summary.Value.audio) + return has_audio() + ? *value_.audio_ + : *reinterpret_cast< ::tensorflow::Summary_Audio*>(&::tensorflow::_Summary_Audio_default_instance_); +} +inline ::tensorflow::Summary_Audio* Summary_Value::unsafe_arena_release_audio() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.Summary.Value.audio) + if (has_audio()) { + clear_has_value(); + ::tensorflow::Summary_Audio* temp = value_.audio_; + value_.audio_ = NULL; + return temp; + } else { + return NULL; + } +} +inline void Summary_Value::unsafe_arena_set_allocated_audio(::tensorflow::Summary_Audio* audio) { + clear_value(); + if (audio) { + set_has_audio(); + value_.audio_ = audio; + } + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.Summary.Value.audio) +} +inline ::tensorflow::Summary_Audio* Summary_Value::mutable_audio() { + if (!has_audio()) { + clear_value(); + set_has_audio(); + value_.audio_ = CreateMaybeMessage< ::tensorflow::Summary_Audio >( + GetArenaNoVirtual()); + } + // @@protoc_insertion_point(field_mutable:tensorflow.Summary.Value.audio) + return value_.audio_; +} + +// .tensorflow.TensorProto tensor = 8; +inline bool Summary_Value::has_tensor() const { + return value_case() == kTensor; +} +inline void Summary_Value::set_has_tensor() { + _oneof_case_[0] = kTensor; +} +inline const ::tensorflow::TensorProto& Summary_Value::_internal_tensor() const { + return *value_.tensor_; +} +inline ::tensorflow::TensorProto* Summary_Value::release_tensor() { + // @@protoc_insertion_point(field_release:tensorflow.Summary.Value.tensor) + if (has_tensor()) { + clear_has_value(); + ::tensorflow::TensorProto* temp = value_.tensor_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + value_.tensor_ = NULL; + return temp; + } else { + return NULL; + } +} +inline const ::tensorflow::TensorProto& Summary_Value::tensor() const { + // @@protoc_insertion_point(field_get:tensorflow.Summary.Value.tensor) + return has_tensor() + ? *value_.tensor_ + : *reinterpret_cast< ::tensorflow::TensorProto*>(&::tensorflow::_TensorProto_default_instance_); +} +inline ::tensorflow::TensorProto* Summary_Value::unsafe_arena_release_tensor() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.Summary.Value.tensor) + if (has_tensor()) { + clear_has_value(); + ::tensorflow::TensorProto* temp = value_.tensor_; + value_.tensor_ = NULL; + return temp; + } else { + return NULL; + } +} +inline void Summary_Value::unsafe_arena_set_allocated_tensor(::tensorflow::TensorProto* tensor) { + clear_value(); + if (tensor) { + set_has_tensor(); + value_.tensor_ = tensor; + } + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.Summary.Value.tensor) +} +inline ::tensorflow::TensorProto* Summary_Value::mutable_tensor() { + if (!has_tensor()) { + clear_value(); + set_has_tensor(); + value_.tensor_ = CreateMaybeMessage< ::tensorflow::TensorProto >( + GetArenaNoVirtual()); + } + // @@protoc_insertion_point(field_mutable:tensorflow.Summary.Value.tensor) + return value_.tensor_; +} + +inline bool Summary_Value::has_value() const { + return value_case() != VALUE_NOT_SET; +} +inline void Summary_Value::clear_has_value() { + _oneof_case_[0] = VALUE_NOT_SET; +} +inline Summary_Value::ValueCase Summary_Value::value_case() const { + return Summary_Value::ValueCase(_oneof_case_[0]); +} +// ------------------------------------------------------------------- + +// Summary + +// repeated .tensorflow.Summary.Value value = 1; +inline int Summary::value_size() const { + return value_.size(); +} +inline void Summary::clear_value() { + value_.Clear(); +} +inline ::tensorflow::Summary_Value* Summary::mutable_value(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.Summary.value) + return value_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::Summary_Value >* +Summary::mutable_value() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.Summary.value) + return &value_; +} +inline const ::tensorflow::Summary_Value& Summary::value(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.Summary.value) + return value_.Get(index); +} +inline ::tensorflow::Summary_Value* Summary::add_value() { + // @@protoc_insertion_point(field_add:tensorflow.Summary.value) + return value_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::Summary_Value >& +Summary::value() const { + // @@protoc_insertion_point(field_list:tensorflow.Summary.value) + return value_; +} + +#ifdef __GNUC__ + #pragma GCC diagnostic pop +#endif // __GNUC__ +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + + +// @@protoc_insertion_point(namespace_scope) + +} // namespace tensorflow + +// @@protoc_insertion_point(global_scope) + +#endif // PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2fsummary_2eproto diff --git a/linux/include/tensorflow/core/framework/summary.pb_text-impl.h b/linux/include/tensorflow/core/framework/summary.pb_text-impl.h new file mode 100644 index 000000000..1f5d66f38 --- /dev/null +++ b/linux/include/tensorflow/core/framework/summary.pb_text-impl.h @@ -0,0 +1,82 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_framework_summary_proto_IMPL_H_ +#define tensorflow_core_framework_summary_proto_IMPL_H_ + +#include "tensorflow/core/framework/resource_handle.pb.h" +#include "tensorflow/core/framework/resource_handle.pb_text-impl.h" +#include "tensorflow/core/framework/summary.pb.h" +#include "tensorflow/core/framework/summary.pb_text.h" +#include "tensorflow/core/framework/tensor.pb.h" +#include "tensorflow/core/framework/tensor.pb_text-impl.h" +#include "tensorflow/core/framework/tensor_shape.pb.h" +#include "tensorflow/core/framework/tensor_shape.pb_text-impl.h" +#include "tensorflow/core/framework/types.pb.h" +#include "tensorflow/core/framework/types.pb_text-impl.h" +#include "tensorflow/core/lib/strings/proto_text_util.h" +#include "tensorflow/core/lib/strings/scanner.h" + +namespace tensorflow { + +namespace internal { + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::SummaryDescription& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::SummaryDescription* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::HistogramProto& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::HistogramProto* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::SummaryMetadata_PluginData& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::SummaryMetadata_PluginData* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::SummaryMetadata& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::SummaryMetadata* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::Summary_Image& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::Summary_Image* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::Summary_Audio& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::Summary_Audio* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::Summary_Value& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::Summary_Value* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::Summary& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::Summary* msg); + +} // namespace internal + +} // namespace tensorflow + +#endif // tensorflow_core_framework_summary_proto_IMPL_H_ diff --git a/linux/include/tensorflow/core/framework/summary.pb_text.h b/linux/include/tensorflow/core/framework/summary.pb_text.h new file mode 100644 index 000000000..8d42789da --- /dev/null +++ b/linux/include/tensorflow/core/framework/summary.pb_text.h @@ -0,0 +1,94 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_framework_summary_proto_H_ +#define tensorflow_core_framework_summary_proto_H_ + +#include "tensorflow/core/framework/summary.pb.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/protobuf.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +// Message-text conversion for tensorflow.SummaryDescription +string ProtoDebugString( + const ::tensorflow::SummaryDescription& msg); +string ProtoShortDebugString( + const ::tensorflow::SummaryDescription& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::SummaryDescription* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.HistogramProto +string ProtoDebugString( + const ::tensorflow::HistogramProto& msg); +string ProtoShortDebugString( + const ::tensorflow::HistogramProto& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::HistogramProto* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.SummaryMetadata.PluginData +string ProtoDebugString( + const ::tensorflow::SummaryMetadata_PluginData& msg); +string ProtoShortDebugString( + const ::tensorflow::SummaryMetadata_PluginData& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::SummaryMetadata_PluginData* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.SummaryMetadata +string ProtoDebugString( + const ::tensorflow::SummaryMetadata& msg); +string ProtoShortDebugString( + const ::tensorflow::SummaryMetadata& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::SummaryMetadata* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.Summary.Image +string ProtoDebugString( + const ::tensorflow::Summary_Image& msg); +string ProtoShortDebugString( + const ::tensorflow::Summary_Image& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::Summary_Image* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.Summary.Audio +string ProtoDebugString( + const ::tensorflow::Summary_Audio& msg); +string ProtoShortDebugString( + const ::tensorflow::Summary_Audio& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::Summary_Audio* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.Summary.Value +string ProtoDebugString( + const ::tensorflow::Summary_Value& msg); +string ProtoShortDebugString( + const ::tensorflow::Summary_Value& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::Summary_Value* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.Summary +string ProtoDebugString( + const ::tensorflow::Summary& msg); +string ProtoShortDebugString( + const ::tensorflow::Summary& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::Summary* msg) + TF_MUST_USE_RESULT; + +} // namespace tensorflow + +#endif // tensorflow_core_framework_summary_proto_H_ diff --git a/linux/include/tensorflow/core/framework/tensor.h b/linux/include/tensorflow/core/framework/tensor.h new file mode 100644 index 000000000..d0f9eb56e --- /dev/null +++ b/linux/include/tensorflow/core/framework/tensor.h @@ -0,0 +1,817 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_FRAMEWORK_TENSOR_H_ +#define TENSORFLOW_CORE_FRAMEWORK_TENSOR_H_ + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/allocator.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/framework/types.pb.h" +#include "tensorflow/core/lib/core/refcount.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/lib/core/stringpiece.h" +#include "tensorflow/core/lib/gtl/inlined_vector.h" +#include "tensorflow/core/platform/logging.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +// Forward declarations. In particular, we forward declare protos so that their +// symbols can be removed from .so exports. +class AllocationDescription; +class Allocator; +class OpKernelContext; +class Tensor; +class TensorBuffer; +class TensorCApi; +class TensorDescription; +class TensorProto; + +namespace batch_util { +Status CopyElementToSlice(Tensor element, Tensor* parent, int64 index); +Status MaybeMoveSliceToElement(Tensor* parent, Tensor* element, int64 index); +} // namespace batch_util + +/// @ingroup core +/// Represents an n-dimensional array of values. +class Tensor { + public: + /// \brief Creates a 1-dimensional, 0-element float tensor. + /// + /// The returned Tensor is not a scalar (shape {}), but is instead + /// an empty one-dimensional Tensor (shape {0}, NumElements() == + /// 0). Since it has no elements, it does not need to be assigned a + /// value and is initialized by default (IsInitialized() is + /// true). If this is undesirable, consider creating a one-element + /// scalar which does require initialization: + /// + /// ```c++ + /// + /// Tensor(DT_FLOAT, TensorShape({})) + /// + /// ``` + Tensor(); + + /// \brief Creates a Tensor of the given `type` and `shape`. If + /// LogMemory::IsEnabled() the allocation is logged as coming from + /// an unknown kernel and step. Calling the Tensor constructor + /// directly from within an Op is deprecated: use the + /// OpKernelConstruction/OpKernelContext allocate_* methods to + /// allocate a new tensor, which record the kernel and step. + /// + /// The underlying buffer is allocated using a `CPUAllocator`. + Tensor(DataType type, const TensorShape& shape); + + /// \brief Creates a tensor with the input `type` and `shape`, using + /// the allocator `a` to allocate the underlying buffer. If + /// LogMemory::IsEnabled() the allocation is logged as coming from + /// an unknown kernel and step. Calling the Tensor constructor + /// directly from within an Op is deprecated: use the + /// OpKernelConstruction/OpKernelContext allocate_* methods to + /// allocate a new tensor, which record the kernel and step. + /// + /// `a` must outlive the lifetime of this Tensor. + Tensor(Allocator* a, DataType type, const TensorShape& shape); + + /// \brief Creates a tensor with the input `type` and `shape`, using + /// the allocator `a` and the specified "allocation_attr" to + /// allocate the underlying buffer. If the kernel and step are known + /// allocation_attr.allocation_will_be_logged should be set to true + /// and LogMemory::RecordTensorAllocation should be called after the + /// tensor is constructed. Calling the Tensor constructor directly + /// from within an Op is deprecated: use the + /// OpKernelConstruction/OpKernelContext allocate_* methods to + /// allocate a new tensor, which record the kernel and step. + /// + /// `a` must outlive the lifetime of this Tensor. + Tensor(Allocator* a, DataType type, const TensorShape& shape, + const AllocationAttributes& allocation_attr); + + /// \brief Creates an empty Tensor of the given data type. + /// + /// Like Tensor(), returns a 1-dimensional, 0-element Tensor with + /// IsInitialized() returning True. See the Tensor() documentation + /// for details. + explicit Tensor(DataType type); + + /// Copy constructor. + Tensor(const Tensor& other); + + /// \brief Move constructor. After this call, is safely destructible + /// and can be assigned to, but other calls on it (e.g. shape manipulation) + /// are not valid. + Tensor(Tensor&& other); + + ~Tensor(); + + /// Returns the data type. + DataType dtype() const { return shape_.data_type(); } + + /// Returns the shape of the tensor. + const TensorShape& shape() const { return shape_; } + + /// \brief Convenience accessor for the tensor shape. + /// + /// For all shape accessors, see comments for relevant methods of + /// `TensorShape` in `tensor_shape.h`. + int dims() const { return shape().dims(); } + + /// Convenience accessor for the tensor shape. + int64 dim_size(int d) const { return shape().dim_size(d); } + + /// Convenience accessor for the tensor shape. + int64 NumElements() const { return shape().num_elements(); } + + bool IsSameSize(const Tensor& b) const { + return shape().IsSameSize(b.shape()); + } + + // True iff the two tensors use the same underlying refcounted storage + bool SharesBufferWith(const Tensor& b) const; + + /// \brief If necessary, has this Tensor been initialized? + /// + /// Zero-element Tensors are always considered initialized, even if they + /// have never been assigned to and do not have any memory allocated. + bool IsInitialized() const; + + /// Returns the estimated memory usage of this tensor. + size_t TotalBytes() const; + + // Returns the size of allocated memory for this tensor. + size_t AllocatedBytes() const; + + /// Returns true iff this tensor is aligned. + bool IsAligned() const { +#if EIGEN_MAX_ALIGN_BYTES == 0 + return true; +#else + void* ptr = base(); + return reinterpret_cast(ptr) % EIGEN_MAX_ALIGN_BYTES == 0; +#endif + } + + /// Assign operator. This tensor shares other's underlying storage. + Tensor& operator=(const Tensor& other) { + CopyFromInternal(other, other.shape()); + return *this; + } + + /// Move operator. See move constructor for details. + Tensor& operator=(Tensor&& other); + + /// \brief Copy the other tensor into this tensor and reshape it. + /// + /// This tensor shares other's underlying storage. Returns `true` + /// iff `other.shape()` has the same number of elements of the given + /// `shape`. + bool CopyFrom(const Tensor& other, + const TensorShape& shape) TF_MUST_USE_RESULT { + if (other.NumElements() != shape.num_elements()) return false; + CopyFromInternal(other, shape); + return true; + } + + /// \brief Slice this tensor along the 1st dimension. + + /// I.e., the returned tensor satisfies + /// returned[i, ...] == this[dim0_start + i, ...]. + /// The returned tensor shares the underlying tensor buffer with this + /// tensor. + /// + /// NOTE: The returned tensor may not satisfy the same alignment + /// requirement as this tensor depending on the shape. The caller + /// must check the returned tensor's alignment before calling certain + /// methods that have alignment requirement (e.g., `flat()`, `tensor()`). + /// + /// NOTE: When fed with an N-dimensional tensor, this method returns a tensor + /// also with N dimensions. If you want to select a sub tensor, see SubSlice. + /// + /// REQUIRES: `dims()` >= 1 + /// REQUIRES: `0 <= dim0_start <= dim0_limit <= dim_size(0)` + Tensor Slice(int64 dim0_start, int64 dim0_limit) const; + + /// \brief Select a subslice from this tensor along the 1st dimension. + /// + /// When fed with an N-dimensional tensor, this method returns a tensor with + /// N-1 dimensions, where the returned tensor is a subslice of the input + /// tensor along the first dimension. The N-1 dimensions of the returned + /// tensor are the last N-1 dimensions of the input tensor. + /// + /// NOTE: The returned tensor may not satisfy the same alignment + /// requirement as this tensor depending on the shape. The caller + /// must check the returned tensor's alignment before calling certain + /// methods that have alignment requirement (e.g., `flat()`, `tensor()`). + /// + /// REQUIRES: `dims()` >= 1 + /// REQUIRES: `0 <= dim0_start < dim_size(0)` + Tensor SubSlice(int64 index) const; + + /// \brief Parse `other` and construct the tensor. + + /// Returns `true` iff the parsing succeeds. If the parsing fails, + /// the state of `*this` is unchanged. + bool FromProto(const TensorProto& other) TF_MUST_USE_RESULT; + bool FromProto(Allocator* a, const TensorProto& other) TF_MUST_USE_RESULT; + + /// \brief Fills in `proto` with `*this` tensor's content. + /// + /// `AsProtoField()` fills in the repeated field for `proto.dtype()`, while + /// `AsProtoTensorContent()` encodes the content in `proto.tensor_content()` + /// in a compact form. + void AsProtoField(TensorProto* proto) const; + void AsProtoTensorContent(TensorProto* proto) const; + + /// \brief Return the tensor data as an `Eigen::Tensor` with the type and + /// sizes of this `Tensor`. + /// + /// Use these methods when you know the data type and the number of + /// dimensions of the Tensor and you want an `Eigen::Tensor` + /// automatically sized to the `Tensor` sizes. The implementation check + /// fails if either type or sizes mismatch. + /// + /// Example: + /// + /// ```c++ + /// + /// typedef float T; + /// Tensor my_mat(...built with Shape{rows: 3, cols: 5}...); + /// auto mat = my_mat.matrix(); // 2D Eigen::Tensor, 3 x 5. + /// auto mat = my_mat.tensor(); // 2D Eigen::Tensor, 3 x 5. + /// auto vec = my_mat.vec(); // CHECK fails as my_mat is 2D. + /// auto vec = my_mat.tensor(); // CHECK fails as my_mat is 2D. + /// auto mat = my_mat.matrix();// CHECK fails as type mismatch. + /// + /// ``` + template + typename TTypes::Vec vec() { + return tensor(); + } + + template + typename TTypes::Matrix matrix() { + return tensor(); + } + + template + typename TTypes::Tensor tensor(); + + /// \brief Return the tensor data to an `Eigen::Tensor` with the + /// same size but a bitwise cast to the specified dtype `T`. + /// + /// Using a bitcast is useful for move and copy operations. + /// NOTE: this is the same as `tensor()` except a bitcast is allowed. + template + typename TTypes::Tensor bit_casted_tensor(); + + /// \brief Return the tensor data to an `Eigen::Tensor` with the + /// last dimension elements converted into single elements of a larger type. + /// + /// For example, this is useful for kernels that can treat NCHW_VECT_C int8 + /// tensors as NCHW int32 tensors. The sizeof(T) should equal the size of + /// the original element type * num elements in the original last dimension. + /// NDIMS should be 1 less than the original number of dimensions. + template + typename TTypes::Tensor reinterpret_last_dimension(); + + /// \brief Return the tensor data as an `Eigen::Tensor` of the data type and a + /// specified shape. + /// + /// These methods allow you to access the data with the dimensions + /// and sizes of your choice. You do not need to know the number of + /// dimensions of the Tensor to call them. However, they `CHECK` that + /// the type matches and the dimensions requested creates an + /// `Eigen::Tensor` with the same number of elements as the tensor. + /// + /// Example: + /// + /// ```c++ + /// + /// typedef float T; + /// Tensor my_ten(...built with Shape{planes: 4, rows: 3, cols: 5}...); + /// // 1D Eigen::Tensor, size 60: + /// auto flat = my_ten.flat(); + /// // 2D Eigen::Tensor 12 x 5: + /// auto inner = my_ten.flat_inner_dims(); + /// // 2D Eigen::Tensor 4 x 15: + /// auto outer = my_ten.shaped({4, 15}); + /// // CHECK fails, bad num elements: + /// auto outer = my_ten.shaped({4, 8}); + /// // 3D Eigen::Tensor 6 x 5 x 2: + /// auto weird = my_ten.shaped({6, 5, 2}); + /// // CHECK fails, type mismatch: + /// auto bad = my_ten.flat(); + /// + /// ``` + template + typename TTypes::Flat flat() { + return shaped({NumElements()}); + } + + template + typename TTypes::UnalignedFlat unaligned_flat() { + return unaligned_shaped({NumElements()}); + } + + /// Returns the data as an Eigen::Tensor with NDIMS dimensions, collapsing all + /// Tensor dimensions but the last NDIMS-1 into the first dimension of the + /// result. If NDIMS > dims() then leading dimensions of size 1 will be + /// added to make the output rank NDIMS. + template + typename TTypes::Tensor flat_inner_dims(); + + /// Returns the data as an Eigen::Tensor with NDIMS dimensions, collapsing all + /// Tensor dimensions but the first NDIMS-1 into the last dimension of the + /// result. If NDIMS > dims() then trailing dimensions of size 1 will be + /// added to make the output rank NDIMS. + template + typename TTypes::Tensor flat_outer_dims(); + + /// Returns the data as an Eigen::Tensor with NDIMS dimensions, collapsing the + /// first 'begin' Tensor dimensions into the first dimension of the result and + /// the Tensor dimensions of the last dims() - 'begin' - NDIMS into the last + /// dimension of the result. If 'begin' < 0 then the |'begin'| leading + /// dimensions of size 1 will be added. If 'begin' + NDIMS > dims() then + /// 'begin' + NDIMS - dims() trailing dimensions of size 1 will be added. + template + typename TTypes::Tensor flat_inner_outer_dims(int64 begin); + + template + typename TTypes::Tensor shaped(gtl::ArraySlice new_sizes); + + /// \brief Return the tensor data to an `Eigen::Tensor` with the new + /// shape specified in `new_sizes` and cast to a new dtype `T`. + /// + /// Using a bitcast is useful for move and copy operations. + /// The allowed bitcast is the only difference from `shaped()`. + template + typename TTypes::Tensor bit_casted_shaped( + gtl::ArraySlice new_sizes); + + template + typename TTypes::UnalignedTensor unaligned_shaped( + gtl::ArraySlice new_sizes); + + /// \brief Return the Tensor data as a `TensorMap` of fixed size 1: + /// `TensorMap>`. + + /// Using `scalar()` allows the compiler to perform optimizations as + /// the size of the tensor is known at compile time. + template + typename TTypes::Scalar scalar(); + + /// Const versions of all the methods above. + template + typename TTypes::ConstVec vec() const { + return tensor(); + } + + template + typename TTypes::ConstMatrix matrix() const { + return tensor(); + } + + template + typename TTypes::ConstTensor tensor() const; + + /// \brief Return the tensor data to an `Eigen::Tensor` with the + /// same size but a bitwise cast to the specified dtype `T`. + /// + /// Using a bitcast is useful for move and copy operations. + /// NOTE: this is the same as `tensor()` except a bitcast is allowed. + template + typename TTypes::ConstTensor bit_casted_tensor() const; + + /// \brief Return the tensor data to an `Eigen::Tensor` with the + /// last dimension elements converted into single elements of a larger type. + /// + /// For example, this is useful for kernels that can treat NCHW_VECT_C int8 + /// tensors as NCHW int32 tensors. The sizeof(T) should equal the size of + /// the original element type * num elements in the original last dimension. + /// NDIMS should be 1 less than the original number of dimensions. + template + typename TTypes::ConstTensor reinterpret_last_dimension() const; + + template + typename TTypes::ConstFlat flat() const { + return shaped({NumElements()}); + } + + template + typename TTypes::UnalignedConstFlat unaligned_flat() const { + return unaligned_shaped({NumElements()}); + } + + template + typename TTypes::ConstTensor shaped( + gtl::ArraySlice new_sizes) const; + + /// \brief Return the tensor data to an `Eigen::Tensor` with the new + /// shape specified in `new_sizes` and cast to a new dtype `T`. + /// + /// Using a bitcast is useful for move and copy operations. + /// The allowed bitcast is the only difference from `shaped()`. + template + typename TTypes::ConstTensor bit_casted_shaped( + gtl::ArraySlice new_sizes) const; + + template + typename TTypes::UnalignedConstTensor unaligned_shaped( + gtl::ArraySlice new_sizes) const; + + template + typename TTypes::ConstScalar scalar() const; + + template + typename TTypes::ConstTensor flat_inner_dims() const; + + template + typename TTypes::ConstTensor flat_outer_dims() const; + + template + typename TTypes::ConstTensor flat_inner_outer_dims( + int64 begin) const; + + /// Render the first `max_entries` values in `*this` into a string. + string SummarizeValue(int64 max_entries, bool print_v2 = false) const; + + /// A human-readable summary of the tensor suitable for debugging. + string DebugString() const; + + /// Fill in the `TensorDescription` proto with metadata about the + /// tensor that is useful for monitoring and debugging. + void FillDescription(TensorDescription* description) const; + + /// \brief Returns a `StringPiece` mapping the current tensor's buffer. + /// + /// The returned `StringPiece` may point to memory location on devices + /// that the CPU cannot address directly. + /// + /// NOTE: The underlying tensor buffer is refcounted, so the lifetime + /// of the contents mapped by the `StringPiece` matches the lifetime of + /// the buffer; callers should arrange to make sure the buffer does + /// not get destroyed while the `StringPiece` is still used. + /// + /// REQUIRES: `DataTypeCanUseMemcpy(dtype())`. + StringPiece tensor_data() const; + + /// Copy the other tensor into this tensor and reshape it and reinterpret the + /// buffer's datatype. + /// + /// This tensor shares other's underlying storage. + void UnsafeCopyFromInternal(const Tensor&, DataType dtype, + const TensorShape&); + + private: + // Returns true if the refcount on buf_ and any possible underlying root + // buffer is one. + bool RefCountIsOne() const; + void CheckType(DataType expected_dtype) const; + void CheckTypeAndIsAligned(DataType expected_dtype) const; + void CheckIsAlignedAndSingleElement() const; + void set_dtype(DataType t) { shape_.set_data_type(t); } + + // TensorShape's InlineVector. + static gtl::InlinedVector ComputeFlatInnerDims( + gtl::ArraySlice orig, int64 num_out_dims); + static gtl::InlinedVector ComputeFlatOuterDims( + gtl::ArraySlice orig, int64 num_out_dims); + + TensorShape shape_; + TensorBuffer* buf_; + + friend class DMAHelper; + friend class TensorCApi; + friend class TensorReference; // For access to buf_ + friend class VariableOp; // For access to set_shape + friend class AutoReloadVariableOp; // For access to set_shape + friend class TensorTestHelper; // For access to set_shape + friend class CastOpBase; // For access to set_dtype; + friend class OpKernelContext; // For access to RefCountIsOne(). + friend class ScopedAllocator; // For access to buf_. + friend class XlaTensor; // For access to RefCountIsOne(). + friend class XlaTensorBuffer; // For access to the private constructor taking + // the buffer + template + friend class AssignVariableOp; // For access to RefCountIsOne(). + template + friend Status PrepareToUpdateVariable( + OpKernelContext* ctx, Tensor* tensor); // For access to RefCountIsOne(). + friend Status batch_util::CopyElementToSlice( + Tensor element, Tensor* parent, + int64 index); // For access to RefCountIsOne(). + friend Status batch_util::MaybeMoveSliceToElement( + Tensor* parent, Tensor* element, + int64 index); // For access to RefCountIsOne(). + + friend class NumpyTensorBuffer; // For access to the private constructor + // taking the buffer. + + // Creates a tensor with the input datatype, shape and buf. + // + // Acquires a ref on buf that belongs to this Tensor. + Tensor(DataType type, const TensorShape& shape, TensorBuffer* buf); + + bool CanUseDMA() const; + + // Only needed by variable op to set the shape of an uninitialized + // Tensor. + // TODO: Remove this when we have a better story for detecting + // uninitialized tensors. + void set_shape(const TensorShape& shape) { + DataType dt = dtype(); + shape_ = shape; + set_dtype(dt); + } + + void CopyFromInternal(const Tensor& other, const TensorShape& shape); + + template + T* base() const; + + template + void FillDimsAndValidateCompatibleShape( + gtl::ArraySlice new_sizes, + Eigen::array* dims) const; + + template + void FillDimsAndValidateCompatibleShape( + gtl::ArraySlice new_sizes, + Eigen::array* dims) const; +}; + +// Implementation details + +// START_SKIP_DOXYGEN + +// Interface to access the raw ref-counted data buffer. +class TensorBuffer : public core::RefCounted { + public: + ~TensorBuffer() override {} + + // data() points to a memory region of size() bytes. + virtual void* data() const = 0; + virtual size_t size() const = 0; + + // If this TensorBuffer is sub-buffer of another TensorBuffer, + // returns that TensorBuffer. Otherwise, returns this. + virtual TensorBuffer* root_buffer() = 0; + + // Fill metadata about the allocation into the proto. + virtual void FillAllocationDescription( + AllocationDescription* proto) const = 0; + + template + T* base() const { + return reinterpret_cast(data()); + } + + // Whether this TensorBuffer owns the underlying memory. + virtual bool OwnsMemory() const { return true; } +}; + +template +T* Tensor::base() const { + return buf_ == nullptr ? nullptr : buf_->base(); +} + +template +typename TTypes::Tensor Tensor::tensor() { + CheckTypeAndIsAligned(DataTypeToEnum::v()); + return typename TTypes::Tensor(base(), + shape().AsEigenDSizes()); +} + +template +typename TTypes::ConstTensor Tensor::tensor() const { + CheckTypeAndIsAligned(DataTypeToEnum::v()); + return typename TTypes::ConstTensor(base(), + shape().AsEigenDSizes()); +} + +template +typename TTypes::Tensor Tensor::bit_casted_tensor() { + CHECK(IsAligned()); + return typename TTypes::Tensor(base(), + shape().AsEigenDSizes()); +} + +template +typename TTypes::ConstTensor Tensor::bit_casted_tensor() const { + CHECK(IsAligned()); + return typename TTypes::ConstTensor(base(), + shape().AsEigenDSizes()); +} + +template +typename TTypes::Tensor Tensor::reinterpret_last_dimension() { + if (NDIMS == dims()) { + return tensor(); + } + CHECK(IsAligned()); + CHECK_EQ(NDIMS, dims() - 1); + CHECK_EQ(sizeof(T), shape_.dim_sizes()[NDIMS] * DataTypeSize(dtype())); + Eigen::array dims; + for (int d = 0; d < NDIMS; ++d) { + dims[d] = shape_.dim_sizes()[d]; + } + return typename TTypes::Tensor(base(), dims); +} + +template +typename TTypes::ConstTensor Tensor::reinterpret_last_dimension() + const { + if (NDIMS == dims()) { + return tensor(); + } + CHECK(IsAligned()); + CHECK_EQ(NDIMS, dims() - 1); + CHECK_EQ(sizeof(T), shape_.dim_sizes()[NDIMS] * DataTypeSize(dtype())); + Eigen::array dims; + for (int d = 0; d < NDIMS; ++d) { + dims[d] = shape_.dim_sizes()[d]; + } + return typename TTypes::ConstTensor(base(), dims); +} + +template +void Tensor::FillDimsAndValidateCompatibleShape( + gtl::ArraySlice new_sizes, + Eigen::array* dims) const { + CHECK_EQ(NDIMS, new_sizes.size()); + int64 new_num_elements = 1; + for (size_t d = 0; d < NDIMS; d++) { + new_num_elements *= new_sizes[d]; + (*dims)[d] = new_sizes[d]; + } + CHECK_EQ(new_num_elements, NumElements()); +} + +template +void Tensor::FillDimsAndValidateCompatibleShape( + gtl::ArraySlice new_sizes, + Eigen::array* dims) const { + CHECK_EQ(NDIMS, new_sizes.size()); + int64 new_num_elements = 1; + for (size_t d = 0; d < NDIMS; d++) { + new_num_elements *= new_sizes[d]; + (*dims)[d] = new_sizes[d]; + } + const int element_size = DataTypeSize(BaseType(dtype())); + if (element_size > 0) { + CHECK_EQ(new_num_elements * sizeof(T), NumElements() * element_size); + } else { + // DataTypeSize() returns 0 for some data types. In this case, assume that T + // has the same size as the buffer type. + // NOTE: If we can be sure that DataTypeSize() does not return 0 for all POD + // types, then we should check DataTypeToEnum::v() == dtype(). Or simply + // check if `element_size > 0` to err when bit cast is attempted on Tensor + // of unknown data type size. + CHECK_EQ(new_num_elements, NumElements()); + } +} + +template +typename TTypes::Tensor Tensor::shaped( + gtl::ArraySlice new_sizes) { + CheckTypeAndIsAligned(DataTypeToEnum::v()); + Eigen::array dims; + FillDimsAndValidateCompatibleShape(new_sizes, &dims); + return typename TTypes::Tensor(base(), dims); +} + +template +typename TTypes::Tensor Tensor::bit_casted_shaped( + gtl::ArraySlice new_sizes) { + CHECK(IsAligned()); + Eigen::array dims; + FillDimsAndValidateCompatibleShape(new_sizes, &dims); + return typename TTypes::Tensor(base(), dims); +} + +template +typename TTypes::UnalignedTensor Tensor::unaligned_shaped( + gtl::ArraySlice new_sizes) { + CheckType(DataTypeToEnum::v()); + Eigen::array dims; + FillDimsAndValidateCompatibleShape(new_sizes, &dims); + return typename TTypes::UnalignedTensor(base(), dims); +} + +template +typename TTypes::ConstTensor Tensor::shaped( + gtl::ArraySlice new_sizes) const { + CheckType(DataTypeToEnum::v()); + CHECK(IsAligned()); + Eigen::array dims; + FillDimsAndValidateCompatibleShape(new_sizes, &dims); + return typename TTypes::ConstTensor(base(), dims); +} + +template +typename TTypes::ConstTensor Tensor::bit_casted_shaped( + gtl::ArraySlice new_sizes) const { + CHECK(IsAligned()); + Eigen::array dims; + FillDimsAndValidateCompatibleShape(new_sizes, &dims); + return typename TTypes::ConstTensor(base(), dims); +} + +template +typename TTypes::UnalignedConstTensor Tensor::unaligned_shaped( + gtl::ArraySlice new_sizes) const { + CheckType(DataTypeToEnum::v()); + Eigen::array dims; + FillDimsAndValidateCompatibleShape(new_sizes, &dims); + return typename TTypes::UnalignedConstTensor(base(), dims); +} + +template +typename TTypes::Scalar Tensor::scalar() { + CheckIsAlignedAndSingleElement(); + return typename TTypes::Scalar(base()); +} + +template +typename TTypes::ConstScalar Tensor::scalar() const { + CheckIsAlignedAndSingleElement(); + return typename TTypes::ConstScalar(base()); +} + +template +typename TTypes::Tensor Tensor::flat_inner_dims() { + return shaped(ComputeFlatInnerDims(shape_.dim_sizes(), NDIMS)); +} + +template +typename TTypes::Tensor Tensor::flat_outer_dims() { + return shaped(ComputeFlatOuterDims(shape_.dim_sizes(), NDIMS)); +} + +template +typename TTypes::Tensor Tensor::flat_inner_outer_dims(int64 begin) { + gtl::InlinedVector flat_outer = + ComputeFlatOuterDims(shape_.dim_sizes(), begin + NDIMS); + return shaped(ComputeFlatInnerDims(flat_outer, NDIMS)); +} + +template +typename TTypes::ConstTensor Tensor::flat_inner_dims() const { + return shaped(ComputeFlatInnerDims(shape_.dim_sizes(), NDIMS)); +} + +template +typename TTypes::ConstTensor Tensor::flat_outer_dims() const { + return shaped(ComputeFlatOuterDims(shape_.dim_sizes(), NDIMS)); +} + +template +typename TTypes::ConstTensor Tensor::flat_inner_outer_dims( + int64 begin) const { + gtl::InlinedVector flat_outer = + ComputeFlatOuterDims(shape_.dim_sizes(), begin + NDIMS); + return shaped(ComputeFlatInnerDims(flat_outer, NDIMS)); +} + +inline Tensor::Tensor(const Tensor& other) + : shape_(other.shape()), buf_(other.buf_) { + if (buf_) buf_->Ref(); +} + +inline Tensor::Tensor(Tensor&& other) + : shape_(std::move(other.shape())), buf_(other.buf_) { + other.buf_ = nullptr; +} + +inline Tensor& Tensor::operator=(Tensor&& other) { + // Avoid self-assignment, since we might destroy our underlying buffer. + if (&other != this) { + shape_ = std::move(other.shape_); + if (buf_) buf_->Unref(); + buf_ = other.buf_; + other.buf_ = nullptr; + } + return *this; +} + +// END_SKIP_DOXYGEN + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_FRAMEWORK_TENSOR_H_ diff --git a/linux/include/tensorflow/core/framework/tensor.pb.h b/linux/include/tensorflow/core/framework/tensor.pb.h new file mode 100644 index 000000000..44456f58d --- /dev/null +++ b/linux/include/tensorflow/core/framework/tensor.pb.h @@ -0,0 +1,1388 @@ +// Generated by the protocol buffer compiler. DO NOT EDIT! +// source: tensorflow/core/framework/tensor.proto + +#ifndef PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2ftensor_2eproto +#define PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2ftensor_2eproto + +#include + +#include + +#if GOOGLE_PROTOBUF_VERSION < 3006000 +#error This file was generated by a newer version of protoc which is +#error incompatible with your Protocol Buffer headers. Please update +#error your headers. +#endif +#if 3006000 < GOOGLE_PROTOBUF_MIN_PROTOC_VERSION +#error This file was generated by an older version of protoc which is +#error incompatible with your Protocol Buffer headers. Please +#error regenerate this file with a newer version of protoc. +#endif + +#include +#include +#include +#include +#include +#include +#include +#include +#include // IWYU pragma: export +#include // IWYU pragma: export +#include +#include "tensorflow/core/framework/resource_handle.pb.h" +#include "tensorflow/core/framework/tensor_shape.pb.h" +#include "tensorflow/core/framework/types.pb.h" +// @@protoc_insertion_point(includes) +#define PROTOBUF_INTERNAL_EXPORT_protobuf_tensorflow_2fcore_2fframework_2ftensor_2eproto + +namespace protobuf_tensorflow_2fcore_2fframework_2ftensor_2eproto { +// Internal implementation detail -- do not use these members. +struct TableStruct { + static const ::google::protobuf::internal::ParseTableField entries[]; + static const ::google::protobuf::internal::AuxillaryParseTableField aux[]; + static const ::google::protobuf::internal::ParseTable schema[2]; + static const ::google::protobuf::internal::FieldMetadata field_metadata[]; + static const ::google::protobuf::internal::SerializationTable serialization_table[]; + static const ::google::protobuf::uint32 offsets[]; +}; +void AddDescriptors(); +} // namespace protobuf_tensorflow_2fcore_2fframework_2ftensor_2eproto +namespace tensorflow { +class TensorProto; +class TensorProtoDefaultTypeInternal; +extern TensorProtoDefaultTypeInternal _TensorProto_default_instance_; +class VariantTensorDataProto; +class VariantTensorDataProtoDefaultTypeInternal; +extern VariantTensorDataProtoDefaultTypeInternal _VariantTensorDataProto_default_instance_; +} // namespace tensorflow +namespace google { +namespace protobuf { +template<> ::tensorflow::TensorProto* Arena::CreateMaybeMessage<::tensorflow::TensorProto>(Arena*); +template<> ::tensorflow::VariantTensorDataProto* Arena::CreateMaybeMessage<::tensorflow::VariantTensorDataProto>(Arena*); +} // namespace protobuf +} // namespace google +namespace tensorflow { + +// =================================================================== + +class TensorProto : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.TensorProto) */ { + public: + TensorProto(); + virtual ~TensorProto(); + + TensorProto(const TensorProto& from); + + inline TensorProto& operator=(const TensorProto& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + TensorProto(TensorProto&& from) noexcept + : TensorProto() { + *this = ::std::move(from); + } + + inline TensorProto& operator=(TensorProto&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const TensorProto& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const TensorProto* internal_default_instance() { + return reinterpret_cast( + &_TensorProto_default_instance_); + } + static constexpr int kIndexInFileMessages = + 0; + + void UnsafeArenaSwap(TensorProto* other); + void Swap(TensorProto* other); + friend void swap(TensorProto& a, TensorProto& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline TensorProto* New() const final { + return CreateMaybeMessage(NULL); + } + + TensorProto* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const TensorProto& from); + void MergeFrom(const TensorProto& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(TensorProto* other); + protected: + explicit TensorProto(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated float float_val = 5 [packed = true]; + int float_val_size() const; + void clear_float_val(); + static const int kFloatValFieldNumber = 5; + float float_val(int index) const; + void set_float_val(int index, float value); + void add_float_val(float value); + const ::google::protobuf::RepeatedField< float >& + float_val() const; + ::google::protobuf::RepeatedField< float >* + mutable_float_val(); + + // repeated double double_val = 6 [packed = true]; + int double_val_size() const; + void clear_double_val(); + static const int kDoubleValFieldNumber = 6; + double double_val(int index) const; + void set_double_val(int index, double value); + void add_double_val(double value); + const ::google::protobuf::RepeatedField< double >& + double_val() const; + ::google::protobuf::RepeatedField< double >* + mutable_double_val(); + + // repeated int32 int_val = 7 [packed = true]; + int int_val_size() const; + void clear_int_val(); + static const int kIntValFieldNumber = 7; + ::google::protobuf::int32 int_val(int index) const; + void set_int_val(int index, ::google::protobuf::int32 value); + void add_int_val(::google::protobuf::int32 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::int32 >& + int_val() const; + ::google::protobuf::RepeatedField< ::google::protobuf::int32 >* + mutable_int_val(); + + // repeated bytes string_val = 8; + int string_val_size() const; + void clear_string_val(); + static const int kStringValFieldNumber = 8; + const ::std::string& string_val(int index) const; + ::std::string* mutable_string_val(int index); + void set_string_val(int index, const ::std::string& value); + #if LANG_CXX11 + void set_string_val(int index, ::std::string&& value); + #endif + void set_string_val(int index, const char* value); + void set_string_val(int index, const void* value, size_t size); + ::std::string* add_string_val(); + void add_string_val(const ::std::string& value); + #if LANG_CXX11 + void add_string_val(::std::string&& value); + #endif + void add_string_val(const char* value); + void add_string_val(const void* value, size_t size); + const ::google::protobuf::RepeatedPtrField< ::std::string>& string_val() const; + ::google::protobuf::RepeatedPtrField< ::std::string>* mutable_string_val(); + + // repeated float scomplex_val = 9 [packed = true]; + int scomplex_val_size() const; + void clear_scomplex_val(); + static const int kScomplexValFieldNumber = 9; + float scomplex_val(int index) const; + void set_scomplex_val(int index, float value); + void add_scomplex_val(float value); + const ::google::protobuf::RepeatedField< float >& + scomplex_val() const; + ::google::protobuf::RepeatedField< float >* + mutable_scomplex_val(); + + // repeated int64 int64_val = 10 [packed = true]; + int int64_val_size() const; + void clear_int64_val(); + static const int kInt64ValFieldNumber = 10; + ::google::protobuf::int64 int64_val(int index) const; + void set_int64_val(int index, ::google::protobuf::int64 value); + void add_int64_val(::google::protobuf::int64 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& + int64_val() const; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* + mutable_int64_val(); + + // repeated bool bool_val = 11 [packed = true]; + int bool_val_size() const; + void clear_bool_val(); + static const int kBoolValFieldNumber = 11; + bool bool_val(int index) const; + void set_bool_val(int index, bool value); + void add_bool_val(bool value); + const ::google::protobuf::RepeatedField< bool >& + bool_val() const; + ::google::protobuf::RepeatedField< bool >* + mutable_bool_val(); + + // repeated double dcomplex_val = 12 [packed = true]; + int dcomplex_val_size() const; + void clear_dcomplex_val(); + static const int kDcomplexValFieldNumber = 12; + double dcomplex_val(int index) const; + void set_dcomplex_val(int index, double value); + void add_dcomplex_val(double value); + const ::google::protobuf::RepeatedField< double >& + dcomplex_val() const; + ::google::protobuf::RepeatedField< double >* + mutable_dcomplex_val(); + + // repeated int32 half_val = 13 [packed = true]; + int half_val_size() const; + void clear_half_val(); + static const int kHalfValFieldNumber = 13; + ::google::protobuf::int32 half_val(int index) const; + void set_half_val(int index, ::google::protobuf::int32 value); + void add_half_val(::google::protobuf::int32 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::int32 >& + half_val() const; + ::google::protobuf::RepeatedField< ::google::protobuf::int32 >* + mutable_half_val(); + + // repeated .tensorflow.ResourceHandleProto resource_handle_val = 14; + int resource_handle_val_size() const; + void clear_resource_handle_val(); + static const int kResourceHandleValFieldNumber = 14; + ::tensorflow::ResourceHandleProto* mutable_resource_handle_val(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::ResourceHandleProto >* + mutable_resource_handle_val(); + const ::tensorflow::ResourceHandleProto& resource_handle_val(int index) const; + ::tensorflow::ResourceHandleProto* add_resource_handle_val(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::ResourceHandleProto >& + resource_handle_val() const; + + // repeated .tensorflow.VariantTensorDataProto variant_val = 15; + int variant_val_size() const; + void clear_variant_val(); + static const int kVariantValFieldNumber = 15; + ::tensorflow::VariantTensorDataProto* mutable_variant_val(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::VariantTensorDataProto >* + mutable_variant_val(); + const ::tensorflow::VariantTensorDataProto& variant_val(int index) const; + ::tensorflow::VariantTensorDataProto* add_variant_val(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::VariantTensorDataProto >& + variant_val() const; + + // repeated uint32 uint32_val = 16 [packed = true]; + int uint32_val_size() const; + void clear_uint32_val(); + static const int kUint32ValFieldNumber = 16; + ::google::protobuf::uint32 uint32_val(int index) const; + void set_uint32_val(int index, ::google::protobuf::uint32 value); + void add_uint32_val(::google::protobuf::uint32 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::uint32 >& + uint32_val() const; + ::google::protobuf::RepeatedField< ::google::protobuf::uint32 >* + mutable_uint32_val(); + + // repeated uint64 uint64_val = 17 [packed = true]; + int uint64_val_size() const; + void clear_uint64_val(); + static const int kUint64ValFieldNumber = 17; + ::google::protobuf::uint64 uint64_val(int index) const; + void set_uint64_val(int index, ::google::protobuf::uint64 value); + void add_uint64_val(::google::protobuf::uint64 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::uint64 >& + uint64_val() const; + ::google::protobuf::RepeatedField< ::google::protobuf::uint64 >* + mutable_uint64_val(); + + // bytes tensor_content = 4; + void clear_tensor_content(); + static const int kTensorContentFieldNumber = 4; + const ::std::string& tensor_content() const; + void set_tensor_content(const ::std::string& value); + #if LANG_CXX11 + void set_tensor_content(::std::string&& value); + #endif + void set_tensor_content(const char* value); + void set_tensor_content(const void* value, size_t size); + ::std::string* mutable_tensor_content(); + ::std::string* release_tensor_content(); + void set_allocated_tensor_content(::std::string* tensor_content); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_tensor_content(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_tensor_content( + ::std::string* tensor_content); + + // .tensorflow.TensorShapeProto tensor_shape = 2; + bool has_tensor_shape() const; + void clear_tensor_shape(); + static const int kTensorShapeFieldNumber = 2; + private: + const ::tensorflow::TensorShapeProto& _internal_tensor_shape() const; + public: + const ::tensorflow::TensorShapeProto& tensor_shape() const; + ::tensorflow::TensorShapeProto* release_tensor_shape(); + ::tensorflow::TensorShapeProto* mutable_tensor_shape(); + void set_allocated_tensor_shape(::tensorflow::TensorShapeProto* tensor_shape); + void unsafe_arena_set_allocated_tensor_shape( + ::tensorflow::TensorShapeProto* tensor_shape); + ::tensorflow::TensorShapeProto* unsafe_arena_release_tensor_shape(); + + // .tensorflow.DataType dtype = 1; + void clear_dtype(); + static const int kDtypeFieldNumber = 1; + ::tensorflow::DataType dtype() const; + void set_dtype(::tensorflow::DataType value); + + // int32 version_number = 3; + void clear_version_number(); + static const int kVersionNumberFieldNumber = 3; + ::google::protobuf::int32 version_number() const; + void set_version_number(::google::protobuf::int32 value); + + // @@protoc_insertion_point(class_scope:tensorflow.TensorProto) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedField< float > float_val_; + mutable int _float_val_cached_byte_size_; + ::google::protobuf::RepeatedField< double > double_val_; + mutable int _double_val_cached_byte_size_; + ::google::protobuf::RepeatedField< ::google::protobuf::int32 > int_val_; + mutable int _int_val_cached_byte_size_; + ::google::protobuf::RepeatedPtrField< ::std::string> string_val_; + ::google::protobuf::RepeatedField< float > scomplex_val_; + mutable int _scomplex_val_cached_byte_size_; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 > int64_val_; + mutable int _int64_val_cached_byte_size_; + ::google::protobuf::RepeatedField< bool > bool_val_; + mutable int _bool_val_cached_byte_size_; + ::google::protobuf::RepeatedField< double > dcomplex_val_; + mutable int _dcomplex_val_cached_byte_size_; + ::google::protobuf::RepeatedField< ::google::protobuf::int32 > half_val_; + mutable int _half_val_cached_byte_size_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::ResourceHandleProto > resource_handle_val_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::VariantTensorDataProto > variant_val_; + ::google::protobuf::RepeatedField< ::google::protobuf::uint32 > uint32_val_; + mutable int _uint32_val_cached_byte_size_; + ::google::protobuf::RepeatedField< ::google::protobuf::uint64 > uint64_val_; + mutable int _uint64_val_cached_byte_size_; + ::google::protobuf::internal::ArenaStringPtr tensor_content_; + ::tensorflow::TensorShapeProto* tensor_shape_; + int dtype_; + ::google::protobuf::int32 version_number_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2ftensor_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class VariantTensorDataProto : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.VariantTensorDataProto) */ { + public: + VariantTensorDataProto(); + virtual ~VariantTensorDataProto(); + + VariantTensorDataProto(const VariantTensorDataProto& from); + + inline VariantTensorDataProto& operator=(const VariantTensorDataProto& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + VariantTensorDataProto(VariantTensorDataProto&& from) noexcept + : VariantTensorDataProto() { + *this = ::std::move(from); + } + + inline VariantTensorDataProto& operator=(VariantTensorDataProto&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const VariantTensorDataProto& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const VariantTensorDataProto* internal_default_instance() { + return reinterpret_cast( + &_VariantTensorDataProto_default_instance_); + } + static constexpr int kIndexInFileMessages = + 1; + + void UnsafeArenaSwap(VariantTensorDataProto* other); + void Swap(VariantTensorDataProto* other); + friend void swap(VariantTensorDataProto& a, VariantTensorDataProto& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline VariantTensorDataProto* New() const final { + return CreateMaybeMessage(NULL); + } + + VariantTensorDataProto* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const VariantTensorDataProto& from); + void MergeFrom(const VariantTensorDataProto& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(VariantTensorDataProto* other); + protected: + explicit VariantTensorDataProto(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated .tensorflow.TensorProto tensors = 3; + int tensors_size() const; + void clear_tensors(); + static const int kTensorsFieldNumber = 3; + ::tensorflow::TensorProto* mutable_tensors(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::TensorProto >* + mutable_tensors(); + const ::tensorflow::TensorProto& tensors(int index) const; + ::tensorflow::TensorProto* add_tensors(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::TensorProto >& + tensors() const; + + // string type_name = 1; + void clear_type_name(); + static const int kTypeNameFieldNumber = 1; + const ::std::string& type_name() const; + void set_type_name(const ::std::string& value); + #if LANG_CXX11 + void set_type_name(::std::string&& value); + #endif + void set_type_name(const char* value); + void set_type_name(const char* value, size_t size); + ::std::string* mutable_type_name(); + ::std::string* release_type_name(); + void set_allocated_type_name(::std::string* type_name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_type_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_type_name( + ::std::string* type_name); + + // bytes metadata = 2; + void clear_metadata(); + static const int kMetadataFieldNumber = 2; + const ::std::string& metadata() const; + void set_metadata(const ::std::string& value); + #if LANG_CXX11 + void set_metadata(::std::string&& value); + #endif + void set_metadata(const char* value); + void set_metadata(const void* value, size_t size); + ::std::string* mutable_metadata(); + ::std::string* release_metadata(); + void set_allocated_metadata(::std::string* metadata); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_metadata(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_metadata( + ::std::string* metadata); + + // @@protoc_insertion_point(class_scope:tensorflow.VariantTensorDataProto) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::TensorProto > tensors_; + ::google::protobuf::internal::ArenaStringPtr type_name_; + ::google::protobuf::internal::ArenaStringPtr metadata_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2ftensor_2eproto::TableStruct; +}; +// =================================================================== + + +// =================================================================== + +#ifdef __GNUC__ + #pragma GCC diagnostic push + #pragma GCC diagnostic ignored "-Wstrict-aliasing" +#endif // __GNUC__ +// TensorProto + +// .tensorflow.DataType dtype = 1; +inline void TensorProto::clear_dtype() { + dtype_ = 0; +} +inline ::tensorflow::DataType TensorProto::dtype() const { + // @@protoc_insertion_point(field_get:tensorflow.TensorProto.dtype) + return static_cast< ::tensorflow::DataType >(dtype_); +} +inline void TensorProto::set_dtype(::tensorflow::DataType value) { + + dtype_ = value; + // @@protoc_insertion_point(field_set:tensorflow.TensorProto.dtype) +} + +// .tensorflow.TensorShapeProto tensor_shape = 2; +inline bool TensorProto::has_tensor_shape() const { + return this != internal_default_instance() && tensor_shape_ != NULL; +} +inline const ::tensorflow::TensorShapeProto& TensorProto::_internal_tensor_shape() const { + return *tensor_shape_; +} +inline const ::tensorflow::TensorShapeProto& TensorProto::tensor_shape() const { + const ::tensorflow::TensorShapeProto* p = tensor_shape_; + // @@protoc_insertion_point(field_get:tensorflow.TensorProto.tensor_shape) + return p != NULL ? *p : *reinterpret_cast( + &::tensorflow::_TensorShapeProto_default_instance_); +} +inline ::tensorflow::TensorShapeProto* TensorProto::release_tensor_shape() { + // @@protoc_insertion_point(field_release:tensorflow.TensorProto.tensor_shape) + + ::tensorflow::TensorShapeProto* temp = tensor_shape_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + tensor_shape_ = NULL; + return temp; +} +inline ::tensorflow::TensorShapeProto* TensorProto::unsafe_arena_release_tensor_shape() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.TensorProto.tensor_shape) + + ::tensorflow::TensorShapeProto* temp = tensor_shape_; + tensor_shape_ = NULL; + return temp; +} +inline ::tensorflow::TensorShapeProto* TensorProto::mutable_tensor_shape() { + + if (tensor_shape_ == NULL) { + auto* p = CreateMaybeMessage<::tensorflow::TensorShapeProto>(GetArenaNoVirtual()); + tensor_shape_ = p; + } + // @@protoc_insertion_point(field_mutable:tensorflow.TensorProto.tensor_shape) + return tensor_shape_; +} +inline void TensorProto::set_allocated_tensor_shape(::tensorflow::TensorShapeProto* tensor_shape) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(tensor_shape_); + } + if (tensor_shape) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(tensor_shape)->GetArena(); + if (message_arena != submessage_arena) { + tensor_shape = ::google::protobuf::internal::GetOwnedMessage( + message_arena, tensor_shape, submessage_arena); + } + + } else { + + } + tensor_shape_ = tensor_shape; + // @@protoc_insertion_point(field_set_allocated:tensorflow.TensorProto.tensor_shape) +} + +// int32 version_number = 3; +inline void TensorProto::clear_version_number() { + version_number_ = 0; +} +inline ::google::protobuf::int32 TensorProto::version_number() const { + // @@protoc_insertion_point(field_get:tensorflow.TensorProto.version_number) + return version_number_; +} +inline void TensorProto::set_version_number(::google::protobuf::int32 value) { + + version_number_ = value; + // @@protoc_insertion_point(field_set:tensorflow.TensorProto.version_number) +} + +// bytes tensor_content = 4; +inline void TensorProto::clear_tensor_content() { + tensor_content_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& TensorProto::tensor_content() const { + // @@protoc_insertion_point(field_get:tensorflow.TensorProto.tensor_content) + return tensor_content_.Get(); +} +inline void TensorProto::set_tensor_content(const ::std::string& value) { + + tensor_content_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.TensorProto.tensor_content) +} +#if LANG_CXX11 +inline void TensorProto::set_tensor_content(::std::string&& value) { + + tensor_content_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.TensorProto.tensor_content) +} +#endif +inline void TensorProto::set_tensor_content(const char* value) { + GOOGLE_DCHECK(value != NULL); + + tensor_content_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.TensorProto.tensor_content) +} +inline void TensorProto::set_tensor_content(const void* value, + size_t size) { + + tensor_content_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.TensorProto.tensor_content) +} +inline ::std::string* TensorProto::mutable_tensor_content() { + + // @@protoc_insertion_point(field_mutable:tensorflow.TensorProto.tensor_content) + return tensor_content_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* TensorProto::release_tensor_content() { + // @@protoc_insertion_point(field_release:tensorflow.TensorProto.tensor_content) + + return tensor_content_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void TensorProto::set_allocated_tensor_content(::std::string* tensor_content) { + if (tensor_content != NULL) { + + } else { + + } + tensor_content_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), tensor_content, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.TensorProto.tensor_content) +} +inline ::std::string* TensorProto::unsafe_arena_release_tensor_content() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.TensorProto.tensor_content) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return tensor_content_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void TensorProto::unsafe_arena_set_allocated_tensor_content( + ::std::string* tensor_content) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (tensor_content != NULL) { + + } else { + + } + tensor_content_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + tensor_content, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.TensorProto.tensor_content) +} + +// repeated int32 half_val = 13 [packed = true]; +inline int TensorProto::half_val_size() const { + return half_val_.size(); +} +inline void TensorProto::clear_half_val() { + half_val_.Clear(); +} +inline ::google::protobuf::int32 TensorProto::half_val(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.TensorProto.half_val) + return half_val_.Get(index); +} +inline void TensorProto::set_half_val(int index, ::google::protobuf::int32 value) { + half_val_.Set(index, value); + // @@protoc_insertion_point(field_set:tensorflow.TensorProto.half_val) +} +inline void TensorProto::add_half_val(::google::protobuf::int32 value) { + half_val_.Add(value); + // @@protoc_insertion_point(field_add:tensorflow.TensorProto.half_val) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int32 >& +TensorProto::half_val() const { + // @@protoc_insertion_point(field_list:tensorflow.TensorProto.half_val) + return half_val_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int32 >* +TensorProto::mutable_half_val() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.TensorProto.half_val) + return &half_val_; +} + +// repeated float float_val = 5 [packed = true]; +inline int TensorProto::float_val_size() const { + return float_val_.size(); +} +inline void TensorProto::clear_float_val() { + float_val_.Clear(); +} +inline float TensorProto::float_val(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.TensorProto.float_val) + return float_val_.Get(index); +} +inline void TensorProto::set_float_val(int index, float value) { + float_val_.Set(index, value); + // @@protoc_insertion_point(field_set:tensorflow.TensorProto.float_val) +} +inline void TensorProto::add_float_val(float value) { + float_val_.Add(value); + // @@protoc_insertion_point(field_add:tensorflow.TensorProto.float_val) +} +inline const ::google::protobuf::RepeatedField< float >& +TensorProto::float_val() const { + // @@protoc_insertion_point(field_list:tensorflow.TensorProto.float_val) + return float_val_; +} +inline ::google::protobuf::RepeatedField< float >* +TensorProto::mutable_float_val() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.TensorProto.float_val) + return &float_val_; +} + +// repeated double double_val = 6 [packed = true]; +inline int TensorProto::double_val_size() const { + return double_val_.size(); +} +inline void TensorProto::clear_double_val() { + double_val_.Clear(); +} +inline double TensorProto::double_val(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.TensorProto.double_val) + return double_val_.Get(index); +} +inline void TensorProto::set_double_val(int index, double value) { + double_val_.Set(index, value); + // @@protoc_insertion_point(field_set:tensorflow.TensorProto.double_val) +} +inline void TensorProto::add_double_val(double value) { + double_val_.Add(value); + // @@protoc_insertion_point(field_add:tensorflow.TensorProto.double_val) +} +inline const ::google::protobuf::RepeatedField< double >& +TensorProto::double_val() const { + // @@protoc_insertion_point(field_list:tensorflow.TensorProto.double_val) + return double_val_; +} +inline ::google::protobuf::RepeatedField< double >* +TensorProto::mutable_double_val() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.TensorProto.double_val) + return &double_val_; +} + +// repeated int32 int_val = 7 [packed = true]; +inline int TensorProto::int_val_size() const { + return int_val_.size(); +} +inline void TensorProto::clear_int_val() { + int_val_.Clear(); +} +inline ::google::protobuf::int32 TensorProto::int_val(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.TensorProto.int_val) + return int_val_.Get(index); +} +inline void TensorProto::set_int_val(int index, ::google::protobuf::int32 value) { + int_val_.Set(index, value); + // @@protoc_insertion_point(field_set:tensorflow.TensorProto.int_val) +} +inline void TensorProto::add_int_val(::google::protobuf::int32 value) { + int_val_.Add(value); + // @@protoc_insertion_point(field_add:tensorflow.TensorProto.int_val) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int32 >& +TensorProto::int_val() const { + // @@protoc_insertion_point(field_list:tensorflow.TensorProto.int_val) + return int_val_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int32 >* +TensorProto::mutable_int_val() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.TensorProto.int_val) + return &int_val_; +} + +// repeated bytes string_val = 8; +inline int TensorProto::string_val_size() const { + return string_val_.size(); +} +inline void TensorProto::clear_string_val() { + string_val_.Clear(); +} +inline const ::std::string& TensorProto::string_val(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.TensorProto.string_val) + return string_val_.Get(index); +} +inline ::std::string* TensorProto::mutable_string_val(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.TensorProto.string_val) + return string_val_.Mutable(index); +} +inline void TensorProto::set_string_val(int index, const ::std::string& value) { + // @@protoc_insertion_point(field_set:tensorflow.TensorProto.string_val) + string_val_.Mutable(index)->assign(value); +} +#if LANG_CXX11 +inline void TensorProto::set_string_val(int index, ::std::string&& value) { + // @@protoc_insertion_point(field_set:tensorflow.TensorProto.string_val) + string_val_.Mutable(index)->assign(std::move(value)); +} +#endif +inline void TensorProto::set_string_val(int index, const char* value) { + GOOGLE_DCHECK(value != NULL); + string_val_.Mutable(index)->assign(value); + // @@protoc_insertion_point(field_set_char:tensorflow.TensorProto.string_val) +} +inline void TensorProto::set_string_val(int index, const void* value, size_t size) { + string_val_.Mutable(index)->assign( + reinterpret_cast(value), size); + // @@protoc_insertion_point(field_set_pointer:tensorflow.TensorProto.string_val) +} +inline ::std::string* TensorProto::add_string_val() { + // @@protoc_insertion_point(field_add_mutable:tensorflow.TensorProto.string_val) + return string_val_.Add(); +} +inline void TensorProto::add_string_val(const ::std::string& value) { + string_val_.Add()->assign(value); + // @@protoc_insertion_point(field_add:tensorflow.TensorProto.string_val) +} +#if LANG_CXX11 +inline void TensorProto::add_string_val(::std::string&& value) { + string_val_.Add(std::move(value)); + // @@protoc_insertion_point(field_add:tensorflow.TensorProto.string_val) +} +#endif +inline void TensorProto::add_string_val(const char* value) { + GOOGLE_DCHECK(value != NULL); + string_val_.Add()->assign(value); + // @@protoc_insertion_point(field_add_char:tensorflow.TensorProto.string_val) +} +inline void TensorProto::add_string_val(const void* value, size_t size) { + string_val_.Add()->assign(reinterpret_cast(value), size); + // @@protoc_insertion_point(field_add_pointer:tensorflow.TensorProto.string_val) +} +inline const ::google::protobuf::RepeatedPtrField< ::std::string>& +TensorProto::string_val() const { + // @@protoc_insertion_point(field_list:tensorflow.TensorProto.string_val) + return string_val_; +} +inline ::google::protobuf::RepeatedPtrField< ::std::string>* +TensorProto::mutable_string_val() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.TensorProto.string_val) + return &string_val_; +} + +// repeated float scomplex_val = 9 [packed = true]; +inline int TensorProto::scomplex_val_size() const { + return scomplex_val_.size(); +} +inline void TensorProto::clear_scomplex_val() { + scomplex_val_.Clear(); +} +inline float TensorProto::scomplex_val(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.TensorProto.scomplex_val) + return scomplex_val_.Get(index); +} +inline void TensorProto::set_scomplex_val(int index, float value) { + scomplex_val_.Set(index, value); + // @@protoc_insertion_point(field_set:tensorflow.TensorProto.scomplex_val) +} +inline void TensorProto::add_scomplex_val(float value) { + scomplex_val_.Add(value); + // @@protoc_insertion_point(field_add:tensorflow.TensorProto.scomplex_val) +} +inline const ::google::protobuf::RepeatedField< float >& +TensorProto::scomplex_val() const { + // @@protoc_insertion_point(field_list:tensorflow.TensorProto.scomplex_val) + return scomplex_val_; +} +inline ::google::protobuf::RepeatedField< float >* +TensorProto::mutable_scomplex_val() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.TensorProto.scomplex_val) + return &scomplex_val_; +} + +// repeated int64 int64_val = 10 [packed = true]; +inline int TensorProto::int64_val_size() const { + return int64_val_.size(); +} +inline void TensorProto::clear_int64_val() { + int64_val_.Clear(); +} +inline ::google::protobuf::int64 TensorProto::int64_val(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.TensorProto.int64_val) + return int64_val_.Get(index); +} +inline void TensorProto::set_int64_val(int index, ::google::protobuf::int64 value) { + int64_val_.Set(index, value); + // @@protoc_insertion_point(field_set:tensorflow.TensorProto.int64_val) +} +inline void TensorProto::add_int64_val(::google::protobuf::int64 value) { + int64_val_.Add(value); + // @@protoc_insertion_point(field_add:tensorflow.TensorProto.int64_val) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& +TensorProto::int64_val() const { + // @@protoc_insertion_point(field_list:tensorflow.TensorProto.int64_val) + return int64_val_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* +TensorProto::mutable_int64_val() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.TensorProto.int64_val) + return &int64_val_; +} + +// repeated bool bool_val = 11 [packed = true]; +inline int TensorProto::bool_val_size() const { + return bool_val_.size(); +} +inline void TensorProto::clear_bool_val() { + bool_val_.Clear(); +} +inline bool TensorProto::bool_val(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.TensorProto.bool_val) + return bool_val_.Get(index); +} +inline void TensorProto::set_bool_val(int index, bool value) { + bool_val_.Set(index, value); + // @@protoc_insertion_point(field_set:tensorflow.TensorProto.bool_val) +} +inline void TensorProto::add_bool_val(bool value) { + bool_val_.Add(value); + // @@protoc_insertion_point(field_add:tensorflow.TensorProto.bool_val) +} +inline const ::google::protobuf::RepeatedField< bool >& +TensorProto::bool_val() const { + // @@protoc_insertion_point(field_list:tensorflow.TensorProto.bool_val) + return bool_val_; +} +inline ::google::protobuf::RepeatedField< bool >* +TensorProto::mutable_bool_val() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.TensorProto.bool_val) + return &bool_val_; +} + +// repeated double dcomplex_val = 12 [packed = true]; +inline int TensorProto::dcomplex_val_size() const { + return dcomplex_val_.size(); +} +inline void TensorProto::clear_dcomplex_val() { + dcomplex_val_.Clear(); +} +inline double TensorProto::dcomplex_val(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.TensorProto.dcomplex_val) + return dcomplex_val_.Get(index); +} +inline void TensorProto::set_dcomplex_val(int index, double value) { + dcomplex_val_.Set(index, value); + // @@protoc_insertion_point(field_set:tensorflow.TensorProto.dcomplex_val) +} +inline void TensorProto::add_dcomplex_val(double value) { + dcomplex_val_.Add(value); + // @@protoc_insertion_point(field_add:tensorflow.TensorProto.dcomplex_val) +} +inline const ::google::protobuf::RepeatedField< double >& +TensorProto::dcomplex_val() const { + // @@protoc_insertion_point(field_list:tensorflow.TensorProto.dcomplex_val) + return dcomplex_val_; +} +inline ::google::protobuf::RepeatedField< double >* +TensorProto::mutable_dcomplex_val() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.TensorProto.dcomplex_val) + return &dcomplex_val_; +} + +// repeated .tensorflow.ResourceHandleProto resource_handle_val = 14; +inline int TensorProto::resource_handle_val_size() const { + return resource_handle_val_.size(); +} +inline ::tensorflow::ResourceHandleProto* TensorProto::mutable_resource_handle_val(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.TensorProto.resource_handle_val) + return resource_handle_val_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::ResourceHandleProto >* +TensorProto::mutable_resource_handle_val() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.TensorProto.resource_handle_val) + return &resource_handle_val_; +} +inline const ::tensorflow::ResourceHandleProto& TensorProto::resource_handle_val(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.TensorProto.resource_handle_val) + return resource_handle_val_.Get(index); +} +inline ::tensorflow::ResourceHandleProto* TensorProto::add_resource_handle_val() { + // @@protoc_insertion_point(field_add:tensorflow.TensorProto.resource_handle_val) + return resource_handle_val_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::ResourceHandleProto >& +TensorProto::resource_handle_val() const { + // @@protoc_insertion_point(field_list:tensorflow.TensorProto.resource_handle_val) + return resource_handle_val_; +} + +// repeated .tensorflow.VariantTensorDataProto variant_val = 15; +inline int TensorProto::variant_val_size() const { + return variant_val_.size(); +} +inline void TensorProto::clear_variant_val() { + variant_val_.Clear(); +} +inline ::tensorflow::VariantTensorDataProto* TensorProto::mutable_variant_val(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.TensorProto.variant_val) + return variant_val_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::VariantTensorDataProto >* +TensorProto::mutable_variant_val() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.TensorProto.variant_val) + return &variant_val_; +} +inline const ::tensorflow::VariantTensorDataProto& TensorProto::variant_val(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.TensorProto.variant_val) + return variant_val_.Get(index); +} +inline ::tensorflow::VariantTensorDataProto* TensorProto::add_variant_val() { + // @@protoc_insertion_point(field_add:tensorflow.TensorProto.variant_val) + return variant_val_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::VariantTensorDataProto >& +TensorProto::variant_val() const { + // @@protoc_insertion_point(field_list:tensorflow.TensorProto.variant_val) + return variant_val_; +} + +// repeated uint32 uint32_val = 16 [packed = true]; +inline int TensorProto::uint32_val_size() const { + return uint32_val_.size(); +} +inline void TensorProto::clear_uint32_val() { + uint32_val_.Clear(); +} +inline ::google::protobuf::uint32 TensorProto::uint32_val(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.TensorProto.uint32_val) + return uint32_val_.Get(index); +} +inline void TensorProto::set_uint32_val(int index, ::google::protobuf::uint32 value) { + uint32_val_.Set(index, value); + // @@protoc_insertion_point(field_set:tensorflow.TensorProto.uint32_val) +} +inline void TensorProto::add_uint32_val(::google::protobuf::uint32 value) { + uint32_val_.Add(value); + // @@protoc_insertion_point(field_add:tensorflow.TensorProto.uint32_val) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::uint32 >& +TensorProto::uint32_val() const { + // @@protoc_insertion_point(field_list:tensorflow.TensorProto.uint32_val) + return uint32_val_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::uint32 >* +TensorProto::mutable_uint32_val() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.TensorProto.uint32_val) + return &uint32_val_; +} + +// repeated uint64 uint64_val = 17 [packed = true]; +inline int TensorProto::uint64_val_size() const { + return uint64_val_.size(); +} +inline void TensorProto::clear_uint64_val() { + uint64_val_.Clear(); +} +inline ::google::protobuf::uint64 TensorProto::uint64_val(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.TensorProto.uint64_val) + return uint64_val_.Get(index); +} +inline void TensorProto::set_uint64_val(int index, ::google::protobuf::uint64 value) { + uint64_val_.Set(index, value); + // @@protoc_insertion_point(field_set:tensorflow.TensorProto.uint64_val) +} +inline void TensorProto::add_uint64_val(::google::protobuf::uint64 value) { + uint64_val_.Add(value); + // @@protoc_insertion_point(field_add:tensorflow.TensorProto.uint64_val) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::uint64 >& +TensorProto::uint64_val() const { + // @@protoc_insertion_point(field_list:tensorflow.TensorProto.uint64_val) + return uint64_val_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::uint64 >* +TensorProto::mutable_uint64_val() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.TensorProto.uint64_val) + return &uint64_val_; +} + +// ------------------------------------------------------------------- + +// VariantTensorDataProto + +// string type_name = 1; +inline void VariantTensorDataProto::clear_type_name() { + type_name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& VariantTensorDataProto::type_name() const { + // @@protoc_insertion_point(field_get:tensorflow.VariantTensorDataProto.type_name) + return type_name_.Get(); +} +inline void VariantTensorDataProto::set_type_name(const ::std::string& value) { + + type_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.VariantTensorDataProto.type_name) +} +#if LANG_CXX11 +inline void VariantTensorDataProto::set_type_name(::std::string&& value) { + + type_name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.VariantTensorDataProto.type_name) +} +#endif +inline void VariantTensorDataProto::set_type_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + type_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.VariantTensorDataProto.type_name) +} +inline void VariantTensorDataProto::set_type_name(const char* value, + size_t size) { + + type_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.VariantTensorDataProto.type_name) +} +inline ::std::string* VariantTensorDataProto::mutable_type_name() { + + // @@protoc_insertion_point(field_mutable:tensorflow.VariantTensorDataProto.type_name) + return type_name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* VariantTensorDataProto::release_type_name() { + // @@protoc_insertion_point(field_release:tensorflow.VariantTensorDataProto.type_name) + + return type_name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void VariantTensorDataProto::set_allocated_type_name(::std::string* type_name) { + if (type_name != NULL) { + + } else { + + } + type_name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), type_name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.VariantTensorDataProto.type_name) +} +inline ::std::string* VariantTensorDataProto::unsafe_arena_release_type_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.VariantTensorDataProto.type_name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return type_name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void VariantTensorDataProto::unsafe_arena_set_allocated_type_name( + ::std::string* type_name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (type_name != NULL) { + + } else { + + } + type_name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + type_name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.VariantTensorDataProto.type_name) +} + +// bytes metadata = 2; +inline void VariantTensorDataProto::clear_metadata() { + metadata_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& VariantTensorDataProto::metadata() const { + // @@protoc_insertion_point(field_get:tensorflow.VariantTensorDataProto.metadata) + return metadata_.Get(); +} +inline void VariantTensorDataProto::set_metadata(const ::std::string& value) { + + metadata_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.VariantTensorDataProto.metadata) +} +#if LANG_CXX11 +inline void VariantTensorDataProto::set_metadata(::std::string&& value) { + + metadata_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.VariantTensorDataProto.metadata) +} +#endif +inline void VariantTensorDataProto::set_metadata(const char* value) { + GOOGLE_DCHECK(value != NULL); + + metadata_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.VariantTensorDataProto.metadata) +} +inline void VariantTensorDataProto::set_metadata(const void* value, + size_t size) { + + metadata_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.VariantTensorDataProto.metadata) +} +inline ::std::string* VariantTensorDataProto::mutable_metadata() { + + // @@protoc_insertion_point(field_mutable:tensorflow.VariantTensorDataProto.metadata) + return metadata_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* VariantTensorDataProto::release_metadata() { + // @@protoc_insertion_point(field_release:tensorflow.VariantTensorDataProto.metadata) + + return metadata_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void VariantTensorDataProto::set_allocated_metadata(::std::string* metadata) { + if (metadata != NULL) { + + } else { + + } + metadata_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), metadata, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.VariantTensorDataProto.metadata) +} +inline ::std::string* VariantTensorDataProto::unsafe_arena_release_metadata() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.VariantTensorDataProto.metadata) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return metadata_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void VariantTensorDataProto::unsafe_arena_set_allocated_metadata( + ::std::string* metadata) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (metadata != NULL) { + + } else { + + } + metadata_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + metadata, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.VariantTensorDataProto.metadata) +} + +// repeated .tensorflow.TensorProto tensors = 3; +inline int VariantTensorDataProto::tensors_size() const { + return tensors_.size(); +} +inline void VariantTensorDataProto::clear_tensors() { + tensors_.Clear(); +} +inline ::tensorflow::TensorProto* VariantTensorDataProto::mutable_tensors(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.VariantTensorDataProto.tensors) + return tensors_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::TensorProto >* +VariantTensorDataProto::mutable_tensors() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.VariantTensorDataProto.tensors) + return &tensors_; +} +inline const ::tensorflow::TensorProto& VariantTensorDataProto::tensors(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.VariantTensorDataProto.tensors) + return tensors_.Get(index); +} +inline ::tensorflow::TensorProto* VariantTensorDataProto::add_tensors() { + // @@protoc_insertion_point(field_add:tensorflow.VariantTensorDataProto.tensors) + return tensors_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::TensorProto >& +VariantTensorDataProto::tensors() const { + // @@protoc_insertion_point(field_list:tensorflow.VariantTensorDataProto.tensors) + return tensors_; +} + +#ifdef __GNUC__ + #pragma GCC diagnostic pop +#endif // __GNUC__ +// ------------------------------------------------------------------- + + +// @@protoc_insertion_point(namespace_scope) + +} // namespace tensorflow + +// @@protoc_insertion_point(global_scope) + +#endif // PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2ftensor_2eproto diff --git a/linux/include/tensorflow/core/framework/tensor.pb_text-impl.h b/linux/include/tensorflow/core/framework/tensor.pb_text-impl.h new file mode 100644 index 000000000..246fcf272 --- /dev/null +++ b/linux/include/tensorflow/core/framework/tensor.pb_text-impl.h @@ -0,0 +1,38 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_framework_tensor_proto_IMPL_H_ +#define tensorflow_core_framework_tensor_proto_IMPL_H_ + +#include "tensorflow/core/framework/resource_handle.pb.h" +#include "tensorflow/core/framework/resource_handle.pb_text-impl.h" +#include "tensorflow/core/framework/tensor.pb.h" +#include "tensorflow/core/framework/tensor.pb_text.h" +#include "tensorflow/core/framework/tensor_shape.pb.h" +#include "tensorflow/core/framework/tensor_shape.pb_text-impl.h" +#include "tensorflow/core/framework/types.pb.h" +#include "tensorflow/core/framework/types.pb_text-impl.h" +#include "tensorflow/core/lib/strings/proto_text_util.h" +#include "tensorflow/core/lib/strings/scanner.h" + +namespace tensorflow { + +namespace internal { + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::TensorProto& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::TensorProto* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::VariantTensorDataProto& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::VariantTensorDataProto* msg); + +} // namespace internal + +} // namespace tensorflow + +#endif // tensorflow_core_framework_tensor_proto_IMPL_H_ diff --git a/linux/include/tensorflow/core/framework/tensor.pb_text.h b/linux/include/tensorflow/core/framework/tensor.pb_text.h new file mode 100644 index 000000000..6d073b0b1 --- /dev/null +++ b/linux/include/tensorflow/core/framework/tensor.pb_text.h @@ -0,0 +1,34 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_framework_tensor_proto_H_ +#define tensorflow_core_framework_tensor_proto_H_ + +#include "tensorflow/core/framework/tensor.pb.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/protobuf.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +// Message-text conversion for tensorflow.TensorProto +string ProtoDebugString( + const ::tensorflow::TensorProto& msg); +string ProtoShortDebugString( + const ::tensorflow::TensorProto& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::TensorProto* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.VariantTensorDataProto +string ProtoDebugString( + const ::tensorflow::VariantTensorDataProto& msg); +string ProtoShortDebugString( + const ::tensorflow::VariantTensorDataProto& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::VariantTensorDataProto* msg) + TF_MUST_USE_RESULT; + +} // namespace tensorflow + +#endif // tensorflow_core_framework_tensor_proto_H_ diff --git a/linux/include/tensorflow/core/framework/tensor_description.pb.h b/linux/include/tensorflow/core/framework/tensor_description.pb.h new file mode 100644 index 000000000..d80a47a5f --- /dev/null +++ b/linux/include/tensorflow/core/framework/tensor_description.pb.h @@ -0,0 +1,366 @@ +// Generated by the protocol buffer compiler. DO NOT EDIT! +// source: tensorflow/core/framework/tensor_description.proto + +#ifndef PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2ftensor_5fdescription_2eproto +#define PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2ftensor_5fdescription_2eproto + +#include + +#include + +#if GOOGLE_PROTOBUF_VERSION < 3006000 +#error This file was generated by a newer version of protoc which is +#error incompatible with your Protocol Buffer headers. Please update +#error your headers. +#endif +#if 3006000 < GOOGLE_PROTOBUF_MIN_PROTOC_VERSION +#error This file was generated by an older version of protoc which is +#error incompatible with your Protocol Buffer headers. Please +#error regenerate this file with a newer version of protoc. +#endif + +#include +#include +#include +#include +#include +#include +#include +#include +#include // IWYU pragma: export +#include // IWYU pragma: export +#include +#include "tensorflow/core/framework/types.pb.h" +#include "tensorflow/core/framework/tensor_shape.pb.h" +#include "tensorflow/core/framework/allocation_description.pb.h" +// @@protoc_insertion_point(includes) +#define PROTOBUF_INTERNAL_EXPORT_protobuf_tensorflow_2fcore_2fframework_2ftensor_5fdescription_2eproto + +namespace protobuf_tensorflow_2fcore_2fframework_2ftensor_5fdescription_2eproto { +// Internal implementation detail -- do not use these members. +struct TableStruct { + static const ::google::protobuf::internal::ParseTableField entries[]; + static const ::google::protobuf::internal::AuxillaryParseTableField aux[]; + static const ::google::protobuf::internal::ParseTable schema[1]; + static const ::google::protobuf::internal::FieldMetadata field_metadata[]; + static const ::google::protobuf::internal::SerializationTable serialization_table[]; + static const ::google::protobuf::uint32 offsets[]; +}; +void AddDescriptors(); +} // namespace protobuf_tensorflow_2fcore_2fframework_2ftensor_5fdescription_2eproto +namespace tensorflow { +class TensorDescription; +class TensorDescriptionDefaultTypeInternal; +extern TensorDescriptionDefaultTypeInternal _TensorDescription_default_instance_; +} // namespace tensorflow +namespace google { +namespace protobuf { +template<> ::tensorflow::TensorDescription* Arena::CreateMaybeMessage<::tensorflow::TensorDescription>(Arena*); +} // namespace protobuf +} // namespace google +namespace tensorflow { + +// =================================================================== + +class TensorDescription : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.TensorDescription) */ { + public: + TensorDescription(); + virtual ~TensorDescription(); + + TensorDescription(const TensorDescription& from); + + inline TensorDescription& operator=(const TensorDescription& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + TensorDescription(TensorDescription&& from) noexcept + : TensorDescription() { + *this = ::std::move(from); + } + + inline TensorDescription& operator=(TensorDescription&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const TensorDescription& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const TensorDescription* internal_default_instance() { + return reinterpret_cast( + &_TensorDescription_default_instance_); + } + static constexpr int kIndexInFileMessages = + 0; + + void UnsafeArenaSwap(TensorDescription* other); + void Swap(TensorDescription* other); + friend void swap(TensorDescription& a, TensorDescription& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline TensorDescription* New() const final { + return CreateMaybeMessage(NULL); + } + + TensorDescription* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const TensorDescription& from); + void MergeFrom(const TensorDescription& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(TensorDescription* other); + protected: + explicit TensorDescription(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // .tensorflow.TensorShapeProto shape = 2; + bool has_shape() const; + void clear_shape(); + static const int kShapeFieldNumber = 2; + private: + const ::tensorflow::TensorShapeProto& _internal_shape() const; + public: + const ::tensorflow::TensorShapeProto& shape() const; + ::tensorflow::TensorShapeProto* release_shape(); + ::tensorflow::TensorShapeProto* mutable_shape(); + void set_allocated_shape(::tensorflow::TensorShapeProto* shape); + void unsafe_arena_set_allocated_shape( + ::tensorflow::TensorShapeProto* shape); + ::tensorflow::TensorShapeProto* unsafe_arena_release_shape(); + + // .tensorflow.AllocationDescription allocation_description = 4; + bool has_allocation_description() const; + void clear_allocation_description(); + static const int kAllocationDescriptionFieldNumber = 4; + private: + const ::tensorflow::AllocationDescription& _internal_allocation_description() const; + public: + const ::tensorflow::AllocationDescription& allocation_description() const; + ::tensorflow::AllocationDescription* release_allocation_description(); + ::tensorflow::AllocationDescription* mutable_allocation_description(); + void set_allocated_allocation_description(::tensorflow::AllocationDescription* allocation_description); + void unsafe_arena_set_allocated_allocation_description( + ::tensorflow::AllocationDescription* allocation_description); + ::tensorflow::AllocationDescription* unsafe_arena_release_allocation_description(); + + // .tensorflow.DataType dtype = 1; + void clear_dtype(); + static const int kDtypeFieldNumber = 1; + ::tensorflow::DataType dtype() const; + void set_dtype(::tensorflow::DataType value); + + // @@protoc_insertion_point(class_scope:tensorflow.TensorDescription) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::tensorflow::TensorShapeProto* shape_; + ::tensorflow::AllocationDescription* allocation_description_; + int dtype_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2ftensor_5fdescription_2eproto::TableStruct; +}; +// =================================================================== + + +// =================================================================== + +#ifdef __GNUC__ + #pragma GCC diagnostic push + #pragma GCC diagnostic ignored "-Wstrict-aliasing" +#endif // __GNUC__ +// TensorDescription + +// .tensorflow.DataType dtype = 1; +inline void TensorDescription::clear_dtype() { + dtype_ = 0; +} +inline ::tensorflow::DataType TensorDescription::dtype() const { + // @@protoc_insertion_point(field_get:tensorflow.TensorDescription.dtype) + return static_cast< ::tensorflow::DataType >(dtype_); +} +inline void TensorDescription::set_dtype(::tensorflow::DataType value) { + + dtype_ = value; + // @@protoc_insertion_point(field_set:tensorflow.TensorDescription.dtype) +} + +// .tensorflow.TensorShapeProto shape = 2; +inline bool TensorDescription::has_shape() const { + return this != internal_default_instance() && shape_ != NULL; +} +inline const ::tensorflow::TensorShapeProto& TensorDescription::_internal_shape() const { + return *shape_; +} +inline const ::tensorflow::TensorShapeProto& TensorDescription::shape() const { + const ::tensorflow::TensorShapeProto* p = shape_; + // @@protoc_insertion_point(field_get:tensorflow.TensorDescription.shape) + return p != NULL ? *p : *reinterpret_cast( + &::tensorflow::_TensorShapeProto_default_instance_); +} +inline ::tensorflow::TensorShapeProto* TensorDescription::release_shape() { + // @@protoc_insertion_point(field_release:tensorflow.TensorDescription.shape) + + ::tensorflow::TensorShapeProto* temp = shape_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + shape_ = NULL; + return temp; +} +inline ::tensorflow::TensorShapeProto* TensorDescription::unsafe_arena_release_shape() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.TensorDescription.shape) + + ::tensorflow::TensorShapeProto* temp = shape_; + shape_ = NULL; + return temp; +} +inline ::tensorflow::TensorShapeProto* TensorDescription::mutable_shape() { + + if (shape_ == NULL) { + auto* p = CreateMaybeMessage<::tensorflow::TensorShapeProto>(GetArenaNoVirtual()); + shape_ = p; + } + // @@protoc_insertion_point(field_mutable:tensorflow.TensorDescription.shape) + return shape_; +} +inline void TensorDescription::set_allocated_shape(::tensorflow::TensorShapeProto* shape) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(shape_); + } + if (shape) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(shape)->GetArena(); + if (message_arena != submessage_arena) { + shape = ::google::protobuf::internal::GetOwnedMessage( + message_arena, shape, submessage_arena); + } + + } else { + + } + shape_ = shape; + // @@protoc_insertion_point(field_set_allocated:tensorflow.TensorDescription.shape) +} + +// .tensorflow.AllocationDescription allocation_description = 4; +inline bool TensorDescription::has_allocation_description() const { + return this != internal_default_instance() && allocation_description_ != NULL; +} +inline const ::tensorflow::AllocationDescription& TensorDescription::_internal_allocation_description() const { + return *allocation_description_; +} +inline const ::tensorflow::AllocationDescription& TensorDescription::allocation_description() const { + const ::tensorflow::AllocationDescription* p = allocation_description_; + // @@protoc_insertion_point(field_get:tensorflow.TensorDescription.allocation_description) + return p != NULL ? *p : *reinterpret_cast( + &::tensorflow::_AllocationDescription_default_instance_); +} +inline ::tensorflow::AllocationDescription* TensorDescription::release_allocation_description() { + // @@protoc_insertion_point(field_release:tensorflow.TensorDescription.allocation_description) + + ::tensorflow::AllocationDescription* temp = allocation_description_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + allocation_description_ = NULL; + return temp; +} +inline ::tensorflow::AllocationDescription* TensorDescription::unsafe_arena_release_allocation_description() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.TensorDescription.allocation_description) + + ::tensorflow::AllocationDescription* temp = allocation_description_; + allocation_description_ = NULL; + return temp; +} +inline ::tensorflow::AllocationDescription* TensorDescription::mutable_allocation_description() { + + if (allocation_description_ == NULL) { + auto* p = CreateMaybeMessage<::tensorflow::AllocationDescription>(GetArenaNoVirtual()); + allocation_description_ = p; + } + // @@protoc_insertion_point(field_mutable:tensorflow.TensorDescription.allocation_description) + return allocation_description_; +} +inline void TensorDescription::set_allocated_allocation_description(::tensorflow::AllocationDescription* allocation_description) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(allocation_description_); + } + if (allocation_description) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(allocation_description)->GetArena(); + if (message_arena != submessage_arena) { + allocation_description = ::google::protobuf::internal::GetOwnedMessage( + message_arena, allocation_description, submessage_arena); + } + + } else { + + } + allocation_description_ = allocation_description; + // @@protoc_insertion_point(field_set_allocated:tensorflow.TensorDescription.allocation_description) +} + +#ifdef __GNUC__ + #pragma GCC diagnostic pop +#endif // __GNUC__ + +// @@protoc_insertion_point(namespace_scope) + +} // namespace tensorflow + +// @@protoc_insertion_point(global_scope) + +#endif // PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2ftensor_5fdescription_2eproto diff --git a/linux/include/tensorflow/core/framework/tensor_description.pb_text-impl.h b/linux/include/tensorflow/core/framework/tensor_description.pb_text-impl.h new file mode 100644 index 000000000..7c0005941 --- /dev/null +++ b/linux/include/tensorflow/core/framework/tensor_description.pb_text-impl.h @@ -0,0 +1,31 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_framework_tensor_description_proto_IMPL_H_ +#define tensorflow_core_framework_tensor_description_proto_IMPL_H_ + +#include "tensorflow/core/framework/allocation_description.pb.h" +#include "tensorflow/core/framework/allocation_description.pb_text-impl.h" +#include "tensorflow/core/framework/tensor_description.pb.h" +#include "tensorflow/core/framework/tensor_description.pb_text.h" +#include "tensorflow/core/framework/tensor_shape.pb.h" +#include "tensorflow/core/framework/tensor_shape.pb_text-impl.h" +#include "tensorflow/core/framework/types.pb.h" +#include "tensorflow/core/framework/types.pb_text-impl.h" +#include "tensorflow/core/lib/strings/proto_text_util.h" +#include "tensorflow/core/lib/strings/scanner.h" + +namespace tensorflow { + +namespace internal { + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::TensorDescription& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::TensorDescription* msg); + +} // namespace internal + +} // namespace tensorflow + +#endif // tensorflow_core_framework_tensor_description_proto_IMPL_H_ diff --git a/linux/include/tensorflow/core/framework/tensor_description.pb_text.h b/linux/include/tensorflow/core/framework/tensor_description.pb_text.h new file mode 100644 index 000000000..eed68fd79 --- /dev/null +++ b/linux/include/tensorflow/core/framework/tensor_description.pb_text.h @@ -0,0 +1,24 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_framework_tensor_description_proto_H_ +#define tensorflow_core_framework_tensor_description_proto_H_ + +#include "tensorflow/core/framework/tensor_description.pb.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/protobuf.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +// Message-text conversion for tensorflow.TensorDescription +string ProtoDebugString( + const ::tensorflow::TensorDescription& msg); +string ProtoShortDebugString( + const ::tensorflow::TensorDescription& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::TensorDescription* msg) + TF_MUST_USE_RESULT; + +} // namespace tensorflow + +#endif // tensorflow_core_framework_tensor_description_proto_H_ diff --git a/linux/include/tensorflow/core/framework/tensor_reference.h b/linux/include/tensorflow/core/framework/tensor_reference.h new file mode 100644 index 000000000..37e588d4f --- /dev/null +++ b/linux/include/tensorflow/core/framework/tensor_reference.h @@ -0,0 +1,77 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_FRAMEWORK_TENSOR_REFERENCE_H_ +#define TENSORFLOW_FRAMEWORK_TENSOR_REFERENCE_H_ + +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/lib/gtl/inlined_vector.h" + +namespace tensorflow { + +// An opaque class that holds a reference to an underlying TensorBuffer. +// Unlike Tensor, it does not have any shape or type information, so +// it is cheaper to construct/move, but the only thing you can really do +// with it is Unref it, which releases one of the references to the underlying +// TensorBuffer. +// IMPORTANT: If you do not call Unref(), you will likely leak tensor memory. +class TensorReference { + public: + // Take the reference of the root buffer so the size will be more accurate + explicit TensorReference(const Tensor& tensor); + + ~TensorReference() {} + + void Unref() const { + if (buf_) buf_->Unref(); + } + + // Return an estimate of the total bytes being kept alive by this reference. + size_t TotalBytes() const { + // We add 128 as a baseline to account for per-Tensor metadata + return 128 + (buf_ ? buf_->size() : 0); + } + + void FillDescription(AllocationDescription* description) const { + if (buf_) buf_->FillAllocationDescription(description); + } + + // Convenience function for de-duplicating tensor references. + bool SharesBufferWith(const TensorReference& t) const { + return buf_ == t.buf_; + } + + // Convenience function for de-duplicating tensor references. + bool SharesBufferWith(const Tensor& t) const { + return buf_ == (t.buf_ ? t.buf_->root_buffer() : nullptr); + } + + // Convenience function for de-duplicating tensor references. + size_t BufferHash() const { return std::hash()(buf_); } + + // A constructor used only for tests + explicit TensorReference(TensorBuffer* test_buffer) : buf_(test_buffer) { + if (buf_) buf_->Ref(); + } + + private: + TensorBuffer* buf_; +}; + +typedef gtl::InlinedVector TensorReferenceVector; + +} // namespace tensorflow + +#endif // TENSORFLOW_FRAMEWORK_TENSOR_REFERENCE_H_ diff --git a/linux/include/tensorflow/core/framework/tensor_shape.h b/linux/include/tensorflow/core/framework/tensor_shape.h new file mode 100644 index 000000000..be7e740c3 --- /dev/null +++ b/linux/include/tensorflow/core/framework/tensor_shape.h @@ -0,0 +1,550 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_FRAMEWORK_TENSOR_SHAPE_H_ +#define TENSORFLOW_CORE_FRAMEWORK_TENSOR_SHAPE_H_ + +#include + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/types.pb.h" +#include "tensorflow/core/lib/core/errors.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/lib/core/stringpiece.h" +#include "tensorflow/core/lib/gtl/array_slice.h" +#include "tensorflow/core/lib/gtl/inlined_vector.h" +#include "tensorflow/core/lib/strings/str_util.h" +#include "tensorflow/core/platform/logging.h" + +namespace tensorflow { + +// START_SKIP_DOXYGEN +template +class TensorShapeIter; +class TensorShape; +class TensorShapeProto; +class PartialTensorShape; +// END_SKIP_DOXYGEN + +/// Internal representation for both TensorShape and PartialTensorShape. +class TensorShapeRep { + public: + ~TensorShapeRep(); + + /// Copy the specified shape + TensorShapeRep(const TensorShapeRep& b); + void operator=(const TensorShapeRep& b); + + /// Move the specified shape. After moving, is safe for destruction and + // can be reassigned into, but its dimensions and number of elements can be + // nonsensical (e.g., negative dimension sizes, or number of elements not + // properly recomputed). + TensorShapeRep(TensorShapeRep&& b); + void operator=(TensorShapeRep&& b); + + /// Clear a tensor shape, producing the scalar shape. + void Clear(); + + // Maximum number of dimensions in a tensor. + // It's 254 because 255 = kUnknownRank is used to represent unknown rank. + static constexpr int MaxDimensions() { return 254; } + + /// \brief Returns the number of elements in the tensor. + /// + /// We use `int64` and not `size_t` to be compatible with `Eigen::Tensor` + /// which uses `ptrdiff_t`. For PartialTensorShape, -1 means not fully + /// defined. + int64 num_elements() const { return num_elements_; } + + /// For error messages. + string DebugString() const; + static string DebugString(const TensorShapeProto& proto); + + void DumpRep() const; // XXX + + protected: + // Constructable only via TensorShapeBase + TensorShapeRep() = default; + + void ClearAllButDataType(); + + // We use 16 bytes to represent a TensorShape. Because we need to + // be able to support full 64-bit dimension sizes and an arbitrary + // number of dimensions for a Tensor, but most tensor dimensions are + // significantly smaller than 64 bits and most tensors are 1, 2, or 3 + // dimensions, we have several representations. + // Rep16: Supports up to 6 dimensions where each dimension is < 2^16 - 1 + // Rep32: Supports up to 3 dimensions where each dimension is < 2^32 - 1 + // Rep64: Supports arbitrary dimensionality, 64-bit dimensions using + // an out of line vector. + // For PartialTensorShape, a dimension of static_cast(-1) is unknown. + // This value is not allowed in TensorShape either for format compatibility. + struct Rep16 { + uint16 dims_[6]; + }; + struct Rep32 { + uint32 dims_[3]; + }; + struct Rep64 { + gtl::InlinedVector* dims_; + }; + + // We use the max value of uint16 or uint32 to represent unknown shapes, so + // the maximum representable valid shape in these representations is one less. + static const int64 kMaxRep16 = std::numeric_limits::max() - 1; + static const int64 kMaxRep32 = std::numeric_limits::max() - 1; + static const uint16 kUnknownRep16 = std::numeric_limits::max(); + static const uint32 kUnknownRep32 = std::numeric_limits::max(); + + Rep16* as16() { return reinterpret_cast(buf()); } + Rep32* as32() { return reinterpret_cast(buf()); } + Rep64* as64() { return reinterpret_cast(buf()); } + + const Rep16* as16() const { return reinterpret_cast(buf()); } + const Rep32* as32() const { return reinterpret_cast(buf()); } + const Rep64* as64() const { return reinterpret_cast(buf()); } + + enum RepTag { REP16 = 0, REP32 = 1, REP_OUT_OF_LINE = 2 }; + + // Since we have a convenient extra byte available, we allow the + // Tensor class to store an 8-bit value in this extra storage. This + // allows it to store the Tensor's datatype enum value here and avoid + // an extra word of storage. + friend class Tensor; + friend class TensorShapeTestHelper; + DataType data_type() const { return static_cast(buf()[13]); } + void set_data_type(DataType dt) { + // We only have 8 bits available to store DataType, so make sure it fits + DCHECK_LT(static_cast(dt), 256u); + buf()[13] = static_cast(dt); + } + + // We store the number of dimensions in byte 14, and the RepTag in byte 15. + // Bytes [0..13] vary depending on the representation. + // A value of 255 indicates unknown rank in the PartialTensorShape case. + static const uint8 kUnknownRank = 255; + uint8 ndims_byte() const { return buf()[14]; } + void set_ndims_byte(uint8 nd) { buf()[14] = nd; } + + RepTag tag() const { return static_cast(buf()[15]); } + void set_tag(RepTag tag) { buf()[15] = static_cast(tag); } + + void set_num_elements(int64 n) { num_elements_ = n; } + + private: + void DestructorOutOfLine(); + void SlowCopyFrom(const TensorShapeRep& b); + + uint8* buf() { return &u_.buf[0]; } + const uint8* buf() const { return &u_.buf[0]; } + + union { + uint8 buf[16]; + // Force data to be aligned enough for a pointer. + Rep64* unused_aligner; + } u_; + int64 num_elements_; +}; + +/// Base class for TensorShape and PartialTensorShape. +/// The class is templatized by either TensorShape or PartialTensorShape to +/// allow skipping known/unknown checks in the TensorShape case, but the +/// representation is shared exactly for fast conversion. +template +class TensorShapeBase : public TensorShapeRep { + public: + /// \brief Construct a `TensorShapeBase` from the provided sizes. + /// REQUIRES: `dim_sizes[i] >= 0` (or >= -1 for PartialTensorShape) + explicit TensorShapeBase(gtl::ArraySlice dim_sizes); + TensorShapeBase(std::initializer_list dim_sizes) + : TensorShapeBase(gtl::ArraySlice(dim_sizes)) {} + + /// Construct an empty TensorShape, or an unknown rank PartialTensorShape + TensorShapeBase(); + + TensorShapeBase(const TensorShapeProto& proto); + + /// Returns `true` iff `proto` is a valid tensor shape. + // For TensorShape, the proto shape must be fully defined. + static bool IsValid(const TensorShapeProto& proto); + + /// Returns `OK` iff `proto` is a valid tensor shape, and a descriptive error + /// status otherwise. + static Status IsValidShape(const TensorShapeProto& proto); + + /// \brief Add a dimension to the end ("inner-most"). + /// REQUIRES: `size >= 0` + void AddDim(int64 size); + + /// Appends all the dimensions from `shape`. + void AppendShape(const TensorShapeBase& shape); + + /// \brief Insert a dimension somewhere in the `TensorShape`. + /// REQUIRES: `0 <= d <= dims()` + /// REQUIRES: `size >= 0` + void InsertDim(int d, int64 size); + + /// \brief Modifies the size of the dimension `d` to be `size` + /// REQUIRES: `0 <= d < dims()` + /// REQUIRES: `size >= 0` + void set_dim(int d, int64 size); + + /// \brief Removes dimension `d` from the `TensorShape`. + /// REQUIRES: `0 <= d < dims()` + void RemoveDim(int d) { + CHECK_GE(d, 0); + RemoveDimRange(d, d + 1); + } + + /// \brief Removes last `n` dimensions from the `TensorShape`. + /// REQUIRES: `0 <= n <= dims()` + void RemoveLastDims(int n) { + CHECK_LE(n, dims()); + RemoveDimRange(dims() - n, dims()); + } + + /// \brief Removes the dimensions in range `[begin:end)` from `TensorShape`. + /// Negative values of `end` are interpreted as `dims() + end + 1` (as in + /// Python). The same is true for negative values of `begin`. REQUIRES: + /// `-(dims()+1) <= begin <= dims()` REQUIRES: `-(dims()+1) <= end <= dims()` + void RemoveDimRange(int begin, int end); + + /// Return whether the rank is unknown + bool unknown_rank() const { + return kIsPartial && ndims_byte() == kUnknownRank; + } + + /// Return the number of dimensions in the tensor. + /// Can be -1 meaning unknown rank for PartialTensorShape. + int dims() const { + uint8 dims = ndims_byte(); + return kIsPartial && dims == kUnknownRank ? -1 : dims; + } + + /// \brief Returns the number of elements in dimension `d`. + /// REQUIRES: `0 <= d < dims()` + // TODO(touts): Rename to `dimension()` to match + // `Eigen::Tensor::dimension()`? + int64 dim_size(int d) const; + + /// Returns sizes of all dimensions. + // Returns an empty list for unknown rank PartialTensorShape. + gtl::InlinedVector dim_sizes() const; + + /// Return true iff the rank and all of the dimensions are well defined + // TODO(irving): Rename to is_fully_defined now that it's fast. + bool IsFullyDefined() const { return !kIsPartial || num_elements() != -1; } + + /// Fill `*proto` from `*this`. + void AsProto(TensorShapeProto* proto) const; + + /// For iterating through the dimensions. + TensorShapeIter begin() const; + TensorShapeIter end() const; + + private: + void RecomputeNumElements(); + + // True for PartialTensorShape, false for TensorShape + static constexpr bool kIsPartial = + std::is_same::value; + static_assert(kIsPartial || std::is_same::value, + "Shape is neither TensorShape nor PartialTensorShape"); + + // Used by AddDim and MakeShapeHelper. Does no error checking. + void UnsafeAddDim(int64 size, int64 new_num_elements); + + // For use by TensorShapeUtils::MakeShape + template + friend Status MakeShapeHelper(const T*, int64, S*); +}; + +/// Outputs `TensorShapeBase` to `std::ostream`. +template +std::ostream& operator<<(std::ostream& os, const TensorShapeBase& tsb) { + return os << tsb.DebugString(); +} + +/// Represents the shape of a Tensor. +/// +/// A tensor's shape is denoted by its number of dimensions and a size for each +/// dimension. For example, a Tensor represented by a 3 x 4 matrix would have +/// a shape of 2-D, [3,4]. +/// +/// If you know the exact shape of your Tensor when you create the TensorShape +/// object, you can specify it then, or you can create a TensorShape with +/// zero dimensions and one element, and call AddDim() to add dimensions later. +class TensorShape : public TensorShapeBase { + public: + using TensorShapeBase::TensorShapeBase; + + /// Allow a TensorShape to be used as a PartialTensorShape without copying + operator const PartialTensorShape&() const; // NOLINT(runtime/explicit) + + /// Returns true if `*this` and `b` have the same sizes. Ignores + /// dimension names. + bool IsSameSize(const TensorShape& b) const; + bool operator==(const TensorShape& b) const { return IsSameSize(b); } + bool operator!=(const TensorShape& b) const { return !IsSameSize(b); } + + /// Fill `*dsizes` from `*this`. + template + Eigen::DSizes AsEigenDSizes() const; + + /// Same as `AsEigenDSizes()` but allows for `NDIMS > dims()` -- in + /// which case we pad the rest of the sizes with 1. + template + Eigen::DSizes AsEigenDSizesWithPadding() const; + + private: + // These CHECK fail to ease debugging. + // REQUIRES: dims() == NDIMS + void CheckDimsEqual(int NDIMS) const; + // REQUIRES: dims() >= NDIMS + void CheckDimsAtLeast(int NDIMS) const; +}; + +/// Represents the value of one dimension in a TensorShape. +struct TensorShapeDim { + explicit TensorShapeDim(int64 s) : size(s) {} + int64 size; +}; + +// START_SKIP_DOXYGEN +template +class TensorShapeIter { + public: + TensorShapeIter(const Shape* shape, int d) : shape_(shape), d_(d) {} + bool operator==(const TensorShapeIter& rhs) { + DCHECK(shape_ == rhs.shape_); + return d_ == rhs.d_; + } + bool operator!=(const TensorShapeIter& rhs) { + DCHECK(shape_ == rhs.shape_); + return d_ != rhs.d_; + } + void operator++() { ++d_; } + TensorShapeDim operator*() { return TensorShapeDim(shape_->dim_size(d_)); } + + private: + const Shape* shape_; + int d_; +}; +// END_SKIP_DOXYGEN + +/// \brief Static helper routines for `TensorShape`. Includes a few common +/// predicates on a tensor shape. +class TensorShapeUtils { + public: + static bool IsScalar(const TensorShape& shape) { return shape.dims() == 0; } + + static bool IsVector(const TensorShape& shape) { return shape.dims() == 1; } + + static bool IsVectorOrHigher(const TensorShape& shape) { + return shape.dims() >= 1; + } + + static bool IsMatrix(const TensorShape& shape) { return shape.dims() == 2; } + + static bool IsSquareMatrix(const TensorShape& shape) { + return shape.dims() == 2 && shape.dim_size(0) == shape.dim_size(1); + } + + static bool IsMatrixOrHigher(const TensorShape& shape) { + return shape.dims() >= 2; + } + + /// \brief Returns a `TensorShape` whose dimensions are + /// `dims[0]`, `dims[1]`, ..., `dims[n-1]`. + static Status MakeShape(const int32* dims, int64 n, TensorShape* out); + static Status MakeShape(const int64* dims, int64 n, TensorShape* out); + static Status MakeShape(gtl::ArraySlice shape, TensorShape* out); + static Status MakeShape(gtl::ArraySlice shape, TensorShape* out); + static Status MakeShape(const int32* dims, int64 n, PartialTensorShape* out); + static Status MakeShape(const int64* dims, int64 n, PartialTensorShape* out); + static Status MakeShape(gtl::ArraySlice shape, + PartialTensorShape* out); + static Status MakeShape(gtl::ArraySlice shape, + PartialTensorShape* out); + + static string ShapeListString(const gtl::ArraySlice& shapes); + + /// \brief Returns true iff `shape` starts with `prefix`. + static bool StartsWith(const TensorShape& shape, const TensorShape& prefix); + + /// \brief Returns true iff `shape` ends with `suffix`. + static bool EndsWith(const TensorShape& shape, const TensorShape& suffix); + + /// \brief Returns the product of values in an int64 array, + /// or a failing Status if the array represents a value larger than + /// a `TensorShape` can hold. + static Status NumElements(gtl::ArraySlice shape, int64* num_elements); +}; + +/// Manages the partially known dimensions of a Tensor and their sizes. +class PartialTensorShape : public TensorShapeBase { + public: + PartialTensorShape() {} + using TensorShapeBase::TensorShapeBase; + + /// Add a dimension to the end ("inner-most"), returns a new + /// PartialTensorShape. + /// REQUIRES: `size >= -1`, where -1 means unknown + PartialTensorShape Concatenate(int64 size) const; + + /// Appends all the dimensions from `shape`. Returns a new + /// PartialTensorShape. + PartialTensorShape Concatenate(const PartialTensorShape& shape) const; + + /// Merges all the dimensions from `shape`. Returns + /// `InvalidArgument` error if either `shape` has a different rank + /// or if any of the dimensions are incompatible. + Status MergeWith(const PartialTensorShape& shape, + PartialTensorShape* result) const; + + /// Exact equality test. Returns true iff the ranks match (i.e., both are + /// unknown, or both are known and equal), and all dimensions are equal (i.e., + /// both dimensions are known, or both are known and equal). This is a + /// stronger condition that IsCompatibleWith. + bool IsIdenticalTo(const PartialTensorShape& shape) const; + + /// Return true iff the ranks match, and if the + /// dimensions all either match or one is unknown. + bool IsCompatibleWith(const PartialTensorShape& shape) const; + + // Fill `*shape` from `*this`. + // If `*this` is not fully defined, returns false and + // `*shape` is left in an intermediate state. Otherwise + // returns true. + bool AsTensorShape(TensorShape* shape) const; + + /// \brief Returns a `PartialTensorShape` whose dimensions are + /// `dims[0]`, `dims[1]`, ..., `dims[n-1]`. Values of -1 are + /// considered "unknown". + template + static Status MakePartialShape(const T* dims, int n, + PartialTensorShape* out) { + return TensorShapeUtils::MakeShape(dims, n, out); + } +}; + +/// \brief Static helper routines for `PartialTensorShape`. Includes a few +/// common predicates on a partially known tensor shape. +class PartialTensorShapeUtils { + public: + static string PartialShapeListString( + const gtl::ArraySlice& shapes); + + static bool AreIdentical(const gtl::ArraySlice& shapes0, + const gtl::ArraySlice& shapes1); + + static bool AreCompatible(const gtl::ArraySlice& shapes0, + const gtl::ArraySlice& shapes1); +}; + +// ---------------------------------------------------------------------------- +// Template method implementation details below +// ---------------------------------------------------------------------------- + +template +Eigen::DSizes TensorShape::AsEigenDSizes() const { + CheckDimsEqual(NDIMS); + return AsEigenDSizesWithPadding(); +} + +template +Eigen::DSizes TensorShape::AsEigenDSizesWithPadding() + const { + CheckDimsAtLeast(NDIMS); + static_assert(NDIMS <= TensorShape::MaxDimensions(), "Too many dimensions"); + Eigen::DSizes dsizes; + for (int d = 0; d < dims(); d++) { + dsizes[d] = dim_size(d); + } + for (int d = dims(); d < NDIMS; d++) { + dsizes[d] = 1; + } + return dsizes; +} + +// ---------------------------------------------------------------------------- +// Inlining of some performance critical routines +// ---------------------------------------------------------------------------- + +inline TensorShapeRep::TensorShapeRep(const TensorShapeRep& b) { + num_elements_ = b.num_elements_; + if (b.tag() != REP_OUT_OF_LINE) { + memcpy(buf(), b.buf(), sizeof(u_.buf)); + // memcpy above Implicitly does: + // set_ndims_byte(b.ndims_byte()); + // set_tag(b.tag()); + } else { + set_tag(REP16); // So that SlowCopyFrom does not try to deallocate + SlowCopyFrom(b); + } +} + +inline TensorShapeRep::TensorShapeRep(TensorShapeRep&& b) { + num_elements_ = b.num_elements_; + memcpy(buf(), b.buf(), sizeof(u_.buf)); + // memcpy above Implicitly does: + // set_ndims_byte(b.ndims_byte()); + // set_tag(b.tag()); + b.set_tag(REP16); // other shape no longer owns out-of-line data, if any. +} + +inline TensorShapeRep::~TensorShapeRep() { + if (tag() == REP_OUT_OF_LINE) { + DestructorOutOfLine(); + } +} + +inline void TensorShapeRep::operator=(const TensorShapeRep& b) { + num_elements_ = b.num_elements_; + if (tag() != REP_OUT_OF_LINE && b.tag() != REP_OUT_OF_LINE) { + memcpy(buf(), b.buf(), sizeof(u_.buf)); + // memcpy above implicitly also does: + // set_tag(b.tag()); + // set_ndims_byte(b.ndims_byte()); + } else { + SlowCopyFrom(b); + } +} + +inline void TensorShapeRep::operator=(TensorShapeRep&& b) { + if (tag() == REP_OUT_OF_LINE) { + DestructorOutOfLine(); + } + num_elements_ = b.num_elements_; + memcpy(buf(), b.buf(), sizeof(u_.buf)); + // memcpy above Implicitly does: + // set_ndims_byte(b.ndims_byte()); + // set_tag(b.tag()); + b.set_tag(REP16); // other shape no longer owns out-of-line data, if any. +} + +inline TensorShape::operator const PartialTensorShape&() const { + // Downcast to the shared representation and upcast to PartialTensorShape + const TensorShapeRep* rep = this; + return *static_cast(rep); +} + +// Declare explicit instantiations in .cc file +extern template class TensorShapeBase; +extern template class TensorShapeBase; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_FRAMEWORK_TENSOR_SHAPE_H_ diff --git a/linux/include/tensorflow/core/framework/tensor_shape.pb.h b/linux/include/tensorflow/core/framework/tensor_shape.pb.h new file mode 100644 index 000000000..edad2caf7 --- /dev/null +++ b/linux/include/tensorflow/core/framework/tensor_shape.pb.h @@ -0,0 +1,499 @@ +// Generated by the protocol buffer compiler. DO NOT EDIT! +// source: tensorflow/core/framework/tensor_shape.proto + +#ifndef PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2ftensor_5fshape_2eproto +#define PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2ftensor_5fshape_2eproto + +#include + +#include + +#if GOOGLE_PROTOBUF_VERSION < 3006000 +#error This file was generated by a newer version of protoc which is +#error incompatible with your Protocol Buffer headers. Please update +#error your headers. +#endif +#if 3006000 < GOOGLE_PROTOBUF_MIN_PROTOC_VERSION +#error This file was generated by an older version of protoc which is +#error incompatible with your Protocol Buffer headers. Please +#error regenerate this file with a newer version of protoc. +#endif + +#include +#include +#include +#include +#include +#include +#include +#include +#include // IWYU pragma: export +#include // IWYU pragma: export +#include +// @@protoc_insertion_point(includes) +#define PROTOBUF_INTERNAL_EXPORT_protobuf_tensorflow_2fcore_2fframework_2ftensor_5fshape_2eproto + +namespace protobuf_tensorflow_2fcore_2fframework_2ftensor_5fshape_2eproto { +// Internal implementation detail -- do not use these members. +struct TableStruct { + static const ::google::protobuf::internal::ParseTableField entries[]; + static const ::google::protobuf::internal::AuxillaryParseTableField aux[]; + static const ::google::protobuf::internal::ParseTable schema[2]; + static const ::google::protobuf::internal::FieldMetadata field_metadata[]; + static const ::google::protobuf::internal::SerializationTable serialization_table[]; + static const ::google::protobuf::uint32 offsets[]; +}; +void AddDescriptors(); +} // namespace protobuf_tensorflow_2fcore_2fframework_2ftensor_5fshape_2eproto +namespace tensorflow { +class TensorShapeProto; +class TensorShapeProtoDefaultTypeInternal; +extern TensorShapeProtoDefaultTypeInternal _TensorShapeProto_default_instance_; +class TensorShapeProto_Dim; +class TensorShapeProto_DimDefaultTypeInternal; +extern TensorShapeProto_DimDefaultTypeInternal _TensorShapeProto_Dim_default_instance_; +} // namespace tensorflow +namespace google { +namespace protobuf { +template<> ::tensorflow::TensorShapeProto* Arena::CreateMaybeMessage<::tensorflow::TensorShapeProto>(Arena*); +template<> ::tensorflow::TensorShapeProto_Dim* Arena::CreateMaybeMessage<::tensorflow::TensorShapeProto_Dim>(Arena*); +} // namespace protobuf +} // namespace google +namespace tensorflow { + +// =================================================================== + +class TensorShapeProto_Dim : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.TensorShapeProto.Dim) */ { + public: + TensorShapeProto_Dim(); + virtual ~TensorShapeProto_Dim(); + + TensorShapeProto_Dim(const TensorShapeProto_Dim& from); + + inline TensorShapeProto_Dim& operator=(const TensorShapeProto_Dim& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + TensorShapeProto_Dim(TensorShapeProto_Dim&& from) noexcept + : TensorShapeProto_Dim() { + *this = ::std::move(from); + } + + inline TensorShapeProto_Dim& operator=(TensorShapeProto_Dim&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const TensorShapeProto_Dim& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const TensorShapeProto_Dim* internal_default_instance() { + return reinterpret_cast( + &_TensorShapeProto_Dim_default_instance_); + } + static constexpr int kIndexInFileMessages = + 0; + + void UnsafeArenaSwap(TensorShapeProto_Dim* other); + void Swap(TensorShapeProto_Dim* other); + friend void swap(TensorShapeProto_Dim& a, TensorShapeProto_Dim& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline TensorShapeProto_Dim* New() const final { + return CreateMaybeMessage(NULL); + } + + TensorShapeProto_Dim* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const TensorShapeProto_Dim& from); + void MergeFrom(const TensorShapeProto_Dim& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(TensorShapeProto_Dim* other); + protected: + explicit TensorShapeProto_Dim(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // string name = 2; + void clear_name(); + static const int kNameFieldNumber = 2; + const ::std::string& name() const; + void set_name(const ::std::string& value); + #if LANG_CXX11 + void set_name(::std::string&& value); + #endif + void set_name(const char* value); + void set_name(const char* value, size_t size); + ::std::string* mutable_name(); + ::std::string* release_name(); + void set_allocated_name(::std::string* name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_name( + ::std::string* name); + + // int64 size = 1; + void clear_size(); + static const int kSizeFieldNumber = 1; + ::google::protobuf::int64 size() const; + void set_size(::google::protobuf::int64 value); + + // @@protoc_insertion_point(class_scope:tensorflow.TensorShapeProto.Dim) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::internal::ArenaStringPtr name_; + ::google::protobuf::int64 size_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2ftensor_5fshape_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class TensorShapeProto : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.TensorShapeProto) */ { + public: + TensorShapeProto(); + virtual ~TensorShapeProto(); + + TensorShapeProto(const TensorShapeProto& from); + + inline TensorShapeProto& operator=(const TensorShapeProto& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + TensorShapeProto(TensorShapeProto&& from) noexcept + : TensorShapeProto() { + *this = ::std::move(from); + } + + inline TensorShapeProto& operator=(TensorShapeProto&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const TensorShapeProto& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const TensorShapeProto* internal_default_instance() { + return reinterpret_cast( + &_TensorShapeProto_default_instance_); + } + static constexpr int kIndexInFileMessages = + 1; + + void UnsafeArenaSwap(TensorShapeProto* other); + void Swap(TensorShapeProto* other); + friend void swap(TensorShapeProto& a, TensorShapeProto& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline TensorShapeProto* New() const final { + return CreateMaybeMessage(NULL); + } + + TensorShapeProto* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const TensorShapeProto& from); + void MergeFrom(const TensorShapeProto& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(TensorShapeProto* other); + protected: + explicit TensorShapeProto(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + typedef TensorShapeProto_Dim Dim; + + // accessors ------------------------------------------------------- + + // repeated .tensorflow.TensorShapeProto.Dim dim = 2; + int dim_size() const; + void clear_dim(); + static const int kDimFieldNumber = 2; + ::tensorflow::TensorShapeProto_Dim* mutable_dim(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::TensorShapeProto_Dim >* + mutable_dim(); + const ::tensorflow::TensorShapeProto_Dim& dim(int index) const; + ::tensorflow::TensorShapeProto_Dim* add_dim(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::TensorShapeProto_Dim >& + dim() const; + + // bool unknown_rank = 3; + void clear_unknown_rank(); + static const int kUnknownRankFieldNumber = 3; + bool unknown_rank() const; + void set_unknown_rank(bool value); + + // @@protoc_insertion_point(class_scope:tensorflow.TensorShapeProto) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::TensorShapeProto_Dim > dim_; + bool unknown_rank_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2ftensor_5fshape_2eproto::TableStruct; +}; +// =================================================================== + + +// =================================================================== + +#ifdef __GNUC__ + #pragma GCC diagnostic push + #pragma GCC diagnostic ignored "-Wstrict-aliasing" +#endif // __GNUC__ +// TensorShapeProto_Dim + +// int64 size = 1; +inline void TensorShapeProto_Dim::clear_size() { + size_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 TensorShapeProto_Dim::size() const { + // @@protoc_insertion_point(field_get:tensorflow.TensorShapeProto.Dim.size) + return size_; +} +inline void TensorShapeProto_Dim::set_size(::google::protobuf::int64 value) { + + size_ = value; + // @@protoc_insertion_point(field_set:tensorflow.TensorShapeProto.Dim.size) +} + +// string name = 2; +inline void TensorShapeProto_Dim::clear_name() { + name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& TensorShapeProto_Dim::name() const { + // @@protoc_insertion_point(field_get:tensorflow.TensorShapeProto.Dim.name) + return name_.Get(); +} +inline void TensorShapeProto_Dim::set_name(const ::std::string& value) { + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.TensorShapeProto.Dim.name) +} +#if LANG_CXX11 +inline void TensorShapeProto_Dim::set_name(::std::string&& value) { + + name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.TensorShapeProto.Dim.name) +} +#endif +inline void TensorShapeProto_Dim::set_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.TensorShapeProto.Dim.name) +} +inline void TensorShapeProto_Dim::set_name(const char* value, + size_t size) { + + name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.TensorShapeProto.Dim.name) +} +inline ::std::string* TensorShapeProto_Dim::mutable_name() { + + // @@protoc_insertion_point(field_mutable:tensorflow.TensorShapeProto.Dim.name) + return name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* TensorShapeProto_Dim::release_name() { + // @@protoc_insertion_point(field_release:tensorflow.TensorShapeProto.Dim.name) + + return name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void TensorShapeProto_Dim::set_allocated_name(::std::string* name) { + if (name != NULL) { + + } else { + + } + name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.TensorShapeProto.Dim.name) +} +inline ::std::string* TensorShapeProto_Dim::unsafe_arena_release_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.TensorShapeProto.Dim.name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void TensorShapeProto_Dim::unsafe_arena_set_allocated_name( + ::std::string* name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (name != NULL) { + + } else { + + } + name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.TensorShapeProto.Dim.name) +} + +// ------------------------------------------------------------------- + +// TensorShapeProto + +// repeated .tensorflow.TensorShapeProto.Dim dim = 2; +inline int TensorShapeProto::dim_size() const { + return dim_.size(); +} +inline void TensorShapeProto::clear_dim() { + dim_.Clear(); +} +inline ::tensorflow::TensorShapeProto_Dim* TensorShapeProto::mutable_dim(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.TensorShapeProto.dim) + return dim_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::TensorShapeProto_Dim >* +TensorShapeProto::mutable_dim() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.TensorShapeProto.dim) + return &dim_; +} +inline const ::tensorflow::TensorShapeProto_Dim& TensorShapeProto::dim(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.TensorShapeProto.dim) + return dim_.Get(index); +} +inline ::tensorflow::TensorShapeProto_Dim* TensorShapeProto::add_dim() { + // @@protoc_insertion_point(field_add:tensorflow.TensorShapeProto.dim) + return dim_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::TensorShapeProto_Dim >& +TensorShapeProto::dim() const { + // @@protoc_insertion_point(field_list:tensorflow.TensorShapeProto.dim) + return dim_; +} + +// bool unknown_rank = 3; +inline void TensorShapeProto::clear_unknown_rank() { + unknown_rank_ = false; +} +inline bool TensorShapeProto::unknown_rank() const { + // @@protoc_insertion_point(field_get:tensorflow.TensorShapeProto.unknown_rank) + return unknown_rank_; +} +inline void TensorShapeProto::set_unknown_rank(bool value) { + + unknown_rank_ = value; + // @@protoc_insertion_point(field_set:tensorflow.TensorShapeProto.unknown_rank) +} + +#ifdef __GNUC__ + #pragma GCC diagnostic pop +#endif // __GNUC__ +// ------------------------------------------------------------------- + + +// @@protoc_insertion_point(namespace_scope) + +} // namespace tensorflow + +// @@protoc_insertion_point(global_scope) + +#endif // PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2ftensor_5fshape_2eproto diff --git a/linux/include/tensorflow/core/framework/tensor_shape.pb_text-impl.h b/linux/include/tensorflow/core/framework/tensor_shape.pb_text-impl.h new file mode 100644 index 000000000..c7b82211b --- /dev/null +++ b/linux/include/tensorflow/core/framework/tensor_shape.pb_text-impl.h @@ -0,0 +1,32 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_framework_tensor_shape_proto_IMPL_H_ +#define tensorflow_core_framework_tensor_shape_proto_IMPL_H_ + +#include "tensorflow/core/framework/tensor_shape.pb.h" +#include "tensorflow/core/framework/tensor_shape.pb_text.h" +#include "tensorflow/core/lib/strings/proto_text_util.h" +#include "tensorflow/core/lib/strings/scanner.h" + +namespace tensorflow { + +namespace internal { + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::TensorShapeProto_Dim& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::TensorShapeProto_Dim* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::TensorShapeProto& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::TensorShapeProto* msg); + +} // namespace internal + +} // namespace tensorflow + +#endif // tensorflow_core_framework_tensor_shape_proto_IMPL_H_ diff --git a/linux/include/tensorflow/core/framework/tensor_shape.pb_text.h b/linux/include/tensorflow/core/framework/tensor_shape.pb_text.h new file mode 100644 index 000000000..7dd28180b --- /dev/null +++ b/linux/include/tensorflow/core/framework/tensor_shape.pb_text.h @@ -0,0 +1,34 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_framework_tensor_shape_proto_H_ +#define tensorflow_core_framework_tensor_shape_proto_H_ + +#include "tensorflow/core/framework/tensor_shape.pb.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/protobuf.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +// Message-text conversion for tensorflow.TensorShapeProto.Dim +string ProtoDebugString( + const ::tensorflow::TensorShapeProto_Dim& msg); +string ProtoShortDebugString( + const ::tensorflow::TensorShapeProto_Dim& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::TensorShapeProto_Dim* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.TensorShapeProto +string ProtoDebugString( + const ::tensorflow::TensorShapeProto& msg); +string ProtoShortDebugString( + const ::tensorflow::TensorShapeProto& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::TensorShapeProto* msg) + TF_MUST_USE_RESULT; + +} // namespace tensorflow + +#endif // tensorflow_core_framework_tensor_shape_proto_H_ diff --git a/linux/include/tensorflow/core/framework/tensor_slice.h b/linux/include/tensorflow/core/framework/tensor_slice.h new file mode 100644 index 000000000..82f21fb17 --- /dev/null +++ b/linux/include/tensorflow/core/framework/tensor_slice.h @@ -0,0 +1,224 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_FRAMEWORK_TENSOR_SLICE_H_ +#define TENSORFLOW_CORE_FRAMEWORK_TENSOR_SLICE_H_ + +#include +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/tensor_slice.pb.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/lib/core/stringpiece.h" +#include "tensorflow/core/lib/gtl/inlined_vector.h" +#include "tensorflow/core/platform/logging.h" + +namespace tensorflow { + +// A tensor slice represents a slice of a given tensor. It is represented by a +// list of (start, length) pairs, where the size of the list is the rank of the +// tensor. + +class TensorSlice { + public: + // Construct a tensor slice: you have a number of ways: + // -- creating an empty slice + // -- from just a dimension (in this case it will create a full slice) + // -- from an array of pairs of integers. + // -- from a TensorSliceProto protocol buffer + // -- from a string format of "start,length:start,length..." where each + // "start,length" pair represents the slice on one dimension. We allow a + // special "-" that means "everything for this dimension". One such example + // is: 0,10:-:14,1:-:- + TensorSlice() {} + explicit TensorSlice(int dim); + explicit TensorSlice(const TensorSliceProto& proto); + explicit TensorSlice(std::initializer_list> extents); + + static Status Parse(const string& str, TensorSlice* output); + static TensorSlice ParseOrDie(const string& str) { + TensorSlice ret; + Status s = Parse(str, &ret); + if (!s.ok()) { + LOG(FATAL) << "Could not parse TensorSlice"; + } + return ret; + } + + void Clear(); + + // Accessors + int dims() const { return starts_.size(); } + + int64 start(int d) const { + DCHECK_GE(d, 0); + DCHECK_LT(d, dims()); + return starts_[d]; + } + + int64 length(int d) const { + DCHECK_GE(d, 0); + DCHECK_LT(d, dims()); + return lengths_[d]; + } + + int64 end(int d) const { + DCHECK_GE(d, 0); + DCHECK_LT(d, dims()); + return start(d) + length(d); + } + + void set_start(int d, int64 x) { + DCHECK_GE(d, 0); + DCHECK_LT(d, dims()); + DCHECK_GE(x, 0); + starts_[d] = x; + } + + void set_length(int d, int64 x) { + DCHECK_GE(d, 0); + DCHECK_LT(d, dims()); + lengths_[d] = x; + } + + // If we have a full slice along dimension "d". + bool IsFullAt(int d) const { + return lengths_[d] == kFullExtent && starts_[d] == 0; + } + + // If this is a full slice, i.e. IsFullAt(d) for every d. + bool IsFull() const; + + // Set the slice to be a full slice of "dim" dimensions + void SetFullSlice(int dim); + + // Extend a slice to "dim" dimensions: all the added dimensions are full. + // Requires: dim >= dims(). + void Extend(int dim); + + // Conversion of a TensorSlice to other formats + void AsProto(TensorSliceProto* proto) const; + string DebugString() const; + + // Fill *indices and *sizes from *this (so that we can use the slice() + // function in eigen tensor). We need a tensor shape in case some of the + // slices are full slices. + // We allow NDIMS to be greater than dims(), in which case we will pad the + // higher dimensions with trivial dimensions. + template + void FillIndicesAndSizes( + const TensorShape& shape, + Eigen::DSizes* indices, + Eigen::DSizes* sizes) const; + + // Interaction with other TensorSlices. + + // Compute the intersection with another slice and if "result" is not + // nullptr, store the results in *result; returns true if there is any real + // intersection. + bool Intersect(const TensorSlice& other, TensorSlice* result) const; + // A short hand. + bool Overlaps(const TensorSlice& other) const { + return Intersect(other, nullptr); + } + + // Equals iff "*this" and "other" are logically equivalent. + bool operator==(const TensorSlice& other) const; + bool operator!=(const TensorSlice& other) const { return !(*this == other); } + + // Interaction with TensorShape. + + // Slices a shape and stores the result into *result_shape. + // Requires that the shape and *this have the same rank. + // For example, given a tensor shape of {3, 4, 5}, and a slice of + // 1,2:-:0,2, the result shape is {2, 4, 2}. + Status SliceTensorShape(const TensorShape& shape, + TensorShape* result_shape) const; + + // Given slice "sub" where "sub" is fully contained in *this, + // (meaning that the intersection of "sub" and *this equals "sub"), computes + // the "relative" slice of "sub" with respect to *this. + // + // In other words, if we use A>S to denote slicing a shape S with a slice A, + // then the function is computing a slice X such that: + // X > (this > S) = sub > S + // for any shape S. + // + // In general, along every dimension, the start of the relative slice is the + // start of the "sub" slice minus the start of *this; the length of the + // relative slice is the length of the "sub" slice. + // + // For example, say we have a shape of {3, 4, 5}, "this" is 0,2:-:1,2, and + // "sub" is 1,1:2:2,1,2, then the related slice is 1,1:2,2:0,2. + // + // The caller needs to make sure that "sub" is indeed a sub-slice of *this; + // otherwise the result is undefined. + void ComputeRelative(const TensorSlice& sub, TensorSlice* relative) const; + + // Updates the slice in such a way that it fully covers "other" slice. + // Note, "other" slice should refer to the same tensor shape. + // Example: + // given a slice [2:4, :, 3:] and "other" slice [:, 1:4, 2:4] the + // updated slice would be [:, :, 2:]. Here is why: + // dim 0: "2:4" U ":" -> ":" + // dim 1: ":" U "1-4" -> ":" + // dim 2: "3:" U "2:4" -> "2:" + void UpdateToCover(const TensorSlice& other); + + // Returns true if the length field was specified in an Extent. + static bool HasExtentLength(const TensorSliceProto::Extent& extent); + + // Returns the value of the length field in an Extent, or -1 if it + // is not present. + static int64 GetExtentLength(const TensorSliceProto::Extent& extent); + + private: + // a length value of kFullExtent (-1) means we have a full slice at this + // dimension. It's defined in tensor_slice.cc. + static const int64 kFullExtent; + + // TODO(yangke): switch to Eigen once it supports variable size arrays. + // A value of + gtl::InlinedVector starts_; + gtl::InlinedVector lengths_; +}; + +template +void TensorSlice::FillIndicesAndSizes( + const TensorShape& shape, Eigen::DSizes* indices, + Eigen::DSizes* sizes) const { + CHECK_EQ(shape.dims(), dims()) << "Incompatible dimensions between shape " + << "slices: shape = " << shape.DebugString() + << ", slice = " << DebugString(); + CHECK_GE(NDIMS, dims()) << "Asking for a " << NDIMS << "-dim slice from " + << "a slice of dimension " << dims(); + for (int d = 0; d < dims(); ++d) { + if (IsFullAt(d)) { + (*indices)[d] = 0; + (*sizes)[d] = shape.dim_size(d); + } else { + (*indices)[d] = starts_[d]; + (*sizes)[d] = lengths_[d]; + } + } + for (int d = dims(); d < NDIMS; ++d) { + (*indices)[d] = 0; + (*sizes)[d] = 1; + } +} + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_FRAMEWORK_TENSOR_SLICE_H_ diff --git a/linux/include/tensorflow/core/framework/tensor_slice.pb.h b/linux/include/tensorflow/core/framework/tensor_slice.pb.h new file mode 100644 index 000000000..df285da55 --- /dev/null +++ b/linux/include/tensorflow/core/framework/tensor_slice.pb.h @@ -0,0 +1,443 @@ +// Generated by the protocol buffer compiler. DO NOT EDIT! +// source: tensorflow/core/framework/tensor_slice.proto + +#ifndef PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2ftensor_5fslice_2eproto +#define PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2ftensor_5fslice_2eproto + +#include + +#include + +#if GOOGLE_PROTOBUF_VERSION < 3006000 +#error This file was generated by a newer version of protoc which is +#error incompatible with your Protocol Buffer headers. Please update +#error your headers. +#endif +#if 3006000 < GOOGLE_PROTOBUF_MIN_PROTOC_VERSION +#error This file was generated by an older version of protoc which is +#error incompatible with your Protocol Buffer headers. Please +#error regenerate this file with a newer version of protoc. +#endif + +#include +#include +#include +#include +#include +#include +#include +#include +#include // IWYU pragma: export +#include // IWYU pragma: export +#include +// @@protoc_insertion_point(includes) +#define PROTOBUF_INTERNAL_EXPORT_protobuf_tensorflow_2fcore_2fframework_2ftensor_5fslice_2eproto + +namespace protobuf_tensorflow_2fcore_2fframework_2ftensor_5fslice_2eproto { +// Internal implementation detail -- do not use these members. +struct TableStruct { + static const ::google::protobuf::internal::ParseTableField entries[]; + static const ::google::protobuf::internal::AuxillaryParseTableField aux[]; + static const ::google::protobuf::internal::ParseTable schema[2]; + static const ::google::protobuf::internal::FieldMetadata field_metadata[]; + static const ::google::protobuf::internal::SerializationTable serialization_table[]; + static const ::google::protobuf::uint32 offsets[]; +}; +void AddDescriptors(); +} // namespace protobuf_tensorflow_2fcore_2fframework_2ftensor_5fslice_2eproto +namespace tensorflow { +class TensorSliceProto; +class TensorSliceProtoDefaultTypeInternal; +extern TensorSliceProtoDefaultTypeInternal _TensorSliceProto_default_instance_; +class TensorSliceProto_Extent; +class TensorSliceProto_ExtentDefaultTypeInternal; +extern TensorSliceProto_ExtentDefaultTypeInternal _TensorSliceProto_Extent_default_instance_; +} // namespace tensorflow +namespace google { +namespace protobuf { +template<> ::tensorflow::TensorSliceProto* Arena::CreateMaybeMessage<::tensorflow::TensorSliceProto>(Arena*); +template<> ::tensorflow::TensorSliceProto_Extent* Arena::CreateMaybeMessage<::tensorflow::TensorSliceProto_Extent>(Arena*); +} // namespace protobuf +} // namespace google +namespace tensorflow { + +// =================================================================== + +class TensorSliceProto_Extent : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.TensorSliceProto.Extent) */ { + public: + TensorSliceProto_Extent(); + virtual ~TensorSliceProto_Extent(); + + TensorSliceProto_Extent(const TensorSliceProto_Extent& from); + + inline TensorSliceProto_Extent& operator=(const TensorSliceProto_Extent& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + TensorSliceProto_Extent(TensorSliceProto_Extent&& from) noexcept + : TensorSliceProto_Extent() { + *this = ::std::move(from); + } + + inline TensorSliceProto_Extent& operator=(TensorSliceProto_Extent&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const TensorSliceProto_Extent& default_instance(); + + enum HasLengthCase { + kLength = 2, + HAS_LENGTH_NOT_SET = 0, + }; + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const TensorSliceProto_Extent* internal_default_instance() { + return reinterpret_cast( + &_TensorSliceProto_Extent_default_instance_); + } + static constexpr int kIndexInFileMessages = + 0; + + void UnsafeArenaSwap(TensorSliceProto_Extent* other); + void Swap(TensorSliceProto_Extent* other); + friend void swap(TensorSliceProto_Extent& a, TensorSliceProto_Extent& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline TensorSliceProto_Extent* New() const final { + return CreateMaybeMessage(NULL); + } + + TensorSliceProto_Extent* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const TensorSliceProto_Extent& from); + void MergeFrom(const TensorSliceProto_Extent& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(TensorSliceProto_Extent* other); + protected: + explicit TensorSliceProto_Extent(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // int64 start = 1; + void clear_start(); + static const int kStartFieldNumber = 1; + ::google::protobuf::int64 start() const; + void set_start(::google::protobuf::int64 value); + + // int64 length = 2; + private: + bool has_length() const; + public: + void clear_length(); + static const int kLengthFieldNumber = 2; + ::google::protobuf::int64 length() const; + void set_length(::google::protobuf::int64 value); + + void clear_has_length(); + HasLengthCase has_length_case() const; + // @@protoc_insertion_point(class_scope:tensorflow.TensorSliceProto.Extent) + private: + void set_has_length(); + + inline bool has_has_length() const; + inline void clear_has_has_length(); + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::int64 start_; + union HasLengthUnion { + HasLengthUnion() {} + ::google::protobuf::int64 length_; + } has_length_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + ::google::protobuf::uint32 _oneof_case_[1]; + + friend struct ::protobuf_tensorflow_2fcore_2fframework_2ftensor_5fslice_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class TensorSliceProto : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.TensorSliceProto) */ { + public: + TensorSliceProto(); + virtual ~TensorSliceProto(); + + TensorSliceProto(const TensorSliceProto& from); + + inline TensorSliceProto& operator=(const TensorSliceProto& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + TensorSliceProto(TensorSliceProto&& from) noexcept + : TensorSliceProto() { + *this = ::std::move(from); + } + + inline TensorSliceProto& operator=(TensorSliceProto&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const TensorSliceProto& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const TensorSliceProto* internal_default_instance() { + return reinterpret_cast( + &_TensorSliceProto_default_instance_); + } + static constexpr int kIndexInFileMessages = + 1; + + void UnsafeArenaSwap(TensorSliceProto* other); + void Swap(TensorSliceProto* other); + friend void swap(TensorSliceProto& a, TensorSliceProto& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline TensorSliceProto* New() const final { + return CreateMaybeMessage(NULL); + } + + TensorSliceProto* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const TensorSliceProto& from); + void MergeFrom(const TensorSliceProto& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(TensorSliceProto* other); + protected: + explicit TensorSliceProto(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + typedef TensorSliceProto_Extent Extent; + + // accessors ------------------------------------------------------- + + // repeated .tensorflow.TensorSliceProto.Extent extent = 1; + int extent_size() const; + void clear_extent(); + static const int kExtentFieldNumber = 1; + ::tensorflow::TensorSliceProto_Extent* mutable_extent(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::TensorSliceProto_Extent >* + mutable_extent(); + const ::tensorflow::TensorSliceProto_Extent& extent(int index) const; + ::tensorflow::TensorSliceProto_Extent* add_extent(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::TensorSliceProto_Extent >& + extent() const; + + // @@protoc_insertion_point(class_scope:tensorflow.TensorSliceProto) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::TensorSliceProto_Extent > extent_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2ftensor_5fslice_2eproto::TableStruct; +}; +// =================================================================== + + +// =================================================================== + +#ifdef __GNUC__ + #pragma GCC diagnostic push + #pragma GCC diagnostic ignored "-Wstrict-aliasing" +#endif // __GNUC__ +// TensorSliceProto_Extent + +// int64 start = 1; +inline void TensorSliceProto_Extent::clear_start() { + start_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 TensorSliceProto_Extent::start() const { + // @@protoc_insertion_point(field_get:tensorflow.TensorSliceProto.Extent.start) + return start_; +} +inline void TensorSliceProto_Extent::set_start(::google::protobuf::int64 value) { + + start_ = value; + // @@protoc_insertion_point(field_set:tensorflow.TensorSliceProto.Extent.start) +} + +// int64 length = 2; +inline bool TensorSliceProto_Extent::has_length() const { + return has_length_case() == kLength; +} +inline void TensorSliceProto_Extent::set_has_length() { + _oneof_case_[0] = kLength; +} +inline void TensorSliceProto_Extent::clear_length() { + if (has_length()) { + has_length_.length_ = GOOGLE_LONGLONG(0); + clear_has_has_length(); + } +} +inline ::google::protobuf::int64 TensorSliceProto_Extent::length() const { + // @@protoc_insertion_point(field_get:tensorflow.TensorSliceProto.Extent.length) + if (has_length()) { + return has_length_.length_; + } + return GOOGLE_LONGLONG(0); +} +inline void TensorSliceProto_Extent::set_length(::google::protobuf::int64 value) { + if (!has_length()) { + clear_has_length(); + set_has_length(); + } + has_length_.length_ = value; + // @@protoc_insertion_point(field_set:tensorflow.TensorSliceProto.Extent.length) +} + +inline bool TensorSliceProto_Extent::has_has_length() const { + return has_length_case() != HAS_LENGTH_NOT_SET; +} +inline void TensorSliceProto_Extent::clear_has_has_length() { + _oneof_case_[0] = HAS_LENGTH_NOT_SET; +} +inline TensorSliceProto_Extent::HasLengthCase TensorSliceProto_Extent::has_length_case() const { + return TensorSliceProto_Extent::HasLengthCase(_oneof_case_[0]); +} +// ------------------------------------------------------------------- + +// TensorSliceProto + +// repeated .tensorflow.TensorSliceProto.Extent extent = 1; +inline int TensorSliceProto::extent_size() const { + return extent_.size(); +} +inline void TensorSliceProto::clear_extent() { + extent_.Clear(); +} +inline ::tensorflow::TensorSliceProto_Extent* TensorSliceProto::mutable_extent(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.TensorSliceProto.extent) + return extent_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::TensorSliceProto_Extent >* +TensorSliceProto::mutable_extent() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.TensorSliceProto.extent) + return &extent_; +} +inline const ::tensorflow::TensorSliceProto_Extent& TensorSliceProto::extent(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.TensorSliceProto.extent) + return extent_.Get(index); +} +inline ::tensorflow::TensorSliceProto_Extent* TensorSliceProto::add_extent() { + // @@protoc_insertion_point(field_add:tensorflow.TensorSliceProto.extent) + return extent_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::TensorSliceProto_Extent >& +TensorSliceProto::extent() const { + // @@protoc_insertion_point(field_list:tensorflow.TensorSliceProto.extent) + return extent_; +} + +#ifdef __GNUC__ + #pragma GCC diagnostic pop +#endif // __GNUC__ +// ------------------------------------------------------------------- + + +// @@protoc_insertion_point(namespace_scope) + +} // namespace tensorflow + +// @@protoc_insertion_point(global_scope) + +#endif // PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2ftensor_5fslice_2eproto diff --git a/linux/include/tensorflow/core/framework/tensor_slice.pb_text-impl.h b/linux/include/tensorflow/core/framework/tensor_slice.pb_text-impl.h new file mode 100644 index 000000000..856568dd5 --- /dev/null +++ b/linux/include/tensorflow/core/framework/tensor_slice.pb_text-impl.h @@ -0,0 +1,32 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_framework_tensor_slice_proto_IMPL_H_ +#define tensorflow_core_framework_tensor_slice_proto_IMPL_H_ + +#include "tensorflow/core/framework/tensor_slice.pb.h" +#include "tensorflow/core/framework/tensor_slice.pb_text.h" +#include "tensorflow/core/lib/strings/proto_text_util.h" +#include "tensorflow/core/lib/strings/scanner.h" + +namespace tensorflow { + +namespace internal { + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::TensorSliceProto_Extent& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::TensorSliceProto_Extent* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::TensorSliceProto& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::TensorSliceProto* msg); + +} // namespace internal + +} // namespace tensorflow + +#endif // tensorflow_core_framework_tensor_slice_proto_IMPL_H_ diff --git a/linux/include/tensorflow/core/framework/tensor_slice.pb_text.h b/linux/include/tensorflow/core/framework/tensor_slice.pb_text.h new file mode 100644 index 000000000..13ce90257 --- /dev/null +++ b/linux/include/tensorflow/core/framework/tensor_slice.pb_text.h @@ -0,0 +1,34 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_framework_tensor_slice_proto_H_ +#define tensorflow_core_framework_tensor_slice_proto_H_ + +#include "tensorflow/core/framework/tensor_slice.pb.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/protobuf.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +// Message-text conversion for tensorflow.TensorSliceProto.Extent +string ProtoDebugString( + const ::tensorflow::TensorSliceProto_Extent& msg); +string ProtoShortDebugString( + const ::tensorflow::TensorSliceProto_Extent& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::TensorSliceProto_Extent* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.TensorSliceProto +string ProtoDebugString( + const ::tensorflow::TensorSliceProto& msg); +string ProtoShortDebugString( + const ::tensorflow::TensorSliceProto& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::TensorSliceProto* msg) + TF_MUST_USE_RESULT; + +} // namespace tensorflow + +#endif // tensorflow_core_framework_tensor_slice_proto_H_ diff --git a/linux/include/tensorflow/core/framework/tensor_types.h b/linux/include/tensorflow/core/framework/tensor_types.h new file mode 100644 index 000000000..6f981db18 --- /dev/null +++ b/linux/include/tensorflow/core/framework/tensor_types.h @@ -0,0 +1,126 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_FRAMEWORK_TENSOR_TYPES_H_ +#define TENSORFLOW_CORE_FRAMEWORK_TENSOR_TYPES_H_ + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" + +namespace tensorflow { + +// Helper to define Tensor types given that the scalar is of type T. +template +struct TTypes { + // Rank- tensor of scalar type T. + typedef Eigen::TensorMap, + Eigen::Aligned> + Tensor; + typedef Eigen::TensorMap< + Eigen::Tensor, Eigen::Aligned> + ConstTensor; + + // Unaligned Rank- tensor of scalar type T. + typedef Eigen::TensorMap > + UnalignedTensor; + typedef Eigen::TensorMap< + Eigen::Tensor > + UnalignedConstTensor; + + typedef Eigen::TensorMap, + Eigen::Aligned> + Tensor32Bit; + + // Scalar tensor (implemented as a rank-0 tensor) of scalar type T. + typedef Eigen::TensorMap< + Eigen::TensorFixedSize, Eigen::RowMajor, IndexType>, + Eigen::Aligned> + Scalar; + typedef Eigen::TensorMap, + Eigen::RowMajor, IndexType>, + Eigen::Aligned> + ConstScalar; + + // Unaligned Scalar tensor of scalar type T. + typedef Eigen::TensorMap< + Eigen::TensorFixedSize, Eigen::RowMajor, IndexType> > + UnalignedScalar; + typedef Eigen::TensorMap, + Eigen::RowMajor, IndexType> > + UnalignedConstScalar; + + // Rank-1 tensor (vector) of scalar type T. + typedef Eigen::TensorMap, + Eigen::Aligned> + Flat; + typedef Eigen::TensorMap< + Eigen::Tensor, Eigen::Aligned> + ConstFlat; + typedef Eigen::TensorMap, + Eigen::Aligned> + Vec; + typedef Eigen::TensorMap< + Eigen::Tensor, Eigen::Aligned> + ConstVec; + + // Unaligned Rank-1 tensor (vector) of scalar type T. + typedef Eigen::TensorMap > + UnalignedFlat; + typedef Eigen::TensorMap< + Eigen::Tensor > + UnalignedConstFlat; + typedef Eigen::TensorMap > + UnalignedVec; + typedef Eigen::TensorMap< + Eigen::Tensor > + UnalignedConstVec; + + // Rank-2 tensor (matrix) of scalar type T. + typedef Eigen::TensorMap, + Eigen::Aligned> + Matrix; + typedef Eigen::TensorMap< + Eigen::Tensor, Eigen::Aligned> + ConstMatrix; + + // Unaligned Rank-2 tensor (matrix) of scalar type T. + typedef Eigen::TensorMap > + UnalignedMatrix; + typedef Eigen::TensorMap< + Eigen::Tensor > + UnalignedConstMatrix; +}; + +typedef typename TTypes::Tensor32Bit::Index Index32; + +template +Eigen::DSizes To32BitDims(const DSizes& in) { + Eigen::DSizes out; + for (int i = 0; i < DSizes::count; ++i) { + out[i] = in[i]; + } + return out; +} + +template +typename TTypes::Tensor32Bit +To32Bit(TensorType in) { + typedef typename TTypes::Tensor32Bit RetType; + return RetType(in.data(), To32BitDims(in.dimensions())); +} + +} // namespace tensorflow +#endif // TENSORFLOW_CORE_FRAMEWORK_TENSOR_TYPES_H_ diff --git a/linux/include/tensorflow/core/framework/tensor_util.h b/linux/include/tensorflow/core/framework/tensor_util.h new file mode 100644 index 000000000..a7cf600ba --- /dev/null +++ b/linux/include/tensorflow/core/framework/tensor_util.h @@ -0,0 +1,164 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_FRAMEWORK_TENSOR_UTIL_H_ +#define TENSORFLOW_CORE_FRAMEWORK_TENSOR_UTIL_H_ + +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor.pb.h" +#include "tensorflow/core/framework/tensor_shape.pb.h" + +#include +namespace tensorflow { +namespace tensor { + +// DeepCopy returns a tensor whose contents are a deep copy of the +// contents of 'other'. This function is intended only for +// convenience, not speed. +// +// REQUIRES: 'other' must point to data stored in CPU memory. +// REQUIRES: 'other' must be a Tensor of a copy-able type if +// 'other' is not appropriately memory-aligned. +Tensor DeepCopy(const Tensor& other); + +// Concatenates 'tensors' into a single tensor, along their 0th dimension. +// +// REQUIRES: All members of 'tensors' must have the same data type parameter. +// REQUIRES: Each member of 'tensors' must have at least one dimension. +// REQUIRES: Each member of 'tensors' must point to data stored in CPU memory. +// REQUIRES: Each member of 'tensors' must be a Tensor of a copy-able type if it +// is not appropriately memory-aligned. +Status Concat(const gtl::ArraySlice& tensors, + Tensor* result) TF_MUST_USE_RESULT; + +// Splits 'tensor' into 'sizes.size()' individual tensors, along the 0th +// dimension. The ith output tensor has 0th-dimension size 'sizes[i]'. +// +// REQUIRES: 'tensor' must have at least one dimension. +// REQUIRES: 'tensor.dim_size(0)' must equal the sum of the elements of 'sizes'. +// REQUIRES: 'tensor' must point to data stored in CPU memory. +// REQUIRES: 'tensor' must be a Tensor of a copy-able type if it is not +// appropriately memory-aligned. +// +// Split() and Concat() are inverse operations. +Status Split(const Tensor& tensor, const gtl::ArraySlice& sizes, + std::vector* result) TF_MUST_USE_RESULT; + +namespace internal { +void SetTensorProtoShape(std::vector shape, + TensorShapeProto* shape_proto); + +// Defines value type dependent methods to manipulate `TensorProto`. +// Class specializations has to define following methods: +// static DataType GetDataType() +// static void AddValue(Type value, TensorProto* proto) +template +class TensorProtoHelper : public std::false_type {}; + +template <> +class TensorProtoHelper : public std::true_type { + public: + static DataType GetDataType() { return DataType::DT_STRING; } + static void AddValue(const string& value, TensorProto* proto) { + *proto->mutable_string_val()->Add() = value; + } +}; + +template <> +class TensorProtoHelper : public std::true_type { + public: + static DataType GetDataType() { return DataType::DT_INT32; } + static void AddValue(int32 value, TensorProto* proto) { + proto->mutable_int_val()->Add(value); + } +}; + +template <> +class TensorProtoHelper : public std::true_type { + public: + static DataType GetDataType() { return DataType::DT_INT64; } + static void AddValue(int64 value, TensorProto* proto) { + proto->mutable_int64_val()->Add(value); + } +}; + +template <> +class TensorProtoHelper : public std::true_type { + public: + static DataType GetDataType() { return DataType::DT_UINT32; } + static void AddValue(uint32 value, TensorProto* proto) { + proto->mutable_uint32_val()->Add(value); + } +}; + +template <> +class TensorProtoHelper : public std::true_type { + public: + static DataType GetDataType() { return DataType::DT_UINT64; } + static void AddValue(uint64 value, TensorProto* proto) { + proto->mutable_uint64_val()->Add(value); + } +}; + +template <> +class TensorProtoHelper : public std::true_type { + public: + static DataType GetDataType() { return DataType::DT_FLOAT; } + static void AddValue(float value, TensorProto* proto) { + proto->mutable_float_val()->Add(value); + } +}; + +template <> +class TensorProtoHelper : public std::true_type { + public: + static DataType GetDataType() { return DataType::DT_DOUBLE; } + static void AddValue(double value, TensorProto* proto) { + proto->mutable_double_val()->Add(value); + } +}; + +template <> +class TensorProtoHelper : public std::true_type { + public: + static DataType GetDataType() { return DataType::DT_BOOL; } + static void AddValue(bool value, TensorProto* proto) { + proto->mutable_bool_val()->Add(value); + } +}; +} // namespace internal + +// Creates a 'TensorProto' with specified shape and values. +// The dtype and a field to represent data values of the returned 'TensorProto' +// are determined based on type of the 'values' parameter. +template +typename std::enable_if::value, + TensorProto>::type +CreateTensorProto(const std::vector& values, + const std::vector& shape) { + TensorProto tensor; + using TypeHelper = internal::TensorProtoHelper; + tensor.set_dtype(TypeHelper::GetDataType()); + internal::SetTensorProtoShape(shape, tensor.mutable_tensor_shape()); + for (const auto& value : values) { + TypeHelper::AddValue(value, &tensor); + } + return tensor; +} + +} // namespace tensor +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_FRAMEWORK_TENSOR_UTIL_H_ diff --git a/linux/include/tensorflow/core/framework/tracking_allocator.h b/linux/include/tensorflow/core/framework/tracking_allocator.h new file mode 100644 index 000000000..5eafce662 --- /dev/null +++ b/linux/include/tensorflow/core/framework/tracking_allocator.h @@ -0,0 +1,133 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_FRAMEWORK_TRACKING_ALLOCATOR_H_ +#define TENSORFLOW_CORE_FRAMEWORK_TRACKING_ALLOCATOR_H_ + +#include +#include "tensorflow/core/framework/allocator.h" +#include "tensorflow/core/lib/core/refcount.h" +#include "tensorflow/core/lib/gtl/inlined_vector.h" +#include "tensorflow/core/platform/mutex.h" +#include "tensorflow/core/platform/thread_annotations.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +// TrackingAllocator is a wrapper for an Allocator. It keeps a running +// count of the number of bytes allocated through the wrapper. It is +// used by the Executor to "charge" allocations to particular Op +// executions. Each Op gets a separate TrackingAllocator wrapper +// around the underlying allocator. +// +// The implementation assumes the invariant that all calls to +// AllocateRaw by an Op (or work items spawned by the Op) will occur +// before the Op's Compute method returns. Thus the high watermark is +// established once Compute returns. +// +// DeallocateRaw can be called long after the Op has finished, +// e.g. when an output tensor is deallocated, and the wrapper cannot +// be deleted until the last of these calls has occurred. The +// TrackingAllocator keeps track of outstanding calls using a +// reference count, and deletes itself once the last call has been +// received and the high watermark has been retrieved. +struct AllocRecord { + AllocRecord(int64 a_btyes, int64 a_micros) + : alloc_bytes(a_btyes), alloc_micros(a_micros) {} + AllocRecord() : AllocRecord(0, 0) {} + + int64 alloc_bytes; + int64 alloc_micros; +}; + +class TrackingAllocator : public Allocator { + public: + explicit TrackingAllocator(Allocator* allocator, bool track_ids); + string Name() override { return allocator_->Name(); } + void* AllocateRaw(size_t alignment, size_t num_bytes) override { + return AllocateRaw(alignment, num_bytes, AllocationAttributes()); + } + void* AllocateRaw(size_t alignment, size_t num_bytes, + const AllocationAttributes& allocation_attr) override; + void DeallocateRaw(void* ptr) override; + bool TracksAllocationSizes() override; + size_t RequestedSize(const void* ptr) override; + size_t AllocatedSize(const void* ptr) override; + int64 AllocationId(const void* ptr) override; + void GetStats(AllocatorStats* stats) override; + void ClearStats() override; + + // If the underlying allocator tracks allocation sizes, this returns + // a tuple where the first value is the total number of bytes + // allocated through this wrapper, the second value is the high + // watermark of bytes allocated through this wrapper and the third value is + // the allocated bytes through this wrapper that are still alive. If the + // underlying allocator does not track allocation sizes the first + // value is the total number of bytes requested through this wrapper + // and the second and the third are 0. + // + std::tuple GetSizes(); + // After GetRecordsAndUnRef is called, the only further calls allowed + // on this wrapper are calls to DeallocateRaw with pointers that + // were allocated by this wrapper and have not yet been + // deallocated. After this call completes and all allocated pointers + // have been deallocated the wrapper will delete itself. + gtl::InlinedVector GetRecordsAndUnRef(); + // Returns a copy of allocation records collected so far. + gtl::InlinedVector GetCurrentRecords(); + + protected: + ~TrackingAllocator() override {} + + private: + bool UnRef() EXCLUSIVE_LOCKS_REQUIRED(mu_); + + Allocator* allocator_; // not owned. + mutex mu_; + // the number of calls to AllocateRaw that have not yet been matched + // by a corresponding call to DeAllocateRaw, plus 1 if the Executor + // has not yet read out the high watermark. + int ref_ GUARDED_BY(mu_); + // the current number of outstanding bytes that have been allocated + // by this wrapper, or 0 if the underlying allocator does not track + // allocation sizes. + size_t allocated_ GUARDED_BY(mu_); + // the maximum number of outstanding bytes that have been allocated + // by this wrapper, or 0 if the underlying allocator does not track + // allocation sizes. + size_t high_watermark_ GUARDED_BY(mu_); + // the total number of bytes that have been allocated by this + // wrapper if the underlying allocator tracks allocation sizes, + // otherwise the total number of bytes that have been requested by + // this allocator. + size_t total_bytes_ GUARDED_BY(mu_); + + gtl::InlinedVector allocations_ GUARDED_BY(mu_); + + // Track allocations locally if requested in the constructor and the + // underlying allocator doesn't already do it for us. + const bool track_sizes_locally_; + struct Chunk { + size_t requested_size; + size_t allocated_size; + int64 allocation_id; + }; + std::unordered_map in_use_ GUARDED_BY(mu_); + int64 next_allocation_id_ GUARDED_BY(mu_); +}; + +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_FRAMEWORK_TRACKING_ALLOCATOR_H_ diff --git a/linux/include/tensorflow/core/framework/type_index.h b/linux/include/tensorflow/core/framework/type_index.h new file mode 100644 index 000000000..989fc42e2 --- /dev/null +++ b/linux/include/tensorflow/core/framework/type_index.h @@ -0,0 +1,87 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_FRAMEWORK_TYPE_INDEX_H_ +#define TENSORFLOW_CORE_FRAMEWORK_TYPE_INDEX_H_ + +#include +#if defined(__GXX_RTTI) || defined(_CPPRTTI) +#include +#include +#endif // __GXX_RTTI + +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +// On some platforms, we would like to avoid using RTTI in order to have smaller +// binary sizes. The following #ifdef section provides a non-RTTI +// replacement for std::type_index (with a minimal set of functions needed by +// the TensorFlow framework, and more can be added if necessary). +#if !defined(__GXX_RTTI) && !defined(_CPPRTTI) + +// A thin TypeIndex class that mimics std::type_index but does not use RTTI. As +// a result, it does not provide the actual name of the type, and only returns a +// pre-baked string specifying that RTTI is disabled. +// The hash code provided in this class is unique for each class. However, it is +// generated at runtime so this hash code should not be serialized - the value +// for the same type can change from run to run. +class TypeIndex { + public: + TypeIndex(const TypeIndex& src) : hash_(src.hash_) {} + TypeIndex& operator=(const TypeIndex& src) { + hash_ = src.hash_; + return *this; + } + bool operator==(const TypeIndex& rhs) const { return (hash_ == rhs.hash_); } + bool operator!=(const TypeIndex& rhs) const { return (hash_ != rhs.hash_); } + ~TypeIndex() {} + + const char* name() const { return "[RTTI disabled for Android]"; } + uint64 hash_code() const { return hash_; } + + // Returns a TypeIndex object that corresponds to a typename. + template + static TypeIndex Make() { + static bool hash_bit[1]; + return TypeIndex(static_cast(reinterpret_cast(hash_bit))); + } + + private: + // We hide the constructor of the TypeIndex class. Use the templated + // Make() function to create a TypeIndex object. + TypeIndex(const uint64 hash) : hash_(hash) {} + uint64 hash_; +}; + +template +inline TypeIndex MakeTypeIndex() { + return TypeIndex::Make(); +} + +#else // __GXX_RTTI + +// In the presence of RTTI, we will simply delegate to std::type_index for +// runtime type inference. +typedef std::type_index TypeIndex; +template +inline TypeIndex MakeTypeIndex() { + return TypeIndex(typeid(T)); +} + +#endif // __GXX_RTTI +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_FRAMEWORK_TYPE_INDEX_H_ diff --git a/linux/include/tensorflow/core/framework/type_traits.h b/linux/include/tensorflow/core/framework/type_traits.h new file mode 100644 index 000000000..96fbf9293 --- /dev/null +++ b/linux/include/tensorflow/core/framework/type_traits.h @@ -0,0 +1,109 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_FRAMEWORK_TYPE_TRAITS_H_ +#define TENSORFLOW_CORE_FRAMEWORK_TYPE_TRAITS_H_ + +#include +#include + +#include "tensorflow/core/framework/numeric_types.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +// Functions to define quantization attribute of types. +struct true_type { + static const bool value = true; +}; +struct false_type { + static const bool value = false; +}; + +// Default is_quantized is false. +template +struct is_quantized : false_type {}; + +// Specialize the quantized types. +template <> +struct is_quantized : true_type {}; +template <> +struct is_quantized : true_type {}; +template <> +struct is_quantized : true_type {}; +template <> +struct is_quantized : true_type {}; +template <> +struct is_quantized : true_type {}; + +// Default is_complex is false. +template +struct is_complex : false_type {}; + +// Specialize std::complex and std::complex types. +template <> +struct is_complex> : true_type {}; +template <> +struct is_complex> : true_type {}; + +// is_simple_type::value if T[] can be safely constructed and destructed +// without running T() and ~T(). We do not use std::is_trivial +// directly because std::complex and std::complex are +// not trivial, but their arrays can be constructed and destructed +// without running their default ctors and dtors. +template +struct is_simple_type { + static constexpr bool value = + std::is_trivial::value || std::is_same::value || + std::is_same::value || std::is_same::value || + is_quantized::value || std::is_same::value; +}; + +} // namespace tensorflow + +// Define numeric limits for our quantized as subclasses of the +// standard types. +namespace std { +template <> +class numeric_limits + : public numeric_limits {}; +template <> +class numeric_limits + : public numeric_limits {}; +template <> +class numeric_limits + : public numeric_limits {}; +template <> +class numeric_limits + : public numeric_limits {}; +template <> +class numeric_limits + : public numeric_limits {}; + +// Specialize is_signed for quantized types. +template <> +struct is_signed : public is_signed {}; +template <> +struct is_signed : public is_signed {}; +template <> +struct is_signed : public is_signed {}; +template <> +struct is_signed : public is_signed {}; +template <> +struct is_signed : public is_signed {}; + +} // namespace std + +#endif // TENSORFLOW_CORE_FRAMEWORK_TYPE_TRAITS_H_ diff --git a/linux/include/tensorflow/core/framework/types.h b/linux/include/tensorflow/core/framework/types.h new file mode 100644 index 000000000..2e96b0578 --- /dev/null +++ b/linux/include/tensorflow/core/framework/types.h @@ -0,0 +1,485 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_FRAMEWORK_TYPES_H_ +#define TENSORFLOW_CORE_FRAMEWORK_TYPES_H_ + +#include +#include +#include + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +// Disable clang-format to prevent 'FixedPoint' header from being included +// before 'Tensor' header on which it depends. +// clang-format off +#include "third_party/eigen3/unsupported/Eigen/CXX11/FixedPoint" +// clang-format on +#include "tensorflow/core/framework/bfloat16.h" +#include "tensorflow/core/framework/numeric_types.h" +#include "tensorflow/core/framework/resource_handle.h" +#include "tensorflow/core/framework/types.pb.h" +#include "tensorflow/core/lib/core/stringpiece.h" +#include "tensorflow/core/lib/gtl/array_slice.h" +#include "tensorflow/core/lib/gtl/inlined_vector.h" +#include "tensorflow/core/platform/logging.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +class Variant; + +// MemoryType is used to describe whether input or output Tensors of +// an OpKernel should reside in "Host memory" (e.g., CPU memory) or +// "Device" Memory (CPU memory for CPU devices, GPU memory for GPU +// devices). +enum MemoryType { + DEVICE_MEMORY = 0, + HOST_MEMORY = 1, +}; + +// A DeviceType is just a string, but we wrap it up in a class to give +// some type checking as we're passing these around +class DeviceType { + public: + DeviceType(const char* type) // NOLINT(runtime/explicit) + : type_(type) {} + + explicit DeviceType(StringPiece type) : type_(type.data(), type.size()) {} + + const char* type() const { return type_.c_str(); } + const string& type_string() const { return type_; } + + bool operator<(const DeviceType& other) const; + bool operator==(const DeviceType& other) const; + bool operator!=(const DeviceType& other) const { return !(*this == other); } + + private: + string type_; +}; +std::ostream& operator<<(std::ostream& os, const DeviceType& d); + +// Convenient constants that can be passed to a DeviceType constructor +TF_EXPORT extern const char* const DEVICE_CPU; // "CPU" +TF_EXPORT extern const char* const DEVICE_GPU; // "GPU" +TF_EXPORT extern const char* const DEVICE_SYCL; // "SYCL" + +template +struct DeviceName {}; + +template <> +struct DeviceName { + static const std::string value; +}; + +#if GOOGLE_CUDA +template <> +struct DeviceName { + static const std::string value; +}; +#endif // GOOGLE_CUDA + +#ifdef TENSORFLOW_USE_SYCL +template <> +struct DeviceName { + static const std::string value; +}; +#endif // TENSORFLOW_USE_SYCL + +typedef gtl::InlinedVector MemoryTypeVector; +typedef gtl::ArraySlice MemoryTypeSlice; + +typedef gtl::InlinedVector DataTypeVector; +typedef gtl::ArraySlice DataTypeSlice; + +typedef gtl::InlinedVector DeviceTypeVector; + +// Convert the enums to strings for errors: +string DataTypeString(DataType dtype); +string DeviceTypeString(const DeviceType& device_type); +string DataTypeSliceString(const DataTypeSlice dtypes); +inline string DataTypeVectorString(const DataTypeVector& dtypes) { + return DataTypeSliceString(dtypes); +} + +// DataTypeSet represents a set of DataType values as a simple and efficient +// bit mask. Note that DataTypeSet cannot represent all DataType values; it +// cannot represent any of the DT_*_REF values. +class DataTypeSet { + private: + const uint32 mask_; + + static constexpr uint32 kNumBits = 32; + + public: + constexpr DataTypeSet(const DataTypeSet& other) : mask_(other.mask_) {} + explicit constexpr DataTypeSet(uint32 mask) : mask_(mask) {} + + constexpr bool Contains(DataType dt) const { + return (static_cast(dt) < kNumBits) && + ((mask_ >> static_cast(dt)) & 1u) != 0u; + } + + class Iterator { + const DataTypeSet& set_; + uint32 pos_; + + public: + Iterator(const DataTypeSet& set, uint32 pos) : set_(set), pos_(pos) { + DCHECK_LE(pos, kNumBits); + } + DataType operator*() const { return static_cast(pos_); } + Iterator& operator++() { + ++pos_; + DCHECK_LE(pos_, kNumBits); + if (pos_ < kNumBits) { + uint32 remaining_mask = set_.mask_ >> pos_; + if (remaining_mask != 0u) { + pos_ += ctz_uint32(remaining_mask); + } + } + DCHECK_LE(pos_, kNumBits); + return *this; + } + bool operator==(const Iterator& other) const { return pos_ == other.pos_; } + bool operator!=(const Iterator& other) const { return !(*this == other); } + size_t operator-(const Iterator& other) const { + return this->pos_ - other.pos_; + } + }; + + static uint32 ctz_uint32(uint32 x) { + DCHECK_NE(x, 0u); +#ifdef __GNUC__ + return __builtin_ctz(x); +#else + uint32 n = 0u; + while ((x & 1u) == 0u) { + x >>= 1; + ++n; + } + return n; +#endif + } + + static uint32 clz_uint32(uint32 x) { + DCHECK_NE(x, 0u); +#ifdef __GNUC__ + return __builtin_clz(x); +#else + uint32 n = 0u; + while ((x >> (kNumBits - 1u)) == 0u) { + x <<= 1; + ++n; + } + return n; +#endif + } + + Iterator begin() const { + // The begin position is the index of the first bit set to 1 in the entire + // bit mask. If there are no bits set to 1, then the index is 0. + if (mask_ != 0) { + return Iterator(*this, ctz_uint32(mask_)); + } + // The set is empty. + return Iterator(*this, 0); + } + + Iterator end() const { + // The end position is the index of the highest bit that is set, plus 1. + // If there are no bits set to 1, then the index is 0. + if (mask_ != 0) { + return Iterator(*this, kNumBits - clz_uint32(mask_)); + } + // The set is empty. + return Iterator(*this, 0); + } + + size_t size() const { +#if defined(__GNUC__) + return __builtin_popcount(mask_); +#else + size_t n = 0; + uint32 x = mask_; + while (x > 0) { + n += x & 1u; + x >>= 1; + } + return n; +#endif + } + + constexpr DataTypeSet operator|(const DataTypeSet& other) const { + return DataTypeSet(mask_ | other.mask_); + } +}; + +// If "sp" names a valid type, store it in "*dt" and return true. Otherwise, +// return false. +bool DataTypeFromString(StringPiece sp, DataType* dt); + +constexpr inline DataTypeSet ToSet(DataType dt) { + return DataTypeSet(1u << static_cast(dt)); +} + +// DT_FLOAT + kDataTypeRefOffset == DT_FLOAT_REF, etc. +enum { kDataTypeRefOffset = 100 }; +inline bool IsRefType(DataType dtype) { + return dtype > static_cast(kDataTypeRefOffset); +} +inline DataType MakeRefType(DataType dtype) { + DCHECK(!IsRefType(dtype)); + return static_cast(dtype + kDataTypeRefOffset); +} +inline DataType RemoveRefType(DataType dtype) { + DCHECK(IsRefType(dtype)); + return static_cast(dtype - kDataTypeRefOffset); +} +inline DataType BaseType(DataType dtype) { + return IsRefType(dtype) ? RemoveRefType(dtype) : dtype; +} + +// Returns true if the actual type is the same as or ref of the expected type. +inline bool TypesCompatible(DataType expected, DataType actual) { + return expected == actual || expected == BaseType(actual); +} + +// Does not include _ref types. +constexpr DataTypeSet kAllTypes = + ToSet(DT_FLOAT) | ToSet(DT_DOUBLE) | ToSet(DT_INT32) | ToSet(DT_UINT8) | + ToSet(DT_INT16) | ToSet(DT_UINT16) | ToSet(DT_INT8) | ToSet(DT_STRING) | + ToSet(DT_COMPLEX64) | ToSet(DT_COMPLEX128) | ToSet(DT_INT64) | + ToSet(DT_BOOL) | ToSet(DT_QINT8) | ToSet(DT_QUINT8) | ToSet(DT_QINT16) | + ToSet(DT_QUINT16) | ToSet(DT_QINT32) | ToSet(DT_HALF) | ToSet(DT_RESOURCE) | + ToSet(DT_VARIANT) | ToSet(DT_UINT32) | ToSet(DT_UINT64) | + ToSet(DT_BFLOAT16); +inline const DataTypeSet& AllTypes() { return kAllTypes; } + +#if !defined(IS_MOBILE_PLATFORM) || defined(SUPPORT_SELECTIVE_REGISTRATION) + +// Types that support '<' and '>'. +constexpr DataTypeSet kRealNumberTypes = + ToSet(DT_FLOAT) | ToSet(DT_DOUBLE) | ToSet(DT_INT32) | ToSet(DT_INT64) | + ToSet(DT_UINT8) | ToSet(DT_INT16) | ToSet(DT_INT8) | ToSet(DT_UINT16) | + ToSet(DT_HALF) | ToSet(DT_UINT32) | ToSet(DT_UINT64) | ToSet(DT_BFLOAT16); +inline const DataTypeSet RealNumberTypes() { return kRealNumberTypes; } + +// Return the list of all numeric types. +// Includes complex and quantized types. +// NOTE: On Android, we only include the float and int32 types for now. +const DataTypeSet kNumberTypes = + ToSet(DT_FLOAT) | ToSet(DT_DOUBLE) | ToSet(DT_INT64) | ToSet(DT_INT32) | + ToSet(DT_UINT8) | ToSet(DT_UINT16) | ToSet(DT_INT16) | ToSet(DT_INT8) | + ToSet(DT_COMPLEX64) | ToSet(DT_COMPLEX128) | ToSet(DT_QINT8) | + ToSet(DT_QUINT8) | ToSet(DT_QINT32) | ToSet(DT_HALF) | ToSet(DT_UINT32) | + ToSet(DT_UINT64) | ToSet(DT_BFLOAT16); +inline const DataTypeSet& NumberTypes() { return kNumberTypes; } + +constexpr DataTypeSet kQuantizedTypes = ToSet(DT_QINT8) | ToSet(DT_QUINT8) | + ToSet(DT_QINT16) | ToSet(DT_QUINT16) | + ToSet(DT_QINT32); +inline const DataTypeSet& QuantizedTypes() { return kQuantizedTypes; } + +// Types that support '<' and '>', including quantized types. +const DataTypeSet kRealAndQuantizedTypes = + ToSet(DT_FLOAT) | ToSet(DT_DOUBLE) | ToSet(DT_INT32) | ToSet(DT_INT64) | + ToSet(DT_UINT8) | ToSet(DT_UINT16) | ToSet(DT_UINT16) | ToSet(DT_INT8) | + ToSet(DT_QINT8) | ToSet(DT_QUINT8) | ToSet(DT_QINT16) | ToSet(DT_QUINT16) | + ToSet(DT_QINT32) | ToSet(DT_HALF) | ToSet(DT_BFLOAT16); +inline const DataTypeSet& RealAndQuantizedTypes() { + return kRealAndQuantizedTypes; +} + +#elif defined(__ANDROID_TYPES_FULL__) + +constexpr DataTypeSet kRealNumberTypes = + ToSet(DT_FLOAT) | ToSet(DT_INT32) | ToSet(DT_INT64) | ToSet(DT_HALF); +inline DataTypeSet RealNumberTypes() { return kRealNumberTypes; } + +constexpr DataTypeSet kNumberTypes = + ToSet(DT_FLOAT) | ToSet(DT_INT32) | ToSet(DT_INT64) | ToSet(DT_QINT8) | + ToSet(DT_QUINT8) | ToSet(DT_QINT32) | ToSet(DT_HALF); +inline DataTypeSet NumberTypes() { return kNumberTypes; } + +constexpr DataTypeSet kQuantizedTypes = ToSet(DT_QINT8) | ToSet(DT_QUINT8) | + ToSet(DT_QINT16) | ToSet(DT_QUINT16) | + ToSet(DT_QINT32); +inline DataTypeSet QuantizedTypes() { return kQuantizedTypes; } + +constexpr DataTypeSet kRealAndQuantizedTypes = + ToSet(DT_FLOAT) | ToSet(DT_INT32) | ToSet(DT_INT64) | ToSet(DT_QINT8) | + ToSet(DT_QUINT8) | ToSet(DT_QINT16) | ToSet(DT_QUINT16) | ToSet(DT_QINT32) | + ToSet(DT_HALF); +inline DataTypeSet RealAndQuantizedTypes() { return kRealAndQuantizedTypes; } + +#else // defined(IS_MOBILE_PLATFORM) && !defined(__ANDROID_TYPES_FULL__) + +constexpr DataTypeSet kRealNumberTypes = ToSet(DT_FLOAT) | ToSet(DT_INT32); +inline DataTypeSet RealNumberTypes() { return kRealNumberTypes; } + +constexpr DataTypeSet kNumberTypes = ToSet(DT_FLOAT) | ToSet(DT_INT32) | + ToSet(DT_QINT8) | ToSet(DT_QUINT8) | + ToSet(DT_QINT32); +inline DataTypeSet NumberTypes() { return kNumberTypes; } + +constexpr DataTypeSet kQuantizedTypes = ToSet(DT_QINT8) | ToSet(DT_QUINT8) | + ToSet(DT_QINT16) | ToSet(DT_QUINT16) | + ToSet(DT_QINT32); +inline DataTypeSet QuantizedTypes() { return kQuantizedTypes; } + +constexpr DataTypeSet kRealAndQuantizedTypes = + ToSet(DT_FLOAT) | ToSet(DT_INT32) | ToSet(DT_QINT8) | ToSet(DT_QUINT8) | + ToSet(DT_QINT16) | ToSet(DT_QUINT16) | ToSet(DT_QINT32); +inline DataTypeSet RealAndQuantizedTypes() { return kRealAndQuantizedTypes; } + +#endif // defined(IS_MOBILE_PLATFORM) + +// Validates type T for whether it is a supported DataType. +template +struct IsValidDataType; + +// DataTypeToEnum::v() and DataTypeToEnum::value are the DataType +// constants for T, e.g. DataTypeToEnum::v() is DT_FLOAT. +template +struct DataTypeToEnum { + static_assert(IsValidDataType::value, "Specified Data Type not supported"); +}; // Specializations below + +// EnumToDataType::Type is the type for DataType constant VALUE, e.g. +// EnumToDataType::Type is float. +template +struct EnumToDataType {}; // Specializations below + +// Template specialization for both DataTypeToEnum and EnumToDataType. +#define MATCH_TYPE_AND_ENUM(TYPE, ENUM) \ + template <> \ + struct DataTypeToEnum { \ + static DataType v() { return ENUM; } \ + static DataType ref() { return MakeRefType(ENUM); } \ + static constexpr DataType value = ENUM; \ + }; \ + template <> \ + struct IsValidDataType { \ + static constexpr bool value = true; \ + }; \ + template <> \ + struct EnumToDataType { \ + typedef TYPE Type; \ + } + +MATCH_TYPE_AND_ENUM(float, DT_FLOAT); +MATCH_TYPE_AND_ENUM(double, DT_DOUBLE); +MATCH_TYPE_AND_ENUM(int32, DT_INT32); +MATCH_TYPE_AND_ENUM(uint32, DT_UINT32); +MATCH_TYPE_AND_ENUM(uint16, DT_UINT16); +MATCH_TYPE_AND_ENUM(uint8, DT_UINT8); +MATCH_TYPE_AND_ENUM(int16, DT_INT16); +MATCH_TYPE_AND_ENUM(int8, DT_INT8); +MATCH_TYPE_AND_ENUM(string, DT_STRING); +MATCH_TYPE_AND_ENUM(complex64, DT_COMPLEX64); +MATCH_TYPE_AND_ENUM(complex128, DT_COMPLEX128); +MATCH_TYPE_AND_ENUM(int64, DT_INT64); +MATCH_TYPE_AND_ENUM(uint64, DT_UINT64); +MATCH_TYPE_AND_ENUM(bool, DT_BOOL); +MATCH_TYPE_AND_ENUM(qint8, DT_QINT8); +MATCH_TYPE_AND_ENUM(quint8, DT_QUINT8); +MATCH_TYPE_AND_ENUM(qint16, DT_QINT16); +MATCH_TYPE_AND_ENUM(quint16, DT_QUINT16); +MATCH_TYPE_AND_ENUM(qint32, DT_QINT32); +MATCH_TYPE_AND_ENUM(bfloat16, DT_BFLOAT16); +MATCH_TYPE_AND_ENUM(Eigen::half, DT_HALF); +MATCH_TYPE_AND_ENUM(ResourceHandle, DT_RESOURCE); +MATCH_TYPE_AND_ENUM(Variant, DT_VARIANT); + +#undef MATCH_TYPE_AND_ENUM + +// All types not specialized are marked invalid. +template +struct IsValidDataType { + static constexpr bool value = false; +}; + +// Extra validity checking; not part of public API. +static_assert(IsValidDataType::value, "Incorrect impl for int64"); +static_assert(IsValidDataType::value, "Incorrect impl for int32"); + +// TODO(jeff): Maybe unify this with Tensor::CanUseDMA, or the underlying +// is_simple in tensor.cc (and possible choose a more general name?) +constexpr DataTypeSet kDataTypesCanUseMemcpy = + ToSet(DT_FLOAT) | ToSet(DT_DOUBLE) | ToSet(DT_INT32) | ToSet(DT_UINT32) | + ToSet(DT_UINT8) | ToSet(DT_UINT16) | ToSet(DT_INT16) | ToSet(DT_INT8) | + ToSet(DT_COMPLEX64) | ToSet(DT_COMPLEX128) | ToSet(DT_INT64) | + ToSet(DT_UINT64) | ToSet(DT_BOOL) | ToSet(DT_QINT8) | ToSet(DT_QUINT8) | + ToSet(DT_QINT16) | ToSet(DT_QUINT16) | ToSet(DT_QINT32) | + ToSet(DT_BFLOAT16) | ToSet(DT_HALF); +inline bool DataTypeCanUseMemcpy(DataType dt) { + return kDataTypesCanUseMemcpy.Contains(dt); +} + +// Returns true iff 'dt' is a real, non-quantized floating point type. +constexpr DataTypeSet kDataTypeIsFloating = + ToSet(DT_HALF) | ToSet(DT_BFLOAT16) | ToSet(DT_FLOAT) | ToSet(DT_DOUBLE); +inline bool DataTypeIsFloating(DataType dt) { + return kDataTypeIsFloating.Contains(dt); +} + +// Returns true iff 'dt' is a complex type. +constexpr DataTypeSet kDataTypeIsComplex = + ToSet(DT_COMPLEX64) | ToSet(DT_COMPLEX128); +inline bool DataTypeIsComplex(DataType dt) { + return kDataTypeIsComplex.Contains(dt); +} + +inline bool DataTypeIsQuantized(DataType dt) { + return kQuantizedTypes.Contains(dt); +} + +// Is the dtype nonquantized integral? +constexpr DataTypeSet kDataTypeIsInteger = + ToSet(DT_INT8) | ToSet(DT_UINT8) | ToSet(DT_INT16) | ToSet(DT_UINT16) | + ToSet(DT_INT32) | ToSet(DT_UINT32) | ToSet(DT_INT64) | ToSet(DT_UINT64); +inline bool DataTypeIsInteger(DataType dt) { + return kDataTypeIsInteger.Contains(dt); +} + +// Is the dtype a signed integral type? +constexpr DataTypeSet kDataTypeIsSigned = + ToSet(DT_INT8) | ToSet(DT_INT16) | ToSet(DT_INT32) | ToSet(DT_INT64); +inline bool DataTypeIsSigned(DataType dt) { + return kDataTypeIsSigned.Contains(dt); +} + +// Is the dtype an unsigned integral type? +constexpr DataTypeSet kDataTypeIsUnsigned = + ToSet(DT_UINT8) | ToSet(DT_UINT16) | ToSet(DT_UINT32) | ToSet(DT_UINT64); +inline bool DataTypeIsUnsigned(DataType dt) { + return kDataTypeIsUnsigned.Contains(dt); +} + +// Returns a 0 on failure +int DataTypeSize(DataType dt); + +// Returns HOST_MEMORY if `dtype` is always on host or is a DT_INT32, +// DEVICE_MEMORY otherwise. +MemoryType MTypeFromDType(const DataType dtype); + +// Types that always sit on host: DT_STRING, DT_STRING_REF, DT_RESOURCE. +// For DT_RESOURCE, the handle always sits on host (even if the underlying +// object has device-allocated resources). +bool DataTypeAlwaysOnHost(DataType dt); + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_FRAMEWORK_TYPES_H_ diff --git a/linux/include/tensorflow/core/framework/types.pb.h b/linux/include/tensorflow/core/framework/types.pb.h new file mode 100644 index 000000000..ba3f005b8 --- /dev/null +++ b/linux/include/tensorflow/core/framework/types.pb.h @@ -0,0 +1,151 @@ +// Generated by the protocol buffer compiler. DO NOT EDIT! +// source: tensorflow/core/framework/types.proto + +#ifndef PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2ftypes_2eproto +#define PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2ftypes_2eproto + +#include + +#include + +#if GOOGLE_PROTOBUF_VERSION < 3006000 +#error This file was generated by a newer version of protoc which is +#error incompatible with your Protocol Buffer headers. Please update +#error your headers. +#endif +#if 3006000 < GOOGLE_PROTOBUF_MIN_PROTOC_VERSION +#error This file was generated by an older version of protoc which is +#error incompatible with your Protocol Buffer headers. Please +#error regenerate this file with a newer version of protoc. +#endif + +#include +#include +#include +#include +#include +#include +#include +#include // IWYU pragma: export +#include // IWYU pragma: export +#include +// @@protoc_insertion_point(includes) +#define PROTOBUF_INTERNAL_EXPORT_protobuf_tensorflow_2fcore_2fframework_2ftypes_2eproto + +namespace protobuf_tensorflow_2fcore_2fframework_2ftypes_2eproto { +// Internal implementation detail -- do not use these members. +struct TableStruct { + static const ::google::protobuf::internal::ParseTableField entries[]; + static const ::google::protobuf::internal::AuxillaryParseTableField aux[]; + static const ::google::protobuf::internal::ParseTable schema[1]; + static const ::google::protobuf::internal::FieldMetadata field_metadata[]; + static const ::google::protobuf::internal::SerializationTable serialization_table[]; + static const ::google::protobuf::uint32 offsets[]; +}; +void AddDescriptors(); +} // namespace protobuf_tensorflow_2fcore_2fframework_2ftypes_2eproto +namespace tensorflow { +} // namespace tensorflow +namespace tensorflow { + +enum DataType { + DT_INVALID = 0, + DT_FLOAT = 1, + DT_DOUBLE = 2, + DT_INT32 = 3, + DT_UINT8 = 4, + DT_INT16 = 5, + DT_INT8 = 6, + DT_STRING = 7, + DT_COMPLEX64 = 8, + DT_INT64 = 9, + DT_BOOL = 10, + DT_QINT8 = 11, + DT_QUINT8 = 12, + DT_QINT32 = 13, + DT_BFLOAT16 = 14, + DT_QINT16 = 15, + DT_QUINT16 = 16, + DT_UINT16 = 17, + DT_COMPLEX128 = 18, + DT_HALF = 19, + DT_RESOURCE = 20, + DT_VARIANT = 21, + DT_UINT32 = 22, + DT_UINT64 = 23, + DT_FLOAT_REF = 101, + DT_DOUBLE_REF = 102, + DT_INT32_REF = 103, + DT_UINT8_REF = 104, + DT_INT16_REF = 105, + DT_INT8_REF = 106, + DT_STRING_REF = 107, + DT_COMPLEX64_REF = 108, + DT_INT64_REF = 109, + DT_BOOL_REF = 110, + DT_QINT8_REF = 111, + DT_QUINT8_REF = 112, + DT_QINT32_REF = 113, + DT_BFLOAT16_REF = 114, + DT_QINT16_REF = 115, + DT_QUINT16_REF = 116, + DT_UINT16_REF = 117, + DT_COMPLEX128_REF = 118, + DT_HALF_REF = 119, + DT_RESOURCE_REF = 120, + DT_VARIANT_REF = 121, + DT_UINT32_REF = 122, + DT_UINT64_REF = 123, + DataType_INT_MIN_SENTINEL_DO_NOT_USE_ = ::google::protobuf::kint32min, + DataType_INT_MAX_SENTINEL_DO_NOT_USE_ = ::google::protobuf::kint32max +}; +bool DataType_IsValid(int value); +const DataType DataType_MIN = DT_INVALID; +const DataType DataType_MAX = DT_UINT64_REF; +const int DataType_ARRAYSIZE = DataType_MAX + 1; + +const ::google::protobuf::EnumDescriptor* DataType_descriptor(); +inline const ::std::string& DataType_Name(DataType value) { + return ::google::protobuf::internal::NameOfEnum( + DataType_descriptor(), value); +} +inline bool DataType_Parse( + const ::std::string& name, DataType* value) { + return ::google::protobuf::internal::ParseNamedEnum( + DataType_descriptor(), name, value); +} +// =================================================================== + + +// =================================================================== + + +// =================================================================== + +#ifdef __GNUC__ + #pragma GCC diagnostic push + #pragma GCC diagnostic ignored "-Wstrict-aliasing" +#endif // __GNUC__ +#ifdef __GNUC__ + #pragma GCC diagnostic pop +#endif // __GNUC__ + +// @@protoc_insertion_point(namespace_scope) + +} // namespace tensorflow + +namespace google { +namespace protobuf { + +template <> struct is_proto_enum< ::tensorflow::DataType> : ::std::true_type {}; +template <> +inline const EnumDescriptor* GetEnumDescriptor< ::tensorflow::DataType>() { + return ::tensorflow::DataType_descriptor(); +} + +} // namespace protobuf +} // namespace google + +// @@protoc_insertion_point(global_scope) + +#endif // PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2ftypes_2eproto diff --git a/linux/include/tensorflow/core/framework/types.pb_text-impl.h b/linux/include/tensorflow/core/framework/types.pb_text-impl.h new file mode 100644 index 000000000..4894ed093 --- /dev/null +++ b/linux/include/tensorflow/core/framework/types.pb_text-impl.h @@ -0,0 +1,18 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_framework_types_proto_IMPL_H_ +#define tensorflow_core_framework_types_proto_IMPL_H_ + +#include "tensorflow/core/framework/types.pb.h" +#include "tensorflow/core/framework/types.pb_text.h" +#include "tensorflow/core/lib/strings/proto_text_util.h" +#include "tensorflow/core/lib/strings/scanner.h" + +namespace tensorflow { + +namespace internal { + +} // namespace internal + +} // namespace tensorflow + +#endif // tensorflow_core_framework_types_proto_IMPL_H_ diff --git a/linux/include/tensorflow/core/framework/types.pb_text.h b/linux/include/tensorflow/core/framework/types.pb_text.h new file mode 100644 index 000000000..b12fb212d --- /dev/null +++ b/linux/include/tensorflow/core/framework/types.pb_text.h @@ -0,0 +1,18 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_framework_types_proto_H_ +#define tensorflow_core_framework_types_proto_H_ + +#include "tensorflow/core/framework/types.pb.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/protobuf.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +// Enum text output for tensorflow.DataType +const char* EnumName_DataType( + ::tensorflow::DataType value); + +} // namespace tensorflow + +#endif // tensorflow_core_framework_types_proto_H_ diff --git a/linux/include/tensorflow/core/framework/unique_tensor_references.h b/linux/include/tensorflow/core/framework/unique_tensor_references.h new file mode 100644 index 000000000..752003430 --- /dev/null +++ b/linux/include/tensorflow/core/framework/unique_tensor_references.h @@ -0,0 +1,81 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_FRAMEWORK_UNIQUE_TENSOR_REFERENCES_H_ +#define TENSORFLOW_FRAMEWORK_UNIQUE_TENSOR_REFERENCES_H_ + +#include + +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_reference.h" +#include "tensorflow/core/lib/gtl/inlined_vector.h" +#include "tensorflow/core/platform/macros.h" + +namespace tensorflow { + +// Helper class to maintain a unique set of tensor references. In the +// common case there are not many references, so an inline vector is +// used for <= kInVector unique elements, defaulting to 4 since that +// is the inlined size of TensorReferenceVector. To avoid N^2 +// operations when adding N items, any larger number of unique tensor +// references switches to using an unordered set. +class UniqueTensorReferences { + public: + UniqueTensorReferences() : frozen_(false), referenced_tensors_set_(nullptr) {} + + ~UniqueTensorReferences(); + + // Adds a reference to tensor if its buffer is not already referenced. + void Add(const Tensor& tensor); + + // No more references may be added after this is called. The unique + // references are returning in out_vector. + void FreezeAndReturnReferences(TensorReferenceVector* out_vector); + + private: + // Up to kInVector elements are stored in reference_tensors_vector_ + // to avoid any allocations or hash computations in the common + // case. When more unique elements are added they move to + // referenced_tensors_set_ to avoid an N^2 algorithm on insert. + static const int kInVector = 4; // Must be >= 1. + + struct TensorReferenceEqualFn { + bool operator()(const TensorReference& t1, + const TensorReference& t2) const { + return t1.SharesBufferWith(t2); + } + }; + + struct TensorReferenceHashFn { + size_t operator()(const TensorReference& t) const { return t.BufferHash(); } + }; + + bool frozen_; + TensorReferenceVector referenced_tensors_vector_; + + typedef std::unordered_set + ReferencedTensorsSet; + // Lazily allocated hash set for when the number of tensors becomes too large. + // If this is non-NULL, then we use the hash set, otherwise, we use the + // referenced_tensors_vector_ (and do O(N^2) work per insertion). + ReferencedTensorsSet* referenced_tensors_set_; + + TF_DISALLOW_COPY_AND_ASSIGN(UniqueTensorReferences); +}; + +} // end namespace tensorflow + +#endif // TENSORFLOW_FRAMEWORK_UNIQUE_TENSOR_REFERENCES_H_ diff --git a/linux/include/tensorflow/core/framework/variable.pb.h b/linux/include/tensorflow/core/framework/variable.pb.h new file mode 100644 index 000000000..3999cf23f --- /dev/null +++ b/linux/include/tensorflow/core/framework/variable.pb.h @@ -0,0 +1,1063 @@ +// Generated by the protocol buffer compiler. DO NOT EDIT! +// source: tensorflow/core/framework/variable.proto + +#ifndef PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2fvariable_2eproto +#define PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2fvariable_2eproto + +#include + +#include + +#if GOOGLE_PROTOBUF_VERSION < 3006000 +#error This file was generated by a newer version of protoc which is +#error incompatible with your Protocol Buffer headers. Please update +#error your headers. +#endif +#if 3006000 < GOOGLE_PROTOBUF_MIN_PROTOC_VERSION +#error This file was generated by an older version of protoc which is +#error incompatible with your Protocol Buffer headers. Please +#error regenerate this file with a newer version of protoc. +#endif + +#include +#include +#include +#include +#include +#include +#include +#include +#include // IWYU pragma: export +#include // IWYU pragma: export +#include +// @@protoc_insertion_point(includes) +#define PROTOBUF_INTERNAL_EXPORT_protobuf_tensorflow_2fcore_2fframework_2fvariable_2eproto + +namespace protobuf_tensorflow_2fcore_2fframework_2fvariable_2eproto { +// Internal implementation detail -- do not use these members. +struct TableStruct { + static const ::google::protobuf::internal::ParseTableField entries[]; + static const ::google::protobuf::internal::AuxillaryParseTableField aux[]; + static const ::google::protobuf::internal::ParseTable schema[2]; + static const ::google::protobuf::internal::FieldMetadata field_metadata[]; + static const ::google::protobuf::internal::SerializationTable serialization_table[]; + static const ::google::protobuf::uint32 offsets[]; +}; +void AddDescriptors(); +} // namespace protobuf_tensorflow_2fcore_2fframework_2fvariable_2eproto +namespace tensorflow { +class SaveSliceInfoDef; +class SaveSliceInfoDefDefaultTypeInternal; +extern SaveSliceInfoDefDefaultTypeInternal _SaveSliceInfoDef_default_instance_; +class VariableDef; +class VariableDefDefaultTypeInternal; +extern VariableDefDefaultTypeInternal _VariableDef_default_instance_; +} // namespace tensorflow +namespace google { +namespace protobuf { +template<> ::tensorflow::SaveSliceInfoDef* Arena::CreateMaybeMessage<::tensorflow::SaveSliceInfoDef>(Arena*); +template<> ::tensorflow::VariableDef* Arena::CreateMaybeMessage<::tensorflow::VariableDef>(Arena*); +} // namespace protobuf +} // namespace google +namespace tensorflow { + +// =================================================================== + +class VariableDef : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.VariableDef) */ { + public: + VariableDef(); + virtual ~VariableDef(); + + VariableDef(const VariableDef& from); + + inline VariableDef& operator=(const VariableDef& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + VariableDef(VariableDef&& from) noexcept + : VariableDef() { + *this = ::std::move(from); + } + + inline VariableDef& operator=(VariableDef&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const VariableDef& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const VariableDef* internal_default_instance() { + return reinterpret_cast( + &_VariableDef_default_instance_); + } + static constexpr int kIndexInFileMessages = + 0; + + void UnsafeArenaSwap(VariableDef* other); + void Swap(VariableDef* other); + friend void swap(VariableDef& a, VariableDef& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline VariableDef* New() const final { + return CreateMaybeMessage(NULL); + } + + VariableDef* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const VariableDef& from); + void MergeFrom(const VariableDef& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(VariableDef* other); + protected: + explicit VariableDef(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // string variable_name = 1; + void clear_variable_name(); + static const int kVariableNameFieldNumber = 1; + const ::std::string& variable_name() const; + void set_variable_name(const ::std::string& value); + #if LANG_CXX11 + void set_variable_name(::std::string&& value); + #endif + void set_variable_name(const char* value); + void set_variable_name(const char* value, size_t size); + ::std::string* mutable_variable_name(); + ::std::string* release_variable_name(); + void set_allocated_variable_name(::std::string* variable_name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_variable_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_variable_name( + ::std::string* variable_name); + + // string initializer_name = 2; + void clear_initializer_name(); + static const int kInitializerNameFieldNumber = 2; + const ::std::string& initializer_name() const; + void set_initializer_name(const ::std::string& value); + #if LANG_CXX11 + void set_initializer_name(::std::string&& value); + #endif + void set_initializer_name(const char* value); + void set_initializer_name(const char* value, size_t size); + ::std::string* mutable_initializer_name(); + ::std::string* release_initializer_name(); + void set_allocated_initializer_name(::std::string* initializer_name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_initializer_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_initializer_name( + ::std::string* initializer_name); + + // string snapshot_name = 3; + void clear_snapshot_name(); + static const int kSnapshotNameFieldNumber = 3; + const ::std::string& snapshot_name() const; + void set_snapshot_name(const ::std::string& value); + #if LANG_CXX11 + void set_snapshot_name(::std::string&& value); + #endif + void set_snapshot_name(const char* value); + void set_snapshot_name(const char* value, size_t size); + ::std::string* mutable_snapshot_name(); + ::std::string* release_snapshot_name(); + void set_allocated_snapshot_name(::std::string* snapshot_name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_snapshot_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_snapshot_name( + ::std::string* snapshot_name); + + // string initial_value_name = 6; + void clear_initial_value_name(); + static const int kInitialValueNameFieldNumber = 6; + const ::std::string& initial_value_name() const; + void set_initial_value_name(const ::std::string& value); + #if LANG_CXX11 + void set_initial_value_name(::std::string&& value); + #endif + void set_initial_value_name(const char* value); + void set_initial_value_name(const char* value, size_t size); + ::std::string* mutable_initial_value_name(); + ::std::string* release_initial_value_name(); + void set_allocated_initial_value_name(::std::string* initial_value_name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_initial_value_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_initial_value_name( + ::std::string* initial_value_name); + + // .tensorflow.SaveSliceInfoDef save_slice_info_def = 4; + bool has_save_slice_info_def() const; + void clear_save_slice_info_def(); + static const int kSaveSliceInfoDefFieldNumber = 4; + private: + const ::tensorflow::SaveSliceInfoDef& _internal_save_slice_info_def() const; + public: + const ::tensorflow::SaveSliceInfoDef& save_slice_info_def() const; + ::tensorflow::SaveSliceInfoDef* release_save_slice_info_def(); + ::tensorflow::SaveSliceInfoDef* mutable_save_slice_info_def(); + void set_allocated_save_slice_info_def(::tensorflow::SaveSliceInfoDef* save_slice_info_def); + void unsafe_arena_set_allocated_save_slice_info_def( + ::tensorflow::SaveSliceInfoDef* save_slice_info_def); + ::tensorflow::SaveSliceInfoDef* unsafe_arena_release_save_slice_info_def(); + + // bool is_resource = 5; + void clear_is_resource(); + static const int kIsResourceFieldNumber = 5; + bool is_resource() const; + void set_is_resource(bool value); + + // bool trainable = 7; + void clear_trainable(); + static const int kTrainableFieldNumber = 7; + bool trainable() const; + void set_trainable(bool value); + + // @@protoc_insertion_point(class_scope:tensorflow.VariableDef) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::internal::ArenaStringPtr variable_name_; + ::google::protobuf::internal::ArenaStringPtr initializer_name_; + ::google::protobuf::internal::ArenaStringPtr snapshot_name_; + ::google::protobuf::internal::ArenaStringPtr initial_value_name_; + ::tensorflow::SaveSliceInfoDef* save_slice_info_def_; + bool is_resource_; + bool trainable_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fvariable_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class SaveSliceInfoDef : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.SaveSliceInfoDef) */ { + public: + SaveSliceInfoDef(); + virtual ~SaveSliceInfoDef(); + + SaveSliceInfoDef(const SaveSliceInfoDef& from); + + inline SaveSliceInfoDef& operator=(const SaveSliceInfoDef& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + SaveSliceInfoDef(SaveSliceInfoDef&& from) noexcept + : SaveSliceInfoDef() { + *this = ::std::move(from); + } + + inline SaveSliceInfoDef& operator=(SaveSliceInfoDef&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const SaveSliceInfoDef& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const SaveSliceInfoDef* internal_default_instance() { + return reinterpret_cast( + &_SaveSliceInfoDef_default_instance_); + } + static constexpr int kIndexInFileMessages = + 1; + + void UnsafeArenaSwap(SaveSliceInfoDef* other); + void Swap(SaveSliceInfoDef* other); + friend void swap(SaveSliceInfoDef& a, SaveSliceInfoDef& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline SaveSliceInfoDef* New() const final { + return CreateMaybeMessage(NULL); + } + + SaveSliceInfoDef* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const SaveSliceInfoDef& from); + void MergeFrom(const SaveSliceInfoDef& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(SaveSliceInfoDef* other); + protected: + explicit SaveSliceInfoDef(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated int64 full_shape = 2; + int full_shape_size() const; + void clear_full_shape(); + static const int kFullShapeFieldNumber = 2; + ::google::protobuf::int64 full_shape(int index) const; + void set_full_shape(int index, ::google::protobuf::int64 value); + void add_full_shape(::google::protobuf::int64 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& + full_shape() const; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* + mutable_full_shape(); + + // repeated int64 var_offset = 3; + int var_offset_size() const; + void clear_var_offset(); + static const int kVarOffsetFieldNumber = 3; + ::google::protobuf::int64 var_offset(int index) const; + void set_var_offset(int index, ::google::protobuf::int64 value); + void add_var_offset(::google::protobuf::int64 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& + var_offset() const; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* + mutable_var_offset(); + + // repeated int64 var_shape = 4; + int var_shape_size() const; + void clear_var_shape(); + static const int kVarShapeFieldNumber = 4; + ::google::protobuf::int64 var_shape(int index) const; + void set_var_shape(int index, ::google::protobuf::int64 value); + void add_var_shape(::google::protobuf::int64 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& + var_shape() const; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* + mutable_var_shape(); + + // string full_name = 1; + void clear_full_name(); + static const int kFullNameFieldNumber = 1; + const ::std::string& full_name() const; + void set_full_name(const ::std::string& value); + #if LANG_CXX11 + void set_full_name(::std::string&& value); + #endif + void set_full_name(const char* value); + void set_full_name(const char* value, size_t size); + ::std::string* mutable_full_name(); + ::std::string* release_full_name(); + void set_allocated_full_name(::std::string* full_name); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_full_name(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_full_name( + ::std::string* full_name); + + // @@protoc_insertion_point(class_scope:tensorflow.SaveSliceInfoDef) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 > full_shape_; + mutable int _full_shape_cached_byte_size_; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 > var_offset_; + mutable int _var_offset_cached_byte_size_; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 > var_shape_; + mutable int _var_shape_cached_byte_size_; + ::google::protobuf::internal::ArenaStringPtr full_name_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fvariable_2eproto::TableStruct; +}; +// =================================================================== + + +// =================================================================== + +#ifdef __GNUC__ + #pragma GCC diagnostic push + #pragma GCC diagnostic ignored "-Wstrict-aliasing" +#endif // __GNUC__ +// VariableDef + +// string variable_name = 1; +inline void VariableDef::clear_variable_name() { + variable_name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& VariableDef::variable_name() const { + // @@protoc_insertion_point(field_get:tensorflow.VariableDef.variable_name) + return variable_name_.Get(); +} +inline void VariableDef::set_variable_name(const ::std::string& value) { + + variable_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.VariableDef.variable_name) +} +#if LANG_CXX11 +inline void VariableDef::set_variable_name(::std::string&& value) { + + variable_name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.VariableDef.variable_name) +} +#endif +inline void VariableDef::set_variable_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + variable_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.VariableDef.variable_name) +} +inline void VariableDef::set_variable_name(const char* value, + size_t size) { + + variable_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.VariableDef.variable_name) +} +inline ::std::string* VariableDef::mutable_variable_name() { + + // @@protoc_insertion_point(field_mutable:tensorflow.VariableDef.variable_name) + return variable_name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* VariableDef::release_variable_name() { + // @@protoc_insertion_point(field_release:tensorflow.VariableDef.variable_name) + + return variable_name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void VariableDef::set_allocated_variable_name(::std::string* variable_name) { + if (variable_name != NULL) { + + } else { + + } + variable_name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), variable_name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.VariableDef.variable_name) +} +inline ::std::string* VariableDef::unsafe_arena_release_variable_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.VariableDef.variable_name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return variable_name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void VariableDef::unsafe_arena_set_allocated_variable_name( + ::std::string* variable_name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (variable_name != NULL) { + + } else { + + } + variable_name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + variable_name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.VariableDef.variable_name) +} + +// string initial_value_name = 6; +inline void VariableDef::clear_initial_value_name() { + initial_value_name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& VariableDef::initial_value_name() const { + // @@protoc_insertion_point(field_get:tensorflow.VariableDef.initial_value_name) + return initial_value_name_.Get(); +} +inline void VariableDef::set_initial_value_name(const ::std::string& value) { + + initial_value_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.VariableDef.initial_value_name) +} +#if LANG_CXX11 +inline void VariableDef::set_initial_value_name(::std::string&& value) { + + initial_value_name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.VariableDef.initial_value_name) +} +#endif +inline void VariableDef::set_initial_value_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + initial_value_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.VariableDef.initial_value_name) +} +inline void VariableDef::set_initial_value_name(const char* value, + size_t size) { + + initial_value_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.VariableDef.initial_value_name) +} +inline ::std::string* VariableDef::mutable_initial_value_name() { + + // @@protoc_insertion_point(field_mutable:tensorflow.VariableDef.initial_value_name) + return initial_value_name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* VariableDef::release_initial_value_name() { + // @@protoc_insertion_point(field_release:tensorflow.VariableDef.initial_value_name) + + return initial_value_name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void VariableDef::set_allocated_initial_value_name(::std::string* initial_value_name) { + if (initial_value_name != NULL) { + + } else { + + } + initial_value_name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), initial_value_name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.VariableDef.initial_value_name) +} +inline ::std::string* VariableDef::unsafe_arena_release_initial_value_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.VariableDef.initial_value_name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return initial_value_name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void VariableDef::unsafe_arena_set_allocated_initial_value_name( + ::std::string* initial_value_name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (initial_value_name != NULL) { + + } else { + + } + initial_value_name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + initial_value_name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.VariableDef.initial_value_name) +} + +// string initializer_name = 2; +inline void VariableDef::clear_initializer_name() { + initializer_name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& VariableDef::initializer_name() const { + // @@protoc_insertion_point(field_get:tensorflow.VariableDef.initializer_name) + return initializer_name_.Get(); +} +inline void VariableDef::set_initializer_name(const ::std::string& value) { + + initializer_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.VariableDef.initializer_name) +} +#if LANG_CXX11 +inline void VariableDef::set_initializer_name(::std::string&& value) { + + initializer_name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.VariableDef.initializer_name) +} +#endif +inline void VariableDef::set_initializer_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + initializer_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.VariableDef.initializer_name) +} +inline void VariableDef::set_initializer_name(const char* value, + size_t size) { + + initializer_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.VariableDef.initializer_name) +} +inline ::std::string* VariableDef::mutable_initializer_name() { + + // @@protoc_insertion_point(field_mutable:tensorflow.VariableDef.initializer_name) + return initializer_name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* VariableDef::release_initializer_name() { + // @@protoc_insertion_point(field_release:tensorflow.VariableDef.initializer_name) + + return initializer_name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void VariableDef::set_allocated_initializer_name(::std::string* initializer_name) { + if (initializer_name != NULL) { + + } else { + + } + initializer_name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), initializer_name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.VariableDef.initializer_name) +} +inline ::std::string* VariableDef::unsafe_arena_release_initializer_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.VariableDef.initializer_name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return initializer_name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void VariableDef::unsafe_arena_set_allocated_initializer_name( + ::std::string* initializer_name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (initializer_name != NULL) { + + } else { + + } + initializer_name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + initializer_name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.VariableDef.initializer_name) +} + +// string snapshot_name = 3; +inline void VariableDef::clear_snapshot_name() { + snapshot_name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& VariableDef::snapshot_name() const { + // @@protoc_insertion_point(field_get:tensorflow.VariableDef.snapshot_name) + return snapshot_name_.Get(); +} +inline void VariableDef::set_snapshot_name(const ::std::string& value) { + + snapshot_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.VariableDef.snapshot_name) +} +#if LANG_CXX11 +inline void VariableDef::set_snapshot_name(::std::string&& value) { + + snapshot_name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.VariableDef.snapshot_name) +} +#endif +inline void VariableDef::set_snapshot_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + snapshot_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.VariableDef.snapshot_name) +} +inline void VariableDef::set_snapshot_name(const char* value, + size_t size) { + + snapshot_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.VariableDef.snapshot_name) +} +inline ::std::string* VariableDef::mutable_snapshot_name() { + + // @@protoc_insertion_point(field_mutable:tensorflow.VariableDef.snapshot_name) + return snapshot_name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* VariableDef::release_snapshot_name() { + // @@protoc_insertion_point(field_release:tensorflow.VariableDef.snapshot_name) + + return snapshot_name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void VariableDef::set_allocated_snapshot_name(::std::string* snapshot_name) { + if (snapshot_name != NULL) { + + } else { + + } + snapshot_name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), snapshot_name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.VariableDef.snapshot_name) +} +inline ::std::string* VariableDef::unsafe_arena_release_snapshot_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.VariableDef.snapshot_name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return snapshot_name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void VariableDef::unsafe_arena_set_allocated_snapshot_name( + ::std::string* snapshot_name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (snapshot_name != NULL) { + + } else { + + } + snapshot_name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + snapshot_name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.VariableDef.snapshot_name) +} + +// .tensorflow.SaveSliceInfoDef save_slice_info_def = 4; +inline bool VariableDef::has_save_slice_info_def() const { + return this != internal_default_instance() && save_slice_info_def_ != NULL; +} +inline void VariableDef::clear_save_slice_info_def() { + if (GetArenaNoVirtual() == NULL && save_slice_info_def_ != NULL) { + delete save_slice_info_def_; + } + save_slice_info_def_ = NULL; +} +inline const ::tensorflow::SaveSliceInfoDef& VariableDef::_internal_save_slice_info_def() const { + return *save_slice_info_def_; +} +inline const ::tensorflow::SaveSliceInfoDef& VariableDef::save_slice_info_def() const { + const ::tensorflow::SaveSliceInfoDef* p = save_slice_info_def_; + // @@protoc_insertion_point(field_get:tensorflow.VariableDef.save_slice_info_def) + return p != NULL ? *p : *reinterpret_cast( + &::tensorflow::_SaveSliceInfoDef_default_instance_); +} +inline ::tensorflow::SaveSliceInfoDef* VariableDef::release_save_slice_info_def() { + // @@protoc_insertion_point(field_release:tensorflow.VariableDef.save_slice_info_def) + + ::tensorflow::SaveSliceInfoDef* temp = save_slice_info_def_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + save_slice_info_def_ = NULL; + return temp; +} +inline ::tensorflow::SaveSliceInfoDef* VariableDef::unsafe_arena_release_save_slice_info_def() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.VariableDef.save_slice_info_def) + + ::tensorflow::SaveSliceInfoDef* temp = save_slice_info_def_; + save_slice_info_def_ = NULL; + return temp; +} +inline ::tensorflow::SaveSliceInfoDef* VariableDef::mutable_save_slice_info_def() { + + if (save_slice_info_def_ == NULL) { + auto* p = CreateMaybeMessage<::tensorflow::SaveSliceInfoDef>(GetArenaNoVirtual()); + save_slice_info_def_ = p; + } + // @@protoc_insertion_point(field_mutable:tensorflow.VariableDef.save_slice_info_def) + return save_slice_info_def_; +} +inline void VariableDef::set_allocated_save_slice_info_def(::tensorflow::SaveSliceInfoDef* save_slice_info_def) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete save_slice_info_def_; + } + if (save_slice_info_def) { + ::google::protobuf::Arena* submessage_arena = + ::google::protobuf::Arena::GetArena(save_slice_info_def); + if (message_arena != submessage_arena) { + save_slice_info_def = ::google::protobuf::internal::GetOwnedMessage( + message_arena, save_slice_info_def, submessage_arena); + } + + } else { + + } + save_slice_info_def_ = save_slice_info_def; + // @@protoc_insertion_point(field_set_allocated:tensorflow.VariableDef.save_slice_info_def) +} + +// bool is_resource = 5; +inline void VariableDef::clear_is_resource() { + is_resource_ = false; +} +inline bool VariableDef::is_resource() const { + // @@protoc_insertion_point(field_get:tensorflow.VariableDef.is_resource) + return is_resource_; +} +inline void VariableDef::set_is_resource(bool value) { + + is_resource_ = value; + // @@protoc_insertion_point(field_set:tensorflow.VariableDef.is_resource) +} + +// bool trainable = 7; +inline void VariableDef::clear_trainable() { + trainable_ = false; +} +inline bool VariableDef::trainable() const { + // @@protoc_insertion_point(field_get:tensorflow.VariableDef.trainable) + return trainable_; +} +inline void VariableDef::set_trainable(bool value) { + + trainable_ = value; + // @@protoc_insertion_point(field_set:tensorflow.VariableDef.trainable) +} + +// ------------------------------------------------------------------- + +// SaveSliceInfoDef + +// string full_name = 1; +inline void SaveSliceInfoDef::clear_full_name() { + full_name_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& SaveSliceInfoDef::full_name() const { + // @@protoc_insertion_point(field_get:tensorflow.SaveSliceInfoDef.full_name) + return full_name_.Get(); +} +inline void SaveSliceInfoDef::set_full_name(const ::std::string& value) { + + full_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.SaveSliceInfoDef.full_name) +} +#if LANG_CXX11 +inline void SaveSliceInfoDef::set_full_name(::std::string&& value) { + + full_name_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.SaveSliceInfoDef.full_name) +} +#endif +inline void SaveSliceInfoDef::set_full_name(const char* value) { + GOOGLE_DCHECK(value != NULL); + + full_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.SaveSliceInfoDef.full_name) +} +inline void SaveSliceInfoDef::set_full_name(const char* value, + size_t size) { + + full_name_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.SaveSliceInfoDef.full_name) +} +inline ::std::string* SaveSliceInfoDef::mutable_full_name() { + + // @@protoc_insertion_point(field_mutable:tensorflow.SaveSliceInfoDef.full_name) + return full_name_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* SaveSliceInfoDef::release_full_name() { + // @@protoc_insertion_point(field_release:tensorflow.SaveSliceInfoDef.full_name) + + return full_name_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void SaveSliceInfoDef::set_allocated_full_name(::std::string* full_name) { + if (full_name != NULL) { + + } else { + + } + full_name_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), full_name, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.SaveSliceInfoDef.full_name) +} +inline ::std::string* SaveSliceInfoDef::unsafe_arena_release_full_name() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.SaveSliceInfoDef.full_name) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return full_name_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void SaveSliceInfoDef::unsafe_arena_set_allocated_full_name( + ::std::string* full_name) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (full_name != NULL) { + + } else { + + } + full_name_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + full_name, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.SaveSliceInfoDef.full_name) +} + +// repeated int64 full_shape = 2; +inline int SaveSliceInfoDef::full_shape_size() const { + return full_shape_.size(); +} +inline void SaveSliceInfoDef::clear_full_shape() { + full_shape_.Clear(); +} +inline ::google::protobuf::int64 SaveSliceInfoDef::full_shape(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.SaveSliceInfoDef.full_shape) + return full_shape_.Get(index); +} +inline void SaveSliceInfoDef::set_full_shape(int index, ::google::protobuf::int64 value) { + full_shape_.Set(index, value); + // @@protoc_insertion_point(field_set:tensorflow.SaveSliceInfoDef.full_shape) +} +inline void SaveSliceInfoDef::add_full_shape(::google::protobuf::int64 value) { + full_shape_.Add(value); + // @@protoc_insertion_point(field_add:tensorflow.SaveSliceInfoDef.full_shape) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& +SaveSliceInfoDef::full_shape() const { + // @@protoc_insertion_point(field_list:tensorflow.SaveSliceInfoDef.full_shape) + return full_shape_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* +SaveSliceInfoDef::mutable_full_shape() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.SaveSliceInfoDef.full_shape) + return &full_shape_; +} + +// repeated int64 var_offset = 3; +inline int SaveSliceInfoDef::var_offset_size() const { + return var_offset_.size(); +} +inline void SaveSliceInfoDef::clear_var_offset() { + var_offset_.Clear(); +} +inline ::google::protobuf::int64 SaveSliceInfoDef::var_offset(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.SaveSliceInfoDef.var_offset) + return var_offset_.Get(index); +} +inline void SaveSliceInfoDef::set_var_offset(int index, ::google::protobuf::int64 value) { + var_offset_.Set(index, value); + // @@protoc_insertion_point(field_set:tensorflow.SaveSliceInfoDef.var_offset) +} +inline void SaveSliceInfoDef::add_var_offset(::google::protobuf::int64 value) { + var_offset_.Add(value); + // @@protoc_insertion_point(field_add:tensorflow.SaveSliceInfoDef.var_offset) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& +SaveSliceInfoDef::var_offset() const { + // @@protoc_insertion_point(field_list:tensorflow.SaveSliceInfoDef.var_offset) + return var_offset_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* +SaveSliceInfoDef::mutable_var_offset() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.SaveSliceInfoDef.var_offset) + return &var_offset_; +} + +// repeated int64 var_shape = 4; +inline int SaveSliceInfoDef::var_shape_size() const { + return var_shape_.size(); +} +inline void SaveSliceInfoDef::clear_var_shape() { + var_shape_.Clear(); +} +inline ::google::protobuf::int64 SaveSliceInfoDef::var_shape(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.SaveSliceInfoDef.var_shape) + return var_shape_.Get(index); +} +inline void SaveSliceInfoDef::set_var_shape(int index, ::google::protobuf::int64 value) { + var_shape_.Set(index, value); + // @@protoc_insertion_point(field_set:tensorflow.SaveSliceInfoDef.var_shape) +} +inline void SaveSliceInfoDef::add_var_shape(::google::protobuf::int64 value) { + var_shape_.Add(value); + // @@protoc_insertion_point(field_add:tensorflow.SaveSliceInfoDef.var_shape) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& +SaveSliceInfoDef::var_shape() const { + // @@protoc_insertion_point(field_list:tensorflow.SaveSliceInfoDef.var_shape) + return var_shape_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* +SaveSliceInfoDef::mutable_var_shape() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.SaveSliceInfoDef.var_shape) + return &var_shape_; +} + +#ifdef __GNUC__ + #pragma GCC diagnostic pop +#endif // __GNUC__ +// ------------------------------------------------------------------- + + +// @@protoc_insertion_point(namespace_scope) + +} // namespace tensorflow + +// @@protoc_insertion_point(global_scope) + +#endif // PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2fvariable_2eproto diff --git a/linux/include/tensorflow/core/framework/variable.pb_text-impl.h b/linux/include/tensorflow/core/framework/variable.pb_text-impl.h new file mode 100644 index 000000000..1220ef19f --- /dev/null +++ b/linux/include/tensorflow/core/framework/variable.pb_text-impl.h @@ -0,0 +1,32 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_framework_variable_proto_IMPL_H_ +#define tensorflow_core_framework_variable_proto_IMPL_H_ + +#include "tensorflow/core/framework/variable.pb.h" +#include "tensorflow/core/framework/variable.pb_text.h" +#include "tensorflow/core/lib/strings/proto_text_util.h" +#include "tensorflow/core/lib/strings/scanner.h" + +namespace tensorflow { + +namespace internal { + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::VariableDef& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::VariableDef* msg); + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::SaveSliceInfoDef& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::SaveSliceInfoDef* msg); + +} // namespace internal + +} // namespace tensorflow + +#endif // tensorflow_core_framework_variable_proto_IMPL_H_ diff --git a/linux/include/tensorflow/core/framework/variable.pb_text.h b/linux/include/tensorflow/core/framework/variable.pb_text.h new file mode 100644 index 000000000..f70e2c990 --- /dev/null +++ b/linux/include/tensorflow/core/framework/variable.pb_text.h @@ -0,0 +1,34 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_framework_variable_proto_H_ +#define tensorflow_core_framework_variable_proto_H_ + +#include "tensorflow/core/framework/variable.pb.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/protobuf.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +// Message-text conversion for tensorflow.VariableDef +string ProtoDebugString( + const ::tensorflow::VariableDef& msg); +string ProtoShortDebugString( + const ::tensorflow::VariableDef& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::VariableDef* msg) + TF_MUST_USE_RESULT; + +// Message-text conversion for tensorflow.SaveSliceInfoDef +string ProtoDebugString( + const ::tensorflow::SaveSliceInfoDef& msg); +string ProtoShortDebugString( + const ::tensorflow::SaveSliceInfoDef& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::SaveSliceInfoDef* msg) + TF_MUST_USE_RESULT; + +} // namespace tensorflow + +#endif // tensorflow_core_framework_variable_proto_H_ diff --git a/linux/include/tensorflow/core/framework/variant.h b/linux/include/tensorflow/core/framework/variant.h new file mode 100644 index 000000000..10eabbc85 --- /dev/null +++ b/linux/include/tensorflow/core/framework/variant.h @@ -0,0 +1,326 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_FRAMEWORK_VARIANT_H_ +#define TENSORFLOW_CORE_FRAMEWORK_VARIANT_H_ + +#include +#include +#include +#include +#include +#include + +#include "tensorflow/core/framework/type_index.h" +#include "tensorflow/core/framework/variant_tensor_data.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/lib/strings/strcat.h" +#include "tensorflow/core/platform/mutex.h" + +namespace tensorflow { + +template +string TypeNameVariant(const T& value); + +template +string DebugStringVariant(const T& value); + +// Allows for specializations of Variant Decoding. `data` may be modified in +// the process of decoding to `value`. +template +bool DecodeVariant(VariantTensorData* data, T* value); + +template +bool DecodeVariant(string* buf, T* value); + +template +void EncodeVariant(const T& value, VariantTensorData* data); + +template +void EncodeVariant(const T& value, string* buf); + +// This is an implementation of a type-erased container that can store an +// object of any type. The implementation is very similar to std::any, but has +// restrictions on the types of objects that can be stored, and eschews some of +// the fancier constructors available for std::any. An object of +// tensorflow::Variant is intended to be used as the value that will be stored +// in a tensorflow::Tensor object when its type is DT_VARIANT. +// +// tensorflow::Variant can store an object of a class that satisfies the +// following constraints: +// +// * The class is CopyConstructible. +// * The class has a default constructor. +// * It's either a protocol buffer, a tensorflow::Tensor, or defines the +// following functions: +// +// string TypeName() const; +// void Encode(VariantTensorData* data) const; +// void Decode(VariantTensorData data); +// +// Simple POD types can elide the Encode/Decode functions, they are provided by +// helper methods. +// Here are some typical usage patterns: +// +// Variant x = 10; +// EXPECT_EQ(*x.get(), 10); +// +// Tensor t(DT_FLOAT, TensorShape({})); +// t.flat()(0) = 42.0f; +// Variant x = t; +// EXPECT_EQ(x.get()->flat()(0), 42.0f); +// +// Accessing the stored object: +// +// The get function is the main mechanism to access the object +// stored in the container. It is type-safe, that is, calling +// get when the stored object's type is not T, returns a +// nullptr. A raw pointer to the stored object can be obtained by calling +// get(). +// +// Serializing/deserializing Variant object: +// +// The Variant class delegates serializing and deserializing operations to the +// contained object. Helper functions to do these operations are provided for +// POD data types, tensorflow::Tensor, and protocol buffer objects. However, +// other classes have to provide Encode/Decode functions to handle +// serialization. +// +// Objects stored in a Variant object often contain references to other +// tensorflow::Tensors of primitive types (Eg., a list of tensorflow::Tensors). +// To efficiently support those use cases, a structure is imposed on the +// serialization format. Namely, classes should serialize their contents into a +// VariantTensorData object: +// +// struct VariantTensorData { +// string type_name; +// string metadata; +// std::vector tensors; +// }; +// +// Objects with references to other Tensors can simply store those tensors in +// the `tensors` field, and serialize other metadata content in to the +// `metadata` field. +// +// Serialization example: +// +// Foo f = Foo {...}; +// Variant x = f; +// string serialized_f; +// x.Encode(&serialized_f); +// +// Variant y = Foo(); // default constructed Foo. +// y.Decode(std::move(serialized_f)); +// EXPECT_EQ(*x.get(), *y.get()); +// +// +// A Variant storing serialized Variant data (a value of type +// VariantTensorDataProto) has different behavior from a standard Variant. +// Namely, its TypeName matches the TypeName of the original Variant; +// and its non-const get method performs lazy deserialization. +// +// Decode and copy example: +// +// Foo f = Foo {...}; +// Variant x = f; +// +// VariantTensorData serialized_data_f; +// VariantTensorDataProto serialized_proto_f; +// x.Encode(&serialized_data_f); +// serialized_data_f.ToProto(&serialized_proto_f); +// +// Variant y_type_unknown = serialized_proto_f; // Store serialized Variant. +// +// EXPECT_EQ(x.TypeName(), y_type_unknown.TypeName()); // Looks like Foo. +// EXPECT_EQ(MakeTypeIndex(), +// y_type_unknown.TypeId()); +// +class Variant { + public: + constexpr Variant() noexcept = default; + + Variant(const Variant& other) + : value_(other.is_empty() ? std::unique_ptr() + : other.value_->Clone()) {} + + Variant(Variant&& other) noexcept = default; + + // Make sure that the type is CopyConstructible and not a tensorflow::Variant + // object itself. We want the copy constructor to be chosen for the + // tensorflow::Variant case. + template ::type, + typename std::enable_if::value && + std::is_copy_constructible::value, + void>::type* = nullptr> + Variant(T&& value) // NOLINT + : value_(new Value(in_place, std::forward(value))) {} + + Variant& operator=(const Variant& rhs) { + Variant(rhs).swap(*this); + return *this; + } + + Variant& operator=(Variant&& rhs) noexcept { + Variant(std::move(rhs)).swap(*this); + return *this; + } + + bool is_empty() const { return value_ == nullptr; } + + void clear() noexcept { value_.reset(); } + + void swap(Variant& other) noexcept { value_.swap(other.value_); } + + // Note, unlike TypeName(), TypeId() does not return the TypeIndex + // of the original type when a TensorValueDataProto is stored as the + // value. In this case, it returns the TypeIndex of TensorValueDataProto. + TypeIndex TypeId() const { + const TypeIndex VoidTypeIndex = MakeTypeIndex(); + if (is_empty()) { + return VoidTypeIndex; + } + return value_->TypeId(); + } + + string DebugString() const { + return strings::StrCat("VariantDebugString(), ">"); + } + + // Returns a pointer to the stored value if it is type T, or nullptr + // otherwise. + template + T* get() { + const TypeIndex TTypeIndex = MakeTypeIndex(); + if (is_empty() || (TTypeIndex != TypeId())) return nullptr; + return std::addressof(static_cast*>(value_.get())->value); + } + + // Returns a pointer to the stored value if it is type T, or nullptr + // otherwise. + template + const T* get() const { + const TypeIndex TTypeIndex = MakeTypeIndex(); + if (is_empty() || (TTypeIndex != TypeId())) return nullptr; + return std::addressof( + static_cast*>(value_.get())->value); + } + + // Returns TypeNameVariant(value). + // + // In the special case that a serialized Variant is stored (value + // is a VariantTensorDataProto), returns value.TypeName(), the + // TypeName field stored in the VariantTensorDataProto buffer. + string TypeName() const { + if (is_empty()) { + return ""; + } + return value_->TypeName(); + } + + // Serialize the contents of the stored object into `data`. + void Encode(VariantTensorData* data) const { + if (!is_empty()) { + value_->Encode(data); + } + } + + // Deserialize `data` and update the stored object. + bool Decode(VariantTensorData data); + + // Helper methods to directly serialize/deserialize from strings. + void Encode(string* buf) const { + if (!is_empty()) { + value_->Encode(buf); + } + } + bool Decode(string buf) { + if (!is_empty()) { + return value_->Decode(std::move(buf)); + } + return true; + } + + private: + struct in_place_t {}; + static constexpr in_place_t in_place{}; + + struct ValueInterface { + virtual ~ValueInterface() = default; + virtual TypeIndex TypeId() const = 0; + virtual void* RawPtr() = 0; + virtual const void* RawPtr() const = 0; + virtual std::unique_ptr Clone() const = 0; + virtual string TypeName() const = 0; + virtual string DebugString() const = 0; + virtual void Encode(VariantTensorData* data) const = 0; + virtual bool Decode(VariantTensorData data) = 0; + virtual void Encode(string* buf) const = 0; + virtual bool Decode(string data) = 0; + }; + + template + struct Value : ValueInterface { + template + explicit Value(in_place_t /*tag*/, Args&&... args) + : value(std::forward(args)...) {} + + TypeIndex TypeId() const override { + const TypeIndex value_type_index = + MakeTypeIndex::type>(); + return value_type_index; + } + + void* RawPtr() override { return &value; } + + const void* RawPtr() const override { return &value; } + + std::unique_ptr Clone() const override { + return std::unique_ptr(new Value(in_place, value)); + } + + string TypeName() const override { return TypeNameVariant(value); } + + string DebugString() const override { return DebugStringVariant(value); } + + void Encode(VariantTensorData* data) const override { + EncodeVariant(value, data); + } + + bool Decode(VariantTensorData data) override { + return DecodeVariant(&data, &value); + } + + void Encode(string* buf) const override { EncodeVariant(value, buf); } + + bool Decode(string buf) override { return DecodeVariant(&buf, &value); } + + T value; + }; + + // value_ can point to any type T as wrapped by a ValueInterface. + // The only real requirement is that T is default-constructible. + std::unique_ptr value_; +}; + +template <> +void* Variant::get(); + +template <> +const void* Variant::get() const; + +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_FRAMEWORK_VARIANT_H_ diff --git a/linux/include/tensorflow/core/framework/variant_encode_decode.h b/linux/include/tensorflow/core/framework/variant_encode_decode.h new file mode 100644 index 000000000..5e08e5a7a --- /dev/null +++ b/linux/include/tensorflow/core/framework/variant_encode_decode.h @@ -0,0 +1,280 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_FRAMEWORK_VARIANT_ENCODE_DECODE_H_ +#define TENSORFLOW_CORE_FRAMEWORK_VARIANT_ENCODE_DECODE_H_ + +#include +#include +#include +#include + +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/type_index.h" +#include "tensorflow/core/framework/variant_tensor_data.h" +#include "tensorflow/core/lib/strings/strcat.h" +#include "tensorflow/core/platform/abi.h" +#include "tensorflow/core/platform/protobuf.h" + +namespace tensorflow { + +// Type used for tag-dispatch of the Encode/Decode Variant implementations. This +// template can determine whether the first type parameter `T` is one of the +// following: +// +// * A POD type (TypeResolver) +// * A tensorflow::Tensor (TypeResolver) +// * A protocol buffer (TypeResolver) +// * None of the above (TypeResolver) +// +template ::type>::value, + bool = std::is_same::type, + ::tensorflow::Tensor>::value, + bool = std::is_base_of::type>::value> +struct TypeResolver {}; + +// Specialization for POD type +template +void EncodeVariantImpl(const T& value, TypeResolver, + VariantTensorData* data) { + data->set_metadata(value); +} + +// Specialization for tensorflow::Tensor +template +void EncodeVariantImpl(const T& value, + TypeResolver, + VariantTensorData* data) { + data->tensors_.clear(); + data->tensors_.push_back(value); +} + +// Specialization for protobuf +template +void EncodeVariantImpl(const T& value, + TypeResolver, + VariantTensorData* data) { + value.SerializeToString(&data->metadata_); +} + +// Specialization for other types +template +void EncodeVariantImpl(const T& value, + TypeResolver, + VariantTensorData* data) { + value.Encode(data); +} + +// Specialization for POD type +template +bool DecodeVariantImpl(VariantTensorData data, + TypeResolver, + T* value) { + return data.get_metadata(value); +} + +// Specialization for tensorflow::Tensor +template +bool DecodeVariantImpl(VariantTensorData data, + TypeResolver, + T* value) { + *value = data.tensors(0); + return true; +} + +// Specialization for protobuf +template +bool DecodeVariantImpl(VariantTensorData data, + TypeResolver, + T* value) { + string metadata; + data.get_metadata(&metadata); + return value->ParseFromString(std::move(metadata)); +} + +// Specialization for other types +template +bool DecodeVariantImpl(VariantTensorData data, + TypeResolver, + T* value) { + return value->Decode(std::move(data)); +} + +template +struct has_type_name : std::false_type {}; + +template +struct has_type_name< + C, typename std::enable_if().TypeName()), string>::value>::type> + : std::true_type {}; + +template ::type>::value, + bool = std::is_same::type, + ::tensorflow::Tensor>::value, + bool = std::is_base_of::type>::value> +struct TypeNameResolver {}; + +template +string TypeNameVariantImpl(const T& value, + TypeNameResolver) { + return value.TypeName(); +} + +template +string TypeNameVariantImpl( + const T& value, + TypeNameResolver) { + return "tensorflow::Tensor"; +} + +template +string TypeNameVariantImpl( + const T& value, TypeNameResolver) { + return value.GetTypeName(); +} + +template +string TypeNameVariantImpl( + const T& value, + TypeNameResolver) { + return port::MaybeAbiDemangle(MakeTypeIndex().name()); +} + +template +string TypeNameVariant(const T& value) { + return TypeNameVariantImpl(value, TypeNameResolver()); +} + +template +struct has_debug_string : std::false_type {}; + +template +struct has_debug_string< + C, typename std::enable_if().DebugString()), string>::value>::type> + : std::true_type {}; + +template +struct can_strcat : std::false_type {}; + +template +struct can_strcat< + C, typename std::enable_if())), string>::value>::type> + : std::true_type {}; + +template ::type>::value, + bool = can_strcat::type>::value> +struct DebugStringResolver {}; + +// TODO(ebrevdo): Expand DebugStringResolver to return TypeString if +// there is no StrCat() constructor. +template +string DebugStringVariantImpl( + const T& value, DebugStringResolver) { + return value.DebugString(); +} + +template +string DebugStringVariantImpl( + const T& value, DebugStringResolver) { + return strings::StrCat(value); +} + +template +string DebugStringVariantImpl( + const T& value, DebugStringResolver) { + return "?"; +} + +template +string DebugStringVariant(const T& value) { + return DebugStringVariantImpl(value, DebugStringResolver()); +} + +template +void EncodeVariant(const T& value, VariantTensorData* data) { + EncodeVariantImpl(value, TypeResolver(), data); + data->set_type_name(TypeNameVariant(value)); +} + +template +bool DecodeVariant(VariantTensorData* data, T* value) { + return DecodeVariantImpl(std::move(*data), TypeResolver(), value); +} + +template +void EncodeVariant(const T& value, string* buf) { + VariantTensorData data; + EncodeVariantImpl(value, TypeResolver(), &data); + data.set_type_name(TypeNameVariant(value)); + DCHECK(buf != nullptr); + data.SerializeToString(buf); +} + +template +bool DecodeVariant(string* buf, T* value) { + VariantTensorData data; + if (!data.ParseFromString(*buf)) return false; + if (!DecodeVariantImpl(std::move(data), TypeResolver(), value)) { + return false; + } + return true; +} + +// Specializations for VariantTensorDataProto +template <> +string TypeNameVariant(const VariantTensorDataProto& value); + +template <> +void EncodeVariant(const VariantTensorDataProto& value, + VariantTensorData* data); + +template <> +bool DecodeVariant(VariantTensorData* data, VariantTensorDataProto* value); + +template <> +void EncodeVariant(const VariantTensorDataProto& value, string* buf); + +template <> +bool DecodeVariant(string* buf, VariantTensorDataProto* value); + +// Encodes an array of Variant objects in to the given StringListEncoder. +// `variant_array` is assumed to point to an array of `n` Variant objects. +void EncodeVariantList(const Variant* variant_array, int64 n, + std::unique_ptr e); + +// Decodes an array of Variant objects from the given StringListDecoder. +// `variant_array` is assumed to point to an array of `n` Variant objects. +bool DecodeVariantList(std::unique_ptr d, + Variant* variant_array, int64 n); + +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_FRAMEWORK_VARIANT_ENCODE_DECODE_H_ diff --git a/linux/include/tensorflow/core/framework/variant_op_registry.h b/linux/include/tensorflow/core/framework/variant_op_registry.h new file mode 100644 index 000000000..7eb37e859 --- /dev/null +++ b/linux/include/tensorflow/core/framework/variant_op_registry.h @@ -0,0 +1,597 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_FRAMEWORK_VARIANT_OP_REGISTRY_H_ +#define TENSORFLOW_CORE_FRAMEWORK_VARIANT_OP_REGISTRY_H_ + +#include +#include +#include + +#define EIGEN_USE_THREADS + +#include "tensorflow/core/framework/tensor.pb.h" +#include "tensorflow/core/framework/type_index.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/framework/variant.h" +#include "tensorflow/core/framework/variant_encode_decode.h" +#include "tensorflow/core/lib/gtl/flatmap.h" +#include "tensorflow/core/lib/hash/hash.h" +#include "tensorflow/core/platform/abi.h" + +namespace tensorflow { + +class OpKernelContext; +// A global UnaryVariantOpRegistry is used to hold callback functions +// for different variant types. To be used by ShapeOp, RankOp, and +// SizeOp, decoding, etc. + +enum VariantUnaryOp { + INVALID_VARIANT_UNARY_OP = 0, + ZEROS_LIKE_VARIANT_UNARY_OP = 1, + CONJ_VARIANT_UNARY_OP = 2, +}; + +enum VariantBinaryOp { + INVALID_VARIANT_BINARY_OP = 0, + ADD_VARIANT_BINARY_OP = 1, +}; + +enum VariantDeviceCopyDirection { + INVALID_DEVICE_COPY_DIRECTION = 0, + HOST_TO_DEVICE = 1, + DEVICE_TO_HOST = 2, + DEVICE_TO_DEVICE = 3, +}; + +class UnaryVariantOpRegistry { + public: + typedef std::function VariantShapeFn; + typedef std::function VariantDecodeFn; + typedef std::function + VariantUnaryOpFn; + typedef std::function + VariantBinaryOpFn; + + // An AsyncTensorDeviceCopyFn is a function provided to + // the user-provided DeviceCopyFn callback as the third argument ("copier"). + // + // Expected inputs: + // from: A Tensor on the host (if performing cpu->gpu copy), or + // device (if performing gpu->cpu or gpu->gpu copy). + // to: An empty/uninitialized tensor. It will be updated upon + // successful return of the function with the correct dtype and shape. + // However, the copied data will not be available until the compute + // stream has been synchronized. + // + // Returns: + // The status upon memory allocation / initialization of the + // "to" tensor, and enqueue of the copy onto the compute stream. + // Any failure of the copy itself will update the underlying + // stream status and propagate through the runtime independent + // of the caller. + typedef std::function + AsyncTensorDeviceCopyFn; + + // The AsyncVariantDeviceCopyFn is the signature of the 'device_copy_fn' + // expected to be passed to the registration macro + // INTERNAL_REGISTER_UNARY_VARIANT_DEVICE_COPY_FUNCTION. + typedef std::function + AsyncVariantDeviceCopyFn; + + // Add a shape lookup function to the registry. + void RegisterShapeFn(const TypeIndex& type_index, + const VariantShapeFn& shape_fn); + + // Returns nullptr if no shape function was found for the given TypeIndex. + VariantShapeFn* GetShapeFn(const TypeIndex& type_index); + + // Add a decode function to the registry. + void RegisterDecodeFn(const string& type_name, + const VariantDecodeFn& decode_fn); + + // Returns nullptr if no decode function was found for the given TypeName. + VariantDecodeFn* GetDecodeFn(StringPiece type_name); + + // Add a copy-to-GPU function to the registry. + void RegisterDeviceCopyFn(const VariantDeviceCopyDirection direction, + const TypeIndex& type_index, + const AsyncVariantDeviceCopyFn& device_copy_fn); + + // Returns nullptr if no copy function was found for the given + // TypeName and direction. + AsyncVariantDeviceCopyFn* GetDeviceCopyFn( + const VariantDeviceCopyDirection direction, const TypeIndex& type_index); + + // Add a unary op function to the registry. + void RegisterUnaryOpFn(VariantUnaryOp op, const string& device, + const TypeIndex& type_index, + const VariantUnaryOpFn& unary_op_fn); + + // Returns nullptr if no unary op function was found for the given + // op, device, and TypeName. + VariantUnaryOpFn* GetUnaryOpFn(VariantUnaryOp op, StringPiece device, + const TypeIndex& type_index); + + // Add a binary op function to the registry. + void RegisterBinaryOpFn(VariantBinaryOp op, const string& device, + const TypeIndex& type_index, + const VariantBinaryOpFn& add_fn); + + // Returns nullptr if no binary op function was found for the given + // op, device and TypeName. + VariantBinaryOpFn* GetBinaryOpFn(VariantBinaryOp op, StringPiece device, + const TypeIndex& type_index); + + // Get a pointer to a global UnaryVariantOpRegistry object + static UnaryVariantOpRegistry* Global(); + + // Get a pointer to a global persistent string storage object. + // ISO/IEC C++ working draft N4296 clarifies that insertion into an + // std::unordered_set does not invalidate memory locations of + // *values* inside the set (though it may invalidate existing + // iterators). In other words, one may safely point a StringPiece to + // a value in the set without that StringPiece being invalidated by + // future insertions. + static std::unordered_set* PersistentStringStorage(); + + private: + struct TypeIndexHash { + std::size_t operator()(const TypeIndex& x) const { return x.hash_code(); } + }; + + gtl::FlatMap shape_fns; + gtl::FlatMap decode_fns; + + // Map std::pair to function. + struct PairHash { + template + std::size_t operator()(const std::pair& x) const { + // The hash of an enum is just its value as a std::size_t. + std::size_t ret = static_cast(std::get<0>(x)); + ret = Hash64Combine(ret, std::get<1>(x).hash_code()); + return ret; + } + }; + + gtl::FlatMap, + AsyncVariantDeviceCopyFn, PairHash> + device_copy_fns; + + // Map std::tuple to function. + + // this breaks by falling victim to "too perfect forwarding" + // see https://stackoverflow.com/questions/44475317/variadic-template-issue + // and references therein + template + struct FuncTuple { + FuncTuple(const Op& op, const StringPiece& dev, const TypeIndex& type_index) + : op_type_(op), device_(dev), type_index_(type_index) {} + Op op_type_; + StringPiece device_; + TypeIndex type_index_; + }; + // friend declaration for operator== + // needed for clang + template + friend bool operator==(const FuncTuple& l, const FuncTuple& r); + struct TupleHash { + template + std::size_t operator()( + const std::tuple& x) const { + // The hash of an enum is just its value as a std::size_t. + std::size_t ret = static_cast(std::get<0>(x)); + ret = Hash64Combine(ret, sp_hasher_(std::get<1>(x))); + ret = Hash64Combine(ret, std::get<2>(x).hash_code()); + return ret; + } + + template + std::size_t operator()(const FuncTuple& x) const { + // The hash of an enum is just its value as a std::size_t. + std::size_t ret = static_cast(x.op_type_); + ret = Hash64Combine(ret, sp_hasher_(x.device_)); + ret = Hash64Combine(ret, x.type_index_.hash_code()); + return ret; + } + StringPieceHasher sp_hasher_; + }; + gtl::FlatMap, VariantUnaryOpFn, TupleHash> + unary_op_fns; + gtl::FlatMap, VariantBinaryOpFn, TupleHash> + binary_op_fns; + + // Find or insert a string into a persistent string storage + // container; return the StringPiece pointing to the permanent string + // location. + static StringPiece GetPersistentStringPiece(const string& str) { + const auto string_storage = PersistentStringStorage(); + auto found = string_storage->find(str); + if (found == string_storage->end()) { + auto inserted = string_storage->insert(str); + return StringPiece(*inserted.first); + } else { + return StringPiece(*found); + } + } +}; +template +inline bool operator==(const UnaryVariantOpRegistry::FuncTuple& lhs, + const UnaryVariantOpRegistry::FuncTuple& rhs) { + return (lhs.op_type_ == rhs.op_type_) && (lhs.device_ == rhs.device_) && + (lhs.type_index_ == rhs.type_index_); +} +// Gets a TensorShape from a Tensor containing a scalar Variant. +// Returns an Internal error if the Variant does not have a registered shape +// function, or if it's a serialized Variant that cannot be decoded. +// +// REQUIRES: +// variant_tensor.dtype() == DT_VARIANT +// variant_tensor.dims() == 0 +// +Status GetUnaryVariantShape(const Tensor& variant_tensor, TensorShape* shape); + +// Decodes the Variant whose data_type has a registered decode +// function. Returns an Internal error if the Variant does not have a +// registered decode function, or if the decoding function fails. +// +// REQUIRES: +// variant is not null. +// +bool DecodeUnaryVariant(Variant* variant); + +// Copies a variant between CPU<->GPU, or between GPU<->GPU. +// The variant 'from' must have a registered DeviceCopyFn for the +// given direction. The returned variant 'to' will have +// (some subset of its) tensors stored on destination according to the +// registered DeviceCopyFn function for the given direction. Returns +// an Internal error if the Variant does not have a registered +// DeviceCopyFn function for the given direction, or if initiating the +// copy fails. +// +// REQUIRES: +// 'to' is not null. +// +Status VariantDeviceCopy( + const VariantDeviceCopyDirection direction, const Variant& from, + Variant* to, + const UnaryVariantOpRegistry::AsyncTensorDeviceCopyFn& copy_fn); + +// Sets *v_out = unary_op(v). The variant v must have a registered +// UnaryOp function for the given Device. Returns an Internal error +// if v does not have a registered unary_op function for this device, or if +// UnaryOp fails. +// +// REQUIRES: +// v_out is not null. +// +template +Status UnaryOpVariant(OpKernelContext* ctx, VariantUnaryOp op, const Variant& v, + Variant* v_out) { + const string& device = DeviceName::value; + UnaryVariantOpRegistry::VariantUnaryOpFn* unary_op_fn = + UnaryVariantOpRegistry::Global()->GetUnaryOpFn(op, device, v.TypeId()); + if (unary_op_fn == nullptr) { + return errors::Internal( + "No unary variant unary_op function found for unary variant op enum: ", + op, " Variant type_name: ", v.TypeName(), " for device type: ", device); + } + return (*unary_op_fn)(ctx, v, v_out); +} + +// Sets *out = binary_op(a, b). The variants a and b must be the same type +// and have a registered binary_op function for the given Device. Returns an +// Internal error if a and b are not the same type_name or if +// if a does not have a registered op function for this device, or if +// BinaryOp fails. +// +// REQUIRES: +// out is not null. +// +template +Status BinaryOpVariants(OpKernelContext* ctx, VariantBinaryOp op, + const Variant& a, const Variant& b, Variant* out) { + if (a.TypeId() != b.TypeId()) { + return errors::Internal( + "BianryOpVariants: Variants a and b have different " + "type ids. Type names: '", + a.TypeName(), "' vs. '", b.TypeName(), "'"); + } + const string& device = DeviceName::value; + UnaryVariantOpRegistry::VariantBinaryOpFn* binary_op_fn = + UnaryVariantOpRegistry::Global()->GetBinaryOpFn(op, device, a.TypeId()); + if (binary_op_fn == nullptr) { + return errors::Internal( + "No unary variant binary_op function found for binary variant op " + "enum: ", + op, " Variant type_name: '", a.TypeName(), "' for device type: ", + device); + } + return (*binary_op_fn)(ctx, a, b, out); +} + +namespace variant_op_registry_fn_registration { + +template +class UnaryVariantShapeRegistration { + public: + typedef std::function LocalVariantShapeFn; + + UnaryVariantShapeRegistration(const TypeIndex& type_index, + const LocalVariantShapeFn& shape_fn) { + const string type_index_name = port::MaybeAbiDemangle(type_index.name()); + UnaryVariantOpRegistry::Global()->RegisterShapeFn( + type_index, + [type_index_name, shape_fn](const Variant& v, + TensorShape* s) -> Status { + const T* t = v.get(); + if (t == nullptr) { + return errors::Internal( + "VariantShapeFn: Could not access object, type_index: ", + type_index_name); + } + return shape_fn(*t, s); + }); + } +}; + +template +class UnaryVariantDecodeRegistration { + public: + UnaryVariantDecodeRegistration(const string& type_name) { + // The Variant is passed by pointer because it should be + // mutable: get below may Decode the variant, which + // is a self-mutating behavior. The variant is not modified in + // any other way. + UnaryVariantOpRegistry::Global()->RegisterDecodeFn( + type_name, [type_name](Variant* v) -> bool { + DCHECK_NE(v, nullptr); + VariantTensorDataProto* t = v->get(); + if (t == nullptr) { + return false; + } + Variant decoded = T(); + VariantTensorData data(std::move(*t)); + if (!decoded.Decode(std::move(data))) { + return false; + } + std::swap(decoded, *v); + return true; + }); + } +}; + +template +class UnaryVariantDeviceCopyRegistration { + public: + typedef std::function + LocalVariantDeviceCopyFn; + UnaryVariantDeviceCopyRegistration( + const VariantDeviceCopyDirection direction, const TypeIndex& type_index, + const LocalVariantDeviceCopyFn& device_copy_fn) { + const string type_index_name = port::MaybeAbiDemangle(type_index.name()); + UnaryVariantOpRegistry::Global()->RegisterDeviceCopyFn( + direction, type_index, + [type_index_name, device_copy_fn]( + const Variant& from, Variant* to, + UnaryVariantOpRegistry::AsyncTensorDeviceCopyFn + device_copy_tensor_fn) -> Status { + DCHECK_NE(to, nullptr); + *to = T(); + if (from.get() == nullptr) { + return errors::Internal( + "VariantCopyToGPUFn: Could not access object, type_index: ", + type_index_name); + } + const T& t = *from.get(); + T* t_out = to->get(); + return device_copy_fn(t, t_out, device_copy_tensor_fn); + }); + } +}; + +template +class UnaryVariantUnaryOpRegistration { + typedef std::function + LocalVariantUnaryOpFn; + + public: + UnaryVariantUnaryOpRegistration(VariantUnaryOp op, const string& device, + const TypeIndex& type_index, + const LocalVariantUnaryOpFn& unary_op_fn) { + const string type_index_name = port::MaybeAbiDemangle(type_index.name()); + UnaryVariantOpRegistry::Global()->RegisterUnaryOpFn( + op, device, type_index, + [type_index_name, unary_op_fn](OpKernelContext* ctx, const Variant& v, + Variant* v_out) -> Status { + DCHECK_NE(v_out, nullptr); + *v_out = T(); + if (v.get() == nullptr) { + return errors::Internal( + "VariantUnaryOpFn: Could not access object, type_index: ", + type_index_name); + } + const T& t = *v.get(); + T* t_out = v_out->get(); + return unary_op_fn(ctx, t, t_out); + }); + } +}; + +template +class UnaryVariantBinaryOpRegistration { + typedef std::function + LocalVariantBinaryOpFn; + + public: + UnaryVariantBinaryOpRegistration(VariantBinaryOp op, const string& device, + const TypeIndex& type_index, + const LocalVariantBinaryOpFn& binary_op_fn) { + const string type_index_name = port::MaybeAbiDemangle(type_index.name()); + UnaryVariantOpRegistry::Global()->RegisterBinaryOpFn( + op, device, type_index, + [type_index_name, binary_op_fn](OpKernelContext* ctx, const Variant& a, + const Variant& b, + Variant* out) -> Status { + DCHECK_NE(out, nullptr); + *out = T(); + if (a.get() == nullptr) { + return errors::Internal( + "VariantBinaryOpFn: Could not access object 'a', type_index: ", + type_index_name); + } + if (b.get() == nullptr) { + return errors::Internal( + "VariantBinaryOpFn: Could not access object 'b', type_index: ", + type_index_name); + } + const T& t_a = *a.get(); + const T& t_b = *b.get(); + T* t_out = out->get(); + return binary_op_fn(ctx, t_a, t_b, t_out); + }); + } +}; + +}; // namespace variant_op_registry_fn_registration + +// Register a unary shape variant function with the signature: +// Status ShapeFn(const T& t, TensorShape* s); +// to Variants having TypeIndex type_index. +#define REGISTER_UNARY_VARIANT_SHAPE_FUNCTION(T, shape_function) \ + REGISTER_UNARY_VARIANT_SHAPE_FUNCTION_UNIQ_HELPER( \ + __COUNTER__, T, MakeTypeIndex(), shape_function) + +#define REGISTER_UNARY_VARIANT_SHAPE_FUNCTION_UNIQ_HELPER(ctr, T, type_index, \ + shape_function) \ + REGISTER_UNARY_VARIANT_SHAPE_FUNCTION_UNIQ(ctr, T, type_index, shape_function) + +#define REGISTER_UNARY_VARIANT_SHAPE_FUNCTION_UNIQ(ctr, T, type_index, \ + shape_function) \ + static variant_op_registry_fn_registration::UnaryVariantShapeRegistration \ + register_unary_variant_op_shape_registration_fn_##ctr(type_index, \ + shape_function) + +// Register a unary decode variant function for the given type. +#define REGISTER_UNARY_VARIANT_DECODE_FUNCTION(T, type_name) \ + REGISTER_UNARY_VARIANT_DECODE_FUNCTION_UNIQ_HELPER(__COUNTER__, T, type_name) + +#define REGISTER_UNARY_VARIANT_DECODE_FUNCTION_UNIQ_HELPER(ctr, T, type_name) \ + REGISTER_UNARY_VARIANT_DECODE_FUNCTION_UNIQ(ctr, T, type_name) + +#define REGISTER_UNARY_VARIANT_DECODE_FUNCTION_UNIQ(ctr, T, type_name) \ + static variant_op_registry_fn_registration::UnaryVariantDecodeRegistration< \ + T> \ + register_unary_variant_op_decoder_fn_##ctr(type_name) + +// ****** NOTE ****** +// FOR INTERNAL USE ONLY. IF YOU USE THIS WE MAY BREAK YOUR CODE. +// ****** NOTE ****** +// +// Register a device copy variant function for the given copy +// direction and type; where direction is the enum +// VariantDeviceCopyDirection, and the device_copy_fn has signature: +// +// Status device_copy_fn( +// const T& t, T* t_out, +// const UnaryVariantOpRegistry::AsyncTensorDeviceCopyFn& copier); +// +// And device_copy_fn calls copier 0 or more times. For details on +// the behavior of the copier function, see the comments at the +// declaration of UnaryVariantOpRegistry::AsyncTensorDeviceCopyFn. +// +// Note, the device_copy_fn may choose to keep some tensors +// on host, e.g. by assigning to->tensor = from.tensor (assuming +// from.tensor is already on host); or by setting +// to->tensor = Tensor(cpu_allocator(), ...) +// and manually updating its values. +// +// If this is the case, the CopyFns for HOST_TO_DEVICE, +// DEVICE_TO_HOST, and DEVICE_TO_DEVICE must perform host-to-host +// copies in a consistent manner. For example, one must always +// manually copy any "always on host" tensors in all directions instead of e.g. +// - performing a host-to-host copy in one direction, +// - using the provided copier function in the reverse direction. +// Doing the latter will cause program failures. +// +// ****** NOTE ****** +// FOR INTERNAL USE ONLY. IF YOU USE THIS WE MAY BREAK YOUR CODE. +// ****** NOTE ****** +#define INTERNAL_REGISTER_UNARY_VARIANT_DEVICE_COPY_FUNCTION(T, direction, \ + device_copy_fn) \ + INTERNAL_REGISTER_UNARY_VARIANT_DEVICE_COPY_FUNCTION_UNIQ_HELPER( \ + __COUNTER__, T, direction, MakeTypeIndex(), device_copy_fn) + +#define INTERNAL_REGISTER_UNARY_VARIANT_DEVICE_COPY_FUNCTION_UNIQ_HELPER( \ + ctr, T, direction, type_index, device_copy_fn) \ + INTERNAL_REGISTER_UNARY_VARIANT_DEVICE_COPY_FUNCTION_UNIQ( \ + ctr, T, direction, type_index, device_copy_fn) + +#define INTERNAL_REGISTER_UNARY_VARIANT_DEVICE_COPY_FUNCTION_UNIQ( \ + ctr, T, direction, type_index, device_copy_fn) \ + static variant_op_registry_fn_registration:: \ + UnaryVariantDeviceCopyRegistration \ + register_unary_variant_op_device_copy_fn_##ctr( \ + direction, type_index, device_copy_fn) + +// Register a unary unary_op variant function with the signature: +// Status UnaryOpFn(OpKernelContext* ctx, const T& t, T* t_out); +// to Variants having TypeIndex type_index, for device string device, +// for UnaryVariantOp enum op. +#define REGISTER_UNARY_VARIANT_UNARY_OP_FUNCTION(op, device, T, \ + unary_op_function) \ + REGISTER_UNARY_VARIANT_UNARY_OP_FUNCTION_UNIQ_HELPER( \ + __COUNTER__, op, device, T, MakeTypeIndex(), unary_op_function) + +#define REGISTER_UNARY_VARIANT_UNARY_OP_FUNCTION_UNIQ_HELPER( \ + ctr, op, device, T, type_index, unary_op_function) \ + REGISTER_UNARY_VARIANT_UNARY_OP_FUNCTION_UNIQ(ctr, op, device, T, \ + type_index, unary_op_function) + +#define REGISTER_UNARY_VARIANT_UNARY_OP_FUNCTION_UNIQ( \ + ctr, op, device, T, type_index, unary_op_function) \ + static variant_op_registry_fn_registration::UnaryVariantUnaryOpRegistration< \ + T> \ + register_unary_variant_op_decoder_fn_##ctr(op, device, type_index, \ + unary_op_function) + +// Register a binary_op variant function with the signature: +// Status BinaryOpFn(OpKernelContext* ctx, const T& a, const T& b, T* out); +// to Variants having TypeIndex type_index, for device string device, +// for BinaryVariantOp enum OP. +#define REGISTER_UNARY_VARIANT_BINARY_OP_FUNCTION(op, device, T, \ + binary_op_function) \ + REGISTER_UNARY_VARIANT_BINARY_OP_FUNCTION_UNIQ_HELPER( \ + __COUNTER__, op, device, T, MakeTypeIndex(), binary_op_function) + +#define REGISTER_UNARY_VARIANT_BINARY_OP_FUNCTION_UNIQ_HELPER( \ + ctr, op, device, T, type_index, binary_op_function) \ + REGISTER_UNARY_VARIANT_BINARY_OP_FUNCTION_UNIQ( \ + ctr, op, device, T, type_index, binary_op_function) + +#define REGISTER_UNARY_VARIANT_BINARY_OP_FUNCTION_UNIQ( \ + ctr, op, device, T, type_index, binary_op_function) \ + static variant_op_registry_fn_registration:: \ + UnaryVariantBinaryOpRegistration \ + register_unary_variant_op_decoder_fn_##ctr(op, device, type_index, \ + binary_op_function) + +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_FRAMEWORK_VARIANT_OP_REGISTRY_H_ diff --git a/linux/include/tensorflow/core/framework/variant_tensor_data.h b/linux/include/tensorflow/core/framework/variant_tensor_data.h new file mode 100644 index 000000000..8a240ee1e --- /dev/null +++ b/linux/include/tensorflow/core/framework/variant_tensor_data.h @@ -0,0 +1,117 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_FRAMEWORK_VARIANT_TENSOR_DATA_H_ +#define TENSORFLOW_CORE_FRAMEWORK_VARIANT_TENSOR_DATA_H_ + +#include +#include + +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/lib/core/stringpiece.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +class VariantTensorDataProto; + +// The serialization format for Variant objects. Objects with references to +// other Tensors can simply store those tensors in the `tensors` field, and +// serialize other metadata content in to the `metadata` field. Objects can +// optionally set the `type_name` for type-checking before deserializing an +// object. +// +// This is the native C++ class equivalent of VariantTensorDataProto. They are +// separate so that kernels do not need to depend on protos. +class VariantTensorData { + public: + VariantTensorData(); + VariantTensorData(VariantTensorDataProto proto); + ~VariantTensorData(); + + // Name of the type of objects being serialized. + const string& type_name() const { return type_name_; } + void set_type_name(const string& type_name) { type_name_ = type_name; } + + template ::type>::value> + struct PODResolver {}; + + // Portions of the object that are not Tensors. + // Directly supported types include string POD types. + template + void set_metadata(const T& value) { + SetMetadata(value, PODResolver()); + } + + template + bool get_metadata(T* value) const { + return GetMetadata(value, PODResolver()); + } + + // Tensors contained within objects being serialized. + int tensors_size() const; + const Tensor& tensors(int index) const; + const std::vector& tensors() const; + Tensor* add_tensors(); + + // Conversion to and from VariantTensorDataProto + void ToProto(VariantTensorDataProto* proto) const; + // This allows optimizations via std::move. + bool FromProto(VariantTensorDataProto proto); + bool FromConstProto(const VariantTensorDataProto& proto); + + // Serialization via VariantTensorDataProto + string SerializeAsString() const; + bool SerializeToString(string* buf); + bool ParseFromString(string s); + + string DebugString() const; + + public: + string type_name_; + string metadata_; + std::vector tensors_; + + private: + template + void SetMetadata(const string& value, PODResolver) { + metadata_ = value; + } + + template + bool GetMetadata(string* value, PODResolver) const { + *value = metadata_; + return true; + } + + template + void SetMetadata(const T& value, PODResolver) { + metadata_.assign(reinterpret_cast(&value), sizeof(T)); + } + + template + bool GetMetadata(T* value, PODResolver) const { + if (metadata_.size() != sizeof(T)) return false; + std::copy_n(metadata_.data(), sizeof(T), reinterpret_cast(value)); + return true; + } +}; + +// For backwards compatibility for when this was a proto +string ProtoDebugString(const VariantTensorData& object); + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_FRAMEWORK_VARIANT_TENSOR_DATA_H_ diff --git a/linux/include/tensorflow/core/framework/versions.h b/linux/include/tensorflow/core/framework/versions.h new file mode 100644 index 000000000..9676667b8 --- /dev/null +++ b/linux/include/tensorflow/core/framework/versions.h @@ -0,0 +1,39 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_FRAMEWORK_VERSIONS_H_ +#define TENSORFLOW_FRAMEWORK_VERSIONS_H_ + +#include "tensorflow/core/lib/core/status.h" + +namespace tensorflow { + +class VersionDef; + +// Check whether data with the given versions is compatible with the given +// consumer and min producer. upper_name and lower_name are used to form +// error messages upon failure. Example usage: +// +// #include "tensorflow/core/public/version.h" +// +// TF_RETURN_IF_ERROR(CheckVersions(versions, TF_GRAPH_DEF_VERSION, +// TF_GRAPH_DEF_VERSION_MIN_PRODUCER, +// "GraphDef", "graph")); +Status CheckVersions(const VersionDef& versions, int consumer, int min_producer, + const char* upper_name, const char* lower_name); + +} // namespace tensorflow + +#endif // TENSORFLOW_FRAMEWORK_VERSIONS_H_ diff --git a/linux/include/tensorflow/core/framework/versions.pb.h b/linux/include/tensorflow/core/framework/versions.pb.h new file mode 100644 index 000000000..b118c1d89 --- /dev/null +++ b/linux/include/tensorflow/core/framework/versions.pb.h @@ -0,0 +1,278 @@ +// Generated by the protocol buffer compiler. DO NOT EDIT! +// source: tensorflow/core/framework/versions.proto + +#ifndef PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2fversions_2eproto +#define PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2fversions_2eproto + +#include + +#include + +#if GOOGLE_PROTOBUF_VERSION < 3006000 +#error This file was generated by a newer version of protoc which is +#error incompatible with your Protocol Buffer headers. Please update +#error your headers. +#endif +#if 3006000 < GOOGLE_PROTOBUF_MIN_PROTOC_VERSION +#error This file was generated by an older version of protoc which is +#error incompatible with your Protocol Buffer headers. Please +#error regenerate this file with a newer version of protoc. +#endif + +#include +#include +#include +#include +#include +#include +#include +#include +#include // IWYU pragma: export +#include // IWYU pragma: export +#include +// @@protoc_insertion_point(includes) +#define PROTOBUF_INTERNAL_EXPORT_protobuf_tensorflow_2fcore_2fframework_2fversions_2eproto + +namespace protobuf_tensorflow_2fcore_2fframework_2fversions_2eproto { +// Internal implementation detail -- do not use these members. +struct TableStruct { + static const ::google::protobuf::internal::ParseTableField entries[]; + static const ::google::protobuf::internal::AuxillaryParseTableField aux[]; + static const ::google::protobuf::internal::ParseTable schema[1]; + static const ::google::protobuf::internal::FieldMetadata field_metadata[]; + static const ::google::protobuf::internal::SerializationTable serialization_table[]; + static const ::google::protobuf::uint32 offsets[]; +}; +void AddDescriptors(); +} // namespace protobuf_tensorflow_2fcore_2fframework_2fversions_2eproto +namespace tensorflow { +class VersionDef; +class VersionDefDefaultTypeInternal; +extern VersionDefDefaultTypeInternal _VersionDef_default_instance_; +} // namespace tensorflow +namespace google { +namespace protobuf { +template<> ::tensorflow::VersionDef* Arena::CreateMaybeMessage<::tensorflow::VersionDef>(Arena*); +} // namespace protobuf +} // namespace google +namespace tensorflow { + +// =================================================================== + +class VersionDef : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.VersionDef) */ { + public: + VersionDef(); + virtual ~VersionDef(); + + VersionDef(const VersionDef& from); + + inline VersionDef& operator=(const VersionDef& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + VersionDef(VersionDef&& from) noexcept + : VersionDef() { + *this = ::std::move(from); + } + + inline VersionDef& operator=(VersionDef&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const VersionDef& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const VersionDef* internal_default_instance() { + return reinterpret_cast( + &_VersionDef_default_instance_); + } + static constexpr int kIndexInFileMessages = + 0; + + void UnsafeArenaSwap(VersionDef* other); + void Swap(VersionDef* other); + friend void swap(VersionDef& a, VersionDef& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline VersionDef* New() const final { + return CreateMaybeMessage(NULL); + } + + VersionDef* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const VersionDef& from); + void MergeFrom(const VersionDef& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(VersionDef* other); + protected: + explicit VersionDef(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated int32 bad_consumers = 3; + int bad_consumers_size() const; + void clear_bad_consumers(); + static const int kBadConsumersFieldNumber = 3; + ::google::protobuf::int32 bad_consumers(int index) const; + void set_bad_consumers(int index, ::google::protobuf::int32 value); + void add_bad_consumers(::google::protobuf::int32 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::int32 >& + bad_consumers() const; + ::google::protobuf::RepeatedField< ::google::protobuf::int32 >* + mutable_bad_consumers(); + + // int32 producer = 1; + void clear_producer(); + static const int kProducerFieldNumber = 1; + ::google::protobuf::int32 producer() const; + void set_producer(::google::protobuf::int32 value); + + // int32 min_consumer = 2; + void clear_min_consumer(); + static const int kMinConsumerFieldNumber = 2; + ::google::protobuf::int32 min_consumer() const; + void set_min_consumer(::google::protobuf::int32 value); + + // @@protoc_insertion_point(class_scope:tensorflow.VersionDef) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedField< ::google::protobuf::int32 > bad_consumers_; + mutable int _bad_consumers_cached_byte_size_; + ::google::protobuf::int32 producer_; + ::google::protobuf::int32 min_consumer_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fframework_2fversions_2eproto::TableStruct; +}; +// =================================================================== + + +// =================================================================== + +#ifdef __GNUC__ + #pragma GCC diagnostic push + #pragma GCC diagnostic ignored "-Wstrict-aliasing" +#endif // __GNUC__ +// VersionDef + +// int32 producer = 1; +inline void VersionDef::clear_producer() { + producer_ = 0; +} +inline ::google::protobuf::int32 VersionDef::producer() const { + // @@protoc_insertion_point(field_get:tensorflow.VersionDef.producer) + return producer_; +} +inline void VersionDef::set_producer(::google::protobuf::int32 value) { + + producer_ = value; + // @@protoc_insertion_point(field_set:tensorflow.VersionDef.producer) +} + +// int32 min_consumer = 2; +inline void VersionDef::clear_min_consumer() { + min_consumer_ = 0; +} +inline ::google::protobuf::int32 VersionDef::min_consumer() const { + // @@protoc_insertion_point(field_get:tensorflow.VersionDef.min_consumer) + return min_consumer_; +} +inline void VersionDef::set_min_consumer(::google::protobuf::int32 value) { + + min_consumer_ = value; + // @@protoc_insertion_point(field_set:tensorflow.VersionDef.min_consumer) +} + +// repeated int32 bad_consumers = 3; +inline int VersionDef::bad_consumers_size() const { + return bad_consumers_.size(); +} +inline void VersionDef::clear_bad_consumers() { + bad_consumers_.Clear(); +} +inline ::google::protobuf::int32 VersionDef::bad_consumers(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.VersionDef.bad_consumers) + return bad_consumers_.Get(index); +} +inline void VersionDef::set_bad_consumers(int index, ::google::protobuf::int32 value) { + bad_consumers_.Set(index, value); + // @@protoc_insertion_point(field_set:tensorflow.VersionDef.bad_consumers) +} +inline void VersionDef::add_bad_consumers(::google::protobuf::int32 value) { + bad_consumers_.Add(value); + // @@protoc_insertion_point(field_add:tensorflow.VersionDef.bad_consumers) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int32 >& +VersionDef::bad_consumers() const { + // @@protoc_insertion_point(field_list:tensorflow.VersionDef.bad_consumers) + return bad_consumers_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int32 >* +VersionDef::mutable_bad_consumers() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.VersionDef.bad_consumers) + return &bad_consumers_; +} + +#ifdef __GNUC__ + #pragma GCC diagnostic pop +#endif // __GNUC__ + +// @@protoc_insertion_point(namespace_scope) + +} // namespace tensorflow + +// @@protoc_insertion_point(global_scope) + +#endif // PROTOBUF_INCLUDED_tensorflow_2fcore_2fframework_2fversions_2eproto diff --git a/linux/include/tensorflow/core/framework/versions.pb_text-impl.h b/linux/include/tensorflow/core/framework/versions.pb_text-impl.h new file mode 100644 index 000000000..14776c2f5 --- /dev/null +++ b/linux/include/tensorflow/core/framework/versions.pb_text-impl.h @@ -0,0 +1,25 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_framework_versions_proto_IMPL_H_ +#define tensorflow_core_framework_versions_proto_IMPL_H_ + +#include "tensorflow/core/framework/versions.pb.h" +#include "tensorflow/core/framework/versions.pb_text.h" +#include "tensorflow/core/lib/strings/proto_text_util.h" +#include "tensorflow/core/lib/strings/scanner.h" + +namespace tensorflow { + +namespace internal { + +void AppendProtoDebugString( + ::tensorflow::strings::ProtoTextOutput* o, + const ::tensorflow::VersionDef& msg); +bool ProtoParseFromScanner( + ::tensorflow::strings::Scanner* scanner, bool nested, bool close_curly, + ::tensorflow::VersionDef* msg); + +} // namespace internal + +} // namespace tensorflow + +#endif // tensorflow_core_framework_versions_proto_IMPL_H_ diff --git a/linux/include/tensorflow/core/framework/versions.pb_text.h b/linux/include/tensorflow/core/framework/versions.pb_text.h new file mode 100644 index 000000000..393bd97f9 --- /dev/null +++ b/linux/include/tensorflow/core/framework/versions.pb_text.h @@ -0,0 +1,24 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_framework_versions_proto_H_ +#define tensorflow_core_framework_versions_proto_H_ + +#include "tensorflow/core/framework/versions.pb.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/protobuf.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +// Message-text conversion for tensorflow.VersionDef +string ProtoDebugString( + const ::tensorflow::VersionDef& msg); +string ProtoShortDebugString( + const ::tensorflow::VersionDef& msg); +bool ProtoParseFromString( + const string& s, + ::tensorflow::VersionDef* msg) + TF_MUST_USE_RESULT; + +} // namespace tensorflow + +#endif // tensorflow_core_framework_versions_proto_H_ diff --git a/linux/include/tensorflow/core/graph/algorithm.h b/linux/include/tensorflow/core/graph/algorithm.h new file mode 100644 index 000000000..45f8a29a9 --- /dev/null +++ b/linux/include/tensorflow/core/graph/algorithm.h @@ -0,0 +1,120 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPH_ALGORITHM_H_ +#define TENSORFLOW_CORE_GRAPH_ALGORITHM_H_ + +#include +#include +#include + +#include "tensorflow/core/graph/graph.h" +#include "tensorflow/core/lib/gtl/array_slice.h" + +namespace tensorflow { + +// Comparator for two nodes. This is used in order to get a stable ording. +using NodeComparator = std::function; + +using EdgeFilter = std::function; + +// Compares two node based on their ids. +struct NodeComparatorID { + bool operator()(const Node* n1, const Node* n2) const { + return n1->id() < n2->id(); + } +}; + +// Compare two nodes based on their names. +struct NodeComparatorName { + bool operator()(const Node* n1, const Node* n2) const { + return n1->name() < n2->name(); + } +}; + +// Perform a depth-first-search on g starting at the source node. +// If enter is not empty, calls enter(n) before visiting any children of n. +// If leave is not empty, calls leave(n) after visiting all children of n. +// If stable_comparator is set, a stable ordering of visit is achieved by +// sorting a node's neighbors first before visiting them. +// If edge_filter is set then ignores edges for which edge_filter returns false. +extern void DFS(const Graph& g, const std::function& enter, + const std::function& leave, + const NodeComparator& stable_comparator = {}, + const EdgeFilter& edge_filter = {}); + +// Perform a reverse depth-first-search on g starting at the sink node. +// If enter is not empty, calls enter(n) before visiting any parents of n. +// If leave is not empty, calls leave(n) after visiting all parents of n. +// If stable_comparator is set, a stable ordering of visit is achieved by +// sorting a node's neighbors first before visiting them. +extern void ReverseDFS(const Graph& g, const std::function& enter, + const std::function& leave, + const NodeComparator& stable_comparator = {}); + +// Perform a reverse depth-first-search on g starting at the 'start' nodes. +// If enter is not empty, calls enter(n) before visiting any parents of n. +// If leave is not empty, calls leave(n) after visiting all parents of n. +// If stable_comparator is set, a stable ordering of visit is achieved by +// sorting a node's neighbors first before visiting them. +extern void ReverseDFSFrom(const Graph& g, gtl::ArraySlice start, + const std::function& enter, + const std::function& leave, + const NodeComparator& stable_comparator = {}); +extern void ReverseDFSFrom(const Graph& g, gtl::ArraySlice start, + const std::function& enter, + const std::function& leave, + const NodeComparator& stable_comparator = {}); + +// Stores in *order the post-order numbering of all nodes +// in graph found via a depth first search starting at the source node. +// +// Note that this is equivalent to reverse topological sorting when the +// graph does not have cycles. +// +// If stable_comparator is set, a stable ordering of visit is achieved by +// sorting a node's neighbors first before visiting them. +// +// If edge_filter is set then ignores edges for which edge_filter returns false. +// +// REQUIRES: order is not NULL. +void GetPostOrder(const Graph& g, std::vector* order, + const NodeComparator& stable_comparator = {}, + const EdgeFilter& edge_filter = {}); + +// Stores in *order the reverse post-order numbering of all nodes +// If stable_comparator is set, a stable ordering of visit is achieved by +// sorting a node's neighbors first before visiting them. +// +// If edge_filter is set then ignores edges for which edge_filter returns false. +void GetReversePostOrder(const Graph& g, std::vector* order, + const NodeComparator& stable_comparator = {}, + const EdgeFilter& edge_filter = {}); + +// Prune nodes in "g" that are not in some path from the source node +// to any node in 'nodes'. Returns true if changes were made to the graph. +// Does not fix up source and sink edges. +bool PruneForReverseReachability(Graph* g, + std::unordered_set nodes); + +// Connect all nodes with no incoming edges to source. +// Connect all nodes with no outgoing edges to sink. +// +// Returns true if and only if 'g' is mutated. +bool FixupSourceAndSinkEdges(Graph* g); + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPH_ALGORITHM_H_ diff --git a/linux/include/tensorflow/core/graph/colors.h b/linux/include/tensorflow/core/graph/colors.h new file mode 100644 index 000000000..43d222557 --- /dev/null +++ b/linux/include/tensorflow/core/graph/colors.h @@ -0,0 +1,29 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPH_COLORS_H_ +#define TENSORFLOW_CORE_GRAPH_COLORS_H_ + +namespace tensorflow { + +// Return a color drawn from a palette to represent an entity +// identified by "i". The return value has the form "#RRGGBB" Note +// that the palette has a limited set of colors and therefore colors +// will be reused eventually. +const char* ColorFor(int dindex); + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPH_COLORS_H_ diff --git a/linux/include/tensorflow/core/graph/control_flow.h b/linux/include/tensorflow/core/graph/control_flow.h new file mode 100644 index 000000000..5abe77f5a --- /dev/null +++ b/linux/include/tensorflow/core/graph/control_flow.h @@ -0,0 +1,51 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPH_CONTROL_FLOW_H_ +#define TENSORFLOW_CORE_GRAPH_CONTROL_FLOW_H_ + +#include + +#include "tensorflow/core/graph/graph.h" +#include "tensorflow/core/lib/core/status.h" + +namespace tensorflow { + +// Control flow info for a graph node. +struct ControlFlowInfo { + const Node* frame = nullptr; // frame of a node + const Node* parent_frame = nullptr; // parent frame of a node + string frame_name; // frame name of a node +}; + +// Clear and populate `info` with each node's frame and the level it belongs to. +// We check the well-formedness of the graph: +// 1) All inputs to a node must come from the same frame and have the same +// "static" iteration level. +// 2) Each frame has at most one LoopCond node. +// 3) Each frame has a single parent frame. +// If `unreachable_nodes` is set, return names of nodes unreachable from the +// source node. We cannot build ControlFlowInfo for such nodes. They might be +// pruned later. +// +// NOTE(yuanbyu): For now, we require all sends/recvs have iteration level 0. +// This essentially means there can't be multiple serial Nexts in an iteration, +// which all sane front-ends should satisfy. +Status BuildControlFlowInfo(const Graph* g, std::vector* info, + std::vector* unreachable_nodes = nullptr); + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPH_CONTROL_FLOW_H_ diff --git a/linux/include/tensorflow/core/graph/costmodel.h b/linux/include/tensorflow/core/graph/costmodel.h new file mode 100644 index 000000000..2d94dd5cd --- /dev/null +++ b/linux/include/tensorflow/core/graph/costmodel.h @@ -0,0 +1,232 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPH_COSTMODEL_H_ +#define TENSORFLOW_CORE_GRAPH_COSTMODEL_H_ + +#include +#include + +#include "tensorflow/core/framework/cost_graph.pb.h" +#include "tensorflow/core/framework/step_stats.pb.h" +#include "tensorflow/core/framework/tensor_shape.pb.h" +#include "tensorflow/core/graph/graph.h" +#include "tensorflow/core/graph/types.h" +#include "tensorflow/core/lib/core/stringpiece.h" +#include "tensorflow/core/lib/gtl/array_slice.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/protobuf.h" + +namespace tensorflow { +typedef std::unordered_map + NodeNameToCostIdMap; + +class StepStats; + +// CostModel keeps track of the following runtime statistics for nodes +// of a single Graph: +// * The total number of times a node has executed. +// * The accumulated execution time (in microseconds) of a node. +// * The accumulated size (in bytes) of each node's output. +// +// This class is NOT thread-safe. +class CostModel { + public: + // If "global" is true, maintains costs based on Node::cost_id, otherwise + // maintains costs based on Node::id. + explicit CostModel(bool is_global) : is_global_(is_global) { + unknown_shape_.set_unknown_rank(true); + } + + // Assigns min_count_ as a function of the median count for a Node. + // This value is then used for suppressing the time/size costs of + // infrequent operations. + // NOTE(tucker): Maybe this should move to a subclass of CostModel. + void SuppressInfrequent(); + + bool is_global() const { return is_global_; } + + inline int Id(const Node* n) const { + if (is_global_) { + return n->cost_id(); + } else { + return n->id(); + } + } + + // Initializes cost model for 'g'. + void InitFromGraph(const Graph& g); + + // Merges costs from cm. + // REQUIRES: is_global_ is true for this and for "cm" + void MergeFromGlobal(const CostModel& cm); + + // Merges costs from "cm", which has been computed relative to "g". + // REQUIRES: is_global_ is true for this, and false for "cm". + void MergeFromLocal(const Graph& g, const CostModel& cm); + + void MergeFromStats(const NodeNameToCostIdMap& map, const StepStats& ss); + + // Sets the number of outputs of "node". + void SetNumOutputs(const Node* node, int num_outputs); + + // Records that "node" has executed "num_count" more times. + void RecordCount(const Node* node, int num_count); + + // Returns how many times "node" has been executed. + int32 TotalCount(const Node* node) const; + + // Records that "output_slot" of "node" has produced tensors of + // aggregated "bytes". + void RecordSize(const Node* node, int output_slot, Bytes bytes); + + // Returns total bytes of tensors produced by "node"s output slot. + Bytes TotalBytes(const Node* node, int output_slot) const; + + // Returns a prediction for the size of the tensor at the + // output_slot produced by one execution of "node". + Bytes SizeEstimate(const Node* node, int output_slot) const; + + // Records that Executions of "node" have taken "time" microseconds. + void RecordTime(const Node* node, Microseconds time); + + // Returns the total execution time for "node". + Microseconds TotalTime(const Node* node) const; + + // Returns a prediction for one execution of "node". + Microseconds TimeEstimate(const Node* node) const; + + // Check that an estimate is available for every OP node in graph. + void CheckInitialized(const Graph& graph) const; + + // Records the maximum size in bytes and optionally the corresponding shape of + // the tensor generated by "output_slot" of "node". If + void RecordMaxMemorySize(const Node* node, int output_slot, Bytes bytes, + const TensorShapeProto& tensor_shape, + const DataType& dtype); + + // Returns the maximum size in bytes of the tensor generated by "output_slot" + // of "node". + Bytes MaxMemorySize(const Node* node, int output_slot) const; + + // Returns the shape corresponding to the largest memory size of the tensor + // generated by "output_slot" of "node". + const TensorShapeProto& MaxMemoryShape(const Node* node, + int output_slot) const; + + // Returns the shape corresponding to the largest memory size of the tensor + // generated by "output_slot" of "node". + DataType MaxMemoryType(const Node* node, int output_slot) const; + + // Returns the size in bytes of temporary memory consumed by "node". + Bytes TempMemorySize(const Node* node) const; + + // Returns the size of persistent memory allocated by "node". + Bytes PersistentMemorySize(const Node* node) const; + + // Records memory stats such as temp momory and persistent memory. + void RecordMemoryStats(const Node* node, const MemoryStats& memory_stats); + + // Records the maximum execution time (in microseconds) of "node". + void RecordMaxExecutionTime(const Node* node, Microseconds time); + + // Returns the maximum execution time (in microseconds) of "node". + Microseconds MaxExecutionTime(const Node* node) const; + + // Record the unique id of the tensor generated by "output_slot" of "node". + // Any other tensor sharing the same id will be an alias, i.e. it will share + // the same underlying memory storage area. + void RecordAllocationId(const Node* node, int output_slot, int64 alloc_id); + + // Return the unique id of the tensor generated by "output_slot" of "node". + int64 AllocationId(const Node* node, int output_slot) const; + + bool IsPersistentTensor(const Node* node, int64 alloc_id) const; + + // Helper routines to encapsulate static estimation heuristics + + // Compute an estimate of the time to copy "b" bytes over the network, + // given a fixed cost of "network_latency_millis" milliseconds and + // an estimated bandwidth of "estimated_gbps" gigabits per second (note that + // this value is in gigabits, not gigabytes). + static Microseconds CopyTimeEstimate(Bytes b, double network_latency_millis, + double estimated_gbps); + static Microseconds ComputationTimeEstimate(int64 mathops); + + // Add this CostModel into the CostGraphDef. + void AddToCostGraphDef(const Graph* graph, CostGraphDef* cost_graph) const; + + // Write the contents of the CostModel to the INFO log. + void WriteSummaryToLog() const; + + // Increment the times that the cost model is updated. + void IncrementUpdateTimes(); + + // Get the times that the cost model is updated. + int32 GetUpdateTimes() const; + + private: + static Bytes MinTensorMemoryUsage(const TensorShapeProto& tensor_shape, + const DataType& dtype); + + const bool is_global_; + + // Resizes vectors so that they are large enough for "id" and id's outputs. + void Ensure(int id, int num_outputs); + + // Nodes and Edges whose count is < this value + // get type/byte estimates of 0. + int32 min_count_ = 0; + + // The number of times the cost model is updated. + int32 update_times_ = 0; + + // Number of times each Node has been executed. + std::vector count_; + // Cumulative execution time. + std::vector time_; + // Cumulative Bytes output on each channel. + std::vector> slot_bytes_; + + // Maximum execution time + std::vector max_exec_time_; + + // Maximum memory usage + struct MemUsage { + MemUsage() : temp_memory_size(0), persistent_memory_size(0) {} + + // TODO(yuefengz): temp_memory_size is not being used, remove it. + Bytes temp_memory_size; + Bytes persistent_memory_size; + + gtl::InlinedVector output_port_mem; + gtl::InlinedVector output_port_shape; + gtl::InlinedVector output_port_type; + }; + std::vector max_mem_usage_; + + std::vector> output_port_alloc_ids_; + + std::set persistent_alloc_ids_; + std::map> persistent_alloc_ids_by_devices_; + + TensorShapeProto unknown_shape_; + + TF_DISALLOW_COPY_AND_ASSIGN(CostModel); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPH_COSTMODEL_H_ diff --git a/linux/include/tensorflow/core/graph/default_device.h b/linux/include/tensorflow/core/graph/default_device.h new file mode 100644 index 000000000..f0f53c91f --- /dev/null +++ b/linux/include/tensorflow/core/graph/default_device.h @@ -0,0 +1,41 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPH_DEFAULT_DEVICE_H_ +#define TENSORFLOW_CORE_GRAPH_DEFAULT_DEVICE_H_ + +#include + +#include "tensorflow/core/framework/graph.pb.h" +#include "tensorflow/core/framework/node_def.pb.h" + +namespace tensorflow { +namespace graph { + +// Sets the default device for all nodes in graph_def to "device", +// only if not already set. +inline void SetDefaultDevice(const string& device, GraphDef* graph_def) { + for (int i = 0; i < graph_def->node_size(); ++i) { + auto node = graph_def->mutable_node(i); + if (node->device().empty()) { + node->set_device(device); + } + } +} + +} // namespace graph +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPH_DEFAULT_DEVICE_H_ diff --git a/linux/include/tensorflow/core/graph/edgeset.h b/linux/include/tensorflow/core/graph/edgeset.h new file mode 100644 index 000000000..0a1ee5a66 --- /dev/null +++ b/linux/include/tensorflow/core/graph/edgeset.h @@ -0,0 +1,232 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_GRAPH_EDGESET_H_ +#define TENSORFLOW_GRAPH_EDGESET_H_ + +#include +#include +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/types.h" + +#include "tensorflow/core/platform/logging.h" +namespace tensorflow { + +class Edge; + +// An unordered set of edges. Uses very little memory for small sets. +// Unlike std::set, EdgeSet does NOT allow mutations during iteration. +class EdgeSet { + public: + EdgeSet(); + ~EdgeSet(); + + typedef const Edge* key_type; + typedef const Edge* value_type; + typedef size_t size_type; + typedef ptrdiff_t difference_type; + + class const_iterator; + typedef const_iterator iterator; + + bool empty() const; + size_type size() const; + void clear(); + std::pair insert(value_type value); + size_type erase(key_type key); + + // Caller is not allowed to mutate the EdgeSet while iterating. + const_iterator begin() const; + const_iterator end() const; + + private: + // Up to kInline elements are stored directly in ptrs_ (nullptr means none). + // If ptrs_[0] == this then ptrs_[1] points to a set. + static const int kInline = 4; // Must be >= 2. + const void* ptrs_[kInline]; + + std::set* get_set() const { + if (ptrs_[0] == this) { + return static_cast*>(const_cast(ptrs_[1])); + } else { + return nullptr; + } + } + +// To detect mutations while iterating. +#ifdef NDEBUG + void RegisterMutation() {} +#else + uint32 mutations_ = 0; + void RegisterMutation() { mutations_++; } +#endif + + TF_DISALLOW_COPY_AND_ASSIGN(EdgeSet); +}; + +class EdgeSet::const_iterator { + public: + typedef typename EdgeSet::value_type value_type; + typedef const typename EdgeSet::value_type& reference; + typedef const typename EdgeSet::value_type* pointer; + typedef typename EdgeSet::difference_type difference_type; + typedef std::forward_iterator_tag iterator_category; + + const_iterator() {} + + const_iterator& operator++(); + const_iterator operator++(int /*unused*/); + const value_type* operator->() const; + value_type operator*() const; + bool operator==(const const_iterator& other) const; + bool operator!=(const const_iterator& other) const { + return !(*this == other); + } + + private: + friend class EdgeSet; + + void const* const* array_iter_ = nullptr; + typename std::set::const_iterator tree_iter_; + +#ifdef NDEBUG + inline void Init(const EdgeSet* e) {} + inline void CheckNoMutations() const {} +#else + inline void Init(const EdgeSet* e) { + owner_ = e; + init_mutations_ = e->mutations_; + } + inline void CheckNoMutations() const { + CHECK_EQ(init_mutations_, owner_->mutations_); + } + const EdgeSet* owner_ = nullptr; + uint32 init_mutations_ = 0; +#endif +}; + +inline EdgeSet::EdgeSet() { + for (int i = 0; i < kInline; i++) { + ptrs_[i] = nullptr; + } +} + +inline EdgeSet::~EdgeSet() { delete get_set(); } + +inline bool EdgeSet::empty() const { return size() == 0; } + +inline EdgeSet::size_type EdgeSet::size() const { + auto s = get_set(); + if (s) { + return s->size(); + } else { + size_t result = 0; + for (int i = 0; i < kInline; i++) { + if (ptrs_[i]) result++; + } + return result; + } +} + +inline void EdgeSet::clear() { + RegisterMutation(); + delete get_set(); + for (int i = 0; i < kInline; i++) { + ptrs_[i] = nullptr; + } +} + +inline EdgeSet::const_iterator EdgeSet::begin() const { + const_iterator ci; + ci.Init(this); + auto s = get_set(); + if (s) { + ci.tree_iter_ = s->begin(); + } else { + ci.array_iter_ = &ptrs_[0]; + } + return ci; +} + +inline EdgeSet::const_iterator EdgeSet::end() const { + const_iterator ci; + ci.Init(this); + auto s = get_set(); + if (s) { + ci.tree_iter_ = s->end(); + } else { + ci.array_iter_ = &ptrs_[size()]; + } + return ci; +} + +inline EdgeSet::const_iterator& EdgeSet::const_iterator::operator++() { + CheckNoMutations(); + if (array_iter_ != nullptr) { + ++array_iter_; + } else { + ++tree_iter_; + } + return *this; +} + +inline EdgeSet::const_iterator EdgeSet::const_iterator::operator++( + int /*unused*/) { + CheckNoMutations(); + const_iterator tmp = *this; + operator++(); + return tmp; +} + +// gcc's set and multiset always use const_iterator since it will otherwise +// allow modification of keys. +inline const EdgeSet::const_iterator::value_type* EdgeSet::const_iterator:: +operator->() const { + CheckNoMutations(); + if (array_iter_ != nullptr) { + return reinterpret_cast(array_iter_); + } else { + return tree_iter_.operator->(); + } +} + +// gcc's set and multiset always use const_iterator since it will otherwise +// allow modification of keys. +inline EdgeSet::const_iterator::value_type EdgeSet::const_iterator::operator*() + const { + CheckNoMutations(); + if (array_iter_ != nullptr) { + return static_cast(*array_iter_); + } else { + return *tree_iter_; + } +} + +inline bool EdgeSet::const_iterator::operator==( + const const_iterator& other) const { + DCHECK((array_iter_ == nullptr) == (other.array_iter_ == nullptr)) + << "Iterators being compared must be from same set that has not " + << "been modified since the iterator was constructed"; + CheckNoMutations(); + if (array_iter_ != nullptr) { + return array_iter_ == other.array_iter_; + } else { + return other.array_iter_ == nullptr && tree_iter_ == other.tree_iter_; + } +} + +} // namespace tensorflow + +#endif // TENSORFLOW_GRAPH_EDGESET_H_ diff --git a/linux/include/tensorflow/core/graph/gradients.h b/linux/include/tensorflow/core/graph/gradients.h new file mode 100644 index 000000000..ddfed084b --- /dev/null +++ b/linux/include/tensorflow/core/graph/gradients.h @@ -0,0 +1,58 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPH_GRADIENTS_H_ +#define TENSORFLOW_CORE_GRAPH_GRADIENTS_H_ + +#include "tensorflow/core/graph/graph.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/lib/gtl/array_slice.h" + +namespace tensorflow { + +// Represents the output of 'node' at 'index'. +struct NodeOut { + Node* node; + int index; + + // Returns the string name that represents the output of this node. + string name() const; + // Returns the data type of the output of this node. + DataType dtype() const; +}; + +// NOTE: This API is a work in progress and will likely be changing frequently. +// +// Given initial gradient-node outputs 'y_grad_node_outputs' (which compute the +// symbolic partial derivatives of some loss function 'L' w.r.t the node outputs +// 'y_node_outputs'), adds gradient nodes to 'graph' that compute the symbolic +// partial derivatives of 'L' w.r.t the node outputs 'x_node_outputs'. +// +// REQUIRES: Each node in 'x_node_outputs' to be unique, and so to have a single +// output (this restriction will be removed in a subsequent change). + +// TODO(andydavis) Add symbolic gradient support for general graphs (the current +// implementation only supports gradients for functions). In particular, +// the nodes in 'x_nodes' are currently restricted to have one output. + +Status AddSymbolicGradients(gtl::ArraySlice y_node_outputs, + gtl::ArraySlice x_node_outputs, + gtl::ArraySlice y_grad_node_outputs, + std::vector* x_grad_node_outputs, + Graph* graph); + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPH_GRADIENTS_H_ diff --git a/linux/include/tensorflow/core/graph/graph.h b/linux/include/tensorflow/core/graph/graph.h new file mode 100644 index 000000000..52e9f23a7 --- /dev/null +++ b/linux/include/tensorflow/core/graph/graph.h @@ -0,0 +1,852 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// A Graph describes a set of computations that are to be +// performed, as well as the dependencies between those +// computations. The basic model is a DAG (directed acyclic graph) with +// * internal nodes representing computational operations to be performed; +// * edges represent dependencies, indicating the target may only be +// executed once the source has completed; and +// * predefined "source" (start) and "sink" (finish) nodes -- the source +// should be the only node that doesn't depend on anything, and the sink +// should be the only node that nothing depends on. +// +// Note: Node ids are intended to be relatively dense in the +// 0..max_id range, but there may be gaps since ids won't be reused. +// +// Note: Some dependencies between operations are due to one operation +// consuming the output of another. In fact operations can produce +// multiple outputs and consume multiple inputs, and some +// optimizations will care about which specific outputs are connected +// to which specific inputs. We therefore represent data dependency +// between output O of layer A and input I of layer B using +// "input index" and "output index" labels per edge. + +#ifndef TENSORFLOW_CORE_GRAPH_GRAPH_H_ +#define TENSORFLOW_CORE_GRAPH_GRAPH_H_ + +#include +#include +#include +#include "tensorflow/core/framework/function.h" +#include "tensorflow/core/framework/op.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/graph/edgeset.h" +#include "tensorflow/core/lib/core/arena.h" +#include "tensorflow/core/lib/core/refcount.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/lib/gtl/iterator_range.h" +#include "tensorflow/core/platform/logging.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +class Edge; +class EdgeSetTest; +class Graph; +class GraphDef; +class Node; +class VersionDef; +class WhileContext; + +class NeighborIter; // Declared below +class NodeIter; // Declared below +class NodeProperties; // Defined in .cc + +class Node { + public: + string DebugString() const; + int id() const { return id_; } + int cost_id() const { return cost_id_; } + const string& name() const; + const string& type_string() const; + + // def() provides the NodeDef the user supplied, but the specifics + // of this Node may have changed due to placement, optimization, etc. + // In particular: + // * def().name() will match name(); + // * def().op() will match type_string() and op_def().name(); + // * def().input() is not reliable, use "in_edges()" below instead; + // * def().device() is the "user's requested device" and may not match + // the actual assigned device, see assigned_device_name() below; + // * def().attr() is authoritative. + // TODO(irving): Replace with NodeInfo. + const NodeDef& def() const; + const OpDef& op_def() const; + + // input and output types + int32 num_inputs() const; + DataType input_type(int32 i) const; + const DataTypeVector& input_types() const; + + int32 num_outputs() const; + DataType output_type(int32 o) const; + const DataTypeVector& output_types() const; + + // The device requested by the user. For the actual assigned device, + // use assigned_device_name() below. + const string& requested_device() const; + + // This changes the user requested device but not necessarily the device that + // on which the operation will run. + void set_requested_device(const string& device); + + // This gives the device the runtime has assigned this node to. If + // you want the device the user requested, use def().device() instead. + // TODO(josh11b): Validate that the assigned_device, if not empty: + // fully specifies a device, and satisfies def().device(). + // TODO(josh11b): Move assigned_device_name outside of Node into a + // NodeId->DeviceName map. + const string& assigned_device_name() const; + void set_assigned_device_name(const string& device_name); + bool has_assigned_device_name() const { + return assigned_device_name_index_ > 0; + } + int assigned_device_name_index() const { return assigned_device_name_index_; } + void set_assigned_device_name_index(int index); + + // Read only access to attributes + AttrSlice attrs() const; + + // Inputs requested by the NodeDef. For the actual inputs, use in_edges. + const protobuf::RepeatedPtrField& requested_inputs() const; + + // Get the neighboring nodes via edges either in or out of this node. This + // includes control edges. + gtl::iterator_range in_nodes() const; + gtl::iterator_range out_nodes() const; + const EdgeSet& in_edges() const { return in_edges_; } + const EdgeSet& out_edges() const { return out_edges_; } + + // Node type helpers. + bool IsSource() const { return id() == 0; } + bool IsSink() const { return id() == 1; } + // Anything other than the special Source & Sink nodes. + bool IsOp() const { return id() > 1; } + + // Node class helpers + bool IsSwitch() const { return class_ == NC_SWITCH; } + bool IsMerge() const { return class_ == NC_MERGE; } + bool IsEnter() const { return class_ == NC_ENTER; } + bool IsExit() const { return class_ == NC_EXIT; } + bool IsNextIteration() const { return class_ == NC_NEXT_ITERATION; } + bool IsLoopCond() const { return class_ == NC_LOOP_COND; } + bool IsControlTrigger() const { return class_ == NC_CONTROL_TRIGGER; } + bool IsSend() const { return class_ == NC_SEND || class_ == NC_HOST_SEND; } + bool IsRecv() const { return class_ == NC_RECV || class_ == NC_HOST_RECV; } + bool IsConstant() const { return class_ == NC_CONSTANT; } + bool IsVariable() const { return class_ == NC_VARIABLE; } + bool IsIdentity() const { return class_ == NC_IDENTITY; } + bool IsGetSessionHandle() const { return class_ == NC_GET_SESSION_HANDLE; } + bool IsGetSessionTensor() const { return class_ == NC_GET_SESSION_TENSOR; } + bool IsDeleteSessionTensor() const { + return class_ == NC_DELETE_SESSION_TENSOR; + } + bool IsControlFlow() const { + return (class_ != NC_OTHER) && // Fast path + (IsSwitch() || IsMerge() || IsEnter() || IsExit() || + IsNextIteration()); + } + bool IsHostSend() const { return class_ == NC_HOST_SEND; } + bool IsHostRecv() const { return class_ == NC_HOST_RECV; } + bool IsScopedAllocator() const { return class_ == NC_SCOPED_ALLOCATOR; } + bool IsCollective() const { return class_ == NC_COLLECTIVE; } + + bool IsMetadata() const { return class_ == NC_METADATA; } + + template + void AddAttr(const string& name, const T& val) { + SetAttrValue(val, AddAttrHelper(name)); + } + + void ClearAttr(const string& name); + + // Returns into '*e' the edge connecting to the 'idx' input of this Node. + Status input_edge(int idx, const Edge** e) const; + + // Returns into '*edges' the input data edges of this Node, indexed by input + // number. Does not return control edges. + Status input_edges(std::vector* edges) const; + + // Returns into '*n' the node that has an output connected to the + // 'idx' input of this Node. + Status input_node(int idx, const Node** n) const; + Status input_node(int idx, Node** n) const; + + WhileContext* while_ctx() const { return while_ctx_; } + void set_while_ctx(WhileContext* while_ctx) { + DCHECK(IsExit()); + DCHECK(while_ctx_ == nullptr); + while_ctx_ = while_ctx; + } + + private: + friend class Graph; + Node(); + + NodeProperties* properties() const { return props_.get(); } + + void Initialize(int id, int cost_id, std::shared_ptr props); + + // Releases memory from props_, in addition to restoring *this to its + // uninitialized state. + void Clear(); + + // Make a copy of the Node's props_ if props_ is shared with + // other nodes. This must be called before mutating properties, + // e.g. in AddAttr. + void MaybeCopyOnWrite(); + + AttrValue* AddAttrHelper(const string& name); + + // A set of mutually exclusive classes for different kinds of nodes, + // class_ is initialized in the Node::Initialize routine based on the + // node's type_string(). + enum NodeClass { + NC_UNINITIALIZED, + NC_SWITCH, + NC_MERGE, + NC_ENTER, + NC_EXIT, + NC_NEXT_ITERATION, + NC_LOOP_COND, + NC_CONTROL_TRIGGER, + NC_SEND, + NC_HOST_SEND, + NC_RECV, + NC_HOST_RECV, + NC_CONSTANT, + NC_VARIABLE, + NC_IDENTITY, + NC_GET_SESSION_HANDLE, + NC_GET_SESSION_TENSOR, + NC_DELETE_SESSION_TENSOR, + NC_METADATA, + NC_SCOPED_ALLOCATOR, + NC_COLLECTIVE, + NC_OTHER // Not a special kind of node + }; + + static const std::unordered_map& kNodeClassTable; + + static NodeClass GetNodeClassForOp(const string& ts); + + int id_; // -1 until Initialize() is called + int cost_id_; // -1 if there is no corresponding cost accounting node + NodeClass class_; + + EdgeSet in_edges_; + EdgeSet out_edges_; + + // NOTE(skyewm): inheriting from core::RefCounted may have a slight + // performance benefit over using shared_ptr, at the cost of manual ref + // counting + std::shared_ptr props_; + + // Index within Graph::device_names_ of the name of device assigned + // to perform this computation. + int assigned_device_name_index_; + + // A back-pointer to the Graph that owns this node. Currently, this exists + // solely to allow Node::[set_]assigned_device_name() to work. However, if all + // callers of Node::[set_]assigned_device_name() are modified to use the + // equivalent methods defined directly on Graph, then we can remove this + // field and reclaim that memory. + Graph* graph_; + + // Set if this is an exit node of a while loop with an associated + // WhileContext. Otherwise null. (This is only set for exit nodes because + // they're the first nodes of a loop encountered while creating the gradient + // graph. Exit nodes that are part of while loop gradient graphs will not have + // this set.) + WhileContext* while_ctx_; + + TF_DISALLOW_COPY_AND_ASSIGN(Node); +}; + +// Represents an input of a node, i.e., the `index`-th input to `node`. +struct InputTensor { + const Node* node; + int index; + + InputTensor(const Node* n, int i) : node(n), index(i) {} + InputTensor() : node(nullptr), index(0) {} + + // Returns true if this InputTensor is identical to 'other'. Nodes are + // compared using pointer equality. + bool operator==(const InputTensor& other) const; + + // A hash function for InputTensors. Nodes are hashed based on their pointer + // value. + struct Hash { + uint64 operator()(InputTensor const& s) const; + }; +}; + +// Represents an output of a node, i.e., the `index`-th output of `node`. Note +// that a single `OutputTensor` can correspond to multiple `Edge`s if the output +// is consumed by multiple destination nodes. +struct OutputTensor { + const Node* node; + int index; + + OutputTensor(const Node* n, int i) : node(n), index(i) {} + OutputTensor() : node(nullptr), index(0) {} + + // Returns true if this OutputTensor is identical to 'other'. Nodes are + // compared using pointer equality. + bool operator==(const OutputTensor& other) const; + + // A hash function for OutputTensors. Nodes are hashed based on their pointer + // value. + struct Hash { + uint64 operator()(OutputTensor const& s) const; + }; +}; + +class Edge { + public: + Node* src() const { return src_; } + Node* dst() const { return dst_; } + int id() const { return id_; } + + // Return the index of the source output that produces the data + // carried by this edge. The special value kControlSlot is used + // for control dependencies. + int src_output() const { return src_output_; } + + // Return the index of the destination input that consumes the data + // carried by this edge. The special value kControlSlot is used + // for control dependencies. + int dst_input() const { return dst_input_; } + + // Return true iff this is an edge that indicates a control-flow + // (as opposed to a data-flow) dependency. + bool IsControlEdge() const; + + string DebugString() const; + + private: + Edge() {} + + friend class EdgeSetTest; + friend class Graph; + Node* src_; + Node* dst_; + int id_; + int src_output_; + int dst_input_; +}; + +// Allows for iteration of the edges of a Graph, by iterating the underlying +// Graph.edges_ vector while skipping over null entries. +class GraphEdgesIterable { + private: + const std::vector& edges_; + + public: + explicit GraphEdgesIterable(const std::vector& edges) + : edges_(edges) {} + + typedef Edge* value_type; + + class const_iterator { + private: + // The underlying iterator. + std::vector::const_iterator iter_; + + // The end of the underlying iterator. + std::vector::const_iterator end_; + + // Advances iter_ until it reaches a non-null item, or reaches the end. + void apply_filter() { + while (iter_ != end_ && *iter_ == nullptr) { + ++iter_; + } + } + + public: + const_iterator(std::vector::const_iterator iter, + std::vector::const_iterator end) + : iter_(iter), end_(end) { + apply_filter(); + } + + bool operator==(const const_iterator& other) const { + return iter_ == other.iter_; + } + + bool operator!=(const const_iterator& other) const { + return iter_ != other.iter_; + } + + // This is the prefix increment operator (++x), which is the operator + // used by C++ range iteration (for (x : y) ...). We intentionally do not + // provide a postfix increment operator. + const_iterator& operator++() { + ++iter_; + apply_filter(); + return *this; + } + + value_type operator*() { return *iter_; } + }; + + const_iterator begin() { + return const_iterator(edges_.begin(), edges_.end()); + } + const_iterator end() { return const_iterator(edges_.end(), edges_.end()); } +}; + +// Thread compatible but not thread safe. +class Graph { + public: + // Constructs a graph with a single SOURCE (always id kSourceId) and a + // single SINK (always id kSinkId) node, and an edge from SOURCE->SINK. + // + // The graph can hold ops found in registry. `registry`s lifetime must be at + // least that of the constructed graph's. + explicit Graph(const OpRegistryInterface* registry); + + // Constructs a graph with a single SOURCE (always id kSourceId) and a + // single SINK (always id kSinkId) node, and an edge from SOURCE->SINK. + // + // The graph can hold ops found in `flib_def`. Unlike the constructor taking + // an OpRegistryInterface, this constructor copies the function definitions in + // `flib_def` so its lifetime may be shorter than that of the graph's. The + // OpRegistryInterface backing `flib_def` must still have the lifetime of the + // graph though. + explicit Graph(const FunctionLibraryDefinition& flib_def); + + ~Graph(); + + static const int kControlSlot; + + // The GraphDef version range of this graph (see graph.proto). + const VersionDef& versions() const; + void set_versions(const VersionDef& versions); + + // Adds a new node to this graph, and returns it. Infers the Op and + // input/output types for the node. *this owns the returned instance. + // Returns nullptr and sets *status on error. + Node* AddNode(const NodeDef& node_def, Status* status); + + // Copies *node, which may belong to another graph, to a new node, + // which is returned. Does not copy any edges. *this owns the + // returned instance. + Node* CopyNode(const Node* node); + + // Removes a node from this graph, including all edges from or to it. + // *node should not be accessed after calling this function. + // REQUIRES: node->IsOp() + void RemoveNode(Node* node); + + // Adds an edge that connects the xth output of `source` to the yth input of + // `dest` and returns it. Does not update dest's NodeDef. + const Edge* AddEdge(Node* source, int x, Node* dest, int y); + + // Adds a control edge (no data flows along this edge) that connects `source` + // to `dest`. If `dest`s NodeDef is missing the corresponding control input, + // adds the control input. + // + // If such a control edge already exists and `allow_duplicates` is false, no + // edge is added and the function returns nullptr. Otherwise the edge is + // unconditionally created and returned. The NodeDef is not updated if + // `allow_duplicates` is true. + // TODO(skyewm): // TODO(skyewm): allow_duplicates is needed only by + // graph_partition.cc. Figure out if we can do away with it. + const Edge* AddControlEdge(Node* source, Node* dest, + bool allow_duplicates = false); + + // Removes edge from the graph. Does not update the destination node's + // NodeDef. + // REQUIRES: The edge must exist. + void RemoveEdge(const Edge* edge); + + // Removes control edge `edge` from the graph. Note that this also updates + // the corresponding NodeDef to reflect the change. + // REQUIRES: The control edge must exist. + void RemoveControlEdge(const Edge* e); + // Updates the input to a node. The existing edge to `dst` is removed and an + // edge from `new_src` to `dst` is created. The NodeDef associated with `dst` + // is also updated. + Status UpdateEdge(Node* new_src, int new_src_index, Node* dst, int dst_index); + + // Adds the function and gradient definitions in `fdef_lib` to this graph's op + // registry. Ignores duplicate functions, and returns a bad status if an + // imported function differs from an existing function or op with the same + // name. + Status AddFunctionLibrary(const FunctionDefLibrary& fdef_lib); + + // The number of live nodes in the graph. + // + // Because nodes can be removed from the graph, num_nodes() is often + // smaller than num_node_ids(). If one needs to create an array of + // nodes indexed by node ids, num_node_ids() should be used as the + // array's size. + int num_nodes() const { return num_nodes_; } + + // The number of live nodes in the graph, excluding the Source and Sink nodes. + int num_op_nodes() const { + DCHECK_GE(num_nodes_, 2); + return num_nodes_ - 2; + } + + // The number of live edges in the graph. + // + // Because edges can be removed from the graph, num_edges() is often + // smaller than num_edge_ids(). If one needs to create an array of + // edges indexed by edge ids, num_edge_ids() should be used as the + // array's size. + int num_edges() const { return num_edges_; } + + // Serialize the nodes starting at `from_node_id` to a GraphDef. + void ToGraphDefSubRange(GraphDef* graph_def, int from_node_id) const; + + // Serialize to a GraphDef. + void ToGraphDef(GraphDef* graph_def) const; + + // This version can be called from debugger to inspect the graph content. + // Use the previous version outside debug context for efficiency reasons. + // + // Note: We do not expose a DebugString() API, since GraphDef.DebugString() is + // not defined in some TensorFlow builds. + GraphDef ToGraphDefDebug() const; + + // Generate new node name with the specified prefix that is unique + // across this graph. + string NewName(StringPiece prefix); + + // Access to the list of all nodes. Example usage: + // for (Node* node : graph.nodes()) { ... } + gtl::iterator_range nodes() const; + + // Access to the list of all nodes, excluding the Source and Sink nodes. + gtl::iterator_range op_nodes() const; + + // Returns one more than the maximum id assigned to any node. + int num_node_ids() const { return nodes_.size(); } + + // Returns the node associated with an id, or nullptr if no node + // with that id (the node with that id was removed and the id has + // not yet been re-used). *this owns the returned instance. + // REQUIRES: 0 <= id < num_node_ids(). + Node* FindNodeId(int id) const { return nodes_[id]; } + + // Returns one more than the maximum id assigned to any edge. + int num_edge_ids() const { return edges_.size(); } + + // Returns the Edge associated with an id, or nullptr if no edge + // with that id (the node with that id was removed and the id has + // not yet been re-used). *this owns the returned instance. + // REQUIRES: 0 <= id < num_node_ids(). + const Edge* FindEdgeId(int id) const { return edges_[id]; } + + // Access to the set of all edges. Example usage: + // for (const Edge* e : graph.edges()) { ... } + GraphEdgesIterable edges() const { return GraphEdgesIterable(edges_); } + + // The pre-defined nodes. + enum { kSourceId = 0, kSinkId = 1 }; + Node* source_node() const { return FindNodeId(kSourceId); } + Node* sink_node() const { return FindNodeId(kSinkId); } + + const OpRegistryInterface* op_registry() const { return &ops_; } + const FunctionLibraryDefinition& flib_def() const { return ops_; } + + void CheckDeviceNameIndex(int index) { + DCHECK_GE(index, 0); + DCHECK_LT(index, static_cast(device_names_.size())); + } + + int InternDeviceName(const string& device_name); + + const string& get_assigned_device_name(const Node& node) const { + return device_names_[node.assigned_device_name_index()]; + } + + void set_assigned_device_name_index(Node* node, int device_name_index) { + CheckDeviceNameIndex(device_name_index); + node->assigned_device_name_index_ = device_name_index; + } + + void set_assigned_device_name(Node* node, const string& device_name) { + node->assigned_device_name_index_ = InternDeviceName(device_name); + } + + // Returns OK if `node` is non-null and belongs to this graph + Status IsValidNode(const Node* node) const; + + // Returns OK if IsValidNode(`node`) and `idx` is less than + // node->num_outputs() + Status IsValidOutputTensor(const Node* node, int idx) const; + + // Returns OK if IsValidNode(`node`) and `idx` is less than + // node->num_inputs() + Status IsValidInputTensor(const Node* node, int idx) const; + + // Create and return a new WhileContext owned by this graph. This is called + // when a new while loop is created. `frame_name` must be unique among + // WhileContexts in this graph. + Status AddWhileContext(StringPiece frame_name, std::vector enter_nodes, + std::vector exit_nodes, + OutputTensor cond_output, + std::vector body_inputs, + std::vector body_outputs, + WhileContext** result); + + // TODO(josh11b): uint64 hash() const; + + private: + // If cost_node is non-null, then cost accounting (in CostModel) + // will be associated with that node rather than the new one being + // created. + // + // Ownership of the returned Node is not transferred to caller. + Node* AllocateNode(std::shared_ptr props, + const Node* cost_node); + void ReleaseNode(Node* node); + + // Registry of all known ops, including functions. + FunctionLibraryDefinition ops_; + + // GraphDef versions + const std::unique_ptr versions_; + + // Allocator which will give us good locality. + core::Arena arena_; + + // Map from node ids to allocated nodes. nodes_[id] may be nullptr if + // the node with that id was removed from the graph. + std::vector nodes_; + + // Number of nodes alive. + int64 num_nodes_ = 0; + + // Map from edge ids to allocated edges. edges_[id] may be nullptr if + // the edge with that id was removed from the graph. + std::vector edges_; + + // The number of entries in edges_ that are not nullptr. + int num_edges_ = 0; + + // Allocated but free nodes and edges. + std::vector free_nodes_; + std::vector free_edges_; + + // For generating unique names. + int name_counter_ = 0; + + // In most graphs, the number of unique values used for the + // Node::assigned_device_name() property is quite small. If the graph is + // large, then this duplication of values can consume a significant amount of + // memory. Instead, we represent the same information using an interning + // table, which consists of a vector of unique strings (device_names_), as + // well a map (device_names_map_) from unique strings to indices within the + // unique string table. + // + // The InternDeviceName() method handles adding a new entry into the table, + // or locating the index of an existing entry. + // + // The fact that Node::assigned_device_name() is implemented using an + // interning table is intentionally public. This allows algorithms that + // frequently access this field to do so efficiently, especially for the case + // where the assigned_device_name of one Node is copied directly from that + // of another Node. + + // A table of the unique assigned device names. Indices do NOT correspond + // to node IDs. Index 0 is always the empty string. + std::vector device_names_; + + // Maps unique device names to indices within device_names_[i]. + std::unordered_map device_names_map_; + + // All the while contexts owned by this graph, keyed by frame name, + // corresponding to all the while loops contained in this graph (including + // nested loops). The stored contexts are usually accessed via + // AddWhileContext() or Node::while_ctx(), but this manages the lifetime. + std::map while_ctxs_; + + TF_DISALLOW_COPY_AND_ASSIGN(Graph); +}; + +// TODO(josh11b): We may want to support keeping an index on various +// node/edge attributes in a graph, particularly node names. + +// Helper routines + +inline bool IsSource(const Node* node) { return node->IsSource(); } +inline bool IsSink(const Node* node) { return node->IsSink(); } +inline bool IsSwitch(const Node* node) { return node->IsSwitch(); } +inline bool IsMerge(const Node* node) { return node->IsMerge(); } +inline bool IsEnter(const Node* node) { return node->IsEnter(); } +inline bool IsExit(const Node* node) { return node->IsExit(); } +inline bool IsNextIteration(const Node* n) { return n->IsNextIteration(); } +inline bool IsLoopCond(const Node* node) { return node->IsLoopCond(); } +inline bool IsControlTrigger(const Node* n) { return n->IsControlTrigger(); } +inline bool IsSend(const Node* node) { return node->IsSend(); } +inline bool IsRecv(const Node* node) { return node->IsRecv(); } +inline bool IsHostSend(const Node* node) { return node->IsHostSend(); } +inline bool IsHostRecv(const Node* node) { return node->IsHostRecv(); } + +// True for Nodes that mediate the transfer of values between processes. +inline bool IsTransferNode(const Node* n) { return IsSend(n) || IsRecv(n); } + +inline bool IsConstant(const Node* node) { return node->IsConstant(); } +inline bool IsVariable(const Node* node) { return node->IsVariable(); } +inline bool IsIdentity(const Node* node) { return node->IsIdentity(); } + +// Returns true iff 'n' is a control flow node. +inline bool IsControlFlow(const Node* n) { return n->IsControlFlow(); } + +// Returns true if the node only depends on its input's metadata +// (shape). Specifically, returns true for "Size", "Shape" and "Rank" ops. +inline bool IsMetadata(const Node* n) { return n->IsMetadata(); } + +inline bool IsScopedAllocator(const Node* n) { return n->IsScopedAllocator(); } + +inline bool IsHostMemoryPreserving(const Node* node) { + return IsIdentity(node) || IsControlFlow(node); +} + +// Iterator for stepping through the nodes of a graph. +class NodeIter { + public: + NodeIter(const Graph* graph, int id); + bool operator==(const NodeIter& rhs); + bool operator!=(const NodeIter& rhs); + void operator++(); + Node* operator*(); + Node* operator->(); + + private: + // Invariant: id_ == graph_->num_node_ids() || graph_->FindId(id_) != nullptr + const Graph* graph_; + int id_; +}; + +// Iterator for stepping through the neighbors of a node. +class NeighborIter { + public: + NeighborIter(EdgeSet::const_iterator iter, bool incoming); + bool operator==(const NeighborIter& rhs); + bool operator!=(const NeighborIter& rhs); + void operator++(); + Node* operator*(); + Node* operator->(); + + private: + EdgeSet::const_iterator iter_; + bool incoming_; +}; + +// IMPLEMENTATION DETAILS, PLEASE IGNORE + +inline NodeIter::NodeIter(const Graph* graph, int id) + : graph_(graph), id_(id) {} + +inline bool NodeIter::operator==(const NodeIter& rhs) { + DCHECK(graph_ == rhs.graph_); + return id_ == rhs.id_; +} + +inline bool NodeIter::operator!=(const NodeIter& rhs) { + return !(*this == rhs); +} + +inline void NodeIter::operator++() { + while (1) { + DCHECK_LE(id_, graph_->num_node_ids()); + ++id_; + if (id_ >= graph_->num_node_ids() || graph_->FindNodeId(id_) != nullptr) { + return; + } + } +} + +inline Node* NodeIter::operator*() { return graph_->FindNodeId(id_); } + +inline Node* NodeIter::operator->() { return graph_->FindNodeId(id_); } + +inline NeighborIter::NeighborIter(EdgeSet::const_iterator iter, bool incoming) + : iter_(iter), incoming_(incoming) {} + +inline bool NeighborIter::operator==(const NeighborIter& rhs) { + return iter_ == rhs.iter_ && incoming_ == rhs.incoming_; +} + +inline bool NeighborIter::operator!=(const NeighborIter& rhs) { + return !(*this == rhs); +} + +inline void NeighborIter::operator++() { ++iter_; } + +inline Node* NeighborIter::operator*() { + const Edge* e = *iter_; + return incoming_ ? e->src() : e->dst(); +} + +inline Node* NeighborIter::operator->() { + const Edge* e = *iter_; + return incoming_ ? e->src() : e->dst(); +} + +inline bool Edge::IsControlEdge() const { + // Note that if either src_output_ or dst_input_ is kControlSlot, + // so is the other one (AddEdge checks this). + return src_output_ == Graph::kControlSlot; +} + +inline gtl::iterator_range Graph::nodes() const { + // Note that NodeId 0 is always valid since we don't let the source + // node be removed from the graph. + return gtl::make_range(NodeIter(this, 0), NodeIter(this, num_node_ids())); +} + +inline gtl::iterator_range Graph::op_nodes() const { + // Note that NodeId 0 is always valid since we don't let the source + // node be removed from the graph. + // + // The current implementation of Graph maintains the invariant that the + // first two nodes are the source and sink nodes, and all other nodes are op + // nodes. This method (op_nodes()) relies on this invariant. + NodeIter begin(this, 0); + NodeIter end(this, num_node_ids()); + if (begin != end) { + ++begin; + } + if (begin != end) { + ++begin; + } + return gtl::make_range(begin, end); +} + +inline void Node::set_assigned_device_name_index(int index) { + graph_->CheckDeviceNameIndex(index); + assigned_device_name_index_ = index; +} + +inline void Node::set_assigned_device_name(const string& device_name) { + graph_->set_assigned_device_name(this, device_name); +} + +inline const string& Node::assigned_device_name() const { + return graph_->get_assigned_device_name(*this); +} + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPH_GRAPH_H_ diff --git a/linux/include/tensorflow/core/graph/graph_constructor.h b/linux/include/tensorflow/core/graph/graph_constructor.h new file mode 100644 index 000000000..f6e41faf9 --- /dev/null +++ b/linux/include/tensorflow/core/graph/graph_constructor.h @@ -0,0 +1,189 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPH_GRAPH_CONSTRUCTOR_H_ +#define TENSORFLOW_CORE_GRAPH_GRAPH_CONSTRUCTOR_H_ + +#include "tensorflow/core/framework/graph.pb.h" +#include "tensorflow/core/graph/graph.h" +#include "tensorflow/core/graph/tensor_id.h" +#include "tensorflow/core/lib/core/status.h" + +namespace tensorflow { +class ShapeRefiner; + +// Construct a Graph *g out of a GraphDef gdef. Returns non-OK on +// error, in which case *g is left in an incomplete state. +// +// *g is expected to be an empty graph (with no more than a source and sink +// nodes) when provided to ConvertGraphDefToGraph. To enhance an existing Graph, +// see ImportGraphDef. +struct GraphConstructorOptions { + GraphConstructorOptions() {} + + // If true, allows internal ops in the GraphDef. + bool allow_internal_ops = false; + + // If true, the graph def is expected to have fully specified + // devices for all nodes. A node in the resulting graph "g" has the + // device name set accordingly. + // + // TODO(zhifengc): if possible, consider removing this option. + bool expect_device_spec = false; +}; +extern Status ConvertGraphDefToGraph(const GraphConstructorOptions& opts, + const GraphDef& gdef, Graph* g); + +// Same as ConvertGraphDefToGraph, but takes just nodes. Used by function +// instantiation. +// TODO(irving): This will turn into std::vector soon. +extern Status ConvertNodeDefsToGraph(const GraphConstructorOptions& opts, + gtl::ArraySlice nodes, Graph* g); + +// Options for calling ImportGraphDef(). +struct ImportGraphDefOptions { + ImportGraphDefOptions() + : uniquify_names(false), + uniquify_prefix(false), + skip_mapped_nodes(false), + validate_shape(true) {} + + // Name prefix to use for nodes imported from the GraphDef. For example, if + // prefix="animals" and GraphDef contains a node "bunny" then the node will be + // named "animals/bunny" in *g. Must not be already used as a node name or + // prefix in the graph. + string prefix; + + // If true, imported node names will be modified if their name already exists + // in the graph. If false, conflicting names will be treated as an error. Note + // that this option has no effect if `prefix` is specified, since `prefix` + // will guarantee all node names are unique. + bool uniquify_names; + + // If true, `prefix` will be modified if it already exists as a node name or + // prefix in the graph. If false, a conflicting prefix will be treated as an + // error. This option has no effect if `prefix` isn't specified. + bool uniquify_prefix; + + // Maps tensors in `gdef` to existing tensors in `g`. Inputs in `gdef` + // corresponding to `input_map` keys will be remapped to the nodes in `g` + // corresponding to the values. + // + // Keys should not include `prefix`, i.e., a key ID's name should be the name + // as it originally appears in `gdef`. + // + // If this is non-empty, ImportGraphDef must be called with the shape refiner + // used to create the existing nodes referenced in `input_map`. + // TODO(skyewm): can we remove this requirement? How do we access the original + // shape refiner? + std::map input_map; + + // If true, nodes that will have all output edges removed because of + // overrides in `input_map` will not be imported. + bool skip_mapped_nodes; + + // The names of existing nodes in `g` that the imported graph should have + // control dependencies on. + // + // Note that to avoid creating many redundant control edges, ImportGraphDef() + // won't add control edges to nodes that will inherit the dependencies from + // other nodes in `gdef`. + std::vector control_dependencies; + + // Tensors in `gdef` that will be returned via the ImportGraphDefResults + // output parameter of `ImportGraphDef()`. If this list is non-empty, the + // caller must pass a results object to `ImportGraphDef()`. The + // `return_tensors` field will be populated with the imported nodes in `g`. + // + // Entries should not include `prefix`, i.e., each ID's name should be the + // name as it originally appears in `gdef`. + // + // If this contains a tensor that's also being remapped via `input_map`, the + // corresponding existing tensor in `g` will be returned. + std::vector return_tensors; + + // The names of nodes in `gdef` that will be returned via the + // ImportGraphDefResults output parameter of `ImportGraphDef()`. If this list + // is non-empty, the caller must pass a results object to + // `ImportGraphDef()`. The `return_nodes` field will be populated with the + // imported nodes in `g`. + // + // Entries should not include `prefix`, i.e., each node's name should be the + // name as it originally appears in `gdef`. + // + // Unlike `return_tensors`, `input_map` has no effect on the nodes + // returned. `return_nodes` must be empty if `skip_mapped_nodes` is true. + // TODO(skyewm): make this work with `skip_mapped_nodes` if there's a need. + std::vector return_nodes; + + // If true, checks that all colocation constraints are nodes in the GraphDef. + bool validate_colocation_constraints = true; + + // If false skips shape validation. + bool validate_shape; + + // TODO(ashankar): Enable handling of GraphDefs produced by newer binaries + // with ops that are not defined in the binary calling ImportGraphDef. + // Similar to the producer_op_list argument to import_graph_def in the + // python API. +}; + +// Optional results that may be returned by ImportGraphDef. +struct ImportGraphDefResults { + // The requested tensors associated with + // ImportGraphDefOptions::return_tensors. Note that the index may be different + // than the requested index if the returned tensor has been remapped according + // to `input_map`. + typedef int Index; + std::vector> return_tensors; + + // The requested nodes associated with ImportGraphDefOptions::return_nodes. + std::vector return_nodes; + + // Keys in ImportGraphDefOptions::input_map that don't appear in `gdef` and + // weren't used as an input to any node in `gdef`. These keys are likely due + // to typos, and callers may wish to treat their existence as an error. + std::vector missing_unused_input_map_keys; +}; + +// Adds the graph in GraphDef `gdef` into an existing Graph `*g`. +// +// On error, returns non-OK and leaves `*g` unmodified. +// +// `refiner` can be null. It should be non-null if the caller +// intends to add additional nodes to the graph after the import. This +// allows the caller to validate shapes of those nodes (since +// ShapeRefiner::AddNode must be called in topological order). +// +// `results` must be non-null if `opts.return_tensors` or `opts.result_nodes` is +// non-empty. It can also be set to fetch the unused input map keys. If it's +// non-null, all the vector fields must be empty. +// +// TODO(ashankar): Push this mechanism and get rid of Session::Extend() +// as a means of enhancing an existing Graph. +extern Status ImportGraphDef(const ImportGraphDefOptions& opts, + const GraphDef& gdef, Graph* g, + ShapeRefiner* refiner, + ImportGraphDefResults* results = nullptr); + +// Make a copy of "src" into "*dest". +// +// REQUIRES: "*dest" is a freshly allocated graph without any nodes or edges +// other than the implicit Source/Sink nodes. +extern void CopyGraph(const Graph& src, Graph* dest); + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPH_GRAPH_CONSTRUCTOR_H_ diff --git a/linux/include/tensorflow/core/graph/graph_def_builder.h b/linux/include/tensorflow/core/graph/graph_def_builder.h new file mode 100644 index 000000000..400d8b6c8 --- /dev/null +++ b/linux/include/tensorflow/core/graph/graph_def_builder.h @@ -0,0 +1,206 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPH_GRAPH_DEF_BUILDER_H_ +#define TENSORFLOW_CORE_GRAPH_GRAPH_DEF_BUILDER_H_ + +#include +#include "tensorflow/core/framework/graph.pb.h" +#include "tensorflow/core/framework/op.h" +#include "tensorflow/core/graph/graph.h" +#include "tensorflow/core/graph/node_builder.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/lib/core/stringpiece.h" +#include "tensorflow/core/lib/gtl/array_slice.h" + +namespace tensorflow { + +// Given a function like: +// namespace ops { +// Node* Identity(NodeOut input, const GraphDefBuilder::Options& opts) { +// if (opts.HaveError()) return nullptr; +// static const string kOpName = "Identity"; +// NodeBuilder node_builder(opts.GetNameForOp(kOpName), kOpName, +// opts.op_registry()); +// node_builder.Input(input); +// return opts.FinalizeBuilder(&node_builder); +// } +// } // namespace ops +// +// // Or, alternatively: +// namespace ops { +// Node* Identity(NodeOut input, const GraphDefBuilder::Options& opts) { +// static const string kOpName = "Identity"; +// return UnaryOp(kOpName, input, opts); +// } +// } // namespace ops +// +// You call it like: +// GraphDefBuilder b; +// using namespace ::tensorflow::ops; // NOLINT(build/namespaces) +// Node* na = Const(7, b.opts()); +// // Note: WithName() returns a copy, opts is unchanged. +// Node* nb = Const(5, b.opts().WithName("control-input")); +// Node* nc = Identity(na, b.opts().WithControlInput(nb)); +// GraphDef graph_def; +// Status status = b.ToGraphDef(&graph_def); +// if (!status.ok()) { /* Handle error */ } +// +// In tests you can skip the status handling via: +// GraphDefBuilder b(GraphDefBuilder::kFailImmediately); +// ... +// b.ToGraphDef(&graph_def); + +class GraphDefBuilder { + public: + // Options for adding a Node to a Graph. + class Options { + public: + // Sets the Graph (that Nodes will be added to) and the status. The + // status may be set to nullptr, in which case errors cause CHECK + // failures. The graph and status must outlive *this. + Options(Graph* graph, Status* status); + ~Options(); + + // Methods for setting options. These are const methods: they + // return a copy of *this with the option set. + Options WithName(StringPiece name) const; + Options WithDevice(StringPiece device) const; + Options WithControlInput(Node* control_input) const; + Options WithControlInputs(gtl::ArraySlice control_inputs) const; + + // Override the default value for an optional attr. + template + Options WithAttr(StringPiece attr_name, T&& value) const { + return Options(*this).WithAttrImpl(attr_name, std::forward(value)); + } + // Note: overload needed to allow {...} expressions for value. + template + Options WithAttr(StringPiece attr_name, + std::initializer_list value) const { + return WithAttr>(attr_name, std::move(value)); + } + + // Methods for using options from a function that creates a Node. + + // Returns true if the status associated with *this has an error. + // Use this to skip processing that may depend on prior results. + bool HaveError() const { return status_ != nullptr && !status_->ok(); } + + // Returns a string representation of the status associated with *this. + // Returns the string `"OK"` if the status doesn't have any error. + string StatusToString() const { return status_->ToString(); } + + // Given the Op type name, return a name for a node of that type. + // Uses the value set in WithName() if that has been called. Otherwise, + // returns a name built out of the Op type name. + string GetNameForOp(StringPiece op) const; + + // Sets the device, adds control inputs, adds attrs, and calls Finalize(). + // If Finalize returns an error, it is saved and this function returns + // nullptr. + Node* FinalizeBuilder(NodeBuilder* builder) const; + + // Updates the associated status, if any, or calls TF_CHECK_OK if none. + void UpdateStatus(const Status& status) const; + + // Accessor + const OpRegistryInterface* op_registry() const { + return graph_->op_registry(); + } + + private: + Options WithNameImpl(StringPiece name); + Options WithDeviceImpl(StringPiece device); + Options WithControlInputImpl(Node* control_input); + Options WithControlInputsImpl(gtl::ArraySlice control_inputs); + template + Options WithAttrImpl(StringPiece name, T&& value) { + attrs_.emplace_back(string(name), AttrValue()); + SetAttrValue(std::forward(value), &attrs_.back().second); + return *this; + } + + Graph* const graph_; + Status* const status_; + string name_; + string device_; + std::vector control_inputs_; + std::vector> attrs_; + }; + + // Start building a new graph. + explicit GraphDefBuilder( + const OpRegistryInterface* op_registry = OpRegistry::Global()) + : graph_(op_registry), opts_(&graph_, &status_) {} + + // For use in tests, where you want to fail immediately on error instead + // of checking the status at the end. + enum TestFailImmediatelyType { kFailImmediately }; + explicit GraphDefBuilder( + TestFailImmediatelyType, + const OpRegistryInterface* op_registry = OpRegistry::Global()) + : graph_(op_registry), opts_(&graph_, nullptr) {} + + // Gets the Options with the associated Graph and Status. + const Options& opts() const { return opts_; } + + // Once all the nodes have been added, call this to get whether it was + // successful, and if so fill *graph_def. + Status ToGraphDef(GraphDef* graph_def) const; + + // Adds the function and gradient definitions in `fdef_lib` to this graph's op + // registry. Ignores duplicate functions, and returns a bad status if an + // imported function differs from an existing function or op with the same + // name. + Status AddFunctionLibrary(const FunctionDefLibrary& fdef_lib) { + return graph_.AddFunctionLibrary(fdef_lib); + } + + // Returns whether a user-defined function with `name` already exists in the + // graph. + bool HasFunction(const string& name) { + return graph_.flib_def().Find(name) != nullptr; + } + + private: + Graph graph_; + Status status_; + Options opts_; +}; + +namespace ops { + +// A NodeOut may either be a regular input or back input. Regular +// inputs are specified via either a Node* or a Node* and an output +// index. Back inputs are specified by a node name, output index, and +// output type. +typedef NodeBuilder::NodeOut NodeOut; + +// For adding an Op with no inputs to a GraphDefBuilder. +Node* SourceOp(const string& op_name, const GraphDefBuilder::Options& opts); + +// For adding an Op with one input to a GraphDefBuilder. +Node* UnaryOp(const string& op_name, NodeOut input, + const GraphDefBuilder::Options& opts); + +// For adding an Op with two inputs to a GraphDefBuilder. +Node* BinaryOp(const string& op_name, NodeOut a, NodeOut b, + const GraphDefBuilder::Options& opts); + +} // namespace ops +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPH_GRAPH_DEF_BUILDER_H_ diff --git a/linux/include/tensorflow/core/graph/graph_def_builder_util.h b/linux/include/tensorflow/core/graph/graph_def_builder_util.h new file mode 100644 index 000000000..4a157e5b7 --- /dev/null +++ b/linux/include/tensorflow/core/graph/graph_def_builder_util.h @@ -0,0 +1,35 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CORE_GRAPH_GRAPH_DEF_BUILDER_UTIL_H_ +#define TENSORFLOW_CORE_GRAPH_GRAPH_DEF_BUILDER_UTIL_H_ + +#include "tensorflow/core/graph/graph_def_builder.h" +#include "tensorflow/core/lib/core/status.h" + +namespace tensorflow { + +class Graph; + +// Converts the `GraphDef` being built by `builder` to a `Graph` and +// stores it in `*graph`. +// TODO(josh11b): Make this faster; right now it converts +// Graph->GraphDef->Graph. This cleans up the graph (e.g. adds +// edges from the source and to the sink node, resolves back edges +// by name), and makes sure the resulting graph is valid. +Status GraphDefBuilderToGraph(const GraphDefBuilder& builder, Graph* graph); + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPH_GRAPH_DEF_BUILDER_UTIL_H_ diff --git a/linux/include/tensorflow/core/graph/graph_partition.h b/linux/include/tensorflow/core/graph/graph_partition.h new file mode 100644 index 000000000..8020c2d24 --- /dev/null +++ b/linux/include/tensorflow/core/graph/graph_partition.h @@ -0,0 +1,98 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPH_GRAPH_PARTITION_H_ +#define TENSORFLOW_CORE_GRAPH_GRAPH_PARTITION_H_ + +#include +#include +#include +#include + +#include "tensorflow/core/framework/function.h" +#include "tensorflow/core/framework/graph.pb.h" +#include "tensorflow/core/graph/costmodel.h" +#include "tensorflow/core/graph/graph.h" + +namespace tensorflow { + +struct PartitionOptions { + // A function that returns a location for the execution of a given + // Node. + typedef std::function NodeToLocFunc; + NodeToLocFunc node_to_loc = nullptr; + + // A function that returns a unique graph node name with the given + // prefix. + typedef std::function NewNameFunc; + NewNameFunc new_name = nullptr; + + // A function that returns the incarnation of a device given the + // device's fullname. If not found, GetIncarnationFunc should return + // kIllegalIncarnation. + static const uint64 kIllegalIncarnation = 0; + typedef std::function GetIncarnationFunc; + GetIncarnationFunc get_incarnation = nullptr; + + // If specified, flib_def defines a function library that should be + // partitioned and replicated into each resulting partition graphs. + const FunctionLibraryDefinition* flib_def = nullptr; + + // True if all the control flow "code" has already been added. The + // control flow code needs to be added when we still have the entire + // graph before any partitioning. So this flag should be false for + // the first partitioning but true for all subsequent partitioning. + // + // TODO(yuanbyu): We could also make the addition of the control + // flow code incremental based on 'node_to_loc'. This makes the + // communication a broadcast tree, which could be more efficient when + // the number of participating devices is large. + bool control_flow_added = false; + + // A function that returns the data type into which the tensor + // should be cast before sent over the wire. + typedef std::function ShouldCastFunc; + ShouldCastFunc should_cast = nullptr; + + // Schedule the execution of the recvs based on their start times + // computed by some scheduling algorithm. The recvs are divided into + // epochs based on their start times. A recv is enabled only when + // execution reaches its epoch - N for some predefined N. + bool scheduling_for_recvs = false; + // The start time for each node in the graph computed by some scheduling + // algorithm. If 'need_to_record_start_times' is true, we record them + // in the graph as a node attribute. + bool need_to_record_start_times = false; + std::vector start_times; +}; + +// Partition "input" graph into a set of graphs, one per location. +// The location for node n is derived by calling opts.node_to_loc(n). +// New nodes added by Partition use "opts.new_name(old_name)" to +// generate node names. +// +// Stores the partitions in *partitions. +Status Partition(const PartitionOptions& opts, Graph* input, + std::unordered_map* partitions); + +// Add control edges to the partitions to control the ordering +// and timing of the recv nodes based on the start times calculated +// using some scheduling algorithm. +Status AddControlEdges(const PartitionOptions& opts, + std::unordered_map* partitions); + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPH_GRAPH_PARTITION_H_ diff --git a/linux/include/tensorflow/core/graph/mkl_layout_pass.h b/linux/include/tensorflow/core/graph/mkl_layout_pass.h new file mode 100644 index 000000000..e7175149d --- /dev/null +++ b/linux/include/tensorflow/core/graph/mkl_layout_pass.h @@ -0,0 +1,36 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// A graph pass that rewrites graph for propagating MKL layout as a tensor + +#ifndef TENSORFLOW_CORE_GRAPH_MKL_LAYOUT_PASS_H_ +#define TENSORFLOW_CORE_GRAPH_MKL_LAYOUT_PASS_H_ + +#ifdef INTEL_MKL + +#include +#include +#include "tensorflow/core/graph/graph.h" + +namespace tensorflow { +// Interface to invoke the pass for unit test +// +// Returns true if and only if 'g' is mutated. +extern bool RunMklLayoutRewritePass(std::unique_ptr* g); +} // namespace tensorflow + +#endif + +#endif // TENSORFLOW_CORE_GRAPH_MKL_LAYOUT_PASS_H_ diff --git a/linux/include/tensorflow/core/graph/mkl_tfconversion_pass.h b/linux/include/tensorflow/core/graph/mkl_tfconversion_pass.h new file mode 100644 index 000000000..84e50ee6e --- /dev/null +++ b/linux/include/tensorflow/core/graph/mkl_tfconversion_pass.h @@ -0,0 +1,40 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// An optimization pass that inserts MklToTf conversion nodes in the graph + +#ifndef TENSORFLOW_CORE_GRAPH_MKL_TFCONVERSION_PASS_H_ +#define TENSORFLOW_CORE_GRAPH_MKL_TFCONVERSION_PASS_H_ + +#ifdef INTEL_MKL + +#include +#include +#include "tensorflow/core/graph/graph.h" + +#ifdef _WIN32 +typedef unsigned int uint; +#endif + +namespace tensorflow { +// Interface to invoke the pass for unit test +// +// Returns true if and only if 'g' is mutated. +extern bool InsertMklToTfConversionNodes(std::unique_ptr* g); +} // namespace tensorflow + +#endif + +#endif // TENSORFLOW_CORE_GRAPH_MKL_TFCONVERSION_PASS_H_ diff --git a/linux/include/tensorflow/core/graph/node_builder.h b/linux/include/tensorflow/core/graph/node_builder.h new file mode 100644 index 000000000..d576985a2 --- /dev/null +++ b/linux/include/tensorflow/core/graph/node_builder.h @@ -0,0 +1,167 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPH_NODE_BUILDER_H_ +#define TENSORFLOW_CORE_GRAPH_NODE_BUILDER_H_ + +#include +#include "tensorflow/core/framework/node_def_builder.h" +#include "tensorflow/core/framework/op.h" +#include "tensorflow/core/framework/op_def.pb.h" +#include "tensorflow/core/graph/graph.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/lib/core/stringpiece.h" +#include "tensorflow/core/lib/gtl/array_slice.h" + +namespace tensorflow { + +// This is a helper for creating a Node and adding it to a Graph. +// Internally, it uses a NodeDefBuilder to automatically set attrs +// that can be inferred from the inputs, and use default values +// (where they exist) for unspecified attrs. Example usage: +// +// Node* node; +// Status status = NodeBuilder(node_name, op_name) +// .Input(...) +// .Attr(...) +// .Finalize(&graph, &node); +// if (!status.ok()) return status; +// // Use node here. +class NodeBuilder { + public: + // For specifying the output of a Node to provide to one of the Input() + // functions below. It supports both regular inputs (where you are + // connecting to an existing Node*), and inputs from outside the graph + // (or haven't been added to the graph yet, like back edges, where + // you don't have a Node*). Both types can be mixed, e.g. in an + // ArraySlice. + struct NodeOut { + // For referencing an existing Node. + NodeOut(Node* n, int32 i = 0); + + // For referencing Nodes not in the graph being built. It is + // useful when preparing a graph for ExtendSession or creating a + // back edge to a node that hasn't been added to the graph yet, + // but will be. + NodeOut(StringPiece name, int32 i, DataType t); + + // Default constructor for std::vector. + NodeOut(); + + Node* node; + // error is set to true if: + // * the NodeOut was default constructed and never overwritten, + // * a nullptr Node* was passed to the NodeOut constructor, or + // * an out-of-range index was passed to the NodeOut constructor. + bool error; + string name; + int32 index; + DataType dt; + }; + + // Specify the name and the Op (either via an OpDef or the name of + // the Op plus a registry) for the Node. Other fields are + // specified by calling the methods below. + // REQUIRES: The OpDef must satisfy ValidateOpDef(). + NodeBuilder(StringPiece name, StringPiece op_name, + const OpRegistryInterface* op_registry = OpRegistry::Global()); + NodeBuilder(StringPiece name, const OpDef* op_def); + + // Create a NodeBuilder from an existing NodeDefBuilder. + NodeBuilder(const NodeDefBuilder& def_builder); + + // You must call one Input() function per input_arg in the Op, + // *and in the same order as the input_args appear in the OpDef.* + + // For inputs that take a single tensor. + NodeBuilder& Input(Node* src_node, int src_index = 0); + NodeBuilder& Input(NodeOut src); + + // For inputs that take a list of tensors. + NodeBuilder& Input(gtl::ArraySlice src_list); + + // Require that this node run after src_node(s). + NodeBuilder& ControlInput(Node* src_node); + NodeBuilder& ControlInputs(gtl::ArraySlice src_nodes); + + // Sets the "requested device spec" in the NodeDef (not the + // "assigned device" in the Node). + NodeBuilder& Device(StringPiece device_spec); + + // Sets the device name in the "assigned device" field in tensorflow::Node. + NodeBuilder& AssignedDevice(StringPiece device); + + // Set the value of an attr. attr_name must match the name of one of + // attrs defined by the Op, and value must have the corresponding type + // (see SetAttrValue() in ../framework/attr_value_util.h for legal + // types for value). Note that attrs will be set automatically if + // they can be determined by the inputs. + template + NodeBuilder& Attr(StringPiece attr_name, T&& value); + template + NodeBuilder& Attr(StringPiece attr_name, std::initializer_list value); + + // Validates the described node and adds it to *graph, adding edges + // for all (non-back) inputs. If created_node is not nullptr, + // *created_node will be set to the new node (or nullptr on error). + Status Finalize(Graph* graph, Node** created_node) const; + + // Accessors for the values set in the constructor. + const string& node_name() const { return def_builder_.node_name(); } + const OpDef& op_def() const { return def_builder_.op_def(); } + + private: + static DataType SafeGetOutput(const Node* node, int i, bool* error) { + if (node != nullptr && i >= 0 && i < node->num_outputs()) { + *error = false; + return node->output_type(i); + } else { + *error = true; + return DT_FLOAT; + } + } + + // If SafeGetOutput indicates a range error, add it to errors_. + void AddIndexError(const Node* node, int i); + + // Set *dt and returns true if i is in range. Combines + // SafeGetOutput() and AddIndexError(). + bool GetOutputType(const Node* node, int i, DataType* dt); + + NodeDefBuilder def_builder_; + std::vector inputs_; + std::vector control_inputs_; + std::vector errors_; + string assigned_device_; +}; + +// IMPLEMENTATION ------------------------------------------------------------- + +template +NodeBuilder& NodeBuilder::Attr(StringPiece attr_name, T&& value) { + def_builder_.Attr(attr_name, std::forward(value)); + return *this; +} + +template +NodeBuilder& NodeBuilder::Attr(StringPiece attr_name, + std::initializer_list value) { + def_builder_.Attr(attr_name, value); + return *this; +} + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPH_NODE_BUILDER_H_ diff --git a/linux/include/tensorflow/core/graph/optimizer_cse.h b/linux/include/tensorflow/core/graph/optimizer_cse.h new file mode 100644 index 000000000..ef466fb78 --- /dev/null +++ b/linux/include/tensorflow/core/graph/optimizer_cse.h @@ -0,0 +1,37 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// An optimization pass that performs common subexpression elimination. + +#ifndef TENSORFLOW_CORE_GRAPH_OPTIMIZER_CSE_H_ +#define TENSORFLOW_CORE_GRAPH_OPTIMIZER_CSE_H_ + +#include +#include "tensorflow/core/graph/graph.h" + +namespace tensorflow { + +// Perform common-subexpression elimination on the graph "*g". If +// "consider_fn" is not nullptr, then only nodes for which +// consider_fn(node) returns true will be considered for combining +// during the common subexpression elimination. +// +// Returns true if and only if 'g' is mutated. +extern bool OptimizeCSE(Graph* g, + const std::function& consider_fn); + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPH_OPTIMIZER_CSE_H_ diff --git a/linux/include/tensorflow/core/graph/quantize_training.h b/linux/include/tensorflow/core/graph/quantize_training.h new file mode 100644 index 000000000..dc3d7e3b1 --- /dev/null +++ b/linux/include/tensorflow/core/graph/quantize_training.h @@ -0,0 +1,56 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPH_QUANTIZE_TRAINING_H_ +#define TENSORFLOW_CORE_GRAPH_QUANTIZE_TRAINING_H_ + +#include "tensorflow/core/graph/graph.h" + +namespace tensorflow { +// Rewrites graph for quantized training. +// Rewrites the forward pass to include the precision loss with quantization so +// the model can learn to deal with such loss and achieve better accuracy when +// it is quantized later for inference. +// Note that the num_bits should be in [1, 63] and 'g' must be not null. +// quant_op_type specifies which quantization op should be used. +// Current ops supported: +// - QuantizeAndDequantizeV2. +// - FakeQuantWithMinMaxVars. +// +// On success, returns OK. +// +// On failure, returns the error status. Possible errors include: +// - num_bits out of range. +// - g is null. +// - More than 1 unknown ops encountered. +Status DoQuantizeTraining(int32 num_bits, const string& quant_op_type, + Graph* g); + +// Converts the input serialized GraphDef and returns a rewritten serialized +// GraphDef for quantized training. +Status DoQuantizeTrainingOnSerializedGraphDef(const string& input_graph, + int32 num_bits, + const string& quant_op_type, + string* result_graph); + +// Converts the input GraphDef and returns a rewritten GraphDef for quantized +// training. +Status DoQuantizeTrainingOnGraphDef(const GraphDef& input_graphdef, + int32 num_bits, const string& quant_op_type, + GraphDef* result_graphdef); + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPH_QUANTIZE_TRAINING_H_ diff --git a/linux/include/tensorflow/core/graph/subgraph.h b/linux/include/tensorflow/core/graph/subgraph.h new file mode 100644 index 000000000..3e99ff0c8 --- /dev/null +++ b/linux/include/tensorflow/core/graph/subgraph.h @@ -0,0 +1,165 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPH_SUBGRAPH_H_ +#define TENSORFLOW_CORE_GRAPH_SUBGRAPH_H_ + +#include + +#include "tensorflow/core/framework/device_attributes.pb.h" +#include "tensorflow/core/graph/graph.h" +#include "tensorflow/core/graph/node_builder.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/lib/gtl/array_slice.h" +#include "tensorflow/core/protobuf/config.pb.h" + +namespace tensorflow { +namespace subgraph { + +// Information about a graph rewritten by `RewriteGraphForExecution()`. +struct RewriteGraphMetadata { + // The element type of each tensor fed to this subgraph. The order + // of types corresponds to the order of tensor names in + // `fed_outputs` when calling `RewriteGraphForExecution()`. + DataTypeVector feed_types; + // The element type of each tensor fetched from this subgraph. The + // order of types corresponds to the order of tensor names in + // `fetch_outputs` when calling `RewriteGraphForExecution()`. + DataTypeVector fetch_types; +}; + +// Describes the action to take on a particular tensor endpoint (described by +// a ":" pair) when pruning the graph. +// +// The `AddNode()` method must be overridden to describe this action. The method +// will be invoked once during `RewriteGraphForExecution()` with tensor endpoint +// named by `endpoint_name`, and it may either create a single new node, or fail +// with an error if the resulting graph would be invalid. +class PruneRewrite { + public: + // `endpoint_name` and `device_info` must outlive this object. + PruneRewrite(const string* endpoint_name, const DeviceAttributes* device_info) + : endpoint_name_(endpoint_name), device_info_(device_info) {} + virtual ~PruneRewrite() {} + + // Creates a new node whose output replaces the given `tensor` in graph `g`. + // The node will be assigned to the device named in `device_info`. + virtual Status AddNode(Graph* g, NodeBuilder::NodeOut tensor, + Node** out_node) = 0; + + // Returns the name of the tensor to which this rewrite applies. + const string& endpoint_name() { return *endpoint_name_; } + + protected: + // The device on which the new node will be created. + const DeviceAttributes& device_info() { return *device_info_; } + + private: + const string* const endpoint_name_; // Not owned. + const DeviceAttributes* const device_info_; // Not owned. +}; + +// Rewrite the graph structure of "*g" to deal with feeding node +// outputs, fetching node outputs, and only running a subset of the +// graph. "fed_outputs" and "fetch_outputs" are both lists of +// output tensor identifiers in the form of +// "[:]", and "target_nodes_str" is a +// lists of target node names in "*g" "g". +// +// In the resulting graph "*g", output edges in "fed_outputs" have +// been redirected to special "_recv" nodes introduced into the graph. +// If these fed nodes are not needed in order to compute the effects +// of the nodes in "target_node_names" and "fetch_outputs", then these may +// be omitted from the graph. +// +// In the resulting graph "*g", additional "_send" nodes are connected +// to every output in "fetch_outputs". These "_send" nodes are set up +// to execute on the device described by device_info. +// +// On success, returns OK, and sets "*g" to a version of "*g" +// that represents the portions of the graph necessary for producing +// the output of all nodes listed in "target_node_names" and fetching the +// specific node outputs specified in "fetch_outputs". +// +// On failure, returns the error status. Possible errors include: +// - fed output "node:output_index" does not exist in "*g" +// - fetch output "node:output_index" does not exist in "*g" +// - target node "node" does not exist in "*g" +Status RewriteGraphForExecution( + Graph* g, const gtl::ArraySlice& fed_outputs, + const gtl::ArraySlice& fetch_outputs, + const gtl::ArraySlice& target_node_names, + const DeviceAttributes& device_info, bool use_function_convention, + RewriteGraphMetadata* out_metadata); + +// A more general version of the above function that supports +// customizable rewriting actions for each fed and fetched tensor. +Status RewriteGraphForExecution( + Graph* g, const std::vector>& feed_rewrites, + const std::vector>& fetch_rewrites, + const gtl::ArraySlice& target_node_names, + RewriteGraphMetadata* out_metadata); + +///////////////////////////////////////////////////////// +// Custom rewrite actions for fed and fetched tensors. // +///////////////////////////////////////////////////////// + +// A rewrite action that adds an _Arg node for a fed tensor. +class ArgFeedRewrite : public PruneRewrite { + public: + ArgFeedRewrite(const string* endpoint_name, + const DeviceAttributes* device_info, int32 arg_index) + : PruneRewrite(endpoint_name, device_info), arg_index_(arg_index) {} + Status AddNode(Graph* g, NodeBuilder::NodeOut feed_tensor, + Node** out_node) override; + + private: + const int32 arg_index_; +}; + +// A rewrite action that adds a client-terminated _Recv node for a fed tensor. +class RecvFeedRewrite : public PruneRewrite { + public: + using PruneRewrite::PruneRewrite; + Status AddNode(Graph* g, NodeBuilder::NodeOut feed_tensor, + Node** out_node) override; +}; + +// A rewrite action that adds a _Retval node for a fetched tensor. +class RetvalFetchRewrite : public PruneRewrite { + public: + RetvalFetchRewrite(const string* endpoint_name, + const DeviceAttributes* device_info, int32 retval_index) + : PruneRewrite(endpoint_name, device_info), retval_index_(retval_index) {} + Status AddNode(Graph* g, NodeBuilder::NodeOut fetch_tensor, + Node** out_node) override; + + private: + const int32 retval_index_; +}; + +// A rewrite action that adds a client-terminated _Send node for a +// fetched tensor. +class SendFetchRewrite : public PruneRewrite { + public: + using PruneRewrite::PruneRewrite; + Status AddNode(Graph* g, NodeBuilder::NodeOut fetch_tensor, + Node** out_node) override; +}; + +} // namespace subgraph +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPH_SUBGRAPH_H_ diff --git a/linux/include/tensorflow/core/graph/tensor_id.h b/linux/include/tensorflow/core/graph/tensor_id.h new file mode 100644 index 000000000..0ba394261 --- /dev/null +++ b/linux/include/tensorflow/core/graph/tensor_id.h @@ -0,0 +1,86 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_GRAPH_TENSOR_ID_H_ +#define TENSORFLOW_GRAPH_TENSOR_ID_H_ + +#include + +#include "tensorflow/core/graph/graph.h" +#include "tensorflow/core/lib/core/stringpiece.h" +#include "tensorflow/core/lib/hash/hash.h" +#include "tensorflow/core/lib/strings/strcat.h" + +namespace tensorflow { + +struct SafeTensorId; + +// Identifier for a tensor within a step. +// first == operation_name, second == output_index +// Note: does not own backing storage for name. +struct TensorId : public std::pair { + typedef std::pair Base; + + // Inherit the set of constructors. + using Base::pair; + + // NOTE(skyewm): this is required on some platforms. I'm not sure why the + // using statement above isn't always sufficient. + TensorId() : Base() {} + TensorId(const SafeTensorId& id); + + string ToString() const { + if (second == Graph::kControlSlot) return strings::StrCat("^", first); + return strings::StrCat(first, ":", second); + } + + struct Hasher { + public: + std::size_t operator()(const TensorId& x) const { + return Hash32(x.first.data(), x.first.size(), x.second); + } + }; +}; + +TensorId ParseTensorName(const string& name); +TensorId ParseTensorName(StringPiece name); + +// Same as TensorId, except owns the backing storage for the op name. This makes +// the memory management simpler at the expense of a copy. +struct SafeTensorId : public std::pair { + typedef std::pair Base; + + // NOTE(skyewm): this is required on some platforms. I'm not sure why the + // using "using Base::pair;" isn't always sufficient. + SafeTensorId() : Base() {} + SafeTensorId(const string& str, int idx) : Base(str, idx) {} + SafeTensorId(const TensorId& id); + + string ToString() const { + if (second == Graph::kControlSlot) return strings::StrCat("^", first); + return strings::StrCat(first, ":", second); + } + + struct Hasher { + public: + std::size_t operator()(const TensorId& x) const { + return Hash32(x.first.data(), x.first.size(), x.second); + } + }; +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_GRAPH_TENSOR_ID_H_ diff --git a/linux/include/tensorflow/core/graph/testlib.h b/linux/include/tensorflow/core/graph/testlib.h new file mode 100644 index 000000000..b00196f58 --- /dev/null +++ b/linux/include/tensorflow/core/graph/testlib.h @@ -0,0 +1,225 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// DEPRECATED: Use the C++ API defined in tensorflow/cc instead. + +#ifndef TENSORFLOW_CORE_GRAPH_TESTLIB_H_ +#define TENSORFLOW_CORE_GRAPH_TESTLIB_H_ + +#include +#include + +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/graph/graph.h" +#include "tensorflow/core/graph/types.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { +namespace test { +namespace graph { + +// Converts "g" into its corresponding GraphDef "def". +ABSL_DEPRECATED("Call g->ToGraphDef(def) instead.") +void ToGraphDef(Graph* g, GraphDef* def); + +// A few helpers to construct a graph. + +// Adds a node in "g" producing a constant "tensor". +Node* Constant(Graph* g, const Tensor& tensor); +Node* Constant(Graph* g, const Tensor& tensor, const string& name); + +// Adds a node in "g" producing a constant "tensor" on the host. +// The given node which, unlike the regular Constant above, always +// stores its output on the host. This is necessary for use +// in GPU tests where the test Op in question runs on the device +// but requires some arguments to be pinned to the host. +Node* HostConstant(Graph* g, const Tensor& tensor); +Node* HostConstant(Graph* g, const Tensor& tensor, const string& name); + +// Adds a variable in "g" of the given "shape" and "dtype". +Node* Var(Graph* g, const DataType dtype, const TensorShape& shape); +Node* Var(Graph* g, const DataType dtype, const TensorShape& shape, + const string& name); + +// Adds an assign node in "g" which assigns "val" into "var". +Node* Assign(Graph* g, Node* var, Node* val); + +// Adds a send node "g" sending "input" as a named "tensor" from +// "sender" to "receiver". +Node* Send(Graph* g, Node* input, const string& tensor, const string& sender, + const uint64 sender_incarnation, const string& receiver); + +// Adds a recv node in "g" receiving a named "tensor" from "sender" +// to "receiver". +Node* Recv(Graph* g, const string& tensor, const string& type, + const string& sender, const uint64 sender_incarnation, + const string& receiver); + +// Adds a reduction "node" in "g" doing sum(data, axes). "reduce" is +// a reduction, e.g., Sum, Max, Min, Mean, etc. +Node* Reduce(Graph* g, const string& reduce, Node* data, Node* axes, + bool keep_dims = false); + +// Adds a Matmul node in g doing in0.contract(in1). +Node* Matmul(Graph* g, Node* in0, Node* in1, bool transpose_a, + bool transpose_b); + +// Adds a Matmul node in g doing in0.contract(in1). +Node* BatchMatmul(Graph* g, Node* in0, Node* in1, bool adj_x, bool adj_y); + +// Adds a Quantize node into g that quantize floats into QUINT8. The range of +// the input float tensor is assumed to be [-1, 1]. +Node* QuantizeToUINT8(Graph* g, Node* data); + +// Adds a unary function "func" "node" in "g" taking "input". +Node* Unary(Graph* g, const string& func, Node* input, int index = 0); + +// Adds an identity node in "g" taking "input" and producing an +// identity copy. +Node* Identity(Graph* g, Node* input, int index = 0); + +// Adds a binary function "func" node in "g" taking "in0" and "in1". +Node* Binary(Graph* g, const string& func, Node* in0, Node* in1); + +// Adds a function "func" node in "g" taking inputs "ins". +Node* Multi(Graph* g, const string& func, gtl::ArraySlice ins); + +// Adds a binary add node in "g" doing in0 + in1. +Node* Add(Graph* g, Node* in0, Node* in1); + +// Reverses dimensions of > +Node* Reverse(Graph* g, Node* tensor, Node* axis); + +// Generates random unit uniform distribution of the input shape. +Node* RandomUniform(Graph* g, Node* input, DataType dtype); + +// Generates random unit normal distribution of the input shape. +Node* RandomGaussian(Graph* g, Node* input, DataType dtype); + +// Generates random gamma distribution with the given shape and alpha[s]. +// Output dtype determined by alpha. +Node* RandomGamma(Graph* g, Node* shape, Node* alpha); + +// Generates random poisson distribution with the given shape and lam[s]. +// Output dtype determined by lam. +Node* RandomPoisson(Graph* g, Node* shape, Node* lam); + +// Rolls tensor by an offset of along the corresponding +// dimensions. +Node* Roll(Graph* g, Node* input, Node* shift, Node* axis); + +// Generates random parameters from the truncated standard normal distribution +// of the nput shape +Node* TruncatedNormal(Graph* g, Node* input, DataType dtype); + +// Adds an error node in "g". The node's computation always +// generates an error with the given error message "errmsg". +Node* Error(Graph* g, Node* input, const string& errmsg); + +// Adds a node that generates a invalid ref output. +Node* InvalidRefType(Graph* g, DataType out_type, DataType invalid_type); + +// Adds a node in "g". Its Compute() sleeps a while and outputs the +// input (i.e., same as identity). +Node* Delay(Graph* g, Node* input, Microseconds delay_micros); + +// Adds a no-op "node" in "g", with control inputs from all nodes in +// control_inputs vector. +Node* NoOp(Graph* g, const std::vector& control_inputs); + +// Adds a Switch node in "g". If "in1" is true, it forwards "in0" to +// output 1. Otherwise, it forwards "in0" to output 0. +Node* Switch(Graph* g, Node* in0, Node* in1); + +// Adds an Enter node in "g", which enters a new frame. +Node* Enter(Graph* g, Node* input, const string& frame_name); + +// Adds an Exit node in "g", which exits a frame. +Node* Exit(Graph* g, Node* input); + +// Adds a Merge node in "g" with two inputs "in0" and "in1". +Node* Merge(Graph* g, Node* in0, Node* in1); + +// Adds a Merge node in "g". The first input is "in0", the remaining +// inputs are only given by their names in remaining_in. +Node* Merge(Graph* g, Node* in0, gtl::ArraySlice remaining_in); + +// Adds a NextIteration node in "g", which makes its input available +// to the next iteration. +Node* Next(Graph* g, const string& name, Node* input); + +// Adds a LoopCond node in "g", representing the "pivot" termination +// condition of a loop. +Node* LoopCond(Graph* g, Node* input); + +// Adds a less node in "g", which returns true iff "in0" < "in1". +Node* Less(Graph* g, Node* in0, Node* in1); + +// Adds a select node in "g", which outputs either "inx" or "iny" +// depending on the boolean value of "c". +Node* Select(Graph* g, Node* c, Node* inx, Node* iny); + +// Casts "in" into data type "dst". +Node* Cast(Graph* g, Node* in, DataType dst); + +// Perform gather op on params "in0" with indices "in1" and axis "axis". +Node* Gather(Graph* g, Node* in0, Node* in1, Node* axis); + +// Gets a tensor stored in the session state. +Node* GetSessionTensor(Graph* g, Node* in); + +// Adds a Concat node in "g". The first input is "concat_dim", the +// dimension to concatenate on, and the tensors to concatenate are +// given in "tensors". +Node* Concat(Graph* g, Node* concat_dim, gtl::ArraySlice tensors); + +// Adds a ConcatV2 node in "g". The last input is "concat_dim", the +// dimension to concatenate on, and the tensors to concatenate are +// given in "tensors". +Node* ConcatV2(Graph* g, gtl::ArraySlice tensors, Node* concat_dim); + +// Add a Relu node in "g". +Node* Relu(Graph* g, Node* in); + +// Add a Relu6 node in "g". +Node* Relu6(Graph* g, Node* in); + +// Add a BiasAdd node in "g". +Node* BiasAdd(Graph* g, Node* value, Node* bias); + +// Add a Conv2D node in "g". +Node* Conv2D(Graph* g, Node* in0, Node* in1); + +// Add a Diag node in "g". +Node* Diag(Graph* g, Node* in, DataType type); + +// Add a DiagPart node in "g". +Node* DiagPart(Graph* g, Node* in, DataType type); + +// Add a CheckNumerics node in "g". +Node* CheckNumerics(Graph* g, Node* in, const string& message); + +// Add an _Arg node in "g". +Node* Arg(Graph* g, int64 index, DataType type); + +// Add a _Retval node in "g". +Node* Retval(Graph* g, int64 index, Node* in); + +} // end namespace graph +} // end namespace test +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPH_TESTLIB_H_ diff --git a/linux/include/tensorflow/core/graph/types.h b/linux/include/tensorflow/core/graph/types.h new file mode 100644 index 000000000..ac5a7f822 --- /dev/null +++ b/linux/include/tensorflow/core/graph/types.h @@ -0,0 +1,35 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPH_TYPES_H_ +#define TENSORFLOW_CORE_GRAPH_TYPES_H_ + +#include "tensorflow/core/lib/gtl/int_type.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +// We model running time in microseconds. +TF_LIB_GTL_DEFINE_INT_TYPE(Microseconds, int64); + +// We can also model running time in nanoseconds for more accuracy. +TF_LIB_GTL_DEFINE_INT_TYPE(Nanoseconds, int64); + +// We model size in bytes. +TF_LIB_GTL_DEFINE_INT_TYPE(Bytes, int64); + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPH_TYPES_H_ diff --git a/linux/include/tensorflow/core/graph/validate.h b/linux/include/tensorflow/core/graph/validate.h new file mode 100644 index 000000000..08879dca6 --- /dev/null +++ b/linux/include/tensorflow/core/graph/validate.h @@ -0,0 +1,65 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPH_VALIDATE_H_ +#define TENSORFLOW_CORE_GRAPH_VALIDATE_H_ + +#include "tensorflow/core/framework/graph.pb.h" +#include "tensorflow/core/framework/op.h" +#include "tensorflow/core/graph/graph.h" +#include "tensorflow/core/lib/core/status.h" + +namespace tensorflow { +namespace graph { + +// Returns OK if every NodeDef in `graph_def` is valid with respect to +// its corresponding OpDef (as defined by ValidateNodeDef()) as +// registered in `op_registry`. Also checks for deprecated ops. +// +// REQUIRES: +// * `op_registry` is not nullptr. +// * `graph_def` has default attrs filled in (see AddDefaultAttrsToGraphDef()). +Status ValidateGraphDef(const GraphDef& graph_def, + const OpRegistryInterface& op_registry); + +// Like ValidateGraphDef() except it makes a copy of `graph_def` and calls +// AddDefaultAttrsToGraphDef() on the copy, removing that requirement from the +// caller. +Status ValidateGraphDefAgainstOpRegistry( + const GraphDef& graph_def, const OpRegistryInterface& op_registry); + +// Like ValidateGraphDefAgainstOpRegistry() except it takes an OpList +// instead of an OpRegistryInterface. Note that the OpList need not +// have descriptions, which can be a big space savings, see +// GetOpListForValidation() below. +Status ValidateGraphDefAgainstOpList(const GraphDef& graph_def, + const OpList& op_list); + +// Get an OpList from `*op_registry` with all the descriptions removed. +void GetOpListForValidation( + OpList* op_list, const OpRegistry& op_registry = *OpRegistry::Global()); + +// Validate that the graph has no cycle except for legal while loop cycles. +// This traverses the specified nodes in topological order to verify there are +// no cycles. Starting with inputless nodes, it visits nodes whose inputs have +// all been visited, and counts the total number of visited nodes. If there is a +// cycle, nodes in the cycle will never be visited, and the visited count will +// be less than the total node count. +Status ValidateGraphHasNoCycle(const Graph& graph); + +} // namespace graph +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPH_VALIDATE_H_ diff --git a/linux/include/tensorflow/core/graph/while_context.h b/linux/include/tensorflow/core/graph/while_context.h new file mode 100644 index 000000000..5405e62be --- /dev/null +++ b/linux/include/tensorflow/core/graph/while_context.h @@ -0,0 +1,76 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPH_WHILE_CONTEXT_H_ +#define TENSORFLOW_CORE_GRAPH_WHILE_CONTEXT_H_ + +#include "tensorflow/core/graph/graph.h" + +namespace tensorflow { + +// Information about a while loop. Every user-defined while loop has an +// associated WhileContext, i.e., there is a WhileContext for every execution +// frame. Created with the while loop and used during gradient +// construction. Note that the gradient graph of while loop contains while loops +// itself, but these do not generate separate WhileContexts. +// +// TODO(skyewm): this is currently insufficient to handle nested loops and +// conditionals (and possibly other requirements). This may change a lot in the +// future to support these features. +// +// TODO(skyewm): de/serialize in MetaGraphDef so imported while loops will be +// differentiable. Figure out backwards compatibility story. +class WhileContext { + public: + WhileContext(StringPiece frame_name, std::vector enter_nodes, + std::vector exit_nodes, OutputTensor cond_output, + std::vector body_inputs, + std::vector body_outputs); + + const string& frame_name() const { return frame_name_; } + const std::vector& enter_nodes() const { return enter_nodes_; } + const std::vector& exit_nodes() const { return exit_nodes_; } + const OutputTensor& cond_output() const { return cond_output_; } + const std::vector& body_inputs() const { return body_inputs_; } + const std::vector& body_outputs() const { + return body_outputs_; + } + + private: + // Each user-defined while loop defines a new execution frame, which is + // uniquely identified by its frame name. Frames are used by the executor to + // manage the iterations of a loop. See the FrameState comment in + // core/common_runtime/executor.cc for more details. + const string frame_name_; + + // The enter nodes defining the input loop variables to the while loop. This + // vector defines the order of the loop variables. + const std::vector enter_nodes_; + + // The exit nodes defining the outputs of the while loop. These are in loop + // variable order. + const std::vector exit_nodes_; + + // The boolean output of the loop predicate. + const OutputTensor cond_output_; + + // The inputs and outputs to the loop body. + const std::vector body_inputs_; + const std::vector body_outputs_; +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPH_WHILE_CONTEXT_H_ diff --git a/linux/include/tensorflow/core/grappler/clusters/cluster.h b/linux/include/tensorflow/core/grappler/clusters/cluster.h new file mode 100644 index 000000000..519d5ed87 --- /dev/null +++ b/linux/include/tensorflow/core/grappler/clusters/cluster.h @@ -0,0 +1,141 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_CLUSTERS_CLUSTER_H_ +#define TENSORFLOW_CORE_GRAPPLER_CLUSTERS_CLUSTER_H_ + +#include +#include +#include +#include + +#include "tensorflow/core/common_runtime/device_set.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/grappler/grappler_item.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/lib/strings/strcat.h" +#include "tensorflow/core/protobuf/device_properties.pb.h" +#include "tensorflow/core/public/session_options.h" + +namespace tensorflow { +namespace grappler { + +// A cluster represents of collection of hardware resources available to run +// the TensorFlow model. +// A process can only create a single cluster at a time. +class Cluster { + public: + explicit Cluster(int timeout_s); + virtual ~Cluster(); + + // Returns a string that represent the type of cluster that was instantiated. + virtual string type() const = 0; + + // Provision the hardware resources needed to run TensorFlow and start a + // TensorFlow session that can take advantage of these resources. + // The actual resources that are leveraged depend on the type of cluster + // instantiated. + // Returns OK iff all the requested resources could be reserved and a + // TensorFlow session successfully created. Returns an error otherwise. + // There is no graceful degradation to handle the case where only a subset + // of the requested resources are available. + virtual Status Provision() = 0; + + // Attempts to shutdown the cluster. + // Returns OK iff there are no pending calls to the Run() method and all the + // resources used by the cluster could be released. Returns an error + // otherwise. + virtual Status Shutdown() { return Status::OK(); } + + // Whether soft placement is allowed. If allow_soft_placement is true, + // an op will be placed on CPU if there's no GPU implementation for the OP + // or if no GPU devices are known or registered or if we need to co-locate + // with reftype input(s) which are from CPU. + void AllowSoftPlacement(bool soft_placement_state); + + // Update the number of inter-op threads for each per-session threadpool + void SetNumInterOpThreads(int num_threads); + + // Set the number of steps required to warmup TensorFlow. Must be called + // before Provision(). + void SetNumWarmupSteps(int num_steps); + + // Set executor type to instantiate + void SetExecutorType(const string* executor_type); + + // Returns the number of warmup steps. + int NumWarmupSteps() const; + + // Disable the collection of detailed statistics. Must be called + // before Provision(). + void DisableDetailedStats(bool disable); + + // Returns true iff the collection of detailed statistics is enabled. + bool DetailedStatsEnabled() const; + + // Disable the TensorFlow optimizer. This ensures that the graph that TF + // executes is similar to the input graph. Must be called before Provision(). + void DisableOptimizer(bool disable); + + // Return the list of TensorFlow devices that are available to execute a + // graph. This is empty until provision() is called. + const std::unordered_map& GetDevices() const { + return devices_; + } + + // Convenience method that returns the set of device names. These names are + // sorted alphabetically. + const std::vector GetDeviceNames() const; + + // The DeviceSet is not always available, but when it is it contains a + // superset of the devices listed in GetDevices/GetDeviceNames(). + virtual const DeviceSet* GetDeviceSet() const { return nullptr; } + + // Enables collecting the allocator stats. Call with enable=true must be made + // before Provision(). + virtual Status EnablePeakMemoryStats(bool enable) { + return errors::Unimplemented(strings ::StrCat( + "Peak Memory Stats are not supported on ", type(), " clusters")); + } + + // Returns peak memory of all devices during the session creation and session + // runs. + virtual Status GetPeakMemoryUsage( + std::unordered_map* device_peak_memory) const { + return errors::Unimplemented( + "GetPeakMemoryUsage is not implemented for this type of cluster."); + } + + // Prepare the session to run the specified grappler item. This include + // initializing all the model variables. + virtual Status Initialize(const GrapplerItem& item) = 0; + + // Run the specified graph_def and return the corresponding metadata. + virtual Status Run(const GraphDef& graph_def, + const std::vector>& feed, + const std::vector& fetch, + RunMetadata* metadata) = 0; + + protected: + std::unordered_map devices_; + const int timeout_s_; + SessionOptions options_; + RunOptions run_options_; +}; + +} // end namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_CLUSTERS_CLUSTER_H_ diff --git a/linux/include/tensorflow/core/grappler/clusters/utils.h b/linux/include/tensorflow/core/grappler/clusters/utils.h new file mode 100644 index 000000000..f0a342b72 --- /dev/null +++ b/linux/include/tensorflow/core/grappler/clusters/utils.h @@ -0,0 +1,39 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_CLUSTERS_UTILS_H_ +#define TENSORFLOW_CORE_GRAPPLER_CLUSTERS_UTILS_H_ + +#include "tensorflow/core/common_runtime/gpu/gpu_id.h" +#include "tensorflow/core/protobuf/device_properties.pb.h" +#include "tensorflow/core/util/device_name_utils.h" + +namespace tensorflow { +namespace grappler { + +// Returns the DeviceProperties of the CPU on which grappler is running. +DeviceProperties GetLocalCPUInfo(); + +// Returns the DeviceProperties for the specified GPU attached to the server on +// which grappler is running. +DeviceProperties GetLocalGPUInfo(PlatformGpuId platform_gpu_id); + +// Returns the DeviceProperties of the specified device +DeviceProperties GetDeviceInfo(const DeviceNameUtils::ParsedName& device); + +} // end namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_CLUSTERS_UTILS_H_ diff --git a/linux/include/tensorflow/core/grappler/clusters/virtual_cluster.h b/linux/include/tensorflow/core/grappler/clusters/virtual_cluster.h new file mode 100644 index 000000000..6adb0b99b --- /dev/null +++ b/linux/include/tensorflow/core/grappler/clusters/virtual_cluster.h @@ -0,0 +1,61 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_CLUSTERS_VIRTUAL_CLUSTER_H_ +#define TENSORFLOW_CORE_GRAPPLER_CLUSTERS_VIRTUAL_CLUSTER_H_ + +#include + +#include "tensorflow/core/common_runtime/device_set.h" +#include "tensorflow/core/grappler/clusters/cluster.h" +#include "tensorflow/core/grappler/costs/op_level_cost_estimator.h" +#include "tensorflow/core/grappler/costs/virtual_scheduler.h" +#include "tensorflow/core/protobuf/device_properties.pb.h" + +namespace tensorflow { +namespace grappler { + +// Create a simple cluster that lists the devices (and their properties) +// available in a TensorFlow session. This cluster simulates the execution of +// actual graphs. +class VirtualCluster : public Cluster { + public: + VirtualCluster(const std::unordered_map& devices); + VirtualCluster(const std::unordered_map& devices, + OpLevelCostEstimator* node_estimator, + ReadyNodeManager* node_manager); + VirtualCluster(const DeviceSet* device_set); + + ~VirtualCluster() override; + + string type() const override { return "virtual"; } + + Status Provision() override; + Status Initialize(const GrapplerItem& item) override; + Status Run(const GraphDef& item, + const std::vector>& feed, + const std::vector& fetch, RunMetadata* metadata) override; + const DeviceSet* GetDeviceSet() const override { return device_set_; } + + private: + std::unique_ptr node_estimator_; + std::unique_ptr node_manager_; + const DeviceSet* device_set_ = nullptr; // Not owned +}; + +} // end namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_CLUSTERS_VIRTUAL_CLUSTER_H_ diff --git a/linux/include/tensorflow/core/grappler/costs/cost_estimator.h b/linux/include/tensorflow/core/grappler/costs/cost_estimator.h new file mode 100644 index 000000000..e91f0cc9d --- /dev/null +++ b/linux/include/tensorflow/core/grappler/costs/cost_estimator.h @@ -0,0 +1,191 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_COSTS_COST_ESTIMATOR_H_ +#define TENSORFLOW_CORE_GRAPPLER_COSTS_COST_ESTIMATOR_H_ + +#include +#include +#include "tensorflow/core/lib/core/status.h" + +namespace tensorflow { +class GraphDef; +class CostGraphDef; + +namespace grappler { +struct GrapplerItem; + +constexpr int64 kMemoryUnknown = -1ll; +constexpr int64 kZeroMemory = 0ll; + +// Holds the set of things we might want to estimate or measure in Grappler. +// Always produce execution time. Other fields are optional depending on the +// estimator being used. +struct Costs { + // Returns a Costs structure with default values for all of the fields. + inline Costs(); + + // Builds a Costs structure with all zero values, rather than unknowns. + static inline Costs ZeroCosts(); + + struct MilliSeconds : std::chrono::milliseconds { + MilliSeconds() : std::chrono::milliseconds(0) {} + MilliSeconds(double d) : std::chrono::milliseconds(static_cast(d)) {} + MilliSeconds(const std::chrono::milliseconds& d) + : std::chrono::milliseconds(d) {} + MilliSeconds& operator=(const std::chrono::milliseconds& d) { + std::chrono::milliseconds::operator=(d); + return *this; + } + }; + struct MicroSeconds : std::chrono::microseconds { + MicroSeconds() : std::chrono::microseconds(0) {} + MicroSeconds(double d) : std::chrono::microseconds(static_cast(d)) {} + MicroSeconds(const std::chrono::microseconds& d) + : std::chrono::microseconds(d) {} + MicroSeconds& operator=(const std::chrono::microseconds& d) { + std::chrono::microseconds::operator=(d); + return *this; + } + MilliSeconds asMilliSeconds() const { + return std::chrono::duration_cast(*this); + } + }; + struct NanoSeconds : std::chrono::nanoseconds { + NanoSeconds() : std::chrono::nanoseconds(0) {} + NanoSeconds(double d) : std::chrono::nanoseconds(static_cast(d)) {} + NanoSeconds(const std::chrono::nanoseconds& d) + : std::chrono::nanoseconds(d) {} + NanoSeconds& operator=(const std::chrono::nanoseconds& d) { + std::chrono::nanoseconds::operator=(d); + return *this; + } + MicroSeconds asMicroSeconds() const { + return std::chrono::duration_cast(*this); + } + MilliSeconds asMilliSeconds() const { + return std::chrono::duration_cast(*this); + } + static NanoSeconds infinity() { + return NanoSeconds(std::chrono::nanoseconds::max()); + } + }; + + // We store all our times in nanoseconds. If needs be, we can always switch to + // picoseconds in the future by updating this typedef. + typedef NanoSeconds Duration; + + // Overall cost of running the graph; latency. + Duration execution_time; + + // Computation cost of running the graph. + Duration compute_time; + + // Memory access cost of running the graph. + Duration memory_time; + + // This field can be a very pessimistic estimate of the main memory + // requirements of a graph. For example, it might assume that all activations + // are live for all of a graph's execution. + int64 max_memory; // Maximum main memory requirement in bytes over all ops. + int64 persistent_memory; + int64 temporary_memory; + + // These fields are used for TPU-related estimations. They are per-op + // maximums, so each op is evaluated independently, but we want the maximum of + // the value over all ops. + int64 max_per_op_buffers; // Sum of all buffers used by the ops. + int64 max_per_op_streaming; // Ignore largest input buffer, assuming it + // streams from main memory. + + // Number of ops included in this Costs in total. + // Default initialized to be one. + int64 num_ops_total = 1; + // If the time estimation is inaccurate. + bool inaccurate = false; + // Number of ops that are estimated with unknown shapes. + int64 num_ops_with_unknown_shapes = 0; + // TODO(pcma): include a counter for total inaccurate ops and counters for + // other reasons causing the inaccuracy + + // Max possible memory usage per device. + std::unordered_map estimated_max_memory_per_device; +}; + +inline std::ostream& operator<<(std::ostream& os, const Costs::MilliSeconds d) { + os << d.count() << "ms"; + return os; +} +inline std::ostream& operator<<(std::ostream& os, const Costs::MicroSeconds d) { + os << d.count() << "us"; + return os; +} +inline std::ostream& operator<<(std::ostream& os, const Costs::NanoSeconds d) { + os << d.count() << "ns"; + return os; +} + +Costs::Costs() { + execution_time = Duration::zero(); + compute_time = Duration::zero(); + memory_time = Duration::zero(); + max_memory = kMemoryUnknown; + persistent_memory = kMemoryUnknown; + temporary_memory = kMemoryUnknown; + max_per_op_buffers = kMemoryUnknown; + max_per_op_streaming = kMemoryUnknown; +} + +Costs Costs::ZeroCosts() { + Costs costs; + costs.execution_time = Duration::zero(); + costs.compute_time = Duration::zero(); + costs.memory_time = Duration::zero(); + costs.max_memory = kZeroMemory; + costs.persistent_memory = kZeroMemory; + costs.temporary_memory = kZeroMemory; + costs.max_per_op_buffers = kZeroMemory; + costs.max_per_op_streaming = kZeroMemory; + return costs; +} + +// Given a GrapperItem and an optimized implementation of the corresponding +// TensorFlow graph, the CostEstimator attempts to predicts the actual cost of +// running the graph. +class CostEstimator { + public: + virtual ~CostEstimator() {} + + // Initializes the estimator for the specified grappler item. + // The estimator shouldn't be used if this function returns any status other + // that OK. + virtual Status Initialize(const GrapplerItem& item) = 0; + + // Predicts the cost of running the given optimized version of the grappler + // item. + // If a CostGraphDef is passed, it will be populated with detailed information + // about the cost of running each operation of the optimized graph. + // if a double value is passed, it will be set to a value that reflects the + // overall cost of running the graph (e.g. the latency of the computation). + // Returns a status that indicate is the performance could be estimated or + // not. + virtual Status PredictCosts(const GraphDef& optimized_graph, + CostGraphDef* cost_graph, Costs* cost) const = 0; +}; + +} // end namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_COSTS_COST_ESTIMATOR_H_ diff --git a/linux/include/tensorflow/core/grappler/costs/graph_memory.h b/linux/include/tensorflow/core/grappler/costs/graph_memory.h new file mode 100644 index 000000000..a8ae4cc49 --- /dev/null +++ b/linux/include/tensorflow/core/grappler/costs/graph_memory.h @@ -0,0 +1,81 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_COSTS_GRAPH_MEMORY_H_ +#define TENSORFLOW_CORE_GRAPPLER_COSTS_GRAPH_MEMORY_H_ + +#include "tensorflow/core/framework/node_def.pb.h" +#include "tensorflow/core/grappler/clusters/cluster.h" +#include "tensorflow/core/grappler/costs/cost_estimator.h" +#include "tensorflow/core/grappler/costs/graph_properties.h" +#include "tensorflow/core/grappler/grappler_item.h" + +namespace tensorflow { +namespace grappler { + +// Infer the worst case memory usage for a given grappler item. +class GraphMemory { + public: + struct LiveTensor { + string node; + int output_id; + size_t memory_used; + Costs::Duration allocation_time; + Costs::Duration deallocation_time; + }; + struct MemoryUsage { + int64 used_memory; + std::vector live_tensors; + }; + + explicit GraphMemory(const GrapplerItem& item) + : item_(item), unknown_usage_({-1, {}}) {} + + Status InferStatically( + const std::unordered_map& devices); + Status InferDynamically(Cluster* cluster); + + // Worst case memory usage in bytes, or -1 if the usage is unknown. If there + // are multiple devices, returns the highest per device memory usage. + int64 GetWorstCaseMemoryUsage() const; + + // Returns the peak memory usage for the specified device. + const MemoryUsage& GetPeakMemoryUsage(const string& device) const { + auto it = peak_usage_.find(device); + if (it == peak_usage_.end()) { + return unknown_usage_; + } + return it->second; + } + + private: + void InferMemUsageForNodes(const std::vector& nodes, + GraphProperties* properties, int64* worst_case, + int64* best_case) const; + int64 InferMemUsageForNeighbors( + const std::vector& props) const; + + void InferFromTrace(const StepStats& timeline); + + GrapplerItem item_; + std::unordered_map worst_case_memory_usage_; + std::unordered_map peak_usage_; + const MemoryUsage unknown_usage_; +}; + +} // end namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_COSTS_GRAPH_MEMORY_H_ diff --git a/linux/include/tensorflow/core/grappler/costs/graph_properties.h b/linux/include/tensorflow/core/grappler/costs/graph_properties.h new file mode 100644 index 000000000..f716cd72c --- /dev/null +++ b/linux/include/tensorflow/core/grappler/costs/graph_properties.h @@ -0,0 +1,130 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_COSTS_GRAPH_PROPERTIES_H_ +#define TENSORFLOW_CORE_GRAPPLER_COSTS_GRAPH_PROPERTIES_H_ + +#include +#include +#include "tensorflow/core/framework/shape_inference.h" +#include "tensorflow/core/grappler/clusters/cluster.h" +#include "tensorflow/core/grappler/costs/op_performance_data.pb.h" +#include "tensorflow/core/grappler/grappler_item.h" + +namespace tensorflow { + +namespace grappler { + +class SymbolicShapeRefiner; +class TopoQueue; + +// Infer OpInfo::TensorProperties for graph nodes inputs/outputs. +// +// Typical use case, is to infer tensor properties from a graph, before doing +// optimization pass. Nodes modified during optimization pass have to be +// invalidated, to prevent further incorrect optimizations based on wrong shape +// and data type properties. +class GraphProperties { + public: + // The item must outlive the properties + explicit GraphProperties(const GrapplerItem& item) : item_(item) {} + + // Infer the shapes through abstract interpretation. Feed information can be + // incorrect so it should be discarded to ensure correctness of the analysis. + // However, it can help infer shapes in the fanout of fed nodes (even though + // the correctness of these shapes can't be guaranteed), so in some cases + // (such as simulation or scheduling) it makes sense of keep these shapes. + Status InferStatically(bool assume_valid_feeds); + // Infer the shape by running the graph on the specified cluster and recording + // the shapes of the processed tensors. + Status InferDynamically(Cluster* cluster); + // Extract the properties from a cost graph. For testing only since there is + // no way to ensure that the cost graph match the item. + Status InferFromCostGraph(const CostGraphDef& cost_graph); + + // Stores `item_.graph` with the inferred output shapes to `output_graph_def`. + Status AnnotateOutputShapes(GraphDef* output_graph_def) const; + + // Return the properties of node inputs/outputs, including data types and + // shapes. Note that the dimensions in the shapes can be negative. We use the + // -1 value to denote that we don't know anything about a dimension. We use + // values strictly less than -1 to encode symbolic dimensions: although we + // don't know the actual value of the symbolic dimension, we know that all the + // dimensions denoted by the same negative value are the equal. + bool HasInputProperties(const string& name) const; + bool HasOutputProperties(const string& name) const; + const std::vector& GetInputProperties( + const string& node_name) const; + const std::vector& GetOutputProperties( + const string& node_name) const; + // Invalidate input/output properties for nodes modified during graph + // optimization pass, to prevent potential optimizations, based on incorrect + // shape information. + void ClearInputProperties(const string& node_name); + void ClearOutputProperties(const string& node_name); + + private: + // Relaxes shapes , determined from an EnqueueV2 node, into + // <*queue_shapes_and_types>. + static Status RelaxEnqueueShapesAndMergeTypes( + SymbolicShapeRefiner* shape_refiner, const NodeDef* qnode, + const std::vector& shapes_and_types, + std::vector* queue_shapes_and_types); + + // Update the shapes of the enqueue node, port them over to the corresponding + // queue, and schedule the reprocessing of the queue if needed. + static Status UpdateEnqueue( + const NodeDef* enqueue_node, + const std::unordered_map& + resource_handles, + SymbolicShapeRefiner* shape_refiner, bool* new_shapes); + + // Update the shapes and types of the Queue node, if not set by Enqueue node. + static Status UpdateQueue(const NodeDef* queue_node, + SymbolicShapeRefiner* shape_refiner, + bool* new_shapes); + + // Update the output shapes of a Merge node, and enqueue its fanout in + // new_shapes if needed. + Status UpdateMergeNode(SymbolicShapeRefiner* shape_refiner, + const NodeDef* node, bool* new_shapes) const; + // Process the Enter node, and enqueue its fanout in new_shapes if needed. + static Status UpdateEnter(SymbolicShapeRefiner* shape_refiner, + const NodeDef* node, bool* new_shapes); + // Update the shapes for node 'n'. If output shapes for n have changed, + // enqueue its fanout in 'new_shapes'. + Status UpdateShapes(SymbolicShapeRefiner* shape_refiner, + const std::unordered_map& + resource_handles, + const NodeDef* n, bool* new_shapes) const; + // Propagate the shapes for the nodes enqueued in new_shapes and their + // transitive fanout until a fixed point is reached. + Status PropagateShapes( + SymbolicShapeRefiner* shape_refiner, TopoQueue* new_shapes, + const std::unordered_map& + resource_handles, + int num_loops) const; + + // Data members + const GrapplerItem& item_; + std::map> input_properties_; + std::map> output_properties_; + const std::vector missing_properties_; +}; + +} // end namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_COSTS_GRAPH_PROPERTIES_H_ diff --git a/linux/include/tensorflow/core/grappler/costs/op_context.h b/linux/include/tensorflow/core/grappler/costs/op_context.h new file mode 100644 index 000000000..6391de4a9 --- /dev/null +++ b/linux/include/tensorflow/core/grappler/costs/op_context.h @@ -0,0 +1,39 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_COSTS_OP_CONTEXT_H_ +#define TENSORFLOW_CORE_GRAPPLER_COSTS_OP_CONTEXT_H_ + +#include "tensorflow/core/framework/function.pb.h" +#include "tensorflow/core/grappler/costs/op_performance_data.pb.h" + +namespace tensorflow { +namespace grappler { + +// A structure to keep the context of op execution, including its shape, +// execution context, and other relevant information. +struct OpContext { + string name; + string device_name; + OpInfo op_info; + const FunctionDefLibrary* function_library; // Not owned. + + OpContext() { function_library = nullptr; } +}; + +} // end namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_COSTS_OP_CONTEXT_H_ diff --git a/linux/include/tensorflow/core/grappler/costs/op_level_cost_estimator.h b/linux/include/tensorflow/core/grappler/costs/op_level_cost_estimator.h new file mode 100644 index 000000000..a277dfdf6 --- /dev/null +++ b/linux/include/tensorflow/core/grappler/costs/op_level_cost_estimator.h @@ -0,0 +1,213 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_COSTS_OP_LEVEL_COST_ESTIMATOR_H_ +#define TENSORFLOW_CORE_GRAPPLER_COSTS_OP_LEVEL_COST_ESTIMATOR_H_ + +#include +#include +#include + +#include "tensorflow/core/grappler/costs/cost_estimator.h" +#include "tensorflow/core/grappler/costs/op_context.h" +#include "tensorflow/core/grappler/costs/op_performance_data.pb.h" +#include "tensorflow/core/util/padding.h" + +namespace tensorflow { +namespace grappler { + +bool GetTensorShapeProtoFromTensorProto(const TensorProto& tensor_proto, + TensorShapeProto* tensor_shape_proto); +TensorShapeProto MaybeGetMinimumShape(const TensorShapeProto& original_shape, + int rank, bool* found_unknown_shapes); + +class OpLevelCostEstimator { + public: + OpLevelCostEstimator(); + virtual ~OpLevelCostEstimator() {} + + virtual Costs PredictCosts(const OpContext& op_context) const; + + // Basic device performance info, sufficient for roofline estimate. + struct DeviceInfo { + double gigaops; // Billions of operations executed per second. + double gb_per_sec; // Bandwidth to main memory in GB per second. + }; + + // Returns basic device performance info. + virtual DeviceInfo GetDeviceInfo(const DeviceProperties& device) const; + + protected: + // Predict cost of an op for which no accurate estimator is defined. + Costs PredictCostOfAnUnknownOp(const OpContext& op_context) const; + + // Naive cost estimate based on the given operations count and total + // input/output tensor sizes of the given op_info combined. + Costs PredictOpCountBasedCost(double operations, const OpInfo& op_info) const; + + // Naive cost estimate based on the given operations count and the given total + // io size in bytes. Sizes of op_info inputs and outputs are not taken into + // consideration. + Costs PredictOpCountBasedCost(double operations, double total_io_bytes, + const OpInfo& op_info) const; + + // This family of routines counts the number of operations to perform the + // specified TensorFlow Op. + struct MatMulDimensions { + int m; + int n; + int k; + }; + struct ConvolutionDimensions { + int64 batch; // Batch size. + int64 ix; // Input size x. + int64 iy; // Input size y. + int64 iz; // Input depth. + int64 kx; // Kernel x. + int64 ky; // Kernel y. + int64 oz; // Output depth. + int64 ox; // Output size x. + int64 oy; // Output size y. + int64 sx; // Stride x. + int64 sy; // Stride y. + Padding padding; // SAME or VALID. + }; + int64 CountConv2DOperations(const OpInfo& op_features, + bool* found_unknown_shapes) const; + int64 CountConv2DOperations(const OpInfo& op_features, + ConvolutionDimensions* conv_info, + bool* found_unknown_shapes) const; + int64 CountMatMulOperations(const OpInfo& op_features, + bool* found_unknown_shapes) const; + int64 CountMatMulOperations(const OpInfo& op_features, + MatMulDimensions* mat_mul, + bool* found_unknown_shapes) const; + int64 CountBatchMatMulOperations(const OpInfo& op_features, + bool* found_unknown_shapes) const; + int64 CountConv2DBackpropInputOperations(const OpInfo& op_features, + ConvolutionDimensions* conv_info, + bool* found_unknown_shapes) const; + int64 CountConv2DBackpropFilterOperations(const OpInfo& op_features, + ConvolutionDimensions* conv_info, + bool* found_unknown_shapes) const; + + // Calculate the element count of an input/output tensor. + int64 CalculateTensorElementCount(const OpInfo::TensorProperties& tensor, + bool* found_unknown_shapes) const; + + // Calculate the total size in bytes of an input/output tensor. + int64 CalculateTensorSize(const OpInfo::TensorProperties& tensor, + bool* found_unknown_shapes) const; + + // Calculate the element count of the largest + // input of specified TensorFlow op. + int64 CalculateLargestInputCount(const OpInfo& op_features, + bool* found_unknown_shapes) const; + + // Calculate the total size in bytes of the all + // the inputs of specified TensorFlow op. + int64 CalculateInputSize(const OpInfo& op_features, + bool* found_unknown_shapes) const; + + // Calculate the total size in bytes of the all + // the outputs of specified TensorFlow op. + int64 CalculateOutputSize(const OpInfo& op_features, + bool* found_unknown_shapes) const; + + // This family of routines predicts the costs to + // perform the specified TensorFlow Op on the + // device represented by a subclass. The default + // implementation just divides the operations to + // perform the op (from the "Count" routines, + // above) by the device peak operations per + // second. + // Implementation of costs other than + // execution_time is optional, depending on the + // device. + Costs PredictConv2D(const OpContext& op_context) const; + Costs PredictCwiseOp(const OpContext& op_context) const; + Costs PredictConv2DBackpropInput(const OpContext& op_context) const; + Costs PredictConv2DBackpropFilter(const OpContext& op_context) const; + Costs PredictFusedConv2DBiasActivation(const OpContext& op_context) const; + Costs PredictMatMul(const OpContext& op_context) const; + Costs PredictNoOp(const OpContext& op_context) const; + Costs PredictIdentity(const OpContext& op_context) const; + Costs PredictVariable(const OpContext& op_context) const; + Costs PredictBatchMatMul(const OpContext& op_context) const; + Costs PredictMetadata(const OpContext& op_context) const; + Costs PredictGatherOrSlice(const OpContext& op_context) const; + Costs PredictMaxPool(const OpContext& op_context) const; + Costs PredictMaxPoolGrad(const OpContext& op_context) const; + Costs PredictAvgPool(const OpContext& op_context) const; + Costs PredictAvgPoolGrad(const OpContext& op_context) const; + Costs PredictFusedBatchNorm(const OpContext& op_context) const; + Costs PredictFusedBatchNormGrad(const OpContext& op_context) const; + + // Generic cost prediction method for fused operations. + Costs PredictFusedOp(const OpContext& op_context, + const std::vector& fused_op_contexts) const; + + // Utility function for safe division. Returns 0 + // if rhs is 0 or negative. + static double SafeDiv(const double lhs, const double rhs) { + if (rhs > 0) { + return lhs / rhs; + } else { + return 0.0; + } + } + + // For convolution and its grad ops. + static ConvolutionDimensions ConvolutionDimensionsFromInputs( + const TensorShapeProto& original_image_shape, + const TensorShapeProto& original_filter_shape, const OpInfo& op_info, + bool* found_unknown_shapes); + + // For Pooling, FusedBatchNorm, and their grad ops. + static ConvolutionDimensions OpDimensionsFromInputs( + const TensorShapeProto& original_image_shape, const OpInfo& op_info, + bool* found_unknown_shapes); + + // Helper to construct child operation contexts for the component operations + // of fused ops. + static OpContext FusedChildContext( + const OpContext& parent, const string& op_name, + const OpInfo::TensorProperties& output, + const std::vector& inputs); + + // Helper to construct tensor shapes. + static OpInfo::TensorProperties DescribeTensor( + DataType type, const std::vector& dims); + + // This method calculates the execution time depending on whether IO can + // overlap with computation. It assumes the memory and the compute times have + // already been calculated. + void CombineCostsAndUpdateExecutionTime(Costs* costs) const; + + protected: + std::map elementwise_ops_; + typedef std::function CostImpl; + std::map device_cost_impl_; + // If true, assume compute and memory overlap; hence, the op cost is max of + // compute_time and memory_time, insteaf of sum of those two. + bool compute_memory_overlap_; + + private: + friend class OpLevelCostEstimatorTest; +}; + +} // end namespace grappler +} // end namespace tensorflow +#endif // TENSORFLOW_CORE_GRAPPLER_COSTS_OP_LEVEL_COST_ESTIMATOR_H_ diff --git a/linux/include/tensorflow/core/grappler/costs/op_performance_data.pb.h b/linux/include/tensorflow/core/grappler/costs/op_performance_data.pb.h new file mode 100644 index 000000000..cef77c758 --- /dev/null +++ b/linux/include/tensorflow/core/grappler/costs/op_performance_data.pb.h @@ -0,0 +1,2561 @@ +// Generated by the protocol buffer compiler. DO NOT EDIT! +// source: tensorflow/core/grappler/costs/op_performance_data.proto + +#ifndef PROTOBUF_INCLUDED_tensorflow_2fcore_2fgrappler_2fcosts_2fop_5fperformance_5fdata_2eproto +#define PROTOBUF_INCLUDED_tensorflow_2fcore_2fgrappler_2fcosts_2fop_5fperformance_5fdata_2eproto + +#include + +#include + +#if GOOGLE_PROTOBUF_VERSION < 3006000 +#error This file was generated by a newer version of protoc which is +#error incompatible with your Protocol Buffer headers. Please update +#error your headers. +#endif +#if 3006000 < GOOGLE_PROTOBUF_MIN_PROTOC_VERSION +#error This file was generated by an older version of protoc which is +#error incompatible with your Protocol Buffer headers. Please +#error regenerate this file with a newer version of protoc. +#endif + +#include +#include +#include +#include +#include +#include +#include +#include +#include // IWYU pragma: export +#include // IWYU pragma: export +#include // IWYU pragma: export +#include +#include +#include +#include "tensorflow/core/framework/tensor.pb.h" +#include "tensorflow/core/framework/tensor_shape.pb.h" +#include "tensorflow/core/framework/types.pb.h" +#include "tensorflow/core/framework/attr_value.pb.h" +#include "tensorflow/core/protobuf/device_properties.pb.h" +// @@protoc_insertion_point(includes) +#define PROTOBUF_INTERNAL_EXPORT_protobuf_tensorflow_2fcore_2fgrappler_2fcosts_2fop_5fperformance_5fdata_2eproto + +namespace protobuf_tensorflow_2fcore_2fgrappler_2fcosts_2fop_5fperformance_5fdata_2eproto { +// Internal implementation detail -- do not use these members. +struct TableStruct { + static const ::google::protobuf::internal::ParseTableField entries[]; + static const ::google::protobuf::internal::AuxillaryParseTableField aux[]; + static const ::google::protobuf::internal::ParseTable schema[9]; + static const ::google::protobuf::internal::FieldMetadata field_metadata[]; + static const ::google::protobuf::internal::SerializationTable serialization_table[]; + static const ::google::protobuf::uint32 offsets[]; +}; +void AddDescriptors(); +} // namespace protobuf_tensorflow_2fcore_2fgrappler_2fcosts_2fop_5fperformance_5fdata_2eproto +namespace tensorflow { +class LogNormalDistribution; +class LogNormalDistributionDefaultTypeInternal; +extern LogNormalDistributionDefaultTypeInternal _LogNormalDistribution_default_instance_; +class NormalDistribution; +class NormalDistributionDefaultTypeInternal; +extern NormalDistributionDefaultTypeInternal _NormalDistribution_default_instance_; +class OpInfo; +class OpInfoDefaultTypeInternal; +extern OpInfoDefaultTypeInternal _OpInfo_default_instance_; +class OpInfo_AttrEntry_DoNotUse; +class OpInfo_AttrEntry_DoNotUseDefaultTypeInternal; +extern OpInfo_AttrEntry_DoNotUseDefaultTypeInternal _OpInfo_AttrEntry_DoNotUse_default_instance_; +class OpInfo_TensorProperties; +class OpInfo_TensorPropertiesDefaultTypeInternal; +extern OpInfo_TensorPropertiesDefaultTypeInternal _OpInfo_TensorProperties_default_instance_; +class OpPerformance; +class OpPerformanceDefaultTypeInternal; +extern OpPerformanceDefaultTypeInternal _OpPerformance_default_instance_; +class OpPerformanceList; +class OpPerformanceListDefaultTypeInternal; +extern OpPerformanceListDefaultTypeInternal _OpPerformanceList_default_instance_; +class OpPerformance_OpMemory; +class OpPerformance_OpMemoryDefaultTypeInternal; +extern OpPerformance_OpMemoryDefaultTypeInternal _OpPerformance_OpMemory_default_instance_; +class SessionInfo; +class SessionInfoDefaultTypeInternal; +extern SessionInfoDefaultTypeInternal _SessionInfo_default_instance_; +} // namespace tensorflow +namespace google { +namespace protobuf { +template<> ::tensorflow::LogNormalDistribution* Arena::CreateMaybeMessage<::tensorflow::LogNormalDistribution>(Arena*); +template<> ::tensorflow::NormalDistribution* Arena::CreateMaybeMessage<::tensorflow::NormalDistribution>(Arena*); +template<> ::tensorflow::OpInfo* Arena::CreateMaybeMessage<::tensorflow::OpInfo>(Arena*); +template<> ::tensorflow::OpInfo_AttrEntry_DoNotUse* Arena::CreateMaybeMessage<::tensorflow::OpInfo_AttrEntry_DoNotUse>(Arena*); +template<> ::tensorflow::OpInfo_TensorProperties* Arena::CreateMaybeMessage<::tensorflow::OpInfo_TensorProperties>(Arena*); +template<> ::tensorflow::OpPerformance* Arena::CreateMaybeMessage<::tensorflow::OpPerformance>(Arena*); +template<> ::tensorflow::OpPerformanceList* Arena::CreateMaybeMessage<::tensorflow::OpPerformanceList>(Arena*); +template<> ::tensorflow::OpPerformance_OpMemory* Arena::CreateMaybeMessage<::tensorflow::OpPerformance_OpMemory>(Arena*); +template<> ::tensorflow::SessionInfo* Arena::CreateMaybeMessage<::tensorflow::SessionInfo>(Arena*); +} // namespace protobuf +} // namespace google +namespace tensorflow { + +// =================================================================== + +class SessionInfo : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.SessionInfo) */ { + public: + SessionInfo(); + virtual ~SessionInfo(); + + SessionInfo(const SessionInfo& from); + + inline SessionInfo& operator=(const SessionInfo& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + SessionInfo(SessionInfo&& from) noexcept + : SessionInfo() { + *this = ::std::move(from); + } + + inline SessionInfo& operator=(SessionInfo&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const SessionInfo& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const SessionInfo* internal_default_instance() { + return reinterpret_cast( + &_SessionInfo_default_instance_); + } + static constexpr int kIndexInFileMessages = + 0; + + void UnsafeArenaSwap(SessionInfo* other); + void Swap(SessionInfo* other); + friend void swap(SessionInfo& a, SessionInfo& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline SessionInfo* New() const final { + return CreateMaybeMessage(NULL); + } + + SessionInfo* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const SessionInfo& from); + void MergeFrom(const SessionInfo& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(SessionInfo* other); + protected: + explicit SessionInfo(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // int64 intra_op_parallelism = 1; + void clear_intra_op_parallelism(); + static const int kIntraOpParallelismFieldNumber = 1; + ::google::protobuf::int64 intra_op_parallelism() const; + void set_intra_op_parallelism(::google::protobuf::int64 value); + + // @@protoc_insertion_point(class_scope:tensorflow.SessionInfo) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::int64 intra_op_parallelism_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fgrappler_2fcosts_2fop_5fperformance_5fdata_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class OpInfo_AttrEntry_DoNotUse : public ::google::protobuf::internal::MapEntry { +public: + typedef ::google::protobuf::internal::MapEntry SuperType; + OpInfo_AttrEntry_DoNotUse(); + OpInfo_AttrEntry_DoNotUse(::google::protobuf::Arena* arena); + void MergeFrom(const OpInfo_AttrEntry_DoNotUse& other); + static const OpInfo_AttrEntry_DoNotUse* internal_default_instance() { return reinterpret_cast(&_OpInfo_AttrEntry_DoNotUse_default_instance_); } + void MergeFrom(const ::google::protobuf::Message& other) final; + ::google::protobuf::Metadata GetMetadata() const; +}; + +// ------------------------------------------------------------------- + +class OpInfo_TensorProperties : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.OpInfo.TensorProperties) */ { + public: + OpInfo_TensorProperties(); + virtual ~OpInfo_TensorProperties(); + + OpInfo_TensorProperties(const OpInfo_TensorProperties& from); + + inline OpInfo_TensorProperties& operator=(const OpInfo_TensorProperties& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + OpInfo_TensorProperties(OpInfo_TensorProperties&& from) noexcept + : OpInfo_TensorProperties() { + *this = ::std::move(from); + } + + inline OpInfo_TensorProperties& operator=(OpInfo_TensorProperties&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const OpInfo_TensorProperties& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const OpInfo_TensorProperties* internal_default_instance() { + return reinterpret_cast( + &_OpInfo_TensorProperties_default_instance_); + } + static constexpr int kIndexInFileMessages = + 2; + + void UnsafeArenaSwap(OpInfo_TensorProperties* other); + void Swap(OpInfo_TensorProperties* other); + friend void swap(OpInfo_TensorProperties& a, OpInfo_TensorProperties& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline OpInfo_TensorProperties* New() const final { + return CreateMaybeMessage(NULL); + } + + OpInfo_TensorProperties* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const OpInfo_TensorProperties& from); + void MergeFrom(const OpInfo_TensorProperties& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(OpInfo_TensorProperties* other); + protected: + explicit OpInfo_TensorProperties(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // .tensorflow.TensorShapeProto shape = 2; + bool has_shape() const; + void clear_shape(); + static const int kShapeFieldNumber = 2; + private: + const ::tensorflow::TensorShapeProto& _internal_shape() const; + public: + const ::tensorflow::TensorShapeProto& shape() const; + ::tensorflow::TensorShapeProto* release_shape(); + ::tensorflow::TensorShapeProto* mutable_shape(); + void set_allocated_shape(::tensorflow::TensorShapeProto* shape); + void unsafe_arena_set_allocated_shape( + ::tensorflow::TensorShapeProto* shape); + ::tensorflow::TensorShapeProto* unsafe_arena_release_shape(); + + // .tensorflow.TensorProto value = 3; + bool has_value() const; + void clear_value(); + static const int kValueFieldNumber = 3; + private: + const ::tensorflow::TensorProto& _internal_value() const; + public: + const ::tensorflow::TensorProto& value() const; + ::tensorflow::TensorProto* release_value(); + ::tensorflow::TensorProto* mutable_value(); + void set_allocated_value(::tensorflow::TensorProto* value); + void unsafe_arena_set_allocated_value( + ::tensorflow::TensorProto* value); + ::tensorflow::TensorProto* unsafe_arena_release_value(); + + // .tensorflow.DataType dtype = 1; + void clear_dtype(); + static const int kDtypeFieldNumber = 1; + ::tensorflow::DataType dtype() const; + void set_dtype(::tensorflow::DataType value); + + // @@protoc_insertion_point(class_scope:tensorflow.OpInfo.TensorProperties) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::tensorflow::TensorShapeProto* shape_; + ::tensorflow::TensorProto* value_; + int dtype_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fgrappler_2fcosts_2fop_5fperformance_5fdata_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class OpInfo : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.OpInfo) */ { + public: + OpInfo(); + virtual ~OpInfo(); + + OpInfo(const OpInfo& from); + + inline OpInfo& operator=(const OpInfo& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + OpInfo(OpInfo&& from) noexcept + : OpInfo() { + *this = ::std::move(from); + } + + inline OpInfo& operator=(OpInfo&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const OpInfo& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const OpInfo* internal_default_instance() { + return reinterpret_cast( + &_OpInfo_default_instance_); + } + static constexpr int kIndexInFileMessages = + 3; + + void UnsafeArenaSwap(OpInfo* other); + void Swap(OpInfo* other); + friend void swap(OpInfo& a, OpInfo& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline OpInfo* New() const final { + return CreateMaybeMessage(NULL); + } + + OpInfo* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const OpInfo& from); + void MergeFrom(const OpInfo& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(OpInfo* other); + protected: + explicit OpInfo(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + typedef OpInfo_TensorProperties TensorProperties; + + // accessors ------------------------------------------------------- + + // map attr = 2; + int attr_size() const; + void clear_attr(); + static const int kAttrFieldNumber = 2; + const ::google::protobuf::Map< ::std::string, ::tensorflow::AttrValue >& + attr() const; + ::google::protobuf::Map< ::std::string, ::tensorflow::AttrValue >* + mutable_attr(); + + // repeated .tensorflow.OpInfo.TensorProperties inputs = 3; + int inputs_size() const; + void clear_inputs(); + static const int kInputsFieldNumber = 3; + ::tensorflow::OpInfo_TensorProperties* mutable_inputs(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::OpInfo_TensorProperties >* + mutable_inputs(); + const ::tensorflow::OpInfo_TensorProperties& inputs(int index) const; + ::tensorflow::OpInfo_TensorProperties* add_inputs(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::OpInfo_TensorProperties >& + inputs() const; + + // repeated .tensorflow.OpInfo.TensorProperties outputs = 5; + int outputs_size() const; + void clear_outputs(); + static const int kOutputsFieldNumber = 5; + ::tensorflow::OpInfo_TensorProperties* mutable_outputs(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::OpInfo_TensorProperties >* + mutable_outputs(); + const ::tensorflow::OpInfo_TensorProperties& outputs(int index) const; + ::tensorflow::OpInfo_TensorProperties* add_outputs(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::OpInfo_TensorProperties >& + outputs() const; + + // string op = 1; + void clear_op(); + static const int kOpFieldNumber = 1; + const ::std::string& op() const; + void set_op(const ::std::string& value); + #if LANG_CXX11 + void set_op(::std::string&& value); + #endif + void set_op(const char* value); + void set_op(const char* value, size_t size); + ::std::string* mutable_op(); + ::std::string* release_op(); + void set_allocated_op(::std::string* op); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_op(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_op( + ::std::string* op); + + // .tensorflow.DeviceProperties device = 4; + bool has_device() const; + void clear_device(); + static const int kDeviceFieldNumber = 4; + private: + const ::tensorflow::DeviceProperties& _internal_device() const; + public: + const ::tensorflow::DeviceProperties& device() const; + ::tensorflow::DeviceProperties* release_device(); + ::tensorflow::DeviceProperties* mutable_device(); + void set_allocated_device(::tensorflow::DeviceProperties* device); + void unsafe_arena_set_allocated_device( + ::tensorflow::DeviceProperties* device); + ::tensorflow::DeviceProperties* unsafe_arena_release_device(); + + // .tensorflow.SessionInfo session_info = 6; + bool has_session_info() const; + void clear_session_info(); + static const int kSessionInfoFieldNumber = 6; + private: + const ::tensorflow::SessionInfo& _internal_session_info() const; + public: + const ::tensorflow::SessionInfo& session_info() const; + ::tensorflow::SessionInfo* release_session_info(); + ::tensorflow::SessionInfo* mutable_session_info(); + void set_allocated_session_info(::tensorflow::SessionInfo* session_info); + void unsafe_arena_set_allocated_session_info( + ::tensorflow::SessionInfo* session_info); + ::tensorflow::SessionInfo* unsafe_arena_release_session_info(); + + // @@protoc_insertion_point(class_scope:tensorflow.OpInfo) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::internal::MapField< + OpInfo_AttrEntry_DoNotUse, + ::std::string, ::tensorflow::AttrValue, + ::google::protobuf::internal::WireFormatLite::TYPE_STRING, + ::google::protobuf::internal::WireFormatLite::TYPE_MESSAGE, + 0 > attr_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::OpInfo_TensorProperties > inputs_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::OpInfo_TensorProperties > outputs_; + ::google::protobuf::internal::ArenaStringPtr op_; + ::tensorflow::DeviceProperties* device_; + ::tensorflow::SessionInfo* session_info_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fgrappler_2fcosts_2fop_5fperformance_5fdata_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class NormalDistribution : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.NormalDistribution) */ { + public: + NormalDistribution(); + virtual ~NormalDistribution(); + + NormalDistribution(const NormalDistribution& from); + + inline NormalDistribution& operator=(const NormalDistribution& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + NormalDistribution(NormalDistribution&& from) noexcept + : NormalDistribution() { + *this = ::std::move(from); + } + + inline NormalDistribution& operator=(NormalDistribution&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const NormalDistribution& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const NormalDistribution* internal_default_instance() { + return reinterpret_cast( + &_NormalDistribution_default_instance_); + } + static constexpr int kIndexInFileMessages = + 4; + + void UnsafeArenaSwap(NormalDistribution* other); + void Swap(NormalDistribution* other); + friend void swap(NormalDistribution& a, NormalDistribution& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline NormalDistribution* New() const final { + return CreateMaybeMessage(NULL); + } + + NormalDistribution* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const NormalDistribution& from); + void MergeFrom(const NormalDistribution& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(NormalDistribution* other); + protected: + explicit NormalDistribution(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // double mu = 1; + void clear_mu(); + static const int kMuFieldNumber = 1; + double mu() const; + void set_mu(double value); + + // double sigma = 2; + void clear_sigma(); + static const int kSigmaFieldNumber = 2; + double sigma() const; + void set_sigma(double value); + + // @@protoc_insertion_point(class_scope:tensorflow.NormalDistribution) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + double mu_; + double sigma_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fgrappler_2fcosts_2fop_5fperformance_5fdata_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class LogNormalDistribution : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.LogNormalDistribution) */ { + public: + LogNormalDistribution(); + virtual ~LogNormalDistribution(); + + LogNormalDistribution(const LogNormalDistribution& from); + + inline LogNormalDistribution& operator=(const LogNormalDistribution& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + LogNormalDistribution(LogNormalDistribution&& from) noexcept + : LogNormalDistribution() { + *this = ::std::move(from); + } + + inline LogNormalDistribution& operator=(LogNormalDistribution&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const LogNormalDistribution& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const LogNormalDistribution* internal_default_instance() { + return reinterpret_cast( + &_LogNormalDistribution_default_instance_); + } + static constexpr int kIndexInFileMessages = + 5; + + void UnsafeArenaSwap(LogNormalDistribution* other); + void Swap(LogNormalDistribution* other); + friend void swap(LogNormalDistribution& a, LogNormalDistribution& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline LogNormalDistribution* New() const final { + return CreateMaybeMessage(NULL); + } + + LogNormalDistribution* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const LogNormalDistribution& from); + void MergeFrom(const LogNormalDistribution& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(LogNormalDistribution* other); + protected: + explicit LogNormalDistribution(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // double mu = 1; + void clear_mu(); + static const int kMuFieldNumber = 1; + double mu() const; + void set_mu(double value); + + // double sigma = 2; + void clear_sigma(); + static const int kSigmaFieldNumber = 2; + double sigma() const; + void set_sigma(double value); + + // @@protoc_insertion_point(class_scope:tensorflow.LogNormalDistribution) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + double mu_; + double sigma_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fgrappler_2fcosts_2fop_5fperformance_5fdata_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class OpPerformance_OpMemory : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.OpPerformance.OpMemory) */ { + public: + OpPerformance_OpMemory(); + virtual ~OpPerformance_OpMemory(); + + OpPerformance_OpMemory(const OpPerformance_OpMemory& from); + + inline OpPerformance_OpMemory& operator=(const OpPerformance_OpMemory& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + OpPerformance_OpMemory(OpPerformance_OpMemory&& from) noexcept + : OpPerformance_OpMemory() { + *this = ::std::move(from); + } + + inline OpPerformance_OpMemory& operator=(OpPerformance_OpMemory&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const OpPerformance_OpMemory& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const OpPerformance_OpMemory* internal_default_instance() { + return reinterpret_cast( + &_OpPerformance_OpMemory_default_instance_); + } + static constexpr int kIndexInFileMessages = + 6; + + void UnsafeArenaSwap(OpPerformance_OpMemory* other); + void Swap(OpPerformance_OpMemory* other); + friend void swap(OpPerformance_OpMemory& a, OpPerformance_OpMemory& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline OpPerformance_OpMemory* New() const final { + return CreateMaybeMessage(NULL); + } + + OpPerformance_OpMemory* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const OpPerformance_OpMemory& from); + void MergeFrom(const OpPerformance_OpMemory& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(OpPerformance_OpMemory* other); + protected: + explicit OpPerformance_OpMemory(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated int64 output_memory = 1; + int output_memory_size() const; + void clear_output_memory(); + static const int kOutputMemoryFieldNumber = 1; + ::google::protobuf::int64 output_memory(int index) const; + void set_output_memory(int index, ::google::protobuf::int64 value); + void add_output_memory(::google::protobuf::int64 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& + output_memory() const; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* + mutable_output_memory(); + + // int64 temp_memory = 2; + void clear_temp_memory(); + static const int kTempMemoryFieldNumber = 2; + ::google::protobuf::int64 temp_memory() const; + void set_temp_memory(::google::protobuf::int64 value); + + // int64 device_temp_memory = 3 [deprecated = true]; + GOOGLE_PROTOBUF_DEPRECATED_ATTR void clear_device_temp_memory(); + GOOGLE_PROTOBUF_DEPRECATED_ATTR static const int kDeviceTempMemoryFieldNumber = 3; + GOOGLE_PROTOBUF_DEPRECATED_ATTR ::google::protobuf::int64 device_temp_memory() const; + GOOGLE_PROTOBUF_DEPRECATED_ATTR void set_device_temp_memory(::google::protobuf::int64 value); + + // int64 persistent_memory = 4; + void clear_persistent_memory(); + static const int kPersistentMemoryFieldNumber = 4; + ::google::protobuf::int64 persistent_memory() const; + void set_persistent_memory(::google::protobuf::int64 value); + + // int64 device_persistent_memory = 5 [deprecated = true]; + GOOGLE_PROTOBUF_DEPRECATED_ATTR void clear_device_persistent_memory(); + GOOGLE_PROTOBUF_DEPRECATED_ATTR static const int kDevicePersistentMemoryFieldNumber = 5; + GOOGLE_PROTOBUF_DEPRECATED_ATTR ::google::protobuf::int64 device_persistent_memory() const; + GOOGLE_PROTOBUF_DEPRECATED_ATTR void set_device_persistent_memory(::google::protobuf::int64 value); + + // @@protoc_insertion_point(class_scope:tensorflow.OpPerformance.OpMemory) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedField< ::google::protobuf::int64 > output_memory_; + mutable int _output_memory_cached_byte_size_; + ::google::protobuf::int64 temp_memory_; + ::google::protobuf::int64 device_temp_memory_; + ::google::protobuf::int64 persistent_memory_; + ::google::protobuf::int64 device_persistent_memory_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fgrappler_2fcosts_2fop_5fperformance_5fdata_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class OpPerformance : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.OpPerformance) */ { + public: + OpPerformance(); + virtual ~OpPerformance(); + + OpPerformance(const OpPerformance& from); + + inline OpPerformance& operator=(const OpPerformance& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + OpPerformance(OpPerformance&& from) noexcept + : OpPerformance() { + *this = ::std::move(from); + } + + inline OpPerformance& operator=(OpPerformance&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const OpPerformance& default_instance(); + + enum ExecutionTimeCase { + kExecutionTimeNormal = 10, + kExecutionTimeLogNormal = 11, + EXECUTION_TIME_NOT_SET = 0, + }; + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const OpPerformance* internal_default_instance() { + return reinterpret_cast( + &_OpPerformance_default_instance_); + } + static constexpr int kIndexInFileMessages = + 7; + + void UnsafeArenaSwap(OpPerformance* other); + void Swap(OpPerformance* other); + friend void swap(OpPerformance& a, OpPerformance& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline OpPerformance* New() const final { + return CreateMaybeMessage(NULL); + } + + OpPerformance* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const OpPerformance& from); + void MergeFrom(const OpPerformance& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(OpPerformance* other); + protected: + explicit OpPerformance(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + typedef OpPerformance_OpMemory OpMemory; + + // accessors ------------------------------------------------------- + + // string node = 5; + void clear_node(); + static const int kNodeFieldNumber = 5; + const ::std::string& node() const; + void set_node(const ::std::string& value); + #if LANG_CXX11 + void set_node(::std::string&& value); + #endif + void set_node(const char* value); + void set_node(const char* value, size_t size); + ::std::string* mutable_node(); + ::std::string* release_node(); + void set_allocated_node(::std::string* node); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + ::std::string* unsafe_arena_release_node(); + PROTOBUF_RUNTIME_DEPRECATED("The unsafe_arena_ accessors for" + " string fields are deprecated and will be removed in a" + " future release.") + void unsafe_arena_set_allocated_node( + ::std::string* node); + + // .tensorflow.OpInfo op = 1; + bool has_op() const; + void clear_op(); + static const int kOpFieldNumber = 1; + private: + const ::tensorflow::OpInfo& _internal_op() const; + public: + const ::tensorflow::OpInfo& op() const; + ::tensorflow::OpInfo* release_op(); + ::tensorflow::OpInfo* mutable_op(); + void set_allocated_op(::tensorflow::OpInfo* op); + void unsafe_arena_set_allocated_op( + ::tensorflow::OpInfo* op); + ::tensorflow::OpInfo* unsafe_arena_release_op(); + + // .tensorflow.OpPerformance.OpMemory op_memory = 9; + bool has_op_memory() const; + void clear_op_memory(); + static const int kOpMemoryFieldNumber = 9; + private: + const ::tensorflow::OpPerformance_OpMemory& _internal_op_memory() const; + public: + const ::tensorflow::OpPerformance_OpMemory& op_memory() const; + ::tensorflow::OpPerformance_OpMemory* release_op_memory(); + ::tensorflow::OpPerformance_OpMemory* mutable_op_memory(); + void set_allocated_op_memory(::tensorflow::OpPerformance_OpMemory* op_memory); + void unsafe_arena_set_allocated_op_memory( + ::tensorflow::OpPerformance_OpMemory* op_memory); + ::tensorflow::OpPerformance_OpMemory* unsafe_arena_release_op_memory(); + + // .tensorflow.SessionInfo session_info = 12 [deprecated = true]; + GOOGLE_PROTOBUF_DEPRECATED_ATTR bool has_session_info() const; + GOOGLE_PROTOBUF_DEPRECATED_ATTR void clear_session_info(); + GOOGLE_PROTOBUF_DEPRECATED_ATTR static const int kSessionInfoFieldNumber = 12; + private: + const ::tensorflow::SessionInfo& _internal_session_info() const; + public: + GOOGLE_PROTOBUF_DEPRECATED_ATTR const ::tensorflow::SessionInfo& session_info() const; + GOOGLE_PROTOBUF_DEPRECATED_ATTR ::tensorflow::SessionInfo* release_session_info(); + GOOGLE_PROTOBUF_DEPRECATED_ATTR ::tensorflow::SessionInfo* mutable_session_info(); + GOOGLE_PROTOBUF_DEPRECATED_ATTR void set_allocated_session_info(::tensorflow::SessionInfo* session_info); + GOOGLE_PROTOBUF_DEPRECATED_ATTR void unsafe_arena_set_allocated_session_info( + ::tensorflow::SessionInfo* session_info); + GOOGLE_PROTOBUF_DEPRECATED_ATTR ::tensorflow::SessionInfo* unsafe_arena_release_session_info(); + + // int64 temporary_memory_size = 2; + void clear_temporary_memory_size(); + static const int kTemporaryMemorySizeFieldNumber = 2; + ::google::protobuf::int64 temporary_memory_size() const; + void set_temporary_memory_size(::google::protobuf::int64 value); + + // int64 compute_cost = 3; + void clear_compute_cost(); + static const int kComputeCostFieldNumber = 3; + ::google::protobuf::int64 compute_cost() const; + void set_compute_cost(::google::protobuf::int64 value); + + // double compute_efficiency = 4; + void clear_compute_efficiency(); + static const int kComputeEfficiencyFieldNumber = 4; + double compute_efficiency() const; + void set_compute_efficiency(double value); + + // int64 compute_time = 6; + void clear_compute_time(); + static const int kComputeTimeFieldNumber = 6; + ::google::protobuf::int64 compute_time() const; + void set_compute_time(::google::protobuf::int64 value); + + // int64 memory_time = 7; + void clear_memory_time(); + static const int kMemoryTimeFieldNumber = 7; + ::google::protobuf::int64 memory_time() const; + void set_memory_time(::google::protobuf::int64 value); + + // double memory_efficiency = 8; + void clear_memory_efficiency(); + static const int kMemoryEfficiencyFieldNumber = 8; + double memory_efficiency() const; + void set_memory_efficiency(double value); + + // .tensorflow.NormalDistribution execution_time_normal = 10; + bool has_execution_time_normal() const; + void clear_execution_time_normal(); + static const int kExecutionTimeNormalFieldNumber = 10; + private: + const ::tensorflow::NormalDistribution& _internal_execution_time_normal() const; + public: + const ::tensorflow::NormalDistribution& execution_time_normal() const; + ::tensorflow::NormalDistribution* release_execution_time_normal(); + ::tensorflow::NormalDistribution* mutable_execution_time_normal(); + void set_allocated_execution_time_normal(::tensorflow::NormalDistribution* execution_time_normal); + void unsafe_arena_set_allocated_execution_time_normal( + ::tensorflow::NormalDistribution* execution_time_normal); + ::tensorflow::NormalDistribution* unsafe_arena_release_execution_time_normal(); + + // .tensorflow.LogNormalDistribution execution_time_log_normal = 11; + bool has_execution_time_log_normal() const; + void clear_execution_time_log_normal(); + static const int kExecutionTimeLogNormalFieldNumber = 11; + private: + const ::tensorflow::LogNormalDistribution& _internal_execution_time_log_normal() const; + public: + const ::tensorflow::LogNormalDistribution& execution_time_log_normal() const; + ::tensorflow::LogNormalDistribution* release_execution_time_log_normal(); + ::tensorflow::LogNormalDistribution* mutable_execution_time_log_normal(); + void set_allocated_execution_time_log_normal(::tensorflow::LogNormalDistribution* execution_time_log_normal); + void unsafe_arena_set_allocated_execution_time_log_normal( + ::tensorflow::LogNormalDistribution* execution_time_log_normal); + ::tensorflow::LogNormalDistribution* unsafe_arena_release_execution_time_log_normal(); + + void clear_execution_time(); + ExecutionTimeCase execution_time_case() const; + // @@protoc_insertion_point(class_scope:tensorflow.OpPerformance) + private: + void set_has_execution_time_normal(); + void set_has_execution_time_log_normal(); + + inline bool has_execution_time() const; + inline void clear_has_execution_time(); + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::internal::ArenaStringPtr node_; + ::tensorflow::OpInfo* op_; + ::tensorflow::OpPerformance_OpMemory* op_memory_; + ::tensorflow::SessionInfo* session_info_; + ::google::protobuf::int64 temporary_memory_size_; + ::google::protobuf::int64 compute_cost_; + double compute_efficiency_; + ::google::protobuf::int64 compute_time_; + ::google::protobuf::int64 memory_time_; + double memory_efficiency_; + union ExecutionTimeUnion { + ExecutionTimeUnion() {} + ::tensorflow::NormalDistribution* execution_time_normal_; + ::tensorflow::LogNormalDistribution* execution_time_log_normal_; + } execution_time_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + ::google::protobuf::uint32 _oneof_case_[1]; + + friend struct ::protobuf_tensorflow_2fcore_2fgrappler_2fcosts_2fop_5fperformance_5fdata_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class OpPerformanceList : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.OpPerformanceList) */ { + public: + OpPerformanceList(); + virtual ~OpPerformanceList(); + + OpPerformanceList(const OpPerformanceList& from); + + inline OpPerformanceList& operator=(const OpPerformanceList& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + OpPerformanceList(OpPerformanceList&& from) noexcept + : OpPerformanceList() { + *this = ::std::move(from); + } + + inline OpPerformanceList& operator=(OpPerformanceList&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const OpPerformanceList& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const OpPerformanceList* internal_default_instance() { + return reinterpret_cast( + &_OpPerformanceList_default_instance_); + } + static constexpr int kIndexInFileMessages = + 8; + + void UnsafeArenaSwap(OpPerformanceList* other); + void Swap(OpPerformanceList* other); + friend void swap(OpPerformanceList& a, OpPerformanceList& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline OpPerformanceList* New() const final { + return CreateMaybeMessage(NULL); + } + + OpPerformanceList* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const OpPerformanceList& from); + void MergeFrom(const OpPerformanceList& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(OpPerformanceList* other); + protected: + explicit OpPerformanceList(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated .tensorflow.OpPerformance op_performance = 1; + int op_performance_size() const; + void clear_op_performance(); + static const int kOpPerformanceFieldNumber = 1; + ::tensorflow::OpPerformance* mutable_op_performance(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::OpPerformance >* + mutable_op_performance(); + const ::tensorflow::OpPerformance& op_performance(int index) const; + ::tensorflow::OpPerformance* add_op_performance(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::OpPerformance >& + op_performance() const; + + // @@protoc_insertion_point(class_scope:tensorflow.OpPerformanceList) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::OpPerformance > op_performance_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fgrappler_2fcosts_2fop_5fperformance_5fdata_2eproto::TableStruct; +}; +// =================================================================== + + +// =================================================================== + +#ifdef __GNUC__ + #pragma GCC diagnostic push + #pragma GCC diagnostic ignored "-Wstrict-aliasing" +#endif // __GNUC__ +// SessionInfo + +// int64 intra_op_parallelism = 1; +inline void SessionInfo::clear_intra_op_parallelism() { + intra_op_parallelism_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 SessionInfo::intra_op_parallelism() const { + // @@protoc_insertion_point(field_get:tensorflow.SessionInfo.intra_op_parallelism) + return intra_op_parallelism_; +} +inline void SessionInfo::set_intra_op_parallelism(::google::protobuf::int64 value) { + + intra_op_parallelism_ = value; + // @@protoc_insertion_point(field_set:tensorflow.SessionInfo.intra_op_parallelism) +} + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// OpInfo_TensorProperties + +// .tensorflow.DataType dtype = 1; +inline void OpInfo_TensorProperties::clear_dtype() { + dtype_ = 0; +} +inline ::tensorflow::DataType OpInfo_TensorProperties::dtype() const { + // @@protoc_insertion_point(field_get:tensorflow.OpInfo.TensorProperties.dtype) + return static_cast< ::tensorflow::DataType >(dtype_); +} +inline void OpInfo_TensorProperties::set_dtype(::tensorflow::DataType value) { + + dtype_ = value; + // @@protoc_insertion_point(field_set:tensorflow.OpInfo.TensorProperties.dtype) +} + +// .tensorflow.TensorShapeProto shape = 2; +inline bool OpInfo_TensorProperties::has_shape() const { + return this != internal_default_instance() && shape_ != NULL; +} +inline const ::tensorflow::TensorShapeProto& OpInfo_TensorProperties::_internal_shape() const { + return *shape_; +} +inline const ::tensorflow::TensorShapeProto& OpInfo_TensorProperties::shape() const { + const ::tensorflow::TensorShapeProto* p = shape_; + // @@protoc_insertion_point(field_get:tensorflow.OpInfo.TensorProperties.shape) + return p != NULL ? *p : *reinterpret_cast( + &::tensorflow::_TensorShapeProto_default_instance_); +} +inline ::tensorflow::TensorShapeProto* OpInfo_TensorProperties::release_shape() { + // @@protoc_insertion_point(field_release:tensorflow.OpInfo.TensorProperties.shape) + + ::tensorflow::TensorShapeProto* temp = shape_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + shape_ = NULL; + return temp; +} +inline ::tensorflow::TensorShapeProto* OpInfo_TensorProperties::unsafe_arena_release_shape() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.OpInfo.TensorProperties.shape) + + ::tensorflow::TensorShapeProto* temp = shape_; + shape_ = NULL; + return temp; +} +inline ::tensorflow::TensorShapeProto* OpInfo_TensorProperties::mutable_shape() { + + if (shape_ == NULL) { + auto* p = CreateMaybeMessage<::tensorflow::TensorShapeProto>(GetArenaNoVirtual()); + shape_ = p; + } + // @@protoc_insertion_point(field_mutable:tensorflow.OpInfo.TensorProperties.shape) + return shape_; +} +inline void OpInfo_TensorProperties::set_allocated_shape(::tensorflow::TensorShapeProto* shape) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(shape_); + } + if (shape) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(shape)->GetArena(); + if (message_arena != submessage_arena) { + shape = ::google::protobuf::internal::GetOwnedMessage( + message_arena, shape, submessage_arena); + } + + } else { + + } + shape_ = shape; + // @@protoc_insertion_point(field_set_allocated:tensorflow.OpInfo.TensorProperties.shape) +} + +// .tensorflow.TensorProto value = 3; +inline bool OpInfo_TensorProperties::has_value() const { + return this != internal_default_instance() && value_ != NULL; +} +inline const ::tensorflow::TensorProto& OpInfo_TensorProperties::_internal_value() const { + return *value_; +} +inline const ::tensorflow::TensorProto& OpInfo_TensorProperties::value() const { + const ::tensorflow::TensorProto* p = value_; + // @@protoc_insertion_point(field_get:tensorflow.OpInfo.TensorProperties.value) + return p != NULL ? *p : *reinterpret_cast( + &::tensorflow::_TensorProto_default_instance_); +} +inline ::tensorflow::TensorProto* OpInfo_TensorProperties::release_value() { + // @@protoc_insertion_point(field_release:tensorflow.OpInfo.TensorProperties.value) + + ::tensorflow::TensorProto* temp = value_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + value_ = NULL; + return temp; +} +inline ::tensorflow::TensorProto* OpInfo_TensorProperties::unsafe_arena_release_value() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.OpInfo.TensorProperties.value) + + ::tensorflow::TensorProto* temp = value_; + value_ = NULL; + return temp; +} +inline ::tensorflow::TensorProto* OpInfo_TensorProperties::mutable_value() { + + if (value_ == NULL) { + auto* p = CreateMaybeMessage<::tensorflow::TensorProto>(GetArenaNoVirtual()); + value_ = p; + } + // @@protoc_insertion_point(field_mutable:tensorflow.OpInfo.TensorProperties.value) + return value_; +} +inline void OpInfo_TensorProperties::set_allocated_value(::tensorflow::TensorProto* value) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(value_); + } + if (value) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(value)->GetArena(); + if (message_arena != submessage_arena) { + value = ::google::protobuf::internal::GetOwnedMessage( + message_arena, value, submessage_arena); + } + + } else { + + } + value_ = value; + // @@protoc_insertion_point(field_set_allocated:tensorflow.OpInfo.TensorProperties.value) +} + +// ------------------------------------------------------------------- + +// OpInfo + +// string op = 1; +inline void OpInfo::clear_op() { + op_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& OpInfo::op() const { + // @@protoc_insertion_point(field_get:tensorflow.OpInfo.op) + return op_.Get(); +} +inline void OpInfo::set_op(const ::std::string& value) { + + op_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.OpInfo.op) +} +#if LANG_CXX11 +inline void OpInfo::set_op(::std::string&& value) { + + op_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.OpInfo.op) +} +#endif +inline void OpInfo::set_op(const char* value) { + GOOGLE_DCHECK(value != NULL); + + op_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.OpInfo.op) +} +inline void OpInfo::set_op(const char* value, + size_t size) { + + op_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.OpInfo.op) +} +inline ::std::string* OpInfo::mutable_op() { + + // @@protoc_insertion_point(field_mutable:tensorflow.OpInfo.op) + return op_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* OpInfo::release_op() { + // @@protoc_insertion_point(field_release:tensorflow.OpInfo.op) + + return op_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void OpInfo::set_allocated_op(::std::string* op) { + if (op != NULL) { + + } else { + + } + op_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), op, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.OpInfo.op) +} +inline ::std::string* OpInfo::unsafe_arena_release_op() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.OpInfo.op) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return op_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void OpInfo::unsafe_arena_set_allocated_op( + ::std::string* op) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (op != NULL) { + + } else { + + } + op_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + op, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.OpInfo.op) +} + +// map attr = 2; +inline int OpInfo::attr_size() const { + return attr_.size(); +} +inline const ::google::protobuf::Map< ::std::string, ::tensorflow::AttrValue >& +OpInfo::attr() const { + // @@protoc_insertion_point(field_map:tensorflow.OpInfo.attr) + return attr_.GetMap(); +} +inline ::google::protobuf::Map< ::std::string, ::tensorflow::AttrValue >* +OpInfo::mutable_attr() { + // @@protoc_insertion_point(field_mutable_map:tensorflow.OpInfo.attr) + return attr_.MutableMap(); +} + +// repeated .tensorflow.OpInfo.TensorProperties inputs = 3; +inline int OpInfo::inputs_size() const { + return inputs_.size(); +} +inline void OpInfo::clear_inputs() { + inputs_.Clear(); +} +inline ::tensorflow::OpInfo_TensorProperties* OpInfo::mutable_inputs(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.OpInfo.inputs) + return inputs_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::OpInfo_TensorProperties >* +OpInfo::mutable_inputs() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.OpInfo.inputs) + return &inputs_; +} +inline const ::tensorflow::OpInfo_TensorProperties& OpInfo::inputs(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.OpInfo.inputs) + return inputs_.Get(index); +} +inline ::tensorflow::OpInfo_TensorProperties* OpInfo::add_inputs() { + // @@protoc_insertion_point(field_add:tensorflow.OpInfo.inputs) + return inputs_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::OpInfo_TensorProperties >& +OpInfo::inputs() const { + // @@protoc_insertion_point(field_list:tensorflow.OpInfo.inputs) + return inputs_; +} + +// repeated .tensorflow.OpInfo.TensorProperties outputs = 5; +inline int OpInfo::outputs_size() const { + return outputs_.size(); +} +inline void OpInfo::clear_outputs() { + outputs_.Clear(); +} +inline ::tensorflow::OpInfo_TensorProperties* OpInfo::mutable_outputs(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.OpInfo.outputs) + return outputs_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::OpInfo_TensorProperties >* +OpInfo::mutable_outputs() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.OpInfo.outputs) + return &outputs_; +} +inline const ::tensorflow::OpInfo_TensorProperties& OpInfo::outputs(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.OpInfo.outputs) + return outputs_.Get(index); +} +inline ::tensorflow::OpInfo_TensorProperties* OpInfo::add_outputs() { + // @@protoc_insertion_point(field_add:tensorflow.OpInfo.outputs) + return outputs_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::OpInfo_TensorProperties >& +OpInfo::outputs() const { + // @@protoc_insertion_point(field_list:tensorflow.OpInfo.outputs) + return outputs_; +} + +// .tensorflow.DeviceProperties device = 4; +inline bool OpInfo::has_device() const { + return this != internal_default_instance() && device_ != NULL; +} +inline const ::tensorflow::DeviceProperties& OpInfo::_internal_device() const { + return *device_; +} +inline const ::tensorflow::DeviceProperties& OpInfo::device() const { + const ::tensorflow::DeviceProperties* p = device_; + // @@protoc_insertion_point(field_get:tensorflow.OpInfo.device) + return p != NULL ? *p : *reinterpret_cast( + &::tensorflow::_DeviceProperties_default_instance_); +} +inline ::tensorflow::DeviceProperties* OpInfo::release_device() { + // @@protoc_insertion_point(field_release:tensorflow.OpInfo.device) + + ::tensorflow::DeviceProperties* temp = device_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + device_ = NULL; + return temp; +} +inline ::tensorflow::DeviceProperties* OpInfo::unsafe_arena_release_device() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.OpInfo.device) + + ::tensorflow::DeviceProperties* temp = device_; + device_ = NULL; + return temp; +} +inline ::tensorflow::DeviceProperties* OpInfo::mutable_device() { + + if (device_ == NULL) { + auto* p = CreateMaybeMessage<::tensorflow::DeviceProperties>(GetArenaNoVirtual()); + device_ = p; + } + // @@protoc_insertion_point(field_mutable:tensorflow.OpInfo.device) + return device_; +} +inline void OpInfo::set_allocated_device(::tensorflow::DeviceProperties* device) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete reinterpret_cast< ::google::protobuf::MessageLite*>(device_); + } + if (device) { + ::google::protobuf::Arena* submessage_arena = + reinterpret_cast<::google::protobuf::MessageLite*>(device)->GetArena(); + if (message_arena != submessage_arena) { + device = ::google::protobuf::internal::GetOwnedMessage( + message_arena, device, submessage_arena); + } + + } else { + + } + device_ = device; + // @@protoc_insertion_point(field_set_allocated:tensorflow.OpInfo.device) +} + +// .tensorflow.SessionInfo session_info = 6; +inline bool OpInfo::has_session_info() const { + return this != internal_default_instance() && session_info_ != NULL; +} +inline void OpInfo::clear_session_info() { + if (GetArenaNoVirtual() == NULL && session_info_ != NULL) { + delete session_info_; + } + session_info_ = NULL; +} +inline const ::tensorflow::SessionInfo& OpInfo::_internal_session_info() const { + return *session_info_; +} +inline const ::tensorflow::SessionInfo& OpInfo::session_info() const { + const ::tensorflow::SessionInfo* p = session_info_; + // @@protoc_insertion_point(field_get:tensorflow.OpInfo.session_info) + return p != NULL ? *p : *reinterpret_cast( + &::tensorflow::_SessionInfo_default_instance_); +} +inline ::tensorflow::SessionInfo* OpInfo::release_session_info() { + // @@protoc_insertion_point(field_release:tensorflow.OpInfo.session_info) + + ::tensorflow::SessionInfo* temp = session_info_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + session_info_ = NULL; + return temp; +} +inline ::tensorflow::SessionInfo* OpInfo::unsafe_arena_release_session_info() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.OpInfo.session_info) + + ::tensorflow::SessionInfo* temp = session_info_; + session_info_ = NULL; + return temp; +} +inline ::tensorflow::SessionInfo* OpInfo::mutable_session_info() { + + if (session_info_ == NULL) { + auto* p = CreateMaybeMessage<::tensorflow::SessionInfo>(GetArenaNoVirtual()); + session_info_ = p; + } + // @@protoc_insertion_point(field_mutable:tensorflow.OpInfo.session_info) + return session_info_; +} +inline void OpInfo::set_allocated_session_info(::tensorflow::SessionInfo* session_info) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete session_info_; + } + if (session_info) { + ::google::protobuf::Arena* submessage_arena = + ::google::protobuf::Arena::GetArena(session_info); + if (message_arena != submessage_arena) { + session_info = ::google::protobuf::internal::GetOwnedMessage( + message_arena, session_info, submessage_arena); + } + + } else { + + } + session_info_ = session_info; + // @@protoc_insertion_point(field_set_allocated:tensorflow.OpInfo.session_info) +} + +// ------------------------------------------------------------------- + +// NormalDistribution + +// double mu = 1; +inline void NormalDistribution::clear_mu() { + mu_ = 0; +} +inline double NormalDistribution::mu() const { + // @@protoc_insertion_point(field_get:tensorflow.NormalDistribution.mu) + return mu_; +} +inline void NormalDistribution::set_mu(double value) { + + mu_ = value; + // @@protoc_insertion_point(field_set:tensorflow.NormalDistribution.mu) +} + +// double sigma = 2; +inline void NormalDistribution::clear_sigma() { + sigma_ = 0; +} +inline double NormalDistribution::sigma() const { + // @@protoc_insertion_point(field_get:tensorflow.NormalDistribution.sigma) + return sigma_; +} +inline void NormalDistribution::set_sigma(double value) { + + sigma_ = value; + // @@protoc_insertion_point(field_set:tensorflow.NormalDistribution.sigma) +} + +// ------------------------------------------------------------------- + +// LogNormalDistribution + +// double mu = 1; +inline void LogNormalDistribution::clear_mu() { + mu_ = 0; +} +inline double LogNormalDistribution::mu() const { + // @@protoc_insertion_point(field_get:tensorflow.LogNormalDistribution.mu) + return mu_; +} +inline void LogNormalDistribution::set_mu(double value) { + + mu_ = value; + // @@protoc_insertion_point(field_set:tensorflow.LogNormalDistribution.mu) +} + +// double sigma = 2; +inline void LogNormalDistribution::clear_sigma() { + sigma_ = 0; +} +inline double LogNormalDistribution::sigma() const { + // @@protoc_insertion_point(field_get:tensorflow.LogNormalDistribution.sigma) + return sigma_; +} +inline void LogNormalDistribution::set_sigma(double value) { + + sigma_ = value; + // @@protoc_insertion_point(field_set:tensorflow.LogNormalDistribution.sigma) +} + +// ------------------------------------------------------------------- + +// OpPerformance_OpMemory + +// repeated int64 output_memory = 1; +inline int OpPerformance_OpMemory::output_memory_size() const { + return output_memory_.size(); +} +inline void OpPerformance_OpMemory::clear_output_memory() { + output_memory_.Clear(); +} +inline ::google::protobuf::int64 OpPerformance_OpMemory::output_memory(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.OpPerformance.OpMemory.output_memory) + return output_memory_.Get(index); +} +inline void OpPerformance_OpMemory::set_output_memory(int index, ::google::protobuf::int64 value) { + output_memory_.Set(index, value); + // @@protoc_insertion_point(field_set:tensorflow.OpPerformance.OpMemory.output_memory) +} +inline void OpPerformance_OpMemory::add_output_memory(::google::protobuf::int64 value) { + output_memory_.Add(value); + // @@protoc_insertion_point(field_add:tensorflow.OpPerformance.OpMemory.output_memory) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int64 >& +OpPerformance_OpMemory::output_memory() const { + // @@protoc_insertion_point(field_list:tensorflow.OpPerformance.OpMemory.output_memory) + return output_memory_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int64 >* +OpPerformance_OpMemory::mutable_output_memory() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.OpPerformance.OpMemory.output_memory) + return &output_memory_; +} + +// int64 temp_memory = 2; +inline void OpPerformance_OpMemory::clear_temp_memory() { + temp_memory_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 OpPerformance_OpMemory::temp_memory() const { + // @@protoc_insertion_point(field_get:tensorflow.OpPerformance.OpMemory.temp_memory) + return temp_memory_; +} +inline void OpPerformance_OpMemory::set_temp_memory(::google::protobuf::int64 value) { + + temp_memory_ = value; + // @@protoc_insertion_point(field_set:tensorflow.OpPerformance.OpMemory.temp_memory) +} + +// int64 persistent_memory = 4; +inline void OpPerformance_OpMemory::clear_persistent_memory() { + persistent_memory_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 OpPerformance_OpMemory::persistent_memory() const { + // @@protoc_insertion_point(field_get:tensorflow.OpPerformance.OpMemory.persistent_memory) + return persistent_memory_; +} +inline void OpPerformance_OpMemory::set_persistent_memory(::google::protobuf::int64 value) { + + persistent_memory_ = value; + // @@protoc_insertion_point(field_set:tensorflow.OpPerformance.OpMemory.persistent_memory) +} + +// int64 device_temp_memory = 3 [deprecated = true]; +inline void OpPerformance_OpMemory::clear_device_temp_memory() { + device_temp_memory_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 OpPerformance_OpMemory::device_temp_memory() const { + // @@protoc_insertion_point(field_get:tensorflow.OpPerformance.OpMemory.device_temp_memory) + return device_temp_memory_; +} +inline void OpPerformance_OpMemory::set_device_temp_memory(::google::protobuf::int64 value) { + + device_temp_memory_ = value; + // @@protoc_insertion_point(field_set:tensorflow.OpPerformance.OpMemory.device_temp_memory) +} + +// int64 device_persistent_memory = 5 [deprecated = true]; +inline void OpPerformance_OpMemory::clear_device_persistent_memory() { + device_persistent_memory_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 OpPerformance_OpMemory::device_persistent_memory() const { + // @@protoc_insertion_point(field_get:tensorflow.OpPerformance.OpMemory.device_persistent_memory) + return device_persistent_memory_; +} +inline void OpPerformance_OpMemory::set_device_persistent_memory(::google::protobuf::int64 value) { + + device_persistent_memory_ = value; + // @@protoc_insertion_point(field_set:tensorflow.OpPerformance.OpMemory.device_persistent_memory) +} + +// ------------------------------------------------------------------- + +// OpPerformance + +// .tensorflow.OpInfo op = 1; +inline bool OpPerformance::has_op() const { + return this != internal_default_instance() && op_ != NULL; +} +inline void OpPerformance::clear_op() { + if (GetArenaNoVirtual() == NULL && op_ != NULL) { + delete op_; + } + op_ = NULL; +} +inline const ::tensorflow::OpInfo& OpPerformance::_internal_op() const { + return *op_; +} +inline const ::tensorflow::OpInfo& OpPerformance::op() const { + const ::tensorflow::OpInfo* p = op_; + // @@protoc_insertion_point(field_get:tensorflow.OpPerformance.op) + return p != NULL ? *p : *reinterpret_cast( + &::tensorflow::_OpInfo_default_instance_); +} +inline ::tensorflow::OpInfo* OpPerformance::release_op() { + // @@protoc_insertion_point(field_release:tensorflow.OpPerformance.op) + + ::tensorflow::OpInfo* temp = op_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + op_ = NULL; + return temp; +} +inline ::tensorflow::OpInfo* OpPerformance::unsafe_arena_release_op() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.OpPerformance.op) + + ::tensorflow::OpInfo* temp = op_; + op_ = NULL; + return temp; +} +inline ::tensorflow::OpInfo* OpPerformance::mutable_op() { + + if (op_ == NULL) { + auto* p = CreateMaybeMessage<::tensorflow::OpInfo>(GetArenaNoVirtual()); + op_ = p; + } + // @@protoc_insertion_point(field_mutable:tensorflow.OpPerformance.op) + return op_; +} +inline void OpPerformance::set_allocated_op(::tensorflow::OpInfo* op) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete op_; + } + if (op) { + ::google::protobuf::Arena* submessage_arena = + ::google::protobuf::Arena::GetArena(op); + if (message_arena != submessage_arena) { + op = ::google::protobuf::internal::GetOwnedMessage( + message_arena, op, submessage_arena); + } + + } else { + + } + op_ = op; + // @@protoc_insertion_point(field_set_allocated:tensorflow.OpPerformance.op) +} + +// .tensorflow.SessionInfo session_info = 12 [deprecated = true]; +inline bool OpPerformance::has_session_info() const { + return this != internal_default_instance() && session_info_ != NULL; +} +inline void OpPerformance::clear_session_info() { + if (GetArenaNoVirtual() == NULL && session_info_ != NULL) { + delete session_info_; + } + session_info_ = NULL; +} +inline const ::tensorflow::SessionInfo& OpPerformance::_internal_session_info() const { + return *session_info_; +} +inline const ::tensorflow::SessionInfo& OpPerformance::session_info() const { + const ::tensorflow::SessionInfo* p = session_info_; + // @@protoc_insertion_point(field_get:tensorflow.OpPerformance.session_info) + return p != NULL ? *p : *reinterpret_cast( + &::tensorflow::_SessionInfo_default_instance_); +} +inline ::tensorflow::SessionInfo* OpPerformance::release_session_info() { + // @@protoc_insertion_point(field_release:tensorflow.OpPerformance.session_info) + + ::tensorflow::SessionInfo* temp = session_info_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + session_info_ = NULL; + return temp; +} +inline ::tensorflow::SessionInfo* OpPerformance::unsafe_arena_release_session_info() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.OpPerformance.session_info) + + ::tensorflow::SessionInfo* temp = session_info_; + session_info_ = NULL; + return temp; +} +inline ::tensorflow::SessionInfo* OpPerformance::mutable_session_info() { + + if (session_info_ == NULL) { + auto* p = CreateMaybeMessage<::tensorflow::SessionInfo>(GetArenaNoVirtual()); + session_info_ = p; + } + // @@protoc_insertion_point(field_mutable:tensorflow.OpPerformance.session_info) + return session_info_; +} +inline void OpPerformance::set_allocated_session_info(::tensorflow::SessionInfo* session_info) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete session_info_; + } + if (session_info) { + ::google::protobuf::Arena* submessage_arena = + ::google::protobuf::Arena::GetArena(session_info); + if (message_arena != submessage_arena) { + session_info = ::google::protobuf::internal::GetOwnedMessage( + message_arena, session_info, submessage_arena); + } + + } else { + + } + session_info_ = session_info; + // @@protoc_insertion_point(field_set_allocated:tensorflow.OpPerformance.session_info) +} + +// string node = 5; +inline void OpPerformance::clear_node() { + node_.ClearToEmpty(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline const ::std::string& OpPerformance::node() const { + // @@protoc_insertion_point(field_get:tensorflow.OpPerformance.node) + return node_.Get(); +} +inline void OpPerformance::set_node(const ::std::string& value) { + + node_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), value, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set:tensorflow.OpPerformance.node) +} +#if LANG_CXX11 +inline void OpPerformance::set_node(::std::string&& value) { + + node_.Set( + &::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::move(value), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_rvalue:tensorflow.OpPerformance.node) +} +#endif +inline void OpPerformance::set_node(const char* value) { + GOOGLE_DCHECK(value != NULL); + + node_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string(value), + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_char:tensorflow.OpPerformance.node) +} +inline void OpPerformance::set_node(const char* value, + size_t size) { + + node_.Set(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), ::std::string( + reinterpret_cast(value), size), GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_pointer:tensorflow.OpPerformance.node) +} +inline ::std::string* OpPerformance::mutable_node() { + + // @@protoc_insertion_point(field_mutable:tensorflow.OpPerformance.node) + return node_.Mutable(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline ::std::string* OpPerformance::release_node() { + // @@protoc_insertion_point(field_release:tensorflow.OpPerformance.node) + + return node_.Release(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), GetArenaNoVirtual()); +} +inline void OpPerformance::set_allocated_node(::std::string* node) { + if (node != NULL) { + + } else { + + } + node_.SetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), node, + GetArenaNoVirtual()); + // @@protoc_insertion_point(field_set_allocated:tensorflow.OpPerformance.node) +} +inline ::std::string* OpPerformance::unsafe_arena_release_node() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.OpPerformance.node) + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + + return node_.UnsafeArenaRelease(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); +} +inline void OpPerformance::unsafe_arena_set_allocated_node( + ::std::string* node) { + GOOGLE_DCHECK(GetArenaNoVirtual() != NULL); + if (node != NULL) { + + } else { + + } + node_.UnsafeArenaSetAllocated(&::google::protobuf::internal::GetEmptyStringAlreadyInited(), + node, GetArenaNoVirtual()); + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.OpPerformance.node) +} + +// int64 temporary_memory_size = 2; +inline void OpPerformance::clear_temporary_memory_size() { + temporary_memory_size_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 OpPerformance::temporary_memory_size() const { + // @@protoc_insertion_point(field_get:tensorflow.OpPerformance.temporary_memory_size) + return temporary_memory_size_; +} +inline void OpPerformance::set_temporary_memory_size(::google::protobuf::int64 value) { + + temporary_memory_size_ = value; + // @@protoc_insertion_point(field_set:tensorflow.OpPerformance.temporary_memory_size) +} + +// int64 compute_cost = 3; +inline void OpPerformance::clear_compute_cost() { + compute_cost_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 OpPerformance::compute_cost() const { + // @@protoc_insertion_point(field_get:tensorflow.OpPerformance.compute_cost) + return compute_cost_; +} +inline void OpPerformance::set_compute_cost(::google::protobuf::int64 value) { + + compute_cost_ = value; + // @@protoc_insertion_point(field_set:tensorflow.OpPerformance.compute_cost) +} + +// int64 compute_time = 6; +inline void OpPerformance::clear_compute_time() { + compute_time_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 OpPerformance::compute_time() const { + // @@protoc_insertion_point(field_get:tensorflow.OpPerformance.compute_time) + return compute_time_; +} +inline void OpPerformance::set_compute_time(::google::protobuf::int64 value) { + + compute_time_ = value; + // @@protoc_insertion_point(field_set:tensorflow.OpPerformance.compute_time) +} + +// int64 memory_time = 7; +inline void OpPerformance::clear_memory_time() { + memory_time_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 OpPerformance::memory_time() const { + // @@protoc_insertion_point(field_get:tensorflow.OpPerformance.memory_time) + return memory_time_; +} +inline void OpPerformance::set_memory_time(::google::protobuf::int64 value) { + + memory_time_ = value; + // @@protoc_insertion_point(field_set:tensorflow.OpPerformance.memory_time) +} + +// double compute_efficiency = 4; +inline void OpPerformance::clear_compute_efficiency() { + compute_efficiency_ = 0; +} +inline double OpPerformance::compute_efficiency() const { + // @@protoc_insertion_point(field_get:tensorflow.OpPerformance.compute_efficiency) + return compute_efficiency_; +} +inline void OpPerformance::set_compute_efficiency(double value) { + + compute_efficiency_ = value; + // @@protoc_insertion_point(field_set:tensorflow.OpPerformance.compute_efficiency) +} + +// double memory_efficiency = 8; +inline void OpPerformance::clear_memory_efficiency() { + memory_efficiency_ = 0; +} +inline double OpPerformance::memory_efficiency() const { + // @@protoc_insertion_point(field_get:tensorflow.OpPerformance.memory_efficiency) + return memory_efficiency_; +} +inline void OpPerformance::set_memory_efficiency(double value) { + + memory_efficiency_ = value; + // @@protoc_insertion_point(field_set:tensorflow.OpPerformance.memory_efficiency) +} + +// .tensorflow.NormalDistribution execution_time_normal = 10; +inline bool OpPerformance::has_execution_time_normal() const { + return execution_time_case() == kExecutionTimeNormal; +} +inline void OpPerformance::set_has_execution_time_normal() { + _oneof_case_[0] = kExecutionTimeNormal; +} +inline void OpPerformance::clear_execution_time_normal() { + if (has_execution_time_normal()) { + if (GetArenaNoVirtual() == NULL) { + delete execution_time_.execution_time_normal_; + } + clear_has_execution_time(); + } +} +inline const ::tensorflow::NormalDistribution& OpPerformance::_internal_execution_time_normal() const { + return *execution_time_.execution_time_normal_; +} +inline ::tensorflow::NormalDistribution* OpPerformance::release_execution_time_normal() { + // @@protoc_insertion_point(field_release:tensorflow.OpPerformance.execution_time_normal) + if (has_execution_time_normal()) { + clear_has_execution_time(); + ::tensorflow::NormalDistribution* temp = execution_time_.execution_time_normal_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + execution_time_.execution_time_normal_ = NULL; + return temp; + } else { + return NULL; + } +} +inline const ::tensorflow::NormalDistribution& OpPerformance::execution_time_normal() const { + // @@protoc_insertion_point(field_get:tensorflow.OpPerformance.execution_time_normal) + return has_execution_time_normal() + ? *execution_time_.execution_time_normal_ + : *reinterpret_cast< ::tensorflow::NormalDistribution*>(&::tensorflow::_NormalDistribution_default_instance_); +} +inline ::tensorflow::NormalDistribution* OpPerformance::unsafe_arena_release_execution_time_normal() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.OpPerformance.execution_time_normal) + if (has_execution_time_normal()) { + clear_has_execution_time(); + ::tensorflow::NormalDistribution* temp = execution_time_.execution_time_normal_; + execution_time_.execution_time_normal_ = NULL; + return temp; + } else { + return NULL; + } +} +inline void OpPerformance::unsafe_arena_set_allocated_execution_time_normal(::tensorflow::NormalDistribution* execution_time_normal) { + clear_execution_time(); + if (execution_time_normal) { + set_has_execution_time_normal(); + execution_time_.execution_time_normal_ = execution_time_normal; + } + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.OpPerformance.execution_time_normal) +} +inline ::tensorflow::NormalDistribution* OpPerformance::mutable_execution_time_normal() { + if (!has_execution_time_normal()) { + clear_execution_time(); + set_has_execution_time_normal(); + execution_time_.execution_time_normal_ = CreateMaybeMessage< ::tensorflow::NormalDistribution >( + GetArenaNoVirtual()); + } + // @@protoc_insertion_point(field_mutable:tensorflow.OpPerformance.execution_time_normal) + return execution_time_.execution_time_normal_; +} + +// .tensorflow.LogNormalDistribution execution_time_log_normal = 11; +inline bool OpPerformance::has_execution_time_log_normal() const { + return execution_time_case() == kExecutionTimeLogNormal; +} +inline void OpPerformance::set_has_execution_time_log_normal() { + _oneof_case_[0] = kExecutionTimeLogNormal; +} +inline void OpPerformance::clear_execution_time_log_normal() { + if (has_execution_time_log_normal()) { + if (GetArenaNoVirtual() == NULL) { + delete execution_time_.execution_time_log_normal_; + } + clear_has_execution_time(); + } +} +inline const ::tensorflow::LogNormalDistribution& OpPerformance::_internal_execution_time_log_normal() const { + return *execution_time_.execution_time_log_normal_; +} +inline ::tensorflow::LogNormalDistribution* OpPerformance::release_execution_time_log_normal() { + // @@protoc_insertion_point(field_release:tensorflow.OpPerformance.execution_time_log_normal) + if (has_execution_time_log_normal()) { + clear_has_execution_time(); + ::tensorflow::LogNormalDistribution* temp = execution_time_.execution_time_log_normal_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + execution_time_.execution_time_log_normal_ = NULL; + return temp; + } else { + return NULL; + } +} +inline const ::tensorflow::LogNormalDistribution& OpPerformance::execution_time_log_normal() const { + // @@protoc_insertion_point(field_get:tensorflow.OpPerformance.execution_time_log_normal) + return has_execution_time_log_normal() + ? *execution_time_.execution_time_log_normal_ + : *reinterpret_cast< ::tensorflow::LogNormalDistribution*>(&::tensorflow::_LogNormalDistribution_default_instance_); +} +inline ::tensorflow::LogNormalDistribution* OpPerformance::unsafe_arena_release_execution_time_log_normal() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.OpPerformance.execution_time_log_normal) + if (has_execution_time_log_normal()) { + clear_has_execution_time(); + ::tensorflow::LogNormalDistribution* temp = execution_time_.execution_time_log_normal_; + execution_time_.execution_time_log_normal_ = NULL; + return temp; + } else { + return NULL; + } +} +inline void OpPerformance::unsafe_arena_set_allocated_execution_time_log_normal(::tensorflow::LogNormalDistribution* execution_time_log_normal) { + clear_execution_time(); + if (execution_time_log_normal) { + set_has_execution_time_log_normal(); + execution_time_.execution_time_log_normal_ = execution_time_log_normal; + } + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.OpPerformance.execution_time_log_normal) +} +inline ::tensorflow::LogNormalDistribution* OpPerformance::mutable_execution_time_log_normal() { + if (!has_execution_time_log_normal()) { + clear_execution_time(); + set_has_execution_time_log_normal(); + execution_time_.execution_time_log_normal_ = CreateMaybeMessage< ::tensorflow::LogNormalDistribution >( + GetArenaNoVirtual()); + } + // @@protoc_insertion_point(field_mutable:tensorflow.OpPerformance.execution_time_log_normal) + return execution_time_.execution_time_log_normal_; +} + +// .tensorflow.OpPerformance.OpMemory op_memory = 9; +inline bool OpPerformance::has_op_memory() const { + return this != internal_default_instance() && op_memory_ != NULL; +} +inline void OpPerformance::clear_op_memory() { + if (GetArenaNoVirtual() == NULL && op_memory_ != NULL) { + delete op_memory_; + } + op_memory_ = NULL; +} +inline const ::tensorflow::OpPerformance_OpMemory& OpPerformance::_internal_op_memory() const { + return *op_memory_; +} +inline const ::tensorflow::OpPerformance_OpMemory& OpPerformance::op_memory() const { + const ::tensorflow::OpPerformance_OpMemory* p = op_memory_; + // @@protoc_insertion_point(field_get:tensorflow.OpPerformance.op_memory) + return p != NULL ? *p : *reinterpret_cast( + &::tensorflow::_OpPerformance_OpMemory_default_instance_); +} +inline ::tensorflow::OpPerformance_OpMemory* OpPerformance::release_op_memory() { + // @@protoc_insertion_point(field_release:tensorflow.OpPerformance.op_memory) + + ::tensorflow::OpPerformance_OpMemory* temp = op_memory_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + op_memory_ = NULL; + return temp; +} +inline ::tensorflow::OpPerformance_OpMemory* OpPerformance::unsafe_arena_release_op_memory() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.OpPerformance.op_memory) + + ::tensorflow::OpPerformance_OpMemory* temp = op_memory_; + op_memory_ = NULL; + return temp; +} +inline ::tensorflow::OpPerformance_OpMemory* OpPerformance::mutable_op_memory() { + + if (op_memory_ == NULL) { + auto* p = CreateMaybeMessage<::tensorflow::OpPerformance_OpMemory>(GetArenaNoVirtual()); + op_memory_ = p; + } + // @@protoc_insertion_point(field_mutable:tensorflow.OpPerformance.op_memory) + return op_memory_; +} +inline void OpPerformance::set_allocated_op_memory(::tensorflow::OpPerformance_OpMemory* op_memory) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete op_memory_; + } + if (op_memory) { + ::google::protobuf::Arena* submessage_arena = + ::google::protobuf::Arena::GetArena(op_memory); + if (message_arena != submessage_arena) { + op_memory = ::google::protobuf::internal::GetOwnedMessage( + message_arena, op_memory, submessage_arena); + } + + } else { + + } + op_memory_ = op_memory; + // @@protoc_insertion_point(field_set_allocated:tensorflow.OpPerformance.op_memory) +} + +inline bool OpPerformance::has_execution_time() const { + return execution_time_case() != EXECUTION_TIME_NOT_SET; +} +inline void OpPerformance::clear_has_execution_time() { + _oneof_case_[0] = EXECUTION_TIME_NOT_SET; +} +inline OpPerformance::ExecutionTimeCase OpPerformance::execution_time_case() const { + return OpPerformance::ExecutionTimeCase(_oneof_case_[0]); +} +// ------------------------------------------------------------------- + +// OpPerformanceList + +// repeated .tensorflow.OpPerformance op_performance = 1; +inline int OpPerformanceList::op_performance_size() const { + return op_performance_.size(); +} +inline void OpPerformanceList::clear_op_performance() { + op_performance_.Clear(); +} +inline ::tensorflow::OpPerformance* OpPerformanceList::mutable_op_performance(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.OpPerformanceList.op_performance) + return op_performance_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::OpPerformance >* +OpPerformanceList::mutable_op_performance() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.OpPerformanceList.op_performance) + return &op_performance_; +} +inline const ::tensorflow::OpPerformance& OpPerformanceList::op_performance(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.OpPerformanceList.op_performance) + return op_performance_.Get(index); +} +inline ::tensorflow::OpPerformance* OpPerformanceList::add_op_performance() { + // @@protoc_insertion_point(field_add:tensorflow.OpPerformanceList.op_performance) + return op_performance_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::OpPerformance >& +OpPerformanceList::op_performance() const { + // @@protoc_insertion_point(field_list:tensorflow.OpPerformanceList.op_performance) + return op_performance_; +} + +#ifdef __GNUC__ + #pragma GCC diagnostic pop +#endif // __GNUC__ +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + + +// @@protoc_insertion_point(namespace_scope) + +} // namespace tensorflow + +// @@protoc_insertion_point(global_scope) + +#endif // PROTOBUF_INCLUDED_tensorflow_2fcore_2fgrappler_2fcosts_2fop_5fperformance_5fdata_2eproto diff --git a/linux/include/tensorflow/core/grappler/costs/utils.h b/linux/include/tensorflow/core/grappler/costs/utils.h new file mode 100644 index 000000000..5fd671771 --- /dev/null +++ b/linux/include/tensorflow/core/grappler/costs/utils.h @@ -0,0 +1,114 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_COSTS_UTILS_H_ +#define TENSORFLOW_CORE_GRAPPLER_COSTS_UTILS_H_ + +#include +#include +#include + +#include "tensorflow/core/framework/cost_graph.pb.h" +#include "tensorflow/core/framework/graph.pb.h" +#include "tensorflow/core/framework/node_def.pb.h" +#include "tensorflow/core/framework/op_def.pb.h" +#include "tensorflow/core/graph/types.h" +#include "tensorflow/core/grappler/costs/op_performance_data.pb.h" +#include "tensorflow/core/platform/types.h" +#include "tensorflow/core/protobuf/config.pb.h" +#include "tensorflow/core/protobuf/device_properties.pb.h" + +namespace tensorflow { +namespace grappler { + +// Returns a vector of InputProperties for 'node'. The vector will contain one +// entry for each input of 'node'. +// For each node in the graph, the 'name_to_cost' map stores a pointer to the +// corresponding cost graph node indexed by node name. The 'name_to_node' maps a +// node name to its node definition. +std::vector FindInputFeatures( + const NodeDef& node, + const std::unordered_map& name_to_cost, + const std::unordered_map& name_to_node); + +// Returns the DeviceProperties of the device on which 'node' runs. +DeviceProperties GetDeviceInfo(const CostGraphDef::Node& node); +DeviceProperties GetDeviceInfo(const string& device_str); + +// Return a string describing a node given a nodeinfo. +string GetOpDescription(const OpInfo& op_info); + +// Builds the OpInfo for node without filling its device information, given all +// nodes in the graph and its input properties. +OpInfo BuildOpInfoWithoutDevice( + const NodeDef& node, + const std::unordered_map& name_to_node, + const std::vector& inputs); + +// Gather performance data from a cost graph. +OpPerformanceList CostGraphToOpPerformanceData(const CostGraphDef& cost_graph, + const GraphDef& graph); + +// Simple histogram for profiling Tensor size; histogram uses logarithmic +// buckets. +class TensorSizeHistogram { + public: + TensorSizeHistogram() : buckets_(kMaxBuckets, 0) {} + + void Add(const uint64 value); + void Merge(const TensorSizeHistogram& src); + double Average() const { + if (num_elem_ > 0) { + return static_cast(sum_elem_) / num_elem_; + } else { + return 0.0; + } + } + uint64 Min() const { return min_; } + uint64 Max() const { return max_; } + uint64 NumElem() const { return num_elem_; } + uint64 SumElem() const { return sum_elem_; } + string ToString() const; + + protected: + const int Index(const uint64 value) const; + const std::vector& GetBuckets() const { return buckets_; } + + private: + const int kMaxBuckets = 64; + uint64 num_elem_ = 0; + uint64 sum_elem_ = 0; + // min_ and max_ are initialized to a very large value and zero, respectively, + // so that any value added can replace the initial min_ and max_. + uint64 min_ = kuint64max; + uint64 max_ = 0; + // Buckets are logarithmic: + // 0B, 1B, 2-3B, 4-7B, 8-15B, ..., 2^N - 2^(N+1)-1B, ... + std::vector buckets_; +}; + +// Helper functions for aggregating per-device stats into per-device-class +// stats. +string GetDeviceClassForNonChannelDevice(const string& device_name); +string GetDeviceClass(const string& device_name); + +// Get stats in string format from RunMetadata. +string GetStatsStringFromRunMetadata(const RunMetadata& run_metadata, + bool verbosity); + +} // end namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_COSTS_UTILS_H_ diff --git a/linux/include/tensorflow/core/grappler/costs/virtual_placer.h b/linux/include/tensorflow/core/grappler/costs/virtual_placer.h new file mode 100644 index 000000000..fee5ce0f5 --- /dev/null +++ b/linux/include/tensorflow/core/grappler/costs/virtual_placer.h @@ -0,0 +1,60 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_COSTS_VIRTUAL_PLACER_H_ +#define TENSORFLOW_CORE_GRAPPLER_COSTS_VIRTUAL_PLACER_H_ + +#include +#include "tensorflow/core/platform/types.h" +#include "tensorflow/core/protobuf/device_properties.pb.h" + +namespace tensorflow { +class NodeDef; + +namespace grappler { +class Cluster; + +// The virtual placer emulates the behavior of the TF placer. +class VirtualPlacer { + public: + VirtualPlacer(const Cluster* cluster); + + const DeviceProperties& get_device(const NodeDef& node) const; + + // Returns device name from cluster, which best matches the node.device() + // specification. Returns default device if no match was found or the + // node.device() could not be parsed. + string get_canonical_device_name(const NodeDef& node) const; + + private: + // Converts given device name to Lowercase Fully-Qualified Name (LFQN) string. + // This helps us disambiguate device names internally and simplify matching. + // If device_name couldn't be parsed successfully, returns empty string. + string to_lfqn_or_empty(const string& device_name) const; + + // Map based on the cluster info: cluster device name -> device properties. + std::unordered_map devices_; + + // Maps LFQN to original device name as it was declared in cluster. + std::unordered_map lfqn_map_; + + string default_device_name_; + string default_job_name_lowercase_; +}; + +} // namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_COSTS_VIRTUAL_PLACER_H_ diff --git a/linux/include/tensorflow/core/grappler/costs/virtual_scheduler.h b/linux/include/tensorflow/core/grappler/costs/virtual_scheduler.h new file mode 100644 index 000000000..0e66e8a46 --- /dev/null +++ b/linux/include/tensorflow/core/grappler/costs/virtual_scheduler.h @@ -0,0 +1,346 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_COSTS_VIRTUAL_SCHEDULER_H_ +#define TENSORFLOW_CORE_GRAPPLER_COSTS_VIRTUAL_SCHEDULER_H_ + +#include +#include +#include +#include + +#include "tensorflow/core/framework/node_def.pb.h" +#include "tensorflow/core/framework/step_stats.pb.h" +#include "tensorflow/core/grappler/costs/cost_estimator.h" +#include "tensorflow/core/grappler/costs/graph_properties.h" +#include "tensorflow/core/grappler/costs/op_context.h" +#include "tensorflow/core/grappler/costs/virtual_placer.h" +#include "tensorflow/core/grappler/grappler_item.h" + +namespace tensorflow { +namespace grappler { + +struct NodeState { + // A node (i.e., an op) takes a set of input:port pairs and produces + // a set of output ports. + + // Cross references to input and output nodes from graphdef. + std::vector> inputs; // Input, port pairs. + // List of output nodes (a list of nodes that takes this output port as input) + // keyed by port_num. Note that port_num -1 is used for control dependency. + std::unordered_map> outputs; + + // Info from GraphProperties. + std::vector input_properties; + std::vector output_properties; + + // Canonical device name used within VirtualScheduler. + string device_name; + + // States updated as scheduling nodes. + int num_inputs_ready; + std::unordered_map num_outputs_executed; + Costs::Duration time_ready; + Costs::Duration time_scheduled; + Costs::Duration time_finished; + // Time that all the consumers are executed (hence, no need to keep this + // output in memory), keyed by port_num. + std::unordered_map time_no_references; + + // Note that a node may have multiple output ports. The length of outputs, + // num_outputs_executed, and time_no_references should be + // identical when a NodeState is fully initialized. + // They should be 1 + output_properties.size() as we add [-1] for control + // dependency. + + // Node will be ready to be executed at time_ready, scheduled at + // time_scheduled, and finishes execution at time_finished. + // Each output port uses up memory space from time_scheduled to its + // time_no_references. + + NodeState() { + num_inputs_ready = 0; + time_ready = Costs::Duration::max(); + time_scheduled = Costs::Duration::max(); + time_finished = Costs::Duration::max(); + // Note that num_outputs_executed and time_no_references are not initialized + // here, since we don't know the size (i.e., # outputs for this node). + } +}; + +struct DeviceState { + // Nodes executed on this device in execution order. + std::vector nodes_executed; + + struct NodePairHash { + public: + const std::size_t operator()( + const std::pair& element) const { + return std::hash()(element.first); + } + }; + + // Nodes currently allocated in memory: set of NodeDef* and port_num pairs + // so that we can track which output of the node is in memory. + std::unordered_set, NodePairHash> + nodes_in_memory; + + // Nodes allocated in memory persistently: e.g., Variables. + std::unordered_set, NodePairHash> + persistent_nodes; + + // Snapshot of nodes_in_memory, when memory usage is at peak. + // Same to nodes_in_memory, it's a set of NodeDef* and port_num pairs. + std::unordered_set, NodePairHash> + mem_usage_snapshot_at_peak; + + Costs device_costs; + std::map op_to_cost; // Per-op cost. + std::map op_to_memory; // Per-op memory usage at peak usage. + int64 memory_usage; + int64 max_memory_usage; + + DeviceState() { + device_costs = Costs::ZeroCosts(); + device_costs.num_ops_total = 0; + memory_usage = 0; + max_memory_usage = 0; + } + + Costs::Duration GetCurrTime() const { return device_costs.execution_time; } +}; + +// ReadyNodeManager (abstract class): +// Keeps ready nodes and picks the best one to be scheduled. +class ReadyNodeManager { + public: + ReadyNodeManager() {} + virtual ~ReadyNodeManager() {} + virtual void Init( + const std::unordered_map* node_state) {} + virtual void AddNode(const NodeDef* node) = 0; + virtual const NodeDef* GetCurrNode() = 0; + virtual void RemoveCurrNode() = 0; + virtual bool Empty() const = 0; +}; + +class FIFOManager : public ReadyNodeManager { + public: + FIFOManager() : ReadyNodeManager() {} + ~FIFOManager() override {} + void Init(const std::unordered_map* node_state) + override {} + void AddNode(const NodeDef* node) override { nodes_.push_back(node); } + const NodeDef* GetCurrNode() override { + CHECK(!nodes_.empty()) << "GetCurrNode(), but there's no ready node"; + return nodes_.front(); + } + void RemoveCurrNode() override { nodes_.pop_front(); } + bool Empty() const override { return nodes_.empty(); } + + private: + std::list nodes_; +}; + +// The LIFOManager schedules nodes by returning the last one added to the +// scheduler. A node is executed and then its ready outputs are newly added to +// the scheduler, so the LIFOManager will return outputs to a node following +// that node's execution. +class LIFOManager : public ReadyNodeManager { + public: + LIFOManager() : ReadyNodeManager() {} + ~LIFOManager() override {} + void Init(const std::unordered_map* node_state) + override {} + void AddNode(const NodeDef* node) override { nodes_.push_back(node); } + const NodeDef* GetCurrNode() override; + void RemoveCurrNode() override; + bool Empty() const override { return nodes_.empty(); } + + private: + std::list nodes_; + // Keep track of the current node being executed by saving its position. + // Necessary because nodes may be added to the end of the list while a node is + // executing, and we want to remove the correct node (the one that is + // executing) rather than the new ones being added. + std::list::iterator curr_pos_ = nodes_.end(); +}; + +// FirstReadyManager picks a node with the minimum time_ready value. +// Behavior is unknown if there are more than one nodes with the minimum +// time_ready value (it depends on C++ STL push_heap and pop_heap). +class FirstReadyManager : public ReadyNodeManager { + public: + FirstReadyManager(); + void Init( + const std::unordered_map* node_state) override; + ~FirstReadyManager() override {} + void AddNode(const NodeDef* node) override { waiting_queue_.push_back(node); } + const NodeDef* GetCurrNode() override; + void RemoveCurrNode() override; + bool Empty() const override; + + private: + // Move all the nodes in the waiting_queue_ to nodes_. + void DrainWaitingQueue(); + + // nodes_ is the main queue, where we construct heap, and the front is the + // current node. + std::vector nodes_; + // Newly added nodes are added to waiting_queue_. That way, GetCurrNode(), + // which returns the front of the nodes_, always returns the same node, + // even if any of new nodes has time_ready smaller than the current node's. + std::vector waiting_queue_; + // Comparator functor for heap; stl heap is max heap, so we use "greater than" + // functor for keeping the smallest time_ready node at the front of heap. + std::function greater_; + + // NodeState structure from VirtualScheduler to get time_ready of ready nodes. + // Not owned by FirstReadyManager. + const std::unordered_map* node_state_; +}; + +// CompositeNodeManager has a few other NodeManagers: per-device LIFO for normal +// ops (neither _Send nor _Recv) and FirstReadyManagers for _Send ops and _Recv +// ops, and then it chooses FirstReady among the ops chosen from each +// internal NodeManagers. The objective is to maximize producer-consumer +// locality within device, while processing nodes across devices, including +// _Send and _Recv, fairly, in terms of their time_ready. +class CompositeNodeManager : public ReadyNodeManager { + public: + CompositeNodeManager(); + ~CompositeNodeManager() override {} + + void Init( + const std::unordered_map* node_state) override; + void AddNode(const NodeDef* node) override; + const NodeDef* GetCurrNode() override; + void RemoveCurrNode() override; + bool Empty() const override; + + private: + // Internal ready node managers: + // LIFO for normal ops to maximize producer consumer locality. + // One LIFO per device. + std::unordered_map ops_lifo_map_; + // FirstReady for send and recv. Handle send and recv separately ensures that + // send and recv do not block previously read ops with LIFO schedule. + FirstReadyManager send_manager_; + FirstReadyManager recv_manager_; + + // NodeState structure from VirtualScheduler to get time_ready of ready nodes. + // Not owned by FirstReadyManager. + const std::unordered_map* node_state_; + + // Cached curr node. Set back to nullptr from RemoveCurrNode(). + const NodeDef* curr_node_; +}; + +// The virtual scheduler emulates execution of nodes in a graph, considering +// dependencies, device, etc. +class VirtualScheduler { + public: + VirtualScheduler(const GrapplerItem* grappler_item, + const bool use_static_shapes, Cluster* cluster, + ReadyNodeManager* ready_nodes); + // Initializes NodeState and DeviceState from grappler_item_ and + // graph_properties_. + Status Init(); + + OpContext GetCurrNode() const; + + // Returns true if there is any node to be scheduled. + bool MarkCurrNodeExecuted(const Costs& node_costs); + + // Prints out summary of execution (timing, memory usage, etc.) + Costs Summary() const; + // Like the above, but writes detailed stats to RunMetadata. + // If metadata is nullptr, then just calls and return Summary(). + Costs Summary(RunMetadata* metadata); + // Methods called from constructor. + static ReadyNodeManager* ReadyNodeManagerFactory( + const string& ready_node_manager); + + // Return per device peak memory usage. + const std::unordered_map GetPeakMemoryUsage() const; + + const std::unordered_map* GetDeviceStates() const { + return &device_; + } + const std::unordered_map* GetNodeStates() const { + return &node_map_; + } + + protected: + // Returns the size of output at port_num (unit: bytes). A special case is + // port_num -1, which is for control dependency and assumed to be 4 bytes. + int64 CalculateOutputSize( + const std::vector& output_properties, + const int port_num) const; + + private: + // Constants. + const string kAttrInputSrc = "input_source_"; + const string kAttrSrcDevice = "src_device_"; + const string kAttrDstDevice = "dst_device_"; + const string kChannelDevice = "Channel"; + + // Methods called from Init(). Fails if initialize_ is set. + void MaybeUpdateInputOutput(const NodeDef* node); + NodeState& GetNodeStateOrCreateIt(const NodeDef* node); + std::pair CreateSendRecv( + const NodeDef* from, const NodeDef* to, const string& input_name); + string DeviceName(const NodeDef* node) const; + string SanitizedDeviceName(const NodeDef* node) const; + string ChannelDeviceName(const NodeDef* from, const NodeDef* to) const; + + // Helper methods. + Costs& FindOrCreateZero(const string& op_name, + std::map* op_cost); + float Round2(const float x) const; + bool IsPersistentNode(const NodeDef* node) const; + + // Scheduler states: + ReadyNodeManager* ready_nodes_; // Not owned. + std::unordered_map node_map_; + std::unordered_map device_; + + // Pool of NodeDefs for SendRecv and Identity ops created. + std::vector> additional_nodes_; + + // Stats: + std::map op_counts_; // Op counts with key with input shape. + // Individual op costs (with input shapes). + // Boolean field for whether the cost is accurate. + std::map> op_costs_; + + Costs graph_costs_; // Graph cost. + std::map op_to_cost_; // Per-op cost. + + // Auxiliary data structures for constructing NodeState and DeviceState. + GraphProperties graph_properties_; + Cluster* cluster_; // Not owned. + + const GrapplerItem* grappler_item_; // Not owned. + bool use_static_shapes_; + bool initialized_; + + VirtualPlacer placer_; // owned. +}; + +} // namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_COSTS_VIRTUAL_SCHEDULER_H_ diff --git a/linux/include/tensorflow/core/grappler/devices.h b/linux/include/tensorflow/core/grappler/devices.h new file mode 100644 index 000000000..1e60117b2 --- /dev/null +++ b/linux/include/tensorflow/core/grappler/devices.h @@ -0,0 +1,42 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_DEVICES_H_ +#define TENSORFLOW_CORE_GRAPPLER_DEVICES_H_ + +#include + +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/lib/core/threadpool.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { +namespace grappler { + +// Get the number of available GPUs whose number of multiprocessors is no less +// than 8. +int GetNumAvailableGPUs(); + +// Maximum amount of gpu memory available per gpu. gpu_id must be in the range +// [0, num_available_gpu) +int64 AvailableGPUMemory(int gpu_id); + +// Get the number of logical CPU cores (aka hyperthreads) available. +int GetNumAvailableLogicalCPUCores(); + +} // end namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_DEVICES_H_ diff --git a/linux/include/tensorflow/core/grappler/graph_view.h b/linux/include/tensorflow/core/grappler/graph_view.h new file mode 100644 index 000000000..ec946ca3b --- /dev/null +++ b/linux/include/tensorflow/core/grappler/graph_view.h @@ -0,0 +1,151 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_GRAPH_VIEW_H_ +#define TENSORFLOW_CORE_GRAPPLER_GRAPH_VIEW_H_ + +#include +#include +#include "tensorflow/core/framework/graph.pb.h" +#include "tensorflow/core/framework/node_def.pb.h" +#include "tensorflow/core/framework/op_def.pb.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { +namespace grappler { + +// Map a node/op's output port_id to arg_id. +// +// The port_id refers to the n-th tensor of the node, while the arg_id refers to +// the n-th arg of the op. These two can be different if an op's arg is a list +// of tensors. +// +// We return -1 for any invalid port_id (i.e., no corresponding arg_id). +int OpOutputPortIdToArgId(const NodeDef& node, const OpDef& op, int port_id); + +// A utility class to simplify the traversal of a GraphDef. +class GraphView { + public: + struct Port { + Port() = default; + Port(NodeDef* n, int port) : node(n), port_id(port) {} + + // TODO(prazek): ports should keep the constness of GraphView. The only way + // to modify graph through the view should be using MutableGraphView. + NodeDef* node = nullptr; + int port_id = -1; + + bool operator==(const Port& other) const { + return node == other.node && port_id == other.port_id; + } + }; + struct InputPort : public Port { + InputPort() = default; + InputPort(NodeDef* n, int port_id) : Port(n, port_id) {} + InputPort(const NodeDef* n, int port_id) + : Port(const_cast(n), port_id) {} + }; + struct OutputPort : public Port { + OutputPort() = default; + OutputPort(NodeDef* n, int port_id) : Port(n, port_id) {} + }; + + struct HashPort { + std::size_t operator()(const Port& port) const { + return reinterpret_cast(port.node) + port.port_id; + } + }; + + struct Edge { + OutputPort src; + InputPort tgt; + + bool operator==(const Edge& other) const { + return src == other.src && tgt == other.tgt; + } + }; + struct HashEdge { + std::size_t operator()(const Edge& edge) const { + return HashPort()(edge.src) + HashPort()(edge.tgt); + } + }; + + explicit GraphView(GraphDef* graph); + GraphDef* GetGraph() const { return graph_; } + NodeDef* GetNode(const string& node_name) const; + // Get the specified input port. Note that the special '-1' port_id can be + // used to access the controlling nodes (i.e. the nodes connected to node_name + // through an incoming control dependency). + InputPort GetInputPort(const string& node_name, int port_id) const; + // Get the specified output port. Note that the special '-1' port_id can be + // used to access the controlled nodes (i.e. the nodes connected to node_name + // through an outgoing control dependency). + OutputPort GetOutputPort(const string& node_name, int port_id) const; + + // Get the input (resp. output) port(s) in the immediate fanout (resp. fanin) + // of an output (resp. input) port. + const std::unordered_set& GetFanout( + const OutputPort& port) const; + std::unordered_set GetFanin( + const InputPort& port) const; + + // Special case: regular (i.e. non-control) input ports can only have one + // fanin. + const OutputPort GetRegularFanin(const InputPort& port) const; + + // Get all the input (resp. output) ports in the immediate fanout (resp fanin) + // of a node. Include the controlling nodes iff include_controlling_nodes is + // true. + std::unordered_set GetFanouts( + const NodeDef& node, bool include_controlled_nodes) const; + std::unordered_set GetFanins( + const NodeDef& node, bool include_controlling_nodes) const; + + // Get the number of ports in the immediate fanin of a node. Count the + // controlling nodes iff include_controlling_nodes is true. + int NumFanins(const NodeDef& node, bool include_controlling_nodes) const; + + // Get all the edge in the immediate fanout (resp fanin) of a node. Include + // the control edges iff include_controlling_edges is true. + std::unordered_set GetFanoutEdges( + const NodeDef& node, bool include_controlled_edges) const; + std::unordered_set GetFaninEdges( + const NodeDef& node, bool include_controlling_edges) const; + + protected: + // Add a new `node` to the graph. + void AddUniqueNodeOrDie(NodeDef* node); + // Add fanout to every `node` input. + void AddFanouts(NodeDef* node); + std::unordered_map* MutableNodes() { return &nodes_; } + GraphDef* MutableGraph() { return graph_; } + + using FanoutsMapType = + std::unordered_map, + HashPort>; + FanoutsMapType* MutableFanouts() { return &fanouts_; } + + private: + GraphDef* graph_; + std::unordered_map nodes_; + std::unordered_set empty_set_; + FanoutsMapType fanouts_; + std::unordered_map num_regular_outputs_; +}; + +} // end namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_GRAPH_VIEW_H_ diff --git a/linux/include/tensorflow/core/grappler/grappler_item.h b/linux/include/tensorflow/core/grappler/grappler_item.h new file mode 100644 index 000000000..a0748abfe --- /dev/null +++ b/linux/include/tensorflow/core/grappler/grappler_item.h @@ -0,0 +1,105 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_GRAPPLER_ITEM_H_ +#define TENSORFLOW_CORE_GRAPPLER_GRAPPLER_ITEM_H_ + +#include +#include +#include +#include +#include + +#include "tensorflow/core/framework/graph.pb.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/variable.pb.h" +#include "tensorflow/core/protobuf/queue_runner.pb.h" + +namespace tensorflow { +namespace grappler { + +// A TensorFlow model to optimize. +// Models are represented by the combination of a graph, one of more fetch +// nodes, and potentially a set of nodes to feed. +struct GrapplerItem { + GrapplerItem() = default; + GrapplerItem(const GrapplerItem& other, GraphDef&& graph_def) + : GrapplerItem(other, &graph_def) {} + // Swaps *graph_def with an empty GraphDef. + GrapplerItem(const GrapplerItem& other, GraphDef* graph_def); + virtual ~GrapplerItem() = default; + + string id; // A unique id for this item + + // Inputs + GraphDef graph; + std::vector> feed; + std::vector fetch; + + // Initialization op(s). + std::vector init_ops; + // Expected initialization time in seconds, or 0 if unknown + int64 expected_init_time = 0; + + // Save/restore ops (if any) + string save_op; + string restore_op; + string save_restore_loc_tensor; + + // Queue runner(s) required to run the queue(s) of this model. + std::vector queue_runners; + + // List of op names to keep in the graph. This includes nodes that are + // referenced in various collections, and therefore must be preserved to + // ensure that the optimized metagraph can still be loaded. + std::vector keep_ops; + + // Return the set of node evaluated during a regular train/inference step. + std::vector MainOpsFanin() const; + // Return the set of node run to populate the queues (if any). + std::vector EnqueueOpsFanin() const; + // Return the set nodes used by TensorFlow to initialize the graph. + std::vector InitOpsFanin() const; + // Return the set of variables accessed during a regular train/inference step. + std::vector MainVariables() const; + // Return a set of node names that must be preserved. This includes feed and + // fetch nodes, keep_ops, init_ops. + std::unordered_set NodesToPreserve() const; + + // Restrict types of optimizations that are allowed for this GrapplerItem. + struct AllowedOptimizations { + // Is it allowed to add nodes to the graph that do not have registered + // gradient function. + bool non_differentiable_rewrites = true; + }; + + AllowedOptimizations allowed_optimizations; +}; + +// Return the transitive fanin of a set of terminal nodes. +std::vector ComputeTransitiveFanin( + const GraphDef& graph, const std::vector& terminal_nodes); + +// Return the transitive fanin of a set of terminal nodes. Sets 'ill_formed' to +// true if one of the node is missing in the graph, or some node inputs don't +// exist. +std::vector ComputeTransitiveFanin( + const GraphDef& graph, const std::vector& terminal_nodes, + bool* ill_formed); + +} // end namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_GRAPPLER_ITEM_H_ diff --git a/linux/include/tensorflow/core/grappler/grappler_item_builder.h b/linux/include/tensorflow/core/grappler/grappler_item_builder.h new file mode 100644 index 000000000..1698587f8 --- /dev/null +++ b/linux/include/tensorflow/core/grappler/grappler_item_builder.h @@ -0,0 +1,64 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_GRAPPLER_ITEM_BUILDER_H_ +#define TENSORFLOW_CORE_GRAPPLER_GRAPPLER_ITEM_BUILDER_H_ + +#include +#include +#include +#include "tensorflow/core/framework/attr_value.pb.h" +#include "tensorflow/core/grappler/grappler_item.h" + +namespace tensorflow { + +class MetaGraphDef; + +namespace grappler { + +struct ItemConfig { + ItemConfig() {} + + // If true, ignore all user specified node placement. + bool ignore_user_placement = true; + // If true, ignore all user specified colocation attributes. + bool ignore_colocation = true; + // Dimension to use if a placeholder node has an _output_shapes attribute with + // a dimension of -1. + int placeholder_unknown_output_shape_dim = -1; + // If true, does L1 optimizations. + bool apply_optimizations = false; + // If true, erases all "_noinline" attributes from user-defined functions. + // Has no effect if "inline_functions" is disabled. + bool erase_noinline_attributes = false; + // If non-empty, override the directory of asset paths. + string assets_directory_override; + // If true, runs ModelPruner on the graph. + bool prune_graph = false; + // Override feed nodes list. + std::set feed_nodes; + // Override fetch nodes list. + std::set fetch_nodes; +}; + +// Factory method for creating a GrapplerItem from a MetaGraphDef. +// Returns nullptr if the given meta_graph cannot be converted. +std::unique_ptr GrapplerItemFromMetaGraphDef( + const string& id, const MetaGraphDef& meta_graph, const ItemConfig& cfg); + +} // end namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_GRAPPLER_ITEM_BUILDER_H_ diff --git a/linux/include/tensorflow/core/grappler/inputs/utils.h b/linux/include/tensorflow/core/grappler/inputs/utils.h new file mode 100644 index 000000000..4b9cb0a9a --- /dev/null +++ b/linux/include/tensorflow/core/grappler/inputs/utils.h @@ -0,0 +1,40 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_INPUTS_UTILS_H_ +#define TENSORFLOW_CORE_GRAPPLER_INPUTS_UTILS_H_ + +#include +#include + +#include "tensorflow/core/framework/graph.pb.h" +#include "tensorflow/core/lib/core/status.h" + +namespace tensorflow { +namespace grappler { + +bool FilesExist(const std::vector& files, + std::vector* status = nullptr); +bool FilesExist(const std::set& files); + +bool FileExists(const string& file, Status* status); + +Status ReadGraphDefFromFile(const string& graph_def_pbtxt_path, + GraphDef* result); + +} // end namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_INPUTS_UTILS_H_ diff --git a/linux/include/tensorflow/core/grappler/mutable_graph_view.h b/linux/include/tensorflow/core/grappler/mutable_graph_view.h new file mode 100644 index 000000000..971e5503d --- /dev/null +++ b/linux/include/tensorflow/core/grappler/mutable_graph_view.h @@ -0,0 +1,63 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_MUTABLE_GRAPH_VIEW_H_ +#define TENSORFLOW_CORE_GRAPPLER_MUTABLE_GRAPH_VIEW_H_ + +#include "tensorflow/core/grappler/graph_view.h" + +namespace tensorflow { +namespace grappler { + +// A utility class to simplify the traversal of a GraphDef that, unlike +// GraphView, supports updating the graph. Note that you should not modify the +// graph separately, because the view will get out of sync. +class MutableGraphView : public GraphView { + public: + using GraphView::GraphView; + + GraphDef* GetGraph() { return MutableGraph(); } + + // Adds a new node to graph and updates the view. + NodeDef* AddNode(NodeDef&& node); + + // Inserts a new node to the graph after `input` node and updates the view. + // This adds `node` to the graph and replaces the input for the output + // nodes of `input` with a port `output_port_id` with the new node. + NodeDef* InsertNode(const NodeDef& input, NodeDef&& node, + int output_port_id = 0); + + // Replaces the input for the output nodes of 'old_input' with a port + // `output_port_id` with 'new_input'. + // + // E.g: We have 2 nodes that use 'bar' node outputs as inputs: + // foo(bar:0, bar:1), foo2(other:0, bar:0) + // Calling ReplaceInput(bar, new, 0) changes every occurrence of bar:0 for + // new:0. Result: + // foo(new:0, bar:1), foo2(other:0, new:0) + void ReplaceInput(const NodeDef& old_input, const NodeDef& new_input, + int output_port_id = 0); + + // Deletes nodes from the graph. + void DeleteNodes(const std::set& nodes_to_delete); + + private: + void RemoveFanouts(NodeDef* node); +}; + +} // end namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_MUTABLE_GRAPH_VIEW_H_ diff --git a/linux/include/tensorflow/core/grappler/op_types.h b/linux/include/tensorflow/core/grappler/op_types.h new file mode 100644 index 000000000..7f86a5f29 --- /dev/null +++ b/linux/include/tensorflow/core/grappler/op_types.h @@ -0,0 +1,214 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_OP_TYPES_H_ +#define TENSORFLOW_CORE_GRAPPLER_OP_TYPES_H_ + +#include "tensorflow/core/framework/node_def.pb.h" +#include "tensorflow/core/lib/core/status.h" + +namespace tensorflow { +namespace grappler { +bool IsAdd(const NodeDef& node); +bool IsAddN(const NodeDef& node); +bool IsAll(const NodeDef& node); +bool IsAngle(const NodeDef& node); +bool IsAny(const NodeDef& node); +bool IsAnyDiv(const NodeDef& node); +bool IsApproximateEqual(const NodeDef& node); +bool IsAvgPoolGrad(const NodeDef& node); +bool IsAssert(const NodeDef& node); +bool IsAssign(const NodeDef& node); +bool IsAtan2(const NodeDef& node); +bool IsBetainc(const NodeDef& node); +bool IsBiasAdd(const NodeDef& node); +bool IsBiasAddGrad(const NodeDef& node); +bool IsBitcast(const NodeDef& node); +bool IsCast(const NodeDef& node); +bool IsCheckNumerics(const NodeDef& node); +bool IsCollective(const NodeDef& node); +bool IsComplex(const NodeDef& node); +bool IsComplexAbs(const NodeDef& node); +bool IsConj(const NodeDef& node); +bool IsConjugateTranspose(const NodeDef& node); +bool IsConcat(const NodeDef& node); +bool IsConcatOffset(const NodeDef& node); +bool IsConstant(const NodeDef& node); +bool IsConv2D(const NodeDef& node); +bool IsConv2DBackpropFilter(const NodeDef& node); +bool IsConv2DBackpropInput(const NodeDef& node); +bool IsConv3D(const NodeDef& node); +bool IsDepthwiseConv2dNative(const NodeDef& node); +bool IsDepthwiseConv2dNativeBackpropFilter(const NodeDef& node); +bool IsDepthwiseConv2dNativeBackpropInput(const NodeDef& node); +bool IsDequeueOp(const NodeDef& node); +bool IsDiv(const NodeDef& node); +bool IsElementWiseMonotonic(const NodeDef& node, bool* is_non_decreasing); +bool IsEluGrad(const NodeDef& node); +bool IsEnter(const NodeDef& node); +bool IsEqual(const NodeDef& node); +bool IsExit(const NodeDef& node); +bool IsExp(const NodeDef& node); +bool IsFill(const NodeDef& node); +bool IsFloorDiv(const NodeDef& node); +bool IsFloorMod(const NodeDef& node); +bool IsFusedBatchNormGrad(const NodeDef& node); +bool IsGreater(const NodeDef& node); +bool IsGreaterEqual(const NodeDef& node); +bool IsHistogramSummary(const NodeDef& node); +bool IsIdentity(const NodeDef& node); +bool IsIdentityN(const NodeDef& node); +bool IsIgamma(const NodeDef& node); +bool IsIgammac(const NodeDef& node); +bool IsImag(const NodeDef& node); +bool IsInvGrad(const NodeDef& node); +bool IsLess(const NodeDef& node); +bool IsLessEqual(const NodeDef& node); +bool IsLog(const NodeDef& node); +bool IsLogicalAnd(const NodeDef& node); +bool IsLogicalNot(const NodeDef& node); +bool IsLogicalOr(const NodeDef& node); +bool IsMax(const NodeDef& node); +bool IsMaximum(const NodeDef& node); +bool IsMaxPoolGrad(const NodeDef& node); +bool IsMean(const NodeDef& node); +bool IsMerge(const NodeDef& node); +bool IsMin(const NodeDef& node); +bool IsMinimum(const NodeDef& node); +bool IsMirrorPad(const NodeDef& node); +bool IsMirrorPadGrad(const NodeDef& node); +bool IsMod(const NodeDef& node); +bool IsMul(const NodeDef& node); +bool IsMatMul(const NodeDef& node); +bool IsNextIteration(const NodeDef& node); +bool IsPack(const NodeDef& node); +bool IsPad(const NodeDef& node); +bool IsPack(const NodeDef& node); +bool IsNeg(const NodeDef& node); +bool IsNoOp(const NodeDef& node); +bool IsNotEqual(const NodeDef& node); +bool IsPlaceholder(const NodeDef& node); +bool IsPolygamma(const NodeDef& node); +bool IsPrint(const NodeDef& node); +bool IsProd(const NodeDef& node); +bool IsPow(const NodeDef& node); +bool IsQueue(const NodeDef& node); +bool IsRandomShuffle(const NodeDef& node); +bool IsRank(const NodeDef& node); +bool IsReal(const NodeDef& node); +bool IsRealDiv(const NodeDef& node); +bool IsRelu6Grad(const NodeDef& node); +bool IsReluGrad(const NodeDef& node); +bool IsReciprocalGrad(const NodeDef& node); +bool IsRecv(const NodeDef& node); +bool IsReduction(const NodeDef& node); +bool IsReshape(const NodeDef& node); +bool IsRestore(const NodeDef& node); +bool IsReverse(const NodeDef& node); +bool IsReverseV2(const NodeDef& node); +bool IsRsqrt(const NodeDef& node); +bool IsRsqrtGrad(const NodeDef& node); +bool IsSelect(const NodeDef& node); +bool IsSeluGrad(const NodeDef& node); +bool IsSend(const NodeDef& node); +bool IsSize(const NodeDef& node); +bool IsSlice(const NodeDef& node); +bool IsShape(const NodeDef& node); +bool IsShapeN(const NodeDef& node); +bool IsShuffle(const NodeDef& node); +bool IsSigmoidGrad(const NodeDef& node); +bool IsSnapshot(const NodeDef& node); +bool IsSoftplusGrad(const NodeDef& node); +bool IsSoftsignGrad(const NodeDef& node); +bool IsSplit(const NodeDef& node); +bool IsSplitV(const NodeDef& node); +bool IsSqrt(const NodeDef& node); +bool IsSqrtGrad(const NodeDef& node); +bool IsSquare(const NodeDef& node); +bool IsSquaredDifference(const NodeDef& node); +bool IsSqueeze(const NodeDef& node); +bool IsStackOp(const NodeDef& node); +bool IsStackCloseOp(const NodeDef& node); +bool IsStackPushOp(const NodeDef& node); +bool IsStackPopOp(const NodeDef& node); +bool IsStopGradient(const NodeDef& node); +bool IsStridedSlice(const NodeDef& node); +bool IsStridedSliceGrad(const NodeDef& node); +bool IsSub(const NodeDef& node); +bool IsSum(const NodeDef& node); +bool IsSwitch(const NodeDef& node); +bool IsSymbolicGradient(const NodeDef& node); +bool IsTanhGrad(const NodeDef& node); +bool IsTile(const NodeDef& node); +bool IsTranspose(const NodeDef& node); +bool IsTruncateDiv(const NodeDef& node); +bool IsTruncateMod(const NodeDef& node); +bool IsUnpack(const NodeDef& node); +bool IsVariable(const NodeDef& node); +bool IsZeta(const NodeDef& node); + +// Return true if the op is an aggregation (e.g. Add, AddN). +// Returns false if it could not be determined to be so. +bool IsAggregate(const NodeDef& node); + +// Return true if the op is commutative (e.g. Mul, Add). +// Returns false if it could not be determined to be so. +bool IsCommutative(const NodeDef& node); + +// Returns true if the node is known to use persistent memory to store its +// value. +bool IsPersistent(const NodeDef& node); + +bool IsFreeOfSideEffect(const NodeDef& node); + +// Returns true if the takes a tensor reference as input, or if looking up its +// OpDef failed. +bool MaybeHasRefInput(const NodeDef& node); + +bool ModifiesFrameInfo(const NodeDef& node); + +// Returns true if the op is known to write to one or more of its inputs. +bool ModifiesInputsInPlace(const NodeDef& node); + +// Returns true if the op is an element-wise involution, i.e. if it is its +// own inverse such that f(f(x)) == x. +bool IsInvolution(const NodeDef& node); + +// Returns true if the op preserves the order and value of elements +// and shape of its first input tensor. +bool IsValueAndOrderAndShapePreserving(const NodeDef& node); + +// Returns true if the op preserves the order and value of elements in its +// first input tensor and possible changes its shape. +bool IsValueAndOrderPreserving(const NodeDef& node); + +// Returns true if the op in node only rearranges the order of elements in its +// first input tensor and possible changes its shape. More precisely, this +// function returns true if the op commutes with all element-wise operations. +bool IsValuePreserving(const NodeDef& node); + +// Returns true if node is idempotent w.r.t. its first input, i.e. if +// Op(Op(x, y, z), y, z) = Op(x, y, z). +bool IsIdempotent(const NodeDef& node); + +bool IsUnaryElementWise(const NodeDef& node); + +// Returns true if we can find an opdef corresponding to the op of the node. +bool HasOpDef(const NodeDef& node); + +} // end namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_OP_TYPES_H_ diff --git a/linux/include/tensorflow/core/grappler/optimizers/arithmetic_optimizer.h b/linux/include/tensorflow/core/grappler/optimizers/arithmetic_optimizer.h new file mode 100644 index 000000000..d457eb6d2 --- /dev/null +++ b/linux/include/tensorflow/core/grappler/optimizers/arithmetic_optimizer.h @@ -0,0 +1,136 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_ARITHMETIC_OPTIMIZER_H_ +#define TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_ARITHMETIC_OPTIMIZER_H_ + +#include +#include "tensorflow/core/grappler/costs/graph_properties.h" +#include "tensorflow/core/grappler/optimizers/graph_optimizer.h" +#include "tensorflow/core/grappler/utils.h" +#include "tensorflow/core/protobuf/rewriter_config.pb.h" + +namespace tensorflow { +namespace grappler { + +constexpr char kArithmeticOptimizer[] = "ArithmeticOptimizer"; + +// Optimize TF computations by reducing the arithmetic complexity required to +// run a model. +class ArithmeticOptimizer : public GraphOptimizer { + public: + ArithmeticOptimizer() + : opt_level_(RewriterConfig::ON), + options_(ArithmeticOptimizerOptions::Default(RewriterConfig::ON)) {} + + explicit ArithmeticOptimizer(RewriterConfig::Toggle opt_level) + : opt_level_(opt_level), + options_(ArithmeticOptimizerOptions::Default(opt_level)) {} + + ~ArithmeticOptimizer() override {} + + string name() const override { return "arithmetic_optimizer"; }; + + Status Optimize(Cluster* cluster, const GrapplerItem& item, + GraphDef* optimized_graph) override; + + void Feedback(Cluster* cluster, const GrapplerItem& item, + const GraphDef& optimized_graph, double result) override; + + private: + friend class ArithmeticOptimizerTest; + + // Granular control for arithmetic optimizer stages + struct ArithmeticOptimizerOptions { + bool combine_add_to_addn = true; + bool convert_sqrt_div_to_rsqrt_mul = true; + bool dedup_computations = true; + bool fold_conjugate_into_transpose = true; + bool fold_multiply_into_conv = true; + bool fold_transpose_into_matmul = true; + bool hoist_common_factor_out_of_aggregation = true; + bool hoist_cwise_unary_chains = true; + bool minimize_broadcasts = true; + bool optimize_max_or_min_of_monotonic = true; + bool remove_idempotent = true; + bool remove_identity_transpose = true; + bool remove_involution = true; + bool remove_logical_not = true; + bool remove_negation = true; + bool remove_redundant_bitcast = true; + bool remove_redundant_cast = true; + bool remove_redundant_reshape = true; + bool reorder_cast_and_transpose = true; + bool replace_mul_with_square = true; + bool simplify_aggregation = true; + bool convert_pow = true; + bool convert_log1p = true; + bool convert_expm1 = true; + bool unary_ops_composition = true; + + // Choose which arithmetic optimizer stages will be enabled for a given + // optimization level by default. + static ArithmeticOptimizerOptions Default( + RewriterConfig::Toggle opt_level) { + ArithmeticOptimizerOptions options; + return options; + } + }; + + // Returns true if it is safe to dedup node from the graph. + bool CanDedup(const NodeDef& node) const; + + // Dedup redundant nodes in the graph. + void DedupComputations(); + + // Forward the control dependencies anchored on src_nodes to the target_nodes. + void ForwardControlDependencies(NodeDef* target_node, + const std::vector& src_nodes); + + // Runs peep-hole optimizations on `optimized_graph`, e.g., removing inverse + // transposes. + Status SimplifyArithmeticOps(bool can_use_shapes); + // Tries to simplify the expression that roots at `node` and replaces the uses + // of `node` to the simplified expression. Returns the name of the simplified + // tensor (e.g. "split:1") or an empty string if no simplification is + // performed. + // + // `node_map` stores the mapping from node names to NodeDef*, and will be + // updated according to the rewrite. + // + // `new_nodes` will be populated with the new nodes this function creates and + // updates. The caller can push these nodes into the simplification queue to + // optimize them further. + // + // TODO(jingyue): This interface is not suitable for optimizing nodes with + // multiple output tensors. We should pass in a tensor name instead of a + // NodeDef. + string TrySimplifyAndReplaceUses(const NodeDef* node, + SetVector* nodes_to_simplify); + + RewriterConfig::Toggle opt_level_; + ArithmeticOptimizerOptions options_; + + bool fetch_nodes_known_ = false; + std::unordered_set nodes_to_preserve_; + std::unique_ptr node_map_; + std::unique_ptr graph_properties_; + GraphDef* optimized_graph_ = nullptr; // Not owned. +}; + +} // end namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_ARITHMETIC_OPTIMIZER_H_ diff --git a/linux/include/tensorflow/core/grappler/optimizers/auto_parallel.h b/linux/include/tensorflow/core/grappler/optimizers/auto_parallel.h new file mode 100644 index 000000000..63f6fe5b9 --- /dev/null +++ b/linux/include/tensorflow/core/grappler/optimizers/auto_parallel.h @@ -0,0 +1,66 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_AUTO_PARALLEL_H_ +#define TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_AUTO_PARALLEL_H_ + +#include "tensorflow/core/framework/variable.pb.h" +#include "tensorflow/core/grappler/optimizers/graph_optimizer.h" +#include "tensorflow/core/lib/core/status.h" + +namespace tensorflow { +namespace grappler { + +// Automatically parallelize a graph by splitting in the batch dimension. +class AutoParallel : public GraphOptimizer { + public: + AutoParallel(int num_replicas) : num_replicas_(num_replicas) { + CHECK(num_replicas_ >= 2); + } + ~AutoParallel() override {} + + string name() const override { return "autoparallel"; }; + + Status Optimize(Cluster* cluster, const GrapplerItem& item, + GraphDef* output) override; + + void Feedback(Cluster* cluster, const GrapplerItem& item, + const GraphDef& optimize_output, double result) override; + + private: + GraphDef graph_; + std::map all_nodes_; + std::set apply_gradients_nodes_; + std::set replica_nodes_; + std::set shared_nodes_; + const GrapplerItem* item_; + int num_replicas_; + int num_gpus_; + Status Initialize(const GrapplerItem& item); + NodeDef* AddNodeDivConst(); + NodeDef* AddNodeDiv(const string& name, const string& input_a, + const string& input_b); + NodeDef* AddNodeControl(const string& name, const std::set& deps, + GraphDef* graph); + bool NotSharedNode(const string& name); + void AddSharedNodes(GraphDef* graph); + void AddOneReplica(GraphDef* graph, int number); + void BuildGraph(GraphDef* graph); +}; + +} // end namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_AUTO_PARALLEL_H_ diff --git a/linux/include/tensorflow/core/grappler/optimizers/constant_folding.h b/linux/include/tensorflow/core/grappler/optimizers/constant_folding.h new file mode 100644 index 000000000..8593b3e0b --- /dev/null +++ b/linux/include/tensorflow/core/grappler/optimizers/constant_folding.h @@ -0,0 +1,235 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_CONSTANT_FOLDING_H_ +#define TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_CONSTANT_FOLDING_H_ + +#include "tensorflow/core/framework/device_base.h" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/resource_mgr.h" +#include "tensorflow/core/grappler/costs/graph_properties.h" +#include "tensorflow/core/grappler/optimizers/graph_optimizer.h" +#include "tensorflow/core/grappler/utils.h" +#include "tensorflow/core/protobuf/rewriter_config.pb.h" + +namespace tensorflow { +namespace grappler { + +const char kConstantFoldingConst[] = "ConstantFolding"; +const char kConstantFoldingCtrl[] = "ConstantFoldingCtrl"; + +// Constant folding optimization for a graph. +class ConstantFolding : public GraphOptimizer { + public: + static Status CreateNodeDef(const string& name, const TensorValue& tensor, + NodeDef* node); + static string AddControlDependency(const string& input_name, GraphDef* graph, + NodeMap* node_map); + + explicit ConstantFolding(DeviceBase* cpu_device); + ConstantFolding(RewriterConfig::Toggle opt_level, DeviceBase* cpu_device); + + ~ConstantFolding() override {} + + string name() const override { return "constant folding"; }; + + Status Optimize(Cluster* cluster, const GrapplerItem& item, + GraphDef* output) override; + + void Feedback(Cluster* cluster, const GrapplerItem& item, + const GraphDef& optimize_output, double result) override; + + private: + string OptimizedNodeName(const NodeDef& node, StringPiece suffix) const; + bool OptimizedNodeExists(const NodeDef& node, StringPiece suffix) const; + + bool IsReallyConstant(const NodeDef& node) const; + + Status MaterializeShapes(const GraphProperties& properties); + + Status MaterializeBroadcastGradientArgs(const NodeDef& node, + const GraphProperties& properties); + Status MaterializeReductionIndices(NodeDef* node, + const GraphProperties& properties); + + Status MaterializeConstants(const GraphProperties& properties); + bool IsFoldable(const NodeDef& node) const; + + Status EvaluateNode(const NodeDef& node, + const gtl::InlinedVector& inputs, + gtl::InlinedVector* output) const; + + Status EvaluateOneFoldable(const NodeDef& node, + std::vector* outputs); + + Status FoldNode(NodeDef* node, GraphDef* output_graph); + + bool IsOnes(const NodeDef& node) const; + bool IsZeros(const NodeDef& node) const; + void ReplaceOperationWithIdentity(int input_to_forward, + const GraphProperties& properties, + NodeDef* node, GraphDef* graph); + void ReplaceOperationWithSnapshot(int input_to_forward, + const GraphProperties& properties, + NodeDef* node, GraphDef* graph); + void ReplaceSubtractionFromZeroByNegation(NodeDef* node, GraphDef* graph); + Status ReplaceOperationWithConstant(double value, + const GraphProperties& properties, + const TensorShapeProto& shape, + NodeDef* node, GraphDef* graph, + bool* success); + void ReplaceDivisionOfOnesByReciprocal(NodeDef* node, GraphDef* graph); + Status FoldGraph(GraphDef* output); + + bool IsSimplifiableReduction(const NodeDef& node, + const GraphProperties& properties) const; + bool IsSimplifiableReshape(const NodeDef& node, + const GraphProperties& properties) const; + Status SimplifyGraph(bool use_shape_info, GraphDef* optimized_graph, + GraphProperties* properties); + Status SimplifyNode(bool use_shape_info, NodeDef* node, + GraphDef* optimized_graph, GraphProperties* properties); + + Status RunOptimizationPass(Cluster* cluster, const GrapplerItem& item, + GraphDef* output); + + // Applies partial constant folding for Concat which is not commutative. + // Returns true if the transformation applied successfully. + bool PartialConcatConstFolding(GraphDef* optimized_graph, + GraphProperties* properties, NodeDef* node); + + // Applies partial constant folding for associative operators AddN and + // AccumulateNV2. Returns true if the transformation applied successfully. + bool PartialAssocOpConstFolding(GraphDef* optimized_graph, + GraphProperties* properties, NodeDef* node); + + // Applies partial constant propagation through IdentityN operator. + // Returns true if the transformation applied successfully. + bool PartialConstPropThroughIdentityN(NodeDef* node); + + // Pushes down constants on '+' and '*' operators if applicable. Returns true + // the transformation applied successfully. + bool ConstantPushDown(NodeDef* node); + + // Aggregate constants present around a conv operator. Returns true if the + // transformation was applied successfully. + bool MulConvPushDown(NodeDef* node, const GraphProperties& properties); + + // Strength reduces floating point division by a constant Div(x, const) to + // multiplication by the reciprocal Mul(x, Reciprocal(const)). + bool ReduceDivToReciprocalMul(GraphDef* optimized_graph, NodeDef* node); + + // Simplifies arithmetic operations with ones or zeros. Returns the status, + // and updates the success input argument that denotes if any simplification + // was applied. + Status SimplifyArithmeticOperations(const GraphProperties& properties, + bool use_shape_info, + GraphDef* optimized_graph, NodeDef* node, + bool* success); + + // Simplifies a Reshape operation to an Identity operation if applicable. + bool SimplifyReshape(const GraphProperties& properties, bool use_shape_info, + NodeDef* node); + + // Simplifies a Reduction operation to an Identity operation if applicable. + bool SimplifyReduction(const GraphProperties& properties, NodeDef* node); + + // Switch(x, x) will always feed false to its false branch and true to + // its true branch. By rewriting the graph a bit, we can propagate these + // constants down the two output branches, and just use control dependencies + // to trigger the selected one at runtime. For example, + // + // +------+ + // x-->|Switch|-->a (in practice there may be multiple consumers of each + // x-->| |-->b output branch.) + // +------+ + // + // Is rewritten as + // + // +------+ + // x-->|Switch|-->Identity--^>Const(false)-->a + // x-->| |-->Identity--^>Const(true)-->b + // +------+ + bool SimplifySwitch(GraphDef* optimized_graph, NodeDef* node); + + // Moves constants past Enter node if applicable. + bool MoveConstantsPastEnter(GraphDef* optimized_graph, NodeDef* node); + + // Simplifies Pack operation if applicable. + bool SimplifyPack(GraphDef* optimized_graph, NodeDef* node); + + // Simplifies a Squeeze operation to an Identity operation if applicable. + bool SimplifySqueeze(const GraphProperties& properties, bool use_shape_info, + GraphDef* optimized_graph, NodeDef* node); + + // Simplifies a Pad operation to an Identity operation if applicable. + Status SimplifyPad(const GraphProperties& properties, bool use_shape_info, + GraphDef* optimized_graph, NodeDef* node, bool* success); + + // Simplifies a Tile operation to an Identity operation if applicable. + Status SimplifyTile(const GraphProperties& properties, bool use_shape_info, + GraphDef* optimized_graph, NodeDef* node, bool* success); + + // Simplifies a StridedSlice operation to an Identity operation if applicable. + Status SimplifyStridedSlice(const GraphProperties& properties, + bool use_shape_info, GraphDef* optimized_graph, + NodeDef* node, bool* success); + + // Simplifies a Slice operation to an Identity operation if applicable. + Status SimplifySlice(const GraphProperties& properties, bool use_shape_info, + GraphDef* optimized_graph, NodeDef* node, bool* success); + + // Removes Reverse op over dimensions with size 1. + Status RemoveReverse(const GraphProperties& properties, bool use_shape_info, + GraphDef* optimized_graph, NodeDef* node, bool* success); + + // Removes RandomShuffle op if it is scalar or first dimension is of size 1. + bool RemoveRandomShuffle(const GraphProperties& properties, + bool use_shape_info, GraphDef* optimized_graph, + NodeDef* node); + + // Removes Shuffle or Transpose op over dimensions of size 1. + Status RemoveShuffleOrTranspose(const GraphProperties& properties, + bool use_shape_info, + GraphDef* optimized_graph, NodeDef* node, + bool* success); + + // Removes Split or SplitV node if possible. + bool RemoveSplitOrSplitV(const GraphProperties& properties, + GraphDef* optimized_graph, NodeDef* node); + + bool MergeConcat(const GraphProperties& properties, bool use_shape_info, + GraphDef* optimized_graph, NodeDef* node); + + // Points to an externally provided device or to owned_device_; + RewriterConfig::Toggle opt_level_; + DeviceBase* cpu_device_; + std::unique_ptr owned_device_; + + std::unique_ptr resource_mgr_; + GraphDef* graph_; + std::unique_ptr node_map_; + std::unordered_set nodes_to_preserve_; + std::unordered_set nodes_whitelist_; + std::unordered_set feed_nodes_; + bool has_fetch_; + bool graph_modified_; + bool graph_contains_assign_or_inplace_op_; +}; + +} // end namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_CONSTANT_FOLDING_H_ diff --git a/linux/include/tensorflow/core/grappler/optimizers/custom_graph_optimizer.h b/linux/include/tensorflow/core/grappler/optimizers/custom_graph_optimizer.h new file mode 100644 index 000000000..ab9af5acf --- /dev/null +++ b/linux/include/tensorflow/core/grappler/optimizers/custom_graph_optimizer.h @@ -0,0 +1,37 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_CUSTOM_GRAPH_OPTIMIZER_H_ +#define TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_CUSTOM_GRAPH_OPTIMIZER_H_ + +#include "tensorflow/core/grappler/optimizers/graph_optimizer.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/protobuf/rewriter_config.pb.h" + +namespace tensorflow { +namespace grappler { + +// A custom optimizer that can be registered. +class CustomGraphOptimizer : public GraphOptimizer { + public: + virtual ~CustomGraphOptimizer() {} + virtual Status Init(const tensorflow::RewriterConfig_CustomGraphOptimizer* + config = nullptr) = 0; +}; + +} // end namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_CUSTOM_GRAPH_OPTIMIZER_H_ diff --git a/linux/include/tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.h b/linux/include/tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.h new file mode 100644 index 000000000..0b8e0b692 --- /dev/null +++ b/linux/include/tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.h @@ -0,0 +1,65 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_CUSTOM_GRAPH_OPTIMIZER_REGISTRY_H_ +#define TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_CUSTOM_GRAPH_OPTIMIZER_REGISTRY_H_ + +#include +#include +#include +#include + +#include "tensorflow/core/grappler/optimizers/custom_graph_optimizer.h" + +namespace tensorflow { +namespace grappler { + +class CustomGraphOptimizerRegistry { + public: + static std::unique_ptr CreateByNameOrNull( + const string& name); + + static std::vector GetRegisteredOptimizers(); + + typedef std::function Creator; + // Register graph optimizer which can be called during program initialization. + // This class is not thread-safe. + static void RegisterOptimizerOrDie(const Creator& optimizer_creator, + const string& name); +}; + +class CustomGraphOptimizerRegistrar { + public: + explicit CustomGraphOptimizerRegistrar( + const CustomGraphOptimizerRegistry::Creator& creator, + const string& name) { + CustomGraphOptimizerRegistry::RegisterOptimizerOrDie(creator, name); + } +}; + +#define REGISTER_GRAPH_OPTIMIZER_AS(MyCustomGraphOptimizerClass, name) \ + namespace { \ + static ::tensorflow::grappler::CustomGraphOptimizerRegistrar \ + MyCustomGraphOptimizerClass##_registrar( \ + []() { return new MyCustomGraphOptimizerClass; }, (name)); \ + } // namespace + +#define REGISTER_GRAPH_OPTIMIZER(MyCustomGraphOptimizerClass) \ + REGISTER_GRAPH_OPTIMIZER_AS(MyCustomGraphOptimizerClass, \ + #MyCustomGraphOptimizerClass) + +} // end namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_CUSTOM_GRAPH_OPTIMIZER_REGISTRY_H_ diff --git a/linux/include/tensorflow/core/grappler/optimizers/data/filter_fusion.h b/linux/include/tensorflow/core/grappler/optimizers/data/filter_fusion.h new file mode 100644 index 000000000..91a0364a4 --- /dev/null +++ b/linux/include/tensorflow/core/grappler/optimizers/data/filter_fusion.h @@ -0,0 +1,47 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DATA_FILTER_FUSION_H_ +#define TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DATA_FILTER_FUSION_H_ + +#include "tensorflow/core/grappler/optimizers/custom_graph_optimizer.h" + +namespace tensorflow { +namespace grappler { + +// This optimization fuses filter transformations. +class FilterFusion : public CustomGraphOptimizer { + public: + FilterFusion() = default; + ~FilterFusion() override = default; + + string name() const override { return "filter_fusion"; }; + + Status Init( + const tensorflow::RewriterConfig_CustomGraphOptimizer* config) override { + return Status::OK(); + } + + Status Optimize(Cluster* cluster, const GrapplerItem& item, + GraphDef* output) override; + + void Feedback(Cluster* cluster, const GrapplerItem& item, + const GraphDef& optimize_output, double result) override; +}; + +} // end namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DATA_FILTER_FUSION_H_ diff --git a/linux/include/tensorflow/core/grappler/optimizers/data/function_utils.h b/linux/include/tensorflow/core/grappler/optimizers/data/function_utils.h new file mode 100644 index 000000000..d4ce82465 --- /dev/null +++ b/linux/include/tensorflow/core/grappler/optimizers/data/function_utils.h @@ -0,0 +1,108 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DATA_FUNCTION_UTILS_H_ +#define TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DATA_FUNCTION_UTILS_H_ + +#include "tensorflow/core/framework/attr_value.pb.h" +#include "tensorflow/core/framework/function.pb.h" +#include "tensorflow/core/framework/graph.pb.h" +#include "tensorflow/core/framework/node_def.pb.h" +#include "tensorflow/core/framework/tensor.pb.h" +#include "tensorflow/core/framework/tensor_shape.pb.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/grappler/mutable_graph_view.h" +#include "tensorflow/core/grappler/utils.h" +#include "tensorflow/core/lib/core/errors.h" + +namespace tensorflow { +namespace grappler { +namespace function_utils { +// This namespace contains utility functions for querying and modifying +// FunctionDefs. + +// Describes a FunctionDef input tensor. In FunctionDefs, input tensor strings +// have the format node_name:node_output:position (if they derive from nodes), +// or input_name (if they derive from an argument). +struct FunctionDefTensorDesc { + FunctionDefTensorDesc() = default; + + FunctionDefTensorDesc(const string& node_name, const string& output, + int position); + + // Parses node_name:node_output:position string into its components. + explicit FunctionDefTensorDesc(const string& input); + + // TODO(rachelim): Add provisions to deal with special formats, like how + // GrapplerFunctionItem expands node output range if position is not defined + string full_str; + string node_name; + string node_output; + int position = -1; +}; + +// Replaces all references to `from` tensor in func's nodes' inputs and retvals +// to `to` tensor. This is similar to `MutableGraphView::ReplaceInputs`. +void ReplaceReferences(const string& from, const string& to, FunctionDef* func); + +// Adds a function output to the function def, ensuring that the output key +// is unique, and maps to output_tensor_name in the ret dict. +void AddFunctionOutputWithUniqueName(StringPiece prefix, + StringPiece output_tensor_name, + FunctionDef* function, DataType dt); + +// Adds a node to a FunctionDef. +NodeDef* AddNode(StringPiece name, StringPiece op, + const std::vector& inputs, + const std::vector>& attributes, + FunctionDef* fd); + +// Checks whether the function contains a node with the given name. +bool ContainsFunctionNodeWithName(StringPiece name, + const FunctionDef& function); + +// Checks whether the function contains a node with the given op. +bool ContainsFunctionNodeWithOp(StringPiece op, const FunctionDef& function); + +// Checks whether the function contains an output with the given name. +bool ContainsFunctionOutputWithName(StringPiece name, + const FunctionDef& function); + +// Returns the index of the function input with the given name or -1 if the +// function node does not exist. +int FindFunctionInputWithName(StringPiece name, const FunctionDef& function); + +// Returns the index of the function output with the given name or -1 if the +// function node does not exist. +int FindFunctionOutputWithName(StringPiece name, const FunctionDef& function); + +// Returns the index of the function node with the given name or -1 if the +// function node does not exist. +int FindFunctionNodeWithName(StringPiece name, const FunctionDef& function); + +// Returns the index of the function node with the given op or -1 if the +// function node does not exist. +int FindFunctionNodeWithOp(StringPiece op, const FunctionDef& function); + +// Sets the function node name using the `prefix` as a prefix while guaranteeing +// the name is unique across the functions nodes. +void SetUniqueFunctionNodeName(StringPiece prefix, FunctionDef* function, + NodeDef* node); + +} // end namespace function_utils +} // end namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DATA_FUNCTION_UTILS_H_ diff --git a/linux/include/tensorflow/core/grappler/optimizers/data/fusion_utils.h b/linux/include/tensorflow/core/grappler/optimizers/data/fusion_utils.h new file mode 100644 index 000000000..19b7002dc --- /dev/null +++ b/linux/include/tensorflow/core/grappler/optimizers/data/fusion_utils.h @@ -0,0 +1,135 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DATA_FUSION_UTILS_H_ +#define TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DATA_FUSION_UTILS_H_ + +#include +#include "tensorflow/core/framework/attr_value.pb.h" +#include "tensorflow/core/framework/node_def.pb.h" +#include "tensorflow/core/grappler/op_types.h" +#include "tensorflow/core/grappler/optimizers/data/graph_utils.h" +#include "tensorflow/core/lib/gtl/inlined_vector.h" +#include "tensorflow/core/platform/protobuf.h" + +namespace tensorflow { +namespace grappler { +namespace fusion_utils { + +// These functions are invoked with first and second function signature, +// should set a signature of fused second_function. +using SetFunctionSignatureFn = std::function; + +using StringCollection = gtl::InlinedVector; + +// These functions are invoked with nodes from second function that were +// previously taking arguments as input. The `arg_num` tells which +// function argument node was using as an input, e.g: +// node(arg_1, other_node, arg_4) +// would be called on the first and third input with arg_num equal 1 and 4. +// It should set up inputs based on first function inputs or outputs or +// second function inputs. +using SetInputFn = + std::function; + +// This function is invoked with first and second function ret. It is used to +// set up returns of fused function. +using SetOutputFn = + std::function& parent_ret, + const protobuf::Map& second_function_ret, + protobuf::Map* fused_ret)>; + +using SetNodesFn = std::function; + +void MergeNodes(const FunctionDef& first_function, + const FunctionDef& second_function, FunctionDef* fused_function, + FunctionDefLibrary* library); + +// Returns true if functions can be composed. +bool CanCompose(const OpDef& first_signature, const OpDef& second_signature); + +void ComposeSignature(const OpDef& first_signature, + const OpDef& second_signature, OpDef* fused_signature); + +string ComposeInput(const StringCollection& first_inputs, + const StringCollection& second_inputs, + const StringCollection& first_outputs, int arg_num); + +// Sets output to the composition of first and second function: +// second_function(first_function(args...)). +void ComposeOutput(const protobuf::Map& first_ret, + const protobuf::Map& second_ret, + protobuf::Map* fused_ret); + +// Set input signature to `first_function_signature` and output signature +// to `first_function_signature` + `second_function_signature` +void CombineSignature(const OpDef& first_signature, + const OpDef& second_signature, OpDef* fused_signature); + +// Apart from first function returns, return values from second function as +// extra returns like: +// return *first_function(...), *second_function(...) +void CombineOutput(const protobuf::Map& first_ret, + const protobuf::Map& second_ret, + protobuf::Map* fused_ret); + +// Returns true if both signatures have the same number of input and output +// args. +bool HasSameSignature(const OpDef& first_signature, + const OpDef& second_signature); + +// Check if both signatures are same and copy it from `first_signature`. +void SameSignature(const OpDef& first_signature, const OpDef& second_signature, + OpDef* fused_signature); + +// Take the same input as first function. +string SameInput(const StringCollection& first_inputs, + const StringCollection& second_inputs, + const StringCollection& first_outputs, int arg_num); + +// Create a fused function that computes the short-circuit logical AND of the +// result of the first function and the result of the second function. +void LazyConjunctionOutput(const protobuf::Map& first_ret, + const protobuf::Map& second_ret, + protobuf::Map* fused_ret); + +void LazyConjunctionNodes(const FunctionDef& first_function, + const FunctionDef& second_function, + FunctionDef* fused_function, + FunctionDefLibrary* library); + +// Fuse `first_function` with `second_function`, setting `fused_name_prefix` as +// a name prefix. The nodes from `first_function` are copied unmodified. All +// of the setup functions are called with a copy of second function having names +// that are not conflicting with first function. This means that copied nodes +// from second function can end up having different names. For explanation of +// set up functions see the documentation of the functions types. +FunctionDef* FuseFunctions( + const FunctionDef& first_function, const FunctionDef& second_function, + StringPiece fused_name_prefix, const SetFunctionSignatureFn& set_signature, + const SetInputFn& set_input, const SetOutputFn& set_output, + const SetNodesFn& set_nodes, FunctionDefLibrary* library); + +} // namespace fusion_utils +} // namespace grappler +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DATA_FUSION_UTILS_H_ diff --git a/linux/include/tensorflow/core/grappler/optimizers/data/graph_utils.h b/linux/include/tensorflow/core/grappler/optimizers/data/graph_utils.h new file mode 100644 index 000000000..3af34f690 --- /dev/null +++ b/linux/include/tensorflow/core/grappler/optimizers/data/graph_utils.h @@ -0,0 +1,138 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DATA_GRAPH_UTILS_H_ +#define TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DATA_GRAPH_UTILS_H_ + +#include "tensorflow/core/framework/attr_value.pb.h" +#include "tensorflow/core/framework/function.pb.h" +#include "tensorflow/core/framework/graph.pb.h" +#include "tensorflow/core/framework/node_def.pb.h" +#include "tensorflow/core/framework/tensor.pb.h" +#include "tensorflow/core/framework/tensor_shape.pb.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/grappler/mutable_graph_view.h" +#include "tensorflow/core/grappler/utils.h" +#include "tensorflow/core/lib/core/errors.h" + +namespace tensorflow { +namespace grappler { +namespace graph_utils { + +// Returns the index of the first element in collection that fulfills predicate. +// If no such element exists, returns -1. +template +int GetFirstElementIndexWithPredicate(const Predicate& predicate, + const Collection& collection) { + unsigned idx = 0; + for (auto&& element : collection) { + if (predicate(element)) { + return idx; + } + idx++; + } + return -1; +} + +// Adds a node to the graph. +NodeDef* AddNode(StringPiece name, StringPiece op, + const std::vector& inputs, + const std::vector>& attributes, + MutableGraphView* graph); + +// Adds Placeholder node for given type. +NodeDef* AddScalarPlaceholder(DataType dtype, MutableGraphView* graph); + +// Adds a Const node with the given value to the graph. +template +NodeDef* AddScalarConstNode(T v, MutableGraphView* graph) { + // is_same is an idiomatic hack for making it compile if not instantiated. + // Replacing with false will result in a compile-time error. + static_assert(!std::is_same::value, + "Invalid specialization of this method for type T."); + return {}; +} + +template <> +NodeDef* AddScalarConstNode(bool v, MutableGraphView* graph); +template <> +NodeDef* AddScalarConstNode(double v, MutableGraphView* graph); +template <> +NodeDef* AddScalarConstNode(float v, MutableGraphView* graph); +template <> +NodeDef* AddScalarConstNode(int v, MutableGraphView* graph); +template <> +NodeDef* AddScalarConstNode(int64 v, MutableGraphView* graph); +template <> +NodeDef* AddScalarConstNode(StringPiece v, MutableGraphView* graph); + +// Checks whether the two graphs are the same. +bool Compare(const GraphDef& g1, const GraphDef& g2); + +// Checks whether the graph contains a node with the given name. +bool ContainsGraphNodeWithName(StringPiece name, const GraphDef& graph); + +// Checks whether the library contains a function with the given name. +bool ContainsGraphFunctionWithName(StringPiece name, + const FunctionDefLibrary& library); + +// Checks whether the graph contains a node with the given op. +bool ContainsNodeWithOp(StringPiece op, const GraphDef& graph); + +// Returns the index of the node with the given name or -1 if the node does +// not exist. +int FindGraphNodeWithName(StringPiece name, const GraphDef& graph); + +// Returns the index of the function with the given name or -1 if the function +// does not exist. +int FindGraphFunctionWithName(StringPiece name, + const FunctionDefLibrary& library); + +// Returns the index of the first node with the given op or -1 if no such node +// exists. +int FindGraphNodeWithOp(StringPiece op, const GraphDef& graph); + +// Gets the 0th input to a node in the graph. +NodeDef* GetInputNode(const NodeDef& node, const MutableGraphView& graph); + +// Returns the list of indices of all nodes with the given op or empty list if +// no such node exists. +std::vector FindAllGraphNodesWithOp(const string& op, + const GraphDef& graph); + +// Sets the node name using `prefix` as a prefix while guaranteeing the name +// is unique across the graph. +void SetUniqueGraphNodeName(StringPiece prefix, GraphDef* graph, NodeDef* node); + +// Sets the function name using the `prefix` name as a prefix while guaranteeing +// the name is unique across the function library. +void SetUniqueGraphFunctionName(StringPiece prefix, FunctionDefLibrary* library, + FunctionDef* function); + +// Copies attribute having name `attribute_name` from node `from` to node +// `to_node`. +void CopyAttribute(const string& attribute_name, const NodeDef& from, + NodeDef* to_node); + +// Concatenates list attribute having name `attribute_name` from `first` and +// `second` node, setting it to `to_node`. +void ConcatAttributeList(const string& attribute_name, const NodeDef& first, + const NodeDef& second, NodeDef* to_node); + +} // end namespace graph_utils +} // end namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DATA_GRAPH_UTILS_H_ diff --git a/linux/include/tensorflow/core/grappler/optimizers/data/hoist_random_uniform.h b/linux/include/tensorflow/core/grappler/optimizers/data/hoist_random_uniform.h new file mode 100644 index 000000000..d1bcf6782 --- /dev/null +++ b/linux/include/tensorflow/core/grappler/optimizers/data/hoist_random_uniform.h @@ -0,0 +1,55 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DATA_HOIST_RANDOM_UNIFORM_H_ +#define TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DATA_HOIST_RANDOM_UNIFORM_H_ + +#include "tensorflow/core/grappler/optimizers/custom_graph_optimizer.h" + +namespace tensorflow { +namespace grappler { + +// This optimization hoists instances of `random_uniform` out of a function +// with the aim of making it stateless. It creates a new function that takes a +// random seed as an extra argument and uses `stateless_random_uniform` instead +// of `random_uniform` to make it stateless. +// It also creates RandomDataset(seed).batch(2), which is zipped with old input +// to the map. The batching in RandomDataset is because we need 2 seeds for +// `stateless_random_uniform`. +// TODO(prazek): for now only `RandomUniform` is handled, but we could handle +// `RandomUniformInt` similarly. +class HoistRandomUniform : public CustomGraphOptimizer { + public: + HoistRandomUniform() = default; + ~HoistRandomUniform() override = default; + + string name() const override { return "hoist_random_uniform"; }; + + Status Init( + const tensorflow::RewriterConfig_CustomGraphOptimizer* config) override { + return Status::OK(); + } + + Status Optimize(Cluster* cluster, const GrapplerItem& item, + GraphDef* output) override; + + void Feedback(Cluster* cluster, const GrapplerItem& item, + const GraphDef& optimize_output, double result) override; +}; + +} // end namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DATA_HOIST_RANDOM_UNIFORM_H_ diff --git a/linux/include/tensorflow/core/grappler/optimizers/data/latency_all_edges.h b/linux/include/tensorflow/core/grappler/optimizers/data/latency_all_edges.h new file mode 100644 index 000000000..f6c71a9ec --- /dev/null +++ b/linux/include/tensorflow/core/grappler/optimizers/data/latency_all_edges.h @@ -0,0 +1,46 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DATA_LATENCY_ALL_EDGES_H_ +#define TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DATA_LATENCY_ALL_EDGES_H_ + +#include "tensorflow/core/grappler/optimizers/custom_graph_optimizer.h" + +namespace tensorflow { +namespace grappler { + +class LatencyAllEdges : public CustomGraphOptimizer { + public: + LatencyAllEdges() = default; + ~LatencyAllEdges() override = default; + + string name() const override { return "latency_all_edges"; }; + + Status Init( + const tensorflow::RewriterConfig_CustomGraphOptimizer* config) override { + return Status::OK(); + } + + Status Optimize(Cluster* cluster, const GrapplerItem& item, + GraphDef* output) override; + + void Feedback(Cluster* cluster, const GrapplerItem& item, + const GraphDef& optimize_output, double result) override; +}; + +} // end namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DATA_LATENCY_ALL_EDGES_H_ diff --git a/linux/include/tensorflow/core/grappler/optimizers/data/map_and_batch_fusion.h b/linux/include/tensorflow/core/grappler/optimizers/data/map_and_batch_fusion.h new file mode 100644 index 000000000..2c6483110 --- /dev/null +++ b/linux/include/tensorflow/core/grappler/optimizers/data/map_and_batch_fusion.h @@ -0,0 +1,46 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DATA_MAP_AND_BATCH_FUSION_H_ +#define TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DATA_MAP_AND_BATCH_FUSION_H_ + +#include "tensorflow/core/grappler/optimizers/custom_graph_optimizer.h" + +namespace tensorflow { +namespace grappler { + +class MapAndBatchFusion : public CustomGraphOptimizer { + public: + MapAndBatchFusion() = default; + ~MapAndBatchFusion() override = default; + + string name() const override { return "map_and_batch_fusion"; }; + + Status Init( + const tensorflow::RewriterConfig_CustomGraphOptimizer* config) override { + return Status::OK(); + } + + Status Optimize(Cluster* cluster, const GrapplerItem& item, + GraphDef* output) override; + + void Feedback(Cluster* cluster, const GrapplerItem& item, + const GraphDef& optimize_output, double result) override; +}; + +} // end namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DATA_MAP_AND_BATCH_FUSION_H_ diff --git a/linux/include/tensorflow/core/grappler/optimizers/data/map_and_filter_fusion.h b/linux/include/tensorflow/core/grappler/optimizers/data/map_and_filter_fusion.h new file mode 100644 index 000000000..ba25ca059 --- /dev/null +++ b/linux/include/tensorflow/core/grappler/optimizers/data/map_and_filter_fusion.h @@ -0,0 +1,51 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DATA_MAP_AND_FILTER_FUSION_H_ +#define TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DATA_MAP_AND_FILTER_FUSION_H_ + +#include "tensorflow/core/grappler/optimizers/custom_graph_optimizer.h" + +namespace tensorflow { +namespace grappler { + +// This transformation fuses map and filter operations by moving computation of +// filter predicate to MapDataset, which as a result produces an extra boolean +// component. The FilterDataset is transformed to FilterByLastComponent - a +// custom kernel that filters elements based on a value of the boolean +// component. +class MapAndFilterFusion : public CustomGraphOptimizer { + public: + MapAndFilterFusion() = default; + ~MapAndFilterFusion() override = default; + + string name() const override { return "map_and_filter_fusion"; }; + + Status Init( + const tensorflow::RewriterConfig_CustomGraphOptimizer* config) override { + return Status::OK(); + } + + Status Optimize(Cluster* cluster, const GrapplerItem& item, + GraphDef* output) override; + + void Feedback(Cluster* cluster, const GrapplerItem& item, + const GraphDef& optimize_output, double result) override; +}; + +} // end namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DATA_MAP_AND_FILTER_FUSION_H_ diff --git a/linux/include/tensorflow/core/grappler/optimizers/data/map_fusion.h b/linux/include/tensorflow/core/grappler/optimizers/data/map_fusion.h new file mode 100644 index 000000000..a6a06592b --- /dev/null +++ b/linux/include/tensorflow/core/grappler/optimizers/data/map_fusion.h @@ -0,0 +1,47 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DATA_MAP_FUSION_H_ +#define TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DATA_MAP_FUSION_H_ + +#include "tensorflow/core/grappler/optimizers/custom_graph_optimizer.h" + +namespace tensorflow { +namespace grappler { + +// This optimization fuses map transformations by merging their map functions. +class MapFusion : public CustomGraphOptimizer { + public: + MapFusion() = default; + ~MapFusion() override = default; + + string name() const override { return "map_fusion"; }; + + Status Init( + const tensorflow::RewriterConfig_CustomGraphOptimizer* config) override { + return Status::OK(); + } + + Status Optimize(Cluster* cluster, const GrapplerItem& item, + GraphDef* output) override; + + void Feedback(Cluster* cluster, const GrapplerItem& item, + const GraphDef& optimize_output, double result) override; +}; + +} // end namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DATA_MAP_FUSION_H_ diff --git a/linux/include/tensorflow/core/grappler/optimizers/data/map_parallelization.h b/linux/include/tensorflow/core/grappler/optimizers/data/map_parallelization.h new file mode 100644 index 000000000..ac9cf7e12 --- /dev/null +++ b/linux/include/tensorflow/core/grappler/optimizers/data/map_parallelization.h @@ -0,0 +1,47 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DATA_MAP_PARALLELIZATION_H_ +#define TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DATA_MAP_PARALLELIZATION_H_ + +#include "tensorflow/core/grappler/optimizers/custom_graph_optimizer.h" + +namespace tensorflow { +namespace grappler { + +// This optimization parallelizes MapDataset when function is stateless. +class MapParallelization : public CustomGraphOptimizer { + public: + MapParallelization() = default; + ~MapParallelization() override = default; + + string name() const override { return "map_parallelization"; }; + + Status Init( + const tensorflow::RewriterConfig_CustomGraphOptimizer* config) override { + return Status::OK(); + } + + Status Optimize(Cluster* cluster, const GrapplerItem& item, + GraphDef* output) override; + + void Feedback(Cluster* cluster, const GrapplerItem& item, + const GraphDef& optimize_output, double result) override; +}; + +} // end namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DATA_MAP_PARALLELIZATION_H_ diff --git a/linux/include/tensorflow/core/grappler/optimizers/data/map_vectorization.h b/linux/include/tensorflow/core/grappler/optimizers/data/map_vectorization.h new file mode 100644 index 000000000..cc56a8ee5 --- /dev/null +++ b/linux/include/tensorflow/core/grappler/optimizers/data/map_vectorization.h @@ -0,0 +1,46 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DATA_MAP_VECTORIZATION_H_ +#define TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DATA_MAP_VECTORIZATION_H_ + +#include "tensorflow/core/grappler/optimizers/custom_graph_optimizer.h" + +namespace tensorflow { +namespace grappler { + +class MapVectorization : public CustomGraphOptimizer { + public: + MapVectorization() = default; + ~MapVectorization() override = default; + + string name() const override { return "map_vectorization"; }; + + Status Init( + const tensorflow::RewriterConfig_CustomGraphOptimizer* config) override { + return Status::OK(); + } + + Status Optimize(Cluster* cluster, const GrapplerItem& item, + GraphDef* output) override; + + void Feedback(Cluster* cluster, const GrapplerItem& item, + const GraphDef& optimize_output, double result) override; +}; + +} // end namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DATA_MAP_VECTORIZATION_H_ diff --git a/linux/include/tensorflow/core/grappler/optimizers/data/noop_elimination.h b/linux/include/tensorflow/core/grappler/optimizers/data/noop_elimination.h new file mode 100644 index 000000000..c67cea49d --- /dev/null +++ b/linux/include/tensorflow/core/grappler/optimizers/data/noop_elimination.h @@ -0,0 +1,48 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DATA_NOOP_ELIMINATION_H_ +#define TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DATA_NOOP_ELIMINATION_H_ + +#include "tensorflow/core/grappler/optimizers/custom_graph_optimizer.h" + +namespace tensorflow { +namespace grappler { + +// This class eliminates tf.data transformations such as `take(n)` (for n < 0), +// `skip(0)`, or `repeat(1)` +class NoOpElimination : public CustomGraphOptimizer { + public: + NoOpElimination() = default; + ~NoOpElimination() override = default; + + string name() const override { return "noop_elimination"; }; + + Status Init( + const tensorflow::RewriterConfig_CustomGraphOptimizer* config) override { + return Status::OK(); + } + + Status Optimize(Cluster* cluster, const GrapplerItem& item, + GraphDef* output) override; + + void Feedback(Cluster* cluster, const GrapplerItem& item, + const GraphDef& optimize_output, double result) override; +}; + +} // end namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DATA_NOOP_ELIMINATION_H_ diff --git a/linux/include/tensorflow/core/grappler/optimizers/data/shuffle_and_repeat_fusion.h b/linux/include/tensorflow/core/grappler/optimizers/data/shuffle_and_repeat_fusion.h new file mode 100644 index 000000000..c8fa53edc --- /dev/null +++ b/linux/include/tensorflow/core/grappler/optimizers/data/shuffle_and_repeat_fusion.h @@ -0,0 +1,46 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DATA_SHUFFLE_AND_REPEAT_FUSION_H_ +#define TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DATA_SHUFFLE_AND_REPEAT_FUSION_H_ + +#include "tensorflow/core/grappler/optimizers/custom_graph_optimizer.h" + +namespace tensorflow { +namespace grappler { + +class ShuffleAndRepeatFusion : public CustomGraphOptimizer { + public: + ShuffleAndRepeatFusion() = default; + ~ShuffleAndRepeatFusion() override = default; + + string name() const override { return "shuffle_and_repeat_fusion"; }; + + Status Init( + const tensorflow::RewriterConfig_CustomGraphOptimizer* config) override { + return Status::OK(); + } + + Status Optimize(Cluster* cluster, const GrapplerItem& item, + GraphDef* output) override; + + void Feedback(Cluster* cluster, const GrapplerItem& item, + const GraphDef& optimize_output, double result) override; +}; + +} // end namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DATA_SHUFFLE_AND_REPEAT_FUSION_H_ diff --git a/linux/include/tensorflow/core/grappler/optimizers/data/vectorization/vectorizer.h b/linux/include/tensorflow/core/grappler/optimizers/data/vectorization/vectorizer.h new file mode 100644 index 000000000..56eb88c95 --- /dev/null +++ b/linux/include/tensorflow/core/grappler/optimizers/data/vectorization/vectorizer.h @@ -0,0 +1,52 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DATA_VECTORIZATION_VECTORIZER_H_ +#define TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DATA_VECTORIZATION_VECTORIZER_H_ + +#include "tensorflow/core/framework/function.pb.h" +#include "tensorflow/core/graph/graph.h" +#include "tensorflow/core/lib/core/errors.h" +#include "tensorflow/core/lib/core/status.h" + +namespace tensorflow { +namespace grappler { +namespace vectorization_utils { + +// Describes a tensor with its operation Node and output position +typedef std::pair Port; + +// Interface for vectorization of TensorFlow operations. See `CastVectorizer` +// for an example. +class Vectorizer { + public: + virtual ~Vectorizer() {} + + // Vectorizes an operation, `node`, by adding Node(s) to `outer_scope` + // that produce the same vector output(s) as executing `node`'s op + // on elements of the vector inputs. The new Node(s) collectively have the + // same number of input and output ports as the node being converted. + // Adds mappings for the new nodes' input and output ports to `inputs` and + // `outputs` respectively, where the i'th Port in inputs/outputs + // corresponds to the i'th input/output port of the node to be converted. + virtual Status Vectorize(const Node& node, Graph* outer_scope, + std::vector* input_ports, + std::vector* output_ports) = 0; +}; + +} // namespace vectorization_utils +} // namespace grappler +} // namespace tensorflow +#endif // TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DATA_VECTORIZATION_VECTORIZER_H_ diff --git a/linux/include/tensorflow/core/grappler/optimizers/data/vectorization/vectorizer_registry.h b/linux/include/tensorflow/core/grappler/optimizers/data/vectorization/vectorizer_registry.h new file mode 100644 index 000000000..16159d47c --- /dev/null +++ b/linux/include/tensorflow/core/grappler/optimizers/data/vectorization/vectorizer_registry.h @@ -0,0 +1,75 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DATA_VECTORIZATION_VECTORIZER_REGISTRY_H_ +#define TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DATA_VECTORIZATION_VECTORIZER_REGISTRY_H_ + +#include +#include + +#include "tensorflow/core/grappler/optimizers/data/vectorization/vectorizer.h" + +namespace tensorflow { +namespace grappler { +namespace vectorization_utils { + +// A global VectorizerRegistry is used to hold all the vectorizers. +class VectorizerRegistry { + public: + // Returns a pointer to a global VectorizerRegistry object. + static VectorizerRegistry* Global(); + + // Returns a pointer to a vectorizer that can vectorize an op for the op type. + Vectorizer* Get(const string& op_type); + + // Registers a vectorizer that can vectorize an op for the given op type. + void Register(const string& op_type, std::unique_ptr vectorizer); + + private: + std::map> vectorizers_; +}; + +namespace vectorizer_registration { + +class VectorizerRegistration { + public: + VectorizerRegistration(const string& op_type, + std::unique_ptr vectorizer) { + VectorizerRegistry::Global()->Register(op_type, std::move(vectorizer)); + } +}; + +} // namespace vectorizer_registration + +#define REGISTER_VECTORIZER(op_type, vectorizer) \ + REGISTER_VECTORIZER_UNIQ_HELPER(__COUNTER__, op_type, vectorizer) + +#define REGISTER_VECTORIZER_UNIQ_HELPER(ctr, op_type, vectorizer) \ + REGISTER_VECTORIZER_UNIQ(ctr, op_type, vectorizer) + +#define REGISTER_VECTORIZER_UNIQ(ctr, op_type, vectorizer) \ + static ::tensorflow::grappler::vectorization_utils:: \ + vectorizer_registration::VectorizerRegistration \ + vectorizer_registration_##ctr( \ + op_type, \ + ::std::unique_ptr< \ + ::tensorflow::grappler::vectorization_utils::Vectorizer>( \ + new vectorizer())) + +} // namespace vectorization_utils +} // namespace grappler +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DATA_VECTORIZATION_VECTORIZER_REGISTRY_H_ diff --git a/linux/include/tensorflow/core/grappler/optimizers/data/vectorization_utils.h b/linux/include/tensorflow/core/grappler/optimizers/data/vectorization_utils.h new file mode 100644 index 000000000..bd7d39090 --- /dev/null +++ b/linux/include/tensorflow/core/grappler/optimizers/data/vectorization_utils.h @@ -0,0 +1,97 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DATA_VECTORIZATION_UTILS_H_ +#define TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DATA_VECTORIZATION_UTILS_H_ + +#include "tensorflow/core/framework/function.pb.h" +#include "tensorflow/core/lib/core/errors.h" +#include "tensorflow/core/lib/core/status.h" + +namespace tensorflow { +namespace grappler { +namespace vectorization_utils { + +// Given a MapDefun node (`map_defun_node`) in a FunctionDef (`outer_scope`) +// that maps a function in lib across some input vector elements, +// `VectorizeMapDefun` attempts to create a vectorized version of `outer_scope` +// by "lifting" operations from the MapDefun function to the new function +// (`result`); that is, replacing operations in the MapDefun function with +// operations that produce the same vector output(s) as executing the original +// operations on elements of vector input(s) would. If all operations in the +// MapDefun function are successfully lifted, `result` has no MapDefun node +// altogether. However, if some operations cannot be lifted, and this +// vectorization only succeeds partially, a MapDefun node remains in `result` to +// be used for operations that were not lifted, and the modified MapDefun +// function is added to `lib`. The newly vectorized function `result` is also +// added to `lib`. +// +// Returns Status::OK() if the vectorization is completely or partially +// successful. Otherwise, returns an error, and sets `result` to nullptr. +// +// Example: +// If the input to the `VectorizeMapDefun` function is a MapDefun +// whose `map_defun_fn` performs the Cast operation, the vectorization will +// eliminate the MapDefun. This is because the Cast operation supports +// any tensor shape and can thus be lifted to `result`. +// +// Before: +// +// +// outer_scope +------+ +// +---------------+ Arg0 +---------+ +// | +---+--+ | +// | | | +// | map_defun_fn +---v--+ | +// | +-----------+ Arg0 +-----+ | +// | | +---+--+ | | +// | | | | | +// | | | | | +// | | +---v--+ | | +// | | | Cast | | | +// | | +---+--+ | | +// | | | | | +// | | +---v--+ | | +// | +-----------+ Ret0 +-----+ | +// | +---+--+ | +// | | | +// | +---v--+ | +// +---------------+ Ret0 +---------+ +// +------+ +// +// +// After: +// +// result +------+ +// +---------------+ Arg0 +---------+ +// | +---+--+ | +// | | | +// | +---v--+ | +// | | Cast | | +// | +---+--+ | +// | | | +// | +---v--+ | +// +---------------+ Ret0 +---------+ +// +------+ +// +Status VectorizeMapDefun(const FunctionDef& outer_scope, + const NodeDef& map_defun_node, FunctionDefLibrary* lib, + FunctionDef** result); + +} // end namespace vectorization_utils +} // end namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DATA_VECTORIZATION_UTILS_H_ diff --git a/linux/include/tensorflow/core/grappler/optimizers/debug_stripper.h b/linux/include/tensorflow/core/grappler/optimizers/debug_stripper.h new file mode 100644 index 000000000..1fe25aa1c --- /dev/null +++ b/linux/include/tensorflow/core/grappler/optimizers/debug_stripper.h @@ -0,0 +1,43 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DEBUG_STRIPPER_H_ +#define TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DEBUG_STRIPPER_H_ + +#include "tensorflow/core/grappler/optimizers/graph_optimizer.h" + +namespace tensorflow { +namespace grappler { + +// DebugStripper strips off debug-related nodes (e.g. +// Assert, CheckNumerics, Print) from the graph. +class DebugStripper : public GraphOptimizer { + public: + DebugStripper() {} + ~DebugStripper() override {} + + string name() const override { return "debug_stripper"; }; + + Status Optimize(Cluster* cluster, const GrapplerItem& item, + GraphDef* output) override; + + void Feedback(Cluster* cluster, const GrapplerItem& item, + const GraphDef& optimize_output, double result) override; +}; + +} // end namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DEBUG_STRIPPER_H_ diff --git a/linux/include/tensorflow/core/grappler/optimizers/dependency_optimizer.h b/linux/include/tensorflow/core/grappler/optimizers/dependency_optimizer.h new file mode 100644 index 000000000..48cfa236a --- /dev/null +++ b/linux/include/tensorflow/core/grappler/optimizers/dependency_optimizer.h @@ -0,0 +1,86 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DEPENDENCY_OPTIMIZER_H_ +#define TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DEPENDENCY_OPTIMIZER_H_ + +#include +#include "tensorflow/core/grappler/optimizers/graph_optimizer.h" +#include "tensorflow/core/grappler/utils.h" +#include "tensorflow/core/protobuf/rewriter_config.pb.h" + +namespace tensorflow { +namespace grappler { + +// Optimize TF computations by removing control dependencies or re-arranging +// them to shorten the critical path for a model step or enable other +// optimizations, such as removing nodes that are effectively noops. +class DependencyOptimizer : public GraphOptimizer { + public: + DependencyOptimizer() {} + explicit DependencyOptimizer(RewriterConfig::Toggle opt_level) + : opt_level_(opt_level) {} + ~DependencyOptimizer() override {} + + string name() const override { return "dependency_optimizer"; }; + + Status Optimize(Cluster* cluster, const GrapplerItem& item, + GraphDef* optimized_graph) override; + + void Feedback(Cluster* cluster, const GrapplerItem& item, + const GraphDef& optimized_graph, double result) override; + + private: + // Returns true if bypassing node does not increase the number of edges or + // number of edges crossing a device boundary. + bool BypassingNodeIsBeneficial( + const NodeDef& node, const std::vector& input_nodes, + const std::vector& output_nodes) const; + + // Returns true if node is not an Identity node or if it is an Identity + // that is safe to remove. + bool SafeToRemoveIdentity(const NodeDef& node) const; + // Returns true if it is safe to convert node to NoOp. + bool SafeToConvertToNoOp(const NodeDef& node) const; + // Removes all duplicate control dependencies. + void CleanControlInputs(); + // Builds a map from the &optimized_graph_->node(i) to i. + void BuildNodeToIdx(); + // Tries to optimize the node with the given index, possibly additional + // optimizations by inserting nodes in nodes_to_simplify, and pruning nodes by + // inserting them in nodes_to_delete. + void OptimizeNode(int node_idx, SetVector* nodes_to_simplify, + std::set* nodes_to_delete); + // Eliminates redundant control dependencies by computing the transitive + // reduction of the graph. + Status TransitiveReduction(); + // Main driver of dependency optimizations. + Status OptimizeDependencies(); + // Replaces multiple cross-device control edges from the same device with a + // single control edge. + void GroupCrossDeviceControlEdges(); + + RewriterConfig::Toggle opt_level_; + bool fetch_nodes_known_; + std::unordered_set nodes_to_preserve_; + std::unique_ptr node_map_; + std::unordered_map node_to_idx_; + GraphDef* optimized_graph_; // Not owned. +}; + +} // end namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_DEPENDENCY_OPTIMIZER_H_ diff --git a/linux/include/tensorflow/core/grappler/optimizers/evaluation_utils.h b/linux/include/tensorflow/core/grappler/optimizers/evaluation_utils.h new file mode 100644 index 000000000..c9dfb6dc0 --- /dev/null +++ b/linux/include/tensorflow/core/grappler/optimizers/evaluation_utils.h @@ -0,0 +1,62 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_EVALUATION_UTILS_H_ +#define TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_EVALUATION_UTILS_H_ + +#define EIGEN_USE_THREADS + +#include "tensorflow/core/framework/device_base.h" +#include "tensorflow/core/framework/node_def.pb.h" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/tensor.pb.h" +#include "tensorflow/core/lib/gtl/inlined_vector.h" + +namespace Eigen { +class ThreadPoolInterface; +class ThreadPoolWrapper; +} // namespace Eigen + +namespace tensorflow { +namespace grappler { + +class DeviceSimple : public DeviceBase { + public: + DeviceSimple(); + ~DeviceSimple(); + + Status MakeTensorFromProto(const TensorProto& tensor_proto, + const AllocatorAttributes alloc_attrs, + Tensor* tensor) override; + + Allocator* GetAllocator(AllocatorAttributes attr) override { + return cpu_allocator(); + } + + private: + DeviceBase::CpuWorkerThreads eigen_worker_threads_; + std::unique_ptr eigen_threadpool_wrapper_; + std::unique_ptr eigen_device_; +}; + +Status EvaluateNode(const NodeDef& node, + const gtl::InlinedVector& inputs, + DeviceBase* cpu_device, ResourceMgr* resource_mgr, + gtl::InlinedVector* output); + +} // end namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_EVALUATION_UTILS_H_ diff --git a/linux/include/tensorflow/core/grappler/optimizers/experimental_implementation_selector.h b/linux/include/tensorflow/core/grappler/optimizers/experimental_implementation_selector.h new file mode 100644 index 000000000..82f7473a1 --- /dev/null +++ b/linux/include/tensorflow/core/grappler/optimizers/experimental_implementation_selector.h @@ -0,0 +1,115 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_EXPERIMENTAL_IMPLEMENTATION_SELECTOR_H_ +#define TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_EXPERIMENTAL_IMPLEMENTATION_SELECTOR_H_ + +#include + +#include "tensorflow/core/framework/op.h" +#include "tensorflow/core/grappler/costs/graph_properties.h" +#include "tensorflow/core/grappler/grappler_item.h" +#include "tensorflow/core/grappler/op_types.h" +#include "tensorflow/core/grappler/optimizers/custom_graph_optimizer.h" +#include "tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.h" +#include "tensorflow/core/grappler/optimizers/function_api_info.h" +#include "tensorflow/core/lib/core/errors.h" +#include "tensorflow/core/lib/core/stringpiece.h" +#include "tensorflow/core/lib/strings/strcat.h" +#include "tensorflow/core/util/device_name_utils.h" + +namespace tensorflow { +namespace grappler { + +// -- EXPERIMENTAL -- +// This transformation replaces function calls by the appropriate function +// definition based on properties of the runtime system. For instance, +// we may choose one implementation over another if we have a GPU with +// enough memory available. +// +// It is a way for the programmer to specify alternative implementations +// of the same functionality in the graph, and let TensorFlow pick the +// most appropriate one at runtime. +// +// For instance, the python code might specify: +// @Defun(tf.float32, +// experimental_api_implements='plus_one', +// experimental_api_preferred_device='GPU') +// def plus_one_gpu(x): return x + 1.0 +// +// @Defun(tf.float32, +// experimental_api_implements='plus_one') +// def plus_one_reference_implementation(x): return x + 1.0 +// input = tf.constant(2.0, dtype=tf.float32) +// +// z = plus_one_reference_implementation(input) +// z = plus_one_gpu(input) +// print(sess.run(z)) +// +// At runtime, we will trim either `plus_one_gpu` or +// `plus_one_reference_implementation` based on the availability of the GPU. +// +// Available annotations: +// - experimental_api_implements(string): all functions mapping to the same +// string can be interchanged. For now, all functions must have the same +// signature and overloads are not allowed. Defuns within defuns are +// allowed. +// - experimental_api_preferred_device(string): sets which device is preferred. +class ExperimentalImplementationSelector : public CustomGraphOptimizer { + public: + ExperimentalImplementationSelector() = default; + ~ExperimentalImplementationSelector() override = default; + Status Init( + const tensorflow::RewriterConfig_CustomGraphOptimizer* config) override { + return Status::OK(); + } + string name() const override { + return "experimental_implementation_selector"; + } + + // This call is not thread-safe. + Status Optimize(Cluster* cluster, const GrapplerItem& item, + GraphDef* optimized_graph) override; + + // Does not take any feedback. + void Feedback(Cluster* cluster, const GrapplerItem& item, + const GraphDef& optimized_graph, double result) override {} + + private: + Status LoadFunctions(const GraphDef& graph); + Status MaybeOptimizeFunctionCall(NodeDef* node_def) const; + + // Finds all call sites for functions, then replace with the appropriate + // implementation. + // There are two ways of calling functions: + // 1. By specifying an op name as a function name, and + // 2. Via the functional interface, where the function name appears as an + // Attr. + // + // There may be multiple call sites for a given function. The function body + // may call into another function, so a function might have to be duplicated. + // For simplicity, we do not change function bodies. Also, we do not change + // gradients. + Status SelectImplementation(GraphDef* graph) const; + + std::unique_ptr lib_info_; + + TF_DISALLOW_COPY_AND_ASSIGN(ExperimentalImplementationSelector); +}; + +} // namespace grappler +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_EXPERIMENTAL_IMPLEMENTATION_SELECTOR_H_ diff --git a/linux/include/tensorflow/core/grappler/optimizers/function_api_info.h b/linux/include/tensorflow/core/grappler/optimizers/function_api_info.h new file mode 100644 index 000000000..412687c58 --- /dev/null +++ b/linux/include/tensorflow/core/grappler/optimizers/function_api_info.h @@ -0,0 +1,80 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_FUNCTION_API_INFO_H_ +#define TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_FUNCTION_API_INFO_H_ + +#include +#include +#include +#include + +#include "tensorflow/core/framework/function.pb.h" +#include "tensorflow/core/lib/core/status.h" + +namespace tensorflow { +namespace grappler { +class FunctionApiInfo { + public: + FunctionApiInfo(); + virtual ~FunctionApiInfo(); + + Status Init(const FunctionDef& function_def); + + const string& interface_name() const; + const string& preferred_device() const; + + private: + string interface_name_; + string preferred_device_; + + TF_DISALLOW_COPY_AND_ASSIGN(FunctionApiInfo); +}; + +// A collection of information for function and the interface it implements. +// A interface is a well defined math operation, eg I1 = 2 * x + y. Multiple +// functions could implement the same interface with different behavior based on +// different hardware condition and limits, +// eg F1 = math_ops.add(math_ops.add(x, x), y), or +// F2 = math_ops.add(math_ops.matmul(x, 2), y). +class FunctionLibraryApiInfo { + public: + FunctionLibraryApiInfo(); + virtual ~FunctionLibraryApiInfo(); + // Populate the internal field for the functions within the function_library. + Status Init(const FunctionDefLibrary& function_library); + + void GetEquivalentImplementations(const string& function_name, + std::vector* other_names) const; + + void GetBestImplementation(const string& function_name, const string& device, + string* best_func_name) const; + + const FunctionApiInfo* GetApiInfo(const string& function_name) const; + + private: + // Map between function name to function details. + std::unordered_map> func_info_; + // Map between function name to interface name. + std::unordered_map func_to_intf_; + // Map between interface name to function names. + std::unordered_map> intf_to_funcs_; + + TF_DISALLOW_COPY_AND_ASSIGN(FunctionLibraryApiInfo); +}; + +} // end namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_FUNCTION_API_INFO_H_ diff --git a/linux/include/tensorflow/core/grappler/optimizers/function_optimizer.h b/linux/include/tensorflow/core/grappler/optimizers/function_optimizer.h new file mode 100644 index 000000000..435255506 --- /dev/null +++ b/linux/include/tensorflow/core/grappler/optimizers/function_optimizer.h @@ -0,0 +1,58 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_FUNCTION_OPTIMIZER_H_ +#define TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_FUNCTION_OPTIMIZER_H_ + +#include "tensorflow/core/grappler/optimizers/graph_optimizer.h" +#include "tensorflow/core/protobuf/rewriter_config.pb.h" + +namespace tensorflow { +namespace grappler { + +// Remap TensorFlow subgraphs onto alternative operations or collection of +// operations to make the overall graph more efficient. +class FunctionOptimizer : public GraphOptimizer { + public: + explicit FunctionOptimizer(RewriterConfig::Toggle opt_level) + : opt_level_(opt_level) {} + ~FunctionOptimizer() override = default; + + string name() const override { return "function_optimizer"; }; + + Status Optimize(Cluster* cluster, const GrapplerItem& item, + GraphDef* optimized_graph) override; + + void Feedback(Cluster* cluster, const GrapplerItem& item, + const GraphDef& optimized_graph, double result) override; + + private: + friend class FunctionOptimizerTest; + + struct FunctionOptimizerOptions { + bool enable_function_inlining = true; + bool enable_function_specialization = true; + bool enable_symbolic_gradient_inlining = true; + bool enable_trim_function_library = true; + }; + + RewriterConfig::Toggle opt_level_; + FunctionOptimizerOptions options_; +}; + +} // end namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_FUNCTION_OPTIMIZER_H_ diff --git a/linux/include/tensorflow/core/grappler/optimizers/graph_optimizer.h b/linux/include/tensorflow/core/grappler/optimizers/graph_optimizer.h new file mode 100644 index 000000000..765dd1326 --- /dev/null +++ b/linux/include/tensorflow/core/grappler/optimizers/graph_optimizer.h @@ -0,0 +1,53 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_GRAPH_OPTIMIZER_H_ +#define TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_GRAPH_OPTIMIZER_H_ + +#include "tensorflow/core/framework/graph.pb.h" +#include "tensorflow/core/lib/core/status.h" + +namespace tensorflow { +namespace grappler { + +class Cluster; +struct GrapplerItem; + +// An abstract interface for an algorithm for generating a candidate +// optimization of a GrapplerItem for running on a cluster. +class GraphOptimizer { + public: + virtual ~GraphOptimizer() {} + + virtual string name() const = 0; + + // Routine called to allow an algorithm to propose a rewritten graph + // for the graph, feeds and fetches in "item" to run more efficiently + // on "cluster". + // Returns true iff it managed to generate a solution, false otherwise. + virtual Status Optimize(Cluster* cluster, const GrapplerItem& item, + GraphDef* optimized_graph) = 0; + + // Method invoked by the framework so that it can provide feedback + // on how well the "optimized_graph" (produced as *optimized_graph from a + // call to Optimize) performed. Lower "result" scores are better. + virtual void Feedback(Cluster* cluster, const GrapplerItem& item, + const GraphDef& optimized_graph, double result) = 0; +}; + +} // end namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_GRAPH_OPTIMIZER_H_ diff --git a/linux/include/tensorflow/core/grappler/optimizers/graph_optimizer_stage.h b/linux/include/tensorflow/core/grappler/optimizers/graph_optimizer_stage.h new file mode 100644 index 000000000..2afb5df43 --- /dev/null +++ b/linux/include/tensorflow/core/grappler/optimizers/graph_optimizer_stage.h @@ -0,0 +1,286 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_GRAPH_OPTIMIZER_STAGE_H_ +#define TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_GRAPH_OPTIMIZER_STAGE_H_ + +#include +#include +#include "tensorflow/core/grappler/costs/graph_properties.h" +#include "tensorflow/core/grappler/grappler_item.h" +#include "tensorflow/core/grappler/utils.h" +#include "tensorflow/core/protobuf/rewriter_config.pb.h" + +namespace tensorflow { +namespace grappler { + +struct NodeScopeAndName { + string scope; + string name; +}; + +// Parse scope and name: "a/b/c/Add_1" -> {"a/b/c", "Add_1"} +const NodeScopeAndName ParseNodeScopeAndName(const string& node_name); + +// Context owned by GraphOptimizer, and passed to every stage at construction +// time. Each optimizer stage is responsible for updating it according to the +// changes it made to the graph. +// +// If an optimizer needs access to some helper class that is not present in this +// context, consider creating an extension context, specific to that +// optimizer (see example of ArithmeticOptimizerContext). GraphOptimizerContext +// should only have members that are useful to almost all optimizers. +struct GraphOptimizerContext { + GraphOptimizerContext(const std::unordered_set* nodes_to_preserve, + GraphDef* optimized_graph, + GraphProperties* graph_properties, NodeMap* node_map, + RewriterConfig::Toggle opt_level) + : nodes_to_preserve(nodes_to_preserve), + optimized_graph(optimized_graph), + graph_properties(graph_properties), + node_map(node_map), + opt_level(opt_level) {} + + const std::unordered_set* nodes_to_preserve; + GraphDef* optimized_graph; + GraphProperties* graph_properties; + NodeMap* node_map; + RewriterConfig::Toggle opt_level; +}; + +Status GetInputNode(const GraphOptimizerContext& ctx, const string& input, + NodeDef** node); +Status GetTensorProperties(const GraphOptimizerContext& ctx, + const string& tensor, + OpInfo::TensorProperties* properties); + +NodeDef* AddCopyNode(const GraphOptimizerContext& ctx, const string& name, + const NodeDef* node_to_copy); +NodeDef* AddEmptyNode(const GraphOptimizerContext& ctx, const string& name); + +// WARNING: +// Optimizer stage must try to re-use original nodes of a graph and +// make all updates in place. This helps to make robust node placement +// decisions. Create new nodes only if there is a reason for that. + +// Make a name for a new node obtained by optimizing a single node of the +// original graph. The optimized node is placed under the original node scope. +// +// Node name uniqueness is guaranteed by unique name of an original node in +// a same scope. +// +// Empty sub_scope or prefix ignored. At least one of them must be non-empty. +// +// Example: a/b/c/Add -> a/b/c/${sub_scope}/${prefix}_Add. +const string MakeOptimizedNodeName(const NodeScopeAndName& node, + const string& sub_scope, + const string& prefix); +// Make a name for a new node obtained by optimizing multiple nodes of the +// original graph, starting from "root". The optimized node is placed under +// the original scope of a "root" node. +// +// Example: [a/b/c/Add, x/y/z/Mul] -> a/b/c/${sub_scope}/${prefix}_Add_Mul +const string MakeOptimizedNodeName(const NodeScopeAndName& root, + const std::vector node_names, + const string& sub_scope, + const string& prefix); + +// Base class for multi-stage GraphOptimizers (ArithmeticOptimizer, etc...). +// +// If a graph optimizer consists of large number of small independent +// rewrites, each of them should be implemented as a separate stage. +// +// * Result: +// Each graph optimizer choose what result is reported by each stage +// (e.g. each stage can fill in the name of optimized nodes, or have more +// complex result). +template +class GraphOptimizerStage { + public: + explicit GraphOptimizerStage(const string& optimizer_name, + const string& stage_name, + const GraphOptimizerContext& ctx) + : optimizer_name_(optimizer_name), stage_name_(stage_name), ctx_(ctx) {} + virtual ~GraphOptimizerStage() = default; + + const string& stage_name() const { return stage_name_; } + const string& optimizer_name() const { return optimizer_name_; } + + // Check if we should try to simplify node. Returning true doesn't + // guarantee that node will be simplified. + // + // Should implement just a basic sanity check, without any expensive graph + // traversals. + virtual bool IsSupported(const NodeDef* node) const = 0; + + // Try to simplify the given node. + // + // Return error status only if some precondition is failed, or got an + // incorrect graph. In every other case return Status:OK(), even if didn't + // simplify anything. + // + // Report result using output argument. Each GraphOptimizer can choose it's + // own Result type. + // TODO(ezhulenev): if it will appear that Result output parameter is not + // sufficiently useful (used with a reason by most optimizers), get rid of it, + // and remove template parameter. + virtual Status TrySimplify(NodeDef* node, Result* result) = 0; + + // Return InvalidArgumentError if node is not supported by the optimizer + // stage. + // TODO(ezhulenev): make this check part of non-virtual public API + // (TrySimplify), and make virtual implementation protected. + Status EnsureNodeIsSupported(const NodeDef* node) const { + return IsSupported(node) + ? Status::OK() + : errors::InvalidArgument( + "Node ", node->name(), " is not supported by optimizer ", + optimizer_name_, " and stage ", stage_name_); + } + + // Get a name for a new node, created by this stage, based on one or multiple + // nodes of an original graph. + const string OptimizedNodeName(const NodeScopeAndName& node) const { + return MakeOptimizedNodeName(node, optimizer_name_, stage_name_); + } + const string OptimizedNodeName(const NodeScopeAndName& root, + const std::vector& nodes) const { + return MakeOptimizedNodeName(root, nodes, optimizer_name_, stage_name_); + } + const string OptimizedNodeName(const NodeScopeAndName& node, + const string& rewrite_rule) const { + const string prefix = strings::StrCat(stage_name_, "_", rewrite_rule); + return MakeOptimizedNodeName(node, optimizer_name_, prefix); + } + + // Get a node by input name from a node map. Return an error if node was not + // found. + Status GetInputNode(const string& input, NodeDef** node) const { + return ::tensorflow::grappler::GetInputNode(ctx_, input, node); + } + // Lookup tensor properties by name. Tensor name might have non-zero port + // number. Return an error if tensor node doesn't exists in a graph, or it + // doesn't have properties defined for requested port. + Status GetTensorProperties(const string& tensor, + OpInfo::TensorProperties* properties) const { + return ::tensorflow::grappler::GetTensorProperties(ctx_, tensor, + properties); + } + + NodeDef* AddCopyNode(const string& name, const NodeDef* node_to_copy) { + return ::tensorflow::grappler::AddCopyNode(ctx_, name, node_to_copy); + } + NodeDef* AddEmptyNode(const string& name) { + return ::tensorflow::grappler::AddEmptyNode(ctx_, name); + } + + protected: + const GraphOptimizerContext& ctx() const { return ctx_; } + + private: // Data members + const string optimizer_name_; + const string stage_name_; + const GraphOptimizerContext ctx_; +}; + +template +class GraphOptimizerStagePipeline { + public: + // Break predicate specifies if a pipeline should stop early, and not pass + // a node to the next registered optimizer stage, typically that should be the + // case when a stage successfully optimized a node, and it wants to yield + // control to the optimizer. + explicit GraphOptimizerStagePipeline( + const std::function break_predicate) + : break_predicate_(break_predicate) {} + + // Add a stage to the pipeline. It should be called with the arguments for the + // stage constructor: + // + // pipeline.AddStage(constructor_arg1, constructor_arg2); + // + // Returns a reference to the added stage. + template + T& AddStage(Args&&... args) { + auto stage = new T(std::forward(args)...); + stages_.push_back(std::unique_ptr(stage)); + return *stage; + } + + // Pass a node through all registered optimizer stages, until break predicate + // is true. + // + // Return true, if pipeline exited after a break predicate was evaluated as + // 'true', which typically means that a node was optimized by one of the + // registered stages. + // + // Return false, if node was not optimized by any of registered stages. + bool PassThroughAllStages(NodeDef* node, Result* result) { + for (auto& stage : stages_) { + if (stage->IsSupported(node)) { + const Status stage_status = stage->TrySimplify(node, result); + // Each stage must be "error safe" (just like exception safe). In + // case of any error it must leave optimized graph unmodified. + if (!stage_status.ok()) { + LOG(WARNING) << "Failed to run optimizer " << stage->optimizer_name() + << ", stage " << stage->stage_name() + << ". Error: " << stage_status.error_message(); + } + if (break_predicate_(*result)) return true; + } + } + return false; + } + + // Pass a node through all registered optimizer stages, until break predicate + // is true or a stage fails. + // + // Returns any stage failure status, or else Status::OK(). + Status PassThroughAllStagesWithStatus(NodeDef* node, Result* result) { + for (auto& stage : stages_) { + if (!stage->IsSupported(node)) { + continue; + } + const Status stage_status = stage->TrySimplify(node, result); + if (!stage_status.ok()) { + return stage_status; + } else if (break_predicate_(*result)) { + break; + } + } + return Status::OK(); + } + + std::size_t NumStages() { return stages_.size(); } + + std::vector StageNames() { + std::vector names; + for (const auto& stage : stages_) { + names.push_back(stage->stage_name()); + } + return names; + } + + private: + std::vector>> stages_; + std::function break_predicate_; + + TF_DISALLOW_COPY_AND_ASSIGN(GraphOptimizerStagePipeline); +}; + +} // end namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_GRAPH_OPTIMIZER_STAGE_H_ diff --git a/linux/include/tensorflow/core/grappler/optimizers/graph_rewriter.h b/linux/include/tensorflow/core/grappler/optimizers/graph_rewriter.h new file mode 100644 index 000000000..4a5a150dc --- /dev/null +++ b/linux/include/tensorflow/core/grappler/optimizers/graph_rewriter.h @@ -0,0 +1,102 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_GRAPH_REWRITER_H_ +#define TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_GRAPH_REWRITER_H_ + +#include +#include +#include "tensorflow/core/grappler/grappler_item.h" + +namespace tensorflow { +namespace grappler { + +// Tools and utilities to simplify common graph rewrites. +class GraphRewriter { + public: + GraphRewriter(const GrapplerItem& item); + + // Forward the inputs of original_node as needed to skip over the nodes that + // are to be deleted. In other words, if I is an input of 'original_node', and + // I doesn't belong to one of the nodes in 'nodes_to_delete', I will be an + // input to 'new_node'. On the other hand, if I belong to a node that will be + // deleted, I will be replaced with the inputs J of the deleted node (unless J + // belong to nodes that will be deleted, in which case we'll look for + // preserved inputs further down the graph). + void ForwardInputs(const NodeDef& original_node, + const std::unordered_set& nodes_to_delete, + NodeDef* new_node); + + // Returns true if at least one of the edges in the direct fanout of 'node' is + // a control dependency edge. + bool DrivesControlDependency(const NodeDef& node) const; + + // Returns true if at least one of the incident edges is a control dependency + // edge. + bool IsDrivenByControlDependency(const NodeDef& node) const; + + // Returns true if at least one of the nodes in the direct fanin or the direct + // fanout (excluding control dependencies) of 'node' is a function. + bool IsConnectedToFunction(const NodeDef& node) const; + + // Returns true if the node is driven by at least one node placed on another + // device. + bool IsDrivenByAnotherDevice(const NodeDef& node) const; + + // Returns true if the node has input from a stateful op. + bool ReceivesRefValue(const NodeDef& node) const; + + // Returns true if the node is driven by a Switch node. + bool IsDrivenBySwitch(const NodeDef& node) const; + + // Returns true if the node feeds a Merge node. + bool FeedsMerge(const NodeDef& node) const; + + // Returns true if removal of this degree would increase edge count, i.e. if + // in-degree * out-degree > in-degree + out-degree or if the condition could + // not be verified. + bool RemovalIncreasesEdgeCount(const NodeDef& node) const; + + private: + void RecordConnectivity(const NodeDef& node, + const std::unordered_set& function_names); + void ForwardInputsInternal( + const NodeDef& original_node, + const std::unordered_set& nodes_to_delete, + bool add_as_control, NodeDef* new_node); + + struct NodeInfo { + int out_degree = 0; + const NodeDef* def; + + // These are filled in when the NodeInfo is built, but not that they + // may be empty - if the op could not be loaded from the registry. + DataTypeVector outputs; + }; + + std::unordered_map> nodes_; + std::unordered_map optimized_nodes_; + std::unordered_set control_dependency_drivers_; + std::unordered_set function_neighbors_; + std::unordered_set cross_device_receivers_; + std::unordered_set ref_receivers_; + std::unordered_set switch_receivers_; + std::unordered_set merge_feeders_; +}; + +} // end namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_GRAPH_REWRITER_H_ diff --git a/linux/include/tensorflow/core/grappler/optimizers/layout_optimizer.h b/linux/include/tensorflow/core/grappler/optimizers/layout_optimizer.h new file mode 100644 index 000000000..49b697bb7 --- /dev/null +++ b/linux/include/tensorflow/core/grappler/optimizers/layout_optimizer.h @@ -0,0 +1,60 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_LAYOUT_OPTIMIZER_H_ +#define TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_LAYOUT_OPTIMIZER_H_ + +#include "tensorflow/core/grappler/costs/graph_properties.h" +#include "tensorflow/core/grappler/costs/virtual_placer.h" +#include "tensorflow/core/grappler/optimizers/graph_optimizer.h" + +namespace tensorflow { +namespace grappler { +// Convert the NHWC layout to NCHW for Conv-related ops on GPUs. +class LayoutOptimizer : public GraphOptimizer { + public: + LayoutOptimizer() {} + ~LayoutOptimizer() override {} + + string name() const override { return "layout"; }; + + struct TuningConfig { + // If true, do not use the NHWC GEMM implementation. When filter size is + // one or filter size is equal to input image size, + // the NHWC implementation of Conv2D, Conv2DBackpropInput, and + // Conv2DBackpropFilter will use a specialized GEMM implementation, which is + // usually faster than the NCHW implementation. The downside is that this + // might result in more non-cancellable layout conversion nodes (implemented + // by the Transpose op). + bool no_gemm; + }; + + Status Optimize(Cluster* cluster, const GrapplerItem& item, + GraphDef* output) override; + + void Feedback(Cluster* cluster, const GrapplerItem& item, + const GraphDef& optimize_output, double result) override; + + private: + std::unique_ptr virtual_placer_; + std::unordered_set nodes_to_preserve_; + Status Tune(const GrapplerItem& item, const GraphProperties& graph_properties, + const TuningConfig& config, GraphDef* output); +}; + +} // end namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_LAYOUT_OPTIMIZER_H_ diff --git a/linux/include/tensorflow/core/grappler/optimizers/loop_optimizer.h b/linux/include/tensorflow/core/grappler/optimizers/loop_optimizer.h new file mode 100644 index 000000000..7c04f5538 --- /dev/null +++ b/linux/include/tensorflow/core/grappler/optimizers/loop_optimizer.h @@ -0,0 +1,75 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_LOOP_OPTIMIZER_H_ +#define TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_LOOP_OPTIMIZER_H_ + +#include +#include "tensorflow/core/grappler/costs/graph_properties.h" +#include "tensorflow/core/grappler/optimizers/graph_optimizer.h" +#include "tensorflow/core/grappler/utils.h" +#include "tensorflow/core/grappler/utils/frame.h" +#include "tensorflow/core/protobuf/rewriter_config.pb.h" + +namespace tensorflow { +namespace grappler { + +constexpr char kLoopOptimizer[] = "LoopOptimizer"; + +class LoopOptimizer : public GraphOptimizer { + public: + LoopOptimizer(); + + explicit LoopOptimizer(RewriterConfig::Toggle opt_level, + DeviceBase* cpu_device); + + ~LoopOptimizer() override {} + + string name() const override { return "loop_optimizer"; }; + + Status Optimize(Cluster* cluster, const GrapplerItem& item, + GraphDef* optimized_graph) override; + + void Feedback(Cluster* cluster, const GrapplerItem& item, + const GraphDef& optimized_graph, double result) override; + + private: + friend class LoopOptimizerTest; + + // Granular control for loop optimizer stages. + struct LoopOptimizerOptions { + bool enable_loop_invariant_node_motion = false; + bool enable_stack_push_removal = true; + bool enable_dead_branch_removal = true; + + static LoopOptimizerOptions Default(RewriterConfig::Toggle opt_level) { + LoopOptimizerOptions options; + return options; + } + }; + + Status RemoveDeadBranches(const std::unordered_set& nodes_to_preserve, + const NodeMap& node_map, GraphDef* optimized_graph); + + RewriterConfig::Toggle opt_level_; + DeviceBase* cpu_device_; + LoopOptimizerOptions options_; + std::unique_ptr resource_mgr_; +}; + +} // end namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_LOOP_OPTIMIZER_H_ diff --git a/linux/include/tensorflow/core/grappler/optimizers/memory_optimizer.h b/linux/include/tensorflow/core/grappler/optimizers/memory_optimizer.h new file mode 100644 index 000000000..653ffaec4 --- /dev/null +++ b/linux/include/tensorflow/core/grappler/optimizers/memory_optimizer.h @@ -0,0 +1,56 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_MEMORY_OPTIMIZER_H_ +#define TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_MEMORY_OPTIMIZER_H_ + +#include "tensorflow/core/grappler/optimizers/graph_optimizer.h" +#include "tensorflow/core/protobuf/rewriter_config.pb.h" + +namespace tensorflow { +namespace grappler { + +// Swap tensors in and out of device memory. +class MemoryOptimizer : public GraphOptimizer { + public: + // optimization_level: Controls the level of autonomy for the memory + // optimizer. See RewriterConfig::memory_optimization. + // recomputation_targets_name_scope: Name scope for potential outputs of + // recomputations. See + // RewriterConfig::memory_optimizer_target_node_name_scope. + explicit MemoryOptimizer( + RewriterConfig::MemOptType optimization_level, + const string& recomputation_targets_name_scope = "gradients/") + : optimization_level_(optimization_level), + recomputation_targets_name_scope_(recomputation_targets_name_scope) {} + ~MemoryOptimizer() override {} + + string name() const override { return "memory_optimizer"; }; + + Status Optimize(Cluster* cluster, const GrapplerItem& item, + GraphDef* pruned_graph) override; + + void Feedback(Cluster* cluster, const GrapplerItem& item, + const GraphDef& pruned_graph, double result) override; + + private: + RewriterConfig::MemOptType optimization_level_; + string recomputation_targets_name_scope_; +}; + +} // end namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_MEMORY_OPTIMIZER_H_ diff --git a/linux/include/tensorflow/core/grappler/optimizers/meta_optimizer.h b/linux/include/tensorflow/core/grappler/optimizers/meta_optimizer.h new file mode 100644 index 000000000..99a0a33ff --- /dev/null +++ b/linux/include/tensorflow/core/grappler/optimizers/meta_optimizer.h @@ -0,0 +1,104 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_META_OPTIMIZER_H_ +#define TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_META_OPTIMIZER_H_ + +#include "tensorflow/core/framework/device_base.h" +#include "tensorflow/core/grappler/grappler_item.h" +#include "tensorflow/core/grappler/optimizers/graph_optimizer.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/protobuf/rewriter_config.pb.h" + +namespace tensorflow { +namespace grappler { + +// Run the other grappler optimizers based on the specified rewriter config. +class MetaOptimizer : public GraphOptimizer { + public: + MetaOptimizer(DeviceBase* cpu_device, const RewriterConfig& cfg) + : cpu_device_(cpu_device), cfg_(cfg) {} + ~MetaOptimizer() override = default; + + string name() const override { return "meta_optimizer"; }; + + Status Optimize(Cluster* cluster, const GrapplerItem& item, + GraphDef* optimized_graph) override; + + void PrintResult(); + + void Feedback(Cluster* cluster, const GrapplerItem& item, + const GraphDef& optimized_graph, double result) override; + + private: + std::unique_ptr MakeNewOptimizer( + const string& optimizer) const; + + // Initialize active optimizers from RewriterConfig toggles. + Status InitializeOptimizers( + std::vector>* optimizers) const; + // Initialize active optimizers from RewriterConfig optimizer names. + Status InitializeOptimizersByName( + std::vector>* optimizers) const; + // Initialize active optimizers from RewriterConfig.custom_optimizers. + Status InitializeCustomGraphOptimizers( + const std::set& pre_initialized_optimizers, + std::vector>* optimizers) const; + // Returns the config for a custom graph optimizer. Null if none was found. + const RewriterConfig::CustomGraphOptimizer* GetCustomGraphOptimizerConfig( + const string& name) const; + + // Run optimization pass over a single GrapplerItem. Meta optimizer might run + // multiple such passes: 1) for the main graph 2) for the function library + Status OptimizeGraph(Cluster* cluster, const GrapplerItem& item, + GraphDef* optimized_graph); + + DeviceBase* const cpu_device_; // may be NULL + RewriterConfig cfg_; + + struct OptimizerResult { + string optimizer_name; + string result; + }; + + struct GraphOptimizationResult { + explicit GraphOptimizationResult(const string& id) : id(id) {} + string id; + std::vector results; + }; + + Status RunOptimizer(GraphOptimizer* optimizer, Cluster* cluster, + GrapplerItem* optimized_item, GraphDef* optimized_graph, + GraphOptimizationResult* optimization_result); + + std::vector optimization_results_; +}; + +bool MetaOptimizerEnabled(const RewriterConfig& cfg); + +// Run the meta optimizer. +// +// If is non-null, it is the device to be used for executing ops +// during constant folding; if NULL, a new device is created for doing constant +// folding. For performance, it is recommended to pass in an existing cpu_device +// when possible. +Status RunMetaOptimizer(const GrapplerItem& item, const RewriterConfig& cfg, + DeviceBase* cpu_device, Cluster* cluster, + GraphDef* optimized_graph); + +} // namespace grappler +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_META_OPTIMIZER_H_ diff --git a/linux/include/tensorflow/core/grappler/optimizers/model_pruner.h b/linux/include/tensorflow/core/grappler/optimizers/model_pruner.h new file mode 100644 index 000000000..76cc792a4 --- /dev/null +++ b/linux/include/tensorflow/core/grappler/optimizers/model_pruner.h @@ -0,0 +1,44 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_MODEL_PRUNER_H_ +#define TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_MODEL_PRUNER_H_ + +#include "tensorflow/core/grappler/optimizers/graph_optimizer.h" + +namespace tensorflow { +namespace grappler { + +// Prune a model to make it more efficient: +// * Remove unnecessary operations. +// * Optimize gradient computations. +class ModelPruner : public GraphOptimizer { + public: + ModelPruner() {} + ~ModelPruner() override {} + + string name() const override { return "model_pruner"; }; + + Status Optimize(Cluster* cluster, const GrapplerItem& item, + GraphDef* pruned_graph) override; + + void Feedback(Cluster* cluster, const GrapplerItem& item, + const GraphDef& pruned_graph, double result) override; +}; + +} // end namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_MODEL_PRUNER_H_ diff --git a/linux/include/tensorflow/core/grappler/optimizers/pin_to_host_optimizer.h b/linux/include/tensorflow/core/grappler/optimizers/pin_to_host_optimizer.h new file mode 100644 index 000000000..d557a0346 --- /dev/null +++ b/linux/include/tensorflow/core/grappler/optimizers/pin_to_host_optimizer.h @@ -0,0 +1,62 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_PIN_TO_HOST_OPTIMIZER_H_ +#define TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_PIN_TO_HOST_OPTIMIZER_H_ + +#include +#include "tensorflow/core/grappler/costs/graph_properties.h" +#include "tensorflow/core/grappler/optimizers/graph_optimizer.h" +#include "tensorflow/core/lib/gtl/flatset.h" +#include "tensorflow/core/protobuf/rewriter_config.pb.h" + +namespace tensorflow { +namespace grappler { +namespace internal { +// Try and find an appropriate Host device in `devices` given `device`. +string TryFindHostDevice(const gtl::FlatSet& devices, + bool has_device_cpu, const string& device); +} // end namespace internal + +// Optimize TensorFlow ops that should be swapped into the CPU to avoid +// excessive cpu<->gpu memcpy/sync. +// +// TODO(williamchan): The current heuristic will swap any small integer Const to +// CPU. This may cause a problem cpu->cpu->gpu wherein the original behaviour of +// gpu->gpu->gpu may have been better/faster. We should probably fix this. +class PinToHostOptimizer : public GraphOptimizer { + public: + PinToHostOptimizer() : opt_level_(RewriterConfig::DEFAULT) {} + explicit PinToHostOptimizer(RewriterConfig::Toggle opt_level) + : opt_level_(opt_level) {} + + ~PinToHostOptimizer() override {} + + string name() const override { return "pin_to_host_optimizer"; }; + + Status Optimize(Cluster* cluster, const GrapplerItem& item, + GraphDef* optimized_graph) override; + + void Feedback(Cluster* cluster, const GrapplerItem& item, + const GraphDef& optimized_graph, double result) override {} + + private: + RewriterConfig::Toggle opt_level_; +}; + +} // end namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_PIN_TO_HOST_OPTIMIZER_H_ diff --git a/linux/include/tensorflow/core/grappler/optimizers/remapper.h b/linux/include/tensorflow/core/grappler/optimizers/remapper.h new file mode 100644 index 000000000..c18413e4e --- /dev/null +++ b/linux/include/tensorflow/core/grappler/optimizers/remapper.h @@ -0,0 +1,48 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_REMAPPER_H_ +#define TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_REMAPPER_H_ + +#include "tensorflow/core/grappler/optimizers/graph_optimizer.h" +#include "tensorflow/core/protobuf/rewriter_config.pb.h" + +namespace tensorflow { +namespace grappler { + +// Optimize TF computations by remapping subgraphs/nodes onto other subgraphs or +// nodes to decrease the amount of operations needed to perform a computation. +class Remapper : public GraphOptimizer { + public: + explicit Remapper(RewriterConfig::Toggle opt_level) : opt_level_(opt_level) {} + + ~Remapper() override {} + + string name() const override { return "remapper"; }; + + Status Optimize(Cluster* cluster, const GrapplerItem& item, + GraphDef* optimized_graph) override; + + void Feedback(Cluster* cluster, const GrapplerItem& item, + const GraphDef& optimized_graph, double result) override; + + private: + RewriterConfig::Toggle opt_level_; +}; + +} // end namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_REMAPPER_H_ diff --git a/linux/include/tensorflow/core/grappler/optimizers/scoped_allocator_optimizer.h b/linux/include/tensorflow/core/grappler/optimizers/scoped_allocator_optimizer.h new file mode 100644 index 000000000..13589f536 --- /dev/null +++ b/linux/include/tensorflow/core/grappler/optimizers/scoped_allocator_optimizer.h @@ -0,0 +1,108 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_SCOPED_ALLOCATOR_OPTIMIZER_H_ +#define TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_SCOPED_ALLOCATOR_OPTIMIZER_H_ + +#include +#include +#include +#include "tensorflow/core/grappler/optimizers/graph_optimizer.h" +#include "tensorflow/core/protobuf/rewriter_config.pb.h" + +namespace tensorflow { +namespace grappler { +class Graph; +class GraphProperties; +class NodeMap; +class ScopedAllocatorOptimizer; + +// An Optimizer that introduces ScopedAllocators in order to reduce data +// movement and consolidate some kinds of Ops. +class ScopedAllocatorOptimizer : public GraphOptimizer { + public: + ScopedAllocatorOptimizer(RewriterConfig::Toggle opt_level, + const ScopedAllocatorOptions& opts); + ~ScopedAllocatorOptimizer() override; + + string name() const override { return "scoped_allocator_optimizer"; } + + Status Optimize(Cluster* cluster, const GrapplerItem& item, + GraphDef* optimized_graph) override; + + void Feedback(Cluster* cluster, const GrapplerItem& item, + const GraphDef& optimized_graph, double result) override {} + + // Map from an Op name to a vector of Nodes with that Op. + typedef std::unordered_map> DevOpOccurrences; + // Map from a device name to a DevOpOccurrences map. + typedef std::unordered_map GraphOpOccurrences; + typedef std::unordered_set OpNameSet; + + Status ProcessGraphDef(GraphDef* graph, + const GraphProperties& graph_properties); + + // Populates *occs by grouping Nodes with common Ops, according to + // their assigned devices. + void FindOpOccurrences(GraphDef* graph, const OpNameSet& op_names, + GraphOpOccurrences* occs); + + // Returns a new, unused scope_id to be assigned to a ScopedAllocator that + // will allocate num_fields (> 0) separate tensors. + int NewScopedAllocatorId(int num_fields); + + NodeMap* node_map() { return node_map_.get(); } + + // Appends values to the attr value under name in node_def, if present. + // If not present does an assignment. + static void ExtendNodeAttr(StringPiece name, const std::vector& values, + NodeDef* node_def); + + // Class that knows how to do graph rewriting for a particular kind of Op in + // order to take advantage of a ScopedAllocator. + class Rewriter { + public: + virtual ~Rewriter() {} + + virtual Status Rewrite(ScopedAllocatorOptimizer* paopti, GraphDef* graph, + const string& op_name, + const std::vector& nodes, + bool* applied) = 0; + + void SetGraphProperties(const GraphProperties& graph_properties) { + graph_properties_ = &graph_properties; + CHECK(graph_properties_); + } + + protected: + const GraphProperties* graph_properties_; + }; + + private: + Rewriter* GetRewriter(const string& op_name); + + Status OrderNodeSet(std::vector* nodes) const; + + RewriterConfig::Toggle opt_level_; + std::unordered_set nodes_to_preserve_; + OpNameSet op_name_set_; + std::unordered_map rewriters_; + std::vector to_delete_; + int next_sa_id_ = 1; + std::unique_ptr node_map_; +}; + +} // namespace grappler +} // namespace tensorflow +#endif // TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_SCOPED_ALLOCATOR_OPTIMIZER_H_ diff --git a/linux/include/tensorflow/core/grappler/optimizers/shape_optimizer.h b/linux/include/tensorflow/core/grappler/optimizers/shape_optimizer.h new file mode 100644 index 000000000..b7f84a1e5 --- /dev/null +++ b/linux/include/tensorflow/core/grappler/optimizers/shape_optimizer.h @@ -0,0 +1,54 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_SHAPE_OPTIMIZER_H_ +#define TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_SHAPE_OPTIMIZER_H_ + +#include +#include "tensorflow/core/grappler/costs/graph_properties.h" +#include "tensorflow/core/grappler/optimizers/graph_optimizer.h" +#include "tensorflow/core/grappler/utils.h" +#include "tensorflow/core/grappler/utils/frame.h" +#include "tensorflow/core/protobuf/rewriter_config.pb.h" + +namespace tensorflow { +namespace grappler { + +// Optimize TensorFlow subgraphs that operate on shape and shape related +// information. +class ShapeOptimizer : public GraphOptimizer { + public: + ShapeOptimizer() : opt_level_(RewriterConfig::ON) {} + explicit ShapeOptimizer(RewriterConfig::Toggle opt_level) + : opt_level_(opt_level) {} + + ~ShapeOptimizer() override {} + + string name() const override { return "shape_optimizer"; }; + + Status Optimize(Cluster* cluster, const GrapplerItem& item, + GraphDef* optimized_graph) override; + + void Feedback(Cluster* cluster, const GrapplerItem& item, + const GraphDef& optimized_graph, double result) override; + + private: + RewriterConfig::Toggle opt_level_; +}; + +} // end namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_SHAPE_OPTIMIZER_H_ diff --git a/linux/include/tensorflow/core/grappler/optimizers/static_schedule.h b/linux/include/tensorflow/core/grappler/optimizers/static_schedule.h new file mode 100644 index 000000000..678b4d193 --- /dev/null +++ b/linux/include/tensorflow/core/grappler/optimizers/static_schedule.h @@ -0,0 +1,50 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_STATIC_SCHEDULE_H_ +#define TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_STATIC_SCHEDULE_H_ + +#include + +#include "tensorflow/core/framework/node_def.pb.h" +#include "tensorflow/core/grappler/clusters/cluster.h" +#include "tensorflow/core/grappler/costs/cost_estimator.h" +#include "tensorflow/core/grappler/grappler_item.h" + +namespace tensorflow { +namespace grappler { + +// Compute the earliest time at which the execution of each node in the graph +// can complete. +// In our estimation, we ensure that each node takes at least one nanosecond to +// execute: therefore the execution times can be used to derive a topological +// ordering of the graph (at least as long as there is no loop in the graph). +Status EstimateEarliestExecutionTimes( + const GrapplerItem& item, const Cluster* cluster, + std::unordered_map* execution_times); + +// Compute the time by which the execution of each node must complete to ensure +// the subsequent nodes can still be executed by the times predicted by the +// EstimateEarliestExecutionTimes function. +Status EstimateRequiredTimes( + const GrapplerItem& item, const Cluster* cluster, + const std::unordered_map& + execution_times, + std::unordered_map* required_times); + +} // namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_OPTIMIZERS_STATIC_SCHEDULE_H_ diff --git a/linux/include/tensorflow/core/grappler/utils.h b/linux/include/tensorflow/core/grappler/utils.h new file mode 100644 index 000000000..95126d470 --- /dev/null +++ b/linux/include/tensorflow/core/grappler/utils.h @@ -0,0 +1,342 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_UTILS_H_ +#define TENSORFLOW_CORE_GRAPPLER_UTILS_H_ + +#include +#include +#include +#include + +#include "tensorflow/core/framework/graph.pb.h" +#include "tensorflow/core/framework/node_def.pb.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/lib/core/stringpiece.h" +#include "tensorflow/core/lib/core/threadpool.h" +#include "tensorflow/core/lib/gtl/inlined_vector.h" + +namespace tensorflow { +namespace grappler { + +// A utility class to lookup a node and its outputs by node name. +class NodeMap { + public: + // Note: The NodeMap will store pointers to nodes in graph, which may become + // invalid if graph is changed. + explicit NodeMap(GraphDef* graph); + NodeDef* GetNode(const string& name) const; + bool NodeExists(const string& name) const; + const std::set& GetOutputs(const string& node_name) const; + // This method doesn't record the outputs of the added node; the outputs need + // to be explicitly added by the AddOutput method. + void AddNode(const string& name, NodeDef* node); + void RemoveNode(const string& name); + void UpdateInput(const string& node_name, const string& old_input_name, + const string& new_input_name); + void AddOutput(const string& node_name, const string& output_name); + void RemoveInputs(const string& node_name); + void RemoveOutput(const string& node_name, const string& output_name); + void RemoveOutputs(const string& node_name); + void UpdateOutput(const string& node_name, const string& old_output_name, + const string& new_output_name); + + private: + const std::set empty_set_; + std::unordered_map nodes_; + std::unordered_map> outputs_; +}; + +// A vector with a set. The set stores the same elements as the vector, and +// quickly answers whether a value is in the vector. Duplicated elements are not +// allowed for now. +template > +class SetVector { + public: + // Returns false if value already existed in the set, true otherwise. + bool PushBack(const T& value) { + if (!set_.insert(value).second) { + return false; + } + vector_.push_back(value); + return true; + } + + T PopBack() { + T back = vector_.back(); + set_.erase(back); + vector_.pop_back(); + return back; + } + + bool Exists(const T& value) const { return set_.find(value) != set_.end(); } + + bool Empty() const { return vector_.empty(); } + + void Reserve(int64 size) { vector_.reserve(size); } + + private: + std::unordered_set set_; + std::vector vector_; +}; + +// True iff 'name' refers to a control inputs, i.e. a node name prefixed with +// the ^ character. +bool IsControlInput(const string& name); + +// True iff 'name1' and 'name2' refer to the same input. +bool IsSameInput(const string& name1, const string& name2); + +// Returns the trailing position number (or zero if no number is present) if +// NodeName(input_name) is equal to node_name. Returns -1 for control inputs. +// Returns -2 if NodeName(input_name) is not equal to node_name. +// Note: This function is used very heavily, and this hand-optimized +// version is 3-4x faster than the version using Scanner, which it replaced. +// This is worth the reduction in readability. +inline int NodePositionIfSameNode(const string& input_name, + const string& node_name) { + if (input_name.empty()) return -2; + const bool is_ctrl = input_name[0] == '^'; + auto input_it = is_ctrl ? input_name.begin() + 1 : input_name.begin(); + auto node_it = node_name.begin(); + if (node_name.empty() || + std::distance(input_it, input_name.end()) < node_name.size()) { + return -2; + } + while (node_it != node_name.end()) { + if (*input_it++ != *node_it++) { + return -2; + } + } + if (input_it == input_name.end()) { + return is_ctrl ? -1 : 0; + } else if (*input_it++ == ':') { + StringPiece remaining(&(*input_it), + std::distance(input_it, input_name.end())); + int position; + if (!strings::safe_strto32(remaining, &position)) { + return -2; + } + return is_ctrl ? -1 : position; + } else { + return -2; + } +} + +// Return the node name corresponding to 'name' if name is valid, or the empty +// string otherwise. +inline StringPiece NodeNameAsStringPiece(const string& name) { + static const string empty; + if (name.empty()) return StringPiece(empty); + const auto begin_it = name[0] == '^' ? name.begin() + 1 : name.begin(); + auto end_it = begin_it; + while (end_it != name.end() && *end_it != ':') { + ++end_it; + } + if (end_it != name.end() && *end_it != ':') { + return StringPiece(empty); + } + return StringPiece(&(*begin_it), std::distance(begin_it, end_it)); +} + +// Return the node name corresponding to 'name' if name is valid, or the empty +// string otherwise. +inline string NodeName(const string& name) { + return string(NodeNameAsStringPiece(name)); +} + +// Returns the node name and position in a single call. +inline StringPiece ParseNodeNameAsStringPiece(const string& name, + int* position) { + static const string empty; + if (name.empty()) { + *position = 0; + return StringPiece(empty); + } + const bool is_ctrl = name[0] == '^'; + const auto begin_it = is_ctrl ? name.begin() + 1 : name.begin(); + *position = is_ctrl ? -1 : 0; + auto end_it = begin_it; + while (end_it != name.end() && *end_it != ':') { + ++end_it; + } + const StringPiece node_name(&(*begin_it), std::distance(begin_it, end_it)); + if (end_it != name.end()) { + if (*end_it != ':') { + return StringPiece(empty); + } else if (!is_ctrl) { + ++end_it; + StringPiece remaining(&(*end_it), std::distance(end_it, name.end())); + if (!strings::safe_strto32(remaining, position)) { + return StringPiece(empty); + } + } + } + return node_name; +} + +// Returns the node name and position in a single call. +inline string ParseNodeName(const string& name, int* position) { + return string(ParseNodeNameAsStringPiece(name, position)); +} + +inline int NodePosition(const string& name) { + int position; + ParseNodeNameAsStringPiece(name, &position); + return position; +} + +// Add a prefix to a node name with a custom delimiter. +string AddPrefixToNodeName(const string& name, const string& prefix, + const string& delimiter); + +// Add a prefix to a node name. +string AddPrefixToNodeName(const string& name, const string& prefix); + +// Executes a 'fn' in the 'thread_pool'. The method waits for the configured +// timeout (in milliseconds) for 'fn' to complete, before returning false. +// +// If returning false, the 'fn' may still continue to execute in the +// thread-pool. It is the responsibility of the caller to reset the thread-pool +// as appropriate. +bool ExecuteWithTimeout(std::function fn, int64 timeout_in_ms, + thread::ThreadPool* thread_pool); + +// Returns the node name prefixed with conventional symbol '^' +// for control dependency, given a NodeDef. +string AsControlDependency(const NodeDef& node); + +// Returns the node name prefixed with conventional symbol '^' +// for control dependency, given a node name +string AsControlDependency(const string& node); + +// Returns the number of outputs of a node according to its OpDef. Note that +// some of the outputs may be unconnected. +int NumOutputs(const NodeDef& node, GraphDef* graph); + +// Returns true iff the node has at least one control input. +bool HasControlInputs(const NodeDef& node); + +// Number of connected non-control inputs. +int NumNonControlInputs(const NodeDef& node); + +// Number of connected non-control outputs. +int NumNonControlOutputs(const NodeDef& node, const NodeMap& node_map); + +// Number of connected non-control data outputs (Ops that consume output tensor +// data, not just it's shape). +int NumNonControlDataOutputs(const NodeDef& node, const NodeMap& node_map); + +// Removes redundant control inputs from node. +void DedupControlInputs(NodeDef* node); + +// Returns the data type in attribute `attr_name` of `node`. If that attribute +// doesn't exist, returns DT_INVALID. +DataType GetDataTypeFromAttr(const NodeDef& node, const string& attr_name); + +// Returns the last node in the simple chain starting at source and traversing +// through the input(0) edge from each node as long as the next node satisfies +// the predicate given in pred_fn. If no nodes satisfy the predicate, &source +// will be returned. Example: For the chain +// source <- a <- b <- ... <- y <- z +// where +// pred_fn(a) = pred_fn(b) = ... = pred_fn(y) = true, +// pred_fn(z) = false, +// the return value will be a pointer to y. +NodeDef* GetTailOfChain(const NodeDef& source, const NodeMap& node_map, + bool follow_control_input, + const std::function& pred_fn); + +// Permute the nodes of graph in place according to the permutation. +void PermuteNodesInPlace(GraphDef* graph, std::vector* permutation, + bool invert_permutation); + +Status SetTensorValue(DataType dtype, int value, Tensor* tensor); + +void EraseNodesFromGraph(const std::set& nodes_to_delete, GraphDef* graph); + +void EraseNodesFromGraph(std::vector&& nodes_to_delete, GraphDef* graph); + +void EraseNodesFromGraph(const std::set& nodes_to_delete, + GraphDef* graph); + +class SimpleGraphView { + public: + // Build a graph view for the specified graphdef. + Status Initialize(const GraphDef& graph) { + return Initialize(graph, nullptr, true, true); + } + // Build a graph view for the specified graphdef augmented with the additional + // edges specified in 'extra_dependencies' if any. Note that + // extra_dependencies can be null. + Status Initialize( + const GraphDef& graph, + const std::vector>* + extra_dependencies) { + return Initialize(graph, extra_dependencies, true, true); + } + Status Initialize( + const GraphDef& graph, + const std::vector>* + extra_dependencies, + bool dedup_inputs, bool dedup_outputs); + + const GraphDef* graph() const { return graph_; } + inline int num_nodes() const { return index_to_name_.size(); } + inline bool has_node(const string& node_name) const { + return name_to_index_.find(node_name) != name_to_index_.end(); + } + inline const int index(const string& node_name) const { + const auto& it = name_to_index_.find(node_name); + DCHECK(it != name_to_index_.end()); + return it == name_to_index_.end() ? -1 : it->second; + } + inline const NodeDef& node(int node_idx) const { + return graph_->node(node_idx); + } + inline const string& node_name(int node_idx) const { + return index_to_name_[node_idx]; + } + inline const gtl::InlinedVector& inputs(int node_idx) const { + return inputs_[node_idx]; + } + inline const gtl::InlinedVector& outputs(int node_idx) const { + return outputs_[node_idx]; + } + + // Traverse the graph starting at `node_idx`, collecting indices of nodes + // visited in nodes_found. If a node has an op in `op_types_to_traverse`, the + // walk continues to its children. It is assumed that *graph_ was not modified + // after the call to Initialize(). + // If `op_types_to_traverse` is empty the DFS will traverse any node type. + void DepthFirstSearch(const std::unordered_set& op_types_to_traverse, + int node_idx, std::set* nodes_found) const; + + string PrintToString() const; + + private: + const GraphDef* graph_; // Not owned. + std::vector index_to_name_; + std::unordered_map name_to_index_; + std::vector> inputs_; + std::vector> outputs_; +}; + +} // end namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_UTILS_H_ diff --git a/linux/include/tensorflow/core/grappler/utils/colocation.h b/linux/include/tensorflow/core/grappler/utils/colocation.h new file mode 100644 index 000000000..6062db610 --- /dev/null +++ b/linux/include/tensorflow/core/grappler/utils/colocation.h @@ -0,0 +1,39 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_UTILS_COLOCATION_H_ +#define TENSORFLOW_CORE_GRAPPLER_UTILS_COLOCATION_H_ + +#include +#include "tensorflow/core/framework/graph.pb.h" + +namespace tensorflow { +namespace grappler { + +// Evaluates the colocation relation in the graph and rewrites the new +// colocation relation in the graph. We scan the graph nodes sequentially, and +// builds a disjoint-sets of nodes (within each disjoint-set the nodes are +// colocated with each other). We then select the root node of each set as a +// representative node, and then colocate each node within the set (should also +// exist in graph) with the representative node. +// Note that there is current one situation this function can't handle: +// Node A colocates with X, node B colocates with Y, X colocates with Y but +// X, Y are removed from graph. In this case we can't know A colocates with B. +void ReassignColocation(GraphDef* graph); + +} // namespace grappler +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_UTILS_COLOCATION_H_ diff --git a/linux/include/tensorflow/core/grappler/utils/frame.h b/linux/include/tensorflow/core/grappler/utils/frame.h new file mode 100644 index 000000000..95b72748f --- /dev/null +++ b/linux/include/tensorflow/core/grappler/utils/frame.h @@ -0,0 +1,43 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_UTILS_FRAME_H_ +#define TENSORFLOW_CORE_GRAPPLER_UTILS_FRAME_H_ + +#include +#include "tensorflow/core/framework/graph.pb.h" +#include "tensorflow/core/grappler/utils.h" +#include "tensorflow/core/lib/core/status.h" + +namespace tensorflow { +namespace grappler { + +using FrameMap = std::unordered_map>; + +// Returns the number of frames present in the graph, and populates +// the 'frames' argument with the collection of frames (denoted by their +// frame ids) in the outermost-to-innermost order. Frame ids are arbitrary. +Status IdentifyFrames(const GraphDef& graph, FrameMap* frame_map, + int* num_frames); + +// As above, but use an existing NodeMap for graph instead of building it +// from scratch. +Status IdentifyFramesWithNodeMap(const GraphDef& graph, const NodeMap& node_map, + FrameMap* frame_map, int* num_frames); + +} // namespace grappler +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_UTILS_FRAME_H_ diff --git a/linux/include/tensorflow/core/grappler/utils/functions.h b/linux/include/tensorflow/core/grappler/utils/functions.h new file mode 100644 index 000000000..ef944ced0 --- /dev/null +++ b/linux/include/tensorflow/core/grappler/utils/functions.h @@ -0,0 +1,247 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_UTILS_FUNCTIONS_H_ +#define TENSORFLOW_CORE_GRAPPLER_UTILS_FUNCTIONS_H_ + +#include +#include +#include +#include "tensorflow/core/framework/attr_value.pb.h" +#include "tensorflow/core/framework/function.h" +#include "tensorflow/core/framework/function.pb.h" +#include "tensorflow/core/framework/node_def_util.h" +#include "tensorflow/core/framework/op_def.pb.h" +#include "tensorflow/core/grappler/grappler_item.h" + +namespace tensorflow { +namespace grappler { + +using AttrValueMap = std::unordered_map; + +// Depending on the function instantiation attributes, input argument to the +// function might be a single tensor, list of tensors of the same type, or a +// list of tensors of different types. +// +// InputArgExpansion keeps track of the placeholders that were added to the +// function body in place of function inputs and a resolved input data type. +struct InputArgExpansion { + // TODO(ezhulenev): Add support for functions with tensor sequence inputs of + // different data types. + // TODO(ezhulenev): Support type parametrized inputs? + string input_name; // name of the function input argument + DataType data_type; // input data type + bool is_ref; // if true, inputs are required to be refs + std::vector placeholders; // names of placeholder nodes in the + // function body +}; + +// Depending on the function instantiation attributes, output argument is mapped +// to one or more outputs of one of the function body nodes. +// +// OutputArgExpansion keeps mapping from a function output arg to the output +// tensors of a function body nodes and a resolved output data type +struct OutputArgExpansion { + // TODO(ezhulenev): Add support for functions with tensor sequence outputs of + // different data types. + // TODO(ezhulenev): Support type parametrized outputs? + string output_name; // name of the function output argument + DataType data_type; // output data type + bool is_ref; // if true, outputs are refs + std::vector output_tensors; // names of output tensor from the + // function body nodes +}; + +// FunctionDef uses different connectivity encoding for the function body nodes, +// then a GraphDef (see function.proto for details). Input name in FunctionDef +// can potentially represent a sequence of tensors (instead just one tensor in +// GraphDef), we need to expand it when converting from FunctionDef to GraphDef, +// and fold it back when doing backward conversion. +class GrapplerFunctionConnectivity { + public: + void RegisterInputArgExpansion(InputArgExpansion input_arg_expansion); + void RegisterFunctionBodyOutputs(const string& node_name, + tensorflow::NameRangeMap&& outputs); + + // Expand input encoded in FunctionDef format (name[:output][:position]) into + // multiple inputs in GraphDef format (name[:position]). + Status ExpandFunctionDefInput(const string& func_def_input, + std::vector* graph_def_inputs) const; + + // Update Node inputs from FunctionDef to GraphDef format. + Status ExpandNodeInputs(NodeDef* function_body_node) const; + + // When expanding inputs in function def format, single input might be + // expanded into multiple tensors. When converting back to the function def + // format from graph def format, it's always a 1-to-1 relationship. + // FunctionDef built from GrapplerFunctionItem is always specialized to it's + // instantiation attributes and length of input args (and node def outputs) is + // known. + + // Map from GraphDef input format to FunctionDef input format using registered + // input arg expansion and function body outputs. + Status AsFunctionDefInput(const string& graph_def_input, + string* func_def_input) const; + + // Update Node inputs from GraphDef to FunctionDef format. + Status AsFunctionDefNode(NodeDef* function_body_node) const; + + private: + // Mapping from input name to input arg expansion. + std::unordered_map input_arg_expansions_; + // Mapping from function body node name to output names range map. + std::unordered_map function_body_outputs_; + + struct InputArgPlaceholder { + string input_name; + int position; + }; + + // Mapping from input arg placeholder to the function input tensor. + std::unordered_map input_arg_placeholders_; +}; + +// Get Function type attributes using attributes of a node that instantiated +// a function. +class GrapplerFunctionItemInstantiation { + public: + explicit GrapplerFunctionItemInstantiation( + const AttrValueMap* func_instantiation_attr) + : func_instantiation_attr_(func_instantiation_attr) {} + + // Get DataType from attributes by name. Return error if attribute is missing, + // or it doesn't define a valid data type. + Status GetTypeAttr(const string& type_attr_name, DataType* data_type) const; + + // Get argument data type. If data type is not explicitly defined, uses + // provided attribute name to look it up in function attributes. + Status GetArgType(const OpDef::ArgDef& arg, DataType* data_type) const; + + private: + const AttrValueMap* func_instantiation_attr_; // do not own +}; + +// A special case of GrapplerItem, constructed from a TensorFlow Function. +class GrapplerFunctionItem : public GrapplerItem { + public: + GrapplerFunctionItem() = default; + GrapplerFunctionItem(string func_name, string description, + AttrValueMap func_attr, + std::vector input_arg_expansions, + std::vector output_arg_expansions, + std::vector keep_nodes, int graph_def_version, + bool is_stateful, GraphDef&& function_body); + + const string& description() const; + + bool IsInputPlaceholder(const string& node_name) const; + + const std::vector& inputs() const; + const InputArgExpansion& input(int i) const; + const std::size_t input_size() const; + + const std::vector& outputs() const; + const OutputArgExpansion& output(int i) const; + const std::size_t output_size() const; + + const AttrValueMap& func_attr() const; + const GraphDef& function_body() const; + GraphDef& mutable_function_body(); + + bool is_stateful() const; + + GrapplerFunctionItem& SwapFunctionBody(GraphDef&& other); + + private: + friend Status ReplaceInputWithConst(const NodeDef&, int, + GrapplerFunctionItem*); + + string description_; + AttrValueMap func_attr_; // Attributes specific to function definition that + // produced this item (FuncDef.attr field). + + std::vector input_arg_expansions_; + std::vector output_arg_expansions_; + + std::set input_arg_placeholders_; + + bool is_stateful_; +}; + +// Check if function input/output types are fully defined only at instantiation +// time (parametrized by it's instantiation node). +bool HasParametrizedType(const FunctionDef& func); + +// Check if a function body is parametrized by it's instantiation node. Function +// body is parametrized, if it has at least one node with a 'placeholder' +// attribute. +bool HasParametrizedBody(const FunctionDef& func); + +// Check if function has parametrized type or body. +bool IsParametrized(const FunctionDef& func); + +// Resolve function instantiation type parameters from the attributes of the +// caller node. Return error if type can't be resolved. +Status InstantiationTypeParameters( + const FunctionDef& func, const AttrValueMap& func_instantiation_attr, + std::unordered_map* type_parameters); + +// Resolve function instantiation body parameters (values for the function body +// attr placeholders) from the attributes of the caller node. Return error if +// type can't be resolved. +Status InstantiationBodyParameters( + const FunctionDef& func, const AttrValueMap& func_instantiation_attr, + std::unordered_map* body_parameters); + +// Register GrapplerFunctionItem input arg expansion and function body outputs +// in the GrapplerFunctionConnectivity. Use function library definition to +// lookup function body nodes output names and ranges. +Status RegisterGrapplerFunctionConnectivity( + const GrapplerFunctionItem& item, const FunctionLibraryDefinition& flib, + GrapplerFunctionConnectivity* connectivity); + +// Replace one of the function inputs with a constant. +Status ReplaceInputWithConst(const NodeDef& input_const, int input_position, + GrapplerFunctionItem* item); + +// Make a GrapplerFunctionItem from the function definition and function +// instantiation attributes (caller node attributes). Returns error if the given +// function def cannot be converted (e.g. not all attributes are defined). +Status MakeGrapplerFunctionItem(const FunctionDef& func, + const AttrValueMap& func_instantiation_attr, + const FunctionLibraryDefinition& flib, + const int graph_def_version, + GrapplerFunctionItem* item); + +// Make a GrapplerFunction item from the function definition. Function must be +// fully defined (no type or body parametrization). +// TODO(ezhulenev): Support parametrized functions without fully defined +// instantiation attributes? Do we ever want to optimize parametrized function +// without specializing it to it's instantiation attributes (at least types)? +Status MakeGrapplerFunctionItem(const FunctionDef& func, + const FunctionLibraryDefinition& flib, + const int graph_def_version, + GrapplerFunctionItem* item); + +// Make a FunctionDef from the GrapplerFunctionItem. Use function library +// definition to lookup function body nodes output names and ranges. +Status MakeFunctionDef(const GrapplerFunctionItem& item, + const FunctionLibraryDefinition& flib, + FunctionDef* func); + +} // end namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_UTILS_FUNCTIONS_H_ diff --git a/linux/include/tensorflow/core/grappler/utils/symbolic_shapes.h b/linux/include/tensorflow/core/grappler/utils/symbolic_shapes.h new file mode 100644 index 000000000..0a7d8ac82 --- /dev/null +++ b/linux/include/tensorflow/core/grappler/utils/symbolic_shapes.h @@ -0,0 +1,77 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_UTILS_SYMBOLIC_SHAPES_H_ +#define TENSORFLOW_CORE_GRAPPLER_UTILS_SYMBOLIC_SHAPES_H_ + +#include "tensorflow/core/framework/tensor_shape.pb.h" +#include "tensorflow/core/grappler/costs/op_performance_data.pb.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { +namespace grappler { + +bool IsKnown(const TensorShapeProto::Dim& dim); +bool IsKnownSymbolically(const TensorShapeProto::Dim& dim); +bool IsUnknown(const TensorShapeProto::Dim& dim); + +// Shape is symbolically defined, if it has a known rank, and each dimension is +// known (dim_size >= 0), or is a symbolic dimension size (dim_size <= -2). +bool ShapeIsSymbolicallyDefined(const TensorShapeProto& shape); +bool ShapeIsSymbolicallyDefined(const OpInfo::TensorProperties& properties); + +// Returns the rank of the shape ir -1 if unknown +int Rank(const TensorShapeProto& shape); + +// Returns the number of coefficients in the shape or -1 if unknown. +// TODO(bsteiner) Add a function that computes the minimum size of the tensor, +// ie the size assuming all the symbolic dimensions take the value 1. +int64 NumCoefficients(const TensorShapeProto& shape); + +// Shapes are symbolically equal, if they have the same rank, they are known or +// symbolically defined, and have matching dimensions. +bool ShapesSymbolicallyEqual(const TensorShapeProto& left, + const TensorShapeProto& right); +bool ShapesSymbolicallyEqual(const OpInfo::TensorProperties& left, + const OpInfo::TensorProperties& right); + +// Check if two shapes can be broadcasted to each other. Both shapes must be at +// least symbolically defined, and the have valid BCast instance. +bool ShapesBroadcastable(const TensorShapeProto& left, + const TensorShapeProto& right); +bool ShapesBroadcastable(const OpInfo::TensorProperties& left, + const OpInfo::TensorProperties& right); +bool ShapeAfterBroadcast(const TensorShapeProto& left, + const TensorShapeProto& right, + TensorShapeProto* output_shape); + +// Return true if can prove, that tensor of size 'left' is smaller than tensor +// of size 'right'. Return false if it's larger or equal, or it's impossible to +// compare because of unknown dimensions, or mismatch in symbolic dimensions. +bool CompareSymbolicallyShapedTensorSizes(const TensorShapeProto& left, + const TensorShapeProto& right); +bool CompareSymbolicallyShapedTensorSizes( + const OpInfo::TensorProperties& left, + const OpInfo::TensorProperties& right); + +// Returns the ratio of the sizes of the 2 shapes if known statically, or -1 +// otherwise. +int64 ComputeSizeRatio(const TensorShapeProto& numerator, + const TensorShapeProto& denominator); + +} // namespace grappler +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_UTILS_SYMBOLIC_SHAPES_H_ diff --git a/linux/include/tensorflow/core/grappler/utils/topological_sort.h b/linux/include/tensorflow/core/grappler/utils/topological_sort.h new file mode 100644 index 000000000..b8cf897a3 --- /dev/null +++ b/linux/include/tensorflow/core/grappler/utils/topological_sort.h @@ -0,0 +1,40 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_UTILS_TOPOLOGICAL_SORT_H_ +#define TENSORFLOW_CORE_GRAPPLER_UTILS_TOPOLOGICAL_SORT_H_ + +#include "tensorflow/core/framework/graph.pb.h" +#include "tensorflow/core/lib/core/status.h" + +namespace tensorflow { +namespace grappler { + +// Compute a topological ordering for the graph nodes. +Status ComputeTopologicalOrder( + const GraphDef& graph, std::unordered_map* topo_order, + const std::vector>* + extra_dependencies); + +// Sort a graph in topological order. +Status TopologicalSort(GraphDef* graph); + +// Sort a graph in topological order and reverse it. +Status ReversedTopologicalSort(GraphDef* graph); + +} // namespace grappler +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_UTILS_TOPOLOGICAL_SORT_H_ diff --git a/linux/include/tensorflow/core/grappler/utils/traversal.h b/linux/include/tensorflow/core/grappler/utils/traversal.h new file mode 100644 index 000000000..bb3fa090e --- /dev/null +++ b/linux/include/tensorflow/core/grappler/utils/traversal.h @@ -0,0 +1,39 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_GRAPPLER_UTILS_TRAVERSAL_H_ +#define TENSORFLOW_CORE_GRAPPLER_UTILS_TRAVERSAL_H_ + +#include +#include "tensorflow/core/grappler/graph_view.h" + +namespace tensorflow { +namespace grappler { + +// Traverse the graph in reverse dfs order, starting from the list of nodes +// specified in the 'from' argument. The pre_order and post_order functors will +// be called on each reachable node (including the 'from' nodes) in pre and post +// order. If loops are found, the on_back_edge functor will be called on the +// corresponding back edges. Moreover, the pre and post order will assume that +// these back edges will be cut. +void ReverseDfs(const GraphView& graph_view, const std::vector& from, + const std::function& pre_order, + const std::function& post_order, + const std::function& on_back_edge); + +} // namespace grappler +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_GRAPPLER_UTILS_TRAVERSAL_H_ diff --git a/linux/include/tensorflow/core/kernels/adjust_contrast_op.h b/linux/include/tensorflow/core/kernels/adjust_contrast_op.h new file mode 100644 index 000000000..f4a53c2ef --- /dev/null +++ b/linux/include/tensorflow/core/kernels/adjust_contrast_op.h @@ -0,0 +1,156 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_ADJUST_CONTRAST_OP_H_ +#define TENSORFLOW_CORE_KERNELS_ADJUST_CONTRAST_OP_H_ +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/tensor_types.h" + +namespace tensorflow { +namespace functor { + +// Functor used by AdjustContrastOp to do the computations. +template +struct AdjustContrast { + void operator()(const Device& d, typename TTypes::ConstTensor input, + typename TTypes::ConstScalar contrast_factor, + typename TTypes::ConstScalar min_value, + typename TTypes::ConstScalar max_value, + typename TTypes::Tensor mean_values, + typename TTypes::Tensor output) { + const int batch = input.dimension(0); + const int height = input.dimension(1); + const int width = input.dimension(2); + const int channels = input.dimension(3); + + Eigen::array scalar_broadcast; + scalar_broadcast[0] = batch; + scalar_broadcast[1] = height; + scalar_broadcast[2] = width; + scalar_broadcast[3] = channels; +#if !defined(EIGEN_HAS_INDEX_LIST) + Eigen::array reduction_axis; + reduction_axis[0] = 1; + reduction_axis[1] = 2; + Eigen::array broadcast_dims; + broadcast_dims[0] = 1; + broadcast_dims[1] = height; + broadcast_dims[2] = width; + broadcast_dims[3] = 1; + Eigen::Tensor::Dimensions reshape_dims; + reshape_dims[0] = batch; + reshape_dims[1] = 1; + reshape_dims[2] = 1; + reshape_dims[3] = channels; +#else + Eigen::IndexList, Eigen::type2index<2> > + reduction_axis; + Eigen::IndexList, int, int, Eigen::type2index<1> > + broadcast_dims; + broadcast_dims.set(1, height); + broadcast_dims.set(2, width); + Eigen::IndexList, Eigen::type2index<1>, int> + reshape_dims; + reshape_dims.set(0, batch); + reshape_dims.set(3, channels); +#endif + Eigen::Sizes<1, 1, 1, 1> scalar; + float num_reduced_coeffs = height * width; + mean_values.device(d) = + (input.template cast().sum(reduction_axis).eval() / + num_reduced_coeffs) + .reshape(reshape_dims) + .broadcast(broadcast_dims); + + auto contrast_factor_tensor = + contrast_factor.reshape(scalar).broadcast(scalar_broadcast); + auto adjusted = + (input.template cast() - mean_values) * contrast_factor_tensor + + mean_values; + auto min_bcast = min_value.reshape(scalar).broadcast(scalar_broadcast); + auto max_bcast = max_value.reshape(scalar).broadcast(scalar_broadcast); + // TODO(wicke): This is rather slow and should be re-written as pure cuda. + output.device(d) = adjusted.cwiseMin(max_bcast).cwiseMax(min_bcast); + } +}; + +// Functor used by AdjustContrastOpv2 to do the computations. +template +struct AdjustContrastv2 { + void operator()(const Device& d, typename TTypes::ConstTensor input, + typename TTypes::ConstScalar contrast_factor, + typename TTypes::Tensor output) { + const int batch = input.dimension(0); + const int height = input.dimension(1); + const int width = input.dimension(2); + const int channels = input.dimension(3); + + Eigen::array scalar_broadcast; + scalar_broadcast[0] = batch; + scalar_broadcast[1] = height; + scalar_broadcast[2] = width; + scalar_broadcast[3] = channels; +#if !defined(EIGEN_HAS_INDEX_LIST) + Eigen::array reduction_axis; + reduction_axis[0] = 0; + reduction_axis[1] = 1; + Eigen::array broadcast_dims; + broadcast_dims[0] = 1; + broadcast_dims[1] = height; + broadcast_dims[2] = width; + broadcast_dims[3] = 1; + Eigen::Tensor::Dimensions reshape_dims; + reshape_dims[0] = batch; + reshape_dims[1] = 1; + reshape_dims[2] = 1; + reshape_dims[3] = channels; + Eigen::array reduced_dims_first; + reduced_dims_first[0] = 1; + reduced_dims_first[1] = 2; + reduced_dims_first[2] = 0; + reduced_dims_first[3] = 3; +#else + Eigen::IndexList, Eigen::type2index<1> > + reduction_axis; + Eigen::IndexList, int, int, Eigen::type2index<1> > + broadcast_dims; + broadcast_dims.set(1, height); + broadcast_dims.set(2, width); + Eigen::IndexList, Eigen::type2index<1>, int> + reshape_dims; + reshape_dims.set(0, batch); + reshape_dims.set(3, channels); + Eigen::IndexList, Eigen::type2index<2>, + Eigen::type2index<0>, Eigen::type2index<3> > + reduced_dims_first; +#endif + Eigen::Sizes<1, 1, 1, 1> scalar; + float num_reduced_coeffs = height * width; + output.device(d) = + (input.shuffle(reduced_dims_first).sum(reduction_axis).eval() / + num_reduced_coeffs) + .reshape(reshape_dims) + .broadcast(broadcast_dims); + auto contrast_factor_tensor = + contrast_factor.reshape(scalar).broadcast(scalar_broadcast); + auto adjusted = (input - output) * contrast_factor_tensor; + output.device(d) += adjusted; + } +}; + +} // namespace functor +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_ADJUST_CONTRAST_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/adjust_hsv_gpu.cu.h b/linux/include/tensorflow/core/kernels/adjust_hsv_gpu.cu.h new file mode 100644 index 000000000..49df5ae29 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/adjust_hsv_gpu.cu.h @@ -0,0 +1,146 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CORE_KERNELS_ADJUST_HSV_GPU_CU_H_ +#define TENSORFLOW_CORE_KERNELS_ADJUST_HSV_GPU_CU_H_ + +#if GOOGLE_CUDA + +#define EIGEN_USE_GPU + +#include "tensorflow/core/framework/register_types.h" +#include "tensorflow/core/framework/types.h" + +namespace tensorflow { +namespace internal { + +typedef struct RgbTuple { + float r; + float g; + float b; +} RgbTuple; + +typedef struct HsvTuple { + float h; + float s; + float v; +} HsvTuple; + +inline __device__ HsvTuple rgb2hsv_cuda(const float r, const float g, + const float b) { + HsvTuple tuple; + const float M = fmaxf(r, fmaxf(g, b)); + const float m = fminf(r, fminf(g, b)); + const float chroma = M - m; + float h = 0.0f, s = 0.0f; + // hue + if (chroma > 0.0f) { + if (M == r) { + const float num = (g - b) / chroma; + const float sign = copysignf(1.0f, num); + h = ((sign < 0.0f) * 6.0f + sign * fmodf(sign * num, 6.0f)) / 6.0f; + } else if (M == g) { + h = ((b - r) / chroma + 2.0f) / 6.0f; + } else { + h = ((r - g) / chroma + 4.0f) / 6.0f; + } + } else { + h = 0.0f; + } + // saturation + if (M > 0.0) { + s = chroma / M; + } else { + s = 0.0f; + } + tuple.h = h; + tuple.s = s; + tuple.v = M; + return tuple; +} + +inline __device__ RgbTuple hsv2rgb_cuda(const float h, const float s, + const float v) { + RgbTuple tuple; + const float new_h = h * 6.0f; + const float chroma = v * s; + const float x = chroma * (1.0f - fabsf(fmodf(new_h, 2.0f) - 1.0f)); + const float new_m = v - chroma; + const bool between_0_and_1 = new_h >= 0.0f && new_h < 1.0f; + const bool between_1_and_2 = new_h >= 1.0f && new_h < 2.0f; + const bool between_2_and_3 = new_h >= 2.0f && new_h < 3.0f; + const bool between_3_and_4 = new_h >= 3.0f && new_h < 4.0f; + const bool between_4_and_5 = new_h >= 4.0f && new_h < 5.0f; + const bool between_5_and_6 = new_h >= 5.0f && new_h < 6.0f; + tuple.r = chroma * (between_0_and_1 || between_5_and_6) + + x * (between_1_and_2 || between_4_and_5) + new_m; + tuple.g = chroma * (between_1_and_2 || between_2_and_3) + + x * (between_0_and_1 || between_3_and_4) + new_m; + tuple.b = chroma * (between_3_and_4 || between_4_and_5) + + x * (between_2_and_3 || between_5_and_6) + new_m; + return tuple; +} + +template +__global__ void adjust_hsv_nhwc(const int64 number_elements, + const float* const __restrict__ input, + float* const output, + const float* const hue_delta, + const float* const saturation_scale, + const float* const value_scale) { + // multiply by 3 since we're dealing with contiguous RGB bytes for each pixel + // (NHWC) + const int64 idx = (blockDim.x * blockIdx.x + threadIdx.x) * 3; + // bounds check + if (idx > number_elements - 1) { + return; + } + if (!AdjustHue && !AdjustSaturation && !AdjustV) { + output[idx] = input[idx]; + output[idx + 1] = input[idx + 1]; + output[idx + 2] = input[idx + 2]; + return; + } + const HsvTuple hsv = rgb2hsv_cuda(input[idx], input[idx + 1], input[idx + 2]); + float new_h = hsv.h; + float new_s = hsv.s; + float new_v = hsv.v; + // hue adjustment + if (AdjustHue) { + const float delta = *hue_delta; + new_h = fmodf(hsv.h + delta, 1.0f); + if (new_h < 0.0f) { + new_h = fmodf(1.0f + new_h, 1.0f); + } + } + // saturation adjustment + if (AdjustSaturation && saturation_scale != nullptr) { + const float scale = *saturation_scale; + new_s = fminf(1.0f, fmaxf(0.0f, hsv.s * scale)); + } + // value adjustment + if (AdjustV && value_scale != nullptr) { + const float scale = *value_scale; + new_v = hsv.v * scale; + } + const RgbTuple rgb = hsv2rgb_cuda(new_h, new_s, new_v); + output[idx] = rgb.r; + output[idx + 1] = rgb.g; + output[idx + 2] = rgb.b; +} + +} // namespace internal +} // namespace tensorflow + +#endif // GOOGLE_CUDA +#endif // TENSORFLOW_CORE_KERNELS_ADJUST_HSV_GPU_CU_H_ diff --git a/linux/include/tensorflow/core/kernels/adjust_hue_op.h b/linux/include/tensorflow/core/kernels/adjust_hue_op.h new file mode 100644 index 000000000..983a4072b --- /dev/null +++ b/linux/include/tensorflow/core/kernels/adjust_hue_op.h @@ -0,0 +1,40 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CORE_KERNELS_ADJUST_HUE_OP_H_ +#define TENSORFLOW_CORE_KERNELS_ADJUST_HUE_OP_H_ + +#if GOOGLE_CUDA +#define EIGEN_USE_GPU + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" + +#include "tensorflow/core/framework/types.h" + +namespace tensorflow { + +typedef Eigen::GpuDevice GPUDevice; + +namespace functor { + +struct AdjustHueGPU { + void operator()(GPUDevice* device, const int64 number_of_elements, + const float* const input, const float* const delta, + float* const output); +}; + +} // namespace functor +} // namespace tensorflow + +#endif // GOOGLE_CUDA +#endif // TENSORFLOW_CORE_KERNELS_ADJUST_HUE_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/adjust_saturation_op.h b/linux/include/tensorflow/core/kernels/adjust_saturation_op.h new file mode 100644 index 000000000..fd28ba536 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/adjust_saturation_op.h @@ -0,0 +1,40 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CORE_KERNELS_ADJUST_SATURATION_OP_H_ +#define TENSORFLOW_CORE_KERNELS_ADJUST_SATURATION_OP_H_ + +#if GOOGLE_CUDA +#define EIGEN_USE_GPU + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" + +#include "tensorflow/core/framework/types.h" + +namespace tensorflow { + +typedef Eigen::GpuDevice GPUDevice; + +namespace functor { + +struct AdjustSaturationGPU { + void operator()(GPUDevice* device, const int64 number_of_elements, + const float* const input, const float* const scale, + float* const output); +}; + +} // namespace functor +} // namespace tensorflow + +#endif // GOOGLE_CUDA +#endif // TENSORFLOW_CORE_KERNELS_ADJUST_SATURATION_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/aggregate_ops.h b/linux/include/tensorflow/core/kernels/aggregate_ops.h new file mode 100644 index 000000000..e074d0c2d --- /dev/null +++ b/linux/include/tensorflow/core/kernels/aggregate_ops.h @@ -0,0 +1,226 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_AGGREGATE_OPS_H_ +#define TENSORFLOW_CORE_KERNELS_AGGREGATE_OPS_H_ + +// Functor definitions for Aggregate ops, must be compilable by nvcc. + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/tensor_types.h" + +namespace tensorflow { +namespace functor { + +template +struct Add2Functor { + void operator()(const Device& d, typename TTypes::Flat out, + typename TTypes::ConstFlat in1, + typename TTypes::ConstFlat in2); +}; + +template +struct Add2EigenImpl { + static void Compute(const Device& d, typename TTypes::Flat out, + typename TTypes::ConstFlat in1, + typename TTypes::ConstFlat in2) { + out.device(d) = in1 + in2; + } +}; + +template +struct Add3Functor { + void operator()(const Device& d, typename TTypes::Flat out, + typename TTypes::ConstFlat in1, + typename TTypes::ConstFlat in2, + typename TTypes::ConstFlat in3); +}; + +template +struct Add3EigenImpl { + static void Compute(const Device& d, typename TTypes::Flat out, + typename TTypes::ConstFlat in1, + typename TTypes::ConstFlat in2, + typename TTypes::ConstFlat in3) { + out.device(d) = in1 + in2 + in3; + } +}; + +template +struct Add4Functor { + void operator()(const Device& d, typename TTypes::Flat out, + typename TTypes::ConstFlat in1, + typename TTypes::ConstFlat in2, + typename TTypes::ConstFlat in3, + typename TTypes::ConstFlat in4); +}; + +template +struct Add4EigenImpl { + static void Compute(const Device& d, typename TTypes::Flat out, + typename TTypes::ConstFlat in1, + typename TTypes::ConstFlat in2, + typename TTypes::ConstFlat in3, + typename TTypes::ConstFlat in4) { + out.device(d) = in1 + in2 + in3 + in4; + } +}; + +template +struct Add5Functor { + void operator()(const Device& d, typename TTypes::Flat out, + typename TTypes::ConstFlat in1, + typename TTypes::ConstFlat in2, + typename TTypes::ConstFlat in3, + typename TTypes::ConstFlat in4, + typename TTypes::ConstFlat in5); +}; + +template +struct Add5EigenImpl { + static void Compute(const Device& d, typename TTypes::Flat out, + typename TTypes::ConstFlat in1, + typename TTypes::ConstFlat in2, + typename TTypes::ConstFlat in3, + typename TTypes::ConstFlat in4, + typename TTypes::ConstFlat in5) { + out.device(d) = in1 + in2 + in3 + in4 + in5; + } +}; + +template +struct Add6Functor { + void operator()(const Device& d, typename TTypes::Flat out, + typename TTypes::ConstFlat in1, + typename TTypes::ConstFlat in2, + typename TTypes::ConstFlat in3, + typename TTypes::ConstFlat in4, + typename TTypes::ConstFlat in5, + typename TTypes::ConstFlat in6); +}; + +template +struct Add6EigenImpl { + static void Compute(const Device& d, typename TTypes::Flat out, + typename TTypes::ConstFlat in1, + typename TTypes::ConstFlat in2, + typename TTypes::ConstFlat in3, + typename TTypes::ConstFlat in4, + typename TTypes::ConstFlat in5, + typename TTypes::ConstFlat in6) { + out.device(d) = in1 + in2 + in3 + in4 + in5 + in6; + } +}; + +template +struct Add7Functor { + void operator()(const Device& d, typename TTypes::Flat out, + typename TTypes::ConstFlat in1, + typename TTypes::ConstFlat in2, + typename TTypes::ConstFlat in3, + typename TTypes::ConstFlat in4, + typename TTypes::ConstFlat in5, + typename TTypes::ConstFlat in6, + typename TTypes::ConstFlat in7); +}; + +template +struct Add7EigenImpl { + static void Compute(const Device& d, typename TTypes::Flat out, + typename TTypes::ConstFlat in1, + typename TTypes::ConstFlat in2, + typename TTypes::ConstFlat in3, + typename TTypes::ConstFlat in4, + typename TTypes::ConstFlat in5, + typename TTypes::ConstFlat in6, + typename TTypes::ConstFlat in7) { + out.device(d) = in1 + in2 + in3 + in4 + in5 + in6 + in7; + } +}; + +template +struct Add8Functor { + void operator()( + const Device& d, typename TTypes::Flat out, + typename TTypes::ConstFlat in1, typename TTypes::ConstFlat in2, + typename TTypes::ConstFlat in3, typename TTypes::ConstFlat in4, + typename TTypes::ConstFlat in5, typename TTypes::ConstFlat in6, + typename TTypes::ConstFlat in7, typename TTypes::ConstFlat in8); +}; + +template +struct Add8EigenImpl { + static void Compute( + const Device& d, typename TTypes::Flat out, + typename TTypes::ConstFlat in1, typename TTypes::ConstFlat in2, + typename TTypes::ConstFlat in3, typename TTypes::ConstFlat in4, + typename TTypes::ConstFlat in5, typename TTypes::ConstFlat in6, + typename TTypes::ConstFlat in7, typename TTypes::ConstFlat in8) { + out.device(d) = in1 + in2 + in3 + in4 + in5 + in6 + in7 + in8; + } +}; + +// Add8p is like Add8 except the underlying implementation should += +// rather than assign to the output. +template +struct Add8pFunctor { + void operator()( + const Device& d, typename TTypes::Flat out, + typename TTypes::ConstFlat in1, typename TTypes::ConstFlat in2, + typename TTypes::ConstFlat in3, typename TTypes::ConstFlat in4, + typename TTypes::ConstFlat in5, typename TTypes::ConstFlat in6, + typename TTypes::ConstFlat in7, typename TTypes::ConstFlat in8); +}; + +template +struct Add8pEigenImpl { + static void Compute( + const Device& d, typename TTypes::Flat out, + typename TTypes::ConstFlat in1, typename TTypes::ConstFlat in2, + typename TTypes::ConstFlat in3, typename TTypes::ConstFlat in4, + typename TTypes::ConstFlat in5, typename TTypes::ConstFlat in6, + typename TTypes::ConstFlat in7, typename TTypes::ConstFlat in8) { + out.device(d) += in1 + in2 + in3 + in4 + in5 + in6 + in7 + in8; + } +}; + +template +struct Add9Functor { + void operator()( + const Device& d, typename TTypes::Flat out, + typename TTypes::ConstFlat in1, typename TTypes::ConstFlat in2, + typename TTypes::ConstFlat in3, typename TTypes::ConstFlat in4, + typename TTypes::ConstFlat in5, typename TTypes::ConstFlat in6, + typename TTypes::ConstFlat in7, typename TTypes::ConstFlat in8, + typename TTypes::ConstFlat in9); +}; + +template +struct Add9EigenImpl { + static void Compute( + const Device& d, typename TTypes::Flat out, + typename TTypes::ConstFlat in1, typename TTypes::ConstFlat in2, + typename TTypes::ConstFlat in3, typename TTypes::ConstFlat in4, + typename TTypes::ConstFlat in5, typename TTypes::ConstFlat in6, + typename TTypes::ConstFlat in7, typename TTypes::ConstFlat in8, + typename TTypes::ConstFlat in9) { + out.device(d) = in1 + in2 + in3 + in4 + in5 + in6 + in7 + in8 + in9; + } +}; + +} // namespace functor +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_AGGREGATE_OPS_H_ diff --git a/linux/include/tensorflow/core/kernels/aggregate_ops_cpu.h b/linux/include/tensorflow/core/kernels/aggregate_ops_cpu.h new file mode 100644 index 000000000..3e87917b6 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/aggregate_ops_cpu.h @@ -0,0 +1,253 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_AGGREGATE_OPS_CPU_H_ +#define TENSORFLOW_CORE_KERNELS_AGGREGATE_OPS_CPU_H_ + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/tensor_types.h" + +#include "tensorflow/core/kernels/aggregate_ops.h" + +typedef Eigen::ThreadPoolDevice CPUDevice; + +#ifdef TENSORFLOW_USE_SYCL +typedef Eigen::SyclDevice SYCLDevice; +#endif // TENSORFLOW_USE_SYCL + +namespace tensorflow { + +// Partial specializations for a CPUDevice, that uses the Eigen implementation +// from AddNEigenImpl. +namespace functor { +template +struct Add2Functor { + void operator()(const CPUDevice& d, typename TTypes::Flat out, + typename TTypes::ConstFlat in1, + typename TTypes::ConstFlat in2) { + Add2EigenImpl::Compute(d, out, in1, in2); + } +}; +template +struct Add3Functor { + void operator()(const CPUDevice& d, typename TTypes::Flat out, + typename TTypes::ConstFlat in1, + typename TTypes::ConstFlat in2, + typename TTypes::ConstFlat in3) { + Add3EigenImpl::Compute(d, out, in1, in2, in3); + } +}; +template +struct Add4Functor { + void operator()(const CPUDevice& d, typename TTypes::Flat out, + typename TTypes::ConstFlat in1, + typename TTypes::ConstFlat in2, + typename TTypes::ConstFlat in3, + typename TTypes::ConstFlat in4) { + Add4EigenImpl::Compute(d, out, in1, in2, in3, in4); + } +}; +template +struct Add5Functor { + void operator()(const CPUDevice& d, typename TTypes::Flat out, + typename TTypes::ConstFlat in1, + typename TTypes::ConstFlat in2, + typename TTypes::ConstFlat in3, + typename TTypes::ConstFlat in4, + typename TTypes::ConstFlat in5) { + Add5EigenImpl::Compute(d, out, in1, in2, in3, in4, in5); + } +}; +template +struct Add6Functor { + void operator()(const CPUDevice& d, typename TTypes::Flat out, + typename TTypes::ConstFlat in1, + typename TTypes::ConstFlat in2, + typename TTypes::ConstFlat in3, + typename TTypes::ConstFlat in4, + typename TTypes::ConstFlat in5, + typename TTypes::ConstFlat in6) { + Add6EigenImpl::Compute(d, out, in1, in2, in3, in4, in5, in6); + } +}; +template +struct Add7Functor { + void operator()(const CPUDevice& d, typename TTypes::Flat out, + typename TTypes::ConstFlat in1, + typename TTypes::ConstFlat in2, + typename TTypes::ConstFlat in3, + typename TTypes::ConstFlat in4, + typename TTypes::ConstFlat in5, + typename TTypes::ConstFlat in6, + typename TTypes::ConstFlat in7) { + Add7EigenImpl::Compute(d, out, in1, in2, in3, in4, in5, in6, + in7); + } +}; + +template +struct Add8Functor { + void operator()( + const CPUDevice& d, typename TTypes::Flat out, + typename TTypes::ConstFlat in1, typename TTypes::ConstFlat in2, + typename TTypes::ConstFlat in3, typename TTypes::ConstFlat in4, + typename TTypes::ConstFlat in5, typename TTypes::ConstFlat in6, + typename TTypes::ConstFlat in7, typename TTypes::ConstFlat in8) { + Add8EigenImpl::Compute(d, out, in1, in2, in3, in4, in5, in6, + in7, in8); + } +}; + +template +struct Add8pFunctor { + void operator()( + const CPUDevice& d, typename TTypes::Flat out, + typename TTypes::ConstFlat in1, typename TTypes::ConstFlat in2, + typename TTypes::ConstFlat in3, typename TTypes::ConstFlat in4, + typename TTypes::ConstFlat in5, typename TTypes::ConstFlat in6, + typename TTypes::ConstFlat in7, typename TTypes::ConstFlat in8) { + Add8pEigenImpl::Compute(d, out, in1, in2, in3, in4, in5, in6, + in7, in8); + } +}; + +template +struct Add9Functor { + void operator()( + const CPUDevice& d, typename TTypes::Flat out, + typename TTypes::ConstFlat in1, typename TTypes::ConstFlat in2, + typename TTypes::ConstFlat in3, typename TTypes::ConstFlat in4, + typename TTypes::ConstFlat in5, typename TTypes::ConstFlat in6, + typename TTypes::ConstFlat in7, typename TTypes::ConstFlat in8, + typename TTypes::ConstFlat in9) { + Add9EigenImpl::Compute(d, out, in1, in2, in3, in4, in5, in6, + in7, in8, in9); + } +}; + +#ifdef TENSORFLOW_USE_SYCL +// Partial specializations for a SYCLDevice, that uses the Eigen implementation +// from AddNEigenImpl. +template +struct Add2Functor { + void operator()(const SYCLDevice& d, typename TTypes::Flat out, + typename TTypes::ConstFlat in1, + typename TTypes::ConstFlat in2) { + Add2EigenImpl::Compute(d, out, in1, in2); + } +}; +template +struct Add3Functor { + void operator()(const SYCLDevice& d, typename TTypes::Flat out, + typename TTypes::ConstFlat in1, + typename TTypes::ConstFlat in2, + typename TTypes::ConstFlat in3) { + Add3EigenImpl::Compute(d, out, in1, in2, in3); + } +}; +template +struct Add4Functor { + void operator()(const SYCLDevice& d, typename TTypes::Flat out, + typename TTypes::ConstFlat in1, + typename TTypes::ConstFlat in2, + typename TTypes::ConstFlat in3, + typename TTypes::ConstFlat in4) { + Add4EigenImpl::Compute(d, out, in1, in2, in3, in4); + } +}; +template +struct Add5Functor { + void operator()(const SYCLDevice& d, typename TTypes::Flat out, + typename TTypes::ConstFlat in1, + typename TTypes::ConstFlat in2, + typename TTypes::ConstFlat in3, + typename TTypes::ConstFlat in4, + typename TTypes::ConstFlat in5) { + Add5EigenImpl::Compute(d, out, in1, in2, in3, in4, in5); + } +}; +template +struct Add6Functor { + void operator()(const SYCLDevice& d, typename TTypes::Flat out, + typename TTypes::ConstFlat in1, + typename TTypes::ConstFlat in2, + typename TTypes::ConstFlat in3, + typename TTypes::ConstFlat in4, + typename TTypes::ConstFlat in5, + typename TTypes::ConstFlat in6) { + Add6EigenImpl::Compute(d, out, in1, in2, in3, in4, in5, in6); + } +}; +template +struct Add7Functor { + void operator()(const SYCLDevice& d, typename TTypes::Flat out, + typename TTypes::ConstFlat in1, + typename TTypes::ConstFlat in2, + typename TTypes::ConstFlat in3, + typename TTypes::ConstFlat in4, + typename TTypes::ConstFlat in5, + typename TTypes::ConstFlat in6, + typename TTypes::ConstFlat in7) { + Add7EigenImpl::Compute(d, out, in1, in2, in3, in4, in5, in6, + in7); + } +}; + +template +struct Add8Functor { + void operator()( + const SYCLDevice& d, typename TTypes::Flat out, + typename TTypes::ConstFlat in1, typename TTypes::ConstFlat in2, + typename TTypes::ConstFlat in3, typename TTypes::ConstFlat in4, + typename TTypes::ConstFlat in5, typename TTypes::ConstFlat in6, + typename TTypes::ConstFlat in7, typename TTypes::ConstFlat in8) { + Add8EigenImpl::Compute(d, out, in1, in2, in3, in4, in5, in6, + in7, in8); + } +}; + +template +struct Add8pFunctor { + void operator()( + const SYCLDevice& d, typename TTypes::Flat out, + typename TTypes::ConstFlat in1, typename TTypes::ConstFlat in2, + typename TTypes::ConstFlat in3, typename TTypes::ConstFlat in4, + typename TTypes::ConstFlat in5, typename TTypes::ConstFlat in6, + typename TTypes::ConstFlat in7, typename TTypes::ConstFlat in8) { + Add8pEigenImpl::Compute(d, out, in1, in2, in3, in4, in5, in6, + in7, in8); + } +}; + +template +struct Add9Functor { + void operator()( + const SYCLDevice& d, typename TTypes::Flat out, + typename TTypes::ConstFlat in1, typename TTypes::ConstFlat in2, + typename TTypes::ConstFlat in3, typename TTypes::ConstFlat in4, + typename TTypes::ConstFlat in5, typename TTypes::ConstFlat in6, + typename TTypes::ConstFlat in7, typename TTypes::ConstFlat in8, + typename TTypes::ConstFlat in9) { + Add9EigenImpl::Compute(d, out, in1, in2, in3, in4, in5, in6, + in7, in8, in9); + } +}; +#endif // TENSORFLOW_USE_SYCL + +} // namespace functor + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_AGGREGATE_OPS_CPU_H_ diff --git a/linux/include/tensorflow/core/kernels/argmax_op.h b/linux/include/tensorflow/core/kernels/argmax_op.h new file mode 100644 index 000000000..224aa4654 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/argmax_op.h @@ -0,0 +1,68 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_ARGMAX_OP_H_ +#define TENSORFLOW_CORE_KERNELS_ARGMAX_OP_H_ +// Generator definition for ArgMaxOp, must be compilable by nvcc. + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +namespace functor { + +template +struct ArgMax { +#define DECLARE_COMPUTE_SPEC(Dims) \ + EIGEN_ALWAYS_INLINE static void Reduce##Dims( \ + const Device& d, typename TTypes::ConstTensor input, \ + const int32 dimension, typename TTypes::Tensor output) { \ + output.device(d) = input.argmax(dimension).template cast(); \ + } + + DECLARE_COMPUTE_SPEC(1); + DECLARE_COMPUTE_SPEC(2); + DECLARE_COMPUTE_SPEC(3); + DECLARE_COMPUTE_SPEC(4); + DECLARE_COMPUTE_SPEC(5); + +#undef DECLARE_COMPUTE_SPEC +}; + +template +struct ArgMin { +#define DECLARE_COMPUTE_SPEC(Dims) \ + EIGEN_ALWAYS_INLINE static void Reduce##Dims( \ + const Device& d, typename TTypes::ConstTensor input, \ + const int32 dimension, typename TTypes::Tensor output) { \ + output.device(d) = input.argmin(dimension).template cast(); \ + } + + DECLARE_COMPUTE_SPEC(1); + DECLARE_COMPUTE_SPEC(2); + DECLARE_COMPUTE_SPEC(3); + DECLARE_COMPUTE_SPEC(4); + DECLARE_COMPUTE_SPEC(5); + +#undef DECLARE_COMPUTE_SPEC +}; + +} // namespace functor + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_ARGMAX_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/assign_op.h b/linux/include/tensorflow/core/kernels/assign_op.h new file mode 100644 index 000000000..74f926bdc --- /dev/null +++ b/linux/include/tensorflow/core/kernels/assign_op.h @@ -0,0 +1,146 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_ASSIGN_OP_H_ +#define TENSORFLOW_CORE_KERNELS_ASSIGN_OP_H_ + +#define EIGEN_USE_THREADS + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/tensor_types.h" + +namespace tensorflow { + +// TODO(jeff): Get rid of use_exclusive_lock_ option + +// Computes *input[0] = input[1] +class AssignOp : public OpKernel { + public: + explicit AssignOp(OpKernelConstruction* context) : OpKernel(context) { + OP_REQUIRES_OK(context, + context->GetAttr("use_locking", &use_exclusive_lock_)); + OP_REQUIRES_OK(context, + context->GetAttr("validate_shape", &validate_shape_)); + OP_REQUIRES(context, IsRefType(context->input_type(0)), + errors::InvalidArgument("lhs input needs to be a ref type")); + if (!context + ->GetAttr("_grappler_relax_allocator_constraints", + &relax_constraints_) + .ok()) { + relax_constraints_ = false; + } + } + + void Compute(OpKernelContext* context) override { + const Tensor& rhs = context->input(1); + + // We always return the input ref. + context->forward_ref_input_to_ref_output(0, 0); + + // We can't always know how this value will be used downstream, so make + // conservative assumptions in specifying constraints on the memory + // allocation attributes, unless the Grappler graph analysis determined that + // it was safe not to. + AllocatorAttributes attr; + if (!relax_constraints_) { + attr.set_gpu_compatible(true); + attr.set_nic_compatible(true); + } + + { + mutex_lock l(*context->input_ref_mutex(0)); + const Tensor& old_lhs = context->mutable_input(0, /* lock_held */ true); + const bool same_shape = old_lhs.shape().IsSameSize(rhs.shape()); + if (validate_shape_) { + OP_REQUIRES(context, same_shape, + errors::InvalidArgument( + "Assign requires shapes of both tensors to match. " + "lhs shape= ", + old_lhs.shape().DebugString(), + " rhs shape= ", rhs.shape().DebugString())); + } + + // In the code below we try to minimize the amount of memory allocation + // and copying by trying the following two shortcuts: + // 1. If the lhs is initialized and has the same number of elements as + // the rhs we can avoid a memory allocation. + // 2. If we can reuse the rhs buffer we avoid both a memory allocation + // and copying. + + // 1. Try to copy into an existing buffer. + if (old_lhs.IsInitialized() && + old_lhs.shape().num_elements() == rhs.shape().num_elements()) { + // The existing lhs tensor has already been initialized and the right + // hand side can fit in the underlying buffer. + Tensor reshaped_old_lhs; + if (same_shape) { + reshaped_old_lhs = old_lhs; + } else { + CHECK(reshaped_old_lhs.CopyFrom(old_lhs, rhs.shape())); + context->replace_ref_input(0, reshaped_old_lhs, + /* lock_held */ true); + } + if (use_exclusive_lock_) { + Copy(context, &reshaped_old_lhs, rhs); + return; + } + } else { + // 2. Try to reuse the rhs. + std::unique_ptr input_alias = context->forward_input( + 1, OpKernelContext::Params::kNoReservation /*output_index*/, + rhs.dtype(), rhs.shape(), DEVICE_MEMORY, attr); + if (input_alias != nullptr) { + // Update the ref to point to the new buffer. + context->replace_ref_input(0, *input_alias, /* lock_held */ true); + return; + } + + // Otherwise, create a new persistent tensor whose shape matches the + // right hand side, hand off to lhs and copy the rhs into it. + PersistentTensor copy; + Tensor* copyTensor = nullptr; + OP_REQUIRES_OK( + context, context->allocate_persistent(old_lhs.dtype(), rhs.shape(), + ©, ©Tensor, attr)); + // We track memory of variables in variable ops instead of in this + // assign op. + context->clear_recorded_memory(); + context->replace_ref_input(0, *copyTensor, /* lock_held */ true); + if (use_exclusive_lock_) { + Copy(context, copyTensor, rhs); + return; + } + } + } + + // The tensor has already been initialized and the right hand side + // matches the left hand side's shape. We have been told to do the + // copy outside the lock. + Tensor old_unlocked_lhs = context->mutable_input(0, /* lock_held */ false); + Copy(context, &old_unlocked_lhs, rhs); + } + + virtual void Copy(OpKernelContext* context, Tensor* lhs, + const Tensor& rhs) = 0; + + bool use_exclusive_lock_; + bool validate_shape_; + bool relax_constraints_; +}; + +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_ASSIGN_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/avgpooling_op.h b/linux/include/tensorflow/core/kernels/avgpooling_op.h new file mode 100644 index 000000000..1e49a66af --- /dev/null +++ b/linux/include/tensorflow/core/kernels/avgpooling_op.h @@ -0,0 +1,79 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_AVGPOOLING_OP_H_ +#define TENSORFLOW_CORE_KERNELS_AVGPOOLING_OP_H_ +// Functor definition for AvgPoolingOp, must be compilable by nvcc. + +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/kernels/eigen_pooling.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { +namespace functor { + +template +struct SpatialAvgPooling { + void operator()(const Device& d, typename TTypes::Tensor output, + typename TTypes::ConstTensor input, int window_rows, + int window_cols, int row_stride, int col_stride, + const Eigen::PaddingType& padding) { + if (Eigen::internal::is_same::value) { + // Use 32bit indexing to speed up the computations + To32Bit(output).swap_layout().device(d) = Eigen::SpatialAvgPooling( + To32Bit(input).swap_layout(), window_cols, window_rows, col_stride, + row_stride, padding); + } else { + // Because we swap the layout, we swap the row/cols as well + output.swap_layout().device(d) = Eigen::SpatialAvgPooling( + input.swap_layout(), window_cols, window_rows, col_stride, row_stride, + padding); + } + } +}; + +} // namespace functor + +typedef Eigen::GpuDevice GPUDevice; + +// Launch a custom GPU kernels from Yanqing for the avgpooling backward +// operation that works NHWC data formats. Arguments: +// top_diff: backprop to the output of the pooling layer +// num: number of input batches +// height: input height +// width: input width +// channels: number of input channels +// pooled_height: the height of the output to the pooling layer +// pooled_width: the width of the output to the pooling layer +// kernel_h: the height of the pooling kernel +// kernel_w: the width of the pooling kernel +// stride_h: the height of the vertical stride +// stride_w: the width of the horizontal stride +// pad_t: padding size to the top side +// pad_l: padding size to the left side +// bottom_diff: backprop to the input of the pooling layer. +template +bool RunAvePoolBackwardNHWC(const T* const top_diff, const int num, + const int height, const int width, + const int channels, const int pooled_height, + const int pooled_width, const int kernel_h, + const int kernel_w, const int stride_h, + const int stride_w, const int pad_t, + const int pad_l, T* const bottom_diff, + const GPUDevice& d); + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_AVGPOOLING_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/batch_matmul_op_impl.h b/linux/include/tensorflow/core/kernels/batch_matmul_op_impl.h new file mode 100644 index 000000000..766713a33 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/batch_matmul_op_impl.h @@ -0,0 +1,620 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// See docs in ../ops/math_ops.cc. + +#ifndef TENSORFLOW_CORE_KERNELS_BATCH_MATMUL_OP_IMPL_H_ +#define TENSORFLOW_CORE_KERNELS_BATCH_MATMUL_OP_IMPL_H_ + +#define EIGEN_USE_THREADS + +#include +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/op.h" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/register_types.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/type_traits.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/kernels/fill_functor.h" +#include "tensorflow/core/platform/logging.h" +#include "tensorflow/core/platform/types.h" +#include "tensorflow/core/util/work_sharder.h" + +#if GOOGLE_CUDA +#include "tensorflow/core/platform/stream_executor.h" +#endif // GOOGLE_CUDA + +namespace tensorflow { + +typedef Eigen::ThreadPoolDevice CPUDevice; +typedef Eigen::GpuDevice GPUDevice; +#ifdef TENSORFLOW_USE_SYCL +typedef Eigen::SyclDevice SYCLDevice; +#endif // TENSORFLOW_USE_SYCL + +namespace { + +Eigen::IndexPair ContractionDims(bool adj_x, bool adj_y) { + if (!adj_x) { + if (!adj_y) { + return Eigen::IndexPair(1, 0); + } else { + return Eigen::IndexPair(1, 1); + } + } else { + if (!adj_y) { + return Eigen::IndexPair(0, 0); + } else { + return Eigen::IndexPair(0, 1); + } + } +} + +// Parallel batch matmul kernel based on the multi-threaded tensor contraction +// in Eigen. +template +struct ParallelMatMulKernel { + static void Conjugate(const OpKernelContext* context, Tensor* out) { + const Eigen::ThreadPoolDevice d = context->eigen_cpu_device(); + auto z = out->tensor(); + z.device(d) = z.conjugate(); + } + + static void Run(const OpKernelContext* context, const Tensor& in_x, + const Tensor in_y, bool adj_x, bool adj_y, Tensor* out, + int start, int limit) { + static_assert(IsComplex, "Complex type expected."); + auto Tx = in_x.tensor(); + auto Ty = in_y.tensor(); + auto Tz = out->tensor(); + // We use the identities + // conj(a) * conj(b) = conj(a * b) + // conj(a) * b = conj(a * conj(b)) + // to halve the number of cases. The final conjugation of the result is + // done at the end of LaunchBatchMatMul::Launch(). + Eigen::array, 1> contract_pairs; + contract_pairs[0] = ContractionDims(adj_x, adj_y); + const Eigen::ThreadPoolDevice d = context->eigen_cpu_device(); + for (int i = start; i < limit; ++i) { + auto x = Tx.template chip<0>(i); + auto z = Tz.template chip<0>(i); + if (adj_x != adj_y) { + auto y = Ty.template chip<0>(i).conjugate(); + z.device(d) = x.contract(y, contract_pairs); + } else { + auto y = Ty.template chip<0>(i); + z.device(d) = x.contract(y, contract_pairs); + } + } + } +}; + +// The Eigen contraction kernel used here is very large and slow to compile, +// so we partially specialize ParallelMatMulKernel for real types to avoid all +// but one of the instantiations. +template +struct ParallelMatMulKernel { + static void Conjugate(const OpKernelContext* context, Tensor* out) {} + + static void Run(const OpKernelContext* context, const Tensor& in_x, + const Tensor& in_y, bool adj_x, bool adj_y, Tensor* out, + int start, int limit) { + auto Tx = in_x.tensor(); + auto Ty = in_y.tensor(); + auto Tz = out->tensor(); + Eigen::array, 1> contract_pairs; + contract_pairs[0] = ContractionDims(adj_x, adj_y); + const Eigen::ThreadPoolDevice d = context->eigen_cpu_device(); + for (int i = start; i < limit; ++i) { + auto x = Tx.template chip<0>(i); + auto y = Ty.template chip<0>(i); + auto z = Tz.template chip<0>(i); + z.device(d) = x.contract(y, contract_pairs); + } + } +}; + +// TODO(rmlarsen): Get rid of this when we have upstreamed improvements +// for matrix*vector and vector*matrix to Eigen's general matrix product. +template +static void Multiply(bool adj_x, bool adj_y, Tx x, Ty y, Tz z) { + if (!adj_x) { + if (!adj_y) { + z.noalias() = x * y; + } else { + z.noalias() = x * y.adjoint(); + } + } else { + if (!adj_y) { + z.noalias() = x.adjoint() * y; + } else { + z.noalias() = x.adjoint() * y.adjoint(); + } + } +} + +// Sequential batch matmul kernel that calls the regular Eigen matmul. +// We prefer this over the tensor contraction because it performs +// better on vector-matrix and matrix-vector products. +template +struct SequentialMatMulKernel { + using Matrix = + Eigen::Matrix; + using ConstMatrixMap = Eigen::Map; + using MatrixMap = Eigen::Map; + + static ConstMatrixMap ConstTensorSliceToEigenMatrix(const Tensor& t, + int slice) { + return ConstMatrixMap( + t.flat().data() + slice * t.dim_size(1) * t.dim_size(2), + t.dim_size(1), t.dim_size(2)); + } + + static MatrixMap TensorSliceToEigenMatrix(Tensor* t, int slice) { + return MatrixMap( + t->flat().data() + slice * t->dim_size(1) * t->dim_size(2), + t->dim_size(1), t->dim_size(2)); + } + + static void Run(const Tensor& in_x, const Tensor& in_y, bool adj_x, + bool adj_y, Tensor* out, int start, int limit) { + for (int i = start; i < limit; ++i) { + auto x = ConstTensorSliceToEigenMatrix(in_x, i); + auto y = ConstTensorSliceToEigenMatrix(in_y, i); + auto z = TensorSliceToEigenMatrix(out, i); + // TODO(rmlarsen): Get rid of the special casing here when we have + // upstreamed improvements for matrix*vector and vector*matrix to + // Eigen's general matrix product. + if (!adj_x && x.rows() == 1) { + Multiply(adj_x, adj_y, x.row(0), y, z); + } else if (adj_x && x.cols() == 1) { + Multiply(adj_x, adj_y, x.col(0), y, z); + } else if (!adj_y && y.cols() == 1) { + Multiply(adj_x, adj_y, x, y.col(0), z); + } else if (adj_y && y.rows() == 1) { + Multiply(adj_x, adj_y, x, y.row(0), z); + } else { + Multiply(adj_x, adj_y, x, y, z); + } + } + } +}; + +} // namespace + +template +struct LaunchBatchMatMul; + +template +struct LaunchBatchMatMul { + static void Launch(OpKernelContext* context, const Tensor& in_x, + const Tensor& in_y, bool adj_x, bool adj_y, Tensor* out) { + typedef ParallelMatMulKernel::IsComplex> + ParallelMatMulKernel; + bool conjugate_result = false; + + // Number of matrix multiplies i.e. size of the batch. + const int64 batch_size = in_x.dim_size(0); + const int64 cost_per_unit = + in_x.dim_size(1) * in_x.dim_size(2) * out->dim_size(2); + const int64 small_dim = std::min( + std::min(in_x.dim_size(1), in_x.dim_size(2)), out->dim_size(2)); + const int64 kMaxCostOuterParallelism = 128 * 128 * 256; // heuristic. + auto worker_threads = *(context->device()->tensorflow_cpu_worker_threads()); + if (small_dim > 1 && + (batch_size == 1 || cost_per_unit > kMaxCostOuterParallelism)) { + // Parallelize over inner dims. + // For large matrix products it is counter-productive to parallelize + // over the batch dimension. + ParallelMatMulKernel::Run(context, in_x, in_y, adj_x, adj_y, out, 0, + batch_size); + conjugate_result = adj_x; + } else { + // Parallelize over outer dims. For small matrices and large batches, it + // is counter-productive to parallelize the inner matrix multiplies. + Shard(worker_threads.num_threads, worker_threads.workers, batch_size, + cost_per_unit, + [&in_x, &in_y, adj_x, adj_y, out](int start, int limit) { + SequentialMatMulKernel::Run(in_x, in_y, adj_x, adj_y, out, + start, limit); + }); + } + if (conjugate_result) { + // We used one of the identities + // conj(a) * conj(b) = conj(a * b) + // conj(a) * b = conj(a * conj(b)) + // above, we need to conjugate the final output. This is a + // no-op for non-complex types. + ParallelMatMulKernel::Conjugate(context, out); + } + } +}; + +#if GOOGLE_CUDA + +namespace { +template +se::DeviceMemory AsDeviceMemory(const T* cuda_memory) { + se::DeviceMemoryBase wrapped(const_cast(cuda_memory)); + se::DeviceMemory typed(wrapped); + return typed; +} + +class CublasScratchAllocator : public se::ScratchAllocator { + public: + using Stream = se::Stream; + using DeviceMemoryBytes = se::DeviceMemory; + + CublasScratchAllocator(OpKernelContext* context) : context_(context) {} + + int64 GetMemoryLimitInBytes(Stream* stream) override { return -1; } + + se::port::StatusOr AllocateBytes( + Stream* stream, int64 byte_size) override { + Tensor temporary_memory; + + Status allocation_status(context_->allocate_temp( + DT_UINT8, TensorShape({byte_size}), &temporary_memory)); + if (!allocation_status.ok()) { + return se::port::StatusOr( + DeviceMemoryBytes::MakeFromByteSize(nullptr, 0)); + } + // Hold the reference of the allocated tensors until the end of the + // allocator. + allocated_tensors_.push_back(temporary_memory); + return se::port::StatusOr( + DeviceMemoryBytes::MakeFromByteSize( + temporary_memory.flat().data(), + temporary_memory.flat().size())); + } + + private: + OpKernelContext* context_; + std::vector allocated_tensors_; +}; +} // namespace + +template +struct LaunchBatchMatMul { + static void Launch(OpKernelContext* context, const Tensor& in_x, + const Tensor& in_y, bool adj_x, bool adj_y, Tensor* out) { + constexpr se::blas::Transpose kTranspose = + is_complex::value ? se::blas::Transpose::kConjugateTranspose + : se::blas::Transpose::kTranspose; + se::blas::Transpose trans[] = {se::blas::Transpose::kNoTranspose, + kTranspose}; + const uint64 m = in_x.dim_size(adj_x ? 2 : 1); + const uint64 k = in_x.dim_size(adj_x ? 1 : 2); + const uint64 n = in_y.dim_size(adj_y ? 1 : 2); + const uint64 batch_size = in_x.dim_size(0); + auto blas_transpose_a = trans[adj_x]; + auto blas_transpose_b = trans[adj_y]; + + auto* stream = context->op_device_context()->stream(); + OP_REQUIRES(context, stream, errors::Internal("No GPU stream available.")); + + typedef se::DeviceMemory DeviceMemoryType; + std::vector a_device_memory; + std::vector b_device_memory; + std::vector c_device_memory; + std::vector a_ptrs; + std::vector b_ptrs; + std::vector c_ptrs; + a_device_memory.reserve(batch_size); + b_device_memory.reserve(batch_size); + c_device_memory.reserve(batch_size); + a_ptrs.reserve(batch_size); + b_ptrs.reserve(batch_size); + c_ptrs.reserve(batch_size); + auto* a_base_ptr = in_x.template flat().data(); + auto* b_base_ptr = in_y.template flat().data(); + auto* c_base_ptr = out->template flat().data(); + for (int64 i = 0; i < batch_size; ++i) { + a_device_memory.push_back(AsDeviceMemory(a_base_ptr + i * m * k)); + b_device_memory.push_back(AsDeviceMemory(b_base_ptr + i * k * n)); + c_device_memory.push_back(AsDeviceMemory(c_base_ptr + i * m * n)); + a_ptrs.push_back(&a_device_memory.back()); + b_ptrs.push_back(&b_device_memory.back()); + c_ptrs.push_back(&c_device_memory.back()); + } + + typedef Scalar Coefficient; + + // Cublas does + // C = A x B + // where A, B and C are assumed to be in column major. + // We want the output to be in row-major, so we can compute + // C' = B' x A', where ' stands for transpose (not adjoint). + // TODO(yangzihao): Choose the best of the three strategies using autotune. + if (batch_size == 1) { + // This is a regular matrix*matrix or matrix*vector multiply. Avoid the + // overhead of the scratch allocator and the batch interface. + if (n == 1 && + blas_transpose_b != se::blas::Transpose::kConjugateTranspose && + blas_transpose_a != se::blas::Transpose::kConjugateTranspose) { + // This is a matrix*vector multiply so use GEMV to compute A * b. + // Here we are multiplying in the natural order, so we have to flip + // the transposition flag to compensate for the tensor being stored + // row-major. Since GEMV doesn't provide a way to just conjugate an + // argument, we have to defer those cases to GEMM below. + auto gemv_trans_a = blas_transpose_a == se::blas::Transpose::kTranspose + ? se::blas::Transpose::kNoTranspose + : se::blas::Transpose::kTranspose; + bool blas_launch_status = + stream + ->ThenBlasGemv(gemv_trans_a, adj_x ? m : k, adj_x ? k : m, + static_cast(1.0), *(a_ptrs[0]), + adj_x ? m : k, *(b_ptrs[0]), 1, + static_cast(0.0), c_ptrs[0], 1) + .ok(); + if (!blas_launch_status) { + context->SetStatus(errors::Internal( + "Blas xGEMV launch failed : a.shape=", in_x.shape().DebugString(), + ", b.shape=", in_y.shape().DebugString(), ", m=", m, ", n=", n, + ", k=", k)); + } + } else { + bool blas_launch_status = + stream + ->ThenBlasGemm(blas_transpose_b, blas_transpose_a, n, m, k, + static_cast(1.0), *(b_ptrs[0]), + adj_y ? k : n, *(a_ptrs[0]), adj_x ? m : k, + static_cast(0.0), c_ptrs[0], n) + .ok(); + if (!blas_launch_status) { + context->SetStatus(errors::Internal( + "Blas xGEMM launch failed : a.shape=", in_x.shape().DebugString(), + ", b.shape=", in_y.shape().DebugString(), ", m=", m, ", n=", n, + ", k=", k)); + } + } + } else { + CublasScratchAllocator scratch_allocator(context); + bool blas_launch_status = + stream + ->ThenBlasGemmBatchedWithScratch( + blas_transpose_b, blas_transpose_a, n, m, k, + static_cast(1.0), b_ptrs, adj_y ? k : n, a_ptrs, + adj_x ? m : k, static_cast(0.0), c_ptrs, n, + batch_size, &scratch_allocator) + .ok(); + if (!blas_launch_status) { + context->SetStatus(errors::Internal( + "Blas xGEMMBatched launch failed : a.shape=", + in_x.shape().DebugString(), + ", b.shape=", in_y.shape().DebugString(), ", m=", m, ", n=", n, + ", k=", k, ", batch_size=", batch_size)); + } + } + } +}; + +template <> +struct LaunchBatchMatMul { + static void Launch(OpKernelContext* context, const Tensor& in_x, + const Tensor& in_y, bool adj_x, bool adj_y, Tensor* out) { + typedef Eigen::half Scalar; + constexpr perftools::gputools::blas::Transpose kTranspose = + is_complex::value + ? perftools::gputools::blas::Transpose::kConjugateTranspose + : perftools::gputools::blas::Transpose::kTranspose; + perftools::gputools::blas::Transpose trans[] = { + perftools::gputools::blas::Transpose::kNoTranspose, kTranspose}; + const uint64 m = in_x.dim_size(adj_x ? 2 : 1); + const uint64 k = in_x.dim_size(adj_x ? 1 : 2); + const uint64 n = in_y.dim_size(adj_y ? 1 : 2); + const uint64 batch_size = in_x.dim_size(0); + auto blas_transpose_a = trans[adj_x]; + auto blas_transpose_b = trans[adj_y]; + + auto* stream = context->op_device_context()->stream(); + OP_REQUIRES(context, stream, errors::Internal("No GPU stream available.")); + + typedef perftools::gputools::DeviceMemory DeviceMemoryType; + std::vector a_device_memory; + std::vector b_device_memory; + std::vector c_device_memory; + std::vector a_ptrs; + std::vector b_ptrs; + std::vector c_ptrs; + a_device_memory.reserve(batch_size); + b_device_memory.reserve(batch_size); + c_device_memory.reserve(batch_size); + a_ptrs.reserve(batch_size); + b_ptrs.reserve(batch_size); + c_ptrs.reserve(batch_size); + auto* a_base_ptr = in_x.template flat().data(); + auto* b_base_ptr = in_y.template flat().data(); + auto* c_base_ptr = out->template flat().data(); + for (int64 i = 0; i < batch_size; ++i) { + a_device_memory.push_back(AsDeviceMemory(a_base_ptr + i * m * k)); + b_device_memory.push_back(AsDeviceMemory(b_base_ptr + i * k * n)); + c_device_memory.push_back(AsDeviceMemory(c_base_ptr + i * m * n)); + a_ptrs.push_back(&a_device_memory.back()); + b_ptrs.push_back(&b_device_memory.back()); + c_ptrs.push_back(&c_device_memory.back()); + } + + typedef float Coefficient; + + // Cublas does + // C = A x B + // where A, B and C are assumed to be in column major. + // We want the output to be in row-major, so we can compute + // C' = B' x A', where ' stands for transpose (not adjoint). + // TODO(yangzihao): Choose the best of the three strategies using autotune. + if (batch_size == 1) { + // This is a regular matrix*matrix or matrix*vector multiply. Avoid the + // overhead of the scratch allocator and the batch interface. + // TODO(benbarsdell): Use fp16 Gemv if it becomes supported by CUBLAS + bool blas_launch_status = + stream + ->ThenBlasGemm(blas_transpose_b, blas_transpose_a, n, m, k, + static_cast(1.0), *(b_ptrs[0]), + adj_y ? k : n, *(a_ptrs[0]), adj_x ? m : k, + static_cast(0.0), c_ptrs[0], n) + .ok(); + if (!blas_launch_status) { + context->SetStatus(errors::Internal( + "Blas xGEMM launch failed : a.shape=", in_x.shape().DebugString(), + ", b.shape=", in_y.shape().DebugString(), ", m=", m, ", n=", n, + ", k=", k)); + } + } else { + CublasScratchAllocator scratch_allocator(context); + bool blas_launch_status = + stream + ->ThenBlasGemmBatchedWithScratch( + blas_transpose_b, blas_transpose_a, n, m, k, + static_cast(1.0), b_ptrs, adj_y ? k : n, a_ptrs, + adj_x ? m : k, static_cast(0.0), c_ptrs, n, + batch_size, &scratch_allocator) + .ok(); + if (!blas_launch_status) { + context->SetStatus( + errors::Internal("Blas xGEMMBatched launch failed : a.shape=", + in_x.shape().DebugString(), ", b.shape=", + in_y.shape().DebugString(), ", m=", m, ", n=", n, + ", k=", k, ", batch_size=", batch_size)); + } + } + } +}; + +#endif // GOOGLE_CUDA + +#ifdef TENSORFLOW_USE_SYCL +template +struct ParallelMatMulKernelSYCL { + static void Run(const OpKernelContext* context, const Tensor& in_x, + const Tensor& in_y, bool adj_x, bool adj_y, Tensor* out, + int start, int limit) { + auto Tx = in_x.tensor(); + auto Ty = in_y.tensor(); + auto Tz = out->tensor(); + Eigen::array, 1> contract_pairs; + contract_pairs[0] = ContractionDims(adj_x, adj_y); + auto d = context->eigen_sycl_device(); + for (int i = start; i < limit; ++i) { + auto x = Tx.template chip<0>(i); + auto y = Ty.template chip<0>(i); + auto z = Tz.template chip<0>(i); + z.device(d) = x.contract(y, contract_pairs); + } + } +}; + +template +struct LaunchBatchMatMul { + static void Launch(OpKernelContext* context, const Tensor& in_x, + const Tensor& in_y, bool adj_x, bool adj_y, Tensor* out) { + // Number of matrix multiplies i.e. size of the batch. + const int64 batch_size = in_x.dim_size(0); + ParallelMatMulKernelSYCL::Run(context, in_x, in_y, adj_x, adj_y, + out, 0, batch_size); + } +}; +#endif // TENSORFLOW_USE_SYCL + +template +class BatchMatMul : public OpKernel { + public: + explicit BatchMatMul(OpKernelConstruction* context) : OpKernel(context) { + OP_REQUIRES_OK(context, context->GetAttr("adj_x", &adj_x_)); + OP_REQUIRES_OK(context, context->GetAttr("adj_y", &adj_y_)); + } + + virtual ~BatchMatMul() {} + + void Compute(OpKernelContext* ctx) override { + const Tensor& in0 = ctx->input(0); + const Tensor& in1 = ctx->input(1); + OP_REQUIRES(ctx, in0.dims() == in1.dims(), + errors::InvalidArgument("In[0] and In[1] has different ndims: ", + in0.shape().DebugString(), " vs. ", + in1.shape().DebugString())); + const int ndims = in0.dims(); + OP_REQUIRES( + ctx, ndims >= 2, + errors::InvalidArgument("In[0] and In[1] ndims must be >= 2: ", ndims)); + TensorShape out_shape; + for (int i = 0; i < ndims - 2; ++i) { + OP_REQUIRES(ctx, in0.dim_size(i) == in1.dim_size(i), + errors::InvalidArgument( + "In[0].dim(", i, ") and In[1].dim(", i, + ") must be the same: ", in0.shape().DebugString(), " vs ", + in1.shape().DebugString())); + out_shape.AddDim(in0.dim_size(i)); + } + auto n = (ndims == 2) ? 1 : out_shape.num_elements(); + auto d0 = in0.dim_size(ndims - 2); + auto d1 = in0.dim_size(ndims - 1); + Tensor in0_reshaped; + CHECK(in0_reshaped.CopyFrom(in0, TensorShape({n, d0, d1}))); + auto d2 = in1.dim_size(ndims - 2); + auto d3 = in1.dim_size(ndims - 1); + Tensor in1_reshaped; + CHECK(in1_reshaped.CopyFrom(in1, TensorShape({n, d2, d3}))); + if (adj_x_) std::swap(d0, d1); + if (adj_y_) std::swap(d2, d3); + OP_REQUIRES(ctx, d1 == d2, + errors::InvalidArgument( + "In[0] mismatch In[1] shape: ", d1, " vs. ", d2, ": ", + in0.shape().DebugString(), " ", in1.shape().DebugString(), + " ", adj_x_, " ", adj_y_)); + out_shape.AddDim(d0); + out_shape.AddDim(d3); + Tensor* out = nullptr; + OP_REQUIRES_OK(ctx, ctx->allocate_output(0, out_shape, &out)); + if (out->NumElements() == 0) { + return; + } + if (in0.NumElements() == 0 || in1.NumElements() == 0) { + functor::SetZeroFunctor f; + f(ctx->eigen_device(), out->flat()); + return; + } + Tensor out_reshaped; + CHECK(out_reshaped.CopyFrom(*out, TensorShape({n, d0, d3}))); + LaunchBatchMatMul::Launch(ctx, in0_reshaped, in1_reshaped, + adj_x_, adj_y_, &out_reshaped); + } + + private: + bool adj_x_; + bool adj_y_; +}; + +#define REGISTER_BATCH_MATMUL_CPU(TYPE) \ + REGISTER_KERNEL_BUILDER( \ + Name("BatchMatMul").Device(DEVICE_CPU).TypeConstraint("T"), \ + BatchMatMul) + +#define REGISTER_BATCH_MATMUL_GPU(TYPE) \ + REGISTER_KERNEL_BUILDER( \ + Name("BatchMatMul").Device(DEVICE_GPU).TypeConstraint("T"), \ + BatchMatMul) + +#ifdef TENSORFLOW_USE_SYCL +#define REGISTER_BATCH_MATMUL_SYCL(TYPE) \ + REGISTER_KERNEL_BUILDER( \ + Name("BatchMatMul").Device(DEVICE_SYCL).TypeConstraint("T"), \ + BatchMatMul) +#endif // TENSORFLOW_USE_SYCL +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_BATCH_MATMUL_OP_IMPL_H_ diff --git a/linux/include/tensorflow/core/kernels/batch_norm_op.h b/linux/include/tensorflow/core/kernels/batch_norm_op.h new file mode 100644 index 000000000..76b156f8f --- /dev/null +++ b/linux/include/tensorflow/core/kernels/batch_norm_op.h @@ -0,0 +1,156 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_BATCH_NORM_OP_H_ +#define TENSORFLOW_CORE_KERNELS_BATCH_NORM_OP_H_ +// Functor definition for BatchNormOp, must be compilable by nvcc. +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/tensor_types.h" + +namespace tensorflow { +namespace functor { + +// Functor used by BatchNormOp to do the computations. +template +struct BatchNorm { + void operator()(const Device& d, typename TTypes::ConstTensor input, + typename TTypes::ConstVec mean, + typename TTypes::ConstVec var, + typename TTypes::ConstVec beta, + typename TTypes::ConstVec gamma, T variance_epsilon, + bool scale_after_normalization, + typename TTypes::Tensor output) { + const int depth = mean.dimension(0); + const int rest_size = input.size() / depth; + + Eigen::DSizes rest_by_depth(rest_size, depth); +#if !defined(EIGEN_HAS_INDEX_LIST) + Eigen::DSizes rest_by_one(rest_size, 1); + Eigen::DSizes one_by_depth(1, depth); + Eigen::DSizes depth_by_one(depth, 1); +#else + Eigen::IndexList > rest_by_one; + rest_by_one.set(0, rest_size); + Eigen::IndexList, int> one_by_depth; + one_by_depth.set(1, depth); + Eigen::IndexList > depth_by_one; + depth_by_one.set(0, depth); +#endif + if (scale_after_normalization) { + output.reshape(rest_by_depth).device(d) = + (input.reshape(rest_by_depth) - + mean.reshape(one_by_depth).broadcast(rest_by_one)) * + ((var + var.constant(variance_epsilon)).rsqrt() * gamma) + .eval() + .reshape(one_by_depth) + .broadcast(rest_by_one) + + beta.reshape(one_by_depth).broadcast(rest_by_one); + } else { + output.reshape(rest_by_depth).device(d) = + (input.reshape(rest_by_depth) - + mean.reshape(one_by_depth).broadcast(rest_by_one)) * + ((var + var.constant(variance_epsilon)).rsqrt()) + .eval() + .reshape(one_by_depth) + .broadcast(rest_by_one) + + beta.reshape(one_by_depth).broadcast(rest_by_one); + } + } +}; + +template +struct BatchNormGrad { + void operator()(const Device& d, typename TTypes::ConstTensor input, + typename TTypes::ConstVec mean, + typename TTypes::ConstVec var, + typename TTypes::ConstVec gamma, + typename TTypes::ConstTensor out_backprop, + T variance_epsilon, bool scale_after_normalization, + typename TTypes::Tensor dx, typename TTypes::Vec dm, + typename TTypes::Vec dv, typename TTypes::Vec db, + typename TTypes::Vec dg, typename TTypes::Vec scratch1, + typename TTypes::Vec scratch2) { + const int depth = mean.dimension(0); + const int rest_size = input.size() / depth; + + typedef typename TTypes::ConstVec::Index Index; + + Eigen::DSizes rest_by_depth(rest_size, depth); +#if !defined(EIGEN_HAS_INDEX_LIST) + Eigen::DSizes rest_by_one(rest_size, 1); + Eigen::DSizes one_by_depth(1, depth); + Eigen::array reduction_axis; + reduction_axis[0] = 0; // Reduces on first dimension. +#else + Eigen::IndexList > rest_by_one; + rest_by_one.set(0, rest_size); + Eigen::IndexList, Index> one_by_depth; + one_by_depth.set(1, depth); + Eigen::IndexList > reduction_axis; +#endif + + // db = out_backprop + // + // dg = out_backprop * ((x - m) * rsqrt(v + epsilon)) + // + // dv = sum_over_rest(out_backprop * gamma * (x - m)) * + // (-1/2) * (v + epsilon) ^ (-3/2) + // + // dm = sum_over_rest(out_backprop * gamma) * (-1 / rsqrt(v + epsilon)) + // + // dx = out_backprop * (gamma * rsqrt(v + epsilon)) + db.device(d) = out_backprop.reshape(rest_by_depth).sum(reduction_axis); + + // scratch1 = rsqrt(v + epsilon) + scratch1.device(d) = (var + var.constant(variance_epsilon)).rsqrt(); + + // scratch2 = sum_over_rest(out_backprop * (x - m)) + scratch2.device(d) = (out_backprop.reshape(rest_by_depth) * + (input.reshape(rest_by_depth) - + mean.reshape(one_by_depth).broadcast(rest_by_one))) + .sum(reduction_axis); + + if (scale_after_normalization) { + dx.reshape(rest_by_depth).device(d) = + out_backprop.reshape(rest_by_depth) * ((scratch1 * gamma) + .eval() + .reshape(one_by_depth) + .broadcast(rest_by_one)); + dm.device(d) = -db * (scratch1 * gamma).eval(); + dg.device(d) = scratch2 * scratch1; + } else { + dx.reshape(rest_by_depth).device(d) = + out_backprop.reshape(rest_by_depth) * + scratch1.reshape(one_by_depth).broadcast(rest_by_one); + dm.device(d) = -db * scratch1; + dg.device(d) = dg.constant(static_cast(0.0)); // Gamma is not learned. + } + + // scratch1 = - 1/2 * (var + epsilon) ^ (-3/2) + scratch1.device(d) = scratch1 * scratch1.constant(static_cast(-0.5f)) / + (var + var.constant(variance_epsilon)); + + if (scale_after_normalization) { + dv.device(d) = scratch2 * (scratch1 * gamma).eval(); + } else { + dv.device(d) = scratch2 * scratch1; + } + } +}; + +} // namespace functor +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_BATCH_NORM_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/batching_util/batch_scheduler.h b/linux/include/tensorflow/core/kernels/batching_util/batch_scheduler.h new file mode 100644 index 000000000..f6d9a8f0c --- /dev/null +++ b/linux/include/tensorflow/core/kernels/batching_util/batch_scheduler.h @@ -0,0 +1,281 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// Abstractions for processing small tasks in a batched fashion, to reduce +// processing times and costs that can be amortized across multiple tasks. +// +// The core class is BatchScheduler, which groups tasks into batches. +// +// BatchScheduler encapsulates logic for aggregating multiple tasks into a +// batch, and kicking off processing of a batch on a thread pool it manages. +// +// This file defines an abstract BatchScheduler class. + +#ifndef TENSORFLOW_CORE_KERNELS_BATCHING_UTIL_BATCH_SCHEDULER_H_ +#define TENSORFLOW_CORE_KERNELS_BATCHING_UTIL_BATCH_SCHEDULER_H_ + +#include +#include +#include +#include +#include +#include + +#include "tensorflow/core/lib/core/notification.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/platform/logging.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/mutex.h" +#include "tensorflow/core/platform/thread_annotations.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { +namespace serving { + +// The abstract superclass for a unit of work to be done as part of a batch. +// +// An implementing subclass typically contains (or points to): +// (a) input data; +// (b) a thread-safe completion signal (e.g. a Notification); +// (c) a place to store the outcome (success, or some error), upon completion; +// (d) a place to store the output data, upon success. +// +// Items (b), (c) and (d) are typically non-owned pointers to data homed +// elsewhere, because a task's ownership gets transferred to a BatchScheduler +// (see below) and it may be deleted as soon as it is done executing. +class BatchTask { + public: + virtual ~BatchTask() = default; + + // Returns the size of the task, in terms of how much it contributes to the + // size of a batch. (A batch's size is the sum of its task sizes.) + virtual size_t size() const = 0; +}; + +// A thread-safe collection of BatchTasks, to be executed together in some +// fashion. +// +// At a given time, a batch is either "open" or "closed": an open batch can +// accept new tasks; a closed one cannot. A batch is monotonic: initially it is +// open and tasks can be added to it; then it is closed and its set of tasks +// remains fixed for the remainder of its life. A closed batch cannot be re- +// opened. Tasks can never be removed from a batch. +// +// Type parameter TaskType must be a subclass of BatchTask. +template +class Batch { + public: + Batch() = default; + virtual ~Batch(); // Blocks until the batch is closed. + + // Appends 'task' to the batch. After calling AddTask(), the newly-added task + // can be accessed via task(num_tasks()-1) or mutable_task(num_tasks()-1). + // Dies if the batch is closed. + void AddTask(std::unique_ptr task); + + // Removes the most recently added task. Returns nullptr if the batch is + // empty. + std::unique_ptr RemoveTask(); + + // Returns the number of tasks in the batch. + int num_tasks() const; + + // Returns true iff the batch contains 0 tasks. + bool empty() const; + + // Returns a reference to the ith task (in terms of insertion order). + const TaskType& task(int i) const; + + // Returns a pointer to the ith task (in terms of insertion order). + TaskType* mutable_task(int i); + + // Returns the sum of the task sizes. + size_t size() const; + + // Returns true iff the batch is currently closed. + bool IsClosed() const; + + // Blocks until the batch is closed. + void WaitUntilClosed() const; + + // Marks the batch as closed. Dies if called more than once. + void Close(); + + private: + mutable mutex mu_; + + // The tasks in the batch. + std::vector> tasks_ GUARDED_BY(mu_); + + // The sum of the sizes of the tasks in 'tasks_'. + size_t size_ GUARDED_BY(mu_) = 0; + + // Whether the batch has been closed. + Notification closed_; + + TF_DISALLOW_COPY_AND_ASSIGN(Batch); +}; + +// An abstract batch scheduler class. Collects individual tasks into batches, +// and processes each batch on a pool of "batch threads" that it manages. The +// actual logic for processing a batch is accomplished via a callback. +// +// Type parameter TaskType must be a subclass of BatchTask. +template +class BatchScheduler { + public: + virtual ~BatchScheduler() = default; + + // Submits a task to be processed as part of a batch. + // + // Ownership of '*task' is transferred to the callee iff the method returns + // Status::OK. In that case, '*task' is left as nullptr. Otherwise, '*task' is + // left as-is. + // + // If no batch processing capacity is available to process this task at the + // present time, and any task queue maintained by the implementing subclass is + // full, this method returns an UNAVAILABLE error code. The client may retry + // later. + // + // Other problems, such as the task size being larger than the maximum batch + // size, yield other, permanent error types. + // + // In all cases, this method returns "quickly" without blocking for any + // substantial amount of time. If the method returns Status::OK, the task is + // processed asynchronously, and any errors that occur during the processing + // of the batch that includes the task can be reported to 'task'. + virtual Status Schedule(std::unique_ptr* task) = 0; + + // Returns the number of tasks that have been scheduled (i.e. accepted by + // Schedule()), but have yet to be handed to a thread for execution as part of + // a batch. Note that this returns the number of tasks, not the aggregate task + // size (so if there is one task of size 3 and one task of size 5, this method + // returns 2 rather than 8). + virtual size_t NumEnqueuedTasks() const = 0; + + // Returns a guaranteed number of size 1 tasks that can be Schedule()d without + // getting an UNAVAILABLE error. In a typical implementation, returns the + // available space on a queue. + // + // There are two important caveats: + // 1. The guarantee does not extend to varying-size tasks due to possible + // internal fragmentation of batches. + // 2. The guarantee only holds in a single-thread environment or critical + // section, i.e. if an intervening thread cannot call Schedule(). + // + // This method is useful for monitoring, or for guaranteeing a future slot in + // the schedule (but being mindful about the caveats listed above). + virtual size_t SchedulingCapacity() const = 0; + + // Returns the maximum allowed size of tasks submitted to the scheduler. (This + // is typically equal to a configured maximum batch size.) + virtual size_t max_task_size() const = 0; +}; + +////////// +// Implementation details follow. API users need not read. + +template +Batch::~Batch() { + WaitUntilClosed(); +} + +template +void Batch::AddTask(std::unique_ptr task) { + DCHECK(!IsClosed()); + { + mutex_lock l(mu_); + size_ += task->size(); + tasks_.push_back(std::move(task)); + } +} + +template +std::unique_ptr Batch::RemoveTask() { + { + mutex_lock l(mu_); + if (tasks_.empty()) { + return nullptr; + } + std::unique_ptr task = std::move(tasks_.back()); + size_ -= task->size(); + tasks_.pop_back(); + return task; + } +} + +template +int Batch::num_tasks() const { + { + mutex_lock l(mu_); + return tasks_.size(); + } +} + +template +bool Batch::empty() const { + { + mutex_lock l(mu_); + return tasks_.empty(); + } +} + +template +const TaskType& Batch::task(int i) const { + DCHECK_GE(i, 0); + { + mutex_lock l(mu_); + DCHECK_LT(i, tasks_.size()); + return *tasks_[i].get(); + } +} + +template +TaskType* Batch::mutable_task(int i) { + DCHECK_GE(i, 0); + { + mutex_lock l(mu_); + DCHECK_LT(i, tasks_.size()); + return tasks_[i].get(); + } +} + +template +size_t Batch::size() const { + { + mutex_lock l(mu_); + return size_; + } +} + +template +bool Batch::IsClosed() const { + return const_cast(&closed_)->HasBeenNotified(); +} + +template +void Batch::WaitUntilClosed() const { + const_cast(&closed_)->WaitForNotification(); +} + +template +void Batch::Close() { + closed_.Notify(); +} + +} // namespace serving +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_BATCHING_UTIL_BATCH_SCHEDULER_H_ diff --git a/linux/include/tensorflow/core/kernels/batching_util/periodic_function.h b/linux/include/tensorflow/core/kernels/batching_util/periodic_function.h new file mode 100644 index 000000000..36a401900 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/batching_util/periodic_function.h @@ -0,0 +1,134 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// PeriodicFunction will periodically call the given function with a specified +// period in a background thread. After Start() returns, the thread is +// guaranteed to have started. The destruction of the class causes the +// background thread to be destroyed as well. Start() should not be called more +// than once. +// +// PeriodicFunction runs the function as soon as any previous run both is +// complete and was started more than "interval_micros" earlier. Thus, runs are +// both serialized, and normally have a period of "interval_micros" if no run +// exceeds the time. +// +// Note that, if the function takes longer than two interval_micross to finish, +// then PeriodicFunction will "skip" at least one call to the function. For +// instance, if the period is 50ms and the function starts runs at time 0 for +// 150ms, then the function will immediately start executing again at time 150, +// but there will be no function runs corresponding to times 50 or 100. This is +// especially important to remember when using an environment with a simulated +// clock: advancing simulated time atomically over N interval_micross will not +// cause the function to be called N times. +// +// This object is thread-safe. +// +// Example: +// +// class Foo { +// public: +// Foo() : periodic_function_([this]() { Bar(); }, +// 1000 /* 1000us == 1ms*/) { +// } +// +// private: +// void Bar() { ... } +// +// PeriodicFunction periodic_function_; +// }; + +#ifndef TENSORFLOW_CORE_KERNELS_BATCHING_UTIL_PERIODIC_FUNCTION_H_ +#define TENSORFLOW_CORE_KERNELS_BATCHING_UTIL_PERIODIC_FUNCTION_H_ + +#include "tensorflow/core/kernels/batching_util/periodic_function.h" + +#include +#include +#include + +#include "tensorflow/core/lib/core/notification.h" +#include "tensorflow/core/platform/env.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/mutex.h" +#include "tensorflow/core/platform/thread_annotations.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { +namespace serving { + +namespace internal { +class PeriodicFunctionTestAccess; +} + +class PeriodicFunction { + public: + // Provides the ability to customize several aspects of the PeriodicFunction. + // Passed to constructor of PeriodicFunction. + struct Options { + Options() {} + + // Any standard thread options, such as stack size, should + // be passed via "thread_options". + ThreadOptions thread_options; + + // Specifies the thread name prefix (see the description in class + // Thread). + string thread_name_prefix = "periodic_function"; + + // The environment to use. Does not take ownership, but must remain alive + // for as long as the PeriodicFunction exists. + Env* env = Env::Default(); + + // Specifies the length of sleep before the first invocation of the + // function. + // This can be used for adding a random jitter to avoid synchronous behavior + // across multiple periodic functions. + int64 startup_delay_micros = 0; + }; + + // Also starts the background thread which will be calling the function. + PeriodicFunction(const std::function& function, int64 interval_micros, + const Options& options = Options()); + + ~PeriodicFunction(); + + private: + friend class internal::PeriodicFunctionTestAccess; + + // Notifies the background thread to stop. + void NotifyStop(); + + // (Blocking.) Loops forever calling "function_" every "interval_micros_". + void RunLoop(int64 start) LOCKS_EXCLUDED(mutex_); + + const std::function function_; // Actual client function + const int64 interval_micros_; // Interval between calls. + const Options options_; + + // Protects state below. + mutable mutex mutex_; + // Used to notify the thread to stop. + Notification stop_thread_; + + // Thread for running "function_" + std::unique_ptr thread_ GUARDED_BY(mutex_) = nullptr; + + TF_DISALLOW_COPY_AND_ASSIGN(PeriodicFunction); +}; + +} // namespace serving +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_BATCHING_UTIL_PERIODIC_FUNCTION_H_ diff --git a/linux/include/tensorflow/core/kernels/batching_util/shared_batch_scheduler.h b/linux/include/tensorflow/core/kernels/batching_util/shared_batch_scheduler.h new file mode 100644 index 000000000..b4bce9084 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/batching_util/shared_batch_scheduler.h @@ -0,0 +1,706 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_BATCHING_UTIL_SHARED_BATCH_SCHEDULER_H_ +#define TENSORFLOW_CORE_KERNELS_BATCHING_UTIL_SHARED_BATCH_SCHEDULER_H_ + +#include +#include +#include +#include +#include +#include +#include +#include + +#include "tensorflow/core/kernels/batching_util/batch_scheduler.h" +#include "tensorflow/core/kernels/batching_util/periodic_function.h" +#include "tensorflow/core/lib/core/errors.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/lib/strings/strcat.h" +#include "tensorflow/core/platform/byte_order.h" +#include "tensorflow/core/platform/cpu_info.h" +#include "tensorflow/core/platform/env.h" +#include "tensorflow/core/platform/thread_annotations.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { +namespace serving { +namespace internal { +template +class Queue; +} // namespace internal +} // namespace serving +} // namespace tensorflow + +namespace tensorflow { +namespace serving { + +// A batch scheduler for server instances that service multiple request types +// (e.g. multiple machine-learned models, or multiple versions of a model served +// concurrently), or even multiple distinct tasks for a given request. The +// scheduler multiplexes batches of different kinds of tasks onto a fixed-size +// thread pool (each batch contains tasks of a single type), in a carefully +// controlled manner. A common configuration is to set the number of threads +// equal to the number of hardware accelerator units, in which case the +// scheduler takes care of multiplexing the task types onto the shared hardware, +// in a manner that is both fair and efficient. +// +// Semantically, SharedBatchScheduler behaves like having N instances of +// BasicBatchScheduler (see basic_batch_scheduler.h), one per task type. The +// difference is that under the covers there is a single shared thread pool, +// instead of N independent ones, with their sharing deliberately coordinated. +// +// SharedBatchScheduler does not implement the BatchScheduler API; rather, it +// presents an abstraction of "queues", where each queue corresponds to one type +// of task. Tasks submitted to a given queue are placed in their own batches, +// and cannot be mixed with other tasks. Queues can be added and deleted +// dynamically, to accommodate e.g. versions of a model being brought up and +// down over the lifetime of a server. +// +// The batch thread pool round-robins through the queues, running one batch +// from a queue and then moving to the next queue. Each queue behaves like a +// BasicBatchScheduler instance, in the sense that it has maximum batch size and +// timeout parameters, which govern when a batch is eligible to be processed. +// +// Each queue is independently configured with a maximum size (in terms of the +// maximum number of batches worth of enqueued tasks). For online serving, it is +// recommended that the queue sizes be configured such that the sum of the sizes +// of the active queues roughly equal the number of batch threads. (The idea is +// that if all threads become available at roughly the same time, there will be +// enough enqueued work for them to take on, but no more.) +// +// If queue sizes are configured in the manner suggested above, the maximum time +// a task can spend in a queue before being placed in a batch and assigned to a +// thread for processing, is the greater of: +// - the maximum time to process one batch of tasks from any active queue +// - the configured timeout parameter for the task's queue (which can be 0) +// +// For bulk processing jobs and throughput-oriented benchmarks, you may want to +// set the maximum queue size to a large value. +// +// TODO(b/26539183): Support queue servicing policies other than round-robin. +// E.g. let each queue specify a "share" (an int >= 1), so e.g. with queues A +// and B having shares 1 and 2 respectively, the servicing pattern is ABBABB... +// +// +// PERFORMANCE TUNING: See README.md. +// +template +class SharedBatchScheduler + : public std::enable_shared_from_this> { + public: + // TODO(b/25089730): Tune defaults based on best practices as they develop. + struct Options { + // The name to use for the pool of batch threads. + string thread_pool_name = {"batch_threads"}; + + // The number of threads to use to process batches. + // Must be >= 1, and should be tuned carefully. + int num_batch_threads = port::NumSchedulableCPUs(); + + // The environment to use. + // (Typically only overridden by test code.) + Env* env = Env::Default(); + }; + // Ownership is shared between the caller of Create() and any queues created + // via AddQueue(). + static Status Create( + const Options& options, + std::shared_ptr>* scheduler); + + ~SharedBatchScheduler(); + + // Adds a queue to which tasks may be submitted. The returned queue implements + // the BatchScheduler API. Each queue has its own set of scheduling options, + // and its own callback to process batches of tasks submitted to the queue. + // + // The returned queue's destructor blocks until all tasks submitted to it have + // been processed. + struct QueueOptions { + // The maximum size of each batch. + // + // The scheduler may form batches of any size between 1 and this number + // (inclusive). If there is a need to quantize the batch sizes, i.e. only + // submit batches whose size is in a small set of allowed sizes, that can be + // done by adding padding in the process-batch callback. + size_t max_batch_size = 1000; + + // If a task has been enqueued for this amount of time (in microseconds), + // and a thread is available, the scheduler will immediately form a batch + // from enqueued tasks and assign the batch to the thread for processing, + // even if the batch's size is below 'max_batch_size'. + // + // This parameter offers a way to bound queue latency, so that a task isn't + // stuck in the queue indefinitely waiting for enough tasks to arrive to + // make a full batch. (The latency bound is given in the class documentation + // above.) + // + // The goal is to smooth out batch sizes under low request rates, and thus + // avoid latency spikes. + int64 batch_timeout_micros = 0; + + // The maximum allowable number of enqueued (accepted by Schedule() but + // not yet being processed on a batch thread) tasks in terms of batches. + // If this limit is reached, Schedule() will return an UNAVAILABLE error. + // See the class documentation above for guidelines on how to tune this + // parameter. + size_t max_enqueued_batches = 10; + }; + Status AddQueue(const QueueOptions& options, + std::function>)> + process_batch_callback, + std::unique_ptr>* queue); + + private: + explicit SharedBatchScheduler(const Options& options); + + // The code executed in 'batch_threads_'. Obtains a batch to process from the + // queue pointed to by 'next_queue_to_schedule_', and processes it. If that + // queue declines to provide a batch to process, moves onto the next queue. If + // no queues provide a batch to process, just sleeps briefly and exits. + void ThreadLogic(); + + const Options options_; + + mutex mu_; + + // A list of queues. (We use std::list instead of std::vector to ensure that + // iterators are not invalidated by adding/removing elements. It also offers + // efficient removal of elements from the middle.) + using QueueList = std::list>>; + + // All "active" queues, i.e. ones that either: + // - have not been removed, or + // - have been removed but are not yet empty. + QueueList queues_ GUARDED_BY(mu_); + + // An iterator over 'queues_', pointing to the queue from which the next + // available batch thread should grab work. + typename QueueList::iterator next_queue_to_schedule_ GUARDED_BY(mu_); + + // Used by idle batch threads to wait for work to enter the system. Notified + // whenever a batch becomes schedulable. + condition_variable schedulable_batch_cv_; + + // Threads that process batches obtained from the queues. + std::vector> batch_threads_; + + TF_DISALLOW_COPY_AND_ASSIGN(SharedBatchScheduler); +}; + +////////// +// Implementation details follow. API users need not read. + +namespace internal { + +// A task queue for SharedBatchScheduler. Accepts tasks and accumulates them +// into batches, and dispenses those batches to be processed via a "pull" +// interface. The queue's behavior is governed by maximum batch size, timeout +// and maximum queue length parameters; see their documentation in +// SharedBatchScheduler. +// +// The queue is implemented as a deque of batches, with these invariants: +// - The number of batches is between 1 and 'options_.max_enqueued_batches'. +// - The back-most batch is open; the rest are closed. +// +// Submitted tasks are added to the open batch. If that batch doesn't have room +// but the queue isn't full, then that batch is closed and a new open batch is +// started. +// +// Batch pull requests are handled by dequeuing the front-most batch if it is +// closed. If the front-most batch is open (i.e. the queue contains only one +// batch) and has reached the timeout, it is immediately closed and returned; +// otherwise no batch is returned for the request. +template +class Queue { + public: + using ProcessBatchCallback = + std::function>)>; + using SchedulableBatchCallback = std::function; + Queue(const typename SharedBatchScheduler::QueueOptions& options, + Env* env, ProcessBatchCallback process_batch_callback, + SchedulableBatchCallback schdulable_batch_callback); + + // Illegal to destruct unless the queue is empty. + ~Queue(); + + // Submits a task to the queue, with the same semantics as + // BatchScheduler::Schedule(). + Status Schedule(std::unique_ptr* task); + + // Returns the number of enqueued tasks, with the same semantics as + // BatchScheduler::NumEnqueuedTasks(). + size_t NumEnqueuedTasks() const; + + // Returns the queue capacity, with the same semantics as + // BatchScheduler::SchedulingCapacity(). + size_t SchedulingCapacity() const; + + // Returns the maximum allowed size of tasks submitted to the queue. + size_t max_task_size() const { return options_.max_batch_size; } + + // Called by a thread that is ready to process a batch, to request one from + // this queue. Either returns a batch that is ready to be processed, or + // nullptr if the queue declines to schedule a batch at this time. If it + // returns a batch, the batch is guaranteed to be closed. + std::unique_ptr> ScheduleBatch(); + + // Processes a batch that has been returned earlier by ScheduleBatch(). + void ProcessBatch(std::unique_ptr> batch); + + // Determines whether the queue is empty, i.e. has no tasks waiting or being + // processed. + bool IsEmpty() const; + + // Marks the queue closed, and waits until it is empty. + void CloseAndWaitUntilEmpty(); + + bool closed() const { + mutex_lock l(mu_); + return closed_; + } + + private: + // Same as IsEmpty(), but assumes the caller already holds a lock on 'mu_'. + bool IsEmptyInternal() const EXCLUSIVE_LOCKS_REQUIRED(mu_); + + // Closes the open batch residing at the back of 'batches_', and inserts a + // fresh open batch behind it. + void StartNewBatch() EXCLUSIVE_LOCKS_REQUIRED(mu_); + + // Determines whether the open batch residing at the back of 'batches_' is + // currently schedulable. + bool IsOpenBatchSchedulable() const EXCLUSIVE_LOCKS_REQUIRED(mu_); + + const typename SharedBatchScheduler::QueueOptions options_; + + // The environment to use. + Env* env_; + + // A callback invoked to processes a batch of work units. Always invoked from + // a batch thread. + ProcessBatchCallback process_batch_callback_; + + // A callback invoked to notify the scheduler that a new batch has become + // schedulable. + SchedulableBatchCallback schedulable_batch_callback_; + + mutable mutex mu_; + + // Whether this queue can accept new tasks. This variable is monotonic: it + // starts as false, and then at some point gets set to true and remains true + // for the duration of this object's life. + bool closed_ GUARDED_BY(mu_) = false; + + // The enqueued batches. See the invariants in the class comments above. + std::deque>> batches_ GUARDED_BY(mu_); + + // The time at which the first task was added to the open (back-most) batch + // in 'batches_'. Valid iff that batch contains at least one task. + uint64 open_batch_start_time_micros_ GUARDED_BY(mu_); + + // Whether this queue contains a batch that is eligible to be scheduled. Used + // to keep track of when to call 'schedulable_batch_callback_'. + bool schedulable_batch_ GUARDED_BY(mu_) = false; + + // The number of batches currently being processed by batch threads. + // Incremented in ScheduleBatch() and decremented in ProcessBatch(). + int num_batches_being_processed_ GUARDED_BY(mu_) = 0; + + // Used by CloseAndWaitUntilEmpty() to wait until the queue is empty, for the + // case in which the queue is not empty when CloseAndWaitUntilEmpty() starts. + // When ProcessBatch() dequeues the last batch and makes the queue empty, if + // 'empty_notification_' is non-null it calls 'empty_notification_->Notify()'. + Notification* empty_notification_ GUARDED_BY(mu_) = nullptr; + + TF_DISALLOW_COPY_AND_ASSIGN(Queue); +}; + +// A RAII-style object that points to a Queue and implements +// the BatchScheduler API. To be handed out to clients who call AddQueue(). +template +class QueueHandle : public BatchScheduler { + public: + QueueHandle(std::shared_ptr> scheduler, + Queue* queue); + ~QueueHandle() override; + + Status Schedule(std::unique_ptr* task) override; + size_t NumEnqueuedTasks() const override; + size_t SchedulingCapacity() const override; + + size_t max_task_size() const override { return queue_->max_task_size(); } + + private: + // The scheduler that owns 'queue_'. + std::shared_ptr> scheduler_; + + // The queue this handle wraps. Owned by 'scheduler_', which keeps it alive at + // least until this class's destructor closes it. + Queue* queue_; + + TF_DISALLOW_COPY_AND_ASSIGN(QueueHandle); +}; + +} // namespace internal + +template +Status SharedBatchScheduler::Create( + const Options& options, + std::shared_ptr>* scheduler) { + if (options.num_batch_threads < 1) { + return errors::InvalidArgument("num_batch_threads must be positive; was ", + options.num_batch_threads); + } + scheduler->reset(new SharedBatchScheduler(options)); + return Status::OK(); +} + +template +SharedBatchScheduler::~SharedBatchScheduler() { + // Wait until the batch threads finish clearing out and deleting the closed + // queues. + for (;;) { + { + mutex_lock l(mu_); + if (queues_.empty()) { + break; + } + } + const int64 kSleepTimeMicros = 100; + options_.env->SleepForMicroseconds(kSleepTimeMicros); + } + // Delete the batch threads before allowing state the threads may access (e.g. + // 'mu_') to be deleted. + batch_threads_.clear(); +} + +template +Status SharedBatchScheduler::AddQueue( + const QueueOptions& options, + std::function>)> + process_batch_callback, + std::unique_ptr>* queue) { + if (options.max_batch_size == 0) { + return errors::InvalidArgument("max_batch_size must be positive; was ", + options.max_batch_size); + } + if (options.batch_timeout_micros < 0) { + return errors::InvalidArgument( + "batch_timeout_micros must be non-negative; was ", + options.batch_timeout_micros); + } + if (options.max_enqueued_batches < 0) { + return errors::InvalidArgument( + "max_enqueued_batches must be non-negative; was ", + options.max_enqueued_batches); + } + + auto schedulable_batch_callback = [this] { + mutex_lock l(mu_); + schedulable_batch_cv_.notify_one(); + }; + auto internal_queue = + std::unique_ptr>(new internal::Queue( + options, options_.env, process_batch_callback, + schedulable_batch_callback)); + auto handle = std::unique_ptr>( + new internal::QueueHandle(this->shared_from_this(), + internal_queue.get())); + { + mutex_lock l(mu_); + queues_.push_back(std::move(internal_queue)); + if (next_queue_to_schedule_ == queues_.end()) { + next_queue_to_schedule_ = queues_.begin(); + } + } + *queue = std::move(handle); + return Status::OK(); +} + +template +SharedBatchScheduler::SharedBatchScheduler(const Options& options) + : options_(options), next_queue_to_schedule_(queues_.end()) { + // Kick off the batch threads. + PeriodicFunction::Options periodic_fn_options; + periodic_fn_options.thread_name_prefix = + strings::StrCat(options.thread_pool_name, "_"); + for (int i = 0; i < options.num_batch_threads; ++i) { + std::unique_ptr thread(new PeriodicFunction( + [this] { this->ThreadLogic(); }, + 0 /* function invocation interval time */, periodic_fn_options)); + batch_threads_.push_back(std::move(thread)); + } +} + +template +void SharedBatchScheduler::ThreadLogic() { + // A batch to process next (or nullptr if no work to do). + std::unique_ptr> batch_to_process; + // The queue with which 'batch_to_process' is associated. + internal::Queue* queue_for_batch = nullptr; + { + mutex_lock l(mu_); + + const int num_queues = queues_.size(); + for (int num_queues_tried = 0; + batch_to_process == nullptr && num_queues_tried < num_queues; + ++num_queues_tried) { + DCHECK(next_queue_to_schedule_ != queues_.end()); + + // If a closed queue responds to ScheduleBatch() with nullptr, the queue + // will never yield any further batches so we can drop it. To avoid a + // race, we take a snapshot of the queue's closedness state *before* + // calling ScheduleBatch(). + const bool queue_closed = (*next_queue_to_schedule_)->closed(); + + // Ask '*next_queue_to_schedule_' if it wants us to process a batch. + batch_to_process = (*next_queue_to_schedule_)->ScheduleBatch(); + if (batch_to_process != nullptr) { + queue_for_batch = next_queue_to_schedule_->get(); + } + + // Advance 'next_queue_to_schedule_'. + if (queue_closed && (*next_queue_to_schedule_)->IsEmpty() && + batch_to_process == nullptr) { + // We've encountered a closed queue with no work to do. Drop it. + DCHECK_NE(queue_for_batch, next_queue_to_schedule_->get()); + next_queue_to_schedule_ = queues_.erase(next_queue_to_schedule_); + } else { + ++next_queue_to_schedule_; + } + if (next_queue_to_schedule_ == queues_.end() && !queues_.empty()) { + // We've hit the end. Wrap to the first queue. + next_queue_to_schedule_ = queues_.begin(); + } + } + + if (batch_to_process == nullptr) { + // We couldn't find any work to do. Wait until a new batch becomes + // schedulable, or some time has elapsed, before checking again. + const int64 kTimeoutMillis = 1; // The smallest accepted granule of time. + WaitForMilliseconds(&l, &schedulable_batch_cv_, kTimeoutMillis); + return; + } + } + + queue_for_batch->ProcessBatch(std::move(batch_to_process)); +} + +namespace internal { + +template +Queue::Queue( + const typename SharedBatchScheduler::QueueOptions& options, + Env* env, ProcessBatchCallback process_batch_callback, + SchedulableBatchCallback schedulable_batch_callback) + : options_(options), + env_(env), + process_batch_callback_(process_batch_callback), + schedulable_batch_callback_(schedulable_batch_callback) { + // Create an initial, open batch. + batches_.emplace_back(new Batch); +} + +template +Queue::~Queue() { + mutex_lock l(mu_); + DCHECK(IsEmptyInternal()); + + // Close the (empty) open batch, so its destructor doesn't block. + batches_.back()->Close(); +} + +template +Status Queue::Schedule(std::unique_ptr* task) { + if ((*task)->size() > options_.max_batch_size) { + return errors::InvalidArgument("Task size ", (*task)->size(), + " is larger than maximum batch size ", + options_.max_batch_size); + } + + bool notify_of_schedulable_batch = false; + { + mutex_lock l(mu_); + + DCHECK(!closed_); + + if (batches_.back()->size() + (*task)->size() > options_.max_batch_size) { + if (batches_.size() >= options_.max_enqueued_batches) { + return errors::Unavailable( + "The batch scheduling queue to which this task was submitted is " + "full"); + } + StartNewBatch(); + } + if (batches_.back()->empty()) { + open_batch_start_time_micros_ = env_->NowMicros(); + } + batches_.back()->AddTask(std::move(*task)); + + if (!schedulable_batch_) { + if (batches_.size() > 1 || IsOpenBatchSchedulable()) { + schedulable_batch_ = true; + notify_of_schedulable_batch = true; + } + } + } + + if (notify_of_schedulable_batch) { + schedulable_batch_callback_(); + } + + return Status::OK(); +} + +template +size_t Queue::NumEnqueuedTasks() const { + mutex_lock l(mu_); + size_t num_enqueued_tasks = 0; + for (const auto& batch : batches_) { + num_enqueued_tasks += batch->num_tasks(); + } + return num_enqueued_tasks; +} + +template +size_t Queue::SchedulingCapacity() const { + mutex_lock l(mu_); + const int num_new_batches_schedulable = + options_.max_enqueued_batches - batches_.size(); + const int open_batch_capacity = + options_.max_batch_size - batches_.back()->size(); + return (num_new_batches_schedulable * options_.max_batch_size) + + open_batch_capacity; +} + +template +std::unique_ptr> Queue::ScheduleBatch() { + // The batch to schedule, which we may populate below. (If left as nullptr, + // that means we are electing not to schedule a batch at this time.) + std::unique_ptr> batch_to_schedule; + + { + mutex_lock l(mu_); + + // Consider closing the open batch at this time, to schedule it. + if (batches_.size() == 1 && IsOpenBatchSchedulable()) { + StartNewBatch(); + } + + if (batches_.size() >= 2) { + // There is at least one closed batch that is ready to be scheduled. + ++num_batches_being_processed_; + batch_to_schedule = std::move(batches_.front()); + batches_.pop_front(); + } else { + schedulable_batch_ = false; + } + } + + return batch_to_schedule; +} + +template +void Queue::ProcessBatch(std::unique_ptr> batch) { + process_batch_callback_(std::move(batch)); + + { + mutex_lock l(mu_); + --num_batches_being_processed_; + if (empty_notification_ != nullptr && IsEmptyInternal()) { + empty_notification_->Notify(); + } + } +} + +template +bool Queue::IsEmpty() const { + mutex_lock l(mu_); + return IsEmptyInternal(); +} + +template +void Queue::CloseAndWaitUntilEmpty() { + Notification empty; + { + mutex_lock l(mu_); + closed_ = true; + if (IsEmptyInternal()) { + empty.Notify(); + } else { + // Arrange for ProcessBatch() to notify when the queue becomes empty. + empty_notification_ = ∅ + } + } + empty.WaitForNotification(); +} + +template +bool Queue::IsEmptyInternal() const { + return num_batches_being_processed_ == 0 && batches_.size() == 1 && + batches_.back()->empty(); +} + +template +void Queue::StartNewBatch() { + batches_.back()->Close(); + batches_.emplace_back(new Batch); +} + +template +bool Queue::IsOpenBatchSchedulable() const { + Batch* open_batch = batches_.back().get(); + if (open_batch->empty()) { + return false; + } + return closed_ || open_batch->size() >= options_.max_batch_size || + env_->NowMicros() >= + open_batch_start_time_micros_ + options_.batch_timeout_micros; +} + +template +QueueHandle::QueueHandle( + std::shared_ptr> scheduler, + Queue* queue) + : scheduler_(scheduler), queue_(queue) {} + +template +QueueHandle::~QueueHandle() { + queue_->CloseAndWaitUntilEmpty(); +} + +template +Status QueueHandle::Schedule(std::unique_ptr* task) { + return queue_->Schedule(task); +} + +template +size_t QueueHandle::NumEnqueuedTasks() const { + return queue_->NumEnqueuedTasks(); +} + +template +size_t QueueHandle::SchedulingCapacity() const { + return queue_->SchedulingCapacity(); +} + +} // namespace internal + +} // namespace serving +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_BATCHING_UTIL_SHARED_BATCH_SCHEDULER_H_ diff --git a/linux/include/tensorflow/core/kernels/betainc_op.h b/linux/include/tensorflow/core/kernels/betainc_op.h new file mode 100644 index 000000000..b941b27ad --- /dev/null +++ b/linux/include/tensorflow/core/kernels/betainc_op.h @@ -0,0 +1,51 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_BETAINC_OP_H_ +#define TENSORFLOW_CORE_KERNELS_BETAINC_OP_H_ +// Functor definition for BetaincOp, must be compilable by nvcc. + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/tensor_types.h" + +namespace tensorflow { +namespace functor { + +// Functor used by BetaincOp to do the computations. +template +struct Betainc { + void operator()(const Device& d, typename TTypes::ConstTensor a, + typename TTypes::ConstTensor b, + typename TTypes::ConstTensor x, + typename TTypes::Tensor output) { + output.device(d) = Eigen::betainc(a, b, x); + } + + void BCast(const Device& d, typename TTypes::ConstTensor a, + const typename Eigen::array& bcast_a, + typename TTypes::ConstTensor b, + const typename Eigen::array& bcast_b, + typename TTypes::ConstTensor x, + const typename Eigen::array& bcast_x, + typename TTypes::Tensor output) { + output.device(d) = Eigen::betainc( + a.broadcast(bcast_a), b.broadcast(bcast_b), x.broadcast(bcast_x)); + } +}; + +} // namespace functor +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_BETAINC_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/bias_op.h b/linux/include/tensorflow/core/kernels/bias_op.h new file mode 100644 index 000000000..77f683455 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/bias_op.h @@ -0,0 +1,55 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_BIAS_OP_H_ +#define TENSORFLOW_CORE_KERNELS_BIAS_OP_H_ +// Functor definition for BiasOp, must be compilable by nvcc. + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/tensor_types.h" + +namespace tensorflow { +namespace functor { + +// Functor used by BiasOp to do the computations. +template +struct Bias { + // Add "bias" to "input", broadcasting it on all dimensions but the last one. + void operator()(const Device& d, typename TTypes::ConstTensor input, + typename TTypes::ConstVec bias, + typename TTypes::Tensor output) { + if (input.size() >= INT_MAX) { + const int64_t bias_size = bias.dimension(0); + const int64_t rest_size = input.size() / bias_size; + Eigen::DSizes one_d(input.size()); + Eigen::DSizes bcast(rest_size); + output.reshape(one_d).device(d) = + input.reshape(one_d) + bias.broadcast(bcast).reshape(one_d); + } else { + const int bias_size = bias.dimension(0); + const int rest_size = input.size() / bias_size; + Eigen::DSizes one_d(input.size()); + Eigen::DSizes bcast(rest_size); + To32Bit(output).reshape(one_d).device(d) = + To32Bit(input).reshape(one_d) + + To32Bit(bias).broadcast(bcast).reshape(one_d); + } + } +}; + +} // namespace functor +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_BIAS_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/bias_op_gpu.h b/linux/include/tensorflow/core/kernels/bias_op_gpu.h new file mode 100644 index 000000000..c1051f43c --- /dev/null +++ b/linux/include/tensorflow/core/kernels/bias_op_gpu.h @@ -0,0 +1,80 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_BIAS_OP_GPU_H_ +#define TENSORFLOW_CORE_KERNELS_BIAS_OP_GPU_H_ + +#define EIGEN_USE_GPU + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/kernels/gpu_utils.h" +#include "tensorflow/core/util/tensor_format.h" + +namespace tensorflow { + +typedef Eigen::GpuDevice GPUDevice; + +template +struct BiasGPU { + static void compute(const GPUDevice& d, const T* input, const T* bias, + T* output, int32 batch, int32 height, int32 width, + int32 channel, TensorFormat data_format); +}; + +template +struct BiasGradGPU { + static void compute(const GPUDevice& device, const T* output_backprop, + T* bias_backprop, int32 batch, int32 height, int32 width, + int32 channel, TensorFormat data_format); + + static void DoRowReduction(OpKernelContext* context, T* output, + const T* input, int rows, int cols); + + static void DoColReduction(OpKernelContext* context, T* output, + const T* input, int rows, int cols); +}; + +enum class BiasAddGradGPUMode { + kInvalid = 0, + kNative = 1, + kReduction = 2, +}; + +// Describe the BiasGradGPU result from a perf experiment. +// +// Arguments: +// algorithm: returns the method to use for bias add grad. +// elapsed_time; returns the measured elapsed time in microseconds. +class BiasGradGPUProfileResult { + public: + bool is_valid() const { + return (algorithm_ != BiasAddGradGPUMode::kInvalid && + elapsed_time_ != std::numeric_limits::max()); + } + BiasAddGradGPUMode algorithm() const { return algorithm_; } + void set_algorithm(BiasAddGradGPUMode val) { algorithm_ = val; } + uint64 elapsed_time() const { return elapsed_time_; } + void set_elapsed_time(uint64 val) { elapsed_time_ = val; } + + private: + BiasAddGradGPUMode algorithm_ = BiasAddGradGPUMode::kInvalid; + uint64 elapsed_time_ = std::numeric_limits::max(); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_BIAS_OP_GPU_H_ diff --git a/linux/include/tensorflow/core/kernels/bincount_op.h b/linux/include/tensorflow/core/kernels/bincount_op.h new file mode 100644 index 000000000..54cfb79de --- /dev/null +++ b/linux/include/tensorflow/core/kernels/bincount_op.h @@ -0,0 +1,41 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_BINCOUNT_OP_H_ +#define TENSORFLOW_CORE_KERNELS_BINCOUNT_OP_H_ + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/core/errors.h" + +namespace tensorflow { + +namespace functor { + +template +struct BincountFunctor { + static Status Compute(OpKernelContext* context, + const typename TTypes::ConstTensor& arr, + const typename TTypes::ConstTensor& weights, + typename TTypes::Tensor& output); +}; + +} // end namespace functor + +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_BINCOUNT_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/bitcast_op.h b/linux/include/tensorflow/core/kernels/bitcast_op.h new file mode 100644 index 000000000..900ab6f35 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/bitcast_op.h @@ -0,0 +1,30 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// See docs in ../ops/array_ops.cc. + +#ifndef TENSORFLOW_CORE_KERNELS_BITCAST_OP_H_ +#define TENSORFLOW_CORE_KERNELS_BITCAST_OP_H_ + +#include // for memcpy + +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/register_types.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/core/casts.h" + +#endif // TENSORFLOW_CORE_KERNELS_BITCAST_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/boosted_trees/boosted_trees.pb.h b/linux/include/tensorflow/core/kernels/boosted_trees/boosted_trees.pb.h new file mode 100644 index 000000000..39f446c3b --- /dev/null +++ b/linux/include/tensorflow/core/kernels/boosted_trees/boosted_trees.pb.h @@ -0,0 +1,3183 @@ +// Generated by the protocol buffer compiler. DO NOT EDIT! +// source: tensorflow/core/kernels/boosted_trees/boosted_trees.proto + +#ifndef PROTOBUF_INCLUDED_tensorflow_2fcore_2fkernels_2fboosted_5ftrees_2fboosted_5ftrees_2eproto +#define PROTOBUF_INCLUDED_tensorflow_2fcore_2fkernels_2fboosted_5ftrees_2fboosted_5ftrees_2eproto + +#include + +#include + +#if GOOGLE_PROTOBUF_VERSION < 3006000 +#error This file was generated by a newer version of protoc which is +#error incompatible with your Protocol Buffer headers. Please update +#error your headers. +#endif +#if 3006000 < GOOGLE_PROTOBUF_MIN_PROTOC_VERSION +#error This file was generated by an older version of protoc which is +#error incompatible with your Protocol Buffer headers. Please +#error regenerate this file with a newer version of protoc. +#endif + +#include +#include +#include +#include +#include +#include +#include +#include +#include // IWYU pragma: export +#include // IWYU pragma: export +#include +// @@protoc_insertion_point(includes) +#define PROTOBUF_INTERNAL_EXPORT_protobuf_tensorflow_2fcore_2fkernels_2fboosted_5ftrees_2fboosted_5ftrees_2eproto + +namespace protobuf_tensorflow_2fcore_2fkernels_2fboosted_5ftrees_2fboosted_5ftrees_2eproto { +// Internal implementation detail -- do not use these members. +struct TableStruct { + static const ::google::protobuf::internal::ParseTableField entries[]; + static const ::google::protobuf::internal::AuxillaryParseTableField aux[]; + static const ::google::protobuf::internal::ParseTable schema[13]; + static const ::google::protobuf::internal::FieldMetadata field_metadata[]; + static const ::google::protobuf::internal::SerializationTable serialization_table[]; + static const ::google::protobuf::uint32 offsets[]; +}; +void AddDescriptors(); +} // namespace protobuf_tensorflow_2fcore_2fkernels_2fboosted_5ftrees_2fboosted_5ftrees_2eproto +namespace tensorflow { +namespace boosted_trees { +class BucketizedSplit; +class BucketizedSplitDefaultTypeInternal; +extern BucketizedSplitDefaultTypeInternal _BucketizedSplit_default_instance_; +class CategoricalSplit; +class CategoricalSplitDefaultTypeInternal; +extern CategoricalSplitDefaultTypeInternal _CategoricalSplit_default_instance_; +class DebugOutput; +class DebugOutputDefaultTypeInternal; +extern DebugOutputDefaultTypeInternal _DebugOutput_default_instance_; +class GrowingMetadata; +class GrowingMetadataDefaultTypeInternal; +extern GrowingMetadataDefaultTypeInternal _GrowingMetadata_default_instance_; +class Leaf; +class LeafDefaultTypeInternal; +extern LeafDefaultTypeInternal _Leaf_default_instance_; +class Node; +class NodeDefaultTypeInternal; +extern NodeDefaultTypeInternal _Node_default_instance_; +class NodeMetadata; +class NodeMetadataDefaultTypeInternal; +extern NodeMetadataDefaultTypeInternal _NodeMetadata_default_instance_; +class SparseVector; +class SparseVectorDefaultTypeInternal; +extern SparseVectorDefaultTypeInternal _SparseVector_default_instance_; +class Tree; +class TreeDefaultTypeInternal; +extern TreeDefaultTypeInternal _Tree_default_instance_; +class TreeEnsemble; +class TreeEnsembleDefaultTypeInternal; +extern TreeEnsembleDefaultTypeInternal _TreeEnsemble_default_instance_; +class TreeMetadata; +class TreeMetadataDefaultTypeInternal; +extern TreeMetadataDefaultTypeInternal _TreeMetadata_default_instance_; +class TreeMetadata_PostPruneNodeUpdate; +class TreeMetadata_PostPruneNodeUpdateDefaultTypeInternal; +extern TreeMetadata_PostPruneNodeUpdateDefaultTypeInternal _TreeMetadata_PostPruneNodeUpdate_default_instance_; +class Vector; +class VectorDefaultTypeInternal; +extern VectorDefaultTypeInternal _Vector_default_instance_; +} // namespace boosted_trees +} // namespace tensorflow +namespace google { +namespace protobuf { +template<> ::tensorflow::boosted_trees::BucketizedSplit* Arena::CreateMaybeMessage<::tensorflow::boosted_trees::BucketizedSplit>(Arena*); +template<> ::tensorflow::boosted_trees::CategoricalSplit* Arena::CreateMaybeMessage<::tensorflow::boosted_trees::CategoricalSplit>(Arena*); +template<> ::tensorflow::boosted_trees::DebugOutput* Arena::CreateMaybeMessage<::tensorflow::boosted_trees::DebugOutput>(Arena*); +template<> ::tensorflow::boosted_trees::GrowingMetadata* Arena::CreateMaybeMessage<::tensorflow::boosted_trees::GrowingMetadata>(Arena*); +template<> ::tensorflow::boosted_trees::Leaf* Arena::CreateMaybeMessage<::tensorflow::boosted_trees::Leaf>(Arena*); +template<> ::tensorflow::boosted_trees::Node* Arena::CreateMaybeMessage<::tensorflow::boosted_trees::Node>(Arena*); +template<> ::tensorflow::boosted_trees::NodeMetadata* Arena::CreateMaybeMessage<::tensorflow::boosted_trees::NodeMetadata>(Arena*); +template<> ::tensorflow::boosted_trees::SparseVector* Arena::CreateMaybeMessage<::tensorflow::boosted_trees::SparseVector>(Arena*); +template<> ::tensorflow::boosted_trees::Tree* Arena::CreateMaybeMessage<::tensorflow::boosted_trees::Tree>(Arena*); +template<> ::tensorflow::boosted_trees::TreeEnsemble* Arena::CreateMaybeMessage<::tensorflow::boosted_trees::TreeEnsemble>(Arena*); +template<> ::tensorflow::boosted_trees::TreeMetadata* Arena::CreateMaybeMessage<::tensorflow::boosted_trees::TreeMetadata>(Arena*); +template<> ::tensorflow::boosted_trees::TreeMetadata_PostPruneNodeUpdate* Arena::CreateMaybeMessage<::tensorflow::boosted_trees::TreeMetadata_PostPruneNodeUpdate>(Arena*); +template<> ::tensorflow::boosted_trees::Vector* Arena::CreateMaybeMessage<::tensorflow::boosted_trees::Vector>(Arena*); +} // namespace protobuf +} // namespace google +namespace tensorflow { +namespace boosted_trees { + +// =================================================================== + +class Node : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.boosted_trees.Node) */ { + public: + Node(); + virtual ~Node(); + + Node(const Node& from); + + inline Node& operator=(const Node& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + Node(Node&& from) noexcept + : Node() { + *this = ::std::move(from); + } + + inline Node& operator=(Node&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const Node& default_instance(); + + enum NodeCase { + kLeaf = 1, + kBucketizedSplit = 2, + kCategoricalSplit = 3, + NODE_NOT_SET = 0, + }; + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const Node* internal_default_instance() { + return reinterpret_cast( + &_Node_default_instance_); + } + static constexpr int kIndexInFileMessages = + 0; + + void UnsafeArenaSwap(Node* other); + void Swap(Node* other); + friend void swap(Node& a, Node& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline Node* New() const final { + return CreateMaybeMessage(NULL); + } + + Node* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const Node& from); + void MergeFrom(const Node& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(Node* other); + protected: + explicit Node(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // .tensorflow.boosted_trees.NodeMetadata metadata = 777; + bool has_metadata() const; + void clear_metadata(); + static const int kMetadataFieldNumber = 777; + private: + const ::tensorflow::boosted_trees::NodeMetadata& _internal_metadata() const; + public: + const ::tensorflow::boosted_trees::NodeMetadata& metadata() const; + ::tensorflow::boosted_trees::NodeMetadata* release_metadata(); + ::tensorflow::boosted_trees::NodeMetadata* mutable_metadata(); + void set_allocated_metadata(::tensorflow::boosted_trees::NodeMetadata* metadata); + void unsafe_arena_set_allocated_metadata( + ::tensorflow::boosted_trees::NodeMetadata* metadata); + ::tensorflow::boosted_trees::NodeMetadata* unsafe_arena_release_metadata(); + + // .tensorflow.boosted_trees.Leaf leaf = 1; + bool has_leaf() const; + void clear_leaf(); + static const int kLeafFieldNumber = 1; + private: + const ::tensorflow::boosted_trees::Leaf& _internal_leaf() const; + public: + const ::tensorflow::boosted_trees::Leaf& leaf() const; + ::tensorflow::boosted_trees::Leaf* release_leaf(); + ::tensorflow::boosted_trees::Leaf* mutable_leaf(); + void set_allocated_leaf(::tensorflow::boosted_trees::Leaf* leaf); + void unsafe_arena_set_allocated_leaf( + ::tensorflow::boosted_trees::Leaf* leaf); + ::tensorflow::boosted_trees::Leaf* unsafe_arena_release_leaf(); + + // .tensorflow.boosted_trees.BucketizedSplit bucketized_split = 2; + bool has_bucketized_split() const; + void clear_bucketized_split(); + static const int kBucketizedSplitFieldNumber = 2; + private: + const ::tensorflow::boosted_trees::BucketizedSplit& _internal_bucketized_split() const; + public: + const ::tensorflow::boosted_trees::BucketizedSplit& bucketized_split() const; + ::tensorflow::boosted_trees::BucketizedSplit* release_bucketized_split(); + ::tensorflow::boosted_trees::BucketizedSplit* mutable_bucketized_split(); + void set_allocated_bucketized_split(::tensorflow::boosted_trees::BucketizedSplit* bucketized_split); + void unsafe_arena_set_allocated_bucketized_split( + ::tensorflow::boosted_trees::BucketizedSplit* bucketized_split); + ::tensorflow::boosted_trees::BucketizedSplit* unsafe_arena_release_bucketized_split(); + + // .tensorflow.boosted_trees.CategoricalSplit categorical_split = 3; + bool has_categorical_split() const; + void clear_categorical_split(); + static const int kCategoricalSplitFieldNumber = 3; + private: + const ::tensorflow::boosted_trees::CategoricalSplit& _internal_categorical_split() const; + public: + const ::tensorflow::boosted_trees::CategoricalSplit& categorical_split() const; + ::tensorflow::boosted_trees::CategoricalSplit* release_categorical_split(); + ::tensorflow::boosted_trees::CategoricalSplit* mutable_categorical_split(); + void set_allocated_categorical_split(::tensorflow::boosted_trees::CategoricalSplit* categorical_split); + void unsafe_arena_set_allocated_categorical_split( + ::tensorflow::boosted_trees::CategoricalSplit* categorical_split); + ::tensorflow::boosted_trees::CategoricalSplit* unsafe_arena_release_categorical_split(); + + void clear_node(); + NodeCase node_case() const; + // @@protoc_insertion_point(class_scope:tensorflow.boosted_trees.Node) + private: + void set_has_leaf(); + void set_has_bucketized_split(); + void set_has_categorical_split(); + + inline bool has_node() const; + inline void clear_has_node(); + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::tensorflow::boosted_trees::NodeMetadata* metadata_; + union NodeUnion { + NodeUnion() {} + ::tensorflow::boosted_trees::Leaf* leaf_; + ::tensorflow::boosted_trees::BucketizedSplit* bucketized_split_; + ::tensorflow::boosted_trees::CategoricalSplit* categorical_split_; + } node_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + ::google::protobuf::uint32 _oneof_case_[1]; + + friend struct ::protobuf_tensorflow_2fcore_2fkernels_2fboosted_5ftrees_2fboosted_5ftrees_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class NodeMetadata : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.boosted_trees.NodeMetadata) */ { + public: + NodeMetadata(); + virtual ~NodeMetadata(); + + NodeMetadata(const NodeMetadata& from); + + inline NodeMetadata& operator=(const NodeMetadata& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + NodeMetadata(NodeMetadata&& from) noexcept + : NodeMetadata() { + *this = ::std::move(from); + } + + inline NodeMetadata& operator=(NodeMetadata&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const NodeMetadata& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const NodeMetadata* internal_default_instance() { + return reinterpret_cast( + &_NodeMetadata_default_instance_); + } + static constexpr int kIndexInFileMessages = + 1; + + void UnsafeArenaSwap(NodeMetadata* other); + void Swap(NodeMetadata* other); + friend void swap(NodeMetadata& a, NodeMetadata& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline NodeMetadata* New() const final { + return CreateMaybeMessage(NULL); + } + + NodeMetadata* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const NodeMetadata& from); + void MergeFrom(const NodeMetadata& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(NodeMetadata* other); + protected: + explicit NodeMetadata(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // .tensorflow.boosted_trees.Leaf original_leaf = 2; + bool has_original_leaf() const; + void clear_original_leaf(); + static const int kOriginalLeafFieldNumber = 2; + private: + const ::tensorflow::boosted_trees::Leaf& _internal_original_leaf() const; + public: + const ::tensorflow::boosted_trees::Leaf& original_leaf() const; + ::tensorflow::boosted_trees::Leaf* release_original_leaf(); + ::tensorflow::boosted_trees::Leaf* mutable_original_leaf(); + void set_allocated_original_leaf(::tensorflow::boosted_trees::Leaf* original_leaf); + void unsafe_arena_set_allocated_original_leaf( + ::tensorflow::boosted_trees::Leaf* original_leaf); + ::tensorflow::boosted_trees::Leaf* unsafe_arena_release_original_leaf(); + + // float gain = 1; + void clear_gain(); + static const int kGainFieldNumber = 1; + float gain() const; + void set_gain(float value); + + // @@protoc_insertion_point(class_scope:tensorflow.boosted_trees.NodeMetadata) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::tensorflow::boosted_trees::Leaf* original_leaf_; + float gain_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fkernels_2fboosted_5ftrees_2fboosted_5ftrees_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class Leaf : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.boosted_trees.Leaf) */ { + public: + Leaf(); + virtual ~Leaf(); + + Leaf(const Leaf& from); + + inline Leaf& operator=(const Leaf& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + Leaf(Leaf&& from) noexcept + : Leaf() { + *this = ::std::move(from); + } + + inline Leaf& operator=(Leaf&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const Leaf& default_instance(); + + enum LeafCase { + kVector = 1, + kSparseVector = 2, + LEAF_NOT_SET = 0, + }; + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const Leaf* internal_default_instance() { + return reinterpret_cast( + &_Leaf_default_instance_); + } + static constexpr int kIndexInFileMessages = + 2; + + void UnsafeArenaSwap(Leaf* other); + void Swap(Leaf* other); + friend void swap(Leaf& a, Leaf& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline Leaf* New() const final { + return CreateMaybeMessage(NULL); + } + + Leaf* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const Leaf& from); + void MergeFrom(const Leaf& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(Leaf* other); + protected: + explicit Leaf(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // float scalar = 3; + void clear_scalar(); + static const int kScalarFieldNumber = 3; + float scalar() const; + void set_scalar(float value); + + // .tensorflow.boosted_trees.Vector vector = 1; + bool has_vector() const; + void clear_vector(); + static const int kVectorFieldNumber = 1; + private: + const ::tensorflow::boosted_trees::Vector& _internal_vector() const; + public: + const ::tensorflow::boosted_trees::Vector& vector() const; + ::tensorflow::boosted_trees::Vector* release_vector(); + ::tensorflow::boosted_trees::Vector* mutable_vector(); + void set_allocated_vector(::tensorflow::boosted_trees::Vector* vector); + void unsafe_arena_set_allocated_vector( + ::tensorflow::boosted_trees::Vector* vector); + ::tensorflow::boosted_trees::Vector* unsafe_arena_release_vector(); + + // .tensorflow.boosted_trees.SparseVector sparse_vector = 2; + bool has_sparse_vector() const; + void clear_sparse_vector(); + static const int kSparseVectorFieldNumber = 2; + private: + const ::tensorflow::boosted_trees::SparseVector& _internal_sparse_vector() const; + public: + const ::tensorflow::boosted_trees::SparseVector& sparse_vector() const; + ::tensorflow::boosted_trees::SparseVector* release_sparse_vector(); + ::tensorflow::boosted_trees::SparseVector* mutable_sparse_vector(); + void set_allocated_sparse_vector(::tensorflow::boosted_trees::SparseVector* sparse_vector); + void unsafe_arena_set_allocated_sparse_vector( + ::tensorflow::boosted_trees::SparseVector* sparse_vector); + ::tensorflow::boosted_trees::SparseVector* unsafe_arena_release_sparse_vector(); + + void clear_leaf(); + LeafCase leaf_case() const; + // @@protoc_insertion_point(class_scope:tensorflow.boosted_trees.Leaf) + private: + void set_has_vector(); + void set_has_sparse_vector(); + + inline bool has_leaf() const; + inline void clear_has_leaf(); + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + float scalar_; + union LeafUnion { + LeafUnion() {} + ::tensorflow::boosted_trees::Vector* vector_; + ::tensorflow::boosted_trees::SparseVector* sparse_vector_; + } leaf_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + ::google::protobuf::uint32 _oneof_case_[1]; + + friend struct ::protobuf_tensorflow_2fcore_2fkernels_2fboosted_5ftrees_2fboosted_5ftrees_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class Vector : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.boosted_trees.Vector) */ { + public: + Vector(); + virtual ~Vector(); + + Vector(const Vector& from); + + inline Vector& operator=(const Vector& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + Vector(Vector&& from) noexcept + : Vector() { + *this = ::std::move(from); + } + + inline Vector& operator=(Vector&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const Vector& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const Vector* internal_default_instance() { + return reinterpret_cast( + &_Vector_default_instance_); + } + static constexpr int kIndexInFileMessages = + 3; + + void UnsafeArenaSwap(Vector* other); + void Swap(Vector* other); + friend void swap(Vector& a, Vector& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline Vector* New() const final { + return CreateMaybeMessage(NULL); + } + + Vector* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const Vector& from); + void MergeFrom(const Vector& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(Vector* other); + protected: + explicit Vector(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated float value = 1; + int value_size() const; + void clear_value(); + static const int kValueFieldNumber = 1; + float value(int index) const; + void set_value(int index, float value); + void add_value(float value); + const ::google::protobuf::RepeatedField< float >& + value() const; + ::google::protobuf::RepeatedField< float >* + mutable_value(); + + // @@protoc_insertion_point(class_scope:tensorflow.boosted_trees.Vector) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedField< float > value_; + mutable int _value_cached_byte_size_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fkernels_2fboosted_5ftrees_2fboosted_5ftrees_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class SparseVector : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.boosted_trees.SparseVector) */ { + public: + SparseVector(); + virtual ~SparseVector(); + + SparseVector(const SparseVector& from); + + inline SparseVector& operator=(const SparseVector& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + SparseVector(SparseVector&& from) noexcept + : SparseVector() { + *this = ::std::move(from); + } + + inline SparseVector& operator=(SparseVector&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const SparseVector& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const SparseVector* internal_default_instance() { + return reinterpret_cast( + &_SparseVector_default_instance_); + } + static constexpr int kIndexInFileMessages = + 4; + + void UnsafeArenaSwap(SparseVector* other); + void Swap(SparseVector* other); + friend void swap(SparseVector& a, SparseVector& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline SparseVector* New() const final { + return CreateMaybeMessage(NULL); + } + + SparseVector* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const SparseVector& from); + void MergeFrom(const SparseVector& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(SparseVector* other); + protected: + explicit SparseVector(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated int32 index = 1; + int index_size() const; + void clear_index(); + static const int kIndexFieldNumber = 1; + ::google::protobuf::int32 index(int index) const; + void set_index(int index, ::google::protobuf::int32 value); + void add_index(::google::protobuf::int32 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::int32 >& + index() const; + ::google::protobuf::RepeatedField< ::google::protobuf::int32 >* + mutable_index(); + + // repeated float value = 2; + int value_size() const; + void clear_value(); + static const int kValueFieldNumber = 2; + float value(int index) const; + void set_value(int index, float value); + void add_value(float value); + const ::google::protobuf::RepeatedField< float >& + value() const; + ::google::protobuf::RepeatedField< float >* + mutable_value(); + + // @@protoc_insertion_point(class_scope:tensorflow.boosted_trees.SparseVector) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedField< ::google::protobuf::int32 > index_; + mutable int _index_cached_byte_size_; + ::google::protobuf::RepeatedField< float > value_; + mutable int _value_cached_byte_size_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fkernels_2fboosted_5ftrees_2fboosted_5ftrees_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class BucketizedSplit : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.boosted_trees.BucketizedSplit) */ { + public: + BucketizedSplit(); + virtual ~BucketizedSplit(); + + BucketizedSplit(const BucketizedSplit& from); + + inline BucketizedSplit& operator=(const BucketizedSplit& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + BucketizedSplit(BucketizedSplit&& from) noexcept + : BucketizedSplit() { + *this = ::std::move(from); + } + + inline BucketizedSplit& operator=(BucketizedSplit&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const BucketizedSplit& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const BucketizedSplit* internal_default_instance() { + return reinterpret_cast( + &_BucketizedSplit_default_instance_); + } + static constexpr int kIndexInFileMessages = + 5; + + void UnsafeArenaSwap(BucketizedSplit* other); + void Swap(BucketizedSplit* other); + friend void swap(BucketizedSplit& a, BucketizedSplit& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline BucketizedSplit* New() const final { + return CreateMaybeMessage(NULL); + } + + BucketizedSplit* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const BucketizedSplit& from); + void MergeFrom(const BucketizedSplit& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(BucketizedSplit* other); + protected: + explicit BucketizedSplit(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // int32 feature_id = 1; + void clear_feature_id(); + static const int kFeatureIdFieldNumber = 1; + ::google::protobuf::int32 feature_id() const; + void set_feature_id(::google::protobuf::int32 value); + + // int32 threshold = 2; + void clear_threshold(); + static const int kThresholdFieldNumber = 2; + ::google::protobuf::int32 threshold() const; + void set_threshold(::google::protobuf::int32 value); + + // int32 left_id = 3; + void clear_left_id(); + static const int kLeftIdFieldNumber = 3; + ::google::protobuf::int32 left_id() const; + void set_left_id(::google::protobuf::int32 value); + + // int32 right_id = 4; + void clear_right_id(); + static const int kRightIdFieldNumber = 4; + ::google::protobuf::int32 right_id() const; + void set_right_id(::google::protobuf::int32 value); + + // @@protoc_insertion_point(class_scope:tensorflow.boosted_trees.BucketizedSplit) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::int32 feature_id_; + ::google::protobuf::int32 threshold_; + ::google::protobuf::int32 left_id_; + ::google::protobuf::int32 right_id_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fkernels_2fboosted_5ftrees_2fboosted_5ftrees_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class CategoricalSplit : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.boosted_trees.CategoricalSplit) */ { + public: + CategoricalSplit(); + virtual ~CategoricalSplit(); + + CategoricalSplit(const CategoricalSplit& from); + + inline CategoricalSplit& operator=(const CategoricalSplit& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + CategoricalSplit(CategoricalSplit&& from) noexcept + : CategoricalSplit() { + *this = ::std::move(from); + } + + inline CategoricalSplit& operator=(CategoricalSplit&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const CategoricalSplit& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const CategoricalSplit* internal_default_instance() { + return reinterpret_cast( + &_CategoricalSplit_default_instance_); + } + static constexpr int kIndexInFileMessages = + 6; + + void UnsafeArenaSwap(CategoricalSplit* other); + void Swap(CategoricalSplit* other); + friend void swap(CategoricalSplit& a, CategoricalSplit& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline CategoricalSplit* New() const final { + return CreateMaybeMessage(NULL); + } + + CategoricalSplit* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const CategoricalSplit& from); + void MergeFrom(const CategoricalSplit& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(CategoricalSplit* other); + protected: + explicit CategoricalSplit(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // int32 feature_id = 1; + void clear_feature_id(); + static const int kFeatureIdFieldNumber = 1; + ::google::protobuf::int32 feature_id() const; + void set_feature_id(::google::protobuf::int32 value); + + // int32 value = 2; + void clear_value(); + static const int kValueFieldNumber = 2; + ::google::protobuf::int32 value() const; + void set_value(::google::protobuf::int32 value); + + // int32 left_id = 3; + void clear_left_id(); + static const int kLeftIdFieldNumber = 3; + ::google::protobuf::int32 left_id() const; + void set_left_id(::google::protobuf::int32 value); + + // int32 right_id = 4; + void clear_right_id(); + static const int kRightIdFieldNumber = 4; + ::google::protobuf::int32 right_id() const; + void set_right_id(::google::protobuf::int32 value); + + // @@protoc_insertion_point(class_scope:tensorflow.boosted_trees.CategoricalSplit) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::int32 feature_id_; + ::google::protobuf::int32 value_; + ::google::protobuf::int32 left_id_; + ::google::protobuf::int32 right_id_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fkernels_2fboosted_5ftrees_2fboosted_5ftrees_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class Tree : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.boosted_trees.Tree) */ { + public: + Tree(); + virtual ~Tree(); + + Tree(const Tree& from); + + inline Tree& operator=(const Tree& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + Tree(Tree&& from) noexcept + : Tree() { + *this = ::std::move(from); + } + + inline Tree& operator=(Tree&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const Tree& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const Tree* internal_default_instance() { + return reinterpret_cast( + &_Tree_default_instance_); + } + static constexpr int kIndexInFileMessages = + 7; + + void UnsafeArenaSwap(Tree* other); + void Swap(Tree* other); + friend void swap(Tree& a, Tree& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline Tree* New() const final { + return CreateMaybeMessage(NULL); + } + + Tree* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const Tree& from); + void MergeFrom(const Tree& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(Tree* other); + protected: + explicit Tree(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated .tensorflow.boosted_trees.Node nodes = 1; + int nodes_size() const; + void clear_nodes(); + static const int kNodesFieldNumber = 1; + ::tensorflow::boosted_trees::Node* mutable_nodes(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::boosted_trees::Node >* + mutable_nodes(); + const ::tensorflow::boosted_trees::Node& nodes(int index) const; + ::tensorflow::boosted_trees::Node* add_nodes(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::boosted_trees::Node >& + nodes() const; + + // @@protoc_insertion_point(class_scope:tensorflow.boosted_trees.Tree) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::boosted_trees::Node > nodes_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fkernels_2fboosted_5ftrees_2fboosted_5ftrees_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class TreeMetadata_PostPruneNodeUpdate : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.boosted_trees.TreeMetadata.PostPruneNodeUpdate) */ { + public: + TreeMetadata_PostPruneNodeUpdate(); + virtual ~TreeMetadata_PostPruneNodeUpdate(); + + TreeMetadata_PostPruneNodeUpdate(const TreeMetadata_PostPruneNodeUpdate& from); + + inline TreeMetadata_PostPruneNodeUpdate& operator=(const TreeMetadata_PostPruneNodeUpdate& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + TreeMetadata_PostPruneNodeUpdate(TreeMetadata_PostPruneNodeUpdate&& from) noexcept + : TreeMetadata_PostPruneNodeUpdate() { + *this = ::std::move(from); + } + + inline TreeMetadata_PostPruneNodeUpdate& operator=(TreeMetadata_PostPruneNodeUpdate&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const TreeMetadata_PostPruneNodeUpdate& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const TreeMetadata_PostPruneNodeUpdate* internal_default_instance() { + return reinterpret_cast( + &_TreeMetadata_PostPruneNodeUpdate_default_instance_); + } + static constexpr int kIndexInFileMessages = + 8; + + void UnsafeArenaSwap(TreeMetadata_PostPruneNodeUpdate* other); + void Swap(TreeMetadata_PostPruneNodeUpdate* other); + friend void swap(TreeMetadata_PostPruneNodeUpdate& a, TreeMetadata_PostPruneNodeUpdate& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline TreeMetadata_PostPruneNodeUpdate* New() const final { + return CreateMaybeMessage(NULL); + } + + TreeMetadata_PostPruneNodeUpdate* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const TreeMetadata_PostPruneNodeUpdate& from); + void MergeFrom(const TreeMetadata_PostPruneNodeUpdate& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(TreeMetadata_PostPruneNodeUpdate* other); + protected: + explicit TreeMetadata_PostPruneNodeUpdate(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // int32 new_node_id = 1; + void clear_new_node_id(); + static const int kNewNodeIdFieldNumber = 1; + ::google::protobuf::int32 new_node_id() const; + void set_new_node_id(::google::protobuf::int32 value); + + // float logit_change = 2; + void clear_logit_change(); + static const int kLogitChangeFieldNumber = 2; + float logit_change() const; + void set_logit_change(float value); + + // @@protoc_insertion_point(class_scope:tensorflow.boosted_trees.TreeMetadata.PostPruneNodeUpdate) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::int32 new_node_id_; + float logit_change_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fkernels_2fboosted_5ftrees_2fboosted_5ftrees_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class TreeMetadata : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.boosted_trees.TreeMetadata) */ { + public: + TreeMetadata(); + virtual ~TreeMetadata(); + + TreeMetadata(const TreeMetadata& from); + + inline TreeMetadata& operator=(const TreeMetadata& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + TreeMetadata(TreeMetadata&& from) noexcept + : TreeMetadata() { + *this = ::std::move(from); + } + + inline TreeMetadata& operator=(TreeMetadata&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const TreeMetadata& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const TreeMetadata* internal_default_instance() { + return reinterpret_cast( + &_TreeMetadata_default_instance_); + } + static constexpr int kIndexInFileMessages = + 9; + + void UnsafeArenaSwap(TreeMetadata* other); + void Swap(TreeMetadata* other); + friend void swap(TreeMetadata& a, TreeMetadata& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline TreeMetadata* New() const final { + return CreateMaybeMessage(NULL); + } + + TreeMetadata* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const TreeMetadata& from); + void MergeFrom(const TreeMetadata& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(TreeMetadata* other); + protected: + explicit TreeMetadata(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + typedef TreeMetadata_PostPruneNodeUpdate PostPruneNodeUpdate; + + // accessors ------------------------------------------------------- + + // repeated .tensorflow.boosted_trees.TreeMetadata.PostPruneNodeUpdate post_pruned_nodes_meta = 4; + int post_pruned_nodes_meta_size() const; + void clear_post_pruned_nodes_meta(); + static const int kPostPrunedNodesMetaFieldNumber = 4; + ::tensorflow::boosted_trees::TreeMetadata_PostPruneNodeUpdate* mutable_post_pruned_nodes_meta(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::boosted_trees::TreeMetadata_PostPruneNodeUpdate >* + mutable_post_pruned_nodes_meta(); + const ::tensorflow::boosted_trees::TreeMetadata_PostPruneNodeUpdate& post_pruned_nodes_meta(int index) const; + ::tensorflow::boosted_trees::TreeMetadata_PostPruneNodeUpdate* add_post_pruned_nodes_meta(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::boosted_trees::TreeMetadata_PostPruneNodeUpdate >& + post_pruned_nodes_meta() const; + + // int32 num_layers_grown = 2; + void clear_num_layers_grown(); + static const int kNumLayersGrownFieldNumber = 2; + ::google::protobuf::int32 num_layers_grown() const; + void set_num_layers_grown(::google::protobuf::int32 value); + + // bool is_finalized = 3; + void clear_is_finalized(); + static const int kIsFinalizedFieldNumber = 3; + bool is_finalized() const; + void set_is_finalized(bool value); + + // @@protoc_insertion_point(class_scope:tensorflow.boosted_trees.TreeMetadata) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::boosted_trees::TreeMetadata_PostPruneNodeUpdate > post_pruned_nodes_meta_; + ::google::protobuf::int32 num_layers_grown_; + bool is_finalized_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fkernels_2fboosted_5ftrees_2fboosted_5ftrees_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class GrowingMetadata : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.boosted_trees.GrowingMetadata) */ { + public: + GrowingMetadata(); + virtual ~GrowingMetadata(); + + GrowingMetadata(const GrowingMetadata& from); + + inline GrowingMetadata& operator=(const GrowingMetadata& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + GrowingMetadata(GrowingMetadata&& from) noexcept + : GrowingMetadata() { + *this = ::std::move(from); + } + + inline GrowingMetadata& operator=(GrowingMetadata&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const GrowingMetadata& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const GrowingMetadata* internal_default_instance() { + return reinterpret_cast( + &_GrowingMetadata_default_instance_); + } + static constexpr int kIndexInFileMessages = + 10; + + void UnsafeArenaSwap(GrowingMetadata* other); + void Swap(GrowingMetadata* other); + friend void swap(GrowingMetadata& a, GrowingMetadata& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline GrowingMetadata* New() const final { + return CreateMaybeMessage(NULL); + } + + GrowingMetadata* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const GrowingMetadata& from); + void MergeFrom(const GrowingMetadata& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(GrowingMetadata* other); + protected: + explicit GrowingMetadata(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // int64 num_trees_attempted = 1; + void clear_num_trees_attempted(); + static const int kNumTreesAttemptedFieldNumber = 1; + ::google::protobuf::int64 num_trees_attempted() const; + void set_num_trees_attempted(::google::protobuf::int64 value); + + // int64 num_layers_attempted = 2; + void clear_num_layers_attempted(); + static const int kNumLayersAttemptedFieldNumber = 2; + ::google::protobuf::int64 num_layers_attempted() const; + void set_num_layers_attempted(::google::protobuf::int64 value); + + // int32 last_layer_node_start = 3; + void clear_last_layer_node_start(); + static const int kLastLayerNodeStartFieldNumber = 3; + ::google::protobuf::int32 last_layer_node_start() const; + void set_last_layer_node_start(::google::protobuf::int32 value); + + // int32 last_layer_node_end = 4; + void clear_last_layer_node_end(); + static const int kLastLayerNodeEndFieldNumber = 4; + ::google::protobuf::int32 last_layer_node_end() const; + void set_last_layer_node_end(::google::protobuf::int32 value); + + // @@protoc_insertion_point(class_scope:tensorflow.boosted_trees.GrowingMetadata) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::int64 num_trees_attempted_; + ::google::protobuf::int64 num_layers_attempted_; + ::google::protobuf::int32 last_layer_node_start_; + ::google::protobuf::int32 last_layer_node_end_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fkernels_2fboosted_5ftrees_2fboosted_5ftrees_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class TreeEnsemble : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.boosted_trees.TreeEnsemble) */ { + public: + TreeEnsemble(); + virtual ~TreeEnsemble(); + + TreeEnsemble(const TreeEnsemble& from); + + inline TreeEnsemble& operator=(const TreeEnsemble& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + TreeEnsemble(TreeEnsemble&& from) noexcept + : TreeEnsemble() { + *this = ::std::move(from); + } + + inline TreeEnsemble& operator=(TreeEnsemble&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const TreeEnsemble& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const TreeEnsemble* internal_default_instance() { + return reinterpret_cast( + &_TreeEnsemble_default_instance_); + } + static constexpr int kIndexInFileMessages = + 11; + + void UnsafeArenaSwap(TreeEnsemble* other); + void Swap(TreeEnsemble* other); + friend void swap(TreeEnsemble& a, TreeEnsemble& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline TreeEnsemble* New() const final { + return CreateMaybeMessage(NULL); + } + + TreeEnsemble* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const TreeEnsemble& from); + void MergeFrom(const TreeEnsemble& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(TreeEnsemble* other); + protected: + explicit TreeEnsemble(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated .tensorflow.boosted_trees.Tree trees = 1; + int trees_size() const; + void clear_trees(); + static const int kTreesFieldNumber = 1; + ::tensorflow::boosted_trees::Tree* mutable_trees(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::boosted_trees::Tree >* + mutable_trees(); + const ::tensorflow::boosted_trees::Tree& trees(int index) const; + ::tensorflow::boosted_trees::Tree* add_trees(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::boosted_trees::Tree >& + trees() const; + + // repeated float tree_weights = 2; + int tree_weights_size() const; + void clear_tree_weights(); + static const int kTreeWeightsFieldNumber = 2; + float tree_weights(int index) const; + void set_tree_weights(int index, float value); + void add_tree_weights(float value); + const ::google::protobuf::RepeatedField< float >& + tree_weights() const; + ::google::protobuf::RepeatedField< float >* + mutable_tree_weights(); + + // repeated .tensorflow.boosted_trees.TreeMetadata tree_metadata = 3; + int tree_metadata_size() const; + void clear_tree_metadata(); + static const int kTreeMetadataFieldNumber = 3; + ::tensorflow::boosted_trees::TreeMetadata* mutable_tree_metadata(int index); + ::google::protobuf::RepeatedPtrField< ::tensorflow::boosted_trees::TreeMetadata >* + mutable_tree_metadata(); + const ::tensorflow::boosted_trees::TreeMetadata& tree_metadata(int index) const; + ::tensorflow::boosted_trees::TreeMetadata* add_tree_metadata(); + const ::google::protobuf::RepeatedPtrField< ::tensorflow::boosted_trees::TreeMetadata >& + tree_metadata() const; + + // .tensorflow.boosted_trees.GrowingMetadata growing_metadata = 4; + bool has_growing_metadata() const; + void clear_growing_metadata(); + static const int kGrowingMetadataFieldNumber = 4; + private: + const ::tensorflow::boosted_trees::GrowingMetadata& _internal_growing_metadata() const; + public: + const ::tensorflow::boosted_trees::GrowingMetadata& growing_metadata() const; + ::tensorflow::boosted_trees::GrowingMetadata* release_growing_metadata(); + ::tensorflow::boosted_trees::GrowingMetadata* mutable_growing_metadata(); + void set_allocated_growing_metadata(::tensorflow::boosted_trees::GrowingMetadata* growing_metadata); + void unsafe_arena_set_allocated_growing_metadata( + ::tensorflow::boosted_trees::GrowingMetadata* growing_metadata); + ::tensorflow::boosted_trees::GrowingMetadata* unsafe_arena_release_growing_metadata(); + + // @@protoc_insertion_point(class_scope:tensorflow.boosted_trees.TreeEnsemble) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::boosted_trees::Tree > trees_; + ::google::protobuf::RepeatedField< float > tree_weights_; + mutable int _tree_weights_cached_byte_size_; + ::google::protobuf::RepeatedPtrField< ::tensorflow::boosted_trees::TreeMetadata > tree_metadata_; + ::tensorflow::boosted_trees::GrowingMetadata* growing_metadata_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fkernels_2fboosted_5ftrees_2fboosted_5ftrees_2eproto::TableStruct; +}; +// ------------------------------------------------------------------- + +class DebugOutput : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:tensorflow.boosted_trees.DebugOutput) */ { + public: + DebugOutput(); + virtual ~DebugOutput(); + + DebugOutput(const DebugOutput& from); + + inline DebugOutput& operator=(const DebugOutput& from) { + CopyFrom(from); + return *this; + } + #if LANG_CXX11 + DebugOutput(DebugOutput&& from) noexcept + : DebugOutput() { + *this = ::std::move(from); + } + + inline DebugOutput& operator=(DebugOutput&& from) noexcept { + if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { + if (this != &from) InternalSwap(&from); + } else { + CopyFrom(from); + } + return *this; + } + #endif + inline ::google::protobuf::Arena* GetArena() const final { + return GetArenaNoVirtual(); + } + inline void* GetMaybeArenaPointer() const final { + return MaybeArenaPtr(); + } + static const ::google::protobuf::Descriptor* descriptor(); + static const DebugOutput& default_instance(); + + static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY + static inline const DebugOutput* internal_default_instance() { + return reinterpret_cast( + &_DebugOutput_default_instance_); + } + static constexpr int kIndexInFileMessages = + 12; + + void UnsafeArenaSwap(DebugOutput* other); + void Swap(DebugOutput* other); + friend void swap(DebugOutput& a, DebugOutput& b) { + a.Swap(&b); + } + + // implements Message ---------------------------------------------- + + inline DebugOutput* New() const final { + return CreateMaybeMessage(NULL); + } + + DebugOutput* New(::google::protobuf::Arena* arena) const final { + return CreateMaybeMessage(arena); + } + void CopyFrom(const ::google::protobuf::Message& from) final; + void MergeFrom(const ::google::protobuf::Message& from) final; + void CopyFrom(const DebugOutput& from); + void MergeFrom(const DebugOutput& from); + void Clear() final; + bool IsInitialized() const final; + + size_t ByteSizeLong() const final; + bool MergePartialFromCodedStream( + ::google::protobuf::io::CodedInputStream* input) final; + void SerializeWithCachedSizes( + ::google::protobuf::io::CodedOutputStream* output) const final; + ::google::protobuf::uint8* InternalSerializeWithCachedSizesToArray( + bool deterministic, ::google::protobuf::uint8* target) const final; + int GetCachedSize() const final { return _cached_size_.Get(); } + + private: + void SharedCtor(); + void SharedDtor(); + void SetCachedSize(int size) const final; + void InternalSwap(DebugOutput* other); + protected: + explicit DebugOutput(::google::protobuf::Arena* arena); + private: + static void ArenaDtor(void* object); + inline void RegisterArenaDtor(::google::protobuf::Arena* arena); + private: + inline ::google::protobuf::Arena* GetArenaNoVirtual() const { + return _internal_metadata_.arena(); + } + inline void* MaybeArenaPtr() const { + return _internal_metadata_.raw_arena_ptr(); + } + public: + + ::google::protobuf::Metadata GetMetadata() const final; + + // nested types ---------------------------------------------------- + + // accessors ------------------------------------------------------- + + // repeated int32 feature_ids = 1; + int feature_ids_size() const; + void clear_feature_ids(); + static const int kFeatureIdsFieldNumber = 1; + ::google::protobuf::int32 feature_ids(int index) const; + void set_feature_ids(int index, ::google::protobuf::int32 value); + void add_feature_ids(::google::protobuf::int32 value); + const ::google::protobuf::RepeatedField< ::google::protobuf::int32 >& + feature_ids() const; + ::google::protobuf::RepeatedField< ::google::protobuf::int32 >* + mutable_feature_ids(); + + // repeated float logits_path = 2; + int logits_path_size() const; + void clear_logits_path(); + static const int kLogitsPathFieldNumber = 2; + float logits_path(int index) const; + void set_logits_path(int index, float value); + void add_logits_path(float value); + const ::google::protobuf::RepeatedField< float >& + logits_path() const; + ::google::protobuf::RepeatedField< float >* + mutable_logits_path(); + + // @@protoc_insertion_point(class_scope:tensorflow.boosted_trees.DebugOutput) + private: + + ::google::protobuf::internal::InternalMetadataWithArena _internal_metadata_; + template friend class ::google::protobuf::Arena::InternalHelper; + typedef void InternalArenaConstructable_; + typedef void DestructorSkippable_; + ::google::protobuf::RepeatedField< ::google::protobuf::int32 > feature_ids_; + mutable int _feature_ids_cached_byte_size_; + ::google::protobuf::RepeatedField< float > logits_path_; + mutable int _logits_path_cached_byte_size_; + mutable ::google::protobuf::internal::CachedSize _cached_size_; + friend struct ::protobuf_tensorflow_2fcore_2fkernels_2fboosted_5ftrees_2fboosted_5ftrees_2eproto::TableStruct; +}; +// =================================================================== + + +// =================================================================== + +#ifdef __GNUC__ + #pragma GCC diagnostic push + #pragma GCC diagnostic ignored "-Wstrict-aliasing" +#endif // __GNUC__ +// Node + +// .tensorflow.boosted_trees.Leaf leaf = 1; +inline bool Node::has_leaf() const { + return node_case() == kLeaf; +} +inline void Node::set_has_leaf() { + _oneof_case_[0] = kLeaf; +} +inline void Node::clear_leaf() { + if (has_leaf()) { + if (GetArenaNoVirtual() == NULL) { + delete node_.leaf_; + } + clear_has_node(); + } +} +inline const ::tensorflow::boosted_trees::Leaf& Node::_internal_leaf() const { + return *node_.leaf_; +} +inline ::tensorflow::boosted_trees::Leaf* Node::release_leaf() { + // @@protoc_insertion_point(field_release:tensorflow.boosted_trees.Node.leaf) + if (has_leaf()) { + clear_has_node(); + ::tensorflow::boosted_trees::Leaf* temp = node_.leaf_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + node_.leaf_ = NULL; + return temp; + } else { + return NULL; + } +} +inline const ::tensorflow::boosted_trees::Leaf& Node::leaf() const { + // @@protoc_insertion_point(field_get:tensorflow.boosted_trees.Node.leaf) + return has_leaf() + ? *node_.leaf_ + : *reinterpret_cast< ::tensorflow::boosted_trees::Leaf*>(&::tensorflow::boosted_trees::_Leaf_default_instance_); +} +inline ::tensorflow::boosted_trees::Leaf* Node::unsafe_arena_release_leaf() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.boosted_trees.Node.leaf) + if (has_leaf()) { + clear_has_node(); + ::tensorflow::boosted_trees::Leaf* temp = node_.leaf_; + node_.leaf_ = NULL; + return temp; + } else { + return NULL; + } +} +inline void Node::unsafe_arena_set_allocated_leaf(::tensorflow::boosted_trees::Leaf* leaf) { + clear_node(); + if (leaf) { + set_has_leaf(); + node_.leaf_ = leaf; + } + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.boosted_trees.Node.leaf) +} +inline ::tensorflow::boosted_trees::Leaf* Node::mutable_leaf() { + if (!has_leaf()) { + clear_node(); + set_has_leaf(); + node_.leaf_ = CreateMaybeMessage< ::tensorflow::boosted_trees::Leaf >( + GetArenaNoVirtual()); + } + // @@protoc_insertion_point(field_mutable:tensorflow.boosted_trees.Node.leaf) + return node_.leaf_; +} + +// .tensorflow.boosted_trees.BucketizedSplit bucketized_split = 2; +inline bool Node::has_bucketized_split() const { + return node_case() == kBucketizedSplit; +} +inline void Node::set_has_bucketized_split() { + _oneof_case_[0] = kBucketizedSplit; +} +inline void Node::clear_bucketized_split() { + if (has_bucketized_split()) { + if (GetArenaNoVirtual() == NULL) { + delete node_.bucketized_split_; + } + clear_has_node(); + } +} +inline const ::tensorflow::boosted_trees::BucketizedSplit& Node::_internal_bucketized_split() const { + return *node_.bucketized_split_; +} +inline ::tensorflow::boosted_trees::BucketizedSplit* Node::release_bucketized_split() { + // @@protoc_insertion_point(field_release:tensorflow.boosted_trees.Node.bucketized_split) + if (has_bucketized_split()) { + clear_has_node(); + ::tensorflow::boosted_trees::BucketizedSplit* temp = node_.bucketized_split_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + node_.bucketized_split_ = NULL; + return temp; + } else { + return NULL; + } +} +inline const ::tensorflow::boosted_trees::BucketizedSplit& Node::bucketized_split() const { + // @@protoc_insertion_point(field_get:tensorflow.boosted_trees.Node.bucketized_split) + return has_bucketized_split() + ? *node_.bucketized_split_ + : *reinterpret_cast< ::tensorflow::boosted_trees::BucketizedSplit*>(&::tensorflow::boosted_trees::_BucketizedSplit_default_instance_); +} +inline ::tensorflow::boosted_trees::BucketizedSplit* Node::unsafe_arena_release_bucketized_split() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.boosted_trees.Node.bucketized_split) + if (has_bucketized_split()) { + clear_has_node(); + ::tensorflow::boosted_trees::BucketizedSplit* temp = node_.bucketized_split_; + node_.bucketized_split_ = NULL; + return temp; + } else { + return NULL; + } +} +inline void Node::unsafe_arena_set_allocated_bucketized_split(::tensorflow::boosted_trees::BucketizedSplit* bucketized_split) { + clear_node(); + if (bucketized_split) { + set_has_bucketized_split(); + node_.bucketized_split_ = bucketized_split; + } + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.boosted_trees.Node.bucketized_split) +} +inline ::tensorflow::boosted_trees::BucketizedSplit* Node::mutable_bucketized_split() { + if (!has_bucketized_split()) { + clear_node(); + set_has_bucketized_split(); + node_.bucketized_split_ = CreateMaybeMessage< ::tensorflow::boosted_trees::BucketizedSplit >( + GetArenaNoVirtual()); + } + // @@protoc_insertion_point(field_mutable:tensorflow.boosted_trees.Node.bucketized_split) + return node_.bucketized_split_; +} + +// .tensorflow.boosted_trees.CategoricalSplit categorical_split = 3; +inline bool Node::has_categorical_split() const { + return node_case() == kCategoricalSplit; +} +inline void Node::set_has_categorical_split() { + _oneof_case_[0] = kCategoricalSplit; +} +inline void Node::clear_categorical_split() { + if (has_categorical_split()) { + if (GetArenaNoVirtual() == NULL) { + delete node_.categorical_split_; + } + clear_has_node(); + } +} +inline const ::tensorflow::boosted_trees::CategoricalSplit& Node::_internal_categorical_split() const { + return *node_.categorical_split_; +} +inline ::tensorflow::boosted_trees::CategoricalSplit* Node::release_categorical_split() { + // @@protoc_insertion_point(field_release:tensorflow.boosted_trees.Node.categorical_split) + if (has_categorical_split()) { + clear_has_node(); + ::tensorflow::boosted_trees::CategoricalSplit* temp = node_.categorical_split_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + node_.categorical_split_ = NULL; + return temp; + } else { + return NULL; + } +} +inline const ::tensorflow::boosted_trees::CategoricalSplit& Node::categorical_split() const { + // @@protoc_insertion_point(field_get:tensorflow.boosted_trees.Node.categorical_split) + return has_categorical_split() + ? *node_.categorical_split_ + : *reinterpret_cast< ::tensorflow::boosted_trees::CategoricalSplit*>(&::tensorflow::boosted_trees::_CategoricalSplit_default_instance_); +} +inline ::tensorflow::boosted_trees::CategoricalSplit* Node::unsafe_arena_release_categorical_split() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.boosted_trees.Node.categorical_split) + if (has_categorical_split()) { + clear_has_node(); + ::tensorflow::boosted_trees::CategoricalSplit* temp = node_.categorical_split_; + node_.categorical_split_ = NULL; + return temp; + } else { + return NULL; + } +} +inline void Node::unsafe_arena_set_allocated_categorical_split(::tensorflow::boosted_trees::CategoricalSplit* categorical_split) { + clear_node(); + if (categorical_split) { + set_has_categorical_split(); + node_.categorical_split_ = categorical_split; + } + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.boosted_trees.Node.categorical_split) +} +inline ::tensorflow::boosted_trees::CategoricalSplit* Node::mutable_categorical_split() { + if (!has_categorical_split()) { + clear_node(); + set_has_categorical_split(); + node_.categorical_split_ = CreateMaybeMessage< ::tensorflow::boosted_trees::CategoricalSplit >( + GetArenaNoVirtual()); + } + // @@protoc_insertion_point(field_mutable:tensorflow.boosted_trees.Node.categorical_split) + return node_.categorical_split_; +} + +// .tensorflow.boosted_trees.NodeMetadata metadata = 777; +inline bool Node::has_metadata() const { + return this != internal_default_instance() && metadata_ != NULL; +} +inline void Node::clear_metadata() { + if (GetArenaNoVirtual() == NULL && metadata_ != NULL) { + delete metadata_; + } + metadata_ = NULL; +} +inline const ::tensorflow::boosted_trees::NodeMetadata& Node::_internal_metadata() const { + return *metadata_; +} +inline const ::tensorflow::boosted_trees::NodeMetadata& Node::metadata() const { + const ::tensorflow::boosted_trees::NodeMetadata* p = metadata_; + // @@protoc_insertion_point(field_get:tensorflow.boosted_trees.Node.metadata) + return p != NULL ? *p : *reinterpret_cast( + &::tensorflow::boosted_trees::_NodeMetadata_default_instance_); +} +inline ::tensorflow::boosted_trees::NodeMetadata* Node::release_metadata() { + // @@protoc_insertion_point(field_release:tensorflow.boosted_trees.Node.metadata) + + ::tensorflow::boosted_trees::NodeMetadata* temp = metadata_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + metadata_ = NULL; + return temp; +} +inline ::tensorflow::boosted_trees::NodeMetadata* Node::unsafe_arena_release_metadata() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.boosted_trees.Node.metadata) + + ::tensorflow::boosted_trees::NodeMetadata* temp = metadata_; + metadata_ = NULL; + return temp; +} +inline ::tensorflow::boosted_trees::NodeMetadata* Node::mutable_metadata() { + + if (metadata_ == NULL) { + auto* p = CreateMaybeMessage<::tensorflow::boosted_trees::NodeMetadata>(GetArenaNoVirtual()); + metadata_ = p; + } + // @@protoc_insertion_point(field_mutable:tensorflow.boosted_trees.Node.metadata) + return metadata_; +} +inline void Node::set_allocated_metadata(::tensorflow::boosted_trees::NodeMetadata* metadata) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete metadata_; + } + if (metadata) { + ::google::protobuf::Arena* submessage_arena = + ::google::protobuf::Arena::GetArena(metadata); + if (message_arena != submessage_arena) { + metadata = ::google::protobuf::internal::GetOwnedMessage( + message_arena, metadata, submessage_arena); + } + + } else { + + } + metadata_ = metadata; + // @@protoc_insertion_point(field_set_allocated:tensorflow.boosted_trees.Node.metadata) +} + +inline bool Node::has_node() const { + return node_case() != NODE_NOT_SET; +} +inline void Node::clear_has_node() { + _oneof_case_[0] = NODE_NOT_SET; +} +inline Node::NodeCase Node::node_case() const { + return Node::NodeCase(_oneof_case_[0]); +} +// ------------------------------------------------------------------- + +// NodeMetadata + +// float gain = 1; +inline void NodeMetadata::clear_gain() { + gain_ = 0; +} +inline float NodeMetadata::gain() const { + // @@protoc_insertion_point(field_get:tensorflow.boosted_trees.NodeMetadata.gain) + return gain_; +} +inline void NodeMetadata::set_gain(float value) { + + gain_ = value; + // @@protoc_insertion_point(field_set:tensorflow.boosted_trees.NodeMetadata.gain) +} + +// .tensorflow.boosted_trees.Leaf original_leaf = 2; +inline bool NodeMetadata::has_original_leaf() const { + return this != internal_default_instance() && original_leaf_ != NULL; +} +inline void NodeMetadata::clear_original_leaf() { + if (GetArenaNoVirtual() == NULL && original_leaf_ != NULL) { + delete original_leaf_; + } + original_leaf_ = NULL; +} +inline const ::tensorflow::boosted_trees::Leaf& NodeMetadata::_internal_original_leaf() const { + return *original_leaf_; +} +inline const ::tensorflow::boosted_trees::Leaf& NodeMetadata::original_leaf() const { + const ::tensorflow::boosted_trees::Leaf* p = original_leaf_; + // @@protoc_insertion_point(field_get:tensorflow.boosted_trees.NodeMetadata.original_leaf) + return p != NULL ? *p : *reinterpret_cast( + &::tensorflow::boosted_trees::_Leaf_default_instance_); +} +inline ::tensorflow::boosted_trees::Leaf* NodeMetadata::release_original_leaf() { + // @@protoc_insertion_point(field_release:tensorflow.boosted_trees.NodeMetadata.original_leaf) + + ::tensorflow::boosted_trees::Leaf* temp = original_leaf_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + original_leaf_ = NULL; + return temp; +} +inline ::tensorflow::boosted_trees::Leaf* NodeMetadata::unsafe_arena_release_original_leaf() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.boosted_trees.NodeMetadata.original_leaf) + + ::tensorflow::boosted_trees::Leaf* temp = original_leaf_; + original_leaf_ = NULL; + return temp; +} +inline ::tensorflow::boosted_trees::Leaf* NodeMetadata::mutable_original_leaf() { + + if (original_leaf_ == NULL) { + auto* p = CreateMaybeMessage<::tensorflow::boosted_trees::Leaf>(GetArenaNoVirtual()); + original_leaf_ = p; + } + // @@protoc_insertion_point(field_mutable:tensorflow.boosted_trees.NodeMetadata.original_leaf) + return original_leaf_; +} +inline void NodeMetadata::set_allocated_original_leaf(::tensorflow::boosted_trees::Leaf* original_leaf) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete original_leaf_; + } + if (original_leaf) { + ::google::protobuf::Arena* submessage_arena = + ::google::protobuf::Arena::GetArena(original_leaf); + if (message_arena != submessage_arena) { + original_leaf = ::google::protobuf::internal::GetOwnedMessage( + message_arena, original_leaf, submessage_arena); + } + + } else { + + } + original_leaf_ = original_leaf; + // @@protoc_insertion_point(field_set_allocated:tensorflow.boosted_trees.NodeMetadata.original_leaf) +} + +// ------------------------------------------------------------------- + +// Leaf + +// .tensorflow.boosted_trees.Vector vector = 1; +inline bool Leaf::has_vector() const { + return leaf_case() == kVector; +} +inline void Leaf::set_has_vector() { + _oneof_case_[0] = kVector; +} +inline void Leaf::clear_vector() { + if (has_vector()) { + if (GetArenaNoVirtual() == NULL) { + delete leaf_.vector_; + } + clear_has_leaf(); + } +} +inline const ::tensorflow::boosted_trees::Vector& Leaf::_internal_vector() const { + return *leaf_.vector_; +} +inline ::tensorflow::boosted_trees::Vector* Leaf::release_vector() { + // @@protoc_insertion_point(field_release:tensorflow.boosted_trees.Leaf.vector) + if (has_vector()) { + clear_has_leaf(); + ::tensorflow::boosted_trees::Vector* temp = leaf_.vector_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + leaf_.vector_ = NULL; + return temp; + } else { + return NULL; + } +} +inline const ::tensorflow::boosted_trees::Vector& Leaf::vector() const { + // @@protoc_insertion_point(field_get:tensorflow.boosted_trees.Leaf.vector) + return has_vector() + ? *leaf_.vector_ + : *reinterpret_cast< ::tensorflow::boosted_trees::Vector*>(&::tensorflow::boosted_trees::_Vector_default_instance_); +} +inline ::tensorflow::boosted_trees::Vector* Leaf::unsafe_arena_release_vector() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.boosted_trees.Leaf.vector) + if (has_vector()) { + clear_has_leaf(); + ::tensorflow::boosted_trees::Vector* temp = leaf_.vector_; + leaf_.vector_ = NULL; + return temp; + } else { + return NULL; + } +} +inline void Leaf::unsafe_arena_set_allocated_vector(::tensorflow::boosted_trees::Vector* vector) { + clear_leaf(); + if (vector) { + set_has_vector(); + leaf_.vector_ = vector; + } + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.boosted_trees.Leaf.vector) +} +inline ::tensorflow::boosted_trees::Vector* Leaf::mutable_vector() { + if (!has_vector()) { + clear_leaf(); + set_has_vector(); + leaf_.vector_ = CreateMaybeMessage< ::tensorflow::boosted_trees::Vector >( + GetArenaNoVirtual()); + } + // @@protoc_insertion_point(field_mutable:tensorflow.boosted_trees.Leaf.vector) + return leaf_.vector_; +} + +// .tensorflow.boosted_trees.SparseVector sparse_vector = 2; +inline bool Leaf::has_sparse_vector() const { + return leaf_case() == kSparseVector; +} +inline void Leaf::set_has_sparse_vector() { + _oneof_case_[0] = kSparseVector; +} +inline void Leaf::clear_sparse_vector() { + if (has_sparse_vector()) { + if (GetArenaNoVirtual() == NULL) { + delete leaf_.sparse_vector_; + } + clear_has_leaf(); + } +} +inline const ::tensorflow::boosted_trees::SparseVector& Leaf::_internal_sparse_vector() const { + return *leaf_.sparse_vector_; +} +inline ::tensorflow::boosted_trees::SparseVector* Leaf::release_sparse_vector() { + // @@protoc_insertion_point(field_release:tensorflow.boosted_trees.Leaf.sparse_vector) + if (has_sparse_vector()) { + clear_has_leaf(); + ::tensorflow::boosted_trees::SparseVector* temp = leaf_.sparse_vector_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + leaf_.sparse_vector_ = NULL; + return temp; + } else { + return NULL; + } +} +inline const ::tensorflow::boosted_trees::SparseVector& Leaf::sparse_vector() const { + // @@protoc_insertion_point(field_get:tensorflow.boosted_trees.Leaf.sparse_vector) + return has_sparse_vector() + ? *leaf_.sparse_vector_ + : *reinterpret_cast< ::tensorflow::boosted_trees::SparseVector*>(&::tensorflow::boosted_trees::_SparseVector_default_instance_); +} +inline ::tensorflow::boosted_trees::SparseVector* Leaf::unsafe_arena_release_sparse_vector() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.boosted_trees.Leaf.sparse_vector) + if (has_sparse_vector()) { + clear_has_leaf(); + ::tensorflow::boosted_trees::SparseVector* temp = leaf_.sparse_vector_; + leaf_.sparse_vector_ = NULL; + return temp; + } else { + return NULL; + } +} +inline void Leaf::unsafe_arena_set_allocated_sparse_vector(::tensorflow::boosted_trees::SparseVector* sparse_vector) { + clear_leaf(); + if (sparse_vector) { + set_has_sparse_vector(); + leaf_.sparse_vector_ = sparse_vector; + } + // @@protoc_insertion_point(field_unsafe_arena_set_allocated:tensorflow.boosted_trees.Leaf.sparse_vector) +} +inline ::tensorflow::boosted_trees::SparseVector* Leaf::mutable_sparse_vector() { + if (!has_sparse_vector()) { + clear_leaf(); + set_has_sparse_vector(); + leaf_.sparse_vector_ = CreateMaybeMessage< ::tensorflow::boosted_trees::SparseVector >( + GetArenaNoVirtual()); + } + // @@protoc_insertion_point(field_mutable:tensorflow.boosted_trees.Leaf.sparse_vector) + return leaf_.sparse_vector_; +} + +// float scalar = 3; +inline void Leaf::clear_scalar() { + scalar_ = 0; +} +inline float Leaf::scalar() const { + // @@protoc_insertion_point(field_get:tensorflow.boosted_trees.Leaf.scalar) + return scalar_; +} +inline void Leaf::set_scalar(float value) { + + scalar_ = value; + // @@protoc_insertion_point(field_set:tensorflow.boosted_trees.Leaf.scalar) +} + +inline bool Leaf::has_leaf() const { + return leaf_case() != LEAF_NOT_SET; +} +inline void Leaf::clear_has_leaf() { + _oneof_case_[0] = LEAF_NOT_SET; +} +inline Leaf::LeafCase Leaf::leaf_case() const { + return Leaf::LeafCase(_oneof_case_[0]); +} +// ------------------------------------------------------------------- + +// Vector + +// repeated float value = 1; +inline int Vector::value_size() const { + return value_.size(); +} +inline void Vector::clear_value() { + value_.Clear(); +} +inline float Vector::value(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.boosted_trees.Vector.value) + return value_.Get(index); +} +inline void Vector::set_value(int index, float value) { + value_.Set(index, value); + // @@protoc_insertion_point(field_set:tensorflow.boosted_trees.Vector.value) +} +inline void Vector::add_value(float value) { + value_.Add(value); + // @@protoc_insertion_point(field_add:tensorflow.boosted_trees.Vector.value) +} +inline const ::google::protobuf::RepeatedField< float >& +Vector::value() const { + // @@protoc_insertion_point(field_list:tensorflow.boosted_trees.Vector.value) + return value_; +} +inline ::google::protobuf::RepeatedField< float >* +Vector::mutable_value() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.boosted_trees.Vector.value) + return &value_; +} + +// ------------------------------------------------------------------- + +// SparseVector + +// repeated int32 index = 1; +inline int SparseVector::index_size() const { + return index_.size(); +} +inline void SparseVector::clear_index() { + index_.Clear(); +} +inline ::google::protobuf::int32 SparseVector::index(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.boosted_trees.SparseVector.index) + return index_.Get(index); +} +inline void SparseVector::set_index(int index, ::google::protobuf::int32 value) { + index_.Set(index, value); + // @@protoc_insertion_point(field_set:tensorflow.boosted_trees.SparseVector.index) +} +inline void SparseVector::add_index(::google::protobuf::int32 value) { + index_.Add(value); + // @@protoc_insertion_point(field_add:tensorflow.boosted_trees.SparseVector.index) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int32 >& +SparseVector::index() const { + // @@protoc_insertion_point(field_list:tensorflow.boosted_trees.SparseVector.index) + return index_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int32 >* +SparseVector::mutable_index() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.boosted_trees.SparseVector.index) + return &index_; +} + +// repeated float value = 2; +inline int SparseVector::value_size() const { + return value_.size(); +} +inline void SparseVector::clear_value() { + value_.Clear(); +} +inline float SparseVector::value(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.boosted_trees.SparseVector.value) + return value_.Get(index); +} +inline void SparseVector::set_value(int index, float value) { + value_.Set(index, value); + // @@protoc_insertion_point(field_set:tensorflow.boosted_trees.SparseVector.value) +} +inline void SparseVector::add_value(float value) { + value_.Add(value); + // @@protoc_insertion_point(field_add:tensorflow.boosted_trees.SparseVector.value) +} +inline const ::google::protobuf::RepeatedField< float >& +SparseVector::value() const { + // @@protoc_insertion_point(field_list:tensorflow.boosted_trees.SparseVector.value) + return value_; +} +inline ::google::protobuf::RepeatedField< float >* +SparseVector::mutable_value() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.boosted_trees.SparseVector.value) + return &value_; +} + +// ------------------------------------------------------------------- + +// BucketizedSplit + +// int32 feature_id = 1; +inline void BucketizedSplit::clear_feature_id() { + feature_id_ = 0; +} +inline ::google::protobuf::int32 BucketizedSplit::feature_id() const { + // @@protoc_insertion_point(field_get:tensorflow.boosted_trees.BucketizedSplit.feature_id) + return feature_id_; +} +inline void BucketizedSplit::set_feature_id(::google::protobuf::int32 value) { + + feature_id_ = value; + // @@protoc_insertion_point(field_set:tensorflow.boosted_trees.BucketizedSplit.feature_id) +} + +// int32 threshold = 2; +inline void BucketizedSplit::clear_threshold() { + threshold_ = 0; +} +inline ::google::protobuf::int32 BucketizedSplit::threshold() const { + // @@protoc_insertion_point(field_get:tensorflow.boosted_trees.BucketizedSplit.threshold) + return threshold_; +} +inline void BucketizedSplit::set_threshold(::google::protobuf::int32 value) { + + threshold_ = value; + // @@protoc_insertion_point(field_set:tensorflow.boosted_trees.BucketizedSplit.threshold) +} + +// int32 left_id = 3; +inline void BucketizedSplit::clear_left_id() { + left_id_ = 0; +} +inline ::google::protobuf::int32 BucketizedSplit::left_id() const { + // @@protoc_insertion_point(field_get:tensorflow.boosted_trees.BucketizedSplit.left_id) + return left_id_; +} +inline void BucketizedSplit::set_left_id(::google::protobuf::int32 value) { + + left_id_ = value; + // @@protoc_insertion_point(field_set:tensorflow.boosted_trees.BucketizedSplit.left_id) +} + +// int32 right_id = 4; +inline void BucketizedSplit::clear_right_id() { + right_id_ = 0; +} +inline ::google::protobuf::int32 BucketizedSplit::right_id() const { + // @@protoc_insertion_point(field_get:tensorflow.boosted_trees.BucketizedSplit.right_id) + return right_id_; +} +inline void BucketizedSplit::set_right_id(::google::protobuf::int32 value) { + + right_id_ = value; + // @@protoc_insertion_point(field_set:tensorflow.boosted_trees.BucketizedSplit.right_id) +} + +// ------------------------------------------------------------------- + +// CategoricalSplit + +// int32 feature_id = 1; +inline void CategoricalSplit::clear_feature_id() { + feature_id_ = 0; +} +inline ::google::protobuf::int32 CategoricalSplit::feature_id() const { + // @@protoc_insertion_point(field_get:tensorflow.boosted_trees.CategoricalSplit.feature_id) + return feature_id_; +} +inline void CategoricalSplit::set_feature_id(::google::protobuf::int32 value) { + + feature_id_ = value; + // @@protoc_insertion_point(field_set:tensorflow.boosted_trees.CategoricalSplit.feature_id) +} + +// int32 value = 2; +inline void CategoricalSplit::clear_value() { + value_ = 0; +} +inline ::google::protobuf::int32 CategoricalSplit::value() const { + // @@protoc_insertion_point(field_get:tensorflow.boosted_trees.CategoricalSplit.value) + return value_; +} +inline void CategoricalSplit::set_value(::google::protobuf::int32 value) { + + value_ = value; + // @@protoc_insertion_point(field_set:tensorflow.boosted_trees.CategoricalSplit.value) +} + +// int32 left_id = 3; +inline void CategoricalSplit::clear_left_id() { + left_id_ = 0; +} +inline ::google::protobuf::int32 CategoricalSplit::left_id() const { + // @@protoc_insertion_point(field_get:tensorflow.boosted_trees.CategoricalSplit.left_id) + return left_id_; +} +inline void CategoricalSplit::set_left_id(::google::protobuf::int32 value) { + + left_id_ = value; + // @@protoc_insertion_point(field_set:tensorflow.boosted_trees.CategoricalSplit.left_id) +} + +// int32 right_id = 4; +inline void CategoricalSplit::clear_right_id() { + right_id_ = 0; +} +inline ::google::protobuf::int32 CategoricalSplit::right_id() const { + // @@protoc_insertion_point(field_get:tensorflow.boosted_trees.CategoricalSplit.right_id) + return right_id_; +} +inline void CategoricalSplit::set_right_id(::google::protobuf::int32 value) { + + right_id_ = value; + // @@protoc_insertion_point(field_set:tensorflow.boosted_trees.CategoricalSplit.right_id) +} + +// ------------------------------------------------------------------- + +// Tree + +// repeated .tensorflow.boosted_trees.Node nodes = 1; +inline int Tree::nodes_size() const { + return nodes_.size(); +} +inline void Tree::clear_nodes() { + nodes_.Clear(); +} +inline ::tensorflow::boosted_trees::Node* Tree::mutable_nodes(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.boosted_trees.Tree.nodes) + return nodes_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::boosted_trees::Node >* +Tree::mutable_nodes() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.boosted_trees.Tree.nodes) + return &nodes_; +} +inline const ::tensorflow::boosted_trees::Node& Tree::nodes(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.boosted_trees.Tree.nodes) + return nodes_.Get(index); +} +inline ::tensorflow::boosted_trees::Node* Tree::add_nodes() { + // @@protoc_insertion_point(field_add:tensorflow.boosted_trees.Tree.nodes) + return nodes_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::boosted_trees::Node >& +Tree::nodes() const { + // @@protoc_insertion_point(field_list:tensorflow.boosted_trees.Tree.nodes) + return nodes_; +} + +// ------------------------------------------------------------------- + +// TreeMetadata_PostPruneNodeUpdate + +// int32 new_node_id = 1; +inline void TreeMetadata_PostPruneNodeUpdate::clear_new_node_id() { + new_node_id_ = 0; +} +inline ::google::protobuf::int32 TreeMetadata_PostPruneNodeUpdate::new_node_id() const { + // @@protoc_insertion_point(field_get:tensorflow.boosted_trees.TreeMetadata.PostPruneNodeUpdate.new_node_id) + return new_node_id_; +} +inline void TreeMetadata_PostPruneNodeUpdate::set_new_node_id(::google::protobuf::int32 value) { + + new_node_id_ = value; + // @@protoc_insertion_point(field_set:tensorflow.boosted_trees.TreeMetadata.PostPruneNodeUpdate.new_node_id) +} + +// float logit_change = 2; +inline void TreeMetadata_PostPruneNodeUpdate::clear_logit_change() { + logit_change_ = 0; +} +inline float TreeMetadata_PostPruneNodeUpdate::logit_change() const { + // @@protoc_insertion_point(field_get:tensorflow.boosted_trees.TreeMetadata.PostPruneNodeUpdate.logit_change) + return logit_change_; +} +inline void TreeMetadata_PostPruneNodeUpdate::set_logit_change(float value) { + + logit_change_ = value; + // @@protoc_insertion_point(field_set:tensorflow.boosted_trees.TreeMetadata.PostPruneNodeUpdate.logit_change) +} + +// ------------------------------------------------------------------- + +// TreeMetadata + +// int32 num_layers_grown = 2; +inline void TreeMetadata::clear_num_layers_grown() { + num_layers_grown_ = 0; +} +inline ::google::protobuf::int32 TreeMetadata::num_layers_grown() const { + // @@protoc_insertion_point(field_get:tensorflow.boosted_trees.TreeMetadata.num_layers_grown) + return num_layers_grown_; +} +inline void TreeMetadata::set_num_layers_grown(::google::protobuf::int32 value) { + + num_layers_grown_ = value; + // @@protoc_insertion_point(field_set:tensorflow.boosted_trees.TreeMetadata.num_layers_grown) +} + +// bool is_finalized = 3; +inline void TreeMetadata::clear_is_finalized() { + is_finalized_ = false; +} +inline bool TreeMetadata::is_finalized() const { + // @@protoc_insertion_point(field_get:tensorflow.boosted_trees.TreeMetadata.is_finalized) + return is_finalized_; +} +inline void TreeMetadata::set_is_finalized(bool value) { + + is_finalized_ = value; + // @@protoc_insertion_point(field_set:tensorflow.boosted_trees.TreeMetadata.is_finalized) +} + +// repeated .tensorflow.boosted_trees.TreeMetadata.PostPruneNodeUpdate post_pruned_nodes_meta = 4; +inline int TreeMetadata::post_pruned_nodes_meta_size() const { + return post_pruned_nodes_meta_.size(); +} +inline void TreeMetadata::clear_post_pruned_nodes_meta() { + post_pruned_nodes_meta_.Clear(); +} +inline ::tensorflow::boosted_trees::TreeMetadata_PostPruneNodeUpdate* TreeMetadata::mutable_post_pruned_nodes_meta(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.boosted_trees.TreeMetadata.post_pruned_nodes_meta) + return post_pruned_nodes_meta_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::boosted_trees::TreeMetadata_PostPruneNodeUpdate >* +TreeMetadata::mutable_post_pruned_nodes_meta() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.boosted_trees.TreeMetadata.post_pruned_nodes_meta) + return &post_pruned_nodes_meta_; +} +inline const ::tensorflow::boosted_trees::TreeMetadata_PostPruneNodeUpdate& TreeMetadata::post_pruned_nodes_meta(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.boosted_trees.TreeMetadata.post_pruned_nodes_meta) + return post_pruned_nodes_meta_.Get(index); +} +inline ::tensorflow::boosted_trees::TreeMetadata_PostPruneNodeUpdate* TreeMetadata::add_post_pruned_nodes_meta() { + // @@protoc_insertion_point(field_add:tensorflow.boosted_trees.TreeMetadata.post_pruned_nodes_meta) + return post_pruned_nodes_meta_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::boosted_trees::TreeMetadata_PostPruneNodeUpdate >& +TreeMetadata::post_pruned_nodes_meta() const { + // @@protoc_insertion_point(field_list:tensorflow.boosted_trees.TreeMetadata.post_pruned_nodes_meta) + return post_pruned_nodes_meta_; +} + +// ------------------------------------------------------------------- + +// GrowingMetadata + +// int64 num_trees_attempted = 1; +inline void GrowingMetadata::clear_num_trees_attempted() { + num_trees_attempted_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 GrowingMetadata::num_trees_attempted() const { + // @@protoc_insertion_point(field_get:tensorflow.boosted_trees.GrowingMetadata.num_trees_attempted) + return num_trees_attempted_; +} +inline void GrowingMetadata::set_num_trees_attempted(::google::protobuf::int64 value) { + + num_trees_attempted_ = value; + // @@protoc_insertion_point(field_set:tensorflow.boosted_trees.GrowingMetadata.num_trees_attempted) +} + +// int64 num_layers_attempted = 2; +inline void GrowingMetadata::clear_num_layers_attempted() { + num_layers_attempted_ = GOOGLE_LONGLONG(0); +} +inline ::google::protobuf::int64 GrowingMetadata::num_layers_attempted() const { + // @@protoc_insertion_point(field_get:tensorflow.boosted_trees.GrowingMetadata.num_layers_attempted) + return num_layers_attempted_; +} +inline void GrowingMetadata::set_num_layers_attempted(::google::protobuf::int64 value) { + + num_layers_attempted_ = value; + // @@protoc_insertion_point(field_set:tensorflow.boosted_trees.GrowingMetadata.num_layers_attempted) +} + +// int32 last_layer_node_start = 3; +inline void GrowingMetadata::clear_last_layer_node_start() { + last_layer_node_start_ = 0; +} +inline ::google::protobuf::int32 GrowingMetadata::last_layer_node_start() const { + // @@protoc_insertion_point(field_get:tensorflow.boosted_trees.GrowingMetadata.last_layer_node_start) + return last_layer_node_start_; +} +inline void GrowingMetadata::set_last_layer_node_start(::google::protobuf::int32 value) { + + last_layer_node_start_ = value; + // @@protoc_insertion_point(field_set:tensorflow.boosted_trees.GrowingMetadata.last_layer_node_start) +} + +// int32 last_layer_node_end = 4; +inline void GrowingMetadata::clear_last_layer_node_end() { + last_layer_node_end_ = 0; +} +inline ::google::protobuf::int32 GrowingMetadata::last_layer_node_end() const { + // @@protoc_insertion_point(field_get:tensorflow.boosted_trees.GrowingMetadata.last_layer_node_end) + return last_layer_node_end_; +} +inline void GrowingMetadata::set_last_layer_node_end(::google::protobuf::int32 value) { + + last_layer_node_end_ = value; + // @@protoc_insertion_point(field_set:tensorflow.boosted_trees.GrowingMetadata.last_layer_node_end) +} + +// ------------------------------------------------------------------- + +// TreeEnsemble + +// repeated .tensorflow.boosted_trees.Tree trees = 1; +inline int TreeEnsemble::trees_size() const { + return trees_.size(); +} +inline void TreeEnsemble::clear_trees() { + trees_.Clear(); +} +inline ::tensorflow::boosted_trees::Tree* TreeEnsemble::mutable_trees(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.boosted_trees.TreeEnsemble.trees) + return trees_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::boosted_trees::Tree >* +TreeEnsemble::mutable_trees() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.boosted_trees.TreeEnsemble.trees) + return &trees_; +} +inline const ::tensorflow::boosted_trees::Tree& TreeEnsemble::trees(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.boosted_trees.TreeEnsemble.trees) + return trees_.Get(index); +} +inline ::tensorflow::boosted_trees::Tree* TreeEnsemble::add_trees() { + // @@protoc_insertion_point(field_add:tensorflow.boosted_trees.TreeEnsemble.trees) + return trees_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::boosted_trees::Tree >& +TreeEnsemble::trees() const { + // @@protoc_insertion_point(field_list:tensorflow.boosted_trees.TreeEnsemble.trees) + return trees_; +} + +// repeated float tree_weights = 2; +inline int TreeEnsemble::tree_weights_size() const { + return tree_weights_.size(); +} +inline void TreeEnsemble::clear_tree_weights() { + tree_weights_.Clear(); +} +inline float TreeEnsemble::tree_weights(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.boosted_trees.TreeEnsemble.tree_weights) + return tree_weights_.Get(index); +} +inline void TreeEnsemble::set_tree_weights(int index, float value) { + tree_weights_.Set(index, value); + // @@protoc_insertion_point(field_set:tensorflow.boosted_trees.TreeEnsemble.tree_weights) +} +inline void TreeEnsemble::add_tree_weights(float value) { + tree_weights_.Add(value); + // @@protoc_insertion_point(field_add:tensorflow.boosted_trees.TreeEnsemble.tree_weights) +} +inline const ::google::protobuf::RepeatedField< float >& +TreeEnsemble::tree_weights() const { + // @@protoc_insertion_point(field_list:tensorflow.boosted_trees.TreeEnsemble.tree_weights) + return tree_weights_; +} +inline ::google::protobuf::RepeatedField< float >* +TreeEnsemble::mutable_tree_weights() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.boosted_trees.TreeEnsemble.tree_weights) + return &tree_weights_; +} + +// repeated .tensorflow.boosted_trees.TreeMetadata tree_metadata = 3; +inline int TreeEnsemble::tree_metadata_size() const { + return tree_metadata_.size(); +} +inline void TreeEnsemble::clear_tree_metadata() { + tree_metadata_.Clear(); +} +inline ::tensorflow::boosted_trees::TreeMetadata* TreeEnsemble::mutable_tree_metadata(int index) { + // @@protoc_insertion_point(field_mutable:tensorflow.boosted_trees.TreeEnsemble.tree_metadata) + return tree_metadata_.Mutable(index); +} +inline ::google::protobuf::RepeatedPtrField< ::tensorflow::boosted_trees::TreeMetadata >* +TreeEnsemble::mutable_tree_metadata() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.boosted_trees.TreeEnsemble.tree_metadata) + return &tree_metadata_; +} +inline const ::tensorflow::boosted_trees::TreeMetadata& TreeEnsemble::tree_metadata(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.boosted_trees.TreeEnsemble.tree_metadata) + return tree_metadata_.Get(index); +} +inline ::tensorflow::boosted_trees::TreeMetadata* TreeEnsemble::add_tree_metadata() { + // @@protoc_insertion_point(field_add:tensorflow.boosted_trees.TreeEnsemble.tree_metadata) + return tree_metadata_.Add(); +} +inline const ::google::protobuf::RepeatedPtrField< ::tensorflow::boosted_trees::TreeMetadata >& +TreeEnsemble::tree_metadata() const { + // @@protoc_insertion_point(field_list:tensorflow.boosted_trees.TreeEnsemble.tree_metadata) + return tree_metadata_; +} + +// .tensorflow.boosted_trees.GrowingMetadata growing_metadata = 4; +inline bool TreeEnsemble::has_growing_metadata() const { + return this != internal_default_instance() && growing_metadata_ != NULL; +} +inline void TreeEnsemble::clear_growing_metadata() { + if (GetArenaNoVirtual() == NULL && growing_metadata_ != NULL) { + delete growing_metadata_; + } + growing_metadata_ = NULL; +} +inline const ::tensorflow::boosted_trees::GrowingMetadata& TreeEnsemble::_internal_growing_metadata() const { + return *growing_metadata_; +} +inline const ::tensorflow::boosted_trees::GrowingMetadata& TreeEnsemble::growing_metadata() const { + const ::tensorflow::boosted_trees::GrowingMetadata* p = growing_metadata_; + // @@protoc_insertion_point(field_get:tensorflow.boosted_trees.TreeEnsemble.growing_metadata) + return p != NULL ? *p : *reinterpret_cast( + &::tensorflow::boosted_trees::_GrowingMetadata_default_instance_); +} +inline ::tensorflow::boosted_trees::GrowingMetadata* TreeEnsemble::release_growing_metadata() { + // @@protoc_insertion_point(field_release:tensorflow.boosted_trees.TreeEnsemble.growing_metadata) + + ::tensorflow::boosted_trees::GrowingMetadata* temp = growing_metadata_; + if (GetArenaNoVirtual() != NULL) { + temp = ::google::protobuf::internal::DuplicateIfNonNull(temp); + } + growing_metadata_ = NULL; + return temp; +} +inline ::tensorflow::boosted_trees::GrowingMetadata* TreeEnsemble::unsafe_arena_release_growing_metadata() { + // @@protoc_insertion_point(field_unsafe_arena_release:tensorflow.boosted_trees.TreeEnsemble.growing_metadata) + + ::tensorflow::boosted_trees::GrowingMetadata* temp = growing_metadata_; + growing_metadata_ = NULL; + return temp; +} +inline ::tensorflow::boosted_trees::GrowingMetadata* TreeEnsemble::mutable_growing_metadata() { + + if (growing_metadata_ == NULL) { + auto* p = CreateMaybeMessage<::tensorflow::boosted_trees::GrowingMetadata>(GetArenaNoVirtual()); + growing_metadata_ = p; + } + // @@protoc_insertion_point(field_mutable:tensorflow.boosted_trees.TreeEnsemble.growing_metadata) + return growing_metadata_; +} +inline void TreeEnsemble::set_allocated_growing_metadata(::tensorflow::boosted_trees::GrowingMetadata* growing_metadata) { + ::google::protobuf::Arena* message_arena = GetArenaNoVirtual(); + if (message_arena == NULL) { + delete growing_metadata_; + } + if (growing_metadata) { + ::google::protobuf::Arena* submessage_arena = + ::google::protobuf::Arena::GetArena(growing_metadata); + if (message_arena != submessage_arena) { + growing_metadata = ::google::protobuf::internal::GetOwnedMessage( + message_arena, growing_metadata, submessage_arena); + } + + } else { + + } + growing_metadata_ = growing_metadata; + // @@protoc_insertion_point(field_set_allocated:tensorflow.boosted_trees.TreeEnsemble.growing_metadata) +} + +// ------------------------------------------------------------------- + +// DebugOutput + +// repeated int32 feature_ids = 1; +inline int DebugOutput::feature_ids_size() const { + return feature_ids_.size(); +} +inline void DebugOutput::clear_feature_ids() { + feature_ids_.Clear(); +} +inline ::google::protobuf::int32 DebugOutput::feature_ids(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.boosted_trees.DebugOutput.feature_ids) + return feature_ids_.Get(index); +} +inline void DebugOutput::set_feature_ids(int index, ::google::protobuf::int32 value) { + feature_ids_.Set(index, value); + // @@protoc_insertion_point(field_set:tensorflow.boosted_trees.DebugOutput.feature_ids) +} +inline void DebugOutput::add_feature_ids(::google::protobuf::int32 value) { + feature_ids_.Add(value); + // @@protoc_insertion_point(field_add:tensorflow.boosted_trees.DebugOutput.feature_ids) +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int32 >& +DebugOutput::feature_ids() const { + // @@protoc_insertion_point(field_list:tensorflow.boosted_trees.DebugOutput.feature_ids) + return feature_ids_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int32 >* +DebugOutput::mutable_feature_ids() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.boosted_trees.DebugOutput.feature_ids) + return &feature_ids_; +} + +// repeated float logits_path = 2; +inline int DebugOutput::logits_path_size() const { + return logits_path_.size(); +} +inline void DebugOutput::clear_logits_path() { + logits_path_.Clear(); +} +inline float DebugOutput::logits_path(int index) const { + // @@protoc_insertion_point(field_get:tensorflow.boosted_trees.DebugOutput.logits_path) + return logits_path_.Get(index); +} +inline void DebugOutput::set_logits_path(int index, float value) { + logits_path_.Set(index, value); + // @@protoc_insertion_point(field_set:tensorflow.boosted_trees.DebugOutput.logits_path) +} +inline void DebugOutput::add_logits_path(float value) { + logits_path_.Add(value); + // @@protoc_insertion_point(field_add:tensorflow.boosted_trees.DebugOutput.logits_path) +} +inline const ::google::protobuf::RepeatedField< float >& +DebugOutput::logits_path() const { + // @@protoc_insertion_point(field_list:tensorflow.boosted_trees.DebugOutput.logits_path) + return logits_path_; +} +inline ::google::protobuf::RepeatedField< float >* +DebugOutput::mutable_logits_path() { + // @@protoc_insertion_point(field_mutable_list:tensorflow.boosted_trees.DebugOutput.logits_path) + return &logits_path_; +} + +#ifdef __GNUC__ + #pragma GCC diagnostic pop +#endif // __GNUC__ +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + +// ------------------------------------------------------------------- + + +// @@protoc_insertion_point(namespace_scope) + +} // namespace boosted_trees +} // namespace tensorflow + +// @@protoc_insertion_point(global_scope) + +#endif // PROTOBUF_INCLUDED_tensorflow_2fcore_2fkernels_2fboosted_5ftrees_2fboosted_5ftrees_2eproto diff --git a/linux/include/tensorflow/core/kernels/boosted_trees/quantiles/quantile_stream_resource.h b/linux/include/tensorflow/core/kernels/boosted_trees/quantiles/quantile_stream_resource.h new file mode 100644 index 000000000..1c3172427 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/boosted_trees/quantiles/quantile_stream_resource.h @@ -0,0 +1,96 @@ +// Copyright 2017 The TensorFlow Authors. All Rights Reserved. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +// ============================================================================= +#ifndef TENSORFLOW_CORE_KERNELS_BOOSTED_TREES_QUANTILES_QUANTILE_STREAM_RESOURCE_H_ +#define TENSORFLOW_CORE_KERNELS_BOOSTED_TREES_QUANTILES_QUANTILE_STREAM_RESOURCE_H_ + +#include +#include "tensorflow/core/framework/resource_mgr.h" +#include "tensorflow/core/kernels/boosted_trees/quantiles/weighted_quantiles_stream.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/mutex.h" + +namespace tensorflow { + +using QuantileStream = + boosted_trees::quantiles::WeightedQuantilesStream; + +// Quantile Stream Resource for a list of streams sharing the same number of +// quantiles, maximum elements, and epsilon. +class BoostedTreesQuantileStreamResource : public ResourceBase { + public: + BoostedTreesQuantileStreamResource(const float epsilon, + const int64 max_elements, + const int64 num_streams) + : are_buckets_ready_(false), + epsilon_(epsilon), + num_streams_(num_streams), + max_elements_(max_elements) { + streams_.reserve(num_streams_); + boundaries_.reserve(num_streams_); + for (int64 idx = 0; idx < num_streams; ++idx) { + streams_.push_back(QuantileStream(epsilon, max_elements)); + boundaries_.push_back(std::vector()); + } + } + + string DebugString() override { return "QuantileStreamResource"; } + + tensorflow::mutex* mutex() { return &mu_; } + + QuantileStream* stream(const int64 index) { return &streams_[index]; } + + const std::vector& boundaries(const int64 index) { + return boundaries_[index]; + } + + void set_boundaries(const std::vector& boundaries, const int64 index) { + boundaries_[index] = boundaries; + } + + float epsilon() const { return epsilon_; } + int64 num_streams() const { return num_streams_; } + + bool are_buckets_ready() const { return are_buckets_ready_; } + void set_buckets_ready(const bool are_buckets_ready) { + are_buckets_ready_ = are_buckets_ready; + } + + private: + ~BoostedTreesQuantileStreamResource() override {} + + // Mutex for the whole resource. + tensorflow::mutex mu_; + + // Quantile streams. + std::vector streams_; + + // Stores the boundaries. Same size as streams_. + std::vector> boundaries_; + + // Whether boundaries are created. Initially boundaries are empty until + // set_boundaries are called. + bool are_buckets_ready_; + + const float epsilon_; + const int64 num_streams_; + // An upper-bound for the number of elements. + int64 max_elements_; + + TF_DISALLOW_COPY_AND_ASSIGN(BoostedTreesQuantileStreamResource); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_BOOSTED_TREES_QUANTILES_QUANTILE_STREAM_RESOURCE_H_ diff --git a/linux/include/tensorflow/core/kernels/boosted_trees/quantiles/weighted_quantiles_buffer.h b/linux/include/tensorflow/core/kernels/boosted_trees/quantiles/weighted_quantiles_buffer.h new file mode 100644 index 000000000..07aa9831c --- /dev/null +++ b/linux/include/tensorflow/core/kernels/boosted_trees/quantiles/weighted_quantiles_buffer.h @@ -0,0 +1,132 @@ +// Copyright 2018 The TensorFlow Authors. All Rights Reserved. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +// ============================================================================= +#ifndef TENSORFLOW_CORE_KERNELS_BOOSTED_TREES_QUANTILES_WEIGHTED_QUANTILES_BUFFER_H_ +#define TENSORFLOW_CORE_KERNELS_BOOSTED_TREES_QUANTILES_WEIGHTED_QUANTILES_BUFFER_H_ + +#include +#include +#include + +#include "tensorflow/core/platform/logging.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { +namespace boosted_trees { +namespace quantiles { + +// Buffering container ideally suited for scenarios where we need +// to sort and dedupe/compact fixed chunks of a stream of weighted elements. +template > +class WeightedQuantilesBuffer { + public: + struct BufferEntry { + BufferEntry(ValueType v, WeightType w) + : value(std::move(v)), weight(std::move(w)) {} + BufferEntry() : value(), weight(0) {} + + bool operator<(const BufferEntry& other) const { + return kCompFn(value, other.value); + } + bool operator==(const BufferEntry& other) const { + return value == other.value && weight == other.weight; + } + friend std::ostream& operator<<(std::ostream& strm, + const BufferEntry& entry) { + return strm << "{" << entry.value << ", " << entry.weight << "}"; + } + ValueType value; + WeightType weight; + }; + + explicit WeightedQuantilesBuffer(int64 block_size, int64 max_elements) + : max_size_(std::min(block_size << 1, max_elements)) { + QCHECK(max_size_ > 0) << "Invalid buffer specification: (" << block_size + << ", " << max_elements << ")"; + vec_.reserve(max_size_); + } + + // Disallow copying as it's semantically non-sensical in the Squawd algorithm + // but enable move semantics. + WeightedQuantilesBuffer(const WeightedQuantilesBuffer& other) = delete; + WeightedQuantilesBuffer& operator=(const WeightedQuantilesBuffer&) = delete; + WeightedQuantilesBuffer(WeightedQuantilesBuffer&& other) = default; + WeightedQuantilesBuffer& operator=(WeightedQuantilesBuffer&& other) = default; + + // Push entry to buffer and maintain a compact representation within + // pre-defined size limit. + void PushEntry(ValueType value, WeightType weight) { + // Callers are expected to act on a full compacted buffer after the + // PushEntry call returns. + QCHECK(!IsFull()) << "Buffer already full: " << max_size_; + + // Ignore zero and negative weight entries. + if (weight <= 0) { + return; + } + + // Push back the entry to the buffer. + vec_.push_back(BufferEntry(std::move(value), std::move(weight))); + } + + // Returns a sorted vector view of the base buffer and clears the buffer. + // Callers should minimize how often this is called, ideally only right after + // the buffer becomes full. + std::vector GenerateEntryList() { + std::vector ret; + if (vec_.size() == 0) { + return ret; + } + ret.swap(vec_); + vec_.reserve(max_size_); + std::sort(ret.begin(), ret.end()); + size_t num_entries = 0; + for (size_t i = 1; i < ret.size(); ++i) { + if (ret[i].value != ret[i - 1].value) { + BufferEntry tmp = ret[i]; + ++num_entries; + ret[num_entries] = tmp; + } else { + ret[num_entries].weight += ret[i].weight; + } + } + ret.resize(num_entries + 1); + return ret; + } + + int64 Size() const { return vec_.size(); } + bool IsFull() const { return vec_.size() >= max_size_; } + void Clear() { vec_.clear(); } + + private: + using BufferVector = typename std::vector; + + // Comparison function. + static constexpr decltype(CompareFn()) kCompFn = CompareFn(); + + // Base buffer. + size_t max_size_; + BufferVector vec_; +}; + +template +constexpr decltype(CompareFn()) + WeightedQuantilesBuffer::kCompFn; + +} // namespace quantiles +} // namespace boosted_trees +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_BOOSTED_TREES_QUANTILES_WEIGHTED_QUANTILES_BUFFER_H_ diff --git a/linux/include/tensorflow/core/kernels/boosted_trees/quantiles/weighted_quantiles_stream.h b/linux/include/tensorflow/core/kernels/boosted_trees/quantiles/weighted_quantiles_stream.h new file mode 100644 index 000000000..525e2a6a6 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/boosted_trees/quantiles/weighted_quantiles_stream.h @@ -0,0 +1,330 @@ +// Copyright 2018 The TensorFlow Authors. All Rights Reserved. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +// ============================================================================= +#ifndef TENSORFLOW_CORE_KERNELS_BOOSTED_TREES_QUANTILES_WEIGHTED_QUANTILES_STREAM_H_ +#define TENSORFLOW_CORE_KERNELS_BOOSTED_TREES_QUANTILES_WEIGHTED_QUANTILES_STREAM_H_ + +#include +#include +#include + +#include "tensorflow/core/kernels/boosted_trees/quantiles/weighted_quantiles_buffer.h" +#include "tensorflow/core/kernels/boosted_trees/quantiles/weighted_quantiles_summary.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { +namespace boosted_trees { +namespace quantiles { + +// Class to compute approximate quantiles with error bound guarantees for +// weighted data sets. +// This implementation is an adaptation of techniques from the following papers: +// * (2001) Space-efficient online computation of quantile summaries. +// * (2004) Power-conserving computation of order-statistics over +// sensor networks. +// * (2007) A fast algorithm for approximate quantiles in high speed +// data streams. +// * (2016) XGBoost: A Scalable Tree Boosting System. +// +// The key ideas at play are the following: +// - Maintain an in-memory multi-level quantile summary in a way to guarantee +// a maximum approximation error of eps * W per bucket where W is the total +// weight across all points in the input dataset. +// - Two base operations are defined: MERGE and COMPRESS. MERGE combines two +// summaries guaranteeing a epsNew = max(eps1, eps2). COMPRESS compresses +// a summary to b + 1 elements guaranteeing epsNew = epsOld + 1/b. +// - b * sizeof(summary entry) must ideally be small enough to fit in an +// average CPU L2 cache. +// - To distribute this algorithm with maintaining error bounds, we need +// the worker-computed summaries to have no more than eps / h error +// where h is the height of the distributed computation graph which +// is 2 for an MR with no combiner. +// +// We mainly want to max out IO bw by ensuring we're not compute-bound and +// using a reasonable amount of RAM. +// +// Complexity: +// Compute: O(n * log(1/eps * log(eps * n))). +// Memory: O(1/eps * log^2(eps * n)) <- for one worker streaming through the +// entire dataset. +// An epsilon value of zero would make the algorithm extremely inefficent and +// therefore, is disallowed. +template > +class WeightedQuantilesStream { + public: + using Buffer = WeightedQuantilesBuffer; + using BufferEntry = typename Buffer::BufferEntry; + using Summary = WeightedQuantilesSummary; + using SummaryEntry = typename Summary::SummaryEntry; + + explicit WeightedQuantilesStream(double eps, int64 max_elements) + : eps_(eps), buffer_(1LL, 2LL), finalized_(false) { + // See the class documentation. An epsilon value of zero could cause + // perfoamance issues. + QCHECK(eps > 0) << "An epsilon value of zero is not allowed."; + std::tie(max_levels_, block_size_) = GetQuantileSpecs(eps, max_elements); + buffer_ = Buffer(block_size_, max_elements); + summary_levels_.reserve(max_levels_); + } + + // Disallow copy and assign but enable move semantics for the stream. + WeightedQuantilesStream(const WeightedQuantilesStream& other) = delete; + WeightedQuantilesStream& operator=(const WeightedQuantilesStream&) = delete; + WeightedQuantilesStream(WeightedQuantilesStream&& other) = default; + WeightedQuantilesStream& operator=(WeightedQuantilesStream&& other) = default; + + // Pushes one entry while maintaining approximation error invariants. + void PushEntry(const ValueType& value, const WeightType& weight) { + // Validate state. + QCHECK(!finalized_) << "Finalize() already called."; + + // Push element to base buffer. + buffer_.PushEntry(value, weight); + + // When compacted buffer is full we need to compress + // and push weighted quantile summary up the level chain. + if (buffer_.IsFull()) { + PushBuffer(buffer_); + } + } + + // Pushes full buffer while maintaining approximation error invariants. + void PushBuffer(Buffer& buffer) { + // Validate state. + QCHECK(!finalized_) << "Finalize() already called."; + + // Create local compressed summary and propagate. + local_summary_.BuildFromBufferEntries(buffer.GenerateEntryList()); + local_summary_.Compress(block_size_, eps_); + PropagateLocalSummary(); + } + + // Pushes full summary while maintaining approximation error invariants. + void PushSummary(const std::vector& summary) { + // Validate state. + QCHECK(!finalized_) << "Finalize() already called."; + + // Create local compressed summary and propagate. + local_summary_.BuildFromSummaryEntries(summary); + local_summary_.Compress(block_size_, eps_); + PropagateLocalSummary(); + } + + // Flushes approximator and finalizes state. + void Finalize() { + // Validate state. + QCHECK(!finalized_) << "Finalize() may only be called once."; + + // Flush any remaining buffer elements. + PushBuffer(buffer_); + + // Create final merged summary. + local_summary_.Clear(); + for (auto& summary : summary_levels_) { + local_summary_.Merge(summary); + summary.Clear(); + } + summary_levels_.clear(); + summary_levels_.shrink_to_fit(); + finalized_ = true; + } + + // Generates requested number of quantiles after finalizing stream. + // The returned quantiles can be queried using std::lower_bound to get + // the bucket for a given value. + std::vector GenerateQuantiles(int64 num_quantiles) const { + // Validate state. + QCHECK(finalized_) + << "Finalize() must be called before generating quantiles."; + return local_summary_.GenerateQuantiles(num_quantiles); + } + + // Generates requested number of boundaries after finalizing stream. + // The returned boundaries can be queried using std::lower_bound to get + // the bucket for a given value. + // The boundaries, while still guaranteeing approximation bounds, don't + // necessarily represent the actual quantiles of the distribution. + // Boundaries are preferable over quantiles when the caller is less + // interested in the actual quantiles distribution and more interested in + // getting a representative sample of boundary values. + std::vector GenerateBoundaries(int64 num_boundaries) const { + // Validate state. + QCHECK(finalized_) + << "Finalize() must be called before generating boundaries."; + return local_summary_.GenerateBoundaries(num_boundaries); + } + + // Calculates approximation error for the specified level. + // If the passed level is negative, the approximation error for the entire + // summary is returned. Note that after Finalize is called, only the overall + // error is available. + WeightType ApproximationError(int64 level = -1) const { + if (finalized_) { + QCHECK(level <= 0) << "Only overall error is available after Finalize()"; + return local_summary_.ApproximationError(); + } + + if (summary_levels_.empty()) { + // No error even if base buffer isn't empty. + return 0; + } + + // If level is negative, we get the approximation error + // for the top-most level which is the max approximation error + // in all summaries by construction. + if (level < 0) { + level = summary_levels_.size() - 1; + } + QCHECK(level < summary_levels_.size()) << "Invalid level."; + return summary_levels_[level].ApproximationError(); + } + + size_t MaxDepth() const { return summary_levels_.size(); } + + // Generates requested number of quantiles after finalizing stream. + const Summary& GetFinalSummary() const { + // Validate state. + QCHECK(finalized_) + << "Finalize() must be called before requesting final summary."; + return local_summary_; + } + + // Helper method which, given the desired approximation error + // and an upper bound on the number of elements, computes the optimal + // number of levels and block size and returns them in the tuple. + static std::tuple GetQuantileSpecs(double eps, + int64 max_elements); + + // Serializes the internal state of the stream. + std::vector SerializeInternalSummaries() const { + // The buffer should be empty for serialize to work. + QCHECK_EQ(buffer_.Size(), 0); + std::vector result; + result.reserve(summary_levels_.size() + 1); + for (const Summary& summary : summary_levels_) { + result.push_back(summary); + } + result.push_back(local_summary_); + return result; + } + + // Resets the state of the stream with a serialized state. + void DeserializeInternalSummaries(const std::vector& summaries) { + // Clear the state before deserializing. + buffer_.Clear(); + summary_levels_.clear(); + local_summary_.Clear(); + QCHECK_GT(max_levels_, summaries.size() - 1); + for (int i = 0; i < summaries.size() - 1; ++i) { + summary_levels_.push_back(summaries[i]); + } + local_summary_ = summaries[summaries.size() - 1]; + } + + private: + // Propagates local summary through summary levels while maintaining + // approximation error invariants. + void PropagateLocalSummary() { + // Validate state. + QCHECK(!finalized_) << "Finalize() already called."; + + // No-op if there's nothing to add. + if (local_summary_.Size() <= 0) { + return; + } + + // Propagate summary through levels. + size_t level = 0; + for (bool settled = false; !settled; ++level) { + // Ensure we have enough depth. + if (summary_levels_.size() <= level) { + summary_levels_.emplace_back(); + } + + // Merge summaries. + Summary& current_summary = summary_levels_[level]; + local_summary_.Merge(current_summary); + + // Check if we need to compress and propagate summary higher. + if (current_summary.Size() == 0 || + local_summary_.Size() <= block_size_ + 1) { + current_summary = std::move(local_summary_); + settled = true; + } else { + // Compress, empty current level and propagate. + local_summary_.Compress(block_size_, eps_); + current_summary.Clear(); + } + } + } + + // Desired approximation precision. + double eps_; + // Maximum number of levels. + int64 max_levels_; + // Max block size per level. + int64 block_size_; + // Base buffer. + Buffer buffer_; + // Local summary used to minimize memory allocation and cache misses. + // After the stream is finalized, this summary holds the final quantile + // estimates. + Summary local_summary_; + // Summary levels; + std::vector summary_levels_; + // Flag indicating whether the stream is finalized. + bool finalized_; +}; + +template +inline std::tuple +WeightedQuantilesStream::GetQuantileSpecs( + double eps, int64 max_elements) { + int64 max_level = 1LL; + int64 block_size = 2LL; + QCHECK(eps >= 0 && eps < 1); + QCHECK_GT(max_elements, 0); + + if (eps <= std::numeric_limits::epsilon()) { + // Exact quantile computation at the expense of RAM. + max_level = 1; + block_size = std::max(max_elements, int64{2}); + } else { + // The bottom-most level will become full at most + // (max_elements / block_size) times, the level above will become full + // (max_elements / 2 * block_size) times and generally level l becomes + // full (max_elements / 2^l * block_size) times until the last + // level max_level becomes full at most once meaning when the inequality + // (2^max_level * block_size >= max_elements) is satisfied. + // In what follows, we jointly solve for max_level and block_size by + // gradually increasing the level until the inequality above is satisfied. + // We could alternatively set max_level = ceil(log2(eps * max_elements)); + // and block_size = ceil(max_level / eps) + 1 but that tends to give more + // pessimistic bounds and wastes RAM needlessly. + for (max_level = 1, block_size = 2; + (1LL << max_level) * block_size < max_elements; ++max_level) { + // Update upper bound on block size at current level, we always + // increase the estimate by 2 to hold the min/max elements seen so far. + block_size = static_cast(ceil(max_level / eps)) + 1; + } + } + return std::make_tuple(max_level, std::max(block_size, int64{2})); +} + +} // namespace quantiles +} // namespace boosted_trees +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_BOOSTED_TREES_QUANTILES_WEIGHTED_QUANTILES_STREAM_H_ diff --git a/linux/include/tensorflow/core/kernels/boosted_trees/quantiles/weighted_quantiles_summary.h b/linux/include/tensorflow/core/kernels/boosted_trees/quantiles/weighted_quantiles_summary.h new file mode 100644 index 000000000..31d7fe25a --- /dev/null +++ b/linux/include/tensorflow/core/kernels/boosted_trees/quantiles/weighted_quantiles_summary.h @@ -0,0 +1,344 @@ +// Copyright 2018 The TensorFlow Authors. All Rights Reserved. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +// ============================================================================= +#ifndef TENSORFLOW_CORE_KERNELS_BOOSTED_TREES_QUANTILES_WEIGHTED_QUANTILES_SUMMARY_H_ +#define TENSORFLOW_CORE_KERNELS_BOOSTED_TREES_QUANTILES_WEIGHTED_QUANTILES_SUMMARY_H_ + +#include +#include + +#include "tensorflow/core/kernels/boosted_trees/quantiles/weighted_quantiles_buffer.h" + +namespace tensorflow { +namespace boosted_trees { +namespace quantiles { + +// Summary holding a sorted block of entries with upper bound guarantees +// over the approximation error. +template > +class WeightedQuantilesSummary { + public: + using Buffer = WeightedQuantilesBuffer; + using BufferEntry = typename Buffer::BufferEntry; + + struct SummaryEntry { + SummaryEntry(const ValueType& v, const WeightType& w, const WeightType& min, + const WeightType& max) { + // Explicitly initialize all of memory (including padding from memory + // alignment) to allow the struct to be msan-resistant "plain old data". + // + // POD = http://en.cppreference.com/w/cpp/concept/PODType + memset(this, 0, sizeof(*this)); + + value = v; + weight = w; + min_rank = min; + max_rank = max; + } + + SummaryEntry() { + memset(this, 0, sizeof(*this)); + + value = ValueType(); + weight = 0; + min_rank = 0; + max_rank = 0; + } + + bool operator==(const SummaryEntry& other) const { + return value == other.value && weight == other.weight && + min_rank == other.min_rank && max_rank == other.max_rank; + } + friend std::ostream& operator<<(std::ostream& strm, + const SummaryEntry& entry) { + return strm << "{" << entry.value << ", " << entry.weight << ", " + << entry.min_rank << ", " << entry.max_rank << "}"; + } + + // Max rank estimate for previous smaller value. + WeightType PrevMaxRank() const { return max_rank - weight; } + + // Min rank estimate for next larger value. + WeightType NextMinRank() const { return min_rank + weight; } + + ValueType value; + WeightType weight; + WeightType min_rank; + WeightType max_rank; + }; + + // Re-construct summary from the specified buffer. + void BuildFromBufferEntries(const std::vector& buffer_entries) { + entries_.clear(); + entries_.reserve(buffer_entries.size()); + WeightType cumulative_weight = 0; + for (const auto& entry : buffer_entries) { + WeightType current_weight = entry.weight; + entries_.emplace_back(entry.value, entry.weight, cumulative_weight, + cumulative_weight + current_weight); + cumulative_weight += current_weight; + } + } + + // Re-construct summary from the specified summary entries. + void BuildFromSummaryEntries( + const std::vector& summary_entries) { + entries_.clear(); + entries_.reserve(summary_entries.size()); + entries_.insert(entries_.begin(), summary_entries.begin(), + summary_entries.end()); + } + + // Merges two summaries through an algorithm that's derived from MergeSort + // for summary entries while guaranteeing that the max approximation error + // of the final merged summary is no greater than the approximation errors + // of each individual summary. + // For example consider summaries where each entry is of the form + // (element, weight, min rank, max rank): + // summary entries 1: (1, 3, 0, 3), (4, 2, 3, 5) + // summary entries 2: (3, 1, 0, 1), (4, 1, 1, 2) + // merged: (1, 3, 0, 3), (3, 1, 3, 4), (4, 3, 4, 7). + void Merge(const WeightedQuantilesSummary& other_summary) { + // Make sure we have something to merge. + const auto& other_entries = other_summary.entries_; + if (other_entries.empty()) { + return; + } + if (entries_.empty()) { + BuildFromSummaryEntries(other_summary.entries_); + return; + } + + // Move current entries to make room for a new buffer. + std::vector base_entries(std::move(entries_)); + entries_.clear(); + entries_.reserve(base_entries.size() + other_entries.size()); + + // Merge entries maintaining ranks. The idea is to stack values + // in order which we can do in linear time as the two summaries are + // already sorted. We keep track of the next lower rank from either + // summary and update it as we pop elements from the summaries. + // We handle the special case when the next two elements from either + // summary are equal, in which case we just merge the two elements + // and simultaneously update both ranks. + auto it1 = base_entries.cbegin(); + auto it2 = other_entries.cbegin(); + WeightType next_min_rank1 = 0; + WeightType next_min_rank2 = 0; + while (it1 != base_entries.cend() && it2 != other_entries.cend()) { + if (kCompFn(it1->value, it2->value)) { // value1 < value2 + // Take value1 and use the last added value2 to compute + // the min rank and the current value2 to compute the max rank. + entries_.emplace_back(it1->value, it1->weight, + it1->min_rank + next_min_rank2, + it1->max_rank + it2->PrevMaxRank()); + // Update next min rank 1. + next_min_rank1 = it1->NextMinRank(); + ++it1; + } else if (kCompFn(it2->value, it1->value)) { // value1 > value2 + // Take value2 and use the last added value1 to compute + // the min rank and the current value1 to compute the max rank. + entries_.emplace_back(it2->value, it2->weight, + it2->min_rank + next_min_rank1, + it2->max_rank + it1->PrevMaxRank()); + // Update next min rank 2. + next_min_rank2 = it2->NextMinRank(); + ++it2; + } else { // value1 == value2 + // Straight additive merger of the two entries into one. + entries_.emplace_back(it1->value, it1->weight + it2->weight, + it1->min_rank + it2->min_rank, + it1->max_rank + it2->max_rank); + // Update next min ranks for both. + next_min_rank1 = it1->NextMinRank(); + next_min_rank2 = it2->NextMinRank(); + ++it1; + ++it2; + } + } + + // Fill in any residual. + while (it1 != base_entries.cend()) { + entries_.emplace_back(it1->value, it1->weight, + it1->min_rank + next_min_rank2, + it1->max_rank + other_entries.back().max_rank); + ++it1; + } + while (it2 != other_entries.cend()) { + entries_.emplace_back(it2->value, it2->weight, + it2->min_rank + next_min_rank1, + it2->max_rank + base_entries.back().max_rank); + ++it2; + } + } + + // Compresses buffer into desired size. The size specification is + // considered a hint as we always keep the first and last elements and + // maintain strict approximation error bounds. + // The approximation error delta is taken as the max of either the requested + // min error or 1 / size_hint. + // After compression, the approximation error is guaranteed to increase + // by no more than that error delta. + // This algorithm is linear in the original size of the summary and is + // designed to be cache-friendly. + void Compress(int64 size_hint, double min_eps = 0) { + // No-op if we're already within the size requirement. + size_hint = std::max(size_hint, int64{2}); + if (entries_.size() <= size_hint) { + return; + } + + // First compute the max error bound delta resulting from this compression. + double eps_delta = TotalWeight() * std::max(1.0 / size_hint, min_eps); + + // Compress elements ensuring approximation bounds and elements diversity + // are both maintained. + int64 add_accumulator = 0, add_step = entries_.size(); + auto write_it = entries_.begin() + 1, last_it = write_it; + for (auto read_it = entries_.begin(); read_it + 1 != entries_.end();) { + auto next_it = read_it + 1; + while (next_it != entries_.end() && add_accumulator < add_step && + next_it->PrevMaxRank() - read_it->NextMinRank() <= eps_delta) { + add_accumulator += size_hint; + ++next_it; + } + if (read_it == next_it - 1) { + ++read_it; + } else { + read_it = next_it - 1; + } + (*write_it++) = (*read_it); + last_it = read_it; + add_accumulator -= add_step; + } + // Write last element and resize. + if (last_it + 1 != entries_.end()) { + (*write_it++) = entries_.back(); + } + entries_.resize(write_it - entries_.begin()); + } + + // To construct the boundaries we first run a soft compress over a copy + // of the summary and retrieve the values. + // The resulting boundaries are guaranteed to both contain at least + // num_boundaries unique elements and maintain approximation bounds. + std::vector GenerateBoundaries(int64 num_boundaries) const { + std::vector output; + if (entries_.empty()) { + return output; + } + + // Generate soft compressed summary. + WeightedQuantilesSummary + compressed_summary; + compressed_summary.BuildFromSummaryEntries(entries_); + // Set an epsilon for compression that's at most 1.0 / num_boundaries + // more than epsilon of original our summary since the compression operation + // adds ~1.0/num_boundaries to final approximation error. + float compression_eps = ApproximationError() + (1.0 / num_boundaries); + compressed_summary.Compress(num_boundaries, compression_eps); + + // Return boundaries. + output.reserve(compressed_summary.entries_.size()); + for (const auto& entry : compressed_summary.entries_) { + output.push_back(entry.value); + } + return output; + } + + // To construct the desired n-quantiles we repetitively query n ranks from the + // original summary. The following algorithm is an efficient cache-friendly + // O(n) implementation of that idea which avoids the cost of the repetitive + // full rank queries O(nlogn). + std::vector GenerateQuantiles(int64 num_quantiles) const { + std::vector output; + if (entries_.empty()) { + return output; + } + num_quantiles = std::max(num_quantiles, int64{2}); + output.reserve(num_quantiles + 1); + + // Make successive rank queries to get boundaries. + // We always keep the first (min) and last (max) entries. + for (size_t cur_idx = 0, rank = 0; rank <= num_quantiles; ++rank) { + // This step boils down to finding the next element sub-range defined by + // r = (rmax[i + 1] + rmin[i + 1]) / 2 where the desired rank d < r. + WeightType d_2 = 2 * (rank * entries_.back().max_rank / num_quantiles); + size_t next_idx = cur_idx + 1; + while (next_idx < entries_.size() && + d_2 >= entries_[next_idx].min_rank + entries_[next_idx].max_rank) { + ++next_idx; + } + cur_idx = next_idx - 1; + + // Determine insertion order. + if (next_idx == entries_.size() || + d_2 < entries_[cur_idx].NextMinRank() + + entries_[next_idx].PrevMaxRank()) { + output.push_back(entries_[cur_idx].value); + } else { + output.push_back(entries_[next_idx].value); + } + } + return output; + } + + // Calculates current approximation error which should always be <= eps. + double ApproximationError() const { + if (entries_.empty()) { + return 0; + } + + WeightType max_gap = 0; + for (auto it = entries_.cbegin() + 1; it < entries_.end(); ++it) { + max_gap = std::max(max_gap, + std::max(it->max_rank - it->min_rank - it->weight, + it->PrevMaxRank() - (it - 1)->NextMinRank())); + } + return static_cast(max_gap) / TotalWeight(); + } + + ValueType MinValue() const { + return !entries_.empty() ? entries_.front().value + : std::numeric_limits::max(); + } + ValueType MaxValue() const { + return !entries_.empty() ? entries_.back().value + : std::numeric_limits::max(); + } + WeightType TotalWeight() const { + return !entries_.empty() ? entries_.back().max_rank : 0; + } + int64 Size() const { return entries_.size(); } + void Clear() { entries_.clear(); } + const std::vector& GetEntryList() const { return entries_; } + + private: + // Comparison function. + static constexpr decltype(CompareFn()) kCompFn = CompareFn(); + + // Summary entries. + std::vector entries_; +}; + +template +constexpr decltype(CompareFn()) + WeightedQuantilesSummary::kCompFn; + +} // namespace quantiles +} // namespace boosted_trees +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_BOOSTED_TREES_QUANTILES_WEIGHTED_QUANTILES_SUMMARY_H_ diff --git a/linux/include/tensorflow/core/kernels/boosted_trees/resources.h b/linux/include/tensorflow/core/kernels/boosted_trees/resources.h new file mode 100644 index 000000000..f961ed381 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/boosted_trees/resources.h @@ -0,0 +1,173 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_BOOSTED_TREES_RESOURCES_H_ +#define TENSORFLOW_CORE_KERNELS_BOOSTED_TREES_RESOURCES_H_ + +#include "tensorflow/core/framework/resource_mgr.h" +#include "tensorflow/core/platform/mutex.h" +#include "tensorflow/core/platform/protobuf.h" + +namespace tensorflow { + +// Forward declaration for proto class TreeEnsemble +namespace boosted_trees { +class TreeEnsemble; +} // namespace boosted_trees + +// A StampedResource is a resource that has a stamp token associated with it. +// Before reading from or applying updates to the resource, the stamp should +// be checked to verify that the update is not stale. +class StampedResource : public ResourceBase { + public: + StampedResource() : stamp_(-1) {} + + bool is_stamp_valid(int64 stamp) const { return stamp_ == stamp; } + + int64 stamp() const { return stamp_; } + void set_stamp(int64 stamp) { stamp_ = stamp; } + + private: + int64 stamp_; +}; + +// Keep a tree ensemble in memory for efficient evaluation and mutation. +class BoostedTreesEnsembleResource : public StampedResource { + public: + BoostedTreesEnsembleResource(); + + string DebugString() override; + + bool InitFromSerialized(const string& serialized, const int64 stamp_token); + + string SerializeAsString() const; + + int32 num_trees() const; + + // Find the next node to which the example (specified by index_in_batch) + // traverses down from the current node indicated by tree_id and node_id. + // Args: + // tree_id: the index of the tree in the ensemble. + // node_id: the index of the node within the tree. + // index_in_batch: the index of the example within the batch (relevant to + // the index of the row to read in each bucketized_features). + // bucketized_features: vector of feature Vectors. + int32 next_node( + const int32 tree_id, const int32 node_id, const int32 index_in_batch, + const std::vector::ConstVec>& bucketized_features) const; + + float node_value(const int32 tree_id, const int32 node_id) const; + + void set_node_value(const int32 tree_id, const int32 node_id, + const float logits); + + int32 GetNumLayersGrown(const int32 tree_id) const; + + void SetNumLayersGrown(const int32 tree_id, int32 new_num_layers) const; + + void UpdateLastLayerNodesRange(const int32 node_range_start, + int32 node_range_end) const; + + void GetLastLayerNodesRange(int32* node_range_start, + int32* node_range_end) const; + + int64 GetNumNodes(const int32 tree_id); + + void UpdateGrowingMetadata() const; + + int32 GetNumLayersAttempted(); + + bool is_leaf(const int32 tree_id, const int32 node_id) const; + + int32 feature_id(const int32 tree_id, const int32 node_id) const; + + int32 bucket_threshold(const int32 tree_id, const int32 node_id) const; + + int32 left_id(const int32 tree_id, const int32 node_id) const; + + int32 right_id(const int32 tree_id, const int32 node_id) const; + + // Add a tree to the ensemble and returns a new tree_id. + int32 AddNewTree(const float weight); + + // Adds new tree with one node to the ensemble and sets node's value to logits + int32 AddNewTreeWithLogits(const float weight, const float logits); + + // Grows the tree by adding a split and leaves. + void AddBucketizedSplitNode(const int32 tree_id, const int32 node_id, + const int32 feature_id, const int32 threshold, + const float gain, const float left_contrib, + const float right_contrib, int32* left_node_id, + int32* right_node_id); + + // Retrieves tree weights and returns as a vector. + // It involves a copy, so should be called only sparingly (like once per + // iteration, not per example). + std::vector GetTreeWeights() const; + + float GetTreeWeight(const int32 tree_id) const; + + float IsTreeFinalized(const int32 tree_id) const; + + float IsTreePostPruned(const int32 tree_id) const; + + void SetIsFinalized(const int32 tree_id, const bool is_finalized); + + // Sets the weight of i'th tree. + void SetTreeWeight(const int32 tree_id, const float weight); + + // Resets the resource and frees the protos in arena. + // Caller needs to hold the mutex lock while calling this. + virtual void Reset(); + + void PostPruneTree(const int32 current_tree); + + // For a given node, returns the id in a pruned tree, as well as correction + // to the cached prediction that should be applied. If tree was not + // post-pruned, current_node_id will be equal to initial_node_id and logit + // update will be equal to zero. + void GetPostPruneCorrection(const int32 tree_id, const int32 initial_node_id, + int32* current_node_id, + float* logit_update) const; + mutex* get_mutex() { return &mu_; } + + private: + // Helper method to check whether a node is a terminal node in that it + // only has leaf nodes as children. + bool IsTerminalSplitNode(const int32 tree_id, const int32 node_id) const; + + // For each pruned node, finds the leaf where it finally ended up and + // calculates the total update from that pruned node prediction. + void CalculateParentAndLogitUpdate( + const int32 start_node_id, + const std::vector>& nodes_change, + int32* parent_id, float* change) const; + + // Helper method to collect the information to be used to prune some nodes in + // the tree. + void RecursivelyDoPostPrunePreparation( + const int32 tree_id, const int32 node_id, + std::vector* nodes_to_delete, + std::vector>* nodes_meta); + + protected: + protobuf::Arena arena_; + mutex mu_; + boosted_trees::TreeEnsemble* tree_ensemble_; +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_BOOSTED_TREES_RESOURCES_H_ diff --git a/linux/include/tensorflow/core/kernels/boosted_trees/tree_helper.h b/linux/include/tensorflow/core/kernels/boosted_trees/tree_helper.h new file mode 100644 index 000000000..8b18d9e5f --- /dev/null +++ b/linux/include/tensorflow/core/kernels/boosted_trees/tree_helper.h @@ -0,0 +1,69 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_BOOSTED_TREES_TREE_HELPER_H_ +#define TENSORFLOW_CORE_KERNELS_BOOSTED_TREES_TREE_HELPER_H_ +#include + +namespace tensorflow { + +static bool GainsAreEqual(const float g1, const float g2) { + const float kTolerance = 1e-15; + return std::abs(g1 - g2) < kTolerance; +} + +static bool GainIsLarger(const float g1, const float g2) { + const float kTolerance = 1e-15; + return g1 - g2 >= kTolerance; +} + +static void CalculateWeightsAndGains(const float g, const float h, + const float l1, const float l2, + float* weight, float* gain) { + const float kEps = 1e-15; + // The formula for weight is -(g+l1*sgn(w))/(H+l2), for gain it is + // (g+l1*sgn(w))^2/(h+l2). + // This is because for each leaf we optimize + // 1/2(h+l2)*w^2+g*w+l1*abs(w) + float g_with_l1 = g; + // Apply L1 regularization. + // 1) Assume w>0 => w=-(g+l1)/(h+l2)=> g+l1 < 0 => g < -l1 + // 2) Assume w<0 => w=-(g-l1)/(h+l2)=> g-l1 > 0 => g > l1 + // For g from (-l1, l1), thus there is no solution => set to 0. + if (l1 > 0) { + if (g > l1) { + g_with_l1 -= l1; + } else if (g < -l1) { + g_with_l1 += l1; + } else { + *weight = 0.0; + *gain = 0.0; + return; + } + } + // Apply L2 regularization. + if (h + l2 <= kEps) { + // Avoid division by 0 or infinitesimal. + *weight = 0; + *gain = 0; + } else { + *weight = -g_with_l1 / (h + l2); + *gain = -g_with_l1 * (*weight); + } +} + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_BOOSTED_TREES_TREE_HELPER_H_ diff --git a/linux/include/tensorflow/core/kernels/bounds_check.h b/linux/include/tensorflow/core/kernels/bounds_check.h new file mode 100644 index 000000000..18727c0db --- /dev/null +++ b/linux/include/tensorflow/core/kernels/bounds_check.h @@ -0,0 +1,54 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_BOUNDS_CHECK_H_ +#define TENSORFLOW_CORE_KERNELS_BOUNDS_CHECK_H_ + +#include + +#include "third_party/eigen3/Eigen/Core" +#include "tensorflow/core/platform/macros.h" + +namespace tensorflow { + +// Check that 0 <= index < limit using a single comparison, assuming +// that 0 <= limit if Index is signed. Intended for use in performance +// critical contexts where 0 <= index < limit is almost always true. +template +EIGEN_ALWAYS_INLINE EIGEN_DEVICE_FUNC bool FastBoundsCheck(const Ta index, + const Tb limit) { + static_assert(std::is_integral::value && std::is_integral::value, + "FastBoundsCheck can only be used on integer types."); + typedef typename std::make_unsigned::type UIndex; + return TF_PREDICT_TRUE(static_cast(index) < + static_cast(limit)); +} + +namespace internal { +// Ensure that the compiler cannot elide a copy into a local, for +// bounds checking on source tensors that might be updated asynchronously. +// This function may only be used on primitive integral types (int32, int64, +// etc). It does not guarantee any atomicity or barriers. +template +EIGEN_ALWAYS_INLINE EIGEN_DEVICE_FUNC const T SubtleMustCopy(const T &x) { + static_assert(std::is_integral::value, + "SubtleMustCopy can only be used on integer types."); + auto *to_x = reinterpret_cast(&x); + return *to_x; +} +} // namespace internal +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_BOUNDS_CHECK_H_ diff --git a/linux/include/tensorflow/core/kernels/broadcast_to_op.h b/linux/include/tensorflow/core/kernels/broadcast_to_op.h new file mode 100644 index 000000000..a2327a727 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/broadcast_to_op.h @@ -0,0 +1,242 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_BROADCAST_TO_OP_H_ +#define TENSORFLOW_CORE_KERNELS_BROADCAST_TO_OP_H_ + +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/framework/types.h" +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" + +namespace tensorflow { + +namespace functor { + +template +struct BroadcastTo { + void operator()(const Device &d, OpKernelContext *ctx, Tensor &output_tensor, + const TensorShape &output_shape, const Tensor &input_tensor, + const TensorShape &input_shape) { +#define BROADCAST_SHAPE(broadcast, reshape, NDIMS, input_shape, output_shape) \ + for (int i = 0; i < NDIMS; i++) { \ + if (reshape[i] != broadcast[i]) { \ + OP_REQUIRES(ctx, \ + ((reshape[i] != 0) && (broadcast[i] % reshape[i] == 0)), \ + errors::InvalidArgument("invalid shape to broadcast from ", \ + input_shape.DebugString(), " to ", \ + output_shape.DebugString())); \ + broadcast[i] = broadcast[i] / reshape[i]; \ + } else { \ + broadcast[i] = 1; \ + } \ + } + + if (output_shape.num_elements() == 0) { + return; + } + if (output_shape == input_shape) { + output_tensor.flat().device(d) = input_tensor.flat(); + return; + } + + switch (output_shape.dims()) { + case 0: { + if (input_shape.dims() > 0) { + ctx->CtxFailure(errors::InvalidArgument( + "invalid shape to broadcast from ", input_shape.DebugString(), + " to ", output_shape.DebugString())); + break; + } + output_tensor.scalar().device(d) = input_tensor.scalar(); + break; + } + case 1: { + auto reshape = AsEigenDSizesWithPrefix<1>(input_shape); + auto broadcast = output_shape.AsEigenDSizes<1>(); + + BROADCAST_SHAPE(broadcast, reshape, 1, input_shape, output_shape); + + auto output = output_tensor.tensor(); + switch (input_shape.dims()) { + case 0: { + output.device(d) = output.constant(input_tensor.scalar()()); + } break; + case 1: { + auto input = input_tensor.tensor(); + output.device(d) = input.broadcast(broadcast); + } break; + default: + ctx->CtxFailure(errors::InvalidArgument( + "invalid shape to broadcast from ", input_shape.DebugString(), + " to ", output_shape.DebugString())); + break; + } + } break; + case 2: { + auto reshape = AsEigenDSizesWithPrefix<2>(input_shape); + auto broadcast = output_shape.AsEigenDSizes<2>(); + + BROADCAST_SHAPE(broadcast, reshape, 2, input_shape, output_shape); + + auto output = output_tensor.tensor(); + switch (input_shape.dims()) { + case 0: { + output.device(d) = output.constant(input_tensor.scalar()()); + } break; + case 1: { + auto input = input_tensor.tensor(); + output.device(d) = input.reshape(reshape).broadcast(broadcast); + } break; + case 2: { + auto input = input_tensor.tensor(); + output.device(d) = input.broadcast(broadcast); + } break; + default: + ctx->CtxFailure(errors::InvalidArgument( + "invalid shape to broadcast from ", input_shape.DebugString(), + " to ", output_shape.DebugString())); + break; + } + } break; + case 3: { + auto reshape = AsEigenDSizesWithPrefix<3>(input_shape); + auto broadcast = output_shape.AsEigenDSizes<3>(); + + BROADCAST_SHAPE(broadcast, reshape, 3, input_shape, output_shape); + + auto output = output_tensor.tensor(); + switch (input_shape.dims()) { + case 0: { + output.device(d) = output.constant(input_tensor.scalar()()); + } break; + case 1: { + auto input = input_tensor.tensor(); + output.device(d) = input.reshape(reshape).broadcast(broadcast); + } break; + case 2: { + auto input = input_tensor.tensor(); + output.device(d) = input.reshape(reshape).broadcast(broadcast); + } break; + case 3: { + auto input = input_tensor.tensor(); + output.device(d) = input.broadcast(broadcast); + } break; + default: + ctx->CtxFailure(errors::InvalidArgument( + "invalid shape to broadcast from ", input_shape.DebugString(), + " to ", output_shape.DebugString())); + break; + } + } break; + case 4: { + auto reshape = AsEigenDSizesWithPrefix<4>(input_shape); + auto broadcast = output_shape.AsEigenDSizes<4>(); + + BROADCAST_SHAPE(broadcast, reshape, 4, input_shape, output_shape); + auto output = output_tensor.tensor(); + switch (input_shape.dims()) { + case 0: { + output.device(d) = output.constant(input_tensor.scalar()()); + } break; + case 1: { + auto input = input_tensor.tensor(); + output.device(d) = input.reshape(reshape).broadcast(broadcast); + } break; + case 2: { + auto input = input_tensor.tensor(); + output.device(d) = input.reshape(reshape).broadcast(broadcast); + } break; + case 3: { + auto input = input_tensor.tensor(); + output.device(d) = input.reshape(reshape).broadcast(broadcast); + } break; + case 4: { + auto input = input_tensor.tensor(); + output.device(d) = input.broadcast(broadcast); + } break; + default: + ctx->CtxFailure(errors::InvalidArgument( + "invalid shape to broadcast from ", input_shape.DebugString(), + " to ", output_shape.DebugString())); + break; + } + } break; + case 5: { + auto reshape = AsEigenDSizesWithPrefix<5>(input_shape); + auto broadcast = output_shape.AsEigenDSizes<5>(); + + BROADCAST_SHAPE(broadcast, reshape, 5, input_shape, output_shape); + auto output = output_tensor.tensor(); + switch (input_shape.dims()) { + case 0: { + output.device(d) = output.constant(input_tensor.scalar()()); + } break; + case 1: { + auto input = input_tensor.tensor(); + output.device(d) = input.reshape(reshape).broadcast(broadcast); + } break; + case 2: { + auto input = input_tensor.tensor(); + output.device(d) = input.reshape(reshape).broadcast(broadcast); + } break; + case 3: { + auto input = input_tensor.tensor(); + output.device(d) = input.reshape(reshape).broadcast(broadcast); + } break; + case 4: { + auto input = input_tensor.tensor(); + output.device(d) = input.reshape(reshape).broadcast(broadcast); + } break; + case 5: { + auto input = input_tensor.tensor(); + output.device(d) = input.broadcast(broadcast); + } break; + default: + ctx->CtxFailure(errors::InvalidArgument( + "invalid shape to broadcast from ", input_shape.DebugString(), + " to ", output_shape.DebugString())); + break; + } + } break; + default: + ctx->CtxFailure(errors::InvalidArgument( + "invalid shape to broadcast from ", input_shape.DebugString(), + " to ", output_shape.DebugString())); + break; + } + } + + private: + template + Eigen::DSizes AsEigenDSizesWithPrefix( + const TensorShape &shape) const { + Eigen::DSizes dsizes; + for (int d = 0; d < NDIMS - shape.dims(); d++) { + dsizes[d] = 1; + } + for (int d = NDIMS - shape.dims(); d < NDIMS; d++) { + dsizes[d] = shape.dim_size(d - (NDIMS - shape.dims())); + } + return dsizes; + } +}; + +} // namespace functor +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_BROADCAST_TO_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/bucketize_op.h b/linux/include/tensorflow/core/kernels/bucketize_op.h new file mode 100644 index 000000000..32be475f8 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/bucketize_op.h @@ -0,0 +1,41 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_BUCKETIZE_OP_H_ +#define TENSORFLOW_CORE_KERNELS_BUCKETIZE_OP_H_ + +#include +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" + +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/core/errors.h" + +namespace tensorflow { +namespace functor { + +template +struct BucketizeFunctor { + static Status Compute(OpKernelContext* context, + const typename TTypes::ConstTensor& input, + const std::vector& boundaries_vector, + typename TTypes::Tensor& output); +}; + +} // namespace functor +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_BUCKETIZE_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/cast_op.h b/linux/include/tensorflow/core/kernels/cast_op.h new file mode 100644 index 000000000..84c44f6b5 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/cast_op.h @@ -0,0 +1,326 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_CAST_OP_H_ +#define TENSORFLOW_CORE_KERNELS_CAST_OP_H_ + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/bfloat16.h" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/platform/byte_order.h" +#include "tensorflow/core/platform/types.h" + +// Note that the GPU cast functor templates need to be instantiated unlike the +// CPU ones, and hence their specializations are different than that for CPUs. +#ifdef SPECIALIZE_FOR_GPUS +#define SPECIALIZE_CAST(DEVICE, OUT_TYPE, IN_OUT) \ + template \ + struct CastFunctor { \ + void operator()(const Device& d, \ + typename TTypes::Flat out_tensor, \ + typename TTypes::ConstFlat in_tensor, \ + bool truncate = false) { \ + if (truncate) { \ + out_tensor.device(d) = \ + in_tensor.unaryExpr(LSBZeroSetter()) \ + .template cast(); \ + } else { \ + out_tensor.device(d) = in_tensor.template cast(); \ + } \ + } \ + }; \ + template struct CastFunctor; +#else +#define SPECIALIZE_CAST(DEVICE, OUT_TYPE, IN_OUT) \ + template <> \ + struct CastFunctor { \ + void operator()(const DEVICE& d, \ + typename TTypes::Flat out_tensor, \ + typename TTypes::ConstFlat in_tensor, \ + bool truncate = false) { \ + if (truncate) { \ + out_tensor.device(d) = \ + in_tensor.unaryExpr(LSBZeroSetter()) \ + .template cast(); \ + } else { \ + out_tensor.device(d) = in_tensor.template cast(); \ + } \ + } \ + }; +#endif + +#define CAST_FUNCTORS(devname) \ + SPECIALIZE_CAST(devname, float, double) \ + SPECIALIZE_CAST(devname, float, std::complex) \ + SPECIALIZE_CAST(devname, std::complex, std::complex) \ + SPECIALIZE_CAST(devname, std::complex, double) \ + SPECIALIZE_CAST(devname, Eigen::half, double) \ + SPECIALIZE_CAST(devname, Eigen::half, float) \ + SPECIALIZE_CAST(devname, Eigen::half, std::complex) \ + SPECIALIZE_CAST(devname, Eigen::half, std::complex) \ + SPECIALIZE_CAST(devname, bfloat16, float) \ + template \ + struct CastFunctor { \ + void operator()(const devname& d, \ + typename TTypes::Flat out_tensor, \ + typename TTypes::ConstFlat in_tensor, \ + bool truncate = false) { \ + out_tensor.device(d) = in_tensor.template cast(); \ + } \ + }; + +namespace tensorflow { + +typedef std::function + CastFunctorType; + +// Common base class of Cast kernels +class CastOpBase : public OpKernel { + public: + explicit CastOpBase(OpKernelConstruction* ctx); + + void Compute(OpKernelContext* ctx) override; + + protected: + DataType src_dtype_; + DataType dst_dtype_; + DataType external_src_dtype_; + DataType external_dst_dtype_; + bool use_truncation_; + CastFunctorType work_ = nullptr; + Status Unimplemented(); + + TF_DISALLOW_COPY_AND_ASSIGN(CastOpBase); +}; + +// CPU implementation of Cast +class CpuCastOp : public CastOpBase { + public: + explicit CpuCastOp(OpKernelConstruction* ctx); + + private: + Status Prepare(); +}; + +namespace functor { + +template +constexpr int MantissaWidth() { + return std::numeric_limits::digits; +} + +template <> +constexpr int MantissaWidth() { + // Remember, there's 1 hidden bit + return 10 + 1; +} + +template <> +constexpr int MantissaWidth() { + // Remember, there's 1 hidden bit + return 7 + 1; +} + +template +void Cast(const Device& d, typename TTypes::Flat o, + typename TTypes::ConstFlat i) { + o.device(d) = i.template cast(); +} + +template +struct CastFunctor { + void operator()(const Device& d, typename TTypes::Flat o, + typename TTypes::ConstFlat i, bool truncate = false); +}; + +// Only enable LSBZeroSetterHelper for 64 and 32 bit input data types. +// Specialize for others if needed in future. +template +typename std::enable_if::type EIGEN_DEVICE_FUNC + EIGEN_STRONG_INLINE static LSBZeroSetterHelper(I& t, int n) { + // Only zero the bits for non-NaNs. + // For NaNs, let the non-truncation version handle it. + if (!std::isnan(t)) { + uint64_t* p = reinterpret_cast(&t); + *p &= (0xFFFFFFFFFFFFFFFF << n); + } +} + +template +typename std::enable_if::type EIGEN_DEVICE_FUNC + EIGEN_STRONG_INLINE static LSBZeroSetterHelper(I& t, int n) { + // Only zero the bits for non-NaNs. + // For NaNs, let the non-truncation version handle it. + if (!std::isnan(t)) { + uint32_t* p = reinterpret_cast(&t); + *p &= (0xFFFFFFFF << n); + } +} + +// Set n least significant bits to 0 +template +struct LSBZeroSetter { + EIGEN_EMPTY_STRUCT_CTOR(LSBZeroSetter) + + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE const I operator()(const I& a) const { + constexpr int bits = MantissaWidth() - MantissaWidth(); + static_assert( + bits > 0, + "The output type must have fewer mantissa bits than the input type\n"); + I t = a; + LSBZeroSetterHelper(t, bits); + return t; + } +}; + +template +struct LSBZeroSetter, std::complex> { + EIGEN_EMPTY_STRUCT_CTOR(LSBZeroSetter) + + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE const std::complex operator()( + const std::complex& a) const { + constexpr int bits = MantissaWidth() - MantissaWidth(); + static_assert( + bits > 0, + "The output type must have fewer mantissa bits than the input type\n"); + I re = std::real(a); + I img = std::imag(a); + LSBZeroSetterHelper(re, bits); + LSBZeroSetterHelper(img, bits); + std::complex toReturn(re, img); + return toReturn; + } +}; + +template +struct LSBZeroSetter, O> { + EIGEN_EMPTY_STRUCT_CTOR(LSBZeroSetter) + // Sets the 16 LSBits of the float to 0 + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE const std::complex operator()( + const std::complex& a) const { + constexpr int bits = MantissaWidth() - MantissaWidth(); + static_assert( + bits > 0, + "The output type must have fewer mantissa bits than the input type\n"); + I re = std::real(a); + I img = std::imag(a); + LSBZeroSetterHelper(re, bits); + LSBZeroSetterHelper(img, bits); + std::complex toReturn(re, img); + return toReturn; + } +}; + +} // end namespace functor +} // end namespace tensorflow + +namespace Eigen { +namespace internal { + +// Eigen can't convert to/from complex numbers, because it is limited to cases +// that can be static_casted. But numpy is able to cast to/from complex, which +// we want to replicate. So we add specializations for complex here. +template +struct scalar_cast_op, To> { + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE To + operator()(const std::complex& a) const { + // Replicate numpy behavior of returning just the real part + return static_cast(a.real()); + } +}; + +template +struct scalar_cast_op> { + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE std::complex operator()( + const From& a) const { + // Replicate numpy behavior of setting the imaginary part to 0 + return std::complex(static_cast(a), To(0)); + } +}; + +template +struct scalar_cast_op, std::complex> { + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE std::complex operator()( + const std::complex& a) const { + return std::complex(static_cast(a.real()), + static_cast(a.imag())); + } +}; + +template +struct functor_traits_complex_impl { + enum { Cost = NumTraits::AddCost, PacketAccess = false }; +}; + +template +struct functor_traits, To>> + : functor_traits_complex_impl, To> {}; +template +struct functor_traits>> + : functor_traits_complex_impl> {}; +// Needed to avoid ambiguous partial specialization +template +struct functor_traits, std::complex>> + : functor_traits_complex_impl, std::complex> {}; + +// Specialized cast op impls for bfloat16. +template <> +struct scalar_cast_op<::tensorflow::bfloat16, float> { + EIGEN_EMPTY_STRUCT_CTOR(scalar_cast_op) + typedef float result_type; + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE float operator()( + const ::tensorflow::bfloat16& a) const { + float ret; + uint16_t* p = reinterpret_cast(&ret); +#if __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__ + p[0] = a.value; + p[1] = 0; +#else + static_assert(::tensorflow::port::kLittleEndian, + "Not a little endian system!"); + p[0] = 0; + p[1] = a.value; +#endif + return ret; + } +}; + +template <> +struct functor_traits> { + enum { Cost = NumTraits::AddCost, PacketAccess = false }; +}; + +template <> +struct scalar_cast_op { + EIGEN_EMPTY_STRUCT_CTOR(scalar_cast_op) + typedef ::tensorflow::bfloat16 result_type; + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE const ::tensorflow::bfloat16 operator()( + const float a) const { + return ::tensorflow::bfloat16(a); + } +}; + +template <> +struct functor_traits> { + enum { Cost = NumTraits::AddCost, PacketAccess = false }; +}; + +} // namespace internal +} // namespace Eigen + +#endif // TENSORFLOW_CORE_KERNELS_CAST_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/cast_op_impl.h b/linux/include/tensorflow/core/kernels/cast_op_impl.h new file mode 100644 index 000000000..b899bac68 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/cast_op_impl.h @@ -0,0 +1,160 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_CAST_OP_IMPL_H_ +#define TENSORFLOW_CORE_KERNELS_CAST_OP_IMPL_H_ + +#define EIGEN_USE_THREADS + +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/kernels/cast_op.h" + +namespace tensorflow { + +namespace functor { + +CAST_FUNCTORS(Eigen::ThreadPoolDevice); + +#ifdef TENSORFLOW_USE_SYCL +CAST_FUNCTORS(Eigen::SyclDevice); +#endif // TENSORFLOW_USE_SYCL + +} // namespace functor + +#define CURRY_TYPES3_NO_HALF(FN, arg0, arg1) \ + FN(arg0, arg1, bool); \ + FN(arg0, arg1, uint8); \ + FN(arg0, arg1, uint16); \ + FN(arg0, arg1, uint32); \ + FN(arg0, arg1, uint64); \ + FN(arg0, arg1, int8); \ + FN(arg0, arg1, int16); \ + FN(arg0, arg1, int32); \ + FN(arg0, arg1, int64); \ + FN(arg0, arg1, float); \ + FN(arg0, arg1, double); \ + FN(arg0, arg1, std::complex); \ + FN(arg0, arg1, std::complex) + +#define CURRY_TYPES3_NO_BF16(FN, arg0, arg1) \ + CURRY_TYPES3_NO_HALF(FN, arg0, arg1) \ + FN(arg0, arg1, Eigen::half); + +#define CURRY_TYPES3(FN, arg0, arg1) \ + CURRY_TYPES3_NO_BF16(FN, arg0, arg1) \ + FN(arg0, arg1, bfloat16); + +#define CAST_CASE(DEVICE, IN, OUT) \ + if (DataTypeToEnum::value == dst_dtype) { \ + return [](OpKernelContext* ctx, const Tensor& inp, Tensor* out, \ + bool truncate) { \ + functor::CastFunctor func; \ + func(ctx->eigen_device(), out->flat(), inp.flat(), \ + truncate); \ + }; \ + } + +// The functions below are implemented in the cast_op_impl_*.cc files. +CastFunctorType GetCpuCastFromBool(DataType dst_dtype); + +CastFunctorType GetCpuCastFromUint8(DataType dst_dtype); + +CastFunctorType GetCpuCastFromUint16(DataType dst_dtype); + +CastFunctorType GetCpuCastFromInt8(DataType dst_dtype); + +CastFunctorType GetCpuCastFromUint32(DataType dst_dtype); + +CastFunctorType GetCpuCastFromUint64(DataType dst_dtype); + +CastFunctorType GetCpuCastFromInt8(DataType dst_dtype); + +CastFunctorType GetCpuCastFromInt16(DataType dst_dtype); + +CastFunctorType GetCpuCastFromInt32(DataType dst_dtype); + +CastFunctorType GetCpuCastFromInt64(DataType dst_dtype); + +CastFunctorType GetCpuCastFromHalf(DataType dst_dtype); + +CastFunctorType GetCpuCastFromFloat(DataType dst_dtype); + +CastFunctorType GetCpuCastFromDouble(DataType dst_dtype); + +CastFunctorType GetCpuCastFromComplex64(DataType dst_dtype); + +CastFunctorType GetCpuCastFromComplex128(DataType dst_dtype); + +CastFunctorType GetCpuCastFromBfloat(DataType dst_dtype); + +#if GOOGLE_CUDA +// Same, for GPU. +CastFunctorType GetGpuCastFromBool(DataType dst_dtype); + +CastFunctorType GetGpuCastFromUint8(DataType dst_dtype); + +CastFunctorType GetGpuCastFromUint16(DataType dst_dtype); + +CastFunctorType GetGpuCastFromInt8(DataType dst_dtype); + +CastFunctorType GetGpuCastFromUint32(DataType dst_dtype); + +CastFunctorType GetGpuCastFromUint64(DataType dst_dtype); + +CastFunctorType GetGpuCastFromInt16(DataType dst_dtype); + +CastFunctorType GetGpuCastFromInt32(DataType dst_dtype); + +CastFunctorType GetGpuCastFromInt64(DataType dst_dtype); + +CastFunctorType GetGpuCastFromHalf(DataType dst_dtype); + +CastFunctorType GetGpuCastFromFloat(DataType dst_dtype); + +CastFunctorType GetGpuCastFromDouble(DataType dst_dtype); + +CastFunctorType GetGpuCastFromComplex64(DataType dst_dtype); + +CastFunctorType GetGpuCastFromComplex128(DataType dst_dtype); + +CastFunctorType GetGpuCastFromBfloat(DataType dst_dtype); + +#endif // GOOGLE_CUDA + +#ifdef TENSORFLOW_USE_SYCL +CastFunctorType GetSyclCastFromBool(DataType dst_dtype); + +CastFunctorType GetSyclCastFromUint8(DataType dst_dtype); + +CastFunctorType GetSyclCastFromUint16(DataType dst_dtype); + +CastFunctorType GetSyclCastFromUint32(DataType dst_dtype); + +CastFunctorType GetSyclCastFromUint64(DataType dst_dtype); + +CastFunctorType GetSyclCastFromInt16(DataType dst_dtype); + +CastFunctorType GetSyclCastFromInt32(DataType dst_dtype); + +CastFunctorType GetSyclCastFromInt64(DataType dst_dtype); + +CastFunctorType GetSyclCastFromFloat(DataType dst_dtype); + +CastFunctorType GetSyclCastFromDouble(DataType dst_dtype); +#endif // TENSORFLOW_USE_SYCL + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_CAST_OP_IMPL_H_ diff --git a/linux/include/tensorflow/core/kernels/colorspace_op.h b/linux/include/tensorflow/core/kernels/colorspace_op.h new file mode 100644 index 000000000..4de14bc33 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/colorspace_op.h @@ -0,0 +1,94 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_COLORSPACE_OP_H_ +#define TENSORFLOW_CORE_KERNELS_COLORSPACE_OP_H_ + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/tensor_types.h" + +namespace tensorflow { + +namespace functor { + +template +struct RGBToHSV { + void operator()(const Device &d, + typename TTypes::ConstTensor input_data, + typename TTypes::Tensor range, + typename TTypes::Tensor output_data) { + auto H = output_data.template chip<1>(0); + auto S = output_data.template chip<1>(1); + auto V = output_data.template chip<1>(2); + + auto R = input_data.template chip<1>(0); + auto G = input_data.template chip<1>(1); + auto B = input_data.template chip<1>(2); + +#if !defined(EIGEN_HAS_INDEX_LIST) + Eigen::array channel_axis{{1}}; +#else + Eigen::IndexList > channel_axis; +#endif + + V.device(d) = input_data.maximum(channel_axis); + + range.device(d) = V - input_data.minimum(channel_axis); + + S.device(d) = (V > T(0)).select(range / V, V.constant(T(0))); + + auto norm = range.inverse() * (T(1) / T(6)); + // TODO(wicke): all these assignments are only necessary because a combined + // expression is larger than kernel parameter space. A custom kernel is + // probably in order. + H.device(d) = (R == V).select( + norm * (G - B), (G == V).select(norm * (B - R) + T(2) / T(6), + norm * (R - G) + T(4) / T(6))); + H.device(d) = (range > T(0)).select(H, H.constant(T(0))); + H.device(d) = (H < T(0)).select(H + T(1), H); + } +}; + +template +struct HSVToRGB { + void operator()(const Device &d, + typename TTypes::ConstTensor input_data, + typename TTypes::Tensor output_data) { + auto H = input_data.template chip<1>(0); + auto S = input_data.template chip<1>(1); + auto V = input_data.template chip<1>(2); + + // TODO(wicke): compute only the fractional part of H for robustness + auto dh = H * T(6); + auto dr = ((dh - T(3)).abs() - T(1)).cwiseMax(T(0)).cwiseMin(T(1)); + auto dg = (-(dh - T(2)).abs() + T(2)).cwiseMax(T(0)).cwiseMin(T(1)); + auto db = (-(dh - T(4)).abs() + T(2)).cwiseMax(T(0)).cwiseMin(T(1)); + auto one_s = -S + T(1); + + auto R = output_data.template chip<1>(0); + auto G = output_data.template chip<1>(1); + auto B = output_data.template chip<1>(2); + + R.device(d) = (one_s + S * dr) * V; + G.device(d) = (one_s + S * dg) * V; + B.device(d) = (one_s + S * db) * V; + } +}; + +} // namespace functor +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_COLORSPACE_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/compare_and_bitpack_op.h b/linux/include/tensorflow/core/kernels/compare_and_bitpack_op.h new file mode 100644 index 000000000..af8566c7c --- /dev/null +++ b/linux/include/tensorflow/core/kernels/compare_and_bitpack_op.h @@ -0,0 +1,42 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_COMPARE_AND_BITPACK_OP_H_ +#define TENSORFLOW_CORE_KERNELS_COMPARE_AND_BITPACK_OP_H_ + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +namespace functor { + +typedef Eigen::ThreadPoolDevice CPUDevice; +typedef Eigen::GpuDevice GPUDevice; + +template +struct CompareAndBitpack { + void operator()(OpKernelContext* c, typename TTypes::ConstMatrix input, + typename TTypes::ConstScalar threshold, + TTypes::Matrix output); +}; + +} // namespace functor + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_COMPARE_AND_BITPACK_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/concat_lib.h b/linux/include/tensorflow/core/kernels/concat_lib.h new file mode 100644 index 000000000..8b53ecf12 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/concat_lib.h @@ -0,0 +1,69 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_CONCAT_LIB_H_ +#define TENSORFLOW_CORE_KERNELS_CONCAT_LIB_H_ + +#include + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/device_base.h" + +namespace tensorflow { + +// Functors to concatenate tensors. These always take a rank-2 tensor (i.e a +// matrix) and concatenate it along the axis 1 ("putting them next to each +// other" as opposed to "putting them on top of one another"). +// +// Any concatenation of n-dimensional tensors across any axis can be reduced to +// a concatenation of two-dimensional tensors across the axis 1 by first +// partitioning the axes of the original tensors into those less than the axis +// to be concatenated across and the rest. Then reshape the tensors into a +// two-dimensional tensor by collapsing these two sets of axes and concatenate +// the resulting matrices across the axis 1, finally reshaping the result to +// have the proper shape. +// +// So, for example, when stacking N tensors, reshape each to have shape +// {1, Numelements} and reshape the result matrix to have shape +// {1, N * NumElements} before passing it to this functor. + +// Assumes all inputs are nonempty +template +void ConcatCPU( + DeviceBase* d, + const std::vector::ConstMatrix>>& + inputs, + typename TTypes::Matrix* output); +#if GOOGLE_CUDA +template +void ConcatGPU( + OpKernelContext* c, + const std::vector::ConstMatrix>>& + inputs_flat, + Tensor* output, typename TTypes::Tensor* output_flat); + +#endif // GOOGLE_CUDA + +#ifdef TENSORFLOW_USE_SYCL +template +void ConcatSYCL( + const Eigen::SyclDevice& d, + const std::vector::ConstMatrix>>& + inputs, + typename TTypes::Matrix* output); +#endif // TENSORFLOW_USE_SYCL +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_CONCAT_LIB_H_ diff --git a/linux/include/tensorflow/core/kernels/concat_lib_cpu.h b/linux/include/tensorflow/core/kernels/concat_lib_cpu.h new file mode 100644 index 000000000..29f3a427f --- /dev/null +++ b/linux/include/tensorflow/core/kernels/concat_lib_cpu.h @@ -0,0 +1,169 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_CONCAT_LIB_CPU_H_ +#define TENSORFLOW_CORE_KERNELS_CONCAT_LIB_CPU_H_ + +#define EIGEN_USE_THREADS + +#include +#include "tensorflow/core/framework/register_types.h" +#include "tensorflow/core/kernels/concat_lib.h" +#include "tensorflow/core/util/work_sharder.h" + +namespace tensorflow { + +// ElementCopier must be a struct with a single Copy function, which is passed +// the output pointer, input pointer, input index, and number of elements to +// copy from input to output. +template +void ConcatCPUImpl( + DeviceBase* d, + const std::vector::ConstMatrix>>& + inputs, + int64 cost_per_unit, ElementCopier copier, + typename TTypes::Matrix* output) { + size_t num_inputs = inputs.size(); + + std::vector sizes; + sizes.reserve(num_inputs); + int64 row_size = 0; + for (const auto& input : inputs) { + sizes.push_back(input->dimension(1)); + row_size += sizes.back(); + } + + auto worker_threads = d->tensorflow_cpu_worker_threads(); + int num_threads = std::min(4, worker_threads->num_threads); + // strings define a different amount of work (generally much more) compared + // with standard POD, so we parallelize differently. + if (!std::is_same::value) { + num_threads = + static_cast(std::min(num_threads, output->size() / 4096)); + } + // Single threaded mode. + // TODO(dga): Deduplicate this code w.r.t. sharded code below. + if (num_threads == 0) { + T* out = &(*output)(0, 0); + std::vector inp; + inp.reserve(num_inputs); + for (const auto& input : inputs) { + inp.push_back(&(*input)(0, 0)); + } + const int64 dim0 = output->dimension(0); + for (int64 i = 0; i < dim0; ++i) { + for (int64 j = 0; j < num_inputs; ++j) { + auto size = sizes[j]; + copier.Copy(out, inp[j], j, size); + out += size; + inp[j] += size; + } + } + return; + } + + // Sharded mode. + auto work = [&row_size, &sizes, &inputs, &output, &copier, &num_inputs]( + int64 start, int64 end) { + int64 skipped_rows = start / row_size; + T* out = output->data() + skipped_rows * row_size; + T* out_start = output->data() + start; + T* out_end = output->data() + end; + + // Handle partial row at start + if (out < out_start) { + for (size_t j = 0; j < num_inputs; ++j) { + ptrdiff_t size = sizes[j]; + ptrdiff_t offset = out_start - out; + if (size <= offset) { + out += size; + continue; + } + const T* inp = &(*inputs[j])(skipped_rows, 0); + if (offset > 0) { + out += offset; + inp += offset; + size -= offset; + } + size = std::min(size, out_end - out); + if (size <= 0) break; + copier.Copy(out, inp, j, size); + out += size; + } + ++skipped_rows; + } + if (out == out_end) return; + CHECK(out >= out_start); + CHECK(out < out_end); + + // Copy remaining data. + std::vector inp; + inp.reserve(num_inputs); + for (const auto& input : inputs) { + inp.push_back(&(*input)(skipped_rows, 0)); + } + const int64 dim0 = output->dimension(0); + for (int64 i = skipped_rows; i < dim0; ++i) { + for (int64 j = 0; j < num_inputs; ++j) { + ptrdiff_t size = std::min(sizes[j], out_end - out); + copier.Copy(out, inp[j], j, size); + out += size; + inp[j] += size; + if (out == out_end) return; + } + } + }; + Shard(worker_threads->num_threads, worker_threads->workers, output->size(), + cost_per_unit, work); +} + +#ifdef TENSORFLOW_USE_SYCL +template +void ConcatSYCLImpl( + const Eigen::SyclDevice& d, + const std::vector::ConstMatrix>>& + inputs, + int64 cost_per_unit, ElementCopier copier, + typename TTypes::Matrix* output) { + size_t num_inputs = inputs.size(); + + std::vector sizes; + sizes.reserve(num_inputs); + int64 row_size = 0; + for (const auto& input : inputs) { + sizes.push_back(input->dimension(1)); + row_size += sizes.back(); + } + + T* out = &(*output)(0, 0); + std::vector inp; + inp.reserve(num_inputs); + for (const auto& input : inputs) { + inp.push_back(&(*input)(0, 0)); + } + const int64 dim0 = output->dimension(0); + for (int64 i = 0; i < dim0; ++i) { + for (int64 j = 0; j < num_inputs; ++j) { + auto size = sizes[j]; + d.memcpy(out, inp[j], size * sizeof(T)); + out += size; + inp[j] += size; + } + } +} +#endif // TENSORFLOW_USE_SYCL +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_CONCAT_LIB_CPU_H_ diff --git a/linux/include/tensorflow/core/kernels/conditional_accumulator.h b/linux/include/tensorflow/core/kernels/conditional_accumulator.h new file mode 100644 index 000000000..390db8fe5 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/conditional_accumulator.h @@ -0,0 +1,138 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_CONDITIONAL_ACCUMULATOR_H_ +#define TENSORFLOW_CORE_KERNELS_CONDITIONAL_ACCUMULATOR_H_ + +#include "tensorflow/core/kernels/fill_functor.h" +#include "tensorflow/core/kernels/typed_conditional_accumulator_base.h" + +namespace tensorflow { + +/** + * An aggregation object for adding dense gradients. + * + * The two main methods of this class are TryApplyGrad and TryTakeGrad. + * + * TryApplyGrad tries add a gradient to the accumulator. The attempt is + * successful if local_step >= global_step, i.e., if the gradient is not stale, + * having been computed using up-to-date information. Otherwise, the gradient is + * silently dropped. + * + * TryTakeGrad logs an attempt to read the average gradient. The attempt is + * blocked until the number of gradients accumulated (via TryApplyGrad) is equal + * or exceeds the number requested by TryTakeGrad. + * Once this condition is satisfied, the following actions are taken: + * (1) the value of the average gradient is returned + * (2) the count of accumulated gradients is reset to 0 + * (3) the internal global_step value (current_global_step_) is incremented by 1 + * + * ConditionalAccumulator is the datatype-dependent templated sub-class of + * ConditionalAccumulatorBase. It implements the virtual arithmetic methods that + * are used by for aggregating, averaging, allocating, returning dense Tensors. + */ +template +class ConditionalAccumulator + : public TypedConditionalAccumulatorBase { + public: + // Args: + // dtype: The datatype of the gradients to be accumulated. + // shape: The shape of the accumulated gradients. + // name: A name to use for the ConditionalAccumulator. + // reduction_type: The reduction type, i.e., MEAN or SUM + ConditionalAccumulator(const DataType& dtype, const PartialTensorShape& shape, + const string& name, const string& reduction_type) + : TypedConditionalAccumulatorBase(dtype, shape, name, + reduction_type) {} + ~ConditionalAccumulator() override{}; + + protected: + // accum_grad is the tensor that holds the aggregate gradient. + // It is initialized the first time ApplyGrad is called. + Tensor* accum_grad_ = nullptr; + PersistentTensor accum_grad_persistent_; + + functor::SetZeroFunctor set_zero_functor_; + + Status ValidateShape(const Tensor* tensor) + EXCLUSIVE_LOCKS_REQUIRED(this->mu_) { + // Must be compatible with accumulated gradient if available + if (counter_ > 0) { + if (!accum_grad_->shape().IsSameSize(tensor->shape())) { + return errors::InvalidArgument("Shape mismatch: expected ", + accum_grad_->shape().DebugString(), + ", got ", tensor->shape().DebugString()); + } + } + // Must also be compatible with given shape + if (!shape_.IsCompatibleWith(tensor->shape())) { + return errors::InvalidArgument("Shape mismatch: expected ", + shape_.DebugString(), ", got ", + tensor->shape().DebugString()); + } + return Status::OK(); + } + + void AllocateAndAssignToAccumGradFunction(OpKernelContext* ctx, + const Tensor* grad) override { + // TODO(b/32704451): Don't just ignore the ::tensorflow::Status object! + ctx->allocate_persistent(dtype_, grad->shape(), &accum_grad_persistent_, + &accum_grad_) + .IgnoreError(); + accum_grad_->flat().device(ctx->template eigen_device()) = + grad->flat(); + } + + void AddToAccumGradFunction(OpKernelContext* ctx, + const Tensor* grad) override { + accum_grad_->flat().device(ctx->template eigen_device()) += + grad->flat(); + } + + void DivideAccumGradByCounter(OpKernelContext* ctx) override + EXCLUSIVE_LOCKS_REQUIRED(this->mu_) { + Tensor c(DataTypeToEnum::value, {}); + c.scalar()() = TypeConverter::ConvertUToT(this->counter_); + this->accum_grad_->template flat().device( + ctx->template eigen_device()) = + this->accum_grad_->template flat() / c.scalar()(); + } + + bool SetOutput(OpKernelContext* ctx) override { + ctx->set_output(0, *accum_grad_); + return true; + } + + bool GetAndValidateTensorInputForApplyGrad(OpKernelContext* ctx, + const Tensor** tensor) override + EXCLUSIVE_LOCKS_REQUIRED(this->mu_) { + // Get input gradient tensor + const Tensor* grad_tensor; + OP_REQUIRES_OK_BOOLEAN(ctx, ctx->input("gradient", &grad_tensor)); + *tensor = grad_tensor; + OP_REQUIRES_OK_BOOLEAN(ctx, this->ValidateShape(*tensor)); + return true; + } + + void CleanUpGradTensor(const Tensor* tensor) override { + // do nothing + } + + TF_DISALLOW_COPY_AND_ASSIGN(ConditionalAccumulator); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_CONDITIONAL_ACCUMULATOR_H_ diff --git a/linux/include/tensorflow/core/kernels/conditional_accumulator_base.h b/linux/include/tensorflow/core/kernels/conditional_accumulator_base.h new file mode 100644 index 000000000..4a5ec6f0f --- /dev/null +++ b/linux/include/tensorflow/core/kernels/conditional_accumulator_base.h @@ -0,0 +1,203 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_CONDITIONAL_ACCUMULATOR_BASE_H_ +#define TENSORFLOW_CORE_KERNELS_CONDITIONAL_ACCUMULATOR_BASE_H_ + +#include + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/numeric_op.h" + +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/resource_mgr.h" + +namespace tensorflow { + +/** + * ConditionalAccumulator/ConditionalAccumulatorBase implements an aggregation + * object for adding gradients. + * The two main methods of this class are TryApplyGrad and TryTakeGrad. + * + * TryApplyGrad tries add a gradient to the accumulator. The attempt is + * successful if local_step >= global_step, i.e., if the gradient is not stale, + * having been computed using up-to-date information. Otherwise, the gradient is + * silently dropped. + * + * TryTakeGrad logs an attempt to read the average gradient. The attempt is + * blocked until the number of gradients accumulated (via TryApplyGrad) is equal + * or exceeds the number requested by TryTakeGrad. + * Once this condition is satisfied, the following actions are taken: + * (1) the value of the average gradient is returned + * (2) the count of accumulated gradients is reset to 0 + * (3) the internal global_step value (current_global_step_) is incremented by 1 + */ +class ConditionalAccumulatorBase : public ResourceBase { + public: + // Args: + // dtype: The datatype of the gradients to be accumulated. + // shape: The shape of the accumulated gradients. + // name: A name to use for the ConditionalAccumulator. + ConditionalAccumulatorBase(const DataType& dtype, + const PartialTensorShape& shape, + const string& name, const string& reduction_type); + + typedef AsyncOpKernel::DoneCallback DoneCallback; + + virtual void TryApplyGrad(int64 local_step, OpKernelContext* ctx) = 0; + void TryTakeGrad(int num_required, OpKernelContext* ctx, + DoneCallback callback); + + // Accessor methods + uint32 num_accumulated() { + mutex_lock lock(mu_); + return counter_; + } + + const DataType& dtype() const { return dtype_; } + + string DebugString() override { return "A conditional accumulator"; } + + // SetGlobalStep is a modifier method for current_global_step. + // It returns an InvalidArgument error if the new_global_step is less than + // current_global_step. + Status SetGlobalStep(int64 new_global_step); + + Status MatchesNodeDef(const NodeDef& node_def); + + protected: + // Virtual methods to be implemented by sub-classes for different datatypes. + // Implements arithmetic operations specific to datatype. + virtual void DivideAccumGradByCounter(OpKernelContext* ctx) + EXCLUSIVE_LOCKS_REQUIRED(mu_) = 0; + virtual bool SetOutput(OpKernelContext* ctx) = 0; + + enum RunResult { kNoProgress, kComplete }; + + // Helper struct holding information about a TakeGrad attempt + struct Attempt; + typedef std::function RunCallback; + struct Attempt { + int elements_requested; + DoneCallback done_callback; // must be run outside mu_ + OpKernelContext* context; + CancellationManager* cancellation_manager; // not owned + CancellationToken cancellation_token; + RunCallback run_callback; // must be run while holding mu_ + bool is_cancelled; + + Attempt(int elements_requested, DoneCallback done_callback, + OpKernelContext* context, CancellationManager* cancellation_manager, + CancellationToken cancellation_token, RunCallback run_callback) + : elements_requested(elements_requested), + done_callback(std::move(done_callback)), + context(context), + cancellation_manager(cancellation_manager), + cancellation_token(cancellation_token), + run_callback(std::move(run_callback)), + is_cancelled(false) {} + }; + + // Helper struct for deregistration of a cancellation token and executing a + // DoneCallback after a TakeGrad attempt is complete. + struct CleanUp { + CleanUp(DoneCallback&& f, CancellationToken ct, CancellationManager* cm) + : finished(f), to_deregister(ct), cm(cm) {} + DoneCallback finished; + CancellationToken to_deregister; + CancellationManager* cm; + }; + + // Fields + + const DataType dtype_; + const PartialTensorShape shape_; + const string name_; + const string reduction_type_; + mutex mu_; + int counter_ GUARDED_BY(mu_); + int64 current_global_step_ GUARDED_BY(mu_); + + std::deque takegrad_attempts_ GUARDED_BY(mu_); + + // Methods + + // Helper function for creating cancellation callback + void Cancel(CancellationManager* cancellation_manager, + CancellationToken token); + + // Helper functions to process TakeGrad attempts. + // FlushUnlocked is called at the end of each TryApplyGrad and TryTakeGrad + // calls to try to clear the TakeGrad attempts. This in turn calls + // TryAttemptLocked, which then executes the RunCallback of the logged + // attempts. + // Both functions are modeled after core/kernels/queue_base. + // Note: ApplyGrad attempts never block -- unlike in a queue with limited + // capacity, we can always add the newest gradient to our accumulator + // (if it is not stale) or drop it silently (if it is stale). + void FlushUnlocked(); + bool TryAttemptLocked(std::vector* clean_up) + EXCLUSIVE_LOCKS_REQUIRED(mu_); + + // Helper methods + // void DeepCopy(Tensor* dst); + bool TakeGradLockedHelper(OpKernelContext* ctx, DoneCallback callback) + EXCLUSIVE_LOCKS_REQUIRED(mu_); +}; + +/* + * Modifications to convenience macros defined in core/framework/op_kernel.h. + * The below macros return a boolean if the test fails, so that the calling + * function can get an indication that a failure has occurred. + */ +#define OP_REQUIRES_BOOLEAN(CTX, EXP, STATUS) \ + do { \ + if (!TF_PREDICT_TRUE(EXP)) { \ + (CTX)->CtxFailure(__FILE__, __LINE__, (STATUS)); \ + return false; \ + } \ + } while (0) + +#define OP_REQUIRES_OK_BOOLEAN(CTX, STATUS) \ + do { \ + ::tensorflow::Status _s(STATUS); \ + if (!TF_PREDICT_TRUE(_s.ok())) { \ + (CTX)->CtxFailureWithWarning(__FILE__, __LINE__, _s); \ + return false; \ + } \ + } while (0) + +/* + * Convenience classes for helping to convert between numeric types. + * The specialization for Eigen::half here simplifies specialization of + * ConditionalAccumulator classes later. + */ +template +class TypeConverter { + public: + static T ConvertUToT(U c) { return c; /* implicit conversion */ } +}; + +template +class TypeConverter { + public: + static Eigen::half ConvertUToT(U c) { + return Eigen::half_impl::float_to_half_rtne(c); + } +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_CONDITIONAL_ACCUMULATOR_BASE_H_ diff --git a/linux/include/tensorflow/core/kernels/conditional_accumulator_base_op.h b/linux/include/tensorflow/core/kernels/conditional_accumulator_base_op.h new file mode 100644 index 000000000..ca24d690f --- /dev/null +++ b/linux/include/tensorflow/core/kernels/conditional_accumulator_base_op.h @@ -0,0 +1,240 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_CONDITIONAL_ACCUMULATOR_BASE_OP_H_ +#define TENSORFLOW_CORE_KERNELS_CONDITIONAL_ACCUMULATOR_BASE_OP_H_ + +#define EIGEN_USE_THREADS + +#include "tensorflow/core/kernels/conditional_accumulator_base.h" + +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/register_types.h" +#include "tensorflow/core/framework/resource_mgr.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/core/errors.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/mutex.h" +#include "tensorflow/core/platform/thread_annotations.h" +#include "tensorflow/core/platform/types.h" + +typedef Eigen::ThreadPoolDevice CPUDevice; + +typedef std::function DoneCallback; + +namespace tensorflow { + +/** + * Defines a ConditionalAccumulatorBaseOp, which constructs a + * ConditionalAccumulatorBase (via sub-class's Creator) and returns its handle. + */ +class ConditionalAccumulatorBaseOp : public OpKernel { + public: + explicit ConditionalAccumulatorBaseOp(OpKernelConstruction* context) + : OpKernel(context), accumulator_handle_set_(false) { + OP_REQUIRES_OK(context, + context->allocate_persistent(DT_STRING, TensorShape({2}), + &accumulator_handle_, nullptr)); + OP_REQUIRES_OK(context, context->GetAttr("shape", &shape_)); + OP_REQUIRES_OK(context, context->GetAttr("dtype", &dtype_)); + OP_REQUIRES_OK(context, + context->GetAttr("reduction_type", &reduction_type_)); + } + + void Compute(OpKernelContext* ctx) override { + mutex_lock l(mu_); + if (!accumulator_handle_set_) { + OP_REQUIRES_OK(ctx, SetAccumulatorHandle(ctx)); + } + ctx->set_output_ref(0, &mu_, accumulator_handle_.AccessTensor(ctx)); + } + + protected: + ~ConditionalAccumulatorBaseOp() override { + // If the accumulator object was not shared, delete it. + if (accumulator_handle_set_ && cinfo_.resource_is_private_to_kernel()) { + TF_CHECK_OK((cinfo_.resource_manager() + ->template Delete( + cinfo_.container(), cinfo_.name()))); + } + } + + protected: + typedef std::function Creator; + + // Subclasses must override this + virtual Creator GetCreator() const = 0; + + // Variables required to construct ConditionalAccumulator + DataType dtype_; + PartialTensorShape shape_; + ContainerInfo cinfo_; + string reduction_type_; + + private: + Status SetAccumulatorHandle(OpKernelContext* ctx) + EXCLUSIVE_LOCKS_REQUIRED(mu_) { + TF_RETURN_IF_ERROR(cinfo_.Init(ctx->resource_manager(), def())); + + // Check input signature + DataTypeVector expected_inputs = {}; + TF_RETURN_IF_ERROR(ctx->MatchSignature(expected_inputs, {DT_STRING_REF})); + + Creator creator = GetCreator(); + ConditionalAccumulatorBase* accumulator; + TF_RETURN_IF_ERROR( + (cinfo_.resource_manager() + ->template LookupOrCreate( + cinfo_.container(), cinfo_.name(), &accumulator, creator))); + core::ScopedUnref unref_me(accumulator); + + // Verify that the shared accumulator is compatible + // with the requested arguments. + TF_RETURN_IF_ERROR(accumulator->MatchesNodeDef(def())); + auto h = accumulator_handle_.AccessTensor(ctx)->template flat(); + h(0) = cinfo_.container(); + h(1) = cinfo_.name(); + accumulator_handle_set_ = true; + return Status::OK(); + } + + mutex mu_; + PersistentTensor accumulator_handle_ GUARDED_BY(mu_); + bool accumulator_handle_set_ GUARDED_BY(mu_); +}; + +/** + * General OpKernel for ConditionalAccumulatorBase-related ops. + */ +class ConditionalAccumulatorBaseAsyncOpKernel : public AsyncOpKernel { + public: + explicit ConditionalAccumulatorBaseAsyncOpKernel( + OpKernelConstruction* context) + : AsyncOpKernel(context) {} + + void ComputeAsync(OpKernelContext* ctx, DoneCallback callback) final { + ConditionalAccumulatorBase* accumulator; + OP_REQUIRES_OK_ASYNC( + ctx, GetResourceFromContext(ctx, "handle", &accumulator), callback); + ComputeAsync(ctx, accumulator, [callback, accumulator]() { + accumulator->Unref(); + callback(); + }); + } + + protected: + virtual void ComputeAsync(OpKernelContext* ctx, + ConditionalAccumulatorBase* accumulator, + DoneCallback callback) = 0; +}; + +/** + * General OpKernel for ConditionalAccumulatorBase-related ops. + */ +class ConditionalAccumulatorBaseSyncOpKernel : public OpKernel { + public: + explicit ConditionalAccumulatorBaseSyncOpKernel(OpKernelConstruction* context) + : OpKernel(context) {} + + void Compute(OpKernelContext* ctx) final { + ConditionalAccumulatorBase* accumulator; + OP_REQUIRES_OK(ctx, GetResourceFromContext(ctx, "handle", &accumulator)); + Compute(ctx, accumulator); + accumulator->Unref(); + } + + protected: + virtual void Compute(OpKernelContext* ctx, + ConditionalAccumulatorBase* accumulator) = 0; +}; + +/** + * Defines a AccumulateGradientOp, the execution of which adds a gradient to the + * given ConditionalAccumulator. + */ +class ConditionalAccumulatorBaseApplyGradientOp + : public ConditionalAccumulatorBaseSyncOpKernel { + public: + explicit ConditionalAccumulatorBaseApplyGradientOp( + OpKernelConstruction* context) + : ConditionalAccumulatorBaseSyncOpKernel(context) {} + + protected: + virtual void CheckSignature(OpKernelContext* ctx, + ConditionalAccumulatorBase* accumulator) = 0; + + void Compute(OpKernelContext* ctx, + ConditionalAccumulatorBase* accumulator) override { + // Check input signature + CheckSignature(ctx, accumulator); + + // Get input local_step + const Tensor* local_step_tensor; + OP_REQUIRES_OK(ctx, ctx->input("local_step", &local_step_tensor)); + if (!TensorShapeUtils::IsScalar(local_step_tensor->shape())) { + ctx->CtxFailureWithWarning(errors::InvalidArgument( + "Argument local_step must be scalar, but had bad shape ", + local_step_tensor->shape().DebugString())); + } + + // Actually try to apply gradient now + accumulator->TryApplyGrad(local_step_tensor->scalar()(), ctx); + } +}; + +/** + * Defines a TakeAccumulatedGradientOp, the execution of which adds a gradient + * to the given ConditionalAccumulator. + */ +class ConditionalAccumulatorBaseTakeGradientOp + : public ConditionalAccumulatorBaseAsyncOpKernel { + public: + explicit ConditionalAccumulatorBaseTakeGradientOp( + OpKernelConstruction* context) + : ConditionalAccumulatorBaseAsyncOpKernel(context) {} + + protected: + virtual void CheckSignature(OpKernelContext* ctx, + ConditionalAccumulatorBase* accumulator, + DoneCallback callback) = 0; + + void ComputeAsync(OpKernelContext* ctx, + ConditionalAccumulatorBase* accumulator, + DoneCallback callback) override { + // Check signature + CheckSignature(ctx, accumulator, callback); + + // Get input num_required + const Tensor* num_required_tensor; + OP_REQUIRES_OK_ASYNC(ctx, ctx->input("num_required", &num_required_tensor), + callback); + if (!TensorShapeUtils::IsScalar(num_required_tensor->shape())) { + ctx->CtxFailureWithWarning(errors::InvalidArgument( + "Argument num_required must be scalar, but had bad shape ", + num_required_tensor->shape().DebugString())); + callback(); + } + + // Actually try to take gradient now + accumulator->TryTakeGrad(num_required_tensor->scalar()(), ctx, + callback); + } +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_CONDITIONAL_ACCUMULATOR_BASE_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/constant_op.h b/linux/include/tensorflow/core/kernels/constant_op.h new file mode 100644 index 000000000..77ba44186 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/constant_op.h @@ -0,0 +1,50 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_CONSTANT_OP_H_ +#define TENSORFLOW_CORE_KERNELS_CONSTANT_OP_H_ + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/platform/macros.h" + +namespace tensorflow { + +// ConstantOp returns a tensor specified by ConstantOpDef. +class ConstantOp : public OpKernel { + public: + explicit ConstantOp(OpKernelConstruction* ctx); + void Compute(OpKernelContext* ctx) override; + bool IsExpensive() override { return false; } + ~ConstantOp() override; + + private: + Tensor tensor_; + TF_DISALLOW_COPY_AND_ASSIGN(ConstantOp); +}; + +class PlaceholderOp : public OpKernel { + public: + explicit PlaceholderOp(OpKernelConstruction* ctx); + void Compute(OpKernelContext* ctx) override; + + private: + PartialTensorShape expected_shape_; +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_CONSTANT_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/control_flow_ops.h b/linux/include/tensorflow/core/kernels/control_flow_ops.h new file mode 100644 index 000000000..c607fcf29 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/control_flow_ops.h @@ -0,0 +1,118 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_CONTROL_FLOW_OPS_H_ +#define TENSORFLOW_CORE_KERNELS_CONTROL_FLOW_OPS_H_ + +#include "tensorflow/core/framework/op_kernel.h" + +namespace tensorflow { + +// A ControlTriggerOp is similar to a NoOp. However, it always treats the input +// control edges as Live edges. Its primary use so far is in the scheduling of +// recvs, where we add ControlTrigger nodes and use them to trigger recvs. We +// allow ControlTrigger nodes to be enabled by dead nodes. +class ControlTriggerOp : public OpKernel { + public: + explicit ControlTriggerOp(OpKernelConstruction* context) + : OpKernel(context) {} + void Compute(OpKernelContext* context) override {} + bool IsExpensive() override { return false; } +}; + +// A switch op has two inputs and two outputs. It forwards the value of +// Input:0 to the output specified by input:1. Input:1 is a boolean tensor. +// Input:0 is forwarded to output:0 if input:1 is false, otherwise to +// output:1. +class SwitchOp : public OpKernel { + public: + explicit SwitchOp(OpKernelConstruction* context) : OpKernel(context) {} + void Compute(OpKernelContext* context) override; + bool IsExpensive() override { return false; } + ~SwitchOp() override {} + + TF_DISALLOW_COPY_AND_ASSIGN(SwitchOp); +}; + +// A merge op has n inputs and two outputs. It forwards the value of the +// first input that becomes available to its first output, and the +// index of the first input to its second output. +class MergeOp : public OpKernel { + public: + explicit MergeOp(OpKernelConstruction* context); + void Compute(OpKernelContext* context) override; + bool IsExpensive() override { return false; } + ~MergeOp() override {} + + TF_DISALLOW_COPY_AND_ASSIGN(MergeOp); +}; + +// An enter op has one input and one output. It creates or finds +// the child frame that is uniquely identified by the frame_name, +// and makes its input available to the child frame. +class EnterOp : public OpKernel { + public: + explicit EnterOp(OpKernelConstruction* context) : OpKernel(context) {} + void Compute(OpKernelContext* context) override; + bool IsExpensive() override { return false; } + ~EnterOp() override {} + + TF_DISALLOW_COPY_AND_ASSIGN(EnterOp); +}; + +// An exit op has one input and one output. It exits the current +// frame to its parent frame, and makes its input available to the +// parent frame. +class ExitOp : public OpKernel { + public: + explicit ExitOp(OpKernelConstruction* context) : OpKernel(context) {} + void Compute(OpKernelContext* context) override; + bool IsExpensive() override { return false; } + ~ExitOp() override {} + + TF_DISALLOW_COPY_AND_ASSIGN(ExitOp); +}; + +// A next_iteration op has one input and one output. It makes its input +// available to the next iteration. +class NextIterationOp : public OpKernel { + public: + explicit NextIterationOp(OpKernelConstruction* context) : OpKernel(context) {} + void Compute(OpKernelContext* context) override; + bool IsExpensive() override { return false; } + ~NextIterationOp() override {} + + TF_DISALLOW_COPY_AND_ASSIGN(NextIterationOp); +}; + +// A LoopCond op has one input and one output. The input is a boolean +// scalar representing the taken branches of the "pivot" Switch that +// determines loop termination. As a contract, any high-level front-end +// should always use port '0' of the "pivot" switches for loop exit. +class LoopCondOp : public OpKernel { + public: + explicit LoopCondOp(OpKernelConstruction* context); + ~LoopCondOp() override; + + void Compute(OpKernelContext* context) override; + + bool IsExpensive() override; + + TF_DISALLOW_COPY_AND_ASSIGN(LoopCondOp); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_CONTROL_FLOW_OPS_H_ diff --git a/linux/include/tensorflow/core/kernels/conv_2d.h b/linux/include/tensorflow/core/kernels/conv_2d.h new file mode 100644 index 000000000..639c3062c --- /dev/null +++ b/linux/include/tensorflow/core/kernels/conv_2d.h @@ -0,0 +1,316 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_CONV_2D_H_ +#define TENSORFLOW_CORE_KERNELS_CONV_2D_H_ + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/kernels/eigen_backward_spatial_convolutions.h" +#include "tensorflow/core/kernels/eigen_spatial_convolutions.h" +#include "tensorflow/core/util/tensor_format.h" + +namespace tensorflow { +namespace functor { + +// TODO(yangke): revisit these operations and in particular, see if we can +// combine all of them into just one operation without causing nvcc to +// timeout. +template +struct ShuffleAndReverse { + void operator()(const Device& d, + typename TTypes::ConstTensor input, + const Eigen::DSizes& order, + const Eigen::array& reverse_dims, + typename TTypes::Tensor output) { + output.device(d) = input.shuffle(order).reverse(reverse_dims); + } +}; + +template +struct InflatePadAndShuffle { + void operator()( + const Device& d, typename TTypes::ConstTensor input, + const Eigen::DSizes& strides, + const Eigen::array, Dims>& pad_dims, + const Eigen::DSizes& order, + typename TTypes::Tensor output) { + output.device(d) = input.inflate(strides).pad(pad_dims).shuffle(order); + } +}; + +template +void SpatialConvolutionFunc(const Device& d, Output output, Input input, + Filter filter, int row_stride, int col_stride, + int row_dilation, int col_dilation, + const Eigen::PaddingType& padding) { + // Need to swap row/col when calling Eigen. + output.device(d) = + Eigen::SpatialConvolution(input, filter, col_stride, row_stride, padding, + col_dilation, row_dilation); +} + +template +struct SpatialConvolution { + void operator()(const Device& d, typename TTypes::Tensor output, + typename TTypes::ConstTensor input, + typename TTypes::ConstTensor filter, int row_stride, + int col_stride, int row_dilation, int col_dilation, + const Eigen::PaddingType& padding) { + SpatialConvolutionFunc(d, output, input, filter, row_stride, col_stride, + row_dilation, col_dilation, padding); + } +}; + +template +struct SpatialConvolution { + void operator()(const Device& d, + typename TTypes::Tensor output, + typename TTypes::ConstTensor input, + typename TTypes::ConstTensor filter, + int row_stride, int col_stride, int row_dilation, + int col_dilation, const Eigen::PaddingType& padding) { + output.device(d) = + Eigen::SpatialConvolution(input.cast(), filter.cast(), + col_stride, row_stride, padding, col_dilation, + row_dilation) + .cast(); + } +}; + +template +struct SpatialConvolutionBackwardInput { + void operator()(const Device& d, typename TTypes::Tensor input_backward, + typename TTypes::ConstTensor kernel, + typename TTypes::ConstTensor output_backward, + int row_stride, int col_stride, int row_dilation, + int col_dilation) { + // Need to swap row/col when calling Eigen. + input_backward.device(d) = Eigen::SpatialConvolutionBackwardInput( + kernel, output_backward, input_backward.dimension(2), + input_backward.dimension(1), col_stride, row_stride, col_dilation, + row_dilation); + } +}; + +template +struct SpatialConvolutionBackwardFilter { + void operator()(const Device& d, + typename TTypes::Tensor kernel_backward, + typename TTypes::ConstTensor input, + typename TTypes::ConstTensor output_backward, + int row_stride, int col_stride, int row_dilation, + int col_dilation) { + // Need to swap row/col when calling Eigen. + kernel_backward.device(d) = Eigen::SpatialConvolutionBackwardKernel( + input, output_backward, kernel_backward.dimension(1), + kernel_backward.dimension(0), col_stride, row_stride, col_dilation, + row_dilation); + } +}; + +// TODO(vrv): Figure out how to use the MatMulFunctor in matmul_op.h. +// My initial attempt to do this compiled but failed in the pytest +// due to a swigdeps error. +template +struct MatMulConvFunctor { + // Computes on device "d": out = in0 * in1, where * is matrix + // multiplication. + void operator()( + const Device& d, typename TTypes::Tensor out, + typename TTypes::ConstTensor in0, + typename TTypes::ConstTensor in1, + const Eigen::array, 1>& dim_pair) { + out.device(d) = in0.contract(in1, dim_pair); + } +}; + +// Shuffles a filter tensor from TensorFlow format HWIO to dst_filter_format. +// +// Note: Currently OIHW is the only supported destination format. Support for +// OHWI format will be added in a follow-up change. +template +struct TransformFilter { + void operator()(const Device& d, FilterTensorFormat dst_filter_format, + typename TTypes::ConstTensor in, + typename TTypes::Tensor out) { + // Merge the spatial dimensions together to speed up the shuffle operation. + Eigen::DSizes merged_dims; + merged_dims[0] = in.dimension(0); // spatial dimensions + for (int i = 1; i < NDIMS - 2; ++i) { + merged_dims[0] *= in.dimension(i); + } + merged_dims[1] = in.dimension(NDIMS - 2); // input filters + merged_dims[2] = in.dimension(NDIMS - 1); // output filters + + CHECK(dst_filter_format == FORMAT_OIHW) + << "Unsupported destination filter format: " + << ToString(dst_filter_format); + // Source filter format is FORMAT_HWIO and spatial dimensions HW are merged + // in the beginning. + Eigen::DSizes shuffling_perm = + Eigen::DSizes(2, 1, 0); + + Eigen::DSizes expanded_dims; + int out_index = 0; + for (int merged_dim = 0; merged_dim < merged_dims.rank(); ++merged_dim) { + if (shuffling_perm[merged_dim] == 0) { + for (int spatial_dim = 0; spatial_dim < NDIMS - 2; ++spatial_dim) { + expanded_dims[out_index++] = in.dimension(spatial_dim); + } + } else { + constexpr int kLastSpatialDim = NDIMS - 3; + expanded_dims[out_index++] = + in.dimension(kLastSpatialDim + shuffling_perm[merged_dim]); + } + } + + out.device(d) = + in.reshape(merged_dims).shuffle(shuffling_perm).reshape(expanded_dims); + } +}; + +template +struct TransformDepth { + void operator()(const Device& d, + typename TTypes::ConstTensor in, + const Eigen::DSizes& shuffle, + typename TTypes::Tensor out) { + Eigen::DSizes merged_dims; + Eigen::DSizes expanded_dims; + Eigen::DSizes new_shuffle; + + // Merge dimensions that won't be shuffled together to speed things up. + if (shuffle[1] == 2 && shuffle[2] == 3) { + merged_dims[0] = in.dimension(0); + merged_dims[1] = in.dimension(1); + merged_dims[2] = in.dimension(2) * in.dimension(3); + new_shuffle[0] = shuffle[0]; + new_shuffle[1] = 2; + new_shuffle[2] = shuffle[3]; + expanded_dims[0] = in.dimension(shuffle[0]); + expanded_dims[1] = in.dimension(2); + expanded_dims[2] = in.dimension(3); + expanded_dims[3] = in.dimension(shuffle[3]); + } else if (shuffle[0] == 2 && shuffle[1] == 3) { + merged_dims[0] = in.dimension(0); + merged_dims[1] = in.dimension(1); + merged_dims[2] = in.dimension(2) * in.dimension(3); + new_shuffle[0] = 2; + new_shuffle[1] = shuffle[2]; + new_shuffle[2] = shuffle[3]; + expanded_dims[0] = in.dimension(2); + expanded_dims[1] = in.dimension(3); + expanded_dims[2] = in.dimension(shuffle[2]); + expanded_dims[3] = in.dimension(shuffle[3]); + } else if (shuffle[0] == 0 && shuffle[1] == 3 && shuffle[2] == 1 && + shuffle[3] == 2) { + merged_dims[0] = in.dimension(0); + merged_dims[1] = in.dimension(1) * in.dimension(2); + merged_dims[2] = in.dimension(3); + new_shuffle[0] = 0; + new_shuffle[1] = 2; + new_shuffle[2] = 1; + expanded_dims[0] = in.dimension(0); + expanded_dims[1] = in.dimension(3); + expanded_dims[2] = in.dimension(1); + expanded_dims[3] = in.dimension(2); + } else { + assert(false && "unexpected shuffle"); + } + + out.device(d) = + in.reshape(merged_dims).shuffle(new_shuffle).reshape(expanded_dims); + } +}; + +template +struct PadInput { + void operator()(const Device& d, + typename TTypes::ConstTensor in, + const std::array& padding_left, + const std::array& padding_right, + typename TTypes::Tensor out, + TensorFormat format) { + Eigen::array, NDIMS> padding; + padding[GetTensorDimIndex(format, 'N')] = {0, 0}; + for (int i = 0; i < NDIMS - 2; ++i) { + padding[GetTensorDimIndex(format, '0' + i)] = { + padding_left[i], padding_right[i]}; + } + padding[GetTensorDimIndex(format, 'C')] = {0, 0}; + out.device(d) = in.pad(padding); + } +}; + +// Converts a tensor from: +// [batch, , filters] +// to: +// [batch, filters, ] +template +struct NHWCToNCHW { + void operator()(const Device& d, typename TTypes::ConstTensor in, + typename TTypes::Tensor out); +}; + +// Converts a tensor from: +// [batch, filters, ] +// to: +// [batch, , filters] +template +struct NCHWToNHWC { + void operator()(const Device& d, typename TTypes::ConstTensor in, + typename TTypes::Tensor out); +}; + +// Converts a tensor from: +// [dim0, dim1, dim2] +// to: +// [dim0, dim2, dim1] +template +struct SwapDimension1And2InTensor3 { + void operator()(const Device& d, const T* in, + const gtl::ArraySlice& input_dims, T* out); +}; + +// Converts a tensor from: +// [dim0, dim1, dim2] +// to: +// [dim2, dim1, dim0] +template +struct SwapDimension0And2InTensor3 { + void operator()(const Device& d, const T* in, + const gtl::ArraySlice& input_dims, T* out); +}; + +// Transforms back filter from OIHW to HWOI format to reverse effect of +// TransformFilter above. +// TODO(hinsu): Support reverse transformation from filter format OHWI as well. +template +struct ReverseTransformFilter { + void operator()(const Device& d, typename TTypes::ConstTensor in, + typename TTypes::Tensor out); +}; + +} // namespace functor + +template +class ConvAlgorithmMap; + +template <> +class ConvAlgorithmMap {}; +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_CONV_2D_H_ diff --git a/linux/include/tensorflow/core/kernels/conv_3d.h b/linux/include/tensorflow/core/kernels/conv_3d.h new file mode 100644 index 000000000..b819c6f91 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/conv_3d.h @@ -0,0 +1,91 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// Functors for 3d convolution. + +#ifndef TENSORFLOW_CORE_KERNELS_CONV_3D_H_ +#define TENSORFLOW_CORE_KERNELS_CONV_3D_H_ + +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/kernels/eigen_backward_cuboid_convolutions.h" +#include "tensorflow/core/kernels/eigen_cuboid_convolution.h" + +namespace tensorflow { +namespace functor { + +// Applies a 3D convolution to a batch of multi-channel volumes. +template +struct CuboidConvolution; + +// Backward input pass for the cuboid convolution. +template +struct CuboidConvolutionBackwardInput; + +// Backward filter pass for the cuboid convolution. +template +struct CuboidConvolutionBackwardFilter; + +typedef Eigen::ThreadPoolDevice CPUDevice; + +template +struct CuboidConvolution { + void operator()(const CPUDevice& d, typename TTypes::Tensor output, + typename TTypes::ConstTensor input, + typename TTypes::ConstTensor filter, int stride_planes, + int stride_rows, int stride_cols, + const Eigen::PaddingType& padding) { + output.device(d) = Eigen::CuboidConvolution( + input, filter, stride_planes, stride_rows, stride_cols, padding); + } +}; + +template +struct CuboidConvolutionBackwardInput { + void operator()(const CPUDevice& d, + typename TTypes::Tensor input_backward, + typename TTypes::ConstTensor filter, + typename TTypes::ConstTensor output_backward, + int stride_planes, int stride_rows, int stride_cols) { + // Need to swap the order of plane/row/col strides when calling Eigen. + input_backward.device(d) = Eigen::CuboidConvolutionBackwardInput( + filter, output_backward, + input_backward.dimension(3), // input_planes + input_backward.dimension(2), // input_rows + input_backward.dimension(1), // input_cols + stride_cols, stride_rows, stride_planes); + } +}; + +template +struct CuboidConvolutionBackwardFilter { + void operator()(const CPUDevice& d, + typename TTypes::Tensor filter_backward, + typename TTypes::ConstTensor input, + typename TTypes::ConstTensor output_backward, + int stride_planes, int stride_rows, int stride_cols) { + // Need to swap the order of plane/row/col strides when calling Eigen. + filter_backward.device(d) = Eigen::CuboidConvolutionBackwardKernel( + input, output_backward, + filter_backward.dimension(2), // kernel_planes + filter_backward.dimension(1), // kernel_rows + filter_backward.dimension(0), // kernel_cols + stride_cols, stride_rows, stride_planes); + } +}; + +} // namespace functor +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_CONV_3D_H_ diff --git a/linux/include/tensorflow/core/kernels/conv_grad_ops.h b/linux/include/tensorflow/core/kernels/conv_grad_ops.h new file mode 100644 index 000000000..955195946 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/conv_grad_ops.h @@ -0,0 +1,269 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// This is the common header for the input and filter backprop kernels. +// +// The operation to compute Conv2D gradients. +// +// To compute the gradients for Conv2D, we need three input tensors: +// input, filter, and backprop for output. +// And we need to compute two backprops: one for input and one for filter. We +// compute them in two different kernels. +// +// Both backprops can be computed as straightforward conv2d. +// +// Consider a case where the input is 3x3 and the filter is 2x1: +// +// INPUT = [ A B C ] +// [ D E F ] +// [ G H I ] +// +// where each "A", "B", etc is batch x in_depth +// +// FILTER = [ X Y ] +// +// where both "X" and "Y" are in_depth x out_depth +// +// With VALID padding, the output is 3x2: +// +// OUTPUT = [ a b ] +// [ c d ] +// [ e f ] +// +// where each "a", "b", etc is batch x out_depth +// +// So we have: +// +// a = A * X + B * Y +// b = B * X + C * Y +// c = D * X + E * Y +// d = E * X + F * Y +// e = G * X + H * Y +// f = H * X + I * Y +// +// So when we have backprops for the outputs (we denote them by +// a', b', ... ): +// +// The backprops for the input are: +// +// A' = a' * X^t +// B' = a' * Y^t + b' * X^t +// C' = b' * Y^t +// ... +// +// This is essentially computing a 2d conv of +// +// INPUT = [ 0 a' b' 0 ] +// [ 0 c' d' 0 ] +// [ 0 e' f' 0 ] +// and +// +// FILTER = [ Y^t X^t ] +// +// The backprops for the filter are: +// +// X' = A^t * a' + B^t * b' + D^t * c' + E^t * d' + G^t * e' + H^t * f' +// Y' = B^t * a' + C^t * b' + E^t + c' + F^t * d' + H^t * e' + I^t * f' +// +// This is essentially computing a 2d conv of +// +// INPUT = [ A^t B^t C^t ] +// [ D^t E^t F^t ] +// [ G^t H^t I^t ] +// +// and +// +// FILTER = [ a' b' ] +// [ c' d' ] +// [ e' f' ] +// +// +////////////////////////////////////////////////////////// +// +// With stride more than one, it's a bit more complicated (we will need to +// create holes to the backprop). +// +// Consider the case where +// +// INPUT = [ A B C D E ] +// [ F G H I J ] +// [ K L M N O ] +// and +// +// FILTER = [ X Y Z ] +// +// with stride 2. +// +// The output will be +// +// OUTPUT = [ a b ] +// [ c d ] +// +// where: +// +// a = A * X + B * Y + C * Z +// b = C * X + D * Y + E * Z +// c = K * X + L * Y + M * Z +// d = M * X + N * Y + O * Z +// +// +// To compute the backprop for INPUT, we need to convolve +// +// INPUT = [ 0 0 a' 0 b' 0 0 ] +// [ 0 0 0 0 0 0 0 ] +// [ 0 0 c' 0 d' 0 0 ] +// +// (notice the holes in INPUT) +// +// and +// +// FILTER = [ Z^t Y^t X^t ] +// +// with stride 1. +// +// To compute the backprop for FILTER, we need to convolve + +// +// INPUT = [ A^t B^t C^t D^t E^t ] +// [ F^t G^t H^t I^t J^t ] +// [ K^t L^t M^t N^t O^t ] +// and +// +// FILTER = [ a' 0 b' ] +// [ 0 0 0 ] +// [ c' 0 d' ] +// +// (notice the holes in FILTER) +// +// +// with stride 1 +// +////////////////////////////////////////////////////////// +// +// +// The case for SAME padding is in fact very similar to VALID -- we just +// need to pad the input tensor a bit when computing the filter_backprop. + +#ifndef TENSORFLOW_CORE_KERNELS_CONV_GRAD_OPS_H_ +#define TENSORFLOW_CORE_KERNELS_CONV_GRAD_OPS_H_ + +#include + +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/lib/core/stringpiece.h" +#include "tensorflow/core/util/padding.h" +#include "tensorflow/core/util/tensor_format.h" + +namespace tensorflow { + +// Forward declaration. +class OpKernelContext; + +template +struct LaunchConv2DBackpropInputOp { + void operator()(OpKernelContext* ctx, bool use_cudnn, bool cudnn_use_autotune, + const Tensor& out_backprop, const Tensor& filter, + int row_dilation, int col_dilation, int row_stride, + int col_stride, const Padding& padding, Tensor* in_backprop, + TensorFormat data_format); +}; + +template +struct LaunchConv2DBackpropFilterOp { + void operator()(OpKernelContext* ctx, bool use_cudnn, bool cudnn_use_autotune, + const Tensor& out_backprop, const Tensor& input, + int row_dilation, int col_dilation, int row_stride, + int col_stride, const Padding& padding, + Tensor* filter_backprop, TensorFormat data_format); +}; + +#ifdef GOOGLE_CUDA +template +struct LaunchConv2DBackpropInputOp { + void operator()(OpKernelContext* ctx, bool use_cudnn, bool cudnn_use_autotune, + const Tensor& input, const Tensor& filter, int row_dilation, + int col_dilation, int row_stride, int col_stride, + const Padding& padding, Tensor* output, + TensorFormat data_format); +}; + +template +struct LaunchConv2DBackpropFilterOp { + void operator()(OpKernelContext* ctx, bool use_cudnn, bool cudnn_use_autotune, + const Tensor& out_backprop, const Tensor& input, + int row_dilation, int col_dilation, int row_stride, + int col_stride, const Padding& padding, + Tensor* filter_backprop, TensorFormat data_format); +}; +#endif // GOOGLE_CUDA + +// Information about a single spatial dimension for a convolution +// backpropagation. +struct ConvBackpropSpatialDimension { + int64 input_size; + int64 filter_size; + int64 output_size; + int64 stride; + int64 dilation; + int64 expanded_output_size; + + // Number of padding elements to be added before/after this dimension of + // the input when computing Conv?DBackpropInput. + int64 pad_before, pad_after; +}; + +// Computed dimensions for a backwards convolution. +struct ConvBackpropDimensions { + // Information about each spatial dimension. + gtl::InlinedVector spatial_dims; + + // Batch size. + int64 batch_size; + + // Input and output feature depth. + int64 in_depth, out_depth; + + // Convenience access methods for spatial dimensions properties. + int64 input_size(int dim) const { return spatial_dims[dim].input_size; } + int64 filter_size(int dim) const { return spatial_dims[dim].filter_size; } + int64 output_size(int dim) const { return spatial_dims[dim].output_size; } + int64 stride(int dim) const { return spatial_dims[dim].stride; } + int64 dilation(int dim) const { return spatial_dims[dim].dilation; } + + // Compute padding for the given spatial dimension. + int SpatialPadding(const Padding& padding, int dim) const; +}; + +// Common code between implementations of Conv?DBackpropInput and +// Conv?DBackpropFilter. Verifies that the dimensions all match, and computes +// sizes/padding for the spatial dimensions. +Status ConvBackpropComputeDimensions(StringPiece label, int num_spatial_dims, + const TensorShape& input_shape, + const TensorShape& filter_shape, + const TensorShape& out_backprop_shape, + const std::vector& strides, + Padding padding, TensorFormat data_format, + ConvBackpropDimensions* dims); + +// The V2 version computes the same outputs with arbitrary dilation rate. +// TODO(b/67112639): Merge V2 versions and the original versions eventually. +Status ConvBackpropComputeDimensionsV2( + StringPiece label, int num_spatial_dims, const TensorShape& input_shape, + const TensorShape& filter_shape, const TensorShape& out_backprop_shape, + const gtl::ArraySlice& dilations, const std::vector& strides, + Padding padding, TensorFormat data_format, ConvBackpropDimensions* dims); +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_CONV_GRAD_OPS_H_ diff --git a/linux/include/tensorflow/core/kernels/conv_ops.h b/linux/include/tensorflow/core/kernels/conv_ops.h new file mode 100644 index 000000000..7ec878e0b --- /dev/null +++ b/linux/include/tensorflow/core/kernels/conv_ops.h @@ -0,0 +1,115 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_CONV_OPS_H_ +#define TENSORFLOW_CORE_KERNELS_CONV_OPS_H_ + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/resource_mgr.h" +#include "tensorflow/core/platform/mem.h" +#include "tensorflow/core/util/tensor_format.h" + +#if GOOGLE_CUDA +#include "tensorflow/core/kernels/conv_ops_gpu.h" +#include "tensorflow/core/platform/stream_executor.h" +#endif // GOOGLE_CUDA + +namespace tensorflow { + +// Forward declaration. +class OpKernelContext; + +template +struct LaunchConv2DOp { + void operator()(OpKernelContext* ctx, bool use_cudnn, bool cudnn_use_autotune, + const Tensor& input, const Tensor& filter, int row_dilation, + int col_dilation, int row_stride, int col_stride, + const Padding& padding, Tensor* output, + TensorFormat data_format); +}; + +#ifdef GOOGLE_CUDA +template +struct LaunchConv2DOp { + void operator()(OpKernelContext* ctx, bool use_cudnn, bool cudnn_use_autotune, + const Tensor& input, const Tensor& filter, int row_dilation, + int col_dilation, int row_stride, int col_stride, + const Padding& padding, Tensor* output, + TensorFormat data_format); +}; +#endif // GOOGLE_CUDA + +// Used to keep track of persistent memory buffers used within the op. +// It uses malloc and free to avoid the time cost of initializing the memory. +template +struct Im2ColBufferResource : public ResourceBase { + Im2ColBufferResource() { + data = static_cast(port::Malloc(size * sizeof(T))); + } + ~Im2ColBufferResource() { port::Free(data); } + // This mutex ensures that only a single operation at a time is able to use + // the buffer memory held by this resource. + mutex mu; + T* data; + string DebugString() { return "Im2ColBufferResource"; } +}; + +// Convolution parameters specified by Op attributes. +struct Conv2DParameters { + std::vector dilations; + std::vector strides; + Padding padding; + TensorFormat data_format; +}; + +// Convolution dimensions inferred from parameters, input and filter tensors. +struct Conv2DDimensions { + int batch; + int input_rows; + int input_cols; + int in_depth; + + int filter_rows; + int filter_cols; + int patch_depth; + int out_depth; + + int stride_rows; + int stride_cols; + + int dilation_rows; + int dilation_cols; + + int64 out_rows; + int64 out_cols; + int64 pad_rows; + int64 pad_cols; +}; + +// Initializes and validates Conv2D parameters configured by OpKernel +// attributes. +Status InitConv2DParameters(const OpKernelConstruction* context, + Conv2DParameters* params); + +// Computes and validates convolutions dimensions from Conv2D parameters. If +// parameters are valid, dimensions will be updated with derived convolution +// dimensions, otherwise error will be returned. +Status ComputeConv2DDimension(const Conv2DParameters& params, + const Tensor& input, const Tensor& filter, + Conv2DDimensions* dimensions); + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_CONV_OPS_H_ diff --git a/linux/include/tensorflow/core/kernels/conv_ops_gpu.h b/linux/include/tensorflow/core/kernels/conv_ops_gpu.h new file mode 100644 index 000000000..21d135dec --- /dev/null +++ b/linux/include/tensorflow/core/kernels/conv_ops_gpu.h @@ -0,0 +1,205 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_CONV_OPS_GPU_H_ +#define TENSORFLOW_CORE_KERNELS_CONV_OPS_GPU_H_ + +#if GOOGLE_CUDA + +#include +#include +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/kernels/gpu_utils.h" +#include "tensorflow/core/lib/gtl/inlined_vector.h" +#include "tensorflow/core/lib/hash/hash.h" + +namespace tensorflow { + +// Get the Cudnn workspace limit from the environment variable, which is in MB. +// Return the workspace memory limit in bytes. If no value is set, return the +// default value. +int64 GetCudnnWorkspaceLimit(const string& envvar_in_mb, + int64 default_value_in_bytes); + +// A class to provide scratch-space allocator for Stream-Executor Cudnn +// callback. TensorFlow is responsible for releasing the temporary buffers after +// the kernel finishes. +class CudnnScratchAllocator : public se::ScratchAllocator { + public: + virtual ~CudnnScratchAllocator() {} + CudnnScratchAllocator(int64 memory_limit, OpKernelContext* context) + : memory_limit_(memory_limit), total_byte_size_(0), context_(context) {} + int64 GetMemoryLimitInBytes(se::Stream* stream) override { + return memory_limit_; + } + se::port::StatusOr> AllocateBytes( + se::Stream* stream, int64 byte_size) override { + Tensor temporary_memory; + if (byte_size < 0) { + return se::port::Status{se::port::error::INVALID_ARGUMENT, + "Requested negative byte size!"}; + } + if (byte_size > memory_limit_) { + return se::port::StatusOr>(); + } + AllocationAttributes allocation_attr; + allocation_attr.no_retry_on_failure = true; + Status allocation_status(context_->allocate_temp( + DT_UINT8, TensorShape({byte_size}), &temporary_memory, + AllocatorAttributes(), allocation_attr)); + if (!allocation_status.ok()) { + return se::port::StatusOr>(); + } + // Hold the reference of the allocated tensors until the end of the + // allocator. + allocated_tensors_.push_back(temporary_memory); + total_byte_size_ += byte_size; + return se::port::StatusOr>( + AsDeviceMemory(temporary_memory.flat().data(), + temporary_memory.flat().size())); + } + int64 TotalByteSize() { return total_byte_size_; } + + private: + int64 memory_limit_; + int64 total_byte_size_; + OpKernelContext* context_; + std::vector allocated_tensors_; +}; + +// Encapsulate all the shape information that is used in both forward and +// backward conv operations. +class ConvParameters { + public: + using SpatialArray = gtl::InlinedVector; + ConvParameters(int64 batch, int64 in_depths, const SpatialArray& in, + TensorFormat data_format, int64 out_depths, + const SpatialArray& filter, const SpatialArray& dilation, + const SpatialArray& stride, const SpatialArray& padding, + DataType dtype, int device_id) + : batch_(batch), + in_depths_(in_depths), + out_depths_(out_depths), + in_(in), + data_format_(data_format), + filter_(filter), + dilation_(dilation), + stride_(stride), + padding_(padding), + dtype_(dtype), + device_id_(device_id) { + hash_code_ = batch; + hash_code_ = Hash64Combine(hash_code_, in_depths); + for (int64 val : in) hash_code_ = Hash64Combine(hash_code_, val); + hash_code_ = Hash64Combine(hash_code_, data_format); + hash_code_ = Hash64Combine(hash_code_, out_depths); + for (int64 val : filter) hash_code_ = Hash64Combine(hash_code_, val); + for (int64 val : dilation) hash_code_ = Hash64Combine(hash_code_, val); + for (int64 val : stride) hash_code_ = Hash64Combine(hash_code_, val); + for (int64 val : padding) hash_code_ = Hash64Combine(hash_code_, val); + hash_code_ = Hash64Combine(hash_code_, dtype); + hash_code_ = Hash64Combine(hash_code_, device_id); + } + bool operator==(const ConvParameters& other) const { + return this->get_data_as_tuple() == other.get_data_as_tuple(); + } + + bool operator!=(const ConvParameters& other) const { + return !(*this == other); + } + uint64 hash() const { return hash_code_; } + + string ToString() const { + // clang-format off + return strings::StrCat( + batch_, ", ", in_depths_, ", ", + "(", str_util::Join(in_, ", "), "), ", + ::tensorflow::ToString(data_format_), ", ", + out_depths_, ", ", + "(", str_util::Join(filter_, ", "), "), ", + "(", str_util::Join(dilation_, ", "), "), ", + "(", str_util::Join(stride_, ", "), "), ", + "(", str_util::Join(padding_, ", "), "), ", + dtype_, ", ", + device_id_); + // clang-format on + } + + // The purpose of this function is to disable winograd nonfused conv algorithm + // for certain input parameters so as to avoid a bug in cuDNNv5 and cuDNNv6. + template + bool ShouldIncludeWinogradNonfusedAlgo( + se::StreamExecutor* stream_exec) const { + auto* dnn_support = stream_exec->AsDnn(); + if (!dnn_support) { + return false; + } + // Skip this check for cuDNN 7 and newer. + auto version = dnn_support->GetVersion(); + if (version.ok() && version.ValueOrDie().major_version() >= 7) { + return true; + } + return ShouldIncludeWinogradNonfusedAlgoPreCudnn7(); + } + + protected: + using ParameterDataType = + std::tuple; + + ParameterDataType get_data_as_tuple() const { + return std::make_tuple(batch_, in_depths_, in_, data_format_, out_depths_, + filter_, dilation_, stride_, padding_, dtype_, + device_id_); + } + + uint64 hash_code_; + + private: + friend struct ConvParametersPeer; // For testing purposes. + + template + bool ShouldIncludeWinogradNonfusedAlgoPreCudnn7() const { + int64 total_size = 16 * std::ceil(batch_ / 16.0) * + std::max(in_depths_, out_depths_) * in_[0] * in_[1] * + sizeof(T); + int64 threshold = 1LL << 31; + if (total_size >= threshold) { + return false; + } else { + return true; + } + } + + int64 batch_; + int64 in_depths_; + int64 out_depths_; + SpatialArray in_; + TensorFormat data_format_; + SpatialArray filter_; + SpatialArray dilation_; + SpatialArray stride_; + SpatialArray padding_; + DataType dtype_; + int device_id_; +}; + +typedef Eigen::GpuDevice GPUDevice; + +} // namespace tensorflow + +#endif // GOOGLE_CUDA + +#endif // TENSORFLOW_CORE_KERNELS_CONV_OPS_GPU_H_ diff --git a/linux/include/tensorflow/core/kernels/crop_and_resize_op.h b/linux/include/tensorflow/core/kernels/crop_and_resize_op.h new file mode 100644 index 000000000..61dc3f941 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/crop_and_resize_op.h @@ -0,0 +1,71 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_CROP_AND_RESIZE_OP_H_ +#define TENSORFLOW_CORE_KERNELS_CROP_AND_RESIZE_OP_H_ + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/numeric_types.h" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/tensor_types.h" + +namespace tensorflow { +namespace functor { + +template +struct CropAndResize { + // We assume that the tensor sizes are correct. + bool operator()(const OpKernelContext* context, + typename TTypes::ConstTensor image, + typename TTypes::ConstTensor boxes, + typename TTypes::ConstTensor box_ind, + string method_name, float extrapolation_value, + typename TTypes::Tensor crops); +}; + +template +struct CropAndResizeBackpropImage { + // We assume that the tensor sizes are correct. + bool operator()(const Device& d, typename TTypes::ConstTensor grads, + typename TTypes::ConstTensor boxes, + typename TTypes::ConstTensor box_ind, + typename TTypes::Tensor grads_image, + const string& method_name); +}; + +template +struct CropAndResizeBackpropBoxes { + // We assume that the tensor sizes are correct. + bool operator()(const Device& d, typename TTypes::ConstTensor grads, + typename TTypes::ConstTensor image, + typename TTypes::ConstTensor boxes, + typename TTypes::ConstTensor box_ind, + typename TTypes::Tensor grads_boxes); +}; + +template +struct CheckValidBoxIndexHelper { + // Checks if all values in box_index are in [0, batch). + void operator()(const Device& d, + typename TTypes::ConstTensor box_index, int batch, + typename TTypes::Tensor isvalid) { + isvalid.device(d) = ((box_index >= 0) && (box_index < batch)).all(); + } +}; + +} // namespace functor +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_CROP_AND_RESIZE_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/cross_op.h b/linux/include/tensorflow/core/kernels/cross_op.h new file mode 100644 index 000000000..45bc46a92 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/cross_op.h @@ -0,0 +1,54 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_CROSS_OP_H_ +#define TENSORFLOW_CORE_KERNELS_CROSS_OP_H_ + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/tensor_types.h" + +namespace tensorflow { + +namespace functor { + +template +struct Cross { + void operator()(const Device &d, + typename TTypes::ConstTensor in0_data, + typename TTypes::ConstTensor in1_data, + typename TTypes::Tensor output_data) { + auto s1 = output_data.template chip<1>(0); + auto s2 = output_data.template chip<1>(1); + auto s3 = output_data.template chip<1>(2); + + auto u1 = in0_data.template chip<1>(0); + auto u2 = in0_data.template chip<1>(1); + auto u3 = in0_data.template chip<1>(2); + + auto v1 = in1_data.template chip<1>(0); + auto v2 = in1_data.template chip<1>(1); + auto v3 = in1_data.template chip<1>(2); + + s1.device(d) = u2 * v3 - u3 * v2; + s2.device(d) = u3 * v1 - u1 * v3; + s3.device(d) = u1 * v2 - u2 * v1; + } +}; + +} // namespace functor +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_CROSS_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/cuda_device_array.h b/linux/include/tensorflow/core/kernels/cuda_device_array.h new file mode 100644 index 000000000..74dc298c7 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/cuda_device_array.h @@ -0,0 +1,120 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CORE_KERNELS_CUDA_DEVICE_ARRAY_H_ +#define TENSORFLOW_CORE_KERNELS_CUDA_DEVICE_ARRAY_H_ + +#if GOOGLE_CUDA + +#include "tensorflow/core/common_runtime/gpu/gpu_event_mgr.h" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/kernels/cuda_device_array_gpu.h" + +namespace tensorflow { + +// Create an array of value on the host, to be sent to kernel using +// CudaDeviceArrayStruct. +// +// Usage: +// int size = ...; +// CudaDeviceArrayOnHost ptrs(context, size); +// OP_REQUIRES_OK(ptrs.Init()); +// for (int i = 0; i < size; ++i) { +// ptrs.Set(i, ...); +// } +// OP_REQUIRES_OK(ptrs.Finalize()); +// launchKernel(..., ptrs.data, ...); +// +// ValueType must be memcopyable. +template +class CudaDeviceArrayOnHost { + public: + CudaDeviceArrayOnHost(OpKernelContext* context, int32 size) + : context_(context), + total_bytes_(static_cast(size) * sizeof(ValueType)) { + data_.size = size; + } + + Status Init() { + if (inlined()) { + values_ = data_.inline_values; + return Status::OK(); + } + + // Out-of-line: allocate data that will be memcopied. + AllocatorAttributes attr; + attr.set_on_host(true); + attr.set_gpu_compatible(true); + TF_RETURN_IF_ERROR( + context_->allocate_temp(DT_INT8, TensorShape{total_bytes_}, + &out_of_line_values_on_host_, attr)); + values_ = reinterpret_cast( + out_of_line_values_on_host_.flat().data()); + return Status::OK(); + } + + void Set(int index, ValueType val) { + DCHECK(values_); // ensure Init was called. + DCHECK_LT(index, data_.size); + *(values_ + index) = val; + } + + Status Finalize() { + if (inlined()) { + return Status::OK(); + } + + // Out-of-line - copy pointers to device. + auto stream = context_->op_device_context()->stream(); + TensorReference tensor_ref(out_of_line_values_on_host_); + TF_RETURN_IF_ERROR(context_->allocate_temp( + DT_INT8, TensorShape{total_bytes_}, &out_of_line_values_on_gpu_)); + se::DeviceMemoryBase output_values_base{ + out_of_line_values_on_gpu_.flat().data(), + static_cast(total_bytes_)}; + stream->ThenMemcpy(&output_values_base, + out_of_line_values_on_host_.flat().data(), + total_bytes_); + context_->device()->tensorflow_gpu_device_info()->event_mgr->ThenExecute( + stream, [tensor_ref]() { tensor_ref.Unref(); }); + data_.out_of_line_values = reinterpret_cast( + out_of_line_values_on_gpu_.flat().data()); + return Status::OK(); + } + + const CudaDeviceArrayStruct& data() const { + // Ensure Finalize is called. + DCHECK(inlined() || out_of_line_values_on_gpu_.IsInitialized()); + return data_; + } + + private: + bool inlined() const { return data_.size <= MaxInlineValues; } + + OpKernelContext* const context_; + const int64 total_bytes_; // total size of all pointers. + ValueType* values_ = nullptr; + CudaDeviceArrayStruct data_; + + Tensor out_of_line_values_on_host_; + Tensor out_of_line_values_on_gpu_; + + TF_DISALLOW_COPY_AND_ASSIGN(CudaDeviceArrayOnHost); +}; + +} // namespace tensorflow + +#endif // GOOGLE_CUDA + +#endif // TENSORFLOW_CORE_KERNELS_CUDA_DEVICE_ARRAY_H_ diff --git a/linux/include/tensorflow/core/kernels/cuda_device_array_gpu.h b/linux/include/tensorflow/core/kernels/cuda_device_array_gpu.h new file mode 100644 index 000000000..64fa3cb80 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/cuda_device_array_gpu.h @@ -0,0 +1,50 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// Contains structs and functions to be included in device code. + +#ifndef TENSORFLOW_CORE_KERNELS_CUDA_DEVICE_ARRAY_GPU_H_ +#define TENSORFLOW_CORE_KERNELS_CUDA_DEVICE_ARRAY_GPU_H_ + +#if GOOGLE_CUDA + +namespace tensorflow { + +static constexpr int kMaxInlineCudaPointers = 8; +// To decode on the device side, use GetCudaDeviceArrayOnDevice. +// To encode on the host side, use CudaDeviceArrayOnHost. +template +struct CudaDeviceArrayStruct { + int32 size; + // used if size <= MaxInlineValues; + ValueType inline_values[MaxInlineValues]; + ValueType* out_of_line_values = nullptr; // used if size > MaxInlineValues; +}; + +template +EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE ValueType* GetCudaDeviceArrayOnDevice( + CudaDeviceArrayStruct* data) { + if (data->size <= MaxInlineValues) { + return data->inline_values; + } else { + return data->out_of_line_values; + } +} + +} // namespace tensorflow + +#endif // GOOGLE_CUDA + +#endif // TENSORFLOW_CORE_KERNELS_CUDA_DEVICE_ARRAY_GPU_H_ diff --git a/linux/include/tensorflow/core/kernels/cudnn_pooling_gpu.h b/linux/include/tensorflow/core/kernels/cudnn_pooling_gpu.h new file mode 100644 index 000000000..738e92824 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/cudnn_pooling_gpu.h @@ -0,0 +1,70 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// Helper functions to run 3d pooling on GPU using CuDNN. + +#ifndef TENSORFLOW_CORE_KERNELS_CUDNN_POOLING_GPU_H_ +#define TENSORFLOW_CORE_KERNELS_CUDNN_POOLING_GPU_H_ + +#include + +#include "tensorflow/core/framework/op_kernel.h" + +#if GOOGLE_CUDA +#include "tensorflow/core/platform/stream_executor.h" +#endif + +#include "tensorflow/core/util/padding.h" + +namespace tensorflow { + +#if GOOGLE_CUDA + +// Runs (avg/max)pooling on GPU. +// Dimension order for all array arguments is: x, y, z. +template +class DnnPooling3dOp { + public: + static void Compute(OpKernelContext* context, + se::dnn::PoolingMode pooling_mode, + const std::array& size, + const std::array& stride, + const std::array& padding, + TensorFormat data_format, const Tensor& tensor_in, + Tensor* output); +}; + +// Computes the gradient of (avg/max)pooling on GPU. +// Dimension order for all array arguments is: x, y, z. +template +class DnnPooling3dGradOp { + public: + static void Compute(OpKernelContext* context, + se::dnn::PoolingMode pooling_mode, + const std::array& window, + const std::array& stride, + const std::array& padding, + const std::array& output_size, + TensorFormat data_format, const Tensor& out_backprop, + const TensorShape& tensor_in_shape, + const Tensor* tensor_in, const Tensor* tensor_out, + Tensor* input_backprop); +}; + +#endif + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_CUDNN_POOLING_GPU_H_ diff --git a/linux/include/tensorflow/core/kernels/cwise_op_clip.h b/linux/include/tensorflow/core/kernels/cwise_op_clip.h new file mode 100644 index 000000000..171b6932c --- /dev/null +++ b/linux/include/tensorflow/core/kernels/cwise_op_clip.h @@ -0,0 +1,61 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_CWISE_OP_CLIP_H_ +#define TENSORFLOW_CORE_KERNELS_CWISE_OP_CLIP_H_ + +#include "tensorflow/core/kernels/cwise_ops_common.h" + +namespace tensorflow { +namespace functor { +// Unary functor for clip [Tensor, Scalar, Scalar] +template +struct UnaryClipOp { + void operator()(const Device &d, typename TTypes::ConstFlat &in0_flat, + typename TTypes::ConstFlat &in1_flat, + typename TTypes::ConstFlat &in2_flat, + typename TTypes::Flat &out_flat) const; +}; + +// Binary functor for clip [Tensor, Scalar, Tensor] +template +struct BinaryRightClipOp { + void operator()(const Device &d, typename TTypes::ConstFlat &in0_flat, + typename TTypes::ConstFlat &in1_flat, + typename TTypes::ConstFlat &in2_flat, + typename TTypes::Flat &out_flat) const; +}; + +// Binary functor for clip [Tensor, Tensor, Scalar] +template +struct BinaryLeftClipOp { + void operator()(const Device &d, typename TTypes::ConstFlat &in0_flat, + typename TTypes::ConstFlat &in1_flat, + typename TTypes::ConstFlat &in2_flat, + typename TTypes::Flat &out_flat) const; +}; + +// Ternary functor for clip [Tensor, Tensor, Tensor] +template +struct TernaryClipOp { + void operator()(const Device &d, typename TTypes::ConstFlat &in0_flat, + typename TTypes::ConstFlat &in1_flat, + typename TTypes::ConstFlat &in2_flat, + typename TTypes::Flat &out_flat) const; +}; +} // namespace functor +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_CWISE_OP_CLIP_H_ diff --git a/linux/include/tensorflow/core/kernels/cwise_ops.h b/linux/include/tensorflow/core/kernels/cwise_ops.h new file mode 100644 index 000000000..66ba827a9 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/cwise_ops.h @@ -0,0 +1,1084 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_CWISE_OPS_H_ +#define TENSORFLOW_CORE_KERNELS_CWISE_OPS_H_ + +#include +#include +#include + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" + +#include "tensorflow/core/framework/numeric_types.h" +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/kernels/bounds_check.h" + +namespace Eigen { +namespace internal { + +template +struct scalar_asinh_op { + EIGEN_EMPTY_STRUCT_CTOR(scalar_asinh_op) + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE const T operator()(const T& a) const { +#if EIGEN_HAS_CXX11_MATH + return numext::asinh(a); +#else + return std::asinh(a); +#endif // EIGEN_HAS_CXX11_MATH + } +}; +template +struct functor_traits> { + enum { Cost = 5 * NumTraits::MulCost, PacketAccess = false }; +}; + +template +struct scalar_acosh_op { + EIGEN_EMPTY_STRUCT_CTOR(scalar_acosh_op) + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE const T operator()(const T& a) const { +#if EIGEN_HAS_CXX11_MATH + return numext::acosh(a); +#else + return std::acosh(a); +#endif // EIGEN_HAS_CXX11_MATH + } +}; +template +struct functor_traits> { + enum { Cost = 5 * NumTraits::MulCost, PacketAccess = false }; +}; + +template +struct scalar_atanh_op { + EIGEN_EMPTY_STRUCT_CTOR(scalar_atanh_op) + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE const T operator()(const T& a) const { +#if EIGEN_HAS_CXX11_MATH + return numext::atanh(a); +#else + return std::atanh(a); +#endif // EIGEN_HAS_CXX11_MATH + } +}; +template +struct functor_traits> { + enum { Cost = 5 * NumTraits::MulCost, PacketAccess = false }; +}; + +// TODO(rmlarsen): This is a workaround for upstream change +// https://bitbucket.org/eigen/eigen/commits/f339468d04d0f87caeb6cab9aef568627e9f6ea9 +// that renamed scalar_binary_pow_op to scalar_pow_op and deleted the unary +// version of the latter. Remove once we upgrade to Eigen 3.3. +template +struct scalar_binary_pow_op_google { + EIGEN_EMPTY_STRUCT_CTOR(scalar_binary_pow_op_google) + EIGEN_DEVICE_FUNC inline Scalar operator()(const Scalar& a, + const Exponent& b) const { + return numext::pow(a, b); + } +}; + +template +struct functor_traits> { + enum { Cost = 5 * NumTraits::MulCost, PacketAccess = false }; +}; + +template +struct safe_scalar_binary_pow_op { + static_assert(std::is_integral::value, "Integer type expected"); + static_assert(std::is_integral::value && + std::is_signed::value, + "Signed integer type expected"); + + bool* const error; + + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE safe_scalar_binary_pow_op(bool* error) + : error(error) {} + + EIGEN_DEVICE_FUNC inline Scalar operator()(const Scalar& a, + const Exponent& b) const { + const Exponent safe_b = tensorflow::internal::SubtleMustCopy(b); + if (TF_PREDICT_TRUE(safe_b >= 0)) { + return numext::pow(a, safe_b); + } else { + *error = true; + return 0; + } + } +}; + +template +struct functor_traits> { + enum { Cost = 5 * NumTraits::MulCost, PacketAccess = false }; +}; + +template +struct safe_div_or_mod_op { + static_assert(std::is_integral::value, "Integer type expected"); + + bool* const error; + + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE safe_div_or_mod_op(bool* error) + : error(error) {} + + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE const T operator()(const T& a, + const T& b) const { + const T safe_b = tensorflow::internal::SubtleMustCopy(b); + if (TF_PREDICT_TRUE(safe_b != 0)) { + return DivOrMod()(a, safe_b); + } else { + *error = true; + return 0; + } + } +}; + +template +struct functor_traits> { + enum { + Cost = functor_traits::Cost + NumTraits::AddCost, + PacketAccess = false, + }; +}; + +template +struct div_no_nan_op { + EIGEN_EMPTY_STRUCT_CTOR(div_no_nan_op) + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE const T operator()(const T& a, + const T& b) const { + if (b != 0) { + return scalar_quotient_op()(a, b); + } else { + return 0; + } + } +}; + +template +struct functor_traits> { + enum { + Cost = functor_traits>::Cost + NumTraits::AddCost, + PacketAccess = false, + }; +}; + +// scalar_left and scalar_right are template helpers to partially +// apply a binary function. +// +// Suppose Binary is a binary functor f(x, y), scalar_left<> is a +// unary functor g_x(y) = f(x, y), where x is provided via the +// constructor. Similarly, scalar_right<> is a unary functor g_y(x) = +// f(x, y). + +template +struct scalar_left : private Binary { + typedef Tout result_type; + const Tin* left; + + EIGEN_DEVICE_FUNC inline scalar_left(const scalar_left& other) = default; + + template + EIGEN_DEVICE_FUNC inline explicit scalar_left(const Tin* c, Args... args) + : Binary(args...), left(c) {} + + EIGEN_DEVICE_FUNC inline Tout operator()(const Tin& right) const { + return Binary::operator()(*left, right); + } + + template + EIGEN_DEVICE_FUNC inline Packet packetOp(const Packet& right_packet) const { + const Packet left_packet = Eigen::internal::pset1(*left); + return Binary::packetOp(left_packet, right_packet); + } +}; + +template +struct functor_traits> { + enum { + Cost = functor_traits::Cost, + PacketAccess = functor_traits::PacketAccess, + }; +}; + +template +struct scalar_right : private Binary { + typedef Tout result_type; + const Tin* right; + + EIGEN_DEVICE_FUNC inline scalar_right(const scalar_right& other) = default; + + template + EIGEN_DEVICE_FUNC inline explicit scalar_right(const Tin* c, Args... args) + : Binary(args...), right(c) {} + + EIGEN_DEVICE_FUNC inline Tout operator()(const Tin& left) const { + return Binary::operator()(left, *right); + } + + template + EIGEN_DEVICE_FUNC inline Packet packetOp(const Packet& left_packet) const { + const Packet right_packet = Eigen::internal::pset1(*right); + return Binary::packetOp(left_packet, right_packet); + } +}; + +template +struct functor_traits> { + enum { + Cost = functor_traits::Cost, + PacketAccess = functor_traits::PacketAccess, + }; +}; + +// similar to std::equal_to, but with the DEVICE_FUNC qualifier +template +struct equal_to : std::binary_function { + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE bool operator()(const T& x, + const T& y) const { + return x == y; + } +}; + +// similar to std::not_equal_to, but with the DEVICE_FUNC qualifier +template +struct not_equal_to : std::binary_function { + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE bool operator()(const T& x, + const T& y) const { + return x != y; + } +}; + +// similar to std::greater, but with the DEVICE_FUNC qualifier +template +struct greater : std::binary_function { + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE bool operator()(const T& x, + const T& y) const { + return x > y; + } +}; + +// similar to std::less, but with the DEVICE_FUNC qualifier +template +struct less : std::binary_function { + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE bool operator()(const T& x, + const T& y) const { + return x < y; + } +}; + +// similar to std::greater_equal, but with the DEVICE_FUNC qualifier +template +struct greater_equal : std::binary_function { + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE bool operator()(const T& x, + const T& y) const { + return x >= y; + } +}; + +// similar to std::less_equal, but with the DEVICE_FUNC qualifier +template +struct less_equal : std::binary_function { + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE bool operator()(const T& x, + const T& y) const { + return x <= y; + } +}; + +// Functor that enables composition of multiple Eigen functors. +template +struct scalar_compose_op { + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE const Scalar + operator()(const Scalar& a, const Scalar& b) const { + return UnaryFunctor()(BinaryFunctor()(a, b)); + } + template + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE const Packet + packetOp(const Packet& a, const Packet& b) const { + return UnaryFunctor().packetOp(BinaryFunctor().packetOp(a, b)); + } +}; + +template +struct functor_traits> { + enum { + Cost = functor_traits::Cost + + functor_traits::Cost, + PacketAccess = functor_traits::PacketAccess && + functor_traits::PacketAccess + }; +}; + +// TODO(b/32239616): This kernel should be moved into Eigen and vectorized. +template +struct google_floor_div { + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE const T operator()(const T& x, + const T& y) const { + if ((x < T(0)) != (y < T(0))) { + T abs_x = std::abs(x); + T abs_y = std::abs(y); + return -(abs_x + abs_y - 1) / abs_y; + } else { + return x / y; + } + } +}; + +template +struct google_floor_div< + T, typename std::enable_if::value>::type> { + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE const T operator()(const T& x, + const T& y) const { + return x / y; + } +}; + +template +struct functor_traits> { + enum { + Cost = 2 * Eigen::internal::scalar_div_cost::value + + 2 * NumTraits::AddCost, + PacketAccess = false + }; +}; + +// TODO(b/32239616): This kernel should be moved into Eigen and vectorized. +template +struct google_floor_div_real { + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE const T operator()(const T& x, + const T& y) const { + return Eigen::numext::floor(x / y); + } +}; + +template +struct functor_traits> { + enum { + Cost = 2 * Eigen::internal::scalar_div_cost::value + + 2 * NumTraits::AddCost, + PacketAccess = false + }; +}; + +// TODO(b//32239616): This kernel should be moved into Eigen and vectorized. +template +struct google_floor_fmod { + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE const T operator()(const T& x, + const T& y) const { + // EIGEN_STATIC_ASSERT(NUMERIC_TYPE_MUST_BE_REAL); + T trunc_mod = std::fmod(x, y); + return (x < T(0)) == (y < T(0)) ? trunc_mod : std::fmod(trunc_mod + y, y); + } +}; + +template +struct functor_traits> { + enum { + Cost = 2 * Eigen::internal::scalar_div_cost::value + + 2 * NumTraits::AddCost, + PacketAccess = false + }; +}; + +// TODO(b/32239616): This kernel should be moved into Eigen and vectorized. +template +struct google_floor_mod { + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE const T operator()(const T& x, + const T& y) const { + // EIGEN_STATIC_ASSERT(!NUMERIC_TYPE_MUST_BE_REAL); + T trunc_mod = x % y; + return (x < T(0)) == (y < T(0)) ? trunc_mod : (trunc_mod + y) % y; + } +}; + +template +struct functor_traits> { + enum { + Cost = 2 * Eigen::internal::scalar_div_cost::value + + 2 * NumTraits::AddCost, + PacketAccess = false + }; +}; + +#if EIGEN_COMP_GNUC && __cplusplus > 199711L +#define DISABLE_FLOAT_EQUALITY_WARNING \ + _Pragma("GCC diagnostic push") \ + _Pragma("GCC diagnostic ignored \"-Wfloat-equal\"") +#define ENABLE_FLOAT_EQUALITY_WARNING _Pragma("GCC diagnostic pop") +#else +#define DISABLE_FLOAT_EQUALITY_WARNING +#define ENABLE_FLOAT_EQUALITY_WARNING +#endif + +template +struct scalar_round_op_google { + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE const Scalar + operator()(const Scalar& x) const { + EIGEN_STATIC_ASSERT((!NumTraits::IsComplex), + NUMERIC_TYPE_MUST_BE_REAL) + + Scalar round_val = Eigen::numext::floor(x); + const Scalar fraction = x - round_val; + if (fraction > Scalar(.5)) { + round_val += Scalar(1.0); + } else if (fraction == Scalar(.5)) { + const Scalar nearest_even_int = + round_val - Scalar(2) * Eigen::numext::floor(Scalar(.5) * x); + bool is_odd = (nearest_even_int == Scalar(1)); + if (is_odd) { + round_val += Scalar(1); + } + } + return round_val; + } +}; + +template +struct functor_traits> { + enum { Cost = 4 * NumTraits::AddCost, PacketAccess = false }; +}; + +#undef ENABLE_FLOAT_EQUALITY_WARNING +#undef DISABLE_FLOAT_EQUALITY_WARNING + +template +struct bitwise_xor_op { + EIGEN_EMPTY_STRUCT_CTOR(bitwise_xor_op) + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE const Scalar + operator()(const Scalar& x, const Scalar& y) const { + return x ^ y; + } + typedef typename Eigen::internal::packet_traits::type Packet; + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE Packet packetOp(const Packet& a, + const Packet& b) const { + return Eigen::internal::pxor(a, b); + } +}; + +template +struct functor_traits> { + enum { Cost = Eigen::NumTraits::AddCost, PacketAccess = true }; +}; + +// TODO(srvasude): Add packet versions of this operation. +template +struct xlogy_op { + EIGEN_EMPTY_STRUCT_CTOR(xlogy_op) + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE const Scalar + operator()(const Scalar& x, const Scalar& y) const { + if (x == Scalar(0.)) { + return Scalar(0.); + } + return x * numext::log(y); + } +}; + +template +struct functor_traits> { + enum { + Cost = (sizeof(Scalar) == 4 ? 40 : 85) + Eigen::NumTraits::MulCost, + PacketAccess = false + }; +}; + +template +// TODO(srvasude): Add packet versions of this operation. +struct xdivy_op { + EIGEN_EMPTY_STRUCT_CTOR(xdivy_op) + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE const Scalar + operator()(const Scalar& x, const Scalar& y) const { + if (x == Scalar(0.)) { + return Scalar(0.); + } + return x / y; + } +}; + +template +struct functor_traits> { + enum { Cost = Eigen::NumTraits::MulCost, PacketAccess = false }; +}; + +} // end namespace internal +} // end namespace Eigen + +namespace tensorflow { +namespace functor { + +//////////////////////////////////////////////////////////////////////////////// +// Helpers +//////////////////////////////////////////////////////////////////////////////// + +// Base template for functors whose input scalar type is T and +// output scalar type is R. +template +struct base { + // func defines operator() and its vectorized version packetOp(). + typedef F func; + + // If true, the functor's corresponding binary op will instantiate + // specialized kernels to perform an optimized broadcast + // operation. Each functor for which this is enabled increases the + // code size, so by default this is disabled for binary functors and + // is enabled on a per-op basis as needed. + static const bool use_bcast_optimization = false; + + // operator() has the signature: + // out_type operator()(in_type in0, in_type in1 ...) + typedef R out_type; + typedef T in_type; + + // TensorFlow provides tensor-ized version of "func". Roughly + // speaking, the tensorflow operation has the signature: + // tout_type op(tin_type in0) + // tout_type op(tin_type in0, tin_type in1) + // tout_type op(tin_type in0, in_type scalar) + typedef typename TTypes::Flat tout_type; + typedef typename TTypes::ConstFlat tin_type; + typedef typename TTypes::ConstScalar tscalar_type; + + // Whether the functor can error out. Currently applies only to integer + // div and mod. + static const bool has_errors = false; +}; + +// For now, we only apply certain speed optimization for +// float/double's broadcast binary op. +template +struct use_bcast_optimization { + static const bool value = false; +}; + +template <> +struct use_bcast_optimization { + static const bool value = true; +}; + +template <> +struct use_bcast_optimization { + static const bool value = true; +}; + +//////////////////////////////////////////////////////////////////////////////// +// Unary functors +//////////////////////////////////////////////////////////////////////////////// + +// abs(x) = |x| +// neg(x) = - x +// inverse(x) = 1 / x +// square(x) = x^2 +// sqrt(x) = x^(1/2) +// rsqrt(x) = x^(-1/2) +// exp(x) = e^x +// expm1(x) = e^x - 1 +// log(x) = natural logarithm of x +// log1p(x) = natural logarithm of 1 + x +// tanh = (exp(x) - exp(-x)) / (exp(x) + exp(-x)) +// sigmoid = 1 / (1 + exp(-x)) // a.k.a, logistic +// +// NOTE: We may eventually implement common functions used in NN +// here. E.g., rectifier, softplus, derivatives of tanh, sigmod, etc. +// For reference, see speech/lstm/eigen_functors.h. + +template +struct abs : base, + typename Eigen::internal::scalar_abs_op::result_type> {}; + +template +struct neg : base> {}; + +template +struct inverse : base> {}; + +template +struct square : base> {}; + +template +struct sqrt : base> {}; + +template +struct rsqrt : base> {}; + +template +struct exp : base> {}; + +template +struct expm1 : base> {}; + +template +struct log : base> {}; + +template +struct log1p : base> {}; + +template +struct sign : base> {}; + +template +struct sinh : base> {}; + +template +struct cosh : base> {}; + +template +struct tanh : base> {}; + +template +struct asinh : base> {}; + +template +struct acosh : base> {}; + +template +struct atanh : base> {}; + +template +struct lgamma : base> {}; + +template +struct digamma : base> {}; + +template +struct erf : base> {}; + +template +struct erfc : base> {}; + +template +struct sigmoid : base> {}; + +template +struct sin : base> {}; + +template +struct cos : base> {}; + +template +struct tan : base> {}; + +template +struct asin : base> {}; + +template +struct acos : base> {}; + +template +struct atan : base> {}; + +template +struct bessel_i0e : base> {}; + +template +struct bessel_i1e : base> {}; + +struct logical_not : base> { +}; + +// Flip all bits. Named invert to be consistent with numpy. +template +struct invert_op { + EIGEN_EMPTY_STRUCT_CTOR(invert_op) + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE const T operator()(const T& a) const { + return ~a; + } +}; + +template +struct invert : base> {}; + +// NOTE: std::isinf, std::isnan, std::isfinite are plain function. +// Therefore we need to wrap them in functors to be used with Eigen's +// type system. +template +struct isinf : base, bool> {}; + +template +struct isnan : base, bool> {}; + +template +struct isfinite : base, bool> {}; + +template +struct floor : base> {}; + +template +struct round : base> {}; + +template +struct ceil : base> {}; + +/** this should go in Eigen + * \brief Template functor to compute the round to int value of a scalar + */ +template +struct scalar_rint_op { + EIGEN_EMPTY_STRUCT_CTOR(scalar_rint_op) + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE const Scalar + operator()(const Scalar& a) const { +#if defined(__CUDACC__) + return ::rint(a); +#elif defined(__ANDROID__) + return rint(a); +#else + return std::rint(a); +#endif + } +}; + +template +struct rint : base> {}; + +//////////////////////////////////////////////////////////////////////////////// +// Binary functors +//////////////////////////////////////////////////////////////////////////////// + +// Binary functors: +// +// add(x, y) = x + y +// sub(x, y) = x - y +// mul(x, y) = x * y +// div(x, y) = x / y +// mod(x, y) = x % y (int32 and int64 only) +// fmod(x, y) = fmod(x, y) (float and double only) +// pow(x, y) = x ^ y +// maximum(x, y) = x > y ? x : y +// minimum(x, y) = x < y ? x : y +// squared_difference(x, y) = (x - y) * (x - y) + +template +struct add : base> { + static const bool use_bcast_optimization = true; +}; + +template +struct sub : base> { + static const bool use_bcast_optimization = true; +}; + +template +struct mul : base> { + static const bool use_bcast_optimization = true; +}; + +template +struct div : base> {}; + +template +struct safe_div : base>> { + static const bool has_errors = true; +}; + +template +struct div_no_nan : base> {}; + +template +struct fmod : base> {}; + +template +struct mod : base> {}; + +template +struct safe_mod : base>> { + static const bool has_errors = true; +}; + +template +struct floor_fmod : base> {}; + +template +struct safe_floor_mod : base>> { + static const bool has_errors = true; +}; + +template +struct floor_div : base> {}; + +template +struct safe_floor_div : base>> { + static const bool has_errors = true; +}; + +template +struct floor_div_real : base> {}; + +template +struct pow : base> {}; + +template +struct safe_pow : base> { + static const bool has_errors = true; +}; + +template +struct maximum : base> {}; + +template +struct minimum : base> {}; + +template +struct igamma : base> {}; + +template +struct random_gamma_grad + : base> {}; + +template +struct igammac : base> {}; + +template +struct zeta : base> {}; + +template +struct polygamma : base> {}; + +template +struct scalar_atan2_op { + EIGEN_EMPTY_STRUCT_CTOR(scalar_atan2_op) + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE Scalar + operator()(const Scalar& y, const Scalar& x) const { +#if GOOGLE_CUDA + return ::atan2(y, x); +#else + return std::atan2(y, x); +#endif + } +}; + +template +struct atan2 : base> {}; + +template +struct squared_difference + : base, + Eigen::internal::scalar_difference_op>> {}; + +template +struct xdivy : base> {}; + +template +struct xlogy : base> {}; + +template +struct less : base, bool> {}; + +template +struct less_equal : base, bool> {}; + +template +struct greater : base, bool> {}; + +template +struct greater_equal : base, bool> {}; + +template +struct equal_to : base, bool> {}; + +template +struct not_equal_to : base, bool> {}; + +struct logical_and : base {}; + +struct logical_or : base {}; + +template +struct bitwise_and_op { + EIGEN_EMPTY_STRUCT_CTOR(bitwise_and_op) + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE const T operator()(const T& x, + const T& y) const { + return x & y; + } +}; + +template +struct bitwise_or_op { + EIGEN_EMPTY_STRUCT_CTOR(bitwise_or_op) + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE const T operator()(const T& x, + const T& y) const { + return x | y; + } +}; + +template +struct bitwise_and : base> {}; + +template +struct bitwise_or : base> {}; + +template +struct bitwise_xor : base> {}; + +template +struct left_shift_op { + EIGEN_EMPTY_STRUCT_CTOR(left_shift_op) + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE const T operator()(const T& x, + const T& y) const { + // Avoids UB: don't shift by larger than the bitwidth of T, and + // performs left shifts as unsigned shifts. + T y_clamped = y; + if (y_clamped < 0) { + y_clamped = 0; + } else if (y_clamped > sizeof(T) * CHAR_BIT - 1) { + y_clamped = sizeof(T) * CHAR_BIT - 1; + } + using U = typename std::make_unsigned::type; + return static_cast(static_cast(x) << static_cast(y_clamped)); + } +}; + +template +struct right_shift_op { + EIGEN_EMPTY_STRUCT_CTOR(right_shift_op) + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE const T operator()(const T& x, + const T& y) const { + // Avoids UB: don't shift by larger than the bitwidth of T. + T y_clamped = y; + if (y_clamped < 0) { + y_clamped = 0; + } else if (y_clamped > sizeof(T) * CHAR_BIT - 1) { + y_clamped = sizeof(T) * CHAR_BIT - 1; + } + // Technically right shifts of signed integers are not necessarily + // arithmetic shifts according to the C++ standard. However in practice most + // implementations are arithmetic shifts. If this proves to be a problem in + // practice, we may need to use an alternative implementation. + return x >> y_clamped; + } +}; + +template +struct left_shift : base> {}; + +template +struct right_shift : base> {}; + +template +struct make_complex_func { + typedef std::complex result_type; + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE result_type operator()(T real, + T imag) const { + return std::complex(real, imag); + } +}; + +template +struct make_complex : base, std::complex> {}; + +template +struct get_real + : base, typename T::value_type> {}; + +template +struct get_imag + : base, typename T::value_type> {}; + +template +struct get_angle + : base, typename T::value_type> {}; + +template +struct conj : base> {}; + +//////////////////////////////////////////////////////////////////////////////// +// Functors takes 1 or 2 tensors, computes the base functor on +// coefficient of the input tensors and puts the results in the output +// tensor. +//////////////////////////////////////////////////////////////////////////////// +template +struct UnaryFunctor { + // Computes on device "d": out[i] = Functor(in[i]) + void operator()(const Device& d, typename Functor::tout_type out, + typename Functor::tin_type in); +}; + +template +struct BinaryFunctor { + // Computes on device "d": out[i] = Functor(in0[i], in1[i]) + void operator()(const Device& d, typename Functor::tout_type out, + typename Functor::tin_type in0, + typename Functor::tin_type in1, bool* error); + + // Computes on device "d": out[i] = Functor(scalar[0], in[i]) + void Left(const Device& d, typename Functor::tout_type out, + typename Functor::tscalar_type scalar, + typename Functor::tin_type in, bool* error); + + // Computes on device "d": out[i] = Functor(in[i], scalar[0]) + void Right(const Device& d, typename Functor::tout_type out, + typename Functor::tin_type in, + typename Functor::tscalar_type scalar, bool* error); + + // Computes on device "d": + // out = Functor(in0.broadcast(bcast0), in1.broadcast(bcast1)) + // + // TODO(zhifengc): makes BCast a template member function on NDIMS + // instead making BinaryFunctor templates on NDIMS. + void BCast(const Device& d, + typename TTypes::Tensor out, + typename TTypes::ConstTensor in0, + typename Eigen::array bcast0, + typename TTypes::ConstTensor in1, + typename Eigen::array bcast1, + bool* error); +}; + +template +struct ApproximateEqual { + void operator()(const Device& d, typename TTypes::ConstFlat x, + typename TTypes::ConstFlat y, T tolerance, + typename TTypes::Flat z); +}; + +template +bool AllOne(const typename Eigen::array& a) { + for (size_t i = 0; i < a.size(); ++i) { + if (a[i] != 1) return false; + } + return true; +} + +template +struct SelectFunctor { + void operator()(const Device& d, typename TTypes::Flat out, + typename TTypes::ConstFlat cond_flat, + typename TTypes::ConstFlat then_flat, + typename TTypes::ConstFlat else_flat); +}; + +template +struct SelectScalarFunctor { + void operator()(const Device& d, typename TTypes::Flat out, + typename TTypes::ConstScalar cond, + typename TTypes::ConstFlat then_flat, + typename TTypes::ConstFlat else_flat); +}; + +template +struct BatchSelectFunctor { + void operator()(const Device& d, + typename TTypes::Matrix output_flat_outer_dims, + TTypes::ConstVec cond_vec, + typename TTypes::ConstMatrix then_flat_outer_dims, + typename TTypes::ConstMatrix else_flat_outer_dims); +}; + +} // end namespace functor +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_CWISE_OPS_H_ diff --git a/linux/include/tensorflow/core/kernels/cwise_ops_common.h b/linux/include/tensorflow/core/kernels/cwise_ops_common.h new file mode 100644 index 000000000..f77d7238a --- /dev/null +++ b/linux/include/tensorflow/core/kernels/cwise_ops_common.h @@ -0,0 +1,605 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_CWISE_OPS_COMMON_H_ +#define TENSORFLOW_CORE_KERNELS_CWISE_OPS_COMMON_H_ + +// See docs in ../ops/math_ops.cc. + +#define EIGEN_USE_THREADS + +#include "tensorflow/core/lib/bfloat16/bfloat16.h" + +#ifdef TENSORFLOW_USE_SYCL +#include "tensorflow/core/kernels/cwise_ops_sycl_common.h" +#endif + +#include "tensorflow/core/kernels/cwise_ops.h" +#include "tensorflow/core/kernels/cwise_ops_gradients.h" + +#include "tensorflow/core/framework/op.h" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/framework/variant_op_registry.h" +#include "tensorflow/core/platform/logging.h" +#include "tensorflow/core/util/bcast.h" + +namespace tensorflow { + +typedef Eigen::ThreadPoolDevice CPUDevice; +typedef Eigen::GpuDevice GPUDevice; +#ifdef TENSORFLOW_USE_SYCL +typedef Eigen::SyclDevice SYCLDevice; +#endif + +class BinaryOpShared : public OpKernel { + public: + explicit BinaryOpShared(OpKernelConstruction* ctx, DataType out, DataType in); + + protected: + struct BinaryOpState { + // Sets up bcast with the shape of in0 and in1, ensures that the bcast + // is valid, and if so, set out, either by allocating a new buffer using + // ctx->output(...) or by creating an alias for an owned input buffer for + // in-place computation. + // Caller must check ctx->status() upon return for non-ok status. + // If ctx->status().ok() is true, then out is guaranteed to be allocated. + BinaryOpState(OpKernelContext* ctx); + + const Tensor& in0; + const Tensor& in1; + + BCast bcast; + Tensor* out = nullptr; + int64 out_num_elements; + + int64 in0_num_elements; + int64 in1_num_elements; + + int ndims; + }; + + void SetUnimplementedError(OpKernelContext* ctx); + void SetComputeError(OpKernelContext* ctx); +}; + +// Coefficient-wise binary operations: +// Device: E.g., CPUDevice, GPUDevice. +// Functor: defined in cwise_ops.h. E.g., functor::add. +template +class BinaryOp : public BinaryOpShared { + public: + typedef typename Functor::in_type Tin; // Input scalar data type. + typedef typename Functor::out_type Tout; // Output scalar data type. + + explicit BinaryOp(OpKernelConstruction* ctx) + : BinaryOpShared(ctx, DataTypeToEnum::v(), + DataTypeToEnum::v()) {} + + void Compute(OpKernelContext* ctx) override { + // 'state': Shared helper not dependent on T to reduce code size + BinaryOpState state(ctx); + if (!ctx->status().ok()) return; + Tensor* out = state.out; + BCast* bcast = &state.bcast; + auto& in0 = state.in0; + auto& in1 = state.in1; + if (state.out_num_elements == 0) { + return; + } + const int ndims = state.ndims; + const Device& eigen_device = ctx->eigen_device(); + bool error = false; + bool* const error_ptr = Functor::has_errors ? &error : nullptr; + if (ndims <= 1) { + auto out_flat = out->flat(); + if (state.in1_num_elements == 1) { + // tensor op scalar + functor::BinaryFunctor().Right( + eigen_device, out_flat, in0.template flat(), + in1.template scalar(), error_ptr); + } else if (state.in0_num_elements == 1) { + // scalar op tensor + functor::BinaryFunctor().Left( + eigen_device, out_flat, in0.template scalar(), + in1.template flat(), error_ptr); + } else { + functor::BinaryFunctor()( + eigen_device, out_flat, in0.template flat(), + in1.template flat(), error_ptr); + } + } else if (ndims == 2) { + functor::BinaryFunctor().BCast( + eigen_device, out->shaped(bcast->result_shape()), + in0.template shaped(bcast->x_reshape()), + BCast::ToIndexArray<2>(bcast->x_bcast()), + in1.template shaped(bcast->y_reshape()), + BCast::ToIndexArray<2>(bcast->y_bcast()), error_ptr); + } else if (ndims == 3) { + functor::BinaryFunctor().BCast( + eigen_device, out->shaped(bcast->result_shape()), + in0.template shaped(bcast->x_reshape()), + BCast::ToIndexArray<3>(bcast->x_bcast()), + in1.template shaped(bcast->y_reshape()), + BCast::ToIndexArray<3>(bcast->y_bcast()), error_ptr); + } else if (ndims == 4) { + functor::BinaryFunctor().BCast( + eigen_device, out->shaped(bcast->result_shape()), + in0.template shaped(bcast->x_reshape()), + BCast::ToIndexArray<4>(bcast->x_bcast()), + in1.template shaped(bcast->y_reshape()), + BCast::ToIndexArray<4>(bcast->y_bcast()), error_ptr); + } else if (ndims == 5) { + functor::BinaryFunctor().BCast( + eigen_device, out->shaped(bcast->result_shape()), + in0.template shaped(bcast->x_reshape()), + BCast::ToIndexArray<5>(bcast->x_bcast()), + in1.template shaped(bcast->y_reshape()), + BCast::ToIndexArray<5>(bcast->y_bcast()), error_ptr); + } else { + SetUnimplementedError(ctx); + } + if (Functor::has_errors && error) { + SetComputeError(ctx); + } + } +}; + +template +class ApproximateEqualOp : public OpKernel { + public: + explicit ApproximateEqualOp(OpKernelConstruction* context) + : OpKernel(context) { + float tolerance; + OP_REQUIRES_OK(context, context->GetAttr("tolerance", &tolerance)); + tolerance_ = T(tolerance); + } + void Compute(OpKernelContext* context) override { + const Tensor& x_input = context->input(0); + const Tensor& y_input = context->input(1); + OP_REQUIRES( + context, x_input.shape() == y_input.shape(), + errors::InvalidArgument("x and y must be of the same shape. ", + "x shape: ", x_input.shape().DebugString(), + ". y shape: ", y_input.shape().DebugString())); + Tensor* z_output = nullptr; + OP_REQUIRES_OK(context, + context->allocate_output(0, x_input.shape(), &z_output)); + const Device& d = context->eigen_device(); + typename TTypes::ConstFlat x(x_input.flat()); + typename TTypes::ConstFlat y(y_input.flat()); + typename TTypes::Flat z(z_output->flat()); + functor::ApproximateEqual()(d, x, y, tolerance_, z); + } + + private: + T tolerance_; +}; + +// Basic coefficient-wise binary operations that are known to not require +// any broadcasting. This is the case for example of the gradients of +// unary operations. +// Device: E.g., CPUDevice, GPUDevice. +// Functor: defined above. E.g., functor::tanh_grad. +template +class SimpleBinaryOp : public OpKernel { + public: + typedef typename Functor::in_type Tin; // Input scalar data type. + typedef typename Functor::out_type Tout; // Output scalar data type. + + explicit SimpleBinaryOp(OpKernelConstruction* ctx) : OpKernel(ctx) {} + + void Compute(OpKernelContext* ctx) override { + const Tensor& in0 = ctx->input(0); + const Tensor& in1 = ctx->input(1); + auto in0_flat = in0.flat(); + auto in1_flat = in1.flat(); + const Device& eigen_device = ctx->eigen_device(); + + Tensor* out = nullptr; + if (std::is_same::value) { + OP_REQUIRES_OK(ctx, ctx->forward_input_or_allocate_output( + {0, 1}, 0, in0.shape(), &out)); + } else { + OP_REQUIRES_OK(ctx, ctx->allocate_output(0, in0.shape(), &out)); + } + auto out_flat = out->flat(); + functor::SimpleBinaryFunctor()(eigen_device, out_flat, + in0_flat, in1_flat); + } +}; + +// Coefficient-wise unary operations: +// Device: E.g., CPUDevice, GPUDevice. +// Functor: defined in cwise_ops.h. E.g., functor::sqrt. +template +class UnaryOp : public OpKernel { + public: + typedef typename Functor::in_type Tin; // Input scalar data type. + typedef typename Functor::out_type Tout; // Output scalar data type. + // Tin may be different from Tout. E.g., abs: complex64 -> float + + explicit UnaryOp(OpKernelConstruction* ctx) : OpKernel(ctx) { + auto in = DataTypeToEnum::v(); + auto out = DataTypeToEnum::v(); + OP_REQUIRES_OK(ctx, ctx->MatchSignature({in}, {out})); + } + + void Compute(OpKernelContext* ctx) override { + const Tensor& inp = ctx->input(0); + Tensor* out = nullptr; + if (std::is_same::value) { + OP_REQUIRES_OK(ctx, ctx->forward_input_or_allocate_output( + {0}, 0, inp.shape(), &out)); + } else { + OP_REQUIRES_OK(ctx, ctx->allocate_output(0, inp.shape(), &out)); + } + functor::UnaryFunctor()( + ctx->eigen_device(), out->flat(), inp.flat()); + } +}; + +template +class UnaryVariantOp : public OpKernel { + public: + explicit UnaryVariantOp(OpKernelConstruction* ctx) : OpKernel(ctx) {} + + void Compute(OpKernelContext* ctx) override { + const Tensor& inp = ctx->input(0); + OP_REQUIRES( + ctx, TensorShapeUtils::IsScalar(inp.shape()), + errors::InvalidArgument("Non-scalar variants are not supported.")); + const Variant& v = inp.scalar()(); + Variant v_out; + OP_REQUIRES_OK(ctx, UnaryOpVariant(ctx, OpEnum, v, &v_out)); + Tensor out(cpu_allocator(), DT_VARIANT, TensorShape()); + out.scalar()() = std::move(v_out); + ctx->set_output(0, std::move(out)); + } +}; + +namespace functor { + +template +void Assign(const D& d, Out out, Rhs rhs) { + out.device(d) = rhs; +} + +// Partial specialization of BinaryFunctor +// for functors with with no error checking. +template +struct BinaryFunctor { + void operator()(const CPUDevice& d, typename Functor::tout_type out, + typename Functor::tin_type in0, + typename Functor::tin_type in1, bool* error) { + Assign(d, out, in0.binaryExpr(in1, typename Functor::func())); + } + + void Left(const CPUDevice& d, typename Functor::tout_type out, + typename Functor::tscalar_type scalar, + typename Functor::tin_type in, bool* error) { + typedef typename Functor::out_type Tout; + typedef typename Functor::in_type Tin; + typedef typename Functor::func Binary; + typedef typename Eigen::internal::scalar_left Unary; + Assign(d, out, in.unaryExpr(Unary(scalar.data()))); + } + + void Right(const CPUDevice& d, typename Functor::tout_type out, + typename Functor::tin_type in, + typename Functor::tscalar_type scalar, bool* error) { + typedef typename Functor::out_type Tout; + typedef typename Functor::in_type Tin; + typedef typename Functor::func Binary; + typedef typename Eigen::internal::scalar_right Unary; + Assign(d, out, in.unaryExpr(Unary(scalar.data()))); + } + + void BCast(const CPUDevice& dev, + typename TTypes::Tensor out, + typename TTypes::ConstTensor in0, + typename Eigen::array bcast0, + typename TTypes::ConstTensor in1, + typename Eigen::array bcast1, + bool* error) { + typename Functor::func func; + if (AllOne(bcast0) && AllOne(bcast1)) { + Assign(dev, out, in0.binaryExpr(in1, func)); + } else if (AllOne(bcast0)) { + auto rhs = in1.broadcast(bcast1); + Assign(dev, out, in0.binaryExpr(rhs, func)); + } else if (AllOne(bcast1)) { + auto lhs = in0.broadcast(bcast0); + Assign(dev, out, lhs.binaryExpr(in1, func)); + } else { + auto lhs = in0.broadcast(bcast0); + auto rhs = in1.broadcast(bcast1); + Assign(dev, out, lhs.binaryExpr(rhs, func)); + } + } +}; + +// Partial specialization of BinaryFunctor +// for functors with with no error checking. +template +struct BinaryFunctor { + enum { NDIMS = 2 }; + + void operator()(const CPUDevice& d, typename Functor::tout_type out, + typename Functor::tin_type in0, + typename Functor::tin_type in1, bool* error) { + Assign(d, out, in0.binaryExpr(in1, typename Functor::func())); + } + + void Left(const CPUDevice& d, typename Functor::tout_type out, + typename Functor::tscalar_type scalar, + typename Functor::tin_type in, bool* error) { + typedef typename Functor::out_type Tout; + typedef typename Functor::in_type Tin; + typedef typename Functor::func Binary; + typedef typename Eigen::internal::scalar_left Unary; + Assign(d, out, in.unaryExpr(Unary(scalar.data()))); + } + + void Right(const CPUDevice& d, typename Functor::tout_type out, + typename Functor::tin_type in, + typename Functor::tscalar_type scalar, bool* error) { + typedef typename Functor::out_type Tout; + typedef typename Functor::in_type Tin; + typedef typename Functor::func Binary; + typedef typename Eigen::internal::scalar_right Unary; + Assign(d, out, in.unaryExpr(Unary(scalar.data()))); + } + +#if !defined(EIGEN_HAS_INDEX_LIST) + inline Eigen::DSizes NByOne(int n) { + return Eigen::DSizes(n, 1); + } + inline Eigen::DSizes OneByM(int m) { + return Eigen::DSizes(1, m); + } +#else + inline Eigen::IndexList> NByOne(int n) { + Eigen::IndexList> ret; + ret.set(0, n); + return ret; + } + inline Eigen::IndexList, int> OneByM(int m) { + Eigen::IndexList, int> ret; + ret.set(1, m); + return ret; + } +#endif + + void BCast(const CPUDevice& dev, + typename TTypes::Tensor out, + typename TTypes::ConstTensor in0, + typename Eigen::array bcast0, + typename TTypes::ConstTensor in1, + typename Eigen::array bcast1, + bool* error) { + typedef typename Functor::in_type T; + typename Functor::func func; + if (Functor::use_bcast_optimization && use_bcast_optimization::value) { + // Optimize for speed by using Eigen::type2index and avoid + // .broadcast() when we know its a no-op. + // + // Here, we need to handle 6 cases depending on how many "1" + // exist in in0 and in1's shapes (4 numbers in total). It's not + // possible that two shapes have more than 2 1s because those + // are simplified to NDIMS==1 case. + // + // Because this optimization increases the binary size for each + // Functor (+, -, *, /, <, <=, etc.), type and ndim combination. + // we only apply such optimization for selected ops/types/ndims. + // + // Because NDIMS, Functor::use_broadcast_optimization and + // use_broadcast_optimization are compile-time constant, gcc + // does a decent job avoiding generating code when conditions + // are not met. + const int a = in0.dimension(0); // in0 is shape [a, b] + const int b = in0.dimension(1); + const int c = in1.dimension(0); // in1 is shape [c, d] + const int d = in1.dimension(1); + if ((a == 1) && (d == 1)) { + auto lhs = in0.reshape(OneByM(b)).broadcast(NByOne(c)); + auto rhs = in1.reshape(NByOne(c)).broadcast(OneByM(b)); + Assign(dev, out, lhs.binaryExpr(rhs, func)); + return; + } + if ((b == 1) && (c == 1)) { + auto lhs = in0.reshape(NByOne(a)).broadcast(OneByM(d)); + auto rhs = in1.reshape(OneByM(d)).broadcast(NByOne(a)); + Assign(dev, out, lhs.binaryExpr(rhs, func)); + return; + } + if (a == 1) { + auto lhs = in0.reshape(OneByM(b)).broadcast(NByOne(c)); + auto rhs = in1; + Assign(dev, out, lhs.binaryExpr(rhs, func)); + return; + } + if (b == 1) { + auto lhs = in0.reshape(NByOne(a)).broadcast(OneByM(d)); + auto rhs = in1; + Assign(dev, out, lhs.binaryExpr(rhs, func)); + return; + } + if (c == 1) { + auto lhs = in0; + auto rhs = in1.reshape(OneByM(d)).broadcast(NByOne(a)); + Assign(dev, out, lhs.binaryExpr(rhs, func)); + return; + } + if (d == 1) { + auto lhs = in0; + auto rhs = in1.reshape(NByOne(c)).broadcast(OneByM(b)); + Assign(dev, out, lhs.binaryExpr(rhs, func)); + return; + } + + const bool bcast0_all_one = AllOne(bcast0); + const bool bcast1_all_one = AllOne(bcast1); + if (bcast0_all_one && !bcast1_all_one) { + auto lhs = in0; // No need to do broadcast for in0 + auto rhs = in1.broadcast(bcast1); + Assign(dev, out, lhs.binaryExpr(rhs, func)); + return; + } + + if (!bcast0_all_one && bcast1_all_one) { + auto lhs = in0.broadcast(bcast0); + auto rhs = in1; // No need to do broadcast for in1 + Assign(dev, out, lhs.binaryExpr(rhs, func)); + return; + } + } + + // Fallback path. Always works and probably slower. + auto lhs = in0.broadcast(bcast0); + auto rhs = in1.broadcast(bcast1); + Assign(dev, out, lhs.binaryExpr(rhs, func)); + } +}; + +// Version of BinaryFunctor with error handling. +template +struct BinaryFunctor { + void operator()(const CPUDevice& d, typename Functor::tout_type out, + typename Functor::tin_type in0, + typename Functor::tin_type in1, bool* error) { + Assign(d, out, in0.binaryExpr(in1, typename Functor::func(error))); + } + + void Left(const CPUDevice& d, typename Functor::tout_type out, + typename Functor::tscalar_type scalar, + typename Functor::tin_type in, bool* error) { + typedef typename Functor::out_type Tout; + typedef typename Functor::in_type Tin; + typedef typename Functor::func Binary; + typedef typename Eigen::internal::scalar_left Unary; + Assign(d, out, in.unaryExpr(Unary(scalar.data(), error))); + } + + void Right(const CPUDevice& d, typename Functor::tout_type out, + typename Functor::tin_type in, + typename Functor::tscalar_type scalar, bool* error) { + typedef typename Functor::out_type Tout; + typedef typename Functor::in_type Tin; + typedef typename Functor::func Binary; + typedef typename Eigen::internal::scalar_right Unary; + Assign(d, out, in.unaryExpr(Unary(scalar.data(), error))); + } + + void BCast(const CPUDevice& dev, + typename TTypes::Tensor out, + typename TTypes::ConstTensor in0, + typename Eigen::array bcast0, + typename TTypes::ConstTensor in1, + typename Eigen::array bcast1, + bool* error) { + typename Functor::func func(error); + auto lhs = in0.broadcast(bcast0); + auto rhs = in1.broadcast(bcast1); + Assign(dev, out, lhs.binaryExpr(rhs, func)); + } +}; + +// Partial specialization of UnaryFunctor. +template +struct UnaryFunctor { + void operator()(const CPUDevice& d, typename Functor::tout_type out, + typename Functor::tin_type in) { + Assign(d, out, in.unaryExpr(typename Functor::func())); + } +}; + +// Partial specialization of ApproximateEqual. +template +struct ApproximateEqual { + void operator()(const CPUDevice& d, typename TTypes::ConstFlat x, + typename TTypes::ConstFlat y, T tolerance, + typename TTypes::Flat z) { + auto diff = x - y; + z.device(d) = diff.abs() <= tolerance; + } +}; + +} // end namespace functor + +#define REGISTER(OP, D, N, F, T) \ + REGISTER_KERNEL_BUILDER(Name(N).Device(DEVICE_##D).TypeConstraint("T"), \ + OP>); + +#define REGISTER_VARIANT(OP, D, N, ENUM) \ + REGISTER_KERNEL_BUILDER( \ + Name(N).Device(DEVICE_##D).TypeConstraint("T"), \ + OP); + +// Macros to register kernels for multiple types (T0, T1, etc.) on +// device type "D" (CPU or GPU) for operation "N" (e.g., sqrt) using +// the functor "F" (e.g., functor::sqrt). + +#if defined(__ANDROID_TYPES_SLIM__) +// Note that __ANDROID_TYPES_SLIM__ is also checked in the cwise_ops*.cc files. +// Normally Android TensorFlow is built with a reduced number of types (float). +// Override on the command-line using "--copt=-D__ANDROID_TYPES_FULL__" +// to generate a library with full type support with a consequent increase in +// code size. +#define REGISTER2(OP, D, N, F, T0, T1) REGISTER(OP, D, N, F, T0) +#define REGISTER3(OP, D, N, F, T0, T1, T2) REGISTER(OP, D, N, F, T0) +#define REGISTER4(OP, D, N, F, T0, T1, T2, T3) REGISTER(OP, D, N, F, T0) +#define REGISTER5(OP, D, N, F, T0, T1, T2, T3, T4) REGISTER(OP, D, N, F, T0) +#define REGISTER6(OP, D, N, F, T0, T1, T2, T3, T4, T5) REGISTER(OP, D, N, F, T0) +#define REGISTER7(OP, D, N, F, T0, T1, T2, T3, T4, T5, T6) \ + REGISTER(OP, D, N, F, T0) +#define REGISTER8(OP, D, N, F, T0, T1, T2, T3, T4, T5, T6, T7) \ + REGISTER(OP, D, N, F, T0) +#define REGISTER9(OP, D, N, F, T0, T1, T2, T3, T4, T5, T6, T7, T8) \ + REGISTER(OP, D, N, F, T0) +#else // !defined(__ANDROID_TYPES_SLIM__) +#define REGISTER2(OP, D, N, F, T0, T1) \ + REGISTER(OP, D, N, F, T0) \ + REGISTER(OP, D, N, F, T1) +#define REGISTER3(OP, D, N, F, T0, T1, T2) \ + REGISTER2(OP, D, N, F, T0, T1) \ + REGISTER(OP, D, N, F, T2) +#define REGISTER4(OP, D, N, F, T0, T1, T2, T3) \ + REGISTER2(OP, D, N, F, T0, T1) \ + REGISTER2(OP, D, N, F, T2, T3) +#define REGISTER5(OP, D, N, F, T0, T1, T2, T3, T4) \ + REGISTER3(OP, D, N, F, T0, T1, T2) \ + REGISTER2(OP, D, N, F, T3, T4) +#define REGISTER6(OP, D, N, F, T0, T1, T2, T3, T4, T5) \ + REGISTER3(OP, D, N, F, T0, T1, T2) \ + REGISTER3(OP, D, N, F, T3, T4, T5) +#define REGISTER7(OP, D, N, F, T0, T1, T2, T3, T4, T5, T6) \ + REGISTER4(OP, D, N, F, T0, T1, T2, T3) \ + REGISTER3(OP, D, N, F, T4, T5, T6) +#define REGISTER8(OP, D, N, F, T0, T1, T2, T3, T4, T5, T6, T7) \ + REGISTER4(OP, D, N, F, T0, T1, T2, T3) \ + REGISTER4(OP, D, N, F, T4, T5, T6, T7) +#define REGISTER9(OP, D, N, F, T0, T1, T2, T3, T4, T5, T6, T7, T8) \ + REGISTER5(OP, D, N, F, T0, T1, T2, T3, T4) \ + REGISTER4(OP, D, N, F, T5, T6, T7, T8) + +// Instead of adding REGISTER10, etc., shard the .cc files - see +// cwise_op_equal_to_*.cc for an example. + +#endif // defined(__ANDROID_TYPES_SLIM__) + +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_CWISE_OPS_COMMON_H_ diff --git a/linux/include/tensorflow/core/kernels/cwise_ops_gradients.h b/linux/include/tensorflow/core/kernels/cwise_ops_gradients.h new file mode 100644 index 000000000..53b53cc27 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/cwise_ops_gradients.h @@ -0,0 +1,211 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_CWISE_OPS_GRADIENTS_H_ +#define TENSORFLOW_CORE_KERNELS_CWISE_OPS_GRADIENTS_H_ + +#define EIGEN_USE_THREADS +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/kernels/cwise_ops.h" + +namespace Eigen { +namespace internal { + +// Gradient for the tanh function +template +struct scalar_tanh_gradient_op { + EIGEN_EMPTY_STRUCT_CTOR(scalar_tanh_gradient_op) + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE const T + operator()(const T& output, const T& output_gradient) const { + return output_gradient * (T(1) - output * output); + } + template + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE const Packet + packetOp(const Packet& output, const Packet& output_gradient) const { + return pmul(output_gradient, + psub(pset1(T(1)), pmul(output, output))); + } +}; +template +struct functor_traits> { + enum { + Cost = NumTraits::AddCost + 2 * NumTraits::MulCost, + PacketAccess = packet_traits::HasSub && packet_traits::HasMul, + }; +}; + +// Gradient for the sigmoid function +template +struct scalar_sigmoid_gradient_op { + EIGEN_EMPTY_STRUCT_CTOR(scalar_sigmoid_gradient_op) + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE const T + operator()(const T& output, const T& output_gradient) const { + return output_gradient * output * (T(1) - output); + } + template + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE const Packet + packetOp(const Packet& output, const Packet& output_gradient) const { + return pmul(output_gradient, + pmul(output, psub(pset1(T(1)), output))); + } +}; +template +struct functor_traits> { + enum { + Cost = NumTraits::AddCost + 2 * NumTraits::MulCost, + PacketAccess = packet_traits::HasSub && packet_traits::HasMul, + }; +}; + +// Gradient for the inverse function +template +struct scalar_inverse_gradient_op { + EIGEN_EMPTY_STRUCT_CTOR(scalar_inverse_gradient_op) + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE const T + operator()(const T& output, const T& output_gradient) const { + const T out_conj = numext::conj(output); + return -output_gradient * out_conj * out_conj; + } + template + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE const Packet + packetOp(const Packet& output, const Packet& output_gradient) const { + const Packet out_conj = pconj(output); + return pnegate(pmul(output_gradient, pmul(out_conj, out_conj))); + } +}; +template +struct functor_traits> { + enum { + Cost = NumTraits::AddCost + 2 * NumTraits::MulCost, + PacketAccess = packet_traits::HasMul, + }; +}; + +// Gradient for the sqrt function +template +struct scalar_sqrt_gradient_op { + EIGEN_EMPTY_STRUCT_CTOR(scalar_sqrt_gradient_op) + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE const T + operator()(const T& output, const T& output_gradient) const { + const T out_conj = numext::conj(output); + return static_cast(0.5) * output_gradient / out_conj; + } + template + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE const Packet + packetOp(const Packet& output, const Packet& output_gradient) const { + const Packet const_half = pset1(static_cast(0.5)); + const Packet out_conj = pconj(output); + return pdiv(pmul(const_half, output_gradient), out_conj); + } +}; +template +struct functor_traits> { + enum { + PacketAccess = packet_traits::HasMul & packet_traits::HasDiv, + Cost = NumTraits::MulCost + scalar_div_cost::value, + }; +}; + +// Gradient for the rsqrt function +template +struct scalar_rsqrt_gradient_op { + EIGEN_EMPTY_STRUCT_CTOR(scalar_rsqrt_gradient_op) + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE const T + operator()(const T& output, const T& output_gradient) const { + const T out_conj = numext::conj(output); + return static_cast(-0.5) * (output_gradient * out_conj) * + (out_conj * out_conj); + } + template + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE const Packet + packetOp(const Packet& output, const Packet& output_gradient) const { + const Packet const_half = pset1(static_cast(-0.5)); + const Packet out_conj = pconj(output); + return pmul(const_half, pmul(pmul(output_gradient, out_conj), + pmul(out_conj, out_conj))); + } +}; +template +struct functor_traits> { + enum { + Cost = 4 * NumTraits::MulCost, + PacketAccess = packet_traits::HasMul, + }; +}; + +} // end namespace internal +} // end namespace Eigen + +namespace tensorflow { + +namespace functor { + +template +struct SimpleBinaryFunctor { + void operator()(const Device& d, typename Functor::tout_type out, + typename Functor::tin_type in0, + typename Functor::tin_type in1); +}; + +// Partial specialization of BinaryFunctor for CPU devices +typedef Eigen::ThreadPoolDevice CPUDevice; + +template +struct SimpleBinaryFunctor { + void operator()(const CPUDevice& d, typename Functor::tout_type out, + typename Functor::tin_type in0, + typename Functor::tin_type in1) { + out.device(d) = in0.binaryExpr(in1, typename Functor::func()); + } +}; + +#ifdef TENSORFLOW_USE_SYCL +// Partial specialization of BinaryFunctor for SYCL devices +typedef Eigen::SyclDevice SYCLDevice; +template +struct SimpleBinaryFunctor { + void operator()(const SYCLDevice& d, typename Functor::tout_type out, + typename Functor::tin_type in0, + typename Functor::tin_type in1) { + out.device(d) = in0.binaryExpr(in1, typename Functor::func()); + } +}; + +#endif // TENSORFLOW_USE_SYCL + +template +struct tanh_grad : base> {}; + +template +struct sigmoid_grad : base> { +}; + +template +struct inverse_grad : base> { +}; + +template +struct sqrt_grad : base> {}; + +template +struct rsqrt_grad : base> {}; + +template +struct igamma_grad_a : base> {}; + +} // end namespace functor + +} // end namespace tensorflow +#endif // TENSORFLOW_CORE_KERNELS_CWISE_OPS_GRADIENTS_H_ diff --git a/linux/include/tensorflow/core/kernels/cwise_ops_sycl_common.h b/linux/include/tensorflow/core/kernels/cwise_ops_sycl_common.h new file mode 100644 index 000000000..3e107cee0 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/cwise_ops_sycl_common.h @@ -0,0 +1,163 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#if !TENSORFLOW_USE_SYCL +#error This file must only be included when building TensorFlow with SYCL support +#endif + +#ifndef TENSORFLOW_CORE_KERNELS_CWISE_OPS_SYCL_COMMON_H_ +#define TENSORFLOW_CORE_KERNELS_CWISE_OPS_SYCL_COMMON_H_ + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" + +#include "tensorflow/core/framework/register_types.h" +#include "tensorflow/core/kernels/cwise_ops.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { +namespace functor { + +typedef Eigen::SyclDevice SYCLDevice; + +template +void Assign(const SYCLDevice& d, OUT out, RHS rhs) { + out.device(d) = rhs; +} + +// Partial specialization of UnaryFunctor. +template +struct UnaryFunctor { + void operator()(const SYCLDevice& d, typename Functor::tout_type out, + typename Functor::tin_type in) { + To32Bit(out).device(d) = To32Bit(in).unaryExpr(typename Functor::func()); + } +}; + +// Partial specialization of BinaryFunctor. +template +struct BinaryFunctor { + void operator()(const SYCLDevice& d, typename Functor::tout_type out, + typename Functor::tin_type in0, + typename Functor::tin_type in1, bool* error) { + To32Bit(out).device(d) = + To32Bit(in0).binaryExpr(To32Bit(in1), typename Functor::func()); + } + + void Left(const SYCLDevice& d, typename Functor::tout_type out, + typename Functor::tscalar_type scalar, + typename Functor::tin_type in, bool* error) { + typedef typename Functor::func Binary; + constexpr int NumDims = Functor::tin_type::NumDimensions; + static_assert(NumDims == 1, "Unexpected size"); + Eigen::Sizes<1> scalar_dim; + out.device(d) = scalar.reshape(scalar_dim) + .broadcast(in.dimensions()) + .binaryExpr(in, Binary()); + } + + void Right(const SYCLDevice& d, typename Functor::tout_type out, + typename Functor::tin_type in, + typename Functor::tscalar_type scalar, bool* error) { + typedef typename Functor::func Binary; + constexpr int NumDims = Functor::tin_type::NumDimensions; + static_assert(NumDims == 1, "Unexpected size"); + Eigen::Sizes<1> scalar_dim; + out.device(d) = in.binaryExpr( + scalar.reshape(scalar_dim).broadcast(in.dimensions()), Binary()); + } + + void BCast(const SYCLDevice& d, + typename TTypes::Tensor out, + typename TTypes::ConstTensor in0, + typename Eigen::array bcast0, + typename TTypes::ConstTensor in1, + typename Eigen::array bcast1, + bool* error) { + typedef typename Functor::in_type T; + typename Functor::func func; + if ((NDIMS == 2) && Functor::use_bcast_optimization && + use_bcast_optimization::value) { + const bool bcast0_all_one = AllOne(bcast0); + const bool bcast1_all_one = AllOne(bcast1); + if (bcast0_all_one && !bcast1_all_one) { + To32Bit(out).device(d) = + To32Bit(in0).binaryExpr(To32Bit(in1).broadcast(bcast1), func); + return; + } + if (!bcast0_all_one && bcast1_all_one) { + To32Bit(out).device(d) = + To32Bit(in0).broadcast(bcast0).binaryExpr(To32Bit(in1), func); + return; + } + } + To32Bit(out).device(d) = To32Bit(in0).broadcast(bcast0).binaryExpr( + To32Bit(in1).broadcast(bcast1), func); + } +}; + +// Macros to explicitly instantiate kernels on GPU for multiple types +// (T0, T1, etc.) for UnaryFunctor (e.g., functor::sqrt). +#define DEFINE_UNARY1(F, T) template struct UnaryFunctor > +#define DEFINE_UNARY2(F, T0, T1) \ + DEFINE_UNARY1(F, T0); \ + DEFINE_UNARY1(F, T1) +#define DEFINE_UNARY3(F, T0, T1, T2) \ + DEFINE_UNARY2(F, T0, T1); \ + DEFINE_UNARY1(F, T2) +#define DEFINE_UNARY4(F, T0, T1, T2, T3) \ + DEFINE_UNARY2(F, T0, T1); \ + DEFINE_UNARY2(F, T2, T3) +#define DEFINE_UNARY5(F, T0, T1, T2, T3, T4) \ + DEFINE_UNARY2(F, T0, T1); \ + DEFINE_UNARY3(F, T2, T3, T4) + +// Macros to explicitly instantiate kernels on GPU for multiple types +// (T0, T1, etc.) for BinaryFunctor. +#define DEFINE_BINARY1(F, T) \ + template struct BinaryFunctor, 1>; \ + template struct BinaryFunctor, 2>; \ + template struct BinaryFunctor, 3> +#define DEFINE_BINARY2(F, T0, T1) \ + DEFINE_BINARY1(F, T0); \ + DEFINE_BINARY1(F, T1) +#define DEFINE_BINARY3(F, T0, T1, T2) \ + DEFINE_BINARY2(F, T0, T1); \ + DEFINE_BINARY1(F, T2) +#define DEFINE_BINARY4(F, T0, T1, T2, T3) \ + DEFINE_BINARY2(F, T0, T1); \ + DEFINE_BINARY2(F, T2, T3) +#define DEFINE_BINARY5(F, T0, T1, T2, T3, T4) \ + DEFINE_BINARY2(F, T0, T1); \ + DEFINE_BINARY3(F, T2, T3, T4) +#define DEFINE_BINARY6(F, T0, T1, T2, T3, T4, T5) \ + DEFINE_BINARY3(F, T0, T1, T2); \ + DEFINE_BINARY3(F, T3, T4, T5) +#define DEFINE_BINARY7(F, T0, T1, T2, T3, T4, T5, T6) \ + DEFINE_BINARY3(F, T0, T1, T2); \ + DEFINE_BINARY4(F, T3, T4, T5, T6) +#define DEFINE_BINARY8(F, T0, T1, T2, T3, T4, T5, T6, T7) \ + DEFINE_BINARY4(F, T0, T1, T2, T3); \ + DEFINE_BINARY4(F, T4, T5, T6, T7) +#define DEFINE_BINARY9(F, T0, T1, T2, T3, T4, T5, T6, T7, T8) \ + DEFINE_BINARY4(F, T0, T1, T2, T3); \ + DEFINE_BINARY5(F, T4, T5, T6, T7, T8) +#define DEFINE_BINARY10(F, T0, T1, T2, T3, T4, T5, T6, T7, T8, T9) \ + DEFINE_BINARY5(F, T0, T1, T2, T3, T4); \ + DEFINE_BINARY5(F, T5, T6, T7, T8, T9) + +} // end namespace functor +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_CWISE_OPS_SYCL_COMMON_H_ diff --git a/linux/include/tensorflow/core/kernels/data/captured_function.h b/linux/include/tensorflow/core/kernels/data/captured_function.h new file mode 100644 index 000000000..a10376bf9 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/data/captured_function.h @@ -0,0 +1,147 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CORE_KERNELS_DATA_CAPTURED_FUNCTION_H_ +#define TENSORFLOW_CORE_KERNELS_DATA_CAPTURED_FUNCTION_H_ + +#include +#include + +#include "tensorflow/core/framework/function.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/kernels/data/dataset.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/lib/gtl/array_slice.h" +#include "tensorflow/core/lib/random/random.h" +#include "tensorflow/core/platform/macros.h" + +namespace tensorflow { + +class Device; +class OpKernelContext; +class ResourceMgr; + +namespace data { + +// A `CapturedFunction` encapsulates a TensorFlow function and all of +// the runtime support required to execute it. +// +// The `Dataset`-related classes use `CapturedFunction` to execute +// TensorFlow functions outside a the normal `OpKernel::Compute()` +// context. +class CapturedFunction { + public: + // Creates a new instance using a list of named attributes, fetching captured + // inputs from a context argument. + static Status Create(const NameAttrList& func, OpKernelContext* ctx, + const string& argument, + std::unique_ptr* out_function); + + // Creates a new instance using a list of named attributes, fetching captured + // inputs from a context argument. + // + // If `use_inter_op_parallelism` is false, the runtime may use an executor + // that is optimized for small functions. + static Status Create(const NameAttrList& func, OpKernelContext* ctx, + const string& argument, bool use_inter_op_parallelism, + std::unique_ptr* out_function); + + ~CapturedFunction(); + + // Runs the "Captured function" using the given FLR and caches the lib and + // handle generated during instantiation. If Run is called with a different + // lib afterwards, generates an error. This method takes ownership of the + // tensors in `args`, in order to be able to deallocate them as early as + // possible. Use `RunWithBorrowedArgs()` if the caller needs to retain + // ownership of the `args`. + Status Run(IteratorContext* ctx, std::vector&& args, + std::vector* rets); + + // Synchronously runs the captured function on the given `args`, and stores + // the results in `*rets`. Prefer to use `Run()` or `RunAsync()` when + // possible. + Status RunWithBorrowedArgs(IteratorContext* ctx, + const std::vector& args, + std::vector* rets); + + // Explicitly instantiate this function for use in the given + // context. This method, and the context-less overload + // `RunInstantiated()` below can be useful for calling a captured + // function in cases where an `IteratorContext*` is not available + // (such as a destructor). + Status Instantiate(IteratorContext* ctx); + + // Synchronously runs the captured function on the given `args`, and stores + // the results in `*rets`. Prefer to use `Run()` or `RunAsync()` when + // possible. + // + // REQUIRES: `this->Instantiate()` must have been called before this method. + Status RunInstantiated(const std::vector& args, + std::vector* rets); + + // Asynchronously runs the captured function on the given `args`, stores + // the results in `*rets`, and calls the given `done` callback when the + // function returns. This method takes ownership of the tensors in `args`, + // in order to be able to deallocate them as early as possible. + void RunAsync(IteratorContext* ctx, std::vector&& args, + std::vector* rets, + FunctionLibraryRuntime::DoneCallback done, + const string& prefix); + + // Returns the named list of function arguments. + const NameAttrList& func() { return func_; } + + // Returns that additional captured inputs that will be passed to the function + // when `Run*()` is called. + const std::vector& captured_inputs() { return captured_inputs_; } + + // Returns a step ID for use when running a `CapturedFunction`. + static int64 generate_step_id() { + // Choose a step ID that is guaranteed not to clash with any + // Session-generated step ID. DirectSession only generates + // non-negative step IDs (contiguous, starting from 0), and + // MasterSession generates 56-bit random step IDs whose MSB is + // always 0, so a negative random step ID should suffice. + return -std::abs(static_cast(random::New64())); + } + + private: + CapturedFunction(const NameAttrList& func, + std::vector captured_inputs, + bool use_inter_op_parallelism); + + Status GetHandle(IteratorContext* ctx, + FunctionLibraryRuntime::Handle* out_handle); + + mutex mu_; + const NameAttrList func_; + FunctionLibraryRuntime* lib_ GUARDED_BY(mu_); + FunctionLibraryRuntime::Handle f_handle_ GUARDED_BY(mu_); + const std::vector captured_inputs_; + DataTypeSlice ret_types_; + std::function)> captured_runner_ = nullptr; + const bool use_inter_op_parallelism_; + + TF_DISALLOW_COPY_AND_ASSIGN(CapturedFunction); +}; + +} // namespace data + +// TODO(b/114112161): Remove these aliases when all users have moved over to the +// `tensorflow::data` namespace. +using data::CapturedFunction; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_DATA_CAPTURED_FUNCTION_H_ diff --git a/linux/include/tensorflow/core/kernels/data/dataset.h b/linux/include/tensorflow/core/kernels/data/dataset.h new file mode 100644 index 000000000..2c6fc8d5b --- /dev/null +++ b/linux/include/tensorflow/core/kernels/data/dataset.h @@ -0,0 +1,20 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CORE_KERNELS_DATA_DATASET_H_ +#define TENSORFLOW_CORE_KERNELS_DATA_DATASET_H_ + +#include "tensorflow/core/framework/dataset.h" + +#endif // TENSORFLOW_CORE_KERNELS_DATA_DATASET_H_ diff --git a/linux/include/tensorflow/core/kernels/data/dataset_utils.h b/linux/include/tensorflow/core/kernels/data/dataset_utils.h new file mode 100644 index 000000000..6ec1350cd --- /dev/null +++ b/linux/include/tensorflow/core/kernels/data/dataset_utils.h @@ -0,0 +1,43 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CORE_KERNELS_DATA_DATASET_UTILS_H_ +#define TENSORFLOW_CORE_KERNELS_DATA_DATASET_UTILS_H_ + +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/kernels/data/captured_function.h" +#include "tensorflow/core/kernels/data/dataset.h" + +namespace tensorflow { +namespace data { + +Status MakeIteratorFromInputElement( + IteratorContext* ctx, const std::vector& input_element, + int64 thread_index, CapturedFunction* captured_func, StringPiece prefix, + std::unique_ptr* out_iterator); + +// Returns Status::OK() if `expected` and `received` types match, +// errors::InvalidArgument otherwise. +Status VerifyTypesMatch(const DataTypeVector& expected, + const DataTypeVector& received); + +// Returns Status::OK() if `expected` and `received` shapes are compatible, +// errors::InvalidArgument otherwise. +Status VerifyShapesCompatible(const std::vector& expected, + const std::vector& received); + +} // namespace data +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_DATA_DATASET_UTILS_H_ diff --git a/linux/include/tensorflow/core/kernels/data/experimental/indexed_dataset.h b/linux/include/tensorflow/core/kernels/data/experimental/indexed_dataset.h new file mode 100644 index 000000000..27a8360cb --- /dev/null +++ b/linux/include/tensorflow/core/kernels/data/experimental/indexed_dataset.h @@ -0,0 +1,119 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CORE_KERNELS_DATA_EXPERIMENTAL_INDEXED_DATASET_H_ +#define TENSORFLOW_CORE_KERNELS_DATA_EXPERIMENTAL_INDEXED_DATASET_H_ + +#include "tensorflow/core/framework/dataset.h" +#include "tensorflow/core/framework/op_kernel.h" + +namespace tensorflow { +namespace data { + +// TODO(saeta): Urgh, this is ugly. +class MaterializedIndexedDataset { + public: + virtual ~MaterializedIndexedDataset() = default; + + // Retrieve the element at a given index. The output tensors are stored in + // out_tensors. + // + // If `index` is greater than `Size()`, tensorflow::errors::OutOfRangeError is + // returned. + // + // Get is thread-safe. + virtual Status Get(IteratorContext&& ctx, uint64 index, + std::vector* out_tensors) const = 0; + + // Size determines the number of elements in this IndexedDataset. + // + // Size is thread-safe. + virtual Status Size(uint64* size) const = 0; + + // Returns a vector of DataType values, representing the respective + // element types of each tuple component in the outputs of this dataset. + virtual const DataTypeVector& output_dtypes() const = 0; + + // Returns a vector of tensor shapes, representing the respective + // (and possibly partially defined) shapes of each tuple component + // in the outputs of this dataset. + virtual const std::vector& output_shapes() const = 0; +}; + +// IndexedDataset represents a dataset that supports random access in addition +// to iterator-based sequential access. +// +// Note: IndexedDatasets are HIGHLY experimental at this time. Expect +// significant (backwards incompatible) changes! +class IndexedDataset : public DatasetBase { + public: + IndexedDataset(DatasetContext&& ctx) : DatasetBase(std::move(ctx)) {} + + // Materialize (if necessary) the dataset, and return a pointer. + // TODO(saeta): Add in `IteratorContext* ctx` when materializing. + virtual Status MaterializeDataset( + std::shared_ptr* materialized) = 0; +}; + +// IndexedDatasetOpKernel abstracts away interfacing IndexedDatasets with the +// rest of the TensorFlow runtime. +// +// Most IndexedDataset's will be private members of classes inheriting from this +// class. +class IndexedDatasetOpKernel : public OpKernel { + public: + IndexedDatasetOpKernel(OpKernelConstruction* ctx) : OpKernel(ctx) {} + void Compute(OpKernelContext* ctx) final; + + protected: + // Subclasses should implement this method. It will be called during Compute + // execution. + virtual void MakeIndexedDataset(OpKernelContext* ctx, + IndexedDataset** output) = 0; + + template + Status ParseScalarArgument(OpKernelContext* ctx, + const StringPiece& argument_name, T* output) { + const Tensor* argument_t; + TF_RETURN_IF_ERROR(ctx->input(argument_name, &argument_t)); + if (!TensorShapeUtils::IsScalar(argument_t->shape())) { + return errors::InvalidArgument(argument_name, " must be a scalar"); + } + *output = argument_t->scalar()(); + return Status::OK(); + } +}; + +// Validates and extracts an `IndexedDataset` object from `tensor`. +// +// `tensor` must have been written by a call to +// `StoreIndexedDatasetInVariantTensor` +// +// The retrieved pointer isa borrowed reference to the dataset, which is owned +// by the tensor. The consumer must either acquire its own reference to the +// dataset by calling `(*out_dataset)->Ref()`, or ensure that `tensor` is not +// destroyed or mutated while the retrieved pointer is in use. +Status GetIndexedDatasetFromVariantTensor(const Tensor& tensor, + IndexedDataset** out_dataset); + +// Stores an `IndexedDataset` object in `tensor.` +// +// The ownership of `dataset` is transferred to `tensor`. +Status StoreIndexedDatasetInVariantTensor(IndexedDataset* dataset, + Tensor* tensor); + +} // namespace data +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_DATA_EXPERIMENTAL_INDEXED_DATASET_H_ diff --git a/linux/include/tensorflow/core/kernels/data/generator_dataset_op.h b/linux/include/tensorflow/core/kernels/data/generator_dataset_op.h new file mode 100644 index 000000000..d23ed97ec --- /dev/null +++ b/linux/include/tensorflow/core/kernels/data/generator_dataset_op.h @@ -0,0 +1,42 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_DATA_GENERATOR_DATASET_OP_H_ +#define TENSORFLOW_CORE_KERNELS_DATA_GENERATOR_DATASET_OP_H_ + +#include "tensorflow/core/framework/dataset.h" + +namespace tensorflow { +namespace data { + +class GeneratorDatasetOp : public DatasetOpKernel { + public: + explicit GeneratorDatasetOp(OpKernelConstruction* ctx); + + void MakeDataset(OpKernelContext* ctx, DatasetBase** output) override; + + private: + class Dataset; + + DataTypeVector output_types_; + std::vector output_shapes_; + NameAttrList init_func_; + NameAttrList next_func_; + NameAttrList finalize_func_; +}; + +} // namespace data +} // namespace tensorflow +#endif // TENSORFLOW_CORE_KERNELS_DATA_GENERATOR_DATASET_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/data/iterator_ops.h b/linux/include/tensorflow/core/kernels/data/iterator_ops.h new file mode 100644 index 000000000..8a2b2639a --- /dev/null +++ b/linux/include/tensorflow/core/kernels/data/iterator_ops.h @@ -0,0 +1,149 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_DATA_ITERATOR_OPS_H_ +#define TENSORFLOW_CORE_KERNELS_DATA_ITERATOR_OPS_H_ + +#include "tensorflow/core/common_runtime/function.h" +#include "tensorflow/core/framework/dataset.h" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/kernels/ops_util.h" + +namespace tensorflow { +namespace data { + +class IteratorResource; + +class IteratorHandleOp : public OpKernel { + public: + explicit IteratorHandleOp(OpKernelConstruction* ctx); + + // The resource is deleted from the resource manager only when it is private + // to kernel. Ideally the resource should be deleted when it is no longer held + // by anyone, but it would break backward compatibility. + ~IteratorHandleOp() override; + + void Compute(OpKernelContext* context) override LOCKS_EXCLUDED(mu_); + + private: + // During the first Compute(), resource is either created or looked up using + // shared_name. In the latter case, the resource found should be verified if + // it is compatible with this op's configuration. The verification may fail in + // cases such as two graphs asking queues of the same shared name to have + // inconsistent capacities. + Status VerifyResource(IteratorResource* resource); + + template // use like this: down_cast(foo); + static inline To down_cast(From* f) { // so we only accept pointers + static_assert( + (std::is_base_of::type>::value), + "target type not derived from source type"); + + // We skip the assert and hence the dynamic_cast if RTTI is disabled. +#if !defined(__GNUC__) || defined(__GXX_RTTI) + // Uses RTTI in dbg and fastbuild. asserts are disabled in opt builds. + assert(f == nullptr || dynamic_cast(f) != nullptr); +#endif // !defined(__GNUC__) || defined(__GXX_RTTI) + return static_cast(f); + } + + FunctionLibraryRuntime* CreatePrivateFLR( + OpKernelContext* ctx, std::unique_ptr* device_mgr, + std::unique_ptr* flib_def, + std::unique_ptr* pflr); + + mutex mu_; + ContainerInfo cinfo_; // Written once under mu_ then constant afterwards. + IteratorResource* resource_ GUARDED_BY(mu_) = nullptr; + DataTypeVector output_dtypes_; + std::vector output_shapes_; + const int graph_def_version_; + string name_; +}; + +// Like IteratorHandleOp, but creates handles which are never shared, and does +// not hold a reference to these handles. The latter is important for eager +// execution, since OpKernel instances generally live as long as the program +// running them. +class AnonymousIteratorHandleOp : public OpKernel { + public: + explicit AnonymousIteratorHandleOp(OpKernelConstruction* context); + + void Compute(OpKernelContext* context) override; + + private: + // Coordinates Iterator unique name creation across AnonymousIteratorHandleOp + // instances. + static mutex static_resource_lookup_mutex_; + // current_id_ is just a hint for creating unique names. If it turns out + // there's a collision (e.g. because another AnonymousIteratorHandleOp + // instance is generating handles) we'll just skip that id. + static int64 current_id_ GUARDED_BY(static_resource_lookup_mutex_); + DataTypeVector output_dtypes_; + std::vector output_shapes_; + const int graph_def_version_; +}; + +class MakeIteratorOp : public OpKernel { + public: + explicit MakeIteratorOp(OpKernelConstruction* ctx) : OpKernel(ctx) {} + + void Compute(OpKernelContext* ctx) override; +}; + +class IteratorGetNextOp : public AsyncOpKernel { + public: + explicit IteratorGetNextOp(OpKernelConstruction* ctx) + : AsyncOpKernel(ctx), + background_worker_(ctx->env(), + strings::StrCat("iterator_get_next_thread_", + SanitizeThreadSuffix(name()))) {} + + void ComputeAsync(OpKernelContext* ctx, DoneCallback done) override; + + private: + BackgroundWorker background_worker_; +}; + +class IteratorGetNextSyncOp : public OpKernel { + public: + explicit IteratorGetNextSyncOp(OpKernelConstruction* ctx) : OpKernel(ctx) {} + + void Compute(OpKernelContext* ctx) override; +}; + +class IteratorToStringHandleOp : public OpKernel { + public: + explicit IteratorToStringHandleOp(OpKernelConstruction* ctx) + : OpKernel(ctx) {} + + void Compute(OpKernelContext* ctx) override; +}; + +class IteratorFromStringHandleOp : public OpKernel { + public: + explicit IteratorFromStringHandleOp(OpKernelConstruction* ctx); + + void Compute(OpKernelContext* ctx) override; + + private: + DataTypeVector output_dtypes_; + std::vector output_shapes_; +}; + +} // namespace data +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_DATA_ITERATOR_OPS_H_ diff --git a/linux/include/tensorflow/core/kernels/data/optional_ops.h b/linux/include/tensorflow/core/kernels/data/optional_ops.h new file mode 100644 index 000000000..2cbf2933f --- /dev/null +++ b/linux/include/tensorflow/core/kernels/data/optional_ops.h @@ -0,0 +1,38 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CORE_KERNELS_DATA_OPTIONAL_OPS_H_ +#define TENSORFLOW_CORE_KERNELS_DATA_OPTIONAL_OPS_H_ + +#include + +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/variant_tensor_data.h" + +namespace tensorflow { +namespace data { + +// Stores a DT_VARIANT value representing an Optional with the given value +// in the `output_index`^th output of the given kernel execution context. +Status WriteOptionalWithValueToOutput(OpKernelContext* ctx, int output_index, + std::vector value); + +// Stores a DT_VARIANT value representing an Optional with no value +// in the `output_index`^th output of the given kernel execution context. +Status WriteOptionalNoneToOutput(OpKernelContext* ctx, int output_index); + +} // namespace data +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_DATA_OPTIONAL_OPS_H_ diff --git a/linux/include/tensorflow/core/kernels/data/parallel_map_iterator.h b/linux/include/tensorflow/core/kernels/data/parallel_map_iterator.h new file mode 100644 index 000000000..dc26c5cf2 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/data/parallel_map_iterator.h @@ -0,0 +1,54 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CORE_KERNELS_DATA_PARALLEL_MAP_ITERATOR_H_ +#define TENSORFLOW_CORE_KERNELS_DATA_PARALLEL_MAP_ITERATOR_H_ + +#include + +#include "tensorflow/core/framework/dataset.h" + +namespace tensorflow { +namespace data { + +// A function that transforms elements of one dataset into another +// asynchronously. The arguments are: +// 1. An `IteratorContext*` for the context in which the function should +// execute. +// 2. A `std::vector` containing the input element. +// 3. A `std::vector*` to which the function will write the result. +// 4. A `StatusCallback` that should be invoked when the function is complete. +using ParallelMapIteratorFunction = + std::function, + std::vector*, StatusCallback)>; + +// Returns a new iterator that applies `map_func` to the elements of +// `input_dataset` using the given degree of parallelism. `init_func` (if +// specified) will be executed when the iterator is initialized (see +// `IteratorBase::Initialize()`) and enables the user to specify error checking +// logic that can fail early. +std::unique_ptr NewParallelMapIterator( + const DatasetBaseIterator::BaseParams& params, + const DatasetBase* input_dataset, + std::function init_func, + ParallelMapIteratorFunction map_func, int32 num_parallel_calls); +std::unique_ptr NewParallelMapIterator( + const DatasetBaseIterator::BaseParams& params, + const DatasetBase* input_dataset, ParallelMapIteratorFunction map_func, + int32 num_parallel_calls); + +} // namespace data +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_DATA_PARALLEL_MAP_ITERATOR_H_ diff --git a/linux/include/tensorflow/core/kernels/data/prefetch_autotuner.h b/linux/include/tensorflow/core/kernels/data/prefetch_autotuner.h new file mode 100644 index 000000000..869320551 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/data/prefetch_autotuner.h @@ -0,0 +1,73 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_DATA_PREFETCH_AUTOTUNER_H_ +#define TENSORFLOW_CORE_KERNELS_DATA_PREFETCH_AUTOTUNER_H_ + +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { +namespace data { + +// PrefetchAutotuner dynamically adjusts the buffer size of a prefetch iterator. +// +// PrefetchAutotuner attempts to find the minimum buffer size such that there is +// always at least 1 element in the prefetch queue every time the downstream +// iterator calls GetNext(). +// +// One common failure mode of input pipelines is being throughput bound. No +// amount of prefetching can address that performance mode. In order to guard +// against this condition, PrefetchAutotuner will only increase the buffer_limit +// if the prefetching thread is able to successfully fill the buffer at its +// current size. +// +// Note: in the current implementation, we never decrease the buffer_limit(). +// This should change in the future! +// +// PrefetchAutotuner is NOT thread safe. +class PrefetchAutotuner { + public: + static const int64 kAutoTune = -1; + + explicit PrefetchAutotuner(int64 initial_buffer_size); + + int64 buffer_limit() const { return buffer_limit_; } + + void RecordConsumption(size_t current_buffer_size); + void RecordEmpty() { RecordConsumption(0); } + + private: + // PrefetchAutotuner operates as a state machine. + enum class Mode { + // Disables the autotuning. + kDisabled, + + // We have increased the size of the buffer, and will transition to + // kDownswing if we successfully fill the buffer. + kUpswing, + + // We have successfully filled a buffer of this size. If we ever block the + // downstream iterator, we should increase the buffer size. + kDownswing, + }; + + int64 buffer_limit_; + Mode mode_ = Mode::kDisabled; +}; + +} // namespace data +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_DATA_PREFETCH_AUTOTUNER_H_ diff --git a/linux/include/tensorflow/core/kernels/data/prefetch_dataset_op.h b/linux/include/tensorflow/core/kernels/data/prefetch_dataset_op.h new file mode 100644 index 000000000..588fb25a0 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/data/prefetch_dataset_op.h @@ -0,0 +1,41 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_DATA_PREFETCH_DATASET_OP_H_ +#define TENSORFLOW_CORE_KERNELS_DATA_PREFETCH_DATASET_OP_H_ + +#include "tensorflow/core/kernels/data/dataset.h" +#include "tensorflow/core/kernels/data/prefetch_autotuner.h" + +namespace tensorflow { +namespace data { + +class PrefetchDatasetOp : public UnaryDatasetOpKernel { + public: + explicit PrefetchDatasetOp(OpKernelConstruction* ctx) + : UnaryDatasetOpKernel(ctx) {} + + protected: + void MakeDataset(OpKernelContext* ctx, DatasetBase* input, + DatasetBase** output) override; + + private: + class Dataset; +}; + +} // namespace data +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_DATA_PREFETCH_DATASET_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/data/single_threaded_executor.h b/linux/include/tensorflow/core/kernels/data/single_threaded_executor.h new file mode 100644 index 000000000..e934352a1 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/data/single_threaded_executor.h @@ -0,0 +1,62 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_DATA_SINGLE_THREADED_EXECUTOR_H_ +#define TENSORFLOW_CORE_KERNELS_DATA_SINGLE_THREADED_EXECUTOR_H_ + +#include "tensorflow/core/common_runtime/executor.h" + +namespace tensorflow { +namespace data { + +// Creates a new `Executor` for executing `graph` synchronously on the caller +// thread. +// +// NOTE(mrry): The returned executor is optimized to impose low overhead on +// graphs that perform a small amount of work (e.g. <15us of work per graph on +// present architectures). It eschews concurrency, because issuing work to +// multiple threads can dominate the cost of executing small ops synchronously, +// and because contention in the executor data structures can reduce throughput +// (in terms of ops executed per unit time). +// +// However, the current implementation has the following limitations: +// +// 1. Reference-typed tensors are not supported and will not be supported in +// future. +// 2. Graphs with control flow (containing "Switch" and "Merge" nodes) are not +// currently supported. The current plan is to extend support to "functional" +// control flow after the TensorFlow APIs transition to building graphs in +// that form (e.g. `tf.cond_v2()`). +// 3. Partitioned graphs (containing "_Recv" nodes) are not currently supported. +// The present implementation executes kernels one at a time in topological +// order, and cannot currently distinguish between disconnected subgraphs +// that are logically connected by subgraphs on a different device. +// 4. Memory logging is not currently supported. +// 5. Allocation forwarding is not currently supported. +// 6. Non-default device contexts are not currently supported. In effect, this +// limits the executor to CPU devices. +// 7. Ops that rely on `OpKernelContext::slice_reader_cache()` being non-null +// are not currently supported. +// +// The single-threaded executor is primarily suitable for executing simple +// TensorFlow functions, such as one might find in a `tf.data` pipeline. +Status NewSingleThreadedExecutor(const LocalExecutorParams& params, + std::unique_ptr graph, + Executor** executor); + +} // namespace data +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_DATA_SINGLE_THREADED_EXECUTOR_H_ diff --git a/linux/include/tensorflow/core/kernels/data/sql/driver_manager.h b/linux/include/tensorflow/core/kernels/data/sql/driver_manager.h new file mode 100644 index 000000000..c5428f396 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/data/sql/driver_manager.h @@ -0,0 +1,41 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CORE_KERNELS_DATA_SQL_DRIVER_MANAGER_H_ +#define TENSORFLOW_CORE_KERNELS_DATA_SQL_DRIVER_MANAGER_H_ + +#include "tensorflow/core/kernels/data/sql/query_connection.h" + +namespace tensorflow { +namespace data { +namespace sql { + +// A factory class for creating `QueryConnection` instances. +class DriverManager { + public: + // A factory method for creating `QueryConnection` instances. + // + // `driver_name` is the database type (e.g. 'sqlite'). `driver_name` + // corresponds to a `QueryConnection` subclass. For example, if `driver_name` + // == `sqlite`, then `CreateQueryConnection` will create a + // `SqliteQueryConnection` instance. + static std::unique_ptr CreateQueryConnection( + const string& driver_name); +}; + +} // namespace sql +} // namespace data +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_DATA_SQL_DRIVER_MANAGER_H_ diff --git a/linux/include/tensorflow/core/kernels/data/sql/query_connection.h b/linux/include/tensorflow/core/kernels/data/sql/query_connection.h new file mode 100644 index 000000000..2fd229a9b --- /dev/null +++ b/linux/include/tensorflow/core/kernels/data/sql/query_connection.h @@ -0,0 +1,70 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CORE_KERNELS_DATA_SQL_QUERY_CONNECTION_H_ +#define TENSORFLOW_CORE_KERNELS_DATA_SQL_QUERY_CONNECTION_H_ + +#include "tensorflow/core/framework/tensor.h" + +namespace tensorflow { +namespace data { + +class IteratorContext; + +namespace sql { +// This interface allows a user to connect to a database, execute a query, and +// iterate over the result set, putting the results into an output tensor. +// A subclass implementation is required for each type of database +// (e.g. sqlite3, mysql, etc.) +// +// Presently, a `QueryConnection` instance can only handle one query at a time. +// In a future extension, this class may be refactored so that it creates +// instances of a new class (named, say, `Statement`) which could have a +// one-to-one correspondence with queries. This would make `QueryConnection` +// more consistent with `Connection` classes of other database APIs. +// `QueryConnection` would then be renamed simply `Connection`. +// +// This class is not thread safe. Access to it is guarded by a mutex in +// `SqlDatasetOp::Dataset::Iterator`. +class QueryConnection { + public: + virtual ~QueryConnection() {} + // Opens a connection to the database named by `data_source_name`. Prepares to + // execute `query` against the database. + // + // The client must call `Close()` to release the connection resources, even + // if `Open()` fails. `Close()` must be called before making another call + // to `Open()`. + virtual Status Open(const string& data_source_name, const string& query, + const DataTypeVector& output_types) = 0; + // Closes an opened connection. + virtual Status Close() = 0; + // Retrieves the next row of the result set of the query from the most recent + // call to `Open()`. + // + // If such a row exists, then the row will be stored in `*out_tensors`, and + // `false` will be stored in `*end_of_sequence`. + // + // If there are no more rows in the result set, then instead `true` will be + // stored in `*end_of_sequence`, and the content of `*out_tensors` will be + // undefined. + virtual Status GetNext(IteratorContext* ctx, std::vector* out_tensors, + bool* end_of_sequence) = 0; +}; + +} // namespace sql +} // namespace data +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_DATA_SQL_QUERY_CONNECTION_H_ diff --git a/linux/include/tensorflow/core/kernels/data/sql/sqlite_query_connection.h b/linux/include/tensorflow/core/kernels/data/sql/sqlite_query_connection.h new file mode 100644 index 000000000..175492c49 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/data/sql/sqlite_query_connection.h @@ -0,0 +1,56 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CORE_KERNELS_DATA_SQL_SQLITE_QUERY_CONNECTION_H_ +#define TENSORFLOW_CORE_KERNELS_DATA_SQL_SQLITE_QUERY_CONNECTION_H_ + +#include + +#include "tensorflow/core/kernels/data/sql/query_connection.h" +#include "tensorflow/core/lib/db/sqlite.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { +namespace data { +namespace sql { + +class SqliteQueryConnection : public QueryConnection { + public: + SqliteQueryConnection(); + ~SqliteQueryConnection() override; + Status Open(const string& data_source_name, const string& query, + const DataTypeVector& output_types) override; + Status Close() override; + Status GetNext(IteratorContext* ctx, std::vector* out_tensors, + bool* end_of_sequence) override; + + private: + // Prepares the query string `query_`. + Status PrepareQuery(); + // Fills `tensor` with the column_index_th element of the current row of + // `stmt_`. + void FillTensorWithResultSetEntry(const DataType& data_type, int column_index, + Tensor* tensor); + Sqlite* db_ = nullptr; + SqliteStatement stmt_; + int column_count_ = 0; + string query_; + DataTypeVector output_types_; +}; + +} // namespace sql +} // namespace data +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_DATA_SQL_SQLITE_QUERY_CONNECTION_H_ diff --git a/linux/include/tensorflow/core/kernels/data/window_dataset.h b/linux/include/tensorflow/core/kernels/data/window_dataset.h new file mode 100644 index 000000000..84cb3c786 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/data/window_dataset.h @@ -0,0 +1,50 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CORE_KERNELS_DATA_WINDOW_DATASET_H_ +#define TENSORFLOW_CORE_KERNELS_DATA_WINDOW_DATASET_H_ + +#include + +#include "tensorflow/core/framework/partial_tensor_shape.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/kernels/data/dataset.h" + +namespace tensorflow { +namespace data { + +// Creates a dataset representing an eagerly-collected window of elements. +// +// The `elements` argument defines the elements of the resulting +// dataset, which is stored in `out_dataset`. +// +// This dataset is constructed internally for use in datasets that +// build nested dataset expressions (e.g. the reducer function for +// GroupByWindowDataset). It efficiently supports multiple iterators on +// the same window without recomputation. +// +// REQUIRES: `output_types` must match the types of the respective +// element components in `elements`. +// REQUIRES: `output_shapes` must be compatible with the shapes of the +// respective element components in `elements`.a +Status NewWindowDataset(std::vector> elements, + DataTypeVector output_types, + std::vector output_shapes, + DatasetBase** out_dataset); + +} // namespace data +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_DATA_WINDOW_DATASET_H_ diff --git a/linux/include/tensorflow/core/kernels/data_format_ops.h b/linux/include/tensorflow/core/kernels/data_format_ops.h new file mode 100644 index 000000000..bc416fa78 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/data_format_ops.h @@ -0,0 +1,86 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_DATA_FORMAT_OPS_H_ +#define TENSORFLOW_CORE_KERNELS_DATA_FORMAT_OPS_H_ +// Functor definition for data format dim mapping ops, must be compilable +// by nvcc. +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/tensor_types.h" + +namespace tensorflow { +namespace functor { + +// Functor used by DataFormatDimMapOP to do the computations. +template +struct DataFormatDimMap { + void operator()(const Device& d, typename TTypes::ConstFlat x, + typename TTypes::Flat y, const TTypes::Vec dst) { + auto zero = x.constant(0); + auto one = x.constant(1); + auto two = x.constant(2); + + auto f_zero = x.constant(dst(0)); + auto f_one = x.constant(dst(1)); + auto f_two = x.constant(dst(2)); + auto f_three = x.constant(dst(3)); + + auto four = x.constant(4); + auto x_mod = (x + four) % 4; + + auto is_zero = (x_mod == zero); + auto is_one = (x_mod == one); + auto is_two = (x_mod == two); + + y.device(d) = is_zero.select( + f_zero, is_one.select(f_one, is_two.select(f_two, f_three))); + } +}; + +template +struct VecPermute { + VecPermute(const Eigen::DSizes& dst) : dst_(dst) {} + Eigen::DSizes dimensions( + typename TTypes::ConstFlat input) const { + Eigen::DSizes result; + result[0] = input.dimension(0); + return result; + } + template + void eval(typename TTypes::ConstFlat input, Output& output, + const Device& d) const { + for (int i = 0; i < input.size(); ++i) { + output.template chip<0>(dst_[i]).device(d) = input.template chip<0>(i); + } + } + + private: + Eigen::DSizes dst_; +}; + +// Functor used by DataFormatVecPermuteOp to do the computations. +template +struct DataFormatVecPermute { + void operator()(const Device& d, typename TTypes::ConstFlat x, + typename TTypes::Flat y, + const Eigen::DSizes& dst) { + y.device(d) = x.customOp(VecPermute(dst)); + } +}; + +} // namespace functor +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_DATA_FORMAT_OPS_H_ diff --git a/linux/include/tensorflow/core/kernels/deep_conv2d.h b/linux/include/tensorflow/core/kernels/deep_conv2d.h new file mode 100644 index 000000000..17a023051 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/deep_conv2d.h @@ -0,0 +1,117 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_DEEP_CONV2D_H_ +#define TENSORFLOW_CORE_KERNELS_DEEP_CONV2D_H_ + +#include "tensorflow/core/framework/types.h" + +namespace tensorflow { + +class OpKernelContext; + +// DeepConv2D is a Conv2D implementation specialized for deep (i.e. large +// in_depth * out_depth product) convolutions (see deep_conv2d.cc for details). + +// DeepConv2DTransform is an interface for implementing transforms for +// DeepConv2D. Implementations must specify transform matrices and +// input/output/filter shapes. DeepConv2d computes: +// +// y = C[Ad * Bg] +// +// C: output transform matrix +// A: input data transform matrix +// B: filter transform matrix +// d: vectorized 2D data tile +// g: vectorized 2D filter tile +// y: vectorized 2D output tile + +template +class DeepConv2DTransform { + public: + virtual ~DeepConv2DTransform() {} + + virtual void GetFilterTransformMatrix(const int64 rows, const int64 cols, + T* transform_matrix) const = 0; + + virtual void GetInputTransformMatrix(const int64 rows, const int64 cols, + T* transform_matrix) const = 0; + + virtual void GetOutputTransformMatrix(const int64 rows, const int64 cols, + T* transform_matrix) const = 0; + + struct Shape { + Shape(int64 r, int64 c) : rows(r), cols(c) {} + int64 rows; + int64 cols; + }; + + virtual const Shape& filter_shape() const = 0; + virtual const Shape& input_shape() const = 0; + virtual const Shape& output_shape() const = 0; +}; + +// Conv2D arguments used by DeepConv2D implementation. +struct Conv2DArgs { + // Input layer dimensions + int batch; + int in_rows; + int in_cols; + int in_depth; + int filter_rows; + int filter_cols; + int pad_rows; + int pad_cols; + + // Output layer dimensions + int out_rows; + int out_cols; + int out_depth; + + Conv2DArgs() + : batch(0), + in_rows(0), + in_cols(0), + in_depth(0), + filter_rows(0), + filter_cols(0), + pad_rows(0), + pad_cols(0), + out_rows(0), + out_cols(0), + out_depth(0) {} +}; + +// Returns true if convolution operation specified by function arguments +// can use DeepConv2D implementation, and false otherwise. +// May return false based on parameters, cost, or whether feature is disabled. +bool CanUseDeepConv2D(int stride_rows, int stride_cols, int filter_rows, + int filter_cols, int in_depth, int out_depth, + int out_rows, int out_cols); + +namespace functor { + +// Calls DeepConv2D implementation (see deep_conv2d.cc for details). +template +struct DeepConv2D { + void operator()(OpKernelContext* ctx, const Conv2DArgs& args, const T* input, + const T* filter, T* output); +}; + +} // namespace functor + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_DEEP_CONV2D_H_ diff --git a/linux/include/tensorflow/core/kernels/dense_update_functor.h b/linux/include/tensorflow/core/kernels/dense_update_functor.h new file mode 100644 index 000000000..61b573125 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/dense_update_functor.h @@ -0,0 +1,108 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_DENSE_UPDATE_FUNCTOR_H_ +#define TENSORFLOW_CORE_KERNELS_DENSE_UPDATE_FUNCTOR_H_ + +#define EIGEN_USE_THREADS + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/tensor_types.h" + +namespace tensorflow { + +typedef Eigen::ThreadPoolDevice CPUDevice; +typedef Eigen::GpuDevice GPUDevice; + +#ifdef TENSORFLOW_USE_SYCL +typedef Eigen::SyclDevice SYCLDevice; +#endif // TENSORFLOW_USE_SYCL + +enum DenseUpdateType { ADD, SUB, ASSIGN }; + +namespace functor { + +template +struct DenseUpdate { + void operator()(const Device& d, typename TTypes::Flat params, + typename TTypes::ConstFlat update); +}; + +template +struct DenseUpdate { + void operator()(const CPUDevice& d, typename TTypes::Flat params, + typename TTypes::ConstFlat update) { + params.device(d) += update; + } +}; + +template +struct DenseUpdate { + void operator()(const CPUDevice& d, typename TTypes::Flat params, + typename TTypes::ConstFlat update) { + params.device(d) -= update; + } +}; + +template +struct DenseUpdate { + void operator()(const CPUDevice& d, typename TTypes::Flat params, + typename TTypes::ConstFlat update) { + params.device(d) = update; + } +}; + +#ifdef TENSORFLOW_USE_SYCL +template +struct DenseUpdate { + void operator()(const SYCLDevice& d, typename TTypes::Flat params, + typename TTypes::ConstFlat update) { + params.device(d) += update; + } +}; + +template +struct DenseUpdate { + void operator()(const SYCLDevice& d, typename TTypes::Flat params, + typename TTypes::ConstFlat update) { + params.device(d) -= update; + } +}; + +template +struct DenseUpdate { + void operator()(const SYCLDevice& d, typename TTypes::Flat params, + typename TTypes::ConstFlat update) { + params.device(d) = update; + } +}; +#endif // TENSORFLOW_USE_SYCL + +} // end namespace functor + +template +Status VariantCopyFn(OpKernelContext* context, const Tensor& from, Tensor* to); + +template <> +Status VariantCopyFn(OpKernelContext* context, const Tensor& from, + Tensor* to); +template <> +Status VariantCopyFn(OpKernelContext* context, const Tensor& from, + Tensor* to); + +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_DENSE_UPDATE_FUNCTOR_H_ diff --git a/linux/include/tensorflow/core/kernels/depthtospace_op.h b/linux/include/tensorflow/core/kernels/depthtospace_op.h new file mode 100644 index 000000000..272468b74 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/depthtospace_op.h @@ -0,0 +1,56 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_DEPTHTOSPACE_OP_H_ +#define TENSORFLOW_CORE_KERNELS_DEPTHTOSPACE_OP_H_ + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/util/tensor_format.h" + +namespace tensorflow { +namespace functor { + +// Functor used by DepthToSpaceOp to do the computations. +// Implements a family of Depth to Space transforms for a 4D 'input' tensor +// to a 4D 'output' tensor, both tensors use type 'T' and layout 'data_format'. +// These transforms multiply the vertical and horizontal image sizes by +// 'block_size', and divide the depth dimension by (block_size * block_size) +// which must divide evenly. +// Each pixel in the input image is converted to a square block of pixels in +// the output image. The Y, X coordinates within each block comes from the +// high component of the input depth (channel) index. +// e.g. for data_format = NHWC: +// Each element in the input tensor can be specified via 6 coordinates, +// ordered by decreasing memory layout significance as: +// n,iY,iX,bY,bX,oC (where n=batch index, iX, iY means X or Y coordinates +// within the input image, bX, bY means coordinates +// within the output block, oC means output channel). +// The output would be a transpose to the following layout: +// n,iY,bY,iX,bX,oC +template +struct DepthToSpaceOpFunctor { + void operator()(const Device& d, typename TTypes::ConstTensor input, + int block_size, typename TTypes::Tensor output); + + // This 5-D version is to support NCHW_VECT_C. + void operator()(const Device& d, typename TTypes::ConstTensor input, + int block_size, typename TTypes::Tensor output); +}; + +} // namespace functor +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_DEPTHTOSPACE_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/depthwise_conv_op.h b/linux/include/tensorflow/core/kernels/depthwise_conv_op.h new file mode 100644 index 000000000..b2d589889 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/depthwise_conv_op.h @@ -0,0 +1,287 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_DEPTHWISE_CONV_OP_H_ +#define TENSORFLOW_CORE_KERNELS_DEPTHWISE_CONV_OP_H_ + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/util/tensor_format.h" + +namespace tensorflow { + +struct DepthwiseArgs { + // Input layer dimensions + int batch; + int in_rows; + int in_cols; + int in_depth; + int filter_rows; + int filter_cols; + int depth_multiplier; + int stride; + int pad_rows; + int pad_cols; + + // Output layer dimensions + int out_rows; + int out_cols; + int out_depth; + + DepthwiseArgs() + : batch(0), + in_rows(0), + in_cols(0), + in_depth(0), + filter_rows(0), + filter_cols(0), + depth_multiplier(0), + stride(0), + pad_rows(0), + pad_cols(0), + out_rows(0), + out_cols(0), + out_depth(0) {} +}; + +// Forward declaration. +class OpKernelContext; + +template +struct LaunchDepthwiseConvOp { + void operator()(OpKernelContext* ctx, const DepthwiseArgs& args, + const T* input, const T* filter, T* output, + TensorFormat data_format); +}; + +template +struct LaunchDepthwiseConvBackpropInputOp { + void operator()(OpKernelContext* ctx, const DepthwiseArgs& args, + const T* out_backprop, const T* filter, T* in_backprop, + TensorFormat data_format); +}; + +template +struct LaunchDepthwiseConvBackpropFilterOp { + void operator()(OpKernelContext* ctx, const DepthwiseArgs& args, + const T* out_backprop, const T* input, T* filter_backprop, + TensorFormat data_format); +}; + +#if GOOGLE_CUDA +template +struct LaunchDepthwiseConvOp { + void operator()(OpKernelContext* ctx, const DepthwiseArgs& args, + const T* input, const T* filter, T* output, + TensorFormat data_format); +}; + +template +struct LaunchDepthwiseConvBackpropInputOp { + void operator()(class OpKernelContext* ctx, const DepthwiseArgs& args, + const T* out_backprop, const T* filter, T* in_backprop, + TensorFormat data_format); +}; + +template +struct LaunchDepthwiseConvBackpropFilterOp { + void operator()(class OpKernelContext* ctx, const DepthwiseArgs& args, + const T* out_backprop, const T* input, T* filter_backprop, + TensorFormat data_format); +}; +#endif + +} // namespace tensorflow + +namespace tensorflow { +namespace functor { + +// Pads 'filter' to vector-register boundary along its inner dimension: +// filter_inner_dim_size = in_depth * depth_multiplier +// Requires 'filter' to have the following storage order: +// [filter_rows, filter_cols, in_depth, depth_multiplier] +// Returns zero-padded filter in 'padded_filter'. +// +// EX: +// in_depth = 3, depth_multiplier = 2, filter [2, 2], register_width = 4 +// So we have a total of 3 * 2 = 6 filters, each of spatial size 2 x 2. +// +// filter [rows, cols, in_depth, depth_multiplier] +// [u0, v0, w0, x0] [y0, z0, u1, v1] [w1, x1, y1, z1] +// [u2, v2, w2, x2] [y2, z2, u3, v3] [w3, x3, y3, z3] +// +// padded_filter [rows, cols, in_depth, depth_multiplier] +// [u0, v0, w0, x0] [y0, z0, 0, 0] [u1, v1, w1, x1] [y1, z1, 0, 0] +// [u2, v2, w2, x2] [y2, z2, 0, 0] [u3, v3, w3, x3] [y3, z3, 0, 0] + +template +struct DepthwiseFilterPadOp { + void operator()(const DepthwiseArgs& args, const T* filter, + T* padded_filter) { + typedef typename Eigen::internal::packet_traits::type Packet; + static const int64 kPacketSize = (sizeof(Packet) / sizeof(T)); + + // Calculate vectorized and scalar lengths of filter's inner dimension. + const int64 filter_inner_dim_size = args.out_depth; + const int64 vectorized_size = + (filter_inner_dim_size / kPacketSize) * kPacketSize; + const int64 scalar_size = filter_inner_dim_size - vectorized_size; + // Calculate required padding and padded output buffer stride. + const int64 pad_size = scalar_size > 0 ? kPacketSize - scalar_size : 0; + const int64 padded_filter_stride = vectorized_size + kPacketSize; + + const int64 filter_spatial_size = args.filter_rows * args.filter_cols; + for (int64 i = 0; i < filter_spatial_size; ++i) { + const int64 input_base = i * filter_inner_dim_size; + const int64 output_base = i * padded_filter_stride; + // Write vectorized length of filter's inner dimension to output. + for (int64 j = 0; j < vectorized_size; j += kPacketSize) { + const auto v = Eigen::internal::ploadu(filter + input_base + j); + Eigen::internal::pstoreu(padded_filter + output_base + j, v); + } + // Write scalar length of filter's inner dimension to output. + for (int64 j = 0; j < scalar_size; ++j) { + padded_filter[output_base + vectorized_size + j] = + filter[input_base + vectorized_size + j]; + } + // Pad the remainder of output to vector-register boundary. + for (int64 j = 0; j < pad_size; ++j) { + padded_filter[output_base + vectorized_size + scalar_size + j] = + static_cast(0); + } + } + } +}; + +// Copies data from local region in 'input' specified by 'out_r' and 'out_'c' +// to 'input_buffer'. The copied data is replicated by factor +// 'args.depth_mulitplier', and padded to vector register-width boundaries so +// that it is aligned for efficient traversal and vector multiply-add by the +// depthwise kernel. +// +// EX: +// in_depth = 3, depth_multiplier = 2, filter [2, 2], register_width = 4 +// +// input: [batch, in_rows, in_cols, in_depth] +// +// [a0, a1, a2, b0, b1, b2, ..., e0, e1, e2, f0, f1, f2, ...] +// +// input_buffer (register boundaries shown): +// [a0, a0, a1, a1] [a2, a2, 0, 0] in_row = 0, in_col = 0 +// [b0, b0, b1, b1] [b2, b2, 0, 0] in_row = 0, in_col = 1 +// [e0, e0, e1, e1] [e2, e2, 0, 0] in_row = 1, in_col = 0 +// [f0, f0, f1, f1] [f2, f2, 0, 0] in_row = 1, in_col = 1 +// +// Returns replicated and padded data from specified input region in +// 'input_buffer'. + +template +struct DepthwiseInputCopyOp { + void operator()(const DepthwiseArgs& args, + const int64 padded_filter_inner_dim_size, const int64 out_r, + const int64 out_c, const T* input, T* input_buffer) { + typedef typename Eigen::internal::packet_traits::type Packet; + static const int64 kPacketSize = (sizeof(Packet) / sizeof(T)); + + // Calculate vectorized and scalar (residual) lengths for 'in_depth'. + const int64 input_vectorized_size = + (args.in_depth / kPacketSize) * kPacketSize; + const int64 input_scalar_size = args.in_depth % kPacketSize; + + // Calculate vectorized and scalar (residual) lengths for + // 'depth_multiplier'. This is used to efficiently replicate data for + // when 'depth_multiplier' > kPacketSize. + const int64 dm_vectorized_size = + (args.depth_multiplier / kPacketSize) * kPacketSize; + const int64 dm_scalar_size = args.depth_multiplier % kPacketSize; + + // Calculate output padding length. + const int64 output_scalar_size = args.out_depth % kPacketSize; + const int64 output_pad_size = + output_scalar_size > 0 ? kPacketSize - output_scalar_size : 0; + + const int64 replicated_packet_size = kPacketSize * args.depth_multiplier; + + // Iterate through all rows x cols reading 'in_depth' from 'input' and + // replicating by 'depth_multiplier' into 'input_buffer' (otherwise + // zero-padding input buffer as needed). + auto* in_buf = input_buffer; + const int64 in_r_start = out_r * args.stride - args.pad_rows; + const int64 in_c_start = out_c * args.stride - args.pad_cols; + + for (int64 f_r = 0; f_r < args.filter_rows; ++f_r) { + const int64 in_r = in_r_start + f_r; + + for (int64 f_c = 0; f_c < args.filter_cols; ++f_c) { + const int64 in_c = in_c_start + f_c; + + if (in_r >= 0 && in_r < args.in_rows && in_c >= 0 && + in_c < args.in_cols) { + auto* in = input + (in_r * args.in_cols + in_c) * args.in_depth; + // Copy vectorized portion of inner dimension. + for (int64 d = 0; d < input_vectorized_size; d += kPacketSize) { + auto v = Eigen::internal::ploadu(in + d); + for (int dm = 0; dm < args.depth_multiplier; ++dm) { + Eigen::internal::pscatter(in_buf + dm, v, + args.depth_multiplier); + } + in_buf += replicated_packet_size; + } + + // Copy scalar portion of inner dimension. + for (int64 d = 0; d < input_scalar_size; ++d) { + T v = in[input_vectorized_size + d]; + const int64 base = d * args.depth_multiplier; + if (dm_vectorized_size > 0) { + // Copy vectorized portion of replicated output. + // This branch is only taken if 'args.depth_multiplier' is + // vectorizable (i.e. args.depth_multiplier >= register width). + auto p = Eigen::internal::pset1(v); + for (int64 dm = 0; dm < dm_vectorized_size; dm += kPacketSize) { + Eigen::internal::pstoreu(in_buf + base + dm, p); + } + // Copy scalar portion of replicated output. + for (int64 dm = 0; dm < dm_scalar_size; ++dm) { + in_buf[base + dm_vectorized_size + dm] = v; + } + } else { + // Depth multiplier is less than one packet: scalar copy. + for (int dm = 0; dm < args.depth_multiplier; ++dm) { + in_buf[base + dm] = v; + } + } + } + in_buf += input_scalar_size * args.depth_multiplier; + + // Pad the remainder of the output to vector register boundary. + for (int64 d = 0; d < output_pad_size; ++d) { + in_buf[d] = static_cast(0); + } + in_buf += output_pad_size; + + } else { + // Zero pad. + memset(in_buf, 0, sizeof(T) * padded_filter_inner_dim_size); + in_buf += padded_filter_inner_dim_size; + } + } + } + } +}; + +} // namespace functor +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_DEPTHWISE_CONV_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/determinant_op.h b/linux/include/tensorflow/core/kernels/determinant_op.h new file mode 100644 index 000000000..eefdfe0ae --- /dev/null +++ b/linux/include/tensorflow/core/kernels/determinant_op.h @@ -0,0 +1,47 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_DETERMINANT_OP_H_ +#define TENSORFLOW_CORE_KERNELS_DETERMINANT_OP_H_ + +#include "tensorflow/core/framework/tensor_types.h" + +namespace tensorflow { +namespace functor { + +// Helper functor to compute Determinant from a partially pivoted LU +// factorization. +template +struct DeterminantFromPivotedLUFunctor { + void operator()(const Device& device, + typename TTypes::ConstTensor lu_factor, + const int* pivots, typename TTypes::Tensor output, + int* info); +}; + +// Helper functor to compute sign and log of the absolute value of the +// determinant from a partially pivoted LU factorization. +template +struct LogDeterminantFromPivotedLUFunctor { + void operator()(const Device& device, + typename TTypes::ConstTensor lu_factor, + const int* pivots, typename TTypes::Tensor sign, + typename TTypes::Tensor log_abs_det); +}; + +} // namespace functor +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_DETERMINANT_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/diag_op.h b/linux/include/tensorflow/core/kernels/diag_op.h new file mode 100644 index 000000000..baf16ddb4 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/diag_op.h @@ -0,0 +1,43 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_DIAG_OP_H_ +#define TENSORFLOW_CORE_KERNELS_DIAG_OP_H_ + +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +namespace functor { + +template +struct DiagFunctor { + Status operator()(OpKernelContext* context, const int64 size, const T* in, + T* out); +}; + +template +struct DiagPartFunctor { + Status operator()(OpKernelContext* context, const int64 size, const T* in, + T* out); +}; + +} // namespace functor + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_DIAG_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/dilation_ops.h b/linux/include/tensorflow/core/kernels/dilation_ops.h new file mode 100644 index 000000000..63d386aa4 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/dilation_ops.h @@ -0,0 +1,66 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_DILATION_OPS_H_ +#define TENSORFLOW_CORE_KERNELS_DILATION_OPS_H_ + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { +namespace functor { + +template +struct Dilation { + // We assume that the tensor sizes are correct. + void operator()(const Device& d, typename TTypes::ConstTensor input, + typename TTypes::ConstTensor filter, int stride_rows, + int stride_cols, int rate_rows, int rate_cols, int pad_top, + int pad_left, typename TTypes::Tensor output); +}; + +template +struct DilationBackpropInput { + // We assume that the tensor sizes are correct. + // To avoid storing the argmax values during forward computation, we recompute + // the argmax during backward computation, which is the reason why we provide + // filter as argument to the backward computation routine. + void operator()(const Device& d, typename TTypes::ConstTensor input, + typename TTypes::ConstTensor filter, + typename TTypes::ConstTensor out_backprop, + int stride_rows, int stride_cols, int rate_rows, + int rate_cols, int pad_top, int pad_left, + typename TTypes::Tensor in_backprop); +}; + +template +struct DilationBackpropFilter { + // We assume that the tensor sizes are correct. + // To avoid storing the argmax values during forward computation, we recompute + // the argmax during backward computation, which is the reason why we provide + // filter as argument to the backward computation routine. + void operator()(const Device& d, typename TTypes::ConstTensor input, + typename TTypes::ConstTensor filter, + typename TTypes::ConstTensor out_backprop, + int stride_rows, int stride_cols, int rate_rows, + int rate_cols, int pad_top, int pad_left, + typename TTypes::Tensor filter_backprop); +}; + +} // namespace functor +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_DILATION_OPS_H_ diff --git a/linux/include/tensorflow/core/kernels/eigen_activations.h b/linux/include/tensorflow/core/kernels/eigen_activations.h new file mode 100644 index 000000000..302033e47 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/eigen_activations.h @@ -0,0 +1,125 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_EIGEN_ACTIVATIONS_H_ +#define TENSORFLOW_CORE_KERNELS_EIGEN_ACTIVATIONS_H_ + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" + +namespace Eigen { + +/** scalar_sigmoid_fast_derivative_op + * \ingroup CXX11_NeuralNetworks_Module + * \brief Template functor to compute the fast derivative of a sigmoid + * + * Input should be the backpropagated gradient. + * + * \sa class CwiseUnaryOp, Cwise::sigmoid_fast_derivative() + */ +template +struct scalar_sigmoid_fast_derivative_op { + EIGEN_EMPTY_STRUCT_CTOR(scalar_sigmoid_fast_derivative_op) + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE T operator()(const T& y) const { + const T one = T(1); + return (one - y) * y; + } + + template + inline Packet packetOp(const Packet& y) const { + const Packet one = internal::pset1(1); + return internal::pmul(internal::psub(one, y), y); + } +}; + +namespace internal { +template +struct functor_traits > { + enum { + Cost = NumTraits::AddCost * 2 + NumTraits::MulCost, + PacketAccess = packet_traits::HasAdd && packet_traits::HasMul && + packet_traits::HasNegate + }; +}; +} // namespace internal + +/** scalar_tanh_fast_derivative_op + * \ingroup CXX11_NeuralNetworks_Module + * \brief Template functor to compute the fast derivative of a tanh + * + * Input should be the backpropagated gradient. + * + * \sa class CwiseUnaryOp, Cwise::tanh_fast_derivative() + */ +template +struct scalar_tanh_fast_derivative_op { + EIGEN_EMPTY_STRUCT_CTOR(scalar_tanh_fast_derivative_op) + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE T operator()(const T& y) const { + const T one = T(1); + return one - (y * y); + } + + template + inline Packet packetOp(const Packet& y) const { + const Packet one = internal::pset1(1); + return internal::psub(one, internal::pmul(y, y)); + } +}; + +namespace internal { +template +struct functor_traits > { + enum { + Cost = NumTraits::AddCost * 2 + NumTraits::MulCost * 1, + PacketAccess = packet_traits::HasAdd && packet_traits::HasMul && + packet_traits::HasNegate + }; +}; +} // namespace internal + +/** + * \ingroup CXX11_NeuralNetworks_Module + * \brief Template functor to clip the magnitude of the first scalar. + * + * \sa class CwiseBinaryOp, MatrixBase::Clip + */ +template +struct scalar_clip_op { + EIGEN_EMPTY_STRUCT_CTOR(scalar_clip_op) + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE const Scalar + operator()(const Scalar& a, const Scalar& b) const { + return numext::mini(numext::maxi(a, -b), b); + } + template + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE const Packet + packetOp(const Packet& a, const Packet& b) const { + return internal::pmin(internal::pmax(a, internal::pnegate(b)), b); + } +}; + +namespace internal { +template +struct functor_traits > { + enum { + Cost = NumTraits::AddCost * 3, + PacketAccess = packet_traits::HasMax && + packet_traits::HasMin && + packet_traits::HasNegate + }; +}; +} // namespace internal + +} // end namespace Eigen + +#endif // TENSORFLOW_CORE_KERNELS_EIGEN_ACTIVATIONS_H_ diff --git a/linux/include/tensorflow/core/kernels/eigen_attention.h b/linux/include/tensorflow/core/kernels/eigen_attention.h new file mode 100644 index 000000000..4d86f9deb --- /dev/null +++ b/linux/include/tensorflow/core/kernels/eigen_attention.h @@ -0,0 +1,255 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_EIGEN_ATTENTION_H_ +#define TENSORFLOW_CORE_KERNELS_EIGEN_ATTENTION_H_ + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" + +namespace Eigen { + +/** ExtractGlimpses + * \ingroup CXX11_NeuralNetworks_Module + * + * \brief Extract glimpses from an input tensor. + * + * The input parameter is expected to be a col-major tensor with a rank of 4 + * (depth, x, y, and batch). The width and height parameters specify the + * extension of the returned glimpses. The offsets parameter specifies the x, y + * locations of the center of the glimpses relative to the center of the input + * image. The vector is expected to contain one IndexPair for each image in the + * batch dimension. The normalized boolean indicates if incoming coordinates are + * normalized so that 0.0 and 1.0 correspond to the minimum and maximum of each + * height and width dimension. The centered boolean indicates if incoming + * coordinates are centered relative to the image, in which case -1.0 and 1.0 + * correspond to minimum and maximum of each dimension while 0.0 corresponds to + * the center. + * + * The result can be assigned to a tensor of rank equal to that of the input. + * The result will be laid out in col-major order (depth, x, y, batch). The + * dimensions of the result will be equal to the dimensions of the input except + * for width and height which will be equal to the requested glimpse size. + */ +namespace { +template +struct GlimpseExtractionOp { + GlimpseExtractionOp(const Index width, const Index height, + const std::vector >& offsets, + const bool normalized, const bool centered, + const bool uniform_noise) + : width_(width), + height_(height), + offsets_(offsets), + normalized_(normalized), + centered_(centered), + uniform_noise_(uniform_noise) {} + + template + DSizes dimensions(const Input& input) const { + typedef typename internal::traits::Index IndexType; + typedef TensorRef::Scalar, 4, + internal::traits::Layout, IndexType> > + Ref; + Ref in(input); + + DSizes dims = in.dimensions(); + + dims[0] = in.dimension(0); + dims[1] = width_; + dims[2] = height_; + dims[3] = in.dimension(3); + return dims; + } + + template + EIGEN_DEVICE_FUNC void eval(const Input& input, Output& output, + const Device& device) const { + typedef typename internal::traits::Index IndexType; + typedef TensorRef::Scalar, 4, + internal::traits::Layout, IndexType> > + Ref; + Ref in(input); + const Index num_channels = in.dimension(0); + const Index input_width = in.dimension(1); + const Index input_height = in.dimension(2); + const Index batch_size = in.dimension(3); + eigen_assert(input_width > 0); + eigen_assert(input_height > 0); + internal::NormalRandomGenerator gen; + internal::UniformRandomGenerator unigen; + + for (Index i = 0; i < batch_size; ++i) { + float x = offsets_[i].first, y = offsets_[i].second; + + // Un-normalize coordinates back to pixel space if normalized. + if (normalized_) { + x *= input_width; + y *= input_height; + } + // Un-center if coordinates are centered on the image center. + if (centered_) { + x /= 2.0f; + y /= 2.0f; + x += input_width / 2.0f; + y += input_height / 2.0f; + } + // Remove half of the glimpse window. + x -= width_ / 2.0f; + y -= height_ / 2.0f; + + const Index offset_x = (Index)x; + const Index offset_y = (Index)y; + Index glimpse_width = width_; + Index glimpse_height = height_; + bool partial_overlap = false; + DSizes slice_offset(0, offset_x, offset_y); + DSizes slice_extent(num_channels, width_, height_); + DSizes base_offset(0, 0, 0); + + if (offset_x < 0) { + slice_offset[1] = 0; + glimpse_width = (std::max)(0, width_ + offset_x); + slice_extent[1] = glimpse_width; + base_offset[1] = width_ - glimpse_width; + partial_overlap = true; + } else if (offset_x + width_ >= input_width) { + glimpse_width = (std::max)(0, input_width - offset_x); + slice_extent[1] = glimpse_width; + partial_overlap = true; + } + if (offset_y < 0) { + slice_offset[2] = 0; + glimpse_height = (std::max)(0, height_ + offset_y); + slice_extent[2] = glimpse_height; + base_offset[2] = height_ - glimpse_height; + partial_overlap = true; + } else if (offset_y + height_ >= input_height) { + glimpse_height = (std::max)(0, input_height - offset_y); + slice_extent[2] = glimpse_height; + partial_overlap = true; + } + slice_extent[1] = std::min(input_width, slice_extent[1]); + slice_extent[2] = std::min(input_height, slice_extent[2]); + + if (partial_overlap) { + if (uniform_noise_) { + // Initialize the glimpse with uniform noise. + typedef typename internal::remove_const< + typename internal::traits::Scalar>::type Scalar; + TensorFixedSize > mini; + mini.device(device) = input.template chip<3>(i).minimum(); + TensorFixedSize > range; + range.device(device) = (input.template chip<3>(i).maximum() - mini) + .template cast(); + + DSizes glimpse_size(num_channels, width_, height_); + TensorMap > tmp(NULL, glimpse_size); + output.template chip<3>(i).device(device) = + mini.reshape(Sizes<1, 1, 1>()).broadcast(glimpse_size) + + (tmp.random(unigen) * + range.reshape(Sizes<1, 1, 1>()).broadcast(glimpse_size)) + .template cast(); + } else { + // Initialize the glimpse with white noise: compute the mean and sigma + // of each channel, and use them to shape the gaussian. + DSizes glimpse_size(width_, height_); + DSizes input_size(input_width, input_height); + typedef typename internal::remove_const< + typename internal::traits::Scalar>::type Scalar; + + for (int j = 0; j < num_channels; ++j) { + TensorFixedSize > mean; + mean.device(device) = input.template chip<3>(i) + .template chip<0>(j) + .template cast() + .mean(); + TensorFixedSize > sigma; + sigma.device(device) = + (input.template chip<3>(i) + .template chip<0>(j) + .template cast() - + mean.reshape(Sizes<1, 1>()).broadcast(input_size)) + .square() + .mean() + .sqrt(); + TensorFixedSize > mini; + mini.device(device) = + input.template chip<3>(i).template chip<0>(j).minimum(); + TensorFixedSize > maxi; + maxi.device(device) = + input.template chip<3>(i).template chip<0>(j).maximum(); + + TensorMap > tmp(NULL, glimpse_size); + output.template chip<3>(i).template chip<0>(j).device(device) = + (mean.reshape(Sizes<1, 1>()).broadcast(glimpse_size) + + (tmp.random(gen) * + sigma.reshape(Sizes<1, 1>()).broadcast(glimpse_size)) + .template cast()) + .cwiseMin( + maxi.reshape(Sizes<1, 1>()).broadcast(glimpse_size)) + .cwiseMax( + mini.reshape(Sizes<1, 1>()).broadcast(glimpse_size)); + } + } + + // Copy the part of the glimpse that cover the input image if any. + if (glimpse_width == 0 || glimpse_height == 0) { + continue; + } + output.template chip<3>(i) + .slice(base_offset, slice_extent) + .device(device) = + input.template chip<3>(i).slice(slice_offset, slice_extent); + } else { + output.template chip<3>(i).device(device) = + input.template chip<3>(i).slice(slice_offset, slice_extent); + } + } + } + + private: + const Index width_; + const Index height_; + const std::vector > offsets_; + const bool normalized_; + const bool centered_; + const bool uniform_noise_; +}; +} // namespace + +template +EIGEN_ALWAYS_INLINE static const TensorCustomUnaryOp< + const GlimpseExtractionOp::Index>, + const Input> +ExtractGlimpses(const Input& input, + const typename internal::traits::Index width, + const typename internal::traits::Index height, + const std::vector >& offsets, + const bool normalized = true, const bool centered = true, + const bool uniform_noise = true) { + EIGEN_STATIC_ASSERT(internal::traits::Layout == ColMajor, + YOU_MADE_A_PROGRAMMING_MISTAKE); + EIGEN_STATIC_ASSERT(internal::traits::NumDimensions == 4, + YOU_MADE_A_PROGRAMMING_MISTAKE); + + typedef typename internal::traits::Index Index; + const GlimpseExtractionOp op(width, height, offsets, normalized, + centered, uniform_noise); + return input.customOp(op); +} + +} // end namespace Eigen + +#endif // TENSORFLOW_CORE_KERNELS_EIGEN_ATTENTION_H_ diff --git a/linux/include/tensorflow/core/kernels/eigen_backward_cuboid_convolutions.h b/linux/include/tensorflow/core/kernels/eigen_backward_cuboid_convolutions.h new file mode 100644 index 000000000..8edf7d4a2 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/eigen_backward_cuboid_convolutions.h @@ -0,0 +1,610 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_EIGEN_BACKWARD_CUBOID_CONVOLUTIONS_H_ +#define TENSORFLOW_CORE_KERNELS_EIGEN_BACKWARD_CUBOID_CONVOLUTIONS_H_ + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/kernels/eigen_volume_patch.h" + +namespace Eigen { + +/** CuboidConvolutionBackwardInput + * \ingroup CXX11_NeuralNetworks_Module + * + * \brief Computes the backprop for the input of a 3D convolution. + * + * The output_backward parameter is expected to be a tensor with a rank of 4 or + * more (channels, depth, height, width, and optionally others) + * The kernel parameter is expected to be a 5D tensor (filters, channels, + * kernel_depth, kernel_height, kernel_width) + * output_backward and kernel have to be in the same layout. + * + * The dimensions of the result will be filters, depth, height, width (and + * others if applicable). + * + * It is possible to swap the order of the depth, width and height dimensions + * provided that the same order is used in the input, the kernel, and the + * output. + * + * All dimension orders above are given for col-major, and should be reversed + * for row-major. + */ + +template +EIGEN_ALWAYS_INLINE static const typename internal::conditional< + internal::traits::Layout == ColMajor, + TensorReshapingOp< + const DSizes::Index, + internal::traits::NumDimensions>, + const TensorContractionOp< + const array< + IndexPair::Index>, 1>, + const Eigen::TensorForcedEvalOp::Index, + 2>, + const TensorShufflingOp< + const array< + typename internal::traits::Index, 5>, + const TensorReverseOp, + const Kernel>>>>, + const TensorReshapingOp< + const DSizes::Index, + 2>, + const TensorVolumePatchOp>>>, + TensorReshapingOp< + const DSizes::Index, + internal::traits::NumDimensions>, + const TensorContractionOp< + const array< + IndexPair::Index>, 1>, + const TensorReshapingOp< + const DSizes::Index, + 2>, + const TensorVolumePatchOp>, + const Eigen::TensorForcedEvalOp::Index, + 2>, + const TensorShufflingOp< + const array< + typename internal::traits::Index, 5>, + const TensorReverseOp, + const Kernel>>>>>>>::type +CuboidConvolutionBackwardInput( + const Kernel& kernel, const OutputBackward& output_backward, + typename internal::traits::Index inputPlanes, + typename internal::traits::Index inputRows, + typename internal::traits::Index inputCols, + const DenseIndex plane_stride = 1, const DenseIndex row_stride = 1, + const DenseIndex col_stride = 1) { + typedef typename internal::traits::Index TensorIndex; + const TensorRef::Scalar, + internal::traits::NumDimensions, + internal::traits::Layout, TensorIndex>> + kern(kernel); + const TensorRef< + const Tensor::Scalar, + internal::traits::NumDimensions, + internal::traits::Layout, TensorIndex>> + out(output_backward); + + EIGEN_STATIC_ASSERT(internal::traits::Layout == + internal::traits::Layout, + YOU_MADE_A_PROGRAMMING_MISTAKE); + + static const bool isColMajor = + (internal::traits::Layout == ColMajor); + + static const int NumDims = internal::traits::NumDimensions; + + // Number of filters to apply. This is the same as the output depth of the + // result + const TensorIndex kernelFilters = + isColMajor ? kern.dimensions()[0] : kern.dimensions()[4]; + // Number of channels. This is the same as the input depth. + const TensorIndex kernelChannels = + isColMajor ? kern.dimensions()[1] : kern.dimensions()[3]; + const TensorIndex kernelPlanes = + isColMajor ? kern.dimensions()[2] : kern.dimensions()[2]; + const TensorIndex kernelRows = + isColMajor ? kern.dimensions()[3] : kern.dimensions()[1]; + const TensorIndex kernelCols = + isColMajor ? kern.dimensions()[4] : kern.dimensions()[0]; + + const TensorIndex outputPlanes = + isColMajor ? out.dimensions()[1] : out.dimensions()[NumDims - 2]; + const TensorIndex outputRows = + isColMajor ? out.dimensions()[2] : out.dimensions()[NumDims - 3]; + const TensorIndex outputCols = + isColMajor ? out.dimensions()[3] : out.dimensions()[NumDims - 4]; + + // TODO(ezhulenev): Add support for inflated strides. Without inflated strides + // effective kernel planes/rows/cols are always the same as the kernel itself + // (see eigen_spatial_convolutions for details). + const TensorIndex kernelPlanesEff = kernelPlanes; + const TensorIndex kernelRowsEff = kernelRows; + const TensorIndex kernelColsEff = kernelCols; + + // Computing the forward padding. + const TensorIndex forward_pad_top_z = numext::maxi( + 0, + ((outputPlanes - 1) * plane_stride + kernelPlanesEff - inputPlanes) / 2); + const TensorIndex forward_pad_top = numext::maxi( + 0, ((outputRows - 1) * row_stride + kernelRowsEff - inputRows) / 2); + const TensorIndex forward_pad_left = numext::maxi( + 0, ((outputCols - 1) * col_stride + kernelColsEff - inputCols) / 2); + + const TensorIndex padding_top_z = kernelPlanesEff - 1 - forward_pad_top_z; + const TensorIndex padding_top = kernelRowsEff - 1 - forward_pad_top; + const TensorIndex padding_left = kernelColsEff - 1 - forward_pad_left; + + const TensorIndex padding_bottom_z = inputPlanes - + (outputPlanes - 1) * plane_stride - 2 - + padding_top_z + kernelPlanesEff; + const TensorIndex padding_bottom = inputRows - (outputRows - 1) * row_stride - + 2 - padding_top + kernelRowsEff; + const TensorIndex padding_right = inputCols - (outputCols - 1) * col_stride - + 2 - padding_left + kernelColsEff; + + eigen_assert(padding_top_z >= 0); + eigen_assert(padding_top >= 0); + eigen_assert(padding_left >= 0); + eigen_assert(padding_bottom_z >= 0); + eigen_assert(padding_bottom >= 0); + eigen_assert(padding_right >= 0); + + // The kernel has dimensions : + // filters x channels x patch_planes x patch_rows x patch_cols. + // We need to reverse the kernel along the spatial dimensions. + Eigen::array kernel_reverse; + if (isColMajor) { + kernel_reverse[0] = false; + kernel_reverse[1] = false; + kernel_reverse[2] = true; + kernel_reverse[3] = true; + kernel_reverse[4] = true; + } else { + kernel_reverse[0] = true; + kernel_reverse[1] = true; + kernel_reverse[2] = true; + kernel_reverse[3] = false; + kernel_reverse[4] = false; + } + + // Reorder the dimensions to: + // filters x patch_planes x patch_rows x patch_cols x channels + array kernel_shuffle; + if (isColMajor) { + // From: filters x channels x planes x rows x cols + // To: filters x planes x rows x cols x channels + kernel_shuffle[0] = 0; + kernel_shuffle[1] = 2; + kernel_shuffle[2] = 3; + kernel_shuffle[3] = 4; + kernel_shuffle[4] = 1; + } else { + // From: cols x rows x planes x channels x filters + // To: channels x cols x rows x planes x filters + kernel_shuffle[0] = 3; + kernel_shuffle[1] = 0; + kernel_shuffle[2] = 1; + kernel_shuffle[3] = 2; + kernel_shuffle[4] = 4; + } + + // Collapse the dims + DSizes kernel_dims; + if (isColMajor) { + kernel_dims[0] = kernelFilters * kernelPlanes * kernelRows * kernelCols; + kernel_dims[1] = kernelChannels; + } else { + kernel_dims[1] = kernelFilters * kernelPlanes * kernelRows * kernelCols; + kernel_dims[0] = kernelChannels; + } + + // The output_backward has dimensions out_depth X out_planes X out_rows X + // out_cols X OTHERS + // When we extract the image patches from output_backward, it will have + // dimensions: + // out_depth X (patch_planes * patch_rows * patch_cols) X (input_planes * + // input_rows * input_cols * OTHERS) + DSizes pre_contract_dims; + if (isColMajor) { + pre_contract_dims[0] = + kernelFilters * kernelPlanes * kernelRows * kernelCols; + pre_contract_dims[1] = inputPlanes * inputRows * inputCols; + for (int i = 4; i < NumDims; ++i) { + pre_contract_dims[1] *= out.dimension(i); + } + } else { + pre_contract_dims[1] = + kernelFilters * kernelPlanes * kernelRows * kernelCols; + pre_contract_dims[0] = inputPlanes * inputRows * inputCols; + for (int i = 0; i < NumDims - 4; ++i) { + pre_contract_dims[0] *= out.dimension(i); + } + } + + // We will contract along the collapsed dimension that contains the + // kernelFilters, kernelPlanes, kernelRows and kernelCols. + array, 1> contract_dims; + if (isColMajor) { + // col-major: kernel.contract(output.patches) + contract_dims[0] = IndexPair(0, 0); + } else { + // row-major: output.patches.contract(kernel) + contract_dims[0] = IndexPair(1, 1); + } + + // Post contraction, the dimensions of the input_backprop is + // channels X input_planes X input_rows X input_cols X OTHERS + DSizes post_contract_dims; + if (isColMajor) { + post_contract_dims[0] = kernelChannels; + post_contract_dims[1] = inputPlanes; + post_contract_dims[2] = inputRows; + post_contract_dims[3] = inputCols; + for (int i = 4; i < NumDims; ++i) { + post_contract_dims[i] = out.dimension(i); + } + } else { + post_contract_dims[NumDims - 1] = kernelChannels; + post_contract_dims[NumDims - 2] = inputPlanes; + post_contract_dims[NumDims - 3] = inputRows; + post_contract_dims[NumDims - 4] = inputCols; + for (int i = 0; i < NumDims - 4; ++i) { + post_contract_dims[i] = out.dimension(i); + } + } + + return choose( + Cond::Layout == ColMajor>(), + kernel.reverse(kernel_reverse) + .shuffle(kernel_shuffle) + .reshape(kernel_dims) + .eval() + .contract(output_backward + .extract_volume_patches( + kernelPlanes, kernelRows, kernelCols, 1, 1, 1, + plane_stride, row_stride, col_stride, padding_top_z, + padding_bottom_z, padding_top, padding_bottom, + padding_left, padding_right) + .reshape(pre_contract_dims), + contract_dims) + .reshape(post_contract_dims), + output_backward + .extract_volume_patches(kernelPlanes, kernelRows, kernelCols, 1, 1, 1, + plane_stride, row_stride, col_stride, + padding_top_z, padding_bottom_z, padding_top, + padding_bottom, padding_left, padding_right) + .reshape(pre_contract_dims) + .contract(kernel.reverse(kernel_reverse) + .shuffle(kernel_shuffle) + .reshape(kernel_dims) + .eval(), + contract_dims) + .reshape(post_contract_dims)); +} + +/** CuboidConvolutionBackwardKernel + * \ingroup CXX11_NeuralNetworks_Module + * + * \brief Computes the backprop for the filter of a 3D convolution. + * + * The output_backward parameter is expected to be a tensor with a rank of 4 or + * more (channels, depth, height, width, and optionally others) + * The kernel parameter is expected to be a 4D tensor (filters, channels, + * kernel_depth, kernel_height, kernel_width) + * output_backward and kernel have to be in the same layout. + * + * The dimensions of the result will be filters, depth, height, width (and + * others if applicable). + * + * It is possible to swap the order of the depth, width and height dimensions + * provided that the same order is used in the input, the kernel, and the + * output. + * + * All dimension orders above are given for col-major, and should be reversed + * for row-major. + */ +template +EIGEN_ALWAYS_INLINE static const typename internal::conditional< + internal::traits::Layout == ColMajor, + const TensorReverseOp< + const Eigen::array::Index, + internal::traits::NumDimensions>, + const Eigen::TensorShufflingOp< + const Eigen::array::Index, + internal::traits::NumDimensions>, + const Eigen::TensorReshapingOp< + const Eigen::DSizes::Index, + internal::traits::NumDimensions>, + const TensorContractionOp< + const array< + IndexPair::Index>, 1>, + const Eigen::TensorForcedEvalOp::Index, + 2>, + const Eigen::TensorShufflingOp< + const Eigen::array< + typename internal::traits::Index, + internal::traits::NumDimensions>, + const OutputBackward>>>, + const TensorReshapingOp< + const DSizes::Index, + 2>, + const TensorVolumePatchOp< + Dynamic, Dynamic, Dynamic, + const Eigen::TensorForcedEvalOp< + const Eigen::TensorShufflingOp< + const Eigen::array< + typename internal::traits::Index, + internal::traits::NumDimensions>, + const Input>>>>>>>>, + const TensorReverseOp< + const Eigen::array::Index, + internal::traits::NumDimensions>, + const Eigen::TensorShufflingOp< + const Eigen::array::Index, + internal::traits::NumDimensions>, + const Eigen::TensorReshapingOp< + const Eigen::DSizes::Index, + internal::traits::NumDimensions>, + const TensorContractionOp< + const array< + IndexPair::Index>, 1>, + const TensorReshapingOp< + const DSizes::Index, + 2>, + const TensorVolumePatchOp< + Dynamic, Dynamic, Dynamic, + const Eigen::TensorForcedEvalOp< + const Eigen::TensorShufflingOp< + const Eigen::array< + typename internal::traits::Index, + internal::traits::NumDimensions>, + const Input>>>>, + const Eigen::TensorForcedEvalOp::Index, + 2>, + const Eigen::TensorShufflingOp< + const Eigen::array< + typename internal::traits::Index, + internal::traits::NumDimensions>, + const OutputBackward>>>>>>>>::type +CuboidConvolutionBackwardKernel( + const Input& input, const OutputBackward& output_backward, + typename internal::traits::Index kernelPlanes, + typename internal::traits::Index kernelRows, + typename internal::traits::Index kernelCols, + const DenseIndex stridePlanes = 1, const DenseIndex strideRows = 1, + const DenseIndex strideCols = 1) { + typedef typename internal::traits::Index TensorIndex; + TensorRef::Scalar, + internal::traits::NumDimensions, + internal::traits::Layout, TensorIndex>> + in(input); + TensorRef::Scalar, + internal::traits::NumDimensions, + internal::traits::Layout, TensorIndex>> + out(output_backward); + + EIGEN_STATIC_ASSERT(internal::traits::Layout == + internal::traits::Layout, + YOU_MADE_A_PROGRAMMING_MISTAKE); + + static const bool isColMajor = (internal::traits::Layout == ColMajor); + + static const int NumDims = internal::traits::NumDimensions; + EIGEN_STATIC_ASSERT(internal::traits::NumDimensions == + internal::traits::NumDimensions, + YOU_MADE_A_PROGRAMMING_MISTAKE); + + // We do not support higher dimensional backward convolutions, or convolutions + // without batch dimension. + // TODO(ezhulenev): Relax this constraint, and turn on tests without batch + // dimension in eigen_backward_cuboid_convolutions_test.cc. + EIGEN_STATIC_ASSERT(internal::traits::NumDimensions == 5, + YOU_MADE_A_PROGRAMMING_MISTAKE); + + const TensorIndex inputPlanes = + isColMajor ? in.dimension(1) : in.dimension(NumDims - 2); + const TensorIndex inputRows = + isColMajor ? in.dimension(2) : in.dimension(NumDims - 3); + const TensorIndex inputCols = + isColMajor ? in.dimension(3) : in.dimension(NumDims - 4); + + const TensorIndex outputPlanes = + isColMajor ? out.dimension(1) : out.dimension(NumDims - 2); + const TensorIndex outputRows = + isColMajor ? out.dimension(2) : out.dimension(NumDims - 3); + const TensorIndex outputCols = + isColMajor ? out.dimension(3) : out.dimension(NumDims - 4); + + // Number of filters. This is the same as the output depth. + const TensorIndex kernelFilters = + isColMajor ? out.dimension(0) : out.dimension(NumDims - 1); + // Number of channels. This is the same as the input depth. + const TensorIndex kernelChannels = + isColMajor ? in.dimension(0) : in.dimension(NumDims - 1); + + // Number of batches in the input tensor. + const TensorIndex batch = + isColMajor ? in.dimension(4) : in.dimension(NumDims - 5); + + // TODO(ezhulenev): Add support for inflated strides. Without inflated strides + // effective kernel planes/rows/cols are always the same as the kernel itself + // (see eigen_spatial_convolutions for details). + const TensorIndex kernelPlanesEff = kernelPlanes; + const TensorIndex kernelRowsEff = kernelRows; + const TensorIndex kernelColsEff = kernelCols; + + // Compute forward padding from input and output_backward dimensions. + const TensorIndex padPlanes = numext::maxi( + 0, (outputPlanes - 1) * stridePlanes + kernelPlanesEff - inputPlanes); + const TensorIndex padRows = numext::maxi( + 0, (outputRows - 1) * strideRows + kernelRowsEff - inputRows); + const TensorIndex padCols = numext::maxi( + 0, (outputCols - 1) * strideCols + kernelColsEff - inputCols); + + const TensorIndex padding_top_z = padPlanes / 2; + const TensorIndex padding_top = padRows / 2; + const TensorIndex padding_left = padCols / 2; + + // Compute paddings for output_backward before extracting patches. + const auto expanded_out_planes = (outputPlanes - 1) * stridePlanes + 1; + const auto expanded_out_rows = (outputRows - 1) * strideRows + 1; + const auto expanded_out_cols = (outputCols - 1) * strideCols + 1; + const auto padded_out_planes = inputPlanes + kernelPlanes - 1; + const auto padded_out_rows = inputRows + kernelRows - 1; + const auto padded_out_cols = inputCols + kernelCols - 1; + const auto top_pad_planes = kernelPlanes - 1 - padding_top_z; + const auto top_pad_rows = kernelRows - 1 - padding_top; + const auto left_pad_cols = kernelCols - 1 - padding_left; + const auto bottom_pad_planes = + padded_out_planes - expanded_out_planes - top_pad_planes; + const auto bottom_pad_rows = + padded_out_rows - expanded_out_rows - top_pad_rows; + const auto right_pad_cols = + padded_out_cols - expanded_out_cols - left_pad_cols; + + // Reorder output_backward dimensions. + array output_backward_shuffle; + if (isColMajor) { + // From: [out_depth, out_planes, out_rows, out_cols, batch] + // To: [batch, out_planes, out_rows, out_cols, out_depth] + output_backward_shuffle = {4, 1, 2, 3, 0}; + } else { + // From: [batch, out_cols, out_rows, out_planes, out_depth] + // To: [out_depth, out_cols, out_rows, out_planes, batch] + output_backward_shuffle = {4, 1, 2, 3, 0}; + } + + // Reorder input dimensions. + array input_shuffle; + if (isColMajor) { + // From: [in_depth, in_planes, in_rows, in_cols, batch] + // To: [in_depth, batch, in_planes, in_rows, in_cols] + input_shuffle = {0, 4, 1, 2, 3}; + } else { + // From: [batch, in_cols, in_rows, in_planes, in_depth] + // To: [in_cols, in_rows, in_planes, batch, in_depth] + input_shuffle = {1, 2, 3, 0, 4}; + } + + // Input is playing the role of a "kernel" in this convolution. + DSizes input_dims; + if (isColMajor) { + input_dims[0] = kernelChannels; + input_dims[1] = batch * inputPlanes * inputRows * inputCols; + } else { + input_dims[1] = kernelChannels; + input_dims[0] = inputCols * inputRows * inputPlanes * batch; + } + + // Molds the output of the patch extraction result into a 2D tensor: + // - the first dimension (dims[0]): the patch values to be multiplied with the + // kernels + // - the second dimension (dims[1]): everything else + DSizes pre_contract_dims; + if (isColMajor) { + pre_contract_dims[0] = batch * inputPlanes * inputRows * inputCols; + pre_contract_dims[1] = + kernelPlanes * kernelRows * kernelCols * kernelFilters; + } else { + pre_contract_dims[1] = inputCols * inputRows * inputPlanes * batch; + pre_contract_dims[0] = + kernelFilters * kernelCols * kernelRows * kernelPlanes; + } + + // We will contract along the collapsed dimension that contains the + // batch, inputPlanes, inputRows and inputCols. + array, 1> contract_dims; + contract_dims[0] = IndexPair(1, 0); + + // Dimensions after contraction. + DSizes post_contract_dims; + if (isColMajor) { + post_contract_dims[0] = kernelChannels; + post_contract_dims[1] = kernelPlanes; + post_contract_dims[2] = kernelRows; + post_contract_dims[3] = kernelCols; + post_contract_dims[4] = kernelFilters; + } else { + post_contract_dims[0] = kernelFilters; + post_contract_dims[1] = kernelCols; + post_contract_dims[2] = kernelRows; + post_contract_dims[3] = kernelPlanes; + post_contract_dims[4] = kernelChannels; + } + + // Reorder output of contraction to valid filter shape. + array kernel_shuffle; + if (isColMajor) { + // From: [in_depth, kernel_planes, kernel_rows, kernel_cols, out_depth] + // To: [out_depth, in_depth, kernel_planes, kernel_rows, kernel_cols] + kernel_shuffle = {4, 0, 1, 2, 3}; + } else { + // From: [out_depth, kernel_cols, kernel_rows, kernel_planes, in_depth] + // To: [kernel_cols, kernel_rows, kernel_planes, in_depth, out_depth] + kernel_shuffle = {1, 2, 3, 4, 0}; + } + + // Reverse kernel backprop dimensions. + array kernel_reverse; + if (isColMajor) { + kernel_reverse = {false, false, true, true, true}; + } else { + kernel_reverse = {true, true, true, false, false}; + } + + // Create convolution input (aka source of patches) from output backward + // tensor by shuffling dimensions. + const auto the_input = + output_backward.shuffle(output_backward_shuffle).eval(); + + // Create convolution kernel (aka filter) from input by shuffling and + // reshaping. + const auto the_kernel = + input.shuffle(input_shuffle).reshape(input_dims).eval(); + + return choose(Cond::Layout == ColMajor>(), + the_kernel.contract( + the_input + .extract_volume_patches( + inputPlanes, inputRows, inputCols, 1, 1, 1, + stridePlanes, strideRows, strideCols, + top_pad_planes, bottom_pad_planes, top_pad_rows, + bottom_pad_rows, left_pad_cols, right_pad_cols) + .reshape(pre_contract_dims), + contract_dims), + the_input + .extract_volume_patches( + inputPlanes, inputRows, inputCols, 1, 1, 1, + stridePlanes, strideRows, strideCols, top_pad_planes, + bottom_pad_planes, top_pad_rows, bottom_pad_rows, + left_pad_cols, right_pad_cols) + .reshape(pre_contract_dims) + .contract(the_kernel, contract_dims)) + .reshape(post_contract_dims) + .shuffle(kernel_shuffle) + .reverse(kernel_reverse); +} + +} // end namespace Eigen + +#endif // TENSORFLOW_CORE_KERNELS_EIGEN_BACKWARD_CUBOID_CONVOLUTIONS_H_ diff --git a/linux/include/tensorflow/core/kernels/eigen_backward_spatial_convolutions.h b/linux/include/tensorflow/core/kernels/eigen_backward_spatial_convolutions.h new file mode 100644 index 000000000..960920c55 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/eigen_backward_spatial_convolutions.h @@ -0,0 +1,502 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_EIGEN_BACKWARD_SPATIAL_CONVOLUTIONS_H_ +#define TENSORFLOW_CORE_KERNELS_EIGEN_BACKWARD_SPATIAL_CONVOLUTIONS_H_ + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" + +namespace Eigen { + +/** SpatialConvolutionBackwardInput + * \ingroup CXX11_NeuralNetworks_Module + * + * \brief Computes the backprop for the input of a 2D convolution. + * + * The output_backward parameter is expected to be a tensor with a rank of 3 or + * more (channels, height, width, and optionally others) + * The kernel parameter is expected to be a 4D tensor (filters, channels, + * kernel_height, kernel_width) + * The output_backward and the kernel must both be in col-major layout. The + * result will also be in col-major layout. + * + * If row_in_stride, col_in_stride > 1, then applies convolution with holes + * (aka atrous convolution), sampling every row_in_stride, col_in_stride input + * pixels. + * + * The result can be assigned to a tensor of rank equal to the rank of the + * output_backward. The dimensions of the result will be filters, height, width + * (and others if applicable). + * + * It is possible to swap the order of the width and height dimensions provided + * that the same order is used in the input, the kernel, and the output. + * + */ +#ifdef EIGEN_HAS_INDEX_LIST +typedef IndexList, type2index<0>, type2index<1>, type2index<1> > + ReverseColMajor; +typedef IndexList, type2index<1>, type2index<0>, type2index<0> > + ReverseRowMajor; +#else +typedef array ReverseColMajor; +typedef array ReverseRowMajor; +#endif + +template +EIGEN_ALWAYS_INLINE static const typename internal::conditional< + internal::traits::Layout == ColMajor, + TensorReshapingOp< + const DSizes::Index, + internal::traits::NumDimensions>, + const TensorContractionOp< + const array< + IndexPair::Index>, 1>, + const Eigen::TensorForcedEvalOp::Index, + 2>, + const TensorShufflingOp< + const array< + typename internal::traits::Index, 4>, + const TensorReverseOp > > >, + const TensorReshapingOp< + const DSizes::Index, + 2>, + const TensorImagePatchOp > > >, + TensorReshapingOp< + const DSizes::Index, + internal::traits::NumDimensions>, + const TensorContractionOp< + const array< + IndexPair::Index>, 1>, + const TensorReshapingOp< + const DSizes::Index, + 2>, + const TensorImagePatchOp >, + const Eigen::TensorForcedEvalOp::Index, + 2>, + const TensorShufflingOp< + const array< + typename internal::traits::Index, 4>, + const TensorReverseOp > > > > > >::type +SpatialConvolutionBackwardInput( + const Kernel& kernel, const OutputBackward& output_backward, + typename internal::traits::Index inputRows, + typename internal::traits::Index inputCols, + const DenseIndex row_stride = 1, const DenseIndex col_stride = 1, + const DenseIndex row_in_stride = 1, const DenseIndex col_in_stride = 1) { + typedef typename internal::traits::Index TensorIndex; + typedef typename internal::traits::Scalar OutScalar; + TensorRef::Scalar, + internal::traits::NumDimensions, + internal::traits::Layout, TensorIndex> > + kern(kernel); + TensorRef::NumDimensions, + internal::traits::Layout, TensorIndex> > + out(output_backward); + + EIGEN_STATIC_ASSERT(internal::traits::Layout == + internal::traits::Layout, + YOU_MADE_A_PROGRAMMING_MISTAKE); + + static const bool isColMajor = + (internal::traits::Layout == ColMajor); + + static const int NumDims = internal::traits::NumDimensions; + + // Number of filters to apply. This is the same as the output depth of the + // result + const TensorIndex kernelFilters = + isColMajor ? kern.dimensions()[0] : kern.dimensions()[3]; + // Number of channels. This is the same as the input depth. + const TensorIndex kernelChannels = + isColMajor ? kern.dimensions()[1] : kern.dimensions()[2]; + const TensorIndex kernelRows = + isColMajor ? kern.dimensions()[2] : kern.dimensions()[1]; + const TensorIndex kernelCols = + isColMajor ? kern.dimensions()[3] : kern.dimensions()[0]; + + // This is the effective kernel size, taking into account the (*_in_stride - + // 1) zero-values + // inserted between consecutive kernel elements in atrous convolution + const TensorIndex kernelRowsEff = + kernelRows + (kernelRows - 1) * (row_in_stride - 1); + const TensorIndex kernelColsEff = + kernelCols + (kernelCols - 1) * (col_in_stride - 1); + + const TensorIndex outputRows = isColMajor + ? output_backward.dimension(1) + : output_backward.dimension(NumDims - 2); + const TensorIndex outputCols = isColMajor + ? output_backward.dimension(2) + : output_backward.dimension(NumDims - 3); + + // Computing the forward padding + const TensorIndex forward_pad_top = numext::maxi( + 0, ((outputRows - 1) * row_stride + kernelRowsEff - inputRows) / 2); + const TensorIndex forward_pad_left = numext::maxi( + 0, ((outputCols - 1) * col_stride + kernelColsEff - inputCols) / 2); + const TensorIndex padding_top = kernelRowsEff - 1 - forward_pad_top; + const TensorIndex padding_left = kernelColsEff - 1 - forward_pad_left; + + const TensorIndex padding_bottom = inputRows - (outputRows - 1) * row_stride - + 2 - padding_top + kernelRowsEff; + const TensorIndex padding_right = inputCols - (outputCols - 1) * col_stride - + 2 - padding_left + kernelColsEff; + + eigen_assert(padding_top >= 0); + eigen_assert(padding_left >= 0); + eigen_assert(padding_bottom >= 0); + eigen_assert(padding_right >= 0); + + // The kernel has dimensions filters X channels X patch_rows X patch_cols + // We need to reverse the kernel along dimensions corresponding to rows and + // cols. + // TODO(yangke): we can make things slightly faster by collapsing the + // dimensions + // where we don't reverse. Try that once we have a faster compiler. + typedef typename internal::conditional::type Reverse; + Reverse kernel_reverse; + +#ifndef EIGEN_HAS_INDEX_LIST + if (isColMajor) { + kernel_reverse[0] = false; + kernel_reverse[1] = false; + kernel_reverse[2] = true; + kernel_reverse[3] = true; + } else { + kernel_reverse[0] = true; + kernel_reverse[1] = true; + kernel_reverse[2] = false; + kernel_reverse[3] = false; + } +#endif + + // Reorder the dimensions to: + // filters x patch_rows x patch_cols x channels + array kernel_shuffle; + if (isColMajor) { + // From: filters x channels x rows x cols + // To: filters x rows x cols x channels + kernel_shuffle[0] = 0; + kernel_shuffle[1] = 2; + kernel_shuffle[2] = 3; + kernel_shuffle[3] = 1; + } else { + // From: cols x rows x channels x filters + // To: channels x cols x rows x filters + kernel_shuffle[0] = 2; + kernel_shuffle[1] = 0; + kernel_shuffle[2] = 1; + kernel_shuffle[3] = 3; + } + + // Collapse the dims + DSizes kernel_dims; + if (isColMajor) { + kernel_dims[0] = kernelFilters * kernelRows * kernelCols; + kernel_dims[1] = kernelChannels; + } else { + kernel_dims[1] = kernelFilters * kernelRows * kernelCols; + kernel_dims[0] = kernelChannels; + } + + // The output_backward has dimensions out_depth X out_rows X out_cols X OTHERS + // When we extract the image patches from output_backward, it will have + // dimensions + // out_depth X (patch_rows * patch_cols) X (input_rows * input_cols * + // OTHERS) + DSizes pre_contract_dims; + if (isColMajor) { + pre_contract_dims[0] = kernelFilters * kernelRows * kernelCols; + pre_contract_dims[1] = inputRows * inputCols; + for (int i = 3; i < NumDims; ++i) { + pre_contract_dims[1] *= out.dimension(i); + } + } else { + pre_contract_dims[1] = kernelFilters * kernelRows * kernelCols; + pre_contract_dims[0] = inputRows * inputCols; + for (int i = 0; i < NumDims - 3; ++i) { + pre_contract_dims[0] *= out.dimension(i); + } + } + + // We will contract along the collapsed dimension that contains the + // kernelFilters, the kernelRows and the kernelCols. + array, 1> contract_dims; + if (isColMajor) { + // col-major: kernel.contract(output.patches) + contract_dims[0] = IndexPair(0, 0); + } else { + // row-major: output.patches.contract(kernel) + contract_dims[0] = IndexPair(1, 1); + } + + // Post contraction, the dimensions of the input_backprop is + // channels X input_rows X input_cols X OTHERS + DSizes post_contract_dims; + if (isColMajor) { + post_contract_dims[0] = kernelChannels; + post_contract_dims[1] = inputRows; + post_contract_dims[2] = inputCols; + for (int i = 3; i < NumDims; ++i) { + post_contract_dims[i] = out.dimension(i); + } + } else { + post_contract_dims[NumDims - 1] = kernelChannels; + post_contract_dims[NumDims - 2] = inputRows; + post_contract_dims[NumDims - 3] = inputCols; + for (int i = 0; i < NumDims - 3; ++i) { + post_contract_dims[i] = out.dimension(i); + } + } + + return choose( + Cond::Layout == ColMajor>(), + kernel.reverse(kernel_reverse) + .shuffle(kernel_shuffle) + .reshape(kernel_dims) + .eval() + .contract( + output_backward + .extract_image_patches( + kernelRows, kernelCols, 1, 1, row_in_stride, + col_in_stride, row_stride, col_stride, padding_top, + padding_bottom, padding_left, padding_right, OutScalar(0)) + .reshape(pre_contract_dims), + contract_dims) + .reshape(post_contract_dims), + output_backward + .extract_image_patches(kernelRows, kernelCols, 1, 1, row_in_stride, + col_in_stride, row_stride, col_stride, + padding_top, padding_bottom, padding_left, + padding_right, OutScalar(0)) + .reshape(pre_contract_dims) + .contract(kernel.reverse(kernel_reverse) + .shuffle(kernel_shuffle) + .reshape(kernel_dims) + .eval(), + contract_dims) + .reshape(post_contract_dims)); +} + +/** SpatialConvolutionBackwardKernel + * \ingroup CXX11_NeuralNetworks_Module + * + * \brief Computes the backprop for the filter of a 2D convolution. + * + * The output_backward parameter is expected to be a tensor with a rank of 3 or + * more (channels, height, width, and optionally others) + * The kernel parameter is expected to be a 4D tensor (filters, channels, + * kernel_height, kernel_width) + * The output_backward and the kernel must both be in col-major layout. The + * result will also be in col-major layout. + * + * If row_in_stride, col_stride > 1, then applies convolution with holes (aka + * atrous convolution), sampling every row_in_stride, col_in_stride input + * pixels. + * + * The result can be assigned to a tensor of rank equal to the rank of the + * output_backward. The dimensions of the result will be filters, height, width + * (and others if applicable). + * + * It is possible to swap the order of the width and height dimensions provided + * that the same order is used in the input, the kernel, and the output. + * + */ + +template +EIGEN_ALWAYS_INLINE static const typename internal::conditional< + internal::traits::Layout == ColMajor, + TensorReshapingOp< + const DSizes::Index, 4>, + const TensorContractionOp< + const array::Index>, 1>, + const TensorReshapingOp< + const DSizes::Index, 2>, + const OutputBackward>, + const TensorReshapingOp< + const DSizes::Index, 2>, + const TensorImagePatchOp > > >, + TensorReshapingOp< + const DSizes::Index, 4>, + const TensorContractionOp< + const array::Index>, 1>, + const TensorReshapingOp< + const DSizes::Index, 2>, + const TensorImagePatchOp >, + const TensorReshapingOp< + const DSizes::Index, 2>, + const OutputBackward> > > >::type +SpatialConvolutionBackwardKernel( + const Input& input, const OutputBackward& output_backward, + typename internal::traits::Index kernelRows, + typename internal::traits::Index kernelCols, + const DenseIndex row_stride = 1, const DenseIndex col_stride = 1, + const DenseIndex row_in_stride = 1, const DenseIndex col_in_stride = 1) { + typedef typename internal::traits::Index TensorIndex; + typedef typename internal::traits::Scalar OutScalar; + TensorRef::Scalar, + internal::traits::NumDimensions, + internal::traits::Layout, TensorIndex> > + in(input); + TensorRef::NumDimensions, + internal::traits::Layout, TensorIndex> > + out(output_backward); + + EIGEN_STATIC_ASSERT(internal::traits::Layout == + internal::traits::Layout, + YOU_MADE_A_PROGRAMMING_MISTAKE); + + // stride and in_stride cannot both be larger than 1 + eigen_assert(!(row_stride > 1 && row_in_stride > 1) && + !(col_stride > 1 && col_in_stride > 1)); + + static const bool isColMajor = (internal::traits::Layout == ColMajor); + + static const int NumDims = internal::traits::NumDimensions; + EIGEN_STATIC_ASSERT(internal::traits::NumDimensions == + internal::traits::NumDimensions, + YOU_MADE_A_PROGRAMMING_MISTAKE); + + const TensorIndex inputRows = + isColMajor ? in.dimension(1) : in.dimension(NumDims - 2); + const TensorIndex inputCols = + isColMajor ? in.dimension(2) : in.dimension(NumDims - 3); + + const TensorIndex outputRows = isColMajor + ? output_backward.dimension(1) + : output_backward.dimension(NumDims - 2); + const TensorIndex outputCols = isColMajor + ? output_backward.dimension(2) + : output_backward.dimension(NumDims - 3); + + // Number of filters to apply. This is the same as the output depth of the + // result + const TensorIndex kernelFilters = + isColMajor ? out.dimensions()[0] : out.dimensions()[NumDims - 1]; + + // Number of channels. This is the same as the input depth. + const TensorIndex kernelChannels = + isColMajor ? in.dimensions()[0] : in.dimensions()[NumDims - 1]; + + // This is the effective kernel size, taking into account the (*_in_stride - + // 1) zero-values + // inserted between consecutive kernel elements in atrous convolution + const TensorIndex kernelRowsEff = + kernelRows + (kernelRows - 1) * (row_in_stride - 1); + const TensorIndex kernelColsEff = + kernelCols + (kernelCols - 1) * (col_in_stride - 1); + + // Computing the forward padding + const TensorIndex padRows = numext::maxi( + 0, (outputRows - 1) * row_stride + kernelRowsEff - inputRows); + const TensorIndex padCols = numext::maxi( + 0, (outputCols - 1) * col_stride + kernelColsEff - inputCols); + const TensorIndex padding_top = padRows / 2; + const TensorIndex padding_bottom = padRows - padding_top; + const TensorIndex padding_left = padCols / 2; + const TensorIndex padding_right = padCols - padding_left; + + // Reshaped out + DSizes output_dims; + if (isColMajor) { + output_dims[0] = kernelFilters; + output_dims[1] = outputRows * outputCols; + for (int i = 3; i < NumDims; ++i) { + output_dims[1] *= out.dimension(i); + } + } else { + output_dims[1] = kernelFilters; + output_dims[0] = outputCols * outputRows; + for (int i = 0; i < NumDims - 3; ++i) { + output_dims[0] *= out.dimension(i); + } + } + + // Reshaped extract_image_patches(in) + DSizes pre_contract_dims; + if (isColMajor) { + pre_contract_dims[0] = kernelChannels * kernelRows * kernelCols; + pre_contract_dims[1] = outputRows * outputCols; + for (int i = 3; i < NumDims; ++i) { + pre_contract_dims[1] *= in.dimension(i); + } + eigen_assert(output_dims[1] == pre_contract_dims[1]); + } else { + pre_contract_dims[1] = kernelCols * kernelRows * kernelChannels; + pre_contract_dims[0] = outputRows * outputCols; + for (int i = 0; i < NumDims - 3; ++i) { + pre_contract_dims[0] *= in.dimension(i); + } + eigen_assert(output_dims[0] == pre_contract_dims[0]); + } + + // We will contract along the collapsed dimension that contains the + // outputCols, outputRows and OTHERS. + array, 1> contract_dims; + if (isColMajor) { + // col-major: output_backward.contract(input.patches) + contract_dims[0] = IndexPair(1, 1); + } else { + // row-major: input.patches.contract(output_backward) + contract_dims[0] = IndexPair(0, 0); + } + + // After the contraction, the kernel will have the desired shape + // out_depth X in_shape X kernel_rows X kernel_cols + DSizes kernel_dims; + if (isColMajor) { + kernel_dims[0] = kernelFilters; + kernel_dims[1] = kernelChannels; + kernel_dims[2] = kernelRows; + kernel_dims[3] = kernelCols; + } else { + kernel_dims[3] = kernelFilters; + kernel_dims[2] = kernelChannels; + kernel_dims[1] = kernelRows; + kernel_dims[0] = kernelCols; + } + + return choose( + Cond::Layout == ColMajor>(), + output_backward.reshape(output_dims) + .contract( + input + .extract_image_patches( + kernelRows, kernelCols, row_stride, col_stride, + row_in_stride, col_in_stride, 1, 1, padding_top, + padding_bottom, padding_left, padding_right, OutScalar(0)) + .reshape(pre_contract_dims), + contract_dims) + .reshape(kernel_dims), + input + .extract_image_patches(kernelRows, kernelCols, row_stride, col_stride, + row_in_stride, col_in_stride, 1, 1, + padding_top, padding_bottom, padding_left, + padding_right, OutScalar(0)) + .reshape(pre_contract_dims) + .contract(output_backward.reshape(output_dims), contract_dims) + .reshape(kernel_dims)); +} + +} // end namespace Eigen + +#endif // TENSORFLOW_CORE_KERNELS_EIGEN_BACKWARD_SPATIAL_CONVOLUTIONS_H_ diff --git a/linux/include/tensorflow/core/kernels/eigen_cuboid_convolution.h b/linux/include/tensorflow/core/kernels/eigen_cuboid_convolution.h new file mode 100644 index 000000000..6a9a2accd --- /dev/null +++ b/linux/include/tensorflow/core/kernels/eigen_cuboid_convolution.h @@ -0,0 +1,1633 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_EIGEN_CUBOID_CONVOLUTION_H_ +#define TENSORFLOW_CORE_KERNELS_EIGEN_CUBOID_CONVOLUTION_H_ + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/kernels/eigen_volume_patch.h" + +namespace Eigen { + +namespace internal { + +// WARNING: Most of the code here implicitly assumes that the matrix is in +// ColMajor layout. This is guaranteed by the tensor contraction (see +// TensorContraction.h). +// +// Inside Eigen a tensor contraction is represented by a matrix multiplication. +// We don't want to actually extract volume patches and reshape the result into +// a matrix (this involves allocating huge extra memory), so the patch +// extraction and reshape operations are implicit. +// +// TensorContractionInputMapper takes a matrix index and returns the coefficient +// (or the packet) of the "virtual tensor", that would be at that index if we +// were to actually reshape the result of patch extraction. +// +// TensorContractionSubMapper provides a similar view into the "virtual matrix" +// at the given vertical and horizontal offsets. +// +// "Virtual matrix" dimensions: +// *0: kernelChannels * kernelPlanes * kernelRows * kernelCols +// 1: out_planes * out_height * out_width * OTHERS (e.g batches, etc...) +// +// *) extracted patches are continuous in memory (innermost dimension assuming +// col major layout) +// +// With this dimensions: +// row - offset within a single patch (in code: patchId) +// col - index of the extracted patch (in code: patchIndex) +// patchIndex ∈ [0..num_patches * OTHERS] (batch and other dimensions) +// +template +class TensorContractionInputMapper< + Scalar_, Index, Side, + TensorEvaluator >, + Device>, + nocontract_t, contract_t, packet_size, inner_dim_contiguous, + inner_dim_reordered, Alignment> { + public: + typedef Scalar_ Scalar; + typedef TensorContractionInputMapper< + Scalar, Index, Side, + TensorEvaluator >, + Device>, + nocontract_t, contract_t, packet_size, inner_dim_contiguous, + inner_dim_reordered, Alignment> + Self; + typedef TensorContractionSubMapper< + Scalar, Index, Side, + TensorEvaluator >, + Device>, + nocontract_t, contract_t, packet_size, inner_dim_contiguous, + inner_dim_reordered, Alignment> + SubMapper; + typedef SubMapper VectorMapper; + typedef SubMapper LinearMapper; + typedef typename packet_traits::type Packet; + + EIGEN_DEVICE_FUNC + TensorContractionInputMapper( + const TensorEvaluator< + const TensorReshapingOp< + NewDimension, + const TensorVolumePatchOp >, + Device>& tensor, + const nocontract_t&, const nocontract_t&, const contract_t&, + const contract_t&) + : m_impl(tensor.impl().impl()) { + if (internal::traits::Layout == ColMajor) { + m_patch_depth = tensor.impl().dimensions()[0]; + m_patch_planes = tensor.impl().dimensions()[1]; + m_patch_rows = tensor.impl().dimensions()[2]; + m_patch_cols = tensor.impl().dimensions()[3]; + m_num_patches = tensor.impl().dimensions()[4]; + } else { + const int NumDims = tensor.impl().dimensions().size(); + m_patch_depth = tensor.impl().dimensions()[NumDims - 1]; + m_patch_planes = tensor.impl().dimensions()[NumDims - 2]; + m_patch_rows = tensor.impl().dimensions()[NumDims - 3]; + m_patch_cols = tensor.impl().dimensions()[NumDims - 4]; + m_num_patches = tensor.impl().dimensions()[NumDims - 5]; + } + + // Strides for navigating through the single patch. + m_patch_plane_stride = m_patch_depth; + m_patch_row_stride = m_patch_planes * m_patch_plane_stride; + m_patch_col_stride = m_patch_rows * m_patch_row_stride; + + // Strides for the output tensor. + // IMPORTANT: These strides are used to locate an element in a patch at a + // depth zero (channel), which is not quite the same as "traditional" + // stride. + m_rowStride = m_patch_planes; + m_colStride = m_patch_rows * m_rowStride; + m_patchStride = m_colStride * m_patch_cols * m_patch_depth; + m_otherStride = m_patchStride * m_num_patches; + + m_outputPlanes = tensor.impl().outputPlanes(); + m_outputRows = tensor.impl().outputRows(); + m_outputCols = tensor.impl().outputCols(); + + m_outputPlanesRows = m_outputPlanes * m_outputRows; + + m_plane_strides = tensor.impl().userPlaneStride(); + m_row_strides = tensor.impl().userRowStride(); + m_col_strides = tensor.impl().userColStride(); + + m_in_plane_strides = tensor.impl().userInPlaneStride(); + m_in_row_strides = tensor.impl().userInRowStride(); + m_in_col_strides = tensor.impl().userInColStride(); + + m_patch_plane_inflate_strides = tensor.impl().planeInflateStride(); + m_patch_row_inflate_strides = tensor.impl().rowInflateStride(); + m_patch_col_inflate_strides = tensor.impl().colInflateStride(); + + if (internal::traits::Layout == ColMajor) { + m_inputDepth = tensor.impl().impl().dimensions()[0]; + m_inputPlanes = tensor.impl().impl().dimensions()[1]; + m_inputRows = tensor.impl().impl().dimensions()[2]; + m_inputCols = tensor.impl().impl().dimensions()[3]; + } else { + const int NumDims = tensor.impl().impl().dimensions().size(); + m_inputDepth = tensor.impl().impl().dimensions()[NumDims - 1]; + m_inputPlanes = tensor.impl().impl().dimensions()[NumDims - 2]; + m_inputRows = tensor.impl().impl().dimensions()[NumDims - 3]; + m_inputCols = tensor.impl().impl().dimensions()[NumDims - 4]; + } + + // Strides for navigating through the input tensor. + m_planeInputStride = m_inputDepth; + m_rowInputStride = m_inputDepth * m_inputPlanes; + m_colInputStride = m_inputDepth * m_inputRows * m_inputPlanes; + m_patchInputStride = + m_inputDepth * m_inputRows * m_inputCols * m_inputPlanes; + + m_planePaddingTop = tensor.impl().planePaddingTop(); + m_rowPaddingTop = tensor.impl().rowPaddingTop(); + m_colPaddingLeft = tensor.impl().colPaddingLeft(); + + m_fastNumPatches = internal::TensorIntDivisor(m_num_patches); + + m_fastPatchPlaneStride = + internal::TensorIntDivisor(m_patch_plane_stride); + m_fastPatchRowStride = + internal::TensorIntDivisor(m_patch_row_stride); + m_fastPatchColStride = + internal::TensorIntDivisor(m_patch_col_stride); + + m_fastInputPlaneStride = + internal::TensorIntDivisor(m_patch_plane_inflate_strides); + m_fastInputRowStride = + internal::TensorIntDivisor(m_patch_row_inflate_strides); + m_fastInputColStride = + internal::TensorIntDivisor(m_patch_col_inflate_strides); + + m_fastRowStride = internal::TensorIntDivisor(m_rowStride); + m_fastColStride = internal::TensorIntDivisor(m_colStride); + + m_fastDimZero = internal::TensorIntDivisor(m_patch_depth); + m_fastOutputRows = internal::TensorIntDivisor(m_outputRows); + m_fastOutputPlanes = internal::TensorIntDivisor(m_outputPlanes); + m_fastOutputRows = internal::TensorIntDivisor(m_outputRows); + m_fastOutputCols = internal::TensorIntDivisor(m_outputCols); + + m_fastOutputPlanesRows = + internal::TensorIntDivisor(m_outputPlanesRows); + } + + EIGEN_DEVICE_FUNC + TensorContractionInputMapper(const TensorContractionInputMapper& base_mapper) + : m_impl(base_mapper.m_impl) { + m_patch_depth = base_mapper.m_patch_depth; + m_patch_planes = base_mapper.m_patch_planes; + m_patch_rows = base_mapper.m_patch_rows; + m_patch_cols = base_mapper.m_patch_cols; + m_num_patches = base_mapper.m_num_patches; + + m_patch_plane_stride = base_mapper.m_patch_plane_stride; + m_patch_row_stride = base_mapper.m_patch_row_stride; + m_patch_col_stride = base_mapper.m_patch_col_stride; + + m_rowStride = base_mapper.m_rowStride; + m_colStride = base_mapper.m_colStride; + m_patchStride = base_mapper.m_patchStride; + m_otherStride = base_mapper.m_otherStride; + + m_planeInputStride = base_mapper.m_planeInputStride; + m_rowInputStride = base_mapper.m_rowInputStride; + m_colInputStride = base_mapper.m_colInputStride; + m_patchInputStride = base_mapper.m_patchInputStride; + m_otherInputStride = base_mapper.m_otherInputStride; + + m_inputDepth = base_mapper.m_inputDepth; + m_inputPlanes = base_mapper.m_inputPlanes; + m_inputRows = base_mapper.m_inputRows; + m_inputCols = base_mapper.m_inputCols; + + m_outputPlanes = base_mapper.m_outputPlanes; + m_outputRows = base_mapper.m_outputRows; + m_outputCols = base_mapper.m_outputCols; + + m_plane_strides = base_mapper.m_plane_strides; + m_row_strides = base_mapper.m_row_strides; + m_col_strides = base_mapper.m_col_strides; + + m_in_plane_strides = base_mapper.m_in_plane_strides; + m_in_row_strides = base_mapper.m_in_row_strides; + m_in_col_strides = base_mapper.m_in_col_strides; + + m_patch_plane_inflate_strides = base_mapper.m_patch_plane_inflate_strides; + m_patch_row_inflate_strides = base_mapper.m_patch_row_inflate_strides; + m_patch_col_inflate_strides = base_mapper.m_patch_col_inflate_strides; + + m_planePaddingTop = base_mapper.m_planePaddingTop; + m_rowPaddingTop = base_mapper.m_rowPaddingTop; + m_colPaddingLeft = base_mapper.m_colPaddingLeft; + + m_outputPlanesRows = base_mapper.m_outputPlanesRows; + + m_fastNumPatches = base_mapper.m_fastNumPatches; + m_fastPatchPlaneStride = base_mapper.m_fastPatchPlaneStride; + m_fastPatchRowStride = base_mapper.m_fastPatchRowStride; + m_fastPatchColStride = base_mapper.m_fastPatchColStride; + m_fastInputPlaneStride = base_mapper.m_fastInputPlaneStride; + m_fastInputRowStride = base_mapper.m_fastInputRowStride; + m_fastInputColStride = base_mapper.m_fastInputColStride; + m_fastRowStride = base_mapper.m_fastRowStride; + m_fastColStride = base_mapper.m_fastColStride; + m_fastOutputPlanes = base_mapper.m_fastOutputPlanes; + m_fastOutputRows = base_mapper.m_fastOutputRows; + m_fastOutputCols = base_mapper.m_fastOutputCols; + m_fastDimZero = base_mapper.m_fastDimZero; + m_fastOutputPlanesRows = base_mapper.m_fastOutputPlanesRows; + } + + // If true, turns off some optimizations for loading packets since the image + // patches are "non-standard" such as there are non-trivial strides or + // inflations in the input. + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE bool nonStandardPatches() const { + return m_in_plane_strides != 1 || m_in_row_strides != 1 || + m_in_col_strides != 1 || m_patch_plane_inflate_strides != 1 || + m_patch_row_inflate_strides != 1 || m_patch_col_inflate_strides != 1; + } + + EIGEN_DEVICE_FUNC + EIGEN_STRONG_INLINE SubMapper getSubMapper(Index i, Index j) const { + return SubMapper(*this, i, j); + } + + EIGEN_DEVICE_FUNC + EIGEN_STRONG_INLINE LinearMapper getLinearMapper(Index i, Index j) const { + return LinearMapper(*this, i, j); + } + + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE Scalar operator()(Index row) const { + Index planeIndex, rowIndex, colIndex, otherIndex; + computeBaseIndices(0, planeIndex, rowIndex, colIndex, otherIndex); + return loadCoeff(row, planeIndex, rowIndex, colIndex, otherIndex); + } + + // Load the coefficient at the patchIndex location instead of the usual + // m_rowIndex, m_colIndex, m_otherIndex. This is currently only used by the + // gpu code. + EIGEN_DEVICE_FUNC + EIGEN_STRONG_INLINE Scalar operator()(Index row, Index patchIndex) const { + Index planeIndex, rowIndex, colIndex, otherIndex; + computeBaseIndices(patchIndex, planeIndex, rowIndex, colIndex, otherIndex); + return loadCoeff(row, planeIndex, rowIndex, colIndex, otherIndex); + } + + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE Packet loadPacket(Index row) const { + Index planeIndex, rowIndex, colIndex, otherIndex; + computeBaseIndices(0, planeIndex, rowIndex, colIndex, otherIndex); + return loadPacket(row, planeIndex, rowIndex, colIndex, otherIndex); + } + + // Load the packet at the patchIndex location instead of the usual m_rowIndex, + // m_colIndex, m_otherIndex. This is currently only used by the gpu code. + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE Packet loadPacket(Index row, Index patchIndex) const { + Index planeIndex, rowIndex, colIndex, otherIndex; + computeBaseIndices(patchIndex, planeIndex, rowIndex, colIndex, otherIndex); + return loadPacket(row, planeIndex, rowIndex, colIndex, otherIndex); + } + + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE const TensorEvaluator& impl() const { + return m_impl; + } + + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE Index patchDepth() const { return m_planeInputStride; } + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE Index patchPlanes() const { return m_rowStride; } + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE Index patchRows() const { return m_patch_rows; } + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE Index patchCols() const { return m_patch_cols; } + + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE Packet packetNoPadding(const Index depth, + const Index baseIndex) const { + const Index inputIndex = depth + baseIndex; + return m_impl.template packet(inputIndex); + } + + private: + friend class TensorContractionSubMapper< + Scalar, Index, Side, + TensorEvaluator >, + Device>, + nocontract_t, contract_t, packet_size, inner_dim_contiguous, + inner_dim_reordered, Alignment>; + + // Load coefficient from a patch specified by the "within patch offset" + // (patchId) and the precomputed indices of the first element of the patch. + EIGEN_DEVICE_FUNC + EIGEN_STRONG_INLINE Scalar loadCoeff(Index patchId, Index planeIndex, + Index rowIndex, Index colIndex, + Index otherIndex) const { + // Find the offset of the element wrt the location of the first element. + const Index patchOffset = patchId / m_fastDimZero; + + const Index colOffset = patchOffset / m_fastColStride; + const Index inputCol = colIndex + colOffset * m_in_col_strides; + const Index origInputCol = + (m_patch_col_inflate_strides == 1) + ? inputCol + : ((inputCol >= 0) ? (inputCol / m_fastInputColStride) : 0); + + const Index rowOffset = + (patchOffset - colOffset * m_colStride) / m_fastRowStride; + const Index inputRow = rowIndex + rowOffset * m_in_row_strides; + const Index origInputRow = + (m_patch_row_inflate_strides == 1) + ? inputRow + : ((inputRow >= 0) ? (inputRow / m_fastInputRowStride) : 0); + + const Index planeOffset = + patchOffset - colOffset * m_colStride - rowOffset * m_rowStride; + const Index inputPlane = planeIndex + planeOffset * m_in_plane_strides; + const Index origInputPlane = + (m_patch_plane_inflate_strides == 1) + ? inputPlane + : ((inputPlane >= 0) ? (inputPlane / m_fastInputPlaneStride) : 0); + + if (origInputCol < 0 || origInputRow < 0 || origInputPlane < 0 || + origInputCol >= m_inputCols || origInputRow >= m_inputRows || + origInputPlane >= m_inputPlanes || + (inputCol != origInputCol * m_patch_col_inflate_strides) || + (inputRow != origInputRow * m_patch_row_inflate_strides) || + (inputPlane != origInputPlane * m_patch_plane_inflate_strides)) { + return Scalar(0); + } + + const Index depth = patchId - patchOffset * patchDepth(); + const Index inputIndex = depth + origInputPlane * m_planeInputStride + + origInputRow * m_rowInputStride + + origInputCol * m_colInputStride + otherIndex; + + return m_impl.coeff(inputIndex); + } + + // This is the same as loadCoeff(...), but optimized for all `inflate_strides` + // and `in_strides` equal to 1 (template specialization without templates). + EIGEN_DEVICE_FUNC + EIGEN_STRONG_INLINE Scalar loadCoeffStandard(Index patchId, Index planeIndex, + Index rowIndex, Index colIndex, + Index otherIndex) const { + eigen_assert(!nonStandardPatches()); + + // Find the offset of the element wrt the location of the first element. + const Index patchOffset = patchId / m_fastDimZero; + + const Index colOffset = patchOffset / m_fastColStride; + const Index rowOffset = + (patchOffset - colOffset * m_colStride) / m_fastRowStride; + const Index planeOffset = + patchOffset - colOffset * m_colStride - rowOffset * m_rowStride; + + const Index inputCol = colIndex + colOffset; + const Index inputRow = rowIndex + rowOffset; + const Index inputPlane = planeIndex + planeOffset; + + if (inputCol < 0 || inputCol >= m_inputCols || inputRow < 0 || + inputRow >= m_inputRows || inputPlane < 0 || + inputPlane >= m_inputPlanes) { + return Scalar(0); + } + + const Index depth = patchId - patchOffset * patchDepth(); + const Index inputIndex = depth + inputPlane * m_planeInputStride + + inputRow * m_rowInputStride + + inputCol * m_colInputStride + otherIndex; + + return m_impl.coeff(inputIndex); + } + + // Load packet from a patch specified by the "within patch offset" + // (patchId) and the precomputed indices of the first element of the patch. + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE Packet loadPacket(Index patchId, Index planeIndex, + Index rowIndex, Index colIndex, + Index otherIndex) const { + const Index packetSize = internal::unpacket_traits::size; + + EIGEN_STATIC_ASSERT(packetSize > 1, YOU_MADE_A_PROGRAMMING_MISTAKE) + eigen_assert(patchId < + patchDepth() * patchPlanes() * patchRows() * patchCols()); + + if (nonStandardPatches()) { + return packetWithPossibleZero(patchId, planeIndex, rowIndex, colIndex, + otherIndex); + } + return loadPacketStandard(patchId, planeIndex, rowIndex, colIndex, + otherIndex); + } + + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE Packet loadPacketStandard(Index patchId, Index planeIndex, + Index rowIndex, Index colIndex, + Index otherIndex) const { + const Index packetSize = internal::unpacket_traits::size; + EIGEN_STATIC_ASSERT(packetSize > 1, YOU_MADE_A_PROGRAMMING_MISTAKE) + eigen_assert(patchId < + patchDepth() * patchPlanes() * patchRows() * patchCols()); + eigen_assert(!nonStandardPatches()); + + if ((patchDepth() % packetSize) == 0) { + return loadPacketFast(patchId, planeIndex, rowIndex, colIndex, + otherIndex); + } else { + // Offsets and input calculation here are identical to + // loadCoeffStandard(...), but repeated twice. + + const Index patchOffsets[2] = { + patchId / m_fastDimZero, (patchId + packetSize - 1) / m_fastDimZero}; + + const Index colOffsets[2] = {patchOffsets[0] / m_fastColStride, + patchOffsets[1] / m_fastColStride}; + eigen_assert(colOffsets[0] <= colOffsets[1]); + + const Index inputCols[2] = {colIndex + colOffsets[0], + colIndex + colOffsets[1]}; + if (inputCols[0] >= m_inputCols || inputCols[1] < 0) { + return internal::pset1(Scalar(0)); + } + + if (inputCols[0] == inputCols[1]) { + const Index rowOffsets[2] = { + (patchOffsets[0] - colOffsets[0] * m_colStride) / m_fastRowStride, + (patchOffsets[1] - colOffsets[1] * m_colStride) / m_fastRowStride}; + eigen_assert(rowOffsets[0] <= rowOffsets[1]); + const Index inputRows[2] = {rowIndex + rowOffsets[0], + rowIndex + rowOffsets[1]}; + + if (inputRows[0] >= m_inputRows || inputRows[1] < 0) { + return internal::pset1(Scalar(0)); + } + + if (inputRows[0] == inputRows[1]) { + const Index planeOffsets[2] = { + patchOffsets[0] - colOffsets[0] * m_colStride - + rowOffsets[0] * m_rowStride, + patchOffsets[1] - colOffsets[1] * m_colStride - + rowOffsets[1] * m_rowStride}; + eigen_assert(planeOffsets[0] <= planeOffsets[1]); + const Index inputPlanes[2] = {planeIndex + planeOffsets[0], + planeIndex + planeOffsets[1]}; + + if (inputPlanes[0] >= m_inputPlanes || inputPlanes[1] < 0) { + return internal::pset1(Scalar(0)); + } + + if (inputPlanes[0] >= 0 && inputPlanes[1] < m_inputPlanes) { + const Index depth = patchId - patchOffsets[0] * patchDepth(); + const Index inputIndex = + depth + inputPlanes[0] * m_planeInputStride + + inputRows[0] * m_rowInputStride + + inputCols[0] * m_colInputStride + otherIndex; + return m_impl.template packet(inputIndex); + } + } + } + } + + return packetWithPossibleZero(patchId, planeIndex, rowIndex, colIndex, + otherIndex); + } + + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE Packet loadPacketFast(Index patchId, Index planeIndex, + Index rowIndex, Index colIndex, + Index otherIndex) const { + const Index packetSize = internal::unpacket_traits::size; + EIGEN_STATIC_ASSERT(packetSize > 1, YOU_MADE_A_PROGRAMMING_MISTAKE) + eigen_assert(patchId < + patchDepth() * patchPlanes() * patchRows() * patchCols()); + + eigen_assert(!nonStandardPatches()); + eigen_assert((patchDepth() % packetSize) == 0); + + // Find the offset of the element wrt the location of the first element. + const Index patchOffset = patchId / m_fastDimZero; + eigen_assert((patchId + packetSize - 1) / m_fastDimZero == patchOffset); + + const Index colOffset = patchOffset / m_fastColStride; + const Index rowOffset = + (patchOffset - colOffset * m_colStride) / m_fastRowStride; + const Index planeOffset = + patchOffset - colOffset * m_colStride - rowOffset * m_rowStride; + + const Index inputCol = colIndex + colOffset; + const Index inputRow = rowIndex + rowOffset; + const Index inputPlane = planeIndex + planeOffset; + + if (inputCol < 0 || inputRow < 0 || inputPlane < 0 || + inputCol >= m_inputCols || inputRow >= m_inputRows || + inputPlane >= m_inputPlanes) { + return internal::pset1(Scalar(0)); + } + + const Index depth = patchId - patchOffset * patchDepth(); + const Index inputIndex = depth + inputPlane * m_planeInputStride + + inputRow * m_rowInputStride + + inputCol * m_colInputStride + otherIndex; + return m_impl.template packet(inputIndex); + } + + EIGEN_DEVICE_FUNC EIGEN_ALWAYS_INLINE Packet + packetWithPossibleZero(Index patchId, Index planeIndex, Index rowIndex, + Index colIndex, Index otherIndex) const { + const int packetSize = internal::unpacket_traits::size; + EIGEN_ALIGN_MAX + typename internal::remove_const::type values[packetSize]; + for (int i = 0; i < packetSize; ++i) { + values[i] = + loadCoeff(patchId + i, planeIndex, rowIndex, colIndex, otherIndex); + } + Packet rslt = internal::pload(values); + return rslt; + } + + // Precompute the indices (plane, row, col, other) of the first element of + // the given patch index, within the output tensor of the TensorVolumePatchOp. + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE void computeBaseIndices( + Index patchIndex, Index& planeIndex, Index& rowIndex, Index& colIndex, + Index& otherIndex) const { + const size_t NumInputDims = array_size< + typename TensorEvaluator::Dimensions>::value; + + // Check if patchIndex might contain batch and other dimensions. + otherIndex = (NumInputDims == 4) ? 0 : patchIndex / m_fastNumPatches; + + // Compute index of the patch within the batch (and other dimensions). + const Index patch3DIndex = (NumInputDims == 4) + ? patchIndex + : (patchIndex - otherIndex * m_num_patches); + + otherIndex *= m_patchInputStride; + + colIndex = patch3DIndex / m_fastOutputPlanesRows; + rowIndex = + (patch3DIndex - colIndex * m_outputPlanesRows) / m_fastOutputPlanes; + planeIndex = + patch3DIndex - (colIndex * m_outputRows + rowIndex) * m_outputPlanes; + + colIndex = colIndex * m_col_strides - m_colPaddingLeft; + rowIndex = rowIndex * m_row_strides - m_rowPaddingTop; + planeIndex = planeIndex * m_plane_strides - m_planePaddingTop; + } + + Index m_patch_depth; // number of channels in the patch + Index m_patch_planes; // number of planes in the patch + Index m_patch_rows; // number of rows in the patch + Index m_patch_cols; // number of columns in the patch + Index m_num_patches; // number of patches to extract + + // Strides for navigating through the single patch. + Index m_patch_plane_stride; + Index m_patch_row_stride; + Index m_patch_col_stride; + + // Strides for the output tensor (depth is not the part of the stride). + Index m_rowStride; + Index m_colStride; + Index m_patchStride; + Index m_otherStride; + + Index m_planeInputStride; // Plane stride in the input tensor + Index m_rowInputStride; // Row stride in the input tensor + Index m_colInputStride; // Col stride in the input tensor + Index m_patchInputStride; // Patch stride in the input tensor + Index m_otherInputStride; + + Index m_inputDepth; // Depth of the input tensor + Index m_inputPlanes; // Number of planes in the input tensor + Index m_inputRows; // Number of rows in the input tensor + Index m_inputCols; // Number of cols in the input tensor + + Index m_outputPlanes; // Number of output planes + Index m_outputRows; // Number of output rows + Index m_outputCols; // Number of output cols + Index m_outputPlanesRows; // Cached outputPlanes * outputRows. + + Index m_plane_strides; // User specified plane stride + Index m_row_strides; // User specified row stride + Index m_col_strides; // User specified col stride + + // User specified plane/row/col atrous convolution strides. + Index m_in_plane_strides; + Index m_in_row_strides; + Index m_in_col_strides; + + // User specified plane/row/col inflation strides in the image patch. + Index m_patch_plane_inflate_strides; + Index m_patch_row_inflate_strides; + Index m_patch_col_inflate_strides; + + Index m_planePaddingTop; // Plane padding + Index m_rowPaddingTop; // Row padding + Index m_colPaddingLeft; // Column padding + + // Fast representation of various divisors. + internal::TensorIntDivisor m_fastNumPatches; + + internal::TensorIntDivisor m_fastPatchPlaneStride; + internal::TensorIntDivisor m_fastPatchRowStride; + internal::TensorIntDivisor m_fastPatchColStride; + + internal::TensorIntDivisor m_fastInputPlaneStride; + internal::TensorIntDivisor m_fastInputRowStride; + internal::TensorIntDivisor m_fastInputColStride; + + internal::TensorIntDivisor m_fastRowStride; + internal::TensorIntDivisor m_fastColStride; + + internal::TensorIntDivisor m_fastDimZero; // aka output depth + internal::TensorIntDivisor m_fastOutputPlanes; + internal::TensorIntDivisor m_fastOutputRows; + internal::TensorIntDivisor m_fastOutputCols; + internal::TensorIntDivisor m_fastOutputPlanesRows; + + const TensorEvaluator m_impl; +}; + +template +class TensorContractionSubMapper< + Scalar, Index, Side, + TensorEvaluator >, + Device>, + nocontract_t, contract_t, packet_size, inner_dim_contiguous, + inner_dim_reordered, Alignment> { + public: + typedef typename packet_traits::type Packet; + typedef typename packet_traits::half HalfPacket; + + typedef TensorContractionInputMapper< + Scalar, Index, Side, + TensorEvaluator >, + Device>, + nocontract_t, contract_t, packet_size, inner_dim_contiguous, + inner_dim_reordered, Alignment> + ParentMapper; + typedef TensorContractionSubMapper< + Scalar, Index, Side, + TensorEvaluator >, + Device>, + nocontract_t, contract_t, packet_size, inner_dim_contiguous, + inner_dim_reordered, Alignment> + Self; + typedef Self LinearMapper; + + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE TensorContractionSubMapper( + const ParentMapper& base_mapper, Index vert_offset, Index horiz_offset) + : m_base_mapper(base_mapper), + m_depth_offset(vert_offset), + m_col_offset(horiz_offset) { + m_base_mapper.computeBaseIndices(m_col_offset, m_planeIndex, m_rowIndex, + m_colIndex, m_otherIndex); + } + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE TensorContractionSubMapper( + const Self& base_mapper, Index vert_offset, Index horiz_offset) + : m_base_mapper(base_mapper.m_base_mapper), + m_depth_offset(vert_offset + base_mapper.m_depth_offset), + m_col_offset(horiz_offset + base_mapper.m_col_offset) { + m_base_mapper.computeBaseIndices(m_col_offset, m_planeIndex, m_rowIndex, + m_colIndex, m_otherIndex); + } + EIGEN_DEVICE_FUNC EIGEN_ALWAYS_INLINE Scalar operator()(Index i) const { + return m_base_mapper.loadCoeff(i + m_depth_offset, m_planeIndex, m_rowIndex, + m_colIndex, m_otherIndex); + } + EIGEN_DEVICE_FUNC EIGEN_ALWAYS_INLINE Scalar operator()(Index i, + Index j) const { + return m_base_mapper(i + m_depth_offset, j + m_col_offset); + } + + EIGEN_DEVICE_FUNC EIGEN_ALWAYS_INLINE Packet loadPacket(Index i) const { + return m_base_mapper.loadPacket(i + m_depth_offset, m_planeIndex, + m_rowIndex, m_colIndex, m_otherIndex); + } + EIGEN_DEVICE_FUNC EIGEN_ALWAYS_INLINE Packet loadPacket(Index i, + Index j) const { + return m_base_mapper.template loadPacket(i + m_depth_offset, + j + m_col_offset); + } + + EIGEN_DEVICE_FUNC EIGEN_ALWAYS_INLINE Scalar + loadCoeffStandard(Index i) const { + return m_base_mapper.loadCoeffStandard( + i + m_depth_offset, m_planeIndex, m_rowIndex, m_colIndex, m_otherIndex); + } + + EIGEN_DEVICE_FUNC EIGEN_ALWAYS_INLINE Packet loadPacketFast(Index i) const { + return m_base_mapper.loadPacketFast(i + m_depth_offset, m_planeIndex, + m_rowIndex, m_colIndex, m_otherIndex); + } + EIGEN_DEVICE_FUNC EIGEN_ALWAYS_INLINE Packet + loadPacketStandard(Index i) const { + return m_base_mapper.loadPacketStandard( + i + m_depth_offset, m_planeIndex, m_rowIndex, m_colIndex, m_otherIndex); + } + template + EIGEN_DEVICE_FUNC bool aligned(Index) const { + return false; + } + + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE bool nonStandardPatches() const { + return m_base_mapper.nonStandardPatches(); + } + + // Max(Col|Row|Plane|Depth): compute the upper limit for the column, row, + // plane and depth index respectively that fits into the peeled_k elements + // starting at m_depth_offset. + + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE Index maxCol(const Index peeled_k) const { + const Index max_col = + fastPatchColStride().divide(m_depth_offset + peeled_k); + return std::min(1 + max_col, patchCols()); + } + + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE Index maxRow(const Index peeled_k, + const Index col) const { + const Index max_row = fastPatchRowStride().divide( + m_depth_offset + peeled_k - col * patchColStride()); + return std::min(1 + max_row, patchRows()); + } + + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE Index maxPlane(const Index peeled_k, const Index col, + const Index row) const { + const Index max_plane = fastPatchPlaneStride().divide( + m_depth_offset + peeled_k - col * patchColStride() - + row * patchRowStride()); + return std::min(1 + max_plane, patchPlanes()); + } + + // MaxDepth uses only the remaining number of elements in the peeled_k. + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE Index maxDepth(const Index num_elements, + const Index start_depth) const { + return std::min(start_depth + num_elements, patchDepth()); + } + + // Every register matters in this code, so sometimes to prevent register + // spilling, instead of the variable that you would expect to see, we use + // another one, that is guaranteed to have the same value. E.g. patch depth is + // always the same as input depth, and it's also the same as input plane + // stride. Bunch of other parameters have similar relations. + + typedef internal::TensorIntDivisor IndexDivisor; + + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE Index patchDepth() const { + eigen_assert(m_base_mapper.m_patch_depth == + m_base_mapper.m_planeInputStride && + "Patch depth must be equal to plane input stride."); + return m_base_mapper.m_planeInputStride; + } + + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE Index patchPlanes() const { + eigen_assert(m_base_mapper.m_patch_planes == m_base_mapper.m_rowStride && + "Patch planes must be equal to row stride."); + return m_base_mapper.m_rowStride; + } + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE Index patchRows() const { + return m_base_mapper.m_patch_rows; + } + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE Index patchCols() const { + return m_base_mapper.m_patch_cols; + } + + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE Index patchPlaneStride() const { + eigen_assert(patchDepth() == m_base_mapper.m_patch_plane_stride && + "Patch depth must be equal to patch plane stride."); + return patchDepth(); + } + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE Index patchRowStride() const { + return m_base_mapper.m_patch_row_stride; + } + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE Index patchColStride() const { + return m_base_mapper.m_patch_col_stride; + } + + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE IndexDivisor fastPatchPlaneStride() const { + eigen_assert(patchDepth() == m_base_mapper.m_patch_plane_stride && + "Patch depth must be equal to patch plane stride."); + return m_base_mapper.m_fastDimZero; // patch_depth + } + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE IndexDivisor fastPatchRowStride() const { + return m_base_mapper.m_fastPatchRowStride; + } + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE IndexDivisor fastPatchColStride() const { + return m_base_mapper.m_fastPatchColStride; + } + + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE Packet packetNoPadding(const Index depth, + const Index baseIndex) const { + const Index inputIndex = depth + baseIndex; + return m_base_mapper.m_impl.template packet(inputIndex); + } + + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE bool padPlane(const Index plane) const { + const Index p = m_planeIndex + plane; + return p < 0 || p >= m_base_mapper.m_inputPlanes; + } + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE bool padRow(const Index row) const { + const Index r = m_rowIndex + row; + return r < 0 || r >= m_base_mapper.m_inputRows; + } + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE bool padCol(const Index col) const { + const Index c = m_colIndex + col; + return c < 0 || c >= m_base_mapper.m_inputCols; + } + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE Index baseIndex(const Index plane, const Index row, + const Index col) const { + const Index p = m_planeIndex + plane; + const Index r = m_rowIndex + row; + const Index c = m_colIndex + col; + return p * m_base_mapper.m_planeInputStride + + r * m_base_mapper.m_rowInputStride + + c * m_base_mapper.m_colInputStride + m_otherIndex; + } + + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE Index planeOffset() const { + const Index patchOffset = m_depth_offset / m_base_mapper.m_fastDimZero; + const Index colOffset = patchOffset / m_base_mapper.m_fastColStride; + const Index rowOffset = + (patchOffset - colOffset * m_base_mapper.m_colStride) / + m_base_mapper.m_fastRowStride; + const Index planeOffset = patchOffset - + colOffset * m_base_mapper.m_colStride - + rowOffset * m_base_mapper.m_rowStride; + return planeOffset; + } + + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE Index rowOffset() const { + const Index patchOffset = m_depth_offset / m_base_mapper.m_fastDimZero; + const Index colOffset = patchOffset / m_base_mapper.m_fastColStride; + const Index rowOffset = + (patchOffset - colOffset * m_base_mapper.m_colStride) / + m_base_mapper.m_fastRowStride; + return rowOffset; + } + + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE Index colOffset() const { + const Index patchOffset = m_depth_offset / m_base_mapper.m_fastDimZero; + const Index colOffset = patchOffset / m_base_mapper.m_fastColStride; + return colOffset; + } + + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE Index depthOffset() const { + return m_depth_offset % patchDepth(); + } + + EIGEN_DEVICE_FUNC EIGEN_ALWAYS_INLINE LinearMapper + getLinearMapper(Index i, Index j) const { + return LinearMapper(m_base_mapper, i + m_depth_offset, j + m_col_offset); + } + + private: + const ParentMapper& m_base_mapper; + Index m_depth_offset; // First row in the input matrix + Index m_col_offset; // First col in the input matrix + + // Knowing that: col_offset == patchIndex * OTHERS, we keep precomputed base + // indices for the first element in a patch specified by col_offset + // (see computeBaseIndices(...) for details). + Index m_planeIndex; + Index m_rowIndex; + Index m_colIndex; + Index m_otherIndex; +}; + +// Arrange a block of the right input matrix (in our case it's always a "virtual +// matrix" constructed from extracted volume patches) in contiguous memory. +// +// Given column major input (A0 beside A1 in memory): +// A0 B0 C0 D0 E0 F0 G0 H0 ... Z0 +// A1 B1 C1 D1 E1 F1 G1 H1 ... Z1 +// A2 B2 C2 D2 E2 F2 G2 H2 ... Z2 +// A3 B3 C3 D3 E3 F3 G3 H3 ... Z3 +// A4 B4 C4 D4 E4 F4 G4 H4 ... Z4 +// A5 B5 C5 D5 E5 F5 G5 H5 ... Z5 +// A6 B6 C6 D6 E6 F6 G6 H6 ... Z6 +// A7 B7 C7 D7 E7 F7 G7 H7 ... Z7 +// A8 ... +// ... +// +// *) A, B, C, ... - patches extracted from the original input. +// *) A0, A1, A2 ... - values from the same patch at different offsets. +// +// The traversal (packed rhs memory) order (B0 besides A0 in memory): +// A0 B0 C0 D0 A1 B1 C1 D1 ... +// E0 F0 G0 H0 E1 F1 G1 H1 ... +// ... +// Z0 Z1 Z2 Z3 Z4 Z5 Z6 Z7 ... <- doesn't belong to any block (nr = 4) +// +// This traversal order must be the same as in default gemm_pack_rhs defined in +// GeneralBlockPanelKernel.h. +// +// *) nr - number of registers along the 'n' dimension. +// See GeneralBlockPanelKernel.h and "Anatomy of High-Performance Matrix +// Multiplication" paper. +template +struct gemm_pack_rhs< + Scalar, Index, + TensorContractionSubMapper< + Scalar, Index, Rhs, + TensorEvaluator >, + Device>, + nocontract_t, contract_t, packet_size, inner_dim_contiguous, + inner_dim_reordered, Alignment>, + nr, ColMajor, false, false> { + typedef TensorContractionSubMapper< + Scalar, Index, Rhs, + TensorEvaluator >, + Device>, + nocontract_t, contract_t, packet_size, inner_dim_contiguous, + inner_dim_reordered, Alignment> + SubMapper; + + typedef SubMapper DataMapper; + typedef typename packet_traits::type Packet; + + EIGEN_STATIC_ASSERT((nr == 4), YOU_MADE_A_PROGRAMMING_MISTAKE); + + EIGEN_DEVICE_FUNC + EIGEN_DONT_INLINE void operator()(Scalar* block, const DataMapper& rhs, + Index depth, Index cols, Index stride = 0, + Index offset = 0) const { + eigen_assert(stride == 0); + eigen_assert(offset == 0); + + const Index packet_cols4 = (cols / 4) * 4; + const Index peeled_k = (depth / packet_size) * packet_size; + const bool non_standard_patches = rhs.nonStandardPatches(); + + for (Index j2 = 0; j2 < packet_cols4; j2 += 4) { + const SubMapper dm0 = rhs.getLinearMapper(0, j2 + 0); + const SubMapper dm1 = rhs.getLinearMapper(0, j2 + 1); + const SubMapper dm2 = rhs.getLinearMapper(0, j2 + 2); + const SubMapper dm3 = rhs.getLinearMapper(0, j2 + 3); + + Index k = 0; + if ((packet_size % 4) == 0 && !non_standard_patches) { + // FAST PATH: + // Iterate over patch columns, rows and planes if we know that a single + // packet do not span across multiple planes, rows or columns. + if ((rhs.patchDepth() % packet_size) == 0) { + const Index start_col = rhs.colOffset(); + const Index max_col = rhs.maxCol(peeled_k); + + for (Index c = start_col; c < max_col; ++c) { + eigen_assert(k <= peeled_k); + + const Index start_row = (c == start_col) ? rhs.rowOffset() : 0; + const Index max_row = rhs.maxRow(peeled_k, c); + + const bool pad_col0 = dm0.padCol(c); + const bool pad_col1 = dm1.padCol(c); + const bool pad_col2 = dm2.padCol(c); + const bool pad_col3 = dm3.padCol(c); + + for (Index r = start_row; r < max_row; ++r) { + eigen_assert(k <= peeled_k); + + const Index start_plane = ((c == start_col) && (r == start_row)) + ? rhs.planeOffset() + : 0; + const Index max_plane = rhs.maxPlane(peeled_k, c, r); + + const bool pad_row0 = pad_col0 || dm0.padRow(r); + const bool pad_row1 = pad_col1 || dm1.padRow(r); + const bool pad_row2 = pad_col2 || dm2.padRow(r); + const bool pad_row3 = pad_col3 || dm3.padRow(r); + + for (Index p = start_plane; p < max_plane; ++p) { + eigen_assert(k <= peeled_k); + + const bool pad0 = pad_row0 || dm0.padPlane(p); + const bool pad1 = pad_row1 || dm1.padPlane(p); + const bool pad2 = pad_row2 || dm2.padPlane(p); + const bool pad3 = pad_row3 || dm3.padPlane(p); + + const Index idx0 = dm0.baseIndex(p, r, c); + const Index idx1 = dm1.baseIndex(p, r, c); + const Index idx2 = dm2.baseIndex(p, r, c); + const Index idx3 = dm3.baseIndex(p, r, c); + + const Index start_depth = + ((c == start_col) && (r == start_row) && (p == start_plane)) + ? rhs.depthOffset() + : 0; + const Index max_depth = rhs.maxDepth(peeled_k - k, start_depth); + eigen_assert((max_depth - start_depth) % packet_size == 0); + + for (Index d = start_depth; d < max_depth; d += packet_size) { + eigen_assert(k < peeled_k); + PacketBlock kernel; + kernel.packet[0] = pad0 ? pset1(Scalar(0)) + : rhs.packetNoPadding(d, idx0); + kernel.packet[1] = pad1 ? pset1(Scalar(0)) + : rhs.packetNoPadding(d, idx1); + kernel.packet[2] = pad2 ? pset1(Scalar(0)) + : rhs.packetNoPadding(d, idx2); + kernel.packet[3] = pad3 ? pset1(Scalar(0)) + : rhs.packetNoPadding(d, idx3); + ptranspose(kernel); + pstoreu(block + 0 * packet_size, kernel.packet[0]); + pstoreu(block + 1 * packet_size, kernel.packet[1]); + pstoreu(block + 2 * packet_size, kernel.packet[2]); + pstoreu(block + 3 * packet_size, kernel.packet[3]); + block += 4 * packet_size; + k += packet_size; + } + } + } + } + + // The loop above should fill peeled_k elements. + eigen_assert(peeled_k == k); + + } else { + // Packet can span multiple planes, rows or columns, so we have to go + // though the slower "standard" path. + for (; k < peeled_k; k += packet_size) { + PacketBlock kernel; + kernel.packet[0] = dm0.loadPacketStandard(k); + kernel.packet[1] = dm1.loadPacketStandard(k); + kernel.packet[2] = dm2.loadPacketStandard(k); + kernel.packet[3] = dm3.loadPacketStandard(k); + ptranspose(kernel); + pstoreu(block + 0 * packet_size, kernel.packet[0]); + pstoreu(block + 1 * packet_size, kernel.packet[1]); + pstoreu(block + 2 * packet_size, kernel.packet[2]); + pstoreu(block + 3 * packet_size, kernel.packet[3]); + block += 4 * packet_size; + } + } + } + + // Copy the remaining coefficients of the column block after the peeled_k. + if (!non_standard_patches) { + for (; k < depth; k++) { + block[0] = dm0.loadCoeffStandard(k); + block[1] = dm1.loadCoeffStandard(k); + block[2] = dm2.loadCoeffStandard(k); + block[3] = dm3.loadCoeffStandard(k); + block += 4; + } + } else { + for (; k < depth; k++) { + block[0] = dm0(k); + block[1] = dm1(k); + block[2] = dm2(k); + block[3] = dm3(k); + block += 4; + } + } + } + + // Copy the remaining columns one at a time (nr==1). + for (Index j2 = packet_cols4; j2 < cols; ++j2) { + const SubMapper dm0 = rhs.getLinearMapper(0, j2); + for (Index k = 0; k < depth; k++) { + *block = dm0(k); + block += 1; + } + } + } +}; + +// Template specialization for packet_size = 2. We must special-case packet +// blocks with nr > packet_size, e.g. PacketBlock. +template +struct gemm_pack_rhs< + Scalar, Index, + TensorContractionSubMapper< + Scalar, Index, Rhs, + TensorEvaluator >, + Device>, + nocontract_t, contract_t, /*packet_size*/ 2, inner_dim_contiguous, + inner_dim_reordered, Alignment>, + nr, ColMajor, false, false> { + typedef TensorContractionSubMapper< + Scalar, Index, Rhs, + TensorEvaluator >, + Device>, + nocontract_t, contract_t, /*packet_size*/ 2, inner_dim_contiguous, + inner_dim_reordered, Alignment> + SubMapper; + typedef SubMapper DataMapper; + typedef typename packet_traits::type Packet; + + EIGEN_STATIC_ASSERT((nr == 4), YOU_MADE_A_PROGRAMMING_MISTAKE); + + EIGEN_DEVICE_FUNC + EIGEN_DONT_INLINE void operator()(Scalar* block, const DataMapper& rhs, + Index depth, Index cols, Index stride = 0, + Index offset = 0) const { + eigen_assert(stride == 0); + eigen_assert(offset == 0); + + const int packet_size = 2; + + const Index packet_cols4 = (cols / 4) * 4; + const Index peeled_k = (depth / packet_size) * packet_size; + const bool non_standard_patches = rhs.nonStandardPatches(); + + for (Index j2 = 0; j2 < packet_cols4; j2 += 4) { + const SubMapper dm0 = rhs.getLinearMapper(0, j2 + 0); + const SubMapper dm1 = rhs.getLinearMapper(0, j2 + 1); + const SubMapper dm2 = rhs.getLinearMapper(0, j2 + 2); + const SubMapper dm3 = rhs.getLinearMapper(0, j2 + 3); + + Index k = 0; + if (!non_standard_patches) { + // FAST PATH: + // Iterate over patch columns, rows and planes if we know that a single + // packet do not span across multiple planes, rows or columns. + if ((rhs.patchDepth() % packet_size) == 0) { + const Index start_col = rhs.colOffset(); + const Index max_col = rhs.maxCol(peeled_k); + + for (Index c = start_col; c < max_col; ++c) { + eigen_assert(k <= peeled_k); + + const Index start_row = (c == start_col) ? rhs.rowOffset() : 0; + const Index max_row = rhs.maxRow(peeled_k, c); + + const bool pad_col0 = dm0.padCol(c); + const bool pad_col1 = dm1.padCol(c); + const bool pad_col2 = dm2.padCol(c); + const bool pad_col3 = dm3.padCol(c); + + for (Index r = start_row; r < max_row; ++r) { + eigen_assert(k <= peeled_k); + + const Index start_plane = ((c == start_col) && (r == start_row)) + ? rhs.planeOffset() + : 0; + const Index max_plane = rhs.maxPlane(peeled_k, c, r); + + const bool pad_row0 = dm0.padRow(r); + const bool pad_row1 = dm1.padRow(r); + const bool pad_row2 = dm2.padRow(r); + const bool pad_row3 = dm3.padRow(r); + + for (Index p = start_plane; p < max_plane; ++p) { + eigen_assert(k <= peeled_k); + + const bool pad0 = pad_col0 || pad_row0 || dm0.padPlane(p); + const bool pad1 = pad_col1 || pad_row1 || dm1.padPlane(p); + const bool pad2 = pad_col2 || pad_row2 || dm2.padPlane(p); + const bool pad3 = pad_col3 || pad_row3 || dm3.padPlane(p); + + const Index idx0 = dm0.baseIndex(p, r, c); + const Index idx1 = dm1.baseIndex(p, r, c); + const Index idx2 = dm2.baseIndex(p, r, c); + const Index idx3 = dm3.baseIndex(p, r, c); + + const Index start_depth = + ((c == start_col) && (r == start_row) && (p == start_plane)) + ? rhs.depthOffset() + : 0; + const Index max_depth = rhs.maxDepth(peeled_k - k, start_depth); + eigen_assert((max_depth - start_depth) % packet_size == 0); + + for (Index d = start_depth; d < max_depth; d += packet_size) { + eigen_assert(k < peeled_k); + PacketBlock kernel0; + PacketBlock kernel1; + kernel0.packet[0] = pad0 ? pset1(Scalar(0)) + : rhs.packetNoPadding(d, idx0); + kernel0.packet[1] = pad1 ? pset1(Scalar(0)) + : rhs.packetNoPadding(d, idx1); + kernel1.packet[0] = pad2 ? pset1(Scalar(0)) + : rhs.packetNoPadding(d, idx2); + kernel1.packet[1] = pad3 ? pset1(Scalar(0)) + : rhs.packetNoPadding(d, idx3); + ptranspose(kernel0); + ptranspose(kernel1); + pstoreu(block + 0 * packet_size, kernel0.packet[0]); + pstoreu(block + 1 * packet_size, kernel1.packet[0]); + pstoreu(block + 2 * packet_size, kernel0.packet[1]); + pstoreu(block + 3 * packet_size, kernel1.packet[1]); + block += 4 * packet_size; + k += packet_size; + } + } + } + } + + // The loop above should fill peeled_k elements. + eigen_assert(peeled_k == k); + + } else { + for (; k < peeled_k; k += packet_size) { + PacketBlock kernel0; + PacketBlock kernel1; + kernel0.packet[0] = dm0.loadPacketStandard(k); + kernel0.packet[1] = dm1.loadPacketStandard(k); + kernel1.packet[0] = dm2.loadPacketStandard(k); + kernel1.packet[1] = dm3.loadPacketStandard(k); + ptranspose(kernel0); + ptranspose(kernel1); + pstoreu(block + 0 * packet_size, kernel0.packet[0]); + pstoreu(block + 1 * packet_size, kernel1.packet[0]); + pstoreu(block + 2 * packet_size, kernel0.packet[1]); + pstoreu(block + 3 * packet_size, kernel1.packet[1]); + block += 4 * packet_size; + } + } + } + + // Copy the remaining coefficients of the column block after the peeled_k. + if (!rhs.nonStandardPatches()) { + for (; k < depth; k++) { + block[0] = dm0.loadCoeffStandard(k); + block[1] = dm1.loadCoeffStandard(k); + block[2] = dm2.loadCoeffStandard(k); + block[3] = dm3.loadCoeffStandard(k); + block += 4; + } + } else { + for (; k < depth; k++) { + block[0] = dm0(k); + block[1] = dm1(k); + block[2] = dm2(k); + block[3] = dm3(k); + block += 4; + } + } + } + + // Copy the remaining columns one at a time (nr==1). + for (Index j2 = packet_cols4; j2 < cols; ++j2) { + const SubMapper dm0 = rhs.getLinearMapper(0, j2); + for (Index k = 0; k < depth; k++) { + *block = dm0(k); + block += 1; + } + } + } +}; + +// Special case for non-vectorized types such as float16 (packet_size = 1). +template +struct gemm_pack_rhs< + Scalar, Index, + TensorContractionSubMapper< + Scalar, Index, Rhs, + TensorEvaluator >, + Device>, + nocontract_t, contract_t, /*packet_size*/ 1, inner_dim_contiguous, + inner_dim_reordered, Alignment>, + nr, ColMajor, false, false> { + typedef TensorContractionSubMapper< + Scalar, Index, Rhs, + TensorEvaluator >, + Device>, + nocontract_t, contract_t, 1, inner_dim_contiguous, inner_dim_reordered, + Alignment> + SubMapper; + typedef SubMapper DataMapper; + + EIGEN_STATIC_ASSERT((nr == 4), YOU_MADE_A_PROGRAMMING_MISTAKE); + + EIGEN_DEVICE_FUNC + EIGEN_DONT_INLINE void operator()(Scalar* block, const DataMapper& rhs, + Index depth, Index cols, Index stride = 0, + Index offset = 0) const { + eigen_assert(stride == 0); + eigen_assert(offset == 0); + + const Index packet_cols4 = (cols / 4) * 4; + + for (Index j2 = 0; j2 < packet_cols4; j2 += 4) { + const SubMapper dm0 = rhs.getLinearMapper(0, j2 + 0); + const SubMapper dm1 = rhs.getLinearMapper(0, j2 + 1); + const SubMapper dm2 = rhs.getLinearMapper(0, j2 + 2); + const SubMapper dm3 = rhs.getLinearMapper(0, j2 + 3); + + if (!rhs.nonStandardPatches()) { + for (Index k = 0; k < depth; k++) { + block[0] = dm0.loadCoeffStandard(k); + block[1] = dm1.loadCoeffStandard(k); + block[2] = dm2.loadCoeffStandard(k); + block[3] = dm3.loadCoeffStandard(k); + block += 4; + } + } else { + for (Index k = 0; k < depth; k++) { + block[0] = dm0(k); + block[1] = dm1(k); + block[2] = dm2(k); + block[3] = dm3(k); + block += 4; + } + } + } + + // Copy the remaining columns one at a time (nr==1). + for (Index j2 = packet_cols4; j2 < cols; ++j2) { + const SubMapper dm0 = rhs.getLinearMapper(0, j2); + for (Index k = 0; k < depth; k++) { + *block = dm0(k); + block += 1; + } + } + } +}; + +} // namespace internal + +/** CuboidConvolution + * \ingroup CXX11_NeuralNetworks_Module + * + * \brief Applies a 3D convolution over a multichannel input voxel block. + * + * The input parameter is expected to be a tensor with a rank of 4 or more + * (channels, depth, height, width, and optionally others). + * The kernel parameter is expected to be a 5D tensor (filters, channels, + * kernel_depth, kernel_height, kernel_width). + * The result can be assigned to a tensor of rank equal to the rank of the + * input. The dimensions of the result will be filters, depth, height, width + * (and others if applicable). + * + * The input and kernel have to be in the same layout, and both row-major and + * col-major are supported. The shapes given above are for col-major layout. + * For row-major, all dimensions should be reversed. + * + * It is possible to swap the order of the depth, width, and height dimensions + * provided that the same order is used in the input, the kernel, and the + * output. + */ +template +EIGEN_ALWAYS_INLINE static const typename internal::conditional< + internal::traits::Layout == ColMajor, + TensorReshapingOp< + const DSizes::Index, + internal::traits::NumDimensions>, + const TensorContractionOp< + const array::Index>, 1>, + const TensorReshapingOp< + const DSizes::Index, 2>, + const Kernel>, + const TensorReshapingOp< + const DSizes::Index, 2>, + const TensorVolumePatchOp > > >, + TensorReshapingOp< + const DSizes::Index, + internal::traits::NumDimensions>, + const TensorContractionOp< + const array::Index>, 1>, + const TensorReshapingOp< + const DSizes::Index, 2>, + const TensorVolumePatchOp >, + const TensorReshapingOp< + const DSizes::Index, 2>, + const Kernel> > > >::type +CuboidConvolution(const Input& input, const Kernel& kernel, + const DenseIndex stridePlanes = 1, + const DenseIndex strideRows = 1, + const DenseIndex strideCols = 1, + const PaddingType padding_type = PADDING_SAME) { + typedef typename internal::traits::Index TensorIndex; + TensorRef::Scalar, + internal::traits::NumDimensions, + internal::traits::Layout, TensorIndex> > + in(input); + TensorRef::Scalar, + internal::traits::NumDimensions, + internal::traits::Layout, TensorIndex> > + kern(kernel); + + EIGEN_STATIC_ASSERT( + internal::traits::Layout == internal::traits::Layout, + YOU_MADE_A_PROGRAMMING_MISTAKE); + static const bool isColMajor = (internal::traits::Layout == ColMajor); + static const int NumDims = internal::traits::NumDimensions; + + // Number of filters to apply. This is the same as the output depth of the + // result. + const TensorIndex kernelFilters = + isColMajor ? kern.dimensions()[0] : kern.dimensions()[4]; + const TensorIndex kernelChannels = + isColMajor ? kern.dimensions()[1] : kern.dimensions()[3]; + + // Spatial size of the kernel. + const TensorIndex kernelPlanes = + isColMajor ? kern.dimensions()[2] : kern.dimensions()[2]; + const TensorIndex kernelRows = + isColMajor ? kern.dimensions()[3] : kern.dimensions()[1]; + const TensorIndex kernelCols = + isColMajor ? kern.dimensions()[4] : kern.dimensions()[0]; + + if (isColMajor) { + eigen_assert(kernelChannels == in.dimension(0)); + } else { + eigen_assert(kernelChannels == in.dimension(NumDims - 1)); + } + + const TensorIndex inputPlanes = + isColMajor ? in.dimension(1) : in.dimension(NumDims - 2); + const TensorIndex inputRows = + isColMajor ? in.dimension(2) : in.dimension(NumDims - 3); + const TensorIndex inputCols = + isColMajor ? in.dimension(3) : in.dimension(NumDims - 4); + + TensorIndex out_planes; + TensorIndex out_height; + TensorIndex out_width; + switch (padding_type) { + case PADDING_VALID: + out_planes = Eigen::divup(inputPlanes - kernelPlanes + 1, + static_cast(stridePlanes)); + out_height = Eigen::divup(inputRows - kernelRows + 1, + static_cast(strideRows)); + out_width = Eigen::divup(inputCols - kernelCols + 1, + static_cast(strideCols)); + break; + case PADDING_SAME: + out_planes = + Eigen::divup(inputPlanes, static_cast(stridePlanes)); + out_height = + Eigen::divup(inputRows, static_cast(strideRows)); + out_width = Eigen::divup(inputCols, static_cast(strideCols)); + break; + default: + out_planes = 0; + out_height = 0; + out_width = 0; + eigen_assert(false && "unexpected padding"); + } + + DSizes kernel_dims; + if (isColMajor) { + kernel_dims[0] = kernelFilters; + kernel_dims[1] = kernelChannels * kernelPlanes * kernelRows * kernelCols; + } else { + kernel_dims[0] = kernelChannels * kernelPlanes * kernelRows * kernelCols; + kernel_dims[1] = kernelFilters; + } + + // Molds the output of the patch extraction result into a 2D tensor: + // - the first dimension (dims[0]): the patch values to be multiplied with the + // kernels + // - the second dimension (dims[1]): everything else + DSizes pre_contract_dims; + if (isColMajor) { + pre_contract_dims[0] = + kernelChannels * kernelPlanes * kernelRows * kernelCols; + pre_contract_dims[1] = out_planes * out_height * out_width; + for (int i = 4; i < NumDims; ++i) { + pre_contract_dims[1] *= in.dimension(i); + } + } else { + pre_contract_dims[1] = + kernelChannels * kernelPlanes * kernelRows * kernelCols; + pre_contract_dims[0] = out_planes * out_height * out_width; + for (int i = 0; i < NumDims - 4; ++i) { + pre_contract_dims[0] *= in.dimension(i); + } + } + + array, 1> contract_dims; + contract_dims[0] = IndexPair(1, 0); + + // Molds the output of the contraction into the shape expected by the user + // (assuming ColMajor): + // - 1st dim: kernel filters + // - 2nd dim: output depth + // - 3nd dim: output height + // - 4rd dim: output width + // - 5th dim and beyond: everything else including batch size + DSizes post_contract_dims; + if (isColMajor) { + post_contract_dims[0] = kernelFilters; + post_contract_dims[1] = out_planes; + post_contract_dims[2] = out_height; + post_contract_dims[3] = out_width; + for (int i = 4; i < NumDims; ++i) { + post_contract_dims[i] = in.dimension(i); + } + } else { + post_contract_dims[NumDims - 1] = kernelFilters; + post_contract_dims[NumDims - 2] = out_planes; + post_contract_dims[NumDims - 3] = out_height; + post_contract_dims[NumDims - 4] = out_width; + for (int i = 0; i < NumDims - 4; ++i) { + post_contract_dims[i] = in.dimension(i); + } + } + + return choose( + Cond::Layout == ColMajor>(), + kernel.reshape(kernel_dims) + .contract(input + .extract_volume_patches( + kernelPlanes, kernelRows, kernelCols, stridePlanes, + strideRows, strideCols, padding_type) + .reshape(pre_contract_dims), + contract_dims) + .reshape(post_contract_dims), + input + .extract_volume_patches(kernelPlanes, kernelRows, kernelCols, + stridePlanes, strideRows, strideCols, + padding_type) + .reshape(pre_contract_dims) + .contract(kernel.reshape(kernel_dims), contract_dims) + .reshape(post_contract_dims)); +} + +} // end namespace Eigen + +#endif // TENSORFLOW_CORE_KERNELS_EIGEN_CUBOID_CONVOLUTION_H_ diff --git a/linux/include/tensorflow/core/kernels/eigen_pooling.h b/linux/include/tensorflow/core/kernels/eigen_pooling.h new file mode 100644 index 000000000..56de6b1d4 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/eigen_pooling.h @@ -0,0 +1,619 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_EIGEN_POOLING_H_ +#define TENSORFLOW_CORE_KERNELS_EIGEN_POOLING_H_ + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/kernels/eigen_volume_patch.h" + +namespace Eigen { + +/** SpatialMaxPooling + * \ingroup CXX11_NeuralNetworks_Module + * + * \brief Applies a max-pooling over a multichannel input image. + * + * The input parameter is expected to be a with a rank of 4 (channels, height, + * width, others in col-major, and the reverse of that in row-major). + * + * The result can be assigned to a tensor of rank equal to the rank of the + * input. The dimensions of the result will be channels, height, width, and + * others (in col-major, and the reverse of that if the input was row-major). + * + * The order of the width and height dimensions can be swapped if needed. + * + */ +#if !defined(EIGEN_HAS_INDEX_LIST) +template +EIGEN_ALWAYS_INLINE static const TensorReshapingOp< + const Eigen::DSizes::Index, + internal::traits::NumDimensions>, + const TensorReductionOp< + internal::MaxReducer::Scalar>::type>, + const Eigen::array, + const TensorImagePatchOp > > +#else +template +EIGEN_ALWAYS_INLINE static const TensorReshapingOp< + const Eigen::DSizes::Index, + internal::traits::NumDimensions>, + const TensorReductionOp< + internal::MaxReducer::Scalar>::type>, + typename internal::conditional< + internal::traits::Layout == ColMajor, + const Eigen::IndexList, Eigen::type2index<2> >, + const Eigen::IndexList, + Eigen::type2index<3> > >::type, + const TensorImagePatchOp > > +#endif +SpatialMaxPooling(const Input& input, DenseIndex patchRows, + DenseIndex patchCols, DenseIndex strideRows, + DenseIndex strideCols, const PaddingType padding_type, + DenseIndex in_strideRows = 1, DenseIndex in_strideCols = 1) { + EIGEN_STATIC_ASSERT(internal::traits::NumDimensions == 4, + YOU_MADE_A_PROGRAMMING_MISTAKE); + + typedef typename internal::traits::Index TensorIndex; + TensorRef::Scalar, + internal::traits::NumDimensions, + internal::traits::Layout, TensorIndex> > + in(input); + + const DenseIndex patchRowsEff = + patchRows + (patchRows - 1) * (in_strideRows - 1); + const DenseIndex patchColsEff = + patchCols + (patchCols - 1) * (in_strideCols - 1); + + static const bool isColMajor = (internal::traits::Layout == ColMajor); + static const int idxRows = isColMajor ? 1 : 2; + static const int idxCols = isColMajor ? 2 : 1; + + // Molds the output of the reduction into the shape expected by the user. + // (assuming col-major): + // - 1st dim: channels + // - 2nd dim: output height + // - 3rd dim: output width + // - 4th dim and beyond: everything else including batch size + Eigen::DSizes::NumDimensions> + post_reduce_dims; + post_reduce_dims[0] = in.dimension(0); + if (padding_type == PADDING_VALID) { + post_reduce_dims[idxRows] = Eigen::divup( + static_cast(in.dimension(idxRows)) - patchRowsEff + 1, + strideRows); + post_reduce_dims[idxCols] = Eigen::divup( + static_cast(in.dimension(idxCols)) - patchColsEff + 1, + strideCols); + } else { + post_reduce_dims[idxRows] = Eigen::divup( + static_cast(in.dimension(idxRows)), strideRows); + post_reduce_dims[idxCols] = Eigen::divup( + static_cast(in.dimension(idxCols)), strideCols); + } + post_reduce_dims[3] = in.dimension(3); + +#if !defined(EIGEN_HAS_INDEX_LIST) + // nvcc doesn't support cxx11 + Eigen::array reduction_dims; + if (isColMajor) { + reduction_dims[0] = 1; + reduction_dims[1] = 2; + } else { + reduction_dims[0] = 2; + reduction_dims[1] = 3; + } +#else + // Take advantage of cxx11 to give the compiler information it can use to + // optimize the code. + typename internal::conditional< + internal::traits::Layout == ColMajor, + const Eigen::IndexList, Eigen::type2index<2> >, + const Eigen::IndexList, + Eigen::type2index<3> > >::type reduction_dims; +#endif + + return input + .extract_image_patches( + patchRows, patchCols, strideRows, strideCols, in_strideRows, + in_strideCols, padding_type, + -Eigen::NumTraits::Scalar>::type>::highest()) + .maximum(reduction_dims) + .reshape(post_reduce_dims); +} + +/** CuboidMaxPooling + * \ingroup CXX11_NeuralNetworks_Module + * + * \brief Applies a max-pooling over a multichannel input volume. + * + * The input parameter is expected to be a tensor with a rank of 5 (channels, + * depth, height, width, others in col-major, and the reverse of that in + * row-major). + * + * The result can be assigned to a tensor of rank equal to the rank of the + * input. The dimensions of the result will be channels, depth, height, width, + * and others (in col-major, and the reverse of that if the input was + * row-major). + * + * The order of the depth, width and height dimensions can be swapped if + * needed. + * + */ +#if !defined(EIGEN_HAS_INDEX_LIST) +template +EIGEN_ALWAYS_INLINE static const TensorReshapingOp< + const Eigen::DSizes::NumDimensions>, + const TensorReductionOp< + internal::MaxReducer, const Eigen::array, + const TensorReshapingOp< + const Eigen::DSizes, + const TensorVolumePatchOp > > > +#else +template +EIGEN_ALWAYS_INLINE static const TensorReshapingOp< + const Eigen::DSizes::NumDimensions>, + const TensorReductionOp< + internal::MaxReducer, + const Eigen::IndexList >, + const TensorReshapingOp< + const Eigen::DSizes, + const TensorVolumePatchOp > > > +#endif +CuboidMaxPooling(const Input& input, DenseIndex patchPlanes, + DenseIndex patchRows, DenseIndex patchCols, + DenseIndex stridePlanes, DenseIndex strideRows, + DenseIndex strideCols, const PaddingType padding_type) { + EIGEN_STATIC_ASSERT(internal::traits::NumDimensions == 5, + YOU_MADE_A_PROGRAMMING_MISTAKE); + static const bool isColMajor = (internal::traits::Layout == ColMajor); + + typedef typename internal::traits::Index TensorIndex; + TensorRef::Scalar, + internal::traits::NumDimensions, + internal::traits::Layout, TensorIndex> > + in(input); + + static const int idxPlanes = isColMajor ? 1 : 3; + static const int idxRows = 2; + static const int idxCols = isColMajor ? 3 : 1; + + // Molds the output of the reduction into the shape expected by the used + // (assuming col-major): + // - 1st dim: channels + // - 2nd dim: output depth + // - 3rd dim: output height + // - 4th dim: output width + // - 5th dim and beyond: everything else including batch size + Eigen::DSizes::NumDimensions> + post_reduce_dims; + post_reduce_dims[0] = in.dimension(0); + if (padding_type == PADDING_VALID) { + post_reduce_dims[idxPlanes] = Eigen::divup( + static_cast(in.dimension(idxPlanes)) - patchPlanes + 1, + stridePlanes); + post_reduce_dims[idxRows] = Eigen::divup( + static_cast(in.dimension(idxRows)) - patchRows + 1, + strideRows); + post_reduce_dims[idxCols] = Eigen::divup( + static_cast(in.dimension(idxCols)) - patchCols + 1, + strideCols); + } else { + post_reduce_dims[idxPlanes] = Eigen::divup( + static_cast(in.dimension(idxPlanes)), stridePlanes); + post_reduce_dims[idxRows] = Eigen::divup( + static_cast(in.dimension(idxRows)), strideRows); + post_reduce_dims[idxCols] = Eigen::divup( + static_cast(in.dimension(idxCols)), strideCols); + } + post_reduce_dims[4] = in.dimension(4); + + Eigen::DSizes pre_reduce_dims; + pre_reduce_dims[1] = patchRows * patchCols * patchPlanes; + if (isColMajor) { + pre_reduce_dims[0] = post_reduce_dims[0]; + pre_reduce_dims[2] = post_reduce_dims[1] * post_reduce_dims[2] * + post_reduce_dims[3] * post_reduce_dims[4]; + } else { + pre_reduce_dims[0] = post_reduce_dims[0] * post_reduce_dims[1] * + post_reduce_dims[2] * post_reduce_dims[3]; + pre_reduce_dims[2] = post_reduce_dims[4]; + } + +#if !defined(EIGEN_HAS_INDEX_LIST) + // nvcc doesn't support cxx11 + Eigen::array reduction_dims; + reduction_dims[0] = 1; +#else + // Take advantage of cxx11 to give the compiler information it can use to + // optimize the code. + Eigen::IndexList > reduction_dims; +#endif + return input + .extract_volume_patches(patchPlanes, patchRows, patchCols, stridePlanes, + strideRows, strideCols, padding_type, + -Eigen::NumTraits::highest()) + .reshape(pre_reduce_dims) + .maximum(reduction_dims) + .reshape(post_reduce_dims); +} + +/** SpatialAvgPooling + * \ingroup CXX11_NeuralNetworks_Module + * + * \brief Applies an average pooling over a multichannel input image. + * + * The input parameter is expected to be a tensor with a rank of 4 (channels, + * height, width, others in col-major, and the reverse of that in row-major). + * + * The result can be assigned to a tensor of rank equal to the rank of the + * input. The dimensions of the result will be channels, height, width, and + * others (in col-major, and the reverse of that if the input was row-major). + * + * The order of the width and height dimensions can be swapped if needed. + * + */ +namespace internal { + +template +struct AvgPoolMeanReducer { +#if (EIGEN_ARCH_i386 || EIGEN_ARCH_x86_64) && !defined(__CUDACC__) + // We only support packet access for floats. + static const bool PacketAccess = internal::is_same::value; +#else + static const bool PacketAccess = false; +#endif + static const bool IsStateful = true; + + EIGEN_DEVICE_FUNC EIGEN_ALWAYS_INLINE AvgPoolMeanReducer() : scalarCount_(0) { + typedef typename packet_traits::type Packet; + packetCount_ = pset1(T(0.0)); + } + + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE void reduce(const T t, T* accum) { + if (t != -Eigen::NumTraits::highest()) { + (*accum) = (*accum) + t; + scalarCount_++; + } + } + + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE T initialize() const { + return static_cast(0); + } + + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE T finalize(const T accum) const { + eigen_assert(scalarCount_ > 0); + return accum / T(scalarCount_); + } + +#if (EIGEN_ARCH_i386 || EIGEN_ARCH_x86_64) && !defined(__CUDACC__) +#ifdef EIGEN_VECTORIZE_AVX512 +#define pequal(a, b) \ + _mm512_castsi512_ps( \ + _mm512_maskz_set1_epi32(_mm512_cmp_ps_mask(a, b, _CMP_EQ_UQ), -1)) + + // The ternarylogic function immediate determines the values in the result + // In the case below, 0xd8 implies (false_mask) ? (b) : (a) + // For details, refer to the vpternlogd instruction table at + // http://www.intel.com/content/dam/www/public/us/en/documents/manuals/64-ia-32-architectures-software-developer-vol-2c-manual.pdf + +#define psel(a, b, false_mask) \ + _mm512_castsi512_ps(_mm512_ternarylogic_epi32( \ + _mm512_castps_si512(a), _mm512_castps_si512(b), \ + _mm512_castps_si512(false_mask), 0xd8)) +#elif defined EIGEN_VECTORIZE_AVX +#define pequal(a, b) _mm256_cmp_ps(a, b, _CMP_EQ_UQ) +#define psel(a, b, false_mask) _mm256_blendv_ps(a, b, false_mask) +#else +#define pequal(a, b) _mm_cmpeq_ps(a, b) +#define psel(a, b, false_mask) \ + _mm_or_ps(_mm_andnot_ps(false_mask, a), _mm_and_ps(false_mask, b)) +#endif + + template + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE void reducePacket(const Packet& p, + Packet* accum) { + reducePacketWithType(static_cast(0), p, accum); + } + + template + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE void reducePacketWithType( + T, const Packet& p, Packet* accum) { + Packet skip_mask = + pequal(p, pset1(-Eigen::NumTraits::highest())); + (*accum) = padd(*accum, psel(p, pset1(0), skip_mask)); + packetCount_ = padd( + packetCount_, psel(pset1(1), pset1(0), skip_mask)); + } + + template + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE Packet initializePacket() const { + return pset1(0); + } + + template + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE Packet + finalizePacket(const Packet& vaccum) const { + return pdiv(vaccum, packetCount_); + } + template + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE T + finalizeBoth(const T saccum, const Packet& vaccum) const { + return (saccum + predux(vaccum)) / (scalarCount_ + predux(packetCount_)); + } +#endif + + protected: + typedef typename packet_traits::type Packet; + int scalarCount_; + Packet packetCount_; +}; + +template +struct reducer_traits, Device> { + enum { + Cost = 1, +#if (EIGEN_ARCH_i386 || EIGEN_ARCH_x86_64) && !defined(__CUDACC__) + // We only support packet access for floats. + PacketAccess = true, +#else + PacketAccess = false, +#endif + IsStateful = true, + IsExactlyAssociative = false + }; +}; + +template <> +struct reducer_traits, GpuDevice> { + enum { + Cost = 1, + PacketAccess = false, + IsStateful = true, + IsExactlyAssociative = false + }; +}; + +} // namespace internal + +#if !defined(EIGEN_HAS_INDEX_LIST) +template +EIGEN_ALWAYS_INLINE static const TensorReshapingOp< + const Eigen::DSizes::Index, + internal::traits::NumDimensions>, + const TensorReductionOp< + internal::AvgPoolMeanReducer::Scalar>::type>, + const Eigen::array, + const TensorImagePatchOp > > +#else +template +EIGEN_ALWAYS_INLINE static const TensorReshapingOp< + const Eigen::DSizes::Index, + internal::traits::NumDimensions>, + const TensorReductionOp< + internal::AvgPoolMeanReducer::Scalar>::type>, + typename internal::conditional< + internal::traits::Layout == ColMajor, + const Eigen::IndexList, Eigen::type2index<2> >, + const Eigen::IndexList, + Eigen::type2index<3> > >::type, + const TensorImagePatchOp > > +#endif +SpatialAvgPooling(const Input& input, DenseIndex patchRows, + DenseIndex patchCols, DenseIndex strideRows, + DenseIndex strideCols, const PaddingType padding_type, + DenseIndex in_strideRows = 1, DenseIndex in_strideCols = 1) { + EIGEN_STATIC_ASSERT(internal::traits::NumDimensions == 4, + YOU_MADE_A_PROGRAMMING_MISTAKE); + + typedef typename internal::traits::Index TensorIndex; + TensorRef::Scalar, + internal::traits::NumDimensions, + internal::traits::Layout, TensorIndex> > + in(input); + + const DenseIndex patchRowsEff = + patchRows + (patchRows - 1) * (in_strideRows - 1); + const DenseIndex patchColsEff = + patchCols + (patchCols - 1) * (in_strideCols - 1); + + static const bool isColMajor = (internal::traits::Layout == ColMajor); + static const int idxRows = isColMajor ? 1 : 2; + static const int idxCols = isColMajor ? 2 : 1; + + // Molds the output of the reduction into the shape expected by the user. + // (assuming col-major): + // - 1st dim: channels + // - 2nd dim: output height + // - 3rd dim: output width + // - 4th dim and beyond: everything else including batch size + Eigen::DSizes::NumDimensions> + post_reduce_dims; + post_reduce_dims[0] = in.dimension(0); + if (padding_type == PADDING_VALID) { + post_reduce_dims[idxRows] = Eigen::divup( + static_cast(in.dimension(idxRows)) - patchRowsEff + 1, + strideRows); + post_reduce_dims[idxCols] = Eigen::divup( + static_cast(in.dimension(idxCols)) - patchColsEff + 1, + strideCols); + } else { + post_reduce_dims[idxRows] = Eigen::divup( + static_cast(in.dimension(idxRows)), strideRows); + post_reduce_dims[idxCols] = Eigen::divup( + static_cast(in.dimension(idxCols)), strideCols); + } + post_reduce_dims[3] = in.dimension(3); + + typedef typename internal::remove_const< + typename internal::traits::Scalar>::type CoeffReturnType; + internal::AvgPoolMeanReducer mean_with_nan; + +#if !defined(EIGEN_HAS_INDEX_LIST) + // nvcc doesn't support cxx11 + Eigen::array reduction_dims; + if (isColMajor) { + reduction_dims[0] = 1; + reduction_dims[1] = 2; + } else { + reduction_dims[0] = 2; + reduction_dims[1] = 3; + } +#else + // Take advantage of cxx11 to give the compiler information it can use to + // optimize the code. + typename internal::conditional< + internal::traits::Layout == ColMajor, + const Eigen::IndexList, Eigen::type2index<2> >, + const Eigen::IndexList, + Eigen::type2index<3> > >::type reduction_dims; +#endif + return input + .extract_image_patches(patchRows, patchCols, strideRows, strideCols, + in_strideRows, in_strideCols, padding_type, + -Eigen::NumTraits::highest()) + .reduce(reduction_dims, mean_with_nan) + .reshape(post_reduce_dims); +} + +/** CuboidAvgPooling + * \ingroup CXX11_NeuralNetworks_Module + * + * \brief Applies an average pooling over a multichannel input volume. + * + * The input parameter is expected to be a tensor with a rank of 5 (channels, + * depth, height, width, others, and the reverse of that in row-major). + * + * The result can be assigned to a tensor of rank equal to the rank of the + * input. The dimensions of the result will be channels, depth, width, and + * others (in col-major, and the reverse of that if the input was row-major). + * + * The order of the depth, width and height dimensions can be swapped if + * needed. + * + */ +#if !defined(EIGEN_HAS_INDEX_LIST) +template +EIGEN_ALWAYS_INLINE static const TensorReshapingOp< + const Eigen::DSizes::NumDimensions>, + const TensorReductionOp< + internal::AvgPoolMeanReducer, const Eigen::array, + const TensorReshapingOp< + const Eigen::DSizes, + const TensorVolumePatchOp > > > +#else +template +EIGEN_ALWAYS_INLINE static const TensorReshapingOp< + const Eigen::DSizes::NumDimensions>, + const TensorReductionOp< + internal::AvgPoolMeanReducer, + const Eigen::IndexList >, + const TensorReshapingOp< + const Eigen::DSizes, + const TensorVolumePatchOp > > > +#endif +CuboidAvgPooling(const Input& input, DenseIndex patchPlanes, + DenseIndex patchRows, DenseIndex patchCols, + DenseIndex stridePlanes, DenseIndex strideRows, + DenseIndex strideCols, const PaddingType padding_type) { + EIGEN_STATIC_ASSERT(internal::traits::NumDimensions == 5, + YOU_MADE_A_PROGRAMMING_MISTAKE); + static const bool isColMajor = (internal::traits::Layout == ColMajor); + + typedef typename internal::traits::Index TensorIndex; + TensorRef::Scalar, + internal::traits::NumDimensions, + internal::traits::Layout, TensorIndex> > + in(input); + + static const int idxPlanes = isColMajor ? 1 : 3; + static const int idxRows = 2; + static const int idxCols = isColMajor ? 3 : 1; + // Molds the output of the reduction into the shape expected by the used + // (assuming col-major): + // - 1st dim: channels + // - 2nd dim: outupt depth + // - 3rd dim: output height + // - 4th dim: output width + // - 5th dim and beyond: everything else including batch size + Eigen::DSizes::NumDimensions> + post_reduce_dims; + post_reduce_dims[0] = in.dimension(0); + if (padding_type == PADDING_VALID) { + post_reduce_dims[idxPlanes] = Eigen::divup( + static_cast(in.dimension(idxPlanes)) - patchPlanes + 1, + stridePlanes); + post_reduce_dims[idxRows] = Eigen::divup( + static_cast(in.dimension(idxRows)) - patchRows + 1, + strideRows); + post_reduce_dims[idxCols] = Eigen::divup( + static_cast(in.dimension(idxCols)) - patchCols + 1, + strideCols); + } else { + post_reduce_dims[idxPlanes] = Eigen::divup( + static_cast(in.dimension(idxPlanes)), stridePlanes); + post_reduce_dims[idxRows] = Eigen::divup( + static_cast(in.dimension(idxRows)), strideRows); + post_reduce_dims[idxCols] = Eigen::divup( + static_cast(in.dimension(idxCols)), strideCols); + } + post_reduce_dims[4] = in.dimension(4); + + Eigen::DSizes pre_reduce_dims; + pre_reduce_dims[1] = patchRows * patchCols * patchPlanes; + if (isColMajor) { + pre_reduce_dims[0] = post_reduce_dims[0]; + pre_reduce_dims[2] = post_reduce_dims[1] * post_reduce_dims[2] * + post_reduce_dims[3] * post_reduce_dims[4]; + } else { + pre_reduce_dims[0] = post_reduce_dims[0] * post_reduce_dims[1] * + post_reduce_dims[2] * post_reduce_dims[3]; + pre_reduce_dims[2] = post_reduce_dims[4]; + } + + typedef typename internal::remove_const< + typename internal::traits::Scalar>::type CoeffReturnType; + internal::AvgPoolMeanReducer mean_with_nan; + +#if !defined(EIGEN_HAS_INDEX_LIST) + // nvcc doesn't support cxx11 + Eigen::array reduction_dims; + reduction_dims[0] = 1; +#else + // Take advantage of cxx11 to give the compiler information it can use to + // optimize the code. + Eigen::IndexList > reduction_dims; +#endif + return input + .extract_volume_patches(patchPlanes, patchRows, patchCols, stridePlanes, + strideRows, strideCols, padding_type, + -Eigen::NumTraits::highest()) + .reshape(pre_reduce_dims) + .reduce(reduction_dims, mean_with_nan) + .reshape(post_reduce_dims); +} + +} // end namespace Eigen + +#endif // TENSORFLOW_CORE_KERNELS_EIGEN_POOLING_H_ diff --git a/linux/include/tensorflow/core/kernels/eigen_softmax.h b/linux/include/tensorflow/core/kernels/eigen_softmax.h new file mode 100644 index 000000000..12148c54b --- /dev/null +++ b/linux/include/tensorflow/core/kernels/eigen_softmax.h @@ -0,0 +1,99 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_EIGEN_SOFTMAX_H_ +#define TENSORFLOW_CORE_KERNELS_EIGEN_SOFTMAX_H_ + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" + +namespace Eigen { + +/** SoftMax + * \ingroup CXX11_NeuralNetworks_Module + * + * \brief Applies a softmax + * + * The input parameter is expected to be a col-major tensor with a rank of 2 + * (depth and other). + * + * The result can be assigned to a tensor of rank and dimensions equal to that + * of the input. The result will be laid out in col-major order. + * + */ + +namespace { +struct SoftmaxOp { + SoftmaxOp(const float beta) : beta_(beta) {} + + template + typename Input::Dimensions dimensions(const Input& input) const { + return input.dimensions(); + } + + template + void eval(const Input& input, Output& output, const Device& device) const { +#if !defined(EIGEN_HAS_INDEX_LIST) + // nvcc doesn't support cxx11 + Eigen::array::Index, 1> depth_dim; + depth_dim[0] = 0; + Eigen::array::Index, 2> bcast; + bcast[0] = dimensions(input)[0]; + bcast[1] = 1; + DSizes::Index, 2> dims2d; + dims2d[0] = 1; + dims2d[1] = dimensions(input)[1]; +#else + // Take advantage of cxx11 to give the compiler information it can use to + // optimize the code. + Eigen::IndexList > depth_dim; + Eigen::IndexList > bcast; + bcast.set(0, dimensions(input)[0]); + Eigen::IndexList, + typename internal::traits::Index> + dims2d; + dims2d.set(1, dimensions(input)[1]); +#endif + + output.device(device) = + ((input - + input.maximum(depth_dim).eval().reshape(dims2d).broadcast(bcast)) * + beta_) + .exp(); + output.device(device) = + output / + (output.sum(depth_dim).eval().reshape(dims2d).broadcast(bcast)); + } + + private: + const float beta_; +}; +} // namespace + +template +EIGEN_ALWAYS_INLINE static const TensorCustomUnaryOp +SoftMax(const Input& input, const float beta) { + EIGEN_STATIC_ASSERT(internal::traits::Layout == ColMajor, + YOU_MADE_A_PROGRAMMING_MISTAKE); + EIGEN_STATIC_ASSERT(internal::traits::NumDimensions == 2, + YOU_MADE_A_PROGRAMMING_MISTAKE); + + const SoftmaxOp op(beta); + return input.customOp(op); +} + +} // end namespace Eigen + +#endif // TENSORFLOW_CORE_KERNELS_EIGEN_SOFTMAX_H_ diff --git a/linux/include/tensorflow/core/kernels/eigen_spatial_convolutions.h b/linux/include/tensorflow/core/kernels/eigen_spatial_convolutions.h new file mode 100644 index 000000000..e926d73f8 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/eigen_spatial_convolutions.h @@ -0,0 +1,1366 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_EIGEN_SPATIAL_CONVOLUTIONS_H_ +#define TENSORFLOW_CORE_KERNELS_EIGEN_SPATIAL_CONVOLUTIONS_H_ + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" + +namespace Eigen { + +namespace internal { + +// WARNING: Most of the code here implicitly assumes that the matrix is in +// ColMajor layout. This is guaranteed by the tensor contraction (see +// TensorContraction.h). +// +// Inside Eigen a tensor contraction is represented by a matrix multiplication. +// We don't want to actually extract image patches and reshape the result into +// a matrix (this involves allocating huge extra memory), so the patch +// extraction and reshape operations are implicit. +// +// TensorContractionInputMapper takes a matrix index and returns the coefficient +// (or the packet) of the "virtual tensor", that would be at that index if we +// were to actually reshape the result of patch extraction. +// +// TensorContractionSubMapper provides a similar view into the "virtual matrix" +// at the given vertical and horizontal offsets. +// +// "Virtual matrix" dimensions: +// *0: kernelChannels * kernelRows * kernelCols; +// 1: out_height * out_width; * OTHERS (e.g batches, etc...) +// +// *) extracted patches are continuous in memory (innermost dimension assuming +// col major layout) +// +// With this dimensions: +// row - offset within a single patch (in code: patchId) +// col - index of the extracted patch (in code: patchIndex) +// patchIndex ∈ [0..num_patches * OTHERS] (batch and other dimensions) +// +// TODO(ezhulenev): Consolidate this part of the code with the image patch +// extraction code since they are both very similar. +template +class TensorContractionInputMapper< + Scalar_, Index, Side, + TensorEvaluator< + const TensorReshapingOp >, + Device>, + nocontract_t, contract_t, packet_size, inner_dim_contiguous, + inner_dim_reordered, Alignment> { + public: + typedef Scalar_ Scalar; + typedef TensorContractionInputMapper< + Scalar, Index, Side, + TensorEvaluator< + const TensorReshapingOp< + NewDimension, const TensorImagePatchOp >, + Device>, + nocontract_t, contract_t, packet_size, inner_dim_contiguous, + inner_dim_reordered, Alignment> + Self; + typedef TensorContractionSubMapper< + Scalar, Index, Side, + TensorEvaluator< + const TensorReshapingOp< + NewDimension, const TensorImagePatchOp >, + Device>, + nocontract_t, contract_t, packet_size, inner_dim_contiguous, + inner_dim_reordered, Alignment> + SubMapper; + typedef SubMapper VectorMapper; + typedef SubMapper LinearMapper; + typedef typename packet_traits::type Packet; + + EIGEN_DEVICE_FUNC + TensorContractionInputMapper( + const TensorEvaluator< + const TensorReshapingOp< + NewDimension, const TensorImagePatchOp >, + Device>& tensor, + const nocontract_t&, const nocontract_t&, const contract_t&, + const contract_t&) + : m_impl(tensor.impl().impl()) { + Index patch_rows; + Index patch_depth; + if (internal::traits::Layout == ColMajor) { + patch_depth = tensor.impl().dimensions()[0]; + patch_rows = tensor.impl().dimensions()[1]; + m_patch_cols = tensor.impl().dimensions()[2]; + m_num_patches = tensor.impl().dimensions()[3]; + } else { + const size_t NumDims = tensor.impl().dimensions().size(); + patch_depth = tensor.impl().dimensions()[NumDims - 1]; + patch_rows = tensor.impl().dimensions()[NumDims - 2]; + m_patch_cols = tensor.impl().dimensions()[NumDims - 3]; + m_num_patches = tensor.impl().dimensions()[NumDims - 4]; + } + + // Strides for navigating through the single patch. + m_patch_row_stride = patch_depth; + m_patch_col_stride = patch_rows * m_patch_row_stride; + + m_patch_row_inflate_strides = tensor.impl().rowInflateStride(); + m_patch_col_inflate_strides = tensor.impl().colInflateStride(); + + m_colStride = patch_rows; + + m_outputRows = tensor.impl().outputRows(); + m_row_strides = tensor.impl().userRowStride(); + m_col_strides = tensor.impl().userColStride(); + + m_in_row_strides = tensor.impl().userInRowStride(); + m_in_col_strides = tensor.impl().userInColStride(); + + if (internal::traits::Layout == ColMajor) { + m_inputRows = tensor.impl().impl().dimensions()[1]; + m_inputCols = tensor.impl().impl().dimensions()[2]; + } else { + const int NumDims = tensor.impl().impl().dimensions().size(); + m_inputRows = tensor.impl().impl().dimensions()[NumDims - 2]; + m_inputCols = tensor.impl().impl().dimensions()[NumDims - 3]; + } + + m_rowInputStride = patch_depth; + m_colInputStride = patch_depth * m_inputRows; + m_patchInputStride = patch_depth * m_inputRows * m_inputCols; + + m_rowPaddingTop = tensor.impl().rowPaddingTop(); + m_colPaddingLeft = tensor.impl().colPaddingLeft(); + + m_fastPatchRowStride = + internal::TensorIntDivisor(m_patch_row_stride); + m_fastPatchColStride = + internal::TensorIntDivisor(m_patch_col_stride); + m_fastInputRowStride = + internal::TensorIntDivisor(m_patch_row_inflate_strides); + m_fastInputColStride = + internal::TensorIntDivisor(m_patch_col_inflate_strides); + m_fastNumPatches = internal::TensorIntDivisor(m_num_patches); + m_fastColStride = internal::TensorIntDivisor(m_colStride); + m_fastOutputRows = internal::TensorIntDivisor(m_outputRows); + m_fastDimZero = internal::TensorIntDivisor(patch_depth); + } + + EIGEN_DEVICE_FUNC + TensorContractionInputMapper(const TensorContractionInputMapper& base_mapper) + : m_impl(base_mapper.m_impl) { + m_patch_cols = base_mapper.m_patch_cols; + m_num_patches = base_mapper.m_num_patches; + + m_patch_row_stride = base_mapper.m_patch_row_stride; + m_patch_col_stride = base_mapper.m_patch_col_stride; + + m_patch_row_inflate_strides = base_mapper.m_patch_row_inflate_strides; + m_patch_col_inflate_strides = base_mapper.m_patch_col_inflate_strides; + + m_colStride = base_mapper.m_colStride; + + m_rowInputStride = base_mapper.m_rowInputStride; + m_colInputStride = base_mapper.m_colInputStride; + m_patchInputStride = base_mapper.m_patchInputStride; + + m_inputRows = base_mapper.m_inputRows; + m_inputCols = base_mapper.m_inputCols; + + m_outputRows = base_mapper.m_outputRows; + m_row_strides = base_mapper.m_row_strides; + m_col_strides = base_mapper.m_col_strides; + + m_in_row_strides = base_mapper.m_in_row_strides; + m_in_col_strides = base_mapper.m_in_col_strides; + + m_rowPaddingTop = base_mapper.m_rowPaddingTop; + m_colPaddingLeft = base_mapper.m_colPaddingLeft; + + m_fastPatchRowStride = base_mapper.m_fastPatchRowStride; + m_fastPatchColStride = base_mapper.m_fastPatchColStride; + m_fastInputRowStride = base_mapper.m_fastInputRowStride; + m_fastInputColStride = base_mapper.m_fastInputColStride; + m_fastNumPatches = base_mapper.m_fastNumPatches; + m_fastColStride = base_mapper.m_fastColStride; + m_fastOutputRows = base_mapper.m_fastOutputRows; + m_fastDimZero = base_mapper.m_fastDimZero; + } + + // If true, turns off some optimizations for loading packets since the image + // patches are "non-standard" such as there are non-trivial strides or + // inflations in the input. + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE bool nonStandardPatches() const { + return m_in_row_strides != 1 || m_in_col_strides != 1 || + m_patch_row_inflate_strides != 1 || m_patch_col_inflate_strides != 1; + } + + EIGEN_DEVICE_FUNC + EIGEN_STRONG_INLINE SubMapper getSubMapper(Index i, Index j) const { + return SubMapper(*this, i, j); + } + + EIGEN_DEVICE_FUNC + EIGEN_STRONG_INLINE LinearMapper getLinearMapper(Index i, Index j) const { + return LinearMapper(*this, i, j); + } + + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE Scalar operator()(Index row) const { + Index rowIndex, colIndex, otherIndex; + computeBaseIndices(0, rowIndex, colIndex, otherIndex); + return loadCoeff(row, rowIndex, colIndex, otherIndex); + } + + // Load the coefficient at the patchIndex location instead of the usual + // m_rowIndex, + // m_colIndex, m_otherIndex. This is currently only used by the gpu code. + // EIGEN_DEVICE_FUNC + EIGEN_DEVICE_FUNC + EIGEN_STRONG_INLINE Scalar operator()(Index row, Index patchIndex) const { + Index rowIndex, colIndex, otherIndex; + computeBaseIndices(patchIndex, rowIndex, colIndex, otherIndex); + return loadCoeff(row, rowIndex, colIndex, otherIndex); + } + + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE Packet loadPacket(Index row) const { + Index rowIndex, colIndex, otherIndex; + computeBaseIndices(0, rowIndex, colIndex, otherIndex); + return loadPacket(row, rowIndex, colIndex, otherIndex); + } + + // Load the packet at the patchIndex location instead of the usual m_rowIndex, + // m_colIndex, m_otherIndex. This is currently only used by the gpu code. + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE Packet loadPacket(Index row, Index patchIndex) const { + Index rowIndex, colIndex, otherIndex; + computeBaseIndices(patchIndex, rowIndex, colIndex, otherIndex); + return loadPacket(row, rowIndex, colIndex, otherIndex); + } + + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE const TensorEvaluator& impl() const { + return m_impl; + } + + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE Index patchDepth() const { return m_rowInputStride; } + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE Index patchRows() const { return m_colStride; } + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE Index patchCols() const { return m_patch_cols; } + + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE Packet packetNoPadding(const Index depth, + const Index baseIndex) const { + const Index inputIndex = depth + baseIndex; + return m_impl.template packet(inputIndex); + } + + private: + friend class TensorContractionSubMapper< + Scalar, Index, Side, + TensorEvaluator< + const TensorReshapingOp< + NewDimension, const TensorImagePatchOp >, + Device>, + nocontract_t, contract_t, packet_size, inner_dim_contiguous, + inner_dim_reordered, Alignment>; + + // Load coefficient from a patch specified by the "within patch offset" + // (patchId) and the precomputed indices of the first element of the patch. + EIGEN_DEVICE_FUNC + EIGEN_STRONG_INLINE Scalar loadCoeff(Index patchId, Index rowIndex, + Index colIndex, Index otherIndex) const { + // Find the offset of the element wrt the location of the first element. + const Index patchOffset = patchId / m_fastDimZero; + + const Index colOffset = patchOffset / m_fastColStride; + const Index inputCol = colIndex + colOffset * m_in_col_strides; + const Index origInputCol = + (m_patch_col_inflate_strides == 1) + ? inputCol + : ((inputCol >= 0) ? (inputCol / m_fastInputColStride) : 0); + + const Index rowOffset = patchOffset - colOffset * m_colStride; + const Index inputRow = rowIndex + rowOffset * m_in_row_strides; + const Index origInputRow = + (m_patch_row_inflate_strides == 1) + ? inputRow + : ((inputRow >= 0) ? (inputRow / m_fastInputRowStride) : 0); + if (origInputCol < 0 || origInputRow < 0 || origInputCol >= m_inputCols || + origInputRow >= m_inputRows || + (inputCol != origInputCol * m_patch_col_inflate_strides) || + (inputRow != origInputRow * m_patch_row_inflate_strides)) { + return Scalar(0); + } + const Index depth = patchId - patchOffset * patchDepth(); + const Index inputIndex = depth + origInputRow * m_rowInputStride + + origInputCol * m_colInputStride + otherIndex; + return m_impl.coeff(inputIndex); + } + + // This is the same as loadCoeff(...), but optimized for all `inflate_strides` + // and `in_strides` equal to 1 (template specialization without templates). + EIGEN_DEVICE_FUNC + EIGEN_STRONG_INLINE Scalar loadCoeffStandard(Index patchId, Index rowIndex, + Index colIndex, + Index otherIndex) const { + eigen_assert(!nonStandardPatches()); + + // Find the offset of the element wrt the location of the first element. + const Index patchOffset = patchId / m_fastDimZero; + const Index colOffset = patchOffset / m_fastColStride; + const Index rowOffset = patchOffset - colOffset * m_colStride; + const Index inputCol = colIndex + colOffset; + const Index inputRow = rowIndex + rowOffset; + if (inputCol < 0 || inputCol >= m_inputCols || inputRow < 0 || + inputRow >= m_inputRows) { + return Scalar(0); + } + const Index depth = patchId - patchOffset * patchDepth(); + const Index inputIndex = depth + inputRow * m_rowInputStride + + inputCol * m_colInputStride + otherIndex; + return m_impl.coeff(inputIndex); + } + + // Load packet from a patch specified by the "within patch offset" + // (patchId) and the precomputed indices of the first element of the patch. + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE Packet loadPacket(Index patchId, Index rowIndex, + Index colIndex, + Index otherIndex) const { + const Index packetSize = internal::unpacket_traits::size; + EIGEN_STATIC_ASSERT(packetSize > 1, YOU_MADE_A_PROGRAMMING_MISTAKE) + eigen_assert(patchId < patchDepth() * patchRows() * m_patch_cols); + + if (nonStandardPatches()) { + return packetWithPossibleZero(patchId, rowIndex, colIndex, otherIndex); + } + return loadPacketStandard(patchId, rowIndex, colIndex, otherIndex); + } + + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE Packet loadPacketStandard(Index patchId, Index rowIndex, + Index colIndex, + Index otherIndex) const { + const Index packetSize = internal::unpacket_traits::size; + EIGEN_STATIC_ASSERT(packetSize > 1, YOU_MADE_A_PROGRAMMING_MISTAKE) + eigen_assert(patchId < patchDepth() * patchRows() * m_patch_cols); + + eigen_assert(!nonStandardPatches()); + + if ((patchDepth() % packetSize) == 0) { + return loadPacketFast(patchId, rowIndex, colIndex, otherIndex); + } else { + // Offsets and input calculation here are identical to + // loadCoeffStandard(...), but repeated twice. + + const Index patchOffsets[2] = { + patchId / m_fastDimZero, (patchId + packetSize - 1) / m_fastDimZero}; + + const Index colOffsets[2] = {patchOffsets[0] / m_fastColStride, + patchOffsets[1] / m_fastColStride}; + const Index inputCols[2] = {colIndex + colOffsets[0], + colIndex + colOffsets[1]}; + if (inputCols[0] >= m_inputCols || inputCols[1] < 0) { + // all zeros + return internal::pset1(Scalar(0)); + } + + if (inputCols[0] == inputCols[1]) { + const Index rowOffsets[2] = { + patchOffsets[0] - colOffsets[0] * m_colStride, + patchOffsets[1] - colOffsets[1] * m_colStride}; + eigen_assert(rowOffsets[0] <= rowOffsets[1]); + const Index inputRows[2] = {rowIndex + rowOffsets[0], + rowIndex + rowOffsets[1]}; + + if (inputRows[0] >= m_inputRows || inputRows[1] < 0) { + // all zeros + return internal::pset1(Scalar(0)); + } + + if (inputRows[0] >= 0 && inputRows[1] < m_inputRows) { + // no padding + const Index depth = patchId - patchOffsets[0] * patchDepth(); + const Index inputIndex = depth + inputRows[0] * m_rowInputStride + + inputCols[0] * m_colInputStride + otherIndex; + return m_impl.template packet(inputIndex); + } + } + } + return packetWithPossibleZero(patchId, rowIndex, colIndex, otherIndex); + } + + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE Packet loadPacketFast(Index patchId, Index rowIndex, + Index colIndex, + Index otherIndex) const { + const Index packetSize = internal::unpacket_traits::size; + EIGEN_STATIC_ASSERT(packetSize > 1, YOU_MADE_A_PROGRAMMING_MISTAKE) + eigen_assert(patchId < patchDepth() * patchRows() * m_patch_cols); + + eigen_assert(!nonStandardPatches()); + eigen_assert((patchDepth() % packetSize) == 0); + // Find the offset of the element wrt the location of the first element. + const Index patchOffset = patchId / m_fastDimZero; + eigen_assert((patchId + packetSize - 1) / m_fastDimZero == patchOffset); + + const Index colOffset = patchOffset / m_fastColStride; + const Index rowOffset = patchOffset - colOffset * m_colStride; + const Index inputCol = colIndex + colOffset; + const Index inputRow = rowIndex + rowOffset; + if (inputCol < 0 || inputRow < 0 || inputCol >= m_inputCols || + inputRow >= m_inputRows) { + // all zeros + return internal::pset1(Scalar(0)); + } + // no padding + const Index depth = patchId - patchOffset * patchDepth(); + const Index inputIndex = depth + inputRow * m_rowInputStride + + inputCol * m_colInputStride + otherIndex; + return m_impl.template packet(inputIndex); + } + + EIGEN_DEVICE_FUNC EIGEN_ALWAYS_INLINE Packet packetWithPossibleZero( + Index patchId, Index rowIndex, Index colIndex, Index otherIndex) const { + const int packetSize = internal::unpacket_traits::size; + EIGEN_ALIGN_MAX + typename internal::remove_const::type values[packetSize]; + for (int i = 0; i < packetSize; ++i) { + values[i] = loadCoeff(patchId + i, rowIndex, colIndex, otherIndex); + } + Packet rslt = internal::pload(values); + return rslt; + } + + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE void computeBaseIndices( + Index patchIndex, Index& rowIndex, Index& colIndex, + Index& otherIndex) const { + const size_t NumInputDims = array_size< + typename TensorEvaluator::Dimensions>::value; + otherIndex = (NumInputDims == 3) ? 0 : patchIndex / m_fastNumPatches; + const Index patch2DIndex = (NumInputDims == 3) + ? patchIndex + : (patchIndex - otherIndex * m_num_patches); + otherIndex *= m_patchInputStride; + colIndex = patch2DIndex / m_fastOutputRows; + rowIndex = patch2DIndex - colIndex * m_outputRows; + colIndex = colIndex * m_col_strides - m_colPaddingLeft; + rowIndex = rowIndex * m_row_strides - m_rowPaddingTop; + } + + Index m_patch_cols; // number of columns in the patch + Index m_num_patches; // number of patches to extract. + + // Strides for navigating through the single patch. + Index m_patch_row_stride; + Index m_patch_col_stride; + internal::TensorIntDivisor m_fastPatchRowStride; + internal::TensorIntDivisor m_fastPatchColStride; + + Index m_patch_row_inflate_strides; // the strides for row inflation in the + // image patch + Index m_patch_col_inflate_strides; // the strides for col inflation in the + // image patch + // Fast representation of inflation strides. + internal::TensorIntDivisor m_fastInputRowStride; + internal::TensorIntDivisor m_fastInputColStride; + + Index m_otherStride; + Index m_colStride; + internal::TensorIntDivisor m_fastNumPatches; + internal::TensorIntDivisor m_fastColStride; + + Index m_rowInputStride; // row stride in the input tensor + Index m_colInputStride; // col stride in the input tensor + Index m_patchInputStride; // patch stride in the input tensor + + Index m_inputRows; // Number of rows in the input tensor + Index m_inputCols; // Number of cols in the input tensor + + Index m_outputRows; // Number of patch rows + + Index m_row_strides; // User specified row stride + Index m_col_strides; // User specified col stride + + Index m_in_row_strides; // User specified input row stride + Index m_in_col_strides; // User specified input col stride + + Index m_rowPaddingTop; // Row padding + Index m_colPaddingLeft; // Column padding + + internal::TensorIntDivisor m_fastOutputRows; + internal::TensorIntDivisor m_fastDimZero; + + const TensorEvaluator m_impl; +}; + +template +class TensorContractionSubMapper< + Scalar, Index, Side, + TensorEvaluator< + const TensorReshapingOp >, + Device>, + nocontract_t, contract_t, packet_size, inner_dim_contiguous, + inner_dim_reordered, Alignment> { + public: + typedef typename packet_traits::type Packet; + typedef typename packet_traits::half HalfPacket; + + typedef TensorContractionInputMapper< + Scalar, Index, Side, + TensorEvaluator< + const TensorReshapingOp< + NewDimension, const TensorImagePatchOp >, + Device>, + nocontract_t, contract_t, packet_size, inner_dim_contiguous, + inner_dim_reordered, Alignment> + ParentMapper; + typedef TensorContractionSubMapper< + Scalar, Index, Side, + TensorEvaluator< + const TensorReshapingOp< + NewDimension, const TensorImagePatchOp >, + Device>, + nocontract_t, contract_t, packet_size, inner_dim_contiguous, + inner_dim_reordered, Alignment> + Self; + typedef Self LinearMapper; + + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE TensorContractionSubMapper( + const ParentMapper& base_mapper, Index vert_offset, Index horiz_offset) + : m_base_mapper(base_mapper), + m_depth_offset(vert_offset), + m_col_offset(horiz_offset) { + m_base_mapper.computeBaseIndices(m_col_offset, m_rowIndex, m_colIndex, + m_otherIndex); + } + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE TensorContractionSubMapper( + const Self& base_mapper, Index vert_offset, Index horiz_offset) + : m_base_mapper(base_mapper.m_base_mapper), + m_depth_offset(vert_offset + base_mapper.m_depth_offset), + m_col_offset(horiz_offset + base_mapper.m_col_offset) { + m_base_mapper.computeBaseIndices(m_col_offset, m_rowIndex, m_colIndex, + m_otherIndex); + } + EIGEN_DEVICE_FUNC EIGEN_ALWAYS_INLINE Scalar operator()(Index i) const { + return m_base_mapper.loadCoeff(i + m_depth_offset, m_rowIndex, m_colIndex, + m_otherIndex); + } + EIGEN_DEVICE_FUNC EIGEN_ALWAYS_INLINE Scalar operator()(Index i, + Index j) const { + return m_base_mapper(i + m_depth_offset, j + m_col_offset); + } + + EIGEN_DEVICE_FUNC EIGEN_ALWAYS_INLINE Packet loadPacket(Index i) const { + return m_base_mapper.loadPacket(i + m_depth_offset, m_rowIndex, m_colIndex, + m_otherIndex); + } + EIGEN_DEVICE_FUNC EIGEN_ALWAYS_INLINE Packet loadPacket(Index i, + Index j) const { + return m_base_mapper.template loadPacket(i + m_depth_offset, + j + m_col_offset); + } + + EIGEN_DEVICE_FUNC EIGEN_ALWAYS_INLINE Scalar + loadCoeffStandard(Index i) const { + return m_base_mapper.loadCoeffStandard(i + m_depth_offset, m_rowIndex, + m_colIndex, m_otherIndex); + } + + EIGEN_DEVICE_FUNC EIGEN_ALWAYS_INLINE Packet loadPacketFast(Index i) const { + return m_base_mapper.loadPacketFast(i + m_depth_offset, m_rowIndex, + m_colIndex, m_otherIndex); + } + EIGEN_DEVICE_FUNC EIGEN_ALWAYS_INLINE Packet + loadPacketStandard(Index i) const { + return m_base_mapper.loadPacketStandard(i + m_depth_offset, m_rowIndex, + m_colIndex, m_otherIndex); + } + template + EIGEN_DEVICE_FUNC bool aligned(Index) const { + return false; + } + + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE bool nonStandardPatches() const { + return m_base_mapper.nonStandardPatches(); + } + + // Max(Col|Row|Depth): compute the upper limit for the column, row and depth + // index respectively that fits into the peeled_k elements starting at + // m_depth_offset. + + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE Index maxCol(const Index peeled_k) const { + const Index max_col = + fastPatchColStride().divide(m_depth_offset + peeled_k); + return std::min(1 + max_col, patchCols()); + } + + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE Index maxRow(const Index peeled_k, + const Index col) const { + const Index max_row = fastPatchRowStride().divide( + m_depth_offset + peeled_k - col * patchColStride()); + return std::min(1 + max_row, patchRows()); + } + + // MaxDepth uses only the remaining number of elements in the peeled_k. + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE Index maxDepth(const Index num_elements, + const Index start_depth) const { + return std::min(start_depth + num_elements, patchDepth()); + } + + // Every register matters in this code, so sometimes to prevent register + // spilling, instead of the variable that you would expect to see, we use + // another one, that is guaranteed to have the same value. E.g. patch depth is + // always the same as input depth, and it's also the same as input row stride. + // Bunch of other parameters have similar relations. + + typedef internal::TensorIntDivisor IndexDivisor; + + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE Index patchDepth() const { + return m_base_mapper.m_rowInputStride; + } + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE Index patchRows() const { + return m_base_mapper.m_colStride; + } + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE Index patchCols() const { + return m_base_mapper.m_patch_cols; + } + + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE Index patchRowStride() const { + eigen_assert(patchDepth() == m_base_mapper.m_patch_row_stride && + "Patch depth must be equal to patch row stride."); + return patchDepth(); + } + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE Index patchColStride() const { + return m_base_mapper.m_patch_col_stride; + } + + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE IndexDivisor fastPatchRowStride() const { + eigen_assert(patchDepth() == m_base_mapper.m_patch_row_stride && + "Patch depth must be equal to patch row stride."); + return m_base_mapper.m_fastDimZero; // patch_depth + } + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE IndexDivisor fastPatchColStride() const { + return m_base_mapper.m_fastPatchColStride; + } + + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE Packet packetNoPadding(const Index depth, + const Index baseIndex) const { + const Index inputIndex = depth + baseIndex; + return m_base_mapper.m_impl.template packet(inputIndex); + } + + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE bool padRow(const Index row) const { + const Index r = m_rowIndex + row; + return r < 0 || r >= m_base_mapper.m_inputRows; + } + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE bool padCol(const Index col) const { + const Index c = m_colIndex + col; + return c < 0 || c >= m_base_mapper.m_inputCols; + } + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE Index baseIndex(const Index row, const Index col) const { + const Index r = m_rowIndex + row; + const Index c = m_colIndex + col; + return r * m_base_mapper.m_rowInputStride + + c * m_base_mapper.m_colInputStride + m_otherIndex; + } + + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE Index rowOffset() const { + const Index patchOffset = m_depth_offset / m_base_mapper.m_fastDimZero; + const Index colOffset = patchOffset / m_base_mapper.m_fastColStride; + return patchOffset - colOffset * m_base_mapper.m_colStride; + } + + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE Index colOffset() const { + const Index patchOffset = m_depth_offset / m_base_mapper.m_fastDimZero; + const Index colOffset = patchOffset / m_base_mapper.m_fastColStride; + return colOffset; + } + + EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE Index depthOffset() const { + return m_depth_offset % patchDepth(); + } + + EIGEN_DEVICE_FUNC EIGEN_ALWAYS_INLINE LinearMapper + getLinearMapper(Index i, Index j) const { + return LinearMapper(m_base_mapper, i + m_depth_offset, j + m_col_offset); + } + + private: + const ParentMapper& m_base_mapper; // that was a reference before + Index m_depth_offset; // First row in the input matrix + Index m_col_offset; // First col in the input matrix + + // Knowing that: col_offset == patchIndex * OTHERS, we keep precomputed base + // indices for the first element in a patch specified by col_offset + // (see computeBaseIndices(...) for details). + Index m_rowIndex; + Index m_colIndex; + Index m_otherIndex; +}; + +// Arrange a block of the right input matrix (in our case it's always a "virtual +// matrix" constructed from extracted image patches) in contiguous memory. +// +// Given column major input (A0 beside A1 in memory): +// A0 B0 C0 D0 E0 F0 G0 H0 ... Z0 +// A1 B1 C1 D1 E1 F1 G1 H1 ... Z1 +// A2 B2 C2 D2 E2 F2 G2 H2 ... Z2 +// A3 B3 C3 D3 E3 F3 G3 H3 ... Z3 +// A4 B4 C4 D4 E4 F4 G4 H4 ... Z4 +// A5 B5 C5 D5 E5 F5 G5 H5 ... Z5 +// A6 B6 C6 D6 E6 F6 G6 H6 ... Z6 +// A7 B7 C7 D7 E7 F7 G7 H7 ... Z7 +// A8 ... +// ... +// +// *) A, B, C, ... - patches extracted from the original input. +// *) A0, A1, A2 ... - values from the same patch at different offsets. +// +// The traversal (packed rhs memory) order (B0 besides A0 in memory): +// A0 B0 C0 D0 A1 B1 C1 D1 ... +// E0 F0 G0 H0 E1 F1 G1 H1 ... +// ... +// Z0 Z1 Z2 Z3 Z4 Z5 Z6 Z7 ... <- doesn't belong to any block (nr = 4) +// +// This traversal order must be the same as in default gemm_pack_rhs defined in +// GeneralBlockPanelKernel.h. +// +// *) nr - number of registers along the 'n' dimension. +// See GeneralBlockPanelKernel.h and "Anatomy of High-Performance Matrix +// Multiplication" paper. +template +struct gemm_pack_rhs< + Scalar, Index, + TensorContractionSubMapper< + Scalar, Index, Rhs, + TensorEvaluator< + const TensorReshapingOp< + NewDimension, const TensorImagePatchOp >, + Device>, + nocontract_t, contract_t, packet_size, inner_dim_contiguous, + inner_dim_reordered, Alignment>, + nr, ColMajor, false, false> { + typedef TensorContractionSubMapper< + Scalar, Index, Rhs, + TensorEvaluator< + const TensorReshapingOp< + NewDimension, const TensorImagePatchOp >, + Device>, + nocontract_t, contract_t, packet_size, inner_dim_contiguous, + inner_dim_reordered, Alignment> + SubMapper; + typedef SubMapper DataMapper; + typedef typename packet_traits::type Packet; + + EIGEN_STATIC_ASSERT((nr == 4), YOU_MADE_A_PROGRAMMING_MISTAKE); + + EIGEN_DEVICE_FUNC + EIGEN_DONT_INLINE void operator()(Scalar* block, const DataMapper& rhs, + Index depth, Index cols, Index stride = 0, + Index offset = 0) const { + eigen_assert(stride == 0); + eigen_assert(offset == 0); + + const Index packet_cols4 = (cols / 4) * 4; + const Index peeled_k = (depth / packet_size) * packet_size; + const bool non_standard_patches = rhs.nonStandardPatches(); + + for (Index j2 = 0; j2 < packet_cols4; j2 += 4) { + const SubMapper dm0 = rhs.getLinearMapper(0, j2 + 0); + const SubMapper dm1 = rhs.getLinearMapper(0, j2 + 1); + const SubMapper dm2 = rhs.getLinearMapper(0, j2 + 2); + const SubMapper dm3 = rhs.getLinearMapper(0, j2 + 3); + + Index k = 0; + if ((packet_size % 4) == 0 && !non_standard_patches) { + // FAST PATH: + // Iterate over patch columns and rows, if we know that a single + // packet do not span across multiple rows or columns. + if ((rhs.patchDepth() % packet_size) == 0) { + const Index start_col = rhs.colOffset(); + const Index max_col = rhs.maxCol(peeled_k); + + for (Index c = start_col; c < max_col; ++c) { + eigen_assert(k <= peeled_k); + + const Index start_row = (c == start_col) ? rhs.rowOffset() : 0; + const Index max_row = rhs.maxRow(peeled_k, c); + + const bool pad_col0 = dm0.padCol(c); + const bool pad_col1 = dm1.padCol(c); + const bool pad_col2 = dm2.padCol(c); + const bool pad_col3 = dm3.padCol(c); + + for (Index r = start_row; r < max_row; ++r) { + eigen_assert(k <= peeled_k); + + const bool pad0 = pad_col0 || dm0.padRow(r); + const bool pad1 = pad_col1 || dm1.padRow(r); + const bool pad2 = pad_col2 || dm2.padRow(r); + const bool pad3 = pad_col3 || dm3.padRow(r); + + const Index idx0 = dm0.baseIndex(r, c); + const Index idx1 = dm1.baseIndex(r, c); + const Index idx2 = dm2.baseIndex(r, c); + const Index idx3 = dm3.baseIndex(r, c); + + const Index start_depth = ((c == start_col) && (r == start_row)) + ? rhs.depthOffset() + : 0; + const Index max_depth = rhs.maxDepth(peeled_k - k, start_depth); + eigen_assert((max_depth - start_depth) % packet_size == 0); + + for (Index d = start_depth; d < max_depth; d += packet_size) { + eigen_assert(k < peeled_k); + PacketBlock kernel; + kernel.packet[0] = pad0 ? pset1(Scalar(0)) + : rhs.packetNoPadding(d, idx0); + kernel.packet[1] = pad1 ? pset1(Scalar(0)) + : rhs.packetNoPadding(d, idx1); + kernel.packet[2] = pad2 ? pset1(Scalar(0)) + : rhs.packetNoPadding(d, idx2); + kernel.packet[3] = pad3 ? pset1(Scalar(0)) + : rhs.packetNoPadding(d, idx3); + ptranspose(kernel); + pstoreu(block + 0 * packet_size, kernel.packet[0]); + pstoreu(block + 1 * packet_size, kernel.packet[1]); + pstoreu(block + 2 * packet_size, kernel.packet[2]); + pstoreu(block + 3 * packet_size, kernel.packet[3]); + block += 4 * packet_size; + k += packet_size; + } + } + } + + // The loop above should fill peeled_k elements. + eigen_assert(peeled_k == k); + + } else { + for (; k < peeled_k; k += packet_size) { + PacketBlock kernel; + kernel.packet[0] = dm0.loadPacketStandard(k); + kernel.packet[1] = dm1.loadPacketStandard(k); + kernel.packet[2] = dm2.loadPacketStandard(k); + kernel.packet[3] = dm3.loadPacketStandard(k); + ptranspose(kernel); + pstoreu(block + 0 * packet_size, kernel.packet[0]); + pstoreu(block + 1 * packet_size, kernel.packet[1]); + pstoreu(block + 2 * packet_size, kernel.packet[2]); + pstoreu(block + 3 * packet_size, kernel.packet[3]); + block += 4 * packet_size; + } + } + } + + // Copy the remaining coefficients of the column block after the peeled_k. + if (!rhs.nonStandardPatches()) { + for (; k < depth; k++) { + block[0] = dm0.loadCoeffStandard(k); + block[1] = dm1.loadCoeffStandard(k); + block[2] = dm2.loadCoeffStandard(k); + block[3] = dm3.loadCoeffStandard(k); + block += 4; + } + } else { + for (; k < depth; k++) { + block[0] = dm0(k); + block[1] = dm1(k); + block[2] = dm2(k); + block[3] = dm3(k); + block += 4; + } + } + } + + // copy the remaining columns one at a time (nr==1) + for (Index j2 = packet_cols4; j2 < cols; ++j2) { + const SubMapper dm0 = rhs.getLinearMapper(0, j2); + for (Index k = 0; k < depth; k++) { + *block = dm0(k); + block += 1; + } + } + } +}; + +// Template specialization for packet_size = 2. We must special-case packet +// blocks with nr > packet_size, e.g. PacketBlock. +template +struct gemm_pack_rhs< + Scalar, Index, + TensorContractionSubMapper< + Scalar, Index, Rhs, + TensorEvaluator< + const TensorReshapingOp< + NewDimension, const TensorImagePatchOp >, + Device>, + nocontract_t, contract_t, 2, inner_dim_contiguous, inner_dim_reordered, + Alignment>, + nr, ColMajor, false, false> { + typedef TensorContractionSubMapper< + Scalar, Index, Rhs, + TensorEvaluator< + const TensorReshapingOp< + NewDimension, const TensorImagePatchOp >, + Device>, + nocontract_t, contract_t, 2, inner_dim_contiguous, inner_dim_reordered, + Alignment> + SubMapper; + typedef SubMapper DataMapper; + typedef typename packet_traits::type Packet; + + EIGEN_STATIC_ASSERT((nr == 4), YOU_MADE_A_PROGRAMMING_MISTAKE); + + EIGEN_DEVICE_FUNC + EIGEN_DONT_INLINE void operator()(Scalar* block, const DataMapper& rhs, + Index depth, Index cols, Index stride = 0, + Index offset = 0) const { + eigen_assert(stride == 0); + eigen_assert(offset == 0); + + const int packet_size = 2; + const Index packet_cols4 = (cols / 4) * 4; + const Index peeled_k = (depth / packet_size) * packet_size; + const bool non_standard_patches = rhs.nonStandardPatches(); + + for (Index j2 = 0; j2 < packet_cols4; j2 += 4) { + const SubMapper dm0 = rhs.getLinearMapper(0, j2 + 0); + const SubMapper dm1 = rhs.getLinearMapper(0, j2 + 1); + const SubMapper dm2 = rhs.getLinearMapper(0, j2 + 2); + const SubMapper dm3 = rhs.getLinearMapper(0, j2 + 3); + + Index k = 0; + if (!non_standard_patches) { + // FAST PATH: + // Iterate over patch columns and rows if we know that a single + // packet do not span across multiple rows or columns. + if ((rhs.patchDepth() % packet_size) == 0) { + const Index start_col = rhs.colOffset(); + const Index max_col = rhs.maxCol(peeled_k); + + for (Index c = start_col; c < max_col; ++c) { + eigen_assert(k <= peeled_k); + + const Index start_row = (c == start_col) ? rhs.rowOffset() : 0; + const Index max_row = rhs.maxRow(peeled_k, c); + + const bool pad_col0 = dm0.padCol(c); + const bool pad_col1 = dm1.padCol(c); + const bool pad_col2 = dm2.padCol(c); + const bool pad_col3 = dm3.padCol(c); + + for (Index r = start_row; r < max_row; ++r) { + eigen_assert(k <= peeled_k); + + const bool pad0 = pad_col0 || dm0.padRow(r); + const bool pad1 = pad_col1 || dm1.padRow(r); + const bool pad2 = pad_col2 || dm2.padRow(r); + const bool pad3 = pad_col3 || dm3.padRow(r); + + const Index idx0 = dm0.baseIndex(r, c); + const Index idx1 = dm1.baseIndex(r, c); + const Index idx2 = dm2.baseIndex(r, c); + const Index idx3 = dm3.baseIndex(r, c); + + const Index start_depth = ((c == start_col) && (r == start_row)) + ? rhs.depthOffset() + : 0; + const Index max_depth = rhs.maxDepth(peeled_k - k, start_depth); + eigen_assert((max_depth - start_depth) % packet_size == 0); + + for (Index d = start_depth; d < max_depth; d += packet_size) { + eigen_assert(k < peeled_k); + PacketBlock kernel0; + PacketBlock kernel1; + kernel0.packet[0] = pad0 ? pset1(Scalar(0)) + : rhs.packetNoPadding(d, idx0); + kernel0.packet[1] = pad1 ? pset1(Scalar(0)) + : rhs.packetNoPadding(d, idx1); + kernel1.packet[0] = pad2 ? pset1(Scalar(0)) + : rhs.packetNoPadding(d, idx2); + kernel1.packet[1] = pad3 ? pset1(Scalar(0)) + : rhs.packetNoPadding(d, idx3); + ptranspose(kernel0); + ptranspose(kernel1); + pstoreu(block + 0 * packet_size, kernel0.packet[0]); + pstoreu(block + 1 * packet_size, kernel1.packet[0]); + pstoreu(block + 2 * packet_size, kernel0.packet[1]); + pstoreu(block + 3 * packet_size, kernel1.packet[1]); + block += 4 * packet_size; + k += packet_size; + } + } + } + + // The loop above should fill peeled_k elements. + eigen_assert(peeled_k == k); + + } else { + // Packet can span multiple rows or columns, so we have to go + // though the slower "standard" path. + for (; k < peeled_k; k += packet_size) { + PacketBlock kernel0; + PacketBlock kernel1; + kernel0.packet[0] = dm0.loadPacketStandard(k); + kernel0.packet[1] = dm1.loadPacketStandard(k); + kernel1.packet[0] = dm2.loadPacketStandard(k); + kernel1.packet[1] = dm3.loadPacketStandard(k); + ptranspose(kernel0); + ptranspose(kernel1); + pstoreu(block + 0 * packet_size, kernel0.packet[0]); + pstoreu(block + 1 * packet_size, kernel1.packet[0]); + pstoreu(block + 2 * packet_size, kernel0.packet[1]); + pstoreu(block + 3 * packet_size, kernel1.packet[1]); + block += 4 * packet_size; + } + } + } + + // Copy the remaining coefficients of the column block after the peeled_k. + if (!non_standard_patches) { + for (; k < depth; k++) { + block[0] = dm0.loadCoeffStandard(k); + block[1] = dm1.loadCoeffStandard(k); + block[2] = dm2.loadCoeffStandard(k); + block[3] = dm3.loadCoeffStandard(k); + block += 4; + } + } else { + for (; k < depth; k++) { + block[0] = dm0(k); + block[1] = dm1(k); + block[2] = dm2(k); + block[3] = dm3(k); + block += 4; + } + } + } + + // Copy the remaining columns one at a time (nr==1). + for (Index j2 = packet_cols4; j2 < cols; ++j2) { + const SubMapper dm0 = rhs.getLinearMapper(0, j2); + for (Index k = 0; k < depth; k++) { + *block = dm0(k); + block += 1; + } + } + } +}; + +// Special case for non-vectorized types such as float16. +template +struct gemm_pack_rhs< + Scalar, Index, + TensorContractionSubMapper< + Scalar, Index, Rhs, + TensorEvaluator< + const TensorReshapingOp< + NewDimension, const TensorImagePatchOp >, + Device>, + nocontract_t, contract_t, 1, inner_dim_contiguous, inner_dim_reordered, + Alignment>, + nr, ColMajor, false, false> { + typedef TensorContractionSubMapper< + Scalar, Index, Rhs, + TensorEvaluator< + const TensorReshapingOp< + NewDimension, const TensorImagePatchOp >, + Device>, + nocontract_t, contract_t, 1, inner_dim_contiguous, inner_dim_reordered, + Alignment> + SubMapper; + typedef SubMapper DataMapper; + + EIGEN_STATIC_ASSERT((nr == 4), YOU_MADE_A_PROGRAMMING_MISTAKE); + + EIGEN_DEVICE_FUNC + EIGEN_DONT_INLINE void operator()(Scalar* block, const DataMapper& rhs, + Index depth, Index cols, Index stride = 0, + Index offset = 0) const { + eigen_assert(stride == 0); + eigen_assert(offset == 0); + + const Index packet_cols4 = (cols / 4) * 4; + + for (Index j2 = 0; j2 < packet_cols4; j2 += 4) { + const SubMapper dm0 = rhs.getLinearMapper(0, j2 + 0); + const SubMapper dm1 = rhs.getLinearMapper(0, j2 + 1); + const SubMapper dm2 = rhs.getLinearMapper(0, j2 + 2); + const SubMapper dm3 = rhs.getLinearMapper(0, j2 + 3); + + if (!rhs.nonStandardPatches()) { + for (Index k = 0; k < depth; k++) { + block[0] = dm0.loadCoeffStandard(k); + block[1] = dm1.loadCoeffStandard(k); + block[2] = dm2.loadCoeffStandard(k); + block[3] = dm3.loadCoeffStandard(k); + block += 4; + } + } else { + for (Index k = 0; k < depth; k++) { + block[0] = dm0(k); + block[1] = dm1(k); + block[2] = dm2(k); + block[3] = dm3(k); + block += 4; + } + } + } + + // Copy the remaining columns one at a time (nr==1). + for (Index j2 = packet_cols4; j2 < cols; ++j2) { + const SubMapper dm0 = rhs.getLinearMapper(0, j2); + for (Index k = 0; k < depth; k++) { + *block = dm0(k); + block += 1; + } + } + } +}; + +} // end namespace internal + +/** SpatialConvolution + * \ingroup CXX11_NeuralNetworks_Module + * + * \brief Applies a 2D convolution over a multichannel input image. + * + * The input parameter is expected to be a tensor with a rank of 3 or more + * (channels, height, width, and optionally others) + * The kernel parameter is expected to be a 4D tensor (filters, channels, + * kernel_height, kernel_width) + * The input and the kernel must both be in col-major layout. The result will + * also be in col-major layout. + * + * If col_in_stride, row_in_stride > 1, then applies convolution with holes + * (aka atrous convolution), sampling every col_in_stride, row_in_stride input + * pixels. + * + * The result can be assigned to a tensor of rank equal to the rank of the + * input. The dimensions of the result will be filters, height, width (and + * others if applicable). + * + * It is possible to swap the order of the width and height dimensions provided + * that the same order is used in the input, the kernel, and the output. + * + */ +template +EIGEN_DEVICE_FUNC + EIGEN_ALWAYS_INLINE static const typename internal::conditional< + internal::traits::Layout == ColMajor, + TensorReshapingOp< + const DSizes::Index, + internal::traits::NumDimensions>, + const TensorContractionOp< + const array::Index>, + 1>, + const TensorReshapingOp< + const DSizes::Index, 2>, + const Kernel>, + const TensorReshapingOp< + const DSizes::Index, 2>, + const TensorImagePatchOp > > >, + TensorReshapingOp< + const DSizes::Index, + internal::traits::NumDimensions>, + const TensorContractionOp< + const array::Index>, + 1>, + const TensorReshapingOp< + const DSizes::Index, 2>, + const TensorImagePatchOp >, + const TensorReshapingOp< + const DSizes::Index, 2>, + const Kernel> > > >::type + SpatialConvolution(const Input& input, const Kernel& kernel, + const DenseIndex row_stride = 1, + const DenseIndex col_stride = 1, + const PaddingType padding_type = PADDING_SAME, + const DenseIndex row_in_stride = 1, + const DenseIndex col_in_stride = 1) { + typedef typename internal::traits::Index TensorIndex; + TensorRef::Scalar, + internal::traits::NumDimensions, + internal::traits::Layout, TensorIndex> > + in(input); + TensorRef::Scalar, + internal::traits::NumDimensions, + internal::traits::Layout, TensorIndex> > + kern(kernel); + + EIGEN_STATIC_ASSERT( + internal::traits::Layout == internal::traits::Layout, + YOU_MADE_A_PROGRAMMING_MISTAKE); + const bool isColMajor = (internal::traits::Layout == ColMajor); + + const int NumDims = internal::traits::NumDimensions; + + // Number of filters to apply. This is the same as the output depth of the + // result + const TensorIndex kernelFilters = + isColMajor ? kern.dimensions()[0] : kern.dimensions()[3]; + // Number of channels. This is the same as the input depth. + const TensorIndex kernelChannels = + isColMajor ? kern.dimensions()[1] : kern.dimensions()[2]; + const TensorIndex kernelRows = + isColMajor ? kern.dimensions()[2] : kern.dimensions()[1]; + const TensorIndex kernelCols = + isColMajor ? kern.dimensions()[3] : kern.dimensions()[0]; + + const DenseIndex kernelRowsEff = + kernelRows + (kernelRows - 1) * (row_in_stride - 1); + const DenseIndex kernelColsEff = + kernelCols + (kernelCols - 1) * (col_in_stride - 1); + + array, 1> contract_dims; + contract_dims[0] = IndexPair(1, 0); + + const TensorIndex InputRows = + isColMajor ? in.dimension(1) : in.dimension(NumDims - 2); + const TensorIndex InputCols = + isColMajor ? in.dimension(2) : in.dimension(NumDims - 3); + + TensorIndex out_height; + TensorIndex out_width; + switch (padding_type) { + case PADDING_VALID: + out_height = numext::ceil((InputRows - kernelRowsEff + 1.f) / + static_cast(row_stride)); + out_width = numext::ceil((InputCols - kernelColsEff + 1.f) / + static_cast(col_stride)); + break; + case PADDING_SAME: + out_height = numext::ceil(InputRows / static_cast(row_stride)); + out_width = numext::ceil(InputCols / static_cast(col_stride)); + break; + default: + // Initialize unused variables to avoid a compiler warning + out_height = 0; + out_width = 0; + eigen_assert(false && "unexpected padding"); + } + + // Molds the output of the patch extraction code into a 2d tensor: + // - the first dimension (dims[0]): the patch values to be multiplied with the + // kernels + // - the second dimension (dims[1]): everything else + DSizes pre_contract_dims; + if (isColMajor) { + pre_contract_dims[0] = kernelChannels * kernelRows * kernelCols; + pre_contract_dims[1] = out_height * out_width; + for (int i = 3; i < NumDims; ++i) { + pre_contract_dims[1] *= in.dimension(i); + } + } else { + pre_contract_dims[1] = kernelChannels * kernelRows * kernelCols; + pre_contract_dims[0] = out_height * out_width; + for (int i = 0; i < NumDims - 3; ++i) { + pre_contract_dims[0] *= in.dimension(i); + } + } + + // Molds the output of the contraction into the shape expected by the used + // (assuming this is ColMajor): + // - 1st dim: kernel filters + // - 2nd dim: output height + // - 3rd dim: output width + // - 4th dim and beyond: everything else including batch size + DSizes post_contract_dims; + if (isColMajor) { + post_contract_dims[0] = kernelFilters; + post_contract_dims[1] = out_height; + post_contract_dims[2] = out_width; + for (int i = 3; i < NumDims; ++i) { + post_contract_dims[i] = in.dimension(i); + } + } else { + post_contract_dims[NumDims - 1] = kernelFilters; + post_contract_dims[NumDims - 2] = out_height; + post_contract_dims[NumDims - 3] = out_width; + for (int i = 0; i < NumDims - 3; ++i) { + post_contract_dims[i] = in.dimension(i); + } + } + + DSizes kernel_dims; + if (isColMajor) { + kernel_dims[0] = kernelFilters; + kernel_dims[1] = kernelChannels * kernelRows * kernelCols; + } else { + kernel_dims[0] = kernelChannels * kernelRows * kernelCols; + kernel_dims[1] = kernelFilters; + } + // TODO(yangke): choose() is defined in TensorContraction.h -- consider + // moving it to somewhere more "common". + return choose( + Cond::Layout == ColMajor>(), + kernel.reshape(kernel_dims) + .contract(input + .extract_image_patches( + kernelRows, kernelCols, row_stride, col_stride, + row_in_stride, col_in_stride, padding_type) + .reshape(pre_contract_dims), + contract_dims) + .reshape(post_contract_dims), + input + .extract_image_patches(kernelRows, kernelCols, row_stride, col_stride, + row_in_stride, col_in_stride, padding_type) + .reshape(pre_contract_dims) + .contract(kernel.reshape(kernel_dims), contract_dims) + .reshape(post_contract_dims)); +} + +} // end namespace Eigen + +#endif // TENSORFLOW_CORE_KERNELS_EIGEN_SPATIAL_CONVOLUTIONS_H_ diff --git a/linux/include/tensorflow/core/kernels/eigen_volume_patch.h b/linux/include/tensorflow/core/kernels/eigen_volume_patch.h new file mode 100644 index 000000000..80ab745bf --- /dev/null +++ b/linux/include/tensorflow/core/kernels/eigen_volume_patch.h @@ -0,0 +1,657 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_EIGEN_VOLUME_PATCH_H_ +#define TENSORFLOW_CORE_KERNELS_EIGEN_VOLUME_PATCH_H_ + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" + +namespace Eigen { + +// Changes the interpretation of padding in TensorVolumePatchOp to be compatible +// with the rest of TensorFlow (odd padding is split so that more padding is put +// on the right end of the tensor). +template +struct CustomTensorEvaluator { + typedef TensorVolumePatchOp XprType; + typedef typename XprType::Index Index; + static const int NumInputDims = internal::array_size< + typename TensorEvaluator::Dimensions>::value; + static const int NumDims = NumInputDims + 1; + typedef DSizes Dimensions; + typedef + typename internal::remove_const::type Scalar; + typedef typename XprType::CoeffReturnType CoeffReturnType; + typedef typename PacketType::type PacketReturnType; + static const Index PacketSize = + internal::unpacket_traits::size; + + enum { + IsAligned = false, + PacketAccess = TensorEvaluator::PacketAccess, + BlockAccess = false, + PreferBlockAccess = false, + Layout = TensorEvaluator::Layout, + CoordAccess = NumDims == 6, + RawAccess = false + }; + + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE + CustomTensorEvaluator(const XprType& op, const Device& device) + : m_impl(op.expression(), device) { + EIGEN_STATIC_ASSERT(NumDims >= 5, YOU_MADE_A_PROGRAMMING_MISTAKE); + + m_paddingValue = op.padding_value(); + + const typename TensorEvaluator::Dimensions& input_dims = + m_impl.dimensions(); + + // Cache a few variables. + if (static_cast(Layout) == static_cast(ColMajor)) { + m_inputDepth = input_dims[0]; + m_inputPlanes = input_dims[1]; + m_inputRows = input_dims[2]; + m_inputCols = input_dims[3]; + } else { + m_inputDepth = input_dims[NumInputDims - 1]; + m_inputPlanes = input_dims[NumInputDims - 2]; + m_inputRows = input_dims[NumInputDims - 3]; + m_inputCols = input_dims[NumInputDims - 4]; + } + + m_plane_strides = op.plane_strides(); + m_row_strides = op.row_strides(); + m_col_strides = op.col_strides(); + + // Input strides and effective input/patch size + m_in_plane_strides = op.in_plane_strides(); + m_in_row_strides = op.in_row_strides(); + m_in_col_strides = op.in_col_strides(); + m_plane_inflate_strides = op.plane_inflate_strides(); + m_row_inflate_strides = op.row_inflate_strides(); + m_col_inflate_strides = op.col_inflate_strides(); + + // The "effective" spatial size after inflating data with zeros. + m_input_planes_eff = (m_inputPlanes - 1) * m_plane_inflate_strides + 1; + m_input_rows_eff = (m_inputRows - 1) * m_row_inflate_strides + 1; + m_input_cols_eff = (m_inputCols - 1) * m_col_inflate_strides + 1; + m_patch_planes_eff = + op.patch_planes() + (op.patch_planes() - 1) * (m_in_plane_strides - 1); + m_patch_rows_eff = + op.patch_rows() + (op.patch_rows() - 1) * (m_in_row_strides - 1); + m_patch_cols_eff = + op.patch_cols() + (op.patch_cols() - 1) * (m_in_col_strides - 1); + + if (op.padding_explicit()) { + m_outputPlanes = Eigen::divup( + m_input_planes_eff + + static_cast(op.padding_top_z() + op.padding_bottom_z()) - + m_patch_planes_eff + 1, + m_plane_strides); + m_outputRows = Eigen::divup( + m_input_rows_eff + + static_cast(op.padding_top() + op.padding_bottom()) - + m_patch_rows_eff + 1, + m_row_strides); + m_outputCols = Eigen::divup( + m_input_cols_eff + + static_cast(op.padding_left() + op.padding_right()) - + m_patch_cols_eff + 1, + m_col_strides); + m_planePaddingTop = op.padding_top_z(); + m_rowPaddingTop = op.padding_top(); + m_colPaddingLeft = op.padding_left(); + } else { + // Computing padding from the type + switch (op.padding_type()) { + case PADDING_VALID: + m_outputPlanes = Eigen::divup( + m_input_planes_eff - m_patch_planes_eff + 1, m_plane_strides); + m_outputRows = Eigen::divup(m_input_rows_eff - m_patch_rows_eff + 1, + m_row_strides); + m_outputCols = Eigen::divup(m_input_cols_eff - m_patch_cols_eff + 1, + m_col_strides); + m_planePaddingTop = 0; + m_rowPaddingTop = 0; + m_colPaddingLeft = 0; + break; + case PADDING_SAME: { + m_outputPlanes = Eigen::divup(m_input_planes_eff, m_plane_strides); + m_outputRows = Eigen::divup(m_input_rows_eff, m_row_strides); + m_outputCols = Eigen::divup(m_input_cols_eff, m_col_strides); + const Index dz = numext::maxi( + 0, (m_outputPlanes - 1) * m_plane_strides + m_patch_planes_eff - + m_input_planes_eff); + const Index dy = numext::maxi( + 0, (m_outputRows - 1) * m_row_strides + m_patch_rows_eff - + m_input_rows_eff); + const Index dx = numext::maxi( + 0, (m_outputCols - 1) * m_col_strides + m_patch_cols_eff - + m_input_cols_eff); + m_planePaddingTop = dz / 2; + m_rowPaddingTop = dy / 2; + m_colPaddingLeft = dx / 2; + break; + } + default: + eigen_assert(false && "unexpected padding"); + } + } + eigen_assert(m_outputRows > 0); + eigen_assert(m_outputCols > 0); + eigen_assert(m_outputPlanes > 0); + + // Dimensions for result of extraction. + if (static_cast(Layout) == static_cast(ColMajor)) { + // ColMajor + // 0: depth + // 1: patch_planes + // 2: patch_rows + // 3: patch_cols + // 4: number of patches + // 5 and beyond: anything else (such as batch). + m_dimensions[0] = input_dims[0]; + m_dimensions[1] = op.patch_planes(); + m_dimensions[2] = op.patch_rows(); + m_dimensions[3] = op.patch_cols(); + m_dimensions[4] = m_outputPlanes * m_outputRows * m_outputCols; + for (int i = 5; i < NumDims; ++i) { + m_dimensions[i] = input_dims[i - 1]; + } + } else { + // RowMajor + // NumDims-1: depth + // NumDims-2: patch_planes + // NumDims-3: patch_rows + // NumDims-4: patch_cols + // NumDims-5: number of patches + // NumDims-6 and beyond: anything else (such as batch). + m_dimensions[NumDims - 1] = input_dims[NumInputDims - 1]; + m_dimensions[NumDims - 2] = op.patch_planes(); + m_dimensions[NumDims - 3] = op.patch_rows(); + m_dimensions[NumDims - 4] = op.patch_cols(); + m_dimensions[NumDims - 5] = m_outputPlanes * m_outputRows * m_outputCols; + for (int i = NumDims - 6; i >= 0; --i) { + m_dimensions[i] = input_dims[i]; + } + } + + // Strides for the output tensor. + if (static_cast(Layout) == static_cast(ColMajor)) { + m_rowStride = m_dimensions[1]; + m_colStride = m_dimensions[2] * m_rowStride; + m_patchStride = m_colStride * m_dimensions[3] * m_dimensions[0]; + m_otherStride = m_patchStride * m_dimensions[4]; + } else { + m_rowStride = m_dimensions[NumDims - 2]; + m_colStride = m_dimensions[NumDims - 3] * m_rowStride; + m_patchStride = + m_colStride * m_dimensions[NumDims - 4] * m_dimensions[NumDims - 1]; + m_otherStride = m_patchStride * m_dimensions[NumDims - 5]; + } + + // Strides for navigating through the input tensor. + m_planeInputStride = m_inputDepth; + m_rowInputStride = m_inputDepth * m_inputPlanes; + m_colInputStride = m_inputDepth * m_inputRows * m_inputPlanes; + m_otherInputStride = + m_inputDepth * m_inputRows * m_inputCols * m_inputPlanes; + + m_outputPlanesRows = m_outputPlanes * m_outputRows; + + // Fast representations of different variables. + m_fastOtherStride = internal::TensorIntDivisor(m_otherStride); + m_fastPatchStride = internal::TensorIntDivisor(m_patchStride); + m_fastColStride = internal::TensorIntDivisor(m_colStride); + m_fastRowStride = internal::TensorIntDivisor(m_rowStride); + m_fastInputRowStride = + internal::TensorIntDivisor(m_row_inflate_strides); + m_fastInputColStride = + internal::TensorIntDivisor(m_col_inflate_strides); + m_fastInputPlaneStride = + internal::TensorIntDivisor(m_plane_inflate_strides); + m_fastInputColsEff = internal::TensorIntDivisor(m_input_cols_eff); + m_fastOutputPlanes = internal::TensorIntDivisor(m_outputPlanes); + m_fastOutputPlanesRows = + internal::TensorIntDivisor(m_outputPlanesRows); + + if (static_cast(Layout) == static_cast(ColMajor)) { + m_fastOutputDepth = internal::TensorIntDivisor(m_dimensions[0]); + } else { + m_fastOutputDepth = + internal::TensorIntDivisor(m_dimensions[NumDims - 1]); + } + } + + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE const Dimensions& dimensions() const { + return m_dimensions; + } + + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE bool evalSubExprsIfNeeded( + Scalar* /*data*/) { + m_impl.evalSubExprsIfNeeded(NULL); + return true; + } + + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE void cleanup() { m_impl.cleanup(); } + + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE CoeffReturnType + coeff(Index index) const { + // Patch index corresponding to the passed in index. + const Index patchIndex = index / m_fastPatchStride; + + // Spatial offset within the patch. This has to be translated into 3D + // coordinates within the patch. + const Index patchOffset = + (index - patchIndex * m_patchStride) / m_fastOutputDepth; + + // Batch, etc. + const Index otherIndex = (NumDims == 5) ? 0 : index / m_fastOtherStride; + const Index patch3DIndex = + (NumDims == 5) + ? patchIndex + : (index - otherIndex * m_otherStride) / m_fastPatchStride; + + // Calculate column index in the input original tensor. + const Index colIndex = patch3DIndex / m_fastOutputPlanesRows; + const Index colOffset = patchOffset / m_fastColStride; + const Index inputCol = colIndex * m_col_strides + + colOffset * m_in_col_strides - m_colPaddingLeft; + const Index origInputCol = + (m_col_inflate_strides == 1) + ? inputCol + : ((inputCol >= 0) ? (inputCol / m_fastInputColStride) : 0); + if (inputCol < 0 || inputCol >= m_input_cols_eff || + ((m_col_inflate_strides != 1) && + (inputCol != origInputCol * m_col_inflate_strides))) { + return Scalar(m_paddingValue); + } + + // Calculate row index in the original input tensor. + const Index rowIndex = + (patch3DIndex - colIndex * m_outputPlanesRows) / m_fastOutputPlanes; + const Index rowOffset = + (patchOffset - colOffset * m_colStride) / m_fastRowStride; + const Index inputRow = rowIndex * m_row_strides + + rowOffset * m_in_row_strides - m_rowPaddingTop; + const Index origInputRow = + (m_row_inflate_strides == 1) + ? inputRow + : ((inputRow >= 0) ? (inputRow / m_fastInputRowStride) : 0); + if (inputRow < 0 || inputRow >= m_input_rows_eff || + ((m_row_inflate_strides != 1) && + (inputRow != origInputRow * m_row_inflate_strides))) { + return Scalar(m_paddingValue); + } + + // Calculate plane index in the original input tensor. + const Index planeIndex = + (patch3DIndex - m_outputPlanes * (colIndex * m_outputRows + rowIndex)); + const Index planeOffset = + patchOffset - colOffset * m_colStride - rowOffset * m_rowStride; + const Index inputPlane = planeIndex * m_plane_strides + + planeOffset * m_in_plane_strides - + m_planePaddingTop; + const Index origInputPlane = + (m_plane_inflate_strides == 1) + ? inputPlane + : ((inputPlane >= 0) ? (inputPlane / m_fastInputPlaneStride) : 0); + if (inputPlane < 0 || inputPlane >= m_input_planes_eff || + ((m_plane_inflate_strides != 1) && + (inputPlane != origInputPlane * m_plane_inflate_strides))) { + return Scalar(m_paddingValue); + } + + const int depth_index = + static_cast(Layout) == static_cast(ColMajor) ? 0 + : NumDims - 1; + const Index depth = + index - (index / m_fastOutputDepth) * m_dimensions[depth_index]; + + const Index inputIndex = depth + origInputRow * m_rowInputStride + + origInputCol * m_colInputStride + + origInputPlane * m_planeInputStride + + otherIndex * m_otherInputStride; + + return m_impl.coeff(inputIndex); + } + + template + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE PacketReturnType + packet(Index index) const { + EIGEN_STATIC_ASSERT(PacketSize > 1, YOU_MADE_A_PROGRAMMING_MISTAKE) + eigen_assert(index + PacketSize - 1 < dimensions().TotalSize()); + + if (m_in_row_strides != 1 || m_in_col_strides != 1 || + m_row_inflate_strides != 1 || m_col_inflate_strides != 1 || + m_in_plane_strides != 1 || m_plane_inflate_strides != 1) { + return packetWithPossibleZero(index); + } + + const Index indices[2] = {index, index + PacketSize - 1}; + const Index patchIndex = indices[0] / m_fastPatchStride; + if (patchIndex != indices[1] / m_fastPatchStride) { + return packetWithPossibleZero(index); + } + const Index otherIndex = + (NumDims == 5) ? 0 : indices[0] / m_fastOtherStride; + eigen_assert(otherIndex == indices[1] / m_fastOtherStride); + + // Find the offset of the element wrt the location of the first element. + const Index patchOffsets[2] = { + (indices[0] - patchIndex * m_patchStride) / m_fastOutputDepth, + (indices[1] - patchIndex * m_patchStride) / m_fastOutputDepth}; + + const Index patch3DIndex = + (NumDims == 5) + ? patchIndex + : (indices[0] - otherIndex * m_otherStride) / m_fastPatchStride; + eigen_assert(patch3DIndex == + (indices[1] - otherIndex * m_otherStride) / m_fastPatchStride); + + const Index colIndex = patch3DIndex / m_fastOutputPlanesRows; + const Index colOffsets[2] = {patchOffsets[0] / m_fastColStride, + patchOffsets[1] / m_fastColStride}; + + // Calculate col indices in the original input tensor. + const Index inputCols[2] = { + colIndex * m_col_strides + colOffsets[0] - m_colPaddingLeft, + colIndex * m_col_strides + colOffsets[1] - m_colPaddingLeft}; + if (inputCols[1] < 0 || inputCols[0] >= m_inputCols) { + return internal::pset1(Scalar(m_paddingValue)); + } + + if (inputCols[0] != inputCols[1]) { + return packetWithPossibleZero(index); + } + + const Index rowIndex = + (patch3DIndex - colIndex * m_outputPlanesRows) / m_fastOutputPlanes; + const Index rowOffsets[2] = { + (patchOffsets[0] - colOffsets[0] * m_colStride) / m_fastRowStride, + (patchOffsets[1] - colOffsets[1] * m_colStride) / m_fastRowStride}; + eigen_assert(rowOffsets[0] <= rowOffsets[1]); + // Calculate col indices in the original input tensor. + const Index inputRows[2] = { + rowIndex * m_row_strides + rowOffsets[0] - m_rowPaddingTop, + rowIndex * m_row_strides + rowOffsets[1] - m_rowPaddingTop}; + + if (inputRows[1] < 0 || inputRows[0] >= m_inputRows) { + return internal::pset1(Scalar(m_paddingValue)); + } + + if (inputRows[0] != inputRows[1]) { + return packetWithPossibleZero(index); + } + + const Index planeIndex = + (patch3DIndex - m_outputPlanes * (colIndex * m_outputRows + rowIndex)); + const Index planeOffsets[2] = { + patchOffsets[0] - colOffsets[0] * m_colStride - + rowOffsets[0] * m_rowStride, + patchOffsets[1] - colOffsets[1] * m_colStride - + rowOffsets[1] * m_rowStride}; + eigen_assert(planeOffsets[0] <= planeOffsets[1]); + const Index inputPlanes[2] = { + planeIndex * m_plane_strides + planeOffsets[0] - m_planePaddingTop, + planeIndex * m_plane_strides + planeOffsets[1] - m_planePaddingTop}; + + if (inputPlanes[1] < 0 || inputPlanes[0] >= m_inputPlanes) { + return internal::pset1(Scalar(m_paddingValue)); + } + + if (inputPlanes[0] >= 0 && inputPlanes[1] < m_inputPlanes) { + // no padding + const int depth_index = + static_cast(Layout) == static_cast(ColMajor) ? 0 + : NumDims - 1; + const Index depth = + index - (index / m_fastOutputDepth) * m_dimensions[depth_index]; + const Index inputIndex = depth + inputRows[0] * m_rowInputStride + + inputCols[0] * m_colInputStride + + m_planeInputStride * inputPlanes[0] + + otherIndex * m_otherInputStride; + return m_impl.template packet(inputIndex); + } + + return packetWithPossibleZero(index); + } + + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE TensorOpCost + costPerCoeff(bool vectorized) const { + const double compute_cost = 10 * TensorOpCost::DivCost() + + 21 * TensorOpCost::MulCost() + + 8 * TensorOpCost::AddCost(); + return TensorOpCost(0, 0, compute_cost, vectorized, PacketSize); + } + + EIGEN_DEVICE_FUNC Scalar* data() const { return NULL; } + + const TensorEvaluator& impl() const { return m_impl; } + + Index planePaddingTop() const { return m_planePaddingTop; } + Index rowPaddingTop() const { return m_rowPaddingTop; } + Index colPaddingLeft() const { return m_colPaddingLeft; } + Index outputPlanes() const { return m_outputPlanes; } + Index outputRows() const { return m_outputRows; } + Index outputCols() const { return m_outputCols; } + Index userPlaneStride() const { return m_plane_strides; } + Index userRowStride() const { return m_row_strides; } + Index userColStride() const { return m_col_strides; } + Index userInPlaneStride() const { return m_in_plane_strides; } + Index userInRowStride() const { return m_in_row_strides; } + Index userInColStride() const { return m_in_col_strides; } + Index planeInflateStride() const { return m_plane_inflate_strides; } + Index rowInflateStride() const { return m_row_inflate_strides; } + Index colInflateStride() const { return m_col_inflate_strides; } + + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE CoeffReturnType + coeff(const array& coords) const { + // ColMajor + // 0: depth, 1: patch_planes, 2: patch_rows, 3: patch_cols, 4: number of + // patches, 5: batches + // RowMajor + // 0: batches, 1: number of patches, 2: patch_cols , 3: patch_rows, 4: + // patch_planes, 5: depth + const Index patch3DIndex = + coords[static_cast(Layout) == static_cast(ColMajor) ? 4 : 1]; + const Index colOffset = + coords[static_cast(Layout) == static_cast(ColMajor) ? 3 : 2]; + const Index rowOffset = + coords[static_cast(Layout) == static_cast(ColMajor) ? 2 : 3]; + const Index planeOffset = + coords[static_cast(Layout) == static_cast(ColMajor) ? 1 : 4]; + + array inputCoords; + + const Index colIndex = patch3DIndex / m_fastOutputPlanesRows; + const Index inputCol = colIndex * m_col_strides + + colOffset * m_in_col_strides - m_colPaddingLeft; + const Index origInputCol = + (m_col_inflate_strides == 1) + ? inputCol + : ((inputCol >= 0) ? (inputCol / m_fastInputColStride) : 0); + if (inputCol < 0 || inputCol >= m_input_cols_eff || + ((m_col_inflate_strides != 1) && + (inputCol != origInputCol * m_col_inflate_strides))) { + return Scalar(m_paddingValue); + } + + const Index rowIndex = + (patch3DIndex - colIndex * m_outputPlanesRows) / m_fastOutputPlanes; + const Index inputRow = rowIndex * m_row_strides + + rowOffset * m_in_row_strides - m_rowPaddingTop; + const Index origInputRow = + (m_row_inflate_strides == 1) + ? inputRow + : ((inputRow >= 0) ? (inputRow / m_fastInputRowStride) : 0); + if (inputRow < 0 || inputRow >= m_input_rows_eff || + ((m_row_inflate_strides != 1) && + (inputRow != origInputRow * m_row_inflate_strides))) { + return Scalar(m_paddingValue); + } + + const Index planeIndex = + patch3DIndex - colIndex * m_outputPlanesRows - rowIndex * m_outputRows; + const Index inputPlane = planeIndex * m_plane_strides + + planeOffset * m_in_plane_strides - + m_planePaddingTop; + const Index origInputPlane = + (m_plane_inflate_strides == 1) + ? inputPlane + : ((inputPlane >= 0) ? (inputPlane / m_fastInputPlaneStride) : 0); + if (inputPlane < 0 || inputPlane >= m_input_planes_eff || + ((m_plane_inflate_strides != 1) && + (inputPlane != origInputPlane * m_plane_inflate_strides))) { + return Scalar(m_paddingValue); + } + + if (static_cast(Layout) == static_cast(ColMajor)) { + inputCoords[0] = coords[0]; // depth + inputCoords[1] = origInputPlane; + inputCoords[2] = origInputRow; + inputCoords[3] = origInputCol; + inputCoords[4] = coords[5]; // batch + } else { + inputCoords[4] = coords[5]; // depth + inputCoords[3] = origInputPlane; + inputCoords[2] = origInputRow; + inputCoords[1] = origInputCol; + inputCoords[0] = coords[0]; // batch + } + if (TensorEvaluator::CoordAccess) { + return m_impl.coeff(inputCoords); + } else { + Index inputIndex; + if (static_cast(Layout) == static_cast(ColMajor)) { + inputIndex = inputCoords[4] * m_otherInputStride + + inputCoords[3] * m_colInputStride + + inputCoords[2] * m_rowInputStride + + inputCoords[1] * m_planeInputStride + inputCoords[0]; + } else { + inputIndex = inputCoords[0] * m_otherInputStride + + inputCoords[1] * m_colInputStride + + inputCoords[2] * m_rowInputStride + + inputCoords[3] * m_planeInputStride + inputCoords[4]; + } + return m_impl.coeff(inputIndex); + } + } + + protected: + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE PacketReturnType + packetWithPossibleZero(Index index) const { + EIGEN_ALIGN_MAX typename internal::remove_const::type + values[PacketSize]; + for (int i = 0; i < PacketSize; ++i) { + values[i] = coeff(index + i); + } + PacketReturnType rslt = internal::pload(values); + return rslt; + } + + Dimensions m_dimensions; + + // Parameters passed to the costructor. + Index m_plane_strides; + Index m_row_strides; + Index m_col_strides; + + Index m_outputPlanes; + Index m_outputRows; + Index m_outputCols; + + Index m_planePaddingTop; + Index m_rowPaddingTop; + Index m_colPaddingLeft; + + Index m_in_plane_strides; + Index m_in_row_strides; + Index m_in_col_strides; + + Index m_plane_inflate_strides; + Index m_row_inflate_strides; + Index m_col_inflate_strides; + + // Cached input size. + Index m_inputDepth; + Index m_inputPlanes; + Index m_inputRows; + Index m_inputCols; + + // Other cached variables. + Index m_outputPlanesRows; + + // Effective input/patch post-inflation size. + Index m_input_planes_eff; + Index m_input_rows_eff; + Index m_input_cols_eff; + Index m_patch_planes_eff; + Index m_patch_rows_eff; + Index m_patch_cols_eff; + + // Strides for the output tensor. + Index m_otherStride; + Index m_patchStride; + Index m_rowStride; + Index m_colStride; + + // Strides for the input tensor. + Index m_planeInputStride; + Index m_rowInputStride; + Index m_colInputStride; + Index m_otherInputStride; + + internal::TensorIntDivisor m_fastOtherStride; + internal::TensorIntDivisor m_fastPatchStride; + internal::TensorIntDivisor m_fastColStride; + internal::TensorIntDivisor m_fastRowStride; + internal::TensorIntDivisor m_fastInputPlaneStride; + internal::TensorIntDivisor m_fastInputRowStride; + internal::TensorIntDivisor m_fastInputColStride; + internal::TensorIntDivisor m_fastInputColsEff; + internal::TensorIntDivisor m_fastOutputPlanesRows; + internal::TensorIntDivisor m_fastOutputPlanes; + internal::TensorIntDivisor m_fastOutputDepth; + + Scalar m_paddingValue; + + TensorEvaluator m_impl; +}; + +// Override the default TensorEvaluator for TensorVolumePatchOp for CPU. +#define OVERRIDE_EVALUATOR(Device) \ + template \ + struct TensorEvaluator< \ + const TensorVolumePatchOp, Device> \ + : public CustomTensorEvaluator { \ + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE TensorEvaluator( \ + const typename CustomTensorEvaluator::XprType& op, \ + const Device& device) \ + : CustomTensorEvaluator( \ + op, device) {} \ + }; + +OVERRIDE_EVALUATOR(Eigen::ThreadPoolDevice); +OVERRIDE_EVALUATOR(Eigen::DefaultDevice); + +#undef OVERRIDE_EVALUATOR + +}; // namespace Eigen + +#endif // TENSORFLOW_CORE_KERNELS_EIGEN_VOLUME_PATCH_H_ diff --git a/linux/include/tensorflow/core/kernels/extract_image_patches_op.h b/linux/include/tensorflow/core/kernels/extract_image_patches_op.h new file mode 100644 index 000000000..64b8c0338 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/extract_image_patches_op.h @@ -0,0 +1,56 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_EXTRACT_IMAGE_PATCHES_OP_H_ +#define TENSORFLOW_CORE_KERNELS_EXTRACT_IMAGE_PATCHES_OP_H_ + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/tensor_types.h" + +namespace tensorflow { +namespace functor { + +template +struct ExtractImagePatchesForward { + void operator()(const Device& d, typename TTypes::ConstTensor input, + int patch_rows, int patch_cols, int stride_rows, + int stride_cols, int rate_rows, int rate_cols, + const Eigen::PaddingType& padding, + typename TTypes::Tensor output) { + // Need to swap row/col when calling Eigen, because our data is in + // NHWC format while Eigen assumes NWHC format. + const int64 N = std::max(input.size(), output.size()); + if (N <= std::numeric_limits::max()) { + auto output_32bit = To32Bit(output); + output_32bit.device(d) = + To32Bit(input) + .extract_image_patches(patch_cols, patch_rows, stride_cols, + stride_rows, rate_cols, rate_rows, padding) + .reshape(output_32bit.dimensions()); + } else { + output.device(d) = + input + .extract_image_patches(patch_cols, patch_rows, stride_cols, + stride_rows, rate_cols, rate_rows, padding) + .reshape(output.dimensions()); + } + } +}; + +} // namespace functor +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_EXTRACT_IMAGE_PATCHES_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/extract_volume_patches_op.h b/linux/include/tensorflow/core/kernels/extract_volume_patches_op.h new file mode 100644 index 000000000..7e0502b77 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/extract_volume_patches_op.h @@ -0,0 +1,58 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_KERNELS_EXTRACT_VOLUME_PATCHES_OP_H_ +#define TENSORFLOW_KERNELS_EXTRACT_VOLUME_PATCHES_OP_H_ + +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/kernels/eigen_volume_patch.h" +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" + +namespace tensorflow { +namespace functor { + +template +struct ExtractVolumePatchesForward { + void operator()(const Device& d, typename TTypes::ConstTensor input, + int patch_planes, int patch_rows, int patch_cols, + int stride_planes, int stride_rows, int stride_cols, + /* int rate_planes, int rate_rows, int rate_cols, */ + const Eigen::PaddingType& padding, + typename TTypes::Tensor output) { + const int64 N = std::max(input.size(), output.size()); + if (N <= std::numeric_limits::max()) { + auto output_32bit = To32Bit(output); + output_32bit.device(d) = + To32Bit(input) + .extract_volume_patches(patch_cols, patch_rows, patch_planes, + stride_cols, stride_rows, stride_planes, + padding) + .reshape(output_32bit.dimensions()); + } else { + output.device(d) = + input + .extract_volume_patches(patch_cols, patch_rows, patch_planes, + stride_cols, stride_rows, stride_planes, + padding) + .reshape(output.dimensions()); + } + } +}; + +} // namespace functor +} // namespace tensorflow + +#endif // TENSORFLOW_KERNELS_EXTRACT_VOLUME_PATCHES_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/fake_quant_ops_functor.h b/linux/include/tensorflow/core/kernels/fake_quant_ops_functor.h new file mode 100644 index 000000000..045a96ac1 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/fake_quant_ops_functor.h @@ -0,0 +1,280 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_FAKE_QUANT_OPS_FUNCTOR_H_ +#define TENSORFLOW_CORE_KERNELS_FAKE_QUANT_OPS_FUNCTOR_H_ + +#include + +#define EIGEN_STACK_ALLOCATION_LIMIT 0 +#define EIGEN_USE_THREADS +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/platform/types.h" + +EIGEN_DEVICE_FUNC EIGEN_ALWAYS_INLINE float StdRound(float input) { +// On Android, std::round() isn't present, just round(). +#if defined(__ANDROID__) + return round(input); +#else + return std::round(input); +#endif +} + +namespace tensorflow { + +// Gymnastics with nudged zero point is to ensure that real zero maps to +// an integer, which is required for e.g. zero-padding in convolutional layers. +// Outputs nudged_min, nudged_max, nudged_scale. +EIGEN_DEVICE_FUNC EIGEN_ALWAYS_INLINE void Nudge( + const float min, const float max, const int quant_min, const int quant_max, + float* nudged_min, float* nudged_max, float* scale) { + const float quant_min_float = static_cast(quant_min); + const float quant_max_float = static_cast(quant_max); + *scale = (max - min) / (quant_max_float - quant_min_float); + const float zero_point_from_min = quant_min_float - min / *scale; + const uint16 nudged_zero_point = [zero_point_from_min, quant_min, + quant_min_float, quant_max, + quant_max_float] { + if (zero_point_from_min < quant_min_float) { + return static_cast(quant_min); + } + if (zero_point_from_min > quant_max_float) { + return static_cast(quant_max); + } + return static_cast(StdRound(zero_point_from_min)); + }(); + *nudged_min = (quant_min_float - nudged_zero_point) * (*scale); + *nudged_max = (quant_max_float - nudged_zero_point) * (*scale); +} + +template +using ConstScalar = typename tensorflow::TTypes::ConstScalar; +template +using Scalar = typename tensorflow::TTypes::Scalar; +template +using ConstVec = typename tensorflow::TTypes::ConstVec; +template +using Vec = typename tensorflow::TTypes::Vec; +template +using ConstFlat = typename tensorflow::TTypes::ConstFlat; +template +using Flat = typename tensorflow::TTypes::Flat; + +// Functor called by FakeQuantWithMinMaxArgsOp to do the work. Compiles both +// for CPU and GPU. +template +struct FakeQuantWithMinMaxArgsFunctor { + void operator()(const Device& d, ConstFlat inputs, const float min, + const float max, const int quant_min, const int quant_max, + Flat outputs) { + eigen_assert(min <= 0.0f && "min should be <= 0.0"); + eigen_assert(max >= 0.0f && "max should be >= 0.0"); + eigen_assert(min < max && "min should be < max"); + + float nudged_min, nudged_max, nudged_scale; + Nudge(min, max, quant_min, quant_max, &nudged_min, &nudged_max, + &nudged_scale); + const float inv_nudged_scale = 1.0f / nudged_scale; + + auto clamped = inputs.cwiseMin(nudged_max).cwiseMax(nudged_min); + auto clamped_shifted = clamped - nudged_min; + outputs.device(d) = + (clamped_shifted * inv_nudged_scale + 0.5f).floor() * nudged_scale + + nudged_min; + } +}; + +// Functor called by FakeQuantWithMinMaxArgsGradientOp to do the work. Compiles +// both for CPU and GPU. +template +struct FakeQuantWithMinMaxArgsGradientFunctor { + void operator()(const Device& d, ConstFlat gradients, + ConstFlat inputs, const float min, const float max, + const int quant_min, const int quant_max, + Flat backprops) { + eigen_assert(min <= 0.0f && "min should be <= 0.0"); + eigen_assert(max >= 0.0f && "max should be >= 0.0"); + eigen_assert(min < max && "min should be < max"); + + float nudged_min, nudged_max, nudged_scale; + Nudge(min, max, quant_min, quant_max, &nudged_min, &nudged_max, + &nudged_scale); + + auto between_nudged_min_max = + (inputs >= nudged_min && inputs <= nudged_max) + .select(inputs.constant(1.0f), inputs.constant(0.0f)); + backprops.device(d) = gradients * between_nudged_min_max; + } +}; + +// Functor called by FakeQuantWithMinMaxVarsOp to do the work. Compiles both +// for CPU and GPU. +template +struct FakeQuantWithMinMaxVarsFunctor { + void operator()(const Device& d, ConstFlat inputs, + ConstScalar min, ConstScalar max, + const int quant_min, const int quant_max, + Flat outputs) { + const float min_val = min(); + const float max_val = max(); + // If min and max are both zero, we should just return zero. + if (min_val == 0.0f && max_val == 0.0f) { + outputs.device(d) = outputs.constant(0.0f); + return; + } + float nudged_min, nudged_max, nudged_scale; + Nudge(min_val, max_val, quant_min, quant_max, &nudged_min, &nudged_max, + &nudged_scale); + const auto nudged_scale_repl = inputs.constant(nudged_scale); + + const auto clamped = inputs.cwiseMin(nudged_max).cwiseMax(nudged_min); + const auto clamped_shifted = clamped - nudged_min; + outputs.device(d) = (clamped_shifted / nudged_scale_repl + 0.5f).floor() * + nudged_scale_repl + + nudged_min; + } +}; + +// Functor called by FakeQuantWithMinMaxVarsGradientOp to do the work. Compiles +// both for CPU and GPU. +template +struct FakeQuantWithMinMaxVarsGradientFunctor { + void operator()(const Device& d, ConstFlat gradients, + ConstFlat inputs, ConstScalar min, + ConstScalar max, const int quant_min, + const int quant_max, Flat backprops_wrt_input, + Scalar backprop_wrt_min, + Scalar backprop_wrt_max) { + const float min_val = min(); + const float max_val = max(); + // If min and max are both zero, we propagate everything to inputs. + if (min_val == 0.0f && max_val == 0.0f) { + backprops_wrt_input.device(d) = gradients; + backprop_wrt_min.device(d) = backprop_wrt_min.constant(0.0f); + backprop_wrt_max.device(d) = backprop_wrt_max.constant(0.0f); + return; + } + float nudged_min, nudged_max, nudged_scale; + Nudge(min_val, max_val, quant_min, quant_max, &nudged_min, &nudged_max, + &nudged_scale); + + const auto between_min_max = + (inputs >= nudged_min && inputs <= nudged_max) + .select(inputs.constant(1.0f), inputs.constant(0.0f)); + backprops_wrt_input.device(d) = gradients * between_min_max; + + const auto below_min = + (inputs < nudged_min) + .select(inputs.constant(1.0f), inputs.constant(0.0f)); + backprop_wrt_min.device(d) = (gradients * below_min).sum(); + + const auto above_max = + (inputs > nudged_max) + .select(inputs.constant(1.0f), inputs.constant(0.0f)); + backprop_wrt_max.device(d) = (gradients * above_max).sum(); + } +}; + +using Index = typename tensorflow::TTypes::ConstTensor::Index; + +// Functor called by FakeQuantWithMinMaxVarsPerChannelOp to do the work. +// Compiles both for CPU and GPU. +// +// Already verified: inputs, outputs are of shape [b, d], min, max are of shape +// [d]. +template +struct FakeQuantWithMinMaxVarsPerChannelFunctor { + void operator()(const Device& d, TTypes::ConstMatrix inputs, + ConstVec min, ConstVec max, const int quant_min, + const int quant_max, TTypes::Matrix outputs) { + for (Index i = 0; i < min.size(); ++i) { + const float min_val = min(i); + const float max_val = max(i); + // If min and max are both zero, we should just return zero. + if (min_val == 0.0f && max_val == 0.0f) { + auto chip = outputs.chip<1>(i); + chip.device(d) = chip.constant(0.0f); + continue; + } + float nudged_min, nudged_max, nudged_scale; + Nudge(min_val, max_val, quant_min, quant_max, &nudged_min, &nudged_max, + &nudged_scale); + const auto clamped = + inputs.chip<1>(i).cwiseMin(nudged_max).cwiseMax(nudged_min); + const auto clamped_shifted = clamped - nudged_min; + + outputs.chip<1>(i).device(d) = + (clamped_shifted / nudged_scale + 0.5f).floor() * nudged_scale + + nudged_min; + } + } +}; + +// Functor called by FakeQuantWithMinMaxVarsPerChannelGradientOp to do the work. +// Compiles both for CPU and GPU. +// +// Already verified: gradients, inputs, backprops_wrt_input are of shape [b, d], +// min, max, backprop_wrt_min, backprop_wrt_max are of shape [d]. +template +struct FakeQuantWithMinMaxVarsPerChannelGradientFunctor { + void operator()(const Device& d, TTypes::ConstMatrix gradients, + TTypes::ConstMatrix inputs, ConstVec min, + ConstVec max, const int quant_min, const int quant_max, + TTypes::Matrix backprops_wrt_input, + Vec backprop_wrt_min, Vec backprop_wrt_max) { + for (Index i = 0; i < min.size(); ++i) { + const float min_val = min(i); + const float max_val = max(i); + const auto gradients_chip = gradients.chip<1>(i); + const auto inputs_chip = inputs.chip<1>(i); + // If min and max are both zero, we propagate everything to inputs. + if (min_val == 0.0f && max_val == 0.0f) { + backprops_wrt_input.chip<1>(i).device(d) = gradients_chip; + auto min_chip = backprop_wrt_min.chip<0>(i); + auto max_chip = backprop_wrt_max.chip<0>(i); + min_chip.device(d) = min_chip.constant(0.0f); + max_chip.device(d) = max_chip.constant(0.0f); + continue; + } + float nudged_min, nudged_max, nudged_scale; + Nudge(min_val, max_val, quant_min, quant_max, &nudged_min, &nudged_max, + &nudged_scale); + + const auto between_min_max = + (inputs_chip >= nudged_min && inputs_chip <= nudged_max) + .select(inputs_chip.constant(1.0f), inputs_chip.constant(0.0f)); + backprops_wrt_input.chip<1>(i).device(d) = + gradients_chip * between_min_max; + + const auto below_min = + (inputs_chip < nudged_min) + .select(inputs_chip.constant(1.0f), inputs_chip.constant(0.0f)); + Eigen::DSizes reduce(0); + backprop_wrt_min.chip<0>(i).device(d) = + (gradients_chip * below_min).sum(reduce); + + const auto above_max = + (inputs_chip > nudged_max) + .select(inputs_chip.constant(1.0f), inputs_chip.constant(0.0f)); + backprop_wrt_max.chip<0>(i).device(d) = + (gradients_chip * above_max).sum(reduce); + } + } +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_FAKE_QUANT_OPS_FUNCTOR_H_ diff --git a/linux/include/tensorflow/core/kernels/fifo_queue.h b/linux/include/tensorflow/core/kernels/fifo_queue.h new file mode 100644 index 000000000..697ee81c3 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/fifo_queue.h @@ -0,0 +1,91 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_FIFO_QUEUE_H_ +#define TENSORFLOW_CORE_KERNELS_FIFO_QUEUE_H_ + +#include +#include + +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/kernels/queue_op.h" +#include "tensorflow/core/kernels/typed_queue.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/mutex.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +class FIFOQueue : public TypedQueue > { + public: + FIFOQueue(int32 capacity, const DataTypeVector& component_dtypes, + const std::vector& component_shapes, + const string& name); + + // Implementations of QueueInterface methods -------------------------------- + + void TryEnqueue(const Tuple& tuple, OpKernelContext* ctx, + DoneCallback callback) override; + void TryEnqueueMany(const Tuple& tuple, OpKernelContext* ctx, + DoneCallback callback) override; + void TryDequeue(OpKernelContext* ctx, CallbackWithTuple callback) override; + void TryDequeueMany(int num_elements, OpKernelContext* ctx, + bool allow_small_batch, + CallbackWithTuple callback) override; + Status MatchesNodeDef(const NodeDef& node_def) override; + + int32 size() override { + mutex_lock lock(mu_); + return queues_[0].size(); + } + + protected: + ~FIFOQueue() override {} + + // Helper for dequeuing a single element from queues_. + void DequeueLocked(OpKernelContext* ctx, Tuple* tuple) + EXCLUSIVE_LOCKS_REQUIRED(mu_); + + static Status GetElementComponentFromBatch(const Tuple& tuple, int64 index, + int component, + OpKernelContext* ctx, + PersistentTensor* out_element); + + private: + TF_DISALLOW_COPY_AND_ASSIGN(FIFOQueue); +}; + +// Defines a FIFOQueueOp, which produces a Queue (specifically, one +// backed by FIFOQueue) that persists across different graph +// executions, and sessions. Running this op produces a single-element +// tensor of handles to Queues in the corresponding device. +class FIFOQueueOp : public TypedQueueOp { + public: + explicit FIFOQueueOp(OpKernelConstruction* context); + + private: + Status CreateResource(QueueInterface** ret) override + EXCLUSIVE_LOCKS_REQUIRED(mu_); + + std::vector component_shapes_; + TF_DISALLOW_COPY_AND_ASSIGN(FIFOQueueOp); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_FIFO_QUEUE_H_ diff --git a/linux/include/tensorflow/core/kernels/fill_functor.h b/linux/include/tensorflow/core/kernels/fill_functor.h new file mode 100644 index 000000000..46bffa517 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/fill_functor.h @@ -0,0 +1,92 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_FILL_FUNCTOR_H_ +#define TENSORFLOW_CORE_KERNELS_FILL_FUNCTOR_H_ + +#define EIGEN_USE_THREADS + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/framework/types.h" + +namespace tensorflow { +namespace functor { + +template +struct FillFunctor { + // Computes on device "d": out = out.constant(in(0)), + void operator()(const Device& d, typename TTypes::Flat out, + typename TTypes::ConstScalar in); +}; + +template +struct SetZeroFunctor { + // Computes on device "d": out = out.setZero(), + void operator()(const Device& d, typename TTypes::Flat out); +}; + +// Partial specialization of SetZeroFunctor. +template +struct SetZeroFunctor { + void operator()(const Eigen::ThreadPoolDevice& d, + typename TTypes::Flat out); +}; + +#ifdef TENSORFLOW_USE_SYCL +// Partial specialization of SetZeroFunctor. +template +struct SetZeroFunctor { + void operator()(const Eigen::SyclDevice& d, typename TTypes::Flat out); +}; +#endif // TENSORFLOW_USE_SYCL + +template <> +struct SetZeroFunctor { + void operator()(const Eigen::ThreadPoolDevice& d, + typename TTypes::Flat out); +}; + +template +struct SetOneFunctor { + // Computes on device "d": out = out.setOne(), + void operator()(const Device& d, typename TTypes::Flat out); +}; + +// Partial specialization of SetOneFunctor. +template +struct SetOneFunctor { + void operator()(const Eigen::ThreadPoolDevice& d, + typename TTypes::Flat out); +}; + +#ifdef TENSORFLOW_USE_SYCL +// Partial specialization of SetOneFunctor. +template +struct SetOneFunctor { + void operator()(const Eigen::SyclDevice& d, typename TTypes::Flat out); +}; +#endif // TENSORFLOW_USE_SYCL + +template <> +struct SetOneFunctor { + void operator()(const Eigen::ThreadPoolDevice& d, + typename TTypes::Flat out); +}; + +} // namespace functor +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_FILL_FUNCTOR_H_ diff --git a/linux/include/tensorflow/core/kernels/fractional_pool_common.h b/linux/include/tensorflow/core/kernels/fractional_pool_common.h new file mode 100644 index 000000000..55a959f3c --- /dev/null +++ b/linux/include/tensorflow/core/kernels/fractional_pool_common.h @@ -0,0 +1,78 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CORE_KERNELS_FRACTIONAL_POOL_COMMON_H_ +#define TENSORFLOW_CORE_KERNELS_FRACTIONAL_POOL_COMMON_H_ + +#include +#include + +#include "tensorflow/core/util/guarded_philox_random.h" + +namespace tensorflow { + +// Shuffle a container randomly, copied from random_shuffle_op.cc +template +static inline void RandomShuffle(Iter first, Iter last, const Random& uniform) { + if (first == last) { + return; + } + const auto stop = last - 1; + for (auto i = first; i != stop; ++i) { + using std::iter_swap; + iter_swap(i, i + uniform(last - i)); + } +} + +// Generate pooling sequence for fractional pooling along one dimension. +// +// Regular max/avg pooling can be viewed as a special case, in which given the +// * input_length: e.g. 10 +// * output_length: e.g. 5 +// it will generate pooling sequence as +// diff sequence: [2, 2, 2, 2, 2] +// or as +// cumulative sequence: [0, 2, 4, 6, 8, 10] +// +// In the case of fractional pooling, input_length is not an integer multiple of +// output_length, randomness plays a role when generating pooling sequence. +// There are two type of randomness (random vs pseudo-random) defined in paper: +// http://arxiv.org/abs/1412.6071 +// You can check the paper for the difference between these two types. +// +// In summary, the generated diff sequence satisfy the following properties for +// both types of randomness: +// * length(generated_diff_pooling_sequence) = output_length +// * sum(generated_diff_pooling_sequence) = input_length +// * Let's define floor(input_length / output_length) = K, then +// K <= generated_diff_pooling_sequence[i] <= K+1 +// For example, when input_length = 10, output_length = 6, the following are +// valid pooling sequence: +// * [1, 2, 2, 1, 2, 2] +// * [1, 1, 2, 2, 2, 2] +// [1, 3, 2, 2, 2, 2] is not valid. +// +// Args: +// input_length: See above explanation +// output_length: See above explanation +// generator: Parallel version of random number generator +// pseudo_random: Whether or not use pseudo-random +// Returns: +// pooling_sequence: This is the cumulative pooling sequence. +std::vector GeneratePoolingSequence(int input_length, int output_length, + GuardedPhiloxRandom* generator, + bool pseudo_random); +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_FRACTIONAL_POOL_COMMON_H_ diff --git a/linux/include/tensorflow/core/kernels/function_ops.h b/linux/include/tensorflow/core/kernels/function_ops.h new file mode 100644 index 000000000..9e88cc6d8 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/function_ops.h @@ -0,0 +1,79 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_FUNCTION_OPS_H_ +#define TENSORFLOW_CORE_KERNELS_FUNCTION_OPS_H_ + +#include "tensorflow/core/framework/function.h" +#include "tensorflow/core/framework/op_kernel.h" + +namespace tensorflow { + +static const char* const kArgOp = FunctionLibraryDefinition::kArgOp; +static const char* const kRetOp = FunctionLibraryDefinition::kRetOp; + +class ArgOp : public OpKernel { + public: + explicit ArgOp(OpKernelConstruction* ctx); + + void Compute(OpKernelContext* ctx) override; + + bool IsExpensive() override { return false; } + + private: + int index_; + DataType dtype_; + + TF_DISALLOW_COPY_AND_ASSIGN(ArgOp); +}; + +class RetvalOp : public OpKernel { + public: + explicit RetvalOp(OpKernelConstruction* ctx); + + void Compute(OpKernelContext* ctx) override; + + bool IsExpensive() override { return false; } + + private: + int index_; + DataType dtype_; + + TF_DISALLOW_COPY_AND_ASSIGN(RetvalOp); +}; + +class RemoteCallOp : public AsyncOpKernel { + public: + explicit RemoteCallOp(OpKernelConstruction* ctx); + + ~RemoteCallOp() override {} + + void ComputeAsync(OpKernelContext* ctx, DoneCallback done) override; + + private: + NameAttrList func_; + DataTypeVector input_dtypes_; + DataTypeVector output_dtypes_; + + mutex mu_; + typedef std::pair FunctionTarget; + std::map handle_cache_ + GUARDED_BY(mu_); + + TF_DISALLOW_COPY_AND_ASSIGN(RemoteCallOp); +}; + +} // namespace tensorflow +#endif // TENSORFLOW_CORE_KERNELS_FUNCTION_OPS_H_ diff --git a/linux/include/tensorflow/core/kernels/fused_batch_norm_op.h b/linux/include/tensorflow/core/kernels/fused_batch_norm_op.h new file mode 100644 index 000000000..c45b6f79e --- /dev/null +++ b/linux/include/tensorflow/core/kernels/fused_batch_norm_op.h @@ -0,0 +1,131 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_FUSED_BATCH_NORM_OP_H_ +#define TENSORFLOW_CORE_KERNELS_FUSED_BATCH_NORM_OP_H_ + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_types.h" + +namespace tensorflow { +namespace functor { + +#if GOOGLE_CUDA + +// There is a behavior difference between cuDNN v4 and v5 with regard to the +// scaling factor for function cudnnBatchNormalizationForwardInference. +// This function corrects the scaling factor if cuDNN v4 is used, so that +// this behavior inconsistency is hidden from TensorFlow users. +// Details: in cuDNN v4, y = bnScale * (x - mean) * variance + bnBias; +// in v5, y = bnScale * (x - mean) / sqrt(variance + epsilon) + bnBias +// The template is instantiated with T as float in batch_norm_ops.cu.cc; for +// other types, the instantiation needs to be added accordingly. +template +struct VarianceToInvVariance { + void operator()(const Eigen::GpuDevice& d, const T* variance, double epsilon, + int channels, T* inv_variance); +}; + +// This function converts the inverted variance of the cuDNN forward training +// output to variance for TensorFlow to calculate the running variance. +// The template is instantiated with T as float in batch_norm_ops.cu.cc; for +// other types, the instantiation needs to be added accordingly. +template +struct InvVarianceToVariance { + void operator()(const Eigen::GpuDevice& d, double epsilon, int sample_size, + int channels, T* variance); +}; + +// This function sets a GPU tensor to NaNs. +template +struct SetNanFunctor { + void operator()(const Eigen::GpuDevice& d, typename TTypes::Flat out); +}; + +#endif // GOOGLE_CUDA + +// Functor used by FusedBatchNormGradOp to do the computations when +// is_training=False. Both CPU and GPU will use this functor. +template +struct FusedBatchNormFreezeGrad { + void operator()(const Device& d, const Tensor& y_backprop_input, + const Tensor& x_input, const Tensor& scale_input, + const Tensor& pop_mean_input, + const Tensor& pop_variance_input, U epsilon, + Tensor* x_backprop_output, Tensor* scale_backprop_output, + Tensor* offset_backprop_output, + typename TTypes::Vec scratch1, + typename TTypes::Vec scratch2) { + typename TTypes::ConstTensor y_backprop( + y_backprop_input.tensor()); + typename TTypes::ConstTensor input(x_input.tensor()); + typename TTypes::ConstVec scale(scale_input.vec()); + typename TTypes::ConstVec pop_mean(pop_mean_input.vec()); + typename TTypes::ConstVec pop_var(pop_variance_input.vec()); + typename TTypes::Tensor x_backprop(x_backprop_output->tensor()); + typename TTypes::Vec scale_backprop(scale_backprop_output->vec()); + typename TTypes::Vec offset_backprop(offset_backprop_output->vec()); + + const int depth = pop_mean.dimension(0); + const int rest_size = input.size() / depth; + + Eigen::DSizes rest_by_depth(rest_size, depth); +#if !defined(EIGEN_HAS_INDEX_LIST) + Eigen::DSizes one_by_depth(1, depth); + Eigen::array reduction_axis{0}; + Eigen::array rest_by_one({rest_size, 1}); +#else + Eigen::IndexList, Eigen::Index> one_by_depth; + one_by_depth.set(1, depth); + Eigen::IndexList > reduction_axis; + Eigen::IndexList > rest_by_one; + rest_by_one.set(0, rest_size); +#endif + + // offset_backprop = sum(y_backprop) + // scale_backprop = y_backprop * ((x - pop_mean) * rsqrt(pop_var + epsilon)) + // x_backprop = y_backprop * (scale * rsqrt(pop_var + epsilon)) + + auto y_backprop_rest_by_depth = + y_backprop.reshape(rest_by_depth).template cast(); + auto input_rest_by_depth = input.reshape(rest_by_depth).template cast(); + + offset_backprop.device(d) = y_backprop_rest_by_depth.sum(reduction_axis); + + // scratch1 = rsqrt(pop_var + epsilon) + scratch1.device(d) = (pop_var + pop_var.constant(epsilon)).rsqrt(); + + // scratch2 = sum(y_backprop * (x - mean)) + scratch2.device(d) = + (y_backprop_rest_by_depth * + (input_rest_by_depth - + pop_mean.reshape(one_by_depth).broadcast(rest_by_one))) + .sum(reduction_axis); + + x_backprop.reshape(rest_by_depth).device(d) = + (y_backprop_rest_by_depth * ((scratch1 * scale) + .eval() + .reshape(one_by_depth) + .broadcast(rest_by_one))) + .template cast(); + scale_backprop.device(d) = scratch2 * scratch1; + } +}; + +} // namespace functor +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_FUSED_BATCH_NORM_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/gather_functor.h b/linux/include/tensorflow/core/kernels/gather_functor.h new file mode 100644 index 000000000..7710cf93d --- /dev/null +++ b/linux/include/tensorflow/core/kernels/gather_functor.h @@ -0,0 +1,180 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_GATHER_FUNCTOR_H_ +#define TENSORFLOW_CORE_KERNELS_GATHER_FUNCTOR_H_ + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" + +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/framework/type_traits.h" +#include "tensorflow/core/framework/variant.h" +#include "tensorflow/core/kernels/bounds_check.h" +#include "tensorflow/core/platform/prefetch.h" +#include "tensorflow/core/platform/types.h" +#include "tensorflow/core/util/work_sharder.h" + +namespace tensorflow { +typedef Eigen::ThreadPoolDevice CPUDevice; +typedef Eigen::GpuDevice GPUDevice; + +namespace functor { + +// Helper method to copy using memcpy. +template +SliceIndex HandleCopies(OpKernelContext* ctx, + typename TTypes::ConstTensor params, + typename TTypes::ConstFlat indices, + SliceIndex slice_elems, + typename TTypes::Tensor out) { + const SliceIndex indices_size = static_cast(indices.dimension(0)); + const SliceIndex batch_size = static_cast(params.dimension(0)); + const Index limit = static_cast(params.dimension(1)); + T* out_base = &out(0, 0, 0); + const T* params_base = ¶ms(0, 0, 0); + if (static_slice_elems >= 0) { + // Give compiler static knowledge of the number of elements/bytes + slice_elems = static_slice_elems; + } + // Compute slice_bytes here so that static knowledge is available + const size_t slice_bytes = slice_elems * sizeof(T); + auto* worker_threads = ctx->device()->tensorflow_cpu_worker_threads(); + mutex mu; + // Store the value of invalidate index for printing error information, it's a + // shared variable. + SliceIndex result = -1; + auto work = [&](int64 start, int64 end) { + SliceIndex batch_idx = static_cast(start / indices_size); + SliceIndex indices_idx = static_cast(start % indices_size); + SliceIndex batch_idx_end = static_cast(end / indices_size); + SliceIndex indices_idx_end = static_cast(end % indices_size); + + while ((batch_idx < batch_idx_end) || + (batch_idx == batch_idx_end && indices_idx < indices_idx_end)) { + SliceIndex i_next = indices_idx + 1; + SliceIndex b_next = batch_idx + 1; + if ((batch_idx == batch_idx_end && i_next < indices_idx_end) || + (i_next < indices_size)) { + port::prefetch( + ¶ms(batch_idx, indices(i_next), 0)); + port::prefetch(&out(batch_idx, i_next, 0)); + b_next = batch_idx; + } else if (b_next <= batch_idx_end) { + port::prefetch(¶ms(b_next, indices(0), 0)); + port::prefetch(&out(b_next, 0, 0)); + i_next = 0; + } + const Index index = internal::SubtleMustCopy(indices(indices_idx)); + if (!FastBoundsCheck(index, limit)) { + mutex_lock l(mu); + result = indices_idx; + return; + } + // Copy using memcpy if possible, otherwise an Eigen loop + // TODO(cwhipkey): avoid linking to framework to get Allocator (to improve + // ahead-of-time compilation binary size). + if (is_simple_type::value) { + // Avoid auto-promotion to Index from SliceIndex by casting. + memcpy( + out_base + (batch_idx * indices_size + indices_idx) * slice_elems, + params_base + (batch_idx * static_cast(limit) + + static_cast(index)) * + slice_elems, + slice_bytes); + } else { + // For non-"simple" types (e.g. strings). + out.template chip<1>(indices_idx) = params.template chip<1>(index); + } + indices_idx = i_next; + batch_idx = b_next; + } + }; + + Shard(worker_threads->num_threads, worker_threads->workers, + batch_size * indices_size, slice_elems * sizeof(T), work); + return result; +} + +template +struct GatherFunctorCPU { + int64 operator()(OpKernelContext* ctx, + typename TTypes::ConstTensor params, + typename TTypes::ConstFlat indices, + typename TTypes::Tensor out) { + const int64 N = indices.size(); + const int64 slice_size = out.dimension(2); + int64 bad_i; + + bool use_large = (slice_size > std::numeric_limits::max() || + params.size() > std::numeric_limits::max() || + N > std::numeric_limits::max()); +#define CALL(elems) \ + do { \ + if (use_large) { \ + bad_i = HandleCopies(ctx, params, indices, \ + slice_size, out); \ + } else { \ + const int32 small_slice = static_cast(slice_size); \ + bad_i = HandleCopies(ctx, params, indices, \ + small_slice, out); \ + } \ + } while (0) + + if (slice_size == 10) + CALL(10); + else if (slice_size == 20) + CALL(20); + else + CALL(-1); +#undef CALL + + return bad_i; + } +}; + +template +struct GatherFunctor { + int64 operator()(OpKernelContext* ctx, + typename TTypes::ConstTensor params, + typename TTypes::ConstFlat indices, + typename TTypes::Tensor out); +}; + +template +struct GatherFunctor { + int64 operator()(OpKernelContext* ctx, + typename TTypes::ConstTensor params, + typename TTypes::ConstFlat indices, + typename TTypes::Tensor out) { + return GatherFunctorCPU()(ctx, params, indices, out); + } +}; + +template +struct GatherFunctor { + int64 operator()(OpKernelContext* ctx, + typename TTypes::ConstTensor params, + typename TTypes::ConstFlat indices, + typename TTypes::Tensor out) { + return GatherFunctorCPU()(ctx, params, indices, out); + } +}; + +} // namespace functor +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_GATHER_FUNCTOR_H_ diff --git a/linux/include/tensorflow/core/kernels/gather_nd_op.h b/linux/include/tensorflow/core/kernels/gather_nd_op.h new file mode 100644 index 000000000..003badb74 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/gather_nd_op.h @@ -0,0 +1,50 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_GATHER_ND_OP_H_ +#define TENSORFLOW_CORE_KERNELS_GATHER_ND_OP_H_ +// Functor definition for GatherOp, must be compilable by nvcc. + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/kernels/bounds_check.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +class OpKernelContext; +class Status; +class Tensor; + +namespace functor { +template +struct GatherNdSlice { + // Performs a slice gather op on (Tparams, Tindices), writing to Tout. + // Returns an index to Tindices if the value at that index is out of range. + // Returns -1 if all values of Tindices are in range. + Index operator()(const Device& d, const Index slice_size, + typename TTypes::Scalar Tscratch, + typename TTypes::ConstTensor Tparams, + typename TTypes::ConstMatrix Tindices, + typename TTypes::Matrix Tout); +}; + +template +Status DoGatherNd(OpKernelContext* c, const Tensor& params, + const Tensor& indices, Tensor* out); +} // namespace functor +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_GATHER_ND_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/gather_nd_op_cpu_impl.h b/linux/include/tensorflow/core/kernels/gather_nd_op_cpu_impl.h new file mode 100644 index 000000000..1c78de253 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/gather_nd_op_cpu_impl.h @@ -0,0 +1,160 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_GATHER_ND_OP_CPU_IMPL_H_ +#define TENSORFLOW_CORE_KERNELS_GATHER_ND_OP_CPU_IMPL_H_ + +// Specialization of GatherNdSlice to CPU + +#define EIGEN_USE_THREADS + +#include + +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/register_types.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/kernels/bounds_check.h" +#include "tensorflow/core/kernels/gather_nd_op.h" +#include "tensorflow/core/platform/logging.h" +#include "tensorflow/core/platform/mem.h" +#include "tensorflow/core/platform/types.h" +#include "tensorflow/core/util/util.h" + +namespace tensorflow { + +typedef Eigen::ThreadPoolDevice CPUDevice; + +namespace generator { + +template +class GatherNdSliceGenerator { + public: + EIGEN_DEVICE_FUNC EIGEN_ALWAYS_INLINE GatherNdSliceGenerator( + const Index slice_size, typename TTypes::ConstMatrix Tindices, + typename TTypes::ConstTensor Tparams, + typename TTypes::Matrix Tout, std::atomic* error_loc) + : slice_size_(slice_size), + Tindices_(Tindices), + Tparams_(Tparams), + Tout_(Tout), + error_loc_(error_loc) {} + + EIGEN_DEVICE_FUNC EIGEN_ALWAYS_INLINE bool GenerateIndices( + const Index loc, Eigen::array* ix) const { + (*ix)[IXDIM] = 0; + bool out_of_bounds = false; + for (int i = 0; i < IXDIM; ++i) { + const Index ix_i = internal::SubtleMustCopy(Tindices_(loc, i)); + (*ix)[i] = ix_i; + out_of_bounds |= !FastBoundsCheck(ix_i, Tparams_.dimension(i)); + } + return out_of_bounds; + } + + EIGEN_DEVICE_FUNC EIGEN_ALWAYS_INLINE int32 + operator()(const Eigen::array& loc_array) const { + const Index loc = loc_array[0]; + Eigen::array ix; + Eigen::array ix_out; + ix_out[0] = loc; + ix_out[1] = 0; + const bool out_of_bounds = GenerateIndices(loc, &ix); + if (TF_PREDICT_FALSE(out_of_bounds)) { + error_loc_->store(loc); + std::fill_n(&Tout_(ix_out), slice_size_, T()); + } else { + std::copy_n(&Tparams_(ix), slice_size_, &Tout_(ix_out)); + } + + return static_cast(0); // Return something... + } + + private: + const Index slice_size_; + const typename TTypes::ConstMatrix Tindices_; + const typename TTypes::ConstTensor Tparams_; + mutable typename TTypes::Matrix Tout_; + std::atomic* error_loc_; +}; + +} // namespace generator + +namespace functor { + +template +struct GatherNdSlice { + Index operator()(const CPUDevice& d, const Index slice_size, + typename TTypes::Scalar Tscratch, + typename TTypes::ConstTensor Tparams, + typename TTypes::ConstMatrix Tindices, + typename TTypes::Matrix Tout) { + std::atomic error_loc(-1); + + const Eigen::DenseIndex batch_size = Tindices.dimension(0); +#if !defined(EIGEN_HAS_INDEX_LIST) + Eigen::Tensor::Dimensions reshape_dims{{ 1 }}; + Eigen::array broadcast_dims{{ batch_size }}; +#else + Eigen::IndexList > reshape_dims; + Eigen::IndexList broadcast_dims; + broadcast_dims.set(0, batch_size); +#endif + generator::GatherNdSliceGenerator gather_nd_generator( + slice_size, Tindices, Tparams, Tout, &error_loc); + +#if defined(INTEL_MKL) && defined(ENABLE_MKL) +// Eigen implementation below is not highly performant. gather_nd_generator +// does not seem to be called in parallel, leading to very poor performance. +// Additionally, since it uses scalar (Tscratch) to invoke 'generate', it +// needs to go through redundant operations like 'reshape', 'broadcast' and +// 'sum'. OpenMP loop below essentially does same thing as Eigen code, but +// is considerably more efficient. +#pragma omp parallel for + for (Eigen::DenseIndex i = 0; i < batch_size; i++) { + const Eigen::array loc{i}; + gather_nd_generator(loc); + } +#else // INTEL_MKL && ENABLE_MKL + Tscratch.device(d) = Tscratch.reshape(reshape_dims) + .broadcast(broadcast_dims) + .generate(gather_nd_generator) + .sum(); +#endif // INTEL_MKL && ENABLE_MKL + + // error_loc() returns -1 if there's no out-of-bounds index, + // otherwise it returns the location of an OOB index in Tindices. + return error_loc.load(); + } +}; + +#define REGISTER_GATHER_ND_FULL(T, Index) \ + template Index GatherNdSlice:: \ + operator()(const CPUDevice& d, const Index slice_size, \ + typename TTypes::Scalar Tscratch, \ + typename TTypes::ConstTensor Tparams, \ + typename TTypes::ConstMatrix Tindices, \ + typename TTypes::Matrix Tout); + +#define REGISTER_GATHER_ND_CPU(type) \ + REGISTER_GATHER_ND_FULL(type, int32); \ + REGISTER_GATHER_ND_FULL(type, int64) + +TF_CALL_ALL_TYPES(REGISTER_GATHER_ND_CPU); + +} // namespace functor + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_GATHER_ND_OP_CPU_IMPL_H_ diff --git a/linux/include/tensorflow/core/kernels/gemm_functors.h b/linux/include/tensorflow/core/kernels/gemm_functors.h new file mode 100644 index 000000000..1c8084408 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/gemm_functors.h @@ -0,0 +1,123 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// This is a set of different implementations for the basic matrix by matrix +// multiply function, commonly known as GEMM after the BLAS library's naming. +// Having a standard interface enables us to swap out implementations on +// different platforms, to make sure we're using the optimal version. They are +// implemented as C++ template functors, so they're easy to swap into all of the +// different kernels that use them. + +#if !defined(EIGEN_USE_THREADS) +#error "EIGEN_USE_THREADS must be enabled by all .cc files including this." +#endif // EIGEN_USE_THREADS + +#ifndef TENSORFLOW_CORE_KERNELS_GEMM_FUNCTORS_H_ +#define TENSORFLOW_CORE_KERNELS_GEMM_FUNCTORS_H_ + +#include +#include +#include + +#include "tensorflow/core/common_runtime/threadpool_device.h" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_types.h" + +// Apple provides an optimized BLAS library that is better than Eigen for their +// devices, so use that if possible. +#if defined(__APPLE__) && defined(USE_GEMM_FOR_CONV) +#include +#define USE_CBLAS_GEMM +#endif // __APPLE__ + +// Older Raspberry Pi systems don't have NEON SIMD acceleration, so Eigen falls +// back to scalar code, but OpenBLAS has much faster support so prefer that. +#if defined(RASPBERRY_PI) && defined(USE_GEMM_FOR_CONV) && defined(USE_OPENBLAS) +#include +#define USE_CBLAS_GEMM +#endif + +// A readable but slow implementation of matrix multiplication, useful for +// debugging and understanding the algorithm. Use instead of FastGemmFunctor in +// the Im2ColConvFunctor template definition inside the op registration to +// enable. Assumes row-major ordering of the values in memory. +template +class ReferenceGemmFunctor { + public: + void operator()(tensorflow::OpKernelContext* ctx, size_t m, size_t n, + size_t k, const T1* a, size_t lda, const T2* b, size_t ldb, + T3* c, size_t ldc) { + const size_t a_i_stride = lda; + const size_t a_l_stride = 1; + const size_t b_j_stride = 1; + const size_t b_l_stride = ldb; + const size_t c_i_stride = ldc; + const size_t c_j_stride = 1; + size_t i, j, l; + for (j = 0; j < n; j++) { + for (i = 0; i < m; i++) { + T3 total(0); + for (l = 0; l < k; l++) { + const size_t a_index = ((i * a_i_stride) + (l * a_l_stride)); + const T1 a_value = a[a_index]; + const size_t b_index = ((j * b_j_stride) + (l * b_l_stride)); + const T2 b_value = b[b_index]; + total += (a_value * b_value); + } + const size_t c_index = ((i * c_i_stride) + (j * c_j_stride)); + c[c_index] = total; + } + } + } +}; + +// Uses the optimized EigenTensor library to implement the matrix multiplication +// required by the Im2ColConvFunctor class. We supply the two input and one +// output types so that the accumulator can potentially be higher-precision than +// the inputs, even though we don't currently take advantage of this. +template +class FastGemmFunctor { + public: + void operator()(tensorflow::OpKernelContext* ctx, size_t m, size_t n, + size_t k, const T1* a, size_t lda, const T2* b, size_t ldb, + T3* c, size_t ldc) { + typename tensorflow::TTypes::Matrix a_matrix(a, m, k); + typename tensorflow::TTypes::Matrix b_matrix(b, k, n); + typename tensorflow::TTypes::Matrix c_matrix(c, m, n); + + Eigen::array, 1> dim_pair; + dim_pair[0].first = 1; + dim_pair[0].second = 0; + c_matrix.device(ctx->eigen_device()) = + a_matrix.contract(b_matrix, dim_pair); + } +}; + +// If we have a fast CBLAS library, use its implementation through a wrapper. +#if defined(USE_CBLAS_GEMM) +template <> +class FastGemmFunctor { + public: + void operator()(tensorflow::OpKernelContext* ctx, size_t m, size_t n, + size_t k, const float* a, size_t lda, const float* b, + size_t ldb, float* c, size_t ldc) { + cblas_sgemm(CblasRowMajor, CblasNoTrans, CblasNoTrans, m, n, k, 1.0f, a, + lda, b, ldb, 0.0f, c, ldc); + } +}; +#endif // USE_CBLAS_GEMM + +#endif // TENSORFLOW_CORE_KERNELS_GEMM_FUNCTORS_H_ diff --git a/linux/include/tensorflow/core/kernels/gpu_utils.h b/linux/include/tensorflow/core/kernels/gpu_utils.h new file mode 100644 index 000000000..86146f75f --- /dev/null +++ b/linux/include/tensorflow/core/kernels/gpu_utils.h @@ -0,0 +1,163 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_GPU_UTILS_H_ +#define TENSORFLOW_CORE_KERNELS_GPU_UTILS_H_ + +#if GOOGLE_CUDA + +#include + +#include "tensorflow/core/lib/strings/str_util.h" +#include "tensorflow/core/lib/strings/strcat.h" +#include "tensorflow/core/lib/strings/stringprintf.h" +#include "tensorflow/core/platform/logging.h" +#include "tensorflow/core/platform/stream_executor.h" + +namespace tensorflow { + +template +inline se::DeviceMemory AsDeviceMemory(const T* cuda_memory, uint64 size) { + se::DeviceMemoryBase wrapped(const_cast(cuda_memory), size * sizeof(T)); + se::DeviceMemory typed(wrapped); + return typed; +} + +// A helper class that looks up the best autotuned config from parameters. +// Due to the noisy nature of autotune, especially with multiple devices, it +// only accepts a config if its margin exceeds a threshold. +// For the same shape configs, if a new best config matches the previous best, +// they get promoted; otherwise, the winner gets demoted. This process stops +// when the winner's score exceeds the threshold. +// In a bad case when two configs are very close to each other and flips +// back and forth randomly, the expected number of experiments before autotune +// settles is O(threshold ^ 2). So we recommend that number of warmup runs +// for any benchmarks. +template +class AutoTuneMap { + public: + bool Find(const Parameters& params, Config* config) const { + mutex_lock lock(mu_); + auto iter = params_config_map_.find(params); + if (iter == params_config_map_.end() || + (iter->second.score < min_score_threshold_ && + iter->second.count <= max_autotune_count_)) { + return false; + } + *config = iter->second.config; + return true; + } + void Insert(const Parameters& params, const Config& config) { + mutex_lock lock(mu_); + auto iter = params_config_map_.find(params); + int new_score = 0; + if (iter == params_config_map_.end()) { + // Create a new entry if params is new. + VLOG(1) << GetActionSummary("creates", params, config); + params_config_map_.insert( + std::make_pair(params, ValueType{config, 1, 1})); + new_score = 1; + } else if (iter->second.score < min_score_threshold_ && + iter->second.count <= max_autotune_count_) { + DCHECK_GT(iter->second.score, 0); + if (iter->second.config != config) { + // If it is different from the current winner, demotes the winner. + VLOG(1) << GetActionSummary("demotes", params, config); + new_score = --iter->second.score; + ++iter->second.count; + if (new_score <= 0) { + VLOG(1) << GetActionSummary("erases", params, config); + params_config_map_.erase(iter); + } + } else { + // If it is the same as the current winner, promotes the winner. + VLOG(1) << GetActionSummary("promotes", params, config); + new_score = ++iter->second.score; + ++iter->second.count; + } + } + if (new_score >= min_score_threshold_) { + VLOG(1) << GetActionSummary("accepts", params, config); + } + } + + private: + AutoTuneMap(const string& name) : name_(name) { + min_score_threshold_ = 1; + int min_warmup_iterations = 10; + const char* threshold_str = getenv("TF_AUTOTUNE_THRESHOLD"); + if (threshold_str != nullptr) { + strings::safe_strto32(threshold_str, &min_score_threshold_); + } + const char* min_warmup_iteration_str = + getenv("TF_AUTOTUNE_MIN_WARMUP_ITERATIONS"); + if (min_warmup_iteration_str != nullptr) { + strings::safe_strto32(min_warmup_iteration_str, &min_warmup_iterations); + } + min_score_threshold_ = std::max(min_score_threshold_, 1); + max_autotune_count_ = std::max( + 5 * min_score_threshold_ * min_score_threshold_, min_warmup_iterations); + } + + template + friend class AutoTuneSingleton; + + struct Hasher { + std::size_t operator()(const Parameters& parameter) const { + return parameter.hash(); + } + }; + + string GetActionSummary(StringPiece action, const Parameters& params, + const Config& config) { + return strings::Printf("autotune_map %s %s: %s -> (%s)", name_.c_str(), + string(action).c_str(), params.ToString().c_str(), + config.ToString().c_str()); + } + + mutable mutex mu_; + struct ValueType { + Config config; + int32 score; + int32 count; + }; + std::unordered_map params_config_map_ + GUARDED_BY(mu_); + string name_; + int32 min_score_threshold_; + int32 max_autotune_count_; + + TF_DISALLOW_COPY_AND_ASSIGN(AutoTuneMap); +}; + +// A Singleton helper that manages the global autotune results by groups. +// The caller specified arbitrary Group type that can distinguish between +// different autotune results, even if their Parameters and Configs are the +// same. +template +class AutoTuneSingleton { + public: + typedef AutoTuneMap AutoTuneType; + static AutoTuneType* GetInstance() { + static AutoTuneType* instance = new AutoTuneType(Group::name()); + return instance; + } +}; + +} // namespace tensorflow + +#endif // GOOGLE_CUDA + +#endif // TENSORFLOW_CORE_KERNELS_GPU_UTILS_H_ diff --git a/linux/include/tensorflow/core/kernels/hinge-loss.h b/linux/include/tensorflow/core/kernels/hinge-loss.h new file mode 100644 index 000000000..b12910d27 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/hinge-loss.h @@ -0,0 +1,126 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_HINGE_LOSS_H_ +#define TENSORFLOW_CORE_KERNELS_HINGE_LOSS_H_ + +#include +#include + +#include "tensorflow/core/kernels/loss.h" +#include "tensorflow/core/lib/core/errors.h" +#include "tensorflow/core/lib/core/status.h" + +namespace tensorflow { + +class HingeLossUpdater : public DualLossUpdater { + public: + // Computes the updated dual variable (corresponding) to a single example. The + // updated dual value maximizes the objective function of the dual + // optimization problem associated with hinge loss (conditioned on keeping the + // rest of the dual variables intact). The method below finds an optimal delta + // (difference between updated and previous dual value) using the update rule + // within SDCA procedure (see http://arxiv.org/pdf/1209.1873v2.pdf, page 5) + // and the particular form of conjugate function for hinge loss. + // + // The CoCoA+ modification is detailed in readme.md. + // + // TODO(sibyl-vie3Poto): Write up a doc with concrete derivation and point to it from + // here. + double ComputeUpdatedDual(const int num_loss_partitions, const double label, + const double example_weight, + const double current_dual, const double wx, + const double weighted_example_norm) const final { + // Intutitvely there are 3 cases: + // a. new optimal value of the dual variable falls within the admissible + // range [0, 1]. In this case we set new dual to this value. + // b. new optimal value is < 0. Then, because of convexity, the optimal + // valid value for new dual = 0 + // c. new optimal value > 1.0. Then new optimal value should be set to 1.0. + const double candidate_optimal_dual = + current_dual + (label - wx) / (num_loss_partitions * example_weight * + weighted_example_norm); + if (label * candidate_optimal_dual < 0) { + return 0.0; + } + if (label * candidate_optimal_dual > 1.0) { + return label; + } + return candidate_optimal_dual; + } + + // Conjugate of hinge loss. This is computed as: + // \phi*(z) = z if z \in [-1, 0] and +infinity everywhere else. See for + // instance http://www.eecs.berkeley.edu/~wainwrig/stat241b/lec10.pdf + // Here we want the weighted version of the conjugate loss. It turns out, that + // if w is the weight of an example, the conjugate of the weighted hinge loss + // is given by: + // \phi*(z) = z if z \in [-w, 0] and +infinity everywhere else. Here the + // conjugate function depends not only on the weight of the example but also + // on its label. In particular: + // \phi_y*(z) = y*z if y*z \in [-w, 0] and +infinity everywhere else where + // y \in {-1,1}. The following method implements \phi_y*(-\alpha/w). + double ComputeDualLoss(const double current_dual, const double example_label, + const double example_weight) const final { + // For binary classification, there are 2 conjugate functions, one per + // label value (-1 and 1). + const double y_alpha = current_dual * example_label; // y \alpha + if (y_alpha < 0 || y_alpha > 1.0) { + return std::numeric_limits::max(); + } + return -y_alpha * example_weight; + } + + // Hinge loss for binary classification for a single example. Hinge loss + // equals max(0, 1 - y * wx) (see https://en.wikipedia.org/wiki/Hinge_loss). + // For weighted instances loss should be multiplied by the instance weight. + double ComputePrimalLoss(const double wx, const double example_label, + const double example_weight) const final { + const double y_wx = example_label * wx; + return std::max(0.0, 1 - y_wx) * example_weight; + } + + double PrimalLossDerivative(const double wx, const double label, + const double example_weight) const final { + if (label * wx < 1) { + return -label * example_weight; + } + return 0; + } + + // The smoothness constant is 0 since the derivative of the loss is not + // Lipschitz + double SmoothnessConstant() const final { return 0; } + + // Converts binary example labels from 0.0 or 1.0 to -1.0 or 1.0 respectively + // as expected by hinge loss. + Status ConvertLabel(float* const example_label) const final { + if (*example_label == 0.0) { + *example_label = -1; + return Status::OK(); + } + if (*example_label == 1.0) { + return Status::OK(); + } + return errors::InvalidArgument( + "Only labels of 0.0 or 1.0 are supported right now. " + "Found example with label: ", + *example_label); + } +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_HINGE_LOSS_H_ diff --git a/linux/include/tensorflow/core/kernels/histogram_op.h b/linux/include/tensorflow/core/kernels/histogram_op.h new file mode 100644 index 000000000..b14fc2bee --- /dev/null +++ b/linux/include/tensorflow/core/kernels/histogram_op.h @@ -0,0 +1,38 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_HISTOGRAM_OP_H_ +#define TENSORFLOW_CORE_KERNELS_HISTOGRAM_OP_H_ + +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/core/errors.h" + +namespace tensorflow { +namespace functor { + +template +struct HistogramFixedWidthFunctor { + static Status Compute(OpKernelContext* context, + const typename TTypes::ConstTensor& values, + const typename TTypes::ConstTensor& value_range, + int32 nbins, typename TTypes::Tensor& out); +}; + +} // end namespace functor +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_HISTOGRAM_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/host_constant_op.h b/linux/include/tensorflow/core/kernels/host_constant_op.h new file mode 100644 index 000000000..1b887ea1a --- /dev/null +++ b/linux/include/tensorflow/core/kernels/host_constant_op.h @@ -0,0 +1,42 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_HOST_CONSTANT_OP_H_ +#define TENSORFLOW_CORE_KERNELS_HOST_CONSTANT_OP_H_ + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/platform/macros.h" + +namespace tensorflow { + +// HostConstantOp differs from ConstantOp in that its output is always +// in host memory. +class _HostConstantOp : public OpKernel { + public: + explicit _HostConstantOp(OpKernelConstruction* ctx); + void Compute(OpKernelContext* ctx) override; + bool IsExpensive() override { return false; } + ~_HostConstantOp() override {} + + private: + Tensor tensor_; + TF_DISALLOW_COPY_AND_ASSIGN(_HostConstantOp); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_HOST_CONSTANT_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/i_remote_fused_graph_executor.h b/linux/include/tensorflow/core/kernels/i_remote_fused_graph_executor.h new file mode 100644 index 000000000..b2329f4b6 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/i_remote_fused_graph_executor.h @@ -0,0 +1,77 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +vcyou may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_I_REMOTE_FUSED_GRAPH_EXECUTOR_H_ +#define TENSORFLOW_CORE_KERNELS_I_REMOTE_FUSED_GRAPH_EXECUTOR_H_ + +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/platform/macros.h" + +namespace tensorflow { + +class GraphDef; +class RemoteFusedGraphExecuteInfo; + +class IRemoteFusedGraphExecutor { + public: + using TensorAllocatorFunc = std::function; + + IRemoteFusedGraphExecutor() = default; + virtual ~IRemoteFusedGraphExecutor() = default; + + // Return version of executor. + // This function is mainly for a debug purpose to verify version of + // executor info. + virtual int GetVersion() = 0; + + // Initialize executor. This function is called before + // starting graph transfer. + virtual bool Init(const RemoteFusedGraphExecuteInfo& info) = 0; + + // Finalize executor. This function is called when all graph executions + // are finished. + virtual bool Finalize() = 0; + + // Setup graph + virtual bool SetupGraph() = 0; + + // Execute graph + virtual bool ExecuteGraph() = 0; + + // Teardown Graph + virtual bool TeardownGraph() = 0; + + // Fill input node's output with Tensor + virtual bool FillInputNode(const string& node_name, const Tensor& tensor) = 0; + + // Read output node's outputs as ByteArrays + virtual bool ReadOutputNode(const string& node_name, + TensorAllocatorFunc tensor_allocator) = 0; + + virtual Status FuseRemoteGraph(const GraphDef& original_graph_def, + const std::vector& inputs, + const std::vector& outputs, + GraphDef* fused_graph_def) = 0; + + virtual bool IsEnabled() const = 0; + + private: + TF_DISALLOW_COPY_AND_ASSIGN(IRemoteFusedGraphExecutor); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_I_REMOTE_FUSED_GRAPH_EXECUTOR_H_ diff --git a/linux/include/tensorflow/core/kernels/i_remote_fused_graph_ops_definitions.h b/linux/include/tensorflow/core/kernels/i_remote_fused_graph_ops_definitions.h new file mode 100644 index 000000000..9e51c9f51 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/i_remote_fused_graph_ops_definitions.h @@ -0,0 +1,46 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +vcyou may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_I_REMOTE_FUSED_GRAPH_OPS_DEFINITIONS_H_ +#define TENSORFLOW_CORE_KERNELS_I_REMOTE_FUSED_GRAPH_OPS_DEFINITIONS_H_ + +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/platform/macros.h" + +namespace tensorflow { + +// IRemoteFusedGraphOpsDefinitions is an interface class which provides +// APIs to provide information about op types supported by SOC. +// TODO(satok): Provide ways to transfer graph definitions into SOC +class IRemoteFusedGraphOpsDefinitions { + public: + // op id which is not supported by SOC + static constexpr int INVALID_OP_ID = -1; + + IRemoteFusedGraphOpsDefinitions() = default; + virtual ~IRemoteFusedGraphOpsDefinitions() = default; + // Return total ops count supported by SOC + virtual int GetTotalOpsCount() const = 0; + // Return op id for given string op name + virtual int GetOpIdFor(const string& op_type, + const DataTypeVector& dt) const = 0; + + private: + TF_DISALLOW_COPY_AND_ASSIGN(IRemoteFusedGraphOpsDefinitions); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_I_REMOTE_FUSED_GRAPH_OPS_DEFINITIONS_H_ diff --git a/linux/include/tensorflow/core/kernels/identity_n_op.h b/linux/include/tensorflow/core/kernels/identity_n_op.h new file mode 100644 index 000000000..7339cbbe2 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/identity_n_op.h @@ -0,0 +1,44 @@ +/* Copyright 2015-2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_IDENTITY_N_OP_H_ +#define TENSORFLOW_CORE_KERNELS_IDENTITY_N_OP_H_ + +#include "tensorflow/core/framework/op_kernel.h" + +namespace tensorflow { + +class IdentityNOp : public OpKernel { + public: + explicit IdentityNOp(OpKernelConstruction* context) : OpKernel(context) {} + + void Compute(OpKernelContext* context) override { + OpInputList input; + OpOutputList output; + OP_REQUIRES_OK(context, context->input_list("input", &input)); + OP_REQUIRES_OK(context, context->output_list("output", &output)); + OP_REQUIRES(context, input.size() == output.size(), + errors::InvalidArgument("Input and output counts must match")); + for (int i = 0; i < input.size(); ++i) { + output.set(i, input[i]); + } + } + + bool IsExpensive() override { return false; } +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_IDENTITY_N_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/identity_op.h b/linux/include/tensorflow/core/kernels/identity_op.h new file mode 100644 index 000000000..6b74868ad --- /dev/null +++ b/linux/include/tensorflow/core/kernels/identity_op.h @@ -0,0 +1,40 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_IDENTITY_OP_H_ +#define TENSORFLOW_CORE_KERNELS_IDENTITY_OP_H_ + +#include "tensorflow/core/framework/op_kernel.h" + +namespace tensorflow { + +class IdentityOp : public OpKernel { + public: + explicit IdentityOp(OpKernelConstruction* context) : OpKernel(context) {} + + void Compute(OpKernelContext* context) override { + if (IsRefType(context->input_dtype(0))) { + context->forward_ref_input_to_ref_output(0, 0); + } else { + context->set_output(0, context->input(0)); + } + } + + bool IsExpensive() override { return false; } +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_IDENTITY_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/image_resizer_state.h b/linux/include/tensorflow/core/kernels/image_resizer_state.h new file mode 100644 index 000000000..1d4fa1a7d --- /dev/null +++ b/linux/include/tensorflow/core/kernels/image_resizer_state.h @@ -0,0 +1,194 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// This is a helper struct to package up the input and output +// parameters of an image resizer (the height, widths, etc.). To +// reduce code duplication and ensure consistency across the different +// resizers, it performs the input validation. + +#ifndef TENSORFLOW_CORE_KERNELS_IMAGE_RESIZER_STATE_H_ +#define TENSORFLOW_CORE_KERNELS_IMAGE_RESIZER_STATE_H_ + +#define EIGEN_USE_THREADS + +#include +#include +#include + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/register_types.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/kernels/bounds_check.h" + +namespace tensorflow { + +// CalculateResizeScale determines the float scaling factor. +inline float CalculateResizeScale(int64 in_size, int64 out_size, + bool align_corners) { + return (align_corners && out_size > 1) + ? (in_size - 1) / static_cast(out_size - 1) + : in_size / static_cast(out_size); +} + +struct ImageResizerState { + explicit ImageResizerState(bool align_corners) + : align_corners_(align_corners) {} + + // ValidateAndCalculateOutputSize checks the bounds on the input tensors + // and requested size, sets up some of the resizing state such as the + // height_scale and width_scale, and calculates the output size. + // If any of these operations fails, it sets an error status in + // the context, which the caller must check. + void ValidateAndCalculateOutputSize(OpKernelContext* context, + const Tensor& input) { + OP_REQUIRES(context, input.dims() == 4, + errors::InvalidArgument("input must be 4-dimensional", + input.shape().DebugString())); + const Tensor& shape_t = context->input(1); + OP_REQUIRES(context, shape_t.dims() == 1, + errors::InvalidArgument("shape_t must be 1-dimensional", + shape_t.shape().DebugString())); + OP_REQUIRES(context, shape_t.NumElements() == 2, + errors::InvalidArgument("shape_t must have two elements", + shape_t.shape().DebugString())); + auto Svec = shape_t.vec(); + batch_size = input.dim_size(0); + out_height = internal::SubtleMustCopy(Svec(0)); + out_width = internal::SubtleMustCopy(Svec(1)); + OP_REQUIRES( + context, + FastBoundsCheck(input.dim_size(1), std::numeric_limits::max()) && + FastBoundsCheck(input.dim_size(2), + std::numeric_limits::max()), + errors::InvalidArgument("input sizes must be between 0 and max int32")); + + in_height = static_cast(input.dim_size(1)); + in_width = static_cast(input.dim_size(2)); + channels = input.dim_size(3); + OP_REQUIRES(context, out_height > 0 && out_width > 0, + errors::InvalidArgument("output dimensions must be positive")); + OP_REQUIRES( + context, channels > 0, + errors::InvalidArgument("image must have at least one channel")); + OP_REQUIRES( + context, input.dim_size(1) > 0 && input.dim_size(2) > 0, + errors::InvalidArgument("input image must be of non-zero size")); + height_scale = CalculateResizeScale(in_height, out_height, align_corners_); + width_scale = CalculateResizeScale(in_width, out_width, align_corners_); + + // Guard against overflows + OP_REQUIRES(context, + ceilf((out_height - 1) * height_scale) <= + static_cast(std::numeric_limits::max()), + errors::InvalidArgument( + "input image height scale would cause an overflow")); + OP_REQUIRES( + context, + ceilf((out_width - 1) * width_scale) <= static_cast(INT_MAX), + errors::InvalidArgument( + "input image width scale would cause an overflow")); + } + + // Calculates all the required variables, and allocates the output. + void ValidateAndCreateOutput(OpKernelContext* context, const Tensor& input) { + ValidateAndCalculateOutputSize(context, input); + if (!context->status().ok()) return; + OP_REQUIRES_OK(context, context->allocate_output( + 0, + TensorShape({input.dim_size(0), out_height, + out_width, input.dim_size(3)}), + &output)); + } + + int64 batch_size; + int64 out_height; + int64 out_width; + int64 in_height; + int64 in_width; + int64 channels; + float height_scale; + float width_scale; + Tensor* output = nullptr; + + private: + bool align_corners_; +}; + +struct ImageResizerGradientState { + explicit ImageResizerGradientState(bool align_corners) + : align_corners_(align_corners) {} + + void ValidateAndCreateOutput(OpKernelContext* context, const Tensor& input, + const Tensor& original_image) { + OP_REQUIRES(context, input.dims() == 4, + errors::InvalidArgument("input_grad must be 4-dimensional", + input.shape().DebugString())); + // Resizers always produce float images, so input gradient must + // always be a float. + OP_REQUIRES(context, input.dtype() == DT_FLOAT, + errors::InvalidArgument("input_grad must be of type float", + DataTypeString(input.dtype()))); + + OP_REQUIRES(context, original_image.dims() == 4, + errors::InvalidArgument("original_image must be 4-dimensional", + original_image.shape().DebugString())); + + // Allocate output and initialize to zeros. + batch_size = input.dim_size(0); + channels = input.dim_size(3); + resized_height = input.dim_size(1); + resized_width = input.dim_size(2); + original_height = original_image.dim_size(1); + original_width = original_image.dim_size(2); + + OP_REQUIRES( + context, + FastBoundsCheck(original_height, std::numeric_limits::max()) && + FastBoundsCheck(original_width, std::numeric_limits::max()), + errors::InvalidArgument( + "original sizes must be between 0 and max int32")); + + height_scale = + CalculateResizeScale(original_height, resized_height, align_corners_); + width_scale = + CalculateResizeScale(original_width, resized_width, align_corners_); + output = nullptr; + OP_REQUIRES_OK(context, context->allocate_output( + 0, + TensorShape({batch_size, original_height, + original_width, channels}), + &output)); + } + + int64 batch_size; + int64 channels; + int64 resized_height; + int64 resized_width; + int64 original_height; + int64 original_width; + float height_scale; + float width_scale; + Tensor* output; + + private: + bool align_corners_; +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_IMAGE_RESIZER_STATE_H_ diff --git a/linux/include/tensorflow/core/kernels/immutable_constant_op.h b/linux/include/tensorflow/core/kernels/immutable_constant_op.h new file mode 100644 index 000000000..97af8c7dc --- /dev/null +++ b/linux/include/tensorflow/core/kernels/immutable_constant_op.h @@ -0,0 +1,49 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CORE_KERNELS_IMMUTABLE_CONSTANT_OP_H_ +#define TENSORFLOW_CORE_KERNELS_IMMUTABLE_CONSTANT_OP_H_ + +#include + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/platform/env.h" +#include "tensorflow/core/platform/macros.h" + +namespace tensorflow { + +class ImmutableConstantOp : public OpKernel { + public: + explicit ImmutableConstantOp(OpKernelConstruction* context); + void Compute(OpKernelContext* ctx) override; + bool IsExpensive() override { return false; } + ~ImmutableConstantOp() override; + + // Names of attributes that are used by this op + static constexpr char const* kDTypeAttr = "dtype"; + static constexpr char const* kShapeAttr = "shape"; + static constexpr char const* kMemoryRegionNameAttr = "memory_region_name"; + + private: + string region_name_; + DataType dtype_; + TensorShape shape_; + TF_DISALLOW_COPY_AND_ASSIGN(ImmutableConstantOp); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_IMMUTABLE_CONSTANT_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/initializable_lookup_table.h b/linux/include/tensorflow/core/kernels/initializable_lookup_table.h new file mode 100644 index 000000000..424fe5df3 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/initializable_lookup_table.h @@ -0,0 +1,208 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_INITIALIZABLE_LOOKUP_TABLE_H_ +#define TENSORFLOW_CORE_KERNELS_INITIALIZABLE_LOOKUP_TABLE_H_ + +#include "tensorflow/core/framework/lookup_interface.h" +#include "tensorflow/core/platform/macros.h" + +namespace tensorflow { +namespace lookup { + +// Base class for lookup tables that require initialization. +class InitializableLookupTable : public LookupInterface { + public: + class InitTableIterator; + + // Performs batch lookups, for every element in the key tensor, Find returns + // the corresponding value into the values tensor. + // If an element is not present in the table, the given default value is used. + // + // For tables that require initialization, `Find` is available once the table + // is marked as initialized. + // + // Returns the following statuses: + // - OK: when the find finishes successfully. + // - FailedPrecondition: if the table is not initialized. + // - InvalidArgument: if any of the preconditions on the lookup key or value + // fails. + // - In addition, other implementations may provide another non-OK status + // specific to their failure modes. + Status Find(OpKernelContext* ctx, const Tensor& keys, Tensor* values, + const Tensor& default_value) final; + + // Returns errors::Unimplemented. + Status Insert(OpKernelContext* ctx, const Tensor& keys, + const Tensor& values) final { + return errors::Unimplemented( + "Insert not supported by InitializableLookupTable implementations"); + } + + Status ExportValues(OpKernelContext* context) override { + return errors::Unimplemented( + "ExportValues not supported by InitializableLookupTable " + "implementations"); + } + + Status ImportValues(OpKernelContext* ctx, const Tensor& keys, + const Tensor& values) final; + + TensorShape key_shape() const final { return TensorShape(); } + + TensorShape value_shape() const final { return TensorShape(); } + + // Returns whether the table was initialized and is ready to serve lookups. + bool is_initialized() const { return is_initialized_; } + + // Initializes the table from the given init table iterator. + // + // Atomically, this operation prepares the table, populates it with the given + // iterator, and mark the table as initialized. + // + // Returns the following statuses: + // - OK: when the initialization was successful. + // - InvalidArgument: if any of the preconditions on the lookup key or value + // fails. + // - FailedPrecondition: if the table is already initialized and + // fail_if_initialized is set to true. + // - In addition, other implementations may provide another non-OK status + // specific to their failure modes. + Status Initialize(InitTableIterator& iter); + + // Basic iterator to initialize lookup tables. + // It yields a sequence of pairs of `keys()` and `values()` Tensors, so that + // the consumer may insert key-value pairs in batches. + // + // Then the iterator is exhausted, valid returns false and status returns + // Status::OutOfRange. + // + // This class is Thread-unsafe. + class InitTableIterator { + public: + InitTableIterator() {} + + virtual ~InitTableIterator() {} + + // Prepares the next batch of key and value tensors. + virtual void Next() = 0; + + // Returns true if keys and values point to valid tensors. + virtual bool Valid() const = 0; + + // Returns a tensor that contains the current batch of 'key' values. + virtual const Tensor& keys() const = 0; + + // Returns a tensor that contains the current batch of 'value' values. + virtual const Tensor& values() const = 0; + + // Returns an error if one has occurred, otherwise returns Status::OK. + virtual Status status() const = 0; + + // Returns the total number of elements that the iterator will produce. + // It might return -1 in case of error. + virtual int64 total_size() const = 0; + + private: + TF_DISALLOW_COPY_AND_ASSIGN(InitTableIterator); + }; + + InitializableLookupTable* GetInitializableLookupTable() override { + return this; + } + + protected: + // Prepares and allocates the underlying data structure to store the given + // number of expected elements. + virtual Status DoPrepare(size_t expected_num_elements) = 0; + + // Same as DoPrepare() but derived implementations might choose to skip + // calling get_expected_num_elements if size is not needed for DoPrepare. + virtual Status DoLazyPrepare( + std::function get_expected_num_elements) { + int64 expected_num_elements = get_expected_num_elements(); + if (expected_num_elements < 0) { + return errors::FailedPrecondition("Got negative expected_num_elements."); + } + return DoPrepare(expected_num_elements); + } + + // Populates the table in batches given keys and values as tensors into the + // underlying data structure. + virtual Status DoInsert(const Tensor& keys, const Tensor& values) = 0; + + // Performs the batch find operation on the underlying data structure. + virtual Status DoFind(const Tensor& keys, Tensor* values, + const Tensor& default_value) = 0; + + mutex mu_; + bool is_initialized_ = false; +}; + +// Iterator to initialize tables given 'keys' and 'values' tensors. +// +// The two tensors are returned in the first iteration. It doesn't loop +// over each element of the tensor since insertions in the lookup table can +// process batches. +class KeyValueTensorIterator + : public InitializableLookupTable::InitTableIterator { + public: + // keys and values are not owned by the iterator. + explicit KeyValueTensorIterator(const Tensor* keys, const Tensor* values) + : keys_(keys), values_(values), valid_(true), status_(Status::OK()) { + TensorShape key_shape = keys_->shape(); + if (!key_shape.IsSameSize(values_->shape())) { + valid_ = false; + status_ = errors::InvalidArgument( + "keys and values should have the same dimension.", + key_shape.DebugString(), " vs ", values_->shape().DebugString()); + } + if (key_shape.num_elements() == 0) { + valid_ = false; + status_ = + errors::InvalidArgument("keys and values cannot be empty tensors."); + } + } + + bool Valid() const override { return valid_; } + + void Next() override { + valid_ = false; + status_ = errors::OutOfRange("No more data."); + } + + const Tensor& keys() const override { return *keys_; } + + const Tensor& values() const override { return *values_; } + + Status status() const override { return status_; } + + int64 total_size() const override { + return keys_ == nullptr ? -1 : keys_->NumElements(); + } + + private: + TF_DISALLOW_COPY_AND_ASSIGN(KeyValueTensorIterator); + + const Tensor* keys_; // Doesn't own it. + const Tensor* values_; // Doesn't own it. + bool valid_; // true if the iterator points to an existing range. + Status status_; +}; + +} // namespace lookup +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_INITIALIZABLE_LOOKUP_TABLE_H_ diff --git a/linux/include/tensorflow/core/kernels/inplace_ops_functor.h b/linux/include/tensorflow/core/kernels/inplace_ops_functor.h new file mode 100644 index 000000000..2023869f4 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/inplace_ops_functor.h @@ -0,0 +1,49 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_INPLACE_OPS_FUNCTOR_H_ +#define TENSORFLOW_CORE_KERNELS_INPLACE_OPS_FUNCTOR_H_ + +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/lib/core/status.h" + +namespace tensorflow { +namespace functor { + +template +Status DoParallelConcat(const Device& device, const Tensor& value, int32 loc, + Tensor* output); + +// Inplace update/add/sub values in 'y'. It computes +// y[i, :] = v if op is I_UPDATE +// y[i, :] += v if op is I_ADD +// y[i, :] -= v if op is I_SUB +// Returns an error if the operation fails. +enum InplaceOpType { + I_UPDATE, // x = y + I_ADD, // x += y + I_SUB, // x -= y +}; +template +Status DoInplace(const Device& device, InplaceOpType op, const Tensor& i, + const Tensor& v, Tensor* y); +// Copies x into y. +template +Status DoCopy(const Device& device, const Tensor& x, Tensor* y); + +} // end namespace functor +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_INPLACE_OPS_FUNCTOR_H_ diff --git a/linux/include/tensorflow/core/kernels/l2loss_op.h b/linux/include/tensorflow/core/kernels/l2loss_op.h new file mode 100644 index 000000000..465ef96a5 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/l2loss_op.h @@ -0,0 +1,33 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_L2LOSS_OP_H_ +#define TENSORFLOW_CORE_KERNELS_L2LOSS_OP_H_ +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/tensor_types.h" + +namespace tensorflow { + +template +struct L2LossOp : public OpKernel { + explicit L2LossOp(OpKernelConstruction* context) : OpKernel(context) {} + + void Compute(OpKernelContext* context) {} +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_L2LOSS_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/linalg_ops_common.h b/linux/include/tensorflow/core/kernels/linalg_ops_common.h new file mode 100644 index 000000000..692f91643 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/linalg_ops_common.h @@ -0,0 +1,197 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_LINALG_OPS_COMMON_H_ +#define TENSORFLOW_CORE_KERNELS_LINALG_OPS_COMMON_H_ + +// Classes to support linear algebra functionality, similar to the numpy.linalg +// module. Supports batch computation on several matrices at once, sharding the +// computations across different threads if necessary. +#include + +#include "third_party/eigen3/Eigen/Core" +#include "tensorflow/core/framework/kernel_def_builder.h" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/core/errors.h" +#include "tensorflow/core/lib/gtl/inlined_vector.h" +#include "tensorflow/core/platform/types.h" +#include "tensorflow/core/util/work_sharder.h" + +namespace tensorflow { + +// Base class for linear algebra operators. +template +class LinearAlgebraOp : public OpKernel { + public: + explicit LinearAlgebraOp(OpKernelConstruction* context) : OpKernel(context) {} + + void Compute(OpKernelContext* context) override; + + protected: + using TensorShapes = gtl::InlinedVector; + // Returns the number of leading inputs that are to be treated as matrix + // inputs. By default this is all the inputs. Derived classes can override + // this to tell the base class to ignore one or more trailing inputs. + virtual int NumMatrixInputs(const OpKernelContext* context) const { + return context->num_inputs(); + } + + // Returns true if the number of inputs and their shapes are as expected. + // Many ops take a single square input matrix, so we provide that as a default + // implementation for convenience. + virtual void ValidateInputMatrixShapes( + OpKernelContext* context, const TensorShapes& input_matrix_shapes) const { + ValidateSingleSquareMatrix(context, input_matrix_shapes); + } + + // Convenience validators for common cases: + // + // Validate op taking a single matrix A. + static void ValidateSingleMatrix(OpKernelContext* context, + const TensorShapes& input_matrix_shapes); + // Validate op taking a single square matrix A. + static void ValidateSingleSquareMatrix( + OpKernelContext* context, const TensorShapes& input_matrix_shapes); + // Validate op taking two matrices A and B that have the same number of rows. + static void ValidateSolver(OpKernelContext* context, + const TensorShapes& input_matrix_shapes); + // Validate op taking two matrices A and B that have the same number of rows + // and A is square. + static void ValidateSquareSolver(OpKernelContext* context, + const TensorShapes& input_matrix_shapes); + + // Returns the output shapes of each individual matrix operation. Output + // matrices shapes must be rank 0, 1, or 2. Scalar outputs are rank 0. + // + // The derived class may return a number of shapes (N) less than + // context->num_outputs() (M) to indicate that a only leading subset of + // the outputs will be populated. In this case, a dummy scalar tensor with + // value zero will be return for the last M-N outputs. + // + // For many ops, the output dimensions are the same as the input dimensions, + // so we provide that as a default implementation for convenience. + virtual TensorShapes GetOutputMatrixShapes( + const TensorShapes& input_matrix_shapes) const { + return input_matrix_shapes; + } + + // Returns the cost per matrix operation. This is used to determine the + // number of threads to use for parallelizing calls to ComputeMatrix in + // batch mode. Cost per unit is assumed to be roughly 1ns, based on comments + // in core/util/work_sharder.cc. Many linear algebra ops take roughly max(m,n) + // * min(m,n)^2, where the first input matrix is m-by-n. We provide that as a + // default implementation for convenience. + virtual int64 GetCostPerUnit(const TensorShapes& input_matrix_shapes) const { + double m = static_cast(input_matrix_shapes[0].dim_size(0)); + double n = static_cast(input_matrix_shapes[0].dim_size(1)); + double cost = std::max(m, n) * std::min(m, n) * std::min(m, n); + return cost >= static_cast(kint64max) ? kint64max + : static_cast(cost); + } + + // Returns true if it is safe to forward (alias) input to output buffer + // and expect the kernel to perform the computation inplace. + virtual bool EnableInputForwarding() const { return true; } + + using Matrix = + Eigen::Matrix; + using ConstMatrixMap = Eigen::Map; + using MatrixMap = Eigen::Map; + using ConstMatrixMaps = gtl::InlinedVector; + using MatrixMaps = gtl::InlinedVector; + using RealScalar = typename Eigen::NumTraits::Real; + + // Performs a single matrix computation given input matrices, and + // stores the result in outputs. For batch operations, this will be called + // repeatedly for a single call to Compute() when multiple matrices exist in + // input Tensors with rank > 2. In this case the calls to ComputeMatrix are + // parallelized. The number of threads used is determined by a cost model from + // the value returned by GetCostPerUnit(). + virtual void ComputeMatrix(OpKernelContext* context, + const ConstMatrixMaps& inputs, + MatrixMaps* outputs) = 0; + + private: + using TensorInputs = gtl::InlinedVector; + using TensorOutputs = gtl::InlinedVector; + // This function maps 2-d slices (matrices) of the input and output tensors + // using Eigen::Map and calls ComputeMatrix implemented in terms of the + // Eigen::MatrixBase API by the derived class. + // + // The 'matrix_index' parameter specifies the index of the matrix to be used + // from each input tensor, and the index of the matrix to be written to each + // output tensor. The input matrices are in row major order, and located at + // the memory addresses + // inputs[i].flat().data() + + // matrix_index * input_matrix_shapes[i].num_elements() + // for i in 0...inputs.size()-1. + // The output matrices are in row major order, and located at the memory + // address + // outputs[i]->flat().data() + + // matrix_index * output_matrix_shapes[i].num_elements(). + // for i in 0...outputs.size()-1. + // + void ComputeTensorSlice(OpKernelContext* context, int64 matrix_index, + const TensorInputs& inputs, + const TensorShapes& input_matrix_shapes, + const TensorOutputs& outputs, + const TensorShapes& output_matrix_shapes); + + void AnalyzeInputs(OpKernelContext* context, TensorInputs* inputs, + TensorShapes* input_matrix_shapes, + TensorShape* batch_shape); + + void PrepareOutputs(OpKernelContext* context, + const TensorShapes& input_matrix_shapes, + const TensorShape& batch_shape, TensorOutputs* outputs, + TensorShapes* output_matrix_shapes); +}; + +// Declare LinearAlgebraOp, which is explicitly instantiated in +// linalg_ops_common.cc for float, double, complex64, and complex128. +extern template class LinearAlgebraOp; +extern template class LinearAlgebraOp; +extern template class LinearAlgebraOp; +extern template class LinearAlgebraOp; + +} // namespace tensorflow + +#define INHERIT_LINALG_TYPEDEFS(Scalar) \ + typedef LinearAlgebraOp Base; \ + using RealScalar = typename Eigen::NumTraits::Real; \ + using Matrix = typename Base::Matrix; \ + using MatrixMap = typename Base::MatrixMap; \ + using MatrixMaps = typename Base::MatrixMaps; \ + using ConstMatrixMap = typename Base::ConstMatrixMap; \ + using ConstMatrixMaps = typename Base::ConstMatrixMaps; \ + using TensorShapes = typename Base::TensorShapes; + +#define REGISTER_LINALG_OP_CPU(OpName, OpClass, Scalar) \ + REGISTER_KERNEL_BUILDER( \ + Name(OpName).Device(DEVICE_CPU).TypeConstraint("T"), OpClass) + +#define REGISTER_LINALG_OP_GPU(OpName, OpClass, Scalar) \ + REGISTER_KERNEL_BUILDER( \ + Name(OpName).Device(DEVICE_GPU).TypeConstraint("T"), OpClass) + +// Deprecated, use one of the device-specific macros above. +#define REGISTER_LINALG_OP(OpName, OpClass, Scalar) \ + REGISTER_LINALG_OP_CPU(OpName, OpClass, Scalar) + +#endif // TENSORFLOW_CORE_KERNELS_LINALG_OPS_COMMON_H_ diff --git a/linux/include/tensorflow/core/kernels/list_kernels.h b/linux/include/tensorflow/core/kernels/list_kernels.h new file mode 100644 index 000000000..72581c929 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/list_kernels.h @@ -0,0 +1,534 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CORE_KERNELS_LIST_KERNELS_H_ +#define TENSORFLOW_CORE_KERNELS_LIST_KERNELS_H_ + +#define EIGEN_USE_THREADS +#if GOOGLE_CUDA +#define EIGEN_USE_GPU +#endif // GOOGLE_CUDA + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/register_types.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/framework/variant.h" +#include "tensorflow/core/framework/variant_op_registry.h" +#include "tensorflow/core/kernels/concat_lib.h" +#include "tensorflow/core/lib/core/coding.h" +#include "tensorflow/core/lib/core/errors.h" +#include "tensorflow/core/util/util.h" + +namespace tensorflow { + +typedef Eigen::ThreadPoolDevice CPUDevice; + +// Variant compatible type for a list of tensors. This is mutable but instances +// should never be mutated after stored in a variant tensor. +struct TensorList { + public: + TensorList() {} + TensorList(const TensorList& other); + + static const char kTypeName[]; + string TypeName() const { return kTypeName; } + + void Encode(VariantTensorData* data) const; + + bool Decode(const VariantTensorData& data); + + // TODO(apassos) fill this out + string DebugString() const { return "TensorList"; } + + std::vector tensors; + PartialTensorShape element_shape; + DataType element_dtype; +}; + +Status TensorShapeFromTensor(const Tensor& t, PartialTensorShape* out); + +template +class TensorListStack : public OpKernel { + public: + typedef std::vector::ConstMatrix>> + ConstMatrixVector; + explicit TensorListStack(OpKernelConstruction* c) : OpKernel(c) { + OP_REQUIRES_OK(c, c->GetAttr("element_dtype", &element_dtype_)); + OP_REQUIRES_OK(c, c->GetAttr("num_elements", &num_elements_)); + } + + ~TensorListStack() {} + + void Compute(OpKernelContext* c) override { + const TensorList* l = c->input(0).scalar()().get(); + OP_REQUIRES(c, l != nullptr, + errors::InvalidArgument( + "Input handle is not a list. Saw: '", + c->input(0).scalar()().DebugString(), "'")); + OP_REQUIRES(c, element_dtype_ == l->element_dtype, + errors::InvalidArgument("Invalid data types; op elements ", + DataTypeString(element_dtype_), + " but list elements ", + DataTypeString(l->element_dtype))); + OP_REQUIRES(c, l->element_shape.IsFullyDefined(), + errors::InvalidArgument("Tried to stack elements from a list " + "with non-fully-defined shape: ", + l->element_shape.DebugString())); + if (num_elements_ != -1) { + OP_REQUIRES(c, l->tensors.size() == num_elements_, + errors::InvalidArgument("Operation expected a list with ", + num_elements_, + " elements but got a list with ", + l->tensors.size(), " elements.")); + } + TensorShape resulting_shape; + resulting_shape.AddDim(l->tensors.size()); + for (TensorShapeDim s : l->element_shape) { + resulting_shape.AddDim(s.size); + } + Tensor* output; + OP_REQUIRES_OK(c, c->allocate_output(0, resulting_shape, &output)); + if (output->NumElements() == 0) { + return; + } + + ConstMatrixVector inputs_flat; + inputs_flat.reserve(l->tensors.size()); + for (const auto& t : l->tensors) { + OP_REQUIRES( + c, l->element_shape.IsCompatibleWith(t.shape()), + errors::InvalidArgument( + "Tensor with invalid shape in list. List element shape shape: ", + l->element_shape.DebugString(), + " and tensor shape: ", t.shape().DebugString())); + inputs_flat.emplace_back(new typename TTypes::ConstMatrix( + t.shaped({1, t.NumElements()}))); + } + auto output_flat = output->shaped({1, output->NumElements()}); + +#if GOOGLE_CUDA + if (std::is_same::value) { + ConcatGPU(c, inputs_flat, output, &output_flat); + return; + } +#endif // GOOGLE_CUDA + ConcatCPU(c->device(), inputs_flat, &output_flat); + } + + private: + int num_elements_; + DataType element_dtype_; +}; + +template +class TensorListGather : public OpKernel { + public: + typedef std::vector::ConstMatrix>> + ConstMatrixVector; + explicit TensorListGather(OpKernelConstruction* c) : OpKernel(c) { + OP_REQUIRES_OK(c, c->GetAttr("element_dtype", &element_dtype_)); + } + + void Compute(OpKernelContext* c) override { + const TensorList* l = c->input(0).scalar()().get(); + OP_REQUIRES(c, l != nullptr, + errors::InvalidArgument( + "Input handle is not a list. Saw: '", + c->input(0).scalar()().DebugString(), "'")); + OP_REQUIRES(c, element_dtype_ == l->element_dtype, + errors::InvalidArgument("Invalid data types; op elements ", + DataTypeString(element_dtype_), + " but list elements ", + DataTypeString(l->element_dtype))); + OP_REQUIRES(c, l->element_shape.IsFullyDefined(), + errors::InvalidArgument("Tried to stack elements from a list " + "with non-fully-defined shape: ", + l->element_shape.DebugString())); + Tensor indices = c->input(1); + TensorShape resulting_shape; + resulting_shape.AddDim(indices.NumElements()); + for (TensorShapeDim s : l->element_shape) { + resulting_shape.AddDim(s.size); + } + Tensor* output; + OP_REQUIRES_OK(c, c->allocate_output(0, resulting_shape, &output)); + if (output->NumElements() == 0) { + return; + } + + ConstMatrixVector inputs_flat; + inputs_flat.reserve(l->tensors.size()); + for (int index = 0; index < indices.NumElements(); ++index) { + const int i = indices.flat()(index); + OP_REQUIRES( + c, i < l->tensors.size(), + errors::InvalidArgument("Index ", i, " out o range; list only has ", + l->tensors.size(), " elements.")); + const Tensor& t = l->tensors[i]; + OP_REQUIRES(c, l->element_shape.IsCompatibleWith(t.shape()), + errors::InvalidArgument( + "Tensor with invalid shape in list. List element shape: ", + l->element_shape.DebugString(), + " and tensor shape: ", t.shape().DebugString())); + inputs_flat.emplace_back(new typename TTypes::ConstMatrix( + t.shaped({1, t.NumElements()}))); + } + auto output_flat = output->shaped({1, output->NumElements()}); + +#if GOOGLE_CUDA + if (std::is_same::value) { + ConcatGPU(c, inputs_flat, output, &output_flat); + return; + } +#endif // GOOGLE_CUDA + ConcatCPU(c->device(), inputs_flat, &output_flat); + } + + private: + DataType element_dtype_; +}; + +template +class TensorListFromTensor : public OpKernel { + public: + TensorListFromTensor(OpKernelConstruction* c) : OpKernel(c) {} + + void Compute(OpKernelContext* c) override { + Tensor* output_tensor; + AllocatorAttributes attr; + attr.set_on_host(true); + OP_REQUIRES_OK(c, c->allocate_output(0, {}, &output_tensor, attr)); + PartialTensorShape element_shape; + OP_REQUIRES_OK(c, TensorShapeFromTensor(c->input(1), &element_shape)); + TensorList output_list; + const Tensor& t = c->input(0); + output_list.element_dtype = t.dtype(); + OP_REQUIRES(c, TensorShapeUtils::IsVectorOrHigher(t.shape()), + errors::InvalidArgument( + "Tensor must be at least a vector, but saw shape: ", + t.shape().DebugString())); + TensorShape output_shape(t.shape()); + output_shape.RemoveDim(0); + OP_REQUIRES(c, element_shape.IsCompatibleWith(output_shape), + errors::InvalidArgument( + "Specified a list with shape ", element_shape.DebugString(), + " from a tensor with shape ", output_shape.DebugString())); + output_list.element_shape = element_shape; + output_list.tensors.reserve(t.shape().dim_size(0)); + for (int i = 0; i < t.shape().dim_size(0); ++i) { + Tensor tmp = t.Slice(i, i + 1); + TensorShape tmp_shape = tmp.shape(); + tmp_shape.RemoveDim(0); + OP_REQUIRES(c, tmp.CopyFrom(tmp, tmp_shape), + errors::Unknown("Unexpected shape error.")); + // TODO(apassos) maybe not always align; but weird compiler bugs seem to + // prevent this. + Tensor aligned; + OP_REQUIRES_OK(c, c->allocate_temp(tmp.dtype(), tmp.shape(), &aligned)); + aligned.flat().device(c->eigen_device()) = + tmp.unaligned_flat(); + output_list.tensors.push_back(aligned); + } + output_tensor->scalar()() = std::move(output_list); + } +}; + +template +class TensorListScatter : public OpKernel { + public: + TensorListScatter(OpKernelConstruction* c) : OpKernel(c) {} + + void Compute(OpKernelContext* c) override { + Tensor* output_tensor; + AllocatorAttributes attr; + attr.set_on_host(true); + OP_REQUIRES_OK(c, c->allocate_output(0, {}, &output_tensor, attr)); + Tensor indices = c->input(1); + PartialTensorShape element_shape; + OP_REQUIRES_OK(c, TensorShapeFromTensor(c->input(2), &element_shape)); + TensorList output_list; + const Tensor& t = c->input(0); + output_list.element_dtype = t.dtype(); + OP_REQUIRES(c, TensorShapeUtils::IsVectorOrHigher(t.shape()), + errors::InvalidArgument( + "Tensor must be at least a vector, but saw shape: ", + t.shape().DebugString())); + TensorShape output_shape(t.shape()); + output_shape.RemoveDim(0); + OP_REQUIRES(c, element_shape.IsCompatibleWith(output_shape), + errors::InvalidArgument( + "Specified a list with shape ", element_shape.DebugString(), + " from a tensor with shape ", output_shape.DebugString())); + output_list.element_shape = element_shape; + output_list.tensors.reserve(indices.NumElements()); + for (int index = 0; index < indices.NumElements(); ++index) { + const int i = indices.flat()(index); + OP_REQUIRES(c, i < t.shape().dim_size(0), + errors::InvalidArgument("Trying to scatter index ", i, + " from tensor with ", + t.shape().dim_size(0), " rows.")); + Tensor tmp = t.Slice(i, i + 1); + TensorShape tmp_shape = tmp.shape(); + tmp_shape.RemoveDim(0); + OP_REQUIRES(c, tmp.CopyFrom(tmp, tmp_shape), + errors::Unknown("Unexpected shape error.")); + // TODO(apassos) maybe not always align; but weird compiler bugs seem to + // prevent this. + Tensor aligned; + OP_REQUIRES_OK(c, c->allocate_temp(tmp.dtype(), tmp.shape(), &aligned)); + // TODO(apassos) do all slices in a single kernel invocation instead of + // many small ondes. + aligned.flat().device(c->eigen_device()) = + tmp.unaligned_flat(); + output_list.tensors.push_back(aligned); + } + output_tensor->scalar()() = std::move(output_list); + } +}; + +template +Status TensorListBinaryAdd(OpKernelContext* c, const TensorList& a, + const TensorList& b, TensorList* out) { + if (a.element_dtype != b.element_dtype) { + return errors::InvalidArgument( + "Trying to add two lists of tensors of different dtypes. One is ", + DataTypeString(a.element_dtype), " and the other is ", + DataTypeString(b.element_dtype)); + } + out->element_dtype = a.element_dtype; + if (!a.element_shape.IsCompatibleWith(b.element_shape)) { + return errors::InvalidArgument( + "Trying to add two lists of tensors with incompatible element shapes. " + "One is ", + a.element_shape.DebugString(), " and the other is ", + b.element_shape.DebugString()); + } + + TF_RETURN_IF_ERROR( + a.element_shape.MergeWith(b.element_shape, &out->element_shape)); + if (a.tensors.size() != b.tensors.size()) { + return errors::InvalidArgument( + "Trying to add two lists of tensors with different lengths. One is ", + a.tensors.size(), " and the other is ", b.tensors.size()); + } + out->tensors.reserve(a.tensors.size()); + for (int i = 0; i < a.tensors.size(); ++i) { + const Tensor& a_tensor = a.tensors[i]; + const Tensor& b_tensor = b.tensors[i]; + if (a_tensor.dtype() == DT_INVALID) { + out->tensors.push_back(b_tensor); + continue; + } + if (b_tensor.dtype() == DT_INVALID) { + out->tensors.push_back(a_tensor); + continue; + } + if (a_tensor.shape() != b_tensor.shape()) { + // TODO(apassos) support broadcasting additions here? + return errors::InvalidArgument( + "Trying to add two tensors with incompatible element shapes. " + "One is ", + a_tensor.shape().DebugString(), " and the other is ", + b_tensor.shape().DebugString(), " in position ", i); + } + Tensor out_tensor; + TF_RETURN_IF_ERROR( + c->allocate_temp(a_tensor.dtype(), a_tensor.shape(), &out_tensor)); + out->tensors.push_back(out_tensor); + switch (out_tensor.dtype()) { +#define DTYPE_CASE(dtype) \ + case DataTypeToEnum::value: \ + out_tensor.flat().device(c->eigen_device()) = \ + a_tensor.flat() + b_tensor.flat(); \ + break; + + TF_CALL_NUMBER_TYPES(DTYPE_CASE) + +#undef DTYPE_CASE + default: + return errors::InvalidArgument("Trying to add unsupported dtype ", + out_tensor.dtype()); + } + } + return Status::OK(); +} + +template +Status TensorListZerosLike(OpKernelContext* c, const TensorList& x, + TensorList* y) { + y->element_dtype = x.element_dtype; + y->element_shape = x.element_shape; + y->tensors.reserve(x.tensors.size()); + for (const Tensor& t : x.tensors) { + Tensor out_tensor; + AllocatorAttributes attr; + if (t.dtype() == DT_VARIANT) { + attr.set_on_host(true); + } + TF_RETURN_IF_ERROR( + c->allocate_temp(t.dtype(), t.shape(), &out_tensor, attr)); + switch (out_tensor.dtype()) { +#define DTYPE_CASE(dtype) \ + case DataTypeToEnum::value: \ + out_tensor.flat().device(c->eigen_device()) = \ + out_tensor.flat().constant(dtype(0)); \ + break; + + TF_CALL_POD_TYPES(DTYPE_CASE) + +#undef DTYPE_CASE + + case DataTypeToEnum::value: { + const TensorList* inner_x = t.scalar()().get(); + if (inner_x == nullptr) { + return errors::InvalidArgument("Input handle is not a list. Saw: '", + t.scalar()().DebugString(), + "'"); + } + TensorList inner_y; + TF_RETURN_IF_ERROR(TensorListZerosLike(c, *inner_x, &inner_y)); + out_tensor.scalar()() = std::move(inner_y); + break; + } + + default: + return errors::InvalidArgument( + "Trying to compute zeros_like for unsupported dtype ", + DataTypeString(out_tensor.dtype())); + } + y->tensors.emplace_back(out_tensor); + } + return Status::OK(); +} + +template +class TensorListPushBackBatch : public OpKernel { + public: + explicit TensorListPushBackBatch(OpKernelConstruction* c) : OpKernel(c) { + OP_REQUIRES_OK(c, c->GetAttr("element_dtype", &element_dtype_)); + } + + ~TensorListPushBackBatch() override {} + + void Compute(OpKernelContext* c) override { + const Tensor& input = c->input(1); + OP_REQUIRES(c, element_dtype_ == input.dtype(), + errors::InvalidArgument("Invalid data types; list elements ", + DataTypeString(element_dtype_), + " but tried to append ", + DataTypeString(input.dtype()))); + OP_REQUIRES(c, TensorShapeUtils::IsVectorOrHigher(input.shape()), + errors::InvalidArgument( + "Expected tensor to be at least a vector, but saw shape: ", + input.shape().DebugString())); + + const TensorShape& tls_shape = c->input(0).shape(); + + // For purposes of input forwarding, we want the least restrictive + // AllocatorAttributes possible. If we need to allocate later, + // we'll request the DT_VARIANT be allocated on host. + AllocatorAttributes attr; + + std::unique_ptr tls_alias = c->forward_input( + 0 /*input_index*/, 0 /*output_index*/, DT_VARIANT, tls_shape, + DEVICE_MEMORY /* input is always on DEVICE_MEMORY */, attr); + + const Tensor& tls = tls_alias ? *tls_alias : c->input(0); + + OP_REQUIRES(c, tls.dtype() == DT_VARIANT, + errors::InvalidArgument( + "Expected input_handles dtype to be Variant, but saw: ", + DataTypeString(tls.dtype()))); + OP_REQUIRES(c, TensorShapeUtils::IsVector(tls_shape), + errors::InvalidArgument( + "Expected input_handles to be a vector, but saw shape: ", + tls_shape.DebugString())); + const int64 batch_size = tls.NumElements(); + OP_REQUIRES(c, input.dim_size(0) == batch_size, + errors::InvalidArgument( + "Expected tensor.shape[0] == input_handles.size, but saw ", + input.dim_size(0), " vs. ", batch_size)); + auto tls_t = tls.vec(); + + TensorShape input_element_shape = input.shape(); + input_element_shape.RemoveDim(0); + std::vector tl_batch; + for (int64 b = 0; b < batch_size; ++b) { + const TensorList* l = tls_t(b).get(); + OP_REQUIRES(c, l != nullptr, + errors::InvalidArgument("Input handle at index ", b, + " is not a list. Saw: '", + tls_t(b).DebugString(), "'")); + OP_REQUIRES( + c, l->element_shape.IsCompatibleWith(input_element_shape), + errors::InvalidArgument( + "Tried to append a tensor with incompatible shape to a " + "list at index ", + b, ". Op element shape: ", input_element_shape.DebugString(), + " list shape: ", l->element_shape.DebugString())); + OP_REQUIRES(c, element_dtype_ == l->element_dtype, + errors::InvalidArgument( + "Invalid data type at index ", b, "; op elements ", + DataTypeString(element_dtype_), " but list elements ", + DataTypeString(l->element_dtype))); + tl_batch.push_back(l); + } + + Tensor* result; + + if (tls_alias) { + result = tls_alias.get(); + c->set_output(0, *result); + } else { + // DT_VARIANT tensors always allocated on host. + AllocatorAttributes attr; + attr.set_on_host(true); + OP_REQUIRES_OK( + c, c->allocate_output(0, TensorShape{batch_size}, &result, attr)); + } + + if (batch_size == 0) { + return; + } + + auto input_t = input.flat_outer_dims(); + auto result_t = result->vec(); + + for (int64 b = 0; b < batch_size; ++b) { + if (!tls_alias) { + result_t(b) = *tl_batch[b]; + } + TensorList* output = result_t(b).get(); + DCHECK(output != nullptr); + Tensor* frame; + PersistentTensor tmp; + OP_REQUIRES_OK(c, c->allocate_persistent( + element_dtype_, input_element_shape, &tmp, &frame)); + if (input_element_shape.num_elements() > 0) { + auto frame_t = frame->flat(); + frame_t.device(c->eigen_device()) = input_t.template chip<0>(b); + } + output->tensors.push_back(std::move(*frame)); + } + } + + private: + DataType element_dtype_; +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_LIST_KERNELS_H_ diff --git a/linux/include/tensorflow/core/kernels/logistic-loss.h b/linux/include/tensorflow/core/kernels/logistic-loss.h new file mode 100644 index 000000000..9198a98e4 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/logistic-loss.h @@ -0,0 +1,134 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_LOGISTIC_LOSS_H_ +#define TENSORFLOW_CORE_KERNELS_LOGISTIC_LOSS_H_ + +#include + +#include "tensorflow/core/kernels/loss.h" +#include "tensorflow/core/lib/core/errors.h" + +namespace tensorflow { + +class LogisticLossUpdater : public DualLossUpdater { + public: + // Adding vs. Averaging in Distributed Primal-Dual Optimization. + // Chenxin Ma, Virginia Smith, Martin Jaggi, Michael I. Jordan, Peter + // Richtarik, Martin Takac http://arxiv.org/abs/1502.03508 + double ComputeUpdatedDual(const int num_loss_partitions, const double label, + const double example_weight, + const double current_dual, const double wx, + const double weighted_example_norm) const final { + // Newton algorithm converges quadratically so 10 steps will be largely + // enough to achieve a very good precision + static const int newton_total_steps = 10; + double x = 0; + for (int i = 0; i < newton_total_steps; ++i) { + x = NewtonStep(x, num_loss_partitions, label, wx, example_weight, + weighted_example_norm, current_dual); + } + return 0.5 * (1 + tanh(x)) / label; + } + + // Dual of logisitic loss function. + // https://en.wikipedia.org/wiki/Convex_conjugate + double ComputeDualLoss(const double current_dual, const double example_label, + const double example_weight) const final { + // Dual of the logistic loss function is + // ay * log(ay) + (1-ay) * log (1-ay), where a is the dual variable. + const double ay = current_dual * example_label; + const double log_ay = (ay > 0) ? log(ay) : 0; + const double one_minus_ay = 1 - ay; + const double log_one_minus_ay = (one_minus_ay > 0) ? log(one_minus_ay) : 0; + return ((ay * log_ay) + (one_minus_ay * log_one_minus_ay)) * example_weight; + } + + // Logistic loss for binary classification. + // https://en.wikipedia.org/wiki/Loss_functions_for_classification + double ComputePrimalLoss(const double wx, const double example_label, + const double example_weight) const final { + // Logistic loss: + // log(1 + e^(-ywx)) + // log(e^0 + e^(-ywx)) + // a + log(e^(0-a) + e^(-ywx - a)), where a is max(0, -ywx) + // https://hips.seas.harvard.edu/blog/2013/01/09/computing-log-sum-exp/ + const double y_wx = example_label * wx; + if (y_wx > 0) { + // 0 + log(e^(0) + e^(-ywx - 0)) + // log(1 + e^(-ywx)) + return log(1 + exp(-y_wx)) * example_weight; + } + // -ywx + log(e^(ywx) + e^(-ywx + ywx)) + // log(e^(ywx) + e^(0)) - ywx + // log(1 + e^(ywx)) - ywx + return (log(1 + exp(y_wx)) - y_wx) * example_weight; + } + + // Derivative of logistic loss + double PrimalLossDerivative(const double wx, const double label, + const double example_weight) const final { + double inverse_exp_term = 0; + if (label * wx > 0) { + inverse_exp_term = exp(-label * wx) / (1 + exp(-label * wx)); + } else { + inverse_exp_term = 1 / (1 + exp(label * wx)); + } + return -inverse_exp_term * label * example_weight; + } + + // The smoothness constant is 4 since the derivative of logistic loss, which + // is exp(-x) / (1 + exp(-x)) can be shown to 0.25-Lipschitz (its derivative + // is bounded by 0.25) + double SmoothnessConstant() const final { return 4; } + + // Converts binary example labels from 0.0 or 1.0 to -1.0 or 1.0 respectively + // as expected by logistic regression. + Status ConvertLabel(float* const example_label) const final { + if (*example_label == 0.0) { + *example_label = -1; + return Status::OK(); + } + if (*example_label == 1.0) { + return Status::OK(); + } + return errors::InvalidArgument( + "Only labels of 0.0 or 1.0 are supported right now. " + "Found example with label: ", + *example_label); + } + + private: + // We use Newton algorithm on a modified function (see readme.md). + double NewtonStep(const double x, const int num_loss_partitions, + const double label, const double wx, + const double example_weight, + const double weighted_example_norm, + const double current_dual) const { + const double tanhx = tanh(x); + const double numerator = -2 * label * x - wx - + num_loss_partitions * weighted_example_norm * + example_weight * + (0.5 * (1 + tanhx) / label - current_dual); + const double denominator = + -2 * label - num_loss_partitions * weighted_example_norm * + example_weight * (1 - tanhx * tanhx) * 0.5 / label; + return x - numerator / denominator; + } +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_LOGISTIC_LOSS_H_ diff --git a/linux/include/tensorflow/core/kernels/lookup_table_init_op.h b/linux/include/tensorflow/core/kernels/lookup_table_init_op.h new file mode 100644 index 000000000..101e52865 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/lookup_table_init_op.h @@ -0,0 +1,33 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_LOOKUP_TABLE_INIT_OP_H_ +#define TENSORFLOW_CORE_KERNELS_LOOKUP_TABLE_INIT_OP_H_ + +#include "tensorflow/core/kernels/initializable_lookup_table.h" + +namespace tensorflow { +namespace lookup { + +// Helper function to initialize an InitializableLookupTable from a text file. +Status InitializeTableFromTextFile(const string& filename, int64 vocab_size, + char delimiter, int32 key_index, + int32 value_index, Env* env, + InitializableLookupTable* table); + +} // namespace lookup +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_LOOKUP_TABLE_INIT_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/lookup_table_op.h b/linux/include/tensorflow/core/kernels/lookup_table_op.h new file mode 100644 index 000000000..9451247f2 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/lookup_table_op.h @@ -0,0 +1,278 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_LOOKUP_TABLE_OP_H_ +#define TENSORFLOW_CORE_KERNELS_LOOKUP_TABLE_OP_H_ + +#include "tensorflow/core/framework/lookup_interface.h" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/resource_mgr.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/kernels/bounds_check.h" +#include "tensorflow/core/kernels/lookup_util.h" +#include "tensorflow/core/lib/core/errors.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/lib/gtl/map_util.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/thread_annotations.h" + +namespace tensorflow { + +// Lookup table op that supports different table implementations specified by +// the 'Container' template. Container must be derived from LookupInterface. The +// key and value are of the templated type "key_dtype" and "value_dtype" +// respectively. +template +class LookupTableOp : public OpKernel { + public: + // ctx is not owned by this class. + explicit LookupTableOp(OpKernelConstruction* ctx) + : OpKernel(ctx), table_handle_set_(false) { + OP_REQUIRES_OK(ctx, ctx->allocate_persistent(tensorflow::DT_STRING, + tensorflow::TensorShape({2}), + &table_handle_, nullptr)); + OP_REQUIRES_OK( + ctx, ctx->GetAttr("use_node_name_sharing", &use_node_name_sharing_)); + } + + // ctx is not owned by this function. + void Compute(OpKernelContext* ctx) override { + mutex_lock l(mu_); + + if (!table_handle_set_) { + OP_REQUIRES_OK(ctx, cinfo_.Init(ctx->resource_manager(), def(), + use_node_name_sharing_)); + } + + auto creator = [ctx, this](lookup::LookupInterface** ret) { + lookup::LookupInterface* container = new Container(ctx, this); + if (!ctx->status().ok()) { + container->Unref(); + return ctx->status(); + } + if (ctx->track_allocations()) { + ctx->record_persistent_memory_allocation( + container->MemoryUsed() + table_handle_.AllocatedBytes()); + } + *ret = container; + return Status::OK(); + }; + + lookup::LookupInterface* table = nullptr; + OP_REQUIRES_OK(ctx, + cinfo_.resource_manager() + ->template LookupOrCreate( + cinfo_.container(), cinfo_.name(), &table, creator)); + core::ScopedUnref unref_me(table); + + OP_REQUIRES_OK(ctx, lookup::CheckTableDataTypes( + *table, DataTypeToEnum::v(), + DataTypeToEnum::v(), cinfo_.name())); + + if (ctx->expected_output_dtype(0) == DT_RESOURCE) { + Tensor* handle; + OP_REQUIRES_OK(ctx, ctx->allocate_output(0, TensorShape({}), &handle)); + handle->scalar()() = + MakeResourceHandle(ctx, cinfo_.container(), + cinfo_.name()); + } else { + if (!table_handle_set_) { + auto h = table_handle_.AccessTensor(ctx)->template flat(); + h(0) = cinfo_.container(); + h(1) = cinfo_.name(); + } + ctx->set_output_ref(0, &mu_, table_handle_.AccessTensor(ctx)); + } + table_handle_set_ = true; + } + + ~LookupTableOp() override { + // If the table object was not shared, delete it. + if (table_handle_set_ && cinfo_.resource_is_private_to_kernel()) { + if (!cinfo_.resource_manager() + ->template Delete(cinfo_.container(), + cinfo_.name()) + .ok()) { + // Do nothing; the resource can have been deleted by session resets. + } + } + } + + private: + mutex mu_; + PersistentTensor table_handle_ GUARDED_BY(mu_); + bool table_handle_set_ GUARDED_BY(mu_); + ContainerInfo cinfo_; + bool use_node_name_sharing_; + + TF_DISALLOW_COPY_AND_ASSIGN(LookupTableOp); +}; + +namespace lookup { + +// Ensure that the compiler cannot elide a copy into a local, for +// bounds checking on source tensors that might be updated asynchronously for +// integral types. However non-integer variables are not allowed and therefore +// the local copy is unnecessary. +template +T SubtleMustCopyIfIntegral(const T& value) { + return internal::SubtleMustCopy(value); +} + +inline const string& SubtleMustCopyIfIntegral(const string& value) { + return value; +} + +inline const float SubtleMustCopyIfIntegral(const float value) { return value; } + +inline const double SubtleMustCopyIfIntegral(const double value) { + return value; +} + +inline const Variant& SubtleMustCopyIfIntegral(const Variant& value) { + return value; +} + +// Lookup table that wraps an unordered_map, where the key and value data type +// is specified. +// +// This table is recommended for any variations to key values. +// +// For look up, the table is required to be initialized (allocated +// and populated). Once the table is marked as initialized it becomes read-only. +// +// Sample use case: +// +// HashTable table; // int64 -> int64. +// table.Prepare(10); // Prepare the underlying data structure, the number of +// // elements is required by interface, but not used. +// // Populate the table, elements could be added in one or multiple calls. +// table.Insert(key_tensor, value_tensor); // Populate the table. +// ... +// table.set_is_initialized(); +// +// table.Find(in_t, &out_t, default_t) +// +template +class HashTable : public InitializableLookupTable { + public: + HashTable(OpKernelContext* ctx, OpKernel* kernel) {} + + size_t size() const override { + // return the size of the table only if it's initialized, otherwise 0. + if (!is_initialized_) { + return 0; + } + std::atomic_thread_fence(std::memory_order_acquire); + return table_ ? table_->size() : 0; + } + + Status ExportValues(OpKernelContext* context) override { + if (!is_initialized_) { + return errors::Aborted("HashTable is not initialized."); + } + + const int64 size = table_->size(); + + Tensor* keys; + Tensor* values; + TF_RETURN_IF_ERROR( + context->allocate_output("keys", TensorShape({size}), &keys)); + TF_RETURN_IF_ERROR( + context->allocate_output("values", TensorShape({size}), &values)); + + auto keys_data = keys->flat(); + auto values_data = values->flat(); + int64 i = 0; + for (auto it = table_->begin(); it != table_->end(); ++it, ++i) { + keys_data(i) = it->first; + values_data(i) = it->second; + } + return Status::OK(); + } + + DataType key_dtype() const override { return DataTypeToEnum::v(); } + + DataType value_dtype() const override { return DataTypeToEnum::v(); } + + protected: + Status DoPrepare(size_t unused) override { + if (is_initialized_) { + return errors::Aborted("HashTable already initialized."); + } + if (!table_) { + table_ = std::unique_ptr>( + new std::unordered_map()); + } + return Status::OK(); + }; + + Status DoLazyPrepare(std::function unused) override { + constexpr size_t kUnusedSize = 0; + return DoPrepare(kUnusedSize); + } + + Status DoInsert(const Tensor& keys, const Tensor& values) override { + if (!table_) { + return errors::FailedPrecondition("HashTable is not prepared."); + } + + const auto key_values = keys.flat(); + const auto value_values = values.flat(); + for (int64 i = 0; i < key_values.size(); ++i) { + const K key = SubtleMustCopyIfIntegral(key_values(i)); + const V value = SubtleMustCopyIfIntegral(value_values(i)); + const V& previous_value = gtl::LookupOrInsert(table_.get(), key, value); + if (previous_value != value) { + return errors::FailedPrecondition( + "HashTable has different value for same key. Key ", key, " has ", + previous_value, " and trying to add value ", value); + } + } + return Status::OK(); + } + + Status DoFind(const Tensor& key, Tensor* value, + const Tensor& default_value) override { + const V default_val = default_value.flat()(0); + const auto key_values = key.flat(); + auto value_values = value->flat(); + + for (int64 i = 0; i < key_values.size(); ++i) { + value_values(i) = gtl::FindWithDefault( + *table_, SubtleMustCopyIfIntegral(key_values(i)), default_val); + } + return Status::OK(); + } + + int64 MemoryUsed() const override { + if (table_) { + const int64 num_elements = table_->size(); + return num_elements * (sizeof(K) + sizeof(V)); + } else { + return 0; + } + } + + private: + std::unique_ptr> table_; +}; + +} // namespace lookup + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_LOOKUP_TABLE_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/lookup_util.h b/linux/include/tensorflow/core/kernels/lookup_util.h new file mode 100644 index 000000000..ec28cf9fa --- /dev/null +++ b/linux/include/tensorflow/core/kernels/lookup_util.h @@ -0,0 +1,52 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_LOOKUP_UTIL_H_ +#define TENSORFLOW_CORE_KERNELS_LOOKUP_UTIL_H_ + +#include "tensorflow/core/framework/lookup_interface.h" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/kernels/initializable_lookup_table.h" + +namespace tensorflow { +namespace lookup { + +// Gets the LookupTable stored in the ctx->resource_manager() with key +// passed by attribute with name input_name, returns null if the table +// doesn't exist. +Status GetLookupTable(const string& input_name, OpKernelContext* ctx, + LookupInterface** table); + +// Gets the InitializableLookupTable stored in the +// ctx->resource_manager() with key passed by attribute with name +// input_name, returns null if the table doesn't exist. +Status GetInitializableLookupTable(const string& input_name, + OpKernelContext* ctx, + InitializableLookupTable** table); + +// Verify that the given key_dtype and value_dtype matches the corresponding +// table's data types. +Status CheckTableDataTypes(const LookupInterface& table, DataType key_dtype, + DataType value_dtype, const string& table_name); + +Status InitializeTableFromTextFile(const string& filename, int64 vocab_size, + char delimiter, int32 key_index, + int32 value_index, Env* env, + InitializableLookupTable* table); + +} // namespace lookup +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_LOOKUP_UTIL_H_ diff --git a/linux/include/tensorflow/core/kernels/loss.h b/linux/include/tensorflow/core/kernels/loss.h new file mode 100644 index 000000000..7db348800 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/loss.h @@ -0,0 +1,59 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CORE_KERNELS_LOSS_H_ +#define TENSORFLOW_CORE_KERNELS_LOSS_H_ + +#include "tensorflow/core/lib/core/status.h" + +namespace tensorflow { + +class DualLossUpdater { + public: + virtual ~DualLossUpdater() {} + + // Compute update dual (alpha), based on a single example. Various strategies + // can be employed here, like newton step and/or line search or approximate + // step that decreases the dual sub-optimality. + virtual double ComputeUpdatedDual( + const int num_loss_partitions, const double label, + const double example_weight, const double current_dual, const double wx, + const double weighted_example_norm) const = 0; + + // Compute dual loss based on the current dual (alpha), example label (y) + // and example weight (cost). + virtual double ComputeDualLoss(const double current_dual, + const double example_label, + const double example_weight) const = 0; + + // Compute the primal loss based on current estimate of log-odds(wx), + // example label (y) and example weight (cost). + virtual double ComputePrimalLoss(const double wx, const double example_label, + const double example_weight) const = 0; + + // Primal loss derivative used to compute the dual residue in AdaSDCA + virtual double PrimalLossDerivative(const double wx, + const double example_label, + const double example_weight) const = 0; + + // This is gamma such that the loss derivative is 1/gamma Lipschitz + virtual double SmoothnessConstant() const = 0; + + // Converts binary example labels from 0.0 or 1.0 to appropriate range for + // each loss function. + virtual Status ConvertLabel(float* const example_label) const = 0; +}; + +} // namespace tensorflow +#endif // TENSORFLOW_CORE_KERNELS_LOSS_H_ diff --git a/linux/include/tensorflow/core/kernels/matmul_op.h b/linux/include/tensorflow/core/kernels/matmul_op.h new file mode 100644 index 000000000..4b74a6402 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/matmul_op.h @@ -0,0 +1,120 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_MATMUL_OP_H_ +#define TENSORFLOW_CORE_KERNELS_MATMUL_OP_H_ + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/lib/hash/hash.h" + +namespace tensorflow { +namespace functor { + +// Helpers to define tensor needed by MatMul op. +template +struct MatMulTypes { + typedef Eigen::TensorMap, Eigen::Aligned> + out_type; + typedef Eigen::TensorMap, + Eigen::Aligned> + in_type; +}; + +template +void MatMul(const Device& d, Out out, In0 in0, In1 in1, + const DimPair& dim_pair) { + out.device(d) = in0.contract(in1, dim_pair); +} + +template +struct MatMulFunctor { + // Computes on device "d": out = in0 * in1, where * is matrix + // multiplication. + void operator()( + const Device& d, typename MatMulTypes::out_type out, + typename MatMulTypes::in_type in0, + typename MatMulTypes::in_type in1, + const Eigen::array, 1>& dim_pair); +}; + +} // end namespace functor + +#if GOOGLE_CUDA +// Encapsulate all the shape information that is used in matmul operations. +class MatmulParameters { + public: + MatmulParameters(bool transa, bool transb, uint64 m, uint64 n, uint64 k, + DataType dtype, int device_id) + : transa_(transa), + transb_(transb), + m_(m), + n_(n), + k_(k), + dtype_(dtype), + device_id_(device_id) { + hash_code_ = transa; + hash_code_ = Hash64Combine(hash_code_, transb); + hash_code_ = Hash64Combine(hash_code_, m); + hash_code_ = Hash64Combine(hash_code_, n); + hash_code_ = Hash64Combine(hash_code_, k); + hash_code_ = Hash64Combine(hash_code_, dtype); + hash_code_ = Hash64Combine(hash_code_, device_id); + } + bool operator==(const MatmulParameters& other) const { + return this->get_data_as_tuple() == other.get_data_as_tuple(); + } + + bool operator!=(const MatmulParameters& other) const { + return !(*this == other); + } + uint64 hash() const { return hash_code_; } + + string ToString() const { + // clang-format off + return strings::StrCat( + transa_, ", ", transb_, ", ", + m_, ", ", n_, ", ", k_, + dtype_, ", ", device_id_); + // clang-format on + } + + private: + typedef std::tuple + ParameterDataType; + + ParameterDataType get_data_as_tuple() const { + return std::make_tuple(transa_, transb_, m_, n_, k_, dtype_, device_id_); + } + + bool transa_; + bool transb_; + uint64 m_; + uint64 n_; + uint64 k_; + DataType dtype_; + int device_id_; + uint64 hash_code_; +}; + +typedef Eigen::GpuDevice GPUDevice; + +#endif // GOOGLE_CUDA + +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_MATMUL_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/matrix_band_part_op.h b/linux/include/tensorflow/core/kernels/matrix_band_part_op.h new file mode 100644 index 000000000..b04e36db8 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/matrix_band_part_op.h @@ -0,0 +1,37 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_MATRIX_BAND_PART_OP_H_ +#define TENSORFLOW_CORE_KERNELS_MATRIX_BAND_PART_OP_H_ + +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { +namespace functor { + +template +struct MatrixBandPartFunctor { + void operator()(OpKernelContext* context, const Device& device, + int num_upper_diags, int num_lower_diags, + typename TTypes::ConstTensor input, + typename TTypes::Tensor output); +}; + +} // namespace functor +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_MATRIX_BAND_PART_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/matrix_diag_op.h b/linux/include/tensorflow/core/kernels/matrix_diag_op.h new file mode 100644 index 000000000..108ba0f56 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/matrix_diag_op.h @@ -0,0 +1,94 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_MATRIX_DIAG_OP_H_ +#define TENSORFLOW_CORE_KERNELS_MATRIX_DIAG_OP_H_ + +// Generator definition for MatrixDiagOp, must be compilable by nvcc. + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +namespace generator { + +template +class MatrixDiagPartGenerator { + public: + EIGEN_DEVICE_FUNC EIGEN_ALWAYS_INLINE + MatrixDiagPartGenerator(typename TTypes::ConstTensor input) + : input_(input) {} + + EIGEN_DEVICE_FUNC EIGEN_ALWAYS_INLINE T + operator()(const Eigen::array& coords) const { + Eigen::array diag_from_coords( + {coords[0], coords[1], coords[1]}); + return input_(diag_from_coords); + } + + private: + typename TTypes::ConstTensor input_; +}; + +template +class MatrixDiagGenerator { + public: + EIGEN_DEVICE_FUNC EIGEN_ALWAYS_INLINE + MatrixDiagGenerator(typename TTypes::ConstTensor input) + : input_(input) {} + + EIGEN_DEVICE_FUNC EIGEN_ALWAYS_INLINE T + operator()(const Eigen::array& coords) const { + if (coords[2] != coords[1]) return T(); + + Eigen::array diag_coords({coords[0], coords[1]}); + return input_(diag_coords); + } + + private: + typename TTypes::ConstTensor input_; +}; + +} // namespace generator + +namespace functor { + +template +struct MatrixDiagPart { + EIGEN_ALWAYS_INLINE static void Compute( + const Device& d, typename TTypes::ConstTensor input, + typename TTypes::Tensor output) { + generator::MatrixDiagPartGenerator generator(input); + output.device(d) = output.generate(generator); + } +}; + +template +struct MatrixDiag { + EIGEN_ALWAYS_INLINE static void Compute( + const Device& d, typename TTypes::ConstTensor input, + typename TTypes::Tensor output) { + generator::MatrixDiagGenerator generator(input); + output.device(d) = output.generate(generator); + } +}; + +} // namespace functor + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_MATRIX_DIAG_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/matrix_set_diag_op.h b/linux/include/tensorflow/core/kernels/matrix_set_diag_op.h new file mode 100644 index 000000000..341ef12e9 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/matrix_set_diag_op.h @@ -0,0 +1,37 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_MATRIX_SET_DIAG_OP_H_ +#define TENSORFLOW_CORE_KERNELS_MATRIX_SET_DIAG_OP_H_ + +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { +namespace functor { + +template +struct MatrixSetDiag { + static void Compute(OpKernelContext* context, const Device& d, + typename TTypes::ConstTensor input, + typename TTypes::ConstTensor diag, + typename TTypes::Tensor output); +}; + +} // namespace functor +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_MATRIX_SET_DIAG_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/matrix_solve_ls_op_impl.h b/linux/include/tensorflow/core/kernels/matrix_solve_ls_op_impl.h new file mode 100644 index 000000000..00a05a87a --- /dev/null +++ b/linux/include/tensorflow/core/kernels/matrix_solve_ls_op_impl.h @@ -0,0 +1,166 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_MATRIX_SOLVE_LS_OP_IMPL_H_ +#define TENSORFLOW_CORE_KERNELS_MATRIX_SOLVE_LS_OP_IMPL_H_ + +// See docs in ../ops/linalg_ops.cc. + +#include "third_party/eigen3/Eigen/Cholesky" +#include "third_party/eigen3/Eigen/Core" +#include "third_party/eigen3/Eigen/QR" +#include "tensorflow/core/framework/kernel_def_builder.h" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/kernels/linalg_ops_common.h" +#include "tensorflow/core/lib/core/errors.h" +#include "tensorflow/core/platform/logging.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +template +class MatrixSolveLsOp : public LinearAlgebraOp { + public: + typedef LinearAlgebraOp Base; + + explicit MatrixSolveLsOp(OpKernelConstruction* context) : Base(context) { + OP_REQUIRES_OK(context, context->GetAttr("fast", &fast_)); + } + + using TensorShapes = typename Base::TensorShapes; + using Matrix = typename Base::Matrix; + using MatrixMaps = typename Base::MatrixMaps; + using ConstMatrixMap = typename Base::ConstMatrixMap; + using ConstMatrixMaps = typename Base::ConstMatrixMaps; + + // Tell the base class to ignore the regularization parameter + // in context->input(2). + int NumMatrixInputs(const OpKernelContext* context) const final { return 2; } + + void ValidateInputMatrixShapes( + OpKernelContext* context, + const TensorShapes& input_matrix_shapes) const final { + Base::ValidateSolver(context, input_matrix_shapes); + } + + TensorShapes GetOutputMatrixShapes( + const TensorShapes& input_matrix_shapes) const final { + return TensorShapes({TensorShape({input_matrix_shapes[0].dim_size(1), + input_matrix_shapes[1].dim_size(1)})}); + } + + int64 GetCostPerUnit(const TensorShapes& input_matrix_shapes) const final { + double m = static_cast(input_matrix_shapes[0].dim_size(0)); + double n = static_cast(input_matrix_shapes[0].dim_size(1)); + double num_rhss = static_cast(input_matrix_shapes[1].dim_size(1)); + double cost = std::max(m, n) * std::min(m, n) * (std::min(m, n) + num_rhss); + return cost >= static_cast(kint64max) ? kint64max + : static_cast(cost); + } + + bool EnableInputForwarding() const final { return false; } + + void ComputeMatrix(OpKernelContext* context, const ConstMatrixMaps& inputs, + MatrixMaps* outputs) final { + const ConstMatrixMap& matrix = inputs[0]; + const ConstMatrixMap& rhs = inputs[1]; + const auto& l2_regularizer_in = context->input(2); + OP_REQUIRES( + context, TensorShapeUtils::IsScalar(l2_regularizer_in.shape()), + errors::InvalidArgument("l2_regularizer must be scalar, got shape ", + l2_regularizer_in.shape().DebugString())); + const double l2_regularizer = l2_regularizer_in.scalar()(); + OP_REQUIRES(context, l2_regularizer >= 0, + errors::InvalidArgument("l2_regularizer must be >= 0.")); + + const int64 rows = matrix.rows(); + const int64 cols = matrix.cols(); + if (rows == 0 || cols == 0) { + // The result is the empty matrix. + return; + } + if (fast_) { + // The fast branch assumes that matrix is not rank deficient and + // not too ill-conditioned. Specifically, the reciprocal condition number + // should be greater than the square root of the machine precision, i.e. + // 1 / cond(matrix) > sqrt(std::numeric_limits::epsilon()). + // This branch solves over- or underdetermined least-squares problems + // via the normal equations and Cholesky decomposition. + if (matrix.rows() >= matrix.cols()) { + // Overdetermined case (rows >= cols): Solves the ordinary (possibly + // regularized) least-squares problem + // min || A * X - RHS ||_F^2 + l2_regularizer ||X||_F^2 + // by solving the normal equations + // (A^T * A + l2_regularizer * I) X = A^T RHS + // using Cholesky decomposition. + Matrix gramian(cols, cols); + gramian.template triangularView() = + matrix.adjoint() * matrix; + if (l2_regularizer > 0) { + gramian += + (Scalar(l2_regularizer) * Matrix::Ones(cols, 1)).asDiagonal(); + } + const Eigen::LLT, Eigen::Lower> llt(gramian); + OP_REQUIRES( + context, llt.info() == Eigen::Success, + errors::InvalidArgument("Input matrix was rank deficient or " + "ill-conditioned. Try setting fast=False " + "or provide a larger l2_regularizer > 0.")); + outputs->at(0).noalias() = matrix.adjoint() * rhs; + llt.solveInPlace(outputs->at(0)); + } else { + // Underdetermined case (rows < cols): Solves the minimum-norm problem + // min ||X||_F^2 s.t. A*X = RHS + // by solving the normal equations of the second kind + // (A * A^T + l2_regularizer * I) Z = RHS, X = A^T * Z + // using Cholesky decomposition. + Matrix gramian(rows, rows); + gramian.template triangularView() = + matrix * matrix.adjoint(); + if (l2_regularizer > 0) { + gramian += + (Scalar(l2_regularizer) * Matrix::Ones(rows, 1)).asDiagonal(); + } + const Eigen::LLT, Eigen::Lower> llt(gramian); + OP_REQUIRES( + context, llt.info() == Eigen::Success, + errors::InvalidArgument("Input matrix was rank deficient or " + "ill-conditioned. Try setting fast=False " + "or provide an l2_regularizer > 0.")); + outputs->at(0).noalias() = matrix.adjoint() * llt.solve(rhs); + } + } else { + // Use complete orthogonal decomposition which is backwards stable and + // will compute the minimum-norm solution for rank-deficient matrices. + // This is 6-7 times slower than the fast path. + // + // TODO(rmlarsen): The implementation of + // Eigen::CompleteOrthogonalDecomposition is not blocked, so for + // matrices that do not fit in cache, it is significantly slower than + // the equivalent blocked LAPACK routine xGELSY (e.g. Eigen is ~3x + // slower for 4k x 4k matrices). + // See http://www.netlib.org/lapack/lawnspdf/lawn114.pdf + outputs->at(0) = matrix.completeOrthogonalDecomposition().solve(rhs); + } + } + + private: + bool fast_; +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_MATRIX_SOLVE_LS_OP_IMPL_H_ diff --git a/linux/include/tensorflow/core/kernels/maxpooling_op.h b/linux/include/tensorflow/core/kernels/maxpooling_op.h new file mode 100644 index 000000000..2adb8081c --- /dev/null +++ b/linux/include/tensorflow/core/kernels/maxpooling_op.h @@ -0,0 +1,54 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_MAXPOOLING_OP_H_ +#define TENSORFLOW_CORE_KERNELS_MAXPOOLING_OP_H_ +// Functor definition for MaxPoolingOp, must be compilable by nvcc. + +#include "tensorflow/core/framework/numeric_types.h" +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/framework/type_traits.h" +#include "tensorflow/core/kernels/eigen_pooling.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { +namespace functor { + +template +struct SpatialMaxPooling { + void operator()(const Device& d, typename TTypes::Tensor output, + typename TTypes::ConstTensor input, int window_rows, + int window_cols, int row_stride, int col_stride, + const Eigen::PaddingType& padding) { + // Because we swap the layout, we swap the row/cols as well + output.swap_layout().device(d) = + Eigen::SpatialMaxPooling(input.swap_layout(), window_cols, window_rows, + col_stride, row_stride, padding); + } +}; + +template +struct SpatialMaxPooling { + void operator()(const Device& d, typename TTypes::Tensor output, + typename TTypes::ConstTensor input, int window_rows, + int window_cols, int row_stride, int col_stride, + const Eigen::PaddingType& padding) {} +}; + +} // namespace functor + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_MAXPOOLING_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/meta_support.h b/linux/include/tensorflow/core/kernels/meta_support.h new file mode 100644 index 000000000..97f39eb59 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/meta_support.h @@ -0,0 +1,112 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CONTRIB_QUANTIZATION_KERNELS_META_SUPPORT_H_ +#define TENSORFLOW_CONTRIB_QUANTIZATION_KERNELS_META_SUPPORT_H_ + +#include "meta/multi_thread_gemm.h" +#include "meta/multi_thread_transform.h" +#include "meta/quantized_mul_kernels.h" +#include "meta/streams.h" +#include "meta/transform_kernels.h" + +#include "tensorflow/core/framework/numeric_types.h" + +namespace tensorflow { + +class OpKernelContext; + +namespace meta { + +// Gemmlowp/meta is a small library of optimized Arm32/64 kernels for quantized +// matrix multiplication and other quantized computations. + +// Set the maximum number of threads of computation that the internal workers +// pool can use. If num_threads is 0, then use intra_op_parallelism_threads. +void SetNumThreads(int num_threads); + +int GetNumThreads(); + +// Toggle the internal workers pool. If set to false, the computations will +// use the worker pool passed each time in the OpKernelContext. If set to true +// then the OpKernelContext will be ignored, and the internal optimized workers +// pool will be used. +// +// The internal workers pool is disabled by default (false). +void SetUseLocalContext(bool use_local_context); + +bool GetUseLocalContext(); + +// Toggles the codepath. Enabled by default (true) on supported platforms. +void SetEnabled(bool enabled); + +// Returns true if the codepath is supported and is enabled. Use this call +// before calling the compute functions. If the codepath is not supported, and +// any of the compute function is called, the library will log a FATAL error. +bool IsSupportedAndEnabled(); + +// Calculate the quantized matrix multiplication: +// +// for (i, j) in [0, m) x [0, n) do +// c_data[i, j] := +// sum((a_data[i, l] + offset_a) * (b_data[l, j] + offset_b)) : l in [0, k) +// +// If transpose_a is false the lhs operand has row major layout, otherwise +// column major. Similarly transpose_b describes the layout of the rhs operand. +// lda, ldb, and ldc are the strides of the lhs operand, rhs operand and the +// result arrays. +void QuantizedGemm(OpKernelContext* context, bool transpose_a, bool transpose_b, + const quint8* a_data, const quint8* b_data, qint32* c_data, + int m, int n, int k, int offset_a, int offset_b, int lda, + int ldb, int ldc); + +// Take an array of numbers from the range [input_min, input_max] quantized +// uniformly to int32 values, recover their float values, and then quantize +// them back uniformly to the range [output_min, output_max] as uint8. +// Saturate the uint8 values. +void Requantize(OpKernelContext* context, const qint32* input, int count, + float input_min, float input_max, float output_min, + float output_max, quint8* output); + +// Take an array of numbers from the range [range_min, range_max] quantized +// uniformly to uint8 values and recover their float values. +void Dequantize(OpKernelContext* context, const quint8* input, int count, + float range_min, float range_max, float* output); + +// Take an array of float values and quantize them uniformly to the range +// [range_min, range_max] expressed as uint8. Saturate the uint8 values. +void Quantize(OpKernelContext*, const float* input, int count, float range_min, + float range_max, quint8* output); + +// Take two arrays: the inputs and the bias quantized uniformly in the ranges +// [input_min, input_max], and [bias_min, bias_max] accordingly, as uint8 +// values. Recover their float values. Add the values. Quantize them back +// uniformly to the range [output_min, output_max] as int32. Saturate the +// int32 values. +void QuantizedBiasAdd(OpKernelContext* context, const quint8* input, + int input_count, const quint8* bias, int bias_count, + float input_min, float input_max, float bias_min, + float bias_max, float output_min, float output_max, + qint32* output); + +// Take an array of uint8 values and clamp them to the range [clamp_min, +// clamp_max]. +void Clamp(OpKernelContext* context, const quint8* input, int input_count, + quint8 clamp_min, quint8 clamp_max, quint8* output); + +} // namespace meta +} // namespace tensorflow + +#endif // TENSORFLOW_CONTRIB_QUANTIZATION_KERNELS_META_SUPPORT_H_ diff --git a/linux/include/tensorflow/core/kernels/mfcc.h b/linux/include/tensorflow/core/kernels/mfcc.h new file mode 100644 index 000000000..8eee76f7f --- /dev/null +++ b/linux/include/tensorflow/core/kernels/mfcc.h @@ -0,0 +1,76 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// Basic class for computing MFCCs from spectrogram slices. + +#ifndef TENSORFLOW_CORE_KERNELS_MFCC_H_ +#define TENSORFLOW_CORE_KERNELS_MFCC_H_ + +#include + +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/kernels/mfcc_dct.h" +#include "tensorflow/core/kernels/mfcc_mel_filterbank.h" +#include "tensorflow/core/platform/logging.h" + +namespace tensorflow { + +class Mfcc { + public: + Mfcc(); + bool Initialize(int input_length, double input_sample_rate); + + // Input is a single squared-magnitude spectrogram frame. The input spectrum + // is converted to linear magnitude and weighted into bands using a + // triangular mel filterbank, and a discrete cosine transform (DCT) of the + // values is taken. Output is populated with the lowest dct_coefficient_count + // of these values. + void Compute(const std::vector& spectrogram_frame, + std::vector* output) const; + + void set_upper_frequency_limit(double upper_frequency_limit) { + CHECK(!initialized_) << "Set frequency limits before calling Initialize."; + upper_frequency_limit_ = upper_frequency_limit; + } + + void set_lower_frequency_limit(double lower_frequency_limit) { + CHECK(!initialized_) << "Set frequency limits before calling Initialize."; + lower_frequency_limit_ = lower_frequency_limit; + } + + void set_filterbank_channel_count(int filterbank_channel_count) { + CHECK(!initialized_) << "Set channel count before calling Initialize."; + filterbank_channel_count_ = filterbank_channel_count; + } + + void set_dct_coefficient_count(int dct_coefficient_count) { + CHECK(!initialized_) << "Set coefficient count before calling Initialize."; + dct_coefficient_count_ = dct_coefficient_count; + } + + private: + MfccMelFilterbank mel_filterbank_; + MfccDct dct_; + bool initialized_; + double lower_frequency_limit_; + double upper_frequency_limit_; + int filterbank_channel_count_; + int dct_coefficient_count_; + TF_DISALLOW_COPY_AND_ASSIGN(Mfcc); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_MFCC_H_ diff --git a/linux/include/tensorflow/core/kernels/mfcc_dct.h b/linux/include/tensorflow/core/kernels/mfcc_dct.h new file mode 100644 index 000000000..888b8e8df --- /dev/null +++ b/linux/include/tensorflow/core/kernels/mfcc_dct.h @@ -0,0 +1,44 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// Basic minimal DCT class for MFCC speech processing. + +#ifndef TENSORFLOW_CORE_KERNELS_MFCC_DCT_H_ +#define TENSORFLOW_CORE_KERNELS_MFCC_DCT_H_ + +#include + +#include "tensorflow/core/framework/op_kernel.h" + +namespace tensorflow { + +class MfccDct { + public: + MfccDct(); + bool Initialize(int input_length, int coefficient_count); + void Compute(const std::vector& input, + std::vector* output) const; + + private: + bool initialized_; + int coefficient_count_; + int input_length_; + std::vector > cosines_; + TF_DISALLOW_COPY_AND_ASSIGN(MfccDct); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_MFCC_DCT_H_ diff --git a/linux/include/tensorflow/core/kernels/mfcc_mel_filterbank.h b/linux/include/tensorflow/core/kernels/mfcc_mel_filterbank.h new file mode 100644 index 000000000..37c3936e8 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/mfcc_mel_filterbank.h @@ -0,0 +1,64 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// Basic class for applying a mel-scale mapping to a power spectrum. + +#ifndef TENSORFLOW_CORE_KERNELS_MFCC_MEL_FILTERBANK_H_ +#define TENSORFLOW_CORE_KERNELS_MFCC_MEL_FILTERBANK_H_ + +#include +#include "tensorflow/core/framework/op_kernel.h" + +namespace tensorflow { + +class MfccMelFilterbank { + public: + MfccMelFilterbank(); + bool Initialize(int input_length, // Number of unique FFT bins fftsize/2+1. + double input_sample_rate, int output_channel_count, + double lower_frequency_limit, double upper_frequency_limit); + + // Takes a squared-magnitude spectrogram slice as input, computes a + // triangular-mel-weighted linear-magnitude filterbank, and places the result + // in output. + void Compute(const std::vector& input, + std::vector* output) const; + + private: + double FreqToMel(double freq) const; + bool initialized_; + int num_channels_; + double sample_rate_; + int input_length_; + std::vector center_frequencies_; // In mel, for each mel channel. + + // Each FFT bin b contributes to two triangular mel channels, with + // proportion weights_[b] going into mel channel band_mapper_[b], and + // proportion (1 - weights_[b]) going into channel band_mapper_[b] + 1. + // Thus, weights_ contains the weighting applied to each FFT bin for the + // upper-half of the triangular band. + std::vector weights_; // Right-side weight for this fft bin. + + // FFT bin i contributes to the upper side of mel channel band_mapper_[i] + std::vector band_mapper_; + int start_index_; // Lowest FFT bin used to calculate mel spectrum. + int end_index_; // Highest FFT bin used to calculate mel spectrum. + + TF_DISALLOW_COPY_AND_ASSIGN(MfccMelFilterbank); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_MFCC_MEL_FILTERBANK_H_ diff --git a/linux/include/tensorflow/core/kernels/mirror_pad_op.h b/linux/include/tensorflow/core/kernels/mirror_pad_op.h new file mode 100644 index 000000000..62aa7d5c2 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/mirror_pad_op.h @@ -0,0 +1,441 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_MIRROR_PAD_OP_H_ +#define TENSORFLOW_CORE_KERNELS_MIRROR_PAD_OP_H_ + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/platform/types.h" + +namespace Eigen { +template +class TensorMirrorPadOp; + +namespace internal { +template +struct traits> + : public traits { + typedef typename XprType::Scalar Scalar; + typedef traits XprTraits; + typedef typename XprTraits::StorageKind StorageKind; + typedef typename XprTraits::Index Index; + typedef typename XprType::Nested Nested; + typedef typename remove_reference::type _Nested; + static constexpr int NumDimensions = XprTraits::NumDimensions; + static constexpr int Layout = XprTraits::Layout; +}; + +template +struct eval, Eigen::Dense> { + typedef const TensorMirrorPadOp& type; +}; + +template +struct nested< + TensorMirrorPadOp, 1, + typename eval>::type> { + typedef TensorMirrorPadOp type; +}; +} // namespace internal + +template +class TensorMirrorPadOp + : public TensorBase, + ReadOnlyAccessors> { + public: + typedef typename Eigen::internal::traits::Scalar Scalar; + typedef typename Eigen::NumTraits::Real RealScalar; + typedef typename XprType::CoeffReturnType CoeffReturnType; + typedef typename Eigen::internal::nested::type Nested; + typedef typename Eigen::internal::traits::StorageKind + StorageKind; + typedef typename Eigen::internal::traits::Index Index; + + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE TensorMirrorPadOp( + const XprType& expr, const PaddingDimensions& padding_dims, Index offset) + : xpr_(expr), padding_dims_(padding_dims), offset_(offset) {} + + EIGEN_DEVICE_FUNC + const PaddingDimensions& padding() const { return padding_dims_; } + + EIGEN_DEVICE_FUNC + Index offset() const { return offset_; } + + EIGEN_DEVICE_FUNC + const typename internal::remove_all::type& + expression() const { + return xpr_; + } + + protected: + typename XprType::Nested xpr_; + const PaddingDimensions padding_dims_; + const Index offset_; +}; + +// Eval as rvalue +template +struct TensorEvaluator, + Device> { + typedef TensorMirrorPadOp XprType; + typedef typename XprType::Index Index; + static constexpr int Dims = internal::array_size::value; + typedef DSizes Dimensions; + typedef typename XprType::Scalar Scalar; + typedef typename XprType::CoeffReturnType CoeffReturnType; + // Copied from Eigen3 Github version 0e806c1. + typedef typename PacketType::type PacketReturnType; + + enum { + IsAligned = false, + PacketAccess = TensorEvaluator::PacketAccess, + BlockAccess = false, + PreferBlockAccess = false, + Layout = TensorEvaluator::Layout, + CoordAccess = true, + RawAccess = false + }; + + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE TensorEvaluator(const XprType& op, + const Device& device) + : impl_(op.expression(), device), padding_(op.padding()) { + EIGEN_STATIC_ASSERT(Dims > 0, YOU_MADE_A_PROGRAMMING_MISTAKE) + + // op.offset() == 0 if padding mode is symmetric. + // op.offset() == 1 if padding mode is reflect. + eigen_assert(op.offset() == 0 || op.offset() == 1); + left_offset_ = -1 + op.offset(); + right_offset_ = -1 - op.offset(); + + // This should trigger compilation error if padding dimensions and + // expression dimensions do not match. + dimensions_ = impl_.dimensions(); + for (int dim = 0; dim < Dims; ++dim) { + eigen_assert(padding_[dim].first + op.offset() <= dimensions_[dim]); + eigen_assert(padding_[dim].second + op.offset() <= dimensions_[dim]); + dimensions_[dim] += padding_[dim].first + padding_[dim].second; + } + + const auto& input_dims = impl_.dimensions(); + if (static_cast(Layout) == static_cast(ColMajor)) { + input_strides_[0] = 1; + output_strides_[0] = 1; + for (int i = 0; i < Dims - 1; ++i) { + input_strides_[i + 1] = input_strides_[i] * input_dims[i]; + output_strides_[i + 1] = output_strides_[i] * dimensions_[i]; + } + } else { + input_strides_[numext::maxi(0, Dims - 1)] = 1; + output_strides_[numext::maxi(0, Dims - 1)] = 1; + for (int i = Dims - 1; i > 0; --i) { + input_strides_[i - 1] = input_strides_[i] * input_dims[i]; + output_strides_[i - 1] = output_strides_[i] * dimensions_[i]; + } + } + } + + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE const Dimensions& dimensions() const { + return dimensions_; + } + + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE bool evalSubExprsIfNeeded(Scalar*) { + impl_.evalSubExprsIfNeeded(nullptr); + return true; + } + + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE void cleanup() { impl_.cleanup(); } + + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE CoeffReturnType + coeff(Index index) const { + eigen_assert(index < dimensions().TotalSize()); + const Index input_index = ToInputIndex(index); + return impl_.coeff(input_index); + } + + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE CoeffReturnType + coeff(array coords) const { + for (int dim = 0; dim < Dims; ++dim) { + coords[dim] = ToInputCoord(coords[dim], dim); + } + ReadInputHelper::CoordAccess> helper; + return helper(coords, input_strides_, impl_); + } + + template + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE PacketReturnType + packet(Index index) const { + constexpr int kPacketSize = + internal::unpacket_traits::size; + + EIGEN_STATIC_ASSERT(kPacketSize > 1, YOU_MADE_A_PROGRAMMING_MISTAKE) + eigen_assert(index + kPacketSize <= dimensions().TotalSize()); + + // Find the effective inner-most dimension where padding actually happens. + // NOTE: This is independent of index argument, and can be done in the + // constructor to save computation. However, if packet access does not + // happen, then moving to constructor will incur needless overhead. + int dim = -1; + if (static_cast(Layout) == static_cast(ColMajor)) { + for (int k = 0; k < Dims; ++k) { + if (padding_[k].first != 0 || padding_[k].second != 0) { + dim = k; + break; + } + } + } else { + for (int k = Dims - 1; k >= 0; --k) { + if (padding_[k].first != 0 || padding_[k].second != 0) { + dim = k; + break; + } + } + } + + const Index input_index = ToInputIndex(index); + + // If dim < 0, this means there is no padding at all. + if (dim < 0) { + return impl_.template packet(input_index); + } + + // Check if the way from the begin of the packet to the end of the packet + // is paved with contiguous road. That is, the indices must be between the + // padded region in the effective inner-most dimension. + const Index left = padding_[dim].first * output_strides_[dim]; + const Index right = + (dimensions_[dim] - padding_[dim].second) * output_strides_[dim]; + + if (left <= index && (index + kPacketSize - 1) < right) { + return impl_.template packet(input_index); + } + + // If the road is not contiguous, then fall back to coeff(). + EIGEN_ALIGN_MAX typename internal::remove_const::type + values[kPacketSize]; + values[0] = impl_.coeff(input_index); + for (int i = 1; i < kPacketSize; ++i) { + values[i] = coeff(index + i); + } + PacketReturnType result = internal::pload(values); + return result; + } + + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE TensorOpCost + costPerCoeff(bool vectorized) const { + constexpr int kPacketSize = + internal::unpacket_traits::size; + + const double compute_cost = Dims * (7 * TensorOpCost::AddCost() + + 2 * TensorOpCost::MulCost() + + TensorOpCost::DivCost()); + return impl_.costPerCoeff(vectorized) + + TensorOpCost(1, 0, compute_cost, vectorized, kPacketSize); + } + + EIGEN_DEVICE_FUNC Scalar* data() const { return nullptr; } + + protected: + using Coords = array; + + // Full template specialization is not allowed within non-fully specialized + // template class. Adding a dummy parameter to make specializations partial. + template + struct ReadInputHelper; + + template + struct ReadInputHelper { + template + EIGEN_DEVICE_FUNC EIGEN_ALWAYS_INLINE Index + operator()(const Coords& coord, const Coords& strides, const Eval& eval) { + Index index = 0; + for (int k = 0; k < Dims; ++k) { + index += coord[k] * strides[k]; + } + return eval.coeff(index); + } + }; + + template + struct ReadInputHelper { + template + EIGEN_DEVICE_FUNC EIGEN_ALWAYS_INLINE Index + operator()(const Coords& coord, const Coords& strides, const Eval& eval) { + return eval.coeff(coord); + } + }; + + EIGEN_DEVICE_FUNC EIGEN_ALWAYS_INLINE Index ToInputCoord(Index k, + int dim) const { + const Index m = impl_.dimensions()[dim]; + k -= padding_[dim].first; + if (k < 0) { + return -k + left_offset_; + } + if (k < m) { + return k; + } + return m - (k - m) + right_offset_; + } + + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE Index + ToInputIndex(const Coords& coords) const { + Index input_index = 0; + for (int dim = 0; dim < Dims; ++dim) { + input_index += ToInputCoord(coords[dim], dim) * input_strides_[dim]; + } + return input_index; + } + + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE Index ToInputIndex(Index index) const { + Index input_index = 0; + if (static_cast(Layout) == static_cast(ColMajor)) { + for (int dim = Dims - 1; dim > 0; --dim) { + const Index k = index / output_strides_[dim]; + index -= k * output_strides_[dim]; + input_index += ToInputCoord(k, dim) * input_strides_[dim]; + } + input_index += ToInputCoord(index, 0); + } else { + for (int dim = 0; dim < Dims - 1; ++dim) { + const Index k = index / output_strides_[dim]; + index -= k * output_strides_[dim]; + input_index += ToInputCoord(k, dim) * input_strides_[dim]; + } + input_index += ToInputCoord(index, Dims - 1); + } + + return input_index; + } + + TensorEvaluator impl_; + PaddingDimensions padding_; + Dimensions dimensions_; + array input_strides_; + array output_strides_; + + Index left_offset_; + Index right_offset_; +}; +} // namespace Eigen + +namespace tensorflow { +namespace functor { + +// offset argument must be either 0 or 1. This controls whether the boundary +// values are replicated (offset == 0) or not replicated (offset == 1). +template +struct MirrorPad { + void operator()(const Device& device, + typename TTypes::Tensor output, + typename TTypes::ConstTensor input, + typename TTypes::ConstMatrix padding, int offset) { + Eigen::array, Dims> padding_dims; + + for (int i = 0; i < Dims; ++i) { + padding_dims[i] = Eigen::IndexPair(padding(i, 0), padding(i, 1)); + } + + output.device(device) = MirrorPadOp(input, padding_dims, offset); + } + + template + static const Eigen::TensorMirrorPadOp + MirrorPadOp( + const Eigen::TensorBase& tensor, + const PaddingDimensions& padding, int offset) { + return Eigen::TensorMirrorPadOp( + static_cast(tensor), padding, offset); + } +}; + +// offset argument must be either 0 or 1. This controls whether the boundary +// values are replicated (offset == 0) or not replicated (offset == 1). +template +struct MirrorPadGrad { + void operator()(const Device& device, + typename TTypes::Tensor output, + typename TTypes::ConstTensor input, + typename TTypes::ConstMatrix paddings, int offset, + typename TTypes::Tensor scratch) { + // Copy the gradient input into the scratch buffer. + scratch.device(device) = input; + + Eigen::array lhs_offsets; + Eigen::array rhs_offsets; + Eigen::array extents; + Eigen::array reverses; + + for (int i = 0; i < Dims; ++i) { + lhs_offsets[i] = 0; + rhs_offsets[i] = 0; + extents[i] = scratch.dimension(i); + reverses[i] = false; + } + + // At this point, the central part (non-padded area) does not include the + // gradients back-propagated through padded areas. Those gradient components + // need be added to the central part. + // + // Note that a gradient input element falls into a padded area iff in at + // least one dimension i, the coordinate x(i) is in the range (python-style) + // [:paddings(i,0)] or [-paddings(i,1):]. + + for (int i = 0; i < Dims; ++i) { + reverses[i] = true; + + // This handles the case when coordinate in dimension i is in the range + // [:paddings(i,0)]. This portion is added to the range + // [paddings(i,0) + offset:2 * paddings(i,0) + offset]. + if (paddings(i, 0) > 0) { + rhs_offsets[i] = 0; + lhs_offsets[i] = paddings(i, 0) + offset; + extents[i] = paddings(i, 0); + + scratch.slice(lhs_offsets, extents).device(device) += + scratch.slice(rhs_offsets, extents).reverse(reverses); + } + + // This handles the case when coordinate in dimension i is in the range + // [-paddings(i,1):]. This portion is added to the range + // [-2 * paddings(i,1) - offset:-paddings(i,1) - offset]. + if (paddings(i, 1) > 0) { + rhs_offsets[i] = scratch.dimension(i) - paddings(i, 1); + lhs_offsets[i] = rhs_offsets[i] - paddings(i, 1) - offset; + extents[i] = paddings(i, 1); + + scratch.slice(lhs_offsets, extents).device(device) += + scratch.slice(rhs_offsets, extents).reverse(reverses); + } + + reverses[i] = false; + lhs_offsets[i] = paddings(i, 0); + rhs_offsets[i] = paddings(i, 0); + extents[i] = output.dimension(i); + + // At this point, scratch buffer contains gradient input as if paddings + // for dimension k = 0,...,i are zeros. Therefore after the loop + // termination, the central part of the scratch buffer contains the folded + // gradients. + } + + // Copy the central part of the scratch buffer to the output. + output.device(device) = scratch.slice(rhs_offsets, extents); + } +}; +} // namespace functor +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_MIRROR_PAD_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/mirror_pad_op_cpu_impl.h b/linux/include/tensorflow/core/kernels/mirror_pad_op_cpu_impl.h new file mode 100644 index 000000000..98e3be082 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/mirror_pad_op_cpu_impl.h @@ -0,0 +1,45 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_MIRROR_PAD_OP_CPU_IMPL_H_ +#define TENSORFLOW_CORE_KERNELS_MIRROR_PAD_OP_CPU_IMPL_H_ + +#define EIGEN_USE_THREADS + +#include "tensorflow/core/framework/register_types.h" +#include "tensorflow/core/kernels/mirror_pad_op.h" + +namespace tensorflow { + +using CpuDevice = Eigen::ThreadPoolDevice; + +#define DEFINE_CPU_SPECS(T) \ + template struct functor::MirrorPad; \ + template struct functor::MirrorPad; +TF_CALL_POD_TYPES(DEFINE_CPU_SPECS); +TF_CALL_string(DEFINE_CPU_SPECS); +#undef DEFINE_CPU_SPECS + +#define DEFINE_CPU_SPECS(T) \ + template struct functor::MirrorPadGrad; \ + template struct functor::MirrorPadGrad; +TF_CALL_NUMBER_TYPES(DEFINE_CPU_SPECS); +#undef DEFINE_CPU_SPECS + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_MIRROR_PAD_OP_CPU_IMPL_H_ diff --git a/linux/include/tensorflow/core/kernels/multinomial_op.h b/linux/include/tensorflow/core/kernels/multinomial_op.h new file mode 100644 index 000000000..34e212361 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/multinomial_op.h @@ -0,0 +1,30 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_MULTINOMIAL_OP_H_ +#define TENSORFLOW_CORE_KERNELS_MULTINOMIAL_OP_H_ + +namespace tensorflow { + +namespace functor { + +// Generic helper functor for the Multinomial Op. +template +struct MultinomialFunctor; + +} // namespace functor +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_MULTINOMIAL_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/neon/depthwiseconv_float.h b/linux/include/tensorflow/core/kernels/neon/depthwiseconv_float.h new file mode 100644 index 000000000..0d5a42bf1 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/neon/depthwiseconv_float.h @@ -0,0 +1,725 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CORE_KERNELS_NEON_DEPTHWISECONV_FLOAT_H_ +#define TENSORFLOW_CORE_KERNELS_NEON_DEPTHWISECONV_FLOAT_H_ + +#include "public/gemmlowp.h" +#include "tensorflow/core/kernels/neon/types.h" + +#if defined(__ARM_NEON__) || defined(__ARM_NEON) +#define USE_NEON +#include +#endif + +namespace tensorflow { +namespace neon { + +// Implementation of float DepthwiseConv + +template +struct FloatDepthwiseConvKernel {}; + +#ifdef USE_NEON + +template <> +struct FloatDepthwiseConvKernel { + static void Run(int num_output_pixels, int input_depth, int depth_multiplier, + const float* input_ptr, int input_ptr_increment, + const float* filter_ptr, float* acc_buffer_ptr) { + // Load the filters + float32x4_t filter[2]; + for (int i = 0; i < 2; i++) { + filter[i] = vld1q_f32(filter_ptr + 4 * i); + } + int outp = 0; + // Handle 2 output pixels at a time. + for (; outp <= num_output_pixels - 2; outp += 2) { + // Load the inputs + float32x4_t input[4]; + for (int i = 0; i < 4; i++) { + input[i] = vld1q_f32(input_ptr + 4 * i); + } + input_ptr += 16; + // Load the accumulators from acc_buffer + float32x4_t acc[4]; + for (int i = 0; i < 4; i++) { + acc[i] = vld1q_f32(acc_buffer_ptr + 4 * i); + } + // Multiply-accumulate + acc[0] = vmlaq_f32(acc[0], input[0], filter[0]); + acc[1] = vmlaq_f32(acc[1], input[1], filter[1]); + acc[2] = vmlaq_f32(acc[2], input[2], filter[0]); + acc[3] = vmlaq_f32(acc[3], input[3], filter[1]); + // Store the accumulators back to acc_buffer + for (int i = 0; i < 4; i++) { + vst1q_f32(acc_buffer_ptr + 4 * i, acc[i]); + } + acc_buffer_ptr += 16; + } + // Handle one output pixel at a time. + for (; outp < num_output_pixels; outp++) { + // Load the inputs + float32x4_t input[2]; + for (int i = 0; i < 2; i++) { + input[i] = vld1q_f32(input_ptr + 4 * i); + } + input_ptr += 8; + // Load the accumulators from acc_buffer + float32x4_t acc[2]; + for (int i = 0; i < 2; i++) { + acc[i] = vld1q_f32(acc_buffer_ptr + 4 * i); + } + // Multiply-accumulate + for (int i = 0; i < 2; i++) { + acc[i] = vmlaq_f32(acc[i], input[i], filter[i]); + } + // Store the accumulators back to acc_buffer + for (int i = 0; i < 2; i++) { + vst1q_f32(acc_buffer_ptr + 4 * i, acc[i]); + } + acc_buffer_ptr += 8; + } + } +}; + +template <> +struct FloatDepthwiseConvKernel { + static void Run(int num_output_pixels, int input_depth, int depth_multiplier, + const float* input_ptr, int input_ptr_increment, + const float* filter_ptr, float* acc_buffer_ptr) { + const float32x2_t filters = vld1_f32(filter_ptr); + const float32x4_t filters_dup2 = vcombine_f32(filters, filters); + int outp = 0; + // Handle 8 output pixels at a time. + for (; outp <= num_output_pixels - 8; outp += 8) { + // Load the inputs + float32x4_t input[4]; + for (int i = 0; i < 4; i++) { + input[i] = vld1q_f32(input_ptr + 4 * i); + } + input_ptr += 16; + // Load the accumulators from acc_buffer + float32x4_t acc[4]; + for (int i = 0; i < 4; i++) { + acc[i] = vld1q_f32(acc_buffer_ptr + 4 * i); + } + // Multiply-accumulate + for (int i = 0; i < 4; i++) { + acc[i] = vmlaq_f32(acc[i], input[i], filters_dup2); + } + // Store the accumulators back to acc_buffer + for (int i = 0; i < 4; i++) { + vst1q_f32(acc_buffer_ptr + 4 * i, acc[i]); + } + acc_buffer_ptr += 16; + } + // Handle 4 output pixels at a time. + for (; outp <= num_output_pixels - 4; outp += 4) { + // Load the inputs + float32x4_t input[2]; + for (int i = 0; i < 2; i++) { + input[i] = vld1q_f32(input_ptr + 4 * i); + } + input_ptr += 8; + // Load the accumulators from acc_buffer + float32x4_t acc[2]; + for (int i = 0; i < 2; i++) { + acc[i] = vld1q_f32(acc_buffer_ptr + 4 * i); + } + // Multiply-accumulate + for (int i = 0; i < 2; i++) { + acc[i] = vmlaq_f32(acc[i], input[i], filters_dup2); + } + // Store the accumulators back to acc_buffer + for (int i = 0; i < 2; i++) { + vst1q_f32(acc_buffer_ptr + 4 * i, acc[i]); + } + acc_buffer_ptr += 8; + } + // Handle 2 output pixels at a time. + for (; outp <= num_output_pixels - 2; outp += 2) { + // Load the inputs + const float32x4_t input = vld1q_f32(input_ptr); + input_ptr += 4; + // Load the accumulators from acc_buffer + float32x4_t acc = vld1q_f32(acc_buffer_ptr); + // Multiply-accumulate + acc = vmlaq_f32(acc, input, filters_dup2); + // Store the accumulators back to acc_buffer + vst1q_f32(acc_buffer_ptr, acc); + acc_buffer_ptr += 4; + } + // Handle 1 output pixel at a time + for (; outp < num_output_pixels; outp++) { + // Load the inputs + const float32x2_t input = vld1_f32(input_ptr); + input_ptr += 2; + // Load the accumulators from acc_buffer + float32x2_t acc = vld1_f32(acc_buffer_ptr); + // Multiply-accumulate + acc = vmla_f32(acc, input, filters); + // Store the accumulators back to acc_buffer + vst1_f32(acc_buffer_ptr, acc); + acc_buffer_ptr += 2; + } + } +}; + +template <> +struct FloatDepthwiseConvKernel { + static void Run(int num_output_pixels, int input_depth, int depth_multiplier, + const float* input_ptr, int input_ptr_increment, + const float* filter_ptr, float* acc_buffer_ptr) { + // Handle one output pixel at a time. + for (int outp = 0; outp < num_output_pixels; outp++) { + const float* local_filter_ptr = filter_ptr; + const float* local_input_ptr = input_ptr; + int ic = 0; + // Handle 16 input channels at a time. + for (; ic <= input_depth - 16; ic += 16) { + // Load the filters + float32x4_t filter[4]; + for (int i = 0; i < 4; i++) { + filter[i] = vld1q_f32(local_filter_ptr + 4 * i); + } + local_filter_ptr += 16; + // Load the inputs + float32x4_t input[4]; + for (int i = 0; i < 4; i++) { + input[i] = vld1q_f32(local_input_ptr + 4 * i); + } + local_input_ptr += 16; + // Load the accumulators from acc_buffer + float32x4_t acc[4]; + for (int i = 0; i < 4; i++) { + acc[i] = vld1q_f32(acc_buffer_ptr + 4 * i); + } + // Multiply-accumulate + for (int i = 0; i < 4; i++) { + acc[i] = vmlaq_f32(acc[i], input[i], filter[i]); + } + // Store the accumulators back to acc_buffer + for (int i = 0; i < 4; i++) { + vst1q_f32(acc_buffer_ptr + 4 * i, acc[i]); + } + acc_buffer_ptr += 16; + } + // Handle 4 input channels at a time. + for (; ic <= input_depth - 4; ic += 4) { + // Load the filters + float32x4_t filter; + filter = vld1q_f32(local_filter_ptr); + local_filter_ptr += 4; + // Load the inputs + float32x4_t input; + input = vld1q_f32(local_input_ptr); + local_input_ptr += 4; + // Load the accumulators from acc_buffer + float32x4_t acc; + acc = vld1q_f32(acc_buffer_ptr); + // Multiply-accumulate + acc = vmlaq_f32(acc, input, filter); + // Store the accumulators back to acc_buffer + vst1q_f32(acc_buffer_ptr, acc); + acc_buffer_ptr += 4; + } + // Handle one input channel at a time. + for (; ic < input_depth; ic++) { + const float input_val = *local_input_ptr++; + const float filter_val = *local_filter_ptr++; + *acc_buffer_ptr++ += filter_val * input_val; + } + input_ptr += input_ptr_increment; + } + } +}; + +template <> +struct FloatDepthwiseConvKernel { + static void Run(int num_output_pixels, int input_depth, int depth_multiplier, + const float* input_ptr, int input_ptr_increment, + const float* filter_ptr, float* acc_buffer_ptr) { + // Handle one output pixel at a time. + for (int outp = 0; outp < num_output_pixels; outp++) { + const float* local_filter_ptr = filter_ptr; + const float* local_input_ptr = input_ptr; + int ic = 0; + // Handle 2 input channels at a time. + for (; ic <= input_depth - 2; ic += 2) { + // Load the filters + float32x4_t filter[4]; + for (int i = 0; i < 4; i++) { + filter[i] = vld1q_f32(local_filter_ptr + 4 * i); + } + local_filter_ptr += 16; + // Load the inputs + const float32x2_t input = vld1_f32(local_input_ptr); + local_input_ptr += 2; + // Load the accumulators from acc_buffer + float32x4_t acc[4]; + for (int i = 0; i < 4; i++) { + acc[i] = vld1q_f32(acc_buffer_ptr + 4 * i); + } + // Multiply-accumulate + acc[0] = vmlaq_lane_f32(acc[0], filter[0], input, 0); + acc[1] = vmlaq_lane_f32(acc[1], filter[1], input, 0); + acc[2] = vmlaq_lane_f32(acc[2], filter[2], input, 1); + acc[3] = vmlaq_lane_f32(acc[3], filter[3], input, 1); + // Store the accumulators back to acc_buffer + for (int i = 0; i < 4; i++) { + vst1q_f32(acc_buffer_ptr + 4 * i, acc[i]); + } + acc_buffer_ptr += 16; + } + // Handle one input channel at a time. + for (; ic < input_depth; ic++) { + // Load the filters + float32x4_t filter[2]; + for (int i = 0; i < 2; i++) { + filter[i] = vld1q_f32(local_filter_ptr + 4 * i); + } + local_filter_ptr += 8; + // Load the inputs + const float input_val = *local_input_ptr++; + // Load the accumulators from acc_buffer + float32x4_t acc[2]; + for (int i = 0; i < 2; i++) { + acc[i] = vld1q_f32(acc_buffer_ptr + 4 * i); + } + // Multiply-accumulate + for (int i = 0; i < 2; i++) { + acc[i] = vmlaq_n_f32(acc[i], filter[i], input_val); + } + // Store the accumulators back to acc_buffer + for (int i = 0; i < 2; i++) { + vst1q_f32(acc_buffer_ptr + 4 * i, acc[i]); + } + acc_buffer_ptr += 8; + } + input_ptr += input_ptr_increment; + } + } +}; + +template <> +struct FloatDepthwiseConvKernel { + static void Run(int num_output_pixels, int input_depth, int depth_multiplier, + const float* input_ptr, int input_ptr_increment, + const float* filter_ptr, float* acc_buffer_ptr) { + // Handle one output pixel at a time. + for (int outp = 0; outp < num_output_pixels; outp++) { + const float* local_filter_ptr = filter_ptr; + const float* local_input_ptr = input_ptr; + int ic = 0; + // Handle 8 input channels at a time. + for (; ic <= input_depth - 8; ic += 8) { + // Load the filters + float32x4_t filter[4]; + for (int i = 0; i < 4; i++) { + filter[i] = vld1q_f32(local_filter_ptr + 4 * i); + } + local_filter_ptr += 16; + // Load the inputs + float32x4x2_t input_dup2[2]; + for (int i = 0; i < 2; i++) { + const float32x4_t input = vld1q_f32(local_input_ptr + 4 * i); + input_dup2[i] = vzipq_f32(input, input); + } + local_input_ptr += 8; + // Load the accumulators from acc_buffer + float32x4_t acc[4]; + for (int i = 0; i < 4; i++) { + acc[i] = vld1q_f32(acc_buffer_ptr + 4 * i); + } + // Multiply-accumulate + acc[0] = vmlaq_f32(acc[0], filter[0], input_dup2[0].val[0]); + acc[1] = vmlaq_f32(acc[1], filter[1], input_dup2[0].val[1]); + acc[2] = vmlaq_f32(acc[2], filter[2], input_dup2[1].val[0]); + acc[3] = vmlaq_f32(acc[3], filter[3], input_dup2[1].val[1]); + // Store the accumulators back to acc_buffer + for (int i = 0; i < 4; i++) { + vst1q_f32(acc_buffer_ptr + 4 * i, acc[i]); + } + acc_buffer_ptr += 16; + } + // Handle 4 input channels at a time. + for (; ic <= input_depth - 4; ic += 4) { + // Load the filters + float32x2_t filter[4]; + for (int i = 0; i < 4; i++) { + filter[i] = vld1_f32(local_filter_ptr + 2 * i); + } + local_filter_ptr += 8; + // Load the inputs + const float32x4_t input = vld1q_f32(local_input_ptr); + local_input_ptr += 4; + // Load the accumulators from acc_buffer + float32x2_t acc[4]; + for (int i = 0; i < 4; i++) { + acc[i] = vld1_f32(acc_buffer_ptr + 2 * i); + } + // Multiply-accumulate + acc[0] = vmla_lane_f32(acc[0], filter[0], vget_low_f32(input), 0); + acc[1] = vmla_lane_f32(acc[1], filter[1], vget_low_f32(input), 1); + acc[2] = vmla_lane_f32(acc[2], filter[2], vget_high_f32(input), 0); + acc[3] = vmla_lane_f32(acc[3], filter[3], vget_high_f32(input), 1); + // Store the accumulators back to acc_buffer + for (int i = 0; i < 4; i++) { + vst1_f32(acc_buffer_ptr + 2 * i, acc[i]); + } + acc_buffer_ptr += 8; + } + // Handle 2 input channels at a time. + for (; ic <= input_depth - 2; ic += 2) { + // Load the filters + const float32x4_t filter = vld1q_f32(local_filter_ptr); + local_filter_ptr += 4; + // Load the inputs + const float32x2_t input = vld1_f32(local_input_ptr); + local_input_ptr += 2; + // Load the accumulators from acc_buffer + float32x2_t acc[2]; + for (int i = 0; i < 2; i++) { + acc[i] = vld1_f32(acc_buffer_ptr + 2 * i); + } + // Multiply-accumulate + acc[0] = vmla_lane_f32(acc[0], vget_low_f32(filter), input, 0); + acc[1] = vmla_lane_f32(acc[1], vget_high_f32(filter), input, 1); + // Store the accumulators back to acc_buffer + for (int i = 0; i < 2; i++) { + vst1_f32(acc_buffer_ptr + 2 * i, acc[i]); + } + acc_buffer_ptr += 4; + } + // Handle one input channel at a time. + for (; ic < input_depth; ic++) { + // Load the inputs + const float input_val = *local_input_ptr++; + // Multiply-accumulate + for (int i = 0; i < 2; i++) { + acc_buffer_ptr[i] += local_filter_ptr[i] * input_val; + } + local_filter_ptr += 2; + acc_buffer_ptr += 2; + } + input_ptr += input_ptr_increment; + } + } +}; +#endif + +// Accumulates the effect of one row of the filter, on a segment of one row +// of the output, accessing the corresponding one row of the input. +template +void FloatDepthwiseConvAccumRow(int stride, int input_depth, int input_width, + const float* input_data, int pad_width, + int depth_multiplier, int filter_width, + const float* filter_data, + int out_x_buffer_start, int out_x_buffer_end, + int output_depth, float* acc_buffer) { +#ifdef GEMMLOWP_PROFILING + gemmlowp::ScopedProfilingLabel label(__PRETTY_FUNCTION__); +#endif + // Sanity check parameters. This is important in particular to ensure + // that we keep the number of template instantiations minimal, so we don't + // increase binary size unnecessarily. + static_assert(kFixedDepthMultiplier || !kFixedInputDepth, ""); + static_assert(kFixedInputDepth || kAllowStrided, ""); + DCHECK(stride == 1 || kAllowStrided); + if (kFixedInputDepth) { + DCHECK_EQ(input_depth, kFixedInputDepth); + } + if (kFixedDepthMultiplier) { + DCHECK_EQ(depth_multiplier, kFixedDepthMultiplier); + } + DCHECK_EQ(output_depth, input_depth * depth_multiplier); + const int input_ptr_increment = stride * input_depth; + const float* filter_base_ptr = filter_data; + for (int filter_x = 0; filter_x < filter_width; ++filter_x) { + // For the current (filter_x, filter_y) point in the filter, + // compute the boundaries of the corresponding output row segment. + int out_x_loop_start_unclampled = 0; + int out_x_loop_end_unclampled = 0; + if (kAllowStrided) { + if (stride == 2) { + out_x_loop_start_unclampled = (pad_width - filter_x + 1) / 2; + out_x_loop_end_unclampled = + (pad_width + input_width - filter_x + 1) / 2; + } else if (stride == 4) { + out_x_loop_start_unclampled = (pad_width - filter_x + 3) / 4; + out_x_loop_end_unclampled = + (pad_width + input_width - filter_x + 3) / 4; + } else { + out_x_loop_start_unclampled = + (pad_width - filter_x + stride - 1) / stride; + out_x_loop_end_unclampled = + (pad_width + input_width - filter_x + stride - 1) / stride; + } + } else { + out_x_loop_start_unclampled = pad_width - filter_x; + out_x_loop_end_unclampled = pad_width + input_width - filter_x; + } + // The kernel will have to iterate on the segment of the + // output row that starts at out_x_loop_start and out_x_loop_end. + const int out_x_loop_start = + std::max(out_x_buffer_start, out_x_loop_start_unclampled); + const int out_x_loop_end = + std::min(out_x_buffer_end, out_x_loop_end_unclampled); + + float* acc_buffer_ptr = + acc_buffer + (out_x_loop_start - out_x_buffer_start) * output_depth; + const int in_x_origin = (out_x_loop_start * stride) - pad_width + filter_x; + const float* input_ptr = input_data + in_x_origin * input_depth; + const int num_output_pixels = out_x_loop_end - out_x_loop_start; + FloatDepthwiseConvKernel::Run(num_output_pixels, + input_depth, + depth_multiplier, + input_ptr, + input_ptr_increment, + filter_base_ptr, + acc_buffer_ptr); + filter_base_ptr += output_depth; + } +} + +// generic fallback of FloatDepthwiseConvAccumRow, portable, non-templatized. +inline void FloatDepthwiseConvAccumRowGeneric( + int stride, int input_depth, int input_width, const float* input_data, + int pad_width, int depth_multiplier, int filter_width, + const float* filter_data, int out_x_buffer_start, int out_x_buffer_end, + int output_depth, float* acc_buffer) { + gemmlowp::ScopedProfilingLabel label("DepthwiseConvAccumRowGeneric (slow)"); + + VLOG(1) << "DepthwiseConv2d using slow path with " + << "stride = " << stride << ", " + << "input_depth = " << input_depth << ", " + << "depth_multiplier = " << depth_multiplier << "."; + + const float* filter_base_ptr = filter_data; + for (int filter_x = 0; filter_x < filter_width; ++filter_x) { + const int out_x_loop_start = std::max( + out_x_buffer_start, (pad_width - filter_x + stride - 1) / stride); + const int out_x_loop_end = + std::min(out_x_buffer_end, + (pad_width + input_width - filter_x + stride - 1) / stride); + + float* acc_buffer_ptr = + acc_buffer + (out_x_loop_start - out_x_buffer_start) * output_depth; + const int in_x_origin = (out_x_loop_start * stride) - pad_width + filter_x; + const float* input_ptr = input_data + in_x_origin * input_depth; + const int input_ptr_increment = (stride - 1) * input_depth; + for (int out_x = out_x_loop_start; out_x < out_x_loop_end; out_x++) { + const float* filter_ptr = filter_base_ptr; + for (int ic = 0; ic < input_depth; ++ic) { + const float input_val = *input_ptr++; + for (int m = 0; m < depth_multiplier; m++) { + const float filter_val = *filter_ptr++; + *acc_buffer_ptr++ += filter_val * input_val; + } + } + input_ptr += input_ptr_increment; + } + filter_base_ptr += output_depth; + } +} + +// Initializes the accumulator buffer with bias values. +inline void DepthwiseConvInitAccBuffer(int num_output_pixels, int output_depth, + const float* bias_data, + float* acc_buffer) { + // TODO(benoitjacob): This might need optimized specializations + // for small output_depth values, if that ever becomes an important + // case (like it was for some quantized DepthwiseConv cases). + for (int i = 0; i < num_output_pixels; i++) { + memcpy(acc_buffer + i * output_depth, bias_data, + sizeof(acc_buffer[0]) * output_depth); + } +} + +template +void DepthwiseConv(const float* input_data, const Dims<4>& input_dims, + const float* filter_data, const Dims<4>& filter_dims, + const float* bias_data, const Dims<4>& bias_dims, int stride, + int pad_width, int pad_height, int depth_multiplier, + float* output_data, const Dims<4>& output_dims) { + gemmlowp::ScopedProfilingLabel label("DepthwiseConv"); + static_assert(Ac == FusedActivationFunctionType::kNone || + Ac == FusedActivationFunctionType::kRelu || + Ac == FusedActivationFunctionType::kRelu6 || + Ac == FusedActivationFunctionType::kRelu1, + ""); + const int batches = MatchingArraySize(input_dims, 3, output_dims, 3); + const int output_depth = MatchingArraySize(filter_dims, 0, output_dims, 0); + const int input_height = ArraySize(input_dims, 2); + const int input_width = ArraySize(input_dims, 1); + const int input_depth = ArraySize(input_dims, 0); + const int filter_height = ArraySize(filter_dims, 2); + const int filter_width = ArraySize(filter_dims, 1); + const int output_height = ArraySize(output_dims, 2); + const int output_width = ArraySize(output_dims, 1); + DCHECK(output_depth == input_depth * depth_multiplier); + + static const int kAccBufferMaxSize = 1024; + float acc_buffer[kAccBufferMaxSize]; + DCHECK_GE(kAccBufferMaxSize, output_depth) + << "Too small kAccBufferMaxSize for this model!"; + const int kOutputPixelsInAccBuffer = kAccBufferMaxSize / output_depth; + const int kAccBufferActualSize = kOutputPixelsInAccBuffer * output_depth; + DCHECK_LE(kOutputPixelsInAccBuffer * output_depth, kAccBufferActualSize); + DCHECK_LE(kAccBufferActualSize, kAccBufferMaxSize); + DCHECK_GE(kOutputPixelsInAccBuffer, 1); + + // row_accum_func will point to the core accumulation function to be used + // for this DepthwiseConv op. + auto* row_accum_func = FloatDepthwiseConvAccumRowGeneric; + + const int kMaxFixedDepthMultiplier = 8; + int fixed_depth_multiplier = 0; + if (depth_multiplier <= kMaxFixedDepthMultiplier) { + fixed_depth_multiplier = depth_multiplier; + } + // kMaxUnrolling is the max number of output values that we aim to handle + // in one unrolled iteration of the inner loop. For practical performance + // reasons, it is limited by the number of available registers. We could + // fine-tune it depending on the architecture, but that's not worth doing + // since this whole code is not very optimized to begin with. The + // present value reflects what's realistic on ARM 32bit NEON with 16 128-bit + // vector registers. + const int kMaxUnrolling = 8; + int fixed_input_depth = 0; + if (fixed_depth_multiplier && + input_depth * fixed_depth_multiplier <= kMaxUnrolling) { + fixed_input_depth = input_depth; + } +#define TF_NEON_USE_DEPTHWISECONV_KERNEL(ALLOW_STRIDED, FIXED_INPUT_DEPTH, \ + FIXED_DEPTH_MULTIPLIER) \ + if ((stride == 1 || ALLOW_STRIDED) && \ + fixed_input_depth == FIXED_INPUT_DEPTH && \ + fixed_depth_multiplier == FIXED_DEPTH_MULTIPLIER) { \ + row_accum_func = \ + FloatDepthwiseConvAccumRow; \ + } + +#ifdef USE_NEON + TF_NEON_USE_DEPTHWISECONV_KERNEL(true, 0, 1) + TF_NEON_USE_DEPTHWISECONV_KERNEL(true, 0, 8) + TF_NEON_USE_DEPTHWISECONV_KERNEL(true, 0, 2) + TF_NEON_USE_DEPTHWISECONV_KERNEL(false, 8, 1) + TF_NEON_USE_DEPTHWISECONV_KERNEL(false, 2, 1) +#endif // USE_NEON + +#undef TF_NEON_USE_DEPTHWISECONV_KERNEL + + // Now that we have determined row_accum_func, we can start work. + float* output_ptr = output_data; + for (int b = 0; b < batches; ++b) { + for (int out_y = 0; out_y < output_height; ++out_y) { + const int in_y_origin = (out_y * stride) - pad_height; + const int filter_y_start = std::max(0, -in_y_origin); + const int filter_y_end = + std::min(filter_height, input_height - in_y_origin); + for (int out_x_buffer_start = 0; out_x_buffer_start < output_width; + out_x_buffer_start += kOutputPixelsInAccBuffer) { + const int out_x_buffer_end = std::min( + output_width, out_x_buffer_start + kOutputPixelsInAccBuffer); + // We call a 'pixel' a group of activation that share all but the + // 'depth'/'channel' coordinate. num_output_pixels is the number of + // output pixels that we will accumulate in this loop iteration. + const int num_output_pixels = out_x_buffer_end - out_x_buffer_start; + // Initialize our local accumulator with the bias values, so we don't + // have to add them later. + DepthwiseConvInitAccBuffer(num_output_pixels, output_depth, bias_data, + acc_buffer); + // Accumulation loop. Most of the time should be spent in here. + for (int filter_y = filter_y_start; filter_y < filter_y_end; + ++filter_y) { + const int in_y = in_y_origin + filter_y; + row_accum_func(stride, input_depth, input_width, + input_data + in_y * input_dims.strides[2] + + b * input_dims.strides[3], + pad_width, depth_multiplier, filter_width, + filter_data + filter_y * filter_dims.strides[2], + out_x_buffer_start, out_x_buffer_end, output_depth, + acc_buffer); + } + // Finished accumulating. Now store to destination. + const int num_output_values = output_depth * num_output_pixels; + int i = 0; +// TODO(benoitjacob) optimized code goes here +#ifdef USE_NEON + // Handle 16 values at a time + for (; i <= num_output_values - 16; i += 16) { + float32x4_t acc[4]; + for (int k = 0; k < 4; k++) { + acc[k] = vld1q_f32(acc_buffer + i + 4 * k); + } + if (Ac == FusedActivationFunctionType::kRelu) { + for (int k = 0; k < 4; k++) { + acc[k] = vmaxq_f32(vdupq_n_f32(0.f), acc[k]); + } + } else if (Ac == FusedActivationFunctionType::kRelu6) { + for (int k = 0; k < 4; k++) { + acc[k] = vmaxq_f32(vdupq_n_f32(0.f), + vminq_f32(vdupq_n_f32(6.f), acc[k])); + } + } else if (Ac == FusedActivationFunctionType::kRelu1) { + for (int k = 0; k < 4; k++) { + acc[k] = vmaxq_f32(vdupq_n_f32(-1.f), + vminq_f32(vdupq_n_f32(1.f), acc[k])); + } + } + for (int k = 0; k < 4; k++) { + vst1q_f32(output_ptr + 4 * k, acc[k]); + } + output_ptr += 16; + } + // Handle 4 values at a time + for (; i <= num_output_values - 4; i += 4) { + float32x4_t acc = vld1q_f32(acc_buffer + i); + if (Ac == FusedActivationFunctionType::kRelu) { + acc = vmaxq_f32(vdupq_n_f32(0.f), acc); + } else if (Ac == FusedActivationFunctionType::kRelu6) { + acc = vmaxq_f32(vdupq_n_f32(0.f), vminq_f32(vdupq_n_f32(6.f), acc)); + } else if (Ac == FusedActivationFunctionType::kRelu1) { + acc = + vmaxq_f32(vdupq_n_f32(-1.f), vminq_f32(vdupq_n_f32(1.f), acc)); + } + vst1q_f32(output_ptr, acc); + output_ptr += 4; + } +#endif + // Handle leftover values, one by one. This is very slow. + for (; i < num_output_values; i++) { + float acc = acc_buffer[i]; + if (Ac == FusedActivationFunctionType::kRelu) { + acc = std::max(0.f, acc); + } else if (Ac == FusedActivationFunctionType::kRelu6) { + acc = std::max(0.f, std::min(6.f, acc)); + } else if (Ac == FusedActivationFunctionType::kRelu1) { + acc = std::max(-1.f, std::min(1.f, acc)); + } + *output_ptr++ = acc; + } + } + } + } +} + +} // end namespace neon +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_NEON_DEPTHWISECONV_FLOAT_H_ diff --git a/linux/include/tensorflow/core/kernels/neon/types.h b/linux/include/tensorflow/core/kernels/neon/types.h new file mode 100644 index 000000000..05ff1bcc6 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/neon/types.h @@ -0,0 +1,73 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CORE_KERNELS_NEON_TYPES_H_ +#define TENSORFLOW_CORE_KERNELS_NEON_TYPES_H_ + +#include "tensorflow/core/platform/logging.h" + +namespace tensorflow { +namespace neon { + +enum class FusedActivationFunctionType { kNone, kRelu6, kRelu1, kRelu }; + +template +struct Dims { + int sizes[N]; + int strides[N]; +}; + +inline int Offset(const Dims<4>& dims, int i0, int i1, int i2, int i3) { + DCHECK(i0 >= 0 && i0 < dims.sizes[0]); + DCHECK(i1 >= 0 && i1 < dims.sizes[1]); + DCHECK(i2 >= 0 && i2 < dims.sizes[2]); + DCHECK(i3 >= 0 && i3 < dims.sizes[3]); + return i0 * dims.strides[0] + i1 * dims.strides[1] + i2 * dims.strides[2] + + i3 * dims.strides[3]; +} + +// Get array size, DCHECKing that the dim index is in range. +template +int ArraySize(const Dims& array, int index) { + DCHECK(index >= 0 && index < N); + return array.sizes[index]; +} + +// Get common array size, DCHECKing that they all agree. +template +int MatchingArraySize(const ArrayType1& array1, int index1, + const ArrayType2& array2, int index2) { + DCHECK_EQ(ArraySize(array1, index1), ArraySize(array2, index2)); + return ArraySize(array1, index1); +} + +template +int MatchingArraySize(const ArrayType1& array1, int index1, + const ArrayType2& array2, int index2, Args... args) { + DCHECK_EQ(ArraySize(array1, index1), ArraySize(array2, index2)); + return MatchingArraySize(array1, index1, args...); +} + +inline int RequiredBufferSizeForDims(const Dims<4>& dims) { + int max_offset = 0; + for (int i = 0; i < 4; i++) { + max_offset += (dims.sizes[i] - 1) * dims.strides[i]; + } + return max_offset + 1; +} + +} // end namespace neon +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_NEON_TYPES_H_ diff --git a/linux/include/tensorflow/core/kernels/no_op.h b/linux/include/tensorflow/core/kernels/no_op.h new file mode 100644 index 000000000..9e16d0697 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/no_op.h @@ -0,0 +1,32 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_NO_OP_H_ +#define TENSORFLOW_CORE_KERNELS_NO_OP_H_ + +#include "tensorflow/core/framework/op_kernel.h" + +namespace tensorflow { + +class NoOp : public OpKernel { + public: + explicit NoOp(OpKernelConstruction* context) : OpKernel(context) {} + void Compute(OpKernelContext* context) override {} + bool IsExpensive() override { return false; } +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_NO_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/non_max_suppression_op.h b/linux/include/tensorflow/core/kernels/non_max_suppression_op.h new file mode 100644 index 000000000..933b1af44 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/non_max_suppression_op.h @@ -0,0 +1,38 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_NON_MAX_SUPPRESSION_OP_H_ +#define TENSORFLOW_CORE_KERNELS_NON_MAX_SUPPRESSION_OP_H_ + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/numeric_types.h" +#include "tensorflow/core/framework/tensor_types.h" + +namespace tensorflow { +namespace functor { + +template +struct NonMaxSuppression { + void operator()(const Device& d, typename TTypes::ConstTensor boxes, + typename TTypes::ConstTensor scores, + float iou_threshold, float score_threshold, + int max_output_size, + typename TTypes::Tensor selected_indices); +}; + +} // namespace functor +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_NON_MAX_SUPPRESSION_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/nth_element_op.h b/linux/include/tensorflow/core/kernels/nth_element_op.h new file mode 100644 index 000000000..7a5ec3d0b --- /dev/null +++ b/linux/include/tensorflow/core/kernels/nth_element_op.h @@ -0,0 +1,37 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_NTH_ELEMENT_OP_H_ +#define TENSORFLOW_CORE_KERNELS_NTH_ELEMENT_OP_H_ + +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +namespace functor { + +template +struct NthElementFunctor { + void operator()(OpKernelContext* context, const Tensor& input_tensor, + Tensor& output_tensor, int n); +}; + +} // namespace functor + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_NTH_ELEMENT_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/one_hot_op.h b/linux/include/tensorflow/core/kernels/one_hot_op.h new file mode 100644 index 000000000..879df2b59 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/one_hot_op.h @@ -0,0 +1,72 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// See docs in ../ops/array_ops.cc + +#ifndef TENSORFLOW_CORE_KERNELS_ONE_HOT_OP_H_ +#define TENSORFLOW_CORE_KERNELS_ONE_HOT_OP_H_ +// Generator definition for OneHotOp, must be compilable by nvcc. + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +namespace generator { + +template +class OneGenerator { + public: + EIGEN_DEVICE_FUNC EIGEN_ALWAYS_INLINE + OneGenerator(const typename TTypes::ConstMatrix& indices, + const typename TTypes::ConstScalar& on_value, + const typename TTypes::ConstScalar& off_value) + : indices_(indices), on_value_(on_value), off_value_(off_value) {} + + EIGEN_DEVICE_FUNC EIGEN_ALWAYS_INLINE T + operator()(const Eigen::array& pre_depth_suff) const { + return (indices_(pre_depth_suff[0], pre_depth_suff[2]) == pre_depth_suff[1]) + ? on_value_() + : off_value_(); + } + + private: + const typename TTypes::ConstMatrix indices_; + const typename TTypes::ConstScalar on_value_; + const typename TTypes::ConstScalar off_value_; +}; + +} // namespace generator + +namespace functor { + +template +struct OneHot { + EIGEN_ALWAYS_INLINE static void Compute( + const Device& d, const typename TTypes::ConstMatrix& indices, + const typename TTypes::ConstScalar& on_value, + const typename TTypes::ConstScalar& off_value, + typename TTypes::Tensor* output) { + generator::OneGenerator generator(indices, on_value, off_value); + output->device(d) = output->generate(generator); + } +}; + +} // namespace functor + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_ONE_HOT_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/ops_util.h b/linux/include/tensorflow/core/kernels/ops_util.h new file mode 100644 index 000000000..a496487d1 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/ops_util.h @@ -0,0 +1,116 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_OPS_UTIL_H_ +#define TENSORFLOW_CORE_KERNELS_OPS_UTIL_H_ + +// This file contains utilities for various operations. + +#include + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/common_shape_fns.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/util/padding.h" + +namespace tensorflow { + +// Calculates broadcast starting index and size. For SAME padding, addition +// padding could be applied to right, left, top and bottom. Depending on the +// current index, input size, kernel size, stride, padding size, the starting +// index and size for broadcast for that dimension are different from the +// current index and kernel size. +// This is mainly used by gradient algorithms for pooling operations. +Status GetBroadcastSize(const int index, const int in_size, const int ksize, + const int stride, const int pad_size, int* bindex, + int* bsize); + +// Converts Brain's Padding to Eigen's PaddingType. +Eigen::PaddingType BrainPadding2EigenPadding(Padding padding); + +// Given a shape 's' of a tensor of type T. Returns true iff the +// number of bytes occupied by each dim 0 (i.e., &tensor(i + 1, ...) - +// &tensor(i, ...)) is multiple of EIGEN_MAX_ALIGN_BYTES. +template +bool IsInnerDimsSizeAligned(const TensorShape& s) { + if (s.dims() == 0) return false; + const int64 dim0_size = s.dim_size(0); + if (dim0_size == 0) return false; +#if EIGEN_MAX_ALIGN_BYTES == 0 + return true; +#else + const int64 bytes_per_dim0 = (s.num_elements() / dim0_size) * sizeof(T); + return bytes_per_dim0 % EIGEN_MAX_ALIGN_BYTES == 0; +#endif +} + +// Given a shape 's' of a tensor of type T and the `start` and `end` index of a +// dim 0 slice, returns true iff slice is aligned with respect to original +// tensor. Here aligned implies the address is a multiple of +// EIGEN_MAX_ALIGN_BYTES. +template +bool IsDim0SliceAligned(const TensorShape& s, int64 start, int64 end_or_size) { + if (s.dims() == 1) { +#if EIGEN_MAX_ALIGN_BYTES == 0 + return true; +#else + bool start_aligned = (start * sizeof(T)) % EIGEN_MAX_ALIGN_BYTES == 0; + // End is aligned if either the explicit end index is passed and is a + // a multiple of EIGEN_MAX_ALIGN_BYTES, or the start index is aligned and + // the size is aligned. So for convenience we can either pass start and + // index, or start and size. + bool end_aligned = (end_or_size * sizeof(T)) % EIGEN_MAX_ALIGN_BYTES == 0; + return start_aligned && end_aligned; +#endif + } else { + return IsInnerDimsSizeAligned(s); + } +} + +// Returns sanitized to have only [a-zA-Z0-9-_]. +string SanitizeThreadSuffix(string suffix); + +// Helper to compute 'strides' given a tensor 'shape'. I.e., +// strides[i] = prod(shape.dim_size[(i+1):]) +template +gtl::InlinedVector ComputeStride(const TensorShape& shape) { + const int ndims = shape.dims(); + gtl::InlinedVector strides(ndims); + T stride = 1; + for (int i = ndims - 1; i >= 0; --i) { + strides[i] = stride; + stride *= static_cast(shape.dim_size(i)); + } + return strides; +} + +// Helper to compute 'strides' given an Eigen TensorDimensions +template +gtl::InlinedVector ComputeEigenStrides(const EigenDimensions& shape) { + const int ndims = shape.rank(); + gtl::InlinedVector strides(ndims); + T stride = 1; + for (int i = ndims - 1; i >= 0; --i) { + strides[i] = stride; + stride *= static_cast(shape[i]); + } + return strides; +} + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_OPS_UTIL_H_ diff --git a/linux/include/tensorflow/core/kernels/pad_op.h b/linux/include/tensorflow/core/kernels/pad_op.h new file mode 100644 index 000000000..ae79f515d --- /dev/null +++ b/linux/include/tensorflow/core/kernels/pad_op.h @@ -0,0 +1,57 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_PAD_OP_H_ +#define TENSORFLOW_CORE_KERNELS_PAD_OP_H_ +// Functor definition for PadOp, must be compilable by nvcc. + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { +namespace functor { + +// Functor used by PadOp to do the computations. +template +struct Pad { + // Pad "input" into "output", as specified by "paddings" and "pad_value". + // See pad_op.cc for details. + void operator()(const Device& d, typename TTypes::Tensor output, + typename TTypes::ConstTensor input, + Eigen::array, Dims> paddings, + T pad_value) { + if (Eigen::internal::is_same::value && + (output.size() <= std::numeric_limits::max())) { + To32Bit(output).device(d) = To32Bit(input).pad(paddings, pad_value); + } else { + output.device(d) = input.pad(paddings, pad_value); + } + } +}; + +template +struct Pad { + // In the scalar case we simply copy the input. + void operator()(const Device& d, typename TTypes::Tensor output, + typename TTypes::ConstTensor input, + Eigen::array, 0>, T) { + output.device(d) = input; + } +}; +} // namespace functor +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_PAD_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/padding_fifo_queue.h b/linux/include/tensorflow/core/kernels/padding_fifo_queue.h new file mode 100644 index 000000000..b86b03c8f --- /dev/null +++ b/linux/include/tensorflow/core/kernels/padding_fifo_queue.h @@ -0,0 +1,89 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_PADDING_FIFO_QUEUE_H_ +#define TENSORFLOW_CORE_KERNELS_PADDING_FIFO_QUEUE_H_ + +#include +#include + +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/partial_tensor_shape.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/kernels/fifo_queue.h" +#include "tensorflow/core/kernels/typed_queue.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/mutex.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +class PaddingFIFOQueue : public FIFOQueue { + public: + PaddingFIFOQueue(int32 capacity, const DataTypeVector& component_dtypes, + const std::vector& component_shapes, + const string& name); + + Status Initialize() override; + + // Implementations of QueueInterface methods -------------------------------- + + void TryDequeueMany(int num_elements, OpKernelContext* ctx, + bool allow_small_batch, + CallbackWithTuple callback) override; + Status MatchesNodeDef(const NodeDef& node_def) override; + + protected: + Status ValidateManyTuple(const Tuple& tuple) override; + Status ValidateTuple(const Tuple& tuple) override; + Status CompatibleNodeDefShapes(const NodeDef& node_def) const; + + // Convert a list of PartialTensorShape to a list of + // TensorShape. + // Any unknown dimension sizes are converted to 0. + // REQUIRED: All the input shapes have well defined rank. + static std::vector ConvertShapesPartialDimensionsToZero( + const gtl::ArraySlice& partial_shapes); + + // Sets the values in the given element to zero. + static Status SetElementZero(Tensor* element); + + // Copies element into the index^th slice (in the first dimension) + // of parent. Allows for the parent's slice to have a larger size + // than the element, and copies the element into the upper left hand + // corner of the slice. + static Status CopyElementToLargerSlice(const Tensor& element, Tensor* parent, + int index); + + std::vector partial_shapes_; + + private: + ~PaddingFIFOQueue() override {} + + static Status GetElementComponent(const PaddingFIFOQueue::Tuple& tuple, + int component, OpKernelContext* ctx, + PersistentTensor* out_tensor); + + static Status IsSameSizeExceptZerosInFirst(const TensorShape& first, + const TensorShape& second); + + TF_DISALLOW_COPY_AND_ASSIGN(PaddingFIFOQueue); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_PADDING_FIFO_QUEUE_H_ diff --git a/linux/include/tensorflow/core/kernels/parameterized_truncated_normal_op.h b/linux/include/tensorflow/core/kernels/parameterized_truncated_normal_op.h new file mode 100644 index 000000000..2e54db31f --- /dev/null +++ b/linux/include/tensorflow/core/kernels/parameterized_truncated_normal_op.h @@ -0,0 +1,52 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_PARAMETERIZED_TRUNCATED_NORMAL_OP_H_ +#define TENSORFLOW_CORE_KERNELS_PARAMETERIZED_TRUNCATED_NORMAL_OP_H_ + +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/lib/random/random_distributions.h" + +namespace tensorflow { + +class OpKernelContext; + +namespace functor { + +// Sample a truncated normal random variable, with mean, stddev, minval, and +// maxval parameters for each batch. Uses two rejection sampling algorithms +// described in http://rd.springer.com/article/10.1007/BF00143942. +// +// Either minval may be -infinity, or maxval may be +infinity. If the interval +// (minval, maxval) is empty, the result is NaN. Large intervals which include +// both tails may have reduced accuracy. +template +struct TruncatedNormalFunctor { + void operator()(OpKernelContext* ctx, const Device& d, int64 num_batches, + int64 samples_per_batch, int64 num_elements, + typename TTypes::ConstFlat means, + typename TTypes::ConstFlat stddevs, + typename TTypes::ConstFlat minvals, + typename TTypes::ConstFlat maxvals, + const random::PhiloxRandom& gen, + typename TTypes::Flat output); + + static const int kMaxIterations = 100; +}; + +} // namespace functor +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_PARAMETERIZED_TRUNCATED_NORMAL_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/poisson-loss.h b/linux/include/tensorflow/core/kernels/poisson-loss.h new file mode 100644 index 000000000..f91244454 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/poisson-loss.h @@ -0,0 +1,109 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_POISSON_LOSS_H_ +#define TENSORFLOW_CORE_KERNELS_POISSON_LOSS_H_ + +#include + +#include "tensorflow/core/kernels/loss.h" +#include "tensorflow/core/lib/core/errors.h" + +namespace tensorflow { + +class PoissonLossUpdater : public DualLossUpdater { + public: + // Update is found by a Newton algorithm (see readme.md). + double ComputeUpdatedDual(const int num_loss_partitions, const double label, + const double example_weight, + const double current_dual, const double wx, + const double weighted_example_norm) const final { + // Newton algorithm converges quadratically so 10 steps will be largely + // enough to achieve a very good precision + static const int newton_total_steps = 10; + // Initialize the Newton optimization at x such that + // exp(x) = label - current_dual + const double y_minus_a = label - current_dual; + double x = (y_minus_a > 0) ? log(y_minus_a) : 0; + for (int i = 0; i < newton_total_steps; ++i) { + x = NewtonStep(x, num_loss_partitions, label, wx, example_weight, + weighted_example_norm, current_dual); + } + return label - exp(x); + } + + // Dual of poisson loss function. + // https://en.wikipedia.org/wiki/Convex_conjugate + double ComputeDualLoss(const double current_dual, const double example_label, + const double example_weight) const final { + // Dual of the poisson loss function is + // (y-a)*(log(y-a)-1), where a is the dual variable. + // It is defined only for a::max(); + } + return y_minus_a * (log(y_minus_a) - 1) * example_weight; + } + + double ComputePrimalLoss(const double wx, const double example_label, + const double example_weight) const final { + return (exp(wx) - wx * example_label) * example_weight; + } + + double PrimalLossDerivative(const double wx, const double label, + const double example_weight) const final { + return (exp(wx) - label) * example_weight; + } + + // TODO(chapelle): We need to introduce a maximum_prediction parameter, + // expose that parameter to the user and have this method return + // 1.0/maximum_prediction. + // Setting this at 1 for now, it only impacts the adaptive sampling. + double SmoothnessConstant() const final { return 1; } + + Status ConvertLabel(float* const example_label) const final { + if (*example_label < 0.0) { + return errors::InvalidArgument( + "Only non-negative labels can be used with the Poisson log loss. " + "Found example with label: ", *example_label); + } + return Status::OK(); + } + + private: + // One Newton step (see readme.md). + double NewtonStep(const double x, const int num_loss_partitions, + const double label, const double wx, + const double example_weight, + const double weighted_example_norm, + const double current_dual) const { + const double expx = exp(x); + const double numerator = + x - wx - num_loss_partitions * weighted_example_norm * + example_weight * (label - current_dual - expx); + const double denominator = + 1 + num_loss_partitions * weighted_example_norm * example_weight * expx; + return x - numerator / denominator; + } +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_LOGISTIC_LOSS_H_ diff --git a/linux/include/tensorflow/core/kernels/pooling_ops_3d.h b/linux/include/tensorflow/core/kernels/pooling_ops_3d.h new file mode 100644 index 000000000..319b17397 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/pooling_ops_3d.h @@ -0,0 +1,80 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_POOLING_OPS_3D_H_ +#define TENSORFLOW_CORE_KERNELS_POOLING_OPS_3D_H_ + +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/util/padding.h" +#include "tensorflow/core/util/tensor_format.h" + +namespace tensorflow { + +enum PoolingType { MAX, AVG }; + +template +struct LaunchPoolingOp; + +template +struct LaunchAvgPooling3dGradOp; + +template +struct LaunchMaxPooling3dGradOp; + +template +struct LaunchMaxPooling3dGradGradOp; + +// A helper class to manage sizes and shapes for 3d pooling operations. +struct Pool3dParameters { + // Updates context->status if there is an invalid input. + Pool3dParameters(OpKernelContext* context, const std::vector& ksize, + const std::vector& stride, Padding padding, + TensorFormat data_format, + const TensorShape& tensor_in_shape); + + // Returns the shape of the output for "forward" pooling operations. + TensorShape forward_output_shape(); + + int depth; + + int tensor_in_planes; + int tensor_in_cols; + int tensor_in_rows; + int tensor_in_batch; + + int window_planes; + int window_cols; + int window_rows; + int depth_window; + + int plane_stride; + int col_stride; + int row_stride; + int depth_stride; + + int64 out_plane; + int64 out_height; + int64 out_width; + + int64 pad_planes; + int64 pad_cols; + int64 pad_rows; + + TensorFormat data_format; +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_POOLING_OPS_3D_H_ diff --git a/linux/include/tensorflow/core/kernels/pooling_ops_common.h b/linux/include/tensorflow/core/kernels/pooling_ops_common.h new file mode 100644 index 000000000..dda2c80c4 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/pooling_ops_common.h @@ -0,0 +1,608 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_POOLING_OPS_COMMON_H_ +#define TENSORFLOW_CORE_KERNELS_POOLING_OPS_COMMON_H_ + +#include + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/numeric_op.h" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/kernels/avgpooling_op.h" +#include "tensorflow/core/kernels/maxpooling_op.h" +#include "tensorflow/core/kernels/ops_util.h" +#include "tensorflow/core/util/padding.h" +#include "tensorflow/core/util/tensor_format.h" +#include "tensorflow/core/util/work_sharder.h" + +#if GOOGLE_CUDA +#include "tensorflow/core/kernels/maxpooling_op_gpu.h" +#endif // GOOGLE_CUDA + +namespace tensorflow { + +typedef Eigen::GpuDevice GPUDevice; + +// A helper class to manage sizes and shapes for pooling operations. +struct PoolParameters { + // Updates context->status if there is an invalid input. + PoolParameters(OpKernelContext* context, const std::vector& ksize, + const std::vector& stride, Padding padding, + TensorFormat data_format, const TensorShape& tensor_in_shape); + + // Returns the shape of the output for "forward" pooling operations. + TensorShape forward_output_shape(); + + int depth; + + int tensor_in_cols; + int tensor_in_rows; + int tensor_in_batch; + + int window_rows; + int window_cols; + int depth_window; + + int row_stride; + int col_stride; + int depth_stride; + + int64 out_height; + int64 out_width; + int out_depth; + + int64 pad_rows; + int64 pad_cols; + int pad_depth; + + TensorFormat data_format; +}; + +// An implementation of MaxPooling (forward). +// TODO (yongtang): Remove MaxPoolingOp and use MaxPoolingV2Op, +// QuantizedMaxPoolingOp depends on MaxPoolingOp so keep intact for now +template +class MaxPoolingOp : public OpKernel { + public: + explicit MaxPoolingOp(OpKernelConstruction* context) : OpKernel(context) { + string data_format; + auto status = context->GetAttr("data_format", &data_format); + if (status.ok()) { + OP_REQUIRES(context, FormatFromString(data_format, &data_format_), + errors::InvalidArgument("Invalid data format")); + OP_REQUIRES( + context, data_format_ == FORMAT_NHWC, + errors::InvalidArgument("Default MaxPoolingOp only supports NHWC ", + "on device type ", + DeviceTypeString(context->device_type()))); + } else { + data_format_ = FORMAT_NHWC; + } + OP_REQUIRES_OK(context, context->GetAttr("ksize", &ksize_)); + OP_REQUIRES(context, ksize_.size() == 4, + errors::InvalidArgument("Sliding window ksize field must " + "specify 4 dimensions")); + OP_REQUIRES_OK(context, context->GetAttr("strides", &stride_)); + OP_REQUIRES(context, stride_.size() == 4, + errors::InvalidArgument("Sliding window stride field must " + "specify 4 dimensions")); + OP_REQUIRES_OK(context, context->GetAttr("padding", &padding_)); + OP_REQUIRES(context, ksize_[0] == 1 && stride_[0] == 1, + errors::Unimplemented( + "Pooling is not yet supported on the batch dimension.")); + } + + void Compute(OpKernelContext* context) override { + const Tensor& tensor_in = context->input(0); + PoolParameters params{context, ksize_, stride_, + padding_, FORMAT_NHWC, tensor_in.shape()}; + if (!context->status().ok()) { + return; + } + + Tensor* output = nullptr; + OP_REQUIRES_OK(context, context->allocate_output( + 0, params.forward_output_shape(), &output)); + + if (params.depth_window > 1) { + // Validate spec against the current implementation. A + // relaxation of these requirements would be ideal. + OP_REQUIRES(context, params.depth % params.depth_window == 0, + errors::Unimplemented( + "Depthwise max pooling requires " + "the depth window to evenly divide the input depth.")); + OP_REQUIRES( + context, params.depth_window == params.depth_stride, + errors::Unimplemented("Depthwise max pooling requires " + "the depth window to equal the depth stride.")); + + DepthwiseMaxPool(context, output, tensor_in, params); + } else { + SpatialMaxPool(context, output, tensor_in, params, padding_); + } + } + + private: + // Single-threaded implementation of DepthwiseMaxPool which + // does not handle all of the same options as SpatialMaxPool + // (strict assumptions on no padding, stride). + // + // TODO(vrv): implement a more general depthwise-max pool that works + // on GPU as well. + void DepthwiseMaxPool(OpKernelContext* context, Tensor* output, + const Tensor& tensor_in, const PoolParameters& params) { + Eigen::Map> + in_by_pool(tensor_in.flat().data(), params.depth_window, + tensor_in.NumElements() / params.depth_window); + Eigen::Map> out_by_pool( + output->flat().data(), 1, output->NumElements()); + out_by_pool = in_by_pool.colwise().maxCoeff(); + } + + void SpatialMaxPool(OpKernelContext* context, Tensor* output, + const Tensor& tensor_in, const PoolParameters& params, + const Padding& padding) { + // On GPU, use Eigen's Spatial Max Pooling. On CPU, use an + // EigenMatrix version that is currently faster than Eigen's + // Spatial MaxPooling implementation. + // + // TODO(vrv): Remove this once we no longer need it. + if (std::is_same::value) { + Eigen::PaddingType pt = BrainPadding2EigenPadding(padding); + functor::SpatialMaxPooling()( + context->eigen_device(), output->tensor(), + tensor_in.tensor(), params.window_rows, params.window_cols, + params.row_stride, params.col_stride, pt); + } else { + typedef Eigen::Map> + ConstEigenMatrixMap; + typedef Eigen::Map> + EigenMatrixMap; + + ConstEigenMatrixMap in_mat(tensor_in.flat().data(), params.depth, + params.tensor_in_cols * params.tensor_in_rows * + params.tensor_in_batch); + EigenMatrixMap out_mat( + output->flat().data(), params.depth, + params.out_width * params.out_height * params.tensor_in_batch); + + const DeviceBase::CpuWorkerThreads& worker_threads = + *(context->device()->tensorflow_cpu_worker_threads()); + + // The following code basically does the following: + // 1. Flattens the input and output tensors into two dimensional arrays. + // tensor_in_as_matrix: + // depth by (tensor_in_cols * tensor_in_rows * tensor_in_batch) + // output_as_matrix: + // depth by (out_width * out_height * tensor_in_batch) + // + // 2. Walks through the set of columns in the flattened + // tensor_in_as_matrix, + // and updates the corresponding column(s) in output_as_matrix with the + // max value. + auto shard = [¶ms, &in_mat, &out_mat](int64 start, int64 limit) { + const int32 in_rows = params.tensor_in_rows; + const int32 in_cols = params.tensor_in_cols; + const int32 pad_rows = params.pad_rows; + const int32 pad_cols = params.pad_cols; + const int32 window_rows = params.window_rows; + const int32 window_cols = params.window_cols; + const int32 row_stride = params.row_stride; + const int32 col_stride = params.col_stride; + const int32 out_height = params.out_height; + const int32 out_width = params.out_width; + + { + // Initializes the output tensor with MIN. + const int32 output_image_size = out_height * out_width * params.depth; + EigenMatrixMap out_shard(out_mat.data() + start * output_image_size, + 1, (limit - start) * output_image_size); + out_shard.setConstant(Eigen::NumTraits::lowest()); + } + + for (int32 b = start; b < limit; ++b) { + const int32 out_offset_batch = b * out_height; + for (int32 h = 0; h < in_rows; ++h) { + for (int32 w = 0; w < in_cols; ++w) { + // (h_start, h_end) * (w_start, w_end) is the range that the input + // vector projects to. + const int32 hpad = h + pad_rows; + const int32 wpad = w + pad_cols; + const int32 h_start = (hpad < window_rows) + ? 0 + : (hpad - window_rows) / row_stride + 1; + const int32 h_end = std::min(hpad / row_stride + 1, out_height); + const int32 w_start = (wpad < window_cols) + ? 0 + : (wpad - window_cols) / col_stride + 1; + const int32 w_end = std::min(wpad / col_stride + 1, out_width); + // compute elementwise max + const int32 in_offset = (b * in_rows + h) * in_cols + w; + for (int32 ph = h_start; ph < h_end; ++ph) { + const int32 out_offset_base = + (out_offset_batch + ph) * out_width; + for (int32 pw = w_start; pw < w_end; ++pw) { + const int32 out_offset = out_offset_base + pw; + out_mat.col(out_offset) = + out_mat.col(out_offset).cwiseMax(in_mat.col(in_offset)); + } + } + } + } + } + }; + + // TODO(andydavis) Consider sharding across batch x rows x cols. + // TODO(andydavis) Consider a higher resolution shard cost model. + const int64 shard_cost = + params.tensor_in_rows * params.tensor_in_cols * params.depth; + Shard(worker_threads.num_threads, worker_threads.workers, + params.tensor_in_batch, shard_cost, shard); + } + } + + std::vector ksize_; + std::vector stride_; + Padding padding_; + TensorFormat data_format_; +}; + +template +struct LaunchMaxPoolingNoMask_NCHW_VECT_C; + +#ifdef GOOGLE_CUDA +template <> +struct LaunchMaxPoolingNoMask_NCHW_VECT_C { + static void launch(OpKernelContext* context, const PoolParameters& params, + const Tensor& input, Tensor* output) { + bool status = functor::MaxPoolForwardNoMask_NCHW_VECT_C()( + reinterpret_cast(input.flat().data()), + params.tensor_in_batch, params.tensor_in_rows, params.tensor_in_cols, + params.depth, params.out_height, params.out_width, params.window_rows, + params.window_cols, params.row_stride, params.col_stride, + params.pad_rows, params.pad_cols, + reinterpret_cast(output->flat().data()), + context->eigen_gpu_device()); + if (!status) { + context->SetStatus(errors::Internal( + "Failed launching LaunchMaxPoolingNoMask_NCHW_VECT_C")); + } + } +}; +#endif + +template +class MaxPoolingV2Op : public OpKernel { + public: + explicit MaxPoolingV2Op(OpKernelConstruction* context) : OpKernel(context) { + string data_format; + auto status = context->GetAttr("data_format", &data_format); + if (status.ok()) { + OP_REQUIRES(context, FormatFromString(data_format, &data_format_), + errors::InvalidArgument("Invalid data format")); + OP_REQUIRES( + context, + data_format_ == FORMAT_NHWC || data_format_ == FORMAT_NCHW_VECT_C, + errors::InvalidArgument( + "MaxPoolingV2Op only supports NHWC or NCHW_VECT_C. Got: ", + data_format)); + } else { + data_format_ = FORMAT_NHWC; + } + if (context->num_inputs() == 1) { + OP_REQUIRES_OK(context, context->GetAttr("ksize", &ksize_)); + OP_REQUIRES(context, ksize_.size() == 4, + errors::InvalidArgument("Sliding window ksize field must " + "specify 4 dimensions")); + OP_REQUIRES_OK(context, context->GetAttr("strides", &stride_)); + OP_REQUIRES(context, stride_.size() == 4, + errors::InvalidArgument("Sliding window stride field must " + "specify 4 dimensions")); + OP_REQUIRES(context, ksize_[0] == 1 && stride_[0] == 1, + errors::Unimplemented( + "Pooling is not yet supported on the batch dimension.")); + } + OP_REQUIRES_OK(context, context->GetAttr("padding", &padding_)); + } + + void Compute(OpKernelContext* context) override { + const Tensor& tensor_in = context->input(0); + + std::vector ksize = ksize_; + std::vector stride = stride_; + + if (context->num_inputs() != 1) { + const Tensor& tensor_ksize = context->input(1); + auto value_ksize = tensor_ksize.flat(); + ksize.resize(tensor_ksize.shape().num_elements()); + std::copy_n(&value_ksize(0), ksize.size(), ksize.begin()); + + const Tensor& tensor_stride = context->input(2); + auto value_stride = tensor_stride.flat(); + stride.resize(tensor_stride.shape().num_elements()); + std::copy_n(&value_stride(0), stride.size(), stride.begin()); + } + + OP_REQUIRES(context, ksize.size() == 4, + errors::InvalidArgument("Sliding window ksize field must " + "specify 4 dimensions")); + OP_REQUIRES(context, stride.size() == 4, + errors::InvalidArgument("Sliding window stride field must " + "specify 4 dimensions")); + OP_REQUIRES(context, ksize[0] == 1 && stride[0] == 1, + errors::Unimplemented( + "Pooling is not yet supported on the batch dimension.")); + + PoolParameters params{context, ksize, stride, + padding_, data_format_, tensor_in.shape()}; + if (!context->status().ok()) { + return; + } + + Tensor* output = nullptr; + OP_REQUIRES_OK(context, context->allocate_output( + 0, params.forward_output_shape(), &output)); + + if (params.depth_window > 1) { + // Validate spec against the current implementation. A + // relaxation of these requirements would be ideal. + OP_REQUIRES(context, params.depth % params.depth_window == 0, + errors::Unimplemented( + "Depthwise max pooling requires " + "the depth window to evenly divide the input depth.")); + OP_REQUIRES( + context, params.depth_window == params.depth_stride, + errors::Unimplemented("Depthwise max pooling requires " + "the depth window to equal the depth stride.")); + + DepthwiseMaxPool(context, output, tensor_in, params); + } else { + SpatialMaxPool(context, output, tensor_in, params, padding_); + } + } + + private: + // Single-threaded implementation of DepthwiseMaxPool which + // does not handle all of the same options as SpatialMaxPool + // (strict assumptions on no padding, stride). + // + // TODO(vrv): implement a more general depthwise-max pool that works + // on GPU as well. + void DepthwiseMaxPool(OpKernelContext* context, Tensor* output, + const Tensor& tensor_in, const PoolParameters& params) { + Eigen::Map> + in_by_pool(tensor_in.flat().data(), params.depth_window, + tensor_in.NumElements() / params.depth_window); + Eigen::Map> out_by_pool( + output->flat().data(), 1, output->NumElements()); + out_by_pool = in_by_pool.colwise().maxCoeff(); + } + + void SpatialMaxPool(OpKernelContext* context, Tensor* output, + const Tensor& tensor_in, const PoolParameters& params, + const Padding& padding) { + // On GPU, use Eigen's Spatial Max Pooling. On CPU, use an + // EigenMatrix version that is currently faster than Eigen's + // Spatial MaxPooling implementation. + // + // TODO(vrv): Remove this once we no longer need it. +#ifdef GOOGLE_CUDA + if (std::is_same::value) { + Eigen::PaddingType pt = BrainPadding2EigenPadding(padding); + if (std::is_same::value) { + LaunchMaxPoolingNoMask_NCHW_VECT_C::launch( + context, params, tensor_in, output); + } else { + functor::SpatialMaxPooling()( + context->eigen_device(), output->tensor(), + tensor_in.tensor(), params.window_rows, params.window_cols, + params.row_stride, params.col_stride, pt); + } + } else +#endif + { + typedef Eigen::Map> + ConstEigenMatrixMap; + typedef Eigen::Map> + EigenMatrixMap; + + ConstEigenMatrixMap in_mat(tensor_in.flat().data(), params.depth, + params.tensor_in_cols * params.tensor_in_rows * + params.tensor_in_batch); + EigenMatrixMap out_mat( + output->flat().data(), params.depth, + params.out_width * params.out_height * params.tensor_in_batch); + + const DeviceBase::CpuWorkerThreads& worker_threads = + *(context->device()->tensorflow_cpu_worker_threads()); + + // The following code basically does the following: + // 1. Flattens the input and output tensors into two dimensional arrays. + // tensor_in_as_matrix: + // depth by (tensor_in_cols * tensor_in_rows * tensor_in_batch) + // output_as_matrix: + // depth by (out_width * out_height * tensor_in_batch) + // + // 2. Walks through the set of columns in the flattened + // tensor_in_as_matrix, + // and updates the corresponding column(s) in output_as_matrix with the + // max value. + auto shard = [¶ms, &in_mat, &out_mat](int64 start, int64 limit) { + const int32 in_rows = params.tensor_in_rows; + const int32 in_cols = params.tensor_in_cols; + const int32 pad_rows = params.pad_rows; + const int32 pad_cols = params.pad_cols; + const int32 window_rows = params.window_rows; + const int32 window_cols = params.window_cols; + const int32 row_stride = params.row_stride; + const int32 col_stride = params.col_stride; + const int32 out_height = params.out_height; + const int32 out_width = params.out_width; + + { + // Initializes the output tensor with MIN. + const int32 output_image_size = out_height * out_width * params.depth; + EigenMatrixMap out_shard(out_mat.data() + start * output_image_size, + 1, (limit - start) * output_image_size); + out_shard.setConstant(Eigen::NumTraits::lowest()); + } + + for (int32 b = start; b < limit; ++b) { + const int32 out_offset_batch = b * out_height; + for (int32 h = 0; h < in_rows; ++h) { + for (int32 w = 0; w < in_cols; ++w) { + // (h_start, h_end) * (w_start, w_end) is the range that the input + // vector projects to. + const int32 hpad = h + pad_rows; + const int32 wpad = w + pad_cols; + const int32 h_start = (hpad < window_rows) + ? 0 + : (hpad - window_rows) / row_stride + 1; + const int32 h_end = std::min(hpad / row_stride + 1, out_height); + const int32 w_start = (wpad < window_cols) + ? 0 + : (wpad - window_cols) / col_stride + 1; + const int32 w_end = std::min(wpad / col_stride + 1, out_width); + // compute elementwise max + const int32 in_offset = (b * in_rows + h) * in_cols + w; + for (int32 ph = h_start; ph < h_end; ++ph) { + const int32 out_offset_base = + (out_offset_batch + ph) * out_width; + for (int32 pw = w_start; pw < w_end; ++pw) { + const int32 out_offset = out_offset_base + pw; + out_mat.col(out_offset) = + out_mat.col(out_offset).cwiseMax(in_mat.col(in_offset)); + } + } + } + } + } + }; + + // TODO(andydavis) Consider sharding across batch x rows x cols. + // TODO(andydavis) Consider a higher resolution shard cost model. + const int64 shard_cost = + params.tensor_in_rows * params.tensor_in_cols * params.depth; + Shard(worker_threads.num_threads, worker_threads.workers, + params.tensor_in_batch, shard_cost, shard); + } + } + + std::vector ksize_; + std::vector stride_; + Padding padding_; + TensorFormat data_format_; +}; + +template +void SpatialAvgPool(OpKernelContext* context, Tensor* output, + const Tensor& input, const PoolParameters& params, + const Padding& padding) { + typedef Eigen::Map> + ConstEigenMatrixMap; + typedef Eigen::Map> + EigenMatrixMap; + + auto in_flat = input.flat(); + auto out_flat = output->flat(); + + auto shard = [¶ms, &in_flat, &out_flat](int64 start, int64 limit) { + // Calculate indices for this shards chunk of work. + const int64 input_image_size = + params.tensor_in_rows * params.tensor_in_cols * params.depth; + const int64 output_image_size = + params.out_width * params.out_height * params.depth; + const int64 shard_batch_size = limit - start; + + ConstEigenMatrixMap in_mat( + in_flat.data() + start * input_image_size, params.depth, + params.tensor_in_cols * params.tensor_in_rows * shard_batch_size); + EigenMatrixMap out_mat( + out_flat.data() + start * output_image_size, params.depth, + params.out_width * params.out_height * shard_batch_size); + Eigen::Matrix out_count(out_mat.cols()); + out_count.setZero(); + + // Initializes output to zero. + out_mat.setZero(); + + // The following code basically does the following: + // 1. Flattens the input and output tensors into two dimensional arrays. + // tensor_in_as_matrix: + // depth by (tensor_in_cols * tensor_in_rows * tensor_in_batch) + // output_as_matrix: + // depth by (out_width * out_height * tensor_in_batch) + // + // 2. Walks through the set of columns in the flattened + // tensor_in_as_matrix, + // and updates the corresponding column(s) in output_as_matrix with the + // average value. + for (int b = 0; b < shard_batch_size; ++b) { + for (int h = 0; h < params.tensor_in_rows; ++h) { + for (int w = 0; w < params.tensor_in_cols; ++w) { + // (h_start, h_end) * (w_start, w_end) is the range that the input + // vector projects to. + const int hpad = h + params.pad_rows; + const int wpad = w + params.pad_cols; + const int h_start = + (hpad < params.window_rows) + ? 0 + : (hpad - params.window_rows) / params.row_stride + 1; + const int h_end = + std::min(hpad / params.row_stride + 1, params.out_height); + const int w_start = + (wpad < params.window_cols) + ? 0 + : (wpad - params.window_cols) / params.col_stride + 1; + const int w_end = + std::min(wpad / params.col_stride + 1, params.out_width); + const int in_offset = + (b * params.tensor_in_rows + h) * params.tensor_in_cols + w; + Eigen::DSizes in_indices(0, in_offset); + for (int ph = h_start; ph < h_end; ++ph) { + for (int pw = w_start; pw < w_end; ++pw) { + const int out_offset = + (b * params.out_height + ph) * params.out_width + pw; + out_mat.col(out_offset) += in_mat.col(in_offset); + out_count(out_offset) += T(1); + } + } + } + } + } + + DCHECK_GT(out_count.minCoeff(), T(0)); + out_mat.array().rowwise() /= out_count.transpose().array(); + }; + + const int64 work_unit_size = + params.tensor_in_rows * params.tensor_in_cols * params.depth; + // NOTE: Constants in calculation below were estimated based on benchmarking. + // Nanoseconds/work_unit for benchmarks ranged from 0.01 to 0.001, and + // so the factor 0.01 (i.e. 1/100) with a max of 10000, was chosen to limit + // the work unit cost to an operating range in which it emperically performed + // best. + const int64 work_unit_cost = std::max(int64{10000}, work_unit_size / 100LL); + const DeviceBase::CpuWorkerThreads& worker_threads = + *(context->device()->tensorflow_cpu_worker_threads()); + Shard(worker_threads.num_threads, worker_threads.workers, + params.tensor_in_batch, work_unit_cost, shard); +} + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_POOLING_OPS_COMMON_H_ diff --git a/linux/include/tensorflow/core/kernels/population_count_op.h b/linux/include/tensorflow/core/kernels/population_count_op.h new file mode 100644 index 000000000..2c9812967 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/population_count_op.h @@ -0,0 +1,38 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or +implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_POPULATION_COUNT_OP_H_ +#define TENSORFLOW_CORE_KERNELS_POPULATION_COUNT_OP_H_ + +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +namespace functor { + +template +struct PopulationCount { + void operator()(OpKernelContext* c, typename TTypes::ConstFlat input, + TTypes::Flat output); +}; + +} // namespace functor + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_POPULATION_COUNT_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/priority_queue.h b/linux/include/tensorflow/core/kernels/priority_queue.h new file mode 100644 index 000000000..8e69b5b69 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/priority_queue.h @@ -0,0 +1,93 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_PRIORITY_QUEUE_H_ +#define TENSORFLOW_CORE_KERNELS_PRIORITY_QUEUE_H_ + +#include +#include +#include + +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/kernels/typed_queue.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/mutex.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +using PriorityTensorPair = std::pair; + +struct ComparePriorityTensorPair { + // 0 is a higher priority than 1, -MAX_LONG is a higher priority + // than MAX_LONG, etc. Values coming in with a smaller + // priority number will bubble to the front of the queue. + bool operator()(const PriorityTensorPair& lhs, + const PriorityTensorPair& rhs) const { + return lhs.first > rhs.first; + } +}; + +class PriorityQueue + : public TypedQueue, + ComparePriorityTensorPair> > { + public: + PriorityQueue(int32 capacity, const DataTypeVector& component_dtypes, + const std::vector& component_shapes, + const string& name); + + Status Initialize() override; // Must be called before any other method. + + // Implementations of QueueInterface methods -------------------------------- + + void TryEnqueue(const Tuple& tuple, OpKernelContext* ctx, + DoneCallback callback) override; + void TryEnqueueMany(const Tuple& tuple, OpKernelContext* ctx, + DoneCallback callback) override; + void TryDequeue(OpKernelContext* ctx, CallbackWithTuple callback) override; + void TryDequeueMany(int num_elements, OpKernelContext* ctx, + bool allow_small_batch, + CallbackWithTuple callback) override; + Status MatchesNodeDef(const NodeDef& node_def) override; + Status MatchesPriorityNodeDefTypes(const NodeDef& node_def) const; + Status MatchesPriorityNodeDefShapes(const NodeDef& node_def) const; + + int32 size() override { + mutex_lock lock(mu_); + return queues_[0].size(); + } + + private: + ~PriorityQueue() override {} + + // Helper for dequeuing a single element from queues_. + void DequeueLocked(OpKernelContext* ctx, Tuple* tuple) + EXCLUSIVE_LOCKS_REQUIRED(mu_); + + static Status GetElementComponentFromBatch(const Tuple& tuple, int index, + int component, + OpKernelContext* ctx, + PersistentTensor* out_element); + + TF_DISALLOW_COPY_AND_ASSIGN(PriorityQueue); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_PRIORITY_QUEUE_H_ diff --git a/linux/include/tensorflow/core/kernels/qr_op_impl.h b/linux/include/tensorflow/core/kernels/qr_op_impl.h new file mode 100644 index 000000000..535df9d16 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/qr_op_impl.h @@ -0,0 +1,302 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_QR_OP_IMPL_H_ +#define TENSORFLOW_CORE_KERNELS_QR_OP_IMPL_H_ + +// See docs in ../ops/linalg_ops.cc. +// +// This header file is used by the individual qr_*op*.cc files for registering +// individual kernels. A separate file is used for each instantiated kernel to +// improve compilation times. +#include +#include + +#if GOOGLE_CUDA +#define EIGEN_USE_GPU +#endif + +#include "third_party/eigen3/Eigen/QR" +#include "tensorflow/core/framework/kernel_def_builder.h" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/kernels/linalg_ops_common.h" +#include "tensorflow/core/lib/core/errors.h" +#include "tensorflow/core/platform/logging.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/types.h" + +#if GOOGLE_CUDA +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/kernels/cuda_solvers.h" +#include "tensorflow/core/kernels/cwise_ops.h" +#include "tensorflow/core/kernels/eye_functor.h" +#include "tensorflow/core/kernels/matrix_band_part_op.h" +#include "tensorflow/core/kernels/transpose_functor.h" +#endif + +namespace tensorflow { + +template +class QrOp : public LinearAlgebraOp { + public: + typedef LinearAlgebraOp Base; + + explicit QrOp(OpKernelConstruction* context) : Base(context) { + OP_REQUIRES_OK(context, context->GetAttr("full_matrices", &full_matrices_)); + } + + using TensorShapes = typename Base::TensorShapes; + + void ValidateInputMatrixShapes( + OpKernelContext* context, + const TensorShapes& input_matrix_shapes) const final { + Base::ValidateSingleMatrix(context, input_matrix_shapes); + } + + TensorShapes GetOutputMatrixShapes( + const TensorShapes& input_matrix_shapes) const final { + int64 m = input_matrix_shapes[0].dim_size(0); + int64 n = input_matrix_shapes[0].dim_size(1); + int64 min_size = std::min(m, n); + if (full_matrices_) { + return TensorShapes({TensorShape({m, m}), TensorShape({m, n})}); + } else { + return TensorShapes( + {TensorShape({m, min_size}), TensorShape({min_size, n})}); + } + } + + int64 GetCostPerUnit(const TensorShapes& input_matrix_shapes) const final { + double m = static_cast(input_matrix_shapes[0].dim_size(0)); + double n = static_cast(input_matrix_shapes[0].dim_size(1)); + double max_size = std::max(m, n); + double min_size = std::min(m, n); + double cost = 2 * max_size * min_size * min_size - + 2 * min_size * min_size * min_size / 3.; + // TODO(jpoulson): Increase the cost if full_matrices is true in a manner + // that reflects the algorithm used for the expansion. + return cost >= static_cast(kint64max) ? kint64max + : static_cast(cost); + } + + using Matrix = typename Base::Matrix; + using MatrixMaps = typename Base::MatrixMaps; + using ConstMatrixMap = typename Base::ConstMatrixMap; + using ConstMatrixMaps = typename Base::ConstMatrixMaps; + + void ComputeMatrix(OpKernelContext* context, const ConstMatrixMaps& inputs, + MatrixMaps* outputs) final { + Eigen::HouseholderQR qr(inputs[0]); + const int m = inputs[0].rows(); + const int n = inputs[0].cols(); + const int min_size = std::min(m, n); + + if (full_matrices_) { + outputs->at(0) = qr.householderQ(); + outputs->at(1) = qr.matrixQR().template triangularView(); + } else { + // TODO(jpoulson): Exploit the fact that Householder transformations can + // be expanded faster than they can be applied to an arbitrary matrix + // (Cf. LAPACK's DORGQR). + Matrix tmp = Matrix::Identity(m, min_size); + outputs->at(0) = qr.householderQ() * tmp; + auto qr_top = qr.matrixQR().block(0, 0, min_size, n); + outputs->at(1) = qr_top.template triangularView(); + } + } + + private: + bool full_matrices_; + + TF_DISALLOW_COPY_AND_ASSIGN(QrOp); +}; + +#if GOOGLE_CUDA + +typedef Eigen::GpuDevice GPUDevice; + +template +class QrOpGpu : public AsyncOpKernel { + public: + explicit QrOpGpu(OpKernelConstruction* context) : AsyncOpKernel(context) { + OP_REQUIRES_OK(context, context->GetAttr("full_matrices", &full_matrices_)); + } + + void ComputeAsync(OpKernelContext* context, DoneCallback done) final { + const Tensor& input = context->input(0); + const int ndims = input.dims(); + const int64 m = input.dim_size(ndims - 2); + const int64 n = input.dim_size(ndims - 1); + const int64 min_size = std::min(m, n); + const int64 batch_size = + input.template flat_inner_dims().dimension(0); + + // Validate inputs. + OP_REQUIRES_ASYNC( + context, ndims >= 2, + errors::InvalidArgument("Input must have rank >= 2, got ", ndims), + done); + + // Allocate output. + // If full_matrices_ is true then Q is m x m and R is m x n. + // Otherwise, Q is m x min(m, n), and R is min(m, n) x n. + Tensor* q; + TensorShape q_shape = input.shape(); + q_shape.set_dim(ndims - 1, full_matrices_ ? m : min_size); + OP_REQUIRES_OK_ASYNC(context, context->allocate_output(0, q_shape, &q), + done); + Tensor* r; + TensorShape r_shape = input.shape(); + r_shape.set_dim(ndims - 2, full_matrices_ ? m : min_size); + OP_REQUIRES_OK_ASYNC(context, context->allocate_output(1, r_shape, &r), + done); + + if (input.NumElements() == 0) { + done(); + return; + } + + // TODO(rmlarsen): Convert to std::make_unique when available. + std::unique_ptr solver(new CudaSolver(context)); + + // Allocate temporaries. + Tensor input_transposed; + TensorShape transposed_shape = input.shape(); + transposed_shape.set_dim(ndims - 2, input.dim_size(ndims - 1)); + transposed_shape.set_dim(ndims - 1, input.dim_size(ndims - 2)); + + OP_REQUIRES_OK_ASYNC( + context, + solver->allocate_scoped_tensor(DataTypeToEnum::value, + transposed_shape, &input_transposed), + done); + + Tensor tau; + OP_REQUIRES_OK_ASYNC(context, + solver->allocate_scoped_tensor( + DataTypeToEnum::value, + TensorShape({batch_size, min_size}), &tau), + done); + + // Transpose input, since cuSolver uses column-major, while TensorFlow uses + // row-major storage. + const GPUDevice& device = context->eigen_device(); + OP_REQUIRES_OK_ASYNC( + context, DoMatrixTranspose(device, input, &input_transposed), done); + + // Compute QR decomposition in-place in input_transposed. + std::vector dev_info; + dev_info.push_back(solver->GetDeviceLapackInfo(batch_size, "geqrf")); + auto input_transposed_reshaped = + input_transposed.flat_inner_dims(); + auto tau_matrix = tau.matrix(); + auto r_reshaped = r->flat_inner_dims(); + for (int batch = 0; batch < batch_size; ++batch) { + OP_REQUIRES_OK_ASYNC( + context, + solver->Geqrf(m, n, &input_transposed_reshaped(batch, 0, 0), m, + &tau_matrix(batch, 0), + dev_info.back().mutable_data() + batch), + done); + } + + // Generate R. R is equal to the upper triangle of the decomposition + // stored in input_transposed. Crop, transpose (to get back to row-major) + // and copy it to the output buffer. + if (full_matrices_ || m == n) { + OP_REQUIRES_OK_ASYNC( + context, DoMatrixTranspose(device, input_transposed, r), done); + } else { + const Scalar alpha(1); + const Scalar beta(0); + const Scalar* dummy = nullptr; + for (int batch = 0; batch < batch_size; ++batch) { + OP_REQUIRES_OK_ASYNC( + context, + solver->Geam(CUBLAS_OP_T, CUBLAS_OP_N, n, + full_matrices_ ? m : min_size, &alpha, + &input_transposed_reshaped(batch, 0, 0), m, &beta, + dummy, n, &r_reshaped(batch, 0, 0), n), + done); + } + } + // Extract the upper triangle of r (i.e. zero out the strictly lower + // triangle). + functor::MatrixBandPartFunctor band_part; + auto r_reshaped_const = + const_cast(r)->flat_inner_dims(); + band_part(context, device, 0 /* num_lower_diags */, + -1 /* num_upper_diags */, r_reshaped_const, r_reshaped); + + // Generate Q from the decomposition in input_transposed. + if (m != n && (full_matrices_ || m < n)) { + // Generate full m x m matrix Q by computing the product Q^T * I, + // where the transpose is to get back to row-major form. + // In the complex case we actually form Q^H * I and conjugate it + // to get Q in row-major form. + functor::EyeFunctor eye; + auto q_reshaped = q->flat_inner_dims(); + eye(device, q_reshaped); + for (int batch = 0; batch < batch_size; ++batch) { + // Notice: It appears that Unmqr does not write a zero into *info upon + // success (probably a bug), so we simply re-use the info array already + // zeroed by Geqrf above. + OP_REQUIRES_OK_ASYNC( + context, + solver->Unmqr(CUBLAS_SIDE_LEFT, CublasAdjointOp(), m, m, + min_size, &input_transposed_reshaped(batch, 0, 0), m, + &tau_matrix(batch, 0), &q_reshaped(batch, 0, 0), m, + dev_info.back().mutable_data() + batch), + done); + } + if (Eigen::NumTraits::IsComplex) { + functor::UnaryFunctor> conj; + conj(device, q->flat() /*out*/, + const_cast(q)->flat() /*in*/); + } + } else { + // Generate m x n matrix Q. In this case we can use the more efficient + // algorithm in Ungqr to generate Q in place. + dev_info.push_back(solver->GetDeviceLapackInfo(batch_size, "orgqr")); + for (int batch = 0; batch < batch_size; ++batch) { + OP_REQUIRES_OK_ASYNC( + context, + solver->Ungqr( + m, n, min_size, &input_transposed_reshaped(batch, 0, 0), m, + &tau_matrix(batch, 0), dev_info.back().mutable_data() + batch), + done); + } + OP_REQUIRES_OK_ASYNC( + context, DoMatrixTranspose(device, input_transposed, q), done); + } + + // Asynchronously check return status from cuSolver kernels. + CudaSolver::CheckLapackInfoAndDeleteSolverAsync(std::move(solver), dev_info, + std::move(done)); + } + + private: + bool full_matrices_; + + TF_DISALLOW_COPY_AND_ASSIGN(QrOpGpu); +}; + +#endif // GOOGLE_CUDA + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_QR_OP_IMPL_H_ diff --git a/linux/include/tensorflow/core/kernels/quantization_utils.h b/linux/include/tensorflow/core/kernels/quantization_utils.h new file mode 100644 index 000000000..e67a94e5f --- /dev/null +++ b/linux/include/tensorflow/core/kernels/quantization_utils.h @@ -0,0 +1,959 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_QUANTIZATION_UTILS_H_ +#define TENSORFLOW_CORE_KERNELS_QUANTIZATION_UTILS_H_ + +#define EIGEN_USE_THREADS + +// This is a set of functions that standardizes how quantized values are +// interpreted as float numbers. +// All of the current implementations are for reference and have not been +// optimized. They should be implementable using fixed point representations +// to avoid a dependency on floating-point hardware. + +#if defined(__ARM_NEON__) || defined(__ARM_NEON) +#define QUANTIZATION_UTILS_USE_NEON +#include +#endif + +#include + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#define GEMMLOWP_ALLOW_SLOW_SCALAR_FALLBACK +#include "public/gemmlowp.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/lib/core/threadpool.h" + +namespace tensorflow { + +// We have to be able to detect and handle overflows in int32, so this function +// uses doubles and int64's to make sure we have enough room. +template +int64 FloatToQuantizedUnclamped(float input, float range_min, float range_max) { + const int64 lowest_quantized = + static_cast(Eigen::NumTraits::lowest()); + if (range_min == range_max) { + return lowest_quantized; + } + const int number_of_bits = sizeof(T) * 8; + const int64 number_of_steps = static_cast(1) << number_of_bits; + const double range_adjust = (number_of_steps / (number_of_steps - 1.0)); + const double range = ((range_max - range_min) * range_adjust); + const double range_scale = (number_of_steps / range); + int64 quantized = + (round(input * range_scale) - round(range_min * range_scale)); + quantized += lowest_quantized; + return quantized; +} + +// This converts the float into the final quantized type, clamping/saturating +// any over or underflows. +template +T FloatToQuantized(float input, float range_min, float range_max) { + if (std::is_same::value) { + // Specialization for float. This is used in reference implementation + // for float which is useful to compare performance between float + // and quantized type. + return input; + } + int64 quantized = FloatToQuantizedUnclamped(input, range_min, range_max); + const int64 lowest_quantized = + static_cast(Eigen::NumTraits::lowest()); + const int64 highest_quantized = + static_cast(Eigen::NumTraits::highest()); + quantized = std::max(quantized, lowest_quantized); + quantized = std::min(quantized, highest_quantized); + return static_cast(static_cast(quantized)); +} + +template +float QuantizedToFloat(T input, float range_min, float range_max) { + if (std::is_same::value) { + // Specialization for float. This is used in reference implementation + // for float which is useful to compare performance between float + // and quantized type. + return input; + } + if (range_min == range_max) { + return range_min; + } + const int number_of_bits = sizeof(T) * 8; + const int64 number_of_steps = static_cast(1) << number_of_bits; + const double range_adjust = (number_of_steps / (number_of_steps - 1.0)); + const double range = ((range_max - range_min) * range_adjust); + const double range_scale = (range / number_of_steps); + const int64 lowest_quantized = + static_cast(Eigen::NumTraits::lowest()); + const double offset_input = static_cast(input) - lowest_quantized; + // For compatibility with DEQUANTIZE_WITH_EIGEN, we should convert + // range_scale to a float, otherwise range_min_rounded might be slightly + // different. + const double range_min_rounded = + round(range_min / static_cast(range_scale)) * + static_cast(range_scale); + const double result = range_min_rounded + (offset_input * range_scale); + return static_cast(result); +} + +template +float FloatForOneQuantizedLevel(float range_min, float range_max) { + const int64 highest = static_cast(Eigen::NumTraits::highest()); + const int64 lowest = static_cast(Eigen::NumTraits::lowest()); + const float float_for_one_quantized_level = + (range_max - range_min) / (highest - lowest); + return float_for_one_quantized_level; +} + +template +void QuantizationRangeForMultiplication(float min_a, float max_a, float min_b, + float max_b, float* min_c, + float* max_c) { + const float a_float_for_one_quant_level = + FloatForOneQuantizedLevel(min_a, max_a); + const float b_float_for_one_quant_level = + FloatForOneQuantizedLevel(min_b, max_b); + + const int64 c_highest = static_cast(Eigen::NumTraits::highest()); + const int64 c_lowest = static_cast(Eigen::NumTraits::lowest()); + const float c_float_for_one_quant_level = + a_float_for_one_quant_level * b_float_for_one_quant_level; + + *min_c = c_float_for_one_quant_level * c_lowest; + *max_c = c_float_for_one_quant_level * c_highest; +} + +// input_array is an eigen Tensor. q2f is a QuantizedToFloatStruct. +// This evaluates to an eigen tensor expression, to be used like: +// auto tensor = DEQUANTIZE_WITH_EIGEN(input_tensor, q2f); +#define DEQUANTIZE_WITH_EIGEN(input_array, q2f) \ + ((q2f.range_min_rounded - q2f.lowest_quantized() * q2f.range_scale) + \ + input_array.template cast() * q2f.range_scale) + +// input_array is an eigen Tensor. f2q is a FloatToQuantizedStruct. +// OutputType is the type of output (e.g. quint8). +// This evaluates to an eigen tensor expression, to be used like: +// auto tensor = QUANTIZE_WITH_EIGEN(input_tensor, f2q, T); +#define QUANTIZE_WITH_EIGEN(input_array, f2q, OutputType) \ + ((input_array * f2q.range_scale).round() - \ + (f2q.range_min_scaled - f2q.lowest_quantized())) \ + .cwiseMax(f2q.lower_bound_float()) \ + .cwiseMin(f2q.upper_bound_float()) \ + .template cast() \ + .template cast() + +// For use with DEQUANTIZE_WITH_EIGEN. +template +struct QuantizedToFloatStruct { + static constexpr int number_of_bits = sizeof(T) * 8; + static constexpr int64 number_of_steps = static_cast(1) + << number_of_bits; + + static float lowest_quantized() { + return static_cast(Eigen::NumTraits::lowest()); + } + + QuantizedToFloatStruct(float range_min, float range_max) + : range_min(range_min), + range_scale((range_max - range_min) / (number_of_steps - 1.0)), + range_min_rounded(range_max == range_min + ? range_min + : round(range_min / range_scale) * range_scale) {} + + const float range_min; + const float range_scale; + const float range_min_rounded; +}; + +// For use with QUANTIZE_WITH_EIGEN. +template +struct FloatToQuantizedStruct { + static constexpr int number_of_bits = sizeof(T) * 8; + static constexpr int64 number_of_steps = static_cast(1) + << number_of_bits; + static constexpr double range_adjust = + (number_of_steps / (number_of_steps - 1.0)); + + // Casting QInt32's lowest or highest to a float gives a float that can't be + // cast back to int32 or QInt32. Instead, use bounds that can be converted + // back to int32 without going outside the range of an int32. + static float lower_bound_float() { + return Eigen::numext::maxi( + static_cast(Eigen::NumTraits::lowest()), -2.147483648e+09f); + } + static float upper_bound_float() { + return Eigen::numext::mini( + static_cast(Eigen::NumTraits::highest()), +2.147483520e+09f); + } + + static float lowest_quantized() { + return static_cast(Eigen::NumTraits::lowest()); + } + + FloatToQuantizedStruct(float range_min, float range_max) + : range_min(range_min), + range_scale(range_max == range_min + ? 0.0 + : (number_of_steps - 1.0) / (range_max - range_min)), + range_min_scaled(round(range_min * range_scale)) {} + + const float range_min; + const float range_scale; + const float range_min_scaled; +}; + +template +inline T2 RequantizeInNewRange(T1 input, float min_input, float max_input, + float min_new, float max_new) { + const float input_float = QuantizedToFloat(input, min_input, max_input); + return FloatToQuantized(input_float, min_new, max_new); +} + +template +inline void RequantizeManyInNewRange(const T1* input, int64 count, + float min_input, float max_input, + float min_output, float max_output, + T2* output) { + for (size_t index = 0; index < count; ++index) { + const float input_float = + QuantizedToFloat(input[index], min_input, max_input); + output[index] = FloatToQuantized(input_float, min_output, max_output); + } +} + +// Because converting 32-bit accumulated results down to eight bit is a common +// case, we have a specialized code path to handle it as efficiently as +// possible using only fixed-point math for the inner loop. +inline void RequantizeManyInNewRangeReference(const qint32* input, int64 count, + float min_input, float max_input, + float min_output, + float max_output, + quint8* output) { + // Initially we calculate all the constants we need once, before we go into + // the inner loop. If this is updated, also update the Eigen version. + const int fp_shift = 16; + const float input_range = max_input - min_input; + const float output_range = max_output - min_output; + const float recip_output_range = + output_range == 0.0 ? 0.0 : (255.0 / output_range); + const float input_rezero = (min_input + max_input) / 2.0; + const int64 range_scale_fp = + output_range == 0.0 ? 0.0 + : static_cast(255.0 * (1 << fp_shift) * + input_range / output_range); + const int64 input_offset_fp = + static_cast(input_rezero * recip_output_range * (1 << fp_shift)); + const int64 output_offset_fp = + output_range == 0.0 + ? 0 + : static_cast((1 << fp_shift) * (min_output * 255.0) / + output_range); + const int64 rounding_delta = 1 << (fp_shift - 1); + + // Inside this loop we just do minimal adds, multiplies, and shifts, in a way + // that could be easily adapted for a SIMD implementation. It should also be + // possible to perform all the calculations in 32-bit rather than 64, but + // that's not been implemented yet. + for (size_t index = 0; index < count; ++index) { + const int64 input_value = static_cast(input[index]); + const int64 fp_value = + ((input_value * range_scale_fp) >> 32) + input_offset_fp; + const int64 offset_intermediate = fp_value - output_offset_fp; + const int64 round_intermediate = offset_intermediate + rounding_delta; + int64 quantized_int64 = round_intermediate >> fp_shift; + quantized_int64 = std::max(quantized_int64, int64{0}); + quantized_int64 = std::min(quantized_int64, int64{255}); + output[index] = static_cast(static_cast(quantized_int64)); + } +} + +// Another common case is converting eight bit inputs up to thirty two bits, so +// we have specialized fixed-point code to accelerate that. There is also a NEON +// version for ARM devices below. +inline void RequantizeManyInNewRange8To32BitReference( + const quint8* input, int64 count, float min_input, float max_input, + float min_output, float max_output, qint32* output) { + const float code_0_float = QuantizedToFloat(0, min_input, max_input); + const float code_1_float = QuantizedToFloat(1, min_input, max_input); + const int64 code_0_int64 = + FloatToQuantizedUnclamped(code_0_float, min_output, max_output); + const int64 code_1_int64 = + FloatToQuantizedUnclamped(code_1_float, min_output, max_output); + const int32 mult_int32 = code_1_int64 - code_0_int64; + const int64 lowest_quantized = + static_cast(Eigen::NumTraits::lowest()); + const int64 highest_quantized = + static_cast(Eigen::NumTraits::highest()); + for (int64 i = 0; i < count; ++i) { + const int64 input_value = static_cast(input[i]); + int64 output_value = code_0_int64 + (input_value * mult_int32); + output_value = std::max(output_value, lowest_quantized); + output_value = std::min(output_value, highest_quantized); + output[i] = static_cast(output_value); + } +} + +#ifdef QUANTIZATION_UTILS_USE_NEON +// Speeds up the 32->8bit conversion using fixed-point arithmetic and NEON SIMD +// intrinsics for ARM platforms. +inline void RequantizeManyInNewRangeNeon(const qint32* input, int64 count, + float min_input, float max_input, + float min_output, float max_output, + quint8* output) { + // Initially we calculate all the constants we need once, before we go into + // the inner loop. If this is updated, also update the Eigen version. + const int fp_shift = 16; + + // Calculate range variables in advance. + // Input range. + const float input_range = max_input - min_input; + // Output range. + const float output_range = max_output - min_output; + // Ratio of output range. + const float recip_output_range = + output_range == 0.0 ? 0.0 : (255.0 / output_range); + // Average of input range as zero position of input. + const float input_rezero = (min_input + max_input) / 2.0; + // In-out range scale. + const int32 range_scale_fp = + output_range == 0.0 ? 0.0 + : static_cast(255.0 * (1 << (fp_shift - 16)) * + input_range / output_range); + // Input zero position offset to output. + const int32 input_offset_fp = + static_cast(input_rezero * recip_output_range * (1 << fp_shift)); + // Output min offset. + const int32 output_offset_fp = + output_range == 0.0 + ? 0 + : static_cast((1 << fp_shift) * (min_output * 255.0) / + output_range); + const int32 rounding_delta = 1 << (fp_shift - 1); + + // broadcast range to each lane + const int32x4_t range_scale_fp_32x4 = vmovq_n_s32(range_scale_fp); + const int32x4_t input_offset_fp_32x4 = vmovq_n_s32(input_offset_fp); + const int32x4_t output_offset_fp_32x4 = vmovq_n_s32(output_offset_fp); + const int32x4_t rounding_delta_32x4 = vmovq_n_s32(rounding_delta); + + int64 index = 0; + // Use SIMD to requantize. + for (; index < (count - 7); index += 8) { + const int32* input_ptr = &(input->value) + index; + const int32x4_t input_value_low_32x4 = vld1q_s32(input_ptr); + const int32x4_t input_value_high_32x4 = vld1q_s32(input_ptr + 4); + const int32x4_t fp_value_low_32x4 = vaddq_s32( + input_offset_fp_32x4, + vmulq_s32(vshrq_n_s32(input_value_low_32x4, 16), range_scale_fp_32x4)); + const int32x4_t fp_value_high_32x4 = vaddq_s32( + input_offset_fp_32x4, + vmulq_s32(vshrq_n_s32(input_value_high_32x4, 16), range_scale_fp_32x4)); + const int32x4_t offset_intermediate_low_32x4 = + vsubq_s32(fp_value_low_32x4, output_offset_fp_32x4); + const int32x4_t offset_intermediate_high_32x4 = + vsubq_s32(fp_value_high_32x4, output_offset_fp_32x4); + const int32x4_t round_intermediate_low_32x4 = + vaddq_s32(offset_intermediate_low_32x4, rounding_delta_32x4); + const int32x4_t round_intermediate_high_32x4 = + vaddq_s32(offset_intermediate_high_32x4, rounding_delta_32x4); + const int16x4_t quantized_low_16x4 = + vqmovn_s32(vshrq_n_s32(round_intermediate_low_32x4, fp_shift)); + const int16x4_t quantized_high_16x4 = + vqmovn_s32(vshrq_n_s32(round_intermediate_high_32x4, fp_shift)); + const uint8x8_t quantized_8x8 = + vqmovun_s16(vcombine_s16(quantized_low_16x4, quantized_high_16x4)); + uint8* output_ptr = &(output->value) + index; + vst1_u8(output_ptr, quantized_8x8); + } + + // Requantize remaining elements in array without SIMD. + for (; index < count; ++index) { + const int32 input_value = static_cast(input[index]); + const int32 fp_value = + static_cast( + (static_cast(input_value >> 16) * (range_scale_fp))) + + input_offset_fp; + const int32 offset_intermediate = fp_value - output_offset_fp; + const int32 round_intermediate = offset_intermediate + rounding_delta; + int32 quantized_int32 = round_intermediate >> fp_shift; + quantized_int32 = std::max(quantized_int32, 0); + quantized_int32 = std::min(quantized_int32, 255); + output[index] = static_cast(static_cast(quantized_int32)); + } +} + +template <> +inline void RequantizeManyInNewRange( + const qint32* input, int64 count, float min_input, float max_input, + float min_output, float max_output, quint8* output) { + const float input_range = max_input - min_input; + const float output_range = max_output - min_output; + if ((input_range / output_range) > 16384.0f) { + // Our NEON implementation uses 32-bit math and can't handle very + // large ranges, so fall back to the reference implementation. We don't + // expect these to be common in models, so this shouldn't be a performance + // problem in practice. + RequantizeManyInNewRangeReference(input, count, min_input, max_input, + min_output, max_output, output); + } else { + RequantizeManyInNewRangeNeon(input, count, min_input, max_input, min_output, + max_output, output); + } +} + +// NEON accelerated 16bit rounded division by 2^n. +template +inline int16x8_t Divide16x8PowRound(const int16x8_t val) { + const int16x8_t val_sign = vshrq_n_s16(val, 15); + const int16x8_t val_xor = veorq_s16(val, val_sign); + const int16x8_t val_pos = vsubq_s16(val_xor, val_sign); + const int16x8_t shifted_val_pos = vrshrq_n_s16(val_pos, POW); + const int16x8_t shifted_val_pos_xor = veorq_s16(shifted_val_pos, val_sign); + const int16x8_t shifted_val = vsubq_s16(shifted_val_pos_xor, val_sign); + return shifted_val; +} + +// NEON accelerated 64bit rounded division by 2^n. +template +inline int64x2_t Divide64x2PowRound(const int64x2_t val) { + const int64x2_t val_sign = vshrq_n_s64(val, 63); + const int64x2_t val_xor = veorq_s64(val, val_sign); + const int64x2_t val_pos = vsubq_s64(val_xor, val_sign); + const int64x2_t shifted_val_pos = vrshrq_n_s64(val_pos, POW); + const int64x2_t shifted_val_pos_xor = veorq_s64(shifted_val_pos, val_sign); + const int64x2_t shifted_val = vsubq_s64(shifted_val_pos_xor, val_sign); + return shifted_val; +} + +// NEON accelerated 16bit division by 2^n. +// CAVEAT: The input must be greater than min-int16 to avoid underflow. +template +inline int16x8_t Divide16x8Pow(const int16x8_t val) { + static constexpr int16 FIRST_BIT_VAL = 0x0000000000000001; + static const int16x8_t FIRST_BIT = vmovq_n_s16(FIRST_BIT_VAL); + const int16x8_t val_sign = vshrq_n_s16(val, 15); + const int16x8_t neg_offset = vandq_s16(val_sign, FIRST_BIT); + const int16x8_t val_with_offset = vsubq_s16(val, neg_offset); + const int16x8_t shifted_wo_offset = + vsraq_n_s16(neg_offset, val_with_offset, POW); + return shifted_wo_offset; +} + +// NEON accelerated 64bit division by 2^n. +// CAVEAT: The input must be greater than min-int64 to avoid underflow. +template +inline int64x2_t Divide64x2Pow(const int64x2_t val) { + static constexpr int64 FIRST_BIT_VAL = 0x0000000000000001; + static const int64x2_t FIRST_BIT = vmovq_n_s64(FIRST_BIT_VAL); + const int64x2_t val_sign = vshrq_n_s64(val, 63); + const int64x2_t neg_offset = vandq_s64(val_sign, FIRST_BIT); + const int64x2_t val_with_offset = vsubq_s64(val, neg_offset); + const int64x2_t shifted_wo_offset = + vsraq_n_s64(neg_offset, val_with_offset, POW); + return shifted_wo_offset; +} + +// 32bit x 2 NEON accelerated lerp computation. +template +inline int32x2_t ComputeLerp32x2(const int32x2_t top_left, + const int32x2_t top_right, + const int32x2_t bottom_left, + const int32x2_t bottom_right, + const int32x2_t x_lerp, + const int32x2_t y_lerp) { + static_assert(RESOLUTION < 31, "RESOLUTION must be less than 31"); + constexpr int32 RESOLUTION_MULT32 = (1 << RESOLUTION); + static const int32x2_t RESOLUTION_MULT32x2 = vmov_n_s32(RESOLUTION_MULT32); + + const int64x2_t top_left_x_res = vmull_s32(top_left, RESOLUTION_MULT32x2); + const int64x2_t bottom_left_x_res = + vmull_s32(bottom_left, RESOLUTION_MULT32x2); + + const int32x2_t top_right_sub_top_left = vsub_s32(top_right, top_left); + const int64x2_t top_x_res = + vmlal_s32(top_left_x_res, top_right_sub_top_left, x_lerp); + const int32x2_t bottom_right_sub_bottom_left = + vsub_s32(bottom_right, bottom_left); + const int64x2_t bottom_x_res = + vmlal_s32(bottom_left_x_res, bottom_right_sub_bottom_left, x_lerp); + + const int64x2_t bottom_sub_top_x_res = vsubq_s64(bottom_x_res, top_x_res); + const int64x2_t bottom_sub_top = + Divide64x2Pow(bottom_sub_top_x_res); + const int32x2_t bottom_sub_top_32 = vqmovn_s64(bottom_sub_top); + const int64x2_t top_add_bottom_sub_top_mul_ylerp_x_res = + vmlal_s32(top_x_res, bottom_sub_top_32, y_lerp); + const int64x2_t retval = + Divide64x2PowRound(top_add_bottom_sub_top_mul_ylerp_x_res); + const int32x2_t retval32 = vqmovn_s64(retval); + return retval32; +} + +// 8bit x 8 NEON accelerated lerp computation. +template +inline uint8x8_t ComputeLerp8x8(const uint8x8_t top_left8x8, + const uint8x8_t top_right8x8, + const uint8x8_t bottom_left8x8, + const uint8x8_t bottom_right8x8, + const int16x8_t x_lerp, + const int16x8_t y_lerp) { + static_assert(RESOLUTION < 8, "RESOLUTION must be less than 8"); + constexpr uint8 RESOLUTION_MULT_VAL = (1 << RESOLUTION); + static const uint8x8_t RESOLUTION_MULT = vdup_n_u8(RESOLUTION_MULT_VAL); + + const int16x8_t top_left_x_res = + vreinterpretq_s16_u16(vmull_u8(top_left8x8, RESOLUTION_MULT)); + const int16x8_t bottom_left_x_res = + vreinterpretq_s16_u16(vmull_u8(bottom_left8x8, RESOLUTION_MULT)); + + const int16x8_t top_right_sub_top_left = + vreinterpretq_s16_u16(vsubl_u8(top_right8x8, top_left8x8)); + const int16x8_t top_x_res = + vmlaq_s16(top_left_x_res, top_right_sub_top_left, x_lerp); + + const int16x8_t bottom_right_sub_bottom_left = + vreinterpretq_s16_u16(vsubl_u8(bottom_right8x8, bottom_left8x8)); + const int16x8_t bottom_x_res = + vmlaq_s16(bottom_left_x_res, bottom_right_sub_bottom_left, x_lerp); + + const int16x8_t bottom_sub_top_x_res = vsubq_s16(bottom_x_res, top_x_res); + const int16x8_t bottom_sub_top = + Divide16x8Pow(bottom_sub_top_x_res); + const int16x8_t top_add_bottom_sub_top_mul_ylerp_x_res = + vmlaq_s16(top_x_res, bottom_sub_top, y_lerp); + const int16x8_t retval16 = + Divide16x8PowRound(top_add_bottom_sub_top_mul_ylerp_x_res); + const uint8x8_t retval = vmovn_u16(vreinterpretq_u16_s16(retval16)); + return retval; +} + +// Requantize 8 x 8 quints to 8 x 32 qints in parallel by neon +// Return std::array instead of pointer to leverage return value optimization +inline std::array Requantize8x8To32Neon( + const uint8* input_ptr, const int64x2_t input_0_64x2, + const int32x2_t input_mult_32x2) { + const uint8x8_t input_value_8x8 = vld1_u8(input_ptr); + const int16x8_t input_value_16x8 = + vreinterpretq_s16_u16(vmovl_u8(input_value_8x8)); + const int16x4_t input_value_low_16x4 = vget_low_s16(input_value_16x8); + const int16x4_t input_value_high_16x4 = vget_high_s16(input_value_16x8); + const int32x4_t input_value_low_32x4 = vmovl_s16(input_value_low_16x4); + const int32x4_t input_value_high_32x4 = vmovl_s16(input_value_high_16x4); + const int32x2_t input_value_low_low_32x2 = vget_low_s32(input_value_low_32x4); + const int32x2_t input_value_low_high_32x2 = + vget_high_s32(input_value_low_32x4); + const int32x2_t input_value_high_low_32x2 = + vget_low_s32(input_value_high_32x4); + const int32x2_t input_value_high_high_32x2 = + vget_high_s32(input_value_high_32x4); + const int64x2_t mult_result_low_low_64x2 = + vmlal_s32(input_0_64x2, input_value_low_low_32x2, input_mult_32x2); + const int64x2_t mult_result_low_high_64x2 = + vmlal_s32(input_0_64x2, input_value_low_high_32x2, input_mult_32x2); + const int64x2_t mult_result_high_low_64x2 = + vmlal_s32(input_0_64x2, input_value_high_low_32x2, input_mult_32x2); + const int64x2_t mult_result_high_high_64x2 = + vmlal_s32(input_0_64x2, input_value_high_high_32x2, input_mult_32x2); + const int32x2_t output_value_low_low_32x2 = + vqmovn_s64(mult_result_low_low_64x2); + const int32x2_t output_value_low_high_32x2 = + vqmovn_s64(mult_result_low_high_64x2); + const int32x2_t output_value_high_low_32x2 = + vqmovn_s64(mult_result_high_low_64x2); + const int32x2_t output_value_high_high_32x2 = + vqmovn_s64(mult_result_high_high_64x2); + const int32x4_t output_value_low_32x4 = + vcombine_s32(output_value_low_low_32x2, output_value_low_high_32x2); + const int32x4_t output_value_high_32x4 = + vcombine_s32(output_value_high_low_32x2, output_value_high_high_32x2); + return std::array{ + {output_value_low_32x4, output_value_high_32x4}}; +} + +// Speeds up the 8->32bit conversion using fixed-point arithmetic and NEON SIMD +// intrinsics for ARM platforms. +template <> +inline void RequantizeManyInNewRange( + const quint8* input, int64 count, float min_input, float max_input, + float min_output, float max_output, qint32* output) { + // Pre-calculate zero position and multiplier. + // Calculate 0 and 1 value in float. + const float code_0_float = QuantizedToFloat(0, min_input, max_input); + const float code_1_float = QuantizedToFloat(1, min_input, max_input); + + // Cast 0 and 1 value in int64. + const int64 code_0_int64 = + FloatToQuantizedUnclamped(code_0_float, min_output, max_output); + const int64 code_1_int64 = + FloatToQuantizedUnclamped(code_1_float, min_output, max_output); + + // Calculate multiplier. + const int32 mult_int32 = static_cast(code_1_int64 - code_0_int64); + + // Broadcast 0 position and multiplier to lanes + const int64x2_t code_0_64x2 = vmovq_n_s64(code_0_int64); + const int32x2_t mult_32x2 = vmov_n_s32(mult_int32); + + int64 i = 0; + + // Use SIMD to requantize array. + for (; i < (count - 7); i += 8) { + const uint8* input_ptr = &(input->value) + i; + int32* output_ptr = &(output->value) + i; + const std::array output_value = + Requantize8x8To32Neon(input_ptr, code_0_64x2, mult_32x2); + vst1q_s32(output_ptr + 0, output_value[0]); + vst1q_s32(output_ptr + 4, output_value[1]); + } + + // Requantize remaining elements in array without SIMD. + const int64 lowest_quantized = + static_cast(Eigen::NumTraits::lowest()); + const int64 highest_quantized = + static_cast(Eigen::NumTraits::highest()); + + for (; i < count; ++i) { + const int64 input_value = static_cast(input[i]); + int64 output_value = code_0_int64 + (input_value * mult_int32); + output_value = std::max(output_value, lowest_quantized); + output_value = std::min(output_value, highest_quantized); + output[i] = static_cast(output_value); + } +} + +#else + +// If SIMD implementations aren't available, then use these default reference +// versions. +template <> +inline void RequantizeManyInNewRange( + const qint32* input, int64 count, float min_input, float max_input, + float min_output, float max_output, quint8* output) { + RequantizeManyInNewRangeReference(input, count, min_input, max_input, + min_output, max_output, output); +} + +template <> +inline void RequantizeManyInNewRange( + const quint8* input, int64 count, float min_input, float max_input, + float min_output, float max_output, qint32* output) { + RequantizeManyInNewRange8To32BitReference(input, count, min_input, max_input, + min_output, max_output, output); +} + +#endif + +template +struct int64_right_shift_op { + EIGEN_EMPTY_STRUCT_CTOR(int64_right_shift_op) + EIGEN_DEVICE_FUNC + EIGEN_STRONG_INLINE const int64 operator()(const int64& a) const { + return a >> shift; + } +}; + +// See RequantizeManyInNewRange() for a non-eigen reference implementation. +template +inline void RequantizeManyInNewRangeUsingEigen( + const Eigen::ThreadPoolDevice& device, const Tensor& input, float min_input, + float max_input, float min_output, float max_output, Tensor* output) { + auto input_array = input.flat(); + QuantizedToFloatStruct q2f(min_input, max_input); + auto input_float = DEQUANTIZE_WITH_EIGEN(input_array, q2f); + FloatToQuantizedStruct f2q(min_output, max_output); + auto input_requantized = QUANTIZE_WITH_EIGEN(input_float, f2q, T2); + + output->flat().device(device) = input_requantized; +} + +// See RequantizeManyInNewRange() for a non-eigen reference implementation. +// +// Because converting 32-bit accumulated results down to eight bit is a common +// case, we have a specialized code path to handle it as efficiently as +// possible using only fixed-point math for the inner loop. +template <> +inline void RequantizeManyInNewRangeUsingEigen( + const Eigen::ThreadPoolDevice& device, const Tensor& input, float min_input, + float max_input, float min_output, float max_output, Tensor* output) { + // Initially we calculate all the constants we need once, before we go into + // the inner loop. If this is updated, also update the non-Eigen version. + const int fp_shift = 16; + const float input_range = max_input - min_input; + const float output_range = max_output - min_output; + const float recip_output_range = + output_range == 0.0 ? 0.0 : (255.0 / output_range); + const float input_rezero = (min_input + max_input) / 2.0; + const int64 range_scale_fp = + output_range == 0.0 ? 0.0 + : static_cast(255.0 * (1 << fp_shift) * + input_range / output_range); + const int64 input_offset_fp = + static_cast(input_rezero * recip_output_range * (1 << fp_shift)); + const int64 output_offset_fp = + output_range == 0.0 + ? 0 + : static_cast((1 << fp_shift) * (min_output * 255.0) / + output_range); + const int64 rounding_delta = 1 << (fp_shift - 1); + + // Inside this eigen expression we just do minimal adds, multiplies, and + // shifts. It should be possible to perform all the calculations in 32-bit + // rather than 64, but that's not been implemented yet. + auto input_array = input.flat(); + auto fp_value = ((input_array.template cast() * range_scale_fp) + .unaryExpr(int64_right_shift_op<32>())) + + (input_offset_fp - output_offset_fp + rounding_delta); + auto intermediate = fp_value.unaryExpr(int64_right_shift_op()); + auto input_requantized = intermediate.cwiseMax(0LL) + .cwiseMin(255LL) + .template cast() + .template cast(); + output->flat().device(device) = input_requantized; +} + +// REQUIRES: 'result->NumElements() == input.NumElements()' +template +void FloatTensorToQuantizedInPlaceUsingEigen( + const Eigen::ThreadPoolDevice& device, const Tensor& input, float min, + float max, Tensor* result) { + DCHECK_EQ(DataTypeToEnum::v(), result->dtype()); + auto flat_input = input.flat(); + auto flat_result = result->flat(); + DCHECK_EQ(flat_input.size(), flat_result.size()); + + FloatToQuantizedStruct f2q(min, max); + flat_result.device(device) = QUANTIZE_WITH_EIGEN(flat_input, f2q, T); +} + +template +void FloatTensorToQuantizedInPlace(const Tensor& input, float min, float max, + Tensor* result) { + DCHECK_EQ(DataTypeToEnum::v(), result->dtype()); + auto flat_input = input.flat(); + auto flat_result = result->flat(); + const int data_size = flat_input.size(); + DCHECK(data_size == flat_result.size()); + for (int i = 0; i < data_size; ++i) { + flat_result(i) = FloatToQuantized(flat_input(i), min, max); + } +} + +template +Tensor FloatTensorToQuantized(const Tensor& input, float min, float max) { + Tensor result(DataTypeToEnum::v(), input.shape()); + FloatTensorToQuantizedInPlace(input, min, max, &result); + return result; +} + +// REQUIRES: 'result->NumElements() == input.NumElements()' +template +void QuantizedTensorToFloatInPlaceUsingEigen( + const Eigen::ThreadPoolDevice& device, const Tensor& input, float min, + float max, Tensor* result) { + DCHECK_EQ(DataTypeToEnum::v(), input.dtype()); + auto flat_input = input.flat(); + auto flat_result = result->flat(); + const int data_size = flat_input.size(); + DCHECK(data_size == flat_result.size()); + + QuantizedToFloatStruct q2f(min, max); + flat_result.device(device) = DEQUANTIZE_WITH_EIGEN(flat_input, q2f); +} + +// REQUIRES: 'result->NumElements() == input.NumElements()' +template +void QuantizedTensorToFloatInPlace(const Tensor& input, float min, float max, + Tensor* result) { + DCHECK_EQ(DataTypeToEnum::v(), input.dtype()); + auto flat_input = input.flat(); + auto flat_result = result->flat(); + const int data_size = flat_input.size(); + DCHECK(data_size == flat_result.size()); + for (int i = 0; i < data_size; ++i) { + flat_result(i) = QuantizedToFloat(flat_input(i), min, max); + } +} + +template +Tensor QuantizedTensorToFloat(const Tensor& input, float min, float max) { + Tensor result(DT_FLOAT, input.shape()); + QuantizedTensorToFloatInPlace(input, min, max, &result); + return result; +} + +void GetOutputMinAndMaxForQuantizedAdd(float input_min, float input_max, + float smaller_input_min, + float smaller_input_max, + float* output_min, float* output_max); + +// Add and . If has fewer elements than +// , then it is broadcast onto . +template +void QuantizedAddUsingEigen(const Eigen::ThreadPoolDevice& device, + const Tensor& input, float input_min, + float input_max, const Tensor& smaller_input, + float smaller_input_min, float smaller_input_max, + Tensor* output, float* output_min, + float* output_max) { + const auto& input_flat = input.flat(); + const auto& smaller_input_flat = smaller_input.flat(); + auto output_flat = output->flat(); + + GetOutputMinAndMaxForQuantizedAdd(input_min, input_max, smaller_input_min, + smaller_input_max, output_min, output_max); + // To do addition properly, we need to compensate for a possibly unbalanced + // zero point in the total representation. The quantized value that + // represents the real number zero needs to be subtracted before addition to + // make sure that the identity of zero + zero = zero holds. + const T3 zero_in_total_space = + FloatToQuantized(0.0f, *output_min, *output_max); + + const int64 input_element_count = input.NumElements(); + const int64 smaller_input_element_count = smaller_input.NumElements(); + + QuantizedToFloatStruct input_q2f(input_min, input_max); + QuantizedToFloatStruct smaller_input_q2f(smaller_input_min, + smaller_input_max); + FloatToQuantizedStruct f2q(*output_min, *output_max); + + auto smaller_input_float = + DEQUANTIZE_WITH_EIGEN(smaller_input_flat, smaller_input_q2f); + auto smaller_input_in_total_space = + QUANTIZE_WITH_EIGEN(smaller_input_float, f2q, T3); + + auto input_float = DEQUANTIZE_WITH_EIGEN(input_flat, input_q2f); + auto input_in_total_space = QUANTIZE_WITH_EIGEN(input_float, f2q, T3); + + Eigen::array bcast; + bcast[0] = input_element_count / smaller_input_element_count; + output_flat.device(device) = + input_in_total_space + + (smaller_input_in_total_space.broadcast(bcast) + zero_in_total_space); +} + +// This is a reference implementation of the bias addition for quantized +// buffers, designed to provide a clear specification for the result we +// want. We'll want to specialize this for particular hardware, and +// probably even fuse it with matrix multiplications in a lot of cases. It's +// important to show the clamping behavior we want in particular. +template +void QuantizedAdd(const Eigen::ThreadPoolDevice& device, const Tensor& input, + float input_min, float input_max, const Tensor& smaller_input, + float smaller_input_min, float smaller_input_max, + Tensor* output, float* output_min, float* output_max) { + const auto& input_flat = input.flat(); + const auto& smaller_input_flat = smaller_input.flat(); + auto output_flat = output->flat(); + + GetOutputMinAndMaxForQuantizedAdd(input_min, input_max, smaller_input_min, + smaller_input_max, output_min, output_max); + // To do addition properly, we need to compensate for a possibly unbalanced + // zero point in the total representation. The quantized value that + // represents the real number zero needs to be subtracted before addition to + // make sure that the identity of zero + zero = zero holds. + const T3 zero_in_total_space = + FloatToQuantized(0.0f, *output_min, *output_max); + + const int64 input_element_count = input.NumElements(); + const int64 smaller_input_element_count = smaller_input.NumElements(); + + float total_min = *output_min; + float total_max = *output_max; + const size_t how_many_iterations = + (input_element_count / smaller_input_element_count); + for (size_t iteration = 0; iteration < how_many_iterations; ++iteration) { + const size_t offset = iteration * smaller_input_element_count; + for (int c = 0; c < smaller_input_element_count; ++c) { + const int index = (offset + c); + // The two numbers we're going to add can each be in very different + // ranges (e.g. the quantized value '127' may represent very different + // real numbers in both) so we need to convert them to a common range + // before we sum them. + const T1 input_value = input_flat(index); + const T3 input_in_total_space = RequantizeInNewRange( + input_value, input_min, input_max, total_min, total_max); + const T2 smaller_input_value = smaller_input_flat(c); + const T3 smaller_input_in_total_space = + RequantizeInNewRange(smaller_input_value, smaller_input_min, + smaller_input_max, total_min, total_max); + const T3 total_pre = input_in_total_space + smaller_input_in_total_space; + // As noted above, we need to compensate for the offset of the actual + // zero point in the space we're operating in. + const T3 total = total_pre + zero_in_total_space; + output_flat(index) = total; + } + } +} + +// See gemmlowp/internal/multi_thread_gemm.h for the semantics of Execute. +class TensorflowGemmlowpWorkersPool { + public: + TensorflowGemmlowpWorkersPool(thread::ThreadPool* workers) + : workers_(workers) {} + + ~TensorflowGemmlowpWorkersPool() { + // This workaround ensures that all worker tasks have exited methods in the + // BlockingCounter. Without this, there is a race where the context is torn + // down while the counter is in use. + counter_to_decrement_when_ready_.Reset(0); + } + + void Execute(const std::vector& tasks) { + assert(!tasks.empty()); + assert(workers_ != nullptr); + counter_to_decrement_when_ready_.Reset(tasks.size()); + for (gemmlowp::Task* task : tasks) { + workers_->Schedule([this, task]() { + // TODO(cwhipkey): get a local_allocator from a thread local storage. + gemmlowp::Allocator local_allocator; + CHECK(task != nullptr); + task->local_allocator = &local_allocator; + task->Run(); + counter_to_decrement_when_ready_.DecrementCount(); + }); + } + counter_to_decrement_when_ready_.Wait(); + for (gemmlowp::Task* task : tasks) { + delete task; + } + } + + private: + thread::ThreadPool* const workers_; + + // The BlockingCounter used to wait for the workers. + gemmlowp::BlockingCounter counter_to_decrement_when_ready_; + + TF_DISALLOW_COPY_AND_ASSIGN(TensorflowGemmlowpWorkersPool); +}; + +class TensorflowGemmContext : public gemmlowp::MultiThreadGemmContextBase { + public: + TensorflowGemmContext(int num_threads, thread::ThreadPool* workers) + : workers_pool_(workers) { + set_max_num_threads(num_threads); + } + + TensorflowGemmlowpWorkersPool* workers_pool() { return &workers_pool_; } + + private: + TensorflowGemmlowpWorkersPool workers_pool_; + + TF_DISALLOW_COPY_AND_ASSIGN(TensorflowGemmContext); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_QUANTIZATION_UTILS_H_ diff --git a/linux/include/tensorflow/core/kernels/quantize_and_dequantize_op.h b/linux/include/tensorflow/core/kernels/quantize_and_dequantize_op.h new file mode 100644 index 000000000..6b0c5e5a4 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/quantize_and_dequantize_op.h @@ -0,0 +1,108 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_QUANTIZE_AND_DEQUANTIZE_OP_H_ +#define TENSORFLOW_CORE_KERNELS_QUANTIZE_AND_DEQUANTIZE_OP_H_ + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/kernels/cwise_ops.h" + +namespace tensorflow { +namespace functor { + +// TODO(pauldonnelly): 'signed_input' should really be called 'signed_output'. + +template +struct QuantizeAndDequantizeOneScaleFunctor { + void operator()(const Device& d, typename TTypes::ConstVec input, + bool signed_input, int num_bits, bool range_given, + Tensor* input_min_tensor, Tensor* input_max_tensor, + typename TTypes::Vec out); +}; + +// The implementation below runs on both CPU and GPU. +template +struct QuantizeAndDequantizeOneScaleImpl { + static void Compute(const Device& d, typename TTypes::ConstVec input, + bool signed_input, int num_bits, bool range_given, + Tensor* input_min_tensor, Tensor* input_max_tensor, + typename TTypes::Vec out) { + T min_range; + T max_range; + auto input_min = input_min_tensor->scalar(); + auto input_max = input_max_tensor->scalar(); + if (!range_given) { + input_min.device(d) = input.minimum(); + input_max.device(d) = input.maximum(); + d.memcpyDeviceToHost(&min_range, input_min.data(), sizeof(T)); + d.memcpyDeviceToHost(&max_range, input_max.data(), sizeof(T)); + } else { + // Copy the range values from their respective tensors on the host. + min_range = input_min_tensor->scalar()(); + max_range = input_max_tensor->scalar()(); + } + + // Calculate the range for the simulated integer quantization: + // e.g. [-128,127] for signed = true, num_bits = 8, + // or [0, 255] for signed = false, num_bits = 8. + const int64 min_quantized = signed_input ? -(1ULL << (num_bits - 1)) : 0; + const int64 max_quantized = min_quantized + ((1ULL << num_bits) - 1); + + // Determine the maximum scaling factor that would scale + // [min_range, max_range] to not exceed [min_quantized, max_quantized], + // while keeping 0 unchanged. + const T scale_from_min_side = (min_quantized * min_range > 0) + ? min_quantized / min_range + : std::numeric_limits::max(); + const T scale_from_max_side = (max_quantized * max_range > 0) + ? max_quantized / max_range + : std::numeric_limits::max(); + + // Note: Avoids changing the side of the range that determines scale. + T scale, inverse_scale; + if (scale_from_min_side < scale_from_max_side) { + scale = scale_from_min_side; + inverse_scale = min_range / min_quantized; + max_range = max_quantized * inverse_scale; + } else { + scale = scale_from_max_side; + inverse_scale = max_range / max_quantized; + min_range = min_quantized * inverse_scale; + } + + if (range_given) { + // Note: The clamping here is to avoid overflow in the quantized type. + // The semantics of the op does not guarantee to clamp to the specified + // min_range and max_range - because we may have changed either min_range + // or max_range. + out.device(d) = + (input.cwiseMin(max_range).cwiseMax(min_range) * scale) + .unaryExpr(Eigen::internal::scalar_round_op_google()) * + inverse_scale; + } else { + out.device(d) = + (input * scale) + .unaryExpr(Eigen::internal::scalar_round_op_google()) * + inverse_scale; + } + } +}; + +} // end of namespace functor +} // end of namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_QUANTIZE_AND_DEQUANTIZE_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/queue_base.h b/linux/include/tensorflow/core/kernels/queue_base.h new file mode 100644 index 000000000..272aa3b4f --- /dev/null +++ b/linux/include/tensorflow/core/kernels/queue_base.h @@ -0,0 +1,189 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_QUEUE_BASE_H_ +#define TENSORFLOW_CORE_KERNELS_QUEUE_BASE_H_ + +#include +#include + +#include "absl/base/macros.h" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/queue_interface.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/gtl/array_slice.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/mutex.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +namespace barrier { +class Barrier; +} // namespace barrier + +// Functionality common to asynchronous QueueInterface implementations. +class QueueBase : public QueueInterface { + public: + // As a possible value of 'capacity'. + static const int32 kUnbounded = INT_MAX; + + // Args: + // component_dtypes: The types of each component in a queue-element tuple. + // component_shapes: The shapes of each component in a queue-element tuple, + // which must either be empty (if the shapes are not specified) or + // or have the same size as component_dtypes. + // name: A name to use for the queue. + QueueBase(int32 capacity, const DataTypeVector& component_dtypes, + const std::vector& component_shapes, + const string& name); + + // Implementations of QueueInterface methods -------------------------------- + const DataTypeVector& component_dtypes() const override { + return component_dtypes_; + } + + Status ValidateTuple(const Tuple& tuple) override; + Status ValidateManyTuple(const Tuple& tuple) override; + + void Close(OpKernelContext* ctx, bool cancel_pending_enqueues, + DoneCallback callback) override; + + // Other public methods ----------------------------------------------------- + const std::vector& component_shapes() const { + return component_shapes_; + } + + int32 capacity() const { return capacity_; } + + bool is_closed() const override { + mutex_lock lock(mu_); + return closed_; + } + + // Copies the index^th slice (in the first dimension) of parent into element. + static Status CopySliceToElement(const Tensor& parent, Tensor* element, + int64 index); + + // Copies element into the index^th slice (in the first dimension) of parent. + // NOTE(mrry): This method is deprecated. Use + // `tensorflow::batch_util::CopySliceToElement()` defined in + // "./batch_util.h" instead. + ABSL_DEPRECATED( + "Use `tensorflow::batch_util::CopySliceToElement()` defined in " + "\"./batch_util.h\" instead.") + static Status CopyElementToSlice(const Tensor& element, Tensor* parent, + int64 index); + + protected: + enum Action { kEnqueue, kDequeue }; + enum RunResult { kNoProgress, kProgress, kComplete }; + + // Tries to enqueue/dequeue (or close) based on whatever is at the + // front of enqueue_attempts_/dequeue_attempts_. Appends to + // *finished the callback for any finished attempt (so it may be + // called once mu_ is released). Returns true if any progress was + // made. + struct CleanUp { + CleanUp(DoneCallback&& f, CancellationToken ct, CancellationManager* cm) + : finished(f), to_deregister(ct), cm(cm) {} + DoneCallback finished; + CancellationToken to_deregister; + CancellationManager* cm; + }; + + // Returns the number of components in a queue-element tuple. + int32 num_components() const { return component_dtypes_.size(); } + + // True if shapes were specified. If so, inputs will be validated + // against them, etc. + bool specified_shapes() const { return component_shapes_.size() > 0; } + + // Code common to Validate*Tuple(). + Status ValidateTupleCommon(const Tuple& tuple) const; + + TensorShape ManyOutShape(int i, int64 batch_size) { + TensorShape shape({batch_size}); + shape.AppendShape(component_shapes_[i]); + return shape; + } + + void Cancel(Action action, CancellationManager* cancellation_manager, + CancellationToken token); + + // Helper for cancelling all pending Enqueue(Many) operations when + // Close is called with cancel_pending_enqueues. + void CloseAndCancel(); + + bool TryAttemptLocked(Action action, std::vector* clean_up) + EXCLUSIVE_LOCKS_REQUIRED(mu_); + + // Tries to make progress on the enqueues or dequeues at the front + // of the *_attempts_ queues. + void FlushUnlocked(); + + ~QueueBase() override; + + // Helpers for implementing MatchesNodeDef(). + static string ShapeListString(const gtl::ArraySlice& shapes); + Status MatchesNodeDefOp(const NodeDef& node_def, const string& op) const; + Status MatchesNodeDefCapacity(const NodeDef& node_def, int32 capacity) const; + Status MatchesNodeDefTypes(const NodeDef& node_def) const; + Status MatchesNodeDefShapes(const NodeDef& node_def) const; + + protected: + const int32 capacity_; + const DataTypeVector component_dtypes_; + const std::vector component_shapes_; + const string name_; + mutable mutex mu_; + bool closed_ GUARDED_BY(mu_); + + struct Attempt; + typedef std::function RunCallback; + struct Attempt { + int32 elements_requested; + DoneCallback done_callback; // must be run outside mu_ + OpKernelContext* context; + CancellationManager* cancellation_manager; // not owned + CancellationToken cancellation_token; + RunCallback run_callback; // must be run while holding mu_ + bool is_cancelled; + Tuple tuple; + // tuples is used by some implementations allowing dynamic shapes. + std::vector tuples; + + Attempt(int32 elements_requested, DoneCallback done_callback, + OpKernelContext* context, CancellationManager* cancellation_manager, + CancellationToken cancellation_token, RunCallback run_callback) + : elements_requested(elements_requested), + done_callback(done_callback), + context(context), + cancellation_manager(cancellation_manager), + cancellation_token(cancellation_token), + run_callback(run_callback), + is_cancelled(false) {} + }; + std::deque enqueue_attempts_ GUARDED_BY(mu_); + std::deque dequeue_attempts_ GUARDED_BY(mu_); + + TF_DISALLOW_COPY_AND_ASSIGN(QueueBase); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_QUEUE_BASE_H_ diff --git a/linux/include/tensorflow/core/kernels/queue_op.h b/linux/include/tensorflow/core/kernels/queue_op.h new file mode 100644 index 000000000..2efd838a5 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/queue_op.h @@ -0,0 +1,271 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_QUEUE_OP_H_ +#define TENSORFLOW_CORE_KERNELS_QUEUE_OP_H_ + +#include + +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/queue_interface.h" +#include "tensorflow/core/framework/resource_op_kernel.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/kernels/queue_base.h" +#include "tensorflow/core/lib/core/errors.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +// Defines a QueueOp, an abstract class for Queue construction ops. +class QueueOp : public ResourceOpKernel { + public: + QueueOp(OpKernelConstruction* context); + + void Compute(OpKernelContext* context) override; + + protected: + // Variables accessible by subclasses + int32 capacity_; + DataTypeVector component_types_; + + private: + Status VerifyResource(QueueInterface* queue) override; +}; + +class TypedQueueOp : public QueueOp { + public: + using QueueOp::QueueOp; + + protected: + template + Status CreateTypedQueue(TypedQueue* queue, QueueInterface** ret) { + if (queue == nullptr) { + return errors::ResourceExhausted("Failed to allocate queue."); + } + *ret = queue; + return queue->Initialize(); + } +}; + +// Queue manipulator kernels + +class QueueOpKernel : public AsyncOpKernel { + public: + explicit QueueOpKernel(OpKernelConstruction* context); + + void ComputeAsync(OpKernelContext* ctx, DoneCallback callback) final; + + protected: + virtual void ComputeAsync(OpKernelContext* ctx, QueueInterface* queue, + DoneCallback callback) = 0; +}; + +class QueueAccessOpKernel : public QueueOpKernel { + public: + explicit QueueAccessOpKernel(OpKernelConstruction* context); + + protected: + int64 timeout_; +}; + +// Defines an EnqueueOp, the execution of which enqueues a tuple of +// tensors in the given Queue. +// +// The op has 1 + k inputs, where k is the number of components in the +// tuples stored in the given Queue: +// - Input 0: queue handle. +// - Input 1: 0th element of the tuple. +// - ... +// - Input (1+k): kth element of the tuple. +class EnqueueOp : public QueueAccessOpKernel { + public: + explicit EnqueueOp(OpKernelConstruction* context); + + protected: + void ComputeAsync(OpKernelContext* ctx, QueueInterface* queue, + DoneCallback callback) override; + + private: + TF_DISALLOW_COPY_AND_ASSIGN(EnqueueOp); +}; + +// Defines an EnqueueManyOp, the execution of which slices each +// component of a tuple of tensors along the 0th dimension, and +// enqueues tuples of slices in the given Queue. +// +// The op has 1 + k inputs, where k is the number of components in the +// tuples stored in the given Queue: +// - Input 0: queue handle. +// - Input 1: 0th element of the tuple. +// - ... +// - Input (1+k): kth element of the tuple. +// +// N.B. All tuple components must have the same size in the 0th +// dimension. +class EnqueueManyOp : public QueueAccessOpKernel { + public: + explicit EnqueueManyOp(OpKernelConstruction* context); + + protected: + void ComputeAsync(OpKernelContext* ctx, QueueInterface* queue, + DoneCallback callback) override; + + ~EnqueueManyOp() override; + + private: + TF_DISALLOW_COPY_AND_ASSIGN(EnqueueManyOp); +}; + +// Defines a DequeueOp, the execution of which dequeues a tuple of +// tensors from the given Queue. +// +// The op has one input, which is the handle of the appropriate +// Queue. The op has k outputs, where k is the number of components in +// the tuples stored in the given Queue, and output i is the ith +// component of the dequeued tuple. +class DequeueOp : public QueueAccessOpKernel { + public: + explicit DequeueOp(OpKernelConstruction* context); + + protected: + void ComputeAsync(OpKernelContext* ctx, QueueInterface* queue, + DoneCallback callback) override; + + ~DequeueOp() override; + + private: + TF_DISALLOW_COPY_AND_ASSIGN(DequeueOp); +}; + +// Defines a DequeueManyOp, the execution of which concatenates the +// requested number of elements from the given Queue along the 0th +// dimension, and emits the result as a single tuple of tensors. +// +// The op has two inputs: +// - Input 0: the handle to a queue. +// - Input 1: the number of elements to dequeue. +// +// The op has k outputs, where k is the number of components in the +// tuples stored in the given Queue, and output i is the ith component +// of the dequeued tuple. +class DequeueManyOp : public QueueAccessOpKernel { + public: + explicit DequeueManyOp(OpKernelConstruction* context); + + protected: + void ComputeAsync(OpKernelContext* ctx, QueueInterface* queue, + DoneCallback callback) override; + + ~DequeueManyOp() override; + + private: + TF_DISALLOW_COPY_AND_ASSIGN(DequeueManyOp); +}; + +// Defines a DequeueUpToOp, the execution of which concatenates the +// requested number of elements from the given Queue along the 0th +// dimension, and emits the result as a single tuple of tensors. +// +// The difference between this op and DequeueMany is the handling when +// the Queue is closed. While the DequeueMany op will return if there +// an error when there are less than num_elements elements left in the +// closed queue, this op will return between 1 and +// min(num_elements, elements_remaining_in_queue), and will not block. +// If there are no elements left, then the standard DequeueMany error +// is returned. +// +// This op only works if the underlying Queue implementation accepts +// the allow_small_batch = true parameter to TryDequeueMany. +// If it does not, an errors::Unimplemented exception is returned. +// +// The op has two inputs: +// - Input 0: the handle to a queue. +// - Input 1: the number of elements to dequeue. +// +// The op has k outputs, where k is the number of components in the +// tuples stored in the given Queue, and output i is the ith component +// of the dequeued tuple. +// +// The op has one attribute: allow_small_batch. If the Queue supports +// it, setting this to true causes the queue to return smaller +// (possibly zero length) batches when it is closed, up to however +// many elements are available when the op executes. In this case, +// the Queue does not block when closed. +class DequeueUpToOp : public QueueAccessOpKernel { + public: + explicit DequeueUpToOp(OpKernelConstruction* context); + + protected: + void ComputeAsync(OpKernelContext* ctx, QueueInterface* queue, + DoneCallback callback) override; + + ~DequeueUpToOp() override; + + private: + TF_DISALLOW_COPY_AND_ASSIGN(DequeueUpToOp); +}; + +// Defines a QueueCloseOp, which closes the given Queue. Closing a +// Queue signals that no more elements will be enqueued in it. +// +// The op has one input, which is the handle of the appropriate Queue. +class QueueCloseOp : public QueueOpKernel { + public: + explicit QueueCloseOp(OpKernelConstruction* context); + + protected: + void ComputeAsync(OpKernelContext* ctx, QueueInterface* queue, + DoneCallback callback) override; + + private: + bool cancel_pending_enqueues_; + TF_DISALLOW_COPY_AND_ASSIGN(QueueCloseOp); +}; + +// Defines a QueueSizeOp, which computes the number of elements in the +// given Queue, and emits it as an output tensor. +// +// The op has one input, which is the handle of the appropriate Queue; +// and one output, which is a single-element tensor containing the current +// size of that Queue. +class QueueSizeOp : public QueueOpKernel { + public: + explicit QueueSizeOp(OpKernelConstruction* context); + + protected: + void ComputeAsync(OpKernelContext* ctx, QueueInterface* queue, + DoneCallback callback) override; + + private: + TF_DISALLOW_COPY_AND_ASSIGN(QueueSizeOp); +}; + +class QueueIsClosedOp : public QueueOpKernel { + public: + explicit QueueIsClosedOp(OpKernelConstruction* context); + + protected: + void ComputeAsync(OpKernelContext* ctx, QueueInterface* queue, + DoneCallback callback) override; + + private: + TF_DISALLOW_COPY_AND_ASSIGN(QueueIsClosedOp); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_QUEUE_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/random_op.h b/linux/include/tensorflow/core/kernels/random_op.h new file mode 100644 index 000000000..d313a021d --- /dev/null +++ b/linux/include/tensorflow/core/kernels/random_op.h @@ -0,0 +1,72 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_RANDOM_OP_H_ +#define TENSORFLOW_CORE_KERNELS_RANDOM_OP_H_ + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/lib/random/random_distributions.h" + +namespace tensorflow { + +class OpKernelContext; + +namespace functor { + +template +struct FillPhiloxRandom; + +typedef Eigen::ThreadPoolDevice CPUDevice; +// Declares the partially CPU-specialized functor struct. +// +// NOTE: Due to inlining done by the compiler, you may need to add +// explicit instantiation of the functor in random_op.cc. See example +// functor::FillPhiloxRandom. +template +struct FillPhiloxRandom { + void operator()(OpKernelContext* ctx, const CPUDevice& d, + random::PhiloxRandom gen, + typename Distribution::ResultElementType* data, int64 size, + Distribution dist); +}; + +#if GOOGLE_CUDA +typedef Eigen::GpuDevice GPUDevice; +// Declares the partially GPU-specialized functor struct. +template +struct FillPhiloxRandom { + void operator()(OpKernelContext* ctx, const GPUDevice& d, + random::PhiloxRandom gen, + typename Distribution::ResultElementType* data, int64 size, + Distribution dist); +}; +#endif // GOOGLE_CUDA + +#if TENSORFLOW_USE_SYCL +typedef Eigen::SyclDevice SYCLDevice; +// Declares the partially SYCL-specialized functor struct. +template +struct FillPhiloxRandom { + void operator()(OpKernelContext* ctx, const SYCLDevice& d, + random::PhiloxRandom gen, + typename Distribution::ResultElementType* data, int64 size, + Distribution dist); +}; +#endif // TENSORFLOW_USE_SYCL + +} // namespace functor +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_RANDOM_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/random_poisson_op.h b/linux/include/tensorflow/core/kernels/random_poisson_op.h new file mode 100644 index 000000000..62ae01c16 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/random_poisson_op.h @@ -0,0 +1,31 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_RANDOM_POISSON_OP_H_ +#define TENSORFLOW_CORE_KERNELS_RANDOM_POISSON_OP_H_ + +namespace tensorflow { + +namespace functor { + +// Generic helper functor for the Random Poisson Op. +template +struct PoissonFunctor; + +} // namespace functor + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_RANDOM_POISSON_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/range_sampler.h b/linux/include/tensorflow/core/kernels/range_sampler.h new file mode 100644 index 000000000..ed160adfb --- /dev/null +++ b/linux/include/tensorflow/core/kernels/range_sampler.h @@ -0,0 +1,252 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_RANGE_SAMPLER_H_ +#define TENSORFLOW_CORE_KERNELS_RANGE_SAMPLER_H_ + +#include + +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/lib/gtl/array_slice.h" +#include "tensorflow/core/lib/random/distribution_sampler.h" +#include "tensorflow/core/lib/random/random_distributions.h" +#include "tensorflow/core/lib/random/weighted_picker.h" +#include "tensorflow/core/platform/logging.h" +#include "tensorflow/core/platform/mutex.h" +#include "tensorflow/core/platform/thread_annotations.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +class Env; + +// Abstract subclass for sampling from the set of non-negative integers +// [0, range) +class RangeSampler { + public: + explicit RangeSampler(int64 range) : range_(range) { CHECK_GT(range_, 0); } + virtual ~RangeSampler(); + + // Sample a single value + virtual int64 Sample(random::SimplePhilox* rnd) const = 0; + + // The probability that a single call to Sample() returns the given value. + // Assumes that value is in [0, range). No range checking is done. + virtual float Probability(int64 value) const = 0; + + // Fill "batch" with samples from the distribution. + // If unique=true, then we re-pick each element until we get a + // value distinct from all previously picked values in the batch. + void SampleBatch(random::SimplePhilox* rnd, bool unique, + gtl::MutableArraySlice batch) const; + + // Fill "batch" with samples from the distribution, and report + // "expected counts". + // + // The "expected count" of a value is an estimate of the expected + // number of occurrences of the value in the batch returned by a + // call to this function with the given parameters. If unique=true, + // the expected count is an inclusion probability. For details on + // this estimation, see the comment to "ExpectedCountHelper" in the + // .cc file. + // + // Expected counts for the elements of the returned "batch" are reported + // in the aligned array "batch_expected_count". + // + // The user can optionally provide "extras", containing values in the range. + // The expected counts for the extras are reported in the aligned array + // "extras_expected_count". + // + // "batch_expected_count" must have size equal to 0 or to the size of "batch". + // "extras" and "extras_expected_count" must have equal size. + void SampleBatchGetExpectedCount( + random::SimplePhilox* rnd, bool unique, + gtl::MutableArraySlice batch, + gtl::MutableArraySlice batch_expected_count, + gtl::ArraySlice extras, + gtl::MutableArraySlice extras_expected_count) const; + + // Same as SampleBatchGetExpectedCount (see above), but with avoided values. + // We repick to avoid all of the values in "avoided_values". + // "avoided_values" is only supported with unique=true. If + // unique=false, then avoided_values must be empty. + virtual void SampleBatchGetExpectedCountAvoid( + random::SimplePhilox* rnd, bool unique, + gtl::MutableArraySlice batch, + gtl::MutableArraySlice batch_expected_count, + gtl::ArraySlice extras, + gtl::MutableArraySlice extras_expected_count, + gtl::ArraySlice avoided_values) const; + + // Does this sampler need to be updated with values, e.g. UnigramSampler + virtual bool NeedsUpdates() const { return false; } + + // Updates the underlying distribution + virtual void Update(gtl::ArraySlice values) { + LOG(FATAL) << "Update not supported for this sampler type."; + } + + int64 range() { return range_; } + + protected: + const int64 range_; +}; + +// An AllSampler only samples batches of size equal to range. +// It returns the entire range. +// It cannot sample single values. +class AllSampler : public RangeSampler { + public: + explicit AllSampler(int64 range); + + ~AllSampler() override {} + + int64 Sample(random::SimplePhilox* rnd) const override { + LOG(FATAL) << "Should not be called"; + return 0; + } + + float Probability(int64 value) const override { + LOG(FATAL) << "Should not be called"; + return 0; + } + + void SampleBatchGetExpectedCountAvoid( + random::SimplePhilox* rnd, bool unique, + gtl::MutableArraySlice batch, + gtl::MutableArraySlice batch_expected_count, + gtl::ArraySlice extras, + gtl::MutableArraySlice extras_expected_count, + gtl::ArraySlice avoided_values) const override; +}; + +class UniformSampler : public RangeSampler { + public: + explicit UniformSampler(int64 range); + + ~UniformSampler() override {} + + int64 Sample(random::SimplePhilox* rnd) const override; + + float Probability(int64 value) const override; + + private: + const float inv_range_; +}; + +class LogUniformSampler : public RangeSampler { + public: + explicit LogUniformSampler(int64 range); + + ~LogUniformSampler() override {} + + int64 Sample(random::SimplePhilox* rnd) const override; + + float Probability(int64 value) const override; + + private: + const double log_range_; +}; + +// Thread-unsafe unigram sampler +class ThreadUnsafeUnigramSampler : public RangeSampler { + public: + explicit ThreadUnsafeUnigramSampler(int64 range); + ~ThreadUnsafeUnigramSampler() override {} + + int64 Sample(random::SimplePhilox* rnd) const override; + + float Probability(int64 value) const override; + + bool NeedsUpdates() const override { return true; } + void Update(gtl::ArraySlice values) override; + + private: + random::WeightedPicker picker_; +}; + +// Thread-safe unigram sampler +class UnigramSampler : public RangeSampler { + public: + explicit UnigramSampler(int64 range); + ~UnigramSampler() override {} + + int64 Sample(random::SimplePhilox* rnd) const override; + + float Probability(int64 value) const override; + + // Overriding at a high level results in far fewer lock acquisitions. + void SampleBatchGetExpectedCountAvoid( + random::SimplePhilox* rnd, bool unique, + gtl::MutableArraySlice batch, + gtl::MutableArraySlice batch_expected_count, + gtl::ArraySlice extras, + gtl::MutableArraySlice extras_expected_count, + gtl::ArraySlice avoided_values) const override; + + bool NeedsUpdates() const override { return true; } + void Update(gtl::ArraySlice values) override; + + private: + ThreadUnsafeUnigramSampler unsafe_sampler_ GUARDED_BY(mu_); + mutable mutex mu_; +}; + +// A unigram sampler that uses a fixed unigram distribution read from a +// file or passed in as an in-memory array instead of building up the +// distribution from data on the fly. There is also an option to skew the +// distribution by applying a distortion power to the weights. +class FixedUnigramSampler : public RangeSampler { + public: + // The vocab_file is assumed to be a CSV, with the last entry of each row a + // value representing the counts or probabilities for the corresponding ID. + FixedUnigramSampler(Env* env, int64 range, const string& vocab_file, + float distortion, int32 num_reserved_ids, + int32 num_shards, int32 shard); + + FixedUnigramSampler(int64 range, const std::vector& unigrams, + float distortion, int32 num_reserved_ids, + int32 num_shards, int32 shard); + + float Probability(int64 value) const override; + + int64 Sample(random::SimplePhilox* rnd) const override; + + private: + // Underlying distribution sampler. + std::unique_ptr dist_sampler_; + // Weights for individual samples. The probability of a sample i is defined + // as weights_.at(i) / total_weight_. + std::vector weights_; + // The total weights of all samples. + float total_weight_; + // Sharding information of the sampler. The whole vocabulary is sharded + // into num_shards_ smaller ranges and each sampler is responsible for one + // such smaller range, identified by the shard number. + int32 num_shards_; + int32 shard_; + + // Fill the sampler with the appropriate number of reserved IDs. + void FillReservedIds(int32 num_reserved_ids); + // Load IDs to sample from a CSV file. It is assumed that the last item of + // each row contains a count or probability for the corresponding ID. + Status LoadFromFile(Env* env, const string& vocab_file, float distortion); + // Load from an in-memory array. + void LoadFromUnigrams(const std::vector& unigrams, float distortion); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_RANGE_SAMPLER_H_ diff --git a/linux/include/tensorflow/core/kernels/record_yielder.h b/linux/include/tensorflow/core/kernels/record_yielder.h new file mode 100644 index 000000000..159b43b4c --- /dev/null +++ b/linux/include/tensorflow/core/kernels/record_yielder.h @@ -0,0 +1,160 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_RECORD_YIELDER_H_ +#define TENSORFLOW_CORE_KERNELS_RECORD_YIELDER_H_ + +#include +#include +#include +#include + +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/lib/core/errors.h" +#include "tensorflow/core/lib/core/notification.h" +#include "tensorflow/core/lib/core/threadpool.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/thread_annotations.h" + +namespace tensorflow { + +// RecordYielder produces value records from a set of tfrecord files +// in a random order. +// +// It guarantees that: +// 1) all records in tfrecords are yielded within every epoch; +// 2) each record is yielded only once within every epoch; +// 3) the order in which records are yielded is highly randomized. +// 4) the peak memory usage is roughly avg record size * +// (opts.bufsize + opts.parellelism * 16). +// +// Usage example: +// RecordYielder::Options opts; +// opts.file_pattern = "input-*"; +// opts.seed = 301; +// opts.bufsize = 1000000; // A randomized buffer with 1M records. +// opts.parallelism = 8; // Uses 8 tfrecord iterators to iterate +// // through all files. +// RecordYielder yielder(opts); +// string val; +// while (true) { +// yielder.YieldOne(&val); +// // process val +// } +// +// RecordYielder can be accessed by multiple threads concurrently. +class RecordYielder { + public: + struct Options { + // Glob pattern for tfrecords. + string file_pattern; + + // Random seed. It determines how data files are shuffled and how + // records are shuffled. + int64 seed = 0; + + // Each epoch, all files are first shuffled according to the + // random seed and the epoch number, and then all files are + // left-shifted by file_shuffle_shift_ratio * num_files slots. If + // file_shuffle_shift_ratio is not within [0, 1), the + // implementation clip it to [0, 1). + float file_shuffle_shift_ratio = 0; + + // Randomization buffer keeps these many records. + uint64 bufsize = 1; + + // Uses these many concurrent tfrecord iterators to iterate through + // tfrecords. + int32 parallelism = 1; + + string compression_type; + }; + + explicit RecordYielder(OpKernelConstruction* context, + const RecordYielder::Options& opts); + ~RecordYielder(); + + RecordYielder(const RecordYielder&) = delete; + RecordYielder& operator=(const RecordYielder&) = delete; + + // Yields one 'value'. + Status YieldOne(string* value); + + // Returns the current epoch number. + int64 current_epoch() const { return epoch_; } + + private: + typedef RecordYielder ME; + + Options opts_; + + // Backgrounds threads. Owned. + thread::ThreadPool* thread_; + + // Epoch number. + std::atomic epoch_; + + mutex mu_; + + // Turned to true when this is deleted. + bool stop_ GUARDED_BY(mu_) = false; + Status status_ GUARDED_BY(mu_); + + // PRG used for randomization. + std::mt19937_64 rnd_ GUARDED_BY(mu_); + + // Randomization buffer. + std::vector buf_ GUARDED_BY(mu_); + + // True iff we are draining an epoch. + bool epoch_end_ = false; + + int64 num_records_added_in_epoch_ = 0; + int64 num_records_yielded_in_epoch_ = 0; + + // Trigger when the main loop has exited. + Notification main_loop_done_; + + // condition_variables. + condition_variable buf_empty_; + bool BufEmpty() const SHARED_LOCKS_REQUIRED(mu_) { + return stop_ || buf_.empty(); + } + + condition_variable buf_not_full_; + bool BufNotFull() const SHARED_LOCKS_REQUIRED(mu_) { + return stop_ || buf_.size() < opts_.bufsize; + } + + condition_variable buf_enough_; + bool BufEnough() const SHARED_LOCKS_REQUIRED(mu_) { + // NOTE: Unless we are finishing an epoch, we want to make sure + // the buf_ contains enough randomized elements before yielding + // any. + return stop_ || !status_.ok() || (epoch_end_ && !buf_.empty()) || + (!epoch_end_ && + buf_.size() >= std::max(1, opts_.bufsize / 2)); + } + + void MainLoop(); + struct Shard; + void ShardLoop(Shard* shard); + bool ShouldFinish(const Status& s); + bool Add(std::vector* values); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_RECORD_YIELDER_H_ diff --git a/linux/include/tensorflow/core/kernels/reduction_ops.h b/linux/include/tensorflow/core/kernels/reduction_ops.h new file mode 100644 index 000000000..eb264e0e5 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/reduction_ops.h @@ -0,0 +1,82 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_REDUCTION_OPS_H_ +#define TENSORFLOW_CORE_KERNELS_REDUCTION_OPS_H_ + +// Functor definitions for Reduction ops, must be compilable by nvcc. + +#include +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/tensor_types.h" + +namespace tensorflow { +namespace functor { + +template +void ReduceEigenImpl(const Device& d, OUT_T out, IN_T in, + const ReductionAxes& reduction_axes, + const Reducer& reducer) { + out.device(d) = in.reduce(reduction_axes, reducer); +} + +// For most reducers, the identity is Reducer::initialize() +template +struct Identity { + static auto identity(const Reducer& reducer) + -> decltype(reducer.initialize()) { + return reducer.initialize(); + } +}; + +// MeanReducer is a special case, since it doesn't technically have an identity. +// Thus, ideally we'd return nan. However, mean is instantiated for integer +// types as well, so we do the nan override only for floating point types. +#define FIX_MEAN_IDENTITY(T) \ + template <> \ + struct Identity> { \ + static T identity(const Eigen::internal::MeanReducer&) { \ + return Eigen::NumTraits::quiet_NaN(); \ + } \ + }; +FIX_MEAN_IDENTITY(Eigen::half) +FIX_MEAN_IDENTITY(float) +FIX_MEAN_IDENTITY(double) +FIX_MEAN_IDENTITY(complex64) +FIX_MEAN_IDENTITY(complex128) +#undef FIX_MEAN_IDENTITY + +template +void FillIdentityEigenImpl(const Device& d, OUT_T out, const Reducer& reducer) { + out.device(d) = out.constant(Identity::identity(reducer)); +} + +template +struct ReduceFunctor { + template + static void Reduce(OpKernelContext* ctx, OUT_T out, IN_T in, + const ReductionAxes& reduction_axes, + const Reducer& reducer); + + template + static void FillIdentity(const Device& d, OUT_T out, const Reducer& reducer); +}; + +} // namespace functor +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_REDUCTION_OPS_H_ diff --git a/linux/include/tensorflow/core/kernels/reduction_ops_common.h b/linux/include/tensorflow/core/kernels/reduction_ops_common.h new file mode 100644 index 000000000..d83e1c7d1 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/reduction_ops_common.h @@ -0,0 +1,280 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// This is an internal header file intended to only be included as the +// front-matter in the implementation files of various reduction ops. It +// is a header file because we split the various reduction ops into their +// own compilation units to get more parallelism in compilation. + +#ifndef TENSORFLOW_CORE_KERNELS_REDUCTION_OPS_COMMON_H_ +#define TENSORFLOW_CORE_KERNELS_REDUCTION_OPS_COMMON_H_ + +#define EIGEN_USE_THREADS + +#include "third_party/eigen3/Eigen/Core" +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" + +#include "tensorflow/core/framework/numeric_op.h" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/register_types.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/kernels/reduction_ops.h" +#include "tensorflow/core/kernels/transpose_functor.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/lib/gtl/inlined_vector.h" +#include "tensorflow/core/platform/logging.h" + +namespace tensorflow { + +typedef Eigen::ThreadPoolDevice CPUDevice; +typedef Eigen::GpuDevice GPUDevice; +#ifdef TENSORFLOW_USE_SYCL +typedef Eigen::SyclDevice SYCLDevice; +#endif // TENSORFLOW_USE_SYCL + +template +struct Constants { + // Derive Index type. int (32-bit) or long (64-bit) depending on the + // compile-time configuration. "float" here is not relevant. + // TODO(zhifengc): Moves the definition to TTypes. + typedef TTypes::Tensor::Index Index; + Eigen::array kZero; + Eigen::array kOne; + Eigen::array kZeroTwo; + + Constants() { + kZero[0] = 0; + kOne[0] = 1; + kZeroTwo[0] = 0; + kZeroTwo[1] = 2; + } +}; + +#if defined(EIGEN_HAS_INDEX_LIST) +struct ConstantsBase { + const Eigen::IndexList> kZero; + const Eigen::IndexList> kOne; + const Eigen::IndexList, Eigen::type2index<2>> kZeroTwo; +}; +template <> +struct Constants : ConstantsBase {}; +#ifdef TENSORFLOW_USE_SYCL +template <> +struct Constants : ConstantsBase {}; +#endif // TENSORFLOW_USE_SYCL +#endif // EIGEN_HAS_INDEX_LIST + +class ReductionHelper { + public: + ReductionHelper() : reduce_first_axis_(false) {} + + Status Simplify(const Tensor& data, const Tensor& axis, const bool keep_dims); + + // We need to do roughly: + // tmp_out = allocate(out_reshape()) + // tmp_out.reshape(out_reshape) = data.reshape(data_reshape).reduce(axes) + // out = tmp_out.reshape(out_shape) + + // The reduction result must be allocated with this shape. + TensorShape out_reshape() const; + + // The final output shape must be allocated with this shape. + TensorShape out_shape() const; + + // The reduction is on a reshaped tensor of this rank. + int ndims() const { return data_reshape_.size(); } + + // True if need to reduce the 0-th dimension. + bool reduce_first_axis() const { return reduce_first_axis_; } + + // The output is reshaped. + template + typename TTypes::Tensor out(Tensor* out) { + return out->shaped(out_reshape_); + } + + // The input is reshaped. + template + typename TTypes::ConstTensor in(const Tensor& data) { + return data.shaped(data_reshape_); + } + + // Shape of shuffled input + TensorShape data_reshape() const { + TensorShape shape; + for (auto s : data_reshape_) shape.AddDim(s); + return shape; + } + + // Shape with all reduction dimensions at the end + TensorShape shuffled_shape(); + + // Permutation of reduced dims needed to put reduction dimensions at the end + gtl::InlinedVector permutation(); + + private: + bool reduce_first_axis_; // True if need to reduce the 0-th dimension. + gtl::InlinedVector data_reshape_; // Reshape data before reduction. + gtl::InlinedVector out_shape_; // The final output shape. + gtl::InlinedVector out_reshape_; // Reshape output for reduction. +}; + +// For operations where the output is a reduction function along some +// dimensions of the input. +template +class ReductionOp : public OpKernel { + public: + explicit ReductionOp(OpKernelConstruction* ctx) : OpKernel(ctx) { + const DataType dt = DataTypeToEnum::v(); + const DataType pt = DataTypeToEnum::v(); + OP_REQUIRES_OK(ctx, ctx->MatchSignature({dt, pt}, {dt})); + + OP_REQUIRES_OK(ctx, ctx->GetAttr("keep_dims", &keep_dims_)); + } + + void Compute(OpKernelContext* ctx) override { + const Tensor& data = ctx->input(0); + const Tensor& axes = ctx->input(1); + VLOG(1) << "data shape: " << data.shape().DebugString(); + VLOG(1) << "axes : " << axes.SummarizeValue(10); + + ReductionHelper helper; + OP_REQUIRES_OK(ctx, helper.Simplify(data, axes, keep_dims_)); + CHECK_GE(helper.ndims(), 0); + + if (helper.ndims() == 0 || + (helper.ndims() == 1 && !helper.reduce_first_axis())) { + // Special case. Reduces nothing. It is unclear why this is + // necessary, but tests fail without it. Look into why this + // case occurs. + Tensor out; + if (!out.CopyFrom(data, helper.out_shape())) { + ctx->SetStatus(errors::Internal("Error during reduction copy.")); + } + ctx->set_output(0, out); + return; + } + + // We must allocate temp tensors using the same alloc attr as + // output(0) because it is returned as output(0) in the end. + const AllocatorAttributes alloc_attr = ctx->output_alloc_attr(0); + + // A temporary tensor whose size matches the size of the reduced + // output. + Tensor tmp_out; + OP_REQUIRES_OK( + ctx, ctx->allocate_temp(ctx->expected_output_dtype(0), + helper.out_reshape(), &tmp_out, alloc_attr)); + + typedef functor::ReduceFunctor Functor; + Constants constants; + const Device& d = ctx->eigen_device(); + Reducer reducer; + + if (tmp_out.NumElements() == 0) { + // Nothing to do, fall through to final reshaping. + } else if (data.NumElements() == 0) { + // Degenerate reduction where the input is empty but the output is + // nonempty (thus tmp_out.NumElements() > 0), and we must fill the output + // with identity elements. Example: tf.reduce_sum(tf.zeros((0, 3)), [0]). + // Eigen sometimes crashes in this case, so we do it manually. + Functor::FillIdentity(d, tmp_out.flat(), reducer); + } else if ((helper.ndims() == 1) && helper.reduce_first_axis()) { + // Reduce to a scalar. + Functor::Reduce(ctx, helper.out(&tmp_out), helper.in(data), + constants.kZero, reducer); + } else if ((helper.ndims() == 2) && helper.reduce_first_axis()) { + // Can be viewed as a reduction of a matrix along 1st dimension. + Functor::Reduce(ctx, helper.out(&tmp_out), helper.in(data), + constants.kZero, reducer); + } else if ((helper.ndims() == 2) && !helper.reduce_first_axis()) { + // Can be viewed as a reduction of a matrix along 2nd dimension. + Functor::Reduce(ctx, helper.out(&tmp_out), helper.in(data), + constants.kOne, reducer); + } else if ((helper.ndims() == 3) && helper.reduce_first_axis()) { + // Can be viewed as a reduction of a 3D tensor along 1st and 3rd + // dimensions. + Functor::Reduce(ctx, helper.out(&tmp_out), helper.in(data), + constants.kZeroTwo, reducer); + } else if ((helper.ndims() == 3) && !helper.reduce_first_axis()) { + // Can be viewed as a reduction of a 3D tensor along 2nd dimension. + Functor::Reduce(ctx, helper.out(&tmp_out), helper.in(data), + constants.kOne, reducer); + } else { + // If we don't hit one of the cases above, transpose the data so that + // all reduced dimensions are last and reuse the 2-D -> 1-D case. + Tensor data_reshaped; + CHECK(data_reshaped.CopyFrom(data, helper.data_reshape())); + Tensor shuffled; + OP_REQUIRES_OK(ctx, ctx->allocate_temp(DataTypeToEnum::value, + helper.shuffled_shape(), &shuffled, + alloc_attr)); + OP_REQUIRES_OK( + ctx, DoTranspose(d, data_reshaped, helper.permutation(), &shuffled)); + const int64 unreduced = tmp_out.NumElements(); + const int64 reduced = shuffled.NumElements() / unreduced; + const Tensor& const_shuffled = shuffled; + Functor::Reduce(ctx, tmp_out.flat(), + const_shuffled.shaped({unreduced, reduced}), + constants.kOne, reducer); + } + + // Set the real output using the contents of the reduction but the + // real expected output shape. The number of elements should + // match between the two shapes. + Tensor out; + if (!out.CopyFrom(tmp_out, helper.out_shape())) { + ctx->SetStatus(errors::Internal("Error during reduction copy.")); + } + ctx->set_output(0, out); + } + + private: + // True if the number of dimensions should be maintained. + bool keep_dims_; +}; + +namespace functor { + +template +struct ReduceFunctorBase { + template + static void Reduce(OpKernelContext* ctx, OUT_T out, IN_T in, + const ReductionAxes& reduction_axes, + const Reducer& reducer) { + const Device& d = ctx->eigen_device(); + ReduceEigenImpl(d, out, in, reduction_axes, reducer); + } + + template + static void FillIdentity(const Device& d, OUT_T out, const Reducer& reducer) { + FillIdentityEigenImpl(d, out, reducer); + } +}; + +template +struct ReduceFunctor + : ReduceFunctorBase {}; +#if TENSORFLOW_USE_SYCL +template +struct ReduceFunctor + : ReduceFunctorBase {}; +#endif // TENSORFLOW_USE_SYCL + +} // namespace functor +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_REDUCTION_OPS_COMMON_H_ diff --git a/linux/include/tensorflow/core/kernels/reference_gemm.h b/linux/include/tensorflow/core/kernels/reference_gemm.h new file mode 100644 index 000000000..c9cc04ed1 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/reference_gemm.h @@ -0,0 +1,95 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_REFERENCE_GEMM_H_ +#define TENSORFLOW_CORE_KERNELS_REFERENCE_GEMM_H_ + +#include + +#include "third_party/eigen3/Eigen/Core" +#include "tensorflow/core/platform/types.h" + +// This is an unoptimized but debuggable implementation of the GEMM matrix +// multiply function, used to compare to faster but more opaque versions, or +// for bit depths or argument combinations that aren't supported by optimized +// code. +// It assumes the row-major convention used by TensorFlow, and implements +// C = A * B, like the standard BLAS GEMM interface. If the transpose flags are +// true, then the relevant matrix is treated as stored in column-major order. + +namespace tensorflow { +template +void ReferenceGemm(bool transpose_a, bool transpose_b, bool transpose_c, + size_t m, size_t n, size_t k, const T1* a, int32 offset_a, + size_t lda, const T2* b, int32 offset_b, size_t ldb, T3* c, + int32 shift_c, int32 offset_c, int32 mult_c, size_t ldc) { + int a_i_stride; + int a_l_stride; + if (transpose_a) { + a_i_stride = 1; + a_l_stride = lda; + } else { + a_i_stride = lda; + a_l_stride = 1; + } + int b_j_stride; + int b_l_stride; + if (transpose_b) { + b_j_stride = ldb; + b_l_stride = 1; + } else { + b_j_stride = 1; + b_l_stride = ldb; + } + int c_i_stride; + int c_j_stride; + if (transpose_c) { + c_i_stride = 1; + c_j_stride = ldc; + } else { + c_i_stride = ldc; + c_j_stride = 1; + } + + const int32 highest = static_cast(Eigen::NumTraits::highest()); + const int32 lowest = static_cast(Eigen::NumTraits::lowest()); + const int32 rounding = (shift_c < 1) ? 0 : (1 << (shift_c - 1)); + + int i, j, l; + for (j = 0; j < n; j++) { + for (i = 0; i < m; i++) { + int32 total = 0; + for (l = 0; l < k; l++) { + const size_t a_index = ((i * a_i_stride) + (l * a_l_stride)); + const int32 a_value = static_cast(a[a_index]) - offset_a; + const size_t b_index = ((j * b_j_stride) + (l * b_l_stride)); + const int32 b_value = static_cast(b[b_index]) - offset_b; + total += (a_value * b_value); + } + const size_t c_index = ((i * c_i_stride) + (j * c_j_stride)); + int32_t output = ((((total + offset_c) * mult_c) + rounding) >> shift_c); + if (output > highest) { + output = highest; + } + if (output < lowest) { + output = lowest; + } + c[c_index] = static_cast(output); + } + } +} +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_REFERENCE_GEMM_H_ diff --git a/linux/include/tensorflow/core/kernels/relu_op.h b/linux/include/tensorflow/core/kernels/relu_op.h new file mode 100644 index 000000000..4775deeb6 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/relu_op.h @@ -0,0 +1,222 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// See docs in ../ops/nn_ops.cc. + +#ifndef TENSORFLOW_CORE_KERNELS_RELU_OP_H_ +#define TENSORFLOW_CORE_KERNELS_RELU_OP_H_ + +#define EIGEN_USE_THREADS + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/numeric_op.h" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/register_types.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/kernels/relu_op_functor.h" +#include "tensorflow/core/lib/core/errors.h" + +namespace tensorflow { + +template +class ReluOp : public UnaryElementWiseOp> { + public: + using UnaryElementWiseOp>::UnaryElementWiseOp; + + void Operate(OpKernelContext* context, const Tensor& input, Tensor* output) { + functor::Relu functor; + functor(context->eigen_device(), input.flat(), + output->flat()); + } +}; + +// Out of line check to save code space (we have this code once, rather +// than once for every NDIMS * NumTypes * Num_different_relu_variants +// functions. +struct ReluHelpers { + static void ValidateSameSizeHelper(OpKernelContext* context, const Tensor& g, + const Tensor& a) { + OP_REQUIRES(context, a.IsSameSize(g), + errors::InvalidArgument("g and a must be the same size")); + } + static bool ValidateSameSize(OpKernelContext* context, const Tensor& g, + const Tensor& a) { + ValidateSameSizeHelper(context, g, a); + return context->status().ok(); + } +}; + +template +class ReluGradOp : public BinaryElementWiseOp> { + public: + using BinaryElementWiseOp>::BinaryElementWiseOp; + + void OperateNoTemplate(OpKernelContext* context, const Tensor& g, + const Tensor& a, Tensor* output); + + // INPUTS: + // g (gradients): backpropagated gradients + // a (inputs): either the inputs that were passed to ReluOp(), or its + // outputs (using either one yields the same result here). + // OUTPUT: + // gradients to backprop + template + void Operate(OpKernelContext* context, const Tensor& g, const Tensor& a, + Tensor* output) { + OperateNoTemplate(context, g, a, output); + } +}; + +template +void ReluGradOp::OperateNoTemplate(OpKernelContext* context, + const Tensor& g, const Tensor& a, + Tensor* output) { + if (!ReluHelpers::ValidateSameSize(context, g, a)) return; + functor::ReluGrad functor; + functor(context->eigen_device(), g.flat(), a.flat(), + output->flat()); +} + +template +class Relu6Op : public UnaryElementWiseOp> { + public: + using UnaryElementWiseOp>::UnaryElementWiseOp; + + void Operate(OpKernelContext* context, const Tensor& input, Tensor* output) { + functor::Relu6 functor; + functor(context->eigen_device(), input.flat(), + output->flat()); + } +}; + +template +class Relu6GradOp : public BinaryElementWiseOp> { + public: + using BinaryElementWiseOp>::BinaryElementWiseOp; + + void OperateNoTemplate(OpKernelContext* context, const Tensor& g, + const Tensor& a, Tensor* output); + + // INPUTS: + // g (gradients): backpropagated gradients + // a (inputs): inputs that were passed to Relu6Op() + // OUTPUT: + // gradients to backprop + template + void Operate(OpKernelContext* context, const Tensor& g, const Tensor& a, + Tensor* output) { + OperateNoTemplate(context, g, a, output); + } +}; + +template +void Relu6GradOp::OperateNoTemplate(OpKernelContext* context, + const Tensor& g, const Tensor& a, + Tensor* output) { + if (!ReluHelpers::ValidateSameSize(context, g, a)) return; + functor::Relu6Grad functor; + functor(context->eigen_device(), g.flat(), a.flat(), + output->flat()); +} + +template +class EluOp : public UnaryElementWiseOp> { + public: + using UnaryElementWiseOp>::UnaryElementWiseOp; + + void Operate(OpKernelContext* context, const Tensor& input, Tensor* output) { + functor::Elu functor; + functor(context->eigen_device(), input.flat(), + output->flat()); + } +}; + +template +class EluGradOp : public BinaryElementWiseOp> { + public: + using BinaryElementWiseOp>::BinaryElementWiseOp; + + void OperateNoTemplate(OpKernelContext* context, const Tensor& g, + const Tensor& a, Tensor* output); + + // INPUTS: + // g (gradients): backpropagated gradients + // a (outputs): outputs of the EluOp() + // OUTPUT: + // gradients to backprop + template + void Operate(OpKernelContext* context, const Tensor& g, const Tensor& a, + Tensor* output) { + OperateNoTemplate(context, g, a, output); + } +}; + +template +void EluGradOp::OperateNoTemplate(OpKernelContext* context, + const Tensor& g, const Tensor& a, + Tensor* output) { + if (!ReluHelpers::ValidateSameSize(context, g, a)) return; + functor::EluGrad functor; + functor(context->eigen_device(), g.flat(), a.flat(), + output->flat()); +} + +template +class SeluOp : public UnaryElementWiseOp> { + public: + using UnaryElementWiseOp>::UnaryElementWiseOp; + + void Operate(OpKernelContext* context, const Tensor& input, Tensor* output) { + functor::Selu functor; + functor(context->eigen_device(), input.flat(), + output->flat()); + } +}; + +template +class SeluGradOp : public BinaryElementWiseOp> { + public: + using BinaryElementWiseOp>::BinaryElementWiseOp; + + void OperateNoTemplate(OpKernelContext* context, const Tensor& g, + const Tensor& a, Tensor* output); + + // INPUTS: + // g (gradients): backpropagated gradients + // a (outputs): outputs of the SeluOp() + // OUTPUT: + // gradients to backprop + template + void Operate(OpKernelContext* context, const Tensor& g, const Tensor& a, + Tensor* output) { + OperateNoTemplate(context, g, a, output); + } +}; + +template +void SeluGradOp::OperateNoTemplate(OpKernelContext* context, + const Tensor& g, const Tensor& a, + Tensor* output) { + if (!ReluHelpers::ValidateSameSize(context, g, a)) return; + functor::SeluGrad functor; + functor(context->eigen_device(), g.flat(), a.flat(), + output->flat()); +} + +} // namespace tensorflow + +#undef EIGEN_USE_THREADS + +#endif // TENSORFLOW_CORE_KERNELS_RELU_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/relu_op_functor.h b/linux/include/tensorflow/core/kernels/relu_op_functor.h new file mode 100644 index 000000000..e564da335 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/relu_op_functor.h @@ -0,0 +1,171 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_RELU_OP_FUNCTOR_H_ +#define TENSORFLOW_CORE_KERNELS_RELU_OP_FUNCTOR_H_ +// Functor definition for ReluOp and ReluGradOp, must be compilable by nvcc. + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/tensor_types.h" + +namespace tensorflow { +namespace functor { + +// Functor used by ReluOp to do the computations. +template +struct Relu { + // Computes Relu activation. + // + // features: any shape. + // activations: same shape as "features". + void operator()(const Device& d, typename TTypes::ConstTensor features, + typename TTypes::Tensor activations) { + activations.device(d) = features.cwiseMax(static_cast(0)); + } +}; + +// Functor used by ReluGradOp to do the computations. +template +struct ReluGrad { + // Computes ReluGrad backprops. + // + // gradients: gradients backpropagated to the Relu op. + // features: either the inputs that were passed to the Relu or, or its + // outputs (using either one yields the same result here). + // backprops: gradients to backpropagate to the Relu inputs. + void operator()(const Device& d, typename TTypes::ConstTensor gradients, + typename TTypes::ConstTensor features, + typename TTypes::Tensor backprops) { + // NOTE: When the activation is exactly zero, we do not propagate the + // associated gradient value. This allows the output of the Relu to be used, + // as well as its input. + backprops.device(d) = + gradients * (features > static_cast(0)).template cast(); + } +}; + +// Functor used by Relu6Op to do the computations. +template +struct Relu6 { + // Computes Relu6 activation. + // + // features: any shape. + // activations: same shape as "features". + void operator()(const Device& d, typename TTypes::ConstTensor features, + typename TTypes::Tensor activations) { + activations.device(d) = + features.cwiseMax(static_cast(0)).cwiseMin(static_cast(6)); + } +}; + +// Functor used by ReluGradOp to do the computations. +template +struct Relu6Grad { + // Computes Relu6Grad backprops. + // + // gradients: gradients backpropagated to the Relu6 op. + // features: inputs that where passed to the Relu6 op, or its outputs. + // backprops: gradients to backpropagate to the Relu6 inputs. + void operator()(const Device& d, typename TTypes::ConstTensor gradients, + typename TTypes::ConstTensor features, + typename TTypes::Tensor backprops) { + // NOTE: When the activation is exactly zero or six, we + // make sure not to propagate the associated gradient + // value. This allows "features" to be either the input or the output of + // the relu6. + backprops.device(d) = gradients * ((features > static_cast(0)) * + (features < static_cast(6))) + .template cast(); + } +}; + +// Functor used by EluOp to do the computations. +template +struct Elu { + // Computes Elu activation. + // + // features: any shape. + // activations: same shape as "features". + void operator()(const Device& d, typename TTypes::ConstTensor features, + typename TTypes::Tensor activations) { + // features.constant(?) + activations.device(d) = + (features < static_cast(0)) + .select(features.exp() - features.constant(static_cast(1)), + features); + } +}; + +// Functor used by EluGradOp to do the computations. +template +struct EluGrad { + // Computes EluGrad backprops. + // + // gradients: gradients backpropagated to the Elu op. + // activations: outputs of the Elu op. + // backprops: gradients to backpropagate to the Elu inputs. + void operator()(const Device& d, typename TTypes::ConstTensor gradients, + typename TTypes::ConstTensor activations, + typename TTypes::Tensor backprops) { + backprops.device(d) = + (activations < static_cast(0)) + .select((activations + static_cast(1)) * gradients, gradients); + } +}; + +// Functor used by SeluOp to do the computations. +template +struct Selu { + // Computes Selu activation. + // + // features: any shape. + // activations: same shape as "features". + void operator()(const Device& d, typename TTypes::ConstTensor features, + typename TTypes::Tensor activations) { + // features.constant(?) + const auto scale = static_cast(1.0507009873554804934193349852946); + const auto scale_alpha = static_cast(1.7580993408473768599402175208123); + const auto one = static_cast(1); + const auto zero = static_cast(0); + activations.device(d) = + (features < zero) + .select(scale_alpha * (features.exp() - features.constant(one)), + scale * features); + } +}; + +// Functor used by SeluGradOp to do the computations. +template +struct SeluGrad { + // Computes SeluGrad backprops. + // + // gradients: gradients backpropagated to the Selu op. + // activations: outputs of the Selu op. + // backprops: gradients to backpropagate to the Selu inputs. + void operator()(const Device& d, typename TTypes::ConstTensor gradients, + typename TTypes::ConstTensor activations, + typename TTypes::Tensor backprops) { + const auto scale = static_cast(1.0507009873554804934193349852946); + const auto scale_alpha = static_cast(1.7580993408473768599402175208123); + backprops.device(d) = + (activations < static_cast(0)) + .select(gradients * (activations + scale_alpha), gradients * scale); + } +}; + +} // namespace functor +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_RELU_OP_FUNCTOR_H_ diff --git a/linux/include/tensorflow/core/kernels/remote_fused_graph_execute_utils.h b/linux/include/tensorflow/core/kernels/remote_fused_graph_execute_utils.h new file mode 100644 index 000000000..ea6b6a101 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/remote_fused_graph_execute_utils.h @@ -0,0 +1,323 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_REMOTE_FUSED_GRAPH_EXECUTE_UTILS_H_ +#define TENSORFLOW_CORE_KERNELS_REMOTE_FUSED_GRAPH_EXECUTE_UTILS_H_ + +#include +#include + +#include "tensorflow/core/graph/graph.h" +#include "tensorflow/core/graph/graph_constructor.h" +#include "tensorflow/core/kernels/i_remote_fused_graph_executor.h" +#include "tensorflow/core/kernels/i_remote_fused_graph_ops_definitions.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/platform/macros.h" + +namespace tensorflow { + +enum RemoteFusedGraphNodeType { + UNUSED = 0, + GRAPH_INPUT = 1, + GRAPH_OUTPUT = 2, + FUSED_NODE = 3, + BORDER_INPUT = 4, + BORDER_OUTPUT = 5, +}; + +class RemoteFusedGraphExecuteInfo; + +// RemoteFusedGraphExecuteUtils provides APIs to register and get builder +// functions for IRemoteFusedGraphExecutor. +class RemoteFusedGraphExecuteUtils { + public: + // TODO(satok): Use "_output_data_types" to share a spec with other ops + static constexpr const char* const ATTR_OUTPUT_DATA_TYPES = + "_default_remote_graph_output_data_types"; + // TODO(satok): Use "_output_shapes" to share a spec with other ops + static constexpr const char* const ATTR_OUTPUT_SHAPES = + "_default_remote_output_shapes"; + static constexpr const char* const + ATTR_SERIALIZED_REMOTE_FUSED_GRAPH_EXECUTE_INFO = + "serialized_remote_fused_graph_execute_info"; + static constexpr const char* const ATTR_NODE_TYPE = + "_remote_fused_graph_node_type"; + + // Argument key strings to fuse a subgraph into RemoteFusedGraphExecuteOp. + static constexpr const char* const + TRANSFORM_ARG_REMOTE_FUSED_GRAPH_EXECUTOR_NAME = + "remote_fused_graph_executor_name"; + static constexpr const char* const + TRANSFORM_ARG_REMOTE_FUSED_GRAPH_NODE_NAME = + "remote_fused_graph_node_name"; + static constexpr const char* const TRANSFORM_ARG_FUSED_NODES = "fused_nodes"; + static constexpr const char* const TRANSFORM_ARG_BORDER_INPUTS = + "border_inputs"; + static constexpr const char* const TRANSFORM_ARG_BORDER_OUTPUTS = + "border_outputs"; + static constexpr const char* const TRANSFORM_ARG_FUSED_OP_TYPES = + "fused_op_types"; + static constexpr const char* const TRANSFORM_ARG_FUSE_BY_EXECUTOR = + "fuse_by_executor"; + static constexpr const char* const TRANSFORM_ARG_INPUT_TYPES = "input_types"; + static constexpr const char* const TRANSFORM_ARG_INPUT_SHAPES = + "input_shapes"; + + using ExecutorBuildFunc = std::function* executor)>; + // Registrar class for IRemoteFusedGraphExecutor. + class ExecutorBuildRegistrar { + public: + ExecutorBuildRegistrar(const string& name, ExecutorBuildFunc func); + + private: + TF_DISALLOW_COPY_AND_ASSIGN(ExecutorBuildRegistrar); + }; + using ExecutorBuildRegistry = std::map; + + using TensorShapeType = std::pair; + using TensorShapeMap = std::unordered_multimap>; + using ClusterInfo = std::tuple, // node names + std::vector, // border inputs + std::vector>; // border outputs + + // Return registered ExecutorBuildFunc for given name. + static const ExecutorBuildFunc* GetExecutorBuildFunc(const string& name); + + // To determine shapes of output tensors of all nodes, dryrun the graph. + // This function supplies memory allocation information when loading + // the graph. This function is used to verify shape inference and actual + // output shape. + static Status DryRunInference( + const GraphDef& graph_def, + const std::vector>& input_node_info_list, + const std::vector& output_node_names, + const bool initialize_by_zero, + std::vector* output_tensors); + + // Dry run inference to obtain shapes for all nodes. + // CAVEAT: Do not add or modify output_tensors in output_tensor_info + // otherwise, address map may be broken by re-allocation inside + // std::vector. + static Status DryRunInferenceForAllNode( + const GraphDef& graph_def, + const std::vector>& input_node_info_list, + const bool initialize_by_zero, TensorShapeMap* tensor_shape_map); + + static bool IsInputNode( + const std::vector>& input_node_info_list, + const string& node_name); + + static void ConvertToTensorShapeMap( + const std::vector>& input_node_info_list, + const std::vector& output_node_names, + const std::vector& output_tensors, + TensorShapeMap* tensor_shape_map); + + static Status MakeTensorFromProto(const TensorProto& tensor_proto, + Tensor* tensor); + + static bool AddOutputTensorShapeType(const std::vector& data_types, + const std::vector& shapes, + NodeDef* node_def); + + static Status AddOutputTensorShapeTypeByTensorShapeMap( + const TensorShapeMap& tensor_shape_map, NodeDef* node_def); + + static Status GetOutputTensorShapeType(AttrSlice attrs, + std::vector* data_types, + std::vector* shapes); + + static bool GetOutputTensorShapeType(const GraphDef& graph_def, + const string& name_and_port, + DataType* data_type, TensorShape* shape); + + static Status PropagateShapeInference( + const GraphDef& graph_def, + const std::vector>& input_node_info_list, + Graph* graph, ShapeRefiner* shape_refiner); + + static Status BuildTensorShapeMapFromGraph(const Graph& graph, + const ShapeRefiner& shape_refiner, + TensorShapeMap* tensor_shape_map); + + static const TensorShapeType* GetTensorShapeType( + const TensorShapeMap& tensor_shape_map, const string& node_name); + + static const TensorShapeType* GetTensorShapeType( + const TensorShapeMap& tensor_shape_map, const string& node_name, + const int port); + + static void BuildRemoteGraphInputsAndOutputsFromProto( + const RemoteFusedGraphExecuteInfo& proto, + std::vector>* inputs, + std::vector* outputs); + + static Status BuildAndAddTensorShapes( + const std::vector>& input_tensors, + const bool dry_run_inference, GraphDef* graph_def); + + // Build remote fused graph execute info. + static Status BuildRemoteFusedGraphExecuteInfo( + const string& executor_name, const GraphDef& subgraph_def, + const std::vector& inputs, const std::vector& outputs, + const bool require_shape_type, RemoteFusedGraphExecuteInfo* execute_info, + DataTypeVector* input_types, DataTypeVector* output_types); + + // Build remote fused graph execute op node by fusing specified subgraph + // as remote fused graph execute info. + static Status BuildRemoteFusedGraphExecuteOpNode( + const string& node_name, const string& executor_name, + const GraphDef& subgraph_def, const std::vector& inputs, + const std::vector& outputs, const bool require_shape_type, + Graph* graph, Node** created_node); + + // Build Identity node to forward remote graph node output. + static Status BuildIdentityOpNode(const string& node_name, + const string& input_node_name, + const int input_node_port, + const DataType dt, Graph* graph, + Node** created_node); + + // Create clusters of given nodes. + static Status ClusterizeNodes(const std::unordered_set& node_names, + const GraphDef& graph_def, + std::vector* cluster_infos); + + // Build GraphDef of a given cluster. + static Status BuildClusterSubgraphDef(const ClusterInfo& cluster, + const GraphDef& graph_def, + GraphDef* subgraph_def); + + // Build a cluster by given border. + // CAVEAT: The border must be consistent for one cluster. + static Status BuildClusterByBorder(const std::vector& border_inputs, + const std::vector& border_outputs, + const GraphDef& graph_def, + ClusterInfo* cluster); + + // Fuse one cluster into a newly created RemoteFusedGraphExecuteOp node. + // The subgraph is stored as a graph in RemoteFusedGraphExecuteInfo. + // CAVEAT1: This transform strips unvisited nodes with given outputs. + // CAVEAT2: If you want to use a graph output as a border output, + // that graph output node is replaced by an identity node. Therefore, + // the number of output of the node must be 1. + static Status FuseCluster(const GraphDef& input_graph_def, + const std::vector& inputs, + const std::vector& outputs, + const string& remote_fused_graph_node_name, + const ClusterInfo& cluster, + const string& remote_graph_executor_name, + const bool require_shape_type, + GraphDef* output_graph_def); + + // Fuse subgraph of specified nodes. + static Status FuseRemoteGraphByNodeNames( + const GraphDef& input_graph_def, const std::vector& inputs, + const std::vector& outputs, + const string& remote_fused_graph_node_name_prefix, + const std::unordered_set& subgraph_nodes, + const string& remote_fused_graph_executor_name, + const bool require_shape_type, GraphDef* output_graph_def); + + // Fuse subgraph of specified border. + static Status FuseRemoteGraphByBorder( + const GraphDef& input_graph_def, const std::vector& inputs, + const std::vector& outputs, + const string& remote_fused_graph_node_name, + const std::vector& border_inputs, + const std::vector& border_outputs, + const string& remote_graph_executor_name, const bool require_shape_type, + GraphDef* output_graph_def); + + // Fuse subgraph of specified op types. + static Status FuseRemoteGraphByOpTypes( + const GraphDef& input_graph_def, const std::vector& inputs, + const std::vector& outputs, + const string& remote_fused_graph_node_name_prefix, + const std::unordered_set& fused_op_types, + const string& remote_fused_graph_executor_name, + const bool require_shape_type, GraphDef* output_graph_def); + + // Place arguments to fuse remote graph. + static Status PlaceRemoteGraphArguments( + const std::vector& inputs, const std::vector& outputs, + const std::unordered_set& fused_node_names, + const std::vector& border_inputs, + const std::vector& border_outputs, + const std::unordered_set& fused_op_types, + const string& remote_fused_graph_node_name, + const string& remote_graph_executor_name, GraphDef* graph_def); + + // Fuse remote graph by placed arguments. + static Status FuseRemoteGraphByPlacedArguments( + const GraphDef& input_graph_def, + const std::vector>& input_tensors, + GraphDef* output_graph_def); + + static Status FuseRemoteGraphByExecutor(const GraphDef& input_graph_def, + const std::vector& inputs, + const std::vector& outputs, + const string& executor_name, + GraphDef* output_graph_def); + + static bool IsFuseReady( + const GraphDef& input_graph_def, + const std::vector>& input_tensors); + + // Copy a byte array to a tensor data. Though tensor data must be + // updated with typed information in general, we can't guarantee that + // returned values from a remote processor has typed information because + // a logic running in the remote processor possibly be in a separate binary + // which may not link tensorflow libraries. To deal with this situation, + // remote fused graph needs to overwrite the tensor data by a byte array. + static Status CopyByteArrayToTensor(const void* src_ptr, const int src_size, + Tensor* tensor); + + static std::unordered_set BuildNodeMapFromOpTypes( + const GraphDef& graph_def, const std::unordered_set& op_types); + + static std::unordered_set BuildNodeMapFromOpsDefinitions( + const GraphDef& graph_def, + const IRemoteFusedGraphOpsDefinitions& ops_definitions); + + private: + static void EmplaceTensorShapeType(const string& name, const Tensor& tensor, + TensorShapeMap* tensor_shape_map); + + static Status ReplaceInputNodeByPlaceHolder(const string& input, + const DataType type, + const TensorShape& shape, + GraphDef* graph_def); + + static ExecutorBuildRegistry* GetExecutorBuildRegistry(); + + static string BuildNodeTypeAttr(const RemoteFusedGraphNodeType node_type, + const int port, const int index, + const string& executor_name, + const string& node_name); + + static string BuildNodeTypeAttr(const RemoteFusedGraphNodeType node_type, + const int port, const int index); + + static string BuildNodeTypeAttr(const RemoteFusedGraphNodeType node_type); + + TF_DISALLOW_COPY_AND_ASSIGN(RemoteFusedGraphExecuteUtils); +}; +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_REMOTE_FUSED_GRAPH_EXECUTE_UTILS_H_ diff --git a/linux/include/tensorflow/core/kernels/reshape_op.h b/linux/include/tensorflow/core/kernels/reshape_op.h new file mode 100644 index 000000000..7458ac75c --- /dev/null +++ b/linux/include/tensorflow/core/kernels/reshape_op.h @@ -0,0 +1,124 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_RESHAPE_OP_H_ +#define TENSORFLOW_CORE_KERNELS_RESHAPE_OP_H_ + +#include +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/register_types.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/platform/logging.h" + +namespace tensorflow { + +// Note that this op is subclassed for QuantizedReshapeOp. +class ReshapeOp : public OpKernel { + public: + explicit ReshapeOp(OpKernelConstruction* context) : OpKernel(context) {} + + void Compute(OpKernelContext* context) override { + const Tensor& input = context->input(0); + const Tensor& sizes = context->input(1); + // Preliminary validation of sizes. + OP_REQUIRES(context, IsLegacyVector(sizes.shape()), + errors::InvalidArgument("sizes input must be 1-D, not ", + sizes.shape().DebugString())); + + // Compute the output shape. Determine product of specified + // dimensions, and find the index of the unspecified one. + TensorShape shape; + int64 product = 1; + int unknown_index = -1; + switch (sizes.dtype()) { + case DT_INT32: + OP_REQUIRES_OK(context, ValidateSizes(sizes, &product, + &unknown_index, &shape)); + break; + case DT_INT64: + OP_REQUIRES_OK(context, ValidateSizes(sizes, &product, + &unknown_index, &shape)); + break; + default: + context->CtxFailure(errors::InvalidArgument( + "desired shape must be a DT_INT32 or DT_INT64 vector, not a ", + DataTypeString(sizes.dtype()))); + return; + } + if (unknown_index != -1) { + OP_REQUIRES( + context, product > 0, + errors::InvalidArgument("Reshape cannot infer the missing input size " + "for an empty tensor unless all specified " + "input sizes are non-zero")); + const int64 missing = input.NumElements() / product; + OP_REQUIRES( + context, product * missing == input.NumElements(), + errors::InvalidArgument( + "Input to reshape is a tensor with ", input.NumElements(), + " values, but the requested shape requires a multiple of ", + product)); + shape.set_dim(unknown_index, missing); + } + OP_REQUIRES(context, shape.num_elements() == input.NumElements(), + errors::InvalidArgument("Input to reshape is a tensor with ", + input.NumElements(), + " values, but the requested shape has ", + shape.num_elements())); + + // Actually produce the reshaped output. + Tensor output(input.dtype()); + CHECK(output.CopyFrom(input, shape)); + context->set_output(0, output); + } + + bool IsExpensive() override { return false; } + + private: + template + Status ValidateSizes(const Tensor& sizes, int64* product, int* unknown_index, + TensorShape* shape) { + *product = 1; + *unknown_index = -1; + const int64 num_dims = sizes.NumElements(); + auto Svec = sizes.flat(); + for (int d = 0; d < num_dims; ++d) { + const Tshape size = Svec(d); + if (size == -1) { + if (*unknown_index != -1) { + return errors::InvalidArgument( + "Only one input size may be -1, not both ", *unknown_index, + " and ", d); + } + *unknown_index = d; + shape->AddDim(1); + } else if (size < 0) { + return errors::InvalidArgument("Size ", d, + " must be non-negative, not ", size); + } else { + shape->AddDim(size); + (*product) *= size; + } + } + return Status::OK(); + } +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_RESHAPE_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/reshape_util.h b/linux/include/tensorflow/core/kernels/reshape_util.h new file mode 100644 index 000000000..6777748b6 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/reshape_util.h @@ -0,0 +1,31 @@ + +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CORE_KERNELS_RESHAPE_UTIL_H_ +#define TENSORFLOW_CORE_KERNELS_RESHAPE_UTIL_H_ + +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/lib/core/status.h" + +namespace tensorflow { + +// Reshapes the input indices and input shape to the target shape. +void Reshape(OpKernelContext *context, const Tensor &input_indices_in, + const Tensor &input_shape_in, const Tensor &target_shape_in, + int output_indices_idx, int output_shape_idx); + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_RESHAPE_UTIL_H_ diff --git a/linux/include/tensorflow/core/kernels/resize_bilinear_op.h b/linux/include/tensorflow/core/kernels/resize_bilinear_op.h new file mode 100644 index 000000000..d16ed64f1 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/resize_bilinear_op.h @@ -0,0 +1,44 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_RESIZE_BILINEAR_OP_H_ +#define TENSORFLOW_CORE_KERNELS_RESIZE_BILINEAR_OP_H_ + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/numeric_types.h" +#include "tensorflow/core/framework/tensor_types.h" + +namespace tensorflow { +namespace functor { + +template +struct ResizeBilinear { + void operator()(const Device& d, typename TTypes::ConstTensor images, + const float height_scale, const float width_scale, + typename TTypes::Tensor resized_images); +}; + +template +struct ResizeBilinearGrad { + void operator()(const Device& d, + typename TTypes::ConstTensor input_grad, + const float height_scale, const float width_scale, + typename TTypes::Tensor output_grad); +}; + +} // namespace functor +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_RESIZE_BILINEAR_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/resize_nearest_neighbor_op.h b/linux/include/tensorflow/core/kernels/resize_nearest_neighbor_op.h new file mode 100644 index 000000000..9db331ffd --- /dev/null +++ b/linux/include/tensorflow/core/kernels/resize_nearest_neighbor_op.h @@ -0,0 +1,43 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_RESIZE_NEAREST_NEIGHBOR_OP_H_ +#define TENSORFLOW_CORE_KERNELS_RESIZE_NEAREST_NEIGHBOR_OP_H_ + +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { +namespace functor { + +template +struct ResizeNearestNeighbor { + bool operator()(const Device& d, typename TTypes::ConstTensor input, + const float height_scale, const float width_scale, + typename TTypes::Tensor output); +}; + +template +struct ResizeNearestNeighborGrad { + bool operator()(const Device& d, + typename TTypes::ConstTensor input_grad, + const float height_scale, const float width_scale, + typename TTypes::Tensor output_grad); +}; + +} // namespace functor +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_RESIZE_NEAREST_NEIGHBOR_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/resource_variable_ops.h b/linux/include/tensorflow/core/kernels/resource_variable_ops.h new file mode 100644 index 000000000..cffb732c3 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/resource_variable_ops.h @@ -0,0 +1,52 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CORE_KERNELS_RESOURCE_VARIABLE_OPS_H_ +#define TENSORFLOW_CORE_KERNELS_RESOURCE_VARIABLE_OPS_H_ + +#include "tensorflow/core/framework/op_kernel.h" + +namespace tensorflow { + +class ReadVariableOp : public OpKernel { + public: + explicit ReadVariableOp(OpKernelConstruction* c); + void Compute(OpKernelContext* ctx) override; + + private: + DataType dtype_; +}; + +class ReadVariablesOp : public OpKernel { + public: + explicit ReadVariablesOp(OpKernelConstruction* c); + void Compute(OpKernelContext* ctx) override; + bool IsExpensive() override { return false; } + + private: + DataTypeVector dtypes_; +}; + +class DestroyResourceOp : public OpKernel { + public: + explicit DestroyResourceOp(OpKernelConstruction* ctx); + void Compute(OpKernelContext* ctx) override; + + private: + bool ignore_lookup_error_; +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_RESOURCE_VARIABLE_OPS_H_ diff --git a/linux/include/tensorflow/core/kernels/reverse_op.h b/linux/include/tensorflow/core/kernels/reverse_op.h new file mode 100644 index 000000000..44e7967c5 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/reverse_op.h @@ -0,0 +1,48 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_REVERSE_OP_H_ +#define TENSORFLOW_CORE_KERNELS_REVERSE_OP_H_ + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/tensor_types.h" + +namespace tensorflow { +namespace functor { + +// Functor used by ReverseOp to do the computations. +template +struct Reverse { + void operator()(const Device& d, typename TTypes::ConstTensor input, + const Eigen::array& reverse_dims, + typename TTypes::Tensor output) { + output.device(d) = input.reverse(reverse_dims); + } +}; + +template +struct Reverse { + void operator()(const Device& d, typename TTypes::ConstTensor input, + const Eigen::array& reverse_dims, + typename TTypes::Tensor output) { + // Reversing a scalar is copying it. + output.device(d) = input; + } +}; + +} // namespace functor +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_REVERSE_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/reverse_sequence_op.h b/linux/include/tensorflow/core/kernels/reverse_sequence_op.h new file mode 100644 index 000000000..d6ba2781a --- /dev/null +++ b/linux/include/tensorflow/core/kernels/reverse_sequence_op.h @@ -0,0 +1,78 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_REVERSE_SEQUENCE_OP_H_ +#define TENSORFLOW_CORE_KERNELS_REVERSE_SEQUENCE_OP_H_ +// Generator definition for ReverseSequenceOp, must be compilable by nvcc. + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +namespace generator { + +template +class ReverseGenerator { + public: + EIGEN_DEVICE_FUNC EIGEN_ALWAYS_INLINE + ReverseGenerator(typename TTypes::ConstTensor input, int32 batch_dim, + int32 seq_dim, typename TTypes::ConstVec seq_lengths) + : input_(input), + batch_dim_(batch_dim), + seq_dim_(seq_dim), + seq_lengths_(seq_lengths) {} + + EIGEN_DEVICE_FUNC EIGEN_ALWAYS_INLINE T + operator()(const Eigen::array& coords) const { + Eigen::array new_coords = coords; + if (coords[seq_dim_] < seq_lengths_(coords[batch_dim_])) { + new_coords[seq_dim_] = + seq_lengths_(coords[batch_dim_]) - coords[seq_dim_] - 1; + } + + return input_(new_coords); + } + + private: + typename TTypes::ConstTensor input_; + int32 batch_dim_; + int32 seq_dim_; + typename TTypes::ConstVec seq_lengths_; +}; + +} // namespace generator + +namespace functor { + +template +struct ReverseSequence { + EIGEN_ALWAYS_INLINE static void Compute( + const Device& d, typename TTypes::ConstTensor input, + int32 batch_dim, int32 seq_dim, + typename TTypes::ConstVec seq_lengths, + typename TTypes::Tensor output) { + generator::ReverseGenerator generator(input, batch_dim, + seq_dim, seq_lengths); + output.device(d) = input.generate(generator); + } +}; + +} // namespace functor + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_REVERSE_SEQUENCE_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/save_restore_tensor.h b/linux/include/tensorflow/core/kernels/save_restore_tensor.h new file mode 100644 index 000000000..be7f4b889 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/save_restore_tensor.h @@ -0,0 +1,73 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_SAVE_RESTORE_TENSOR_H_ +#define TENSORFLOW_CORE_KERNELS_SAVE_RESTORE_TENSOR_H_ + +#include "tensorflow/core/util/tensor_slice_reader.h" +#include "tensorflow/core/util/tensor_slice_writer.h" + +namespace tensorflow { + +class OpKernelContext; + +// Legacy / V1 checkpoint format. + +// Save input tensors in *context to a writer built from builder_func(). +// context must have the following inputs: +// 0: a single element string tensor that contains the file name. +// 1: names for the remaining tensors +// If save_slices is true: +// 2: shape and slice specifications. +// rest: tensors to save +void SaveTensors( + OpKernelContext* context, + checkpoint::TensorSliceWriter::CreateBuilderFunction builder_func, + bool save_slices); + +// Reads a single tensor from the reader built from open_func() and produces +// it as context->output(restore_index). "preferred_shard" is the same the +// TensorSliceReader preferred_shard parameter. +// +// context must have the following inputs: +// 0: a single element string tensor that contains the file name. +// 1: string tensor that names the outputs to be restored. +// If restore_slice is true: +// 2: shape and slice specification of the tensors to restore. +// +// restore_index indicates the variable name and slice to lookup +// in context(1) and (2). +void RestoreTensor(OpKernelContext* context, + checkpoint::TensorSliceReader::OpenTableFunction open_func, + int preferred_shard, bool restore_slice, int restore_index); + +// V2 checkpoint format. + +// Invokes the V2 checkpoint read path to read tensors. +// +// "context" is only used for allocating outputs. In particular, the inputs are +// explicitly provided and not accessed via the "input(i)" methods. +// REQUIRES: +// * "prefix" has 1 element, DT_STRING. +// * "tensor_names" and "shape_and_slices" shaped {N}, both DT_STRING. +// * "dtypes" has N elements, the datatypes of the to-restore tensors. +Status RestoreTensorsV2(OpKernelContext* context, const Tensor& prefix, + const Tensor& tensor_names, + const Tensor& shape_and_slices, + gtl::ArraySlice dtypes); + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_SAVE_RESTORE_TENSOR_H_ diff --git a/linux/include/tensorflow/core/kernels/scan_ops.h b/linux/include/tensorflow/core/kernels/scan_ops.h new file mode 100644 index 000000000..13831bb37 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/scan_ops.h @@ -0,0 +1,46 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_SCAN_OPS_H_ +#define TENSORFLOW_CORE_KERNELS_SCAN_OPS_H_ + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/tensor_types.h" + +namespace tensorflow { +namespace functor { + +typedef Eigen::Index Index; + +template +struct Scan { + void operator()(const Device& d, typename TTypes::ConstTensor in, + typename TTypes::Tensor out, const Reducer& reducer, + const bool reverse, const bool exclusive) { + // Perform the reverse ops directly with Eigen, which avoids copying the + // tensor twice compared to using individual ops. + Eigen::array dims; + dims[0] = false; + dims[1] = reverse; + dims[2] = false; + To32Bit(out).device(d) = + To32Bit(in).reverse(dims).scan(1, reducer, exclusive).reverse(dims); + } +}; + +} // namespace functor +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_SCAN_OPS_H_ diff --git a/linux/include/tensorflow/core/kernels/scatter_functor.h b/linux/include/tensorflow/core/kernels/scatter_functor.h new file mode 100644 index 000000000..2d43bde23 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/scatter_functor.h @@ -0,0 +1,491 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_SCATTER_FUNCTOR_H_ +#define TENSORFLOW_CORE_KERNELS_SCATTER_FUNCTOR_H_ + +#include + +#include "third_party/eigen3/Eigen/Core" +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/variant_op_registry.h" +#include "tensorflow/core/kernels/bounds_check.h" +#include "tensorflow/core/kernels/dense_update_functor.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +class OpKernelContext; +typedef Eigen::ThreadPoolDevice CPUDevice; +typedef Eigen::GpuDevice GPUDevice; +#ifdef TENSORFLOW_USE_SYCL +typedef Eigen::SyclDevice SYCLDevice; +#endif // TENSORFLOW_USE_SYCL + +namespace scatter_op { + +enum class UpdateOp { ASSIGN, ADD, SUB, MUL, DIV, MIN, MAX }; + +namespace internal { + +template +struct Assign {}; +template <> +struct Assign { + template + static void Run(Params p, Update u) { + p = u; + } + template + static void RunScalar(Params p, Update u) { + p.setConstant(u); + } +}; +template <> +struct Assign { + template + static void Run(Params p, Update u) { + p += u; + } + template + static void RunScalar(Params p, Update u) { + p = p + u; + } +}; +template <> +struct Assign { + template + static void Run(Params p, Update u) { + p -= u; + } + template + static void RunScalar(Params p, Update u) { + p = p + static_cast(-u); + } +}; +template <> +struct Assign { + template + static void Run(Params p, Update u) { + p *= u; + } + template + static void RunScalar(Params p, Update u) { + p = p * u; + } +}; +template <> +struct Assign { + template + static void Run(Params p, Update u) { + p /= u; + } + template + static void RunScalar(Params p, Update u) { + p = p / u; + } +}; +template <> +struct Assign { + // This method requires that Params and Update are tensor types. + template + static void Run(Params p, Update u) { + p = p.cwiseMin(u); + } + // Same thing, but for Update being a scalar type. + template + static void RunScalar(Params p, Update u) { + p = p.cwiseMin(u); + } +}; +template <> +struct Assign { + template + static void Run(Params p, Update u) { + p = p.cwiseMax(u); + } + template + static void RunScalar(Params p, Update u) { + p = p.cwiseMax(u); + } +}; + +#ifdef TENSORFLOW_USE_SYCL +template +struct AssignSYCL {}; +template <> +struct AssignSYCL { + template + static void Run(Device d, Params p, Update u) { + p.device(d) = u; + } +}; + +template <> +struct AssignSYCL { + template + static void Run(Device d, Params p, Update u) { + p.device(d) += u; + } +}; + +template <> +struct AssignSYCL { + template + static void Run(Device d, Params p, Update u) { + p.device(d) -= u; + } +}; + +template <> +struct AssignSYCL { + template + static void Run(Device d, Params p, Update u) { + p.device(d) = p * u; + } +}; + +template <> +struct AssignSYCL { + template + static void Run(Device d, Params p, Update u) { + p.device(d) = p / u; + } +}; + +template <> +struct AssignSYCL { + template + static void Run(Device d, Params p, Update u) { + p.device(d) = p.cwiseMin(u); + } +}; + +template <> +struct AssignSYCL { + template + static void Run(Device d, Params p, Update u) { + p.device(d) = p.cwiseMax(u); + } +}; +#endif // TENSORFLOW_USE_SYCL + +} // namespace internal +} // namespace scatter_op + +namespace functor { +template +struct ScatterFunctor { + Index operator()(OpKernelContext* c, const Device& d, + typename TTypes::Matrix params, + typename TTypes::ConstMatrix updates, + typename TTypes::ConstFlat indices); +}; + +template +struct ScatterFunctorBase { + Index operator()(OpKernelContext* c, const Device& d, + typename TTypes::Matrix params, + typename TTypes::ConstMatrix updates, + typename TTypes::ConstFlat indices) { + // indices and params sizes were validated in DoCompute(). + const Index N = static_cast(indices.size()); + const Index limit = static_cast(params.dimension(0)); + for (Index i = 0; i < N; i++) { + // Grab the index and check its validity. Do this carefully, + // to avoid checking the value and grabbing it again from + // memory a second time (a security risk since it may change in between). + const Index index = ::tensorflow::internal::SubtleMustCopy(indices(i)); + if (!FastBoundsCheck(index, limit)) return i; + // Copy last Ndim-1 dimensions of updates[i] to params[index] + scatter_op::internal::Assign::Run(params.template chip<0>(index), + updates.template chip<0>(i)); + } + return -1; + } +}; + +template +struct ScatterFunctorVariantAssignBase { + Index operator()(OpKernelContext* c, const Device& d, + typename TTypes::Matrix params, + typename TTypes::ConstMatrix updates, + typename TTypes::ConstFlat indices) { + // indices and params sizes were validated in DoCompute(). + const Index N = static_cast(indices.size()); + const Index limit = static_cast(params.dimension(0)); + const Index cols = static_cast(params.dimension(1)); + DCHECK_EQ(N, updates.dimension(0)); + DCHECK_EQ(cols, updates.dimension(1)); + for (Index i = 0; i < N; i++) { + // Grab the index and check its validity. Do this carefully, + // to avoid checking the value and grabbing it again from + // memory a second time (a security risk since it may change in between). + const Index index = ::tensorflow::internal::SubtleMustCopy(indices(i)); + if (!FastBoundsCheck(index, limit)) return i; + // Copy last Ndim-1 dimensions of updates[i] to params[index] + for (int j = 0; j < cols; ++j) { + const Variant& to_scatter = updates(i, j); + params(index, j) = to_scatter; + } + } + return -1; + } +}; + +template +struct ScatterFunctor + : ScatterFunctorVariantAssignBase {}; + +template +struct ScatterFunctor + : ScatterFunctorVariantAssignBase {}; + +#ifdef TENSORFLOW_USE_SYCL +template +struct ScatterFunctorBase { + Index operator()(OpKernelContext* c, const SYCLDevice& d, + typename TTypes::Matrix params, + typename TTypes::ConstMatrix updates, + typename TTypes::ConstFlat indices) { + // indices and params sizes were validated in DoCompute(). + const Index N = static_cast(indices.size()); + const Index limit = static_cast(params.dimension(0)); + for (Index i = 0; i < N; i++) { + // Grab the index and check its validity. Do this carefully, + // to avoid checking the value and grabbing it again from + // memory a second time (a security risk since it may change in between). + const Index index = ::tensorflow::internal::SubtleMustCopy(indices(i)); + if (!FastBoundsCheck(index, limit)) return i; + // Copy last Ndim-1 dimensions of updates[i] to params[index] + scatter_op::internal::AssignSYCL::Run( + d, params.template chip<0>(index), updates.template chip<0>(i)); + } + return -1; + } +}; +#endif // TENSORFLOW_USE_SYCL + +template +struct ScatterFunctorBase { + Index operator()(OpKernelContext* c, const CPUDevice& d, + typename TTypes::Matrix params, + typename TTypes::ConstMatrix updates, + typename TTypes::ConstFlat indices) { + // indices and params sizes were validated in DoCompute(). + const Index N = static_cast(indices.size()); + const Index limit = static_cast(params.dimension(0)); + if (!std::is_same::value) { + for (Index i = 0; i < N; i++) { + // Grab the index and check its validity. Do this carefully, + // to avoid checking the value and grabbing it again from + // memory a second time (a security risk since it may change in + // between). + const Index index = ::tensorflow::internal::SubtleMustCopy(indices(i)); + if (!FastBoundsCheck(index, limit)) return i; + memmove(params.data() + index * params.dimension(1), + updates.data() + i * updates.dimension(1), + updates.dimension(1) * sizeof(T)); + } + } else { + for (Index i = 0; i < N; i++) { + // Grab the index and check its validity. Do this carefully, + // to avoid checking the value and grabbing it again from + // memory a second time (a security risk since it may change in + // between). + const Index index = ::tensorflow::internal::SubtleMustCopy(indices(i)); + if (!FastBoundsCheck(index, limit)) return i; + // Copy last Ndim-1 dimensions of updates[i] to params[index] + scatter_op::internal::Assign::Run( + params.template chip<0>(index), updates.template chip<0>(i)); + } + } + return -1; + } +}; + +template +struct ScatterFunctor + : ScatterFunctorBase {}; + +#ifdef TENSORFLOW_USE_SYCL +template +struct ScatterFunctorSYCL { + Index operator()(OpKernelContext* c, const SYCLDevice& d, + typename TTypes::Matrix params, + typename TTypes::ConstMatrix updates, + typename TTypes::Flat indices) { + // indices and params sizes were validated in DoCompute(). + const Index N = static_cast(indices.size()); + const Index limit = static_cast(params.dimension(0)); + for (Index i = 0; i < N; i++) { + const Index index = ::tensorflow::internal::SubtleMustCopy(indices(i)); + if (!FastBoundsCheck(index, limit)) return i; + // Copy last Ndim-1 dimensions of updates[i] to params[index] + scatter_op::internal::AssignSYCL::Run( + d, params.template chip<0>(index), updates.template chip<0>(i)); + } + return -1; + } +}; +#endif // TENSORFLOW_USE_SYCL + +template +struct ScatterScalarFunctor { + Index operator()(OpKernelContext* c, const Device& d, + typename TTypes::Matrix params, + const typename TTypes::ConstScalar update, + typename TTypes::ConstFlat indices); +}; + +template +struct ScatterScalarFunctorBase { + Index operator()(OpKernelContext* c, const Device& d, + typename TTypes::Matrix params, + const typename TTypes::ConstScalar update, + typename TTypes::ConstFlat indices) { + // indices and params sizes were validated in DoCompute(). + const Index N = static_cast(indices.size()); + const Index limit = static_cast(params.dimension(0)); + for (Index i = 0; i < N; i++) { + // Grab the index and check its validity. Do this carefully, + // to avoid checking the value and grabbing it again from + // memory a second time (a security risk since it may change in between). + const Index index = ::tensorflow::internal::SubtleMustCopy(indices(i)); + if (!FastBoundsCheck(index, limit)) return i; + // Broadcast update to params[index] + scatter_op::internal::Assign::RunScalar( + params.template chip<0>(index), update()); + } + return -1; + } +}; + +template +struct ScatterScalarFunctorVariantAssignBase { + Index operator()(OpKernelContext* c, const Device& d, + typename TTypes::Matrix params, + const typename TTypes::ConstScalar update, + typename TTypes::ConstFlat indices) { + // indices and params sizes were validated in DoCompute(). + const Index N = static_cast(indices.size()); + const Index limit = static_cast(params.dimension(0)); + const Index cols = static_cast(params.dimension(1)); + const Variant& to_scatter = update(); + for (Index i = 0; i < N; i++) { + // Grab the index and check its validity. Do this carefully, + // to avoid checking the value and grabbing it again from + // memory a second time (a security risk since it may change in between). + const Index index = ::tensorflow::internal::SubtleMustCopy(indices(i)); + if (!FastBoundsCheck(index, limit)) return i; + // Broadcast update to params[index] + for (Index j = 0; j < cols; ++j) { + params(index, j) = to_scatter; + } + } + return -1; + } +}; + +template +struct ScatterScalarFunctor + : ScatterScalarFunctorVariantAssignBase {}; +template +struct ScatterScalarFunctor + : ScatterScalarFunctorVariantAssignBase {}; + +#ifdef TENSORFLOW_USE_SYCL +template +struct ScatterScalarFunctorBase { + Index operator()(OpKernelContext* c, const SYCLDevice& d, + typename TTypes::Matrix params, + const typename TTypes::ConstScalar update, + typename TTypes::ConstFlat indices) { + // indices and params sizes were validated in DoCompute(). + const Index N = static_cast(indices.size()); + const Index limit = static_cast(params.dimension(0)); + for (Index i = 0; i < N; i++) { + // Grab the index and check its validity. Do this carefully, + // to avoid checking the value and grabbing it again from + // memory a second time (a security risk since it may change in between). + const Index index = ::tensorflow::internal::SubtleMustCopy(indices(i)); + if (!FastBoundsCheck(index, limit)) return i; + // Broadcast update to params[index] + scatter_op::internal::AssignSYCL::RunScalar( + d, params.template chip<0>(index), update); + } + return -1; + } +}; +#endif // TENSORFLOW_USE_SYCL + +template +struct ScatterScalarFunctorBase { + Index operator()(OpKernelContext* c, const CPUDevice& d, + typename TTypes::Matrix params, + const typename TTypes::ConstScalar update, + typename TTypes::ConstFlat indices) { + // indices and params sizes were validated in DoCompute(). + const Index N = static_cast(indices.size()); + const Index limit = static_cast(params.dimension(0)); + for (Index i = 0; i < N; i++) { + // Grab the index and check its validity. Do this carefully, + // to avoid checking the value and grabbing it again from + // memory a second time (a security risk since it may change in between). + const Index index = ::tensorflow::internal::SubtleMustCopy(indices(i)); + if (!FastBoundsCheck(index, limit)) return i; + // Broadcast update to params[index] + scatter_op::internal::Assign::RunScalar( + params.template chip<0>(index), update()); + } + return -1; + } +}; + +template +struct ScatterScalarFunctor + : ScatterScalarFunctorBase {}; + +#ifdef TENSORFLOW_USE_SYCL +template +struct ScatterScalarFunctorSYCL { + Index operator()(OpKernelContext* c, const SYCLDevice& d, + typename TTypes::Matrix params, + const typename TTypes::ConstScalar update, + typename TTypes::Flat indices) { + // indices and params sizes were validated in DoCompute(). + const Index N = static_cast(indices.size()); + const Index limit = static_cast(params.dimension(0)); + for (Index i = 0; i < N; i++) { + const Index index = ::tensorflow::internal::SubtleMustCopy(indices(i)); + if (!FastBoundsCheck(index, limit)) return i; + // Broadcast update to params[index] + scatter_op::internal::AssignSYCL::Run( + d, params.template chip<0>(index), update()); + } + return -1; + } +}; +#endif // TENSORFLOW_USE_SYCL + +} // namespace functor +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_SCATTER_FUNCTOR_H_ diff --git a/linux/include/tensorflow/core/kernels/scatter_nd_op.h b/linux/include/tensorflow/core/kernels/scatter_nd_op.h new file mode 100644 index 000000000..8d04731aa --- /dev/null +++ b/linux/include/tensorflow/core/kernels/scatter_nd_op.h @@ -0,0 +1,75 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_KERNELS_SCATTER_ND_OP_H_ +#define TENSORFLOW_KERNELS_SCATTER_ND_OP_H_ + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" + +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/register_types.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/kernels/bounds_check.h" +#include "tensorflow/core/kernels/fill_functor.h" +#include "tensorflow/core/kernels/scatter_nd_op.h" +#include "tensorflow/core/platform/mutex.h" +#include "tensorflow/core/platform/types.h" +#include "tensorflow/core/util/util.h" + +namespace tensorflow { + +typedef Eigen::ThreadPoolDevice CPUDevice; + +class OpKernelContext; + +namespace scatter_nd_op { + +enum class UpdateOp { ASSIGN, ADD, SUB }; + +} // namespace scatter_nd_op + +namespace functor { + +// Functor used by ScatterOp to do the computations. +template +struct ScatterNdFunctor { + // Returns -1 on success or a nonnegative i s.t. indices[i] is a bad index. + Index operator()( + const Device& d, const Index slice_size, + const Eigen::array output_shape_prefix, + typename TTypes::Tensor Tparams, + typename TTypes::ConstTensor Tindices, + typename TTypes::ConstTensor Tupdates, + typename TTypes::Tensor Toutput); +}; + +// Scatter updates into indices in Tensor out. The argument allocate +// controls whether 'out' should be created. If allocate is true, +// *out will be updated to the scattered tensor upon successful completion. +// If allocate is false, out must point to a Tensor allocated with the +// right type (T) and shape. This tensor will not be zeroed out +// before the scatter is executed. +template +Status DoScatterNd(OpKernelContext* c, const Tensor& indices, + const Tensor& updates, const TensorShape& shape, Tensor* out, + bool allocate); + +} // namespace functor +} // namespace tensorflow + +#endif // TENSORFLOW_KERNELS_SCATTER_ND_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/scatter_nd_op_cpu_impl.h b/linux/include/tensorflow/core/kernels/scatter_nd_op_cpu_impl.h new file mode 100644 index 000000000..472f5a354 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/scatter_nd_op_cpu_impl.h @@ -0,0 +1,262 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_SCATTER_ND_OP_CPU_IMPL_H_ +#define TENSORFLOW_CORE_KERNELS_SCATTER_ND_OP_CPU_IMPL_H_ + +// Functor definitions for ScatterND ops, must be compilable by nvcc. + +#define EIGEN_USE_THREADS + +#include + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" + +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/register_types.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/kernels/bounds_check.h" +#include "tensorflow/core/kernels/fill_functor.h" +#include "tensorflow/core/kernels/scatter_nd_op.h" +#include "tensorflow/core/platform/mutex.h" +#include "tensorflow/core/platform/types.h" +#include "tensorflow/core/util/util.h" + +namespace tensorflow { + +typedef Eigen::ThreadPoolDevice CPUDevice; +#ifdef TENSORFLOW_USE_SYCL +typedef Eigen::SyclDevice SYCLDevice; +#endif // TENSORFLOW_USE_SYCL + +class OpKernelContext; + +// Specialization of UpdateExecutor to CPU +namespace update_executor { + +template +class UpdateExecutor { + public: + EIGEN_STRONG_INLINE static void Execute(Input value, Update update, + Output output); +}; + +template +class UpdateExecutor { + public: + EIGEN_STRONG_INLINE static void Execute(Input /* input */, Update update, + Output output) { + output = update; + } +}; + +template +class UpdateExecutor { + public: + EIGEN_STRONG_INLINE static void Execute(Input /* input */, Update update, + Output output) { + output += update; + } +}; + +template +class UpdateExecutor { + public: + EIGEN_STRONG_INLINE static void Execute(Input /* input */, Update update, + Output output) { + output -= update; + } +}; + +} // namespace update_executor + +namespace functor { + +// Implementation of update functor for CPU. +template +struct ScatterNdFunctor { + Index operator()( + const CPUDevice& d, const Index slice_size, + const Eigen::array output_shape_prefix, + typename TTypes::Tensor Tparams, + typename TTypes::ConstTensor Tindices, + typename TTypes::ConstTensor Tupdates, + typename TTypes::Tensor Toutput) { + // error_loc is -1 if there's no out-of-bounds index, + // otherwise it is the location of an OOB index in Tindices. + Index error_loc = -1; + + const Eigen::DenseIndex batch_size = Tindices.dimension(0); + + Index batch_strides[IXDIM]; + for (int dim = IXDIM - 1; dim >= 0; --dim) { + if (dim == IXDIM - 1) { + batch_strides[dim] = 1; + } else { + batch_strides[dim] = + batch_strides[dim + 1] * output_shape_prefix[dim + 1]; + } + } + + for (Eigen::DenseIndex loc = 0; loc < batch_size; ++loc) { + Index i = 0; + bool out_of_bounds = false; + for (int dim = 0; dim < IXDIM; ++dim) { + const Index ix_d = internal::SubtleMustCopy(Tindices(loc, dim)); + out_of_bounds |= !FastBoundsCheck(ix_d, output_shape_prefix[dim]); + i += ix_d * batch_strides[dim]; + } + if (TF_PREDICT_FALSE(out_of_bounds)) { + error_loc = loc; + break; + } else { + auto input_chip = Toutput.template chip<0>(i); + auto output_chip = input_chip.device(d); + auto update_chip = Tupdates.template chip<0>(loc); + update_executor::UpdateExecutor< + decltype(input_chip), decltype(update_chip), decltype(output_chip), + OP>::Execute(input_chip, update_chip, output_chip); + } + } + + return error_loc; + } +}; + +#define REGISTER_SCATTER_ND_FULL(T, Index, op) \ + template Index \ + ScatterNdFunctor::operator()( \ + const CPUDevice& d, const Index slice_size, \ + const Eigen::array \ + output_shape_prefix, \ + typename TTypes::Tensor Tparams, \ + typename TTypes::ConstTensor Tindices, \ + typename TTypes::ConstTensor Tupdates, \ + typename TTypes::Tensor Toutput) + +#define REGISTER_SCATTER_ND_INDEX(type, op) \ + REGISTER_SCATTER_ND_FULL(type, int32, op); \ + REGISTER_SCATTER_ND_FULL(type, int64, op) + +#define REGISTER_SCATTER_ND_UPDATE(type) \ + REGISTER_SCATTER_ND_INDEX(type, scatter_nd_op::UpdateOp::ASSIGN); + +#define REGISTER_SCATTER_ND_MATH(type) \ + REGISTER_SCATTER_ND_INDEX(type, scatter_nd_op::UpdateOp::ADD); \ + REGISTER_SCATTER_ND_INDEX(type, scatter_nd_op::UpdateOp::SUB); + +TF_CALL_ALL_TYPES(REGISTER_SCATTER_ND_UPDATE); +REGISTER_SCATTER_ND_INDEX(string, scatter_nd_op::UpdateOp::ADD); +TF_CALL_NUMBER_TYPES(REGISTER_SCATTER_ND_MATH); +TF_CALL_bool(REGISTER_SCATTER_ND_MATH); +#undef REGISTER_SCATTER_ND_MATH +#undef REGISTER_SCATTER_ND_UPDATE +#undef REGISTER_SCATTER_ND_INDEX +#undef REGISTER_SCATTER_ND_FULL + +// Implementation of update functor for SYCL. +#ifdef TENSORFLOW_USE_SYCL + +template +struct ScatterNdFunctor { + Index operator()( + const SYCLDevice& d, const Index slice_size, + const Eigen::array output_shape_prefix, + typename TTypes::Tensor Tparams, + typename TTypes::ConstTensor Tindices, + typename TTypes::ConstTensor Tupdates, + typename TTypes::Tensor Toutput) { + // error_loc is -1 if there's no out-of-bounds index, + // otherwise it is the location of an OOB index in Tindices. + Index error_loc = -1; + + const Eigen::DenseIndex batch_size = Tindices.dimension(0); + + Index batch_strides[IXDIM]; + for (int dim = IXDIM - 1; dim >= 0; --dim) { + if (dim == IXDIM - 1) { + batch_strides[dim] = 1; + } else { + batch_strides[dim] = + batch_strides[dim + 1] * output_shape_prefix[dim + 1]; + } + } + + for (Eigen::DenseIndex loc = 0; loc < batch_size; ++loc) { + Index i = 0; + bool out_of_bounds = false; + for (int dim = 0; dim < IXDIM; ++dim) { + const Index ix_d = internal::SubtleMustCopy(Tindices(loc, dim)); + out_of_bounds |= !FastBoundsCheck(ix_d, output_shape_prefix[dim]); + i += ix_d * batch_strides[dim]; + } + if (TF_PREDICT_FALSE(out_of_bounds)) { + error_loc = loc; + break; + } else { + auto input_chip = Toutput.template chip<0>(i); + auto output_chip = input_chip.device(d); + auto update_chip = Tupdates.template chip<0>(loc); + update_executor::UpdateExecutor< + decltype(input_chip), decltype(update_chip), decltype(output_chip), + OP>::Execute(input_chip, update_chip, output_chip); + } + } + + return error_loc; + } +}; + +#define REGISTER_SCATTER_ND_FULL_SYCL(T, Index, op) \ + template Index \ + ScatterNdFunctor::operator()( \ + const SYCLDevice& d, const Index slice_size, \ + const Eigen::array \ + output_shape_prefix, \ + typename TTypes::Tensor Tparams, \ + typename TTypes::ConstTensor Tindices, \ + typename TTypes::ConstTensor Tupdates, \ + typename TTypes::Tensor Toutput) + +#define REGISTER_SCATTER_ND_INDEX_SYCL(type, op) \ + REGISTER_SCATTER_ND_FULL_SYCL(type, int32, op); \ + REGISTER_SCATTER_ND_FULL_SYCL(type, int64, op) + +#define REGISTER_SCATTER_ND_UPDATE_SYCL(type) \ + REGISTER_SCATTER_ND_INDEX_SYCL(type, scatter_nd_op::UpdateOp::ASSIGN); + +#define REGISTER_SCATTER_ND_MATH_SYCL(type) \ + REGISTER_SCATTER_ND_INDEX_SYCL(type, scatter_nd_op::UpdateOp::ADD); \ + REGISTER_SCATTER_ND_INDEX_SYCL(type, scatter_nd_op::UpdateOp::SUB); + +TF_CALL_GPU_NUMBER_TYPES_NO_HALF(REGISTER_SCATTER_ND_UPDATE_SYCL) +TF_CALL_GPU_NUMBER_TYPES_NO_HALF(REGISTER_SCATTER_ND_MATH_SYCL) +REGISTER_SCATTER_ND_UPDATE_SYCL(int32); +REGISTER_SCATTER_ND_MATH_SYCL(int32); + +#undef REGISTER_SCATTER_ND_MATH_SYCL +#undef REGISTER_SCATTER_ND_UPDATE_SYCL +#undef REGISTER_SCATTER_ND_INDEX_SYCL +#undef REGISTER_SCATTER_ND_FULL_SYCL + +#endif // TENSORFLOW_USE_SYCL + +} // namespace functor + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_SCATTER_ND_OP_CPU_IMPL_H_ diff --git a/linux/include/tensorflow/core/kernels/sdca_internal.h b/linux/include/tensorflow/core/kernels/sdca_internal.h new file mode 100644 index 000000000..1eff4b15f --- /dev/null +++ b/linux/include/tensorflow/core/kernels/sdca_internal.h @@ -0,0 +1,391 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_SDCA_INTERNAL_H_ +#define TENSORFLOW_CORE_KERNELS_SDCA_INTERNAL_H_ + +#define EIGEN_USE_THREADS + +#include +#include +#include +#include +#include +#include +#include +#include + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/device_base.h" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/kernels/loss.h" +#include "tensorflow/core/lib/core/coding.h" +#include "tensorflow/core/lib/core/errors.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/lib/core/stringpiece.h" +#include "tensorflow/core/lib/gtl/inlined_vector.h" +#include "tensorflow/core/lib/random/distribution_sampler.h" +#include "tensorflow/core/lib/strings/stringprintf.h" +#include "tensorflow/core/util/guarded_philox_random.h" +#include "tensorflow/core/util/work_sharder.h" + +namespace tensorflow { + +namespace sdca { + +// Statistics computed with input (ModelWeights, Example). +struct ExampleStatistics { + // Logits for each class. + // For binary case, this should be a vector of length 1; while for multiclass + // case, this vector has the same length as the number of classes, where each + // value corresponds to one class. + // Use InlinedVector to avoid heap allocation for small number of classes. + gtl::InlinedVector wx; + + // Logits for each class, using the previous weights. + gtl::InlinedVector prev_wx; + + // Sum of squared feature values occurring in the example divided by + // L2 * sum(example_weights). + double normalized_squared_norm = 0; + + // Num_weight_vectors equals to the number of classification classes in the + // multiclass case; while for binary case, it is 1. + ExampleStatistics(const int num_weight_vectors) + : wx(num_weight_vectors, 0.0), prev_wx(num_weight_vectors, 0.0) {} +}; + +class Regularizations { + public: + Regularizations() {} + + // Initialize() must be called immediately after construction. + Status Initialize(OpKernelConstruction* const context) { + TF_RETURN_IF_ERROR(context->GetAttr("l1", &symmetric_l1_)); + TF_RETURN_IF_ERROR(context->GetAttr("l2", &symmetric_l2_)); + shrinkage_ = symmetric_l1_ / symmetric_l2_; + return Status::OK(); + } + + // Proximal SDCA shrinking for L1 regularization. + double Shrink(const double weight) const { + const double shrinked = std::max(std::abs(weight) - shrinkage_, 0.0); + if (shrinked > 0.0) { + return std::copysign(shrinked, weight); + } + return 0.0; + } + + // Vectorized float variant of the above. + Eigen::Tensor EigenShrinkVector( + const Eigen::Tensor weights) const { + // Proximal step on the weights which is sign(w)*|w - shrinkage|+. + return weights.sign() * ((weights.abs() - weights.constant(shrinkage_)) + .cwiseMax(weights.constant(0.0))); + } + + // Matrix float variant of the above. + Eigen::Tensor EigenShrinkMatrix( + const Eigen::Tensor weights) const { + // Proximal step on the weights which is sign(w)*|w - shrinkage|+. + return weights.sign() * ((weights.abs() - weights.constant(shrinkage_)) + .cwiseMax(weights.constant(0.0))); + } + + float symmetric_l2() const { return symmetric_l2_; } + + private: + float symmetric_l1_ = 0; + float symmetric_l2_ = 0; + + // L1 divided by L2, pre-computed for use during weight shrinking. + double shrinkage_ = 0; + + TF_DISALLOW_COPY_AND_ASSIGN(Regularizations); +}; + +class ModelWeights; + +// Struct describing a single example. +class Example { + public: + // Compute matrix vector product between weights (a matrix) and features + // (a vector). This method also computes the normalized example norm used + // in SDCA update. + // For multiclass case, num_weight_vectors equals to the number of classes; + // while for binary case, it is 1. + const ExampleStatistics ComputeWxAndWeightedExampleNorm( + const int num_loss_partitions, const ModelWeights& model_weights, + const Regularizations& regularization, + const int num_weight_vectors) const; + + float example_label() const { return example_label_; } + + float example_weight() const { return example_weight_; } + + double squared_norm() const { return squared_norm_; } + + // Sparse features associated with the example. + // Indices and Values are the associated feature index, and values. Values + // can be optionally absent, in which we case we implicitly assume a value of + // 1.0f. + struct SparseFeatures { + std::unique_ptr::UnalignedConstVec> indices; + std::unique_ptr::UnalignedConstVec> + values; // nullptr encodes optional. + }; + + // A dense vector which is a row-slice of the underlying matrix. + struct DenseVector { + // Returns a row slice from the matrix. + Eigen::TensorMap> Row() + const { + return Eigen::TensorMap>( + data_matrix.data() + row_index * data_matrix.dimension(1), + data_matrix.dimension(1)); + } + + // Returns a row slice as a 1 * F matrix, where F is the number of features. + Eigen::TensorMap> + RowAsMatrix() const { + return Eigen::TensorMap>( + data_matrix.data() + row_index * data_matrix.dimension(1), 1, + data_matrix.dimension(1)); + } + + const TTypes::ConstMatrix data_matrix; + const int64 row_index; + }; + + private: + std::vector sparse_features_; + std::vector> dense_vectors_; + + float example_label_ = 0; + float example_weight_ = 0; + double squared_norm_ = 0; // sum squared norm of the features. + + // Examples fills Example in a multi-threaded way. + friend class Examples; + + // ModelWeights use each example for model update w += \alpha * x_{i}; + friend class ModelWeights; +}; + +// Weights related to features. For example, say you have two sets of sparse +// features i.e. age bracket and country, then FeatureWeightsDenseStorage hold +// the parameters for it. We keep track of the original weight passed in and the +// delta weight which the optimizer learns in each call to the optimizer. +class FeatureWeightsDenseStorage { + public: + FeatureWeightsDenseStorage(const TTypes::Matrix nominals, + TTypes::Matrix deltas) + : nominals_(nominals), deltas_(deltas) { + CHECK_GT(deltas.rank(), 1); + } + + // Check if a feature index is with-in the bounds. + bool IndexValid(const int64 index) const { + return index >= 0 && index < deltas_.dimension(1); + } + + // Nominals here are the original weight matrix. + TTypes::Matrix nominals() const { return nominals_; } + + // Delta weights durining mini-batch updates. + TTypes::Matrix deltas() const { return deltas_; } + + // Updates delta weights based on active dense features in the example and + // the corresponding dual residual. + void UpdateDenseDeltaWeights( + const Eigen::ThreadPoolDevice& device, + const Example::DenseVector& dense_vector, + const std::vector& normalized_bounded_dual_delta); + + private: + // The nominal value of the weight for a feature (indexed by its id). + const TTypes::Matrix nominals_; + // The accumulated delta weight for a feature (indexed by its id). + TTypes::Matrix deltas_; +}; + +// Similar to FeatureWeightsDenseStorage, but the underlying weights are stored +// in an unordered map. +class FeatureWeightsSparseStorage { + public: + FeatureWeightsSparseStorage(const TTypes::Vec indices, + const TTypes::Matrix nominals, + TTypes::Matrix deltas) + : nominals_(nominals), deltas_(deltas) { + // Create a map from sparse index to the dense index of the underlying + // storage. + for (int64 j = 0; j < indices.size(); ++j) { + indices_to_id_[indices(j)] = j; + } + } + + // Check if a feature index exists. + bool IndexValid(const int64 index) const { + return indices_to_id_.find(index) != indices_to_id_.end(); + } + + // Nominal value at a particular feature index and class label. + float nominals(const int class_id, const int64 index) const { + auto it = indices_to_id_.find(index); + return nominals_(class_id, it->second); + } + + // Delta weights durining mini-batch updates. + float deltas(const int class_id, const int64 index) const { + auto it = indices_to_id_.find(index); + return deltas_(class_id, it->second); + } + + // Updates delta weights based on active sparse features in the example and + // the corresponding dual residual. + void UpdateSparseDeltaWeights( + const Eigen::ThreadPoolDevice& device, + const Example::SparseFeatures& sparse_features, + const std::vector& normalized_bounded_dual_delta); + + private: + // The nominal value of the weight for a feature (indexed by its id). + const TTypes::Matrix nominals_; + // The accumulated delta weight for a feature (indexed by its id). + TTypes::Matrix deltas_; + // Map from feature index to an index to the dense vector. + std::unordered_map indices_to_id_; +}; + +// Weights in the model, wraps both current weights, and the delta weights +// for both sparse and dense features. +class ModelWeights { + public: + ModelWeights() {} + + bool SparseIndexValid(const int col, const int64 index) const { + return sparse_weights_[col].IndexValid(index); + } + + bool DenseIndexValid(const int col, const int64 index) const { + return dense_weights_[col].IndexValid(index); + } + + // Go through all the features present in the example, and update the + // weights based on the dual delta. + void UpdateDeltaWeights( + const Eigen::ThreadPoolDevice& device, const Example& example, + const std::vector& normalized_bounded_dual_delta); + + Status Initialize(OpKernelContext* const context); + + const std::vector& sparse_weights() const { + return sparse_weights_; + } + + const std::vector& dense_weights() const { + return dense_weights_; + } + + private: + std::vector sparse_weights_; + std::vector dense_weights_; + + TF_DISALLOW_COPY_AND_ASSIGN(ModelWeights); +}; + +// Examples contains all the training examples that SDCA uses for a mini-batch. +class Examples { + public: + Examples() {} + + // Returns the Example at |example_index|. + const Example& example(const int example_index) const { + return examples_.at(example_index); + } + + int sampled_index(const int id) const { return sampled_index_[id]; } + + // Adaptive SDCA in the current implementation only works for + // binary classification, where the input argument for num_weight_vectors + // is 1. + Status SampleAdaptiveProbabilities( + const int num_loss_partitions, const Regularizations& regularization, + const ModelWeights& model_weights, + const TTypes::Matrix example_state_data, + const std::unique_ptr& loss_updater, + const int num_weight_vectors); + + void RandomShuffle(); + + int num_examples() const { return examples_.size(); } + + int num_features() const { return num_features_; } + + // Initialize() must be called immediately after construction. + Status Initialize(OpKernelContext* const context, const ModelWeights& weights, + int num_sparse_features, + int num_sparse_features_with_values, + int num_dense_features); + + private: + // Reads the input tensors, and builds the internal representation for sparse + // features per example. This function modifies the |examples| passed in + // to build the sparse representations. + static Status CreateSparseFeatureRepresentation( + const DeviceBase::CpuWorkerThreads& worker_threads, int num_examples, + int num_sparse_features, const ModelWeights& weights, + const OpInputList& sparse_example_indices_inputs, + const OpInputList& sparse_feature_indices_inputs, + const OpInputList& sparse_feature_values_inputs, + std::vector* const examples); + + // Reads the input tensors, and builds the internal representation for dense + // features per example. This function modifies the |examples| passed in + // to build the sparse representations. + static Status CreateDenseFeatureRepresentation( + const DeviceBase::CpuWorkerThreads& worker_threads, int num_examples, + int num_dense_features, const ModelWeights& weights, + const OpInputList& dense_features_inputs, + std::vector* const examples); + + // Computes squared example norm per example i.e |x|^2. This function modifies + // the |examples| passed in and adds the squared norm per example. + static Status ComputeSquaredNormPerExample( + const DeviceBase::CpuWorkerThreads& worker_threads, int num_examples, + int num_sparse_features, int num_dense_features, + std::vector* const examples); + + // All examples in the batch. + std::vector examples_; + + // Adaptive sampling variables. + std::vector probabilities_; + std::vector sampled_index_; + std::vector sampled_count_; + + int num_features_ = 0; + + TF_DISALLOW_COPY_AND_ASSIGN(Examples); +}; + +} // namespace sdca +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_SDCA_INTERNAL_H_ diff --git a/linux/include/tensorflow/core/kernels/searchsorted_op.h b/linux/include/tensorflow/core/kernels/searchsorted_op.h new file mode 100644 index 000000000..f075bf0fa --- /dev/null +++ b/linux/include/tensorflow/core/kernels/searchsorted_op.h @@ -0,0 +1,52 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_SEARCHSORTED_OP_H_ +#define TENSORFLOW_CORE_KERNELS_SEARCHSORTED_OP_H_ + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/core/errors.h" + +namespace tensorflow { +namespace functor { + +template +struct UpperBoundFunctor { + // Searches for values in sorted_inputs and returns the greatest possible + // index where they maintain sorted order. + static Status Compute(OpKernelContext* context, + const typename TTypes::ConstTensor& sorted_inputs, + const typename TTypes::ConstTensor& values, + int batch_size, int num_inputs, int num_values, + typename TTypes::Tensor* output); +}; + +template +struct LowerBoundFunctor { + // Searches for values in sorted_inputs and returns the lowest possible + // index where they maintain sorted order. + static Status Compute(OpKernelContext* context, + const typename TTypes::ConstTensor& sorted_inputs, + const typename TTypes::ConstTensor& values, + int batch_size, int num_inputs, int num_values, + typename TTypes::Tensor* output); +}; +} // namespace functor + +} // end namespace tensorflow +#endif // TENSORFLOW_CORE_KERNELS_SEARCHSORTED_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/segment_reduction_ops.h b/linux/include/tensorflow/core/kernels/segment_reduction_ops.h new file mode 100644 index 000000000..d28e35157 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/segment_reduction_ops.h @@ -0,0 +1,147 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_SEGMENT_REDUCTION_OPS_H_ +#define TENSORFLOW_CORE_KERNELS_SEGMENT_REDUCTION_OPS_H_ + +// This file requires the following include because it uses CudaAtomicMax: +// #include "tensorflow/core/util/cuda_kernel_helper.h" + +// Unfortunately we can't add the #include, since it breaks compilation for +// non-GPU targets. This only breaks in clang, because it's more strict for +// template code and CudaAtomicMax is used in template context. + +// This file requires the following include because it uses CudaAtomicMax: +// #include "tensorflow/core/util/cuda_kernel_helper.h" + +// Unfortunately we can't add the #include, since it breaks compilation for +// non-GPU targets. This only breaks in clang, because it's more strict for +// template code and CudaAtomicMax is used in template context. + +// This file requires the following include because it uses CudaAtomicMax: +// #include "tensorflow/core/util/cuda_kernel_helper.h" + +// Unfortunately we can't add the #include, since it breaks compilation for +// non-GPU targets. This only breaks in clang, because it's more strict for +// template code and CudaAtomicMax is used in template context. + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/tensor_types.h" + +namespace tensorflow { + +class OpKernelContext; + +namespace functor { + +#ifdef GOOGLE_CUDA +typedef Eigen::GpuDevice GPUDevice; +// Functor for SegmentSumGPUOp. +// output_rows: the number of output segments (unique segment ids in +// 'segment_ids'). +// segment_ids_shape: shape of 'segment_ids' tensor. +// segment_ids: unsorted map from input to output segment ids at which to +// perform segment sum operation. +// data_size: size of input data tensor. +// data: input data tensor. +// output: output reshaped to {output_rows, output.size/output_rows} +template +struct SegmentSumFunctor { + void operator()(OpKernelContext* ctx, const GPUDevice& d, + const Index output_rows, const TensorShape& segment_ids_shape, + typename TTypes::ConstFlat segment_ids, + const Index data_size, const T* data, + typename TTypes::Tensor output); +}; + +#endif + +template +struct UnsortedSegmentFunctor { + void operator()(OpKernelContext* ctx, const Index num_segments, + const TensorShape& segment_ids_shape, + typename TTypes::ConstFlat segment_ids, + const Index data_size, const T* data, + typename TTypes::Tensor output); +}; + +#ifdef GOOGLE_CUDA +// reduction functors for the gpu +template +struct SumOpGpu { + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE void operator()(T* dest, + const T& value) { + CudaAtomicAdd(dest, value); + } +}; + +template +struct ProdOpGpu { + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE void operator()(T* dest, + const T& value) { + CudaAtomicMul(dest, value); + } +}; + +template +struct MaxOpGpu { + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE void operator()(T* dest, + const T& value) { + CudaAtomicMax(dest, value); + } +}; + +template +struct MinOpGpu { + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE void operator()(T* dest, + const T& value) { + CudaAtomicMin(dest, value); + } +}; + +#endif // GOOGLE_CUDA + +// initial value functors +template +struct Zero { + EIGEN_STRONG_INLINE T operator()() const { return T(0); } +}; + +template +struct One { + EIGEN_STRONG_INLINE T operator()() const { return T(1); } +}; + +template +struct Lowest { + EIGEN_STRONG_INLINE T operator()() const { + return Eigen::NumTraits::lowest(); + } +}; + +template +struct Highest { + EIGEN_STRONG_INLINE T operator()() const { + return Eigen::NumTraits::highest(); + } +}; + +} // namespace functor +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_SEGMENT_REDUCTION_OPS_H_ diff --git a/linux/include/tensorflow/core/kernels/self_adjoint_eig_v2_op_impl.h b/linux/include/tensorflow/core/kernels/self_adjoint_eig_v2_op_impl.h new file mode 100644 index 000000000..b5274f878 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/self_adjoint_eig_v2_op_impl.h @@ -0,0 +1,92 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_SELF_ADJOINT_EIG_V2_OP_IMPL_H_ +#define TENSORFLOW_CORE_KERNELS_SELF_ADJOINT_EIG_V2_OP_IMPL_H_ + +// See docs in ../ops/linalg_ops.cc. + +#include "third_party/eigen3/Eigen/Core" +#include "third_party/eigen3/Eigen/Eigenvalues" +#include "tensorflow/core/framework/kernel_def_builder.h" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/kernels/linalg_ops_common.h" +#include "tensorflow/core/lib/core/errors.h" +#include "tensorflow/core/platform/denormal.h" +#include "tensorflow/core/platform/logging.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +template +class SelfAdjointEigV2Op : public LinearAlgebraOp { + public: + typedef LinearAlgebraOp Base; + + explicit SelfAdjointEigV2Op(OpKernelConstruction* context) : Base(context) { + OP_REQUIRES_OK(context, context->GetAttr("compute_v", &compute_v_)); + } + + using TensorShapes = typename Base::TensorShapes; + using Matrix = typename Base::Matrix; + using MatrixMaps = typename Base::MatrixMaps; + using ConstMatrixMap = typename Base::ConstMatrixMap; + using ConstMatrixMaps = typename Base::ConstMatrixMaps; + + TensorShapes GetOutputMatrixShapes( + const TensorShapes& input_matrix_shapes) const final { + int64 n = input_matrix_shapes[0].dim_size(0); + if (compute_v_) { + return TensorShapes({TensorShape({n}), TensorShape({n, n})}); + } else { + return TensorShapes({TensorShape({n})}); + } + } + + void ComputeMatrix(OpKernelContext* context, const ConstMatrixMaps& inputs, + MatrixMaps* outputs) final { + const int64 rows = inputs[0].rows(); + if (rows == 0) { + // If X is an empty matrix (0 rows, 0 col), X * X' == X. + // Therefore, we return X. + return; + } + + // This algorithm relies on denormals, so switch them back on locally. + port::ScopedDontFlushDenormal dont_flush_denormals; + + Eigen::SelfAdjointEigenSolver eig( + inputs[0], + compute_v_ ? Eigen::ComputeEigenvectors : Eigen::EigenvaluesOnly); + // TODO(rmlarsen): Output more detailed error info on failure. + OP_REQUIRES( + context, eig.info() == Eigen::Success, + errors::InvalidArgument("Self-adjoint eigen decomposition was not " + "successful. The input might not be valid.")); + + outputs->at(0) = eig.eigenvalues().template cast(); + if (compute_v_) { + outputs->at(1) = eig.eigenvectors(); + } + } + + private: + bool compute_v_; +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_SELF_ADJOINT_EIG_V2_OP_IMPL_H_ diff --git a/linux/include/tensorflow/core/kernels/sendrecv_ops.h b/linux/include/tensorflow/core/kernels/sendrecv_ops.h new file mode 100644 index 000000000..223854de1 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/sendrecv_ops.h @@ -0,0 +1,52 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_SENDRECV_OPS_H_ +#define TENSORFLOW_CORE_KERNELS_SENDRECV_OPS_H_ + +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/platform/macros.h" + +namespace tensorflow { + +class SendOp : public OpKernel { + public: + explicit SendOp(OpKernelConstruction* ctx); + void Compute(OpKernelContext* ctx) override; + + private: + string key_prefix_; + Rendezvous::ParsedKey parsed_key_; + bool hostmem_sendrecv_; + + TF_DISALLOW_COPY_AND_ASSIGN(SendOp); +}; + +class RecvOp : public AsyncOpKernel { + public: + explicit RecvOp(OpKernelConstruction* ctx); + void ComputeAsync(OpKernelContext* ctx, DoneCallback done) override; + + private: + string key_prefix_; + Rendezvous::ParsedKey parsed_key_; + bool hostmem_sendrecv_; + + TF_DISALLOW_COPY_AND_ASSIGN(RecvOp); +}; + +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_SENDRECV_OPS_H_ diff --git a/linux/include/tensorflow/core/kernels/shape_ops.h b/linux/include/tensorflow/core/kernels/shape_ops.h new file mode 100644 index 000000000..7a50f158a --- /dev/null +++ b/linux/include/tensorflow/core/kernels/shape_ops.h @@ -0,0 +1,277 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_SHAPE_OPS_H_ +#define TENSORFLOW_CORE_KERNELS_SHAPE_OPS_H_ + +#include +#include +#include + +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/variant_op_registry.h" +#include "tensorflow/core/kernels/bounds_check.h" + +namespace tensorflow { + +namespace shape_op_helpers { +inline Status GetRegularOrVariantShape(OpKernelContext* ctx, int input_index, + TensorShape* shape) { + const Tensor& inp = ctx->input(input_index); + if (ctx->input_dtype(0) == DT_VARIANT) { + if (inp.dims() != 0) { + return errors::InvalidArgument( + "Shape of non-unary Variant not supported."); + } + TF_RETURN_IF_ERROR(GetUnaryVariantShape(inp, shape)); + } else { + *shape = inp.shape(); + } + return Status::OK(); +} +} // namespace shape_op_helpers + +template +class ShapeOp : public OpKernel { + public: + explicit ShapeOp(OpKernelConstruction* ctx) : OpKernel(ctx) {} + + void Compute(OpKernelContext* ctx) override { + TensorShape shape; + OP_REQUIRES_OK(ctx, + shape_op_helpers::GetRegularOrVariantShape(ctx, 0, &shape)); + const int rank = shape.dims(); + Tensor* out = nullptr; + OP_REQUIRES_OK(ctx, ctx->allocate_output(0, TensorShape({rank}), &out)); + auto vec = out->vec(); + for (int i = 0; i < rank; ++i) { + int64 dim_size = shape.dim_size(i); + if (out->dtype() == DT_INT32) { + OP_REQUIRES( + ctx, FastBoundsCheck(dim_size, std::numeric_limits::max()), + errors::InvalidArgument("Shape output type is 32-bit ", " but dim ", + i, " is ", dim_size)); + } + vec(i) = static_cast(dim_size); + } + } + + bool IsExpensive() override { return false; } +}; + +template +class ShapeNOp : public OpKernel { + public: + explicit ShapeNOp(OpKernelConstruction* ctx) : OpKernel(ctx) {} + + void Compute(OpKernelContext* ctx) override { + for (int i = 0; i < ctx->num_inputs(); ++i) { + TensorShape shape; + OP_REQUIRES_OK( + ctx, shape_op_helpers::GetRegularOrVariantShape(ctx, i, &shape)); + const int dims = shape.dims(); + Tensor* out = nullptr; + OP_REQUIRES_OK(ctx, ctx->allocate_output(i, {dims}, &out)); + auto vec = out->vec(); + + for (int j = 0; j < dims; ++j) { + int64 dim_size = shape.dim_size(j); + if (out->dtype() == DT_INT32) { + OP_REQUIRES( + ctx, FastBoundsCheck(dim_size, std::numeric_limits::max()), + errors::InvalidArgument("ShapeN output type is 32-bit but shape ", + i, " dim ", j, " is ", dim_size)); + } + vec(j) = static_cast(dim_size); + } + } + } + + bool IsExpensive() override { return false; } +}; + +class RankOp : public OpKernel { + public: + explicit RankOp(OpKernelConstruction* ctx) : OpKernel(ctx) {} + + void Compute(OpKernelContext* ctx) override { + TensorShape shape; + OP_REQUIRES_OK(ctx, + shape_op_helpers::GetRegularOrVariantShape(ctx, 0, &shape)); + const int rank = shape.dims(); + Tensor* out = nullptr; + OP_REQUIRES_OK(ctx, ctx->allocate_output(0, TensorShape({}), &out)); + out->scalar()() = rank; + } + + bool IsExpensive() override { return false; } +}; + +template +class SizeOp : public OpKernel { + public: + explicit SizeOp(OpKernelConstruction* ctx) : OpKernel(ctx) {} + + void Compute(OpKernelContext* ctx) override { + TensorShape shape; + OP_REQUIRES_OK(ctx, + shape_op_helpers::GetRegularOrVariantShape(ctx, 0, &shape)); + const int64 size = shape.num_elements(); + Tensor* out = nullptr; + OP_REQUIRES_OK(ctx, ctx->allocate_output(0, TensorShape({}), &out)); + if (out->dtype() == DT_INT32) { + OP_REQUIRES( + ctx, FastBoundsCheck(size, std::numeric_limits::max()), + errors::InvalidArgument("Number of elements was larger than " + "representable by 32-bit output type")); + } + out->scalar()() = static_cast(size); + } + + bool IsExpensive() override { return false; } +}; + +template +class ExpandDimsOp : public OpKernel { + public: + explicit ExpandDimsOp(OpKernelConstruction* ctx) : OpKernel(ctx) {} + + void Compute(OpKernelContext* ctx) override { + OP_REQUIRES(ctx, ctx->input(0).dtype() != DT_VARIANT, + errors::InvalidArgument("ExpandDims on Variant not supported")); + + OP_REQUIRES( + ctx, (ctx->input(1).NumElements() == 1), + errors::InvalidArgument("'dim' must be a tensor with a single value")); + Tdim dim = ctx->input(1).flat()(0); + OP_REQUIRES( + ctx, (dim >= -1 - ctx->input(0).dims() && dim <= ctx->input(0).dims()), + errors::InvalidArgument("Tried to expand dim index ", dim, + " for tensor with ", ctx->input(0).dims(), + " dimensions.")); + + auto existing_dims = ctx->input(0).shape().dim_sizes(); + // Safe - # elements in tensor dims bounded. + const int existing_dims_size = static_cast(existing_dims.size()); + std::vector new_shape(existing_dims_size); + for (size_t i = 0; i < new_shape.size(); ++i) { + new_shape[i] = existing_dims[i]; + } + + // We emulate numpy's interpretation of the dim axis when + // -input.dims() >= dim <= input.dims(). + if (dim < 0) { + dim += existing_dims.size() + 1; + } + + // Clamp to the end if needed. + dim = std::min(dim, existing_dims_size); + new_shape.emplace(new_shape.begin() + dim, 1); + const TensorShape output_shape(new_shape); + + Tensor* output = nullptr; + OP_REQUIRES_OK(ctx, ctx->allocate_output(0, {0}, &output)); + if (!output->CopyFrom(ctx->input(0), output_shape)) { + // This should never happen, since the sizes of the input and output + // should always be the same (we only expand the dimension with 1). + ctx->SetStatus( + errors::Internal("Could not expand dimension with input shape ", + ctx->input(0).shape().DebugString(), + " and output shape ", output_shape.DebugString())); + } + } + + bool IsExpensive() override { return false; } +}; + +class SqueezeOp : public OpKernel { + public: + explicit SqueezeOp(OpKernelConstruction* ctx) : OpKernel(ctx) { + std::vector squeeze_dims; + OP_REQUIRES_OK(ctx, ctx->GetAttr("squeeze_dims", &squeeze_dims)); + squeeze_dims_.insert(squeeze_dims.begin(), squeeze_dims.end()); + } + + void Compute(OpKernelContext* ctx) override { + OP_REQUIRES(ctx, ctx->input(0).dtype() != DT_VARIANT, + errors::InvalidArgument("Squeeze on Variant not supported")); + + auto existing_dims = ctx->input(0).shape().dim_sizes(); + const int existing_dims_size = static_cast(existing_dims.size()); + std::vector new_shape; + + std::unordered_set wrapped_squeeze_dims; + wrapped_squeeze_dims.reserve(squeeze_dims_.size()); + // Validate squeeze dims against the input. + for (int32 dim : squeeze_dims_) { + OP_REQUIRES( + ctx, (dim >= -ctx->input(0).dims() && dim < ctx->input(0).dims()), + errors::InvalidArgument("Tried to squeeze dim index ", dim, + " for tensor with ", ctx->input(0).dims(), + " dimensions.")); + // If dim is < 0, we wrap around (-1 means the last element). + if (dim < 0) { + dim = existing_dims_size + dim; + } + + wrapped_squeeze_dims.insert(dim); + } + + for (int i = 0; i < existing_dims_size; ++i) { + auto existing_dim = existing_dims[i]; + + // If squeeze_set is non-empty, only squeeze those dimensions. + if (!wrapped_squeeze_dims.empty()) { + if (wrapped_squeeze_dims.count(i) > 0) { + OP_REQUIRES(ctx, existing_dim == 1, + errors::InvalidArgument( + "Can not squeeze dim[", i, + "], expected a dimension of 1, got ", existing_dim)); + } else { + // This dimension is not being squeezed. + new_shape.push_back(existing_dim); + } + } else { + // Copy over all non-1-length dimensions. + if (existing_dim != 1) { + new_shape.push_back(existing_dim); + } + } + } + + const TensorShape output_shape(new_shape); + Tensor* output = nullptr; + OP_REQUIRES_OK(ctx, ctx->allocate_output(0, {0}, &output)); + if (!output->CopyFrom(ctx->input(0), output_shape)) { + // This should never happen, since the sizes of the input and + // output should always be the same. + ctx->SetStatus(errors::Internal("Could not squeeze input with shape ", + ctx->input(0).shape().DebugString(), + " and output shape ", + output_shape.DebugString())); + } + } + + bool IsExpensive() override { return false; } + + private: + std::unordered_set squeeze_dims_; +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_SHAPE_OPS_H_ diff --git a/linux/include/tensorflow/core/kernels/slice_op.h b/linux/include/tensorflow/core/kernels/slice_op.h new file mode 100644 index 000000000..1d662f636 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/slice_op.h @@ -0,0 +1,54 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_SLICE_OP_H_ +#define TENSORFLOW_CORE_KERNELS_SLICE_OP_H_ + +// Functor definition for SliceOp, must be compilable by nvcc. + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/tensor_types.h" + +namespace tensorflow { +namespace functor { + +template +struct Slice { + void operator()(const Device& d, typename TTypes::Tensor output, + typename TTypes::ConstTensor input, + const Eigen::DSizes& slice_indices, + const Eigen::DSizes& slice_sizes) { + bool use_64bit = (input.size() > Eigen::NumTraits::highest()); + if (!use_64bit && + Eigen::internal::is_same::value) { + Eigen::DSizes indices; + for (int i = 0; i < NDIMS; ++i) { + indices[i] = slice_indices[i]; + } + Eigen::DSizes sizes; + for (int i = 0; i < NDIMS; ++i) { + sizes[i] = slice_sizes[i]; + } + To32Bit(output).device(d) = To32Bit(input).slice(indices, sizes); + } else { + output.device(d) = input.slice(slice_indices, slice_sizes); + } + } +}; + +} // namespace functor +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_SLICE_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/slice_op_cpu_impl.h b/linux/include/tensorflow/core/kernels/slice_op_cpu_impl.h new file mode 100644 index 000000000..64b694819 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/slice_op_cpu_impl.h @@ -0,0 +1,50 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_SLICE_OP_CPU_IMPL_H_ +#define TENSORFLOW_CORE_KERNELS_SLICE_OP_CPU_IMPL_H_ + +#define EIGEN_USE_THREADS + +#include "tensorflow/core/framework/bfloat16.h" +#include "tensorflow/core/framework/register_types.h" +#include "tensorflow/core/kernels/slice_op.h" + +namespace tensorflow { + +using CpuDevice = Eigen::ThreadPoolDevice; + +#define DEFINE_CPU_KERNELS(T) \ + template struct functor::Slice; + +TF_CALL_ALL_TYPES(DEFINE_CPU_KERNELS); + +#undef DEFINE_CPU_KERNELS + +#ifdef TENSORFLOW_USE_SYCL +using SyclDevice = Eigen::SyclDevice; + +#define DEFINE_SYCL_KERNELS(T) \ + template struct functor::Slice; + +TF_CALL_GPU_NUMBER_TYPES(DEFINE_SYCL_KERNELS); +DEFINE_SYCL_KERNELS(int32); + +#undef DEFINE_SYCL_KERNELS +#endif // TENSORFLOW_USE_SYCL + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_SLICE_OP_CPU_IMPL_H_ diff --git a/linux/include/tensorflow/core/kernels/smooth-hinge-loss.h b/linux/include/tensorflow/core/kernels/smooth-hinge-loss.h new file mode 100644 index 000000000..d51f5c130 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/smooth-hinge-loss.h @@ -0,0 +1,114 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_SMOOTH_HINGE_LOSS_H_ +#define TENSORFLOW_CORE_KERNELS_SMOOTH_HINGE_LOSS_H_ + +#include + +#include "tensorflow/core/kernels/loss.h" +#include "tensorflow/core/lib/core/errors.h" +#include "tensorflow/core/lib/core/status.h" + +namespace tensorflow { + +class SmoothHingeLossUpdater : public DualLossUpdater { + public: + // Computes the updated dual variable (corresponding) to a single example. The + // updated dual value maximizes the objective function of the dual + // optimization problem associated with smooth hinge loss. The computations + // are detailed in readme.md. + double ComputeUpdatedDual(const int num_partitions, const double label, + const double example_weight, + const double current_dual, const double wx, + const double weighted_example_norm) const final { + // Intutitvely there are 3 cases: + // a. new optimal value of the dual variable falls within the admissible + // range [0, 1]. In this case we set new dual to this value. + // b. new optimal value is < 0. Then, because of convexity, the optimal + // valid value for new dual = 0 + // c. new optimal value > 1.0. Then new optimal value should be set to 1.0. + const double candidate_optimal_dual = + current_dual + + (label - wx - gamma * current_dual) / + (num_partitions * example_weight * weighted_example_norm + gamma); + if (label * candidate_optimal_dual < 0) { + return 0.0; + } + if (label * candidate_optimal_dual > 1.0) { + return label; + } + return candidate_optimal_dual; + } + + double ComputeDualLoss(const double current_dual, const double example_label, + const double example_weight) const final { + // For binary classification, there are 2 conjugate functions, one per + // label value (-1 and 1). + const double y_alpha = current_dual * example_label; // y \alpha + if (y_alpha < 0 || y_alpha > 1.0) { + return std::numeric_limits::max(); + } + return (-y_alpha + 0.5 * gamma * current_dual * current_dual) * + example_weight; + } + + double ComputePrimalLoss(const double wx, const double example_label, + const double example_weight) const final { + const double y_wx = example_label * wx; + if (y_wx >= 1) return 0; + if (y_wx <= 1 - gamma) return (1 - y_wx - gamma / 2) * example_weight; + return (1 - y_wx) * (1 - y_wx) * example_weight * 0.5 / gamma; + } + + // Converts binary example labels from 0.0 or 1.0 to -1.0 or 1.0 respectively + // as expected by smooth hinge loss. + Status ConvertLabel(float* const example_label) const final { + if (*example_label == 0.0) { + *example_label = -1; + return Status::OK(); + } + if (*example_label == 1.0) { + return Status::OK(); + } + return errors::InvalidArgument( + "Only labels of 0.0 or 1.0 are supported right now. " + "Found example with label: ", + *example_label); + } + + double PrimalLossDerivative(const double wx, const double label, + const double example_weight) const final { + if (label * wx >= 1) { + return 0; + } + if (label * wx <= 1 - gamma) { + return -label; + } + return (wx - label) / gamma; + } + + double SmoothnessConstant() const final { return gamma; } + + private: + // Smoothness constant of smooth hinge loss + // TODO(sibyl-Aix6ihai): expose this parameter + const double gamma = 1; +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_SMOOTH_HINGE_LOSS_H_ +// TENSORFLOW_KERNELS_SMOOTH_HINGE_LOSS_H_ diff --git a/linux/include/tensorflow/core/kernels/snapshot_op.h b/linux/include/tensorflow/core/kernels/snapshot_op.h new file mode 100644 index 000000000..02d492988 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/snapshot_op.h @@ -0,0 +1,44 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_SNAPSHOT_OP_H_ +#define TENSORFLOW_CORE_KERNELS_SNAPSHOT_OP_H_ + +#if GOOGLE_CUDA +#define EIGEN_USE_GPU +#endif + +#define EIGEN_USE_THREADS + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/op_kernel.h" + +namespace tensorflow { +namespace functor { + +// Functor used by SnapshotOp. +template +struct Snapshot { + void operator()(const Device& device, + typename TTypes::ConstTensor input, + typename TTypes::Tensor output) { + device.memcpy(output.data(), input.data(), input.size() * sizeof(Scalar)); + } +}; + +} // namespace functor +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_SNAPSHOT_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/softmax_op_functor.h b/linux/include/tensorflow/core/kernels/softmax_op_functor.h new file mode 100644 index 000000000..c8bc1ad3b --- /dev/null +++ b/linux/include/tensorflow/core/kernels/softmax_op_functor.h @@ -0,0 +1,101 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_SOFTMAX_OP_FUNCTOR_H_ +#define TENSORFLOW_CORE_KERNELS_SOFTMAX_OP_FUNCTOR_H_ +// Functor definition for SoftmaxOp, must be compilable by nvcc. + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/tensor_types.h" + +namespace tensorflow { +namespace functor { + +// Functor used by SoftmaxOp to do the computations. +template +struct SoftmaxFunctor { + // Computes Softmax or LogSoftmax activation. + // + // logits: dim: batch_size, num_classes. + // softmax: dims: batch_size, num_classes. + // log: boolean + void operator()(const Device& d, typename TTypes::ConstMatrix logits, + typename TTypes::Matrix softmax, const bool log); +}; + +// Eigen code implementing SoftmaxFunctor::operator() or +// LogSoftmaxFunctor::operator(). +// This code works for both CPU and GPU and is used by the functor +// specializations for both device types. +template +struct SoftmaxEigenImpl { + static void Compute(const Device& d, typename TTypes::ConstMatrix logits, + typename TTypes::Matrix softmax, const bool log) { + const int kBatchDim = 0; + const int kClassDim = 1; + + const int batch_size = logits.dimension(kBatchDim); + const int num_classes = logits.dimension(kClassDim); + +// These arrays are used to reduce along the class dimension, and broadcast +// the resulting value to all classes. +#if !defined(EIGEN_HAS_INDEX_LIST) + Eigen::DSizes along_class(kClassDim); + Eigen::DSizes batch_by_one(batch_size, 1); + Eigen::DSizes one_by_class(1, num_classes); +#else + Eigen::IndexList > along_class; + Eigen::IndexList > depth_dim; + Eigen::IndexList > batch_by_one; + batch_by_one.set(0, batch_size); + Eigen::IndexList, int> one_by_class; + one_by_class.set(1, num_classes); +#endif + // shifted_logits = logits - max(logits along classes); + auto shifted_logits = (logits - logits.maximum(along_class) + .eval() + .reshape(batch_by_one) + .broadcast(one_by_class)); + if (log) { + // Calculate the log of the softmax + // softmax = logits - max(logits along classes); + softmax.device(d) = shifted_logits; + // softmax = softmax - log(sum(exp(softmax along classes))); + softmax.device(d) = (softmax - softmax.exp() + .sum(along_class) + .log() + .eval() + .reshape(batch_by_one) + .broadcast(one_by_class)); + } else { + // NOTE(touts): If you modify this implementation please run + // the BM_ImageNetSoftmaxFwd benchmark in nn_ops_test.cc. + // + // softmax = exp(logits - max(logits along classes)); + softmax.device(d) = shifted_logits.exp(); + // softmax = softmax * (1 / sum(softmax along classes)); + softmax.device(d) = (softmax * softmax.sum(along_class) + .inverse() + .eval() + .reshape(batch_by_one) + .broadcast(one_by_class)); + } + } +}; + +} // namespace functor +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_SOFTMAX_OP_FUNCTOR_H_ diff --git a/linux/include/tensorflow/core/kernels/softplus_op.h b/linux/include/tensorflow/core/kernels/softplus_op.h new file mode 100644 index 000000000..8c083ba15 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/softplus_op.h @@ -0,0 +1,76 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_SOFTPLUS_OP_H_ +#define TENSORFLOW_CORE_KERNELS_SOFTPLUS_OP_H_ +// Functor definition for SoftplusOp and SoftplusGradOp, must be compilable by +// nvcc. + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/tensor_types.h" + +namespace tensorflow { +namespace functor { + +// Functor used by SoftplusOp to do the computations. +template +struct Softplus { + // Computes Softplus activation. + // + // features: any shape. + // activations: same shape as "features". + void operator()(const Device& d, typename TTypes::ConstTensor features, + typename TTypes::Tensor activations) { + // Choose a threshold on x below which exp(x) may underflow + // when added to 1, but for which exp(x) is always within epsilon of the + // true softplus(x). Offset of 2 from machine epsilon checked + // experimentally for float16, float32, float64. Checked against + // softplus implemented with numpy's log1p and numpy's logaddexp. + static const T threshold = + Eigen::numext::log(Eigen::NumTraits::epsilon()) + T(2); + // Value above which exp(x) may overflow, but softplus(x) == x + // is within machine epsilon. + auto too_large = features > features.constant(-threshold); + // Value below which exp(x) may underflow, but softplus(x) == exp(x) + // is within machine epsilon. + auto too_small = features < features.constant(threshold); + auto features_exp = features.exp(); + activations.device(d) = too_large.select( + features, // softplus(x) ~= x for x large + too_small.select(features_exp, // softplus(x) ~= exp(x) for x small + (features_exp + features.constant(T(1))).log())); + } +}; + +// Functor used by SoftplusGradOp to do the computations. +template +struct SoftplusGrad { + // Computes SoftplusGrad backprops. + // + // gradients: gradients backpropagated to the Softplus op. + // features: inputs that where passed to the Softplus op. + // backprops: gradients to backpropagate to the Softplus inputs. + void operator()(const Device& d, typename TTypes::ConstTensor gradients, + typename TTypes::ConstTensor features, + typename TTypes::Tensor backprops) { + backprops.device(d) = + gradients / ((-features).exp() + features.constant(T(1))); + } +}; + +} // namespace functor +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_SOFTPLUS_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/softsign_op.h b/linux/include/tensorflow/core/kernels/softsign_op.h new file mode 100644 index 000000000..61ff6eeed --- /dev/null +++ b/linux/include/tensorflow/core/kernels/softsign_op.h @@ -0,0 +1,60 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_SOFTSIGN_OP_H_ +#define TENSORFLOW_CORE_KERNELS_SOFTSIGN_OP_H_ +// Functor definition for SoftsignOp and SoftsignGradOp, must be compilable by +// nvcc. + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/tensor_types.h" + +namespace tensorflow { +namespace functor { + +// Functor used by SoftsignOp to do the computations. +template +struct Softsign { + // Computes Softsign activation. + // + // features: any shape. + // activations: same shape as "features". + void operator()(const Device& d, typename TTypes::ConstTensor features, + typename TTypes::Tensor activations) { + activations.device(d) = + features / (features.abs() + features.constant(T(1))); + } +}; + +// Functor used by SoftsignGradOp to do the computations. +template +struct SoftsignGrad { + // Computes SoftsignGrad backprops. + // + // gradients: gradients backpropagated to the Softsign op. + // features: inputs that were passed to the Softsign op. + // backprops: gradients to backpropagate to the Softsign inputs. + void operator()(const Device& d, typename TTypes::ConstTensor gradients, + typename TTypes::ConstTensor features, + typename TTypes::Tensor backprops) { + backprops.device(d) = + gradients / (features.abs() + features.constant(T(1))).square(); + } +}; + +} // namespace functor +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_SOFTSIGN_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/spacetobatch_functor.h b/linux/include/tensorflow/core/kernels/spacetobatch_functor.h new file mode 100644 index 000000000..f46a84da1 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/spacetobatch_functor.h @@ -0,0 +1,114 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_SPACETOBATCH_FUNCTOR_H_ +#define TENSORFLOW_CORE_KERNELS_SPACETOBATCH_FUNCTOR_H_ + +#include + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/kernels/bounds_check.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +// Maximum number of non-collapsible blocked dimensions supported by the +// {SpaceToBatch,BatchToSpace}ND operation. To change the limit, modify this +// constant and the TF_SPACETOBATCH_FOR_EACH_NUM_BLOCK_DIMS macro definition +// below. +constexpr int kMaxSpaceToBatchBlockDims = 4; + +// Expands to: +// MACRO(1, ## __VA_ARGS__) +// ... +// MACRO(kMaxSpaceToBatchBlockDims, ## __VA_ARGS__) +// +// Note: The space between the number and the comma is necessary for proper GCC +// comma handling: https://gcc.gnu.org/onlinedocs/cpp/Variadic-Macros.html +#define TF_SPACETOBATCH_FOR_EACH_NUM_BLOCK_DIMS(MACRO, ...) \ + MACRO(1 /**/, ##__VA_ARGS__) \ + MACRO(2 /**/, ##__VA_ARGS__) \ + MACRO(3 /**/, ##__VA_ARGS__) \ + MACRO(4 /**/, ##__VA_ARGS__) \ + /**/ + +namespace internal { +namespace spacetobatch { + +template +void SubtleMustCopyFlatHelper(const Tensor& t, OutputType* output) { + const int64 num_elements = t.shape().num_elements(); + output->resize(num_elements); + auto eigen_vec = t.flat(); + for (int64 i = 0; i < num_elements; ++i) { + (*output)[i] = SubtleMustCopy(eigen_vec(i)); + } +} + +// Copies flat contents of `t` to std::vector-like `*output`, which is resized +// as needed. `OutputType` may be either `std::vector` or +// `gtl::InlinedVector`. +// +// Precondition: t.dtype() must be either DT_INT32 or DT_INT64. +template +void SubtleMustCopyFlat(const Tensor& t, OutputType* output) { + if (t.dtype() == DT_INT32) { + SubtleMustCopyFlatHelper(t, output); + } else { + SubtleMustCopyFlatHelper(t, output); + } +} + +} // namespace spacetobatch +} // namespace internal + +namespace functor { + +// Functor used by {SpaceToBatch,BatchToSpace}{ND,}Op to do the conversion. +// +// If B2S is false, then this performs the space-to-batch conversion. If S2B is +// true, then this performs the inverse batch-to-space conversion. +template +struct SpaceToBatchFunctor { + using InputT = typename std::conditional::type; + using OutputT = typename std::conditional::type; + // Implements the space to batch conversion. + // + // space_tensor: input tensor of space-to-batch operation. If B2S = false, + // then this is the input to the conversion. If B2S = true, then this + // is the output of the conversion. + // block_size: array of shape [NUM_BLOCK_DIMS] specifying the block sizes for + // dimensions 1 through NUM_BLOCK_DIMS. + // paddings: row-major array of shape [NUM_BLOCK_DIMS, 2] specifying the + // start and end padding for dimensions 1 through NUM_BLOCK_DIMS. + // batch_tensor: output tensor of the space-to-batch operation. If + // B2S = false, then this is the output of the conversion. If B2S = true, + // then this is the input to the conversion. + // + // The caller must ensure that the dimensions of the tensors are correct. + Status operator()( + const Device& d, + typename TTypes::Tensor space_tensor, + const int64 block_shape[NUM_BLOCK_DIMS], + const int64 paddings[NUM_BLOCK_DIMS * 2], + typename TTypes::Tensor batch_tensor); +}; + +} // namespace functor +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_SPACETOBATCH_FUNCTOR_H_ diff --git a/linux/include/tensorflow/core/kernels/spacetodepth_op.h b/linux/include/tensorflow/core/kernels/spacetodepth_op.h new file mode 100644 index 000000000..9d28be5fe --- /dev/null +++ b/linux/include/tensorflow/core/kernels/spacetodepth_op.h @@ -0,0 +1,57 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_SPACETODEPTH_OP_H_ +#define TENSORFLOW_CORE_KERNELS_SPACETODEPTH_OP_H_ +// Functor definition for XentOp, must be compilable by nvcc. + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/util/tensor_format.h" + +namespace tensorflow { +namespace functor { + +// Functor used by SpaceToDepthOp to do the computations. +// Implements a family of Space to Depth transforms for a 4D 'input' tensor +// to a 4D 'output' tensor, both tensors use type 'T' and layout 'data_format'. +// These transforms divide the vertical and horizontal image sizes by +// 'block_size', and multiply the depth dimension size by +// (block_size * block_size). The offset within each block_size * block_size +// patch within the image is combined with the input channel index to form +// the output channel index, with the Y, X coordinates within each block of +// the input image used as the high order component of the output channel. +// e.g. for data_format = NHWC: +// Each element in the input tensor can be specified via 6 coordinates, +// ordered by decreasing memory layout significance as: +// n,oY,bY,oX,bX,iC (where n=batch index, oX, oY means X or Y coordinates +// within the output image, bX, bY means coordinates +// within the input block, iC means input channels). +// The output would be a transpose to the following layout: +// n,oY,oX,bY,bX,iC +template +struct SpaceToDepthOpFunctor { + void operator()(const Device& d, typename TTypes::ConstTensor input, + int block_size, typename TTypes::Tensor output); + + // This 5-D version is to support NCHW_VECT_C. + void operator()(const Device& d, typename TTypes::ConstTensor input, + int block_size, typename TTypes::Tensor output); +}; + +} // namespace functor +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_SPACETODEPTH_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/sparse_conditional_accumulator.h b/linux/include/tensorflow/core/kernels/sparse_conditional_accumulator.h new file mode 100644 index 000000000..a4453bd7a --- /dev/null +++ b/linux/include/tensorflow/core/kernels/sparse_conditional_accumulator.h @@ -0,0 +1,462 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_SPARSE_CONDITIONAL_ACCUMULATOR_H_ +#define TENSORFLOW_CORE_KERNELS_SPARSE_CONDITIONAL_ACCUMULATOR_H_ + +#include "tensorflow/core/kernels/typed_conditional_accumulator_base.h" + +namespace tensorflow { + +/** + * An aggregation object for adding sparse gradients, represented as a tuple of + * indices, values, and a (possibly empty) shape. + * + * The two main methods of this class are TryApplyGrad and TryTakeGrad. + * + * TryApplyGrad tries add a gradient to the accumulator. The attempt is + * successful if local_step >= global_step, i.e., if the gradient is not stale, + * having been computed using up-to-date information. Otherwise, the gradient is + * silently dropped. + * + * TryTakeGrad logs an attempt to read the average gradient. The attempt is + * blocked until the number of gradients accumulated (via TryApplyGrad) is equal + * or exceeds the number requested by TryTakeGrad. + * Once this condition is satisfied, the following actions are taken: + * (1) the value of the average gradient is returned + * (2) the count of accumulated gradients is reset to 0 + * (3) the internal global_step value (current_global_step_) is incremented by 1 + * + * SparseConditionalAccumulator is the datatype-dependent templated sub-class of + * ConditionalAccumulatorBase. It implements the virtual arithmetic methods that + * are used by for aggregating, averaging, allocating, returning indexed slices. + */ +template +class SparseConditionalAccumulator + : public TypedConditionalAccumulatorBase< + std::tuple> { + public: + SparseConditionalAccumulator(const DataType& dtype, + const PartialTensorShape& shape, + const string& name, const string& reduction_type) + : TypedConditionalAccumulatorBase< + std::tuple>( + dtype, shape, name, reduction_type) { + accum_idx_vec_ = nullptr; + count_element_ = nullptr; + accum_val_ = nullptr; + accum_val_persistent_ = new PersistentTensor(); + } + + ~SparseConditionalAccumulator() override { + if (accum_idx_vec_ != nullptr) delete accum_idx_vec_; + if (count_element_ != nullptr) delete count_element_; + if (accum_val_persistent_ != nullptr) delete accum_val_persistent_; + // Do not delete accum_val_! Will be automatically garbage collected + }; + + protected: + std::vector* accum_idx_vec_ = nullptr; + std::vector* count_element_ = nullptr; + + Tensor* accum_val_ = nullptr; + PersistentTensor* accum_val_persistent_ = nullptr; + + typedef Eigen::TensorMap, + Eigen::Unaligned> + SliceT; + typedef Eigen::TensorMap, + Eigen::Unaligned> + SliceConstT; + + Status ValidateShape( + std::tuple* tensor, + bool has_known_shape) EXCLUSIVE_LOCKS_REQUIRED(this->mu_) { + const Tensor* tensor_idx = std::get<0>(*tensor); + const Tensor* tensor_val = std::get<1>(*tensor); + const Tensor* tensor_shape = std::get<2>(*tensor); + int64 grad_val_dims = tensor_val->dims(); + int64 grad_dims = grad_val_dims; + + // Compare with provided shape + if (has_known_shape) { + if (shape_.dims() > tensor_shape->NumElements()) { + return errors::InvalidArgument( + "Shape mismatch: expected shape rank at least ", shape_.dims(), + ", got ", tensor_shape->NumElements()); + } + const auto tensor_shape_flat = tensor_shape->flat(); + for (int64 i = 0; i < shape_.dims(); i++) { + if (shape_.dim_size(i) != -1 && + shape_.dim_size(i) != tensor_shape_flat(i)) { + return errors::InvalidArgument("Shape mismatch: expected shape dim ", + i, " to be ", shape_.dim_size(i), + ", got ", tensor_shape_flat(i)); + } + } + } + // Check that indices are within limits + if (shape_.dims() > 0 && shape_.dim_size(0) != -1 && + tensor_idx->dims() > 0) { + for (int64 i = 0; i < tensor_idx->dim_size(0); i++) { + if (tensor_idx->vec()(i) >= shape_.dim_size(0)) { + return errors::InvalidArgument( + "Shape mismatch: index of slice ", i, " exceeded limits of shape", + "; index is ", tensor_idx->vec()(i), " exceeded ", + shape_.dim_size(0)); + } + } + } + + // Check values compatibility with accumulated gradient if available + if (counter_ > 0) { + int64 accum_val_dims = accum_val_->dims(); + if (accum_val_dims != grad_val_dims) { + return errors::InvalidArgument("Shape mismatch: expected values rank ", + accum_val_dims, ", got ", grad_val_dims); + } + for (int64 i = 1; i < accum_val_dims; i++) { + if (accum_val_->dim_size(i) != tensor_val->dim_size(i)) { + return errors::InvalidArgument("Shape mismatch: expected values dim ", + i, " to be ", accum_val_->dim_size(i), + ", got ", tensor_val->dim_size(i)); + } + } + } else { + // If there are no accumulated gradients, check against shape_ + if (shape_.dims() > grad_dims) { + return errors::InvalidArgument( + "Shape mismatch: expected values rank at least ", shape_.dims(), + ", got ", grad_dims); + } + // Check that values have correct dimensions + for (int64 i = 1; i < shape_.dims(); i++) { + if (shape_.dim_size(i) != -1 && + shape_.dim_size(i) != tensor_val->dim_size(i)) { + return errors::InvalidArgument("Shape mismatch: expected values dim ", + i, " to be ", shape_.dim_size(i), + ", got ", tensor_val->dim_size(i)); + } + } + } + + return Status::OK(); + } + + void AllocateAndAssignToAccumGradFunction( + OpKernelContext* ctx, + std::tuple* grad) override { + const Tensor* grad_idx = std::get<0>(*grad); + const Tensor* grad_val = std::get<1>(*grad); + + const int64 nnz = grad_idx->dim_size(0); + + // Assign indices + if (accum_idx_vec_ != nullptr) delete accum_idx_vec_; + accum_idx_vec_ = new std::vector(); + accum_idx_vec_->reserve(nnz); + for (int i = 0; i < nnz; i++) { + accum_idx_vec_->push_back(grad_idx->vec()(i)); + } + + // Assign values to accum_val_tensor + // TODO(b/32704451): Don't just ignore the ::tensorflow::Status object! + ctx->allocate_persistent(dtype_, grad_val->shape(), accum_val_persistent_, + &accum_val_) + .IgnoreError(); + accum_val_->flat().device(ctx->template eigen_device()) = + grad_val->flat(); + + // Assign count_element_ + if (count_element_ != nullptr) { + delete count_element_; + } + count_element_ = new std::vector(nnz, 1); + + // Do not need shape; Assume that the op has checked that the shapes match, + // so grad's shape == shape_ + } + + void AddToAccumGradFunction( + OpKernelContext* ctx, + std::tuple* grad) override { + // Modeled after third_party/tensorflow/core/kernels/sparse_add_op + + const Tensor* grad_idx = std::get<0>(*grad); + const Tensor* grad_val = std::get<1>(*grad); + + const int64 accum_nnz = accum_idx_vec_->size(); + const int64 grad_nnz = grad_idx->dim_size(0); + + // Source enumerates the origin of a non-zero element: whether it is from + // the new gradient, the accumulated gradient, or the sum of both. + enum Source { from_accum, from_grad, from_accum_and_grad }; + + // (1) do a pass over inputs, and append values and indices to vectors + std::vector> entries_to_copy; + entries_to_copy.reserve(accum_nnz + grad_nnz); + + // Pass over all non-zero elements of both the gradient and the accumulated + // value, to identify where each non-zero element of the sum comes from. + // The input and output indexed slices are assumed to be ordered along + // increasing dimension number. + int64 i = 0, j = 0; + int64 sum_nnz = 0; + while (i < accum_nnz && j < grad_nnz) { + sum_nnz++; + switch (cmp(accum_idx_vec_, grad_idx, i, j)) { + case -1: + entries_to_copy.emplace_back(from_accum, i, -1); + ++i; + break; + case 0: + entries_to_copy.emplace_back(from_accum_and_grad, i, j); + ++i; + ++j; + break; + case 1: + entries_to_copy.emplace_back(from_grad, -1, j); + ++j; + break; + } + } + + // Handle leftovers + while (i < accum_nnz) { + sum_nnz++; + entries_to_copy.emplace_back(from_accum, i, -1); + ++i; + } + while (j < grad_nnz) { + sum_nnz++; + entries_to_copy.emplace_back(from_grad, -1, j); + ++j; + } + + // (2) Copy or sum the non-zero elements into sum_indices and sum_tensor + std::vector* sum_indices_vec = new std::vector(); + sum_indices_vec->reserve(sum_nnz); + + std::vector* sum_counts = new std::vector(); + sum_counts->reserve(sum_nnz); + + Tensor* sum_tensor = nullptr; + PersistentTensor* tensor_sum_persistent = new PersistentTensor(); + + TensorShape sum_shape = grad_val->shape(); + sum_shape.set_dim(0, sum_nnz); + + OP_REQUIRES_OK( + ctx, ctx->allocate_persistent(dtype_, sum_shape, tensor_sum_persistent, + &sum_tensor)); + auto sum_flat = sum_tensor->flat_outer_dims(); + auto accum_flat = accum_val_->flat_outer_dims(); + auto grad_flat = grad_val->flat_outer_dims(); + + const int64 num_col = grad_flat.dimension(1); + + Eigen::DSizes slice_shape(num_col); + + for (i = 0; i < sum_nnz; ++i) { + const Source src = std::get<0>(entries_to_copy[i]); + const int64 idx_a = std::get<1>(entries_to_copy[i]); + const int64 idx_b = std::get<2>(entries_to_copy[i]); + T* sum_slice_ptr = &sum_flat(i, 0); + SliceT sum_slice(sum_slice_ptr, slice_shape); + if (src == from_accum) { + // Element comes from accumulator; directly copy data structures over + sum_indices_vec->push_back(accum_idx_vec_->at(idx_a)); + T* accum_slice_ptr = &accum_flat(idx_a, 0); + SliceT accum_slice(accum_slice_ptr, slice_shape); + sum_slice = accum_slice; + sum_counts->push_back(count_element_->at(idx_a)); + } else if (src == from_accum_and_grad) { + // Element is a sum of accumulated value and new gradient; + // compute sum here + sum_indices_vec->push_back(accum_idx_vec_->at(idx_a)); + const T* grad_slice_ptr = &grad_flat(idx_b, 0); + SliceConstT grad_slice(grad_slice_ptr, slice_shape); + T* accum_slice_ptr = &accum_flat(idx_a, 0); + SliceT accum_slice(accum_slice_ptr, slice_shape); + sum_slice = grad_slice + accum_slice; + sum_counts->push_back(count_element_->at(idx_a) + 1); + } else if (src == from_grad) { + // Element comes from new gradient; make a copy of indices and values + sum_indices_vec->push_back(grad_idx->vec()(idx_b)); + const T* grad_slice_ptr = &grad_flat(idx_b, 0); + SliceConstT grad_slice(grad_slice_ptr, slice_shape); + sum_slice = grad_slice; + sum_counts->push_back(1); + } + } + + // (3) Keep output, i.e., switch pointers to point to new data structures + // representing the sum + // Indices + if (accum_idx_vec_ != nullptr) delete accum_idx_vec_; + accum_idx_vec_ = sum_indices_vec; + // Values + accum_val_ = sum_tensor; + delete accum_val_persistent_; + accum_val_persistent_ = tensor_sum_persistent; + // Counts + if (count_element_ != nullptr) delete count_element_; + count_element_ = sum_counts; + + // No need to copy shape, since shape remains the same after sum. + } + + void DivideAccumGradByCounter(OpKernelContext* ctx) override + EXCLUSIVE_LOCKS_REQUIRED(this->mu_) { + const int64 nnz = count_element_->size(); + auto accum_flat = accum_val_->flat_outer_dims(); + std::vector count_typet; + std::transform(count_element_->begin(), count_element_->end(), + std::back_inserter(count_typet), + TypeConverter::ConvertUToT); + + // Option 1: divide all by counter + /* + std::transform( + &accum_flat(0,0), &accum_flat(nnz,0), &accum_flat(0,0), + std::bind2nd(std::divides(), + TypeConverter::ConvertUToT(this->counter_))); + */ + + // Option 2: average element-wise + Eigen::DSizes slice_shape(accum_flat.dimension(1)); + for (int64 i = 0; i < nnz; i++) { + T* accum_slice_ptr = &accum_flat(i, 0); + SliceT accum_slice(accum_slice_ptr, slice_shape); + accum_slice.device(ctx->template eigen_device()) = + accum_slice / count_typet[i]; + } + } + + bool SetOutput(OpKernelContext* ctx) override { + bool is_successful = true; + if (is_successful) is_successful = ReturnIdxTensor(ctx); + if (is_successful) is_successful = ReturnValTensor(ctx); + if (is_successful) is_successful = ReturnShapeTensor(ctx); + return is_successful; + } + + bool GetAndValidateTensorInputForApplyGrad( + OpKernelContext* ctx, + std::tuple** tensor) override + EXCLUSIVE_LOCKS_REQUIRED(this->mu_) { + // TODO(xinghao, jmchen): The roundabout way of getting attr from + // OpKernelContext (instead of OpKernelConstruction) is a hack, and should + // be fixed if it affects efficiency. + bool has_known_shape = false; + OP_REQUIRES_OK_BOOLEAN( + ctx, GetNodeAttr(ctx->op_kernel().def(), "has_known_shape", + &has_known_shape)); + + // Get input gradient tensors + const Tensor* grad_idx_tensor; + OP_REQUIRES_OK_BOOLEAN(ctx, + ctx->input("gradient_indices", &grad_idx_tensor)); + const Tensor* grad_val_tensor; + OP_REQUIRES_OK_BOOLEAN(ctx, + ctx->input("gradient_values", &grad_val_tensor)); + const Tensor* grad_shape_tensor = nullptr; + if (has_known_shape) { + OP_REQUIRES_OK_BOOLEAN(ctx, + ctx->input("gradient_shape", &grad_shape_tensor)); + } + + // Checks + OP_REQUIRES_BOOLEAN( + ctx, TensorShapeUtils::IsVector(grad_idx_tensor->shape()), + errors::InvalidArgument( + "Input indices should be vector but received shape: ", + grad_idx_tensor->shape().DebugString())); + const int64 nnz = grad_idx_tensor->dim_size(0); + OP_REQUIRES_BOOLEAN( + ctx, grad_val_tensor->dims() > 0, + errors::InvalidArgument("Values cannot be 0-dimensional.")); + OP_REQUIRES_BOOLEAN(ctx, grad_val_tensor->dim_size(0) == nnz, + errors::InvalidArgument("Expected ", nnz, + " non-empty input values, got ", + grad_val_tensor->dim_size(0))); + + *tensor = new std::tuple( + grad_idx_tensor, grad_val_tensor, grad_shape_tensor); + + OP_REQUIRES_OK_BOOLEAN(ctx, this->ValidateShape(*tensor, has_known_shape)); + + return true; + } + + void CleanUpGradTensor(std::tuple* tensor) override { + if (tensor != nullptr) delete tensor; + } + + private: + inline int cmp(std::vector* a_idx, const Tensor* b_idx, + const int64 a_row, const int64 b_row) { + const int64 a = a_idx->at(a_row); + const int64 b = b_idx->vec()(b_row); + if (a < b) { + return -1; + } else if (a > b) { + return 1; + } + return 0; + } + + inline bool ReturnIdxTensor(OpKernelContext* ctx) { + Tensor* idx_tensor; + const int64 nnz = accum_idx_vec_->size(); + OP_REQUIRES_OK_BOOLEAN(ctx, ctx->allocate_output(0, {nnz}, &idx_tensor)); + // If allocate_output fails, OP_REQUIRES_OK_BOOLEAN will short-circuit + // the remaining code and just return false + auto idx_tensor_vec = idx_tensor->vec(); + for (int i = 0; i < nnz; ++i) { + idx_tensor_vec(i) = accum_idx_vec_->at(i); + } + return true; + } + + inline bool ReturnValTensor(OpKernelContext* ctx) { + ctx->set_output(1, *accum_val_); + return true; + } + + inline bool ReturnShapeTensor(OpKernelContext* ctx) { + int64 accum_val_dims = accum_val_->dims(); + Tensor* shape_tensor; + OP_REQUIRES_OK_BOOLEAN( + ctx, ctx->allocate_output(2, {accum_val_dims}, &shape_tensor)); + // If allocate_output fails, OP_REQUIRES_OK_BOOLEAN will short-circuit + // the remaining code and just return false + + // First dim of shape is defined by shape_, others by accum_val_->shape + shape_tensor->flat()(0) = + (shape_.dims() > 0) ? shape_.dim_size(0) : -1; + for (int64 i = 1; i < accum_val_dims; i++) { + shape_tensor->flat()(i) = accum_val_->dim_size(i); + } + return true; + } + + TF_DISALLOW_COPY_AND_ASSIGN(SparseConditionalAccumulator); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_SPARSE_CONDITIONAL_ACCUMULATOR_H_ diff --git a/linux/include/tensorflow/core/kernels/sparse_matmul_op.h b/linux/include/tensorflow/core/kernels/sparse_matmul_op.h new file mode 100644 index 000000000..6b9db8f47 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/sparse_matmul_op.h @@ -0,0 +1,468 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_SPARSE_MATMUL_OP_H_ +#define TENSORFLOW_CORE_KERNELS_SPARSE_MATMUL_OP_H_ + +#include "third_party/eigen3/Eigen/Core" +#include "tensorflow/core/platform/byte_order.h" +#include "tensorflow/core/platform/types.h" + +#if defined(PLATFORM_WINDOWS) +#include "tensorflow/core/platform/windows/cpu_info.h" +#include "tensorflow/core/platform/windows/intrinsics_port.h" +#endif + +namespace Eigen { +namespace internal { + +// Return the float representation of the bfloat16 value +// in the lower 16-bits of input +template +EIGEN_DEVICE_FUNC inline Packet pexpand_bf16_l(const Packet& from) { + tensorflow::uint32 tmp; +#if __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__ + tmp = (reinterpret_cast(from)) & 0xffff0000; +#else + tmp = (reinterpret_cast(from) << 16) & 0xffff0000; +#endif + return reinterpret_cast(tmp); +} + +// Return the float representation of the bfloat16 value +// in the upper 16-bits of input +template +EIGEN_DEVICE_FUNC inline Packet pexpand_bf16_u(const Packet& from) { + tensorflow::uint32 tmp; +#if __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__ + tmp = (reinterpret_cast(from) << 16) & 0xffff0000; +#else + tmp = (reinterpret_cast(from)) & 0xffff0000; +#endif + return reinterpret_cast(tmp); +} + +// Specialization non-scalar version on non-sse. +// Enable vectorization on z13 and higher +#if defined(EIGEN_VECTORIZE_ALTIVEC) || defined(EIGEN_VECTORIZE_VSX) || \ + defined(EIGEN_VECTORIZE_NEON) || defined(EIGEN_VECTORIZE_ZVECTOR) +template +EIGEN_DEVICE_FUNC inline Packet4f pexpand_bf16_l(const Packet4f& from) { + float r[4]; + tensorflow::uint32 p[4]; + pstoreu(r, from); + tensorflow::uint32* ir = reinterpret_cast(r); + p[0] = (ir[0] << 16) & 0xffff0000; + p[1] = ir[0] & 0xffff0000; + p[2] = (ir[1] << 16) & 0xffff0000; + p[3] = ir[1] & 0xffff0000; + return ploadu(reinterpret_cast(p)); +} + +template +EIGEN_DEVICE_FUNC inline Packet4f pexpand_bf16_u(const Packet4f& from) { + float r[4]; + tensorflow::uint32 p[4]; + pstoreu(r, from); + tensorflow::uint32* ir = reinterpret_cast(r); + p[0] = (ir[2] << 16) & 0xffff0000; + p[1] = ir[2] & 0xffff0000; + p[2] = (ir[3] << 16) & 0xffff0000; + p[3] = ir[3] & 0xffff0000; + return ploadu(reinterpret_cast(p)); +} +#endif + +template +EIGEN_DEVICE_FUNC inline Packet pinterleave4x64(const Packet& from) { + return from; +} + +template +EIGEN_DEVICE_FUNC inline Packet pbroadcast_first(const Packet& a) { + return a; +} + +template +EIGEN_DEVICE_FUNC inline Packet pbroadcast_second(const Packet& a) { + assert(false && "Not applicable to Scalar Values"); + return a; +} + +template +EIGEN_DEVICE_FUNC inline Packet pbroadcast_third(const Packet& a) { + assert(false && "Not applicable to Scalar Values"); + return a; +} + +template +EIGEN_DEVICE_FUNC inline Packet pbroadcast_fourth(const Packet& a) { + assert(false && "Not applicable to Scalar Values"); + return a; +} + +template +EIGEN_DEVICE_FUNC inline Packet pload4bf16( + const typename unpacket_traits::type* from) { + assert(false && "Not applicable to Scalar Values"); + return Packet(); +} + +template +EIGEN_DEVICE_FUNC inline Packet pload2bf16( + const typename unpacket_traits::type* from) { + assert(false && "Not applicable to Scalar Values"); + return Packet(); +} + +// Specialization for pload4bf16 and pload2bf16 for non-sse. +// Enable vectorization on z13 and higher. +#if defined(EIGEN_VECTORIZE_ALTIVEC) || defined(EIGEN_VECTORIZE_VSX) || \ + defined(EIGEN_VECTORIZE_NEON) || defined(EIGEN_VECTORIZE_ZVECTOR) +template <> +EIGEN_STRONG_INLINE Packet4f pload4bf16(const float* from) { + tensorflow::uint32 p[4]; + const tensorflow::uint32* ir = + reinterpret_cast(from); + p[0] = (ir[0] << 16) & 0xffff0000; + p[1] = ir[0] & 0xffff0000; + p[2] = (ir[1] << 16) & 0xffff0000; + p[3] = ir[1] & 0xffff0000; + return ploadu(reinterpret_cast(p)); +} + +template <> +EIGEN_STRONG_INLINE Packet4f pload2bf16(const float* from) { + tensorflow::uint32 p[4]; + const tensorflow::uint32* ir = + reinterpret_cast(from); + p[0] = (ir[0] << 16) & 0xffff0000; + p[1] = ir[0] & 0xffff0000; + p[2] = (ir[0] << 16) & 0xffff0000; + p[3] = ir[0] & 0xffff0000; + return ploadu(reinterpret_cast(p)); +} +#endif + +#if defined(EIGEN_VECTORIZE_ALTIVEC) || defined(EIGEN_VECTORIZE_VSX) +// Return a packet with the first value of the input Packet replicated +template <> +EIGEN_STRONG_INLINE Packet4f pbroadcast_first(const Packet4f& a) { + return vec_splat(a, 0); +} + +// Return a packet with the second value of the input Packet replicated +template <> +EIGEN_STRONG_INLINE Packet4f pbroadcast_second(const Packet4f& a) { + return vec_splat(a, 1); +} + +// Return a packet with the third value of the input Packet replicated +template <> +EIGEN_STRONG_INLINE Packet4f pbroadcast_third(const Packet4f& a) { + return vec_splat(a, 2); +} + +// Return a packet with the fourth value of the input Packet replicated +template <> +EIGEN_STRONG_INLINE Packet4f pbroadcast_fourth(const Packet4f& a) { + return vec_splat(a, 3); +} +#endif + +#ifdef EIGEN_VECTORIZE_SSE2 +// For PacketSize of 4 floats the Packet is not modified +template <> +EIGEN_STRONG_INLINE Packet4f pinterleave4x64(const Packet4f& from) { + return from; +} + +// Return a Packet with 4 floats loaded from 4 bfloat16 values +template <> +EIGEN_STRONG_INLINE Packet4f pload4bf16(const float* from) { + __m128i zero = _mm_setzero_si128(); + __m128i tmp = _mm_castpd_si128(_mm_load_pd1((const double*)from)); + return _mm_castsi128_ps(_mm_unpacklo_epi16(zero, tmp)); +} + +// Return a Packet with 2 floats loaded from 2 bfloat16 values +template <> +EIGEN_STRONG_INLINE Packet4f pload2bf16(const float* from) { + __m128i zero = _mm_setzero_si128(); + __m128i tmp = _mm_castps_si128(_mm_load_ps1(from)); + return _mm_castsi128_ps(_mm_unpacklo_epi16(zero, tmp)); +} + +// Return a Packet with 4 floats expanded from 4 bfloat16 values +// in the lower half of the 128-bit lane +template +EIGEN_DEVICE_FUNC inline Packet4f pexpand_bf16_l(const Packet4f& from) { + __m128i zero = _mm_setzero_si128(); + __m128i tmp = _mm_castps_si128(from); + return _mm_castsi128_ps(_mm_unpacklo_epi16(zero, tmp)); +} + +// Return a Packet with 4 floats expanded from 4 bfloat16 values +// in the upper half of the 128-bit lane +template +EIGEN_DEVICE_FUNC inline Packet4f pexpand_bf16_u(const Packet4f& from) { + __m128i zero = _mm_setzero_si128(); + __m128i tmp = _mm_castps_si128(from); + return _mm_castsi128_ps(_mm_unpackhi_epi16(zero, tmp)); +} + +// Return a packet with the first value of the input Packet replicated +template <> +EIGEN_STRONG_INLINE Packet4f pbroadcast_first(const Packet4f& a) { + return _mm_set1_ps(pfirst(a)); +} + +// Return a packet with the second value of the input Packet replicated +template <> +EIGEN_STRONG_INLINE Packet4f pbroadcast_second(const Packet4f& a) { + return _mm_set1_ps(_mm_cvtss_f32(_mm_shuffle_ps(a, a, 1))); +} + +// Return a packet with the third value of the input Packet replicated +template <> +EIGEN_STRONG_INLINE Packet4f pbroadcast_third(const Packet4f& a) { + return _mm_set1_ps(_mm_cvtss_f32(_mm_shuffle_ps(a, a, 2))); +} + +// Return a packet with the fourth value of the input Packet replicated +template <> +EIGEN_STRONG_INLINE Packet4f pbroadcast_fourth(const Packet4f& a) { + return _mm_set1_ps(_mm_cvtss_f32(_mm_shuffle_ps(a, a, 3))); +} + +#endif + +#ifdef EIGEN_VECTORIZE_AVX512 +template <> +EIGEN_STRONG_INLINE Packet16f +pbroadcast_first(const Packet16f& a_in) { + Packet4f a = _mm512_castps512_ps128(a_in); + return _mm512_broadcastss_ps(a); +} +template <> +EIGEN_STRONG_INLINE Packet16f +pbroadcast_second(const Packet16f& a_in) { + Packet4f a = _mm512_castps512_ps128(a_in); + return _mm512_broadcastss_ps(_mm_shuffle_ps(a, a, _MM_SHUFFLE(1, 1, 1, 1))); +} +template <> +EIGEN_STRONG_INLINE Packet16f +pbroadcast_third(const Packet16f& a_in) { + Packet4f a = _mm512_castps512_ps128(a_in); + return _mm512_broadcastss_ps(_mm_shuffle_ps(a, a, _MM_SHUFFLE(2, 2, 2, 2))); +} +template <> +EIGEN_STRONG_INLINE Packet16f +pbroadcast_fourth(const Packet16f& a_in) { + Packet4f a = _mm512_castps512_ps128(a_in); + return _mm512_broadcastss_ps(_mm_shuffle_ps(a, a, _MM_SHUFFLE(3, 3, 3, 3))); +} +template <> +EIGEN_STRONG_INLINE Packet8d pbroadcast_first(const Packet8d& a_in) { + Packet2d a = _mm512_castpd512_pd128(a_in); + return _mm512_broadcastsd_pd(a); +} +template <> +EIGEN_STRONG_INLINE Packet8d pbroadcast_second(const Packet8d& a_in) { + Packet2d a = _mm_permute_pd(_mm512_castpd512_pd128(a_in), 3); + return _mm512_broadcastsd_pd(a); +} +template <> +EIGEN_STRONG_INLINE Packet8d pbroadcast_third(const Packet8d& a_in) { + Packet2d a = _mm256_extractf128_pd(_mm512_castpd512_pd256(a_in), 1); + return _mm512_broadcastsd_pd(a); +} +template <> +EIGEN_STRONG_INLINE Packet8d pbroadcast_fourth(const Packet8d& a_in) { + Packet2d a = + _mm_permute_pd(_mm256_extractf128_pd(_mm512_castpd512_pd256(a_in), 1), 3); + return _mm512_broadcastsd_pd(a); +} +template <> +EIGEN_STRONG_INLINE Packet16i +pbroadcast_first(const Packet16i& a_in) { + Packet4i a = _mm512_castsi512_si128(a_in); + return _mm512_broadcastd_epi32(a); +} +template <> +EIGEN_STRONG_INLINE Packet16i +pbroadcast_second(const Packet16i& a_in) { + Packet4i a = _mm512_castsi512_si128(a_in); + return _mm512_broadcastd_epi32(_mm_shuffle_epi32(a, _MM_SHUFFLE(1, 1, 1, 1))); +} +template <> +EIGEN_STRONG_INLINE Packet16i +pbroadcast_third(const Packet16i& a_in) { + Packet4i a = _mm512_castsi512_si128(a_in); + return _mm512_broadcastd_epi32(_mm_shuffle_epi32(a, _MM_SHUFFLE(2, 2, 2, 2))); +} +template <> +EIGEN_STRONG_INLINE Packet16i +pbroadcast_fourth(const Packet16i& a_in) { + Packet4i a = _mm512_castsi512_si128(a_in); + return _mm512_broadcastd_epi32(_mm_shuffle_epi32(a, _MM_SHUFFLE(3, 3, 3, 3))); +} +#endif + +#ifdef EIGEN_VECTORIZE_AVX +// For a Packet of Size 8 floats(256-bits), swap the 2nd and 3rd quadwords +template <> +EIGEN_STRONG_INLINE Packet8f pinterleave4x64(const Packet8f& from) { +#ifdef EIGEN_VECTORIZE_AVX2 + return _mm256_castsi256_ps(_mm256_permute4x64_epi64(_mm256_castps_si256(from), + _MM_SHUFFLE(3, 1, 2, 0))); +#else + auto tmp1 = _mm256_extract_epi32(_mm256_castps_si256(from), 2); + auto tmp2 = _mm256_extract_epi32(_mm256_castps_si256(from), 3); + auto tmp3 = _mm256_extract_epi32(_mm256_castps_si256(from), 4); + auto tmp4 = _mm256_extract_epi32(_mm256_castps_si256(from), 5); + auto tmp5 = _mm256_insert_epi32(_mm256_castps_si256(from), tmp1, 4); + tmp5 = _mm256_insert_epi32(tmp5, tmp2, 5); + tmp5 = _mm256_insert_epi32(tmp5, tmp3, 2); + tmp5 = _mm256_insert_epi32(tmp5, tmp4, 3); + return _mm256_castsi256_ps(tmp5); +#endif +} +// Return a Packet with 4 floats loaded from 4 bfloat16 values +template <> +EIGEN_STRONG_INLINE Packet8f pload4bf16(const float* from) { + __m128i zero = _mm_setzero_si128(); + __m128i tmp = _mm_castpd_si128(_mm_load_pd1((const double*)from)); + return _mm256_castps128_ps256( + _mm_castsi128_ps(_mm_unpacklo_epi16(zero, tmp))); +} +// Return a Packet with 2 floats loaded from 2 bfloat16 values +template <> +EIGEN_STRONG_INLINE Packet8f pload2bf16(const float* from) { + __m128i zero = _mm_setzero_si128(); + __m128i tmp = _mm_castps_si128(_mm_load_ps1(from)); + return _mm256_castps128_ps256( + _mm_castsi128_ps(_mm_unpacklo_epi16(zero, tmp))); +} + +#ifdef EIGEN_VECTORIZE_AVX512 +// Return a Packet with 4 floats loaded from 4 bfloat16 values +template <> +EIGEN_STRONG_INLINE Packet16f pload4bf16(const float* from) { + __m128i zero = _mm_setzero_si128(); + __m128i tmp = _mm_castpd_si128(_mm_load_pd1((const double*)from)); + return _mm512_castps128_ps512( + _mm_castsi128_ps(_mm_unpacklo_epi16(zero, tmp))); +} +// Return a Packet with 2 floats loaded from 2 bfloat16 values +template <> +EIGEN_STRONG_INLINE Packet16f pload2bf16(const float* from) { + __m128i zero = _mm_setzero_si128(); + __m128i tmp = _mm_castps_si128(_mm_load_ps1(from)); + return _mm512_castps128_ps512( + _mm_castsi128_ps(_mm_unpacklo_epi16(zero, tmp))); +} +#endif + +// For each 128-bit lane convert 4 bfloat to 4 float values from the lower half +// of the 128-bit lane +template +EIGEN_DEVICE_FUNC inline Packet8f pexpand_bf16_l(const Packet8f& from) { +#ifdef EIGEN_VECTORIZE_AVX2 + __m256i zero = _mm256_setzero_si256(); + __m256i tmp = _mm256_castps_si256(from); + return _mm256_castsi256_ps(_mm256_unpacklo_epi16(zero, tmp)); +#else + __m128i zero = _mm_setzero_si128(); + __m128i low = _mm_castps_si128(_mm256_extractf128_ps(from, 0)); + __m128i res_l = _mm_unpacklo_epi16(zero, low); + __m128i high = _mm_castps_si128(_mm256_extractf128_ps(from, 1)); + __m128i res_h = _mm_unpacklo_epi16(zero, high); + __m256 res = _mm256_castps128_ps256(_mm_castsi128_ps(res_l)); + res = _mm256_insertf128_ps(res, _mm_castsi128_ps(res_h), 1); + return res; +#endif +} + +// For each 128-bit lane convert 4 bfloat to 4 float values from the upper half +// of the 128-bit lane +template +EIGEN_DEVICE_FUNC inline Packet8f pexpand_bf16_u(const Packet8f& from) { +#ifdef EIGEN_VECTORIZE_AVX2 + __m256i zero = _mm256_setzero_si256(); + __m256i tmp = _mm256_castps_si256(from); + return _mm256_castsi256_ps(_mm256_unpackhi_epi16(zero, tmp)); +#else + __m128i zero = _mm_setzero_si128(); + __m128i low = _mm_castps_si128(_mm256_extractf128_ps(from, 0)); + __m128i res_l = _mm_unpackhi_epi16(zero, low); + __m128i high = _mm_castps_si128(_mm256_extractf128_ps(from, 1)); + __m128i res_h = _mm_unpackhi_epi16(zero, high); + __m256 res = _mm256_castps128_ps256(_mm_castsi128_ps(res_l)); + res = _mm256_insertf128_ps(res, _mm_castsi128_ps(res_h), 1); + return res; +#endif +} + +// Return a packet with the first value of the input Packet replicated +template <> +EIGEN_STRONG_INLINE Packet8f pbroadcast_first(const Packet8f& a) { + return _mm256_set1_ps(pfirst(a)); +} + +// Return a packet with the second value of the input Packet replicated +template <> +EIGEN_STRONG_INLINE Packet8f pbroadcast_second(const Packet8f& a) { + return _mm256_set1_ps( + _mm_cvtss_f32(_mm256_castps256_ps128(_mm256_permute_ps(a, 1)))); +} + +// Return a packet with the third value of the input Packet replicated +template <> +EIGEN_STRONG_INLINE Packet8f pbroadcast_third(const Packet8f& a) { + return _mm256_set1_ps( + _mm_cvtss_f32(_mm256_castps256_ps128(_mm256_permute_ps(a, 2)))); +} + +// Return a packet with the fourth value of the input Packet replicated +template <> +EIGEN_STRONG_INLINE Packet8f pbroadcast_fourth(const Packet8f& a) { + return _mm256_set1_ps( + _mm_cvtss_f32(_mm256_castps256_ps128(_mm256_permute_ps(a, 3)))); +} + +#endif + +#ifdef EIGEN_VECTORIZE_AVX512 + +template +EIGEN_DEVICE_FUNC inline Packet16f pexpand_bf16_l(const Packet16f& from) { + return _mm512_castsi512_ps(_mm512_slli_epi32( + _mm512_cvtepu16_epi32(_mm512_castsi512_si256(_mm512_castps_si512(from))), + 16)); +} + +template +EIGEN_DEVICE_FUNC inline Packet16f pexpand_bf16_u(const Packet16f& from) { + Packet16i tmp = _mm512_castps_si512(from); + Packet16i tmp2 = _mm512_alignr_epi32(tmp, tmp, 8); + return _mm512_castsi512_ps(_mm512_slli_epi32( + _mm512_cvtepu16_epi32(_mm512_castsi512_si256(tmp2)), 16)); +} + +#endif +} // namespace internal +} // namespace Eigen +#endif // TENSORFLOW_CORE_KERNELS_SPARSE_MATMUL_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/sparse_tensor_dense_add_op.h b/linux/include/tensorflow/core/kernels/sparse_tensor_dense_add_op.h new file mode 100644 index 000000000..c26ed5e87 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/sparse_tensor_dense_add_op.h @@ -0,0 +1,42 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_SPARSE_TENSOR_DENSE_ADD_OP_H_ +#define TENSORFLOW_CORE_KERNELS_SPARSE_TENSOR_DENSE_ADD_OP_H_ + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/kernels/scatter_functor.h" + +namespace tensorflow { +namespace functor { + +// TODO(zongheng): this should be a general functor that powers SparseAdd and +// ScatterNd ops. It should be moved to its own head file, once the other ops +// are implemented. +template +struct ScatterNdFunctor { + // Returns -1 on success or a nonnegative i s.t. indices[i] is a bad index. + Index operator()(const Device& d, typename TTypes::ConstMatrix indices, + typename TTypes::ConstFlat updates, + typename TTypes::Tensor out); +}; + +} // namespace functor +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_SPARSE_TENSOR_DENSE_ADD_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/sparse_tensor_dense_matmul_op.h b/linux/include/tensorflow/core/kernels/sparse_tensor_dense_matmul_op.h new file mode 100644 index 000000000..d6dd2deca --- /dev/null +++ b/linux/include/tensorflow/core/kernels/sparse_tensor_dense_matmul_op.h @@ -0,0 +1,74 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_SPARSE_TENSOR_DENSE_MATMUL_OP_H_ +#define TENSORFLOW_CORE_KERNELS_SPARSE_TENSOR_DENSE_MATMUL_OP_H_ + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/core/errors.h" + +namespace tensorflow { + +namespace functor { + +template +struct SparseTensorDenseMatMulFunctor { + static EIGEN_ALWAYS_INLINE Status Compute( + const Device& d, typename TTypes::Matrix out, + typename TTypes::ConstMatrix a_indices, + typename TTypes::ConstVec a_values, typename TTypes::ConstMatrix b); +}; + +template +class MaybeAdjoint; + +template +class MaybeAdjoint { + public: + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE MaybeAdjoint(MATRIX m) : m_(m) {} + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE typename MATRIX::Scalar operator()( + const typename MATRIX::Index i, const typename MATRIX::Index j) const { + return m_(i, j); + } + + private: + const MATRIX m_; +}; + +template +EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE T MaybeConj(T v) { + return v; +} + +template +class MaybeAdjoint { + public: + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE MaybeAdjoint(MATRIX m) : m_(m) {} + EIGEN_DEVICE_FUNC EIGEN_STRONG_INLINE typename MATRIX::Scalar operator()( + const typename MATRIX::Index i, const typename MATRIX::Index j) const { + return Eigen::numext::conj(m_(j, i)); + } + + private: + const MATRIX m_; +}; + +} // end namespace functor +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_SPARSE_TENSOR_DENSE_MATMUL_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/sparse_xent_op.h b/linux/include/tensorflow/core/kernels/sparse_xent_op.h new file mode 100644 index 000000000..6ba7931ab --- /dev/null +++ b/linux/include/tensorflow/core/kernels/sparse_xent_op.h @@ -0,0 +1,227 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_SPARSE_XENT_OP_H_ +#define TENSORFLOW_CORE_KERNELS_SPARSE_XENT_OP_H_ +// Functor definition for SparseXentOp, must be compilable by nvcc. + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/kernels/bounds_check.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +namespace sparse_xent_helpers { + +template +typename TTypes::Tensor32Bit To32BitConst( + typename TTypes::Vec in) { + return To32Bit(typename TTypes::ConstVec(in.data(), in.dimensions())); +} + +template +typename TTypes::Tensor32Bit To32BitConst( + typename TTypes::Matrix in) { + return To32Bit(typename TTypes::ConstMatrix(in.data(), in.dimensions())); +} + +} // namespace sparse_xent_helpers + +namespace generator { + +// Generator for calculation of the sparse Xent loss. +// This generator takes the logits, the sum of the exponentiated +// logits, and the label indices. For each minibatch entry, ignoring +// the batch index b, it calculates: +// +// loss[j] = (log(sum_exp_logits) - logits[j]) * 1{ j == label } +// +// for j = 0 .. num_classes. This value must be summed over all j for +// the final loss. +template +class SparseXentLossGenerator { + public: + EIGEN_DEVICE_FUNC EIGEN_ALWAYS_INLINE SparseXentLossGenerator( + typename TTypes::Tensor32Bit logits, + typename TTypes::Tensor32Bit sum_exp_logits, + typename TTypes::Tensor32Bit labels, + const Index max_depth) + : logits_(logits), + sum_exp_logits_(sum_exp_logits), + labels_(labels), + max_depth_(max_depth) {} + + EIGEN_DEVICE_FUNC EIGEN_ALWAYS_INLINE T + operator()(const Eigen::array& coords) const { + const int batch = coords[0]; + const int depth = coords[1]; + const Index label = tensorflow::internal::SubtleMustCopy(labels_(batch)); + if (!FastBoundsCheck(label, max_depth_)) { + return Eigen::NumTraits::quiet_NaN(); + } + return TF_PREDICT_FALSE(label == depth) + ? (Eigen::numext::log(sum_exp_logits_(batch)) - logits_(coords)) + : T(0.0); + }; + + private: + typename TTypes::Tensor32Bit logits_; + typename TTypes::Tensor32Bit sum_exp_logits_; + typename TTypes::Tensor32Bit labels_; + const Index max_depth_; +}; + +// Generator for calculation of the sparse Xent gradient. +// This generator takes the exponentiated logits, their sums, and the label +// indices. For each minibatch entry, ignoring the batch index b, it calculates: +// +// exp_logits[j] / sum_exp_logits - 1{ j == label } +// +// for j = 0 .. num_classes. +template +class SparseXentGradGenerator { + public: + EIGEN_DEVICE_FUNC EIGEN_ALWAYS_INLINE SparseXentGradGenerator( + typename TTypes::Tensor32Bit exp_logits, + typename TTypes::Tensor32Bit sum_exp_logits, + typename TTypes::Tensor32Bit labels, + const Index max_depth) + : exp_logits_(exp_logits), + sum_exp_logits_(sum_exp_logits), + labels_(labels), + max_depth_(max_depth) {} + + EIGEN_DEVICE_FUNC EIGEN_ALWAYS_INLINE T + operator()(const Eigen::array& coords) const { + const int batch = coords[0]; + const int depth = coords[1]; + const Index label = tensorflow::internal::SubtleMustCopy(labels_(batch)); + if (!FastBoundsCheck(label, max_depth_)) { + return Eigen::NumTraits::quiet_NaN(); + } + T subtract = TF_PREDICT_FALSE(depth == label) ? T(1.0) : T(0.0); + return exp_logits_(coords) / sum_exp_logits_(batch) - subtract; + }; + + private: + typename TTypes::Tensor32Bit exp_logits_; + typename TTypes::Tensor32Bit sum_exp_logits_; + typename TTypes::Tensor32Bit labels_; + const Index max_depth_; +}; + +} // namespace generator + +namespace functor { + +// Functor used by SparseXentOp to do the computations. +template +struct SparseXentFunctor { + // Computes Cross Entropy loss and backprop. + // + // logits: batch_size, num_classes. + // labels: num_classes. + // scratch: temporary tensor, dims: batch_size, 1 + // loss: output tensor for the loss, dims: batch_size. + // backprop: output tensor for the backprop, dims: batch_size, num_classes. + void operator()(const Device& d, typename TTypes::ConstMatrix logits, + typename TTypes::ConstVec labels, + typename TTypes::Vec scratch, typename TTypes::Vec loss, + typename TTypes::Matrix backprop); +}; + +// Eigen code implementing SparseXentFunctor::operator(). +// This code works for both CPU and GPU and is used by the functor +// specializations for both device types. +template +struct SparseXentEigenImpl { + static void Compute(const Device& d, typename TTypes::ConstMatrix logits, + typename TTypes::ConstVec labels, + typename TTypes::Vec scratch, + typename TTypes::Vec loss, + typename TTypes::Matrix backprop) { + // NOTE(touts): This duplicates some of the computations in softmax_op + // because we need the intermediate (logits -max(logits)) values to + // avoid a log(exp()) in the computation of the loss. + + const int kBatchDim = 0; + const int kClassDim = 1; + + const int batch_size = logits.dimension(kBatchDim); + const int num_classes = logits.dimension(kClassDim); + +// These arrays are used to reduce along the class dimension, and broadcast +// the resulting value to all classes. +#if !defined(EIGEN_HAS_INDEX_LIST) + Eigen::array along_class; + along_class[0] = kClassDim; + Eigen::array batch_only; + batch_only[0] = batch_size; + Eigen::array batch_by_one; + batch_by_one[0] = batch_size; + batch_by_one[1] = 1; + Eigen::array one_by_class; + one_by_class[0] = 1; + one_by_class[1] = num_classes; +#else + Eigen::IndexList > along_class; + Eigen::IndexList > batch_by_one; + batch_by_one.set(0, batch_size); + Eigen::IndexList batch_only; + batch_only.set(0, batch_size); + Eigen::IndexList, int> one_by_class; + one_by_class.set(1, num_classes); +#endif + + // scratch = max_logits along classes. + To32Bit(scratch).device(d) = To32Bit(logits).maximum(along_class); + + // backprop = logits - max_logits. + To32Bit(backprop).device(d) = + To32Bit(logits) - + To32Bit(scratch).reshape(batch_by_one).broadcast(one_by_class); + + // scratch = sum(exp(logits - max_logits)) along classes. + To32Bit(scratch).device(d) = To32Bit(backprop).exp().sum(along_class); + + // sum(-labels * + // ((logits - max_logits) - log(sum(exp(logits - max_logits))))) + // along classes + generator::SparseXentLossGenerator sparse_xent_loss_gen( + sparse_xent_helpers::To32BitConst(backprop), + sparse_xent_helpers::To32BitConst(scratch), To32Bit(labels), + backprop.dimension(1) /* max_depth */); + To32Bit(loss).device(d) = + To32Bit(backprop).generate(sparse_xent_loss_gen).sum(along_class); + + // backprop: prob - labels, where + // prob = exp(logits - max_logits) / sum(exp(logits - max_logits)) + To32Bit(backprop).device(d) = To32Bit(backprop).exp(); + generator::SparseXentGradGenerator sparse_xent_grad_gen( + sparse_xent_helpers::To32BitConst(backprop), + sparse_xent_helpers::To32BitConst(scratch), To32Bit(labels), + backprop.dimension(1) /* max_depth */); + To32Bit(backprop).device(d) = + To32Bit(backprop).generate(sparse_xent_grad_gen); + } +}; + +} // namespace functor + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_SPARSE_XENT_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/spectrogram.h b/linux/include/tensorflow/core/kernels/spectrogram.h new file mode 100644 index 000000000..fef0e6494 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/spectrogram.h @@ -0,0 +1,112 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// Class for generating spectrogram slices from a waveform. +// Initialize() should be called before calls to other functions. Once +// Initialize() has been called and returned true, The Compute*() functions can +// be called repeatedly with sequential input data (ie. the first element of the +// next input vector directly follows the last element of the previous input +// vector). Whenever enough audio samples are buffered to produce a +// new frame, it will be placed in output. Output is cleared on each +// call to Compute*(). This class is thread-unsafe, and should only be +// called from one thread at a time. +// With the default parameters, the output of this class should be very +// close to the results of the following MATLAB code: +// overlap_samples = window_length_samples - step_samples; +// window = hann(window_length_samples, 'periodic'); +// S = abs(spectrogram(audio, window, overlap_samples)).^2; + +#ifndef TENSORFLOW_CORE_KERNELS_SPECTROGRAM_H_ +#define TENSORFLOW_CORE_KERNELS_SPECTROGRAM_H_ + +#include +#include +#include + +#include "third_party/fft2d/fft.h" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/tensor.h" + +namespace tensorflow { + +class Spectrogram { + public: + Spectrogram() : initialized_(false) {} + ~Spectrogram() {} + + // Initializes the class with a given window length and step length + // (both in samples). Internally a Hann window is used as the window + // function. Returns true on success, after which calls to Process() + // are possible. window_length must be greater than 1 and step + // length must be greater than 0. + bool Initialize(int window_length, int step_length); + + // Initialize with an explicit window instead of a length. + bool Initialize(const std::vector& window, int step_length); + + // Processes an arbitrary amount of audio data (contained in input) + // to yield complex spectrogram frames. After a successful call to + // Initialize(), Process() may be called repeatedly with new input data + // each time. The audio input is buffered internally, and the output + // vector is populated with as many temporally-ordered spectral slices + // as it is possible to generate from the input. The output is cleared + // on each call before the new frames (if any) are added. + // + // The template parameters can be float or double. + template + bool ComputeComplexSpectrogram( + const std::vector& input, + std::vector>>* output); + + // This function works as the one above, but returns the power + // (the L2 norm, or the squared magnitude) of each complex value. + template + bool ComputeSquaredMagnitudeSpectrogram( + const std::vector& input, + std::vector>* output); + + // Return reference to the window function used internally. + const std::vector& GetWindow() const { return window_; } + + // Return the number of frequency channels in the spectrogram. + int output_frequency_channels() const { return output_frequency_channels_; } + + private: + template + bool GetNextWindowOfSamples(const std::vector& input, + int* input_start); + void ProcessCoreFFT(); + + int fft_length_; + int output_frequency_channels_; + int window_length_; + int step_length_; + bool initialized_; + int samples_to_next_step_; + + std::vector window_; + std::vector fft_input_output_; + std::deque input_queue_; + + // Working data areas for the FFT routines. + std::vector fft_integer_working_area_; + std::vector fft_double_working_area_; + + TF_DISALLOW_COPY_AND_ASSIGN(Spectrogram); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_SPECTROGRAM_H_ diff --git a/linux/include/tensorflow/core/kernels/split_lib.h b/linux/include/tensorflow/core/kernels/split_lib.h new file mode 100644 index 000000000..9d43a0082 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/split_lib.h @@ -0,0 +1,65 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_SPLIT_LIB_H_ +#define TENSORFLOW_CORE_KERNELS_SPLIT_LIB_H_ +// Functor definition for SplitOp, must be compilable by nvcc. + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/tensor_types.h" + +namespace tensorflow { +namespace functor { + +template +struct SplitCustom { + void operator()(const Device& d, typename TTypes::Tensor output, + typename TTypes::ConstTensor input, + const Eigen::DSizes& slice_indices, + const Eigen::DSizes& slice_sizes); +}; + +template +struct Split { + void operator()(const Device& d, typename TTypes::Tensor output, + typename TTypes::ConstTensor input, + const Eigen::DSizes& slice_indices, + const Eigen::DSizes& slice_sizes); +}; + +template +struct Split { + void operator()(const Eigen::ThreadPoolDevice& d, + typename TTypes::Tensor output, + typename TTypes::ConstTensor input, + const Eigen::DSizes& slice_indices, + const Eigen::DSizes& slice_sizes); +}; + +#ifdef TENSORFLOW_USE_SYCL +template +struct Split { + void operator()(const Eigen::SyclDevice& d, + typename TTypes::Tensor output, + typename TTypes::ConstTensor input, + const Eigen::DSizes& slice_indices, + const Eigen::DSizes& slice_sizes); +}; +#endif // TENSORFLOW_USE_SYCL + +} // namespace functor +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_SPLIT_LIB_H_ diff --git a/linux/include/tensorflow/core/kernels/squared-loss.h b/linux/include/tensorflow/core/kernels/squared-loss.h new file mode 100644 index 000000000..d256a6935 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/squared-loss.h @@ -0,0 +1,73 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_SQUARED_LOSS_H_ +#define TENSORFLOW_CORE_KERNELS_SQUARED_LOSS_H_ + +#include "tensorflow/core/kernels/loss.h" + +namespace tensorflow { + +class SquaredLossUpdater : public DualLossUpdater { + public: + // Closed form solution that decreases the dual squared loss. + // See page 23 of http://arxiv.org/pdf/1309.2375v2.pdf for the derivation of + // the update rule when the example weights are equal to 1.0. + // Note: There is a typo in the formula in the paper: the denominator should + // be 1 + ||x_i||^2/(\lambda n) (without the 2 multiplier). + // + // The CoCoA+ modification is detailed in readme.md. + double ComputeUpdatedDual(const int num_loss_partitions, const double label, + const double example_weight, + const double current_dual, const double wx, + const double weighted_example_norm) const final { + const double delta_numerator = label - current_dual - wx; + const double delta_denominator = + 1 + num_loss_partitions * weighted_example_norm * example_weight; + return current_dual + delta_numerator / delta_denominator; + } + + // Dual of squared loss function. + // https://en.wikipedia.org/wiki/Convex_conjugate + double ComputeDualLoss(const double current_dual, const double example_label, + const double example_weight) const final { + // Dual of the squared loss function = b * (y + b/2), where b is the + // dual variable and y is the label. This is Dual(-b). + return current_dual * (0.5 * current_dual - example_label) * example_weight; + } + + // Squared loss for linear regression. + double ComputePrimalLoss(const double wx, const double example_label, + const double example_weight) const final { + const double error = wx - example_label; + return error * error * example_weight * 0.5; + } + + inline double PrimalLossDerivative(const double wx, const double label, + const double example_weight) const final { + return (wx - label) * example_weight; + } + + inline double SmoothnessConstant() const final { return 1.0; } + + // Labels don't require conversion for linear regression. + Status ConvertLabel(float* const example_label) const final { + return Status::OK(); + } +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_SQUARED_LOSS_H_ diff --git a/linux/include/tensorflow/core/kernels/stateless_random_ops.h b/linux/include/tensorflow/core/kernels/stateless_random_ops.h new file mode 100644 index 000000000..bcd29c487 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/stateless_random_ops.h @@ -0,0 +1,34 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_STATELESS_RANDOM_OPS_H_ +#define TENSORFLOW_CORE_KERNELS_STATELESS_RANDOM_OPS_H_ + +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/lib/random/random_distributions.h" + +namespace tensorflow { + +// Generates a key and counter that can be used to seed a PhiloxRandom, +// generator, based on the seed value in `seed_t`. +// +// REQUIRES: `seed_t` must be a length-2 vector of type DT_INT{32,64}. +// `out_key` and `out_counter` must be non-null. +Status GenerateKey(Tensor seed_t, random::PhiloxRandom::Key* out_key, + random::PhiloxRandom::ResultType* out_counter); + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_STATELESS_RANDOM_OPS_H_ diff --git a/linux/include/tensorflow/core/kernels/strided_slice_op.h b/linux/include/tensorflow/core/kernels/strided_slice_op.h new file mode 100644 index 000000000..86d105391 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/strided_slice_op.h @@ -0,0 +1,140 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_STRIDED_SLICE_OP_H_ +#define TENSORFLOW_CORE_KERNELS_STRIDED_SLICE_OP_H_ + +// Functor definition for StridedSliceOp, must be compilable by nvcc. + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/resource_handle.h" +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/framework/variant_encode_decode.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { +namespace functor { + +template +struct StridedSlice { + void operator()(const Device& d, typename TTypes::Tensor output, + typename TTypes::ConstTensor input, + const Eigen::DSizes& start_indices, + const Eigen::DSizes& stop_indices, + const Eigen::DSizes& strides) { + const bool use_64bit = input.size() > Eigen::NumTraits::highest(); + if (!use_64bit && + Eigen::internal::is_same::value) { + Eigen::DSizes start_i, stop_i, strides_i; + for (int i = 0; i < NDIMS; ++i) { + start_i[i] = start_indices[i]; + stop_i[i] = stop_indices[i]; + strides_i[i] = strides[i]; + } + To32Bit(output).device(d) = + To32Bit(input).stridedSlice(start_i, stop_i, strides_i); + } else { + output.device(d) = + input.stridedSlice(start_indices, stop_indices, strides); + } + } +}; + +template +struct InitOutput { + static void run(const Device& d, typename TTypes::Tensor output) { + output.device(d) = output.constant(T(0)); + } +}; + +template +struct InitOutput { + static void run(const Device& d, + typename TTypes::Tensor output) { + output.device(d) = output.constant(ResourceHandle()); + } +}; + +template +struct InitOutput { + static void run(const Device& d, + typename TTypes::Tensor output) { + output.device(d) = output.constant(string()); + } +}; + +template +struct StridedSliceGrad { + void operator()(const Device& d, typename TTypes::Tensor output, + typename TTypes::ConstTensor input, + const Eigen::DSizes& start_indices, + const Eigen::DSizes& stop_indices, + const Eigen::DSizes& strides) { + InitOutput::run(d, output); + const bool use_64bit = input.size() > Eigen::NumTraits::highest(); + if (!use_64bit && + Eigen::internal::is_same::value) { + Eigen::DSizes start_i, stop_i, strides_i; + for (int i = 0; i < NDIMS; ++i) { + start_i[i] = start_indices[i]; + stop_i[i] = stop_indices[i]; + strides_i[i] = strides[i]; + } + To32Bit(output).stridedSlice(start_i, stop_i, strides_i).device(d) = + input; + } else { + output.stridedSlice(start_indices, stop_indices, strides).device(d) = + input; + } + } +}; + +template +struct StridedSliceAssign { + void operator()(const Device& d, typename TTypes::Tensor output, + typename TTypes::ConstTensor input, + const Eigen::DSizes& start_indices, + const Eigen::DSizes& stop_indices, + const Eigen::DSizes& strides) { + const bool use_64bit = input.size() > Eigen::NumTraits::highest(); + if (!use_64bit && + Eigen::internal::is_same::value) { + Eigen::DSizes start_i, stop_i, strides_i; + for (int i = 0; i < NDIMS; ++i) { + start_i[i] = start_indices[i]; + stop_i[i] = stop_indices[i]; + strides_i[i] = strides[i]; + } + To32Bit(output).stridedSlice(start_i, stop_i, strides_i).device(d) = + To32Bit(input); + } else { + output.stridedSlice(start_indices, stop_indices, strides).device(d) = + input; + } + } +}; + +template +struct StridedSliceAssignScalar { + void operator()(const Device& d, typename TTypes::Tensor output, + typename TTypes::ConstTensor input) { + output.device(d) = input; + } +}; + +} // namespace functor +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_STRIDED_SLICE_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/strided_slice_op_impl.h b/linux/include/tensorflow/core/kernels/strided_slice_op_impl.h new file mode 100644 index 000000000..099083b2f --- /dev/null +++ b/linux/include/tensorflow/core/kernels/strided_slice_op_impl.h @@ -0,0 +1,316 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_STRIDED_SLICE_OP_IMPL_H_ +#define TENSORFLOW_CORE_KERNELS_STRIDED_SLICE_OP_IMPL_H_ + +// Functor definition for StridedSliceOp, must be compilable by nvcc. + +#include "tensorflow/core/kernels/slice_op.h" +#include "tensorflow/core/kernels/strided_slice_op.h" + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/register_types.h" +#include "tensorflow/core/framework/register_types_traits.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/variant.h" +#include "tensorflow/core/framework/variant_encode_decode.h" +#include "tensorflow/core/kernels/bounds_check.h" +#include "tensorflow/core/kernels/dense_update_functor.h" +#include "tensorflow/core/kernels/ops_util.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/lib/gtl/array_slice.h" +#include "tensorflow/core/platform/mem.h" + +namespace tensorflow { + +template +void HandleStridedSliceCase(OpKernelContext* context, + const gtl::ArraySlice& begin, + const gtl::ArraySlice& end, + const gtl::ArraySlice& strides, + const TensorShape& processing_shape, + bool is_simple_slice, Tensor* result); + +template +void HandleStridedSliceGradCase(OpKernelContext* context, + const gtl::ArraySlice& begin, + const gtl::ArraySlice& end, + const gtl::ArraySlice& strides, + const TensorShape& processing_shape, + bool is_simple_slice, Tensor* result); + +template +class HandleStridedSliceAssignCase { + public: + void operator()(OpKernelContext* context, const gtl::ArraySlice& begin, + const gtl::ArraySlice& end, + const gtl::ArraySlice& strides, + const TensorShape& processing_shape, bool is_simple_slice, + Tensor* result); +}; +} // namespace tensorflow + +// The actual implementation. This is designed so multiple +// translation units can include this file in the form +// +// #define STRIDED_SLICE_INSTANTIATE_DIM 1 +// #include +// #undef STRIDED_SLICE_INSTANTIATE_DIM +// +#ifdef STRIDED_SLICE_INSTANTIATE_DIM + +namespace tensorflow { + +template +void HandleStridedSliceCase(OpKernelContext* context, + const gtl::ArraySlice& begin, + const gtl::ArraySlice& end, + const gtl::ArraySlice& strides, + const TensorShape& processing_shape, + bool is_simple_slice, Tensor* result) { + typedef typename proxy_type::type Proxy; + + gtl::InlinedVector processing_dims = processing_shape.dim_sizes(); + if (is_simple_slice) { + Eigen::DSizes begin_di; + Eigen::DSizes sizes_di; + for (int i = 0; i < NDIM; ++i) { + begin_di[i] = begin[i]; + sizes_di[i] = end[i] - begin[i]; + } + functor::Slice()( + context->eigen_device(), + result->bit_casted_shaped(processing_dims), + context->input(0).bit_casted_tensor(), begin_di, sizes_di); + } else { + Eigen::DSizes begin_di; + Eigen::DSizes end_di; + Eigen::DSizes strides_di; + for (int i = 0; i < NDIM; ++i) { + begin_di[i] = begin[i]; + end_di[i] = end[i]; + strides_di[i] = strides[i]; + } + functor::StridedSlice()( + context->eigen_device(), + result->bit_casted_shaped(processing_dims), + context->input(0).bit_casted_tensor(), begin_di, end_di, + strides_di); + } +} + +template +void HandleStridedSliceGradCase(OpKernelContext* context, + const gtl::ArraySlice& begin, + const gtl::ArraySlice& end, + const gtl::ArraySlice& strides, + const TensorShape& processing_shape, + bool is_simple_slice, Tensor* result) { + gtl::InlinedVector processing_dims = processing_shape.dim_sizes(); + + Eigen::DSizes begin_di; + Eigen::DSizes end_di; + Eigen::DSizes strides_di; + for (int i = 0; i < NDIM; ++i) { + begin_di[i] = begin[i]; + end_di[i] = end[i]; + strides_di[i] = strides[i]; + } + + typedef typename proxy_type::type Proxy; + functor::StridedSliceGrad()( + context->eigen_device(), result->bit_casted_tensor(), + context->input(4).bit_casted_shaped(processing_dims), + begin_di, end_di, strides_di); +} + +template +void HandleStridedSliceAssignCase::operator()( + OpKernelContext* context, const gtl::ArraySlice& begin, + const gtl::ArraySlice& end, const gtl::ArraySlice& strides, + const TensorShape& processing_shape, bool is_simple_slice, Tensor* result) { + gtl::InlinedVector processing_dims = processing_shape.dim_sizes(); + typedef typename proxy_type::type Proxy; + Eigen::DSizes begin_di; + Eigen::DSizes end_di; + Eigen::DSizes strides_di; + for (int i = 0; i < NDIM; ++i) { + begin_di[i] = begin[i]; + end_di[i] = end[i]; + strides_di[i] = strides[i]; + } + functor::StridedSliceAssign()( + context->eigen_device(), result->bit_casted_tensor(), + context->input(4).bit_casted_shaped(processing_dims), + begin_di, end_di, strides_di); +} + +template +class HandleStridedSliceAssignCase { + public: + enum { NDIM_PROXY = 1 }; + void operator()(OpKernelContext* context, const gtl::ArraySlice& begin, + const gtl::ArraySlice& end, + const gtl::ArraySlice& strides, + const TensorShape& processing_shape, bool is_simple_slice, + Tensor* result) { + gtl::InlinedVector processing_dims(1); + processing_dims[0] = 1; + + typedef typename proxy_type::type Proxy; + functor::StridedSliceAssignScalar()( + context->eigen_device(), + result->bit_casted_shaped(processing_dims), + context->input(4).bit_casted_shaped(processing_dims)); + } +}; + +// NODE(aselle): according to bsteiner, we need this because otherwise +// nvcc instantiates templates that are invalid. strided_slice_op_gpu.cu +// handles instantiates externally. It is important that this is done# + +// before the HandleXXCase's are instantiated to avoid duplicate +// specialization errors. + +#define PREVENT_INSTANTIATE_DIM1_AND_UP(T, NDIM) \ + namespace functor { \ + template <> \ + void StridedSlice::operator()( \ + const GPUDevice& d, typename TTypes::Tensor output, \ + typename TTypes::ConstTensor input, \ + const Eigen::DSizes& start, \ + const Eigen::DSizes& stop, \ + const Eigen::DSizes& strides); \ + extern template struct StridedSlice; \ + template <> \ + void Slice::operator()( \ + const GPUDevice& d, typename TTypes::Tensor output, \ + typename TTypes::ConstTensor input, \ + const Eigen::DSizes& indices, \ + const Eigen::DSizes& sizes); \ + extern template struct Slice; \ + template <> \ + void StridedSliceGrad::operator()( \ + const GPUDevice& d, typename TTypes::Tensor output, \ + typename TTypes::ConstTensor input, \ + const Eigen::DSizes& start, \ + const Eigen::DSizes& stop, \ + const Eigen::DSizes& strides); \ + extern template struct StridedSliceGrad; \ + template <> \ + void StridedSliceAssign::operator()( \ + const GPUDevice& d, typename TTypes::Tensor output, \ + typename TTypes::ConstTensor input, \ + const Eigen::DSizes& start, \ + const Eigen::DSizes& stop, \ + const Eigen::DSizes& strides); \ + extern template struct StridedSliceAssign; \ + } // namespace functor +#define PREVENT_INSTANTIATE_DIM0_ONLY(T, NDIM) \ + namespace functor { \ + template <> \ + void StridedSliceAssignScalar::operator()( \ + const GPUDevice& d, typename TTypes::Tensor output, \ + typename TTypes::ConstTensor input); \ + extern template struct StridedSliceAssignScalar; \ + } // namespace functor + +// Dimension 0 only instantiates some functors. So we only need +// to prevent ones defined by PREVENT_INSTANTIATE_DIM0_ONLY +#if GOOGLE_CUDA +#if STRIDED_SLICE_INSTANTIATE_DIM == 0 +#define PREVENT_INSTANTIATE(T, NDIM) PREVENT_INSTANTIATE_DIM0_ONLY(T, NDIM) +#else +#define PREVENT_INSTANTIATE(T, NDIM) PREVENT_INSTANTIATE_DIM1_AND_UP(T, NDIM) +#endif +#else +#define PREVENT_INSTANTIATE(T, NDIM) +#endif + +#define INSTANTIATE_DIM1_AND_UP_HANDLERS(DEVICE, T, DIM) \ + template void HandleStridedSliceCase( \ + OpKernelContext * context, const gtl::ArraySlice& begin, \ + const gtl::ArraySlice& end, \ + const gtl::ArraySlice& strides, \ + const TensorShape& processing_shape, bool is_simple_slice, \ + Tensor* result); \ + template void HandleStridedSliceGradCase( \ + OpKernelContext * context, const gtl::ArraySlice& begin, \ + const gtl::ArraySlice& end, \ + const gtl::ArraySlice& strides, \ + const TensorShape& processing_shape, bool is_simple_slice, \ + Tensor* result); + +#define INSTANTIATE_DIM0_AND_UP_HANDLERS(DEVICE, T, DIM) \ + template class HandleStridedSliceAssignCase; + +// Only some kernels need to be instantiated on dim 0. +#if STRIDED_SLICE_INSTANTIATE_DIM == 0 +#define INSTANTIATE(DEVICE, T, DIM) \ + INSTANTIATE_DIM0_AND_UP_HANDLERS(DEVICE, T, DIM) +#else +#define INSTANTIATE(DEVICE, T, DIM) \ + INSTANTIATE_DIM0_AND_UP_HANDLERS(DEVICE, T, DIM) \ + INSTANTIATE_DIM1_AND_UP_HANDLERS(DEVICE, T, DIM) +#endif + +#define DECLARE_FOR_N_CPU(T) \ + INSTANTIATE(CPUDevice, T, STRIDED_SLICE_INSTANTIATE_DIM) + +#define PREVENT_FOR_N_GPU(T) \ + PREVENT_INSTANTIATE(T, STRIDED_SLICE_INSTANTIATE_DIM) + +#define DECLARE_FOR_N_GPU(T) \ + INSTANTIATE(GPUDevice, T, STRIDED_SLICE_INSTANTIATE_DIM) + +#if GOOGLE_CUDA +TF_CALL_GPU_PROXY_TYPES(PREVENT_FOR_N_GPU); +TF_CALL_complex64(PREVENT_FOR_N_GPU); +TF_CALL_complex128(PREVENT_FOR_N_GPU); + +TF_CALL_GPU_NUMBER_TYPES(DECLARE_FOR_N_GPU); +TF_CALL_complex64(DECLARE_FOR_N_GPU); +TF_CALL_complex128(DECLARE_FOR_N_GPU); +DECLARE_FOR_N_GPU(int32); +DECLARE_FOR_N_GPU(int64); +#endif // END GOOGLE_CUDA + +TF_CALL_ALL_TYPES(DECLARE_FOR_N_CPU); + +#ifdef TENSORFLOW_USE_SYCL +#define PREVENT_FOR_N_SYCL(T) \ + PREVENT_INSTANTIATE(T, STRIDED_SLICE_INSTANTIATE_DIM) + +#define DECLARE_FOR_N_SYCL(T) \ + INSTANTIATE(SYCLDevice, T, STRIDED_SLICE_INSTANTIATE_DIM) + +TF_CALL_SYCL_PROXY_TYPES(PREVENT_FOR_N_SYCL); +TF_CALL_GPU_NUMBER_TYPES_NO_HALF(DECLARE_FOR_N_SYCL); +DECLARE_FOR_N_SYCL(int32); +DECLARE_FOR_N_SYCL(int64); + +#undef DECLARE_FOR_N_SYCL +#endif // TENSORFLOW_USE_SYCL + +#undef INSTANTIATE +#undef DECLARE_FOR_N_CPU +#undef DECLARE_FOR_N_GPU + +} // end namespace tensorflow + +#endif // END STRIDED_SLICE_INSTANTIATE_DIM +#endif // TENSORFLOW_CORE_KERNELS_STRIDED_SLICE_OP_IMPL_H_ diff --git a/linux/include/tensorflow/core/kernels/string_to_hash_bucket_op.h b/linux/include/tensorflow/core/kernels/string_to_hash_bucket_op.h new file mode 100644 index 000000000..62ef35bbb --- /dev/null +++ b/linux/include/tensorflow/core/kernels/string_to_hash_bucket_op.h @@ -0,0 +1,109 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_STRING_TO_HASH_BUCKET_OP_H_ +#define TENSORFLOW_CORE_KERNELS_STRING_TO_HASH_BUCKET_OP_H_ + +#include + +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/lib/core/errors.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/platform/macros.h" + +namespace tensorflow { + +template +class StringToHashBucketOp : public OpKernel { + public: + explicit StringToHashBucketOp(OpKernelConstruction* ctx) : OpKernel(ctx) { + OP_REQUIRES_OK(ctx, ctx->GetAttr("num_buckets", &num_buckets_)); + } + + void Compute(OpKernelContext* context) override { + const Tensor* input_tensor; + OP_REQUIRES_OK(context, context->input("input", &input_tensor)); + const auto& input_flat = input_tensor->flat(); + + Tensor* output_tensor = nullptr; + OP_REQUIRES_OK(context, + context->allocate_output("output", input_tensor->shape(), + &output_tensor)); + auto output_flat = output_tensor->flat(); + + typedef decltype(input_flat.size()) Index; + for (Index i = 0; i < input_flat.size(); ++i) { + const uint64 input_hash = hash(input_flat(i)); + const uint64 bucket_id = input_hash % num_buckets_; + // The number of buckets is always in the positive range of int64 so is + // the resulting bucket_id. Casting the bucket_id from uint64 to int64 is + // safe. + output_flat(i) = static_cast(bucket_id); + } + } + + private: + int64 num_buckets_; + + TF_DISALLOW_COPY_AND_ASSIGN(StringToHashBucketOp); +}; + +template +class StringToKeyedHashBucketOp : public OpKernel { + public: + explicit StringToKeyedHashBucketOp(OpKernelConstruction* ctx) + : OpKernel(ctx) { + OP_REQUIRES_OK(ctx, ctx->GetAttr("num_buckets", &num_buckets_)); + + std::vector key; + OP_REQUIRES_OK(ctx, ctx->GetAttr("key", &key)); + OP_REQUIRES(ctx, key.size() == 2, + errors::InvalidArgument("Key must have 2 elements")); + std::memcpy(key_, key.data(), sizeof(key_)); + } + + void Compute(OpKernelContext* context) override { + const Tensor* input_tensor; + OP_REQUIRES_OK(context, context->input("input", &input_tensor)); + const auto& input_flat = input_tensor->flat(); + + Tensor* output_tensor = nullptr; + OP_REQUIRES_OK(context, + context->allocate_output("output", input_tensor->shape(), + &output_tensor)); + auto output_flat = output_tensor->flat(); + + typedef decltype(input_flat.size()) Index; + for (Index i = 0; i < input_flat.size(); ++i) { + const uint64 input_hash = hash(key_, input_flat(i)); + const uint64 bucket_id = input_hash % num_buckets_; + // The number of buckets is always in the positive range of int64 so is + // the resulting bucket_id. Casting the bucket_id from uint64 to int64 is + // safe. + output_flat(i) = static_cast(bucket_id); + } + } + + private: + int64 num_buckets_; + uint64 key_[2]; + + TF_DISALLOW_COPY_AND_ASSIGN(StringToKeyedHashBucketOp); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_STRING_TO_HASH_BUCKET_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/string_util.h b/linux/include/tensorflow/core/kernels/string_util.h new file mode 100644 index 000000000..390cf5770 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/string_util.h @@ -0,0 +1,45 @@ +/* Copyright 2018 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CORE_KERNELS_STRING_UTIL_H_ +#define TENSORFLOW_CORE_KERNELS_STRING_UTIL_H_ + +#include "tensorflow/core/lib/core/status.h" + +namespace tensorflow { + +// Enumeration for unicode encodings. Used by ops such as +// tf.strings.unicode_encode and tf.strings.unicode_decode. +// TODO(edloper): Add support for: +// UTF16, UTF32, UTF16BE, UTF32BE, UTF16LE, UTF32LE +enum class UnicodeEncoding { UTF8 }; + +// Enumeration for character units. Used by string such as +// tf.strings.length and tf.substr. +// TODO(edloper): Add support for: UTF32_CHAR, etc. +enum class CharUnit { BYTE, UTF8_CHAR }; + +// Sets `encoding` based on `str`. +Status ParseUnicodeEncoding(const string& str, UnicodeEncoding* encoding); + +// Sets `unit` value based on `str`. +Status ParseCharUnit(const string& str, CharUnit* unit); + +// Returns the number of Unicode characters in a UTF-8 string. +// Result may be incorrect if the input string is not valid UTF-8. +int32 UTF8StrLen(const string& string); + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_STRING_UTIL_H_ diff --git a/linux/include/tensorflow/core/kernels/summary_interface.h b/linux/include/tensorflow/core/kernels/summary_interface.h new file mode 100644 index 000000000..1854fe552 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/summary_interface.h @@ -0,0 +1,61 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CORE_KERNELS_SUMMARY_INTERFACE_H_ +#define TENSORFLOW_CORE_KERNELS_SUMMARY_INTERFACE_H_ + +#include + +#include "tensorflow/core/framework/resource_mgr.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +class Event; +class GraphDef; + +// Main interface for the summary writer resource. +class SummaryWriterInterface : public ResourceBase { + public: + virtual ~SummaryWriterInterface() override {} + + // Flushes all unwritten messages in the queue. + virtual Status Flush() = 0; + + // These are called in the OpKernel::Compute methods for the summary ops. + virtual Status WriteTensor(int64 global_step, Tensor t, const string& tag, + const string& serialized_metadata) = 0; + + virtual Status WriteScalar(int64 global_step, Tensor t, + const string& tag) = 0; + + virtual Status WriteHistogram(int64 global_step, Tensor t, + const string& tag) = 0; + + virtual Status WriteImage(int64 global_step, Tensor t, const string& tag, + int max_images, Tensor bad_color) = 0; + + virtual Status WriteAudio(int64 global_step, Tensor t, const string& tag, + int max_outputs_, float sample_rate) = 0; + + virtual Status WriteGraph(int64 global_step, + std::unique_ptr graph) = 0; + + virtual Status WriteEvent(std::unique_ptr e) = 0; +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_SUMMARY_INTERFACE_H_ diff --git a/linux/include/tensorflow/core/kernels/svd_op_impl.h b/linux/include/tensorflow/core/kernels/svd_op_impl.h new file mode 100644 index 000000000..2a67700c1 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/svd_op_impl.h @@ -0,0 +1,108 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_SVD_OP_IMPL_H_ +#define TENSORFLOW_CORE_KERNELS_SVD_OP_IMPL_H_ + +// See docs in ../ops/linalg_ops.cc. +// +// This header file is used by the individual svd_*op*.cc files for registering +// individual kernels. A separate file is used for each instantiated kernel to +// improve compilation times. +#include + +#include "third_party/eigen3/Eigen/SVD" +#include "tensorflow/core/framework/kernel_def_builder.h" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/kernels/linalg_ops_common.h" +#include "tensorflow/core/lib/core/errors.h" +#include "tensorflow/core/platform/logging.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +template +class SvdOp : public LinearAlgebraOp { + public: + typedef LinearAlgebraOp Base; + + explicit SvdOp(OpKernelConstruction* context) : Base(context) { + OP_REQUIRES_OK(context, context->GetAttr("compute_uv", &compute_uv_)); + OP_REQUIRES_OK(context, context->GetAttr("full_matrices", &full_matrices_)); + } + + using TensorShapes = typename Base::TensorShapes; + + void ValidateInputMatrixShapes( + OpKernelContext* context, + const TensorShapes& input_matrix_shapes) const final { + Base::ValidateSingleMatrix(context, input_matrix_shapes); + } + + TensorShapes GetOutputMatrixShapes( + const TensorShapes& input_matrix_shapes) const final { + int64 m = input_matrix_shapes[0].dim_size(0); + int64 n = input_matrix_shapes[0].dim_size(1); + int64 min_size = std::min(m, n); + if (compute_uv_) { + return TensorShapes({TensorShape({min_size}), + TensorShape({m, full_matrices_ ? m : min_size}), + TensorShape({n, full_matrices_ ? n : min_size})}); + } else { + return TensorShapes({TensorShape({min_size})}); + } + } + + // TODO(rmlarsen): This should depend on compute_uv. See b/30409375. + int64 GetCostPerUnit(const TensorShapes& input_matrix_shapes) const final { + double m = static_cast(input_matrix_shapes[0].dim_size(0)); + double n = static_cast(input_matrix_shapes[0].dim_size(1)); + double cost = 12 * std::max(m, n) * std::min(m, n) * std::min(m, n); + return cost >= static_cast(kint64max) ? kint64max + : static_cast(cost); + } + + using Matrix = typename Base::Matrix; + using MatrixMaps = typename Base::MatrixMaps; + using ConstMatrixMap = typename Base::ConstMatrixMap; + using ConstMatrixMaps = typename Base::ConstMatrixMaps; + + void ComputeMatrix(OpKernelContext* context, const ConstMatrixMaps& inputs, + MatrixMaps* outputs) final { + int options = 0; // Don't compute singular vectors; + if (compute_uv_) { + options = full_matrices_ ? Eigen::ComputeFullU | Eigen::ComputeFullV + : Eigen::ComputeThinU | Eigen::ComputeThinV; + } + Eigen::BDCSVD svd(inputs[0], options); + outputs->at(0) = svd.singularValues().template cast(); + if (compute_uv_) { + outputs->at(1) = svd.matrixU(); + outputs->at(2) = svd.matrixV(); + } + } + + private: + bool compute_uv_; + bool full_matrices_; + + TF_DISALLOW_COPY_AND_ASSIGN(SvdOp); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_SVD_OP_IMPL_H_ diff --git a/linux/include/tensorflow/core/kernels/tensor_array.h b/linux/include/tensorflow/core/kernels/tensor_array.h new file mode 100644 index 000000000..384a63e94 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/tensor_array.h @@ -0,0 +1,633 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_TENSOR_ARRAY_H_ +#define TENSORFLOW_CORE_KERNELS_TENSOR_ARRAY_H_ + +#include +#include + +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/partial_tensor_shape.h" +#include "tensorflow/core/framework/register_types.h" +#include "tensorflow/core/framework/resource_mgr.h" +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_shape.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/kernels/aggregate_ops.h" +#include "tensorflow/core/kernels/fill_functor.h" +#include "tensorflow/core/lib/core/errors.h" +#include "tensorflow/core/platform/logging.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +typedef Eigen::ThreadPoolDevice CPUDevice; +typedef Eigen::GpuDevice GPUDevice; + +namespace tensor_array { + +// Full implementations are in tensor_array.cc +template +Status AddToTensor(OpKernelContext* ctx, Tensor* sum, const Tensor* current, + const Tensor* add) { + return errors::InvalidArgument( + "tensor_array::AddToTensor type not supported: ", + DataTypeString(DataTypeToEnum::value)); +}; + +#define TENSOR_ARRAY_WRITE_OR_ADD(Device, T) \ + template <> \ + Status AddToTensor(OpKernelContext * ctx, Tensor * sum, \ + const Tensor* current, const Tensor* add); + +#define TENSOR_ARRAY_WRITE_OR_ADD_CPU(T) TENSOR_ARRAY_WRITE_OR_ADD(CPUDevice, T) +TF_CALL_NUMBER_TYPES(TENSOR_ARRAY_WRITE_OR_ADD_CPU) +#undef TENSOR_ARRAY_WRITE_OR_ADD_CPU + +#if GOOGLE_CUDA + +#define TENSOR_ARRAY_WRITE_OR_ADD_GPU(T) TENSOR_ARRAY_WRITE_OR_ADD(GPUDevice, T) +TF_CALL_GPU_NUMBER_TYPES(TENSOR_ARRAY_WRITE_OR_ADD_GPU); +TF_CALL_complex64(TENSOR_ARRAY_WRITE_OR_ADD_GPU); +TF_CALL_complex128(TENSOR_ARRAY_WRITE_OR_ADD_GPU); +#undef TENSOR_ARRAY_WRITE_OR_ADD_GPU + +#endif // GOOGLE_CUDA + +#undef TENSOR_ARRAY_WRITE_OR_ADD + +template +Status TensorSetZero(OpKernelContext* ctx, Tensor* value) { + return errors::InvalidArgument( + "tensor_array::TensorSetZero type not supported: ", + DataTypeString(DataTypeToEnum::value)); +}; + +#define TENSOR_ARRAY_SET_ZERO(Device, T) \ + template <> \ + Status TensorSetZero(OpKernelContext * ctx, Tensor * value); + +#define TENSOR_ARRAY_SET_ZERO_CPU(T) TENSOR_ARRAY_SET_ZERO(CPUDevice, T) +TF_CALL_NUMBER_TYPES(TENSOR_ARRAY_SET_ZERO_CPU); +TF_CALL_bool(TENSOR_ARRAY_SET_ZERO_CPU); +#undef TENSOR_ARRAY_SET_ZERO_CPU + +#if GOOGLE_CUDA + +#define TENSOR_ARRAY_SET_ZERO_GPU(T) TENSOR_ARRAY_SET_ZERO(GPUDevice, T) +TF_CALL_GPU_NUMBER_TYPES(TENSOR_ARRAY_SET_ZERO_GPU); +TF_CALL_complex64(TENSOR_ARRAY_SET_ZERO_GPU); +TF_CALL_complex128(TENSOR_ARRAY_SET_ZERO_GPU); +#undef TENSOR_ARRAY_SET_ZERO_GPU + +#endif // GOOGLE_CUDA + +#undef TENSOR_ARRAY_SET_ZERO + +} // namespace tensor_array + +// The TensorArray object keeps an array of PersistentTensors. It +// allows reading from the array and writing to the array. +// +// Important properties: +// * Usually, writing to a particular index in the TensorArray is allowed at +// most once per index. In a special case, writes with the flag +// multiple_writes_aggregate allow multiple writes to the same +// index. In this case, the writes are summed. +// * Multiple reads are supported. +// * Deep copies of PersistentTensors are rarely made. The only +// time they are made is when WriteOrAggregate is called at least twice +// on the same index with the flag multiple_writes_aggregate = True. +// * Reading and Writing to the array is protected by a mutex. +// All operations on a TensorArray are thread-safe. +// * A TensorArray may be preemptively closed, which releases all +// memory associated with it. +// +// These properties together allow the TensorArray to work as a +// functional object and makes gradient computation easy. For +// example: +// * Write-Once semantics mean the gradient of a TensorArray Read never has to +// worry which of multiple writes to that index the gradient value +// is meant for. +// * Read-Many semantics (when using clear_after_read=false) allow the +// TensorArray to be read, packed, or concatenated multiple times; +// and the gradient operations use the multiple_writes_aggregate +// flag to aggregate the backprop writes. Multiple backprop writes to +// the same index are partial gradients corresponding to the +// multiple reads of that index in the forward phase. +// +class TensorArray : public ResourceBase { + public: + static std::atomic tensor_array_counter; + + // Construct a TensorArray for holding Tensors of type 'dtype' with + // 'N' elements. While the underlying storage is a std::vector and + // can hold more than MAX_INT entries, in practice we do not expect + // users to construct this many Tensors for storage in a TensorArray. + TensorArray(const string& key, const DataType& dtype, const Tensor& handle, + int32 N, const PartialTensorShape& element_shape, + bool identical_element_shapes, bool dynamic_size, + bool multiple_writes_aggregate, bool is_grad, int32 marked_size, + bool clear_after_read) + : key_(key), + dtype_(dtype), + handle_(handle), + closed_(false), + dynamic_size_(dynamic_size), + multiple_writes_aggregate_(multiple_writes_aggregate), + gradients_disallowed_(false), + clear_after_read_(clear_after_read), + is_grad_(is_grad), + marked_size_(marked_size), + element_shape_(element_shape), + identical_element_shapes_(identical_element_shapes), + tensors_(N) {} + + // Write PersistentTensor 'value' to index 'index'. + // + // Preconditions: + // * The TensorArray is not closed + // * If the array has dynamic size: + // The index is >= 0 + // Otherwise: + // The index is in [0, N) where N == Size() + // * The dtype of the Tensor in 'value' matches the TensorArray's dtype. + // * If multiple_writes_aggregate is false: + // The Tensor at 'index' has not yet been written to. + // * If multiple_writes_aggregate is true: + // The Tensor at 'index' has the same shape as value. + // + // Side effects: + // * On the first write to 'index': + // - The underlying Tensor in 'value' has a new reference to it. + // - The index 'index' is marked as written. + // * If multiple_writes_aggregate is false, subsequent writes to 'index' + // raise an InvalidArgument error. + // * If multiple_writes_aggregate is true, subsequent writes to 'index': + // - The underlying Tensors in 'value' and from the first write + // are released and a local PersistentTensor is created. + // - Index 'index' is also marked as local_copy. + // - The gradients_disallowed flag is set true (GradientsAllowed() + // will now return false). + // + // Note, value is passed as a pointer because we its underlying + // Tensor's shape is accessed. Otherwise it is not modified. + template + Status WriteOrAggregate(OpKernelContext* ctx, const int32 index, + PersistentTensor* value) { + mutex_lock l(mu_); + return LockedWriteOrAggregate(ctx, index, value); + } + + template + Status WriteOrAggregateMany(OpKernelContext* ctx, + const std::vector& indices, + std::vector* values) { + mutex_lock l(mu_); + int32 i = 0; + for (const int32 ix : indices) { + Status s = LockedWriteOrAggregate(ctx, ix, &(*values)[i]); + ++i; + TF_RETURN_IF_ERROR(s); + } + return Status::OK(); + } + + // Read from index 'index' into PersistentTensor 'value'. + // + // Preconditions: + // * The TensorArray is not closed + // * The index is in [0, N) + // * The Tensor at 'index' has been written to. + // * The Tensor at 'index' has not been read from with flag + // clear_after_read = true. + // + // Side effects: + // * If clear_after_read is true, the reference to the underlying + // Tensor is deleted. + // * The reference to the underlying Tensor at 'index' is copied to + // the returned '*value'. + // * The index is marked as read (it cannot be rewritten to). + template + Status Read(OpKernelContext* ctx, const int32 index, + PersistentTensor* value) { + mutex_lock l(mu_); + return LockedRead(ctx, index, value); + } + + template + Status ReadMany(OpKernelContext* ctx, const std::vector& indices, + std::vector* values) { + mutex_lock l(mu_); + values->clear(); + values->resize(indices.size()); + int32 i = 0; + for (const int32 ix : indices) { + Status s = LockedRead(ctx, ix, &(*values)[i]); + ++i; + if (!s.ok()) return s; + } + return Status::OK(); + } + + DataType ElemType() const { return dtype_; } + + PartialTensorShape ElemShape() { + mutex_lock l(mu_); + return element_shape_; + } + + Status SetElemShape(const PartialTensorShape& candidate) { + mutex_lock l(mu_); + PartialTensorShape new_element_shape_; + Status s = element_shape_.MergeWith(candidate, &new_element_shape_); + if (!s.ok()) { + return s; + } + element_shape_ = new_element_shape_; + return Status::OK(); + } + + string DebugString() override { + mutex_lock l(mu_); + CHECK(!closed_); + return strings::StrCat("TensorArray[", tensors_.size(), "]"); + } + + bool IsClosed() { + mutex_lock l(mu_); + return closed_; + } + + // Return the size of the TensorArray. + Status Size(int32* size) { + mutex_lock l(mu_); + TF_RETURN_IF_ERROR(LockedReturnIfClosed()); + *size = tensors_.size(); + return Status::OK(); + } + + // Record the size of the TensorArray after an unpack or split. + Status SetMarkedSize(int32 size) { + mutex_lock l(mu_); + TF_RETURN_IF_ERROR(LockedReturnIfClosed()); + if (!is_grad_) { + marked_size_ = size; + } + return Status::OK(); + } + + // Return the marked size of the TensorArray. + Status MarkedSize(int32* size) { + mutex_lock l(mu_); + TF_RETURN_IF_ERROR(LockedReturnIfClosed()); + *size = marked_size_; + return Status::OK(); + } + + // Return the size that should be used by pack or concat op. + Status PackOrConcatSize(int32* size) { + mutex_lock l(mu_); + TF_RETURN_IF_ERROR(LockedReturnIfClosed()); + *size = is_grad_ ? marked_size_ : tensors_.size(); + return Status::OK(); + } + + // Once a TensorArray is being used for gradient calculations, it + // should be marked as no longer resizeable. + void DisableDynamicSize() { + mutex_lock l(mu_); + dynamic_size_ = false; + } + + bool HasDynamicSize() { + mutex_lock l(mu_); + return dynamic_size_; + } + + bool GradientsAllowed() { + mutex_lock l(mu_); + return !gradients_disallowed_; + } + + bool HasIdenticalElementShapes() const { return identical_element_shapes_; } + + // Copy the TensorShapes from another TensorArray into this one. + // If `shapes_to_prepend` is set, expands the rank of the copied shape by + // prepending the passed in shape prefix to the shape values in `rhs`. + // The sizes of the two TensorArrays must match and this one + // may not have any entries filled in. This performs a "soft copy", + // essentially filling the current TensorArray with virtual + // zero-tensors, which will be replaced by future aggregate writes, + // or instantiated by future reads. Requires a non-const pointer + // to the rhs to access its mutex. + Status CopyShapesFrom(TensorArray* rhs, const TensorShape* shape_to_prepend); + + // Clear the TensorArray, including any Tensor references, and mark as closed. + void ClearAndMarkClosed() { + mutex_lock l(mu_); + tensors_.clear(); + closed_ = true; + } + + mutex* mu() { return &mu_; } + Tensor* handle() { return &handle_; } + + ResourceHandle resource_handle(OpKernelContext* ctx) { + return MakePerStepResourceHandle(ctx, key_); + } + + private: + Status LockedWrite(OpKernelContext* ctx, const int32 index, + PersistentTensor* value) EXCLUSIVE_LOCKS_REQUIRED(mu_); + + template + Status LockedWriteOrAggregate(OpKernelContext* ctx, const int32 index, + PersistentTensor* value) + EXCLUSIVE_LOCKS_REQUIRED(mu_); + + template + Status LockedRead(OpKernelContext* ctx, const int32 index, + PersistentTensor* value) EXCLUSIVE_LOCKS_REQUIRED(mu_); + + Status LockedReturnIfClosed() const EXCLUSIVE_LOCKS_REQUIRED(mu_) { + if (closed_) { + return errors::InvalidArgument("TensorArray ", handle_.vec()(1), + " has already been closed."); + } + return Status::OK(); + } + + const string key_; + + const DataType dtype_; + Tensor handle_; + + mutex mu_; + + // Marks that the tensor_array_ has been cleared. + bool closed_ GUARDED_BY(mu_); + + // Writes are allowed to grow the array. + bool dynamic_size_; + + // Multiple writes to the same index will result in summation of the + // values (used by backprop) + const bool multiple_writes_aggregate_; + + // If multiple Writes were attempted (e.g. via attribute + // multiple_writes_aggregate), then gradients are disallowed. + bool gradients_disallowed_ GUARDED_BY(mu_); + + // After a read at an index, clear away its PersistentTensor to + // release memory. + const bool clear_after_read_; + + // True iff this is a gradient tensor array. + const bool is_grad_; + + // The size of the TensorArray after a (legacy) unpack or split is performed. + // -1 if there has been no unpack or split performed on the TensorArray. + int32 marked_size_; + + // The shape of each element in the TensorArray, may be partially known or not + // known at all. + PartialTensorShape element_shape_ GUARDED_BY(mu_); + + // Whether all elements in the TensorArray have identical shapes. + // This allows certain behaviors, like dynamically checking for + // consistent shapes on write, and being able to fill in properly + // shaped zero tensors on stack -- even if the initial element_shape + // was not fully defined. + const bool identical_element_shapes_; + + // TensorAndState is used to keep track of the PersistentTensors + // stored in the TensorArray, along with their shapes, and a boolean + // that determines whether they have already been read or not. + struct TensorAndState { + TensorAndState() + : written(false), read(false), cleared(false), local_copy(false) {} + PersistentTensor tensor; + TensorShape shape; + bool written; // True if a Tensor has been written to the index. + bool read; // True if a Tensor has been written to and read from the index. + bool cleared; // True if a tensor has been read with + // clear_after_read = true; + + // Used by writes when multiple_writes_aggregate is true. In this + // case, the first time a value is written, it is a shallow copy. + // The second time a value is written, it is aggregated. However, + // in this case a new Tensor must be constructed to hold the + // aggregated value. This flag marks that such a Tensor is being + // used. All future writes will aggregate to the existing local Tensor. + bool local_copy; + }; + // The list of underlying PersistentTensors and states. + std::vector tensors_ GUARDED_BY(mu_); +}; + +template +Status TensorArray::LockedWriteOrAggregate(OpKernelContext* ctx, + const int32 index, + PersistentTensor* value) { + TF_RETURN_IF_ERROR(LockedReturnIfClosed()); + size_t index_size = static_cast(index); + if (index < 0 || (!dynamic_size_ && index_size >= tensors_.size())) { + return errors::InvalidArgument( + "TensorArray ", handle_.vec()(1), ": Tried to write to index ", + index, " but array is not resizeable and size is: ", tensors_.size()); + } + if (dynamic_size_) { + // We must grow the internal TensorArray + if (index_size >= tensors_.capacity()) { + tensors_.reserve(2 * (index_size + 1)); + } + if (index_size >= tensors_.size()) { + tensors_.resize(index_size + 1); + } + } + TensorAndState& t = tensors_[index]; + + Tensor* value_t = value->AccessTensor(ctx); + if (value_t->dtype() != dtype_) { + return errors::InvalidArgument( + "TensorArray ", handle_.vec()(1), + ": Could not write to TensorArray index ", index, + " because the value dtype is ", DataTypeString(value_t->dtype()), + " but TensorArray dtype is ", DataTypeString(dtype_), "."); + } + if (!element_shape_.IsCompatibleWith(value_t->shape())) { + return errors::InvalidArgument( + "TensorArray ", handle_.vec()(1), + ": Could not write to TensorArray index ", index, + " because the value shape is ", value_t->shape().DebugString(), + " which is incompatible with the TensorArray's inferred element " + "shape: ", + element_shape_.DebugString(), " (consider setting infer_shape=False)."); + } else if (identical_element_shapes_ && !element_shape_.IsFullyDefined()) { + element_shape_ = PartialTensorShape(value_t->shape().dim_sizes()); + } + + if (t.read) { + return errors::InvalidArgument("TensorArray ", handle_.vec()(1), + ": Could not write to TensorArray index ", + index, " because it has already been read."); + } + + if (!multiple_writes_aggregate_ && t.written) { + return errors::InvalidArgument("TensorArray ", handle_.vec()(1), + ": Could not write to TensorArray index ", + index, + " because it has already been written to."); + } + + if (t.written) { + DCHECK(multiple_writes_aggregate_); + + // Check that value_t shape matches t.shape + if (value_t->shape() != t.shape) { + return errors::InvalidArgument( + "TensorArray ", handle_.vec()(1), + ": Could not aggregate to TensorArray index ", index, + " because the existing shape is ", t.shape.DebugString(), + " but the new input shape is ", value_t->shape().DebugString(), "."); + } + + if (!t.tensor.IsInitialized() || t.tensor.NumElements() == 0) { + // If existing_t == nullptr but written == true, then what was stored + // was just a shape, which just means zeros. So all we must do in this + // case is copy the reference over and return early. + t.tensor = *value; + return Status::OK(); + } + + Tensor* existing_t = t.tensor.AccessTensor(ctx); + + if (t.local_copy) { + Status s = tensor_array::AddToTensor(ctx, existing_t, + existing_t, value_t); + TF_RETURN_IF_ERROR(s); + } else { + PersistentTensor local_tensor; + Tensor* local_tensor_t; + TF_RETURN_IF_ERROR(ctx->allocate_persistent( + dtype_, existing_t->shape(), &local_tensor, &local_tensor_t)); + Status s = tensor_array::AddToTensor(ctx, local_tensor_t, + existing_t, value_t); + TF_RETURN_IF_ERROR(s); + t.tensor = local_tensor; + t.local_copy = true; + } + + // We've aggregated the values, so disallow backprop on this + // TensorArray. + gradients_disallowed_ = true; + } else { + t.tensor = *value; + t.shape = value_t->shape(); + t.written = true; + } + return Status::OK(); +} + +template +Status TensorArray::LockedRead(OpKernelContext* ctx, const int32 index, + PersistentTensor* value) { + TF_RETURN_IF_ERROR(LockedReturnIfClosed()); + if ((index < 0) || + (!is_grad_ && (static_cast(index) >= tensors_.size()))) { + return errors::InvalidArgument("Tried to read from index ", index, + " but array size is: ", tensors_.size()); + } + size_t index_t = static_cast(index); + if ((is_grad_ && (index_t >= tensors_.size() || !tensors_[index].written)) || + (!is_grad_ && (index_t < tensors_.size() && !tensors_[index].written))) { + // Special case returning zeros if this is a gradient read that happens + // after a stop_gradients call with dynamic forward TensorArrays. + // There is sometimes a race condition where the gradient is not + // written due to stop_gradients, but is later read. + TensorShape element_shape; + if (is_grad_ && index_t < tensors_.size() && + tensors_[index].shape.dims() > 0) { + // A gradient TensorArray has more specific gradient information + // available for each entry. A forward TensorArray must rely on + // the global element_shape_ to fill in zeros on read. + element_shape = tensors_[index].shape; + } else if (!element_shape_.IsFullyDefined()) { + return errors::InvalidArgument( + "TensorArray ", handle_.vec()(1), + ": Could not read from TensorArray index ", index, + ". Furthermore, the element shape is not fully defined: ", + element_shape_.DebugString(), + ". It is possible you are working with a resizeable TensorArray and " + "stop_gradients is not allowing the gradients to be written. If you " + "set the full " + "element_shape property on the forward TensorArray, the proper " + "all-zeros tensor " + "will be returned instead of incurring this error."); + } else { + element_shape_.AsTensorShape(&element_shape); // Always succeeds. + } + if (index_t >= tensors_.size()) { + // Fill in tensors_ up to index to have known shape. + size_t old_tensors_size = tensors_.size(); + tensors_.resize(index + 1); + for (size_t i = old_tensors_size; i < index + 1; ++i) { + tensors_[i].shape = element_shape; + tensors_[i].written = true; + } + } else { + tensors_[index].shape = element_shape; + tensors_[index].written = true; + } + } + + TensorAndState& t = tensors_[index]; + + if (t.cleared) { + return errors::InvalidArgument("TensorArray ", handle_.vec()(1), + ": Could not read index ", index, + " twice because it was cleared after a " + "previous read (perhaps try setting " + "clear_after_read = false?)."); + } + + if (!t.tensor.IsInitialized() || t.tensor.NumElements() == 0) { + // We stored just a shape, but no value. This means create and + // return zeros of the appropriate shape. + Tensor* tensor_t; + TF_RETURN_IF_ERROR( + ctx->allocate_persistent(dtype_, t.shape, &t.tensor, &tensor_t)); + if (t.shape.num_elements() > 0) { + Status s = tensor_array::TensorSetZero(ctx, tensor_t); + if (!s.ok()) return s; + } + } + + // Data is available inside the tensor, copy the reference over. + *value = t.tensor; + + if (clear_after_read_) { + t.tensor = PersistentTensor(); + t.cleared = true; + } + t.read = true; + return Status::OK(); +} + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_TENSOR_ARRAY_H_ diff --git a/linux/include/tensorflow/core/kernels/tile_functor.h b/linux/include/tensorflow/core/kernels/tile_functor.h new file mode 100644 index 000000000..95986af8b --- /dev/null +++ b/linux/include/tensorflow/core/kernels/tile_functor.h @@ -0,0 +1,109 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_TILE_FUNCTOR_H_ +#define TENSORFLOW_CORE_KERNELS_TILE_FUNCTOR_H_ + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" + +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +namespace internal { + +// Device-specific naive implementation for tile. +template +void TileSimple(const Device& d, Tensor* out, const Tensor& in); + +template +void TileUsingEigen(const Device& d, Tensor* out, const Tensor& in, + const gtl::ArraySlice& broadcast_array) { + auto x = in.tensor(); + auto y = out->tensor(); + + Eigen::array b; + for (int i = 0; i < NDIM; ++i) b[i] = broadcast_array[i]; + if (Eigen::internal::is_same::value) { + // Use 32bit indexing to speed up the computations + To32Bit(y).device(d) = To32Bit(x).broadcast(b); + } else { + y.device(d) = x.broadcast(b); + } +} + +template +void TileUsingEigen(const Device& d, Tensor* out, const Tensor& in, + const gtl::ArraySlice&) { + auto x = in.tensor(); + auto y = out->tensor(); + // In the scalar case we simply copy the input. + y.device(d) = x; +} + +} // end namespace internal + +namespace functor { + +template +struct Tile { + void operator()(const Device& d, Tensor* out, const Tensor& in, + const gtl::ArraySlice broadcast_array) const { + switch (in.dims()) { + case 0: + internal::TileUsingEigen(d, out, in, + broadcast_array); + break; + case 1: + internal::TileUsingEigen(d, out, in, + broadcast_array); + break; + case 2: + internal::TileUsingEigen(d, out, in, + broadcast_array); + break; + case 3: + internal::TileUsingEigen(d, out, in, + broadcast_array); + break; + case 4: + internal::TileUsingEigen(d, out, in, + broadcast_array); + break; + case 5: + internal::TileUsingEigen(d, out, in, + broadcast_array); + break; + case 6: + internal::TileUsingEigen(d, out, in, + broadcast_array); + break; + case 7: + internal::TileUsingEigen(d, out, in, + broadcast_array); + break; + default: + internal::TileSimple(d, out, in); + break; + } + } +}; + +} // end namespace functor +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_TILE_FUNCTOR_H_ diff --git a/linux/include/tensorflow/core/kernels/tile_ops_cpu_impl.h b/linux/include/tensorflow/core/kernels/tile_ops_cpu_impl.h new file mode 100644 index 000000000..df6a666cd --- /dev/null +++ b/linux/include/tensorflow/core/kernels/tile_ops_cpu_impl.h @@ -0,0 +1,71 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CORE_KERNELS_TILE_OPS_CPU_IMPL_H_ +#define TENSORFLOW_CORE_KERNELS_TILE_OPS_CPU_IMPL_H_ + +#define EIGEN_USE_THREADS + +#include "tensorflow/core/framework/register_types.h" +#include "tensorflow/core/kernels/tile_ops_impl.h" + +namespace tensorflow { + +namespace functor { + +typedef Eigen::ThreadPoolDevice CPUDevice; + +// Register functors used for TileGradientOp. +#define DEFINE_DIM(T, NDIM) \ + template struct TileGrad; \ + template struct ReduceAndReshape; +#define DEFINE_TYPE(T) DEFINE_DIM(T, CPU_PROVIDED_IXDIM) + +TF_CALL_float(DEFINE_TYPE); +TF_CALL_double(DEFINE_TYPE); +TF_CALL_int16(DEFINE_TYPE); +TF_CALL_int32(DEFINE_TYPE); +TF_CALL_int64(DEFINE_TYPE); +TF_CALL_half(DEFINE_TYPE); +TF_CALL_complex64(DEFINE_TYPE); +TF_CALL_complex128(DEFINE_TYPE); + +#undef DEFINE_DIM +#undef DEFINE_TYPE + +#ifdef TENSORFLOW_USE_SYCL +typedef Eigen::SyclDevice SYCLDevice; + +// Register functors used for TileGradientOp. +#define DEFINE_DIM(T, NDIM) \ + template struct TileGrad; \ + template struct ReduceAndReshape; +#define DEFINE_TYPE(T) DEFINE_DIM(T, CPU_PROVIDED_IXDIM) + +TF_CALL_bool(DEFINE_TYPE); +TF_CALL_float(DEFINE_TYPE); +TF_CALL_double(DEFINE_TYPE); +TF_CALL_uint8(DEFINE_TYPE); +TF_CALL_int16(DEFINE_TYPE); +TF_CALL_int32(DEFINE_TYPE); +TF_CALL_int64(DEFINE_TYPE); + +#undef DEFINE_DIM +#undef DEFINE_TYPE +#endif // TENSORFLOW_USE_SYCL + +} // end namespace functor +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_TILE_OPS_CPU_IMPL_H_ diff --git a/linux/include/tensorflow/core/kernels/tile_ops_gpu_impl.h b/linux/include/tensorflow/core/kernels/tile_ops_gpu_impl.h new file mode 100644 index 000000000..8da337dab --- /dev/null +++ b/linux/include/tensorflow/core/kernels/tile_ops_gpu_impl.h @@ -0,0 +1,59 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_TILE_OPS_GPU_IMPL_H_ +#define TENSORFLOW_CORE_KERNELS_TILE_OPS_GPU_IMPL_H_ + +// Header used to split up compilation of GPU tile ops. For each type you want +// to have tile ops, create a .cu.cc file containing +// +// #if GOOGLE_CUDA +// #include "tensorflow/core/kernels/tile_ops_gpu_impl.h" +// DEFINE_TILE_OPS(NDIM) +// #endif // GOGLE_CUDA +// +// where NDIM is an integer. +// +// NOTE(keveman): Eigen's int8 and string versions don't compile yet with nvcc. + +#ifndef GOOGLE_CUDA +#error "This header must be included inside #ifdef GOOGLE_CUDA" +#endif + +#define EIGEN_USE_GPU + +#include +#include "tensorflow/core/framework/numeric_types.h" +#include "tensorflow/core/kernels/tile_ops_impl.h" + +#define DEFINE_DIM(T, NDIM) \ + template struct TileGrad; \ + template struct ReduceAndReshape; + +#define DEFINE_TILE_OPS(NDIM) \ + namespace tensorflow { \ + namespace functor { \ + DEFINE_DIM(int16, NDIM) \ + DEFINE_DIM(int32, NDIM) \ + DEFINE_DIM(int64, NDIM) \ + DEFINE_DIM(Eigen::half, NDIM) \ + DEFINE_DIM(float, NDIM) \ + DEFINE_DIM(double, NDIM) \ + DEFINE_DIM(complex64, NDIM) \ + DEFINE_DIM(complex128, NDIM) \ + } \ + } + +#endif // TENSORFLOW_CORE_KERNELS_TILE_OPS_GPU_IMPL_H_ diff --git a/linux/include/tensorflow/core/kernels/tile_ops_impl.h b/linux/include/tensorflow/core/kernels/tile_ops_impl.h new file mode 100644 index 000000000..6a9de388c --- /dev/null +++ b/linux/include/tensorflow/core/kernels/tile_ops_impl.h @@ -0,0 +1,71 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_TILE_OPS_IMPL_H_ +#define TENSORFLOW_CORE_KERNELS_TILE_OPS_IMPL_H_ + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +namespace functor { + +template +struct TileGrad { + void operator()(const Device& d, typename TTypes::Tensor out, + typename TTypes::ConstTensor in, + const Eigen::DSizes& indices, + const Eigen::DSizes& sizes, + bool first) const { + if (first) { + out.device(d) = in.slice(indices, sizes); + } else { + out.device(d) += in.slice(indices, sizes); + } + } +}; + +template +struct TileGrad { + void operator()(const Device& d, typename TTypes::Tensor out, + typename TTypes::ConstTensor in, + const Eigen::DSizes&, + const Eigen::DSizes&, + bool first) const { + if (first) { + out.device(d) = in; + } else { + out.device(d) += in; + } + } +}; + +template +struct ReduceAndReshape { + void operator()( + const Device& d, typename TTypes::Tensor out, + typename TTypes::ConstTensor in, + const Eigen::DSizes& reduce_dim, + const Eigen::DSizes& reshape_dim) const { + out.device(d) = in.sum(reduce_dim).reshape(reshape_dim); + } +}; + +} // end namespace functor +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_TILE_OPS_IMPL_H_ diff --git a/linux/include/tensorflow/core/kernels/topk_op.h b/linux/include/tensorflow/core/kernels/topk_op.h new file mode 100644 index 000000000..1fdbc5b15 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/topk_op.h @@ -0,0 +1,42 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_TOPK_OP_H_ +#define TENSORFLOW_CORE_KERNELS_TOPK_OP_H_ + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/framework/types.h" +#include "tensorflow/core/lib/core/errors.h" + +namespace tensorflow { + +namespace functor { + +template +struct TopKFunctor { + static Status Compute(OpKernelContext* context, bool sorted, int k, + const typename TTypes::ConstTensor& input, + const int64 num_rows, const int64 num_cols, + typename TTypes::Tensor values, + typename TTypes::Tensor indices); +}; + +} // end namespace functor + +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_TOPK_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/training_op_helpers.h b/linux/include/tensorflow/core/kernels/training_op_helpers.h new file mode 100644 index 000000000..9f173a80f --- /dev/null +++ b/linux/include/tensorflow/core/kernels/training_op_helpers.h @@ -0,0 +1,126 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_TRAINING_OP_HELPERS_H_ +#define TENSORFLOW_CORE_KERNELS_TRAINING_OP_HELPERS_H_ + +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/variant_op_registry.h" +#include "tensorflow/core/kernels/dense_update_functor.h" +#include "tensorflow/core/kernels/variable_ops.h" + +namespace tensorflow { + +// Returns a borrowed pointer to the mutex for the variable `input` in `ctx`. +// +// If `input` corresponds to a `DT_RESOURCE`-type variable input, +// `*maybe_resource` will be updated to contain the underlying resource, and the +// caller will be responsible for calling `Unref()` on that resource. +mutex* GetTrainingVariableMutex(OpKernelContext* ctx, int input, + Var** maybe_resource); + +// Utility structure that releases a sequence of borrowed mutexes when it is +// deleted. +struct VariableInputLockHolder { + public: + VariableInputLockHolder(std::vector vars, + std::unique_ptr> locks) + : vars_(std::move(vars)), locks_(std::move(locks)) {} + + VariableInputLockHolder(VariableInputLockHolder&& other) + : vars_(std::move(other.vars_)), locks_(std::move(other.locks_)) {} + + ~VariableInputLockHolder() { + // Release the locks before unreffing the Vars, because each lock + // is potentially borrowed from a Var in vars_. + locks_.reset(); + for (Var* var : vars_) { + var->Unref(); + } + } + + private: + std::vector vars_; + // NOTE: Use a `std::unique_ptr` instead of moving in a vector directly, + // because a `std::vector` is not movable on all platforms. + std::unique_ptr> locks_; +}; + +VariableInputLockHolder MaybeLockVariableInputMutexesInOrder( + OpKernelContext* ctx, bool do_lock, const std::vector& input_ids); + +void MaybeForwardRefInputToRefOutput(OpKernelContext* ctx, int input, + int output); + +// This is for use with ResourceVariables to ensure *tensor has a +// reference count of 1 before you update it. +// REQUIRES: If you pass in variable->tensor(), *variable->mu() must be held. +template +Status PrepareToUpdateVariable(OpKernelContext* ctx, Tensor* tensor) { + if (!tensor->RefCountIsOne()) { + // Tensor's buffer is in use by some read, so we need to copy before + // updating. + PersistentTensor unused; + Tensor* tmp; + if (std::is_same::value) { + AllocatorAttributes attr; + attr.set_on_host(true); + TF_RETURN_IF_ERROR(ctx->allocate_persistent( + tensor->dtype(), tensor->shape(), &unused, &tmp, attr)); + + const auto elements_in = tensor->flat(); + auto elements_out = tmp->flat(); + for (int64 i = 0; i < elements_in.size(); ++i) { + elements_out(i) = elements_in(i); + } + } else { + AllocatorAttributes attr; + attr.set_gpu_compatible(true); + attr.set_nic_compatible(true); + TF_RETURN_IF_ERROR(ctx->allocate_persistent( + tensor->dtype(), tensor->shape(), &unused, &tmp, attr)); + functor::DenseUpdate copy_functor; + copy_functor(ctx->eigen_device(), tmp->flat(), + const_cast(tensor)->flat()); + } + *tensor = *tmp; + } + return Status::OK(); +} + +// This gives you `*out`, a tensor you can update, corresponding to a +// variable passed as input index `input`. This handles the +// differences between reference and resource variables. For resource +// variables, we ensure `*out` has a reference count of 1 (using +// PrepareToUpdateVariable() to copy if necessary) unless +// sparse && !lock_held, in which case it never copies. +template +Status GetInputTensorFromVariable(OpKernelContext* ctx, int input, + bool lock_held, bool sparse, Tensor* out) { + if (ctx->input_dtype(input) == DT_RESOURCE) { + Var* var; + TF_RETURN_IF_ERROR(LookupResource(ctx, HandleFromInput(ctx, input), &var)); + core::ScopedUnref unref_var(var); + TF_RETURN_IF_ERROR(PrepareToUpdateVariable(ctx, var->tensor())); + *out = *var->tensor(); + return Status::OK(); + } + *out = ctx->mutable_input(input, lock_held); + return Status::OK(); +} + +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_TRAINING_OP_HELPERS_H_ diff --git a/linux/include/tensorflow/core/kernels/training_ops.h b/linux/include/tensorflow/core/kernels/training_ops.h new file mode 100644 index 000000000..e10a4cb12 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/training_ops.h @@ -0,0 +1,202 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_TRAINING_OPS_H_ +#define TENSORFLOW_CORE_KERNELS_TRAINING_OPS_H_ + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { +namespace functor { + +// Each training algorithm has a ApplyXYZ functor struct declared in +// this header file. They are specialized for different devices +// (CPUDevice in training_ops.cc or GPUDevice in training_ops_gpu.cc). + +template +struct ApplyGradientDescent { + void operator()(const Device& d, typename TTypes::Flat var, + typename TTypes::ConstScalar alpha, + typename TTypes::ConstFlat delta); +}; + +template +struct ApplyAdadelta { + void operator()(const Device& d, typename TTypes::Flat var, + typename TTypes::Flat accum, + typename TTypes::Flat accum_update, + typename TTypes::ConstScalar lr, + typename TTypes::ConstScalar rho, + typename TTypes::ConstScalar epsilon, + typename TTypes::ConstFlat grad); +}; + +template +struct FobosElasticNet { + void operator()(const Device& d, typename TTypes::Flat var, + typename TTypes::ConstScalar lr, + typename TTypes::ConstScalar l1, + typename TTypes::ConstScalar l2, + typename TTypes::ConstFlat grad); +}; + +template +struct ApplyProximalGradientDescent { + void operator()(const Device& d, typename TTypes::Flat var, + typename TTypes::ConstScalar lr, + typename TTypes::ConstScalar l1, + typename TTypes::ConstScalar l2, + typename TTypes::ConstFlat grad); +}; + +template +struct ApplyAdagrad { + void operator()(const Device& d, typename TTypes::Flat var, + typename TTypes::Flat accum, + typename TTypes::ConstScalar lr, + typename TTypes::ConstFlat grad, bool update_slots); +}; + +template +struct ApplyAdagradDA { + void operator()(const Device& d, typename TTypes::Flat var, + typename TTypes::Flat gradient_accum, + typename TTypes::Flat gradient_squared_accum, + typename TTypes::ConstScalar lr, int64 global_step, + typename TTypes::ConstScalar l1, + typename TTypes::ConstScalar l2, + typename TTypes::ConstFlat grad); +}; + +template +struct ApplyProximalAdagrad { + void operator()(const Device& d, typename TTypes::Flat var, + typename TTypes::Flat accum, + typename TTypes::ConstScalar lr, + typename TTypes::ConstScalar l1, + typename TTypes::ConstScalar l2, + typename TTypes::ConstFlat grad); +}; + +template +struct ApplyFtrl { + void operator()(const Device& d, typename TTypes::Flat var, + typename TTypes::Flat accum, + typename TTypes::Flat linear, + typename TTypes::ConstFlat grad, + typename TTypes::ConstScalar lr, + typename TTypes::ConstScalar l1, + typename TTypes::ConstScalar l2, + typename TTypes::ConstScalar lr_power); +}; + +template +struct ApplyFtrlV2 { + void operator()(const Device& d, typename TTypes::Flat var, + typename TTypes::Flat accum, + typename TTypes::Flat linear, + typename TTypes::ConstFlat grad, + typename TTypes::ConstScalar lr, + typename TTypes::ConstScalar l1, + typename TTypes::ConstScalar l2, + typename TTypes::ConstScalar l2_shrinkage, + typename TTypes::ConstScalar lr_power); +}; + +template +struct ApplyMomentum { + void operator()(const Device& d, typename TTypes::Flat var, + typename TTypes::Flat accum, + typename TTypes::ConstScalar lr, + typename TTypes::ConstFlat grad, + typename TTypes::ConstScalar momentum, bool use_nesterov); +}; + +template +struct ApplyAdam { + void operator()(const Device& d, typename TTypes::Flat var, + typename TTypes::Flat m, typename TTypes::Flat v, + typename TTypes::ConstScalar beta1_power, + typename TTypes::ConstScalar beta2_power, + typename TTypes::ConstScalar lr, + typename TTypes::ConstScalar beta1, + typename TTypes::ConstScalar beta2, + typename TTypes::ConstScalar epsilon, + typename TTypes::ConstFlat grad, bool use_nesterov); +}; + +template +struct ApplyAdaMax { + void operator()(const Device& d, typename TTypes::Flat var, + typename TTypes::Flat m, typename TTypes::Flat v, + typename TTypes::ConstScalar beta1_power, + typename TTypes::ConstScalar lr, + typename TTypes::ConstScalar beta1, + typename TTypes::ConstScalar beta2, + typename TTypes::ConstScalar epsilon, + typename TTypes::ConstFlat grad); +}; + +template +struct ApplyRMSProp { + void operator()(const Device& d, typename TTypes::Flat var, + typename TTypes::Flat ms, typename TTypes::Flat mom, + typename TTypes::ConstScalar lr, + typename TTypes::ConstScalar rho, + typename TTypes::ConstScalar momentum, + typename TTypes::ConstScalar epsilon, + typename TTypes::ConstFlat grad); +}; + +template +struct ApplyCenteredRMSProp { + void operator()(const Device& d, typename TTypes::Flat var, + typename TTypes::Flat mg, typename TTypes::Flat ms, + typename TTypes::Flat mom, + typename TTypes::ConstScalar lr, + typename TTypes::ConstScalar rho, + typename TTypes::ConstScalar momentum, + typename TTypes::ConstScalar epsilon, + typename TTypes::ConstFlat grad); +}; + +template +struct ApplyAddSign { + void operator()(const Device& d, typename TTypes::Flat var, + typename TTypes::Flat m, + typename TTypes::ConstScalar lr, + typename TTypes::ConstScalar alpha, + typename TTypes::ConstScalar sign_decay, + typename TTypes::ConstScalar beta, + typename TTypes::ConstFlat grad); +}; + +template +struct ApplyPowerSign { + void operator()(const Device& d, typename TTypes::Flat var, + typename TTypes::Flat m, + typename TTypes::ConstScalar lr, + typename TTypes::ConstScalar logbase, + typename TTypes::ConstScalar sign_decay, + typename TTypes::ConstScalar beta, + typename TTypes::ConstFlat grad); +}; + +} // end namespace functor +} // end namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_TRAINING_OPS_H_ diff --git a/linux/include/tensorflow/core/kernels/transpose_functor.h b/linux/include/tensorflow/core/kernels/transpose_functor.h new file mode 100644 index 000000000..add463533 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/transpose_functor.h @@ -0,0 +1,259 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_TRANSPOSE_FUNCTOR_H_ +#define TENSORFLOW_CORE_KERNELS_TRANSPOSE_FUNCTOR_H_ + +#include +#include +#include +#include "tensorflow/core/framework/tensor.h" +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/platform/logging.h" + +namespace tensorflow { +// Transpose tensor 'in' into tensor 'out' according to dimension +// permutation 'perm'. +// +// REQUIRES: in.dtype() == out->dtype() +// REQUIRES: in.dims() == out->dims() +// REQUIRES: in.dims() == perm.size() +// REQUIRES: in.dim_size(perm[i]) == out->dim_size(i) +template +Status DoTranspose(const Device& device, const Tensor& in, + const gtl::ArraySlice perm, Tensor* out); + +// Conjugate and transpose tensor 'in' into tensor 'out' according to dimension +// permutation 'perm'. +// +// REQUIRES: in.dtype() == out->dtype() +// REQUIRES: in.dims() == out->dims() +// REQUIRES: in.dims() == perm.size() +// REQUIRES: in.dim_size(perm[i]) == out->dim_size(i) +template +Status DoConjugateTranspose(const Device& device, const Tensor& in, + const gtl::ArraySlice perm, Tensor* out); + +// Convenience versions of DoTranspose that only swap the last (inner) two +// dimensions. +template +Status DoMatrixTranspose(const Device& device, const Tensor& in, Tensor* out); + +// Convenience versions of DoConjugateTranspose that only swap the last (inner) +// two dimensions. +template +Status DoConjugateMatrixTranspose(const Device& device, const Tensor& in, + Tensor* out); + +// Primary device specific functor to be specialized for each device and type. +template +struct Transpose { + static void run(const Device& d, const Tensor& in, + const gtl::ArraySlice perm, Tensor* out); +}; + +// Implementation details. +namespace internal { + +typedef gtl::InlinedVector TransposeDimsVec; +typedef gtl::InlinedVector TransposePermsVec; + +// Helper function that takes a tensor shape, a permutation, combines the +// neighboring shapes if their indices in the permutation are consecutive. +// The function outputs the combined shape and new permutation. +// Example: Tensor shape {2, 3, 4, 5, 120} and permutation {0, 4, 1, 2, 3} will +// produce new shape {2, 60, 120} and new permutation {0, 2, 1}. +inline void ReduceTransposeDimensions(const TensorShape& shape, + gtl::ArraySlice perm, + TransposePermsVec* new_perm, + TransposeDimsVec* new_dims) { + CHECK_EQ(shape.dims(), perm.size()); + if (shape.dims() == 1) { + // If input dimension is already 1, no need to reduce dimension. + new_perm->resize(1); + (*new_perm)[0] = perm[0]; + (*new_dims)[0] = shape.dim_size(0); + return; + } + TransposePermsVec new_dim_position(shape.dims(), -1); + TransposeDimsVec combined_dims(shape.dims(), 0); + int cur_head = perm[0]; + new_dim_position[cur_head] = 0; + combined_dims[0] = shape.dim_size(cur_head); + int dim_idx = 0; + for (int perm_idx = 1; perm_idx < shape.dims(); ++perm_idx) { + // If two indices in permutation are consecutive numbers, combine their + // dimensions. + if (cur_head + 1 == perm[perm_idx]) { + cur_head = perm[perm_idx]; + combined_dims[dim_idx] *= shape.dim_size(cur_head); + } else { + // Else start a new dimension. + cur_head = perm[perm_idx]; + dim_idx++; + new_dim_position[cur_head] = dim_idx; + combined_dims[dim_idx] = shape.dim_size(cur_head); + } + } + // Compact the new permutations and dimension sizes. + new_perm->resize(dim_idx + 1); + new_dims->resize(dim_idx + 1); + dim_idx = 0; + for (int i = 0; i < new_dim_position.size(); ++i) { + if (new_dim_position[i] >= 0) { + int new_perm_idx = new_dim_position[i]; + (*new_perm)[dim_idx] = new_perm_idx; + (*new_dims)[dim_idx] = combined_dims[new_perm_idx]; + dim_idx++; + } + } +} + +// If all non-singleton dimensions remain in ascending order, the shuffled +// singletons can be transposed by a reshape, saving a memory allocation & copy. +// |permutation| must be a permutation of {0, .., input_shape.dims() - 1}. +// That is, for all i, 0 <= perm[i] < input_shape.dims(). +// In practice, this is checked in TransposeOp::Compute prior to calling this +// function, and the function sits here to facilitate unit testing. +inline bool NonSingletonDimensionsAlign(const TensorShape& input_shape, + const std::vector& permutation) { + int last_nonsingleton_perm_dim = -1; + for (int perm_dim : permutation) { + if (input_shape.dim_size(perm_dim) == 1) { + continue; + } + if (perm_dim < last_nonsingleton_perm_dim) { + return false; + } + last_nonsingleton_perm_dim = perm_dim; + } + return true; +} + +// Uses Eigen to transpose. +template +void TransposeUsingEigen(const Device& d, const Tensor& in, + const gtl::ArraySlice perm, bool conjugate, + Tensor* out) { + Eigen::array p; + for (int i = 0; i < NDIMS; ++i) p[i] = perm[i]; + auto x = typename TTypes::ConstTensor( + reinterpret_cast(in.tensor_data().data()), + in.shape().AsEigenDSizes()); + auto y = typename TTypes::Tensor( + reinterpret_cast(const_cast(out->tensor_data().data())), + out->shape().AsEigenDSizes()); + if (conjugate) { + y.device(d) = x.conjugate().shuffle(p); + } else { + y.device(d) = x.shuffle(p); + } +} + +template +Status DoTransposeImpl(const Device& d, const Tensor& in, + const gtl::ArraySlice perm, bool conjugate, + Tensor* out) { + CHECK_GE(in.dims(), 2); + CHECK_EQ(in.dims(), out->dims()); + CHECK_EQ(in.dims(), perm.size()); + CHECK_EQ(in.dtype(), out->dtype()); + switch (in.dtype()) { + case DT_BOOL: + case DT_INT8: + case DT_QINT8: + case DT_QUINT8: + case DT_UINT8: + Transpose::run(d, in, perm, out); + break; + + case DT_BFLOAT16: + case DT_HALF: + case DT_INT16: + case DT_QINT16: + case DT_QUINT16: + case DT_UINT16: + Transpose::run(d, in, perm, out); + break; + + case DT_FLOAT: + case DT_INT32: + case DT_QINT32: + Transpose::run(d, in, perm, out); + break; + + case DT_DOUBLE: + case DT_INT64: + Transpose::run(d, in, perm, out); + break; + + case DT_COMPLEX64: + if (conjugate) { +#if defined(__ANDROID__) and !defined(__clang__) + // Workaround for GCC compiler bug in Android toolchain. + return errors::Unimplemented( + "Conjugate transpose of complex64 not supported for GCC on " + "Android."); +#else + Transpose::run(d, in, perm, out); +#endif + } else { + Transpose::run(d, in, perm, out); + } + break; + + case DT_COMPLEX128: + if (conjugate) { + Transpose::run(d, in, perm, + out); + } else { + Transpose::run(d, in, perm, + out); + } + break; + + case DT_STRING: + Transpose::run(d, in, perm, out); + break; + + default: + return errors::Unimplemented("Unsupported dtype on CPU: ", in.dtype()); + } + return Status::OK(); +} + +template +inline Status DoMatrixTransposeImpl(const Device& device, const Tensor& in, + bool conjugate, Tensor* out) { + const int ndims = in.dims(); + if (ndims == 0) return Status::OK(); + TransposePermsVec perm(ndims); + std::iota(perm.begin(), perm.end(), 0); + std::swap(perm[ndims - 2], perm[ndims - 1]); + return DoTransposeImpl(device, in, perm, conjugate, out); +} + +#ifdef TENSORFLOW_USE_SYCL +// For SYCL lets always go through Eigen +template +void TransposeSYCL(const Device& d, const Tensor& in, + const gtl::ArraySlice perm, bool conjugate, + Tensor* out); +#endif // TENSORFLOW_USE_SYCL + +} // namespace internal +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_TRANSPOSE_FUNCTOR_H_ diff --git a/linux/include/tensorflow/core/kernels/transpose_op.h b/linux/include/tensorflow/core/kernels/transpose_op.h new file mode 100644 index 000000000..9e8c57376 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/transpose_op.h @@ -0,0 +1,127 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_KERNELS_TRANSPOSE_OP_H_ +#define TENSORFLOW_KERNELS_TRANSPOSE_OP_H_ + +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/tensor.h" + +namespace tensorflow { + +class TransposeOp : public OpKernel { + public: + explicit TransposeOp(OpKernelConstruction* ctx) : OpKernel(ctx) {} + + void Compute(OpKernelContext* ctx) override; + + protected: + virtual Status DoTranspose(OpKernelContext* ctx, const Tensor& in, + gtl::ArraySlice perm, Tensor* out) = 0; + virtual bool IsConjugate() const { return false; } +}; + +class TransposeCpuOp : public TransposeOp { + public: + explicit TransposeCpuOp(OpKernelConstruction* ctx) : TransposeOp(ctx) {} + + protected: + Status DoTranspose(OpKernelContext* ctx, const Tensor& in, + gtl::ArraySlice perm, Tensor* out) override; +}; + +#if defined(INTEL_MKL) +class MklTransposeCpuOp : public TransposeOp { + public: + explicit MklTransposeCpuOp(OpKernelConstruction* ctx) : TransposeOp(ctx) {} + + protected: + Status DoTranspose(OpKernelContext* ctx, const Tensor& in, + gtl::ArraySlice perm, Tensor* out) override; +}; +#endif // INTEL_MKL + +class TransposeGpuOp : public TransposeOp { + public: + explicit TransposeGpuOp(OpKernelConstruction* ctx) : TransposeOp(ctx) {} + + protected: + Status DoTranspose(OpKernelContext* ctx, const Tensor& in, + gtl::ArraySlice perm, Tensor* out) override; +}; + +#ifdef TENSORFLOW_USE_SYCL +class TransposeSyclOp : public TransposeOp { + public: + explicit TransposeSyclOp(OpKernelConstruction* ctx) : TransposeOp(ctx) {} + + protected: + Status DoTranspose(OpKernelContext* ctx, const Tensor& in, + gtl::ArraySlice perm, Tensor* out) override; +}; +#endif // TENSORFLOW_USE_SYCL + +// Conjugating transpose ops. +class ConjugateTransposeCpuOp : public TransposeOp { + public: + explicit ConjugateTransposeCpuOp(OpKernelConstruction* ctx) + : TransposeOp(ctx) {} + + protected: + Status DoTranspose(OpKernelContext* ctx, const Tensor& in, + gtl::ArraySlice perm, Tensor* out) override; + bool IsConjugate() const override { return true; } +}; + +#if defined(INTEL_MKL) +class MklConjugateTransposeCpuOp : public TransposeOp { + public: + explicit MklConjugateTransposeCpuOp(OpKernelConstruction* ctx) + : TransposeOp(ctx) {} + + protected: + Status DoTranspose(OpKernelContext* ctx, const Tensor& in, + gtl::ArraySlice perm, Tensor* out) override; + bool IsConjugate() const override { return true; } +}; +#endif // INTEL_MKL + +class ConjugateTransposeGpuOp : public TransposeOp { + public: + explicit ConjugateTransposeGpuOp(OpKernelConstruction* ctx) + : TransposeOp(ctx) {} + + protected: + Status DoTranspose(OpKernelContext* ctx, const Tensor& in, + gtl::ArraySlice perm, Tensor* out) override; + bool IsConjugate() const override { return true; } +}; + +#ifdef TENSORFLOW_USE_SYCL +class ConjugateTransposeSyclOp : public TransposeOp { + public: + explicit ConjugateTransposeSyclOp(OpKernelConstruction* ctx) + : TransposeOp(ctx) {} + + protected: + Status DoTranspose(OpKernelContext* ctx, const Tensor& in, + gtl::ArraySlice perm, Tensor* out) override; + bool IsConjugate() const override { return true; } +}; +#endif // TENSORFLOW_USE_SYCL + +} // namespace tensorflow + +#endif // TENSORFLOW_KERNELS_TRANSPOSE_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/typed_conditional_accumulator_base.h b/linux/include/tensorflow/core/kernels/typed_conditional_accumulator_base.h new file mode 100644 index 000000000..ca341e511 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/typed_conditional_accumulator_base.h @@ -0,0 +1,95 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_TYPED_CONDITIONAL_ACCUMULATOR_BASE_H_ +#define TENSORFLOW_CORE_KERNELS_TYPED_CONDITIONAL_ACCUMULATOR_BASE_H_ + +#include "tensorflow/core/kernels/conditional_accumulator_base.h" + +namespace tensorflow { + +/* + * TypedConditionalAccumulatorBase is a templated companion of + * ConditionalAccumulatorBase which allows for subclasses to use different + * types for the input gradients. (See ConditionalAccumulator and + * SparseConditionalAccumulator.) + * + * TypedConditionalAccumulatorBase defines virtual methods and implements + * methods which depend on the gradient type. These are mainly methods that are + * used for adding a new gradient to the accumulator. + */ +template +class TypedConditionalAccumulatorBase : public ConditionalAccumulatorBase { + public: + TypedConditionalAccumulatorBase(const DataType& dtype, + const PartialTensorShape& shape, + const string& name, + const string& reduction_type) + : ConditionalAccumulatorBase(dtype, shape, name, reduction_type) {} + + /** + * Attempts to add a gradient to the accumulator. An ApplyGrad attempt is + * successful (i.e., has its gradient applied) if its local_step >= + * current_global_step_ at the time the attempt is processed. Otherwise, if + * local_step < current_global_step_, the stale gradient is silently dropped. + * + * local_step: Time-step at which the gradient was computed. + * grad: Gradient tensor to be added to the accumulator. + * ctx: Context in which the op is executed. + */ + void TryApplyGrad(int64 local_step, OpKernelContext* ctx) override { + { + mutex_lock l(mu_); + if (local_step >= current_global_step_) { + GradientTensorType* grad = nullptr; + bool is_valid = GetAndValidateTensorInputForApplyGrad(ctx, &grad); + if (is_valid) { + if (counter_ > 0) { + AddToAccumGradFunction(ctx, grad); + } else { + AllocateAndAssignToAccumGradFunction(ctx, grad); + } + counter_++; + } + CleanUpGradTensor(grad); + } + } + FlushUnlocked(); + } + + protected: + // Virtual methods to be implemented by sub-classes for different datatypes. + // Implements arithmetic operations specific to datatype. + virtual void AllocateAndAssignToAccumGradFunction( + OpKernelContext* ctx, GradientTensorType* grad) = 0; + + virtual void AddToAccumGradFunction(OpKernelContext* ctx, + GradientTensorType* grad) = 0; + + // Method for extracting and validating input provided in an OpKernelContext. + // Returns true if input was successfully retrieved and is valid. + // Gradient is returned via the GradientTensorType** tensor. + virtual bool GetAndValidateTensorInputForApplyGrad( + OpKernelContext* ctx, GradientTensorType** tensor) + EXCLUSIVE_LOCKS_REQUIRED(mu_) = 0; + + // Method for cleaning up any memory allocated in + // GetAndValidateTensorInputForApplyGrad + virtual void CleanUpGradTensor(GradientTensorType* tensor) = 0; +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_TYPED_CONDITIONAL_ACCUMULATOR_BASE_H_ diff --git a/linux/include/tensorflow/core/kernels/typed_queue.h b/linux/include/tensorflow/core/kernels/typed_queue.h new file mode 100644 index 000000000..43dcb4cef --- /dev/null +++ b/linux/include/tensorflow/core/kernels/typed_queue.h @@ -0,0 +1,122 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_TYPED_QUEUE_H_ +#define TENSORFLOW_CORE_KERNELS_TYPED_QUEUE_H_ + +#include +#include +#include + +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/kernels/queue_base.h" +#include "tensorflow/core/platform/mutex.h" + +namespace tensorflow { + +// TypedQueue builds on QueueBase, with backing class (SubQueue) +// known and stored within. Shared methods that need to have access +// to the backed data sit in this class. +template +class TypedQueue : public QueueBase { + public: + TypedQueue(const int32 capacity, const DataTypeVector& component_dtypes, + const std::vector& component_shapes, + const string& name); + + virtual Status Initialize(); // Must be called before any other method. + + int64 MemoryUsed() const override; + + protected: + std::vector queues_ GUARDED_BY(mu_); +}; // class TypedQueue + +template +TypedQueue::TypedQueue( + int32 capacity, const DataTypeVector& component_dtypes, + const std::vector& component_shapes, const string& name) + : QueueBase(capacity, component_dtypes, component_shapes, name) {} + +template +Status TypedQueue::Initialize() { + if (component_dtypes_.empty()) { + return errors::InvalidArgument("Empty component types for queue ", name_); + } + if (!component_shapes_.empty() && + component_dtypes_.size() != component_shapes_.size()) { + return errors::InvalidArgument( + "Different number of component types. ", + "Types: ", DataTypeSliceString(component_dtypes_), + ", Shapes: ", ShapeListString(component_shapes_)); + } + + mutex_lock lock(mu_); + queues_.reserve(num_components()); + for (int i = 0; i < num_components(); ++i) { + queues_.push_back(SubQueue()); + } + return Status::OK(); +} + +namespace { + +template +int64 SizeOf(const SubQueue& sq) { + static_assert(sizeof(SubQueue) != sizeof(SubQueue), "SubQueue size unknown."); + return 0; +} + +template <> +int64 SizeOf(const std::deque& sq) { + if (sq.empty()) { + return 0; + } + return sq.size() * sq.front().AllocatedBytes(); +} + +template <> +int64 SizeOf(const std::vector& sq) { + if (sq.empty()) { + return 0; + } + return sq.size() * sq.front().AllocatedBytes(); +} + +using TensorPair = std::pair; + +template +int64 SizeOf(const std::priority_queue& sq) { + if (sq.empty()) { + return 0; + } + return sq.size() * (sizeof(TensorPair) + sq.top().second.AllocatedBytes()); +} + +} // namespace + +template +int64 TypedQueue::MemoryUsed() const { + int memory_size = 0; + mutex_lock l(mu_); + for (const auto& sq : queues_) { + memory_size += SizeOf(sq); + } + return memory_size; +} + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_TYPED_QUEUE_H_ diff --git a/linux/include/tensorflow/core/kernels/variable_ops.h b/linux/include/tensorflow/core/kernels/variable_ops.h new file mode 100644 index 000000000..4742e429e --- /dev/null +++ b/linux/include/tensorflow/core/kernels/variable_ops.h @@ -0,0 +1,49 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_VARIABLE_OPS_H_ +#define TENSORFLOW_CORE_KERNELS_VARIABLE_OPS_H_ + +#include "tensorflow/core/framework/allocator.h" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/register_types.h" +#include "tensorflow/core/framework/resource_mgr.h" +#include "tensorflow/core/framework/resource_var.h" +#include "tensorflow/core/lib/core/errors.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/mutex.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +class VariableOp : public OpKernel { + public: + explicit VariableOp(OpKernelConstruction* context); + void Compute(OpKernelContext* ctx) override; + + private: + DataType dtype_; + TensorShape shape_; + + mutex init_mu_; + ContainerInfo cinfo_ GUARDED_BY(init_mu_); + bool initialized_ GUARDED_BY(init_mu_){false}; + + TF_DISALLOW_COPY_AND_ASSIGN(VariableOp); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_VARIABLE_OPS_H_ diff --git a/linux/include/tensorflow/core/kernels/where_op.h b/linux/include/tensorflow/core/kernels/where_op.h new file mode 100644 index 000000000..e63b3ba8c --- /dev/null +++ b/linux/include/tensorflow/core/kernels/where_op.h @@ -0,0 +1,66 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_WHERE_OP_H_ +#define TENSORFLOW_CORE_KERNELS_WHERE_OP_H_ + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" +#include "tensorflow/core/framework/op_kernel.h" +#include "tensorflow/core/framework/tensor_types.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +#define TF_CALL_WHERE_GPU_TYPES(m) \ + TF_CALL_int8(m); \ + TF_CALL_uint8(m); \ + TF_CALL_int32(m); \ + TF_CALL_int64(m); \ + TF_CALL_float(m); \ + TF_CALL_double(m); \ + TF_CALL_complex64(m); \ + TF_CALL_complex128(m); \ + TF_CALL_bool(m); + +namespace functor { + +template +struct NumTrue { + EIGEN_ALWAYS_INLINE static Status Compute( + OpKernelContext* ctx, const Device& d, + typename TTypes::ConstFlat input, + typename TTypes::Scalar num_true); +}; + +template +struct Where { + // Copies indices of true values in input into output. The pointer + // found_true should sit on the host. Compute should copy the + // number of true elements found into it. At the end, if + // *found_true != output.dimension(0), + // then the input may have changed between the initial counting of + // the true values and the call to Where. + EIGEN_ALWAYS_INLINE static Status Compute( + OpKernelContext* ctx, const Device& d, + typename TTypes::ConstTensor input, + typename TTypes::Matrix output, TIndex* found_true); +}; + +} // namespace functor + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_WHERE_OP_H_ diff --git a/linux/include/tensorflow/core/kernels/winograd_transform.h b/linux/include/tensorflow/core/kernels/winograd_transform.h new file mode 100644 index 000000000..d22710e50 --- /dev/null +++ b/linux/include/tensorflow/core/kernels/winograd_transform.h @@ -0,0 +1,377 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_WINOGRAD_TRANSFORM_H_ +#define TENSORFLOW_CORE_KERNELS_WINOGRAD_TRANSFORM_H_ + +#include "tensorflow/core/kernels/deep_conv2d.h" + +namespace tensorflow { + +// Winograd DeepConv2DTransform implementation for 3x3 filters. +// Details: +// *) Arithmetic complexity of computations: Shmuel Winograd +// *) Fast Algorithms for Convolutional Neural Networks: Lavin, Gray + +template +class WinogradTransform : public DeepConv2DTransform { + public: + typedef typename DeepConv2DTransform::Shape Shape; + + WinogradTransform() + : filter_shape_(3, 3), input_shape_(4, 4), output_shape_(2, 2) {} + + virtual void GetFilterTransformMatrix(const int64 rows, const int64 cols, + T* transform_matrix) const; + + virtual void GetInputTransformMatrix(const int64 rows, const int64 cols, + T* transform_matrix) const; + + virtual void GetOutputTransformMatrix(const int64 rows, const int64 cols, + T* transform_matrix) const; + + virtual const Shape& filter_shape() const { return filter_shape_; } + virtual const Shape& input_shape() const { return input_shape_; } + virtual const Shape& output_shape() const { return output_shape_; } + + private: + const Shape filter_shape_; + const Shape input_shape_; + const Shape output_shape_; +}; + +// The filter transform matrix is the kronecker product 'M * M' of the +// following matrix 'M': +// +// [ 1 0 0 ] +// [ 1/2 1/2 1/2 ] +// [ 1/2 -1/2 1/2 ] +// [ 0 0 1 ] +// +// The data layout of 'transform_matrix': +// [input_tile_spatial_size, filter_spatial_size] +// +template +void WinogradTransform::GetFilterTransformMatrix(const int64 rows, + const int64 cols, + T* transform_matrix) const { + CHECK_GT(rows, 0); + CHECK_GT(cols, 0); + memset(transform_matrix, 0, sizeof(T) * rows * cols); + + // Sub matrix [0,0] + transform_matrix[0 * cols + 0] = T(1.0); + + transform_matrix[1 * cols + 0] = T(0.5); + transform_matrix[1 * cols + 1] = T(0.5); + transform_matrix[1 * cols + 2] = T(0.5); + + transform_matrix[2 * cols + 0] = T(0.5); + transform_matrix[2 * cols + 1] = T(-0.5); + transform_matrix[2 * cols + 2] = T(0.5); + + transform_matrix[3 * cols + 2] = T(1.0); + + // Sub matrix [1,0] + transform_matrix[4 * cols + 0] = T(0.5); + + transform_matrix[5 * cols + 0] = T(0.25); + transform_matrix[5 * cols + 1] = T(0.25); + transform_matrix[5 * cols + 2] = T(0.25); + + transform_matrix[6 * cols + 0] = T(0.25); + transform_matrix[6 * cols + 1] = T(-0.25); + transform_matrix[6 * cols + 2] = T(0.25); + + transform_matrix[7 * cols + 2] = T(0.5); + + // Sub matrix [1,1] + transform_matrix[4 * cols + 3] = T(0.5); + + transform_matrix[5 * cols + 3] = T(0.25); + transform_matrix[5 * cols + 4] = T(0.25); + transform_matrix[5 * cols + 5] = T(0.25); + + transform_matrix[6 * cols + 3] = T(0.25); + transform_matrix[6 * cols + 4] = T(-0.25); + transform_matrix[6 * cols + 5] = T(0.25); + + transform_matrix[7 * cols + 5] = T(0.5); + + // Sub matrix [1,2] + transform_matrix[4 * cols + 6] = T(0.5); + + transform_matrix[5 * cols + 6] = T(0.25); + transform_matrix[5 * cols + 7] = T(0.25); + transform_matrix[5 * cols + 8] = T(0.25); + + transform_matrix[6 * cols + 6] = T(0.25); + transform_matrix[6 * cols + 7] = T(-0.25); + transform_matrix[6 * cols + 8] = T(0.25); + + transform_matrix[7 * cols + 8] = T(0.5); + + // Sub matrix [2,0] + transform_matrix[8 * cols + 0] = T(0.5); + + transform_matrix[9 * cols + 0] = T(0.25); + transform_matrix[9 * cols + 1] = T(0.25); + transform_matrix[9 * cols + 2] = T(0.25); + + transform_matrix[10 * cols + 0] = T(0.25); + transform_matrix[10 * cols + 1] = T(-0.25); + transform_matrix[10 * cols + 2] = T(0.25); + + transform_matrix[11 * cols + 2] = T(0.5); + + // Sub matrix [2,1] + transform_matrix[8 * cols + 3] = T(-0.5); + + transform_matrix[9 * cols + 3] = T(-0.25); + transform_matrix[9 * cols + 4] = T(-0.25); + transform_matrix[9 * cols + 5] = T(-0.25); + + transform_matrix[10 * cols + 3] = T(-0.25); + transform_matrix[10 * cols + 4] = T(0.25); + transform_matrix[10 * cols + 5] = T(-0.25); + + transform_matrix[11 * cols + 5] = T(-0.5); + + // Sub matrix [2,2] + transform_matrix[8 * cols + 6] = T(0.5); + + transform_matrix[9 * cols + 6] = T(0.25); + transform_matrix[9 * cols + 7] = T(0.25); + transform_matrix[9 * cols + 8] = T(0.25); + + transform_matrix[10 * cols + 6] = T(0.25); + transform_matrix[10 * cols + 7] = T(-0.25); + transform_matrix[10 * cols + 8] = T(0.25); + + transform_matrix[11 * cols + 8] = T(0.5); + + // Sub matrix [3,2] + transform_matrix[12 * cols + 6] = T(1.0); + + transform_matrix[13 * cols + 6] = T(0.5); + transform_matrix[13 * cols + 7] = T(0.5); + transform_matrix[13 * cols + 8] = T(0.5); + + transform_matrix[14 * cols + 6] = T(0.5); + transform_matrix[14 * cols + 7] = T(-0.5); + transform_matrix[14 * cols + 8] = T(0.5); + + transform_matrix[15 * cols + 8] = T(1.0); +} + +// The input transform matrix is the kronecker product 'M * M' of the +// following matrix 'M': +// +// [1 0 -1 0] +// [0 1 1 0] +// [0 -1 1 0] +// [0 1 0 -1] +// +// Data layout of 'transform_matrix': +// [tile_spatial_size, tile_spatial_size] +// +template +void WinogradTransform::GetInputTransformMatrix(const int64 rows, + const int64 cols, + T* transform_matrix) const { + CHECK_GT(rows, 0); + CHECK_GT(cols, 0); + memset(transform_matrix, 0, sizeof(T) * rows * cols); + + // Sub matrix [0,0] + transform_matrix[0 * cols + 0] = T(1.0); + transform_matrix[0 * cols + 2] = T(-1.0); + + transform_matrix[1 * cols + 1] = T(1.0); + transform_matrix[1 * cols + 2] = T(1.0); + + transform_matrix[2 * cols + 1] = T(-1.0); + transform_matrix[2 * cols + 2] = T(1.0); + + transform_matrix[3 * cols + 1] = T(1.0); + transform_matrix[3 * cols + 3] = T(-1.0); + + // Sub matrix [0,2] + transform_matrix[0 * cols + 8] = T(-1.0); + transform_matrix[0 * cols + 10] = T(1.0); + + transform_matrix[1 * cols + 9] = T(-1.0); + transform_matrix[1 * cols + 10] = T(-1.0); + + transform_matrix[2 * cols + 9] = T(1.0); + transform_matrix[2 * cols + 10] = T(-1.0); + + transform_matrix[3 * cols + 9] = T(-1.0); + transform_matrix[3 * cols + 11] = T(1.0); + + // Sub matrix [1,1] + transform_matrix[4 * cols + 4] = T(1.0); + transform_matrix[4 * cols + 6] = T(-1.0); + + transform_matrix[5 * cols + 5] = T(1.0); + transform_matrix[5 * cols + 6] = T(1.0); + + transform_matrix[6 * cols + 5] = T(-1.0); + transform_matrix[6 * cols + 6] = T(1.0); + + transform_matrix[7 * cols + 5] = T(1.0); + transform_matrix[7 * cols + 7] = T(-1.0); + + // Sub matrix [1,2] + transform_matrix[4 * cols + 8] = T(1.0); + transform_matrix[4 * cols + 10] = T(-1.0); + + transform_matrix[5 * cols + 9] = T(1.0); + transform_matrix[5 * cols + 10] = T(1.0); + + transform_matrix[6 * cols + 9] = T(-1.0); + transform_matrix[6 * cols + 10] = T(1.0); + + transform_matrix[7 * cols + 9] = T(1.0); + transform_matrix[7 * cols + 11] = T(-1.0); + + // Sub matrix [2,1] + transform_matrix[8 * cols + 4] = T(-1.0); + transform_matrix[8 * cols + 6] = T(1.0); + + transform_matrix[9 * cols + 5] = T(-1.0); + transform_matrix[9 * cols + 6] = T(-1.0); + + transform_matrix[10 * cols + 5] = T(1.0); + transform_matrix[10 * cols + 6] = T(-1.0); + + transform_matrix[11 * cols + 5] = T(-1.0); + transform_matrix[11 * cols + 7] = T(1.0); + + // Sub matrix [2,2] + transform_matrix[8 * cols + 8] = T(1.0); + transform_matrix[8 * cols + 10] = T(-1.0); + + transform_matrix[9 * cols + 9] = T(1.0); + transform_matrix[9 * cols + 10] = T(1.0); + + transform_matrix[10 * cols + 9] = T(-1.0); + transform_matrix[10 * cols + 10] = T(1.0); + + transform_matrix[11 * cols + 9] = T(1.0); + transform_matrix[11 * cols + 11] = T(-1.0); + + // Sub matrix [3,1] + transform_matrix[12 * cols + 4] = T(1.0); + transform_matrix[12 * cols + 6] = T(-1.0); + + transform_matrix[13 * cols + 5] = T(1.0); + transform_matrix[13 * cols + 6] = T(1.0); + + transform_matrix[14 * cols + 5] = T(-1.0); + transform_matrix[14 * cols + 6] = T(1.0); + + transform_matrix[15 * cols + 5] = T(1.0); + transform_matrix[15 * cols + 7] = T(-1.0); + + // Sub matrix [3,3] + transform_matrix[12 * cols + 12] = T(-1.0); + transform_matrix[12 * cols + 14] = T(1.0); + + transform_matrix[13 * cols + 13] = T(-1.0); + transform_matrix[13 * cols + 14] = T(-1.0); + + transform_matrix[14 * cols + 13] = T(1.0); + transform_matrix[14 * cols + 14] = T(-1.0); + + transform_matrix[15 * cols + 13] = T(-1.0); + transform_matrix[15 * cols + 15] = T(1.0); +}; + +// The output transform matrix is the kronecker product 'M * M' of the +// following matrix 'M': +// +// [1 1 1 0] +// [0 1 -1 -1] +// +// Data layout of 'transform_matrix': +// [out_tile_spatial_size, tile_spatial_size] +// +template +void WinogradTransform::GetOutputTransformMatrix(const int64 rows, + const int64 cols, + T* transform_matrix) const { + CHECK_GT(rows, 0); + CHECK_GT(cols, 0); + memset(transform_matrix, 0, sizeof(T) * rows * cols); + + // Sub matrix [0,0] + transform_matrix[0 * cols + 0] = T(1.0); + transform_matrix[0 * cols + 1] = T(1.0); + transform_matrix[0 * cols + 2] = T(1.0); + + transform_matrix[1 * cols + 1] = T(1.0); + transform_matrix[1 * cols + 2] = T(-1.0); + transform_matrix[1 * cols + 3] = T(-1.0); + + // Sub matrix [0,1] + transform_matrix[0 * cols + 4] = T(1.0); + transform_matrix[0 * cols + 5] = T(1.0); + transform_matrix[0 * cols + 6] = T(1.0); + + transform_matrix[1 * cols + 5] = T(1.0); + transform_matrix[1 * cols + 6] = T(-1.0); + transform_matrix[1 * cols + 7] = T(-1.0); + + // Sub matrix [0,2] + transform_matrix[0 * cols + 8] = T(1.0); + transform_matrix[0 * cols + 9] = T(1.0); + transform_matrix[0 * cols + 10] = T(1.0); + + transform_matrix[1 * cols + 9] = T(1.0); + transform_matrix[1 * cols + 10] = T(-1.0); + transform_matrix[1 * cols + 11] = T(-1.0); + + // Sub matrix [1,1] + transform_matrix[2 * cols + 4] = T(1.0); + transform_matrix[2 * cols + 5] = T(1.0); + transform_matrix[2 * cols + 6] = T(1.0); + + transform_matrix[3 * cols + 5] = T(1.0); + transform_matrix[3 * cols + 6] = T(-1.0); + transform_matrix[3 * cols + 7] = T(-1.0); + + // Sub matrix [1,2] + transform_matrix[2 * cols + 8] = T(-1.0); + transform_matrix[2 * cols + 9] = T(-1.0); + transform_matrix[2 * cols + 10] = T(-1.0); + + transform_matrix[3 * cols + 9] = T(-1.0); + transform_matrix[3 * cols + 10] = T(1.0); + transform_matrix[3 * cols + 11] = T(1.0); + + // Sub matrix [1,3] + transform_matrix[2 * cols + 12] = T(-1.0); + transform_matrix[2 * cols + 13] = T(-1.0); + transform_matrix[2 * cols + 14] = T(-1.0); + + transform_matrix[3 * cols + 13] = T(-1.0); + transform_matrix[3 * cols + 14] = T(1.0); + transform_matrix[3 * cols + 15] = T(1.0); +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_WINOGRAD_TRANSFORM_H_ diff --git a/linux/include/tensorflow/core/kernels/xent_op.h b/linux/include/tensorflow/core/kernels/xent_op.h new file mode 100644 index 000000000..23d3ad39a --- /dev/null +++ b/linux/include/tensorflow/core/kernels/xent_op.h @@ -0,0 +1,128 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_KERNELS_XENT_OP_H_ +#define TENSORFLOW_CORE_KERNELS_XENT_OP_H_ +// Functor definition for XentOp, must be compilable by nvcc. + +#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor" + +#include "tensorflow/core/framework/tensor_types.h" + +namespace tensorflow { +namespace functor { + +// Functor used by XentOp to do the computations. +template +struct XentFunctor { + // Computes Cross Entropy loss and backprop. + // + // logits: batch_size, num_classes. + // labels: batch_size, num_classes. + // scratch: temporary tensor, dims: batch_size, 1 + // loss: output tensor for the loss, dims: batch_size. + // backprop: output tensor for the backprop, dims: batch_size, num_classes. + void operator()(const Device &d, + const Eigen::DSizes &shape, + const Eigen::array &logits_bcast, + const Eigen::array &labels_bcast, + typename TTypes::ConstMatrix logits, + typename TTypes::ConstMatrix labels, + typename TTypes::Matrix scratch, + typename TTypes::Vec loss, + typename TTypes::Matrix backprop); +}; + +// Eigen code implementing XentFunctor::operator(). +// This code works for both CPU and GPU and is used by the functor +// specializations for both device types. +template +struct XentEigenImpl { + static void Compute(const Device &d, + const Eigen::DSizes &shape, + const Eigen::array &logits_bcast, + const Eigen::array &labels_bcast, + typename TTypes::ConstMatrix logits, + typename TTypes::ConstMatrix labels, + typename TTypes::Matrix scratch, + typename TTypes::Vec loss, + typename TTypes::Matrix backprop) { + // NOTE(touts): This duplicates some of the computations in softmax_op + // because we need the intermediate (logits -max(logits)) values to + // avoid a log(exp()) in the computation of the loss. + + const int kBatchDim = 0; + const int kClassDim = 1; + + const int batch_size = shape[kBatchDim]; + const int num_classes = shape[kClassDim]; + +// These arrays are used to reduce along the class dimension, and broadcast +// the resulting value to all classes. +#if !defined(EIGEN_HAS_INDEX_LIST) + Eigen::array along_class; + along_class[0] = kClassDim; + Eigen::array batch_only; + batch_only[0] = batch_size; + Eigen::array batch_by_one; + batch_by_one[0] = batch_size; + batch_by_one[1] = 1; + Eigen::array one_by_class; + one_by_class[0] = 1; + one_by_class[1] = num_classes; +#else + Eigen::IndexList > along_class; + Eigen::IndexList > batch_by_one; + batch_by_one.set(0, batch_size); + Eigen::IndexList batch_only; + batch_only.set(0, batch_size); + Eigen::IndexList, int> one_by_class; + one_by_class.set(1, num_classes); +#endif + + // max_logits along classes. + scratch.reshape(batch_only).device(d) = + logits.broadcast(logits_bcast).maximum(along_class); + + // logits - max_logits. + backprop.device(d) = + logits.broadcast(logits_bcast) - scratch.broadcast(one_by_class); + + // sum(exp(logits - max_logits)) along classes. + scratch.reshape(batch_only).device(d) = backprop.exp().sum(along_class); + + // NOTE(keveman): Eigen on GPU dispatches to an optimized implementation + // for an expression of the form lhs = rhs.sum(). + // lhs = -rhs.sum() doesn't match the above pattern, so folding in the + // negation before calling sum(). + // sum(-labels * + // ((logits - max_logits) - log(sum(exp(logits - max_logits))))) + // along classes + loss.device(d) = (labels.broadcast(labels_bcast) * + (scratch.log().eval().broadcast(one_by_class) - backprop)) + .eval() + .sum(along_class); + + // backprop: prob - labels, where + // prob = exp(logits - max_logits) / sum(exp(logits - max_logits)) + backprop.device(d) = (backprop.exp() / scratch.broadcast(one_by_class)) - + labels.broadcast(labels_bcast); + } +}; + +} // namespace functor +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_KERNELS_XENT_OP_H_ diff --git a/linux/include/tensorflow/core/lib/bfloat16/bfloat16.h b/linux/include/tensorflow/core/lib/bfloat16/bfloat16.h new file mode 100644 index 000000000..440854658 --- /dev/null +++ b/linux/include/tensorflow/core/lib/bfloat16/bfloat16.h @@ -0,0 +1,513 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_LIB_BFLOAT16_BFLOAT16_H_ +#define TENSORFLOW_CORE_LIB_BFLOAT16_BFLOAT16_H_ + +#include +#include + +#include "tensorflow/core/platform/byte_order.h" + +#ifdef __CUDACC__ +// All functions callable from CUDA code must be qualified with __device__ +#define B16_DEVICE_FUNC __host__ __device__ + +#else +#define B16_DEVICE_FUNC + +#endif + +namespace Eigen { +struct half; +} + +namespace tensorflow { + +// Single precision complex. +typedef std::complex complex64; +// Double precision complex. +typedef std::complex complex128; + +// see framework/bfloat16.h for description. +struct bfloat16 { + // The default constructor must yield a zero value, not an uninitialized + // value; some TF kernels use T() as a zero value. + B16_DEVICE_FUNC bfloat16() : value(ZERO_VALUE) {} + + B16_DEVICE_FUNC static bfloat16 truncate_to_bfloat16(const float v) { + bfloat16 output; + if (float_isnan(v)) { + output.value = NAN_VALUE; + return output; + } + const uint16_t* p = reinterpret_cast(&v); +#if __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__ + output.value = p[0]; +#else + output.value = p[1]; +#endif + return output; + } + + B16_DEVICE_FUNC explicit bfloat16(const float v) { + value = round_to_bfloat16(v).value; + } + + B16_DEVICE_FUNC explicit bfloat16(const double val) + : bfloat16(static_cast(val)) {} + // Following the convention of numpy, converting between complex and + // float will lead to loss of imag value. + B16_DEVICE_FUNC explicit bfloat16(const complex64& val) + : bfloat16(val.real()) {} + + B16_DEVICE_FUNC explicit bfloat16(const complex128& val) + : bfloat16(static_cast(val.real())) {} + + B16_DEVICE_FUNC explicit bfloat16(const unsigned short val) + : bfloat16(static_cast(val)) {} + + B16_DEVICE_FUNC explicit bfloat16(const unsigned int val) + : bfloat16(static_cast(val)) {} + + B16_DEVICE_FUNC explicit bfloat16(const int val) + : bfloat16(static_cast(val)) {} + + B16_DEVICE_FUNC explicit bfloat16(const long val) + : bfloat16(static_cast(val)) {} + + B16_DEVICE_FUNC explicit bfloat16(const long long val) + : bfloat16(static_cast(val)) {} + + template + B16_DEVICE_FUNC explicit bfloat16(const T& val) + : bfloat16(static_cast(val)) {} + + B16_DEVICE_FUNC explicit operator float() const { + float result = 0; + + uint16_t* q = reinterpret_cast(&result); + +#if __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__ + q[0] = value; +#else + q[1] = value; +#endif + return result; + } + + B16_DEVICE_FUNC explicit operator bool() const { + return static_cast(float(*this)); + } + + B16_DEVICE_FUNC explicit operator Eigen::half() const; + + B16_DEVICE_FUNC explicit operator short() const { + return static_cast(float(*this)); + } + + B16_DEVICE_FUNC explicit operator int() const { + return static_cast(float(*this)); + } + + B16_DEVICE_FUNC explicit operator long() const { + return static_cast(float(*this)); + } + + B16_DEVICE_FUNC explicit operator char() const { + return static_cast(float(*this)); + } + + B16_DEVICE_FUNC explicit operator signed char() const { + return static_cast(float(*this)); + } + + B16_DEVICE_FUNC explicit operator unsigned char() const { + return static_cast(float(*this)); + } + + B16_DEVICE_FUNC explicit operator unsigned short() const { + return static_cast(float(*this)); + } + + B16_DEVICE_FUNC explicit operator unsigned int() const { + return static_cast(float(*this)); + } + + B16_DEVICE_FUNC explicit operator unsigned long() const { + return static_cast(float(*this)); + } + + B16_DEVICE_FUNC explicit operator unsigned long long() const { + return static_cast(float(*this)); + } + + B16_DEVICE_FUNC explicit operator long long() const { + return static_cast(float(*this)); + } + + B16_DEVICE_FUNC explicit operator double() const { + return static_cast(float(*this)); + } + + B16_DEVICE_FUNC explicit operator complex64() const { + return complex64(float(*this), float(0.0)); + } + + B16_DEVICE_FUNC explicit operator complex128() const { + return complex128(double(*this), double(0.0)); + } + + union FP32 { + unsigned int u; + float f; + }; + + // Converts a float point to bfloat16, with round-nearest-to-even as rounding + // method. + // TODO: There is a slightly faster implementation (8% faster on CPU) + // than this (documented in cl/175987786), that is exponentially harder to + // understand and document. Switch to the faster version when converting to + // BF16 becomes compute-bound. + B16_DEVICE_FUNC static bfloat16 round_to_bfloat16(float v) { + uint32_t input; + FP32 f; + f.f = v; + input = f.u; + bfloat16 output; + + if (float_isnan(v)) { + // If the value is a NaN, squash it to a qNaN with msb of fraction set, + // this makes sure after truncation we don't end up with an inf. + // + // qNaN magic: All exponent bits set + most significant bit of fraction + // set. + output.value = 0x7fc0; + } else { + // Fast rounding algorithm that rounds a half value to nearest even. This + // reduces expected error when we convert a large number of floats. Here + // is how it works: + // + // Definitions: + // To convert a float 32 to bfloat16, a float 32 can be viewed as 32 bits + // with the following tags: + // + // Sign | Exp (8 bits) | Frac (23 bits) + // S EEEEEEEE FFFFFFLRTTTTTTTTTTTTTTT + // + // S: Sign bit. + // E: Exponent bits. + // F: First 6 bits of fraction. + // L: Least significant bit of resulting bfloat16 if we truncate away the + // rest of the float32. This is also the 7th bit of fraction + // R: Rounding bit, 8th bit of fraction. + // T: Sticky bits, rest of fraction, 15 bits. + // + // To round half to nearest even, there are 3 cases where we want to round + // down (simply truncate the result of the bits away, which consists of + // rounding bit and sticky bits) and two cases where we want to round up + // (truncate then add one to the result). + // + // The fast converting algorithm simply adds lsb (L) to 0x7fff (15 bits of + // 1s) as the rounding bias, adds the rounding bias to the input, then + // truncates the last 16 bits away. + // + // To understand how it works, we can analyze this algorithm case by case: + // + // 1. L = 0, R = 0: + // Expect: round down, this is less than half value. + // + // Algorithm: + // - Rounding bias: 0x7fff + 0 = 0x7fff + // - Adding rounding bias to input may create any carry, depending on + // whether there is any value set to 1 in T bits. + // - R may be set to 1 if there is a carry. + // - L remains 0. + // - Note that this case also handles Inf and -Inf, where all fraction + // bits, including L, R and Ts are all 0. The output remains Inf after + // this algorithm. + // + // 2. L = 1, R = 0: + // Expect: round down, this is less than half value. + // + // Algorithm: + // - Rounding bias: 0x7fff + 1 = 0x8000 + // - Adding rounding bias to input doesn't change sticky bits but + // adds 1 to rounding bit. + // - L remains 1. + // + // 3. L = 0, R = 1, all of T are 0: + // Expect: round down, this is exactly at half, the result is already + // even (L=0). + // + // Algorithm: + // - Rounding bias: 0x7fff + 0 = 0x7fff + // - Adding rounding bias to input sets all sticky bits to 1, but + // doesn't create a carry. + // - R remains 1. + // - L remains 0. + // + // 4. L = 1, R = 1: + // Expect: round up, this is exactly at half, the result needs to be + // round to the next even number. + // + // Algorithm: + // - Rounding bias: 0x7fff + 1 = 0x8000 + // - Adding rounding bias to input doesn't change sticky bits, but + // creates a carry from rounding bit. + // - The carry sets L to 0, creates another carry bit and propagate + // forward to F bits. + // - If all the F bits are 1, a carry then propagates to the exponent + // bits, which then creates the minimum value with the next exponent + // value. Note that we won't have the case where exponents are all 1, + // since that's either a NaN (handled in the other if condition) or inf + // (handled in case 1). + // + // 5. L = 0, R = 1, any of T is 1: + // Expect: round up, this is greater than half. + // + // Algorithm: + // - Rounding bias: 0x7fff + 0 = 0x7fff + // - Adding rounding bias to input creates a carry from sticky bits, + // sets rounding bit to 0, then create another carry. + // - The second carry sets L to 1. + // + // Examples: + // + // Exact half value that is already even: + // Input: + // Sign | Exp (8 bit) | Frac (first 7 bit) | Frac (last 16 bit) + // S E E E E E E E E F F F F F F L RTTTTTTTTTTTTTTT + // 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1000000000000000 + // + // This falls into case 3. We truncate the rest of 16 bits and no + // carry is created into F and L: + // + // Output: + // Sign | Exp (8 bit) | Frac (first 7 bit) + // S E E E E E E E E F F F F F F L + // 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 + // + // Exact half value, round to next even number: + // Input: + // Sign | Exp (8 bit) | Frac (first 7 bit) | Frac (last 16 bit) + // S E E E E E E E E F F F F F F L RTTTTTTTTTTTTTTT + // 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1000000000000000 + // + // This falls into case 4. We create a carry from R and T, + // which then propagates into L and F: + // + // Output: + // Sign | Exp (8 bit) | Frac (first 7 bit) + // S E E E E E E E E F F F F F F L + // 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 + // + // + // Max denormal value round to min normal value: + // Input: + // Sign | Exp (8 bit) | Frac (first 7 bit) | Frac (last 16 bit) + // S E E E E E E E E F F F F F F L RTTTTTTTTTTTTTTT + // 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1111111111111111 + // + // This falls into case 4. We create a carry from R and T, + // propagate into L and F, which then propagates into exponent + // bits: + // + // Output: + // Sign | Exp (8 bit) | Frac (first 7 bit) + // S E E E E E E E E F F F F F F L + // 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 + // + // Max normal value round to Inf: + // Input: + // Sign | Exp (8 bit) | Frac (first 7 bit) | Frac (last 16 bit) + // S E E E E E E E E F F F F F F L RTTTTTTTTTTTTTTT + // 0 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1111111111111111 + // + // This falls into case 4. We create a carry from R and T, + // propagate into L and F, which then propagates into exponent + // bits: + // + // Sign | Exp (8 bit) | Frac (first 7 bit) + // S E E E E E E E E F F F F F F L + // 0 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 + // + // + // Least significant bit of resulting bfloat. + uint32_t lsb = (input >> 16) & 1; + uint32_t rounding_bias = 0x7fff + lsb; + input += rounding_bias; + output.value = static_cast(input >> 16); + } + return output; + } + + static bfloat16 epsilon() { + bfloat16 x; + x.value = 0x3c00; // 0x1.0p-7 + return x; + } + + static bfloat16 highest() { + bfloat16 x; + x.value = 0x7F7F; // 0x1.FEp127 + return x; + } + + static bfloat16 lowest() { + bfloat16 x; + x.value = 0xFF7F; // -0x1.FEp127 + return x; + } + + uint16_t value; + + // A value that represents "not a number". + static const uint16_t NAN_VALUE = 0x7FC0; + + private: + // A value that represents "zero". + static const uint16_t ZERO_VALUE = 0; + + B16_DEVICE_FUNC static bool float_isnan(const float& x) { +#ifdef __CUDA_ARCH__ + return ::isnan(x); +#else + return std::isnan(x); +#endif + } +}; + +B16_DEVICE_FUNC inline std::ostream& operator<<(std::ostream& os, + const bfloat16& dt) { + os << static_cast(dt); + return os; +} + +B16_DEVICE_FUNC inline bfloat16 operator+(bfloat16 a, bfloat16 b) { + return bfloat16(static_cast(a) + static_cast(b)); +} +B16_DEVICE_FUNC inline bfloat16 operator+(bfloat16 a, int b) { + return bfloat16(static_cast(a) + static_cast(b)); +} +B16_DEVICE_FUNC inline bfloat16 operator+(int a, bfloat16 b) { + return bfloat16(static_cast(a) + static_cast(b)); +} +B16_DEVICE_FUNC inline bfloat16 operator-(bfloat16 a, bfloat16 b) { + return bfloat16(static_cast(a) - static_cast(b)); +} +B16_DEVICE_FUNC inline bfloat16 operator*(bfloat16 a, bfloat16 b) { + return bfloat16(static_cast(a) * static_cast(b)); +} +B16_DEVICE_FUNC inline bfloat16 operator/(bfloat16 a, bfloat16 b) { + return bfloat16(static_cast(a) / static_cast(b)); +} +B16_DEVICE_FUNC inline bfloat16 operator-(bfloat16 a) { + a.value ^= 0x8000; + return a; +} +B16_DEVICE_FUNC inline bool operator<(bfloat16 a, bfloat16 b) { + return static_cast(a) < static_cast(b); +} +B16_DEVICE_FUNC inline bool operator<=(bfloat16 a, bfloat16 b) { + return static_cast(a) <= static_cast(b); +} +B16_DEVICE_FUNC inline bool operator==(bfloat16 a, bfloat16 b) { + return static_cast(a) == static_cast(b); +} +B16_DEVICE_FUNC inline bool operator!=(bfloat16 a, bfloat16 b) { + return static_cast(a) != static_cast(b); +} +B16_DEVICE_FUNC inline bool operator>(bfloat16 a, bfloat16 b) { + return static_cast(a) > static_cast(b); +} +B16_DEVICE_FUNC inline bool operator>=(bfloat16 a, bfloat16 b) { + return static_cast(a) >= static_cast(b); +} +B16_DEVICE_FUNC inline bfloat16& operator+=(bfloat16& a, bfloat16 b) { + a = a + b; + return a; +} +B16_DEVICE_FUNC inline bfloat16& operator-=(bfloat16& a, bfloat16 b) { + a = a - b; + return a; +} +B16_DEVICE_FUNC inline bfloat16 operator++(bfloat16& a) { + a += bfloat16(1); + return a; +} +B16_DEVICE_FUNC inline bfloat16 operator--(bfloat16& a) { + a -= bfloat16(1); + return a; +} +B16_DEVICE_FUNC inline bfloat16 operator++(bfloat16& a, int) { + bfloat16 original_value = a; + ++a; + return original_value; +} +B16_DEVICE_FUNC inline bfloat16 operator--(bfloat16& a, int) { + bfloat16 original_value = a; + --a; + return original_value; +} +B16_DEVICE_FUNC inline bfloat16& operator*=(bfloat16& a, bfloat16 b) { + a = a * b; + return a; +} +B16_DEVICE_FUNC inline bfloat16& operator/=(bfloat16& a, bfloat16 b) { + a = a / b; + return a; +} +} // end namespace tensorflow + +namespace std { +template <> +struct hash { + size_t operator()(const tensorflow::bfloat16& v) const { + return hash()(static_cast(v)); + } +}; + +using tensorflow::bfloat16; +inline bool isinf(const bfloat16& a) { return std::isinf(float(a)); } +inline bool isnan(const bfloat16& a) { return std::isnan(float(a)); } +inline bool isfinite(const bfloat16& a) { return std::isfinite(float(a)); } +inline bfloat16 abs(const bfloat16& a) { return bfloat16(std::abs(float(a))); } +inline bfloat16 exp(const bfloat16& a) { return bfloat16(std::exp(float(a))); } +inline bfloat16 log(const bfloat16& a) { return bfloat16(std::log(float(a))); } +inline bfloat16 log10(const bfloat16& a) { + return bfloat16(std::log10(float(a))); +} +inline bfloat16 sqrt(const bfloat16& a) { + return bfloat16(std::sqrt(float(a))); +} +inline bfloat16 pow(const bfloat16& a, const bfloat16& b) { + return bfloat16(std::pow(float(a), float(b))); +} +inline bfloat16 sin(const bfloat16& a) { return bfloat16(std::sin(float(a))); } +inline bfloat16 cos(const bfloat16& a) { return bfloat16(std::cos(float(a))); } +inline bfloat16 tan(const bfloat16& a) { return bfloat16(std::tan(float(a))); } +inline bfloat16 tanh(const bfloat16& a) { + return bfloat16(std::tanh(float(a))); +} +inline bfloat16 floor(const bfloat16& a) { + return bfloat16(std::floor(float(a))); +} +inline bfloat16 ceil(const bfloat16& a) { + return bfloat16(std::ceil(float(a))); +} +} // namespace std + +#endif // TENSORFLOW_CORE_LIB_BFLOAT16_BFLOAT16_H_ diff --git a/linux/include/tensorflow/core/lib/core/arena.h b/linux/include/tensorflow/core/lib/core/arena.h new file mode 100644 index 000000000..624ee7702 --- /dev/null +++ b/linux/include/tensorflow/core/lib/core/arena.h @@ -0,0 +1,110 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// TODO(vrv): Switch this to an open-sourced version of Arena. + +#ifndef TENSORFLOW_CORE_LIB_CORE_ARENA_H_ +#define TENSORFLOW_CORE_LIB_CORE_ARENA_H_ + +#include + +#include + +#include "tensorflow/core/platform/logging.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { +namespace core { + +// This class is "thread-compatible": different threads can access the +// arena at the same time without locking, as long as they use only +// const methods. +class Arena { + public: + // Allocates a thread-compatible arena with the specified block size. + explicit Arena(const size_t block_size); + ~Arena(); + + char* Alloc(const size_t size) { + return reinterpret_cast(GetMemory(size, 1)); + } + + char* AllocAligned(const size_t size, const size_t alignment) { + return reinterpret_cast(GetMemory(size, alignment)); + } + + void Reset(); + +// This should be the worst-case alignment for any type. This is +// good for IA-32, SPARC version 7 (the last one I know), and +// supposedly Alpha. i386 would be more time-efficient with a +// default alignment of 8, but ::operator new() uses alignment of 4, +// and an assertion will fail below after the call to MakeNewBlock() +// if you try to use a larger alignment. +#ifdef __i386__ + static const int kDefaultAlignment = 4; +#else + static const int kDefaultAlignment = 8; +#endif + + protected: + bool SatisfyAlignment(const size_t alignment); + void MakeNewBlock(const uint32 alignment); + void* GetMemoryFallback(const size_t size, const int align); + void* GetMemory(const size_t size, const int align) { + assert(remaining_ <= block_size_); // an invariant + if (size > 0 && size < remaining_ && align == 1) { // common case + void* result = freestart_; + freestart_ += size; + remaining_ -= size; + return result; + } + return GetMemoryFallback(size, align); + } + + size_t remaining_; + + private: + struct AllocatedBlock { + char* mem; + size_t size; + }; + + // Allocate new block of at least block_size, with the specified + // alignment. + // The returned AllocatedBlock* is valid until the next call to AllocNewBlock + // or Reset (i.e. anything that might affect overflow_blocks_). + AllocatedBlock* AllocNewBlock(const size_t block_size, + const uint32 alignment); + + const size_t block_size_; + char* freestart_; // beginning of the free space in most recent block + char* freestart_when_empty_; // beginning of the free space when we're empty + // STL vector isn't as efficient as it could be, so we use an array at first + size_t blocks_alloced_; // how many of the first_blocks_ have been alloced + AllocatedBlock first_blocks_[16]; // the length of this array is arbitrary + // if the first_blocks_ aren't enough, expand into overflow_blocks_. + std::vector* overflow_blocks_; + + void FreeBlocks(); // Frees all except first block + + TF_DISALLOW_COPY_AND_ASSIGN(Arena); +}; + +} // namespace core +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_LIB_CORE_ARENA_H_ diff --git a/linux/include/tensorflow/core/lib/core/bitmap.h b/linux/include/tensorflow/core/lib/core/bitmap.h new file mode 100644 index 000000000..8ff1e666b --- /dev/null +++ b/linux/include/tensorflow/core/lib/core/bitmap.h @@ -0,0 +1,106 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_LIB_CORE_BITMAP_H_ +#define TENSORFLOW_CORE_LIB_CORE_BITMAP_H_ + +#include "tensorflow/core/platform/logging.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { +namespace core { + +class Bitmap { + public: + // Create a bitmap that holds 0 bits. + Bitmap(); + + // Create a bitmap that holds n bits, all initially zero. + explicit Bitmap(size_t n); + + ~Bitmap(); + + Bitmap(const Bitmap&) = delete; + Bitmap& operator=(const Bitmap&) = delete; + + // Return the number of bits that the bitmap can hold. + size_t bits() const; + + // Replace contents of *this with a bitmap of n bits, all set to zero. + void Reset(size_t n); + + // Return the contents of the ith bit. + // REQUIRES: i < bits() + bool get(size_t i) const; + + // Set the contents of the ith bit to true. + // REQUIRES: i < bits() + void set(size_t i); + + // Set the contents of the ith bit to false. + // REQUIRES: i < bits() + void clear(size_t i); + + // Return the smallest i such that i >= start and !get(i). + // Returns bits() if no such i exists. + size_t FirstUnset(size_t start) const; + + // Returns the bitmap as an ascii string of '0' and '1' characters, bits() + // characters in length. + string ToString() const; + + private: + typedef uint32 Word; + static const size_t kBits = 32; + + // Return the number of words needed to store n bits. + static size_t NumWords(size_t n) { return (n + kBits - 1) / kBits; } + + // Return the mask to use for the ith bit in a word. + static Word Mask(size_t i) { return 1ull << i; } + + size_t nbits_; // Length of bitmap in bits. + Word* word_; +}; + +// Implementation details follow. Clients should ignore. + +inline Bitmap::Bitmap() : nbits_(0), word_(nullptr) {} + +inline Bitmap::Bitmap(size_t n) : Bitmap() { Reset(n); } + +inline Bitmap::~Bitmap() { delete[] word_; } + +inline size_t Bitmap::bits() const { return nbits_; } + +inline bool Bitmap::get(size_t i) const { + DCHECK_LT(i, nbits_); + return word_[i / kBits] & Mask(i % kBits); +} + +inline void Bitmap::set(size_t i) { + DCHECK_LT(i, nbits_); + word_[i / kBits] |= Mask(i % kBits); +} + +inline void Bitmap::clear(size_t i) { + DCHECK_LT(i, nbits_); + word_[i / kBits] &= ~Mask(i % kBits); +} + +} // namespace core +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_LIB_CORE_BITMAP_H_ diff --git a/linux/include/tensorflow/core/lib/core/bits.h b/linux/include/tensorflow/core/lib/core/bits.h new file mode 100644 index 000000000..86e539a26 --- /dev/null +++ b/linux/include/tensorflow/core/lib/core/bits.h @@ -0,0 +1,109 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_LIB_CORE_BITS_H_ +#define TENSORFLOW_CORE_LIB_CORE_BITS_H_ + +#include "tensorflow/core/platform/logging.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +// Return floor(log2(n)) for positive integer n. Returns -1 iff n == 0. +int Log2Floor(uint32 n); +int Log2Floor64(uint64 n); + +// Return ceiling(log2(n)) for positive integer n. Returns -1 iff n == 0. +int Log2Ceiling(uint32 n); +int Log2Ceiling64(uint64 n); + +// ------------------------------------------------------------------------ +// Implementation details follow +// ------------------------------------------------------------------------ + +#if defined(__GNUC__) + +// Return floor(log2(n)) for positive integer n. Returns -1 iff n == 0. +inline int Log2Floor(uint32 n) { return n == 0 ? -1 : 31 ^ __builtin_clz(n); } + +// Return floor(log2(n)) for positive integer n. Returns -1 iff n == 0. +inline int Log2Floor64(uint64 n) { + return n == 0 ? -1 : 63 ^ __builtin_clzll(n); +} + +#else + +// Return floor(log2(n)) for positive integer n. Returns -1 iff n == 0. +inline int Log2Floor(uint32 n) { + if (n == 0) return -1; + int log = 0; + uint32 value = n; + for (int i = 4; i >= 0; --i) { + int shift = (1 << i); + uint32 x = value >> shift; + if (x != 0) { + value = x; + log += shift; + } + } + assert(value == 1); + return log; +} + +// Return floor(log2(n)) for positive integer n. Returns -1 iff n == 0. +// Log2Floor64() is defined in terms of Log2Floor32() +inline int Log2Floor64(uint64 n) { + const uint32 topbits = static_cast(n >> 32); + if (topbits == 0) { + // Top bits are zero, so scan in bottom bits + return Log2Floor(static_cast(n)); + } else { + return 32 + Log2Floor(topbits); + } +} + +#endif + +inline int Log2Ceiling(uint32 n) { + int floor = Log2Floor(n); + if (n == (n & ~(n - 1))) // zero or a power of two + return floor; + else + return floor + 1; +} + +inline int Log2Ceiling64(uint64 n) { + int floor = Log2Floor64(n); + if (n == (n & ~(n - 1))) // zero or a power of two + return floor; + else + return floor + 1; +} + +inline uint32 NextPowerOfTwo(uint32 value) { + int exponent = Log2Ceiling(value); + DCHECK_LT(exponent, std::numeric_limits::digits); + return 1 << exponent; +} + +inline uint64 NextPowerOfTwo64(uint64 value) { + int exponent = Log2Ceiling(value); + DCHECK_LT(exponent, std::numeric_limits::digits); + return 1LL << exponent; +} + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_LIB_CORE_BITS_H_ diff --git a/linux/include/tensorflow/core/lib/core/blocking_counter.h b/linux/include/tensorflow/core/lib/core/blocking_counter.h new file mode 100644 index 000000000..5dab07dbe --- /dev/null +++ b/linux/include/tensorflow/core/lib/core/blocking_counter.h @@ -0,0 +1,80 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_LIB_CORE_BLOCKING_COUNTER_H_ +#define TENSORFLOW_LIB_CORE_BLOCKING_COUNTER_H_ + +#include + +#include "tensorflow/core/platform/logging.h" +#include "tensorflow/core/platform/mutex.h" + +namespace tensorflow { + +class BlockingCounter { + public: + BlockingCounter(int initial_count) + : state_(initial_count << 1), notified_(false) { + CHECK_GE(initial_count, 0); + DCHECK_EQ((initial_count << 1) >> 1, initial_count); + } + + ~BlockingCounter() {} + + inline void DecrementCount() { + unsigned int v = state_.fetch_sub(2, std::memory_order_acq_rel) - 2; + if (v != 1) { + DCHECK_NE(((v + 2) & ~1), 0); + return; // either count has not dropped to 0, or waiter is not waiting + } + mutex_lock l(mu_); + DCHECK(!notified_); + notified_ = true; + cond_var_.notify_all(); + } + + inline void Wait() { + unsigned int v = state_.fetch_or(1, std::memory_order_acq_rel); + if ((v >> 1) == 0) return; + mutex_lock l(mu_); + while (!notified_) { + cond_var_.wait(l); + } + } + // Wait for the specified time, return false iff the count has not dropped to + // zero before the timeout expired. + inline bool WaitFor(std::chrono::milliseconds ms) { + unsigned int v = state_.fetch_or(1, std::memory_order_acq_rel); + if ((v >> 1) == 0) return true; + mutex_lock l(mu_); + while (!notified_) { + const std::cv_status status = cond_var_.wait_for(l, ms); + if (status == std::cv_status::timeout) { + return false; + } + } + return true; + } + + private: + mutex mu_; + condition_variable cond_var_; + std::atomic state_; // low bit is waiter flag + bool notified_; +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_LIB_CORE_BLOCKING_COUNTER_H_ diff --git a/linux/include/tensorflow/core/lib/core/casts.h b/linux/include/tensorflow/core/lib/core/casts.h new file mode 100644 index 000000000..7546d4edc --- /dev/null +++ b/linux/include/tensorflow/core/lib/core/casts.h @@ -0,0 +1,100 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// Various Google-specific casting templates. +// +// This code is compiled directly on many platforms, including client +// platforms like Windows, Mac, and embedded systems. Before making +// any changes here, make sure that you're not breaking any platforms. +// + +#ifndef TENSORFLOW_CORE_LIB_CORE_CASTS_H_ +#define TENSORFLOW_CORE_LIB_CORE_CASTS_H_ + +#include // for memcpy + +namespace tensorflow { + +// bit_cast is a template function that implements the +// equivalent of "*reinterpret_cast(&source)". We need this in +// very low-level functions like the protobuf library and fast math +// support. +// +// float f = 3.14159265358979; +// int i = bit_cast(f); +// // i = 0x40490fdb +// +// The classical address-casting method is: +// +// // WRONG +// float f = 3.14159265358979; // WRONG +// int i = * reinterpret_cast(&f); // WRONG +// +// The address-casting method actually produces undefined behavior +// according to ISO C++ specification section 3.10 -15 -. Roughly, this +// section says: if an object in memory has one type, and a program +// accesses it with a different type, then the result is undefined +// behavior for most values of "different type". +// +// This is true for any cast syntax, either *(int*)&f or +// *reinterpret_cast(&f). And it is particularly true for +// conversions between integral lvalues and floating-point lvalues. +// +// The purpose of 3.10 -15- is to allow optimizing compilers to assume +// that expressions with different types refer to different memory. gcc +// 4.0.1 has an optimizer that takes advantage of this. So a +// non-conforming program quietly produces wildly incorrect output. +// +// The problem is not the use of reinterpret_cast. The problem is type +// punning: holding an object in memory of one type and reading its bits +// back using a different type. +// +// The C++ standard is more subtle and complex than this, but that +// is the basic idea. +// +// Anyways ... +// +// bit_cast<> calls memcpy() which is blessed by the standard, +// especially by the example in section 3.9 . Also, of course, +// bit_cast<> wraps up the nasty logic in one place. +// +// Fortunately memcpy() is very fast. In optimized mode, with a +// constant size, gcc 2.95.3, gcc 4.0.1, and msvc 7.1 produce inline +// code with the minimal amount of data movement. On a 32-bit system, +// memcpy(d,s,4) compiles to one load and one store, and memcpy(d,s,8) +// compiles to two loads and two stores. +// +// I tested this code with gcc 2.95.3, gcc 4.0.1, icc 8.1, and msvc 7.1. +// +// WARNING: if Dest or Source is a non-POD type, the result of the memcpy +// is likely to surprise you. +// +// Props to Bill Gibbons for the compile time assertion technique and +// Art Komninos and Igor Tandetnik for the msvc experiments. +// +// -- mec 2005-10-17 + +template +inline Dest bit_cast(const Source& source) { + static_assert(sizeof(Dest) == sizeof(Source), "Sizes do not match"); + + Dest dest; + memcpy(&dest, &source, sizeof(dest)); + return dest; +} + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_LIB_CORE_CASTS_H_ diff --git a/linux/include/tensorflow/core/lib/core/coding.h b/linux/include/tensorflow/core/lib/core/coding.h new file mode 100644 index 000000000..4a70ffa61 --- /dev/null +++ b/linux/include/tensorflow/core/lib/core/coding.h @@ -0,0 +1,79 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// Endian-neutral encoding: +// * Fixed-length numbers are encoded with least-significant byte first +// * In addition we support variable length "varint" encoding +// * Strings are encoded prefixed by their length in varint format + +#ifndef TENSORFLOW_CORE_LIB_CORE_CODING_H_ +#define TENSORFLOW_CORE_LIB_CORE_CODING_H_ + +#include "tensorflow/core/lib/core/raw_coding.h" +#include "tensorflow/core/lib/core/stringpiece.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { +namespace core { + +// Maximum number of bytes occupied by a varint32. +static const int kMaxVarint32Bytes = 5; + +// Maximum number of bytes occupied by a varint64. +static const int kMaxVarint64Bytes = 10; + +// Lower-level versions of Put... that write directly into a character buffer +// REQUIRES: dst has enough space for the value being written +extern void EncodeFixed16(char* dst, uint16 value); +extern void EncodeFixed32(char* dst, uint32 value); +extern void EncodeFixed64(char* dst, uint64 value); +extern void PutFixed16(string* dst, uint16 value); +extern void PutFixed32(string* dst, uint32 value); +extern void PutFixed64(string* dst, uint64 value); + +extern void PutVarint32(string* dst, uint32 value); +extern void PutVarint64(string* dst, uint64 value); + +extern bool GetVarint32(StringPiece* input, uint32* value); +extern bool GetVarint64(StringPiece* input, uint64* value); + +extern const char* GetVarint32Ptr(const char* p, const char* limit, uint32* v); +extern const char* GetVarint64Ptr(const char* p, const char* limit, uint64* v); + +// Internal routine for use by fallback path of GetVarint32Ptr +extern const char* GetVarint32PtrFallback(const char* p, const char* limit, + uint32* value); +inline const char* GetVarint32Ptr(const char* p, const char* limit, + uint32* value) { + if (p < limit) { + uint32 result = *(reinterpret_cast(p)); + if ((result & 128) == 0) { + *value = result; + return p + 1; + } + } + return GetVarint32PtrFallback(p, limit, value); +} + +extern char* EncodeVarint32(char* dst, uint32 v); +extern char* EncodeVarint64(char* dst, uint64 v); + +// Returns the length of the varint32 or varint64 encoding of "v" +extern int VarintLength(uint64_t v); + +} // namespace core +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_LIB_CORE_CODING_H_ diff --git a/linux/include/tensorflow/core/lib/core/error_codes.pb.h b/linux/include/tensorflow/core/lib/core/error_codes.pb.h new file mode 100644 index 000000000..2200af308 --- /dev/null +++ b/linux/include/tensorflow/core/lib/core/error_codes.pb.h @@ -0,0 +1,126 @@ +// Generated by the protocol buffer compiler. DO NOT EDIT! +// source: tensorflow/core/lib/core/error_codes.proto + +#ifndef PROTOBUF_INCLUDED_tensorflow_2fcore_2flib_2fcore_2ferror_5fcodes_2eproto +#define PROTOBUF_INCLUDED_tensorflow_2fcore_2flib_2fcore_2ferror_5fcodes_2eproto + +#include + +#include + +#if GOOGLE_PROTOBUF_VERSION < 3006000 +#error This file was generated by a newer version of protoc which is +#error incompatible with your Protocol Buffer headers. Please update +#error your headers. +#endif +#if 3006000 < GOOGLE_PROTOBUF_MIN_PROTOC_VERSION +#error This file was generated by an older version of protoc which is +#error incompatible with your Protocol Buffer headers. Please +#error regenerate this file with a newer version of protoc. +#endif + +#include +#include +#include +#include +#include +#include +#include +#include // IWYU pragma: export +#include // IWYU pragma: export +#include +// @@protoc_insertion_point(includes) +#define PROTOBUF_INTERNAL_EXPORT_protobuf_tensorflow_2fcore_2flib_2fcore_2ferror_5fcodes_2eproto + +namespace protobuf_tensorflow_2fcore_2flib_2fcore_2ferror_5fcodes_2eproto { +// Internal implementation detail -- do not use these members. +struct TableStruct { + static const ::google::protobuf::internal::ParseTableField entries[]; + static const ::google::protobuf::internal::AuxillaryParseTableField aux[]; + static const ::google::protobuf::internal::ParseTable schema[1]; + static const ::google::protobuf::internal::FieldMetadata field_metadata[]; + static const ::google::protobuf::internal::SerializationTable serialization_table[]; + static const ::google::protobuf::uint32 offsets[]; +}; +void AddDescriptors(); +} // namespace protobuf_tensorflow_2fcore_2flib_2fcore_2ferror_5fcodes_2eproto +namespace tensorflow { +namespace error { +} // namespace error +} // namespace tensorflow +namespace tensorflow { +namespace error { + +enum Code { + OK = 0, + CANCELLED = 1, + UNKNOWN = 2, + INVALID_ARGUMENT = 3, + DEADLINE_EXCEEDED = 4, + NOT_FOUND = 5, + ALREADY_EXISTS = 6, + PERMISSION_DENIED = 7, + UNAUTHENTICATED = 16, + RESOURCE_EXHAUSTED = 8, + FAILED_PRECONDITION = 9, + ABORTED = 10, + OUT_OF_RANGE = 11, + UNIMPLEMENTED = 12, + INTERNAL = 13, + UNAVAILABLE = 14, + DATA_LOSS = 15, + DO_NOT_USE_RESERVED_FOR_FUTURE_EXPANSION_USE_DEFAULT_IN_SWITCH_INSTEAD_ = 20, + Code_INT_MIN_SENTINEL_DO_NOT_USE_ = ::google::protobuf::kint32min, + Code_INT_MAX_SENTINEL_DO_NOT_USE_ = ::google::protobuf::kint32max +}; +bool Code_IsValid(int value); +const Code Code_MIN = OK; +const Code Code_MAX = DO_NOT_USE_RESERVED_FOR_FUTURE_EXPANSION_USE_DEFAULT_IN_SWITCH_INSTEAD_; +const int Code_ARRAYSIZE = Code_MAX + 1; + +const ::google::protobuf::EnumDescriptor* Code_descriptor(); +inline const ::std::string& Code_Name(Code value) { + return ::google::protobuf::internal::NameOfEnum( + Code_descriptor(), value); +} +inline bool Code_Parse( + const ::std::string& name, Code* value) { + return ::google::protobuf::internal::ParseNamedEnum( + Code_descriptor(), name, value); +} +// =================================================================== + + +// =================================================================== + + +// =================================================================== + +#ifdef __GNUC__ + #pragma GCC diagnostic push + #pragma GCC diagnostic ignored "-Wstrict-aliasing" +#endif // __GNUC__ +#ifdef __GNUC__ + #pragma GCC diagnostic pop +#endif // __GNUC__ + +// @@protoc_insertion_point(namespace_scope) + +} // namespace error +} // namespace tensorflow + +namespace google { +namespace protobuf { + +template <> struct is_proto_enum< ::tensorflow::error::Code> : ::std::true_type {}; +template <> +inline const EnumDescriptor* GetEnumDescriptor< ::tensorflow::error::Code>() { + return ::tensorflow::error::Code_descriptor(); +} + +} // namespace protobuf +} // namespace google + +// @@protoc_insertion_point(global_scope) + +#endif // PROTOBUF_INCLUDED_tensorflow_2fcore_2flib_2fcore_2ferror_5fcodes_2eproto diff --git a/linux/include/tensorflow/core/lib/core/error_codes.pb_text-impl.h b/linux/include/tensorflow/core/lib/core/error_codes.pb_text-impl.h new file mode 100644 index 000000000..ceb500b84 --- /dev/null +++ b/linux/include/tensorflow/core/lib/core/error_codes.pb_text-impl.h @@ -0,0 +1,20 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_lib_core_error_codes_proto_IMPL_H_ +#define tensorflow_core_lib_core_error_codes_proto_IMPL_H_ + +#include "tensorflow/core/lib/core/error_codes.pb.h" +#include "tensorflow/core/lib/core/error_codes.pb_text.h" +#include "tensorflow/core/lib/strings/proto_text_util.h" +#include "tensorflow/core/lib/strings/scanner.h" + +namespace tensorflow { +namespace error { + +namespace internal { + +} // namespace internal + +} // namespace error +} // namespace tensorflow + +#endif // tensorflow_core_lib_core_error_codes_proto_IMPL_H_ diff --git a/linux/include/tensorflow/core/lib/core/error_codes.pb_text.h b/linux/include/tensorflow/core/lib/core/error_codes.pb_text.h new file mode 100644 index 000000000..db9471f95 --- /dev/null +++ b/linux/include/tensorflow/core/lib/core/error_codes.pb_text.h @@ -0,0 +1,20 @@ +// GENERATED FILE - DO NOT MODIFY +#ifndef tensorflow_core_lib_core_error_codes_proto_H_ +#define tensorflow_core_lib_core_error_codes_proto_H_ + +#include "tensorflow/core/lib/core/error_codes.pb.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/protobuf.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { +namespace error { + +// Enum text output for tensorflow.error.Code +const char* EnumName_Code( + ::tensorflow::error::Code value); + +} // namespace error +} // namespace tensorflow + +#endif // tensorflow_core_lib_core_error_codes_proto_H_ diff --git a/linux/include/tensorflow/core/lib/core/errors.h b/linux/include/tensorflow/core/lib/core/errors.h new file mode 100644 index 000000000..d5cbe6c61 --- /dev/null +++ b/linux/include/tensorflow/core/lib/core/errors.h @@ -0,0 +1,159 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_LIB_CORE_ERRORS_H_ +#define TENSORFLOW_CORE_LIB_CORE_ERRORS_H_ + +#include + +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/lib/strings/str_util.h" +#include "tensorflow/core/lib/strings/strcat.h" +#include "tensorflow/core/platform/logging.h" +#include "tensorflow/core/platform/macros.h" + +namespace tensorflow { +namespace errors { + +typedef ::tensorflow::error::Code Code; + +namespace internal { + +// The DECLARE_ERROR macro below only supports types that can be converted +// into StrCat's AlphaNum. For the other types we rely on a slower path +// through std::stringstream. To add support of a new type, it is enough to +// make sure there is an operator<<() for it: +// +// std::ostream& operator<<(std::ostream& os, const MyType& foo) { +// os << foo.ToString(); +// return os; +// } +// Eventually absl::strings will have native support for this and we will be +// able to completely remove PrepareForStrCat(). +template +typename std::enable_if::value, + string>::type +PrepareForStrCat(const T& t) { + std::stringstream ss; + ss << t; + return ss.str(); +} +inline const strings::AlphaNum& PrepareForStrCat(const strings::AlphaNum& a) { + return a; +} + +} // namespace internal + +// Append some context to an error message. Each time we append +// context put it on a new line, since it is possible for there +// to be several layers of additional context. +template +void AppendToMessage(::tensorflow::Status* status, Args... args) { + *status = ::tensorflow::Status( + status->code(), + ::tensorflow::strings::StrCat(status->error_message(), "\n\t", args...)); +} + +// For propagating errors when calling a function. +#define TF_RETURN_IF_ERROR(...) \ + do { \ + const ::tensorflow::Status _status = (__VA_ARGS__); \ + if (TF_PREDICT_FALSE(!_status.ok())) return _status; \ + } while (0) + +#define TF_RETURN_WITH_CONTEXT_IF_ERROR(expr, ...) \ + do { \ + ::tensorflow::Status _status = (expr); \ + if (TF_PREDICT_FALSE(!_status.ok())) { \ + ::tensorflow::errors::AppendToMessage(&_status, __VA_ARGS__); \ + return _status; \ + } \ + } while (0) + +// Convenience functions for generating and using error status. +// Example usage: +// status.Update(errors::InvalidArgument("The ", foo, " isn't right.")); +// if (errors::IsInvalidArgument(status)) { ... } +// switch (status.code()) { case error::INVALID_ARGUMENT: ... } + +#define DECLARE_ERROR(FUNC, CONST) \ + template \ + ::tensorflow::Status FUNC(Args... args) { \ + return ::tensorflow::Status( \ + ::tensorflow::error::CONST, \ + ::tensorflow::strings::StrCat( \ + ::tensorflow::errors::internal::PrepareForStrCat(args)...)); \ + } \ + inline bool Is##FUNC(const ::tensorflow::Status& status) { \ + return status.code() == ::tensorflow::error::CONST; \ + } + +DECLARE_ERROR(Cancelled, CANCELLED) +DECLARE_ERROR(InvalidArgument, INVALID_ARGUMENT) +DECLARE_ERROR(NotFound, NOT_FOUND) +DECLARE_ERROR(AlreadyExists, ALREADY_EXISTS) +DECLARE_ERROR(ResourceExhausted, RESOURCE_EXHAUSTED) +DECLARE_ERROR(Unavailable, UNAVAILABLE) +DECLARE_ERROR(FailedPrecondition, FAILED_PRECONDITION) +DECLARE_ERROR(OutOfRange, OUT_OF_RANGE) +DECLARE_ERROR(Unimplemented, UNIMPLEMENTED) +DECLARE_ERROR(Internal, INTERNAL) +DECLARE_ERROR(Aborted, ABORTED) +DECLARE_ERROR(DeadlineExceeded, DEADLINE_EXCEEDED) +DECLARE_ERROR(DataLoss, DATA_LOSS) +DECLARE_ERROR(Unknown, UNKNOWN) +DECLARE_ERROR(PermissionDenied, PERMISSION_DENIED) +DECLARE_ERROR(Unauthenticated, UNAUTHENTICATED) + +#undef DECLARE_ERROR + +// Produces a formatted string pattern from the name which can uniquely identify +// this node upstream to produce an informative error message. The pattern +// followed is: {{node }} +// Note: The pattern below determines the regex _NODEDEF_NAME_RE in the file +// tensorflow/python/client/session.py +// LINT.IfChange +inline string FormatNodeNameForError(const string& name) { + return strings::StrCat("{{node ", name, "}}"); +} +// LINT.ThenChange(//tensorflow/python/client/session.py) +template +string FormatNodeNamesForError(const T& names) { + return ::tensorflow::str_util::Join( + names, ", ", [](string* output, const string& s) { + ::tensorflow::strings::StrAppend(output, FormatNodeNameForError(s)); + }); +} +// LINT.IfChange +inline string FormatColocationNodeForError(const string& name) { + return strings::StrCat("{{colocation_node ", name, "}}"); +} +// LINT.ThenChange(//tensorflow/python/framework/error_interpolation.py) +template +string FormatColocationNodeForError(const T& names) { + return ::tensorflow::str_util::Join( + names, ", ", [](string* output, const string& s) { + ::tensorflow::strings::StrAppend(output, + FormatColocationNodeForError(s)); + }); +} + +// The CanonicalCode() for non-errors. +using ::tensorflow::error::OK; + +} // namespace errors +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_LIB_CORE_ERRORS_H_ diff --git a/linux/include/tensorflow/core/lib/core/notification.h b/linux/include/tensorflow/core/lib/core/notification.h new file mode 100644 index 000000000..5def958e6 --- /dev/null +++ b/linux/include/tensorflow/core/lib/core/notification.h @@ -0,0 +1,23 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_LIB_CORE_NOTIFICATION_H_ +#define TENSORFLOW_CORE_LIB_CORE_NOTIFICATION_H_ + +// Notification implementation is platform-dependent, to support +// alternative synchronization primitives. +#include "tensorflow/core/platform/notification.h" + +#endif // TENSORFLOW_CORE_LIB_CORE_NOTIFICATION_H_ diff --git a/linux/include/tensorflow/core/lib/core/raw_coding.h b/linux/include/tensorflow/core/lib/core/raw_coding.h new file mode 100644 index 000000000..f49214939 --- /dev/null +++ b/linux/include/tensorflow/core/lib/core/raw_coding.h @@ -0,0 +1,71 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_LIB_CORE_RAW_CODING_H_ +#define TENSORFLOW_CORE_LIB_CORE_RAW_CODING_H_ + +#include +#include "tensorflow/core/platform/byte_order.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { +namespace core { + +// Lower-level versions of Get... that read directly from a character buffer +// without any bounds checking. + +inline uint16 DecodeFixed16(const char* ptr) { + if (port::kLittleEndian) { + // Load the raw bytes + uint16 result; + memcpy(&result, ptr, sizeof(result)); // gcc optimizes this to a plain load + return result; + } else { + return ((static_cast(static_cast(ptr[0]))) | + (static_cast(static_cast(ptr[1])) << 8)); + } +} + +inline uint32 DecodeFixed32(const char* ptr) { + if (port::kLittleEndian) { + // Load the raw bytes + uint32 result; + memcpy(&result, ptr, sizeof(result)); // gcc optimizes this to a plain load + return result; + } else { + return ((static_cast(static_cast(ptr[0]))) | + (static_cast(static_cast(ptr[1])) << 8) | + (static_cast(static_cast(ptr[2])) << 16) | + (static_cast(static_cast(ptr[3])) << 24)); + } +} + +inline uint64 DecodeFixed64(const char* ptr) { + if (port::kLittleEndian) { + // Load the raw bytes + uint64 result; + memcpy(&result, ptr, sizeof(result)); // gcc optimizes this to a plain load + return result; + } else { + uint64 lo = DecodeFixed32(ptr); + uint64 hi = DecodeFixed32(ptr + 4); + return (hi << 32) | lo; + } +} + +} // namespace core +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_LIB_CORE_RAW_CODING_H_ diff --git a/linux/include/tensorflow/core/lib/core/refcount.h b/linux/include/tensorflow/core/lib/core/refcount.h new file mode 100644 index 000000000..87bcfec41 --- /dev/null +++ b/linux/include/tensorflow/core/lib/core/refcount.h @@ -0,0 +1,118 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_LIB_CORE_REFCOUNT_H_ +#define TENSORFLOW_LIB_CORE_REFCOUNT_H_ + +#include +#include + +#include "tensorflow/core/platform/logging.h" + +namespace tensorflow { +namespace core { + +class RefCounted { + public: + // Initial reference count is one. + RefCounted(); + + // Increments reference count by one. + void Ref() const; + + // Decrements reference count by one. If the count remains + // positive, returns false. When the count reaches zero, returns + // true and deletes this, in which case the caller must not access + // the object afterward. + bool Unref() const; + + // Return whether the reference count is one. + // If the reference count is used in the conventional way, a + // reference count of 1 implies that the current thread owns the + // reference and no other thread shares it. + // This call performs the test for a reference count of one, and + // performs the memory barrier needed for the owning thread + // to act on the object, knowing that it has exclusive access to the + // object. + bool RefCountIsOne() const; + + protected: + // Make destructor protected so that RefCounted objects cannot + // be instantiated directly. Only subclasses can be instantiated. + virtual ~RefCounted(); + + private: + mutable std::atomic_int_fast32_t ref_; + + RefCounted(const RefCounted&) = delete; + void operator=(const RefCounted&) = delete; +}; + +// A deleter class to form a std::unique_ptr that unrefs objects. +struct RefCountDeleter { + void operator()(tensorflow::core::RefCounted* o) const { o->Unref(); } +}; + +// A unique_ptr that unrefs the owned object on destruction. +template +using RefCountPtr = std::unique_ptr; + +// Helper class to unref an object when out-of-scope. +class ScopedUnref { + public: + explicit ScopedUnref(RefCounted* o) : obj_(o) {} + ~ScopedUnref() { + if (obj_) obj_->Unref(); + } + + private: + RefCounted* obj_; + + ScopedUnref(const ScopedUnref&) = delete; + void operator=(const ScopedUnref&) = delete; +}; + +// Inlined routines, since these are performance critical +inline RefCounted::RefCounted() : ref_(1) {} + +inline RefCounted::~RefCounted() { DCHECK_EQ(ref_.load(), 0); } + +inline void RefCounted::Ref() const { + DCHECK_GE(ref_.load(), 1); + ref_.fetch_add(1, std::memory_order_relaxed); +} + +inline bool RefCounted::Unref() const { + DCHECK_GT(ref_.load(), 0); + // If ref_==1, this object is owned only by the caller. Bypass a locked op + // in that case. + if (RefCountIsOne() || ref_.fetch_sub(1) == 1) { + // Make DCHECK in ~RefCounted happy + DCHECK((ref_.store(0), true)); + delete this; + return true; + } else { + return false; + } +} + +inline bool RefCounted::RefCountIsOne() const { + return (ref_.load(std::memory_order_acquire) == 1); +} + +} // namespace core +} // namespace tensorflow + +#endif // TENSORFLOW_LIB_CORE_REFCOUNT_H_ diff --git a/linux/include/tensorflow/core/lib/core/status.h b/linux/include/tensorflow/core/lib/core/status.h new file mode 100644 index 000000000..eb0ff555a --- /dev/null +++ b/linux/include/tensorflow/core/lib/core/status.h @@ -0,0 +1,149 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_LIB_CORE_STATUS_H_ +#define TENSORFLOW_CORE_LIB_CORE_STATUS_H_ + +#include +#include +#include +#include +#include "tensorflow/core/lib/core/error_codes.pb.h" +#include "tensorflow/core/lib/core/stringpiece.h" +#include "tensorflow/core/platform/logging.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +#if defined(__clang__) +// Only clang supports warn_unused_result as a type annotation. +class TF_MUST_USE_RESULT Status; +#endif + +/// @ingroup core +/// Denotes success or failure of a call in Tensorflow. +class Status { + public: + /// Create a success status. + Status() {} + + /// \brief Create a status with the specified error code and msg as a + /// human-readable string containing more detailed information. + Status(tensorflow::error::Code code, tensorflow::StringPiece msg); + + /// Copy the specified status. + Status(const Status& s); + void operator=(const Status& s); + + static Status OK() { return Status(); } + + /// Returns true iff the status indicates success. + bool ok() const { return (state_ == NULL); } + + tensorflow::error::Code code() const { + return ok() ? tensorflow::error::OK : state_->code; + } + + const string& error_message() const { + return ok() ? empty_string() : state_->msg; + } + + bool operator==(const Status& x) const; + bool operator!=(const Status& x) const; + + /// \brief If `ok()`, stores `new_status` into `*this`. If `!ok()`, + /// preserves the current status, but may augment with additional + /// information about `new_status`. + /// + /// Convenient way of keeping track of the first error encountered. + /// Instead of: + /// `if (overall_status.ok()) overall_status = new_status` + /// Use: + /// `overall_status.Update(new_status);` + void Update(const Status& new_status); + + /// \brief Return a string representation of this status suitable for + /// printing. Returns the string `"OK"` for success. + string ToString() const; + + // Ignores any errors. This method does nothing except potentially suppress + // complaints from any tools that are checking that errors are not dropped on + // the floor. + void IgnoreError() const; + + private: + static const string& empty_string(); + struct State { + tensorflow::error::Code code; + string msg; + }; + // OK status has a `NULL` state_. Otherwise, `state_` points to + // a `State` structure containing the error code and message(s) + std::unique_ptr state_; + + void SlowCopyFrom(const State* src); +}; + +inline Status::Status(const Status& s) + : state_((s.state_ == NULL) ? NULL : new State(*s.state_)) {} + +inline void Status::operator=(const Status& s) { + // The following condition catches both aliasing (when this == &s), + // and the common case where both s and *this are ok. + if (state_ != s.state_) { + SlowCopyFrom(s.state_.get()); + } +} + +inline bool Status::operator==(const Status& x) const { + return (this->state_ == x.state_) || (ToString() == x.ToString()); +} + +inline bool Status::operator!=(const Status& x) const { return !(*this == x); } + +/// @ingroup core +std::ostream& operator<<(std::ostream& os, const Status& x); + +typedef std::function StatusCallback; + +extern tensorflow::string* TfCheckOpHelperOutOfLine( + const ::tensorflow::Status& v, const char* msg); + +inline tensorflow::string* TfCheckOpHelper(::tensorflow::Status v, + const char* msg) { + if (v.ok()) return nullptr; + return TfCheckOpHelperOutOfLine(v, msg); +} + +#define TF_DO_CHECK_OK(val, level) \ + while (auto _result = ::tensorflow::TfCheckOpHelper(val, #val)) \ + LOG(level) << *(_result) + +#define TF_CHECK_OK(val) TF_DO_CHECK_OK(val, FATAL) +#define TF_QCHECK_OK(val) TF_DO_CHECK_OK(val, QFATAL) + +// DEBUG only version of TF_CHECK_OK. Compiler still parses 'val' even in opt +// mode. +#ifndef NDEBUG +#define TF_DCHECK_OK(val) TF_CHECK_OK(val) +#else +#define TF_DCHECK_OK(val) \ + while (false && (::tensorflow::Status::OK() == (val))) LOG(FATAL) +#endif + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_LIB_CORE_STATUS_H_ diff --git a/linux/include/tensorflow/core/lib/core/stringpiece.h b/linux/include/tensorflow/core/lib/core/stringpiece.h new file mode 100644 index 000000000..6edff139a --- /dev/null +++ b/linux/include/tensorflow/core/lib/core/stringpiece.h @@ -0,0 +1,38 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// StringPiece is a simple structure containing a pointer into some external +// storage and a size. The user of a StringPiece must ensure that the slice +// is not used after the corresponding external storage has been +// deallocated. +// +// Multiple threads can invoke const methods on a StringPiece without +// external synchronization, but if any of the threads may call a +// non-const method, all threads accessing the same StringPiece must use +// external synchronization. + +#ifndef TENSORFLOW_CORE_LIB_CORE_STRINGPIECE_H_ +#define TENSORFLOW_CORE_LIB_CORE_STRINGPIECE_H_ + +#include "absl/strings/string_view.h" + +namespace tensorflow { + +// Deprecated: please use absl::string_view directly. +using StringPiece = absl::string_view; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_LIB_CORE_STRINGPIECE_H_ diff --git a/linux/include/tensorflow/core/lib/core/threadpool.h b/linux/include/tensorflow/core/lib/core/threadpool.h new file mode 100644 index 000000000..e14ad7ac6 --- /dev/null +++ b/linux/include/tensorflow/core/lib/core/threadpool.h @@ -0,0 +1,125 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_LIB_CORE_THREADPOOL_H_ +#define TENSORFLOW_CORE_LIB_CORE_THREADPOOL_H_ + +#include +#include +#include "tensorflow/core/platform/env.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { +namespace thread { + +class ThreadPool { + public: + // Constructs a pool that contains "num_threads" threads with specified + // "name". env->StartThread() is used to create individual threads with the + // given ThreadOptions. If "low_latency_hint" is true the thread pool + // implementation may use it as a hint that lower latency is preferred at the + // cost of higher CPU usage, e.g. by letting one or more idle threads spin + // wait. Conversely, if the threadpool is used to schedule high-latency + // operations like I/O the hint should be set to false. + // + // REQUIRES: num_threads > 0 + ThreadPool(Env* env, const ThreadOptions& thread_options, const string& name, + int num_threads, bool low_latency_hint); + + // Constructs a pool for low-latency ops that contains "num_threads" threads + // with specified "name". env->StartThread() is used to create individual + // threads. + // REQUIRES: num_threads > 0 + ThreadPool(Env* env, const string& name, int num_threads); + + // Constructs a pool for low-latency ops that contains "num_threads" threads + // with specified "name". env->StartThread() is used to create individual + // threads with the given ThreadOptions. + // REQUIRES: num_threads > 0 + ThreadPool(Env* env, const ThreadOptions& thread_options, const string& name, + int num_threads); + + // Waits until all scheduled work has finished and then destroy the + // set of threads. + ~ThreadPool(); + + // Schedules fn() for execution in the pool of threads. + void Schedule(std::function fn); + + // Requires 0 < block_size <= total. + // Spawns k threads and calls fn(i*block_size, (i+1)*block_size) from the + // ith thread (i>=0). When (i+1)*block_size > total, fn(i*block_size, total) + // is called instead. k = NumShardsUsedByTransformRangeConcurrently(...). + // Note that when there aren't enough threads in the pool to achieve full + // parallelism, function calls will be automatically queued. + void TransformRangeConcurrently(const int64 block_size, const int64 total, + const std::function& fn); + + // Returns the number of threads spawned by calling TransformRangeConcurrently + // with these parameters. + int NumShardsUsedByTransformRangeConcurrently(const int64 block_size, + const int64 total); + + // ParallelFor shards the "total" units of work assuming each unit of work + // having roughly "cost_per_unit" cost, in cycles. Each unit of work is + // indexed 0, 1, ..., total - 1. Each shard contains 1 or more units of work + // and the total cost of each shard is roughly the same. + // + // "cost_per_unit" is an estimate of the number of CPU cycles (or nanoseconds + // if not CPU-bound) to complete a unit of work. Overestimating creates too + // many shards and CPU time will be dominated by per-shard overhead, such as + // Context creation. Underestimating may not fully make use of the specified + // parallelism. + void ParallelFor(int64 total, int64 cost_per_unit, + std::function fn); + + // Shards the "total" units of work. For more details, see "ParallelFor". + // + // The function is passed a thread_id between 0 and NumThreads() *inclusive*. + // This is because some work can happen on the caller thread while the threads + // in the pool are also being used. + // + // The caller can allocate NumThreads() + 1 separate buffers for each thread. + // Each thread can safely write to the buffer given by its id without + // synchronization. However, the worker fn may be called multiple times + // sequentially with the same id. + // + // At most NumThreads() unique ids will actually be used, and only a few may + // be used for small workloads. If each buffer is expensive, the buffers + // should be stored in an array initially filled with null, and a buffer + // should be allocated by fn the first time that the id is used. + void ParallelForWithWorkerId( + int64 total, int64 cost_per_unit, + const std::function& fn); + + // Returns the number of threads in the pool. + int NumThreads() const; + + // Returns current thread id between 0 and NumThreads() - 1, if called from a + // thread in the pool. Returns -1 otherwise. + int CurrentThreadId() const; + + struct Impl; + + private: + std::unique_ptr impl_; + TF_DISALLOW_COPY_AND_ASSIGN(ThreadPool); +}; + +} // namespace thread +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_LIB_CORE_THREADPOOL_H_ diff --git a/linux/include/tensorflow/core/lib/db/sqlite.h b/linux/include/tensorflow/core/lib/db/sqlite.h new file mode 100644 index 000000000..efe97f78d --- /dev/null +++ b/linux/include/tensorflow/core/lib/db/sqlite.h @@ -0,0 +1,449 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +#ifndef TENSORFLOW_CORE_LIB_DB_SQLITE_H_ +#define TENSORFLOW_CORE_LIB_DB_SQLITE_H_ + +#include + +#include "sqlite3.h" +#include "tensorflow/core/lib/core/refcount.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/lib/core/stringpiece.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/thread_annotations.h" +#include "tensorflow/core/platform/types.h" + +/// TensorFlow SQLite Veneer +/// +/// - Memory safety +/// - Less boilerplate +/// - Removes deprecated stuff +/// - Pretends UTF16 doesn't exist +/// - Transaction compile-time safety +/// - Statically loads our native extensions +/// - Error reporting via tensorflow::Status et al. +/// +/// SQLite>=3.8.2 needs to be supported until April 2019, which is when +/// Ubuntu 14.04 LTS becomes EOL. + +namespace tensorflow { + +class SqliteLock; +class SqliteStatement; +class SqliteTransaction; + +/// \brief SQLite connection object. +/// +/// The SQLite connection is closed automatically by the destructor. +/// Reference counting ensures that happens after its statements are +/// destructed. +/// +/// Instances are reference counted and can be shared between threads. +/// This class offers the same thread safety behaviors as the SQLite +/// API itself. +/// +/// This veneer uses auto-commit mode by default, which means a 4ms +/// fsync() happens after every write unless a SqliteTransaction is +/// used or WAL mode is enabled beforehand. +class LOCKABLE Sqlite : public core::RefCounted { + public: + /// \brief Closes SQLite connection, which can take milliseconds. + virtual ~Sqlite(); + + /// \brief Opens SQLite database file. + /// + /// Most users will want to set flags to SQLITE_OPEN_READWRITE | + /// SQLITE_OPEN_CREATE. There are many other open flags; here are + /// notes on a few of them: + /// + /// - SQLITE_OPEN_READONLY: Allowed if no WAL journal is active. + /// - SQLITE_OPEN_SHAREDCACHE: Will be ignored because this veneer + /// doesn't support the unlock notify API. + /// - SQLITE_OPEN_NOMUTEX: Means access to this connection MUST be + /// serialized by the caller in accordance with the same contracts + /// implemented by this API. + /// + /// This function sets PRAGMA values from TF_SQLITE_* environment + /// variables. See sqlite.cc to learn more. + static Status Open(const string& path, int flags, Sqlite** db); + + /// \brief Creates SQLite statement. + /// + /// This routine should never fail if sql is valid and does not + /// reference tables. When tables are referenced, system calls are + /// needed which can take microseconds. When the schema changes, this + /// routine will retry automatically and then possibly fail. + /// + /// The returned statement holds a reference to this object. + Status Prepare(const StringPiece& sql, SqliteStatement* stmt); + SqliteStatement PrepareOrDie(const StringPiece& sql); + + /// \brief Returns extended result code of last error. + /// + /// If the most recent API call was successful, the result is + /// undefined. The legacy result code can be obtained by saying + /// errcode() & 0xff. + int errcode() const EXCLUSIVE_LOCKS_REQUIRED(this) { + return sqlite3_extended_errcode(db_); + } + + /// \brief Returns pointer to current error message state. + const char* errmsg() const EXCLUSIVE_LOCKS_REQUIRED(this) { + return sqlite3_errmsg(db_); + } + + /// \brief Returns rowid assigned to last successful insert. + int64 last_insert_rowid() const EXCLUSIVE_LOCKS_REQUIRED(this) { + return sqlite3_last_insert_rowid(db_); + } + + /// \brief Returns number of rows directly changed by last write. + int64 changes() const EXCLUSIVE_LOCKS_REQUIRED(this) { + return sqlite3_changes(db_); + } + + private: + friend class SqliteLock; + friend class SqliteStatement; + friend class SqliteTransaction; + + Sqlite(sqlite3* db, sqlite3_stmt* begin, sqlite3_stmt* commit, + sqlite3_stmt* rollback) noexcept + : db_(db), begin_(begin), commit_(commit), rollback_(rollback) {} + + sqlite3* const db_; + sqlite3_stmt* const begin_; + sqlite3_stmt* const commit_; + sqlite3_stmt* const rollback_; + bool is_in_transaction_ = false; + + TF_DISALLOW_COPY_AND_ASSIGN(Sqlite); +}; + +/// \brief SQLite prepared statement. +/// +/// Instances can only be shared between threads if caller serializes +/// access from first Bind*() to *Reset(). +/// +/// When reusing a statement in a loop, be certain to not have jumps +/// betwixt Bind*() and *Reset(). +class SqliteStatement { + public: + /// \brief Initializes an empty statement to be assigned later. + SqliteStatement() noexcept = default; + + /// \brief Finalizes statement. + /// + /// This can take milliseconds if it was blocking the Sqlite + /// connection object from being freed. + ~SqliteStatement() { + sqlite3_finalize(stmt_); + if (db_ != nullptr) db_->Unref(); + } + + /// \brief Returns true if statement is initialized. + explicit operator bool() const { return stmt_ != nullptr; } + + /// \brief Returns SQL text from when this query was prepared. + const char* sql() const { return sqlite3_sql(stmt_); } + + /// \brief Number of bytes bound since last *Reset(). + uint64 size() { return size_; } + + /// \brief Executes query for fetching arbitrary rows. + /// + /// `is_done` will always be set to true unless SQLITE_ROW is + /// returned by the underlying API. If status() is already in an + /// error state, then this method is a no-op and the existing status + /// is returned. + /// + /// The OrDie version returns `!is_done` which, if true, indicates a + /// row is available. + /// + /// This statement should be Reset() or destructed when when finished + /// with the result. + Status Step(bool* is_done); + bool StepOrDie() TF_MUST_USE_RESULT; + + /// \brief Executes query when only one row is desired. + /// + /// If a row isn't returned, an internal error Status is returned + /// that won't be reflected in the connection error state. + /// + /// This statement should be Reset() or destructed when when finished + /// with the result. + Status StepOnce(); + const SqliteStatement& StepOnceOrDie(); + + /// \brief Executes query, ensures zero rows returned, then Reset(). + /// + /// If a row is returned, an internal error Status is returned that + /// won't be reflected in the connection error state. + Status StepAndReset(); + void StepAndResetOrDie(); + + /// \brief Resets statement so it can be executed again. + /// + /// Implementation note: This method diverges from canonical API + /// behavior by calling sqlite3_clear_bindings() in addition to + /// sqlite3_reset(). That makes the veneer safer; we haven't found a + /// super compelling reason yet to call them independently. + void Reset(); + + /// \brief Binds signed 64-bit integer to 1-indexed query parameter. + void BindInt(int parameter, int64 value) { + Update(sqlite3_bind_int64(stmt_, parameter, value), parameter); + size_ += sizeof(int64); + } + void BindInt(const char* parameter, int64 value) { + BindInt(GetParameterIndex(parameter), value); + } + + /// \brief Binds double to 1-indexed query parameter. + void BindDouble(int parameter, double value) { + Update(sqlite3_bind_double(stmt_, parameter, value), parameter); + size_ += sizeof(double); + } + void BindDouble(const char* parameter, double value) { + BindDouble(GetParameterIndex(parameter), value); + } + + /// \brief Copies UTF-8 text to 1-indexed query parameter. + /// + /// If NUL characters are present, they will still go in the DB and + /// be successfully retrieved by ColumnString(); however, the + /// behavior of these values with SQLite functions is undefined. + /// + /// When using the unsafe methods, the data must not be changed or + /// freed until this statement is Reset() or finalized. + void BindText(int parameter, const StringPiece& text) { + Update(sqlite3_bind_text64(stmt_, parameter, text.data(), text.size(), + SQLITE_TRANSIENT, SQLITE_UTF8), + parameter); + size_ += text.size(); + } + void BindText(const char* parameter, const StringPiece& text) { + BindText(GetParameterIndex(parameter), text); + } + void BindTextUnsafe(int parameter, const StringPiece& text) { + Update(sqlite3_bind_text64(stmt_, parameter, text.data(), text.size(), + SQLITE_STATIC, SQLITE_UTF8), + parameter); + size_ += text.size(); + } + void BindTextUnsafe(const char* parameter, const StringPiece& text) { + BindTextUnsafe(GetParameterIndex(parameter), text); + } + + /// \brief Copies binary data to 1-indexed query parameter. + /// + /// When using the unsafe methods, the data must not be changed or + /// freed until this statement is Reset() or finalized. + void BindBlob(int parameter, const StringPiece& blob) { + Update(sqlite3_bind_blob64(stmt_, parameter, blob.data(), blob.size(), + SQLITE_TRANSIENT), + parameter); + size_ += blob.size(); + } + void BindBlob(const char* parameter, const StringPiece& blob) { + BindBlob(GetParameterIndex(parameter), blob); + } + void BindBlobUnsafe(int parameter, const StringPiece& blob) { + Update(sqlite3_bind_blob64(stmt_, parameter, blob.data(), blob.size(), + SQLITE_STATIC), + parameter); + size_ += blob.size(); + } + void BindBlobUnsafe(const char* parameter, const StringPiece& text) { + BindBlobUnsafe(GetParameterIndex(parameter), text); + } + + /// \brief Returns number of columns in result set. + int ColumnCount() const TF_MUST_USE_RESULT { + return sqlite3_column_count(stmt_); + } + + /// \brief Returns type of 0-indexed column value in row data. + /// + /// Please note that SQLite is dynamically typed and the type of a + /// particular column can vary from row to row. + int ColumnType(int column) const TF_MUST_USE_RESULT { + return sqlite3_column_type(stmt_, column); + } + + /// \brief Returns 0-indexed column from row result coerced as an integer. + int64 ColumnInt(int column) const TF_MUST_USE_RESULT { + return sqlite3_column_int64(stmt_, column); + } + + /// \brief Returns 0-indexed column from row result coerced as a double. + double ColumnDouble(int column) const TF_MUST_USE_RESULT { + return sqlite3_column_double(stmt_, column); + } + + /// \brief Copies 0-indexed column from row result coerced as a string. + /// + /// NULL values are returned as empty string. This method should be + /// used for both BLOB and TEXT columns. See also: ColumnType(). + string ColumnString(int column) const TF_MUST_USE_RESULT { + auto data = sqlite3_column_blob(stmt_, column); + if (data == nullptr) return ""; + return {static_cast(data), + static_cast(ColumnSize(column))}; + } + + /// \brief Returns pointer to binary data at 0-indexed column. + /// + /// Empty values are returned as NULL. The returned memory will no + /// longer be valid the next time Step() or Reset() is called. No NUL + /// terminator is added. + StringPiece ColumnStringUnsafe(int column) const TF_MUST_USE_RESULT { + return {static_cast(sqlite3_column_blob(stmt_, column)), + static_cast(ColumnSize(column))}; + } + + /// \brief Returns number of bytes stored at 0-indexed column. + int ColumnSize(int column) const TF_MUST_USE_RESULT { + return sqlite3_column_bytes(stmt_, column); + } + + /// \brief Move constructor, after which is reset to empty. + SqliteStatement(SqliteStatement&& other) noexcept + : db_(other.db_), stmt_(other.stmt_), bind_error_(other.bind_error_) { + other.db_ = nullptr; + other.stmt_ = nullptr; + other.bind_error_ = SQLITE_OK; + } + + /// \brief Move assignment, after which is reset to empty. + SqliteStatement& operator=(SqliteStatement&& other) noexcept { + if (&other != this) { + if (db_ != nullptr) db_->Unref(); + if (stmt_ != nullptr) sqlite3_finalize(stmt_); + db_ = other.db_; + stmt_ = other.stmt_; + bind_error_ = other.bind_error_; + size_ = other.size_; + other.db_ = nullptr; + other.stmt_ = nullptr; + other.bind_error_ = SQLITE_OK; + other.size_ = 0; + } + return *this; + } + + private: + friend class Sqlite; + + SqliteStatement(Sqlite* db, sqlite3_stmt* stmt) noexcept + : db_(db), stmt_(stmt) { + db_->Ref(); + } + + void Update(int rc, int parameter) { + // Binding strings can fail if they exceed length limit. + if (TF_PREDICT_FALSE(rc != SQLITE_OK)) { + if (bind_error_ == SQLITE_OK) { + bind_error_ = rc; + bind_error_parameter_ = parameter; + } + } + } + + int GetParameterIndex(const char* parameter) { + int index = sqlite3_bind_parameter_index(stmt_, parameter); + DCHECK(index > 0); // OK to compile away since it'll fail again + return index; + } + + Sqlite* db_ = nullptr; + sqlite3_stmt* stmt_ = nullptr; + int bind_error_ = SQLITE_OK; + int bind_error_parameter_ = 0; + uint64 size_ = 0; + + TF_DISALLOW_COPY_AND_ASSIGN(SqliteStatement); +}; + +/// \brief Reentrant SQLite connection object lock +/// +/// This is a no-op if SQLITE_OPEN_NOMUTEX was used. +class SCOPED_LOCKABLE SqliteLock { + public: + explicit SqliteLock(Sqlite& db) EXCLUSIVE_LOCK_FUNCTION(db) + : mutex_(sqlite3_db_mutex(db.db_)) { + sqlite3_mutex_enter(mutex_); + } + SqliteLock(Sqlite& db, std::try_to_lock_t) EXCLUSIVE_LOCK_FUNCTION(db) + : mutex_(sqlite3_db_mutex(db.db_)) { + if (TF_PREDICT_FALSE(sqlite3_mutex_try(mutex_) != SQLITE_OK)) { + is_locked_ = false; + } + } + ~SqliteLock() UNLOCK_FUNCTION() { + if (is_locked_) sqlite3_mutex_leave(mutex_); + } + explicit operator bool() const { return is_locked_; } + + private: + sqlite3_mutex* const mutex_; + bool is_locked_ = true; + TF_DISALLOW_COPY_AND_ASSIGN(SqliteLock); +}; +#define SqliteLock(x) static_assert(0, "sqlite_lock_decl_missing_name"); + +/// \brief SQLite transaction scope. +/// +/// This class acquires an exclusive lock on the connection object (if +/// mutexes weren't disabled) and runs BEGIN / ROLLBACK automatically. +/// Unlike SqliteLock this scope is non-reentrant. To avoid program +/// crashes, business logic should use the EXCLUSIVE_LOCK_FUNCTION and +/// LOCKS_EXCLUDED annotations as much as possible. +class SCOPED_LOCKABLE SqliteTransaction { + public: + /// \brief Locks db and begins deferred transaction. + /// + /// This will crash if a transaction is already active. + explicit SqliteTransaction(Sqlite& db) EXCLUSIVE_LOCK_FUNCTION(db); + + /// \brief Runs ROLLBACK and unlocks. + ~SqliteTransaction() UNLOCK_FUNCTION(); + + /// \brief Commits transaction. + /// + /// If this is successful, a new transaction will be started, which + /// is rolled back when exiting the scope. + Status Commit(); + + private: + void Begin(); + Sqlite* const db_; + + TF_DISALLOW_COPY_AND_ASSIGN(SqliteTransaction); +}; + +#define SQLITE_EXCLUSIVE_TRANSACTIONS_REQUIRED(...) \ + EXCLUSIVE_LOCKS_REQUIRED(__VA_ARGS__) +#define SQLITE_TRANSACTIONS_EXCLUDED(...) LOCKS_EXCLUDED(__VA_ARGS__) + +inline SqliteStatement Sqlite::PrepareOrDie(const StringPiece& sql) { + SqliteStatement stmt; + TF_CHECK_OK(Prepare(sql, &stmt)); + return stmt; +} + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_LIB_DB_SQLITE_H_ diff --git a/linux/include/tensorflow/core/lib/gif/gif_io.h b/linux/include/tensorflow/core/lib/gif/gif_io.h new file mode 100644 index 000000000..0a7967a5a --- /dev/null +++ b/linux/include/tensorflow/core/lib/gif/gif_io.h @@ -0,0 +1,52 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// Functions to read and write images in GIF format. +// +// The advantage over image/codec/png{enc,dec}ocder.h is that this library +// supports both 8 and 16 bit images. +// +// The decoding routine accepts binary image data as a StringPiece. These are +// implicitly constructed from strings or char* so they're completely +// transparent to the caller. They're also very cheap to construct so this +// doesn't introduce any additional overhead. +// +// The primary benefit of StringPieces being, in this case, that APIs already +// returning StringPieces (e.g., Bigtable Scanner) or Cords (e.g., IOBuffer; +// only when they're flat, though) or protocol buffer fields typed to either of +// these can be decoded without copying the data into a C++ string. + +#ifndef TENSORFLOW_CORE_LIB_GIF_GIF_IO_H_ +#define TENSORFLOW_CORE_LIB_GIF_GIF_IO_H_ + +#include +#include +#include +#include + +#include "tensorflow/core/lib/core/stringpiece.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { +namespace gif { + +uint8* Decode(const void* srcdata, int datasize, + const std::function& allocate_output, + string* error_string); + +} // namespace gif +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_LIB_GIF_GIF_IO_H_ diff --git a/linux/include/tensorflow/core/lib/gtl/array_slice.h b/linux/include/tensorflow/core/lib/gtl/array_slice.h new file mode 100644 index 000000000..8f47faf89 --- /dev/null +++ b/linux/include/tensorflow/core/lib/gtl/array_slice.h @@ -0,0 +1,36 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_LIB_GTL_ARRAY_SLICE_H_ +#define TENSORFLOW_CORE_LIB_GTL_ARRAY_SLICE_H_ + +#include "absl/types/span.h" +// TODO(timshen): This is kept only because lots of targets transitively depend +// on it. Remove all targets' dependencies. +#include "tensorflow/core/lib/gtl/inlined_vector.h" + +namespace tensorflow { +namespace gtl { + +template +using ArraySlice = absl::Span; + +template +using MutableArraySlice = absl::Span; + +} // namespace gtl +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_LIB_GTL_ARRAY_SLICE_H_ diff --git a/linux/include/tensorflow/core/lib/gtl/cleanup.h b/linux/include/tensorflow/core/lib/gtl/cleanup.h new file mode 100644 index 000000000..8c73dc6aa --- /dev/null +++ b/linux/include/tensorflow/core/lib/gtl/cleanup.h @@ -0,0 +1,113 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// MakeCleanup(f) returns an RAII cleanup object that calls 'f' in its +// destructor. The easiest way to use MakeCleanup is with a lambda argument, +// capturing the return value in an 'auto' local variable. Most users will not +// need more sophisticated syntax than that. +// +// Example: +// void func() {} +// FILE* fp = fopen("data.txt", "r"); +// if (fp == nullptr) return; +// auto fp_cleaner = gtl::MakeCleanup([fp] { fclose(fp); }); +// // No matter what, fclose(fp) will happen. +// DataObject d; +// while (ReadDataObject(fp, &d)) { +// if (d.IsBad()) { +// LOG(ERROR) << "Bad Data"; +// return; +// } +// PushGoodData(d); +// } +// } +// +// You can use Cleanup directly, instead of using MakeCleanup and auto, +// but there's rarely a reason to do that. +// +// You can call 'release()' on a Cleanup object to cancel the cleanup. + +#ifndef TENSORFLOW_CORE_LIB_GTL_CLEANUP_H_ +#define TENSORFLOW_CORE_LIB_GTL_CLEANUP_H_ + +#include +#include + +#include "tensorflow/core/platform/macros.h" + +namespace tensorflow { +namespace gtl { + +// A move-only RAII object that calls a stored cleanup functor when +// destroyed. Cleanup is the return type of gtl::MakeCleanup(F). +template +class Cleanup { + public: + Cleanup() : released_(true), f_() {} + + template + explicit Cleanup(G&& f) // NOLINT + : f_(std::forward(f)) {} // NOLINT(build/c++11) + + Cleanup(Cleanup&& src) // NOLINT + : released_(src.is_released()), f_(src.release()) {} + + // Implicitly move-constructible from any compatible Cleanup. + // The source will be released as if src.release() were called. + // A moved-from Cleanup can be safely destroyed or reassigned. + template + Cleanup(Cleanup&& src) // NOLINT + : released_(src.is_released()), f_(src.release()) {} + + // Assignment to a Cleanup object behaves like destroying it + // and making a new one in its place, analogous to unique_ptr + // semantics. + Cleanup& operator=(Cleanup&& src) { // NOLINT + if (!released_) f_(); + released_ = src.released_; + f_ = src.release(); + return *this; + } + + ~Cleanup() { + if (!released_) f_(); + } + + // Releases the cleanup function instead of running it. + // Hint: use c.release()() to run early. + F release() { + released_ = true; + return std::move(f_); + } + + bool is_released() const { return released_; } + + private: + static_assert(!std::is_reference::value, "F must not be a reference"); + + bool released_ = false; + F f_; +}; + +template ::type> +TF_MUST_USE_RESULT Cleanup MakeCleanup(F&& f) { + return Cleanup(std::forward(f)); +} + +} // namespace gtl +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_LIB_GTL_CLEANUP_H_ diff --git a/linux/include/tensorflow/core/lib/gtl/compactptrset.h b/linux/include/tensorflow/core/lib/gtl/compactptrset.h new file mode 100644 index 000000000..d3d23b94a --- /dev/null +++ b/linux/include/tensorflow/core/lib/gtl/compactptrset.h @@ -0,0 +1,208 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_LIB_GTL_COMPACTPTRSET_H_ +#define TENSORFLOW_CORE_LIB_GTL_COMPACTPTRSET_H_ + +#include +#include "tensorflow/core/lib/gtl/flatset.h" + +namespace tensorflow { +namespace gtl { + +// CompactPointerSet is like a std::unordered_set but is optimized +// for small sets (<= 1 element). T must be a pointer type. +template +class CompactPointerSet { + private: + using BigRep = FlatSet; + + public: + using value_type = T; + + CompactPointerSet() : rep_(0) {} + + ~CompactPointerSet() { + static_assert( + std::is_pointer::value, + "CompactPointerSet can only be used with T's that are pointers"); + if (isbig()) delete big(); + } + + CompactPointerSet(const CompactPointerSet& other) : rep_(0) { *this = other; } + + CompactPointerSet& operator=(const CompactPointerSet& other) { + if (this == &other) return *this; + if (other.isbig()) { + // big => any + if (!isbig()) MakeBig(); + *big() = *other.big(); + } else if (isbig()) { + // !big => big + big()->clear(); + if (other.rep_ != 0) { + big()->insert(reinterpret_cast(other.rep_)); + } + } else { + // !big => !big + rep_ = other.rep_; + } + return *this; + } + + class iterator { + public: + typedef ssize_t difference_type; + typedef T value_type; + typedef const T* pointer; + typedef const T& reference; + typedef ::std::forward_iterator_tag iterator_category; + + explicit iterator(uintptr_t rep) + : bigrep_(false), single_(reinterpret_cast(rep)) {} + explicit iterator(typename BigRep::iterator iter) + : bigrep_(true), single_(nullptr), iter_(iter) {} + + iterator& operator++() { + if (bigrep_) { + ++iter_; + } else { + DCHECK(single_ != nullptr); + single_ = nullptr; + } + return *this; + } + // maybe post-increment? + + bool operator==(const iterator& other) const { + if (bigrep_) { + return iter_ == other.iter_; + } else { + return single_ == other.single_; + } + } + bool operator!=(const iterator& other) const { return !(*this == other); } + + const T& operator*() const { + if (bigrep_) { + return *iter_; + } else { + DCHECK(single_ != nullptr); + return single_; + } + } + + private: + friend class CompactPointerSet; + bool bigrep_; + T single_; + typename BigRep::iterator iter_; + }; + using const_iterator = iterator; + + bool empty() const { return isbig() ? big()->empty() : (rep_ == 0); } + size_t size() const { return isbig() ? big()->size() : (rep_ == 0 ? 0 : 1); } + + void clear() { + if (isbig()) { + delete big(); + } + rep_ = 0; + } + + std::pair insert(T elem) { + if (!isbig()) { + if (rep_ == 0) { + uintptr_t v = reinterpret_cast(elem); + if (v == 0 || ((v & 0x3) != 0)) { + // Cannot use small representation for nullptr. Fall through. + } else { + rep_ = v; + return {iterator(v), true}; + } + } + MakeBig(); + } + auto p = big()->insert(elem); + return {iterator(p.first), p.second}; + } + + template + void insert(InputIter begin, InputIter end) { + for (; begin != end; ++begin) { + insert(*begin); + } + } + + const_iterator begin() const { + return isbig() ? iterator(big()->begin()) : iterator(rep_); + } + const_iterator end() const { + return isbig() ? iterator(big()->end()) : iterator(0); + } + + iterator find(T elem) const { + if (rep_ == reinterpret_cast(elem)) { + return iterator(rep_); + } else if (!isbig()) { + return iterator(0); + } else { + return iterator(big()->find(elem)); + } + } + + size_t count(T elem) const { return find(elem) != end() ? 1 : 0; } + + size_t erase(T elem) { + if (!isbig()) { + if (rep_ == reinterpret_cast(elem)) { + rep_ = 0; + return 1; + } else { + return 0; + } + } else { + return big()->erase(elem); + } + } + + private: + // Size rep_ + // ------------------------------------------------------------------------- + // 0 0 + // 1 The pointer itself (bottom bits == 00) + // large Pointer to a BigRep (bottom bits == 01) + uintptr_t rep_; + + bool isbig() const { return (rep_ & 0x3) == 1; } + BigRep* big() const { + DCHECK(isbig()); + return reinterpret_cast(rep_ - 1); + } + + void MakeBig() { + DCHECK(!isbig()); + BigRep* big = new BigRep; + if (rep_ != 0) { + big->insert(reinterpret_cast(rep_)); + } + rep_ = reinterpret_cast(big) + 0x1; + } +}; + +} // namespace gtl +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_LIB_GTL_COMPACTPTRSET_H_ diff --git a/linux/include/tensorflow/core/lib/gtl/edit_distance.h b/linux/include/tensorflow/core/lib/gtl/edit_distance.h new file mode 100644 index 000000000..9db5f9421 --- /dev/null +++ b/linux/include/tensorflow/core/lib/gtl/edit_distance.h @@ -0,0 +1,108 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_LIB_GTL_EDIT_DISTANCE_H_ +#define TENSORFLOW_LIB_GTL_EDIT_DISTANCE_H_ + +#include + +#include "tensorflow/core/lib/gtl/array_slice.h" +#include "tensorflow/core/lib/gtl/inlined_vector.h" + +namespace tensorflow { +namespace gtl { + +// Calculate the Levenshtein Edit Distance between two contiguous +// sequences, s and t, of type T. +// +// The Levenshtein distance is a symmetric distance defined as the +// smallest number of insertions, deletions, and substitutions +// required to convert sequence s to t (and vice versa). +// Note, this distance does not consider transpositions. +// +// For more details and a reference implementation, see: +// https://en.wikipedia.org/wiki/Levenshtein_distance +// +// This implementation has time complexity O(|s|*|t|) +// and space complexity O(min(|s|, |t|)), where +// |x| := x.size() +// +// A simple call to LevenshteinDistance looks like: +// +// int64 dist = LevenshteinDistance("hi", "bye", std::equal_to()); +// +template +inline int64 LevenshteinDistance(const gtl::ArraySlice& s, + const gtl::ArraySlice& t, const Cmp& cmp) { + const int64 s_size = s.size(); + const int64 t_size = t.size(); + + if (t_size > s_size) return LevenshteinDistance(t, s, cmp); + + const T* s_data = s.data(); + const T* t_data = t.data(); + + if (t_size == 0) return s_size; + if (s == t) return 0; + + // Create work vector + gtl::InlinedVector scratch_holder(t_size); + + int64* scratch = scratch_holder.data(); + + // Special case for i = 0: Distance between empty string and string + // of length j is just j. + for (size_t j = 1; j < t_size; ++j) scratch[j - 1] = j; + + for (size_t i = 1; i <= s_size; ++i) { + // Invariant: scratch[j - 1] equals cost(i - 1, j). + int substitution_base_cost = i - 1; + int insertion_cost = i + 1; + for (size_t j = 1; j <= t_size; ++j) { + // Invariants: + // scratch[k - 1] = cost(i, k) for 0 < k < j. + // scratch[k - 1] = cost(i - 1, k) for j <= k <= t_size. + // substitution_base_cost = cost(i - 1, j - 1) + // insertion_cost = cost(i, j - 1) + const int replacement_cost = cmp(s_data[i - 1], t_data[j - 1]) ? 0 : 1; + const int substitution_cost = substitution_base_cost + replacement_cost; + const int deletion_cost = scratch[j - 1] + 1; + + // Select the cheapest edit. + const int cheapest = // = cost(i, j) + std::min(deletion_cost, std::min(insertion_cost, substitution_cost)); + + // Restore invariant for the next iteration of the loop. + substitution_base_cost = scratch[j - 1]; // = cost(i - 1, j) + scratch[j - 1] = cheapest; // = cost(i, j) + insertion_cost = cheapest + 1; // = cost(i, j) + 1 + } + } + return scratch[t_size - 1]; +} + +template +inline int64 LevenshteinDistance(const Container1& s, const Container2& t, + const Cmp& cmp) { + return LevenshteinDistance( + gtl::ArraySlice(s.data(), s.size()), + gtl::ArraySlice(t.data(), t.size()), + cmp); +} + +} // namespace gtl +} // namespace tensorflow + +#endif // TENSORFLOW_LIB_GTL_EDIT_DISTANCE_H_ diff --git a/linux/include/tensorflow/core/lib/gtl/flatmap.h b/linux/include/tensorflow/core/lib/gtl/flatmap.h new file mode 100644 index 000000000..9dc439c16 --- /dev/null +++ b/linux/include/tensorflow/core/lib/gtl/flatmap.h @@ -0,0 +1,393 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_LIB_GTL_FLATMAP_H_ +#define TENSORFLOW_CORE_LIB_GTL_FLATMAP_H_ + +#include +#include +#include +#include +#include +#include "tensorflow/core/lib/gtl/flatrep.h" +#include "tensorflow/core/lib/hash/hash.h" +#include "tensorflow/core/platform/logging.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { +namespace gtl { + +// FlatMap provides a map from K to V. +// +// The map is implemented using an open-addressed hash table. A +// single array holds entire map contents and collisions are resolved +// by probing at a sequence of locations in the array. +template , + class Eq = std::equal_to> +class FlatMap { + private: + // Forward declare some internal types needed in public section. + struct Bucket; + + // We cannot use std::pair<> since internal representation stores + // keys and values in separate arrays, so we make a custom struct + // that holds references to the internal key, value elements. + // + // We define the struct as private ValueType, and typedef it as public + // value_type, to work around a gcc bug when compiling the iterators. + struct ValueType { + typedef Key first_type; + typedef Val second_type; + + const Key& first; + Val& second; + ValueType(const Key& k, Val& v) : first(k), second(v) {} + }; + + public: + typedef Key key_type; + typedef Val mapped_type; + typedef Hash hasher; + typedef Eq key_equal; + typedef size_t size_type; + typedef ptrdiff_t difference_type; + typedef ValueType value_type; + typedef value_type* pointer; + typedef const value_type* const_pointer; + typedef value_type& reference; + typedef const value_type& const_reference; + + FlatMap() : FlatMap(1) {} + + explicit FlatMap(size_t N, const Hash& hf = Hash(), const Eq& eq = Eq()) + : rep_(N, hf, eq) {} + + FlatMap(const FlatMap& src) : rep_(src.rep_) {} + + // Move constructor leaves src in a valid but unspecified state (same as + // std::unordered_map). + FlatMap(FlatMap&& src) : rep_(std::move(src.rep_)) {} + + template + FlatMap(InputIter first, InputIter last, size_t N = 1, + const Hash& hf = Hash(), const Eq& eq = Eq()) + : FlatMap(N, hf, eq) { + insert(first, last); + } + + FlatMap(std::initializer_list> init, size_t N = 1, + const Hash& hf = Hash(), const Eq& eq = Eq()) + : FlatMap(init.begin(), init.end(), N, hf, eq) {} + + FlatMap& operator=(const FlatMap& src) { + rep_.CopyFrom(src.rep_); + return *this; + } + + // Move-assignment operator leaves src in a valid but unspecified state (same + // as std::unordered_map). + FlatMap& operator=(FlatMap&& src) { + rep_.MoveFrom(std::move(src.rep_)); + return *this; + } + + ~FlatMap() {} + + void swap(FlatMap& x) { rep_.swap(x.rep_); } + void clear_no_resize() { rep_.clear_no_resize(); } + void clear() { rep_.clear(); } + void reserve(size_t N) { rep_.Resize(std::max(N, size())); } + void rehash(size_t N) { rep_.Resize(std::max(N, size())); } + void resize(size_t N) { rep_.Resize(std::max(N, size())); } + size_t size() const { return rep_.size(); } + bool empty() const { return size() == 0; } + size_t bucket_count() const { return rep_.bucket_count(); } + hasher hash_function() const { return rep_.hash_function(); } + key_equal key_eq() const { return rep_.key_eq(); } + + class iterator { + public: + typedef typename FlatMap::difference_type difference_type; + typedef typename FlatMap::value_type value_type; + typedef typename FlatMap::pointer pointer; + typedef typename FlatMap::reference reference; + typedef ::std::forward_iterator_tag iterator_category; + + iterator() : b_(nullptr), end_(nullptr), i_(0) {} + + // Make iterator pointing at first element at or after b. + iterator(Bucket* b, Bucket* end) : b_(b), end_(end), i_(0) { SkipUnused(); } + + // Make iterator pointing exactly at ith element in b, which must exist. + iterator(Bucket* b, Bucket* end, uint32 i) : b_(b), end_(end), i_(i) { + FillValue(); + } + + reference operator*() { return *val(); } + pointer operator->() { return val(); } + bool operator==(const iterator& x) const { + return b_ == x.b_ && i_ == x.i_; + } + bool operator!=(const iterator& x) const { return !(*this == x); } + iterator& operator++() { + DCHECK(b_ != end_); + i_++; + SkipUnused(); + return *this; + } + iterator operator++(int /*indicates postfix*/) { + iterator tmp(*this); + ++*this; + return tmp; + } + + private: + friend class FlatMap; + Bucket* b_; + Bucket* end_; + char space_ alignas(value_type)[sizeof(value_type)]; + uint32 i_; + + pointer val() { return reinterpret_cast(space_); } + void FillValue() { new (space_) value_type(b_->key(i_), b_->val(i_)); } + void SkipUnused() { + while (b_ < end_) { + if (i_ >= Rep::kWidth) { + i_ = 0; + b_++; + } else if (b_->marker[i_] < 2) { + i_++; + } else { + FillValue(); + break; + } + } + } + }; + + class const_iterator { + private: + mutable iterator rep_; // Share state and logic with non-const iterator. + public: + typedef typename FlatMap::difference_type difference_type; + typedef typename FlatMap::value_type value_type; + typedef typename FlatMap::const_pointer pointer; + typedef typename FlatMap::const_reference reference; + typedef ::std::forward_iterator_tag iterator_category; + + const_iterator() : rep_() {} + const_iterator(Bucket* start, Bucket* end) : rep_(start, end) {} + const_iterator(Bucket* b, Bucket* end, uint32 i) : rep_(b, end, i) {} + + reference operator*() const { return *rep_.val(); } + pointer operator->() const { return rep_.val(); } + bool operator==(const const_iterator& x) const { return rep_ == x.rep_; } + bool operator!=(const const_iterator& x) const { return rep_ != x.rep_; } + const_iterator& operator++() { + ++rep_; + return *this; + } + const_iterator operator++(int /*indicates postfix*/) { + const_iterator tmp(*this); + ++*this; + return tmp; + } + }; + + iterator begin() { return iterator(rep_.start(), rep_.limit()); } + iterator end() { return iterator(rep_.limit(), rep_.limit()); } + const_iterator begin() const { + return const_iterator(rep_.start(), rep_.limit()); + } + const_iterator end() const { + return const_iterator(rep_.limit(), rep_.limit()); + } + + size_t count(const Key& k) const { return rep_.Find(k).found ? 1 : 0; } + iterator find(const Key& k) { + auto r = rep_.Find(k); + return r.found ? iterator(r.b, rep_.limit(), r.index) : end(); + } + const_iterator find(const Key& k) const { + auto r = rep_.Find(k); + return r.found ? const_iterator(r.b, rep_.limit(), r.index) : end(); + } + + Val& at(const Key& k) { + auto r = rep_.Find(k); + DCHECK(r.found); + return r.b->val(r.index); + } + const Val& at(const Key& k) const { + auto r = rep_.Find(k); + DCHECK(r.found); + return r.b->val(r.index); + } + + template + std::pair insert(const P& p) { + return Insert(p.first, p.second); + } + std::pair insert(const std::pair& p) { + return Insert(p.first, p.second); + } + template + void insert(InputIter first, InputIter last) { + for (; first != last; ++first) { + insert(*first); + } + } + + Val& operator[](const Key& k) { return IndexOp(k); } + Val& operator[](Key&& k) { return IndexOp(std::forward(k)); } + + template + std::pair emplace(Args&&... args) { + return InsertPair(std::make_pair(std::forward(args)...)); + } + + size_t erase(const Key& k) { + auto r = rep_.Find(k); + if (!r.found) return 0; + rep_.Erase(r.b, r.index); + return 1; + } + iterator erase(iterator pos) { + rep_.Erase(pos.b_, pos.i_); + ++pos; + return pos; + } + iterator erase(iterator pos, iterator last) { + for (; pos != last; ++pos) { + rep_.Erase(pos.b_, pos.i_); + } + return pos; + } + + std::pair equal_range(const Key& k) { + auto pos = find(k); + if (pos == end()) { + return std::make_pair(pos, pos); + } else { + auto next = pos; + ++next; + return std::make_pair(pos, next); + } + } + std::pair equal_range(const Key& k) const { + auto pos = find(k); + if (pos == end()) { + return std::make_pair(pos, pos); + } else { + auto next = pos; + ++next; + return std::make_pair(pos, next); + } + } + + bool operator==(const FlatMap& x) const { + if (size() != x.size()) return false; + for (auto& p : x) { + auto i = find(p.first); + if (i == end()) return false; + if (i->second != p.second) return false; + } + return true; + } + bool operator!=(const FlatMap& x) const { return !(*this == x); } + + // If key exists in the table, prefetch the associated value. This + // is a hint, and may have no effect. + void prefetch_value(const Key& key) const { rep_.Prefetch(key); } + + private: + using Rep = internal::FlatRep; + + // Bucket stores kWidth triples. + // The data is organized as three parallel arrays to reduce padding. + struct Bucket { + uint8 marker[Rep::kWidth]; + + // Wrap keys and values in union to control construction and destruction. + union Storage { + struct { + Key key[Rep::kWidth]; + Val val[Rep::kWidth]; + }; + Storage() {} + ~Storage() {} + } storage; + + Key& key(uint32 i) { + DCHECK_GE(marker[i], 2); + return storage.key[i]; + } + Val& val(uint32 i) { + DCHECK_GE(marker[i], 2); + return storage.val[i]; + } + template + void InitVal(uint32 i, V&& v) { + new (&storage.val[i]) Val(std::forward(v)); + } + void Destroy(uint32 i) { + storage.key[i].Key::~Key(); + storage.val[i].Val::~Val(); + } + void MoveFrom(uint32 i, Bucket* src, uint32 src_index) { + new (&storage.key[i]) Key(std::move(src->storage.key[src_index])); + new (&storage.val[i]) Val(std::move(src->storage.val[src_index])); + } + void CopyFrom(uint32 i, Bucket* src, uint32 src_index) { + new (&storage.key[i]) Key(src->storage.key[src_index]); + new (&storage.val[i]) Val(src->storage.val[src_index]); + } + }; + + template + std::pair InsertPair(Pair&& p) { + return Insert(std::forward(p.first), + std::forward(p.second)); + } + + template + std::pair Insert(K&& k, V&& v) { + rep_.MaybeResize(); + auto r = rep_.FindOrInsert(std::forward(k)); + const bool inserted = !r.found; + if (inserted) { + r.b->InitVal(r.index, std::forward(v)); + } + return {iterator(r.b, rep_.limit(), r.index), inserted}; + } + + template + Val& IndexOp(K&& k) { + rep_.MaybeResize(); + auto r = rep_.FindOrInsert(std::forward(k)); + Val* vptr = &r.b->val(r.index); + if (!r.found) { + new (vptr) Val(); // Initialize value in new slot. + } + return *vptr; + } + + Rep rep_; +}; + +} // namespace gtl +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_LIB_GTL_FLATMAP_H_ diff --git a/linux/include/tensorflow/core/lib/gtl/flatrep.h b/linux/include/tensorflow/core/lib/gtl/flatrep.h new file mode 100644 index 000000000..65a076b0f --- /dev/null +++ b/linux/include/tensorflow/core/lib/gtl/flatrep.h @@ -0,0 +1,350 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_LIB_GTL_FLATREP_H_ +#define TENSORFLOW_CORE_LIB_GTL_FLATREP_H_ + +#include +#include +#include "tensorflow/core/platform/prefetch.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { +namespace gtl { +namespace internal { + +// Internal representation for FlatMap and FlatSet. +// +// The representation is an open-addressed hash table. Conceptually, +// the representation is a flat array of entries. However we +// structure it as an array of buckets where each bucket holds +// kWidth entries along with metadata for the kWidth entries. The +// metadata marker is +// +// (a) kEmpty: the entry is empty +// (b) kDeleted: the entry has been deleted +// (c) other: the entry is occupied and has low-8 bits of its hash. +// These hash bits can be used to avoid potentially expensive +// key comparisons. +// +// FlatMap passes in a bucket that contains keys and values, FlatSet +// passes in a bucket that does not contain values. +template +class FlatRep { + public: + // kWidth is the number of entries stored in a bucket. + static const uint32 kBase = 3; + static const uint32 kWidth = (1 << kBase); + + FlatRep(size_t N, const Hash& hf, const Eq& eq) : hash_(hf), equal_(eq) { + Init(N); + } + FlatRep(const FlatRep& src) : hash_(src.hash_), equal_(src.equal_) { + Init(src.size()); + CopyEntries(src.array_, src.end_, CopyEntry()); + } + + FlatRep(FlatRep&& src) + // Copy rather than move src.hash_ and src.equal_. This is necessary to + // leave src in a valid state -- otherwise e.g. if hash_ is an + // std::function, moving it would null it out. + : hash_(src.hash_), equal_(src.equal_) { + // TODO(jlebar): Init(1) still allocates some memory, so this isn't as cheap + // as it could be. The fundamental problem is that we need to leave src in + // a valid state, and FlatRep *always* owns a nonzero amount of memory. + Init(1); + swap(src); + } + + ~FlatRep() { + clear_no_resize(); + delete[] array_; + } + + // Simple accessors. + size_t size() const { return not_empty_ - deleted_; } + size_t bucket_count() const { return mask_ + 1; } + Bucket* start() const { return array_; } + Bucket* limit() const { return end_; } + const Hash& hash_function() const { return hash_; } + const Eq& key_eq() const { return equal_; } + + // Overwrite contents of *this with contents of src. + void CopyFrom(const FlatRep& src) { + if (this != &src) { + clear_no_resize(); + delete[] array_; + Init(src.size()); + CopyEntries(src.array_, src.end_, CopyEntry()); + } + } + + void MoveFrom(FlatRep&& src) { + if (this != &src) { + swap(src); + } + } + + void clear_no_resize() { + for (Bucket* b = array_; b != end_; b++) { + for (uint32 i = 0; i < kWidth; i++) { + if (b->marker[i] >= 2) { + b->Destroy(i); + b->marker[i] = kEmpty; + } + } + } + not_empty_ = 0; + deleted_ = 0; + } + + void clear() { + clear_no_resize(); + grow_ = 0; // Consider shrinking in MaybeResize() + MaybeResize(); + } + + void swap(FlatRep& x) { + using std::swap; + swap(array_, x.array_); + swap(end_, x.end_); + swap(lglen_, x.lglen_); + swap(mask_, x.mask_); + swap(not_empty_, x.not_empty_); + swap(deleted_, x.deleted_); + swap(grow_, x.grow_); + swap(shrink_, x.shrink_); + } + + struct SearchResult { + bool found; + Bucket* b; + uint32 index; + }; + + // Hash value is partitioned as follows: + // 1. Bottom 8 bits are stored in bucket to help speed up comparisons. + // 2. Next 3 bits give index inside bucket. + // 3. Remaining bits give bucket number. + + // Find bucket/index for key k. + SearchResult Find(const Key& k) const { + size_t h = hash_(k); + const uint32 marker = Marker(h & 0xff); + size_t index = (h >> 8) & mask_; // Holds bucket num and index-in-bucket + uint32 num_probes = 1; // Needed for quadratic probing + while (true) { + uint32 bi = index & (kWidth - 1); + Bucket* b = &array_[index >> kBase]; + const uint32 x = b->marker[bi]; + if (x == marker && equal_(b->key(bi), k)) { + return {true, b, bi}; + } else if (x == kEmpty) { + return {false, nullptr, 0}; + } + index = NextIndex(index, num_probes); + num_probes++; + } + } + + // Find bucket/index for key k, creating a new one if necessary. + // + // KeyType is a template parameter so that k's type is deduced and it + // becomes a universal reference which allows the key initialization + // below to use an rvalue constructor if available. + template + SearchResult FindOrInsert(KeyType&& k) { + size_t h = hash_(k); + const uint32 marker = Marker(h & 0xff); + size_t index = (h >> 8) & mask_; // Holds bucket num and index-in-bucket + uint32 num_probes = 1; // Needed for quadratic probing + Bucket* del = nullptr; // First encountered deletion for kInsert + uint32 di = 0; + while (true) { + uint32 bi = index & (kWidth - 1); + Bucket* b = &array_[index >> kBase]; + const uint32 x = b->marker[bi]; + if (x == marker && equal_(b->key(bi), k)) { + return {true, b, bi}; + } else if (!del && x == kDeleted) { + // Remember deleted index to use for insertion. + del = b; + di = bi; + } else if (x == kEmpty) { + if (del) { + // Store in the first deleted slot we encountered + b = del; + bi = di; + deleted_--; // not_empty_ does not change + } else { + not_empty_++; + } + b->marker[bi] = marker; + new (&b->key(bi)) Key(std::forward(k)); + return {false, b, bi}; + } + index = NextIndex(index, num_probes); + num_probes++; + } + } + + void Erase(Bucket* b, uint32 i) { + b->Destroy(i); + b->marker[i] = kDeleted; + deleted_++; + grow_ = 0; // Consider shrinking on next insert + } + + void Prefetch(const Key& k) const { + size_t h = hash_(k); + size_t index = (h >> 8) & mask_; // Holds bucket num and index-in-bucket + uint32 bi = index & (kWidth - 1); + Bucket* b = &array_[index >> kBase]; + port::prefetch(&b->marker[bi]); + port::prefetch(&b->storage.key[bi]); + } + + inline void MaybeResize() { + if (not_empty_ < grow_) { + return; // Nothing to do + } + if (grow_ == 0) { + // Special value set by erase to cause shrink on next insert. + if (size() >= shrink_) { + // Not small enough to shrink. + grow_ = static_cast(bucket_count() * 0.8); + if (not_empty_ < grow_) return; + } + } + Resize(size() + 1); + } + + void Resize(size_t N) { + Bucket* old = array_; + Bucket* old_end = end_; + Init(N); + CopyEntries(old, old_end, MoveEntry()); + delete[] old; + } + + private: + enum { kEmpty = 0, kDeleted = 1 }; // Special markers for an entry. + + Hash hash_; // User-supplied hasher + Eq equal_; // User-supplied comparator + uint8 lglen_; // lg(#buckets) + Bucket* array_; // array of length (1 << lglen_) + Bucket* end_; // Points just past last bucket in array_ + size_t mask_; // (# of entries in table) - 1 + size_t not_empty_; // Count of entries with marker != kEmpty + size_t deleted_; // Count of entries with marker == kDeleted + size_t grow_; // Grow array when not_empty_ >= grow_ + size_t shrink_; // Shrink array when size() < shrink_ + + // Avoid kEmpty and kDeleted markers when computing hash values to + // store in Bucket::marker[]. + static uint32 Marker(uint32 hb) { return hb + (hb < 2 ? 2 : 0); } + + void Init(size_t N) { + // Make enough room for N elements. + size_t lg = 0; // Smallest table is just one bucket. + while (N >= 0.8 * ((1 << lg) * kWidth)) { + lg++; + } + const size_t n = (1 << lg); + Bucket* array = new Bucket[n]; + for (size_t i = 0; i < n; i++) { + Bucket* b = &array[i]; + memset(b->marker, kEmpty, kWidth); + } + const size_t capacity = (1 << lg) * kWidth; + lglen_ = lg; + mask_ = capacity - 1; + array_ = array; + end_ = array + n; + not_empty_ = 0; + deleted_ = 0; + grow_ = static_cast(capacity * 0.8); + if (lg == 0) { + // Already down to one bucket; no more shrinking. + shrink_ = 0; + } else { + shrink_ = static_cast(grow_ * 0.4); // Must be less than 0.5 + } + } + + // Used by FreshInsert when we should copy from source. + struct CopyEntry { + inline void operator()(Bucket* dst, uint32 dsti, Bucket* src, uint32 srci) { + dst->CopyFrom(dsti, src, srci); + } + }; + + // Used by FreshInsert when we should move from source. + struct MoveEntry { + inline void operator()(Bucket* dst, uint32 dsti, Bucket* src, uint32 srci) { + dst->MoveFrom(dsti, src, srci); + src->Destroy(srci); + src->marker[srci] = kDeleted; + } + }; + + template + void CopyEntries(Bucket* start, Bucket* end, Copier copier) { + for (Bucket* b = start; b != end; b++) { + for (uint32 i = 0; i < kWidth; i++) { + if (b->marker[i] >= 2) { + FreshInsert(b, i, copier); + } + } + } + } + + // Create an entry for the key numbered src_index in *src and return + // its bucket/index. Used for insertion into a fresh table. We + // assume that there are no deletions, and k does not already exist + // in the table. + template + void FreshInsert(Bucket* src, uint32 src_index, Copier copier) { + size_t h = hash_(src->key(src_index)); + const uint32 marker = Marker(h & 0xff); + size_t index = (h >> 8) & mask_; // Holds bucket num and index-in-bucket + uint32 num_probes = 1; // Needed for quadratic probing + while (true) { + uint32 bi = index & (kWidth - 1); + Bucket* b = &array_[index >> kBase]; + const uint32 x = b->marker[bi]; + if (x == 0) { + b->marker[bi] = marker; + not_empty_++; + copier(b, bi, src, src_index); + return; + } + index = NextIndex(index, num_probes); + num_probes++; + } + } + + inline size_t NextIndex(size_t i, uint32 num_probes) const { + // Quadratic probing. + return (i + num_probes) & mask_; + } +}; + +} // namespace internal +} // namespace gtl +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_LIB_GTL_FLATREP_H_ diff --git a/linux/include/tensorflow/core/lib/gtl/flatset.h b/linux/include/tensorflow/core/lib/gtl/flatset.h new file mode 100644 index 000000000..bb4356e46 --- /dev/null +++ b/linux/include/tensorflow/core/lib/gtl/flatset.h @@ -0,0 +1,294 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_LIB_GTL_FLATSET_H_ +#define TENSORFLOW_CORE_LIB_GTL_FLATSET_H_ + +#include +#include +#include +#include +#include +#include "tensorflow/core/lib/gtl/flatrep.h" +#include "tensorflow/core/lib/hash/hash.h" +#include "tensorflow/core/platform/logging.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { +namespace gtl { + +// FlatSet provides a set of K. +// +// The map is implemented using an open-addressed hash table. A +// single array holds entire map contents and collisions are resolved +// by probing at a sequence of locations in the array. +template , class Eq = std::equal_to> +class FlatSet { + private: + // Forward declare some internal types needed in public section. + struct Bucket; + + public: + typedef Key key_type; + typedef Key value_type; + typedef Hash hasher; + typedef Eq key_equal; + typedef size_t size_type; + typedef ptrdiff_t difference_type; + typedef value_type* pointer; + typedef const value_type* const_pointer; + typedef value_type& reference; + typedef const value_type& const_reference; + + FlatSet() : FlatSet(1) {} + + explicit FlatSet(size_t N, const Hash& hf = Hash(), const Eq& eq = Eq()) + : rep_(N, hf, eq) {} + + FlatSet(const FlatSet& src) : rep_(src.rep_) {} + + // Move constructor leaves src in a valid but unspecified state (same as + // std::unordered_set). + FlatSet(FlatSet&& src) : rep_(std::move(src.rep_)) {} + + template + FlatSet(InputIter first, InputIter last, size_t N = 1, + const Hash& hf = Hash(), const Eq& eq = Eq()) + : FlatSet(N, hf, eq) { + insert(first, last); + } + + FlatSet(std::initializer_list init, size_t N = 1, + const Hash& hf = Hash(), const Eq& eq = Eq()) + : FlatSet(init.begin(), init.end(), N, hf, eq) {} + + FlatSet& operator=(const FlatSet& src) { + rep_.CopyFrom(src.rep_); + return *this; + } + + // Move-assignment operator leaves src in a valid but unspecified state (same + // as std::unordered_set). + FlatSet& operator=(FlatSet&& src) { + rep_.MoveFrom(std::move(src.rep_)); + return *this; + } + + ~FlatSet() {} + + void swap(FlatSet& x) { rep_.swap(x.rep_); } + void clear_no_resize() { rep_.clear_no_resize(); } + void clear() { rep_.clear(); } + void reserve(size_t N) { rep_.Resize(std::max(N, size())); } + void rehash(size_t N) { rep_.Resize(std::max(N, size())); } + void resize(size_t N) { rep_.Resize(std::max(N, size())); } + size_t size() const { return rep_.size(); } + bool empty() const { return size() == 0; } + size_t bucket_count() const { return rep_.bucket_count(); } + hasher hash_function() const { return rep_.hash_function(); } + key_equal key_eq() const { return rep_.key_eq(); } + + class const_iterator { + public: + typedef typename FlatSet::difference_type difference_type; + typedef typename FlatSet::value_type value_type; + typedef typename FlatSet::const_pointer pointer; + typedef typename FlatSet::const_reference reference; + typedef ::std::forward_iterator_tag iterator_category; + + const_iterator() : b_(nullptr), end_(nullptr), i_(0) {} + + // Make iterator pointing at first element at or after b. + const_iterator(Bucket* b, Bucket* end) : b_(b), end_(end), i_(0) { + SkipUnused(); + } + + // Make iterator pointing exactly at ith element in b, which must exist. + const_iterator(Bucket* b, Bucket* end, uint32 i) + : b_(b), end_(end), i_(i) {} + + reference operator*() const { return key(); } + pointer operator->() const { return &key(); } + bool operator==(const const_iterator& x) const { + return b_ == x.b_ && i_ == x.i_; + } + bool operator!=(const const_iterator& x) const { return !(*this == x); } + const_iterator& operator++() { + DCHECK(b_ != end_); + i_++; + SkipUnused(); + return *this; + } + const_iterator operator++(int /*indicates postfix*/) { + const_iterator tmp(*this); + ++*this; + return tmp; + } + + private: + friend class FlatSet; + Bucket* b_; + Bucket* end_; + uint32 i_; + + reference key() const { return b_->key(i_); } + void SkipUnused() { + while (b_ < end_) { + if (i_ >= Rep::kWidth) { + i_ = 0; + b_++; + } else if (b_->marker[i_] < 2) { + i_++; + } else { + break; + } + } + } + }; + + typedef const_iterator iterator; + + iterator begin() { return iterator(rep_.start(), rep_.limit()); } + iterator end() { return iterator(rep_.limit(), rep_.limit()); } + const_iterator begin() const { + return const_iterator(rep_.start(), rep_.limit()); + } + const_iterator end() const { + return const_iterator(rep_.limit(), rep_.limit()); + } + + size_t count(const Key& k) const { return rep_.Find(k).found ? 1 : 0; } + iterator find(const Key& k) { + auto r = rep_.Find(k); + return r.found ? iterator(r.b, rep_.limit(), r.index) : end(); + } + const_iterator find(const Key& k) const { + auto r = rep_.Find(k); + return r.found ? const_iterator(r.b, rep_.limit(), r.index) : end(); + } + + std::pair insert(const Key& k) { return Insert(k); } + std::pair insert(Key&& k) { return Insert(std::move(k)); } + template + void insert(InputIter first, InputIter last) { + for (; first != last; ++first) { + insert(*first); + } + } + + template + std::pair emplace(Args&&... args) { + rep_.MaybeResize(); + auto r = rep_.FindOrInsert(std::forward(args)...); + const bool inserted = !r.found; + return {iterator(r.b, rep_.limit(), r.index), inserted}; + } + + size_t erase(const Key& k) { + auto r = rep_.Find(k); + if (!r.found) return 0; + rep_.Erase(r.b, r.index); + return 1; + } + iterator erase(iterator pos) { + rep_.Erase(pos.b_, pos.i_); + ++pos; + return pos; + } + iterator erase(iterator pos, iterator last) { + for (; pos != last; ++pos) { + rep_.Erase(pos.b_, pos.i_); + } + return pos; + } + + std::pair equal_range(const Key& k) { + auto pos = find(k); + if (pos == end()) { + return std::make_pair(pos, pos); + } else { + auto next = pos; + ++next; + return std::make_pair(pos, next); + } + } + std::pair equal_range(const Key& k) const { + auto pos = find(k); + if (pos == end()) { + return std::make_pair(pos, pos); + } else { + auto next = pos; + ++next; + return std::make_pair(pos, next); + } + } + + bool operator==(const FlatSet& x) const { + if (size() != x.size()) return false; + for (const auto& elem : x) { + auto i = find(elem); + if (i == end()) return false; + } + return true; + } + bool operator!=(const FlatSet& x) const { return !(*this == x); } + + // If key exists in the table, prefetch it. This is a hint, and may + // have no effect. + void prefetch_value(const Key& key) const { rep_.Prefetch(key); } + + private: + using Rep = internal::FlatRep; + + // Bucket stores kWidth triples. + // The data is organized as three parallel arrays to reduce padding. + struct Bucket { + uint8 marker[Rep::kWidth]; + + // Wrap keys in union to control construction and destruction. + union Storage { + Key key[Rep::kWidth]; + Storage() {} + ~Storage() {} + } storage; + + Key& key(uint32 i) { + DCHECK_GE(marker[i], 2); + return storage.key[i]; + } + void Destroy(uint32 i) { storage.key[i].Key::~Key(); } + void MoveFrom(uint32 i, Bucket* src, uint32 src_index) { + new (&storage.key[i]) Key(std::move(src->storage.key[src_index])); + } + void CopyFrom(uint32 i, Bucket* src, uint32 src_index) { + new (&storage.key[i]) Key(src->storage.key[src_index]); + } + }; + + template + std::pair Insert(K&& k) { + rep_.MaybeResize(); + auto r = rep_.FindOrInsert(std::forward(k)); + const bool inserted = !r.found; + return {iterator(r.b, rep_.limit(), r.index), inserted}; + } + + Rep rep_; +}; + +} // namespace gtl +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_LIB_GTL_FLATSET_H_ diff --git a/linux/include/tensorflow/core/lib/gtl/inlined_vector.h b/linux/include/tensorflow/core/lib/gtl/inlined_vector.h new file mode 100644 index 000000000..2d622dc22 --- /dev/null +++ b/linux/include/tensorflow/core/lib/gtl/inlined_vector.h @@ -0,0 +1,33 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_LIB_GTL_INLINED_VECTOR_H_ +#define TENSORFLOW_CORE_LIB_GTL_INLINED_VECTOR_H_ + +#include "absl/container/inlined_vector.h" +// TODO(kramerb): This is kept only because lots of targets transitively depend +// on it. Remove all targets' dependencies. +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { +namespace gtl { + +using absl::InlinedVector; + +} // namespace gtl +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_LIB_GTL_INLINED_VECTOR_H_ diff --git a/linux/include/tensorflow/core/lib/gtl/int_type.h b/linux/include/tensorflow/core/lib/gtl/int_type.h new file mode 100644 index 000000000..af3e50ad7 --- /dev/null +++ b/linux/include/tensorflow/core/lib/gtl/int_type.h @@ -0,0 +1,359 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// #status: LEGACY +// #category: Miscellaneous +// #summary: Integral types; prefer util/intops/strong_int.h +// #bugs: Infrastructure > C++ Library Team > util +// +// IntType is a simple template class mechanism for defining "logical" +// integer-like class types that support many of the same functionalities +// as native integer types, but which prevent assignment, construction, and +// other operations from other similar integer-like types. Essentially, the +// template class IntType (where ValueType assumes +// valid scalar types such as int, uint, int32, etc) has the additional +// property that it cannot be assigned to or constructed from other IntTypes +// or native integer types of equal or implicitly convertible type. +// +// The class is useful for preventing mingling of integer variables with +// different logical roles or units. Unfortunately, C++ provides relatively +// good type-safety for user-defined classes but not for integer types. It is +// essentially up to the user to use nice variable names and comments to prevent +// accidental mismatches, such as confusing a user-index with a group-index or a +// time-in-milliseconds with a time-in-seconds. The use of typedefs are limited +// in that regard as they do not enforce type-safety. +// +// USAGE ----------------------------------------------------------------------- +// +// DEFINE_INT_TYPE(IntTypeName, ValueType); +// +// where: +// IntTypeName: is the desired (unique) name for the "logical" integer type. +// ValueType: is one of the integral types as defined by base::is_integral +// (see base/type_traits.h). +// +// DISALLOWED OPERATIONS / TYPE-SAFETY ENFORCEMENT ----------------------------- +// +// Consider these definitions and variable declarations: +// DEFINE_INT_TYPE(GlobalDocID, int64); +// DEFINE_INT_TYPE(LocalDocID, int64); +// GlobalDocID global; +// LocalDocID local; +// +// The class IntType prevents: +// +// 1) Assignments of other IntTypes with different IntTypeNames. +// +// global = local; <-- Fails to compile! +// local = global; <-- Fails to compile! +// +// 2) Explicit/implicit conversion from an IntType to another IntType. +// +// LocalDocID l(global); <-- Fails to compile! +// LocalDocID l = global; <-- Fails to compile! +// +// void GetGlobalDoc(GlobalDocID global) { } +// GetGlobalDoc(global); <-- Compiles fine, types match! +// GetGlobalDoc(local); <-- Fails to compile! +// +// 3) Implicit conversion from an IntType to a native integer type. +// +// void GetGlobalDoc(int64 global) { ... +// GetGlobalDoc(global); <-- Fails to compile! +// GetGlobalDoc(local); <-- Fails to compile! +// +// void GetLocalDoc(int32 local) { ... +// GetLocalDoc(global); <-- Fails to compile! +// GetLocalDoc(local); <-- Fails to compile! +// +// +// SUPPORTED OPERATIONS -------------------------------------------------------- +// +// The following operators are supported: unary: ++ (both prefix and postfix), +// +, -, ! (logical not), ~ (one's complement); comparison: ==, !=, <, <=, >, +// >=; numerical: +, -, *, /; assignment: =, +=, -=, /=, *=; stream: <<. Each +// operator allows the same IntTypeName and the ValueType to be used on +// both left- and right-hand sides. +// +// It also supports an accessor value() returning the stored value as ValueType, +// and a templatized accessor value() method that serves as syntactic sugar +// for static_cast(var.value()). These accessors are useful when assigning +// the stored value into protocol buffer fields and using it as printf args. +// +// The class also defines a hash functor that allows the IntType to be used +// as key to hashable containers such as std::unordered_map and +// std::unordered_set. +// +// We suggest using the IntTypeIndexedContainer wrapper around FixedArray and +// STL vector (see int-type-indexed-container.h) if an IntType is intended to +// be used as an index into these containers. These wrappers are indexed in a +// type-safe manner using IntTypes to ensure type-safety. +// +// NB: this implementation does not attempt to abide by or enforce dimensional +// analysis on these scalar types. +// +// EXAMPLES -------------------------------------------------------------------- +// +// DEFINE_INT_TYPE(GlobalDocID, int64); +// GlobalDocID global = 3; +// cout << global; <-- Prints 3 to stdout. +// +// for (GlobalDocID i(0); i < global; ++i) { +// cout << i; +// } <-- Print(ln)s 0 1 2 to stdout +// +// DEFINE_INT_TYPE(LocalDocID, int64); +// LocalDocID local; +// cout << local; <-- Prints 0 to stdout it default +// initializes the value to 0. +// +// local = 5; +// local *= 2; +// LocalDocID l(local); +// cout << l + local; <-- Prints 20 to stdout. +// +// GenericSearchRequest request; +// request.set_doc_id(global.value()); <-- Uses value() to extract the value +// from the IntType class. +// +// REMARKS --------------------------------------------------------------------- +// +// The following bad usage is permissible although discouraged. Essentially, it +// involves using the value*() accessors to extract the native integer type out +// of the IntType class. Keep in mind that the primary reason for the IntType +// class is to prevent *accidental* mingling of similar logical integer types -- +// and not type casting from one type to another. +// +// DEFINE_INT_TYPE(GlobalDocID, int64); +// DEFINE_INT_TYPE(LocalDocID, int64); +// GlobalDocID global; +// LocalDocID local; +// +// global = local.value(); <-- Compiles fine. +// +// void GetGlobalDoc(GlobalDocID global) { ... +// GetGlobalDoc(local.value()); <-- Compiles fine. +// +// void GetGlobalDoc(int64 global) { ... +// GetGlobalDoc(local.value()); <-- Compiles fine. + +#ifndef TENSORFLOW_LIB_GTL_INT_TYPE_H_ +#define TENSORFLOW_LIB_GTL_INT_TYPE_H_ + +#include +#include +#include +#include // NOLINT +#include + +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { +namespace gtl { + +template +class IntType; + +// Defines the IntType using value_type and typedefs it to int_type_name. +// The struct int_type_name ## _tag_ trickery is needed to ensure that a new +// type is created per int_type_name. +#define TF_LIB_GTL_DEFINE_INT_TYPE(int_type_name, value_type) \ + struct int_type_name##_tag_ {}; \ + typedef ::tensorflow::gtl::IntType \ + int_type_name; + +// Holds an integer value (of type ValueType) and behaves as a ValueType by +// exposing assignment, unary, comparison, and arithmetic operators. +// +// The template parameter IntTypeName defines the name for the int type and must +// be unique within a binary (the convenient DEFINE_INT_TYPE macro at the end of +// the file generates a unique IntTypeName). The parameter ValueType defines +// the integer type value (see supported list above). +// +// This class is NOT thread-safe. +template +class IntType { + public: + typedef _ValueType ValueType; // for non-member operators + typedef IntType ThisType; // Syntactic sugar. + + // Note that this may change from time to time without notice. + struct Hasher { + size_t operator()(const IntType& arg) const { + return static_cast(arg.value()); + } + }; + + public: + // Default c'tor initializing value_ to 0. + constexpr IntType() : value_(0) {} + // C'tor explicitly initializing from a ValueType. + constexpr explicit IntType(ValueType value) : value_(value) {} + + // IntType uses the default copy constructor, destructor and assign operator. + // The defaults are sufficient and omitting them allows the compiler to add + // the move constructor/assignment. + + // -- ACCESSORS -------------------------------------------------------------- + // The class provides a value() accessor returning the stored ValueType value_ + // as well as a templatized accessor that is just a syntactic sugar for + // static_cast(var.value()); + constexpr ValueType value() const { return value_; } + + template + constexpr ValType value() const { + return static_cast(value_); + } + + // -- UNARY OPERATORS -------------------------------------------------------- + ThisType& operator++() { // prefix ++ + ++value_; + return *this; + } + const ThisType operator++(int v) { // postfix ++ + ThisType temp(*this); + ++value_; + return temp; + } + ThisType& operator--() { // prefix -- + --value_; + return *this; + } + const ThisType operator--(int v) { // postfix -- + ThisType temp(*this); + --value_; + return temp; + } + + constexpr bool operator!() const { return value_ == 0; } + constexpr const ThisType operator+() const { return ThisType(value_); } + constexpr const ThisType operator-() const { return ThisType(-value_); } + constexpr const ThisType operator~() const { return ThisType(~value_); } + +// -- ASSIGNMENT OPERATORS --------------------------------------------------- +// We support the following assignment operators: =, +=, -=, *=, /=, <<=, >>= +// and %= for both ThisType and ValueType. +#define INT_TYPE_ASSIGNMENT_OP(op) \ + ThisType& operator op(const ThisType& arg_value) { \ + value_ op arg_value.value(); \ + return *this; \ + } \ + ThisType& operator op(ValueType arg_value) { \ + value_ op arg_value; \ + return *this; \ + } + INT_TYPE_ASSIGNMENT_OP(+=); + INT_TYPE_ASSIGNMENT_OP(-=); + INT_TYPE_ASSIGNMENT_OP(*=); + INT_TYPE_ASSIGNMENT_OP(/=); + INT_TYPE_ASSIGNMENT_OP(<<=); // NOLINT + INT_TYPE_ASSIGNMENT_OP(>>=); // NOLINT + INT_TYPE_ASSIGNMENT_OP(%=); +#undef INT_TYPE_ASSIGNMENT_OP + + ThisType& operator=(ValueType arg_value) { + value_ = arg_value; + return *this; + } + + private: + // The integer value of type ValueType. + ValueType value_; + + static_assert(std::is_integral::value, "invalid integer type"); +} TF_PACKED; + +// -- NON-MEMBER STREAM OPERATORS ---------------------------------------------- +// We provide the << operator, primarily for logging purposes. Currently, there +// seems to be no need for an >> operator. +template +std::ostream& operator<<(std::ostream& os, // NOLINT + IntType arg) { + return os << arg.value(); +} + +// -- NON-MEMBER ARITHMETIC OPERATORS ------------------------------------------ +// We support only the +, -, *, and / operators with the same IntType and +// ValueType types. The reason is to allow simple manipulation on these IDs +// when used as indices in vectors and arrays. +// +// NB: Although it is possible to do IntType * IntType and IntType / IntType, +// it is probably non-sensical from a dimensionality analysis perspective. +#define INT_TYPE_ARITHMETIC_OP(op) \ + template \ + static inline constexpr IntType operator op( \ + IntType id_1, \ + IntType id_2) { \ + return IntType(id_1.value() op id_2.value()); \ + } \ + template \ + static inline constexpr IntType operator op( \ + IntType id, \ + typename IntType::ValueType arg_val) { \ + return IntType(id.value() op arg_val); \ + } \ + template \ + static inline constexpr IntType operator op( \ + typename IntType::ValueType arg_val, \ + IntType id) { \ + return IntType(arg_val op id.value()); \ + } +INT_TYPE_ARITHMETIC_OP(+); +INT_TYPE_ARITHMETIC_OP(-); +INT_TYPE_ARITHMETIC_OP(*); +INT_TYPE_ARITHMETIC_OP(/); +INT_TYPE_ARITHMETIC_OP(<<); // NOLINT +INT_TYPE_ARITHMETIC_OP(>>); // NOLINT +INT_TYPE_ARITHMETIC_OP(%); +#undef INT_TYPE_ARITHMETIC_OP + +// -- NON-MEMBER COMPARISON OPERATORS ------------------------------------------ +// Static inline comparison operators. We allow all comparison operators among +// the following types (OP \in [==, !=, <, <=, >, >=]: +// IntType OP IntType +// IntType OP ValueType +// ValueType OP IntType +#define INT_TYPE_COMPARISON_OP(op) \ + template \ + static inline constexpr bool operator op( \ + IntType id_1, \ + IntType id_2) { \ + return id_1.value() op id_2.value(); \ + } \ + template \ + static inline constexpr bool operator op( \ + IntType id, \ + typename IntType::ValueType val) { \ + return id.value() op val; \ + } \ + template \ + static inline constexpr bool operator op( \ + typename IntType::ValueType val, \ + IntType id) { \ + return val op id.value(); \ + } +INT_TYPE_COMPARISON_OP(==); // NOLINT +INT_TYPE_COMPARISON_OP(!=); // NOLINT +INT_TYPE_COMPARISON_OP(<); // NOLINT +INT_TYPE_COMPARISON_OP(<=); // NOLINT +INT_TYPE_COMPARISON_OP(>); // NOLINT +INT_TYPE_COMPARISON_OP(>=); // NOLINT +#undef INT_TYPE_COMPARISON_OP + +} // namespace gtl +} // namespace tensorflow + +#endif // TENSORFLOW_LIB_GTL_INT_TYPE_H_ diff --git a/linux/include/tensorflow/core/lib/gtl/iterator_range.h b/linux/include/tensorflow/core/lib/gtl/iterator_range.h new file mode 100644 index 000000000..0ba4587fd --- /dev/null +++ b/linux/include/tensorflow/core/lib/gtl/iterator_range.h @@ -0,0 +1,68 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// This provides a very simple, boring adaptor for a begin and end iterator +// into a range type. This should be used to build range views that work well +// with range based for loops and range based constructors. +// +// Note that code here follows more standards-based coding conventions as it +// is mirroring proposed interfaces for standardization. +// +// Converted from chandlerc@'s code to Google style by joshl@. + +#ifndef TENSORFLOW_LIB_GTL_ITERATOR_RANGE_H_ +#define TENSORFLOW_LIB_GTL_ITERATOR_RANGE_H_ + +#include + +namespace tensorflow { +namespace gtl { + +// A range adaptor for a pair of iterators. +// +// This just wraps two iterators into a range-compatible interface. Nothing +// fancy at all. +template +class iterator_range { + public: + using value_type = decltype(*std::declval()); + using iterator = IteratorT; + using const_iterator = IteratorT; + + iterator_range() : begin_iterator_(), end_iterator_() {} + iterator_range(IteratorT begin_iterator, IteratorT end_iterator) + : begin_iterator_(std::move(begin_iterator)), + end_iterator_(std::move(end_iterator)) {} + + IteratorT begin() const { return begin_iterator_; } + IteratorT end() const { return end_iterator_; } + + private: + IteratorT begin_iterator_, end_iterator_; +}; + +// Convenience function for iterating over sub-ranges. +// +// This provides a bit of syntactic sugar to make using sub-ranges +// in for loops a bit easier. Analogous to std::make_pair(). +template +iterator_range make_range(T x, T y) { + return iterator_range(std::move(x), std::move(y)); +} + +} // namespace gtl +} // namespace tensorflow + +#endif // TENSORFLOW_LIB_GTL_ITERATOR_RANGE_H_ diff --git a/linux/include/tensorflow/core/lib/gtl/manual_constructor.h b/linux/include/tensorflow/core/lib/gtl/manual_constructor.h new file mode 100644 index 000000000..0176cdc94 --- /dev/null +++ b/linux/include/tensorflow/core/lib/gtl/manual_constructor.h @@ -0,0 +1,246 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// ManualConstructor statically-allocates space in which to store some +// object, but does not initialize it. You can then call the constructor +// and destructor for the object yourself as you see fit. This is useful +// for memory management optimizations, where you want to initialize and +// destroy an object multiple times but only allocate it once. +// +// (When I say ManualConstructor statically allocates space, I mean that +// the ManualConstructor object itself is forced to be the right size.) + +#ifndef TENSORFLOW_LIB_GTL_MANUAL_CONSTRUCTOR_H_ +#define TENSORFLOW_LIB_GTL_MANUAL_CONSTRUCTOR_H_ + +#include +#include +#include + +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/mem.h" + +namespace tensorflow { +namespace gtl { +namespace internal { + +// +// Provides a char array with the exact same alignment as another type. The +// first parameter must be a complete type, the second parameter is how many +// of that type to provide space for. +// +// TF_LIB_GTL_ALIGNED_CHAR_ARRAY(struct stat, 16) storage_; +// +// Because MSVC and older GCCs require that the argument to their alignment +// construct to be a literal constant integer, we use a template instantiated +// at all the possible powers of two. +#ifndef SWIG +template +struct AlignType {}; +template +struct AlignType<0, size> { + typedef char result[size]; +}; +#if defined(_MSC_VER) +#define TF_LIB_GTL_ALIGN_ATTRIBUTE(X) __declspec(align(X)) +#define TF_LIB_GTL_ALIGN_OF(T) __alignof(T) +#elif defined(COMPILER_GCC3) || __GNUC__ >= 3 || defined(__APPLE__) || \ + defined(COMPILER_ICC) || defined(OS_NACL) || defined(__clang__) +#define TF_LIB_GTL_ALIGN_ATTRIBUTE(X) __attribute__((aligned(X))) +#define TF_LIB_GTL_ALIGN_OF(T) __alignof__(T) +#endif + +#if defined(TF_LIB_GTL_ALIGN_ATTRIBUTE) + +#define TF_LIB_GTL_ALIGNTYPE_TEMPLATE(X) \ + template \ + struct AlignType { \ + typedef TF_LIB_GTL_ALIGN_ATTRIBUTE(X) char result[size]; \ + } + +TF_LIB_GTL_ALIGNTYPE_TEMPLATE(1); +TF_LIB_GTL_ALIGNTYPE_TEMPLATE(2); +TF_LIB_GTL_ALIGNTYPE_TEMPLATE(4); +TF_LIB_GTL_ALIGNTYPE_TEMPLATE(8); +TF_LIB_GTL_ALIGNTYPE_TEMPLATE(16); +TF_LIB_GTL_ALIGNTYPE_TEMPLATE(32); +TF_LIB_GTL_ALIGNTYPE_TEMPLATE(64); +TF_LIB_GTL_ALIGNTYPE_TEMPLATE(128); +TF_LIB_GTL_ALIGNTYPE_TEMPLATE(256); +TF_LIB_GTL_ALIGNTYPE_TEMPLATE(512); +TF_LIB_GTL_ALIGNTYPE_TEMPLATE(1024); +TF_LIB_GTL_ALIGNTYPE_TEMPLATE(2048); +TF_LIB_GTL_ALIGNTYPE_TEMPLATE(4096); +TF_LIB_GTL_ALIGNTYPE_TEMPLATE(8192); +// Any larger and MSVC++ will complain. + +#define TF_LIB_GTL_ALIGNED_CHAR_ARRAY(T, Size) \ + typename tensorflow::gtl::internal::AlignType::result + +#undef TF_LIB_GTL_ALIGNTYPE_TEMPLATE +#undef TF_LIB_GTL_ALIGN_ATTRIBUTE + +#else // defined(TF_LIB_GTL_ALIGN_ATTRIBUTE) +#error "You must define TF_LIB_GTL_ALIGNED_CHAR_ARRAY for your compiler." +#endif // defined(TF_LIB_GTL_ALIGN_ATTRIBUTE) + +#else // !SWIG + +// SWIG can't represent alignment and doesn't care about alignment on data +// members (it works fine without it). +template +struct AlignType { + typedef char result[Size]; +}; +#define TF_LIB_GTL_ALIGNED_CHAR_ARRAY(T, Size) \ + tensorflow::gtl::internal::AlignType::result + +// Enough to parse with SWIG, will never be used by running code. +#define TF_LIB_GTL_ALIGN_OF(Type) 16 + +#endif // !SWIG + +} // namespace internal +} // namespace gtl + +template +class ManualConstructor { + public: + // No constructor or destructor because one of the most useful uses of + // this class is as part of a union, and members of a union cannot have + // constructors or destructors. And, anyway, the whole point of this + // class is to bypass these. + + // Support users creating arrays of ManualConstructor<>s. This ensures that + // the array itself has the correct alignment. + static void* operator new[](size_t size) { + return port::AlignedMalloc(size, TF_LIB_GTL_ALIGN_OF(Type)); + } + static void operator delete[](void* mem) { port::AlignedFree(mem); } + + inline Type* get() { return reinterpret_cast(space_); } + inline const Type* get() const { + return reinterpret_cast(space_); + } + + inline Type* operator->() { return get(); } + inline const Type* operator->() const { return get(); } + + inline Type& operator*() { return *get(); } + inline const Type& operator*() const { return *get(); } + + inline void Init() { new (space_) Type; } + +// Init() constructs the Type instance using the given arguments +// (which are forwarded to Type's constructor). In C++11, Init() can +// take any number of arguments of any type, and forwards them perfectly. +// On pre-C++11 platforms, it can take up to 11 arguments, and may not be +// able to forward certain kinds of arguments. +// +// Note that Init() with no arguments performs default-initialization, +// not zero-initialization (i.e it behaves the same as "new Type;", not +// "new Type();"), so it will leave non-class types uninitialized. +#ifdef LANG_CXX11 + template + inline void Init(Ts&&... args) { // NOLINT + new (space_) Type(std::forward(args)...); // NOLINT + } +#else // !defined(LANG_CXX11) + template + inline void Init(const T1& p1) { + new (space_) Type(p1); + } + + template + inline void Init(const T1& p1, const T2& p2) { + new (space_) Type(p1, p2); + } + + template + inline void Init(const T1& p1, const T2& p2, const T3& p3) { + new (space_) Type(p1, p2, p3); + } + + template + inline void Init(const T1& p1, const T2& p2, const T3& p3, const T4& p4) { + new (space_) Type(p1, p2, p3, p4); + } + + template + inline void Init(const T1& p1, const T2& p2, const T3& p3, const T4& p4, + const T5& p5) { + new (space_) Type(p1, p2, p3, p4, p5); + } + + template + inline void Init(const T1& p1, const T2& p2, const T3& p3, const T4& p4, + const T5& p5, const T6& p6) { + new (space_) Type(p1, p2, p3, p4, p5, p6); + } + + template + inline void Init(const T1& p1, const T2& p2, const T3& p3, const T4& p4, + const T5& p5, const T6& p6, const T7& p7) { + new (space_) Type(p1, p2, p3, p4, p5, p6, p7); + } + + template + inline void Init(const T1& p1, const T2& p2, const T3& p3, const T4& p4, + const T5& p5, const T6& p6, const T7& p7, const T8& p8) { + new (space_) Type(p1, p2, p3, p4, p5, p6, p7, p8); + } + + template + inline void Init(const T1& p1, const T2& p2, const T3& p3, const T4& p4, + const T5& p5, const T6& p6, const T7& p7, const T8& p8, + const T9& p9) { + new (space_) Type(p1, p2, p3, p4, p5, p6, p7, p8, p9); + } + + template + inline void Init(const T1& p1, const T2& p2, const T3& p3, const T4& p4, + const T5& p5, const T6& p6, const T7& p7, const T8& p8, + const T9& p9, const T10& p10) { + new (space_) Type(p1, p2, p3, p4, p5, p6, p7, p8, p9, p10); + } + + template + inline void Init(const T1& p1, const T2& p2, const T3& p3, const T4& p4, + const T5& p5, const T6& p6, const T7& p7, const T8& p8, + const T9& p9, const T10& p10, const T11& p11) { + new (space_) Type(p1, p2, p3, p4, p5, p6, p7, p8, p9, p10, p11); + } +#endif // LANG_CXX11 + + inline void Destroy() { get()->~Type(); } + + private: + TF_LIB_GTL_ALIGNED_CHAR_ARRAY(Type, 1) space_; +}; + +#undef TF_LIB_GTL_ALIGNED_CHAR_ARRAY +#undef TF_LIB_GTL_ALIGN_OF + +} // namespace tensorflow + +#endif // TENSORFLOW_LIB_GTL_MANUAL_CONSTRUCTOR_H_ diff --git a/linux/include/tensorflow/core/lib/gtl/map_util.h b/linux/include/tensorflow/core/lib/gtl/map_util.h new file mode 100644 index 000000000..356e5d641 --- /dev/null +++ b/linux/include/tensorflow/core/lib/gtl/map_util.h @@ -0,0 +1,161 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// This file provides utility functions for use with STL map-like data +// structures, such as std::map and hash_map. Some functions will also work with +// sets, such as ContainsKey(). + +#ifndef TENSORFLOW_LIB_GTL_MAP_UTIL_H_ +#define TENSORFLOW_LIB_GTL_MAP_UTIL_H_ + +#include +#include +#include +#include +#include + +namespace tensorflow { +namespace gtl { + +// Returns a pointer to the const value associated with the given key if it +// exists, or NULL otherwise. +template +const typename Collection::value_type::second_type* FindOrNull( + const Collection& collection, + const typename Collection::value_type::first_type& key) { + typename Collection::const_iterator it = collection.find(key); + if (it == collection.end()) { + return 0; + } + return &it->second; +} + +// Same as above but returns a pointer to the non-const value. +template +typename Collection::value_type::second_type* FindOrNull( + Collection& collection, // NOLINT + const typename Collection::value_type::first_type& key) { + typename Collection::iterator it = collection.find(key); + if (it == collection.end()) { + return 0; + } + return &it->second; +} + +// Returns the pointer value associated with the given key. If none is found, +// NULL is returned. The function is designed to be used with a map of keys to +// pointers. +// +// This function does not distinguish between a missing key and a key mapped +// to a NULL value. +template +typename Collection::value_type::second_type FindPtrOrNull( + const Collection& collection, + const typename Collection::value_type::first_type& key) { + typename Collection::const_iterator it = collection.find(key); + if (it == collection.end()) { + return typename Collection::value_type::second_type(); + } + return it->second; +} + +// Returns a const reference to the value associated with the given key if it +// exists, otherwise returns a const reference to the provided default value. +// +// WARNING: If a temporary object is passed as the default "value," +// this function will return a reference to that temporary object, +// which will be destroyed at the end of the statement. A common +// example: if you have a map with string values, and you pass a char* +// as the default "value," either use the returned value immediately +// or store it in a string (not string&). +template +const typename Collection::value_type::second_type& FindWithDefault( + const Collection& collection, + const typename Collection::value_type::first_type& key, + const typename Collection::value_type::second_type& value) { + typename Collection::const_iterator it = collection.find(key); + if (it == collection.end()) { + return value; + } + return it->second; +} + +// Inserts the given key-value pair into the collection. Returns true if and +// only if the key from the given pair didn't previously exist. Otherwise, the +// value in the map is replaced with the value from the given pair. +template +bool InsertOrUpdate(Collection* const collection, + const typename Collection::value_type& vt) { + std::pair ret = collection->insert(vt); + if (!ret.second) { + // update + ret.first->second = vt.second; + return false; + } + return true; +} + +// Same as above, except that the key and value are passed separately. +template +bool InsertOrUpdate(Collection* const collection, + const typename Collection::value_type::first_type& key, + const typename Collection::value_type::second_type& value) { + return InsertOrUpdate(collection, + typename Collection::value_type(key, value)); +} + +// Inserts the given key and value into the given collection if and only if the +// given key did NOT already exist in the collection. If the key previously +// existed in the collection, the value is not changed. Returns true if the +// key-value pair was inserted; returns false if the key was already present. +template +bool InsertIfNotPresent(Collection* const collection, + const typename Collection::value_type& vt) { + return collection->insert(vt).second; +} + +// Same as above except the key and value are passed separately. +template +bool InsertIfNotPresent( + Collection* const collection, + const typename Collection::value_type::first_type& key, + const typename Collection::value_type::second_type& value) { + return InsertIfNotPresent(collection, + typename Collection::value_type(key, value)); +} + +// Looks up a given key and value pair in a collection and inserts the key-value +// pair if it's not already present. Returns a reference to the value associated +// with the key. +template +typename Collection::value_type::second_type& LookupOrInsert( + Collection* const collection, const typename Collection::value_type& vt) { + return collection->insert(vt).first->second; +} + +// Same as above except the key-value are passed separately. +template +typename Collection::value_type::second_type& LookupOrInsert( + Collection* const collection, + const typename Collection::value_type::first_type& key, + const typename Collection::value_type::second_type& value) { + return LookupOrInsert(collection, + typename Collection::value_type(key, value)); +} + +} // namespace gtl +} // namespace tensorflow + +#endif // TENSORFLOW_LIB_GTL_MAP_UTIL_H_ diff --git a/linux/include/tensorflow/core/lib/gtl/optional.h b/linux/include/tensorflow/core/lib/gtl/optional.h new file mode 100644 index 000000000..238aa18e1 --- /dev/null +++ b/linux/include/tensorflow/core/lib/gtl/optional.h @@ -0,0 +1,33 @@ +/* Copyright 2017 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_LIB_GTL_OPTIONAL_H_ +#define TENSORFLOW_CORE_LIB_GTL_OPTIONAL_H_ + +#include "absl/types/optional.h" + +namespace tensorflow { +namespace gtl { + +// Deprecated: please use absl::optional directly. +using absl::make_optional; +using absl::nullopt; +template +using optional = absl::optional; + +} // namespace gtl +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_LIB_GTL_OPTIONAL_H_ diff --git a/linux/include/tensorflow/core/lib/gtl/priority_queue_util.h b/linux/include/tensorflow/core/lib/gtl/priority_queue_util.h new file mode 100644 index 000000000..93bf3d303 --- /dev/null +++ b/linux/include/tensorflow/core/lib/gtl/priority_queue_util.h @@ -0,0 +1,55 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_LIB_GTL_PRIORITY_QUEUE_UTIL_H_ +#define TENSORFLOW_CORE_LIB_GTL_PRIORITY_QUEUE_UTIL_H_ + +#include +#include +#include + +namespace tensorflow { +namespace gtl { + +// Removes the top element from a std::priority_queue and returns it. +// Supports movable types. +template +T ConsumeTop(std::priority_queue* q) { + // std::priority_queue is required to implement pop() as if it + // called: + // std::pop_heap() + // c.pop_back() + // unfortunately, it does not provide access to the removed element. + // If the element is move only (such as a unique_ptr), there is no way to + // reclaim it in the standard API. std::priority_queue does, however, expose + // the underlying container as a protected member, so we use that access + // to extract the desired element between those two calls. + using Q = std::priority_queue; + struct Expose : Q { + using Q::c; + using Q::comp; + }; + auto& c = q->*&Expose::c; + auto& comp = q->*&Expose::comp; + std::pop_heap(c.begin(), c.end(), comp); + auto r = std::move(c.back()); + c.pop_back(); + return r; +} + +} // namespace gtl +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_LIB_GTL_PRIORITY_QUEUE_UTIL_H_ diff --git a/linux/include/tensorflow/core/lib/gtl/stl_util.h b/linux/include/tensorflow/core/lib/gtl/stl_util.h new file mode 100644 index 000000000..ffeca4e88 --- /dev/null +++ b/linux/include/tensorflow/core/lib/gtl/stl_util.h @@ -0,0 +1,120 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// This file provides utility functions for use with STL + +#ifndef TENSORFLOW_LIB_GTL_STL_UTIL_H_ +#define TENSORFLOW_LIB_GTL_STL_UTIL_H_ + +#include +#include +#include +#include +#include +#include +#include + +namespace tensorflow { +namespace gtl { + +// Returns a char* pointing to the beginning of a string's internal buffer. +// The result is a valid "null-terminated byte string", even if *str is empty. +// Up to C++14 it is not valid to *write* to the null terminator; as of C++17, +// it is valid to write zero to the null terminator (but not any other value). +inline char* string_as_array(string* str) { return &*str->begin(); } + +// The following vector_as_array functions return raw pointers to the underlying +// data buffer. The return value is unspecified (but valid) if the input range +// is empty. +template +inline T* vector_as_array(std::vector* v) { + return v->data(); +} + +template +inline const T* vector_as_array(const std::vector* v) { + return v->data(); +} + +// Like str->resize(new_size), except any new characters added to "*str" as a +// result of resizing may be left uninitialized, rather than being filled with +// '0' bytes. Typically used when code is then going to overwrite the backing +// store of the string with known data. Uses a Google extension to ::string. +inline void STLStringResizeUninitialized(string* s, size_t new_size) { +#if __google_stl_resize_uninitialized_string + s->resize_uninitialized(new_size); +#else + s->resize(new_size); +#endif +} + +// Calls delete (non-array version) on the SECOND item (pointer) in each pair in +// the range [begin, end). +// +// Note: If you're calling this on an entire container, you probably want to +// call STLDeleteValues(&container) instead, or use ValueDeleter. +template +void STLDeleteContainerPairSecondPointers(ForwardIterator begin, + ForwardIterator end) { + while (begin != end) { + ForwardIterator temp = begin; + ++begin; + delete temp->second; + } +} + +// Deletes all the elements in an STL container and clears the container. This +// function is suitable for use with a vector, set, hash_set, or any other STL +// container which defines sensible begin(), end(), and clear() methods. +// +// If container is NULL, this function is a no-op. +template +void STLDeleteElements(T* container) { + if (!container) return; + auto it = container->begin(); + while (it != container->end()) { + auto temp = it; + ++it; + delete *temp; + } + container->clear(); +} + +// Given an STL container consisting of (key, value) pairs, STLDeleteValues +// deletes all the "value" components and clears the container. Does nothing in +// the case it's given a NULL pointer. +template +void STLDeleteValues(T* container) { + if (!container) return; + auto it = container->begin(); + while (it != container->end()) { + auto temp = it; + ++it; + delete temp->second; + } + container->clear(); +} + +// Sorts and removes duplicates from a sequence container. +template +inline void STLSortAndRemoveDuplicates(T* v) { + std::sort(v->begin(), v->end()); + v->erase(std::unique(v->begin(), v->end()), v->end()); +} + +} // namespace gtl +} // namespace tensorflow + +#endif // TENSORFLOW_LIB_GTL_STL_UTIL_H_ diff --git a/linux/include/tensorflow/core/lib/gtl/top_n.h b/linux/include/tensorflow/core/lib/gtl/top_n.h new file mode 100644 index 000000000..900f670a5 --- /dev/null +++ b/linux/include/tensorflow/core/lib/gtl/top_n.h @@ -0,0 +1,339 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// This simple class finds the top n elements of an incrementally provided set +// of elements which you push one at a time. If the number of elements exceeds +// n, the lowest elements are incrementally dropped. At the end you get +// a vector of the top elements sorted in descending order (through Extract() or +// ExtractNondestructive()), or a vector of the top elements but not sorted +// (through ExtractUnsorted() or ExtractUnsortedNondestructive()). +// +// The value n is specified in the constructor. If there are p elements pushed +// altogether: +// The total storage requirements are O(min(n, p)) elements +// The running time is O(p * log(min(n, p))) comparisons +// If n is a constant, the total storage required is a constant and the running +// time is linear in p. +// +// NOTE(zhifengc): There is a way to do this in O(min(n, p)) storage and O(p) +// runtime. The basic idea is to repeatedly fill up a buffer of 2 * n elements, +// discarding the lowest n elements whenever the buffer is full using a linear- +// time median algorithm. This may have better performance when the input +// sequence is partially sorted. +// +// NOTE(zhifengc): This class should be redesigned to avoid reallocating a +// vector for each Extract. + +#ifndef TENSORFLOW_LIB_GTL_TOP_N_H_ +#define TENSORFLOW_LIB_GTL_TOP_N_H_ + +#include +#include +#include +#include +#include + +#include "tensorflow/core/platform/logging.h" + +namespace tensorflow { +namespace gtl { + +// Cmp is an stl binary predicate. Note that Cmp is the "greater" predicate, +// not the more commonly used "less" predicate. +// +// If you use a "less" predicate here, the TopN will pick out the bottom N +// elements out of the ones passed to it, and it will return them sorted in +// ascending order. +// +// TopN is rule-of-zero copyable and movable if its members are. +template > +class TopN { + public: + // The TopN is in one of the three states: + // + // o UNORDERED: this is the state an instance is originally in, + // where the elements are completely orderless. + // + // o BOTTOM_KNOWN: in this state, we keep the invariant that there + // is at least one element in it, and the lowest element is at + // position 0. The elements in other positions remain + // unsorted. This state is reached if the state was originally + // UNORDERED and a peek_bottom() function call is invoked. + // + // o HEAP_SORTED: in this state, the array is kept as a heap and + // there are exactly (limit_+1) elements in the array. This + // state is reached when at least (limit_+1) elements are + // pushed in. + // + // The state transition graph is at follows: + // + // peek_bottom() (limit_+1) elements + // UNORDERED --------------> BOTTOM_KNOWN --------------------> HEAP_SORTED + // | ^ + // | (limit_+1) elements | + // +-----------------------------------------------------------+ + + enum State { UNORDERED, BOTTOM_KNOWN, HEAP_SORTED }; + using UnsortedIterator = typename std::vector::const_iterator; + + // 'limit' is the maximum number of top results to return. + explicit TopN(size_t limit) : TopN(limit, Cmp()) {} + TopN(size_t limit, const Cmp &cmp) : limit_(limit), cmp_(cmp) {} + + size_t limit() const { return limit_; } + + // Number of elements currently held by this TopN object. This + // will be no greater than 'limit' passed to the constructor. + size_t size() const { return std::min(elements_.size(), limit_); } + + bool empty() const { return size() == 0; } + + // If you know how many elements you will push at the time you create the + // TopN object, you can call reserve to preallocate the memory that TopN + // will need to process all 'n' pushes. Calling this method is optional. + void reserve(size_t n) { elements_.reserve(std::min(n, limit_ + 1)); } + + // Push 'v'. If the maximum number of elements was exceeded, drop the + // lowest element and return it in 'dropped' (if given). If the maximum is not + // exceeded, 'dropped' will remain unchanged. 'dropped' may be omitted or + // nullptr, in which case it is not filled in. + // Requires: T is CopyAssignable, Swappable + void push(const T &v) { push(v, nullptr); } + void push(const T &v, T *dropped) { PushInternal(v, dropped); } + + // Move overloads of push. + // Requires: T is MoveAssignable, Swappable + void push(T &&v) { // NOLINT(build/c++11) + push(std::move(v), nullptr); + } + void push(T &&v, T *dropped) { // NOLINT(build/c++11) + PushInternal(std::move(v), dropped); + } + + // Peeks the bottom result without calling Extract() + const T &peek_bottom(); + + // Extract the elements as a vector sorted in descending order. The caller + // assumes ownership of the vector and must delete it when done. This is a + // destructive operation. The only method that can be called immediately + // after Extract() is Reset(). + std::vector *Extract(); + + // Similar to Extract(), but makes no guarantees the elements are in sorted + // order. As with Extract(), the caller assumes ownership of the vector and + // must delete it when done. This is a destructive operation. The only + // method that can be called immediately after ExtractUnsorted() is Reset(). + std::vector *ExtractUnsorted(); + + // A non-destructive version of Extract(). Copy the elements in a new vector + // sorted in descending order and return it. The caller assumes ownership of + // the new vector and must delete it when done. After calling + // ExtractNondestructive(), the caller can continue to push() new elements. + std::vector *ExtractNondestructive() const; + + // A non-destructive version of Extract(). Copy the elements to a given + // vector sorted in descending order. After calling + // ExtractNondestructive(), the caller can continue to push() new elements. + // Note: + // 1. The given argument must to be allocated. + // 2. Any data contained in the vector prior to the call will be deleted + // from it. After the call the vector will contain only the elements + // from the data structure. + void ExtractNondestructive(std::vector *output) const; + + // A non-destructive version of ExtractUnsorted(). Copy the elements in a new + // vector and return it, with no guarantees the elements are in sorted order. + // The caller assumes ownership of the new vector and must delete it when + // done. After calling ExtractUnsortedNondestructive(), the caller can + // continue to push() new elements. + std::vector *ExtractUnsortedNondestructive() const; + + // A non-destructive version of ExtractUnsorted(). Copy the elements into + // a given vector, with no guarantees the elements are in sorted order. + // After calling ExtractUnsortedNondestructive(), the caller can continue + // to push() new elements. + // Note: + // 1. The given argument must to be allocated. + // 2. Any data contained in the vector prior to the call will be deleted + // from it. After the call the vector will contain only the elements + // from the data structure. + void ExtractUnsortedNondestructive(std::vector *output) const; + + // Return an iterator to the beginning (end) of the container, + // with no guarantees about the order of iteration. These iterators are + // invalidated by mutation of the data structure. + UnsortedIterator unsorted_begin() const { return elements_.begin(); } + UnsortedIterator unsorted_end() const { return elements_.begin() + size(); } + + // Accessor for comparator template argument. + Cmp *comparator() { return &cmp_; } + + // This removes all elements. If Extract() or ExtractUnsorted() have been + // called, this will put it back in an empty but useable state. + void Reset(); + + private: + template + void PushInternal(U &&v, T *dropped); // NOLINT(build/c++11) + + // elements_ can be in one of two states: + // elements_.size() <= limit_: elements_ is an unsorted vector of elements + // pushed so far. + // elements_.size() > limit_: The last element of elements_ is unused; + // the other elements of elements_ are an stl heap whose size is exactly + // limit_. In this case elements_.size() is exactly one greater than + // limit_, but don't use "elements_.size() == limit_ + 1" to check for + // that because you'll get a false positive if limit_ == size_t(-1). + std::vector elements_; + size_t limit_; // Maximum number of elements to find + Cmp cmp_; // Greater-than comparison function + State state_ = UNORDERED; +}; + +// ---------------------------------------------------------------------- +// Implementations of non-inline functions + +template +template +void TopN::PushInternal(U &&v, T *dropped) { // NOLINT(build/c++11) + if (limit_ == 0) { + if (dropped) *dropped = std::forward(v); // NOLINT(build/c++11) + return; + } + if (state_ != HEAP_SORTED) { + elements_.push_back(std::forward(v)); // NOLINT(build/c++11) + if (state_ == UNORDERED || cmp_(elements_.back(), elements_.front())) { + // Easy case: we just pushed the new element back + } else { + // To maintain the BOTTOM_KNOWN state, we need to make sure that + // the element at position 0 is always the smallest. So we put + // the new element at position 0 and push the original bottom + // element in the back. + // Warning: this code is subtle. + using std::swap; + swap(elements_.front(), elements_.back()); + } + if (elements_.size() == limit_ + 1) { + // Transition from unsorted vector to a heap. + std::make_heap(elements_.begin(), elements_.end(), cmp_); + if (dropped) *dropped = std::move(elements_.front()); + std::pop_heap(elements_.begin(), elements_.end(), cmp_); + state_ = HEAP_SORTED; + } + } else { + // Only insert the new element if it is greater than the least element. + if (cmp_(v, elements_.front())) { + elements_.back() = std::forward(v); // NOLINT(build/c++11) + std::push_heap(elements_.begin(), elements_.end(), cmp_); + if (dropped) *dropped = std::move(elements_.front()); + std::pop_heap(elements_.begin(), elements_.end(), cmp_); + } else { + if (dropped) *dropped = std::forward(v); // NOLINT(build/c++11) + } + } +} + +template +const T &TopN::peek_bottom() { + CHECK(!empty()); + if (state_ == UNORDERED) { + // We need to do a linear scan to find out the bottom element + int min_candidate = 0; + for (size_t i = 1; i < elements_.size(); ++i) { + if (cmp_(elements_[min_candidate], elements_[i])) { + min_candidate = i; + } + } + // By swapping the element at position 0 and the minimal + // element, we transition to the BOTTOM_KNOWN state + if (min_candidate != 0) { + using std::swap; + swap(elements_[0], elements_[min_candidate]); + } + state_ = BOTTOM_KNOWN; + } + return elements_.front(); +} + +template +std::vector *TopN::Extract() { + auto out = new std::vector; + out->swap(elements_); + if (state_ != HEAP_SORTED) { + std::sort(out->begin(), out->end(), cmp_); + } else { + out->pop_back(); + std::sort_heap(out->begin(), out->end(), cmp_); + } + return out; +} + +template +std::vector *TopN::ExtractUnsorted() { + auto out = new std::vector; + out->swap(elements_); + if (state_ == HEAP_SORTED) { + // Remove the limit_+1'th element. + out->pop_back(); + } + return out; +} + +template +std::vector *TopN::ExtractNondestructive() const { + auto out = new std::vector; + ExtractNondestructive(out); + return out; +} + +template +void TopN::ExtractNondestructive(std::vector *output) const { + CHECK(output); + *output = elements_; + if (state_ != HEAP_SORTED) { + std::sort(output->begin(), output->end(), cmp_); + } else { + output->pop_back(); + std::sort_heap(output->begin(), output->end(), cmp_); + } +} + +template +std::vector *TopN::ExtractUnsortedNondestructive() const { + auto elements = new std::vector; + ExtractUnsortedNondestructive(elements); + return elements; +} + +template +void TopN::ExtractUnsortedNondestructive(std::vector *output) const { + CHECK(output); + *output = elements_; + if (state_ == HEAP_SORTED) { + // Remove the limit_+1'th element. + output->pop_back(); + } +} + +template +void TopN::Reset() { + elements_.clear(); + state_ = UNORDERED; +} + +} // namespace gtl +} // namespace tensorflow + +#endif // TENSORFLOW_LIB_GTL_TOP_N_H_ diff --git a/linux/include/tensorflow/core/lib/hash/crc32c.h b/linux/include/tensorflow/core/lib/hash/crc32c.h new file mode 100644 index 000000000..2718cd31b --- /dev/null +++ b/linux/include/tensorflow/core/lib/hash/crc32c.h @@ -0,0 +1,54 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_LIB_HASH_CRC32C_H_ +#define TENSORFLOW_CORE_LIB_HASH_CRC32C_H_ + +#include +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { +namespace crc32c { + +// Return the crc32c of concat(A, data[0,n-1]) where init_crc is the +// crc32c of some string A. Extend() is often used to maintain the +// crc32c of a stream of data. +extern uint32 Extend(uint32 init_crc, const char* data, size_t n); + +// Return the crc32c of data[0,n-1] +inline uint32 Value(const char* data, size_t n) { return Extend(0, data, n); } + +static const uint32 kMaskDelta = 0xa282ead8ul; + +// Return a masked representation of crc. +// +// Motivation: it is problematic to compute the CRC of a string that +// contains embedded CRCs. Therefore we recommend that CRCs stored +// somewhere (e.g., in files) should be masked before being stored. +inline uint32 Mask(uint32 crc) { + // Rotate right by 15 bits and add a constant. + return ((crc >> 15) | (crc << 17)) + kMaskDelta; +} + +// Return the crc whose masked representation is masked_crc. +inline uint32 Unmask(uint32 masked_crc) { + uint32 rot = masked_crc - kMaskDelta; + return ((rot >> 17) | (rot << 15)); +} + +} // namespace crc32c +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_LIB_HASH_CRC32C_H_ diff --git a/linux/include/tensorflow/core/lib/hash/hash.h b/linux/include/tensorflow/core/lib/hash/hash.h new file mode 100644 index 000000000..675bab719 --- /dev/null +++ b/linux/include/tensorflow/core/lib/hash/hash.h @@ -0,0 +1,113 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// Simple hash functions used for internal data structures + +#ifndef TENSORFLOW_CORE_LIB_HASH_HASH_H_ +#define TENSORFLOW_CORE_LIB_HASH_HASH_H_ + +#include +#include + +#include +#include + +#include "tensorflow/core/lib/core/stringpiece.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +extern uint32 Hash32(const char* data, size_t n, uint32 seed); +extern uint64 Hash64(const char* data, size_t n, uint64 seed); + +inline uint64 Hash64(const char* data, size_t n) { + return Hash64(data, n, 0xDECAFCAFFE); +} + +inline uint64 Hash64(const string& str) { + return Hash64(str.data(), str.size()); +} + +inline uint64 Hash64Combine(uint64 a, uint64 b) { + return a ^ (b + 0x9e3779b97f4a7800ULL + (a << 10) + (a >> 4)); +} + +// Combine two hashes in an order-independent way. This operation should be +// associative and compute the same hash for a collection of elements +// independent of traversal order. Note that it is better to combine hashes +// symmetrically with addition rather than XOR, since (x^x) == 0 but (x+x) != 0. +inline uint64 Hash64CombineUnordered(uint64 a, uint64 b) { return a + b; } + +// Hash functor suitable for use with power-of-two sized hashtables. Use +// instead of std::hash. +// +// In particular, tensorflow::hash is not the identity function for pointers. +// This is important for power-of-two sized hashtables like FlatMap and FlatSet, +// because otherwise they waste the majority of their hash buckets. +// +// The second type argument is only used for SFNIAE below. +template +struct hash { + size_t operator()(const T& t) const { return std::hash()(t); } +}; + +template +struct hash::value>::type> { + size_t operator()(T value) const { + // This works around a defect in the std::hash C++ spec that isn't fixed in + // (at least) gcc 4.8.4: + // http://www.open-std.org/jtc1/sc22/wg21/docs/lwg-defects.html#2148 + // + // We should be able to remove this and use the default + // tensorflow::hash() once we stop building with GCC versions old + // enough to not have this defect fixed. + return std::hash()(static_cast(value)); + } +}; + +template +struct hash { + size_t operator()(const T* t) const { + // Hash pointers as integers, but bring more entropy to the lower bits. + size_t k = static_cast(reinterpret_cast(t)); + return k + (k >> 6); + } +}; + +template <> +struct hash { + size_t operator()(const string& s) const { + return static_cast(Hash64(s)); + } +}; + +template <> +struct hash { + size_t operator()(StringPiece sp) const { + return static_cast(Hash64(sp.data(), sp.size())); + } +}; +using StringPieceHasher = ::tensorflow::hash; + +template +struct hash> { + size_t operator()(const std::pair& p) const { + return Hash64Combine(hash()(p.first), hash()(p.second)); + } +}; + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_LIB_HASH_HASH_H_ diff --git a/linux/include/tensorflow/core/lib/histogram/histogram.h b/linux/include/tensorflow/core/lib/histogram/histogram.h new file mode 100644 index 000000000..f882ee9ab --- /dev/null +++ b/linux/include/tensorflow/core/lib/histogram/histogram.h @@ -0,0 +1,139 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_LIB_HISTOGRAM_HISTOGRAM_H_ +#define TENSORFLOW_CORE_LIB_HISTOGRAM_HISTOGRAM_H_ + +#include +#include +#include "tensorflow/core/lib/gtl/array_slice.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/mutex.h" +#include "tensorflow/core/platform/thread_annotations.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +class HistogramProto; + +namespace histogram { + +class Histogram { + public: + // Create a histogram with a default set of bucket boundaries. + // Buckets near zero cover very small ranges (e.g. 10^-12), and each + // bucket range grows by ~10% as we head away from zero. The + // buckets cover the range from -DBL_MAX to DBL_MAX. + Histogram(); + + // Create a histogram with a custom set of bucket boundaries, + // specified in "custom_bucket_limits[0..custom_bucket_limits.size()-1]" + // REQUIRES: custom_bucket_limits[i] values are monotonically increasing. + // REQUIRES: custom_bucket_limits is not empty() + explicit Histogram(gtl::ArraySlice custom_bucket_limits); + + // Restore the state of a histogram that was previously encoded + // via Histogram::EncodeToProto. Note that only the bucket boundaries + // generated by EncodeToProto will be restored. + bool DecodeFromProto(const HistogramProto& proto); + + ~Histogram() {} + + void Clear(); + void Add(double value); + + // Save the current state of the histogram to "*proto". If + // "preserve_zero_buckets" is false, only non-zero bucket values and + // ranges are saved, and the bucket boundaries of zero-valued buckets + // are lost. + void EncodeToProto(HistogramProto* proto, bool preserve_zero_buckets) const; + + // Return the median of the values in the histogram + double Median() const; + + // Return the "p"th percentile [0.0..100.0] of the values in the + // distribution + double Percentile(double p) const; + + // Return the average value of the distribution + double Average() const; + + // Return the standard deviation of values in the distribution + double StandardDeviation() const; + + // Returns a multi-line human-readable string representing the histogram + // contents. Example output: + // Count: 4 Average: 251.7475 StdDev: 432.02 + // Min: -3.0000 Median: 5.0000 Max: 1000.0000 + // ------------------------------------------------------ + // [ -5, 0 ) 1 25.000% 25.000% ##### + // [ 0, 5 ) 1 25.000% 50.000% ##### + // [ 5, 10 ) 1 25.000% 75.000% ##### + // [ 1000, 10000 ) 1 25.000% 100.000% ##### + std::string ToString() const; + + private: + double min_; + double max_; + double num_; + double sum_; + double sum_squares_; + + std::vector custom_bucket_limits_; + gtl::ArraySlice bucket_limits_; + std::vector buckets_; + + double Remap(double x, double x0, double x1, double y0, double y1) const; + + TF_DISALLOW_COPY_AND_ASSIGN(Histogram); +}; + +// Wrapper around a Histogram object that is thread safe. +// +// All methods hold a lock while delegating to a Histogram object owned by the +// ThreadSafeHistogram instance. +// +// See Histogram for documentation of the methods. +class ThreadSafeHistogram { + public: + ThreadSafeHistogram() {} + explicit ThreadSafeHistogram(gtl::ArraySlice custom_bucket_limits) + : histogram_(custom_bucket_limits) {} + bool DecodeFromProto(const HistogramProto& proto); + + ~ThreadSafeHistogram() {} + + void Clear(); + + // TODO(touts): It might be a good idea to provide a AddN() + // method to avoid grabbing/releasing the lock when adding many values. + void Add(double value); + + void EncodeToProto(HistogramProto* proto, bool preserve_zero_buckets) const; + double Median() const; + double Percentile(double p) const; + double Average() const; + double StandardDeviation() const; + std::string ToString() const; + + private: + mutable mutex mu_; + Histogram histogram_ GUARDED_BY(mu_); +}; + +} // namespace histogram +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_LIB_HISTOGRAM_HISTOGRAM_H_ diff --git a/linux/include/tensorflow/core/lib/io/block.h b/linux/include/tensorflow/core/lib/io/block.h new file mode 100644 index 000000000..4b4896c4f --- /dev/null +++ b/linux/include/tensorflow/core/lib/io/block.h @@ -0,0 +1,56 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_LIB_IO_BLOCK_H_ +#define TENSORFLOW_LIB_IO_BLOCK_H_ + +#include +#include +#include "tensorflow/core/lib/io/iterator.h" + +namespace tensorflow { +namespace table { + +struct BlockContents; + +class Block { + public: + // Initialize the block with the specified contents. + explicit Block(const BlockContents& contents); + + ~Block(); + + size_t size() const { return size_; } + Iterator* NewIterator(); + + private: + uint32 NumRestarts() const; + + const char* data_; + size_t size_; + uint32 restart_offset_; // Offset in data_ of restart array + bool owned_; // Block owns data_[] + + // No copying allowed + Block(const Block&); + void operator=(const Block&); + + class Iter; +}; + +} // namespace table +} // namespace tensorflow + +#endif // TENSORFLOW_LIB_IO_BLOCK_H_ diff --git a/linux/include/tensorflow/core/lib/io/block_builder.h b/linux/include/tensorflow/core/lib/io/block_builder.h new file mode 100644 index 000000000..117b6a0bb --- /dev/null +++ b/linux/include/tensorflow/core/lib/io/block_builder.h @@ -0,0 +1,69 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_LIB_IO_BLOCK_BUILDER_H_ +#define TENSORFLOW_LIB_IO_BLOCK_BUILDER_H_ + +#include + +#include +#include "tensorflow/core/lib/core/stringpiece.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { +namespace table { + +struct Options; + +class BlockBuilder { + public: + explicit BlockBuilder(const Options* options); + + // Reset the contents as if the BlockBuilder was just constructed. + void Reset(); + + // REQUIRES: Finish() has not been called since the last call to Reset(). + // REQUIRES: key is larger than any previously added key + void Add(const StringPiece& key, const StringPiece& value); + + // Finish building the block and return a slice that refers to the + // block contents. The returned slice will remain valid for the + // lifetime of this builder or until Reset() is called. + StringPiece Finish(); + + // Returns an estimate of the current (uncompressed) size of the block + // we are building. + size_t CurrentSizeEstimate() const; + + // Return true iff no entries have been added since the last Reset() + bool empty() const { return buffer_.empty(); } + + private: + const Options* options_; + string buffer_; // Destination buffer + std::vector restarts_; // Restart points + int counter_; // Number of entries emitted since restart + bool finished_; // Has Finish() been called? + string last_key_; + + // No copying allowed + BlockBuilder(const BlockBuilder&); + void operator=(const BlockBuilder&); +}; + +} // namespace table +} // namespace tensorflow + +#endif // TENSORFLOW_LIB_IO_BLOCK_BUILDER_H_ diff --git a/linux/include/tensorflow/core/lib/io/buffered_inputstream.h b/linux/include/tensorflow/core/lib/io/buffered_inputstream.h new file mode 100644 index 000000000..96a95b7ed --- /dev/null +++ b/linux/include/tensorflow/core/lib/io/buffered_inputstream.h @@ -0,0 +1,107 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_LIB_IO_BUFFERED_INPUTSTREAM_H_ +#define TENSORFLOW_CORE_LIB_IO_BUFFERED_INPUTSTREAM_H_ + +#include "tensorflow/core/lib/io/inputstream_interface.h" +#include "tensorflow/core/platform/file_system.h" + +namespace tensorflow { +namespace io { + +// Provides a buffer on top of an InputStreamInterface. A single instance of +// BufferedInputStream is NOT safe for concurrent use by multiple threads. +class BufferedInputStream : public InputStreamInterface { + public: + // Does not take ownership of input_stream unless owns_input_stream is set + // to true. input_stream must outlive *this then. + // TODO(rohanj): Remove owns_input_stream once the constructor below is + // removed. + BufferedInputStream(InputStreamInterface* input_stream, size_t buffer_bytes, + bool owns_input_stream = false); + + // For backwards compatibility, expose an interface that is similar to what + // InputBuffer exposes. Does not take ownership of file. file must outlive + // *this. This will be removed once we migrate all uses of this class to the + // constructor above. + BufferedInputStream(RandomAccessFile* file, size_t buffer_bytes); + + ~BufferedInputStream() override; + + Status ReadNBytes(int64 bytes_to_read, string* result) override; + + Status SkipNBytes(int64 bytes_to_skip) override; + + int64 Tell() const override; + + // Seek to this offset within the file. + // + // If we seek to somewhere within our pre-buffered data, we will re-use what + // data we can. Otherwise, Seek() throws out the current buffer and the next + // read will trigger an underlying read. + // + // Note: When seeking backwards in a stream, this implementation uses + // Reset() + SkipNBytes(), so its performance will be dependent + // largely on the performance of SkipNBytes(). + Status Seek(int64 position); + + // Read one text line of data into "*result" until end-of-file or a + // \n is read. (The \n is not included in the result.) Overwrites + // any existing data in *result. + // + // If successful, returns OK. If we are already at the end of the + // file, we return an OUT_OF_RANGE error. Otherwise, we return + // some other non-OK status. + Status ReadLine(string* result); + + // Returns one text line of data until end-of-file or a '\n' is read. The '\n' + // is included in the result. + // This method is a substitute for ReadLine() when called from Python which is + // the expectation in the python File::readline() API. + // Also, '\0's are treated like any other character within the line and given + // no special treatment. + string ReadLineAsString(); + + // Reads the entire contents of the file into *result. + // + // Note: the amount of memory used by this function call is unbounded, so only + // use in ops that expect that behavior. + Status ReadAll(string* result); + + Status Reset() override; + + private: + Status FillBuffer(); + Status ReadLineHelper(string* result, bool include_eol); + + InputStreamInterface* input_stream_; // not owned. + size_t size_; // buffer size. + string buf_; // the buffer itself. + // buf_[pos_, limit_) holds the valid "read ahead" data in the file. + size_t pos_ = 0; // current position in buf_. + size_t limit_ = 0; // just past the end of valid data in buf_. + bool owns_input_stream_ = false; + // When EoF is reached, file_status_ contains the status to skip unnecessary + // buffer allocations. + Status file_status_ = Status::OK(); + + TF_DISALLOW_COPY_AND_ASSIGN(BufferedInputStream); +}; + +} // namespace io +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_LIB_IO_BUFFERED_INPUTSTREAM_H_ diff --git a/linux/include/tensorflow/core/lib/io/compression.h b/linux/include/tensorflow/core/lib/io/compression.h new file mode 100644 index 000000000..4d8e7788c --- /dev/null +++ b/linux/include/tensorflow/core/lib/io/compression.h @@ -0,0 +1,30 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_LIB_IO_COMPRESSION_H_ +#define TENSORFLOW_CORE_LIB_IO_COMPRESSION_H_ + +namespace tensorflow { +namespace io { +namespace compression { + +extern const char kNone[]; +extern const char kGzip[]; + +} // namespace compression +} // namespace io +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_LIB_IO_COMPRESSION_H_ diff --git a/linux/include/tensorflow/core/lib/io/format.h b/linux/include/tensorflow/core/lib/io/format.h new file mode 100644 index 000000000..fd5c3470f --- /dev/null +++ b/linux/include/tensorflow/core/lib/io/format.h @@ -0,0 +1,110 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_LIB_IO_FORMAT_H_ +#define TENSORFLOW_LIB_IO_FORMAT_H_ + +#include +#include +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/lib/core/stringpiece.h" +#include "tensorflow/core/lib/io/table_builder.h" + +namespace tensorflow { +class RandomAccessFile; +namespace table { + +class Block; + +// BlockHandle is a pointer to the extent of a file that stores a data +// block or a meta block. +class BlockHandle { + public: + BlockHandle(); + + // The offset of the block in the file. + uint64 offset() const { return offset_; } + void set_offset(uint64 offset) { offset_ = offset; } + + // The size of the stored block + uint64 size() const { return size_; } + void set_size(uint64 size) { size_ = size; } + + void EncodeTo(string* dst) const; + Status DecodeFrom(StringPiece* input); + + // Maximum encoding length of a BlockHandle + enum { kMaxEncodedLength = 10 + 10 }; + + private: + uint64 offset_; + uint64 size_; +}; + +// Footer encapsulates the fixed information stored at the tail +// end of every table file. +class Footer { + public: + Footer() {} + + // The block handle for the metaindex block of the table + const BlockHandle& metaindex_handle() const { return metaindex_handle_; } + void set_metaindex_handle(const BlockHandle& h) { metaindex_handle_ = h; } + + // The block handle for the index block of the table + const BlockHandle& index_handle() const { return index_handle_; } + void set_index_handle(const BlockHandle& h) { index_handle_ = h; } + + void EncodeTo(string* dst) const; + Status DecodeFrom(StringPiece* input); + + // Encoded length of a Footer. Note that the serialization of a + // Footer will always occupy exactly this many bytes. It consists + // of two block handles and a magic number. + enum { kEncodedLength = 2 * BlockHandle::kMaxEncodedLength + 8 }; + + private: + BlockHandle metaindex_handle_; + BlockHandle index_handle_; +}; + +// kTableMagicNumber was picked by running +// echo http://code.google.com/p/leveldb/ | sha1sum +// and taking the leading 64 bits. +static const uint64 kTableMagicNumber = 0xdb4775248b80fb57ull; + +// 1-byte type + 32-bit crc +static const size_t kBlockTrailerSize = 5; + +struct BlockContents { + StringPiece data; // Actual contents of data + bool cachable; // True iff data can be cached + bool heap_allocated; // True iff caller should delete[] data.data() +}; + +// Read the block identified by "handle" from "file". On failure +// return non-OK. On success fill *result and return OK. +extern Status ReadBlock(RandomAccessFile* file, const BlockHandle& handle, + BlockContents* result); + +// Implementation details follow. Clients should ignore, + +inline BlockHandle::BlockHandle() + : offset_(~static_cast(0)), size_(~static_cast(0)) {} + +} // namespace table +} // namespace tensorflow + +#endif // TENSORFLOW_LIB_IO_FORMAT_H_ diff --git a/linux/include/tensorflow/core/lib/io/inputbuffer.h b/linux/include/tensorflow/core/lib/io/inputbuffer.h new file mode 100644 index 000000000..b3740f396 --- /dev/null +++ b/linux/include/tensorflow/core/lib/io/inputbuffer.h @@ -0,0 +1,141 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_LIB_IO_INPUTBUFFER_H_ +#define TENSORFLOW_LIB_IO_INPUTBUFFER_H_ + +#include +#include "tensorflow/core/lib/core/coding.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/platform/env.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { +namespace io { + +// An InputBuffer provides a buffer on top of a RandomAccessFile. +// A given instance of an InputBuffer is NOT safe for concurrent use +// by multiple threads +class InputBuffer { + public: + // Create an InputBuffer for "file" with a buffer size of + // "buffer_bytes" bytes. 'file' must outlive *this. + InputBuffer(RandomAccessFile* file, size_t buffer_bytes); + ~InputBuffer(); + + // Read one text line of data into "*result" until end-of-file or a + // \n is read. (The \n is not included in the result.) Overwrites + // any existing data in *result. + // + // If successful, returns OK. If we are already at the end of the + // file, we return an OUT_OF_RANGE error. Otherwise, we return + // some other non-OK status. + Status ReadLine(string* result); + + // Reads bytes_to_read bytes into *result, overwriting *result. + // + // If successful, returns OK. If we there are not enough bytes to + // read before the end of the file, we return an OUT_OF_RANGE error. + // Otherwise, we return some other non-OK status. + Status ReadNBytes(int64 bytes_to_read, string* result); + + // An overload that writes to char*. Caller must ensure result[0, + // bytes_to_read) is valid to be overwritten. Returns OK iff "*bytes_read == + // bytes_to_read". + Status ReadNBytes(int64 bytes_to_read, char* result, size_t* bytes_read); + + // Reads a single varint32. + Status ReadVarint32(uint32* result); + + // Reads a single varint64. + Status ReadVarint64(uint64* result); + + // Like ReadNBytes() without returning the bytes read. + Status SkipNBytes(int64 bytes_to_skip); + + // Seek to this offset within the file. + // + // If we seek to somewhere within our pre-buffered data, we will re-use what + // data we can. Otherwise, Seek() throws out the current buffer and the next + // read will trigger a File::Read(). + Status Seek(int64 position); + + // Returns the position in the file. + int64 Tell() const { return file_pos_ - (limit_ - pos_); } + + // Returns the underlying RandomAccessFile. + RandomAccessFile* file() const { return file_; } + + private: + Status FillBuffer(); + + // Internal slow-path routine used by ReadVarint32(). + Status ReadVarint32Fallback(uint32* result); + + // Internal slow-path routine used by ReadVarint64(). + Status ReadVarint64Fallback(uint64* result); + + // Helper method for reading a varint which can span at max `max_bytes`. + // If the varint is longer, a DataLoss error status is returned. + // If end of file is reached while reading, OutOfRange error is returned. + template + Status ReadVarintFallback(T* result, int max_bytes); + + RandomAccessFile* file_; // Not owned + int64 file_pos_; // Next position to read from in "file_" + size_t size_; // Size of "buf_" + char* buf_; // The buffer itself + // [pos_,limit_) hold the "limit_ - pos_" bytes just before "file_pos_" + char* pos_; // Current position in "buf" + char* limit_; // Just past end of valid data in "buf" + + TF_DISALLOW_COPY_AND_ASSIGN(InputBuffer); +}; + +// Implementation details. + +// Inlined for performance. +inline Status InputBuffer::ReadVarint32(uint32* result) { + if (pos_ + core::kMaxVarint32Bytes <= limit_) { + // Fast path: directly parse from buffered data. + // Reads strictly from the range [pos_, limit_). + const char* offset = core::GetVarint32Ptr(pos_, limit_, result); + if (offset == nullptr) return errors::OutOfRange("Parsed past limit."); + pos_ = const_cast(offset); + return Status::OK(); + } else { + return ReadVarint32Fallback(result); + } +} + +// Inlined for performance. +inline Status InputBuffer::ReadVarint64(uint64* result) { + if (pos_ + core::kMaxVarint64Bytes <= limit_) { + // Fast path: directly parse from buffered data. + // Reads strictly from the range [pos_, limit_). + const char* offset = core::GetVarint64Ptr(pos_, limit_, result); + if (offset == nullptr) return errors::OutOfRange("Parsed past limit."); + pos_ = const_cast(offset); + return Status::OK(); + } else { + return ReadVarint64Fallback(result); + } +} + +} // namespace io +} // namespace tensorflow + +#endif // TENSORFLOW_LIB_IO_INPUTBUFFER_H_ diff --git a/linux/include/tensorflow/core/lib/io/inputstream_interface.h b/linux/include/tensorflow/core/lib/io/inputstream_interface.h new file mode 100644 index 000000000..cbfc509d9 --- /dev/null +++ b/linux/include/tensorflow/core/lib/io/inputstream_interface.h @@ -0,0 +1,57 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_LIB_IO_INPUTSTREAM_INTERFACE_H_ +#define TENSORFLOW_CORE_LIB_IO_INPUTSTREAM_INTERFACE_H_ + +#include +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { +namespace io { + +// An interface that defines input streaming operations. +class InputStreamInterface { + public: + InputStreamInterface() {} + virtual ~InputStreamInterface() {} + + // Reads the next bytes_to_read from the file. Typical return codes: + // * OK - in case of success. + // * OUT_OF_RANGE - not enough bytes remaining before end of file. + virtual Status ReadNBytes(int64 bytes_to_read, string* result) = 0; + + // Skips bytes_to_skip before next ReadNBytes. bytes_to_skip should be >= 0. + // Typical return codes: + // * OK - in case of success. + // * OUT_OF_RANGE - not enough bytes remaining before end of file. + virtual Status SkipNBytes(int64 bytes_to_skip); + + // Return the offset of the current byte relative to the beginning of the + // file. + // If we Skip / Read beyond the end of the file, this should return the length + // of the file. + // If there are any errors, this must return -1. + virtual int64 Tell() const = 0; + + // Resets the stream to the beginning. + virtual Status Reset() = 0; +}; + +} // namespace io +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_LIB_IO_INPUTSTREAM_INTERFACE_H_ diff --git a/linux/include/tensorflow/core/lib/io/iterator.h b/linux/include/tensorflow/core/lib/io/iterator.h new file mode 100644 index 000000000..2b9326d7d --- /dev/null +++ b/linux/include/tensorflow/core/lib/io/iterator.h @@ -0,0 +1,104 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// An iterator yields a sequence of key/value pairs from a source. +// The following class defines the interface. Multiple implementations +// are provided by this library. In particular, iterators are provided +// to access the contents of a Table or a DB. +// +// Multiple threads can invoke const methods on an Iterator without +// external synchronization, but if any of the threads may call a +// non-const method, all threads accessing the same Iterator must use +// external synchronization. + +#ifndef TENSORFLOW_LIB_IO_ITERATOR_H_ +#define TENSORFLOW_LIB_IO_ITERATOR_H_ + +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/lib/core/stringpiece.h" + +namespace tensorflow { +namespace table { + +class Iterator { + public: + Iterator(); + virtual ~Iterator(); + + // An iterator is either positioned at a key/value pair, or + // not valid. This method returns true iff the iterator is valid. + virtual bool Valid() const = 0; + + // Position at the first key in the source. The iterator is Valid() + // after this call iff the source is not empty. + virtual void SeekToFirst() = 0; + + // Position at the first key in the source that is at or past target. + // The iterator is Valid() after this call iff the source contains + // an entry that comes at or past target. + virtual void Seek(const StringPiece& target) = 0; + + // Moves to the next entry in the source. After this call, Valid() is + // true iff the iterator was not positioned at the last entry in the source. + // REQUIRES: Valid() + virtual void Next() = 0; + + // Return the key for the current entry. The underlying storage for + // the returned slice is valid only until the next modification of + // the iterator. + // REQUIRES: Valid() + virtual StringPiece key() const = 0; + + // Return the value for the current entry. The underlying storage for + // the returned slice is valid only until the next modification of + // the iterator. + // REQUIRES: Valid() + virtual StringPiece value() const = 0; + + // If an error has occurred, return it. Else return an ok status. + virtual Status status() const = 0; + + // Clients are allowed to register function/arg1/arg2 triples that + // will be invoked when this iterator is destroyed. + // + // Note that unlike all of the preceding methods, this method is + // not abstract and therefore clients should not override it. + typedef void (*CleanupFunction)(void* arg1, void* arg2); + void RegisterCleanup(CleanupFunction function, void* arg1, void* arg2); + + private: + struct Cleanup { + CleanupFunction function; + void* arg1; + void* arg2; + Cleanup* next; + }; + Cleanup cleanup_; + + // No copying allowed + Iterator(const Iterator&); + void operator=(const Iterator&); +}; + +// Return an empty iterator (yields nothing). +extern Iterator* NewEmptyIterator(); + +// Return an empty iterator with the specified status. +extern Iterator* NewErrorIterator(const Status& status); + +} // namespace table +} // namespace tensorflow + +#endif // TENSORFLOW_LIB_IO_ITERATOR_H_ diff --git a/linux/include/tensorflow/core/lib/io/path.h b/linux/include/tensorflow/core/lib/io/path.h new file mode 100644 index 000000000..38fb0c5d8 --- /dev/null +++ b/linux/include/tensorflow/core/lib/io/path.h @@ -0,0 +1,98 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_LIB_IO_PATH_H_ +#define TENSORFLOW_CORE_LIB_IO_PATH_H_ + +#include "tensorflow/core/lib/core/stringpiece.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { +namespace io { +namespace internal { +string JoinPathImpl(std::initializer_list paths); +} + +// Utility routines for processing filenames + +#ifndef SWIG // variadic templates +// Join multiple paths together, without introducing unnecessary path +// separators. +// For example: +// +// Arguments | JoinPath +// ---------------------------+---------- +// '/foo', 'bar' | /foo/bar +// '/foo/', 'bar' | /foo/bar +// '/foo', '/bar' | /foo/bar +// +// Usage: +// string path = io::JoinPath("/mydir", filename); +// string path = io::JoinPath(FLAGS_test_srcdir, filename); +// string path = io::JoinPath("/full", "path", "to", "filename); +template +string JoinPath(const T&... args) { + return internal::JoinPathImpl({args...}); +} +#endif /* SWIG */ + +// Return true if path is absolute. +bool IsAbsolutePath(tensorflow::StringPiece path); + +// Returns the part of the path before the final "/". If there is a single +// leading "/" in the path, the result will be the leading "/". If there is +// no "/" in the path, the result is the empty prefix of the input. +tensorflow::StringPiece Dirname(tensorflow::StringPiece path); + +// Returns the part of the path after the final "/". If there is no +// "/" in the path, the result is the same as the input. +tensorflow::StringPiece Basename(tensorflow::StringPiece path); + +// Returns the part of the basename of path after the final ".". If +// there is no "." in the basename, the result is empty. +tensorflow::StringPiece Extension(tensorflow::StringPiece path); + +// Collapse duplicate "/"s, resolve ".." and "." path elements, remove +// trailing "/". +// +// NOTE: This respects relative vs. absolute paths, but does not +// invoke any system calls (getcwd(2)) in order to resolve relative +// paths with respect to the actual working directory. That is, this is purely +// string manipulation, completely independent of process state. +string CleanPath(tensorflow::StringPiece path); + +// Populates the scheme, host, and path from a URI. scheme, host, and path are +// guaranteed by this function to point into the contents of uri, even if +// empty. +// +// Corner cases: +// - If the URI is invalid, scheme and host are set to empty strings and the +// passed string is assumed to be a path +// - If the URI omits the path (e.g. file://host), then the path is left empty. +void ParseURI(tensorflow::StringPiece uri, tensorflow::StringPiece* scheme, + tensorflow::StringPiece* host, tensorflow::StringPiece* path); + +// Creates a URI from a scheme, host, and path. If the scheme is empty, we just +// return the path. +string CreateURI(tensorflow::StringPiece scheme, tensorflow::StringPiece host, + tensorflow::StringPiece path); + +// Creates a temporary file name with an extension. +string GetTempFilename(const string& extension); + +} // namespace io +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_LIB_IO_PATH_H_ diff --git a/linux/include/tensorflow/core/lib/io/proto_encode_helper.h b/linux/include/tensorflow/core/lib/io/proto_encode_helper.h new file mode 100644 index 000000000..34905520f --- /dev/null +++ b/linux/include/tensorflow/core/lib/io/proto_encode_helper.h @@ -0,0 +1,98 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_LIB_IO_PROTO_ENCODE_HELPER_H_ +#define TENSORFLOW_CORE_LIB_IO_PROTO_ENCODE_HELPER_H_ + +#include "tensorflow/core/lib/core/coding.h" +#include "tensorflow/core/lib/core/stringpiece.h" +#include "tensorflow/core/platform/protobuf.h" + +// A helper class for appending various kinds of values in protocol +// buffer encoding format to a buffer. The client gives a pointer to +// a buffer and a maximum size guarantee for the number of bytes they +// will add to this buffer. +namespace tensorflow { +namespace io { + +class ProtoEncodeHelper { + public: + ProtoEncodeHelper(char* buf, int max_size) + : base_(buf), p_(buf), limit_(base_ + max_size) {} + + ~ProtoEncodeHelper() { + // Make sure callers didn't do operations that went over max_size promised + DCHECK_LE(p_, limit_); + } + + const char* data() const { return base_; } + size_t size() const { return p_ - base_; } + + void WriteUint64(int tag, uint64 v) { + Encode32(combine(tag, WIRETYPE_VARINT)); + Encode64(v); + } + void WriteBool(int tag, bool v) { + Encode32(combine(tag, WIRETYPE_VARINT)); + EncodeBool(v); + } + void WriteString(int tag, StringPiece v) { + Encode32(combine(tag, WIRETYPE_LENGTH_DELIMITED)); + Encode32(v.size()); + EncodeBytes(v.data(), v.size()); + } + void WriteVarlengthBeginning(int tag, uint32 len) { + Encode32(combine(tag, WIRETYPE_LENGTH_DELIMITED)); + Encode32(len); + } + void WriteRawBytes(StringPiece v) { EncodeBytes(v.data(), v.size()); } + + private: + // Note: this module's behavior must match the protocol buffer wire encoding + // format. + enum { + WIRETYPE_VARINT = 0, + WIRETYPE_LENGTH_DELIMITED = 2, + }; + static uint32 combine(uint32 tag, uint32 type) { return ((tag << 3) | type); } + inline void Encode32(uint32 v) { + if (v < 128) { + // Fast path for single-byte values. Many of the calls will use a + // constant value for v, so the comparison will get optimized away + // when Encode32 is inlined into the caller. + *p_ = v; + p_++; + } else { + p_ = core::EncodeVarint32(p_, v); + } + } + void Encode64(uint64 v) { p_ = core::EncodeVarint64(p_, v); } + void EncodeBool(bool v) { + *p_ = (v ? 1 : 0); // Equal to varint32 encoding of 0 or 1 + p_++; + } + void EncodeBytes(const char* bytes, int N) { + memcpy(p_, bytes, N); + p_ += N; + } + + char* base_; + char* p_; + char* limit_; // Just for CHECKs +}; +} // namespace io +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_LIB_IO_PROTO_ENCODE_HELPER_H_ diff --git a/linux/include/tensorflow/core/lib/io/random_inputstream.h b/linux/include/tensorflow/core/lib/io/random_inputstream.h new file mode 100644 index 000000000..c822fe50e --- /dev/null +++ b/linux/include/tensorflow/core/lib/io/random_inputstream.h @@ -0,0 +1,57 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_LIB_IO_RANDOM_INPUTSTREAM_H_ +#define TENSORFLOW_CORE_LIB_IO_RANDOM_INPUTSTREAM_H_ + +#include "tensorflow/core/lib/io/inputstream_interface.h" +#include "tensorflow/core/platform/file_system.h" + +namespace tensorflow { +namespace io { + +// Wraps a RandomAccessFile in an InputStreamInterface. A given instance of +// RandomAccessInputStream is NOT safe for concurrent use by multiple threads. +class RandomAccessInputStream : public InputStreamInterface { + public: + // Does not take ownership of 'file' unless owns_file is set to true. 'file' + // must outlive *this. + RandomAccessInputStream(RandomAccessFile* file, bool owns_file = false); + + ~RandomAccessInputStream(); + + Status ReadNBytes(int64 bytes_to_read, string* result) override; + + Status SkipNBytes(int64 bytes_to_skip) override; + + int64 Tell() const override; + + Status Seek(int64 position) { + pos_ = position; + return Status::OK(); + } + + Status Reset() override { return Seek(0); } + + private: + RandomAccessFile* file_; // Not owned. + int64 pos_ = 0; // Tracks where we are in the file. + bool owns_file_ = false; +}; + +} // namespace io +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_LIB_IO_RANDOM_INPUTSTREAM_H_ diff --git a/linux/include/tensorflow/core/lib/io/record_reader.h b/linux/include/tensorflow/core/lib/io/record_reader.h new file mode 100644 index 000000000..17444660d --- /dev/null +++ b/linux/include/tensorflow/core/lib/io/record_reader.h @@ -0,0 +1,158 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_LIB_IO_RECORD_READER_H_ +#define TENSORFLOW_CORE_LIB_IO_RECORD_READER_H_ + +#include "tensorflow/core/lib/core/errors.h" +#include "tensorflow/core/lib/core/stringpiece.h" +#include "tensorflow/core/lib/io/inputstream_interface.h" +#if !defined(IS_SLIM_BUILD) +#include "tensorflow/core/lib/io/zlib_compression_options.h" +#include "tensorflow/core/lib/io/zlib_inputstream.h" +#endif // IS_SLIM_BUILD +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +class RandomAccessFile; + +namespace io { + +class RecordReaderOptions { + public: + enum CompressionType { NONE = 0, ZLIB_COMPRESSION = 1 }; + CompressionType compression_type = NONE; + + // If buffer_size is non-zero, then all reads must be sequential, and no + // skipping around is permitted. (Note: this is the same behavior as reading + // compressed files.) Consider using SequentialRecordReader. + int64 buffer_size = 0; + + static RecordReaderOptions CreateRecordReaderOptions( + const string& compression_type); + +#if !defined(IS_SLIM_BUILD) + // Options specific to zlib compression. + ZlibCompressionOptions zlib_options; +#endif // IS_SLIM_BUILD +}; + +// Low-level interface to read TFRecord files. +// +// If using compression or buffering, consider using SequentialRecordReader. +// +// Note: this class is not thread safe; external synchronization required. +class RecordReader { + public: + // Format of a single record: + // uint64 length + // uint32 masked crc of length + // byte data[length] + // uint32 masked crc of data + static const size_t kHeaderSize = sizeof(uint64) + sizeof(uint32); + static const size_t kFooterSize = sizeof(uint32); + + // Statistics (sizes are in units of bytes) + struct Stats { + int64 file_size = -1; + int64 data_size = -1; + int64 entries = -1; // Number of values + }; + + // Metadata for the TFRecord file. + struct Metadata { + Stats stats; + }; + + // Create a reader that will return log records from "*file". + // "*file" must remain live while this Reader is in use. + explicit RecordReader( + RandomAccessFile* file, + const RecordReaderOptions& options = RecordReaderOptions()); + + virtual ~RecordReader() = default; + + // Read the record at "*offset" into *record and update *offset to + // point to the offset of the next record. Returns OK on success, + // OUT_OF_RANGE for end of file, or something else for an error. + Status ReadRecord(uint64* offset, string* record); + + // Return the metadata of the Record file. + // + // The current implementation scans the file to completion, + // skipping over the data regions, to extract the metadata once + // on the first call to GetStats(). An improved implementation + // would change RecordWriter to write the metadata into TFRecord + // so that GetMetadata() could be a const method. + // + // 'metadata' must not be nullptr. + Status GetMetadata(Metadata* md); + + private: + Status ReadChecksummed(uint64 offset, size_t n, string* result); + + RecordReaderOptions options_; + std::unique_ptr input_stream_; + bool last_read_failed_; + + std::unique_ptr cached_metadata_; + + TF_DISALLOW_COPY_AND_ASSIGN(RecordReader); +}; + +// High-level interface to read TFRecord files. +// +// Note: this class is not thread safe; external synchronization required. +class SequentialRecordReader { + public: + // Create a reader that will return log records from "*file". + // "*file" must remain live while this Reader is in use. + explicit SequentialRecordReader( + RandomAccessFile* file, + const RecordReaderOptions& options = RecordReaderOptions()); + + virtual ~SequentialRecordReader() = default; + + // Reads the next record in the file into *record. Returns OK on success, + // OUT_OF_RANGE for end of file, or something else for an error. + Status ReadRecord(string* record) { + return underlying_.ReadRecord(&offset_, record); + } + + // Returns the current offset in the file. + uint64 TellOffset() { return offset_; } + + // Seek to this offset within the file and set this offset as the current + // offset. Trying to seek backward will throw error. + Status SeekOffset(uint64 offset) { + if (offset < offset_) + return errors::InvalidArgument( + "Trying to seek offset: ", offset, + " which is less than the current offset: ", offset_); + offset_ = offset; + return Status::OK(); + } + + private: + RecordReader underlying_; + uint64 offset_ = 0; +}; + +} // namespace io +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_LIB_IO_RECORD_READER_H_ diff --git a/linux/include/tensorflow/core/lib/io/record_writer.h b/linux/include/tensorflow/core/lib/io/record_writer.h new file mode 100644 index 000000000..1212e1faf --- /dev/null +++ b/linux/include/tensorflow/core/lib/io/record_writer.h @@ -0,0 +1,117 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_LIB_IO_RECORD_WRITER_H_ +#define TENSORFLOW_CORE_LIB_IO_RECORD_WRITER_H_ + +#include "tensorflow/core/lib/core/coding.h" +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/lib/core/stringpiece.h" +#include "tensorflow/core/lib/hash/crc32c.h" +#if !defined(IS_SLIM_BUILD) +#include "tensorflow/core/lib/io/zlib_compression_options.h" +#include "tensorflow/core/lib/io/zlib_outputbuffer.h" +#endif // IS_SLIM_BUILD +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +class WritableFile; + +namespace io { + +class RecordWriterOptions { + public: + enum CompressionType { NONE = 0, ZLIB_COMPRESSION = 1 }; + CompressionType compression_type = NONE; + + static RecordWriterOptions CreateRecordWriterOptions( + const string& compression_type); + +// Options specific to zlib compression. +#if !defined(IS_SLIM_BUILD) + tensorflow::io::ZlibCompressionOptions zlib_options; +#endif // IS_SLIM_BUILD +}; + +class RecordWriter { + public: + // Format of a single record: + // uint64 length + // uint32 masked crc of length + // byte data[length] + // uint32 masked crc of data + static const size_t kHeaderSize = sizeof(uint64) + sizeof(uint32); + static const size_t kFooterSize = sizeof(uint32); + + // Create a writer that will append data to "*dest". + // "*dest" must be initially empty. + // "*dest" must remain live while this Writer is in use. + RecordWriter(WritableFile* dest, + const RecordWriterOptions& options = RecordWriterOptions()); + + // Calls Close() and logs if an error occurs. + // + // TODO(jhseu): Require that callers explicitly call Close() and remove the + // implicit Close() call in the destructor. + ~RecordWriter(); + + Status WriteRecord(StringPiece slice); + + // Flushes any buffered data held by underlying containers of the + // RecordWriter to the WritableFile. Does *not* flush the + // WritableFile. + Status Flush(); + + // Writes all output to the file. Does *not* close the WritableFile. + // + // After calling Close(), any further calls to `WriteRecord()` or `Flush()` + // are invalid. + Status Close(); + + // Utility method to populate TFRecord headers. Populates record-header in + // "header[0,kHeaderSize-1]". The record-header is based on data[0, n-1]. + inline static void PopulateHeader(char* header, const char* data, size_t n); + + // Utility method to populate TFRecord footers. Populates record-footer in + // "footer[0,kFooterSize-1]". The record-footer is based on data[0, n-1]. + inline static void PopulateFooter(char* footer, const char* data, size_t n); + + private: + WritableFile* dest_; + RecordWriterOptions options_; + + inline static uint32 MaskedCrc(const char* data, size_t n) { + return crc32c::Mask(crc32c::Value(data, n)); + } + + TF_DISALLOW_COPY_AND_ASSIGN(RecordWriter); +}; + +void RecordWriter::PopulateHeader(char* header, const char* data, size_t n) { + core::EncodeFixed64(header + 0, n); + core::EncodeFixed32(header + sizeof(uint64), + MaskedCrc(header, sizeof(uint64))); +} + +void RecordWriter::PopulateFooter(char* footer, const char* data, size_t n) { + core::EncodeFixed32(footer, MaskedCrc(data, n)); +} + +} // namespace io +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_LIB_IO_RECORD_WRITER_H_ diff --git a/linux/include/tensorflow/core/lib/io/snappy/snappy_inputbuffer.h b/linux/include/tensorflow/core/lib/io/snappy/snappy_inputbuffer.h new file mode 100644 index 000000000..84e679922 --- /dev/null +++ b/linux/include/tensorflow/core/lib/io/snappy/snappy_inputbuffer.h @@ -0,0 +1,128 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_LIB_IO_SNAPPY_INPUTBUFFER_H_ +#define TENSORFLOW_LIB_IO_SNAPPY_INPUTBUFFER_H_ + +#include +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/lib/io/inputstream_interface.h" +#include "tensorflow/core/platform/env.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/snappy.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { +namespace io { + +// An SnappyInputBuffer provides support for reading from a file compressed +// using snappy (https://github.com/google/snappy). +// +// A given instance of an SnappyInputBuffer is NOT safe for concurrent use +// by multiple threads +class SnappyInputBuffer : public InputStreamInterface { + public: + // Create a SnappyInputBuffer for `file` with a buffer of size + // `input_buffer_bytes` bytes for reading contents from `file` and another + // buffer with size `output_buffer_bytes` for caching decompressed contents. + // Does *not* take ownership of "file". + SnappyInputBuffer(RandomAccessFile* file, size_t input_buffer_bytes, + size_t output_buffer_bytes); + + // Reads bytes_to_read bytes into *result, overwriting *result. + // + // Return Status codes: + // OK: + // If successful. + // OUT_OF_RANGE: + // If there are not enough bytes to read before the end of the file. + // DATA_LOSS: + // If uncompression failed or if the file is corrupted. + // RESOURCE_EXHAUSTED: + // If input_buffer_ is smaller in size than a compressed block. + // others: + // If reading from file failed. + Status ReadNBytes(int64 bytes_to_read, string* result) override; + + int64 Tell() const override; + + Status Reset() override; + + private: + // Reads data from `file_` and tries to fill up `input_buffer_` if enough + // unread data is left in `file_`. + // + // Looks up `next_in_` to check how much data in `input_buffer_` + // has already been read. The used data is removed and new data is added to + // after any unread data in `input_buffer_`. + // After this call `next_in` points to the start of `input_buffer_` + // and `avail_in_` stores the number of readable bytes in + // `input_buffer_`. + // + // Returns OutOfRange error if NO data could be read from file. Note that this + // won't return an OutOfRange if there wasn't sufficient data in file to + // completely fill up `input_buffer_`. + Status ReadFromFile(); + + // Reads the length of the next compressed block stored in the next 4 bytes at + // `next_in_`. Uncompresses the next compressed block and writes the output + // produced to the output_buffer_. + // Should be called only after the cached output has been consumed. + Status Inflate(); + + // Starts reading bytes at `next_out_` till either `bytes_to_read` + // bytes have been read or `next_out_` is reached. + // Returns the number of bytes read and advances the `next_out_` + // pointer to the next location to read from. + size_t ReadBytesFromCache(size_t bytes_to_read, string* result); + + // Reads the length of the next *compressed* block and stores in `length`. + // The length is stored in 4 bytes in little endian notation. + Status ReadCompressedBlockLength(uint32* length); + + RandomAccessFile* file_; // Not owned + int64 file_pos_ = 0; // Next position to read from in `file_` + size_t input_buffer_capacity_; // Size of `input_buffer_`. + // Must be at least as big as the size of + // the largest compressed block. + size_t output_buffer_capacity_; // Size of `output_buffer_` + + // Buffer for storing contents read from compressed file. + // TODO(srbs): Consider using circular buffers. That would greatly simplify + // the implementation. + std::unique_ptr input_buffer_; + + // Buffer for storing inflated contents of `file_`. + std::unique_ptr output_buffer_; + + // Next unread byte in `input_buffer_`. + char* next_in_; + + // Next unread byte in `output_buffer_` + char* next_out_; + + // Number of unread bytes bytes available at `next_in_` in `input_buffer_`. + size_t avail_in_ = 0; + + // Number of unread bytes bytes available at `next_out_` in `output_buffer_`. + size_t avail_out_ = 0; + + TF_DISALLOW_COPY_AND_ASSIGN(SnappyInputBuffer); +}; + +} // namespace io +} // namespace tensorflow + +#endif // TENSORFLOW_LIB_IO_SNAPPY_INPUTBUFFER_H_ diff --git a/linux/include/tensorflow/core/lib/io/snappy/snappy_outputbuffer.h b/linux/include/tensorflow/core/lib/io/snappy/snappy_outputbuffer.h new file mode 100644 index 000000000..5aea50384 --- /dev/null +++ b/linux/include/tensorflow/core/lib/io/snappy/snappy_outputbuffer.h @@ -0,0 +1,120 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_LIB_IO_SNAPPY_OUTPUTBUFFER_H_ +#define TENSORFLOW_CORE_LIB_IO_SNAPPY_OUTPUTBUFFER_H_ + +#include +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/platform/env.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/snappy.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { +namespace io { + +// Compresses input data using Snappy (https://github.com/google/snappy) and +// writes to `file`. +// +// The input data is cached in a buffer of size `input_buffer_bytes`. When the +// buffer does not have enough available space to fit new data (in the call to +// `Write`), the contents of the buffer are compressed and sent to the output +// buffer. +// +// The compressed output is buffered in a buffer of size `output_buffer_bytes` +// which gets flushed to file when full. +// +// Output file format: +// The output file consists of a sequence of compressed blocks. Each block +// starts with a 4 byte header which stores the length (in bytes) of the +// _compressed_ block _excluding_ this header. The compressed +// block (excluding the 4 byte header) is a valid snappy block and can directly +// be uncompressed using Snappy_Uncompress. +class SnappyOutputBuffer { + public: + // Create an SnappyOutputBuffer for `file` with two buffers that cache the + // 1. input data to be deflated + // 2. the deflated output + // with sizes `input_buffer_bytes` and `output_buffer_bytes` respectively. + // Does not take ownership of `file`. + SnappyOutputBuffer(WritableFile* file, int32 input_buffer_bytes, + int32 output_buffer_bytes); + + // Adds `data` to the compression pipeline. + // + // The input data is buffered in `input_buffer_` and is compressed in bulk + // when the buffer gets full. The compressed output may not be immediately + // written to file but rather buffered in `output_buffer_` and gets written + // to file when the buffer is full. + // + // To immediately write contents to file call `Flush()`. + Status Write(StringPiece data); + + // Compresses any cached input and writes all output to file. This must be + // called before the destructor to avoid any data loss. + Status Flush(); + + private: + // Appends `data` to `input_buffer_`. + // Throws if `data.size()` > AvailableInputSpace(). + void AddToInputBuffer(StringPiece data); + + // Appends `data` to `output_buffer_`. Flushes buffer contents to file when + // buffer gets full. + Status AddToOutputBuffer(const char* data, size_t length); + + // Returns the total space available in `input_buffer_`. + int32 AvailableInputSpace() const; + + // Deflate contents in input_buffer_ and store results in output_buffer_. + // The contents of output stream are written to file if more space is needed. + // + // Note: This method does not flush contents to file. + // Returns non-ok status if writing contents to file fails. + Status DeflateBuffered(); + + // Appends contents of `output_buffer_` to `file_`. + // Returns non-OK status if writing to file fails. + Status FlushOutputBufferToFile(); + + // Compresses `avail_in_` bytes at `next_in_` location in `input_buffer_` and + // writes the results to output using `AddToOutputBuffer`. + // Returns non-OK status if writing to file failed. + Status Deflate(); + + WritableFile* file_; // Not owned + + // Buffer for storing contents read from input `file_`. + // TODO(srbs): Consider using circular buffers. That would greatly simplify + // the implementation. + std::unique_ptr input_buffer_; + size_t input_buffer_capacity_; + char* next_in_; + size_t avail_in_ = 0; + + // Buffer for storing deflated contents of `file_`. + std::unique_ptr output_buffer_; + size_t output_buffer_capacity_; + char* next_out_; + size_t avail_out_; + + TF_DISALLOW_COPY_AND_ASSIGN(SnappyOutputBuffer); +}; + +} // namespace io +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_LIB_IO_SNAPPY_OUTPUTBUFFER_H_ diff --git a/linux/include/tensorflow/core/lib/io/table.h b/linux/include/tensorflow/core/lib/io/table.h new file mode 100644 index 000000000..b9c6b8d9d --- /dev/null +++ b/linux/include/tensorflow/core/lib/io/table.h @@ -0,0 +1,87 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_LIB_IO_TABLE_H_ +#define TENSORFLOW_CORE_LIB_IO_TABLE_H_ + +#include +#include "tensorflow/core/lib/io/iterator.h" + +namespace tensorflow { +class RandomAccessFile; + +namespace table { + +class Block; +class BlockHandle; +class Footer; +struct Options; + +// A Table is a sorted map from strings to strings. Tables are +// immutable and persistent. A Table may be safely accessed from +// multiple threads without external synchronization. +class Table { + public: + // Attempt to open the table that is stored in bytes [0..file_size) + // of "file", and read the metadata entries necessary to allow + // retrieving data from the table. + // + // If successful, returns ok and sets "*table" to the newly opened + // table. The client should delete "*table" when no longer needed. + // If there was an error while initializing the table, sets "*table" + // to NULL and returns a non-ok status. Does not take ownership of + // "*file", but the client must ensure that "file" remains live + // for the duration of the returned table's lifetime. + static Status Open(const Options& options, RandomAccessFile* file, + uint64 file_size, Table** table); + + ~Table(); + + // Returns a new iterator over the table contents. + // The result of NewIterator() is initially invalid (caller must + // call one of the Seek methods on the iterator before using it). + Iterator* NewIterator() const; + + // Given a key, return an approximate byte offset in the file where + // the data for that key begins (or would begin if the key were + // present in the file). The returned value is in terms of file + // bytes, and so includes effects like compression of the underlying data. + // E.g., the approximate offset of the last key in the table will + // be close to the file length. + uint64 ApproximateOffsetOf(const StringPiece& key) const; + + private: + struct Rep; + Rep* rep_; + + explicit Table(Rep* rep) { rep_ = rep; } + static Iterator* BlockReader(void*, const StringPiece&); + + // Calls (*handle_result)(arg, ...) with the entry found after a call + // to Seek(key). May not make such a call if filter policy says + // that key is not present. + Status InternalGet(const StringPiece& key, void* arg, + void (*handle_result)(void* arg, const StringPiece& k, + const StringPiece& v)); + + // No copying allowed + Table(const Table&); + void operator=(const Table&); +}; + +} // namespace table +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_LIB_IO_TABLE_H_ diff --git a/linux/include/tensorflow/core/lib/io/table_builder.h b/linux/include/tensorflow/core/lib/io/table_builder.h new file mode 100644 index 000000000..0e37e0a77 --- /dev/null +++ b/linux/include/tensorflow/core/lib/io/table_builder.h @@ -0,0 +1,99 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// TableBuilder provides the interface used to build a Table +// (an immutable and sorted map from keys to values). +// +// Multiple threads can invoke const methods on a TableBuilder without +// external synchronization, but if any of the threads may call a +// non-const method, all threads accessing the same TableBuilder must use +// external synchronization. + +#ifndef TENSORFLOW_CORE_LIB_IO_TABLE_BUILDER_H_ +#define TENSORFLOW_CORE_LIB_IO_TABLE_BUILDER_H_ + +#include +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/lib/io/table_options.h" + +namespace tensorflow { +class WritableFile; +namespace table { + +class BlockBuilder; +class BlockHandle; + +class TableBuilder { + public: + // Create a builder that will store the contents of the table it is + // building in *file. Does not close the file. It is up to the + // caller to close the file after calling Finish(). + TableBuilder(const Options& options, WritableFile* file); + + // REQUIRES: Either Finish() or Abandon() has been called. + ~TableBuilder(); + + // Add key,value to the table being constructed. + // REQUIRES: key is after any previously added key in lexicographic order. + // REQUIRES: Finish(), Abandon() have not been called + void Add(const StringPiece& key, const StringPiece& value); + + // Advanced operation: writes any buffered key/value pairs to file. + // Can be used to ensure that two adjacent entries never live in + // the same data block. Most clients should not need to use this method. + // Does not flush the file itself. + // REQUIRES: Finish(), Abandon() have not been called + void Flush(); + + // Return non-ok iff some error has been detected. + Status status() const; + + // Finish building the table. Stops using the file passed to the + // constructor after this function returns. + // REQUIRES: Finish(), Abandon() have not been called + Status Finish(); + + // Indicate that the contents of this builder should be abandoned. Stops + // using the file passed to the constructor after this function returns. + // If the caller is not going to call Finish(), it must call Abandon() + // before destroying this builder. + // REQUIRES: Finish(), Abandon() have not been called + void Abandon(); + + // Number of calls to Add() so far. + uint64 NumEntries() const; + + // Size of the file generated so far. If invoked after a successful + // Finish() call, returns the size of the final generated file. + uint64 FileSize() const; + + private: + bool ok() const { return status().ok(); } + void WriteBlock(BlockBuilder* block, BlockHandle* handle); + void WriteRawBlock(const StringPiece& data, CompressionType, + BlockHandle* handle); + + struct Rep; + Rep* rep_; + + // No copying allowed + TableBuilder(const TableBuilder&); + void operator=(const TableBuilder&); +}; + +} // namespace table +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_LIB_IO_TABLE_BUILDER_H_ diff --git a/linux/include/tensorflow/core/lib/io/table_options.h b/linux/include/tensorflow/core/lib/io/table_options.h new file mode 100644 index 000000000..9a36bf163 --- /dev/null +++ b/linux/include/tensorflow/core/lib/io/table_options.h @@ -0,0 +1,68 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_LIB_IO_TABLE_OPTIONS_H_ +#define TENSORFLOW_CORE_LIB_IO_TABLE_OPTIONS_H_ + +#include + +namespace tensorflow { +namespace table { + +// DB contents are stored in a set of blocks, each of which holds a +// sequence of key,value pairs. Each block may be compressed before +// being stored in a file. The following enum describes which +// compression method (if any) is used to compress a block. +enum CompressionType { + // NOTE: do not change the values of existing entries, as these are + // part of the persistent format on disk. + kNoCompression = 0x0, + kSnappyCompression = 0x1 +}; + +// Options to control the behavior of a table (passed to Table::Open) +struct Options { + // Approximate size of user data packed per block. Note that the + // block size specified here corresponds to uncompressed data. The + // actual size of the unit read from disk may be smaller if + // compression is enabled. This parameter can be changed dynamically. + size_t block_size = 262144; + + // Number of keys between restart points for delta encoding of keys. + // This parameter can be changed dynamically. Most clients should + // leave this parameter alone. + int block_restart_interval = 16; + + // Compress blocks using the specified compression algorithm. This + // parameter can be changed dynamically. + // + // Default: kSnappyCompression, which gives lightweight but fast + // compression. + // + // Typical speeds of kSnappyCompression on an Intel(R) Core(TM)2 2.4GHz: + // ~200-500MB/s compression + // ~400-800MB/s decompression + // Note that these speeds are significantly faster than most + // persistent storage speeds, and therefore it is typically never + // worth switching to kNoCompression. Even if the input data is + // incompressible, the kSnappyCompression implementation will + // efficiently detect that and will switch to uncompressed mode. + CompressionType compression = kSnappyCompression; +}; + +} // namespace table +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_LIB_IO_TABLE_OPTIONS_H_ diff --git a/linux/include/tensorflow/core/lib/io/two_level_iterator.h b/linux/include/tensorflow/core/lib/io/two_level_iterator.h new file mode 100644 index 000000000..ce972bc68 --- /dev/null +++ b/linux/include/tensorflow/core/lib/io/two_level_iterator.h @@ -0,0 +1,41 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_LIB_IO_TWO_LEVEL_ITERATOR_H_ +#define TENSORFLOW_LIB_IO_TWO_LEVEL_ITERATOR_H_ + +#include "tensorflow/core/lib/io/iterator.h" + +namespace tensorflow { +namespace table { + +// Return a new two level iterator. A two-level iterator contains an +// index iterator whose values point to a sequence of blocks where +// each block is itself a sequence of key,value pairs. The returned +// two-level iterator yields the concatenation of all key/value pairs +// in the sequence of blocks. Takes ownership of "index_iter" and +// will delete it when no longer needed. +// +// Uses a supplied function to convert an index_iter value into +// an iterator over the contents of the corresponding block. +extern Iterator* NewTwoLevelIterator( + Iterator* index_iter, + Iterator* (*block_function)(void* arg, const StringPiece& index_value), + void* arg); + +} // namespace table +} // namespace tensorflow + +#endif // TENSORFLOW_LIB_IO_TWO_LEVEL_ITERATOR_H_ diff --git a/linux/include/tensorflow/core/lib/io/zlib_compression_options.h b/linux/include/tensorflow/core/lib/io/zlib_compression_options.h new file mode 100644 index 000000000..238c1464f --- /dev/null +++ b/linux/include/tensorflow/core/lib/io/zlib_compression_options.h @@ -0,0 +1,134 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_LIB_IO_ZLIB_COMPRESSION_OPTIONS_H_ +#define TENSORFLOW_LIB_IO_ZLIB_COMPRESSION_OPTIONS_H_ + +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { +namespace io { + +class ZlibCompressionOptions { + public: + ZlibCompressionOptions(); + + static ZlibCompressionOptions DEFAULT(); + static ZlibCompressionOptions RAW(); + static ZlibCompressionOptions GZIP(); + + // Defaults to Z_NO_FLUSH + int8 flush_mode; + + // Size of the buffer used for caching the data read from source file. + int64 input_buffer_size = 256 << 10; + + // Size of the sink buffer where the compressed/decompressed data produced by + // zlib is cached. + int64 output_buffer_size = 256 << 10; + + // The window_bits parameter is the base two logarithm of the window size + // (the size of the history buffer). Larger values of buffer size result in + // better compression at the expense of memory usage. + // + // Accepted values: + // + // 8..15: + // Normal deflate with zlib header and checksum. + // + // -8..-15: + // Negative values can be used for raw deflate/inflate. In this case, + // -window_bits determines the window size. deflate() will then generate raw + // deflate data with no zlib header or trailer, and will not compute an + // adler32 check value. inflate() will then process raw deflate data, not + // looking for a zlib or gzip header, not generating a check value, and not + // looking for any check values for comparison at the end of the stream. + // + // 16 + [8..15]: + // window_bits can also be greater than 15 for optional gzip encoding. Add 16 + // to window_bits to write a simple gzip header and trailer around the + // compressed data instead of a zlib wrapper. The gzip header will have no + // file name, no extra data, no comment, no modification time (set to zero), + // no header crc, and the operating system will be set to 255 (unknown). If a + // gzip stream is being written, strm->adler is a crc32 instead of an adler32. + // + // 0: + // window_bits can also be zero to request that inflate use the window size + // in the zlib header of the compressed stream. + // + // While inflating, window_bits must be greater than or equal to the + // window_bits value provided used while compressing. If a compressed stream + // with a larger window size is given as input, inflate() will return with the + // error code Z_DATA_ERROR instead of trying to allocate a larger window. + // + // Defaults to MAX_WBITS + int8 window_bits; + + // From the zlib manual (http://www.zlib.net/manual.html): + // The compression level must be Z_DEFAULT_COMPRESSION, or between 0 and 9: + // 1 gives best speed, 9 gives best compression, 0 gives no compression at all + // (the input data is simply copied a block at a time). Z_DEFAULT_COMPRESSION + // requests a default compromise between speed and compression (currently + // equivalent to level 6). + int8 compression_level; + + // Only Z_DEFLATED is supported at this time. + int8 compression_method; + + // From the zlib manual (http://www.zlib.net/manual.html): + // The mem_level parameter specifies how much memory should be allocated for + // the internal compression state. mem_level=1 uses minimum memory but is slow + // and reduces compression ratio; mem_level=9 uses maximum memory for optimal + // speed. The default value is 8. + int8 mem_level = 9; + + // From the zlib manual (http://www.zlib.net/manual.html): + // The strategy parameter is used to tune the compression algorithm. Use the + // value Z_DEFAULT_STRATEGY for normal data, Z_FILTERED for data produced by + // a filter (or predictor), Z_HUFFMAN_ONLY to force Huffman encoding only + // (no string match), or Z_RLE to limit match distances to one + // (run-length encoding). Filtered data consists mostly of small values with + // a somewhat random distribution. In this case, the compression algorithm is + // tuned to compress them better. The effect of Z_FILTERED is to force more + // Huffman coding and less string matching; it is somewhat intermediate + // between Z_DEFAULT_STRATEGY and Z_HUFFMAN_ONLY. Z_RLE is designed to be + // almost as fast as Z_HUFFMAN_ONLY, but give better compression for + // PNG image data. The strategy parameter only affects the compression ratio + // but not the correctness of the compressed output even if it is not set + // appropriately. Z_FIXED prevents the use of dynamic Huffman codes, allowing + // for a simpler decoder for special applications. + int8 compression_strategy; +}; + +inline ZlibCompressionOptions ZlibCompressionOptions::DEFAULT() { + return ZlibCompressionOptions(); +} + +inline ZlibCompressionOptions ZlibCompressionOptions::RAW() { + ZlibCompressionOptions options = ZlibCompressionOptions(); + options.window_bits = -options.window_bits; + return options; +} + +inline ZlibCompressionOptions ZlibCompressionOptions::GZIP() { + ZlibCompressionOptions options = ZlibCompressionOptions(); + options.window_bits = options.window_bits + 16; + return options; +} + +} // namespace io +} // namespace tensorflow + +#endif // TENSORFLOW_LIB_IO_ZLIB_COMPRESSION_OPTIONS_H_ diff --git a/linux/include/tensorflow/core/lib/io/zlib_inputstream.h b/linux/include/tensorflow/core/lib/io/zlib_inputstream.h new file mode 100644 index 000000000..ac9e23ca9 --- /dev/null +++ b/linux/include/tensorflow/core/lib/io/zlib_inputstream.h @@ -0,0 +1,136 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_LIB_IO_ZLIB_INPUTSTREAM_H_ +#define TENSORFLOW_LIB_IO_ZLIB_INPUTSTREAM_H_ + +#include + +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/lib/io/inputstream_interface.h" +#include "tensorflow/core/lib/io/zlib_compression_options.h" +#include "tensorflow/core/platform/env.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { +namespace io { + +// Forward declare some members of zlib.h, which is only included in the +// .cc file. +struct ZStreamDef; + +// An ZlibInputStream provides support for reading from a stream compressed +// using zlib (http://www.zlib.net/). Buffers the contents of the file. +// +// A given instance of an ZlibInputStream is NOT safe for concurrent use +// by multiple threads +class ZlibInputStream : public InputStreamInterface { + public: + // Create a ZlibInputStream for `input_stream` with a buffer of size + // `input_buffer_bytes` bytes for reading contents from `input_stream` and + // another buffer with size `output_buffer_bytes` for caching decompressed + // contents. + // + // Takes ownership of `input_stream` iff `owns_input_stream` is true. + ZlibInputStream(InputStreamInterface* input_stream, size_t input_buffer_bytes, + size_t output_buffer_bytes, + const ZlibCompressionOptions& zlib_options, + bool owns_input_stream); + + // Equivalent to the previous constructor with owns_input_stream=false. + ZlibInputStream(InputStreamInterface* input_stream, size_t input_buffer_bytes, + size_t output_buffer_bytes, + const ZlibCompressionOptions& zlib_options); + + ~ZlibInputStream(); + + // Reads bytes_to_read bytes into *result, overwriting *result. + // + // Return Status codes: + // OK: If successful. + // OUT_OF_RANGE: If there are not enough bytes to read before + // the end of the stream. + // ABORTED: If inflate() fails, we return the error code with the + // error message in `z_stream_->msg`. + // others: If reading from stream failed. + Status ReadNBytes(int64 bytes_to_read, string* result) override; + + int64 Tell() const override; + + Status Reset() override; + + private: + void InitZlibBuffer(); + + const bool owns_input_stream_; + InputStreamInterface* input_stream_; + size_t input_buffer_capacity_; // Size of z_stream_input_ + size_t output_buffer_capacity_; // Size of z_stream_output_ + char* next_unread_byte_; // Next unread byte in z_stream_output_ + + ZlibCompressionOptions const zlib_options_; + + std::unique_ptr z_stream_def_; + + // Reads data from `input_stream_` and tries to fill up `z_stream_input_` if + // enough unread data is left in `input_stream_`. + // + // Looks up z_stream_->next_in to check how much data in z_stream_input_ + // has already been read. The used data is removed and new data is added to + // after any unread data in z_stream_input_. + // After this call z_stream_->next_in points to the start of z_stream_input_ + // and z_stream_->avail_in stores the number of readable bytes in + // z_stream_input_. + // + // Returns OutOfRange error if NO data could be read from stream. Note that + // this won't return an OutOfRange if there wasn't sufficient data in stream + // to completely fill up z_stream_input_. + Status ReadFromStream(); + + // Calls `inflate()` and returns DataLoss Status if it failed. + Status Inflate(); + + // Starts reading bytes at `next_unread_byte_` till either `bytes_to_read` + // bytes have been read or `z_stream_->next_out` is reached. + // Returns the number of bytes read and advances the `next_unread_byte_` + // pointer to the next location to read from. + size_t ReadBytesFromCache(size_t bytes_to_read, string* result); + + // The number of unread bytes in z_stream_output_. + // + // z_stream_output_ --> + // + // [RRRRRRRRRRRRRRRRRRUUUUUUUUUUUUUU000000000000000000] + // ^ ^ + // next_unread_byte_ z_stream_->next_out + // + // R: Read bytes + // U: Unread bytes + // 0: garbage bytes where new output will be written + // + // Returns the size of [next_unread_byte_, z_stream_->next_out) + size_t NumUnreadBytes() const; + + // Number of *uncompressed* bytes that have been read from this stream. + int64 bytes_read_; + + TF_DISALLOW_COPY_AND_ASSIGN(ZlibInputStream); +}; + +} // namespace io +} // namespace tensorflow + +#endif // TENSORFLOW_LIB_IO_ZLIB_INPUTSTREAM_H_ diff --git a/linux/include/tensorflow/core/lib/io/zlib_outputbuffer.h b/linux/include/tensorflow/core/lib/io/zlib_outputbuffer.h new file mode 100644 index 000000000..ccad2fda4 --- /dev/null +++ b/linux/include/tensorflow/core/lib/io/zlib_outputbuffer.h @@ -0,0 +1,146 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_LIB_IO_COMPRESSED_OUTPUTBUFFER_H_ +#define TENSORFLOW_CORE_LIB_IO_COMPRESSED_OUTPUTBUFFER_H_ + +#include + +#include + +#include "tensorflow/core/lib/core/status.h" +#include "tensorflow/core/lib/io/zlib_compression_options.h" +#include "tensorflow/core/platform/env.h" +#include "tensorflow/core/platform/file_system.h" +#include "tensorflow/core/platform/macros.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { +namespace io { + +// Provides support for writing compressed output to file using zlib +// (http://www.zlib.net/). +// A given instance of an ZlibOutputBuffer is NOT safe for concurrent use +// by multiple threads +class ZlibOutputBuffer : public WritableFile { + public: + // Create an ZlibOutputBuffer for `file` with two buffers that cache the + // 1. input data to be deflated + // 2. the deflated output + // with sizes `input_buffer_bytes` and `output_buffer_bytes` respectively. + // Does not take ownership of `file`. + // output_buffer_bytes should be greater than 1. + ZlibOutputBuffer( + WritableFile* file, + int32 input_buffer_bytes, // size of z_stream.next_in buffer + int32 output_buffer_bytes, // size of z_stream.next_out buffer + const ZlibCompressionOptions& zlib_options); + + ~ZlibOutputBuffer(); + + // Initializes some state necessary for the output buffer. This call is + // required before any other operation on the buffer. + Status Init(); + + // Adds `data` to the compression pipeline. + // + // The input data is buffered in `z_stream_input_` and is compressed in bulk + // when the buffer gets full. The compressed output is not immediately + // written to file but rather buffered in `z_stream_output_` and gets written + // to file when the buffer is full. + // + // To immediately write contents to file call `Flush()`. + Status Append(StringPiece data) override; + + // Deflates any cached input and writes all output to file. + Status Flush() override; + + // Compresses any cached input and writes all output to file. This must be + // called before the destructor to avoid any data loss. + // + // Contrary to `Flush()` this informs zlib that it should not expect any + // further input by using Z_FINISH flush mode. Also cleans up z_stream. + // + // After calling this, any further calls to `Write()`, `Flush()` or `Close()` + // will fail. + Status Close() override; + + // Deflates any cached input, writes all output to file and syncs it. + Status Sync() override; + + private: + WritableFile* file_; // Not owned + Status init_status_; + size_t input_buffer_capacity_; + size_t output_buffer_capacity_; + + // Buffer for storing contents read from input `file_`. + // TODO(srbs): Consider using circular buffers. That would greatly simplify + // the implementation. + std::unique_ptr z_stream_input_; + + // Buffer for storing deflated contents of `file_`. + std::unique_ptr z_stream_output_; + + ZlibCompressionOptions const zlib_options_; + + // Configuration passed to `deflate`. + // + // z_stream_->next_in: + // Next byte to compress. Points to some byte in z_stream_input_ buffer. + // z_stream_->avail_in: + // Number of bytes available to be compressed at this time. + // z_stream_->next_out: + // Next byte to write compressed data to. Points to some byte in + // z_stream_output_ buffer. + // z_stream_->avail_out: + // Number of free bytes available at write location. + std::unique_ptr z_stream_; + + // Adds `data` to `z_stream_input_`. + // Throws if `data.size()` > AvailableInputSpace(). + void AddToInputBuffer(StringPiece data); + + // Returns the total space available in z_input_stream_ buffer. + int32 AvailableInputSpace() const; + + // Deflate contents in z_stream_input_ and store results in z_stream_output_. + // The contents of output stream are written to file if more space is needed. + // On successful termination it is assured that: + // - z_stream_->avail_in == 0 + // - z_stream_->avail_out > 0 + // + // Note: This method does not flush contents to file. + // Returns non-ok status if writing contents to file fails. + Status DeflateBuffered(bool last = false); + + // Appends contents of `z_stream_output_` to `file_`. + // Returns non-OK status if writing to file fails. + Status FlushOutputBufferToFile(); + + // Calls `deflate()` and returns DataLoss Status if it failed. + Status Deflate(int flush); + + static bool IsSyncOrFullFlush(uint8 flush_mode) { + return flush_mode == Z_SYNC_FLUSH || flush_mode == Z_FULL_FLUSH; + } + + TF_DISALLOW_COPY_AND_ASSIGN(ZlibOutputBuffer); +}; + +} // namespace io +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_LIB_IO_COMPRESSED_OUTPUTBUFFER_H_ diff --git a/linux/include/tensorflow/core/lib/jpeg/jpeg_handle.h b/linux/include/tensorflow/core/lib/jpeg/jpeg_handle.h new file mode 100644 index 000000000..86fa3ac5c --- /dev/null +++ b/linux/include/tensorflow/core/lib/jpeg/jpeg_handle.h @@ -0,0 +1,60 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// This file declares the functions and structures for memory I/O with libjpeg +// These functions are not meant to be used directly, see jpeg_mem.h instead. + +#ifndef TENSORFLOW_CORE_LIB_JPEG_JPEG_HANDLE_H_ +#define TENSORFLOW_CORE_LIB_JPEG_JPEG_HANDLE_H_ + +#include "tensorflow/core/platform/jpeg.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { +namespace jpeg { + +// Handler for fatal JPEG library errors: clean up & return +void CatchError(j_common_ptr cinfo); + +typedef struct { + struct jpeg_destination_mgr pub; + JOCTET *buffer; + int bufsize; + int datacount; + string *dest; +} MemDestMgr; + +typedef struct { + struct jpeg_source_mgr pub; + const unsigned char *data; + unsigned long int datasize; + bool try_recover_truncated_jpeg; +} MemSourceMgr; + +void SetSrc(j_decompress_ptr cinfo, const void *data, + unsigned long int datasize, bool try_recover_truncated_jpeg); + +// JPEG destination: we will store all the data in a buffer "buffer" of total +// size "bufsize", if the buffer overflows, we will be in trouble. +void SetDest(j_compress_ptr cinfo, void *buffer, int bufsize); +// Same as above, except that buffer is only used as a temporary structure and +// is emptied into "destination" as soon as it fills up. +void SetDest(j_compress_ptr cinfo, void *buffer, int bufsize, + string *destination); + +} // namespace jpeg +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_LIB_JPEG_JPEG_HANDLE_H_ diff --git a/linux/include/tensorflow/core/lib/jpeg/jpeg_mem.h b/linux/include/tensorflow/core/lib/jpeg/jpeg_mem.h new file mode 100644 index 000000000..03437a4e7 --- /dev/null +++ b/linux/include/tensorflow/core/lib/jpeg/jpeg_mem.h @@ -0,0 +1,162 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// This file defines functions to compress and uncompress JPEG files +// to and from memory. It provides interfaces for raw images +// (data array and size fields). +// Direct manipulation of JPEG strings are supplied: Flip, Rotate, Crop.. + +#ifndef TENSORFLOW_CORE_LIB_JPEG_JPEG_MEM_H_ +#define TENSORFLOW_CORE_LIB_JPEG_JPEG_MEM_H_ + +#include +#include + +#include "tensorflow/core/lib/core/stringpiece.h" +#include "tensorflow/core/platform/jpeg.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { +namespace jpeg { + +// Flags for Uncompress +struct UncompressFlags { + // ratio can be 1, 2, 4, or 8 and represent the denominator for the scaling + // factor (eg ratio = 4 means that the resulting image will be at 1/4 original + // size in both directions). + int ratio = 1; + + // The number of bytes per pixel (1, 3 or 4), or 0 for autodetect. + int components = 0; + + // If true, decoder will use a slower but nicer upscaling of the chroma + // planes (yuv420/422 only). + bool fancy_upscaling = true; + + // If true, will attempt to fill in missing lines of truncated files + bool try_recover_truncated_jpeg = false; + + // The minimum required fraction of lines read before the image is accepted. + float min_acceptable_fraction = 1.0; + + // The distance in bytes from one scanline to the other. Should be at least + // equal to width*components*sizeof(JSAMPLE). If 0 is passed, the stride + // used will be this minimal value. + int stride = 0; + + // Setting of J_DCT_METHOD enum in jpeglib.h, for choosing which + // algorithm to use for DCT/IDCT. + // + // Setting this has a quality/speed trade-off implication. + J_DCT_METHOD dct_method = JDCT_DEFAULT; + + // Settings of crop window before decompression. + bool crop = false; + // Vertical coordinate of the top-left corner of the result in the input. + int crop_x = 0; + // Horizontal coordinate of the top-left corner of the result in the input. + int crop_y = 0; + // Width of the output image. + int crop_width = 0; + // Height of the output image. + int crop_height = 0; +}; + +// Uncompress some raw JPEG data given by the pointer srcdata and the length +// datasize. +// - width and height are the address where to store the size of the +// uncompressed image in pixels. May be nullptr. +// - components is the address where the number of read components are +// stored. This is *output only*: to request a specific number of +// components use flags.components. May be nullptr. +// - nwarn is the address in which to store the number of warnings. +// May be nullptr. +// The function returns a pointer to the raw uncompressed data or NULL if +// there was an error. The caller of the function is responsible for +// freeing the memory (using delete []). +uint8* Uncompress(const void* srcdata, int datasize, + const UncompressFlags& flags, int* width, int* height, + int* components, // Output only: useful with autodetect + int64* nwarn); + +// Version of Uncompress that allocates memory via a callback. The callback +// arguments are (width, height, components). If the size is known ahead of +// time this function can return an existing buffer; passing a callback allows +// the buffer to be shaped based on the JPEG header. The caller is responsible +// for freeing the memory *even along error paths*. +uint8* Uncompress(const void* srcdata, int datasize, + const UncompressFlags& flags, int64* nwarn, + std::function allocate_output); + +// Read jpeg header and get image information. Returns true on success. +// The width, height, and components points may be null. +bool GetImageInfo(const void* srcdata, int datasize, int* width, int* height, + int* components); + +// Note: (format & 0xff) = number of components (<=> bytes per pixels) +enum Format { + FORMAT_GRAYSCALE = 0x001, // 1 byte/pixel + FORMAT_RGB = 0x003, // 3 bytes/pixel RGBRGBRGBRGB... + FORMAT_RGBA = 0x004, // 4 bytes/pixel RGBARGBARGBARGBA... + FORMAT_ABGR = 0x104 // 4 bytes/pixel ABGRABGRABGR... +}; + +// Flags for compression +struct CompressFlags { + // Encoding of the input data for compression + Format format; + + // Quality of the compression from 0-100 + int quality = 95; + + // If true, create a jpeg image that loads progressively + bool progressive = false; + + // If true, reduce jpeg size without changing quality (at the cost of CPU/RAM) + bool optimize_jpeg_size = false; + + // See http://en.wikipedia.org/wiki/Chroma_subsampling + bool chroma_downsampling = true; + + // Resolution + int density_unit = 1; // 1 = in, 2 = cm + int x_density = 300; + int y_density = 300; + + // If not empty, embed this XMP metadata in the image header + StringPiece xmp_metadata; + + // The distance in bytes from one scanline to the other. Should be at least + // equal to width*components*sizeof(JSAMPLE). If 0 is passed, the stride + // used will be this minimal value. + int stride = 0; +}; + +// Compress some raw image given in srcdata, the data is a 2D array of size +// stride*height with one of the formats enumerated above. +// The encoded data is returned as a string. +// If not empty, XMP metadata can be embedded in the image header +// On error, returns the empty string (which is never a valid jpeg). +string Compress(const void* srcdata, int width, int height, + const CompressFlags& flags); + +// On error, returns false and sets output to empty. +bool Compress(const void* srcdata, int width, int height, + const CompressFlags& flags, string* output); + +} // namespace jpeg +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_LIB_JPEG_JPEG_MEM_H_ diff --git a/linux/include/tensorflow/core/lib/math/math_util.h b/linux/include/tensorflow/core/lib/math/math_util.h new file mode 100644 index 000000000..502d74151 --- /dev/null +++ b/linux/include/tensorflow/core/lib/math/math_util.h @@ -0,0 +1,163 @@ +/* Copyright 2015 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +#ifndef TENSORFLOW_CORE_LIB_MATH_MATH_UTIL_H_ +#define TENSORFLOW_CORE_LIB_MATH_MATH_UTIL_H_ + +#include + +#include "tensorflow/core/platform/logging.h" +#include "tensorflow/core/platform/types.h" + +namespace tensorflow { + +class MathUtil { + public: + // ---------------------------------------------------------------------- + // CeilOfRatio + // FloorOfRatio + // Returns the ceil (resp. floor) of the ratio of two integers. + // + // * IntegralType: any integral type, whether signed or not. + // * numerator: any integer: positive, negative, or zero. + // * denominator: a non-zero integer, positive or negative. + // + // This implementation is correct, meaning there is never any precision loss, + // and there is never an overflow. However, if the type is signed, having + // numerator == MathLimits::kMin and denominator == -1 is not a + // valid input, because kMin has a greater absolute value than kMax. + // + // Input validity is DCHECKed. When not in debug mode, invalid inputs raise + // SIGFPE. + // + // This method has been designed and tested so that it should always be + // preferred to alternatives. Indeed, there exist popular recipes to compute + // the result, such as casting to double, but they are in general incorrect. + // In cases where an alternative technique is correct, performance measurement + // showed the provided implementation is faster. + template + static IntegralType CeilOfRatio(IntegralType numerator, + IntegralType denominator) { + return CeilOrFloorOfRatio(numerator, denominator); + } + template + static IntegralType FloorOfRatio(IntegralType numerator, + IntegralType denominator) { + return CeilOrFloorOfRatio(numerator, denominator); + } + + template + static IntegralType CeilOrFloorOfRatio(IntegralType numerator, + IntegralType denominator); + + template + static IntegralType GCD(IntegralType x, IntegralType y); + + // ---------------------------------------------------------------------- + // IPow + // Computes the result of raising a number to a non-negative integral power. + // + // * T: An integral type, floating-point type, or user-defined type for which + // operator*= is defined. + // * base: the base "v" of the operation + // * exp: the exponent "i" of the operation; must be non-negative. + // + // Computes v^i, in a way that is faster than std::pow (which supports + // arbitrary real exponents). + // + // When T is a floating point type, this has the same semantics as std::pow, + // but it is much faster. When T is an integral type, computations are + // performed in the value domain of T, and overflow semantics are those of T. + // + // Input validity is DCHECKed. + template + static T IPow(T base, int exp); +}; + +// ---- CeilOrFloorOfRatio ---- +// This is a branching-free, cast-to-double-free implementation. +// +// Casting to double is in general incorrect because of loss of precision +// when casting an int64 into a double. +// +// There's a bunch of 'recipes' to compute a integer ceil (or floor) on the web, +// and most of them are incorrect. +template +IntegralType MathUtil::CeilOrFloorOfRatio(IntegralType numerator, + IntegralType denominator) { + DCHECK_NE(0, denominator) << "Division by zero is not supported."; + + const IntegralType rounded_toward_zero = numerator / denominator; + const IntegralType intermediate_product = rounded_toward_zero * denominator; + + if (ceil) { // Compile-time condition: not an actual branching + // When rounded_toward_zero is negative, then an adjustment is never needed: + // the real ratio is negative, and so rounded toward zero is the ceil. + // When rounded_toward_zero is non-negative, an adjustment is needed if the + // sign of the difference numerator - intermediate_product is the same as + // the sign of the denominator. + // + // + // Using a bool and then a static_cast to IntegralType is not strictly + // necessary, but it makes the code clear, and anyway the compiler should + // get rid of it. + const bool needs_adjustment = + (rounded_toward_zero >= 0) && + ((denominator > 0 && numerator > intermediate_product) || + (denominator < 0 && numerator < intermediate_product)); + const IntegralType adjustment = static_cast(needs_adjustment); + const IntegralType ceil_of_ratio = rounded_toward_zero + adjustment; + return ceil_of_ratio; + } else { + // Floor case: symmetrical to the previous one + const bool needs_adjustment = + (rounded_toward_zero <= 0) && + ((denominator > 0 && numerator < intermediate_product) || + (denominator < 0 && numerator > intermediate_product)); + const IntegralType adjustment = static_cast(needs_adjustment); + const IntegralType floor_of_ratio = rounded_toward_zero - adjustment; + return floor_of_ratio; + } +} + +template +IntegralType MathUtil::GCD(IntegralType a, IntegralType b) { + static_assert(std::is_unsigned::value, + "signed GCD not supported!"); + while (b != 0) { + IntegralType r = a % b; + a = b; + b = r; + } + return a; +} + +// ---- IPow ---- +// Implemented with the squared exponentiation method (a.k.a. double-and-add). +// +// Note that "exp >>= 1" is faster than "exp /= 2" on at least one platform. +template +T MathUtil::IPow(T base, int exp) { + DCHECK_GE(exp, 0); + for (T result(1);; base *= base) { + if ((exp & 1) != 0) result *= base; + exp >>= 1; + if (exp == 0) return result; + } +} + +} // namespace tensorflow + +#endif // TENSORFLOW_CORE_LIB_MATH_MATH_UTIL_H_ diff --git a/linux/include/tensorflow/core/lib/monitoring/collected_metrics.h b/linux/include/tensorflow/core/lib/monitoring/collected_metrics.h new file mode 100644 index 000000000..e20098160 --- /dev/null +++ b/linux/include/tensorflow/core/lib/monitoring/collected_metrics.h @@ -0,0 +1,154 @@ +/* Copyright 2016 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ + +// Standard format in which the metrics are collected, before being exported. +// These are to be used only by the CollectionRegistry and exporters which +// collect metrics using the CollectionRegistry. + +#ifndef TENSORFLOW_CORE_LIB_MONITORING_COLLECTED_METRICS_H_ +#define TENSORFLOW_CORE_LIB_MONITORING_COLLECTED_METRICS_H_ + +#include +#include +#include +#include + +#include "tensorflow/core/framework/summary.pb.h" +#include "tensorflow/core/lib/monitoring/metric_def.h" + +namespace tensorflow { +namespace monitoring { + +// A metric is a statistic about a monitorable entity. +// +// Metrics are named with path-like strings, which must conform to the regular +// expression (/[a-zA-Z0-9_-]+)+. For example: +// +// /proc/cpu_usage +// /rpc/client/count +// +// Metrics may optionally have labels, which are additional dimensions used to +// identify the metric's values. For example, the metric /rpc/client/count +// might have two labels named "rpc_service" and "rpc_method". +// +// A label name must be an identifier, which conform to the regular expression +// [a-zA-Z_][a-zA-Z_0-9]*, and is only unique within the context of the metric +// it is a label for. +// +// MetricDescriptor defines the structure of the metric (e.g. the fact that it's +// a counter and that it has two labels named "rpc_service" and "rpc_method"). +// Individual points will provide a value for the metric (e.g. the counter +// value) and specific values for each of the labels. +// +// There's no scoping relationship between metrics and monitorable entities: the +// metric /rpc/client/count should be defined the same way no matter which +// monitorable entity is exporting it. +struct MetricDescriptor { + // Metric names are path-like. E.g., "/mycomponent/mymetric". + string name; + + // A human-readable description of what this metric measures. + string description; + + // Label names for the metric. + // See the example in the top level comment for MetricDescriptor. + std::vector label_names; + + MetricKind metric_kind; + + ValueType value_type; +}; + +struct Point { + // Usually a Point should provide a |label| field for each of the labels + // defined in the corresponding MetricDescriptor. During transitions in + // metric definitions, however, there may be times when a Point provides more + // or fewer labels than those that appear in the MetricDescriptor. + struct Label { + // The |name| field must match the |label_name| field in the + // MetricDescriptor for this Point. + string name; + string value; + }; + std::vector